Date: Thu, 12 Oct 2017 11:38:09 +0100 From: Kate Dawson <k4t@3msg.es> To: freebsd-questions@freebsd.org Subject: Re: FreeBSD ZFS file server with SSD HDD Message-ID: <20171012103809.GF24374@apple.rat.burntout.org> In-Reply-To: <CA%2BtpaK3Cga3SKmbKnRts_SSp=D4qk9p%2BaTzNBZeEqDuvQGVd9A@mail.gmail.com> References: <20171011130512.GE24374@apple.rat.burntout.org> <CA%2BtpaK3Cga3SKmbKnRts_SSp=D4qk9p%2BaTzNBZeEqDuvQGVd9A@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--KJY2Ze80yH5MUxol
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Wed, Oct 11, 2017 at 09:41:13AM -0500, Adam Vande More wrote:
> On Wed, Oct 11, 2017 at 8:05 AM, Kate Dawson <k4t@3msg.es> wrote:
>=20
> > Hi,
> >
> > If I look at the output of `zpool iostat -v tank 1 ` I see that every
> > five seconds or so, the numner of write operation go to > 2k
> >
> > I think this shows that the I'm hitting the limit that the spinning disk
> > can provide in this workload.
> >
>=20
> I doubt that is the cause. It is more likely you have
>=20
> vfs.zfs.txg.timeout
This is set to 5=20
I may try decreasing it and seeing how things go for a few hours.
>=20
> set to the default. Have you tried any other zfs or nfs tuning? If so,
> please share those details.
vfs.zfs.l2arc_noprefetch=3D0
zfs get sync tank/exports/xendomu
NAME PROPERTY VALUE SOURCE
tank/exports/xendomu sync disabled local
>=20
> Does gstat reveal anything useful?
>=20
Thanks for this tool... on regular occasions drives show high percentage bu=
sy
dT: 1.001s w: 1.000s filter: eli =
=
L(q) ops/s r/s kBps ms/r w/s kBps =
ms/w %busy Name
5 247 0 0 0.0 247 15160 16.2 69.1| da2.eli =
=20
0 304 0 0 0.0 304 16219 14.1 71.6| da3.eli
0 240 0 0 0.0 240 15108 16.3 66.6| da4.eli
2 231 0 0 0.0 231 16695 16.1 65.0| da5.eli
1 227 0 0 0.0 227 16703 16.5 66.7| da9.eli
0 304 0 0 0.0 304 16219 13.8 72.7| da8.eli
0 193 0 0 0.0 193 15872 20.2 65.8| ada0.eli
3 263 0 0 0.0 263 13878 15.7 70.7| ada2.eli
0 193 0 0 0.0 193 15872 20.4 64.6| ada1.eli
3 269 0 0 0.0 269 13914 14.7 69.9| ada3.eli
0 256 0 0 0.0 256 16139 16.4 71.5| ada4.eli
0 258 0 0 0.0 258 16139 16.6 72.5| ada5.eli
dT: 1.001s w: 1.000s filter: eli
L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name
0 265 6 739 36.8 257 19425 17.5 103.0| da2.eli
0 260 1 108 18.0 258 21643 25.8 87.1| da3.eli
0 268 8 927 29.4 258 19437 20.2 95.0| da4.eli
1 305 14 1754 32.6 289 21379 17.6 101.2| da5.eli
0 300 8 871 34.8 290 21391 20.1 98.4| da9.eli
0 273 15 1786 30.2 257 21635 19.7 94.1| da8.eli
0 292 13 1378 29.5 278 21731 22.9 103.0| ada0.eli
0 285 10 1187 30.2 273 21003 16.5 77.7| ada2.eli
0 290 10 1091 40.0 279 21731 20.6 97.7| ada1.eli
0 269 4 384 33.4 263 21015 19.7 112.2| ada3.eli
0 251 5 619 23.0 245 21343 21.4 89.0| ada4.eli
0 250 7 791 31.7 242 21199 21.7 88.3| ada5.eli
Here is some 30s or so of `zpool iostat tank 1 `
root@blue:/tank/exports/xendomu/restore # zpool iostat tank 1
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
tank 3.39T 2.05T 80 521 7.04M 13.8M
tank 3.39T 2.05T 2 0 160K 0
tank 3.39T 2.05T 8 1.74K 461K 45.4M
tank 3.39T 2.05T 107 1.04K 11.9M 25.0M
tank 3.39T 2.05T 29 0 2.48M 0
tank 3.39T 2.05T 27 0 1.38M 0
tank 3.39T 2.05T 4 0 116K 0
tank 3.39T 2.05T 15 1.52K 1.27M 23.4M
tank 3.39T 2.05T 12 0 846K 0
tank 3.39T 2.05T 901 0 108M 0
tank 3.39T 2.05T 14 0 814K 0
tank 3.39T 2.05T 1 1.34K 95.8K 103M
tank 3.39T 2.05T 249 1.90K 29.2M 114M
tank 3.39T 2.05T 530 0 59.4M 0
tank 3.39T 2.05T 3 0 180K 0
tank 3.39T 2.05T 83 0 7.47M 0
tank 3.39T 2.05T 4 2.80K 355K 106M
tank 3.39T 2.05T 5 229 263K 3.09M
tank 3.39T 2.05T 2 0 116K 0
tank 3.39T 2.05T 4 0 180K 0
tank 3.39T 2.05T 7 0 439K 0
tank 3.39T 2.05T 5 1.46K 447K 24.0M
tank 3.39T 2.05T 1 0 43.9K 0
tank 3.39T 2.05T 106 0 11.7M 0
tank 3.39T 2.05T 15 0 814K 0
tank 3.39T 2.05T 12 0 974K 0
tank 3.39T 2.05T 10 1.74K 675K 29.7M
tank 3.39T 2.05T 6 0 236K 0
tank 3.39T 2.05T 37 0 1.67M 0
tank 3.39T 2.05T 7 0 747K 0
tank 3.39T 2.05T 2 0 67.9K 0
tank 3.39T 2.05T 0 2.58K 0 90.9M
tank 3.39T 2.05T 19 1.38K 1.49M 20.6M
tank 3.39T 2.05T 1 0 140K 0
tank 3.39T 2.05T 0 0 43.9K 0
tank 3.39T 2.05T 3 0 164K 0
tank 3.39T 2.05T 61 3.36K 3.55M 78.4M
regards,=20
Kate Dawson
--=20
"The introduction of a coordinate system to geometry is an act of violence"
--KJY2Ze80yH5MUxol
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAEBCAAGBQJZ30YLAAoJEAGiBQHoGku6sHMP/1cowvRVr4ym7qD3VIhRO9lv
BPZFsnZ+9I2t2GfQx9RwIVGo+RcYzjxrMjaAYqh/Pi9/g7ZHCF8a10NVlPOMpa1y
f1F5CY5FdS+RAh2HngDtWfunec9fASPm4/sHGF88eey7RildUZKelK/y4+HiWCfp
8suJAiPKCpKJWlYsyen/PXQFPGts7URQ50GTC3jlnUIoa87u0O42nzinfh2JsywH
esUBt4hllv9BBZhSZvVga/ZggN0YP52gLC+BU4+88gSazuGTPBuZmFVPMbMjM92R
YHV7frdnvrvgD8bdUTSpYUOza69GiOSIjSDqy4WPUT67FiwzAsuPRoD3kJFChFsG
SmIAKVByAQYf8k6t1BuwwjZNnjpFMPtclYNX6l1XcYq0nPfe77ejDp8C5l5K9fQX
6INA5fQRRoAMjtvMS+zYBG4pUdVPG6EN/buQlcomKQXtv+YbQ7OLbCY584l/vWew
0Qnn1SxLJmGCUF8eocRZpO8mrWlUmclT3rmA1WgRRzOn/XAKWFZNDTCV/mo1iNfE
4jcmlUTNWsv8jAbkHAgwuyhGYQRNNTfXMDwbQT2DCE9ad+1JxA62ElcXcWPx6XhS
8n1O+ho6pDbzcGbfZmzsX1+h+eKAt2C/NUqf3VbAWZCASyIYqkQSYi5UuG3xP5Py
PAWVsFIKbBjhILEIoLmU
=/Ky4
-----END PGP SIGNATURE-----
--KJY2Ze80yH5MUxol--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20171012103809.GF24374>