Date: Wed, 6 Apr 2022 14:57:14 +0000 From: John F Carr <jfc@mit.edu> To: "egoitz@ramattack.net" <egoitz@ramattack.net> Cc: "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org> Subject: Re: Desperate with 870 QVO and ZFS Message-ID: <D7E66B1E-C036-4362-AB69-3875C8F05E83@exchange.mit.edu> In-Reply-To: <4e98275152e23141eae40dbe7ba5571f@ramattack.net> References: <4e98275152e23141eae40dbe7ba5571f@ramattack.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Apr 6, 2022, at 07:15 , egoitz@ramattack.net wrote: >=20 > Good morning, >=20 > I write this post with the expectation that perhaps someone could help me= <d8974688.gif> >=20 > I am running some mail servers with FreeBSD and ZFS. They use 870 QVO (no= t EVO or other Samsung SSD disks) disks as storage. They can easily have fr= om 1500 to 2000 concurrent connections. The machines have 128GB of ram and = the CPU is almost absolutely idle. The disk IO is normally at 30 or 40% per= cent at most. >=20 > The problem I'm facing is that they could be running just fine and sudden= ly at some peak hour, the IO goes to 60 or 70% and the machine becomes extr= emely slow. ZFS is all by default, except the sync parameter which is set d= isabled. Apart from that the ARC is limited to 64GB. But even this is extre= mely odd. The used ARC is near 20GB. I have seen, that meta cache in arc is= very near to the limit that FreeBSD automatically sets depending on the si= ze of the ARC you set. It seems that almost all ARC is used by meta cache. = I have seen this effect in all my mail servers with this hardware and softw= are config. >=20 My system with nvd0: <Samsung SSD 970 EVO 1TB> NVMe namespace has a problem with high write volume. If I build llvm with debugging symbo= ls, which writes about 70 GB, the filesystem nearly grinds to a halt. I ha= ve to use a spinning disk to get decent performance on this workload. Ther= e is some old talk on the mailing lists about certain drives handling TRIM = commands badly. See comment by Ted Ts'o here: https://forums.freebsd.org/t= hreads/ssd-trim-maintenance.56951/ Unfortunately the documentation for adjusting trim settings is out of date.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?D7E66B1E-C036-4362-AB69-3875C8F05E83>