Date: Fri, 5 Apr 2019 09:33:15 +0200 From: "Patrick M. Hausen" <hausen@punkt.de> To: Warner Losh <imp@bsdimp.com> Cc: freebsd-stable@freebsd.org Subject: Re: NVME aborting outstanding i/o Message-ID: <58E4FC01-D154-42D4-BA0F-EF9A2C60DBF7@punkt.de> In-Reply-To: <CANCZdfoPZ9ViQzZ2k8GT5pNw5hjso3rzmYxzU=s%2B3K=ze%2BLZwg@mail.gmail.com> References: <818CF16A-D71C-47C0-8A1B-35C9D8F68F4E@punkt.de> <CF2365AE-23EA-4F18-9520-C998216155D5@punkt.de> <CANCZdfoPZ9ViQzZ2k8GT5pNw5hjso3rzmYxzU=s%2B3K=ze%2BLZwg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi all, > Am 04.04.2019 um 17:11 schrieb Warner Losh <imp@bsdimp.com>: > There's a request that was sent down to the drive. It took longer than 30s to respond. One of them, at least, was a trim request. > […] Thanks for the explanation. This further explains why I was seeing a lot more of those and the system occasionally froze for a couple of seconds after I increased these: vfs.zfs.vdev.async_write_max_active: 10 vfs.zfs.vdev.async_read_max_active: 3 vfs.zfs.vdev.sync_write_max_active: 10 vfs.zfs.vdev.sync_read_max_active: 10 as recommended by Allan Jude reasoning that NVME devices could work on up to 64 requests in parallel. I have since reverted that change and I am running with the defaults. If I understand correctly, this: > hw.nvme.per_cpu_io_queues=0 essentially limits the rate at which the system throws commands at the devices. Correct? So it’s not a real fix and there’s nothing fundamentally wrong with the per CPU queue or interrupt implementation. I will look into new firmware for my Intel devices and try tweaking the vfs.zfs.vdev.trim_max_active and related parameters. Out of curiosity: what happens if I disable TRIM? My knowledge is rather superficial and I just filed that under „TRIM is absolutely essential lest performance will suffer severely and your devices die - plus bad karma, of course …“ ;-) Kind regards, Patrick -- punkt.de GmbH Internet - Dienstleistungen - Beratung Kaiserallee 13a Tel.: 0721 9109-0 Fax: -100 76133 Karlsruhe info@punkt.de http://punkt.de AG Mannheim 108285 Gf: Juergen Egeling
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?58E4FC01-D154-42D4-BA0F-EF9A2C60DBF7>