Date: Tue, 6 Oct 2015 10:18:11 -0500 From: Sean Kelly <smkelly@smkelly.org> To: FreeBSD-STABLE Mailing List <freebsd-stable@freebsd.org> Cc: Jim Harris <jim.harris@gmail.com> Subject: Dell NVMe issues Message-ID: <BC5F191D-FEB2-4ADC-9D6B-240C80B2301C@smkelly.org>
next in thread | raw e-mail | index | archive | help
Back in May, I posted about issues I was having with a Dell PE R630 with = 4x800GB NVMe SSDs. I would get kernel panics due to the inability to = assign all the interrupts because of = https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D199321 = <https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D199321>. Jim Harris = helped fix this issue so I bought several more of these servers, = Including ones with 4x1.6TB drives=E2=80=A6 while the new servers with 4x800GB drives still work, the ones with = 4x1.6TB drives do not. When I do a zpool create tank mirror nvd0 nvd1 mirror nvd2 nvd3 the command never returns and the kernel logs: nvme0: resetting controller nvme0: controller ready did not become 0 within 2000 ms I=E2=80=99ve tried several different things trying to understand where = the actual problem is. WORKS: dd if=3D/dev/nvd0 of=3D/dev/null bs=3D1m WORKS: dd if=3D/dev/zero of=3D/dev/nvd0 bs=3D1m WORKS: newfs /dev/nvd0 FAILS: zpool create tank mirror nvd[01] FAILS: gpart add -t freebsd-zfs nvd[01] && zpool create tank mirror = nvd[01]p1 FAILS: gpart add -t freebsd-zfs -s 1400g nvd[01[ && zpool create tank = nvd[01]p1 WORKS: gpart add -t freebsd-zfs -s 800g nvd[01] && zpool create tank = nvd[01]p1 NOTE: The above commands are more about getting the point across, not = validity. I wiped the disk clean between gpart attempts and used GPT. So it seems like zpool works if I don=E2=80=99t cross past ~800GB. But = other things like dd and newfs work. When I get the kernel messages about the controller resetting and then = not responding, the NVMe subsystem hangs entirely. Since my boot disks = are not NVMe, the system continues to work but no more NVMe stuff can be = done. Further, attempting to reboot hangs and I have to do a power = cycle. Any thoughts on what the deal may be here? 10.2-RELEASE-p5 nvme0@pci0:132:0:0: class=3D0x010802 card=3D0x1f971028 = chip=3D0xa820144d rev=3D0x03 hdr=3D0x00 vendor =3D 'Samsung Electronics Co Ltd' class =3D mass storage subclass =3D NVM --=20 Sean Kelly smkelly@smkelly.org http://smkelly.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BC5F191D-FEB2-4ADC-9D6B-240C80B2301C>