Date: Tue, 6 May 2025 13:31:23 -0400 From: Charles Sprickman <spork@bway.net> To: Colin Percival <cperciva@tarsnap.com> Cc: freebsd-stable@freebsd.org Subject: Re: NVMe (U.2) hot-swap support status? Message-ID: <5530E3D8-B2F8-4101-91B1-D8BAC17D2FDA@bway.net> In-Reply-To: <01000196a3fb8737-9e591147-f4fd-439d-85bc-5f155be4bebb-000000@email.amazonses.com> References: <2E8A7BC6-6C3F-431F-B6F0-2611577B8028@bway.net> <01000196a3fb8737-9e591147-f4fd-439d-85bc-5f155be4bebb-000000@email.amazonses.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> On May 6, 2025, at 1:04=E2=80=AFAM, Colin Percival = <cperciva@tarsnap.com> wrote: >=20 > On 5/5/25 21:54, Charles Sprickman wrote: >> Anyhow, we're trying purchase a few servers (likely this Supermicro: = https://www.supermicro.com/en/products/system/UP/1U/SYS-111E-WR) and = since manufacturers don't seem to validate for FreeBSD these days, I was = hoping for some community input. >> Last time I looked into this NVMe (specifically the "U.2" format for = servers that includes hot swap) was supported in FreeBSD but hot swap = was kind of iffy. I don't think too many FreeBSD users/developers at = that time had the hardware, most people seemed to just be using an = on-board NVMe drive on desktop or laptop systems, and there's not any = real call for hot swap in that segment. >> I have a thread on the forums linked below, but really what I'm = looking to find is some larger orgs that are running NVMe drives and can = confirm hot swap works as expected, if there's any gotchas to be on the = lookout for, etc. Trying not to overcomplicate it, but we don't want to = be stuck with a bunch of servers that might be less stable than our = current SAS/SATA servers. We exclusively use ZFS, if that's useful info. >=20 > I can't speak to *physical* hotplug, but I've put a lot of energy over = the > past few months into making sure that nvme "hotplug" is 100% = functional in > Amazon EC2. If you run into problems it's probably going to be due to > broken firmware, [...] Can you explain a bit here if you've got a few minutes?=20 I'm used to the old world concept of IDE/SCSI/SAS controllers (esp. RAID = controllers) where you've got a lot of processing going on in the card = to go from the drive interface to whatever kind of bus the controller = sits on in the PC. But in the case of NVMe, what even is a "controller"? = Is it not just a card that's combining/arbitrating PCI-e lanes from a = bunch of drives to the PCI-e bus? Where would be the firmware and what = would be the compatibility issues? I admit I don't use any NVMe stuff = anywhere, so I'm not very familiar with it, and especially not with this = setup where we have a "controller" involved... Thanks, Charles [...] > so unless anyone has experience with that specific server > I think the best answer you can get is "it should work but you need to = test > it and find out". >=20 > --=20 > Colin Percival > FreeBSD Release Engineering Lead & EC2 platform maintainer > Founder, Tarsnap | www.tarsnap.com | Online backups for the truly = paranoid >=20 >=20
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5530E3D8-B2F8-4101-91B1-D8BAC17D2FDA>