From owner-freebsd-questions@freebsd.org Tue May 14 20:33:45 2019 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B5DD15980A0 for ; Tue, 14 May 2019 20:33:45 +0000 (UTC) (envelope-from matthias@smormegpa.no) Received: from mailrelay2-1.pub.mailoutpod1-cph3.one.com (mailrelay2-1.pub.mailoutpod1-cph3.one.com [46.30.210.183]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 78F4A839BF for ; Tue, 14 May 2019 20:33:42 +0000 (UTC) (envelope-from matthias@smormegpa.no) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smormegpa.no; s=20140924; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=/YRDLNJkKscTnXY0HLSKjbtqowWwwL9pvWFaHvnY0YE=; b=MxvmJR/hw0Ie6nX3omscBRzDphO5XJmwonP09ebkgs/RCncHlbKew7wYMABCZLRWGCuxlUZjx4tCB d1EBVC736TOwdq3FWVlJuzYGPLWKfwIHrYzX6NO57GZr7DCvC40siyT2awbGmYvVSGtN7XdCWr2r66 leaquXYUPdQOL1nI= X-HalOne-Cookie: 7d8fc7d6fcce33a7b35e3d3bd926443ded8021ea X-HalOne-ID: 4c9d0593-7685-11e9-81c1-d0431ea8a290 Received: from picadelly.monsieur.mathieu (unknown [85.166.11.253]) by mailrelay2.pub.mailoutpod1-cph3.one.com (Halon) with ESMTPSA id 4c9d0593-7685-11e9-81c1-d0431ea8a290; Tue, 14 May 2019 20:17:30 +0000 (UTC) Message-ID: Subject: Re: Suggestions for working with unstable nvme dev names in AWS From: Matthias Oestreicher To: hartzell@alerce.com, Polytropon Cc: freebsd-questions@freebsd.org Date: Tue, 14 May 2019 22:17:40 +0200 In-Reply-To: <23771.5612.105696.170743@alice.local> References: <23770.10599.687213.86492@alice.local> <08660a2a-489f-8172-22ee-47aeba315986@FreeBSD.org> <23770.58821.826610.399467@alice.local> <20190514210203.3d951fb8.freebsd@edvax.de> <23771.5612.105696.170743@alice.local> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 78F4A839BF X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=smormegpa.no header.s=20140924 header.b=MxvmJR/h X-Spamd-Result: default: False [-0.32 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[smormegpa.no:s=20140924]; NEURAL_HAM_MEDIUM(-0.64)[-0.637,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-0.92)[-0.919,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[smormegpa.no]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.67)[0.670,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[smormegpa.no:+]; MX_GOOD(-0.01)[mx1.pub.mailpod3-cph3.one.com,mx2.pub.mailpod3-cph3.one.com,mx3.pub.mailpod3-cph3.one.com]; RCVD_IN_DNSWL_NONE(0.00)[183.210.30.46.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(0.38)[ipnet: 46.30.208.0/21(1.09), asn: 51468(0.82), country: DK(-0.02)]; ASN(0.00)[asn:51468, ipnet:46.30.208.0/21, country:DK]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 May 2019 20:33:45 -0000 Am Dienstag, den 14.05.2019, 12:24 -0700 schrieb George Hartzell: > Polytropon writes: > > On Tue, 14 May 2019 08:59:01 -0700, George Hartzell wrote: > > > Matthew Seaman writes: > > > > [...] but if you > > > > are using ZFS, then shuffling the disks around should not make any > > > > difference. > > > > [...] > > > Yes, once I have them set up (ZFS or labeled), it doesn't matter what > > > device names they end up having. For now I just do the setup by hand, > > > poking around a bit. Same trick in the Linux world, you end up > > > referring to them by their UUID or .... > > > > In addition to what Matthew suggested, you could use UFS-IDs > > in case the disks are initialized with UFS. You can find more > > information here (at the bottom of the page): > > [...] > > Yes. As I mentioned in my response to Matthew, once I have some sort > of filesystem/zpool on the device, it's straightforward (TMTOWTDI). > > The problem is being able to provision the system automatically > without user intervention. > > In the Linux world, I can use e.g. Terraform to set up a pair of > additional volumes and tell it to call them `/dev/sdy` and `/dev/sdz`. > The Linux magic happens and I get pair of symlinks that I can use in > my e.g. Ansible playbooks, that point to whatever the devices came up > as when it booted. I build filesystems on the devices, add them via > their UUID's to `/etc/fstab` and I'm off and running. > > I can't [seem to] do this in the FreeBSD world; even if I name the > devices `/dev/nvme1` (the fast and big one) and `/dev/nvme2` (the slow > and small one), there's no guarantee that they'll have those names > when the machine boots. > > This is a weirdly AWS issue and their peace offering is to stash the > requested device name in the device/controller/"hardware" and provide > a tool that digs it out. > > I'm trying to figure out what I can do about it from FreeBSD. Perhaps > there's already a solution. Perhaps the nvme driver needs to be > extended to provide access to the magic AWS info stash and then > something like Amazon Linux's `ebsnvme-id` can pry it out. > > g. Hei, I'm not familiar with Amazon's AWS, but if your only problem is shiftig device names for UFS filesystems, then on modern systems, GPT labels is the way to go. There has been a lot of confusion over the years, about the many ways to apply different types of labels to devices on FreeBSD, but really GEOM labels, UUIDs, etc, are only useful on old systems where there's no support for GPT. GPT labels are only applied to partitions, not whole drives, but they are extremely flexible. They can be applied and changed at any time, even on mounted filesystems. In comparison to GEOM labels and all other ID types, they will never be hidden if the devices original device name (like nvm0 or nvm1) is in use. At any time will 'gpart show -l' show the GPT labels you applied, and they can be used to manually mount and in /etc/fstab. I have never used any other labels for years and even disables all others in /boot/loader.conf kern.geom.label.disk_ident.enable=0 kern.geom.label.gptid.enable=0 kern.geom.label.ufsid.enable=0 You can apply a GPT label with # gpart modify -l mylabel -i N /dev/nvm1 and then add something like the following to /etc/fstab /dev/gpt/mylabel / ufs rw 1 1 There is only a single limitation with GPT labels and that is they don't work when you use UFS journaling via GEOM, as the GPT label will be the same for e.g /dev/nvm0p1 and /dev/nvm0p1.journal. Another big plus is, they work with every partition type, freebsd-ufs, freebsd-boot, swap, EFI, freebsd-zfs... One label type for everything can avoid some headache imo. Hope that clears up some confusion. Matthias > _______________________________________________ > freebsd-questions@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org"