From owner-freebsd-questions@freebsd.org Sat Oct 1 18:24:14 2016 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1765DA943AB for ; Sat, 1 Oct 2016 18:24:14 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "wonkity.com", Issuer "wonkity.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 44CF1ED4 for ; Sat, 1 Oct 2016 18:24:13 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.15.2/8.15.2) with ESMTPS id u91HdZpZ039311 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 1 Oct 2016 11:39:35 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.15.2/8.15.2/Submit) with ESMTP id u91HdZAC039308; Sat, 1 Oct 2016 11:39:35 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Sat, 1 Oct 2016 11:39:35 -0600 (MDT) From: Warren Block To: Ian Smith cc: David Christensen , Perry Hutchison , freebsd-questions@freebsd.org Subject: Re: FreeBSD-10.3-RELEASE-i386-memstick.img installer changes contents of USB flash drive In-Reply-To: <20161001235138.N6806@sola.nimnet.asn.au> Message-ID: References: <20160929014801.W6806@sola.nimnet.asn.au> <20161001235138.N6806@sola.nimnet.asn.au> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Sat, 01 Oct 2016 11:39:35 -0600 (MDT) X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Oct 2016 18:24:14 -0000 On Sun, 2 Oct 2016, Ian Smith wrote: > > A secondary copy of the partition table is saved at the end of a > > GPT-partitioned drive. This is a problem for GPT when a binary image is > > written to a device that is not exactly the same size as the image. That is > > pretty much always the case with installer images. > > Indeed. MBR/BSD images don't have this problem because all metadata is > at the beginning of and within the image in the case of UFS superblocks. > So an image can be copied to a device like /dev/da0, in the case of the > hitherto dedicated drive /dev/da0a, or as easily to a slice like da0s1 > where it becomes da0s1a, as long as there's enough room on the memstick. > > A GPT image seems more of a whole-disk image rather than (what equally > can serve as) a single partition image. It must have a primary GPT (all > 34 sectors?) but do you know if these images already contain the full 34 > sector secondary GPT? i.e, I don't see why only the last sector of the > copied image file was changed, with no update to any of the prior 33? Most of those 34 blocks (not a fixed size, by the way) contain partition information. But there are global flags and things like the PMBR which could change. Without knowing exactly what that byte does, it's a guess. Note that we don't know when that change is happening. It could be the machine checking the secondary GPT on the memory stick that is helpfully and silently "fixing" something when it recognizes the stick, before it is read. It could be malicious or buggy firmware in any of several places, including the UEFI or BIOS or the memory stick micontroller itself. The point is that we don't know that it is the FreeBSD installer making that change. > > gpart can repair > > ("recover") the backup partition table because it is a copy of the primary > > partition table. This might be required during an install to make sure the > > disk works with particularly strict BIOS or UEFI implementations. Although > > it seems like a reboot would be required afterwards. > > Hmm, well it's already booted from the memstick and installed the OS, so > we know the installer disk works in that environment. I can't see any > need to be modifying the source disk when it's doing / done its job? Agreed, but see above. > And then Perry Hutchinson (cc'd) is having the 'opposite' problem of > wanting to update the 10.3 i386 memstick he'd booted off with gpart > recover and gpart add for another partition, apparently successfully > according to gpart show, only to have the updates disappear on reboot? Pretty sure that's an unrelated problem I've seen myself: http://www.wonkity.com/~wblock/docs/html/labels.html If you change partition labels (or something) and then mount them, that information is lost. If you reboot first, something works right. I remember a few years back that someone helped me understand what was going on, but forget the details (and, unfortunately, who it was). > Personally I'm wanting to get to a memstick with 2 or 3 bootable slices > for installers (amd64 and i386) - as least the amd64 one from dvd1 with > packages - plus a mountable r/w data partition. Using only native tools > currently implies using boot0, thus MBR/BSD schema. Converting a new > GPT-scheme memstick image to ONE slice on my new 16GB memstick is what I > need to figure out, possibly by means of using a tar pipeline similar to > dvd1_to_memstick.sh. I hope to get time to look at 11.0 make release > memstick scripts, but I'd appreciate any clues / ideas meanwhile .. I'd think you could just dd(1) the contents of the single installer disk partition into a big-enough MBR partition. Repeat for each image, then use boot0cfg. It really is a shame we still require separate installers for amd64 and i386, a single installer can install either.