From owner-freebsd-stable@FreeBSD.ORG Sat Feb 11 06:13:06 2012 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 51F941065674 for ; Sat, 11 Feb 2012 06:13:06 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: from email2.allantgroup.com (email2.emsphone.com [199.67.51.116]) by mx1.freebsd.org (Postfix) with ESMTP id F20908FC17 for ; Sat, 11 Feb 2012 06:13:05 +0000 (UTC) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by email2.allantgroup.com (8.14.4/8.14.4) with ESMTP id q1B628di067497 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 11 Feb 2012 00:02:08 -0600 (CST) (envelope-from dan@dan.emsphone.com) Received: from dan.emsphone.com (smmsp@localhost [127.0.0.1]) by dan.emsphone.com (8.14.5/8.14.5) with ESMTP id q1B6282Z090762 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 11 Feb 2012 00:02:08 -0600 (CST) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.14.5/8.14.5/Submit) id q1B627Lp090751; Sat, 11 Feb 2012 00:02:07 -0600 (CST) (envelope-from dan) Date: Sat, 11 Feb 2012 00:02:07 -0600 From: Dan Nelson To: Randy Bush Message-ID: <20120211060207.GK5775@dan.emsphone.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-OS: FreeBSD 8.2-STABLE User-Agent: Mutt/1.5.21 (2010-09-15) X-Virus-Scanned: clamav-milter 0.97.2 at email2.allantgroup.com X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.6 (email2.allantgroup.com [199.67.51.78]); Sat, 11 Feb 2012 00:02:08 -0600 (CST) X-Scanned-By: MIMEDefang 2.68 on 199.67.51.78 Cc: FreeBSD Stable Subject: Re: 9-stable from i386 to amd64 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Feb 2012 06:13:06 -0000 In the last episode (Feb 10), Randy Bush said: > is there a recipe for moving from i386 to amd64? > > on a very remote system, i made the migration from 7.4 to 8.2 to 9.0, all > 32-bit. it was done with repeated > > make buildworld > make kernel.new [0] > nextboot -k kernel.new > reboot > make installworld > etc > > [0] - well, there were some mv(1)s in there :) > > so after it was happy with 9.0 i386, i went to move to amd64 with > > make buildworld TARGET=amd64 > make kernel TARGET=amd64 DESTDIR=kernel.new [0] > nextboot -k kernel.new > reboot > > it did not come back from the reboot, and required a manual reset. i have > no console access to the machine, not my choice. > > clue bat please. You probably got bit by a mismatched /libexec/ld-elf.so. The kernel expects that to be the "native" version, and on a 64-bit kernel it also expects a ld-elf32.so to be the "compat" 32-bit version. When you rebooted onto the 64-bit kernel, it couldn't find /libexec/ld-elf32.so to run any of the 32-bit binaries on the system. My guess is that your reboot attempt died in /sbin/init, prompting for a path to /bin/sh. If you compiled with a static /bin/sh for performance, it probably died very early in /etc/rc. I think copying ld-elf.so over to ld-elf32.so might have been all you needed to boot, but that would end up with a 64-bit kernel running a true 32-bit userland with all the libraries in the "wrong" place, and your "installworld" step would replace them with their 64-bit equivalents and your install would die halfway through, leaving you with a large mess to clean up. The cleanest upgrade path is to prepare your 32-bit root to be bootable by both 32- and 64-bit kernels: copy the ld-elf32.so that was built during your buildworld over to /libexec/ld-elf32.so, and also make copies of /lib and /usr/lib to /lib32 and /usr/lib32 respectively. That way when you reboot to a 64-bit kernel, your 32-bit executables will be running "correctly" out of compat32 paths and your installworld should succeed. When I did all this on a local system, I made judicious use of ZFS snapshots and clones, preserving a bootable clone of my original system plus intermediate versions all the way until I was happy with the result. I've never done it completely remotely, but if you do a trial run or two on a local machine or VM, you should be able to it confidently remotely. -- Dan Nelson dnelson@allantgroup.com