From owner-freebsd-arch@FreeBSD.ORG Sun Mar 13 07:38:17 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 12793106566B; Sun, 13 Mar 2011 07:38:17 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 28EDE8FC12; Sun, 13 Mar 2011 07:38:15 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id JAA05203; Sun, 13 Mar 2011 09:20:05 +0200 (EET) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Pyfav-000082-4T; Sun, 13 Mar 2011 09:20:05 +0200 Message-ID: <4D7C7024.3030506@freebsd.org> Date: Sun, 13 Mar 2011 09:20:04 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110308 Lightning/1.0b2 Thunderbird/3.1.9 MIME-Version: 1.0 To: Nathan Whitehorn X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org Subject: kernel with DDB for installation/live ISOs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Mar 2011 07:38:17 -0000 Nathan, since you are working on a new installer and changing some things about what installation image is and how release is made, I would like to propose/discuss the following change. What about adding option DDB to the kernels booted from installation images? While adding DDB to installed kernels (GENERIC) would/could be a POLA violation and a potential security issue, having DDB in installation kernels would help a lot with debugging issues where users can't even get installation system to boot. Thank you for your work! -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Sun Mar 13 13:13:40 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0C013106566C; Sun, 13 Mar 2011 13:13:40 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id B401C8FC0A; Sun, 13 Mar 2011 13:13:39 +0000 (UTC) Received: from outgoing.leidinger.net (p5B1539CA.dip.t-dialin.net [91.21.57.202]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 271F284400E; Sun, 13 Mar 2011 14:13:35 +0100 (CET) Received: from unknown (IO.Leidinger.net [192.168.2.110]) by outgoing.leidinger.net (Postfix) with ESMTP id DFF9F2AA9; Sun, 13 Mar 2011 14:13:31 +0100 (CET) Date: Sun, 13 Mar 2011 14:13:30 +0100 From: Alexander Leidinger To: Andriy Gapon Message-ID: <20110313141330.000016a2@unknown> In-Reply-To: <4D7C6DFC.8020504@freebsd.org> References: <201103120909.p2C99P2j010783@svn.freebsd.org> <20110312211423.00000f34@unknown> <4D7C6DFC.8020504@freebsd.org> X-Mailer: Claws Mail 3.7.6 (GTK+ 2.16.0; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 271F284400E.A1663 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=0.317, required 6, autolearn=disabled, ALL_TRUSTED -1.00, SARE_LWSHORTT 1.24, TW_SV 0.08) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1300626816.35631@8w0DlGD8BuxBvSkhLNwqfA X-EBL-Spam-Status: No Cc: emulation@freebsd.org, arch@freebsd.org, Artem Belevich Subject: linux or linux32 on i386 (was: Re: svn commit: r219561 - in head/sys: cddl/dev/systrace modules/dtrace modules/dtrace/dtraceall modules/dtrace/systrace_freebsd32 modules/dtrace/systrace_linux32) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Mar 2011 13:13:40 -0000 On Sun, 13 Mar 2011 09:10:52 +0200 Andriy Gapon wrote: > on 13/03/2011 08:35 Artem Belevich said the following: > > On Sat, Mar 12, 2011 at 12:14 PM, Alexander Leidinger > > wrote: > >> But this looks like it will be named linux32 in any case. In the > >> short term I would prefer: > >> ---snip--- > >> #if defined(__amd64__) > >> #define MODNAME "linux32" > >> #elif defined(__i386__) > >> #define MODNAME "linux" > >> #endif > >> ---snip--- > > > > Makes sense. It's what's done with freebsd syscalls -- 'freebsd' for > > native syscalls, freebsd32 for 32-bit compat. > > BTW, in my opinion, it might not make a lot of sense. > That is, we can have native and compat FreeBSD system calls, but > Linux system calls are always compat and never native, whether the > emulation is for the same arch or not. Explicit suffix makes things > clearer. But that's just my opinion. So far we use sys/i386/linux and sys/amd64/linux32 as the directory structure. When the native linux code for amd64 arrives I would expect it is named linux and not linux64. On i386 we also do not refer to linux32 anywhere. It may not be the native ABI, but it is the native code size. I also doubt anyone would expect to see a linux64 on i386. IMO it is too late for a explicit suffix (this would be the onl place which is using it, all other places use plain linux and it would be a lot of changes necessary to change this) and it would be more consistent to use linux instead of linux32 on i386. > A public mailing list may be a better place to discuss this further. I switched to arch and emulation. Bye, Alexander. From owner-freebsd-arch@FreeBSD.ORG Mon Mar 14 14:13:34 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1EC731065670; Mon, 14 Mar 2011 14:13:34 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from argol.doit.wisc.edu (argol.doit.wisc.edu [144.92.197.212]) by mx1.freebsd.org (Postfix) with ESMTP id E61A08FC13; Mon, 14 Mar 2011 14:13:33 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from avs-daemon.smtpauth3.wiscmail.wisc.edu by smtpauth3.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) id <0LI100D00WUK6G00@smtpauth3.wiscmail.wisc.edu>; Mon, 14 Mar 2011 09:13:32 -0500 (CDT) Received: from comporellon.tachypleus.net ([unknown] [76.210.65.155]) by smtpauth3.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) with ESMTPSA id <0LI1007W1WUI2O20@smtpauth3.wiscmail.wisc.edu>; Mon, 14 Mar 2011 09:13:32 -0500 (CDT) Date: Mon, 14 Mar 2011 09:13:30 -0500 From: Nathan Whitehorn To: freebsd-current Current , freebsd-sysinstall@freebsd.org, FreeBSD Arch Message-id: <4D7E228A.4090906@freebsd.org> X-Spam-Report: AuthenticatedSender=yes, SenderIP=76.210.65.155 X-Spam-PmxInfo: Server=avs-13, Version=5.6.0.2009776, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2011.3.14.140017, SenderIP=76.210.65.155 User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110305 Thunderbird/3.1.9 Cc: Subject: HEADS UP: sysinstall is no longer the default installer X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Mar 2011 14:13:34 -0000 I just committed (r219641) changes that make the release infrastructure (src/release/Makefile) use bsdinstall by default instead of sysinstall on install media. A big thank you is in order to everyone who provided advice, criticism, and testing for this project over the last few months! Along with sysinstall, the original sysinstall build stuff has been preserved (now /usr/src/release/Makefile.sysinstall) and will continue to be for the lifetime of the 9.x release series, although it will not be used by default. This change modifies the process of building releases somewhat, so I'll outline changes that people who run snapshot buildbots will have to make below, and some next steps planned with the installer. Changes to release(7) ----------------------------- Release builds work and look slightly different now, so everyone who snapshot tinderboxes will likely find them breaking shortly. The nearest analog to the old make release (with version-control checkouts and a chroot) is src/release/generate-release.sh, which can be run as generate-release.sh head /path/to/chroot/dir. If you want to include ports and documentation on the release media, CVSUP_HOST must be defined in the environment to point to a cvsup mirror. The output is placed in /R in the chroot directory, as before. If the chroot is unimportant (it ensures a total clean-room build, but may not be necessary in most cases), you can get a release build using the regular makefile, like so: cd /usr/src make buildworld buildkernel cd /usr/src/release make obj release By default, this will include ports and documentation if you have them checked out to /usr/ports and /usr/doc, though this behavior can be modified (see the top of the makefile). In addition, some architectures (i386, amd64, powerpc, powerpc64, and maybe ia64) have release media that can be cross-built, so you can set TARGET/TARGET_ARCH appropriately for those. Output goes to .OBJDIR, which is /usr/obj/usr/src/release in the case of the above commands. The equivalent to disc1 is called release.iso, the memstick image (i386, amd64 only) is called memstick, and a directory of distfiles for FTP mirrors is generated named ftp. Next steps -------------- The new installer is feature-complete at this time (except for a merge with the pc-sysinstall code base and the possible addition of ZFS support in the partition editor), so the next steps mostly involve documentation updates to manpages and the handbook. Generation of a bootonly ISO is another thing that should happen soon. Given time (or external patches), I would also like to update sysinstall to use the new-style distribution files so it can be an option on the 9.0 install CDs. Beyond that, please test this code as much as possible, and report any bugs, suspicious behaviors, or bad interfaces (or patch them -- patches for anything are always very welcome!). We have another several months before 9.0, so let's try to find all the bugs long before then. Thanks again to everyone who helped this project with comments and testing, especially to Bjoern Zeeb who got me irritated enough by sysinstall to start working on this project. -Nathan From owner-freebsd-arch@FreeBSD.ORG Mon Mar 14 15:44:34 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B5939106564A; Mon, 14 Mar 2011 15:44:34 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 8AC268FC12; Mon, 14 Mar 2011 15:44:34 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 3E47046B8F; Mon, 14 Mar 2011 11:44:34 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.10]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id CCE928A027; Mon, 14 Mar 2011 11:44:33 -0400 (EDT) From: John Baldwin To: freebsd-current@freebsd.org Date: Mon, 14 Mar 2011 11:44:32 -0400 User-Agent: KMail/1.13.5 (FreeBSD/7.4-CBSD-20110107; KDE/4.4.5; amd64; ; ) References: <4D7E228A.4090906@freebsd.org> In-Reply-To: <4D7E228A.4090906@freebsd.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201103141144.32815.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Mon, 14 Mar 2011 11:44:33 -0400 (EDT) Cc: FreeBSD Arch , Nathan Whitehorn , freebsd-sysinstall@freebsd.org Subject: Re: HEADS UP: sysinstall is no longer the default installer X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Mar 2011 15:44:34 -0000 On Monday, March 14, 2011 10:13:30 am Nathan Whitehorn wrote: > I just committed (r219641) changes that make the release infrastructure > (src/release/Makefile) use bsdinstall by default instead of sysinstall > on install media. A big thank you is in order to everyone who provided > advice, criticism, and testing for this project over the last few months! > > Along with sysinstall, the original sysinstall build stuff has been > preserved (now /usr/src/release/Makefile.sysinstall) and will continue > to be for the lifetime of the 9.x release series, although it will not > be used by default. This change modifies the process of building > releases somewhat, so I'll outline changes that people who run snapshot > buildbots will have to make below, and some next steps planned with the > installer. Please consider supporting using SVN or CVS to obtain docs, ports, and source trees. I have a custom SVN repo at work that is not exported to CVS and available via csup and am able to use the existing SVNROOT SVNBRANCH variables with 'make release'. Having support for this sort of thing would be useful. I have also made much use of LOCAL_PATCHES in the past for building releases, so having support for that would be useful as well. I think for re@ especially it is nice to just do 'make release TAG=7.2' (or some such) and have it DTRT to check out matching ports, doc, and src into the chroot, etc. I think the new process should be similarly automated. Have you tested network installs using PXE or the like? This was fairly easy before as you could copy the '/boot' directory from a bootable ISO and the mfsroot was self-contained. Do you now have to put the entire contents of release.iso up via NFS? Is there a subset you put in the NFS root and then do an NFS or FTP install? -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Mar 14 15:56:17 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E2B581065670; Mon, 14 Mar 2011 15:56:17 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from argol.doit.wisc.edu (argol.doit.wisc.edu [144.92.197.212]) by mx1.freebsd.org (Postfix) with ESMTP id B048F8FC0A; Mon, 14 Mar 2011 15:56:17 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from avs-daemon.smtpauth3.wiscmail.wisc.edu by smtpauth3.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) id <0LI200M001LS6M00@smtpauth3.wiscmail.wisc.edu>; Mon, 14 Mar 2011 10:56:16 -0500 (CDT) Received: from comporellon.tachypleus.net ([unknown] [76.210.65.155]) by smtpauth3.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) with ESMTPSA id <0LI200HZA1LQRB20@smtpauth3.wiscmail.wisc.edu>; Mon, 14 Mar 2011 10:56:16 -0500 (CDT) Date: Mon, 14 Mar 2011 10:56:14 -0500 From: Nathan Whitehorn In-reply-to: <201103141144.32815.jhb@freebsd.org> To: John Baldwin Message-id: <4D7E3A9E.10800@freebsd.org> X-Spam-Report: AuthenticatedSender=yes, SenderIP=76.210.65.155 X-Spam-PmxInfo: Server=avs-14, Version=5.6.0.2009776, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2011.3.14.154539, SenderIP=76.210.65.155 References: <4D7E228A.4090906@freebsd.org> <201103141144.32815.jhb@freebsd.org> User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110305 Thunderbird/3.1.9 Cc: freebsd-current@freebsd.org, freebsd-sysinstall@freebsd.org, FreeBSD Arch Subject: Re: HEADS UP: sysinstall is no longer the default installer X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Mar 2011 15:56:18 -0000 On 03/14/11 10:44, John Baldwin wrote: > On Monday, March 14, 2011 10:13:30 am Nathan Whitehorn wrote: >> I just committed (r219641) changes that make the release infrastructure >> (src/release/Makefile) use bsdinstall by default instead of sysinstall >> on install media. A big thank you is in order to everyone who provided >> advice, criticism, and testing for this project over the last few months! >> >> Along with sysinstall, the original sysinstall build stuff has been >> preserved (now /usr/src/release/Makefile.sysinstall) and will continue >> to be for the lifetime of the 9.x release series, although it will not >> be used by default. This change modifies the process of building >> releases somewhat, so I'll outline changes that people who run snapshot >> buildbots will have to make below, and some next steps planned with the >> installer. > Please consider supporting using SVN or CVS to obtain docs, ports, and source > trees. I have a custom SVN repo at work that is not exported to CVS and > available via csup and am able to use the existing SVNROOT SVNBRANCH variables > with 'make release'. Having support for this sort of thing would be useful. > I have also made much use of LOCAL_PATCHES in the past for building releases, > so having support for that would be useful as well. SVNBRANCH works now, and source comes over SVN, the others via cvsup. Support for a different SVNROOT and regular cvs for ports and docs can certainly be added. In the case of LOCAL_PATCHES, you can just use the regular makefile on your patched tree -- I don't think the chroot and checkouts make much sense in this case. > I think for re@ especially it is nice to just do 'make release TAG=7.2' (or > some such) and have it DTRT to check out matching ports, doc, and src into the > chroot, etc. I think the new process should be similarly automated. The generate-release.sh script likely needs some work. It exists almost purely for the benefit of re@, and I don't know exactly what their requirements are. A list (or patches) would be very welcome. The feature you want here, though, can be obtained now by the CVSUP_TAG and svn branch arguments to generate-release.sh. > Have you tested network installs using PXE or the like? This was fairly easy > before as you could copy the '/boot' directory from a bootable ISO and the > mfsroot was self-contained. Do you now have to put the entire contents of > release.iso up via NFS? Is there a subset you put in the NFS root and then do > an NFS or FTP install? > Yes, I have, and it works well (tested on i386, sparc64, and powerpc). Right now, you need the whole system (which is a regular installworld + the rc.local to give the installer menu, and, optionally the distfiles). For the future, the set of things the installer needs from the userland is intentionally fairly small. I need to do some work anyway to make a minimal system for bootonly CDs and the like, which should also a smaller system for PXE as well. -Nathan From owner-freebsd-arch@FreeBSD.ORG Mon Mar 14 16:06:56 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 87C7E106564A; Mon, 14 Mar 2011 16:06:56 +0000 (UTC) (envelope-from gkeramidas@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 033528FC1B; Mon, 14 Mar 2011 16:06:55 +0000 (UTC) Received: by gwb15 with SMTP id 15so2142372gwb.13 for ; Mon, 14 Mar 2011 09:06:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=AjyZVvpTnzOoCjk5gMTbmfxlZR6dynwnPh9hese2XAQ=; b=HEWe/0ewyVtYeyk5GJU01hJHLXTI9k6ohs305TPG2qTi698ttEB9ZYomnXMJ4hXaDy Hhf8Yh61nzo9sAytFdSFeZuIHxWzWPO4a90qy0iFXjte5Qff6zb48EQLQfjrAulx7DfB JgPgJ/cb8C6jOL5ySk6DvqECHBchLSpHHh6Ws= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=nqVjxvD0AUWqRTp4sPBvnIrLNUzQ6JYQ8VC98C1YENgR2F2eGg/YrubfZlPC3dIXFY uPVKE+DjaNWPD3trKqz1VwlLzh484MEOoRaTpy7XbR8rKu0yH35BOnxSbUetqiREVAoN DKmHV5lr/wZCEgfHdpdshU4HI2BAWPqAqvkaM= MIME-Version: 1.0 Received: by 10.147.58.20 with SMTP id l20mr9854196yak.26.1300117086639; Mon, 14 Mar 2011 08:38:06 -0700 (PDT) Sender: gkeramidas@gmail.com Received: by 10.147.181.12 with HTTP; Mon, 14 Mar 2011 08:38:05 -0700 (PDT) In-Reply-To: <4D7E228A.4090906@freebsd.org> References: <4D7E228A.4090906@freebsd.org> Date: Mon, 14 Mar 2011 16:38:05 +0100 X-Google-Sender-Auth: eS4jBgbfHjjgiz4Up3bKCeLHgMg Message-ID: From: Giorgos Keramidas To: Nathan Whitehorn Content-Type: text/plain; charset=UTF-8 Cc: freebsd-current Current , freebsd-sysinstall@freebsd.org, FreeBSD Arch Subject: Re: HEADS UP: sysinstall is no longer the default installer X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Mar 2011 16:06:56 -0000 On Mon, Mar 14, 2011 at 3:13 PM, Nathan Whitehorn wrote: > Changes to release(7) > ----------------------------- > > Release builds work and look slightly different now, so everyone who > snapshot tinderboxes will likely find them breaking shortly. The nearest > analog to the old make release (with version-control checkouts and a chroot) > is src/release/generate-release.sh, which can be run as generate-release.sh > head /path/to/chroot/dir. If you want to include ports and documentation on > the release media, CVSUP_HOST must be defined in the environment to point to > a cvsup mirror. The output is placed in /R in the chroot directory, as > before. > > If the chroot is unimportant (it ensures a total clean-room build, but may > not be necessary in most cases), you can get a release build using the > regular makefile, like so: > cd /usr/src > make buildworld buildkernel > cd /usr/src/release > make obj release > > By default, this will include ports and documentation if you have them > checked out to /usr/ports and /usr/doc, though this behavior can be modified > (see the top of the makefile). In addition, some architectures (i386, amd64, > powerpc, powerpc64, and maybe ia64) have release media that can be > cross-built, so you can set TARGET/TARGET_ARCH appropriately for those. > Output goes to .OBJDIR, which is /usr/obj/usr/src/release in the case of the > above commands. The equivalent to disc1 is called release.iso, the memstick > image (i386, amd64 only) is called memstick, and a directory of distfiles > for FTP mirrors is generated named ftp. Any "user interface" changes that affect the release.7 manpage and may catch people building their own release images should be updated in the manpage itself too. Some of the stuff I'd like to see fixed in the release.7 manpage are: - The requirement for CVSUP_HOST should be explicitly mentioned in release.7 for releases that have NODOC. Note: I haven't run a release with the new Makefile yet, but is it still possible to use a local CVS mirror, e.g. /home/ncvs for these files instead of a cvsup host that is only accessible over the (potentially much slower) network? - The make variable ${DATE} is automatically set to the build date. We should probably mention this in the default BUILDNAME description (since it's such a generic variable name). - BUILDNAME is automatically set to a default that may have to be documented to the manpage, so that people know what to expect when they type just "make release" and sit back. - There's a ${BASE} variable set to 9.0 that release engineers may have to manually update when they roll-out release and stable branches. This should be documented in the "shortly before the release" checklist we have in http://www.freebsd.org/doc/en_US.ISO8859-1/articles/releng/release-proc.html - The directories that "make release" creates, and the names of the ISO image files should be mentioned in release.7 now that we have a chance to make a batch of useful updates to the text. Naturally, I volunteer to *make* the mdoc changes. As long as someone (e.g. you Nathan?) who is acquainted with the new release building Makefile can hepl me by reviewing the updates and making sure they look reasonably close to the new state of everything. From owner-freebsd-arch@FreeBSD.ORG Mon Mar 14 16:38:58 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A506B106566B; Mon, 14 Mar 2011 16:38:58 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from argol.doit.wisc.edu (argol.doit.wisc.edu [144.92.197.212]) by mx1.freebsd.org (Postfix) with ESMTP id 70EBE8FC12; Mon, 14 Mar 2011 16:38:58 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from avs-daemon.smtpauth3.wiscmail.wisc.edu by smtpauth3.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) id <0LI2002083KXTL00@smtpauth3.wiscmail.wisc.edu>; Mon, 14 Mar 2011 11:38:57 -0500 (CDT) Received: from anacreon.physics.wisc.edu (anacreon.physics.wisc.edu [128.104.160.176]) by smtpauth3.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) with ESMTPSA id <0LI2000OQ3KR2O10@smtpauth3.wiscmail.wisc.edu>; Mon, 14 Mar 2011 11:38:51 -0500 (CDT) Date: Mon, 14 Mar 2011 11:38:51 -0500 From: Nathan Whitehorn In-reply-to: To: Giorgos Keramidas Message-id: <4D7E449B.5020209@freebsd.org> X-Spam-Report: AuthenticatedSender=yes, SenderIP=128.104.160.176 X-Spam-PmxInfo: Server=avs-10, Version=5.6.0.2009776, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2011.3.14.161528, SenderIP=128.104.160.176 References: <4D7E228A.4090906@freebsd.org> User-Agent: Mozilla/5.0 (X11; U; FreeBSD powerpc; en-US; rv:1.9.2.15) Gecko/20110305 Thunderbird/3.1.9 Cc: freebsd-current Current , freebsd-sysinstall@freebsd.org, FreeBSD Arch Subject: Re: HEADS UP: sysinstall is no longer the default installer X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Mar 2011 16:38:58 -0000 On 03/14/11 10:38, Giorgos Keramidas wrote: > On Mon, Mar 14, 2011 at 3:13 PM, Nathan Whitehorn > wrote: >> Changes to release(7) >> ----------------------------- >> >> Release builds work and look slightly different now, so everyone who >> snapshot tinderboxes will likely find them breaking shortly. The nearest >> analog to the old make release (with version-control checkouts and a chroot) >> is src/release/generate-release.sh, which can be run as generate-release.sh >> head /path/to/chroot/dir. If you want to include ports and documentation on >> the release media, CVSUP_HOST must be defined in the environment to point to >> a cvsup mirror. The output is placed in /R in the chroot directory, as >> before. >> >> If the chroot is unimportant (it ensures a total clean-room build, but may >> not be necessary in most cases), you can get a release build using the >> regular makefile, like so: >> cd /usr/src >> make buildworld buildkernel >> cd /usr/src/release >> make obj release >> >> By default, this will include ports and documentation if you have them >> checked out to /usr/ports and /usr/doc, though this behavior can be modified >> (see the top of the makefile). In addition, some architectures (i386, amd64, >> powerpc, powerpc64, and maybe ia64) have release media that can be >> cross-built, so you can set TARGET/TARGET_ARCH appropriately for those. >> Output goes to .OBJDIR, which is /usr/obj/usr/src/release in the case of the >> above commands. The equivalent to disc1 is called release.iso, the memstick >> image (i386, amd64 only) is called memstick, and a directory of distfiles >> for FTP mirrors is generated named ftp. > Any "user interface" changes that affect the release.7 manpage and may > catch people building their own release images should be updated in the > manpage itself too. Yes. I was hoping to update the manpage in the next couple days. > Some of the stuff I'd like to see fixed in the release.7 manpage are: > > - The requirement for CVSUP_HOST should be explicitly mentioned in > release.7 for releases that have NODOC. > > Note: I haven't run a release with the new Makefile yet, but is it > still possible to use a local CVS mirror, e.g. /home/ncvs for these > files instead of a cvsup host that is only accessible over the > (potentially much slower) network? It isn't possible right now. If you have a pre-existing checkout (from whatever source), make release will use that. John Baldwin mentioned the cvs changes to generate-release.sh and I'll try to get those in soon. > - The make variable ${DATE} is automatically set to the build date. We > should probably mention this in the default BUILDNAME description > (since it's such a generic variable name). > > - BUILDNAME is automatically set to a default that may have to be > documented to the manpage, so that people know what to expect when > they type just "make release" and sit back. > > - There's a ${BASE} variable set to 9.0 that release engineers may > have to manually update when they roll-out release and stable > branches. This should be documented in the "shortly before the > release" checklist we have in > http://www.freebsd.org/doc/en_US.ISO8859-1/articles/releng/release-proc.html Some (most) of these variables don't exist anymore. The CD is just always named 'release.iso' at the moment, for instance. That can easily be changed, however. > - The directories that "make release" creates, and the names of the > ISO image files should be mentioned in release.7 now that we have a > chance to make a batch of useful updates to the text. > > Naturally, I volunteer to *make* the mdoc changes. As long as someone > (e.g. you Nathan?) who is acquainted with the new release building > Makefile can hepl me by reviewing the updates and making sure they look > reasonably close to the new state of everything. Sure. I have the feeling that there are going to be a lot of feature and change requests today with regard to release infrastructure, so I'll let the dust clear for a day or two and we can start hashing out the documentation. -Nathan From owner-freebsd-arch@FreeBSD.ORG Mon Mar 14 16:57:26 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8BF1A1065673; Mon, 14 Mar 2011 16:57:26 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 5EB048FC15; Mon, 14 Mar 2011 16:57:26 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id D383946B38; Mon, 14 Mar 2011 12:57:25 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.10]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 6F0548A01B; Mon, 14 Mar 2011 12:57:25 -0400 (EDT) From: John Baldwin To: Nathan Whitehorn Date: Mon, 14 Mar 2011 12:55:16 -0400 User-Agent: KMail/1.13.5 (FreeBSD/7.4-CBSD-20110107; KDE/4.4.5; amd64; ; ) References: <4D7E228A.4090906@freebsd.org> <201103141144.32815.jhb@freebsd.org> <4D7E3A9E.10800@freebsd.org> In-Reply-To: <4D7E3A9E.10800@freebsd.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201103141255.16292.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Mon, 14 Mar 2011 12:57:25 -0400 (EDT) Cc: freebsd-current@freebsd.org, freebsd-sysinstall@freebsd.org, FreeBSD Arch Subject: Re: HEADS UP: sysinstall is no longer the default installer X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Mar 2011 16:57:26 -0000 On Monday, March 14, 2011 11:56:14 am Nathan Whitehorn wrote: > On 03/14/11 10:44, John Baldwin wrote: > > On Monday, March 14, 2011 10:13:30 am Nathan Whitehorn wrote: > >> I just committed (r219641) changes that make the release infrastructure > >> (src/release/Makefile) use bsdinstall by default instead of sysinstall > >> on install media. A big thank you is in order to everyone who provided > >> advice, criticism, and testing for this project over the last few months! > >> > >> Along with sysinstall, the original sysinstall build stuff has been > >> preserved (now /usr/src/release/Makefile.sysinstall) and will continue > >> to be for the lifetime of the 9.x release series, although it will not > >> be used by default. This change modifies the process of building > >> releases somewhat, so I'll outline changes that people who run snapshot > >> buildbots will have to make below, and some next steps planned with the > >> installer. > > Please consider supporting using SVN or CVS to obtain docs, ports, and source > > trees. I have a custom SVN repo at work that is not exported to CVS and > > available via csup and am able to use the existing SVNROOT SVNBRANCH variables > > with 'make release'. Having support for this sort of thing would be useful. > > I have also made much use of LOCAL_PATCHES in the past for building releases, > > so having support for that would be useful as well. > > SVNBRANCH works now, and source comes over SVN, the others via cvsup. > Support for a different SVNROOT and regular cvs for ports and docs can > certainly be added. In the case of LOCAL_PATCHES, you can just use the > regular makefile on your patched tree -- I don't think the chroot and > checkouts make much sense in this case. Hmm, I've actually used LOCAL_PATCHES a lot to test out changes while still doing builds in a chroot (I'm paranoid about not having pollution from the build machine in the release builds so have always used the chroot). Being able to use CVS and a custom CVSROOT and SVNROOT would be good to have. > > I think for re@ especially it is nice to just do 'make release TAG=7.2' (or > > some such) and have it DTRT to check out matching ports, doc, and src into the > > chroot, etc. I think the new process should be similarly automated. > > The generate-release.sh script likely needs some work. It exists almost > purely for the benefit of re@, and I don't know exactly what their > requirements are. A list (or patches) would be very welcome. The feature > you want here, though, can be obtained now by the CVSUP_TAG and svn > branch arguments to generate-release.sh. Note that re@ uses CVS to checkout ports and docs rather than cvsup. There was also logic in the old release Makefile to take a single CVS-style src tag and convert it into suitable tags for docs and ports. An example of the re@ style is found in the bottom of the old release(7): EXAMPLES The following sequence of commands was used to build the FreeBSD 4.9 release: cd /usr cvs co -rRELENG_4_9_0_RELEASE src cd src make buildworld cd release make release CHROOTDIR=/local3/release BUILDNAME=4.9-RELEASE \ CVSROOT=/host/cvs/usr/home/ncvs RELEASETAG=RELENG_4_9_0_RELEASE After running these commands, a complete system suitable for FTP or CD- ROM distribution is available in the /local3/release/R directory. > > Have you tested network installs using PXE or the like? This was fairly easy > > before as you could copy the '/boot' directory from a bootable ISO and the > > mfsroot was self-contained. Do you now have to put the entire contents of > > release.iso up via NFS? Is there a subset you put in the NFS root and then do > > an NFS or FTP install? > > > > Yes, I have, and it works well (tested on i386, sparc64, and powerpc). > Right now, you need the whole system (which is a regular installworld + > the rc.local to give the installer menu, and, optionally the distfiles). > For the future, the set of things the installer needs from the userland > is intentionally fairly small. I need to do some work anyway to make a > minimal system for bootonly CDs and the like, which should also a > smaller system for PXE as well. Ok. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Mar 14 17:26:41 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8B1B3106564A; Mon, 14 Mar 2011 17:26:41 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 4ADC58FC0A; Mon, 14 Mar 2011 17:26:40 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 0E3395DC2; Mon, 14 Mar 2011 17:07:02 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.4/8.14.4) with ESMTP id p2EH717d003020; Mon, 14 Mar 2011 17:07:01 GMT (envelope-from phk@critter.freebsd.dk) To: Nathan Whitehorn From: "Poul-Henning Kamp" In-Reply-To: Your message of "Mon, 14 Mar 2011 09:13:30 EST." <4D7E228A.4090906@freebsd.org> Date: Mon, 14 Mar 2011 17:07:01 +0000 Message-ID: <3019.1300122421@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: freebsd-current Current , freebsd-sysinstall@freebsd.org, FreeBSD Arch Subject: Re: HEADS UP: sysinstall is no longer the default installer X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Mar 2011 17:26:41 -0000 In message <4D7E228A.4090906@freebsd.org>, Nathan Whitehorn writes: >I just committed (r219641) changes that make the release infrastructure >(src/release/Makefile) use bsdinstall by default instead of sysinstall >on install media. A big thank you is in order to everyone who provided >advice, criticism, and testing for this project over the last few months! Free! I am Free at last!!! I need a beer now... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Tue Mar 15 00:39:07 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E56A106566C; Tue, 15 Mar 2011 00:39:07 +0000 (UTC) (envelope-from mack@macktronics.com) Received: from coco.macktronics.com (coco.macktronics.com [209.181.253.65]) by mx1.freebsd.org (Postfix) with ESMTP id 261228FC15; Tue, 15 Mar 2011 00:39:06 +0000 (UTC) Received: from [172.22.30.42] (dulse.macktronics.com [209.181.253.69]) by coco.macktronics.com (Postfix) with ESMTPA id 8B8014AC40; Mon, 14 Mar 2011 19:20:10 -0500 (CDT) Mime-Version: 1.0 (Apple Message framework v1082) Content-Type: text/plain; charset=us-ascii From: Dan Mack In-Reply-To: <4D7E228A.4090906@freebsd.org> Date: Mon, 14 Mar 2011 19:20:10 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <9B41C412-A8C4-48B9-AF45-182E5658A238@macktronics.com> References: <4D7E228A.4090906@freebsd.org> To: Nathan Whitehorn X-Mailer: Apple Mail (2.1082) Cc: freebsd-current Current , freebsd-sysinstall@freebsd.org, FreeBSD Arch Subject: Re: HEADS UP: sysinstall is no longer the default installer X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Mar 2011 00:39:07 -0000 On Mar 14, 2011, at 9:13 AM, Nathan Whitehorn wrote: > I just committed (r219641) changes that make the release = infrastructure (src/release/Makefile) use bsdinstall by default instead = of sysinstall on install media. A big thank you is in order to everyone = who provided advice, criticism, and testing for this project over the = last few months! >=20 > Along with sysinstall, the original sysinstall build stuff has been = preserved (now /usr/src/release/Makefile.sysinstall) and will continue = to be for the lifetime of the 9.x release series, although it will not = be used by default. This change modifies the process of building = releases somewhat, so I'll outline changes that people who run snapshot = buildbots will have to make below, and some next steps planned with the = installer. >=20 > Changes to release(7) > ----------------------------- >=20 > Release builds work and look slightly different now, so everyone who = snapshot tinderboxes will likely find them breaking shortly. The nearest = analog to the old make release (with version-control checkouts and a = chroot) is src/release/generate-release.sh, which can be run as = generate-release.sh head /path/to/chroot/dir. If you want to include = ports and documentation on the release media, CVSUP_HOST must be defined = in the environment to point to a cvsup mirror. The output is placed in = /R in the chroot directory, as before. >=20 > If the chroot is unimportant (it ensures a total clean-room build, but = may not be necessary in most cases), you can get a release build using = the regular makefile, like so: > cd /usr/src > make buildworld buildkernel > cd /usr/src/release > make obj release >=20 >=20 >=20 Thanks! For what it's worth, I built a new release using this new = method and the only problem I ran into was getting dropped to the = "mountroot>" because the memstick's root partition failed to mount. I = am not sure if this has anything to do with your changes or not but I = thought I would bring it up. After mounting my usb stick with : = ufs:/dev/da0a it booted into bsdinstall without issue. I don't know if = this was do to kern.cam.boot_delay not being long enough or if it was a = problem with the creation of the memstick image.=20 During bsdinstall, there were a bunch of console debug messages spewing = alongside the bsdinstall text but they cleared before I could take a = picture.=20 Now we just need a ZFS template for the partition tool :-) Thanks again! Dan -- Dan Mack mack@macktronics.com From owner-freebsd-arch@FreeBSD.ORG Tue Mar 15 12:51:51 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 55D94106566B; Tue, 15 Mar 2011 12:51:51 +0000 (UTC) (envelope-from gkeramidas@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id C4BC08FC0C; Tue, 15 Mar 2011 12:51:50 +0000 (UTC) Received: by gyg13 with SMTP id 13so239103gyg.13 for ; Tue, 15 Mar 2011 05:51:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=fBahzl649A8V1hLtKJybiIhb2L7QK3T5YrdldoD0dgA=; b=i7EuO6kHQJUoLjQUf4/VfOCRxw+hMnEDuYuL/lx6HZNdr2QWPVjimWvH659q8EQM/M Xv0T9K2DrU46IvjdqMTHcQMtMICU2saevad+WdFv5HHw5SpVMHV1rAiPwuuK1Dzvi0R0 tsI8D98AIfe190MKFv/alsS0hlqDzbEzqBLro= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=wZs8fJBXrE4Lg3w3jWcIsDuO7Qow9OX98V71WGSKiv9y7wxSTbr5kkbD+f0/EpuiqT todcqyWSwSRiD8RGSjEXX0PvyPvMWMaN67JDm2tY3sbfDyAqwRZBRbB9sg+Id0Vzi3Yn AFyUgOHmSZRLXtmuK3kAmUuIUmiqndH6La5QI= MIME-Version: 1.0 Received: by 10.150.251.28 with SMTP id y28mr419340ybh.323.1300193510191; Tue, 15 Mar 2011 05:51:50 -0700 (PDT) Sender: gkeramidas@gmail.com Received: by 10.147.181.12 with HTTP; Tue, 15 Mar 2011 05:51:50 -0700 (PDT) In-Reply-To: <4D7E449B.5020209@freebsd.org> References: <4D7E228A.4090906@freebsd.org> <4D7E449B.5020209@freebsd.org> Date: Tue, 15 Mar 2011 13:51:50 +0100 X-Google-Sender-Auth: _XRkofRKRr9bIH5KbhGwyKc4zC0 Message-ID: From: Giorgos Keramidas To: Nathan Whitehorn Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-current Current , freebsd-sysinstall@freebsd.org, FreeBSD Arch Subject: Re: HEADS UP: sysinstall is no longer the default installer X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Mar 2011 12:51:51 -0000 On Mon, Mar 14, 2011 at 5:38 PM, Nathan Whitehorn wrote: > On 03/14/11 10:38, Giorgos Keramidas wrote: >> Naturally, I volunteer to *make* the mdoc changes. =C2=A0As long as some= one >> (e.g. you Nathan?) who is acquainted with the new release building >> Makefile can hepl me by reviewing the updates and making sure they look >> reasonably close to the new state of everything. > > Sure. I have the feeling that there are going to be a lot of feature and > change requests today with regard to release infrastructure, so I'll let = the > dust clear for a day or two and we can start hashing out the documentatio= n. Sounds like a plan to me. Thanks :-) From owner-freebsd-arch@FreeBSD.ORG Tue Mar 15 18:20:54 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5D3881065672; Tue, 15 Mar 2011 18:20:54 +0000 (UTC) (envelope-from xcllnt@mac.com) Received: from asmtpout026.mac.com (asmtpout026.mac.com [17.148.16.101]) by mx1.freebsd.org (Postfix) with ESMTP id 402AC8FC18; Tue, 15 Mar 2011 18:20:54 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from sa-nc-finance-147.static.jnpr.net (natint3.juniper.net [66.129.224.36]) by asmtp026.mac.com (Oracle Communications Messaging Exchange Server 7u4-20.01 64bit (built Nov 21 2010)) with ESMTPSA id <0LI40071906C8NA0@asmtp026.mac.com>; Tue, 15 Mar 2011 10:20:38 -0700 (PDT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.2.15,1.0.148,0.0.0000 definitions=2011-03-15_02:2011-03-14, 2011-03-15, 1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 suspectscore=2 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx engine=6.0.2-1012030000 definitions=main-1103150117 From: Marcel Moolenaar In-reply-to: <4D7E228A.4090906@freebsd.org> Date: Tue, 15 Mar 2011 10:20:41 -0700 Message-id: References: <4D7E228A.4090906@freebsd.org> To: Nathan Whitehorn X-Mailer: Apple Mail (2.1082) Cc: freebsd-current Current , freebsd-sysinstall@freebsd.org, FreeBSD Arch Subject: Re: HEADS UP: sysinstall is no longer the default installer X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Mar 2011 18:20:54 -0000 On Mar 14, 2011, at 7:13 AM, Nathan Whitehorn wrote: > I just committed (r219641) changes that make the release infrastructure (src/release/Makefile) use bsdinstall by default instead of sysinstall on install media. A big thank you is in order to everyone who provided advice, criticism, and testing for this project over the last few months! Thanks Nathan, I checked ia64 and it works well enough. I may come back with a tweak here and there after the dust settles, but so far it's more reliable (and a while lot simpler) than sysinstall is. Great work! -- Marcel Moolenaar xcllnt@mac.com From owner-freebsd-arch@FreeBSD.ORG Sat Mar 19 01:51:53 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3FDD8106564A; Sat, 19 Mar 2011 01:51:53 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from adsum.doit.wisc.edu (adsum.doit.wisc.edu [144.92.197.210]) by mx1.freebsd.org (Postfix) with ESMTP id 1357E8FC13; Sat, 19 Mar 2011 01:51:53 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from avs-daemon.smtpauth1.wiscmail.wisc.edu by smtpauth1.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) id <0LIA00D047UG5H00@smtpauth1.wiscmail.wisc.edu>; Fri, 18 Mar 2011 20:51:52 -0500 (CDT) Received: from comporellon.tachypleus.net ([unknown] [76.210.65.155]) by smtpauth1.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) with ESMTPSA id <0LIA001T47UEB010@smtpauth1.wiscmail.wisc.edu>; Fri, 18 Mar 2011 20:51:52 -0500 (CDT) Date: Fri, 18 Mar 2011 20:51:50 -0500 From: Nathan Whitehorn In-reply-to: To: Marcel Moolenaar Message-id: <4D840C36.4060407@freebsd.org> X-Spam-Report: AuthenticatedSender=yes, SenderIP=76.210.65.155 X-Spam-PmxInfo: Server=avs-12, Version=5.6.0.2009776, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2011.3.19.14228, SenderIP=76.210.65.155 References: <4D7E228A.4090906@freebsd.org> User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110305 Thunderbird/3.1.9 Cc: freebsd-current Current , freebsd-sysinstall@freebsd.org, FreeBSD Arch Subject: Re: HEADS UP: sysinstall is no longer the default installer X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Mar 2011 01:51:53 -0000 On 03/15/11 12:20, Marcel Moolenaar wrote: > On Mar 14, 2011, at 7:13 AM, Nathan Whitehorn wrote: > >> I just committed (r219641) changes that make the release infrastructure (src/release/Makefile) use bsdinstall by default instead of sysinstall on install media. A big thank you is in order to everyone who provided advice, criticism, and testing for this project over the last few months! > Thanks Nathan, > > I checked ia64 and it works well enough. I may come back with a tweak > here and there after the dust settles, but so far it's more reliable > (and a while lot simpler) than sysinstall is. > > Great work! Thanks! The installer doesn't yet know (and I don't know) how to set up the EFI system partition on IA64, so I'll need some input (or code) from you on that point to get things totally up and running. -Nathan From owner-freebsd-arch@FreeBSD.ORG Sat Mar 19 01:52:35 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 299D3106564A; Sat, 19 Mar 2011 01:52:35 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from adsum.doit.wisc.edu (adsum.doit.wisc.edu [144.92.197.210]) by mx1.freebsd.org (Postfix) with ESMTP id F14318FC1B; Sat, 19 Mar 2011 01:52:34 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from avs-daemon.smtpauth1.wiscmail.wisc.edu by smtpauth1.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) id <0LIA00D007VMA500@smtpauth1.wiscmail.wisc.edu>; Fri, 18 Mar 2011 20:52:34 -0500 (CDT) Received: from comporellon.tachypleus.net ([unknown] [76.210.65.155]) by smtpauth1.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) with ESMTPSA id <0LIA001TB7VKB010@smtpauth1.wiscmail.wisc.edu>; Fri, 18 Mar 2011 20:52:33 -0500 (CDT) Date: Fri, 18 Mar 2011 20:52:32 -0500 From: Nathan Whitehorn In-reply-to: <3019.1300122421@critter.freebsd.dk> To: Poul-Henning Kamp Message-id: <4D840C60.3000604@freebsd.org> X-Spam-Report: AuthenticatedSender=yes, SenderIP=76.210.65.155 X-Spam-PmxInfo: Server=avs-13, Version=5.6.0.2009776, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2011.3.19.14228, SenderIP=76.210.65.155 References: <3019.1300122421@critter.freebsd.dk> User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110305 Thunderbird/3.1.9 Cc: freebsd-current Current , freebsd-sysinstall@freebsd.org, FreeBSD Arch Subject: Re: HEADS UP: sysinstall is no longer the default installer X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Mar 2011 01:52:35 -0000 On 03/14/11 12:07, Poul-Henning Kamp wrote: > In message<4D7E228A.4090906@freebsd.org>, Nathan Whitehorn writes: > >> I just committed (r219641) changes that make the release infrastructure >> (src/release/Makefile) use bsdinstall by default instead of sysinstall >> on install media. A big thank you is in order to everyone who provided >> advice, criticism, and testing for this project over the last few months! > Free! > > I am Free at last!!! > > I need a beer now... I'm not sure I like what this implies about my future :) -Nathan From owner-freebsd-arch@FreeBSD.ORG Sat Mar 19 01:56:07 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 04903106566B; Sat, 19 Mar 2011 01:56:07 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from adsum.doit.wisc.edu (adsum.doit.wisc.edu [144.92.197.210]) by mx1.freebsd.org (Postfix) with ESMTP id CAD098FC0C; Sat, 19 Mar 2011 01:56:06 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from avs-daemon.smtpauth1.wiscmail.wisc.edu by smtpauth1.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) id <0LIA00D0081IR900@smtpauth1.wiscmail.wisc.edu>; Fri, 18 Mar 2011 20:56:06 -0500 (CDT) Received: from comporellon.tachypleus.net ([unknown] [76.210.65.155]) by smtpauth1.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) with ESMTPSA id <0LIA001VJ81GB010@smtpauth1.wiscmail.wisc.edu>; Fri, 18 Mar 2011 20:56:05 -0500 (CDT) Date: Fri, 18 Mar 2011 20:56:04 -0500 From: Nathan Whitehorn In-reply-to: <9B41C412-A8C4-48B9-AF45-182E5658A238@macktronics.com> To: Dan Mack Message-id: <4D840D34.7030005@freebsd.org> X-Spam-Report: AuthenticatedSender=yes, SenderIP=76.210.65.155 X-Spam-PmxInfo: Server=avs-11, Version=5.6.0.2009776, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2011.3.19.14822, SenderIP=76.210.65.155 References: <4D7E228A.4090906@freebsd.org> <9B41C412-A8C4-48B9-AF45-182E5658A238@macktronics.com> User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.15) Gecko/20110305 Thunderbird/3.1.9 Cc: freebsd-current Current , freebsd-sysinstall@freebsd.org, FreeBSD Arch Subject: Re: HEADS UP: sysinstall is no longer the default installer X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Mar 2011 01:56:07 -0000 On 03/14/11 19:20, Dan Mack wrote: > On Mar 14, 2011, at 9:13 AM, Nathan Whitehorn wrote: > >> I just committed (r219641) changes that make the release infrastructure (src/release/Makefile) use bsdinstall by default instead of sysinstall on install media. A big thank you is in order to everyone who provided advice, criticism, and testing for this project over the last few months! >> >> Along with sysinstall, the original sysinstall build stuff has been preserved (now /usr/src/release/Makefile.sysinstall) and will continue to be for the lifetime of the 9.x release series, although it will not be used by default. This change modifies the process of building releases somewhat, so I'll outline changes that people who run snapshot buildbots will have to make below, and some next steps planned with the installer. >> >> Changes to release(7) >> ----------------------------- >> >> Release builds work and look slightly different now, so everyone who snapshot tinderboxes will likely find them breaking shortly. The nearest analog to the old make release (with version-control checkouts and a chroot) is src/release/generate-release.sh, which can be run as generate-release.sh head /path/to/chroot/dir. If you want to include ports and documentation on the release media, CVSUP_HOST must be defined in the environment to point to a cvsup mirror. The output is placed in /R in the chroot directory, as before. >> >> If the chroot is unimportant (it ensures a total clean-room build, but may not be necessary in most cases), you can get a release build using the regular makefile, like so: >> cd /usr/src >> make buildworld buildkernel >> cd /usr/src/release >> make obj release >> >> >> > > > Thanks! For what it's worth, I built a new release using this new method and the only problem I ran into was getting dropped to the "mountroot>" because the memstick's root partition failed to mount. I am not sure if this has anything to do with your changes or not but I thought I would bring it up. After mounting my usb stick with : ufs:/dev/da0a it booted into bsdinstall without issue. I don't know if this was do to kern.cam.boot_delay not being long enough or if it was a problem with the creation of the memstick image. Hm. I'd be interested to know if this is repeatable. The memstick stuff is a fairly new feature, and so hasn't been tested to quite the same degree as the ISOs. In case anyone else wants to try it, there is a memstick image (and ISO) here: http://people.freebsd.org/~nwhitehorn/bsdinstall-amd64-20110313/ > During bsdinstall, there were a bunch of console debug messages spewing alongside the bsdinstall text but they cleared before I could take a picture. Yes, there seem to be some LORs in UFS that get triggered if you untar massive quantities of files very fast. They seem harmless, though. > Now we just need a ZFS template for the partition tool :-) Yes. Hopefully this comes in through collaboration with the pc-sysinstall people. Having gptboot and gptzfsboot be the same thing would also help a great deal toward that goal. > Thanks again! > Glad you liked it! -Nathan From owner-freebsd-arch@FreeBSD.ORG Sat Mar 19 05:05:16 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A34111065676; Sat, 19 Mar 2011 05:05:16 +0000 (UTC) (envelope-from xcllnt@mac.com) Received: from asmtpout026.mac.com (asmtpout026.mac.com [17.148.16.101]) by mx1.freebsd.org (Postfix) with ESMTP id 85E908FC08; Sat, 19 Mar 2011 05:05:16 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from dhcp-192-168-2-13.wifi.xcllnt.net (atm.xcllnt.net [70.36.220.6]) by asmtp026.mac.com (Oracle Communications Messaging Exchange Server 7u4-20.01 64bit (built Nov 21 2010)) with ESMTPSA id <0LIA0056TGSM8X80@asmtp026.mac.com>; Fri, 18 Mar 2011 22:05:13 -0700 (PDT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.2.15,1.0.148,0.0.0000 definitions=2011-03-18_06:2011-03-16, 2011-03-18, 1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 suspectscore=3 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx engine=6.0.2-1012030000 definitions=main-1103180221 From: Marcel Moolenaar In-reply-to: <4D840C36.4060407@freebsd.org> Date: Fri, 18 Mar 2011 22:05:11 -0700 Message-id: References: <4D7E228A.4090906@freebsd.org> <4D840C36.4060407@freebsd.org> To: Nathan Whitehorn X-Mailer: Apple Mail (2.1082) Cc: freebsd-current Current , freebsd-sysinstall@freebsd.org, FreeBSD Arch Subject: Re: HEADS UP: sysinstall is no longer the default installer X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Mar 2011 05:05:16 -0000 On Mar 18, 2011, at 6:51 PM, Nathan Whitehorn wrote: > On 03/15/11 12:20, Marcel Moolenaar wrote: >> On Mar 14, 2011, at 7:13 AM, Nathan Whitehorn wrote: >> >>> I just committed (r219641) changes that make the release infrastructure (src/release/Makefile) use bsdinstall by default instead of sysinstall on install media. A big thank you is in order to everyone who provided advice, criticism, and testing for this project over the last few months! >> Thanks Nathan, >> >> I checked ia64 and it works well enough. I may come back with a tweak >> here and there after the dust settles, but so far it's more reliable >> (and a while lot simpler) than sysinstall is. >> >> Great work! > > Thanks! The installer doesn't yet know (and I don't know) how to set up the EFI system partition on IA64, so I'll need some input (or code) from you on that point to get things totally up and running. It's not that hard in general: create a partition that is 100MB in size, give it the right type (i.e. C12A7328-F81F-11d2-BA4B-00A0C93EC93B) and format with dosfs. This has to be the very first partition on a boot device. As part of the installation, we need to copy the EFI loader to a FreeBSD subdirectory. Adding an entry for FreeBSD to the boot menu is where it really gets interesting. The support for writing the EFI environment exists (see libefi), but construction an EFI device path from a device special file probably needs some more code. Getting that to work is interesting for installing on Intel based Apple hardware as well I would presume. Most systems have a system partition, so copying the loader to it is the most important aspect of getting a bootable installation. -- Marcel Moolenaar xcllnt@mac.com From owner-freebsd-arch@FreeBSD.ORG Sat Mar 19 07:24:23 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 89D15106566B for ; Sat, 19 Mar 2011 07:24:23 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) by mx1.freebsd.org (Postfix) with ESMTP id 65BA18FC13 for ; Sat, 19 Mar 2011 07:24:23 +0000 (UTC) Received: from 197.214.32.202.bf.2iij.net ([202.32.214.197] helo=[192.168.12.70]) by vps.hungerhost.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.69) (envelope-from ) id 1Q0pnP-0003TW-K4 for arch@freebsd.org; Sat, 19 Mar 2011 02:37:56 -0400 From: George Neville-Neil Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Sat, 19 Mar 2011 15:37:47 +0900 Message-Id: <132388F1-44D9-45C9-AE05-1799A7A2DCD9@neville-neil.com> To: arch@freebsd.org Mime-Version: 1.0 (Apple Message framework v1082) X-Pgp-Agent: GPGMail 1.3.2 X-Mailer: Apple Mail (2.1082) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com Cc: Subject: Updating our TCP and socket sysctl values... X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Mar 2011 07:24:23 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Howdy, I believe it's time for us to upgrade our sysctl values for TCP sockets = so that they are more in line with the modern world. At the moment we have = these limits on our buffering: kern.ipc.maxsockbuf: 262144 net.inet.tcp.recvbuf_max: 262144 net.inet.tcp.sendbuf_max: 262144 I believe it's time to up these values to something that's in line with = higher speed local networks, such as 10G. Perhaps it's time to move these to 2MB = instead of 256K. Thoughts? Best, George -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (Darwin) iEYEARECAAYFAk2ETzsACgkQYdh2wUQKM9JiJgCeOMcR1R8/uDeI9PXTLHCFHPRr zIcAn28u4CrD2dSoLII+71KxA0XN4EMV =3D4KpR -----END PGP SIGNATURE----- From owner-freebsd-arch@FreeBSD.ORG Sat Mar 19 15:04:06 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 872081065691 for ; Sat, 19 Mar 2011 15:04:06 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 46F3F8FC08 for ; Sat, 19 Mar 2011 15:04:06 +0000 (UTC) Received: from outgoing.leidinger.net (p5B1554DB.dip.t-dialin.net [91.21.84.219]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 486C384400E; Sat, 19 Mar 2011 16:04:02 +0100 (CET) Received: from unknown (IO.Leidinger.net [192.168.2.110]) by outgoing.leidinger.net (Postfix) with ESMTP id 7744B3EAB; Sat, 19 Mar 2011 16:03:59 +0100 (CET) Date: Sat, 19 Mar 2011 16:04:00 +0100 From: Alexander Leidinger To: George Neville-Neil Message-ID: <20110319160400.000043f5@unknown> In-Reply-To: <132388F1-44D9-45C9-AE05-1799A7A2DCD9@neville-neil.com> References: <132388F1-44D9-45C9-AE05-1799A7A2DCD9@neville-neil.com> X-Mailer: Claws Mail 3.7.6 (GTK+ 2.16.0; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 486C384400E.A3C4D X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1, required 6, autolearn=disabled, ALL_TRUSTED -1.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1301151843.01304@xv8KS2sTdL2O03rbb+CL6A X-EBL-Spam-Status: No Cc: arch@freebsd.org Subject: Re: Updating our TCP and socket sysctl values... X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Mar 2011 15:04:06 -0000 On Sat, 19 Mar 2011 15:37:47 +0900 George Neville-Neil wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Howdy, > > I believe it's time for us to upgrade our sysctl values for TCP > sockets so that they are more in line with the modern world. At the > moment we have these limits on our buffering: > > kern.ipc.maxsockbuf: 262144 > net.inet.tcp.recvbuf_max: 262144 > net.inet.tcp.sendbuf_max: 262144 > > I believe it's time to up these values to something that's in line > with higher speed local networks, such as 10G. Perhaps it's time to > move these to 2MB instead of 256K. > > Thoughts? I suggest to read http://www.bufferbloat.net/projects/bloat/wiki/Bufferbloat and do a before/after test to make sure we do not suffer from the described problem. Jim Getty has test descriptions: http://gettys.wordpress.com/category/bufferbloat/ Bye, Alexander. From owner-freebsd-arch@FreeBSD.ORG Sat Mar 19 15:15:51 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 05CD210656A7 for ; Sat, 19 Mar 2011 15:15:51 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id C06838FC08 for ; Sat, 19 Mar 2011 15:15:50 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id p2JF9jv1055338 for ; Sat, 19 Mar 2011 09:09:47 -0600 (MDT) (envelope-from imp@bsdimp.com) Message-ID: <4D84C72A.3030905@bsdimp.com> Date: Sat, 19 Mar 2011 09:09:30 -0600 From: Warner Losh User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20101211 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org References: <132388F1-44D9-45C9-AE05-1799A7A2DCD9@neville-neil.com> In-Reply-To: <132388F1-44D9-45C9-AE05-1799A7A2DCD9@neville-neil.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: Updating our TCP and socket sysctl values... X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Mar 2011 15:15:51 -0000 On 03/19/2011 00:37, George Neville-Neil wrote: > I believe it's time for us to upgrade our sysctl values for TCP sockets so that > they are more in line with the modern world. At the moment we have these limits on > our buffering: > > kern.ipc.maxsockbuf: 262144 > net.inet.tcp.recvbuf_max: 262144 > net.inet.tcp.sendbuf_max: 262144 > > I believe it's time to up these values to something that's in line with higher speed > local networks, such as 10G. Perhaps it's time to move these to 2MB instead of 256K. > > Thoughts? So long as these can be tuned down to the lower end for memory constrained environments, that's fine. In general, I believe that the project has aimed to having the tunables in the system optimized for the fastest hardware, while being usable on the last few generations of hardware and tunable, where possible, for the older hardware. Warner From owner-freebsd-arch@FreeBSD.ORG Sat Mar 19 22:21:30 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0150D106564A for ; Sat, 19 Mar 2011 22:21:30 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id BEC308FC08 for ; Sat, 19 Mar 2011 22:21:29 +0000 (UTC) Received: by gwb15 with SMTP id 15so2283246gwb.13 for ; Sat, 19 Mar 2011 15:21:29 -0700 (PDT) Received: by 10.236.78.74 with SMTP id f50mr3439242yhe.128.1300571682927; Sat, 19 Mar 2011 14:54:42 -0700 (PDT) Received: from [10.0.1.198] ([72.253.42.56]) by mx.google.com with ESMTPS id 68sm2420991yhl.19.2011.03.19.14.54.39 (version=SSLv3 cipher=OTHER); Sat, 19 Mar 2011 14:54:41 -0700 (PDT) Date: Sat, 19 Mar 2011 11:58:12 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: Alexander Leidinger In-Reply-To: <20110319160400.000043f5@unknown> Message-ID: References: <132388F1-44D9-45C9-AE05-1799A7A2DCD9@neville-neil.com> <20110319160400.000043f5@unknown> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: arch@freebsd.org, George Neville-Neil Subject: Re: Updating our TCP and socket sysctl values... X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Mar 2011 22:21:30 -0000 On Sat, 19 Mar 2011, Alexander Leidinger wrote: > On Sat, 19 Mar 2011 15:37:47 +0900 George Neville-Neil > wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Howdy, >> >> I believe it's time for us to upgrade our sysctl values for TCP >> sockets so that they are more in line with the modern world. At the >> moment we have these limits on our buffering: >> >> kern.ipc.maxsockbuf: 262144 >> net.inet.tcp.recvbuf_max: 262144 >> net.inet.tcp.sendbuf_max: 262144 >> >> I believe it's time to up these values to something that's in line >> with higher speed local networks, such as 10G. Perhaps it's time to >> move these to 2MB instead of 256K. >> >> Thoughts? > > I suggest to read > http://www.bufferbloat.net/projects/bloat/wiki/Bufferbloat > and do a before/after test to make sure we do not suffer from the > described problem. Jim Getty has test descriptions: > http://gettys.wordpress.com/category/bufferbloat/ Are they not talking about buffers in non-endpoint devices? Or perhaps even overly large rx queues in endpoints, but not local socket receive buffers? It seems that they are describing situations where excessive buffering masks network conditions until it's too late. Thanks, Jeff > > Bye, > Alexander. > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Sat Mar 19 23:45:34 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E01371065672 for ; Sat, 19 Mar 2011 23:45:34 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 73E6F8FC08 for ; Sat, 19 Mar 2011 23:45:34 +0000 (UTC) Received: by fxm11 with SMTP id 11so5424586fxm.13 for ; Sat, 19 Mar 2011 16:45:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=H2t0wsvmInGeQ8LPJuVyI6tZFtXQEjGGHrLs/TXIZK4=; b=L+AlCk03+14ysmpVQyYKt2x2rftQjLpiMtcO5h3CW9C95tchC1hNOVlQji3LQ6pNaQ c0gHdj0/vPdlJJiiKkpY0d7yiUJAFGixP/l9UGvObZ/RY0LqU+1tInum2NChcc1gttRz BKDjJPHH6S83dB3NfTfr3A51pl/i0avZXdsIY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=qX5Q55OSIyGjdzsikrFY021/AB5ymuldEq8bqHEEmzhvpXKmljj1Ty65p957wIg81D JEaJqaPOdsS05riEkRt5plnjWuYBPjPulH299fR5d+mKbK82MLVOwcKSsXcf2Gj9cn4l C/3h1M0VX6FBL+SncSd1AsvQHygdqJcSig4cA= MIME-Version: 1.0 Received: by 10.223.76.147 with SMTP id c19mr3000014fak.55.1300576435698; Sat, 19 Mar 2011 16:13:55 -0700 (PDT) Received: by 10.223.32.204 with HTTP; Sat, 19 Mar 2011 16:13:55 -0700 (PDT) In-Reply-To: <132388F1-44D9-45C9-AE05-1799A7A2DCD9@neville-neil.com> References: <132388F1-44D9-45C9-AE05-1799A7A2DCD9@neville-neil.com> Date: Sat, 19 Mar 2011 16:13:55 -0700 Message-ID: From: Navdeep Parhar To: George Neville-Neil Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: arch@freebsd.org Subject: Re: Updating our TCP and socket sysctl values... X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Mar 2011 23:45:35 -0000 On Fri, Mar 18, 2011 at 11:37 PM, George Neville-Neil wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Howdy, > > I believe it's time for us to upgrade our sysctl values for TCP sockets s= o that > they are more in line with the modern world. =A0At the moment we have the= se limits on > our buffering: > > kern.ipc.maxsockbuf: 262144 > net.inet.tcp.recvbuf_max: 262144 > net.inet.tcp.sendbuf_max: 262144 > > I believe it's time to up these values to something that's in line with h= igher speed > local networks, such as 10G. =A0Perhaps it's time to move these to 2MB in= stead of 256K. > > Thoughts? 256KB seems adequate for 10G (as long as the consumer can keep draining the socket rcv buffer fast enough). If you consider 2 x bandwidth delay product to be a reasonable socket buffer size then 256K allows for 10G networks with ~100ms delays. Normally the delay is _way_ less than this for 10G and even 256K may be an overkill (but this is ok, the kernel has tcp_do_autorcvbuf on by default) While we're here discussing defaults, what about nmbclusters and nmbjumboXX? Now those haven't kept up with modern machines (imho). Regards, Navdeep