From owner-freebsd-arch@FreeBSD.ORG Sun Jun 12 00:00:24 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CF665106564A; Sun, 12 Jun 2011 00:00:23 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 2DCDD8FC16; Sun, 12 Jun 2011 00:00:22 +0000 (UTC) Received: by wwc33 with SMTP id 33so3768176wwc.31 for ; Sat, 11 Jun 2011 17:00:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=b9xhvmWxpICLZI6oW7xybJ73PzCUh0VXz1Lu3zh7q+o=; b=wDZdCBh1VeYCWubxGRis+CSZfZ1njeUHq/BT2ZrdPHTHYegQXAUjnVf7gHDsIWvQsZ Ire1cppq/rTAEydxJCnA1MRIMQjuyU2KykfrXZ+2X4Vc35yNt8uKcehjVRbCWDmxeB/m bhFzFrFR2LJjU/jg5lL4CuurvcV78s5P55w/o= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=NnyK1/lSAvKfzEDSNCINjFgNYL527iy0MdcdCoDIPUD3OF4+cq8Ol5/F/aUmSuu+XY Dszu9AI0wHl5+8AfxCZcmksSuyzFHnhtxnK66ULyZiuRx15nMMBgeFEkvyvCs6dMfDCs cOZJbamFdt8rw2YYbKW8imjc9cvu1jdSXnHaE= MIME-Version: 1.0 Received: by 10.216.221.67 with SMTP id q45mr1093039wep.97.1307836821753; Sat, 11 Jun 2011 17:00:21 -0700 (PDT) Sender: mdf356@gmail.com Received: by 10.216.52.141 with HTTP; Sat, 11 Jun 2011 17:00:21 -0700 (PDT) In-Reply-To: References: <20110611171834.GA38142@zim.MIT.EDU> <20110611204326.GA51320@zim.MIT.EDU> Date: Sat, 11 Jun 2011 17:00:21 -0700 X-Google-Sender-Auth: Zk9zV-OUhPC7elxqapBDIQ2lM8k Message-ID: From: mdf@FreeBSD.org To: Warner Losh Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: David Schultz , Adrian Chadd , freebsd-arch@freebsd.org Subject: Re: [RFC] shipping kernels with default modules? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Jun 2011 00:00:24 -0000 On Sat, Jun 11, 2011 at 4:07 PM, Warner Losh wrote: > > On Jun 11, 2011, at 2:43 PM, David Schultz wrote: > >> On Sat, Jun 11, 2011, mdf@freebsd.org wrote: >>> On Sat, Jun 11, 2011 at 10:18 AM, David Schultz wrote= : >>>> On Sat, Jun 11, 2011, Adrian Chadd wrote: >>>>> Hi guys, >>>>> >>>>> Has there been any further thought as of late about shipping kernels >>>>> with modules only by default, rather than monolithic kernels? >>>>> >>>>> I tried this experiment a couple years ago and besides a little >>>>> trickery with ACPI module loading, it worked out fine. >>>>> >>>>> Is there any reason we aren't doing this at the moment? Eg by having = a >>>>> default loader modules list populated from the kernel config file? >>>> >>>> I've been doing this for years, and it has come in quite handy. >>>> For instance, when my if_msk gets wedged, the only way to fix it >>>> short of rebooting seems to be reloading the driver. >>>> >>>> One issue, however, is that the boot loader is horrendously slow >>>> at loading modules. =A0(Either that or my BIOS has a braindead int 13h >>>> handler.) =A0Most of these modules aren't actually needed until much >>>> later in the boot process, so a mechanism to load non-essential >>>> modules after the file systems are mounted might provide a good >>>> solution. >>> >>> Indeed, at $WORK we're trying to get shutdown -> restart under 2 >>> minutes. =A0Several seconds of this is moving things *into* the kernel >>> that need to be there (disk drivers), and everything else to a point >>> in init where modules can be loaded in parallel, using the faster disk >>> driver, rather than in serial with slow BIOS handlers. >> >> Have you found that drivers can be reliably loaded in parallel >> these days? =A0I'm always waiting for timeouts on four card readers >> and two optical drives, so that would be a big win for me. =A0IIRC, >> nothing can happen in parallel during boot because the scheduler >> is initialized very late in the process. =A0I'm not a device driver >> person, but I imagine there might be other assumptions that might >> get in the way as well. > > Loading isn't the problem. =A0The timeouts that you are waiting for are p= art of the probe/attach sequence. =A0And that's strictly serialized... > If the timeouts are implemented using sleep(9) then Giant is dropped while waiting for a timeout, and another driver can begin its initialization. Thus the time to load all modules becomes roughly the max of all the timeouts, rather than the sum. Cheers, matthew From owner-freebsd-arch@FreeBSD.ORG Sun Jun 12 02:09:27 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E4536106564A; Sun, 12 Jun 2011 02:09:26 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 926B78FC12; Sun, 12 Jun 2011 02:09:26 +0000 (UTC) Received: from 63.imp.bsdimp.com (63.imp.bsdimp.com [10.0.0.63]) (authenticated bits=0) by harmony.bsdimp.com (8.14.4/8.14.3) with ESMTP id p5C24JJ7058553 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Sat, 11 Jun 2011 20:04:19 -0600 (MDT) (envelope-from imp@bsdimp.com) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: Date: Sat, 11 Jun 2011 20:04:15 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20110611171834.GA38142@zim.MIT.EDU> <20110611204326.GA51320@zim.MIT.EDU> To: mdf@FreeBSD.org X-Mailer: Apple Mail (2.1084) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (harmony.bsdimp.com [10.0.0.6]); Sat, 11 Jun 2011 20:04:19 -0600 (MDT) Cc: David Schultz , Adrian Chadd , freebsd-arch@FreeBSD.org Subject: Re: [RFC] shipping kernels with default modules? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Jun 2011 02:09:27 -0000 On Jun 11, 2011, at 6:00 PM, mdf@FreeBSD.org wrote: > On Sat, Jun 11, 2011 at 4:07 PM, Warner Losh wrote: >>=20 >> On Jun 11, 2011, at 2:43 PM, David Schultz wrote: >>=20 >>> On Sat, Jun 11, 2011, mdf@freebsd.org wrote: >>>> On Sat, Jun 11, 2011 at 10:18 AM, David Schultz = wrote: >>>>> On Sat, Jun 11, 2011, Adrian Chadd wrote: >>>>>> Hi guys, >>>>>>=20 >>>>>> Has there been any further thought as of late about shipping = kernels >>>>>> with modules only by default, rather than monolithic kernels? >>>>>>=20 >>>>>> I tried this experiment a couple years ago and besides a little >>>>>> trickery with ACPI module loading, it worked out fine. >>>>>>=20 >>>>>> Is there any reason we aren't doing this at the moment? Eg by = having a >>>>>> default loader modules list populated from the kernel config = file? >>>>>=20 >>>>> I've been doing this for years, and it has come in quite handy. >>>>> For instance, when my if_msk gets wedged, the only way to fix it >>>>> short of rebooting seems to be reloading the driver. >>>>>=20 >>>>> One issue, however, is that the boot loader is horrendously slow >>>>> at loading modules. (Either that or my BIOS has a braindead int = 13h >>>>> handler.) Most of these modules aren't actually needed until much >>>>> later in the boot process, so a mechanism to load non-essential >>>>> modules after the file systems are mounted might provide a good >>>>> solution. >>>>=20 >>>> Indeed, at $WORK we're trying to get shutdown -> restart under 2 >>>> minutes. Several seconds of this is moving things *into* the = kernel >>>> that need to be there (disk drivers), and everything else to a = point >>>> in init where modules can be loaded in parallel, using the faster = disk >>>> driver, rather than in serial with slow BIOS handlers. >>>=20 >>> Have you found that drivers can be reliably loaded in parallel >>> these days? I'm always waiting for timeouts on four card readers >>> and two optical drives, so that would be a big win for me. IIRC, >>> nothing can happen in parallel during boot because the scheduler >>> is initialized very late in the process. I'm not a device driver >>> person, but I imagine there might be other assumptions that might >>> get in the way as well. >>=20 >> Loading isn't the problem. The timeouts that you are waiting for are = part of the probe/attach sequence. And that's strictly serialized... >>=20 >=20 > If the timeouts are implemented using sleep(9) then Giant is dropped > while waiting for a timeout, and another driver can begin its > initialization. Thus the time to load all modules becomes roughly the > max of all the timeouts, rather than the sum. Except for the whole single threaded nature of all the bus drivers, this = might work. Making them multi-threaded is a lot of work, and likely = won't result in significant gains. The timeouts people complain about = are already all done in parallel at the end anyway. Warner= From owner-freebsd-arch@FreeBSD.ORG Sun Jun 12 02:15:15 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C7D5C106566B; Sun, 12 Jun 2011 02:15:15 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 7F9C28FC12; Sun, 12 Jun 2011 02:15:15 +0000 (UTC) Received: from 63.imp.bsdimp.com (63.imp.bsdimp.com [10.0.0.63]) (authenticated bits=0) by harmony.bsdimp.com (8.14.4/8.14.3) with ESMTP id p5C2BLQb058589 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Sat, 11 Jun 2011 20:11:21 -0600 (MDT) (envelope-from imp@bsdimp.com) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: Date: Sat, 11 Jun 2011 20:10:55 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <48A8C532-0972-49F8-BBA3-B0663E0D5067@bsdimp.com> References: <20110611171834.GA38142@zim.MIT.EDU> <20110611204326.GA51320@zim.MIT.EDU> <9349A935-F13D-4265-A59C-C1E9B35F2B73@bsdimp.com> To: "K. Macy" X-Mailer: Apple Mail (2.1084) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (harmony.bsdimp.com [10.0.0.6]); Sat, 11 Jun 2011 20:11:21 -0600 (MDT) Cc: mdf@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [RFC] shipping kernels with default modules? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Jun 2011 02:15:15 -0000 On Jun 11, 2011, at 5:51 PM, K. Macy wrote: >>> Although I imagine that many drivers silently benefit from being >>> loaded serially, to the best of my knowledge there is nothing >>> architecturally requiring this apart from the fact that the = scheduler >>> isn't started until everything else tied to initialization happens. >>> The absence of any sort of preemption was a bit of a thorn in my = side >>> back when I was working on "xenbus", as the linux implementation >>> relies on the use of multiple thread contexts. I don't know how much >>> effort to date has been put in to making boot fast. >>=20 >> Right now newbus uses Giant for all its locking. That's the biggest = problem preventing parallel probe/attach. Also, each and every bus = calls probe, then calls attach for each device in sequence. Fixing that = would require changing all the bus drivers. >=20 >=20 > Fair enough. That would only be worthwhile in the presence of a > coordinated push to shorten boot / reset times. Agreed. We could be a *LOT* faster if we stalled on first use rather = than wait for all the stragglers. No need to wait for network devices, = or that USB drive that's connected to the system until ifconfig or mount = time. I think that has to be in the mix too. But that dove-tails into = something like launchd where applications wait for the resources they = use to finish initializing. Warner= From owner-freebsd-arch@FreeBSD.ORG Sun Jun 12 02:20:18 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3D9EB106564A for ; Sun, 12 Jun 2011 02:20:18 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-yi0-f54.google.com (mail-yi0-f54.google.com [209.85.218.54]) by mx1.freebsd.org (Postfix) with ESMTP id D46398FC12 for ; Sun, 12 Jun 2011 02:20:17 +0000 (UTC) Received: by yic13 with SMTP id 13so207053yic.13 for ; Sat, 11 Jun 2011 19:20:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=+m/DS11mm+GnelvDYchi7qR0YbHHxNtp3969DYPVjBg=; b=WOg7wLZYbmQZBZuCCFRLYPKVX01s0hPySS2JHA7gmeCt+7vH2cN4Gl76lOLULmEnsi HhwSFnnVJD7R5vijuFsGpxF7vipEBGrcdX9Wf2Kbu+IGo/wBjxaiRGQaAosfddljbCeJ G2jLlJ3l1ENKg1pR5LoGbNHgmmGGXzNE7A8As= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=OeBQv5zLJRDfFvChDX1EyUUVbmFa0qkfmze9+pf/RLMQnlCAZfHJN1T9NCdGuUE3BX vVsZPYztmCMIDlYSXWMClmGxBQJ15G2u6NIdBmC9wsZuOOXS+JTSZb/QDedkBBZ5QjSx q52OmXiVEZYzwRVDrNiXwakfqeTRTB5vaST6o= MIME-Version: 1.0 Received: by 10.150.13.15 with SMTP id 15mr4776904ybm.103.1307845216497; Sat, 11 Jun 2011 19:20:16 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.150.216.3 with HTTP; Sat, 11 Jun 2011 19:20:16 -0700 (PDT) In-Reply-To: <48A8C532-0972-49F8-BBA3-B0663E0D5067@bsdimp.com> References: <20110611171834.GA38142@zim.MIT.EDU> <20110611204326.GA51320@zim.MIT.EDU> <9349A935-F13D-4265-A59C-C1E9B35F2B73@bsdimp.com> <48A8C532-0972-49F8-BBA3-B0663E0D5067@bsdimp.com> Date: Sun, 12 Jun 2011 10:20:16 +0800 X-Google-Sender-Auth: con1sVBqtNCfAcNwzucOHH9XK8s Message-ID: From: Adrian Chadd To: Warner Losh Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: mdf@freebsd.org, "K. Macy" , freebsd-arch@freebsd.org Subject: Re: [RFC] shipping kernels with default modules? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Jun 2011 02:20:18 -0000 On 12 June 2011 10:10, Warner Losh wrote: >>> Right now newbus uses Giant for all its locking. =A0That's the biggest = problem preventing parallel probe/attach. =A0Also, each and every bus calls= probe, then calls attach for each device in sequence. =A0Fixing that would= require changing all the bus drivers. >> >> >> Fair enough. That would only be worthwhile in the presence of a >> coordinated push to shorten boot / reset times. > > Agreed. =A0We could be a *LOT* faster if we stalled on first use rather t= han wait for all the stragglers. =A0No need to wait for network devices, or= that USB drive that's connected to the system until ifconfig or mount time= . =A0I think that has to be in the mix too. =A0But that dove-tails into som= ething like launchd where applications wait for the resources they use to f= inish initializing. How about first getting a modular GENERIC working for different platforms, with the relevant metadata to load in modules that are needed at loader time, or the relevant loader.conf snippet to emulate the GENERIC behaviour (and people can just comment out modules they don't need from that!). Then worry about why loading modules from loader takes so long (if it does, I've never benchmarked it!) Parallelising probe/attach where possible, which may be a good idea, seems like an orthogonal problem. Adrian From owner-freebsd-arch@FreeBSD.ORG Sun Jun 12 06:36:40 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A2930106566C for ; Sun, 12 Jun 2011 06:36:40 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 604508FC16 for ; Sun, 12 Jun 2011 06:36:40 +0000 (UTC) Received: by ywf7 with SMTP id 7so2260849ywf.13 for ; Sat, 11 Jun 2011 23:36:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=f0c432LlABAZcnmjcMdtVMW6ab3DGYXLstua7yn4Tr4=; b=ReKgM+/n8Z2WQiLdERLelqeYrMAwy6cz9LVYPmKEIMB93jszX4dD7Q/RdunX6uSzWV nC01SuRHvQMYRDD43cLVNAfW4VToMr4sXr8jV8O7zWnvN9GqgnRgCZRPGDdofAvR0RwQ 9FTbbWhftqWntODOdv5lnYJOMEi35NbA73CZE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; b=hFecKTYVQ+wkABtpzzbKSWsBNi3fuFJ7wbTZmAYpperb6aiu3nZqCR8IhtE9Tgpor7 bjmC/ul6OerTjC+vIAFIebn9vVRAW65YMXg1DK1I5BdGFG9DdcI8aTkvtkU5roIrtFUj eEgEqc+QY11lXHwEHTl/rcBsuHuF9nlh20I0Y= MIME-Version: 1.0 Received: by 10.151.25.11 with SMTP id c11mr4698580ybj.445.1307860599804; Sat, 11 Jun 2011 23:36:39 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.150.216.3 with HTTP; Sat, 11 Jun 2011 23:36:39 -0700 (PDT) In-Reply-To: References: Date: Sun, 12 Jun 2011 14:36:39 +0800 X-Google-Sender-Auth: wBMx_I08aoCKaC_m-LxSzT1B_Ho Message-ID: From: Adrian Chadd To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Re: [RFC] shipping kernels with default modules? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Jun 2011 06:36:40 -0000 On 11 June 2011 17:21, Adrian Chadd wrote: > Is there any reason we aren't doing this at the moment? Eg by having a > default loader modules list populated from the kernel config file? The immediate problem - how does one get the config parameters from the kernel configuration file to appear in the per-module build setup? Or, in a more general sense, how do you do per-module configuration? Adrian From owner-freebsd-arch@FreeBSD.ORG Sun Jun 12 17:54:33 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A51C61065688; Sun, 12 Jun 2011 17:54:33 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 43EF38FC08; Sun, 12 Jun 2011 17:54:33 +0000 (UTC) Received: from 63.imp.bsdimp.com (63.imp.bsdimp.com [10.0.0.63]) (authenticated bits=0) by harmony.bsdimp.com (8.14.4/8.14.3) with ESMTP id p5CHpCWb065144 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Sun, 12 Jun 2011 11:51:13 -0600 (MDT) (envelope-from imp@bsdimp.com) Mime-Version: 1.0 (Apple Message framework v1084) From: Warner Losh In-Reply-To: Date: Sun, 12 Jun 2011 11:51:11 -0600 Message-Id: References: To: Adrian Chadd X-Mailer: Apple Mail (2.1084) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (harmony.bsdimp.com [10.0.0.6]); Sun, 12 Jun 2011 11:51:13 -0600 (MDT) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-arch@FreeBSD.org Subject: Re: [RFC] shipping kernels with default modules? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Jun 2011 17:54:33 -0000 On Jun 12, 2011, at 12:36 AM, Adrian Chadd wrote: > On 11 June 2011 17:21, Adrian Chadd wrote: >=20 >> Is there any reason we aren't doing this at the moment? Eg by having = a >> default loader modules list populated from the kernel config file? >=20 > The immediate problem - how does one get the config parameters from > the kernel configuration file to appear in the per-module build setup? For options, we've been doing that for years. Config generates the = opt_foo.h files, and they are picked up by the modules. > Or, in a more general sense, how do you do per-module configuration? Generally, the modules follow the global settings, since we've never = tested building some network drivers with INET and some without, for = example. Or did you have something else in mind when you asked? Warner= From owner-freebsd-arch@FreeBSD.ORG Sun Jun 12 18:50:39 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11DF3106566C for ; Sun, 12 Jun 2011 18:50:39 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) by mx1.freebsd.org (Postfix) with ESMTP id 968CF8FC0C for ; Sun, 12 Jun 2011 18:50:38 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 4B73525D3893; Sun, 12 Jun 2011 18:50:37 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 5D9E615A182D; Sun, 12 Jun 2011 18:50:36 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id 81q0skfIZqmC; Sun, 12 Jun 2011 18:50:35 +0000 (UTC) Received: from orange-en1.sbone.de (orange-en1.sbone.de [IPv6:fde9:577b:c1a9:31:cabc:c8ff:fecf:e8e3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 3918F15A1819; Sun, 12 Jun 2011 18:50:34 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: "Bjoern A. Zeeb" In-Reply-To: Date: Sun, 12 Jun 2011 18:50:34 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Warner Losh X-Mailer: Apple Mail (2.1084) Cc: freebsd-arch@FreeBSD.org Subject: Re: [RFC] shipping kernels with default modules? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Jun 2011 18:50:39 -0000 On Jun 12, 2011, at 5:51 PM, Warner Losh wrote: >=20 > On Jun 12, 2011, at 12:36 AM, Adrian Chadd wrote: >=20 >> On 11 June 2011 17:21, Adrian Chadd wrote: >>=20 >>> Is there any reason we aren't doing this at the moment? Eg by having = a >>> default loader modules list populated from the kernel config file? >>=20 >> The immediate problem - how does one get the config parameters from >> the kernel configuration file to appear in the per-module build = setup? >=20 > For options, we've been doing that for years. Config generates the = opt_foo.h files, and they are picked up by the modules. >=20 >> Or, in a more general sense, how do you do per-module configuration? >=20 > Generally, the modules follow the global settings, since we've never = tested building some network drivers with INET and some without, for = example. What do you mean? I have been doing that for 2 or is it 3 years; you = can even see which ones depend on it in sys/conf/makeLINT.mk these days = ;) > Or did you have something else in mind when you asked? --=20 Bjoern A. Zeeb You have to have visions! Stop bit received. Insert coin for new address family. From owner-freebsd-arch@FreeBSD.ORG Sun Jun 12 19:40:42 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8AC20106566C for ; Sun, 12 Jun 2011 19:40:42 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 45CE88FC08 for ; Sun, 12 Jun 2011 19:40:42 +0000 (UTC) Received: from 63.imp.bsdimp.com (63.imp.bsdimp.com [10.0.0.63]) (authenticated bits=0) by harmony.bsdimp.com (8.14.4/8.14.3) with ESMTP id p5CJd75O066059 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Sun, 12 Jun 2011 13:39:08 -0600 (MDT) (envelope-from imp@bsdimp.com) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: Date: Sun, 12 Jun 2011 13:39:02 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <118F0BE0-2F1A-4106-8050-F6FE1321E106@bsdimp.com> References: To: "Bjoern A. Zeeb" X-Mailer: Apple Mail (2.1084) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (harmony.bsdimp.com [10.0.0.6]); Sun, 12 Jun 2011 13:39:08 -0600 (MDT) Cc: freebsd-arch@freebsd.org Subject: Re: [RFC] shipping kernels with default modules? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Jun 2011 19:40:42 -0000 On Jun 12, 2011, at 12:50 PM, Bjoern A. Zeeb wrote: >=20 > On Jun 12, 2011, at 5:51 PM, Warner Losh wrote: >=20 >>=20 >> On Jun 12, 2011, at 12:36 AM, Adrian Chadd wrote: >>=20 >>> On 11 June 2011 17:21, Adrian Chadd wrote: >>>=20 >>>> Is there any reason we aren't doing this at the moment? Eg by = having a >>>> default loader modules list populated from the kernel config file? >>>=20 >>> The immediate problem - how does one get the config parameters from >>> the kernel configuration file to appear in the per-module build = setup? >>=20 >> For options, we've been doing that for years. Config generates the = opt_foo.h files, and they are picked up by the modules. >>=20 >>> Or, in a more general sense, how do you do per-module configuration? >>=20 >> Generally, the modules follow the global settings, since we've never = tested building some network drivers with INET and some without, for = example. >=20 > What do you mean? I have been doing that for 2 or is it 3 years; you = can even see which ones depend on it in sys/conf/makeLINT.mk these days = ;) Sorry, I mean we do generally build ALL of them without INET or ALL of = them with INET. While some people sometimes do things like load em with = INET and say cxgb without INET in the same system, in general this = hasn't been widely tested. Warner= From owner-freebsd-arch@FreeBSD.ORG Sun Jun 12 21:11:26 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ED14B1065672 for ; Sun, 12 Jun 2011 21:11:26 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) by mx1.freebsd.org (Postfix) with ESMTP id 7CD7B8FC08 for ; Sun, 12 Jun 2011 21:11:26 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 7B5DD25D3893; Sun, 12 Jun 2011 21:11:25 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id D8A4C15A19C3; Sun, 12 Jun 2011 21:11:24 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id cl+Tlezq2cxP; Sun, 12 Jun 2011 21:11:23 +0000 (UTC) Received: from orange-en1.sbone.de (orange-en1.sbone.de [IPv6:fde9:577b:c1a9:31:cabc:c8ff:fecf:e8e3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 83AA015A19C1; Sun, 12 Jun 2011 21:11:23 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: "Bjoern A. Zeeb" In-Reply-To: <118F0BE0-2F1A-4106-8050-F6FE1321E106@bsdimp.com> Date: Sun, 12 Jun 2011 21:11:22 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <118F0BE0-2F1A-4106-8050-F6FE1321E106@bsdimp.com> To: Warner Losh X-Mailer: Apple Mail (2.1084) Cc: freebsd-arch@freebsd.org Subject: Re: [RFC] shipping kernels with default modules? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Jun 2011 21:11:27 -0000 On Jun 12, 2011, at 7:39 PM, Warner Losh wrote: >>> Generally, the modules follow the global settings, since we've never = tested building some network drivers with INET and some without, for = example. >>=20 >> What do you mean? I have been doing that for 2 or is it 3 years; you = can even see which ones depend on it in sys/conf/makeLINT.mk these days = ;) >=20 > Sorry, I mean we do generally build ALL of them without INET or ALL of = them with INET. While some people sometimes do things like load em with = INET and say cxgb without INET in the same system, in general this = hasn't been widely tested. Oh, I see. I'd say "if at all unless by accident" in that case;) --=20 Bjoern A. Zeeb You have to have visions! Stop bit received. Insert coin for new address family. From owner-freebsd-arch@FreeBSD.ORG Mon Jun 13 17:16:33 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1A0FA106564A for ; Mon, 13 Jun 2011 17:16:33 +0000 (UTC) (envelope-from opensourcesurvey@cs.ua.edu) Received: from mailapp-2.ua.edu (mailapp-2.ua.edu [130.160.4.237]) by mx1.freebsd.org (Postfix) with ESMTP id DCE438FC14 for ; Mon, 13 Jun 2011 17:16:32 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AhYSAKg+9k2CoC+0/2dsb2JhbABMBhuXd4VuiDZ3qT+WBoh3gzOCcQSHDY5oK4QkAYZq X-IronPort-AV: E=Sophos;i="4.65,359,1304312400"; d="scan'208";a="242962251" Received: from unknown (HELO SHE202502) ([130.160.47.180]) by smtp.ua.edu with ESMTP; 13 Jun 2011 11:47:48 -0500 From: "Jeffrey Carver" To: Date: Mon, 13 Jun 2011 11:47:48 -0500 Message-ID: <00d901cc29e9$a09fae80$e1df0b80$@cs.ua.edu> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 14.0 Thread-Index: Acwp6Z9m9eXKzSFiSymYvDuNWQNXjw== Content-Language: en-us Subject: Participation Requested: Survey about Open-Source Software Development X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Jun 2011 17:16:33 -0000 Hi, Drs. Jeffrey Carver, Rosanna Guadagno, Debra McCallum, and Mr. Amiangshu Bosu, University of Alabama, and Dr. Lorin Hochstein, University of Southern California, are conducting a survey of open-source software developers. This survey seeks to understand how developers on distributed, virtual teams, like open-source projects, interact with each other to accomplish their tasks. You must be at least 19 years of age to complete the survey. The survey should take approximately 15 minutes to complete. If you are actively participating as a developer, please consider completing our survey. Here is the link to the survey: http://goo.gl/HQnux We apologize for inconvenience and if you receive multiple copies of this email. This survey has been approved by The University of Alabama IRB board. Thanks, Dr. Jeffrey Carver Assistant Professor University of Alabama (v) 205-348-9829 (f) 205-348-0219 http://www.cs.ua.edu/~carver From owner-freebsd-arch@FreeBSD.ORG Sat Jun 18 12:50:54 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D1298106566B for ; Sat, 18 Jun 2011 12:50:54 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 8DF718FC0A for ; Sat, 18 Jun 2011 12:50:54 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QXuki-00013I-Lu for freebsd-arch@freebsd.org; Sat, 18 Jun 2011 14:35:52 +0200 Received: from 42-56.dsl.iskon.hr ([89.164.42.56]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 18 Jun 2011 14:35:52 +0200 Received: from ivoras by 42-56.dsl.iskon.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 18 Jun 2011 14:35:52 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Ivan Voras Date: Sat, 18 Jun 2011 14:35:44 +0200 Lines: 23 Message-ID: References: <20110611171834.GA38142@zim.MIT.EDU> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 42-56.dsl.iskon.hr User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.17) Gecko/20110414 Thunderbird/3.1.10 In-Reply-To: <20110611171834.GA38142@zim.MIT.EDU> Subject: Re: [RFC] shipping kernels with default modules? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Jun 2011 12:50:54 -0000 On 11/06/2011 19:18, David Schultz wrote: > OS X has an interesting solution, intended to preserve the > flexibility of dynamic modules, while minimizing boot time. > It provides a kextcache utility, which packages the kernel > and all of the needed modules into a single binary for better > locality on disk. Unlike recompiling the kernel, running > kextcache is fast, and the system runs it automatically when > hardware or driver changes necessitate it. This seems like the goldilocks solution, but also dangerously close to the Linux's horrendeous initrd system where, if you change the disc controller (which nowadays can mean simply switching it in BIOS from ATA to AHCI to RAID modes) and you get a magically unbootable system. I'd go for something like that only if there's a fallback mode acting like "ok, the previous boot failed, now automagically load ALL the modules and try again". You pretty much need to load all the drivers because except for some obvious exceptions (like the sound drivers, VESA, etc.) many can host devices for file systems ("normal" controllers, network cards, iSCSI, GEOM, USB, Infiniband, etc.). Doing it without this kind of fallback mode is a step backwards. From owner-freebsd-arch@FreeBSD.ORG Sat Jun 18 15:21:51 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7FF6E1065670 for ; Sat, 18 Jun 2011 15:21:51 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 1267B8FC08 for ; Sat, 18 Jun 2011 15:21:50 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QXxLK-00056Z-3e for freebsd-arch@freebsd.org; Sat, 18 Jun 2011 17:21:50 +0200 Received: from 27-58.dsl.iskon.hr ([89.164.27.58]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 18 Jun 2011 17:21:50 +0200 Received: from ivoras by 27-58.dsl.iskon.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 18 Jun 2011 17:21:50 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Ivan Voras Date: Sat, 18 Jun 2011 17:21:38 +0200 Lines: 41 Message-ID: References: <4DF3B532.6020908@FreeBSD.org> <20110611201703.GO48734@deviant.kiev.zoral.com.ua> <4DF3D4EB.5030805@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 27-58.dsl.iskon.hr User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.17) Gecko/20110414 Thunderbird/3.1.10 In-Reply-To: <4DF3D4EB.5030805@FreeBSD.org> Subject: Re: [RFC] shipping kernels with default modules? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Jun 2011 15:21:51 -0000 On 11/06/2011 22:49, Doug Barton wrote: > And no one has ever been surprised that assumptions proved invalid in > the light of actual testing? :) Theorizing on this point is of > less-than-zero utility. Who is going to volunteer to do the actual > benchmarking? For what it's worth, I've done the comparison with GEOM_ZERO and "diskinfo -vt". The results are surprising. I've done about 7 iterations of the diskinfo benchmark in each case (module and compiled-in) and analyzed all their outputs collectively with ministat - as there is no actually IO performed, all operations are basically NOPs no matter what seeks are issued. Here's the ministat analysis of "seek times" (x is as module, + is as directly compiled; GENERIC is used in both cases, AMD64 8.2-RELEASE): x 70 0.021 0.031 0.028 0.027585714 0.0016637034 + 98 0.025 0.033 0.031 0.02972449 0.0020548189 Difference at 95.0% confidence 0.00213878 +/- 0.000583402 7.7532% +/- 2.11487% (Student's t, pooled s = 0.00190204) Here's the analysis for bandwidth part of the benchmark (same symbols): N Min Max Median Avg Stddev x 30 1785465 2096170 1909806 1915060.4 77626.781 + 42 1777623 2083800 1819215 1833298.4 62484.689 Difference at 95.0% confidence -81761.9 +/- 32966.2 -4.26942% +/- 1.72142% (Student's t, pooled s = 69161.2) It looks like for both benchmarks, the results are better for the module case (lower seek times, higher bandwidth). As the results were surprising, I did ABAB interleaving of the two cases (module vs built-in).