Date: Fri, 10 Feb 2012 15:10:59 -0800 From: Jeremy Chadwick <freebsd@jdc.parodius.com> To: Alexander Leidinger <Alexander@Leidinger.net> Cc: stable@FreeBSD.org Subject: Re: Reducing the need to compile a custom kernel Message-ID: <20120210231059.GA25777@icarus.home.lan> In-Reply-To: <20120210145604.Horde.ewjpSpjmRSRPNSH0YRHxgAk@webmail.leidinger.net> References: <20120210145604.Horde.ewjpSpjmRSRPNSH0YRHxgAk@webmail.leidinger.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Feb 10, 2012 at 02:56:04PM +0100, Alexander Leidinger wrote:
> during some big discussions in the last monts on various lists, one
> of the problems was that some people would like to use
> freebsd-update but can't as they are using a custom kernel. With all
> the kernel modules we provide, the need for a custom kernel should
> be small, but on the other hand, we do not provide a small
> kernel-skeleton where you can load just the modules you need.
My below comments apply to RELENG_8, amd64. Just noting that here. I
have no plans to upgrade to RELENG_9 or newer.
> This should be easy to change. As a first step I took the generic
> kernel and removed all devices which are available as modules, e.g.
> the USB section consists now only of the USB_DEBUG option (so that
> the module is build like with the current generic kernel). I also
> removed some storage drivers which are not available as a module.
> The rationale is, that I can not remove CAM from the kernel config
> if I let those drivers inside (if those drivers are important
> enough, someone will probably fix the problem and add the missing
> pieces to generate a module).
This is a core (meaning "key") problem as I see it. There is no
definitive list of what drivers can/will work as modules vs. being
compiled in to the kernel statically.
Furthermore, there is evidence on the mailing lists where users
experience different behaviour in drivers when built as modules than if
statically included: em(4) is one such example (I can dig up the post if
needed). Jack often asks me why we include em(4) statically. :-)
> Such a kernel would cover situations where people compile their own
> kernel because they want to get rid of some unused kernel code (and
> maybe even need the memory this frees up).
>
> The question is, is this enough? Or asked differently, why are you
> compiling a custom kernel in a production environment (so I rule out
> debug options zhich are not enabled in GENERIC)? Are there options
> which you add which you can not add as a module (SW_WATCHDOG comes
> to my mind)? If yes, which ones and how important are they for you?
Excellent question, and I would love to share with folks the list of
things we tweak in our kernel config vs. GENERIC:
* Removal: option INET6
* Removal: option SCTP
* Removal: option COMPAT_FREEBSD32
* Removal: option COMPAT_FREEBSD4
* Removal: option COMPAT_FREEBSD5
* Removal: option COMPAT_FREEBSD6
* Removal: option COMPAT_FREEBSD7
* Setting: option PRINTF_BUFR_SIZE=256
* Addition: option KDTRACE_HOOKS
* Addition: option FLOWTABLE
- Note: Despite this, we disable use of flowtable via sysctl due
to severe bugs as we have no indication this has been fixed:
http://lists.freebsd.org/pipermail/freebsd-stable/2010-November/060289.html
* Addition: option BREAK_TO_DEBUGGER
* Addition: option ALT_BREAK_TO_DEBUGGER
* Addition: option KDB
* Addition: option KDB_TRACE
* Addition: option DDB
* Addition: option DDB_NUMSYM
* Addition: option GDB
* Addition: option KTR
* Addition: option KTR_ENTRIES=262144
* Addition: option KTR_COMPILE=(KTR_SCHED)
* Addition: option KTR_MASK=(KTR_SCHED)
* Removal: device ata
- Note: Our config has the following comment:
# NOTE: "device ata" is missing because we use the Modular ATA core
# to only include the ATA-related drivers we need (e.g. AHCI).
* Removal: device atapifd
* Removal: device atapist
* Removal: All device/options relating to SCSI controllers
* Addition: device atacore
* Addition: device ataisa
* Addition: device atapci
* Addition: device ataahci
* Addition: device ataintel
* Removal: device ch
* Removal: device sa
* Removal: All device/options pertaining to RAID controllers
* Removal: All device/options pertaining to PCMCIA/CardBus
* Removal: All device/options pertaining to parallel ports
* Removal: device puc
* Removal: All PCI Ethernet NICs (with and without miibus),
with the exception being em
* Removal: All ISA Ethernet NICs
* Removal: All Wireless NICs
* Removal: device vlan
* Removal: device tun
* Removal: device gif
* Removal: device faith
* Removal: option USB_DEBUG
* Removal: device ulpt
* Removal: All device/options pertaining to USB serial devices
* Removal: All device/options pertaining to USB Ethernet devices
* Removal: All device/options pertaining to USB Wireless devices
* Removal: All device/options pertaining to FireWire
* Addition: device coretemp
* Addition: device smbus
* Addition: device smb
* Addition: device ichsmb
* Addition: device ichwd
- Note: We do not use features of this driver given known problems
with the watchdog firing during ddb> and similar environments. I
have no idea if this has been fixed, but I do remember it being
confirmed as a problem.
* Addition: options ALTQ
* Addition: options ALTQ_CBQ
* Addition: options ALTQ_RED
* Addition: options ALTQ_RIO
* Addition: options ALTQ_HFSC
* Addition: options ALTQ_CDNR
* Addition: options ALTQ_PRIQ
* Addition: options ALTQ_NOPCC
I want to note here: the pf ALTQ options are a pain in the butt, quite
honestly. I've found in the past that removing the ones you don't use
won't result in a successful build, thus one must include them all. We
do need ALTQ support though, for rate-limiting capability. The NOPCC
option is needed for SMP builds, which makes me wonder what the state of
SMP is in this regard -- meaning, on non-SMP builds, is it still safe
to include ALTQ_NOPCC?
Now, most of the device-related ones -- ASSUMING they can be used
reliably as kernel modules -- can probably be ignored (meaning if you
strip them out, per your proposal, that's fine). However, the list of
options above still stands.
Also, there are obviously kernel modules which we're using dynamically,
such as ahci.ko (/boot/loader.conf ahci_load="yes") and some others:
# kldstat
Id Refs Address Size Name
1 20 0xffffffff80100000 1d34f30 kernel
2 1 0xffffffff81e36000 11e48 ahci.ko
3 1 0xffffffff82012000 1313a8 zfs.ko
4 1 0xffffffff82144000 1ff1 opensolaris.ko
5 1 0xffffffff82146000 28c93 pf.ko
6 1 0xffffffff8216f000 7f8 accf_http.ko
7 1 0xffffffff82170000 1f0 accf_data.ko
So in this case, ZFS is loaded dynamically via zfs_enable="yes" in
rc.conf, as is pf. The accf_* combo is pulled in in real-time when one
uses Apache with apache22_http_accept_enable="yes".
All the other options you see that we adjust we have legitimate need
for. I cannot stress (to you and others) how important it is to have
many of the debugging options in there; in the case of a kernel panic,
in almost every case on the mailing lists, the first response from
developers is "can you drop to ddb and..." I realise for folks running
remote/embedded environments these probably aren't what they want, but
for the majority I see them as being needed more and more I watch the
responses from developers.
Finally, just throwing this out there: does anyone know if you can
include ALT_BREAK_TO_DEBUGGER by itself, without BREAK_TO_DEBUGGER?
E.g. we only want <CR>~<Ctrl-b> inducing a break to ddb, not serial
break.
--
| Jeremy Chadwick jdc@parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, US |
| Making life hard for others since 1977. PGP 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20120210231059.GA25777>