Date: Sun, 30 Mar 1997 10:25:38 +0100 (BST) From: Doug Rabson <dfr@nlsystems.com> To: current@freebsd.org Subject: A new Kernel Module System Message-ID: <Pine.BSF.3.95q.970330101633.1828A-100000@kipper.nlsystems.com>
next in thread | raw e-mail | index | archive | help
I want to re-vamp our kernel module system. The current system is fine as
far as it goes but does have some serious limitations. I want to replace
the current system with a new design which should be more flexible and
should allow many more (hopefully all) device drivers to be supported as
loadable modules.
To avoid stepping on anyone's toes too early, I have written a paper
showing how the new system will work. I would appreciate feedback,
flaming or otherwise :-).
--------------------------cut here------------------------
A new Kernel Module System
A proposal to replace the current LKM system with a new
implementation allowing both static and dynamically loaded
modules.
1. The current LKM system
1.1. Description
The current LKM system only supports dynamically loaded modules.
Each module is either one of a small number of specially supported
types or is a `catch all' misc module. The modules are a.out
object files which are linked against the kernel's symbol table
using ld(1). Each module has a single entry point which is called
when the module is loaded and unloaded.
1.2. Lifecycle of an LKM
The user initiates a module load (either by mounting a filesystem
or by explicitly calling modload(8)).
The module is loaded in three stages. First memory in the kernel
address space is allocated for the module. Second, ld(1) is used
to link the module's object file to run at the address which has
been allocated for it. The kernel's symbol table is used to
resolve external symbol references from the module. Lastly the
relocated module is loaded into the kernel.
The first thing the kernel does with the new module is to call its
entry point to inform it that it has been loaded. If this call
returns an error (e.g. because a device probe failed), the module
is discarded. For syscalls, filesystems, device drivers and exec
format handlers, common code in the lkm subsystem handles this
load event.
When the module is no longer needed, it can be unloaded using
modunload(8). The module's entry point is called to inform it of
the event (again this is handled in common code for most modules)
and the kernel's memory is reclaimed.
1.3. Limitations
Since the link stage is performed outside the kernel, modules can
only be loaded after the system is fully initialised (or at least
until after filesystems have been mounted). This makes automatic
module loading during boot hard or impossible. Kernel initiated
module loads (e.g. as a result of detecting a PCI device which is
supported by a driver in a module) are virtually impossible.
Statically loaded drivers initialise themselves using SYSINIT(9)
along with various tables created by config(8) to add their
entries to the various device switch tables. Making a statically
loaded driver into a loadable module requires extra code to mimic
this process. As a result, most drivers cannot be built as
modules.
2. A new module system
2.1. Features
· Support for both statically and dynamically loaded modules.
· Dynamically loaded modules are relocated and linked by the
kernel using a built in kernel symbol table.
· Static loaded modules are identical to dynamic modules in every
way. To include a static module in a kernel, the module's
object file is simply included in the kernel's link.
· Modules initialise and register themselves with the kernel using
SYSINIT(9).
· All devices drivers and filesystems and other subsystems are
implemented as modules.
· Statically loaded modules are informed when the system shuts
down. System shutdown would appear to a statically loaded
module as an unload event. Various drivers use at_shutdown(9)
to tidy up device state before rebooting. This process can
happen from the module's unload handler.
· A desirable feature would be to support dependencies between
modules. Each module would define a symbol table. If a module
depends upon another, the dependant module's symbol table is
used to resolve undefined symbols.
2.2. Kernel configuration
Statically loaded modules are specified by a kernel configuration
file, either implicitly by a controller, disk, tape or device
keyword or explicitly with a new module keyword.
2.3. Devices
Several types of device exist. Currently devices are configured
into a kernel using various tables built by config(8) and ld(1).
To make it easier to add devices and drivers to a running kernel,
I suggest that all drivers use SYSINIT(9) to register themselves
with the system.
2.3.1. ISA devices
Currently ISA devices are included in a kernel by using config(8)
to generate a list of device instances in ioconf.c which reference
drivers statically compiled into the kernel. Few drivers support
dynamic loading and those that do have hardcoded device instances
built into the LKM (see sys/i386/isa/joy.c for an example).
ISA drivers will register themselves by name using SYSINIT(9).
This would happen either at boot time for static drivers or at
module load time for dynamic drivers.
Device instances (struct isa_device) will refer to their driver by
name rather than by pointer. The name to driver mapping is
performed and the device is probed and attached as normal.
Statically configured devices are placed in a table by config(8)
and modules containing their drivers are added to the kernel
Makefile.
When an ISA device is configured dynamically, first the module
which contains its driver is loaded if not already present and
secondly a system call is used to create a new device instance and
to call the driver to probe and attach the new device. It is
probably worth writing a new utility, isaconf(8), which can add
new ISA device instances to the kernel.
A desirable feature for a new module system would be to allow
drivers to `detach' themselves from device instances, allowing a
dynamically loaded driver to be unloaded cleanly.
2.3.2. PCI devices
Currently each PCI driver has a pci_device structure which is
included in a linker set. When a device is detected by the boot
process, all driver probe functions are called in turn until the
device is recognised.
Again, in the new system, drivers should use SYSINIT(9) to
register themselves with the PCI subsystem. Instead of a linker
set, the pci_device (shouldn't this be pci_driver?) structures
would be held on a linked list built by this registration process.
At boot time, the PCI bus is scanned to generate a list of PCI
device instances. The probes of all currently loaded drivers are
called for these device instances. Any devices which are not
assigned drivers are remembered. When a new driver module is
loaded, the PCI subsystem re-scans unassigned devices when the
module registers itself. This is broadly what happens for PCI
drivers contained in LKMs today.
If a driver is unloaded, it releases any resources such as
interrupts allocated for devices attached to it. These devices
become unassigned, as if they were not successfully probed. This
allows driver developers to repeatedly load and unload modules
without rebooting.
2.4. Dynamic loading
Supporting static as well as dynamic modules makes the single
module per object file paradigm of the existing LKM system
difficult to maintain. A better approach is to separate the idea
of a kernel module (a single kernel subsystem) from the idea of a
kernel object file. The boot kernel should be thought of as
simply a kernel object file which contains the modules that were
configured statically. Dependencies between modules are also
better treated as dependencies between object files (since they
are typically linking dependencies).
The new system will use a kernel linker which can load object
files into the kernel address space. After loading, sysinits from
the new object file are run, allowing any modules contained
therein to register themselves. The linker will keep track of
which modules are contained in which object so that when a user
unloads the object, the modules can be informed of the event.
Each object has a symbol table associated with it. When linking a
new object into the kernel, the symbol tables of all the objects
which it depends on are used to resolve undefined references. All
modules implicitly depend on the boot kernel object and therefore
can use symbols from the base kernel. Since objects have private
symbol tables, a symbol with the same name can be used in more
than one object without conflict; this is not a practice to be
encouraged since it prevents modules in those objects from being
linked statically.
Each object has a reference count. Each time a dependant object
is loaded, the reference count is increased. The user-initiated
load also increases the reference count by one. To unload a
module, the user will simply release this reference. The object
will only be actually removed from the kernel if its reference
count reaches zero. This scheme allows the automatic unloading of
dependant modules as well as preventing an object from being
removed while it is still in use.
--
Doug Rabson Mail: dfr@nlsystems.com
Nonlinear Systems Ltd. Phone: +44 181 951 1891
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.3.95q.970330101633.1828A-100000>