From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 25 01:01:19 2005 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6C6C516A4CE for ; Fri, 25 Feb 2005 01:01:19 +0000 (GMT) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7E60F43D2D for ; Fri, 25 Feb 2005 01:01:18 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from [192.168.254.11] (junior-wifi.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.1/8.13.1) with ESMTP id j1P11Apj064574 for ; Thu, 24 Feb 2005 18:01:10 -0700 (MST) (envelope-from scottl@samsco.org) Message-ID: <421E7867.9060101@samsco.org> Date: Thu, 24 Feb 2005 17:59:19 -0700 From: Scott Long User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.5) Gecko/20050218 X-Accept-Language: en-us, en MIME-Version: 1.0 To: hackers@freebsd.org Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-2.8 required=3.8 tests=ALL_TRUSTED autolearn=failed version=3.0.2 X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on pooker.samsco.org Subject: Driver Update Disk discussion X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Feb 2005 01:01:19 -0000 The good news is that more and more vendors are supporting FreeBSD directly, many via paid staff to write and maintain drivers, testing labs for QA, and tech support for end-customers. The bad news is that FreeBSD puts up a number of roadblocks that makes their jobs very difficult. While we do a lot to help them get their sources into the tree, the real problem is that we make it very hard for them to provide driver updates that are asynchronous from FreeBSD releases. It is very common for a vendor to want to put out an updated driver with fixed bugs or updated support for a prior release of the OS. The ideal way to do this is by releasing a 'driver update disk' that contains binary modules of the updated driver, along with scripts or instructions for installing it into the system. This system has to be easy and smooth; the vendors often times deal with customers who are not savvy and are in need of an automated, fool-proof solution. It has to be able to cleanly override any existing modules or in-kernel versions of the modules, it has to be able to survive a system recompile, and it has to work in an environment where the system might not be able to boot or install without the updated module. So, in no particular order, the problems that are present right now include: - sysinstall support. Sysinstall has a minor menu for loading KLD's off of a disk, but whatever gets loaded does not get transfered to the new system at all. It's possible to break into a shell and copy and paste the bits by hand, but that's a very poor situation. A framework needs to be in place that lets sysinstall run pre- and post- install actions on a driver disk, with the vendor able to supply custom scripts for these actions. Support for the scripts interacting with the user should also be available (for custom readme's, EULA's, etc). Sysinstall should also be able to arbitrate between a vendor/user supplied driver and the stock driver. The current point where sysinstall allows a vendor disk is too late; the system is already up and all the drivers in the kernel have already probed and attached. This means that overriding an existing driver is impossible. Solving this likely means that the sysinstall kernel is completely stripped down, and all drivers exist as modules that get loaded after sysinstall has started and given the user the option to load a vendor driver. - runtime support. Where will modules be put, how will they be ensured to not collide with the base system modules, and how will the system ensure that they get loaded on every boot. Most of the pieces are in place right now via the loader knowing about /boot/modules, and /boot/loader.conf able to be modified to point to modules in there. What is missing is a formal definition of how these pieces work, and tools to automate and manage it for the user and vendor. - kernel option support. How do we support vendor modules in a kernel that might be compiled with PAE (rather common these days), SMP, MAC, etc. The loader and /boot infrastructure has no concept of this. It's highly important, though. - source code support. A vendor might want to distribute source code for the module that would be picked up in a kernel re-compile. Patching the src/ tree is the only way to do that now, and that creates problems when interacting with cvsup. Solving this would tie into the problem of compiling drivers out of ports/. In fact, a vendor driver, both in source and in binary form, should probably be treated as a port/package, and I describe below. - kernel api/abi. We are trying to keep the kernel api/abi stable now, which helps a lot. However, there is a chance that these could change for legitimate reasons. How do we protect binary-only modules from this? Linux has a fairly draconian system of hashing all of the exported kernel symbols with the kernel options and the kernel major/minor/subminor versions, and making the linker reject modules that don't have the right hash. Should we follow with something similar, or should we have runtime checks that check symbol/structure signatures? Or should we say that we make no guarantees about a binary-only module working on anything but a -RELEASE kernel? - management. How do ge let the user and vendor manage the updated drivers? The vendor might put out a series of updates during the lifecycle of a particular OS release, so we need to provide infrastructure to make this work. The obvious candidate is the ports/packages system, but does that really do all that we need. I need to sit down and think about this a lot before I can say. It would also need to tie in with sysinstall. - kernel namespace. Devclass collisions seem to already be handled gracefully. devfs collisions aren't, and maybe some seatbelts here would be good. Linker symbol collisions are what I really worry about, since they will prevent a vendor from installing an updated driver on top of an existing driver with a overlapping set of non-static symbols. I know that we do some sort of magic symbol hiding while compiling the kernel, but does that apply to modules also? These are the big things I can think of right now. Solving it is a big task, but it's something that we need to do if we want vendor support to continue and to grow. If anyone is interested in helping with the design and engineering of this (this isn't a call for testers!), please let me know. Scott