From owner-freebsd-hackers@freebsd.org Sun May 15 14:32:34 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 27C39B391F3 for ; Sun, 15 May 2016 14:32:34 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from mail.metricspace.net (mail.metricspace.net [IPv6:2001:470:1f11:617::107]) by mx1.freebsd.org (Postfix) with ESMTP id E76851283 for ; Sun, 15 May 2016 14:32:33 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from [172.16.0.5] (unknown [172.16.0.5]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) (Authenticated sender: eric) by mail.metricspace.net (Postfix) with ESMTPSA id 80CCC148E for ; Sun, 15 May 2016 14:32:27 +0000 (UTC) Subject: Update on EFI work: refactoring ready for testing, GELI coming very soon From: Eric McCorkle Content-Type: text/plain; charset=us-ascii X-Mailer: iPad Mail (13D15) Message-Id: Date: Sun, 15 May 2016 10:32:26 -0400 To: "freebsd-hackers@freebsd.org" Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (1.0) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 May 2016 14:32:34 -0000 Hello everyone, I've been working on a rather significant refactoring of the EFI boot/loader= code. While this was originally in support of adding GELI support, it has g= rown to such a scope that it could be considered a patch in its own right. The branch containing my work can be found here: https://github.com/emc2/fre= ebsd/tree/efize The following is a summary of the changes: * Both boot1 and loader have been redesigned to look for instances of EFI_SI= MPLE_FILESYSTEM_PROTOCOL and load using that interface. In loader, this is a= ccomplished through a new synthetic filesystem driver (called efifs) that is= just a wrapper around EFI_SIMPLE_FILESYSTEM_PROTOCOL. In boot, this is ach= ieved by calling the interface directly. * The efipart and filesystem back ends (including ZFS) have been moved into a= drivers directory, where they have been wrapped up into a filesystem backen= d driver that does the same probing that loader does today, and then install= s an EFI_SIMPLE_FILESYSTEM_INTERFACE on all device handles that host support= ed filesystems. This interface is a wrapper around the filesystem interface= currently used by loader. * boot now uses the same filesystem backend code as loader. This increased i= ts size, necessitating recreation of the FAT templates. The old boot filesy= stem code and boot modules have been discarded. * loader can use any protocol interface installed by boot just fine. These r= emain valid until ExitBootServices is called. Moreover, the probing process= is idempotent, and is run by both boot and loader, with the first one to ru= n actually installing the interfaces. * I had originally hoped to move the entire code base to use the EFI driver m= odel, which would support hotplugging devices. However, the new bcache stuf= f currently requires that all devices be statically detected before the cach= es are used, which is fundamentally incompatible with this way of doing thin= gs. This was the sole blocker of such a transition (the handles.c code requ= ires some minor modification as well, but nothing problematic) * I had also considered altering the device name code to use textual represe= ntations of EFI device paths, but I don't see any real reason for doing so. * I have not touched efinet or nfs in this changeset. The rationale for these changes is as follows: * The model of looking for EFI_SIMPLE_FILESYSTEM_PROTOCOL instances and usin= g them to load things increases interoperability with other systems. For ex= ample, the new boot and loader would work just fine with interfaces installe= d by GRUB or another boot loader, or perhaps custom filesystem modules added= into an open-source firmware implementation like coreboot. * This model works really well for functionality like GELI or custom partiti= on schemes that potentially create new devices. All you do is create one or= more new device nodes, attach device paths and block io interfaces, and cal= l ConnectController on them (for now, it's necessary to make sure the fs_dri= ver runs AFTER) all new nodes have been created. This also vastly simplifies= passing information between stages about filesystems and devices (this is i= mportant for GELI). * This approach can leverage drivers that are mandated by the EFI spec, like= the GPT partition driver. This avoids reimplementing such a driver to supp= ort partition schemes nested inside GELI volumes, for example. * This model provides most of the groundwork for supporting hot plugging of d= evices at boot time. All that is required at this point is a refactoring of= bcache to support adding new devices dynamically (I had to draw the line so= mewhere, and this had already gotten big enough, and I want to focus on GELI= support) * Getting rid of the duplicated and minimized filesystem code in boot1 impro= ves maintainability. Indeed, the actual boot1 code is quite small now. Some notes and future work: * In general, the FreeBSD loader framework and the EFI framework do many sim= ilar things. For the most part, there is a fairly direct mapping, though EFI= interfaces tend to be more tedious. The one thing EFI does decidedly bette= r is support dynamic detection of devices (hotplugging). * There are some interesting possibilities for hotplugging beyond just the o= bvious. For example, loading GELI or other keys off of hotplugged USB stick= s. * I didn't touch efinet or nfs on this patch. It is certainly possible to e= fize them as well, but I don't have a test setup for doing so (or frankly, t= he motivation to do so). * It might make more sense to use the EFI_LOAD_FILE_PROTOCOL to do the actua= l loading, as some applications support this without supporting a full files= ystem interface (some embedded devices do this, as do some network boot prot= ocols). However, this would involve changing the non-EFI boot code and inte= rfaces, which is something I specifically wanted to avoid in this work. This changeset can be tested as is, and should just work. Disclaimer: I reb= ased it to head yesterday, and haven't had time to build and test it yet, bu= t I will. I know it works with ZFS, but I have no UFS systems on which to t= est that functionality. If you intend to test this, I STRONGLY recommend installing an EFI shell on y= our ESP, and then installing the modified boot block under something like bo= ot.tst. This will allow you to run the modified boot block, but fall back t= o a working boot if it fails. Also, I had modified the loader path to /boot= /loader.tst for similar reasons, and it may still be set to that in the code= . I am currently adding GELI support as a proper EFI driver, and should be com= ing in the very near future (the code is already written, in fact). If anyo= ne wants to test the refactoring by itself, any results or comments are cert= ainly appreciated.=