From owner-freebsd-hackers@freebsd.org Sun May 15 16:01:00 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 97DEAB3C970 for ; Sun, 15 May 2016 16:01:00 +0000 (UTC) (envelope-from oliver.pinter@hardenedbsd.org) Received: from mail-wm0-x235.google.com (mail-wm0-x235.google.com [IPv6:2a00:1450:400c:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2204A1B40 for ; Sun, 15 May 2016 16:01:00 +0000 (UTC) (envelope-from oliver.pinter@hardenedbsd.org) Received: by mail-wm0-x235.google.com with SMTP id e201so74974640wme.0 for ; Sun, 15 May 2016 09:01:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=JIazMmnoKT8IQBm+Y/Rd6YaE/qh6TY50UAoKNHScYLI=; b=llqgj2UiPO0nFmCr5oSQwuyv5e2JfVR5Hc7KJk+3IKAzfl6gD1w801Cg2ERfG84rCz eYLFzF6AkUXG3nKAMCKlpjS7T40e+gO2EjDSVvmrPiab2Q+6rHLrT84mQw+cwHlEp87j tGiEphlUs0Z7ymGB8/bVvy3/JHNMbiPo3OsHoEuSg2HjSgw9c6N9U7CGBF94/4B82W35 ssjQnOlVsi/rqKmVAVgGt2h9tnPvrYW5hfcs9aI5Xu37rt+Zvglj8zpurjgj32fykdDr eTYfLTkfCsytv5sYaiXKmgG591I4ZVnRKwKvbjqJJzTyx5l6b8Qi24y2XoncU8JGdMQt bzng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=JIazMmnoKT8IQBm+Y/Rd6YaE/qh6TY50UAoKNHScYLI=; b=RUCJH/v5Sbw2gOL9FMjyXJoeLe2+YaMn1Vk1ghN5kSE0v6eGW0iPVYD3UENP5Ht1Fb v9LAn0xGsDOrx32BHWbmLPIHvvBQKKhRFFooOT7roiotg8XgfOAvU8f0SDz0IjW4AttM grzc/k+0MZsQxpIkSTJox6ijzkjqPIXHcnTR6qN9yKwtYnebvMnfRep1tYbwlooyppbT +cVx42jQDV/La/tZ9N4XNNjRDIfnwHnaS6+DorkEsvlhOwcah5tA1oyaejpjTc6cafIm RZvkvkG1fqadZrNCYYqPeGbjesl0CtiRslGv0mTM1DhCj+ZQPN+/mBC7SW75qmO90YLO IUlA== X-Gm-Message-State: AOPr4FXTqship3IEYDRvsnPGnsVASC9aKAefYoF1C3dLzRuMMGXr+SsZZ1F2voIUpE+M7oazxUX2uQ6C/+F2l14g MIME-Version: 1.0 X-Received: by 10.194.220.230 with SMTP id pz6mr24335318wjc.93.1463328058733; Sun, 15 May 2016 09:00:58 -0700 (PDT) Received: by 10.194.30.35 with HTTP; Sun, 15 May 2016 09:00:58 -0700 (PDT) In-Reply-To: References: Date: Sun, 15 May 2016 18:00:58 +0200 Message-ID: Subject: Re: Update on EFI work: refactoring ready for testing, GELI coming very soon From: Oliver Pinter To: Eric McCorkle , nao@enuenu.org Cc: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 May 2016 16:01:00 -0000 Hi Eric! Could you please take a look at this PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=207940 This feature would be nice too. On 5/15/16, Eric McCorkle wrote: > Hello everyone, > > I've been working on a rather significant refactoring of the EFI boot/loader > code. While this was originally in support of adding GELI support, it has > grown to such a scope that it could be considered a patch in its own right. > > The branch containing my work can be found here: > https://github.com/emc2/freebsd/tree/efize > > The following is a summary of the changes: > * Both boot1 and loader have been redesigned to look for instances of > EFI_SIMPLE_FILESYSTEM_PROTOCOL and load using that interface. In loader, > this is accomplished through a new synthetic filesystem driver (called > efifs) that is just a wrapper around EFI_SIMPLE_FILESYSTEM_PROTOCOL. In > boot, this is achieved by calling the interface directly. > * The efipart and filesystem back ends (including ZFS) have been moved into > a drivers directory, where they have been wrapped up into a filesystem > backend driver that does the same probing that loader does today, and then > installs an EFI_SIMPLE_FILESYSTEM_INTERFACE on all device handles that host > supported filesystems. This interface is a wrapper around the filesystem > interface currently used by loader. > * boot now uses the same filesystem backend code as loader. This increased > its size, necessitating recreation of the FAT templates. The old boot > filesystem code and boot modules have been discarded. > * loader can use any protocol interface installed by boot just fine. These > remain valid until ExitBootServices is called. Moreover, the probing > process is idempotent, and is run by both boot and loader, with the first > one to run actually installing the interfaces. > * I had originally hoped to move the entire code base to use the EFI driver > model, which would support hotplugging devices. However, the new bcache > stuff currently requires that all devices be statically detected before the > caches are used, which is fundamentally incompatible with this way of doing > things. This was the sole blocker of such a transition (the handles.c code > requires some minor modification as well, but nothing problematic) > * I had also considered altering the device name code to use textual > representations of EFI device paths, but I don't see any real reason for > doing so. > * I have not touched efinet or nfs in this changeset. > > The rationale for these changes is as follows: > * The model of looking for EFI_SIMPLE_FILESYSTEM_PROTOCOL instances and > using them to load things increases interoperability with other systems. > For example, the new boot and loader would work just fine with interfaces > installed by GRUB or another boot loader, or perhaps custom filesystem > modules added into an open-source firmware implementation like coreboot. > * This model works really well for functionality like GELI or custom > partition schemes that potentially create new devices. All you do is create > one or more new device nodes, attach device paths and block io interfaces, > and call ConnectController on them (for now, it's necessary to make sure the > fs_driver runs AFTER) all new nodes have been created. This also vastly > simplifies passing information between stages about filesystems and devices > (this is important for GELI). > * This approach can leverage drivers that are mandated by the EFI spec, like > the GPT partition driver. This avoids reimplementing such a driver to > support partition schemes nested inside GELI volumes, for example. > * This model provides most of the groundwork for supporting hot plugging of > devices at boot time. All that is required at this point is a refactoring > of bcache to support adding new devices dynamically (I had to draw the line > somewhere, and this had already gotten big enough, and I want to focus on > GELI support) > * Getting rid of the duplicated and minimized filesystem code in boot1 > improves maintainability. Indeed, the actual boot1 code is quite small > now. > > Some notes and future work: > * In general, the FreeBSD loader framework and the EFI framework do many > similar things. For the most part, there is a fairly direct mapping, though > EFI interfaces tend to be more tedious. The one thing EFI does decidedly > better is support dynamic detection of devices (hotplugging). > * There are some interesting possibilities for hotplugging beyond just the > obvious. For example, loading GELI or other keys off of hotplugged USB > sticks. > * I didn't touch efinet or nfs on this patch. It is certainly possible to > efize them as well, but I don't have a test setup for doing so (or frankly, > the motivation to do so). > * It might make more sense to use the EFI_LOAD_FILE_PROTOCOL to do the > actual loading, as some applications support this without supporting a full > filesystem interface (some embedded devices do this, as do some network boot > protocols). However, this would involve changing the non-EFI boot code and > interfaces, which is something I specifically wanted to avoid in this work. > > This changeset can be tested as is, and should just work. Disclaimer: I > rebased it to head yesterday, and haven't had time to build and test it yet, > but I will. I know it works with ZFS, but I have no UFS systems on which to > test that functionality. > > If you intend to test this, I STRONGLY recommend installing an EFI shell on > your ESP, and then installing the modified boot block under something like > boot.tst. This will allow you to run the modified boot block, but fall back > to a working boot if it fails. Also, I had modified the loader path to > /boot/loader.tst for similar reasons, and it may still be set to that in the > code. > > I am currently adding GELI support as a proper EFI driver, and should be > coming in the very near future (the code is already written, in fact). If > anyone wants to test the refactoring by itself, any results or comments are > certainly appreciated. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" >