Date: Mon, 12 Feb 2018 13:53:59 -0800 From: John Baldwin <jhb@freebsd.org> To: Warner Losh <imp@bsdimp.com> Cc: FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: LUA boot loader coming very soon Message-ID: <27682255.x3aPd9zKdx@ralph.baldwin.cx> In-Reply-To: <CANCZdfoE4fzkPxrMevnxZE76yTZs_5%2BsToVpHjT5ERcfrU%2BTMw@mail.gmail.com> References: <CANCZdfqJzPud2GeMgc=zRNnz%2BQY0MuOft3L3OeQeq5w75NfbQA@mail.gmail.com> <2333272.J6kp8cXjfh@ralph.baldwin.cx> <CANCZdfoE4fzkPxrMevnxZE76yTZs_5%2BsToVpHjT5ERcfrU%2BTMw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Monday, February 12, 2018 02:31:46 PM Warner Losh wrote: > On Mon, Feb 12, 2018 at 10:12 AM, John Baldwin <jhb@freebsd.org> wrote: > > > On Monday, February 12, 2018 08:27:27 AM Warner Losh wrote: > > > Greetings, > > > > > > As you may know, the Lua (http://www.lua.org) boot loader has been in > > the > > > works for some time. It started out life as a GSoC in 2014 by Pedro Souza > > > mentored by Wojciech A. Koszek. Rui Paulo created a svn project branch to > > > try to integrate it. I rebased that effort into a github branch which > > Pedro > > > Arthur fixed up. Over the past year, I've been cleaning up the boot > > loader > > > for other reasons, and found the time was ripe to start integrating this > > > into the tree. However, those integration efforts have taken a while as > > my > > > day-job work on the boot loader took priority. In the mean time, Ed Maste > > > and the FreeBSD Foundation funded Zakary Nafziger to enhance the original > > > GSoC Lua scripts to bring it closer to parity with the evolution of the > > > FORTH menu system since the GSoC project started. > > > > > > I'm pleased to announce that all these threads of development have > > > converged and I'll be pushing the FreeBSD Lua Loader later today. This > > > loader uses Lua as its scripting language instead of FORTH. While > > > co-existance is planned, the timeline for it is looking to be a few weeks > > > and I didn't want to delay pushing this into the tree for that. > > > > > > To try the loader, you'll need to build WITHOUT_FORTH=yes and > > > WITH_LOADER_LUA=yes. Fortunately, you needn't do a full world to do this, > > > you can do it in src/stand and install the result (be sure to have the > > > options for both the build and the install). This will replace your > > current > > > /boot/loader that is scripted with FORTH to one that's scripted with Lua. > > > It will install the lua scripts in /boot/lua. The boot is scripted with > > > /boot/lua/loader.lua instead of /boot/loader.rc. You are strongly advised > > > to create a backup copy of /boot/loader before testing (eg cp > > /boot/loader > > > /boot/loader_forth), since you'll need to boot that from boot2 if > > something > > > goes wrong. I've tested it extensively, though, with userboot.so and it's > > > test program, so all the initial kinks of finding the lua scripts, etc > > have > > > been worked out. > > > > > > While it's possible to build all the /boot/loader variants with Lua, I've > > > just tested a BIOS booting /boot/loader both with and without menus > > > enabled. I've not tested any of the other variants and the instructions > > for > > > testing some of them may be rather tedious (especially UEFI, if you want > > a > > > simple path to back out). Since there's not been full convergence > > testing, > > > you'll almost certainly find bumps in this system. Also, all the > > > build-system APIs are likely not yet final. > > > > > > I put MFC after a month on the commit. Due to the heroic (dare I say > > > almost crazy) work of Kyle Evans on merging all the revs from -current to > > > 11, I'm planning a MFC to 11 after the co-existence issues are hammered > > > out. In 11, FORTH will be the default, and Lua will be built by default, > > > but users will have to do something to use it. 12, both FORTH and Lua > > will > > > be built and installed, with Lua as default (barring unforeseen > > > complications). Once the co-existence stuff goes in, I imagine we'll make > > > the switch to Lua by default shortly after that. In 13, FORTH will be > > > removed unless there's a really really compelling case made to keep it. > > > > > > So please give it a spin and give me any feedback, documentation updates > > > and/or bug fixes. I'm especially interested in reviews from people that > > > have embedded Lua in other projects or experts in Lua that can improve > > the > > > robustness of the menu code. > > > > Do you have some memory usage numbers for LUA vs forth for the different > > BIOS loaders (text/data/bss sizes)? For the EFI case we probably have lots > > of room, but for the non-EFI case we are constrained to 0xa0000 - 0xa000 > > for the text/data/bss and stack (in some cases like PXE booting the top > > can be lower than 0xa0000). I'm not sure if we have any other platforms > > with similar memory constraints. > > > > Lua is about 60-70k larger than FORTH for this application: > > Forth based: > -r-xr-xr-x 1 root wheel 385024 Feb 12 14:14 > /home/imp/roots/amd64/boot/loader > -r-xr-xr-x 1 root wheel 450560 Feb 12 14:14 > /home/imp/roots/amd64/boot/zfsloader > Lua based: > -r-xr-xr-x 1 root wheel 450560 Feb 12 14:20 > /home/imp/roots/amd64/boot/loader > -r-xr-xr-x 1 root wheel 520192 Feb 12 14:20 > /home/imp/roots/amd64/boot/zfsloader > > So with Lua: > % size loader.bin > text data bss dec hex filename > 411840 22264 48304 482408 0x75c68 loader.bin > % size zfsloader.bin > text data bss dec hex filename > 478712 22968 52368 554048 0x87440 zfsloader.bin > > so we're at 555k out of 640k before we look at heap usage. Of course, these > numbers are only going to get a lot worse as we pile-in new crypto > features... I know that the normal loader works, but I've not tried the > zfsloader. Woof, on stable/11 the forth ones are quite a bit smaller: -r-xr-xr-x 1 root wheel 315392 Dec 16 13:39 /boot/loader -r-xr-xr-x 1 root wheel 356352 Dec 16 13:39 /boot/zfsloader % size /usr/obj/usr/src/sys/boot/i386/loader/loader.bin text data bss dec hex filename 274913 22624 46560 344097 0x54021 /usr/obj/usr/src/sys/boot/i386/loader/loader.bin % size /usr/obj/usr/src/sys/boot/i386/zfsloader/zfsloader.bin text data bss dec hex filename 313865 23184 50624 387673 0x5ea59 /usr/obj/usr/src/sys/boot/i386/zfsloader/zfsloader.bin Note that it's not 640k, it's 0xa0000 (640k) - the start address of the loader: 0xa000 (40k), so text/data/bss + stack has 600k of room. The heap isn't in low memory anymore, so we don't have to worry about that. 555k for ZFS loader leaves around 45k for stack. If the ZFS loader code is anything like the kernel code then ZFS probably uses deeper stacks than UFS. :( One of the larger uses of stack in the loader is probably gzipfs, so loading a mfsroot.gz from the loader might be a good test. Unfortunately, there isn't a good way to really detect stack overflow other than silent corruption in bss. If we could create a 'volatile int canary' in the bss and force it to be the last thing in the bss we could check the value of 'canary' in the loader's main loop perhaps and complain if it is ever non-zero. -- John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?27682255.x3aPd9zKdx>