Date: Mon, 12 Feb 2018 15:25:42 -0700 From: Warner Losh <imp@bsdimp.com> To: John Baldwin <jhb@freebsd.org> Cc: FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: LUA boot loader coming very soon Message-ID: <CANCZdfoOMENxi8FMs14XnVKgCtMY3A5=8BFpWsKJszKnnJikCg@mail.gmail.com> In-Reply-To: <27682255.x3aPd9zKdx@ralph.baldwin.cx> References: <CANCZdfqJzPud2GeMgc=zRNnz%2BQY0MuOft3L3OeQeq5w75NfbQA@mail.gmail.com> <2333272.J6kp8cXjfh@ralph.baldwin.cx> <CANCZdfoE4fzkPxrMevnxZE76yTZs_5%2BsToVpHjT5ERcfrU%2BTMw@mail.gmail.com> <27682255.x3aPd9zKdx@ralph.baldwin.cx>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Feb 12, 2018 at 2:53 PM, John Baldwin <jhb@freebsd.org> wrote: > On Monday, February 12, 2018 02:31:46 PM Warner Losh wrote: > > On Mon, Feb 12, 2018 at 10:12 AM, John Baldwin <jhb@freebsd.org> wrote: > > > > > On Monday, February 12, 2018 08:27:27 AM Warner Losh wrote: > > > > Greetings, > > > > > > > > As you may know, the Lua (http://www.lua.org) boot loader has been > in > > > the > > > > works for some time. It started out life as a GSoC in 2014 by Pedro > Souza > > > > mentored by Wojciech A. Koszek. Rui Paulo created a svn project > branch to > > > > try to integrate it. I rebased that effort into a github branch which > > > Pedro > > > > Arthur fixed up. Over the past year, I've been cleaning up the boot > > > loader > > > > for other reasons, and found the time was ripe to start integrating > this > > > > into the tree. However, those integration efforts have taken a while > as > > > my > > > > day-job work on the boot loader took priority. In the mean time, Ed > Maste > > > > and the FreeBSD Foundation funded Zakary Nafziger to enhance the > original > > > > GSoC Lua scripts to bring it closer to parity with the evolution of > the > > > > FORTH menu system since the GSoC project started. > > > > > > > > I'm pleased to announce that all these threads of development have > > > > converged and I'll be pushing the FreeBSD Lua Loader later today. > This > > > > loader uses Lua as its scripting language instead of FORTH. While > > > > co-existance is planned, the timeline for it is looking to be a few > weeks > > > > and I didn't want to delay pushing this into the tree for that. > > > > > > > > To try the loader, you'll need to build WITHOUT_FORTH=yes and > > > > WITH_LOADER_LUA=yes. Fortunately, you needn't do a full world to do > this, > > > > you can do it in src/stand and install the result (be sure to have > the > > > > options for both the build and the install). This will replace your > > > current > > > > /boot/loader that is scripted with FORTH to one that's scripted with > Lua. > > > > It will install the lua scripts in /boot/lua. The boot is scripted > with > > > > /boot/lua/loader.lua instead of /boot/loader.rc. You are strongly > advised > > > > to create a backup copy of /boot/loader before testing (eg cp > > > /boot/loader > > > > /boot/loader_forth), since you'll need to boot that from boot2 if > > > something > > > > goes wrong. I've tested it extensively, though, with userboot.so and > it's > > > > test program, so all the initial kinks of finding the lua scripts, > etc > > > have > > > > been worked out. > > > > > > > > While it's possible to build all the /boot/loader variants with Lua, > I've > > > > just tested a BIOS booting /boot/loader both with and without menus > > > > enabled. I've not tested any of the other variants and the > instructions > > > for > > > > testing some of them may be rather tedious (especially UEFI, if you > want > > > a > > > > simple path to back out). Since there's not been full convergence > > > testing, > > > > you'll almost certainly find bumps in this system. Also, all the > > > > build-system APIs are likely not yet final. > > > > > > > > I put MFC after a month on the commit. Due to the heroic (dare I say > > > > almost crazy) work of Kyle Evans on merging all the revs from > -current to > > > > 11, I'm planning a MFC to 11 after the co-existence issues are > hammered > > > > out. In 11, FORTH will be the default, and Lua will be built by > default, > > > > but users will have to do something to use it. 12, both FORTH and Lua > > > will > > > > be built and installed, with Lua as default (barring unforeseen > > > > complications). Once the co-existence stuff goes in, I imagine we'll > make > > > > the switch to Lua by default shortly after that. In 13, FORTH will be > > > > removed unless there's a really really compelling case made to keep > it. > > > > > > > > So please give it a spin and give me any feedback, documentation > updates > > > > and/or bug fixes. I'm especially interested in reviews from people > that > > > > have embedded Lua in other projects or experts in Lua that can > improve > > > the > > > > robustness of the menu code. > > > > > > Do you have some memory usage numbers for LUA vs forth for the > different > > > BIOS loaders (text/data/bss sizes)? For the EFI case we probably have > lots > > > of room, but for the non-EFI case we are constrained to 0xa0000 - > 0xa000 > > > for the text/data/bss and stack (in some cases like PXE booting the top > > > can be lower than 0xa0000). I'm not sure if we have any other > platforms > > > with similar memory constraints. > > > > > > > Lua is about 60-70k larger than FORTH for this application: > > > > Forth based: > > -r-xr-xr-x 1 root wheel 385024 Feb 12 14:14 > > /home/imp/roots/amd64/boot/loader > > -r-xr-xr-x 1 root wheel 450560 Feb 12 14:14 > > /home/imp/roots/amd64/boot/zfsloader > > Lua based: > > -r-xr-xr-x 1 root wheel 450560 Feb 12 14:20 > > /home/imp/roots/amd64/boot/loader > > -r-xr-xr-x 1 root wheel 520192 Feb 12 14:20 > > /home/imp/roots/amd64/boot/zfsloader > > > > So with Lua: > > % size loader.bin > > text data bss dec hex filename > > 411840 22264 48304 482408 0x75c68 loader.bin > > % size zfsloader.bin > > text data bss dec hex filename > > 478712 22968 52368 554048 0x87440 zfsloader.bin > > > > so we're at 555k out of 640k before we look at heap usage. Of course, > these > > numbers are only going to get a lot worse as we pile-in new crypto > > features... I know that the normal loader works, but I've not tried the > > zfsloader. > > Woof, on stable/11 the forth ones are quite a bit smaller: > > -r-xr-xr-x 1 root wheel 315392 Dec 16 13:39 /boot/loader > -r-xr-xr-x 1 root wheel 356352 Dec 16 13:39 /boot/zfsloader > > % size /usr/obj/usr/src/sys/boot/i386/loader/loader.bin > text data bss dec hex filename > 274913 22624 46560 344097 0x54021 /usr/obj/usr/src/sys/boot/ > i386/loader/loader.bin > % size /usr/obj/usr/src/sys/boot/i386/zfsloader/zfsloader.bin > text data bss dec hex filename > 313865 23184 50624 387673 0x5ea59 /usr/obj/usr/src/sys/boot/ > i386/zfsloader/zfsloader.bin > > Note that it's not 640k, it's 0xa0000 (640k) - the start address of the > loader: 0xa000 (40k), so text/data/bss + stack has 600k of room. The heap > isn't in low memory anymore, so we don't have to worry about that. 555k > for > ZFS loader leaves around 45k for stack. If the ZFS loader code is anything > like the kernel code then ZFS probably uses deeper stacks than UFS. :( > Yea. We support more FS types than in 11 now I think (at least more than in 10), so we might want to look to trimming that. I don't think we need 3 different compressed filesystem formats compiled in by default, for example... > One of the larger uses of stack in the loader is probably gzipfs, so > loading > a mfsroot.gz from the loader might be a good test. Unfortunately, there > isn't > a good way to really detect stack overflow other than silent corruption in > bss. If we could create a 'volatile int canary' in the bss and force it > to be > the last thing in the bss we could check the value of 'canary' in the > loader's > main loop perhaps and complain if it is ever non-zero. > That's not a terrible idea... Assuming that the corruption is such that it's right at the end of bss, and not something a bit before it... Warner
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfoOMENxi8FMs14XnVKgCtMY3A5=8BFpWsKJszKnnJikCg>