Date: Thu, 5 Sep 2024 18:11:47 -0500 From: Kyle Evans <kevans@FreeBSD.org> To: Mark Johnston <markj@freebsd.org>, freebsd-hackers@freebsd.org Cc: imp@freebsd.org, bapt@freebsd.org Subject: Re: adding new flua libraries Message-ID: <f39f748e-015f-44f0-b1df-02caace27aac@FreeBSD.org> In-Reply-To: <Zto2Cf0Rp3kIsuFH@nuc> References: <Zto2Cf0Rp3kIsuFH@nuc>
next in thread | previous in thread | raw e-mail | index | archive | help
On 9/5/24 17:51, Mark Johnston wrote: > FreeBSD ships a Lua 5.4 implementation, flua, in the base system. It's > intended for use by the base system, and so far has a few consumers, but > not many so far. (As an aside, flua's intended scope is not totally > clear to me. Is it only for use by the base system? What compatibility > guarantees does it provide, if any?) > Only by the base system (hence the rename and hiding it in /usr/libexec), no guarantees except for modules we've re-implemented subsets of- if a conflicting name exists in ports, its use must be compatible with that. > A few flua modules wrapping FreeBSD libraries (e.g., libjail) are > available, but they don't provide enough to make flua useful as a > general purpose programming tool. It lacks interfaces for interacting > with the system (e.g., libc/libsys/libutil/etc wrappers) as well as > standard programming facilities (e.g., classes, higher-order functions, > etc.). Here I'm mostly interested in discussing the former. > > I think flua could be a very useful alternative to shell scripts and C > code where performance is not critical. It's very good at wrapping C > interfaces and thus could be used to make FreeBSD features (jails, > bhyve, ctl, pf, zfs, dtrace, ...) more accessible to developers who > don't want to interact with C. > > It's a lot of work to build up a set of flua modules that provide enough > functionality to be generally useful. My feeling is that the easiest > way to get there is to wrap C interfaces directly (to the extent that > that's possible, but it's usually easy) and expose them in flua as a > stable interface. Libraries written in pure Lua (or other languages > that interoperate well with Lua) could be built on top of that. > > I'm inclined to start by wrapping libc and libsys interfaces in a manner > similar to luaposix. There, the namespace is partitioned by C headers, > so posix.unistd contains identifiers from unistd.h, posix.sys.stat > contains identifiers from sys/stat.h, and so on. In fact, flua already > has a small subset of luaposix's functionality. Wrapping C interfaces > isn't much fun, but it's easy, and it saves us having to come up with > names and interfaces (naming things is hard), and helps ensure that the > glue code is relatively small and doesn't impose a large testing burden. > Moreover, C interfaces don't change much and are subject to > well-understood compatibility constraints, which should mean that Lua > interfaces built on top of them are subject to the same constraints. > > Assuming there's general agreement on that approach, the question I'd > really like to answer is, what should the namespace look like? It would > be useful to provide a posix module compatible with luaposix, but that > obviously won't contain FreeBSD-specific functionality. > > I propose having a top-level "freebsd" namespace for all modules > implemented in the base system, excluding posix and contrib libraries > which already define a Lua interface (libucl is the one example I see so > far; we could import sqlite bindings as well). Then, if we follow > luaposix's convention, we could have freebsd.sys.capsicum.* for > Capsicum-related syscalls and constants, freebsd.sys.event.* for kevent > wrappers, and so on. The posix module could simply provide a subset of > freebsd.*. > I think that's fine. Ideally, we probably just import luaposix wholesale now once we get the bootstrap flua module situation sorted and we can just re-export those implementations into the freebsd.* namespace similar to how we discussed luaposix was doing things today, unless those interfaces are weird (I don't remember them being problematic, though). > Maybe it's better to separate C wrappers from native Lua modules though, > so it could be better to have freebsd.c.sys.capsicum.*, etc., and > provide higher-level interfaces for FreeBSD features under freebsd.pf, > freebsd.zfs, freebsd.jail, etc.. I'm not really sure. > > In the interest of prompting discussion a bit, I posted some patches to > add some example wrappers to flua: > https://reviews.freebsd.org/D46553 > https://reviews.freebsd.org/D46554 > https://reviews.freebsd.org/D46556 > Will take a look when I get a bit of time. > Does anyone have opinions on anything I've brought up above? I'm pretty > happy to write a lot of this glue code or find ways to automate it, as > I've already done a fair bit of it, but it's hard to make progress > without having some rigourous conventions for the flua namespace (again, > naming things is hard). > We should also figure out where to document this stuff (policy, mostly- we have a 3lua section for libraries themselves). Thanks, Kyle Evans
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?f39f748e-015f-44f0-b1df-02caace27aac>