Date: Fri, 6 Sep 2024 09:45:19 -0500 From: Kyle Evans <kevans@FreeBSD.org> To: Baptiste Daroussin <bapt@freebsd.org>, Mark Johnston <markj@freebsd.org> Cc: freebsd-hackers@freebsd.org, imp@freebsd.org Subject: Re: adding new flua libraries Message-ID: <2f3039bc-883c-4b11-ab20-d1873c6cc555@FreeBSD.org> In-Reply-To: <qwingl2edkh655hjlfxhssez4ntqrit6cw7xkzj57jnmfbgzv7@jbaucgbobapr> References: <Zto2Cf0Rp3kIsuFH@nuc> <qwingl2edkh655hjlfxhssez4ntqrit6cw7xkzj57jnmfbgzv7@jbaucgbobapr>
next in thread | previous in thread | raw e-mail | index | archive | help
On 9/6/24 04:29, Baptiste Daroussin wrote: > On Thu 05 Sep 18:51, Mark Johnston wrote: >> FreeBSD ships a Lua 5.4 implementation, flua, in the base system. It's >> intended for use by the base system, and so far has a few consumers, but >> not many so far. (As an aside, flua's intended scope is not totally >> clear to me. Is it only for use by the base system? What compatibility >> guarantees does it provide, if any?) >> >> A few flua modules wrapping FreeBSD libraries (e.g., libjail) are >> available, but they don't provide enough to make flua useful as a >> general purpose programming tool. It lacks interfaces for interacting >> with the system (e.g., libc/libsys/libutil/etc wrappers) as well as >> standard programming facilities (e.g., classes, higher-order functions, >> etc.). Here I'm mostly interested in discussing the former. >> >> I think flua could be a very useful alternative to shell scripts and C >> code where performance is not critical. It's very good at wrapping C >> interfaces and thus could be used to make FreeBSD features (jails, >> bhyve, ctl, pf, zfs, dtrace, ...) more accessible to developers who >> don't want to interact with C. >> >> It's a lot of work to build up a set of flua modules that provide enough >> functionality to be generally useful. My feeling is that the easiest >> way to get there is to wrap C interfaces directly (to the extent that >> that's possible, but it's usually easy) and expose them in flua as a >> stable interface. Libraries written in pure Lua (or other languages >> that interoperate well with Lua) could be built on top of that. >> >> I'm inclined to start by wrapping libc and libsys interfaces in a manner >> similar to luaposix. There, the namespace is partitioned by C headers, >> so posix.unistd contains identifiers from unistd.h, posix.sys.stat >> contains identifiers from sys/stat.h, and so on. In fact, flua already >> has a small subset of luaposix's functionality. Wrapping C interfaces >> isn't much fun, but it's easy, and it saves us having to come up with >> names and interfaces (naming things is hard), and helps ensure that the >> glue code is relatively small and doesn't impose a large testing burden. >> Moreover, C interfaces don't change much and are subject to >> well-understood compatibility constraints, which should mean that Lua >> interfaces built on top of them are subject to the same constraints. >> >> Assuming there's general agreement on that approach, the question I'd >> really like to answer is, what should the namespace look like? It would >> be useful to provide a posix module compatible with luaposix, but that >> obviously won't contain FreeBSD-specific functionality. >> >> I propose having a top-level "freebsd" namespace for all modules >> implemented in the base system, excluding posix and contrib libraries >> which already define a Lua interface (libucl is the one example I see so >> far; we could import sqlite bindings as well). Then, if we follow >> luaposix's convention, we could have freebsd.sys.capsicum.* for >> Capsicum-related syscalls and constants, freebsd.sys.event.* for kevent >> wrappers, and so on. The posix module could simply provide a subset of >> freebsd.*. >> >> Maybe it's better to separate C wrappers from native Lua modules though, >> so it could be better to have freebsd.c.sys.capsicum.*, etc., and >> provide higher-level interfaces for FreeBSD features under freebsd.pf, >> freebsd.zfs, freebsd.jail, etc.. I'm not really sure. >> >> In the interest of prompting discussion a bit, I posted some patches to >> add some example wrappers to flua: >> https://reviews.freebsd.org/D46553 >> https://reviews.freebsd.org/D46554 >> https://reviews.freebsd.org/D46556 >> >> Does anyone have opinions on anything I've brought up above? I'm pretty >> happy to write a lot of this glue code or find ways to automate it, as >> I've already done a fair bit of it, but it's hard to make progress >> without having some rigourous conventions for the flua namespace (again, >> naming things is hard). > > I like all of what I read and I am fully aligned. > > What I don't see here, is I think we should have dynamic modules libucl, fbsd & > friends are not dynamic and the reviews I see for the freebsd module reviews, > this is still bundled. > Right, this is an artifact of how flua's evolved that we need to move away from. We didn't have module support at the beginning- that came later, but we didn't really move things out into modules. We still can't move some things because of the bootstrap flua problem I mentioned elsewhere (doing so would block work to do `make sysent` type work as part of the build as it would break cross-builds), but at least libucl/fbsd would be fine. > my proposal for unbundling, I need kldload for a project of mine, so I did: > https://reviews.freebsd.org/D46558 > > Best regards, > Bapt
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2f3039bc-883c-4b11-ab20-d1873c6cc555>