From nobody Fri Sep 6 09:29:20 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4X0WC56gCmz5WTBt for ; Fri, 06 Sep 2024 09:29:21 +0000 (UTC) (envelope-from bapt@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4X0WC56497z4mvp; Fri, 6 Sep 2024 09:29:21 +0000 (UTC) (envelope-from bapt@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1725614961; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Ktw/hE/s58UQU9fQynPuCEbiL4q4+osCuDbAfIP/NQc=; b=UphaF86iapgsgF6T6GkSjeu188tZ6TJ2mY6kYeX9MWBItX9KNDpHOWX+1rTNKzUzKv8iJD OAJyZOW+0L9UGIFS2Tki2FxUHt+XtaND4nJC2S9cQTXZWGvDsx/g83raktNk+4pVQSDryg O9S7pknOQNez8NDChSdOVFftlp9B8u0Mu/yRNZiiE85BuciwQZrLPAe3YT041ZSNoQsSao 9iUY4tYj6lJtl77JCbuOC+uCmObRnMQZl/6YacozbG4aSpBgJdfwF5f1HU7u7z2is9b6es mb9WL8F5J0E0Q8LcWoX8ZxYNpbFcDrrJziSOL9+9KHDnUfwEpmb81dBn8YzFlw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1725614961; a=rsa-sha256; cv=none; b=f9wb1nc2Y85GCnLP3ikIinCW7I9B6A8+PixtMHD++1SBJ9FIIWVqb98ct0gx6Kghe3QrAr SeuP8BDSkKLjDNpIv8WuCgtRMdxs1KQ3qEVHPTrizPY0RQlqhMAQBQSfIejRDnAskWKEBg B9BSn99ZlUo7FO+Yc4ZLrHHMcuvIR+CFO6L2mat12W02Zyoh0B6rtQkiu0AYEmKkAs6L6L Z25FiC76KB/CeGVQRAdkJFCSuCmsolzjRPKFKHi1Ita/QVjIsox1yYpBTw+TMOruH6TsLh f0H4Zd3fytMTOrm900D69QC5DP8FVKN6j2FJWZyryISYP9NXza3T5+YM5cfuHA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1725614961; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Ktw/hE/s58UQU9fQynPuCEbiL4q4+osCuDbAfIP/NQc=; b=WoZhuhNcSg2pG0XGIAb5s9kvVARghrYOmjr0nyVhSrXPXy5ec70VD+TU83Sy/i6/UZjE0Q gQr2jVGZTNO3xg3R6tp3fHDIa1ggAGeg61L6CoaDXd7X3UDhWuOm+Vq63Nay35LuN7kkXd BVtTaeIn5Pir3VgAPahIcRQRfdylS7YlEsfjRwBQIHQs6QufQFp659jKEKm8f24xCk7J71 h6jVV/qkgrQWd/7PN9BCI1MU8+k4VdtB2w/D//HTeQL/mivXLYB6RfxtoMy30tHrVQiNmh cBQnGBxakjrT7FQPvASfJ+VlreUtaSETKXuDbDjsx7rHagQbVrYtDRDdRTUnIw== Received: from aniel.nours.eu (nours.eu [IPv6:2001:41d0:8:3a4d::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: bapt) by smtp.freebsd.org (Postfix) with ESMTPSA id 4X0WC54xgnzKLv; Fri, 6 Sep 2024 09:29:21 +0000 (UTC) (envelope-from bapt@freebsd.org) Received: by aniel.nours.eu (Postfix, from userid 1001) id 7361D23B6AA; Fri, 6 Sep 2024 11:29:20 +0200 (CEST) Date: Fri, 6 Sep 2024 11:29:20 +0200 From: Baptiste Daroussin To: Mark Johnston Cc: freebsd-hackers@freebsd.org, kevans@freebsd.org, imp@freebsd.org Subject: Re: adding new flua libraries Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Thu 05 Sep 18:51, Mark Johnston wrote: > FreeBSD ships a Lua 5.4 implementation, flua, in the base system. It's > intended for use by the base system, and so far has a few consumers, but > not many so far. (As an aside, flua's intended scope is not totally > clear to me. Is it only for use by the base system? What compatibility > guarantees does it provide, if any?) > > A few flua modules wrapping FreeBSD libraries (e.g., libjail) are > available, but they don't provide enough to make flua useful as a > general purpose programming tool. It lacks interfaces for interacting > with the system (e.g., libc/libsys/libutil/etc wrappers) as well as > standard programming facilities (e.g., classes, higher-order functions, > etc.). Here I'm mostly interested in discussing the former. > > I think flua could be a very useful alternative to shell scripts and C > code where performance is not critical. It's very good at wrapping C > interfaces and thus could be used to make FreeBSD features (jails, > bhyve, ctl, pf, zfs, dtrace, ...) more accessible to developers who > don't want to interact with C. > > It's a lot of work to build up a set of flua modules that provide enough > functionality to be generally useful. My feeling is that the easiest > way to get there is to wrap C interfaces directly (to the extent that > that's possible, but it's usually easy) and expose them in flua as a > stable interface. Libraries written in pure Lua (or other languages > that interoperate well with Lua) could be built on top of that. > > I'm inclined to start by wrapping libc and libsys interfaces in a manner > similar to luaposix. There, the namespace is partitioned by C headers, > so posix.unistd contains identifiers from unistd.h, posix.sys.stat > contains identifiers from sys/stat.h, and so on. In fact, flua already > has a small subset of luaposix's functionality. Wrapping C interfaces > isn't much fun, but it's easy, and it saves us having to come up with > names and interfaces (naming things is hard), and helps ensure that the > glue code is relatively small and doesn't impose a large testing burden. > Moreover, C interfaces don't change much and are subject to > well-understood compatibility constraints, which should mean that Lua > interfaces built on top of them are subject to the same constraints. > > Assuming there's general agreement on that approach, the question I'd > really like to answer is, what should the namespace look like? It would > be useful to provide a posix module compatible with luaposix, but that > obviously won't contain FreeBSD-specific functionality. > > I propose having a top-level "freebsd" namespace for all modules > implemented in the base system, excluding posix and contrib libraries > which already define a Lua interface (libucl is the one example I see so > far; we could import sqlite bindings as well). Then, if we follow > luaposix's convention, we could have freebsd.sys.capsicum.* for > Capsicum-related syscalls and constants, freebsd.sys.event.* for kevent > wrappers, and so on. The posix module could simply provide a subset of > freebsd.*. > > Maybe it's better to separate C wrappers from native Lua modules though, > so it could be better to have freebsd.c.sys.capsicum.*, etc., and > provide higher-level interfaces for FreeBSD features under freebsd.pf, > freebsd.zfs, freebsd.jail, etc.. I'm not really sure. > > In the interest of prompting discussion a bit, I posted some patches to > add some example wrappers to flua: > https://reviews.freebsd.org/D46553 > https://reviews.freebsd.org/D46554 > https://reviews.freebsd.org/D46556 > > Does anyone have opinions on anything I've brought up above? I'm pretty > happy to write a lot of this glue code or find ways to automate it, as > I've already done a fair bit of it, but it's hard to make progress > without having some rigourous conventions for the flua namespace (again, > naming things is hard). I like all of what I read and I am fully aligned. What I don't see here, is I think we should have dynamic modules libucl, fbsd & friends are not dynamic and the reviews I see for the freebsd module reviews, this is still bundled. my proposal for unbundling, I need kldload for a project of mine, so I did: https://reviews.freebsd.org/D46558 Best regards, Bapt