From nobody Thu May 11 04:11:15 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QGz3R6hHQz49cQ7; Thu, 11 May 2023 04:11:15 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QGz3R69VMz48ll; Thu, 11 May 2023 04:11:15 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1683778275; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ksQE+3IxSGKncD9rXn1bQ41jb6G/DbmY6h3dELeSC9Y=; b=DWLNq43S7Mp8+CDeQbIQG4dWI5vtqOWvq0cla9OoXargd9DdPHE7RqaxOCBRcN3C2hAcPT bAC4/xsqKrkIYjBs8UFHSGA/yzY+FNRw0uGcglDFZUm+wajzCSRryRiMPdFwtt/b7Qpelb IsRulnAtO5sYIGPbRSlC1ibiWYqCI0Q1rpcQBFCXbYbAsaHqeyv9Eo2DJmcqZq64C2CsqG Ywz/f9zuodSQolJIplUvEPawmOIfhPgBNZXOPrye5VXtEZF7+DVPxHcBbGaKz+enx9bvAw +H7dZX8H0ejtE6WweNVYwnlUv5CfMYey6nW+Qi/v0U+Qkdl+uINHyGrZbNoWGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1683778275; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ksQE+3IxSGKncD9rXn1bQ41jb6G/DbmY6h3dELeSC9Y=; b=HpkE9V6/60U7Y4AHKT6S5TPdXNs9a+XEubzmBRThAVDb1sEFHNuIE+kJNHf5CPq/1kJwJl vAuv+JegZb6uP8KQyTFdqGtiAKDdZgZciBwGIfaoGFAbJzzgfCoQXIW/vtG5XKLQrlqM00 qgr7TbefVk9SE2t1c+Y3Rmzd+gqR8DsCvRSw3Gxf0NveOGp/f3SUHdyxe/pQ4031+F5ka1 zjU54l0W9sniBgNY7XjYPHHn0tzMJ2z/ZE3kRPWBbf1KhT1Df4WqJ/d7uWqIvuZkwma+Qg a1UBGIgUhMjY9JqzV420f2rdYZmCkgdCSy19iOLqX+QcNvxkIzB8AG6Yh54zPA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1683778275; a=rsa-sha256; cv=none; b=DRVYHuIQkDujrHKe37egzLVcS1jfV2TD2CSWgEWpNXphXXoTgLZiSuqaTcvbevC0qTPKsI cO5mO5H9wZSEaPghCgYta6I87k9fYNVKrVMwvijztq/43Yd+0Za/XAPXXPrfygEve5/dBc O72Qfi3opzzjEE6iK4nHV8y72ufiKxUXs1kaHUKFuQVzeOcLBrYV0i/SKsp6ILKW1c6kIO d5uB1bJzVN0VSj5zan9Ro61cf+Cut+Ty4DRWANXeNTF2cWpTzYUcEZuSy1CPC1xduXnpDk /gJu76pMYTc62p0JK1CqDjkSg4qM8r2A3TD7OQl399DRlEYjZ5nzQPHpOaZV2w== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4QGz3R5F01zdbL; Thu, 11 May 2023 04:11:15 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 34B4BFen047147; Thu, 11 May 2023 04:11:15 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 34B4BFEE047146; Thu, 11 May 2023 04:11:15 GMT (envelope-from git) Date: Thu, 11 May 2023 04:11:15 GMT Message-Id: <202305110411.34B4BFEE047146@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Kyle Evans Subject: git: 3cb2f5f369ec - main - lualoader: add support for .lua configuration files List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kevans X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 3cb2f5f369ecfd6d15aeff2b47e21866551a20de Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by kevans: URL: https://cgit.FreeBSD.org/src/commit/?id=3cb2f5f369ecfd6d15aeff2b47e21866551a20de commit 3cb2f5f369ecfd6d15aeff2b47e21866551a20de Author: Kyle Evans AuthorDate: 2023-05-11 04:10:53 +0000 Commit: Kyle Evans CommitDate: 2023-05-11 04:10:53 +0000 lualoader: add support for .lua configuration files If a file is specified in loader_conf_files that ends in '.lua', lualoader will now load and execute that file. These may be used in place of a traditional loader.conf to use more complicated logic, where some values may be set based on others or based on the environment that the C bits has left us with. Lua scripts are run in a limited environment. In particular, it does not get access to any modules or, in-fact, anything except environment variable. A config.buildenv hook has been added so that a local module can add whatever it may need to to the environment. When a global var is set in the lua script, it does not immediately alter the loader environment. Instead, the script's environment is initially empty and processed only if the whole script executes successfully. Effectively, a lua configuration file either takes effect or it does not, an error will not leave it in a half-baked state. Reviewed by: bcr (manpages), imp Differential Revision: https://reviews.freebsd.org/D28450 --- stand/defaults/loader.conf.5 | 3 ++ stand/lua/Makefile | 3 +- stand/lua/config.lua | 90 +++++++++++++++++++++++++++++++++++++++----- stand/lua/config.lua.8 | 8 ++++ stand/lua/hook.lua | 4 +- stand/lua/hook.lua.8 | 4 +- stand/lua/loader.conf.lua.5 | 77 +++++++++++++++++++++++++++++++++++++ 7 files changed, 175 insertions(+), 14 deletions(-) diff --git a/stand/defaults/loader.conf.5 b/stand/defaults/loader.conf.5 index 23a65935c95a..53c69f0f673c 100644 --- a/stand/defaults/loader.conf.5 +++ b/stand/defaults/loader.conf.5 @@ -427,10 +427,13 @@ It is not available in the default Forth-based loader. default settings \(em do not change this file. .It Pa /boot/loader.conf user defined settings. +.It Pa /boot/loader.conf.lua +user defined settings written in lua. .It Pa /boot/loader.conf.local machine-specific settings for sites with a common loader.conf. .El .Sh SEE ALSO +.Xr loader.conf.lua 5 , .Xr rc.conf 5 , .Xr boot 8 , .Xr cpucontrol 8 , diff --git a/stand/lua/Makefile b/stand/lua/Makefile index fe6fd6f63c8e..bc7c294dfda0 100644 --- a/stand/lua/Makefile +++ b/stand/lua/Makefile @@ -2,7 +2,8 @@ .include -MAN= cli.lua.8 \ +MAN= loader.conf.lua.5 \ + cli.lua.8 \ color.lua.8 \ config.lua.8 \ core.lua.8 \ diff --git a/stand/lua/config.lua b/stand/lua/config.lua index 04d5cda1e704..36be2b6b121e 100644 --- a/stand/lua/config.lua +++ b/stand/lua/config.lua @@ -45,6 +45,7 @@ local MSG_FAILSETENV = "Failed to '%s' with value: %s" local MSG_FAILOPENCFG = "Failed to open config: '%s'" local MSG_FAILREADCFG = "Failed to read config: '%s'" local MSG_FAILPARSECFG = "Failed to parse config: '%s'" +local MSG_FAILEXECLUA = "Failed to execute lua conf '%s': '%s'" local MSG_FAILPARSEVAR = "Failed to parse variable '%s': %s" local MSG_FAILEXBEF = "Failed to execute '%s' before loading '%s'" local MSG_FAILEXAF = "Failed to execute '%s' after loading '%s'" @@ -244,13 +245,15 @@ end -- local pattern_table = { { + luaexempt = true, str = "(#.*)", process = function(_, _) end, groups = 1, }, -- module_load="value" { - str = MODULEEXPR .. "_load%s*=%s*$VALUE", + name = MODULEEXPR .. "_load%s*", + val = "%s*$VALUE", process = function(k, v) if modules[k] == nil then modules[k] = {} @@ -261,7 +264,8 @@ local pattern_table = { }, -- module_name="value" { - str = MODULEEXPR .. "_name%s*=%s*$VALUE", + name = MODULEEXPR .. "_name%s*", + val = "%s*$VALUE", process = function(k, v) setKey(k, "name", v) setEnv(k .. "_name", v) @@ -269,7 +273,8 @@ local pattern_table = { }, -- module_type="value" { - str = MODULEEXPR .. "_type%s*=%s*$VALUE", + name = MODULEEXPR .. "_type%s*", + val = "%s*$VALUE", process = function(k, v) setKey(k, "type", v) setEnv(k .. "_type", v) @@ -277,7 +282,8 @@ local pattern_table = { }, -- module_flags="value" { - str = MODULEEXPR .. "_flags%s*=%s*$VALUE", + name = MODULEEXPR .. "_flags%s*", + val = "%s*$VALUE", process = function(k, v) setKey(k, "flags", v) setEnv(k .. "_flags", v) @@ -285,7 +291,8 @@ local pattern_table = { }, -- module_before="value" { - str = MODULEEXPR .. "_before%s*=%s*$VALUE", + name = MODULEEXPR .. "_before%s*", + val = "%s*$VALUE", process = function(k, v) setKey(k, "before", v) setEnv(k .. "_before", v) @@ -293,7 +300,8 @@ local pattern_table = { }, -- module_after="value" { - str = MODULEEXPR .. "_after%s*=%s*$VALUE", + name = MODULEEXPR .. "_after%s*", + val = "%s*$VALUE", process = function(k, v) setKey(k, "after", v) setEnv(k .. "_after", v) @@ -301,7 +309,8 @@ local pattern_table = { }, -- module_error="value" { - str = MODULEEXPR .. "_error%s*=%s*$VALUE", + name = MODULEEXPR .. "_error%s*", + val = "%s*$VALUE", process = function(k, v) setKey(k, "error", v) setEnv(k .. "_error", v) @@ -309,7 +318,9 @@ local pattern_table = { }, -- exec="command" { - str = "exec%s*=%s*" .. QVALEXPR, + luaexempt = true, + name = "exec%s*", + val = "%s*" .. QVALEXPR, process = function(k, _) if cli_execute_unparsed(k) ~= 0 then print(MSG_FAILEXEC:format(k)) @@ -319,7 +330,8 @@ local pattern_table = { }, -- env_var="value" or env_var=[word|num] { - str = "([%w][%w%d-_.]*)%s*=%s*$VALUE", + name = "([%w][%w%d-_.]*)%s*", + val = "%s*$VALUE", process = function(k, v) local pv, msg = processEnvVar(v) if not pv then @@ -328,6 +340,8 @@ local pattern_table = { end if setEnv(k, pv) ~= 0 then print(MSG_FAILSETENV:format(k, v)) + else + return pv end end, }, @@ -486,6 +500,18 @@ local function checkNextboot() loader.setenv("nextboot_enable", "NO") end +local function processEnv(k, v) + for _, val in ipairs(pattern_table) do + if not val.luaexempt and val.name then + local matched = k:match(val.name) + + if matched then + return val.process(matched, v) + end + end + end +end + -- Module exports config.verbose = false @@ -511,7 +537,47 @@ function config.processFile(name, silent) return silent end - return config.parse(text) + if name:match(".lua$") then + local cfg_env = setmetatable({}, { + indices = {}, + __index = function(env, key) + if getmetatable(env).indices[key] then + return rawget(env, key) + end + + return loader.getenv(key) + end, + __newindex = function(env, key, val) + getmetatable(env).indices[key] = true + rawset(env, key, val) + end, + }) + + -- Give local modules a chance to populate the config + -- environment. + hook.runAll("config.buildenv", cfg_env) + local res, err = pcall(load(text, name, "t", cfg_env)) + if res then + for k, v in pairs(cfg_env) do + local t = type(v) + if t ~= "function" and t ~= "table" then + if t ~= "string" then + v = tostring(v) + end + local pval = processEnv(k, v) + if pval then + setEnv(k, pval) + end + end + end + else + print(MSG_FAILEXECLUA:format(name, err)) + end + + return res + else + return config.parse(text) + end end -- silent runs will not return false if we fail to open the file @@ -522,6 +588,9 @@ function config.parse(text) for line in text:gmatch("([^\n]+)") do if line:match("^%s*$") == nil then for _, val in ipairs(pattern_table) do + if val.str == nil then + val.str = val.name .. "=" .. val.val + end local pattern = '^%s*' .. val.str .. '%s*(.*)'; local cgroups = val.groups or 2 local k, v, c = checkPattern(line, pattern) @@ -805,6 +874,7 @@ function config.getModuleInfo() } end +hook.registerType("config.buildenv") hook.registerType("config.loaded") hook.registerType("config.reloaded") hook.registerType("kernel.loaded") diff --git a/stand/lua/config.lua.8 b/stand/lua/config.lua.8 index 4fc51c90045f..cf4316204008 100644 --- a/stand/lua/config.lua.8 +++ b/stand/lua/config.lua.8 @@ -217,11 +217,19 @@ as well as a representation of The following hooks are defined in .Nm : .Bl -tag -width "config.reloaded" -offset indent +.It Fn config.buildenv env .It Fn config.loaded .It Fn config.reloaded .It Fn kernel.loaded .It Fn modules.loaded .El +.Pp +Note that the +.Fn config.buildenv +hook is only invoked when an environment needs to be built to execute a lua +configuration file that has been specified in +.Ev loader_conf_files . +It will be invoked for each configuration file encountered. .Sh SEE ALSO .Xr loader.conf 5 , .Xr loader 8 , diff --git a/stand/lua/hook.lua b/stand/lua/hook.lua index 9c7e4deeb7e4..6826bcb5f7f1 100644 --- a/stand/lua/hook.lua +++ b/stand/lua/hook.lua @@ -60,7 +60,7 @@ end -- Takes a hooktype and runs all functions associated with that specific hook -- type in the order that they were registered in. This ordering should likely -- not be relied upon. -function hook.runAll(hooktype) +function hook.runAll(hooktype, data) local selected_hooks = registered_hooks[hooktype] if selected_hooks == nil then -- This message, and the print() above, should only be seen by @@ -74,7 +74,7 @@ function hook.runAll(hooktype) end if #selected_hooks > 0 then for _, func in ipairs(selected_hooks) do - func() + func(data) end end return #selected_hooks diff --git a/stand/lua/hook.lua.8 b/stand/lua/hook.lua.8 index 1bc9ca96b509..f53d643d79bd 100644 --- a/stand/lua/hook.lua.8 +++ b/stand/lua/hook.lua.8 @@ -26,7 +26,7 @@ .\" .\" $FreeBSD$ .\" -.Dd June 9, 2018 +.Dd April 28, 2020 .Dt HOOK.LUA 8 .Os .Sh NAME @@ -39,6 +39,8 @@ Hooks are functions used to attach custom behaviors at pre-defined points in loader execution. These pre-defined points are what we refer to as .Dq hook types . +Hooks may also take an optional data parameter, which may or may not be +populated by the caller. .Pp Before using the functionality provided by .Nm , diff --git a/stand/lua/loader.conf.lua.5 b/stand/lua/loader.conf.lua.5 new file mode 100644 index 000000000000..5c8938c321f9 --- /dev/null +++ b/stand/lua/loader.conf.lua.5 @@ -0,0 +1,77 @@ +.\" +.\" SPDX-License-Identifier: BSD-2-Clause-FreeBSD +.\" +.\" Copyright (c) 2020 Kyle Evans +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.Dd May 10, 2023 +.Dt LOADER.CONF.LUA 5 +.Os +.Sh NAME +.Nm loader.conf.lua +.Nd Lua-based system bootstrap configuration file +.Sh DESCRIPTION +When the lua-based +.Xr loader 8 +encounters a filename in +.Va loader_conf_files +that has a +.Dq .lua +suffix, it will be loaded and executed by the lua interpreter in a limited +environment. +.Pp +The limited environment does not contain the ability to reference or load other +lua modules. +Existing loader environment variables may be referenced as if they were already +defined global variables. +.Pp +A lua configuration file may set any global variable, which will subsequently +be processed and added to the environment after execution of the configuration +file has completed. +Other than the +.Ar exec +setting, all variables described in +.Xr loader.conf 5 +operate the same in the +.Nm +environment. +Note that the settings describing module options can only be set in the +environment; there is currently no way for a +.Pa loader.conf.lua +to fetch them. +At this time, global table and function values are ignored. +.Pp +The +.Fn config.buildenv +hook will be run with an empty environment provided to it that may be populated +by a custom +.Pa local.lua . +.Sh SEE ALSO +.Xr loader.conf 5 +.Sh AUTHORS +The mechanism for loading +.Nm +files was originally written by +.An Kyle Evans Aq Mt kevans@FreeBSD.org .