From nobody Tue Feb 18 22:04:57 2025 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YyD8p70yyz5pFqT; Tue, 18 Feb 2025 22:04:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YyD8n6qQPz3SwH; Tue, 18 Feb 2025 22:04:57 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1739916298; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=50dQR47jnrFxo7xqfK0qYy0TFBxEKVg2ofv9I30vxgQ=; b=PL0VDvSSCFzamxt0BA/TZS9RDfijD0gC2QpGkkDRoZTGsEXvLmGs3QInkcz9af07ThZ80y h6ycSWyEVgsbQljFKEBOPHDPuYG7EfajVTfcMAmzvzGEBqQ+uf4wM5tjgfGZ+FpsMaVGvE BueMowVYCbq/+WjcVV+hfR2wqdhG1pndp7pb7tDM5QLT10HyTrgCwf5Ld9ub8f34uS9lw1 vrHoFreyUNKnx9xUZcwzW1vtGPbrMqTJc/uejI50RdDgZGgz0GgpiXq2oE7GVmqskussOW Ayxm8XSPkxJi8xC+o2VgLJyvlG/LO7I1lHOte2MsAjhg9HcoWdN90WL8pf+PYQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1739916298; a=rsa-sha256; cv=none; b=eW8dcTRcn5vik/BqnVz/AIKGAq7ztzjX3mvfnSMVCS4X6IZYLdYPgxGV31EkGRyXMmRJHg HuwBmx2zFnspwZbMFDa4CTD/Mx5Y9n/eHq8jvjJHMMcTRAwGAjU0Dy4M8ZsMceQKZBOEW+ WUyZ2IgtccI38hfaGPNRdZo8wsPGNLx0wHwMUB1CRAMT/3j0ZQB1xyeK3PafN+ymF3Hv1T uWWjVFV+BTUudp7rksf6FTq5X7qaklhmb+ClkuWCZnyWllMaggV6As3BUJzwEOVW0RriFN FQF2Bqp2GAZXSMuKjgGFPXqcBiwikvPtULfTFkeIcEOLxX6//LbR/WbcvUvUpA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1739916298; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=50dQR47jnrFxo7xqfK0qYy0TFBxEKVg2ofv9I30vxgQ=; b=Pji3LIfmmjiurc0fBTNElPfOXp3bxg+CXGfFmGzrsIxeGBe6L93hSi961TI7cs0AglGtww UDTJ8d0wWZLlii7tzXx5X05bKFNwqBQLV+V9c2hKtq/JBAnWPAXw/4TQRLYO+YgQ1pve8D QfnefT1AZvNIORcwKGsbNYtgjpfMDbv109iuG53PbFlp4t6zTkqhb/dvpEYtwqruwLEvEx DaW2Ut8ZDPMQz0mXdyLlAdnldB2YttrGQ05Y4J6izh1qioDjVCsshbp8aOZ9M1mNJSV7+d xKDkZr4eac5A828sBXhdHepcP81/pLqXIgr3dtsOE0RgiHvv1XYqxAZhC5kn1Q== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4YyD8n5kJ8zf6l; Tue, 18 Feb 2025 22:04:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 51IM4vHg009450; Tue, 18 Feb 2025 22:04:57 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 51IM4vMH009447; Tue, 18 Feb 2025 22:04:57 GMT (envelope-from git) Date: Tue, 18 Feb 2025 22:04:57 GMT Message-Id: <202502182204.51IM4vMH009447@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Brooks Davis Subject: git: 42d075f4b7b7 - main - makesyscalls: Restore support for cpp in input List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: brooks X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 42d075f4b7b74eb3c4f943d6bdcc2aeb56be6388 Auto-Submitted: auto-generated The branch main has been updated by brooks: URL: https://cgit.FreeBSD.org/src/commit/?id=42d075f4b7b74eb3c4f943d6bdcc2aeb56be6388 commit 42d075f4b7b74eb3c4f943d6bdcc2aeb56be6388 Author: Brooks Davis AuthorDate: 2025-02-18 22:02:19 +0000 Commit: Brooks Davis CommitDate: 2025-02-18 22:03:13 +0000 makesyscalls: Restore support for cpp in input Allow patterns like this in syscalls.master: #if 0 91 AUE_NULL RESERVED #else 91 AUE_NULL STD|CAPENABLED { int newsyscall(void); } #endif makesyscalls.lua and it's predecessor makesyscalls.sh (really an awk script with a tiny shell prolog) used a single pass parsing model where lines beginning with `#` were emitted into most generated files as they were read. I belive this was initially there to allow includes to be listed in syscalls.master, but Hyrum's Law[0] applies and people are using it for things like architecture-specific syscall definitions. This use of CPP macro is unsound and there are a number of sharp edges in both the new and old implementations. The macros are unsound because not all the files were generate are run through CPP (or if they are not in the same context) and this will increasingly be true as we generate more things. Sharp edges include the fact that anything before the first syscall would be printed at a different scope (e.g., before an array is declared). In this patch I collect each non-#include CPP directive and attach them to the syscall table or individual entries. All entries before the first syscall and after the last are attached to the prolog and epilog members. Within the syscall table all entries are attached to the next system calls's prolog member. In generators, each prolog entry is printed regardless of the system call's visibiilty which replicates the naive single pass model's behavior (including lots of empty blocks of #if/#else/#endif in the output). Unlike makesyscalls.lua, I discard none #define entries at the top of the file and print a warning as their usefulness appears limited. [0] https://www.hyrumslaw.com Reported by: kevans Reviewed by: kevans Sponsored by: DARPA, AFRL Pull Request: https://github.com/freebsd/freebsd-src/pull/1575 --- sys/kern/syscalls.master | 7 ++++--- sys/tools/syscalls/core/freebsd-syscall.lua | 23 ++++++++++++++++++----- sys/tools/syscalls/scripts/init_sysent.lua | 3 +++ sys/tools/syscalls/scripts/syscalls.lua | 4 ++++ sys/tools/syscalls/scripts/sysproto_h.lua | 12 ++++++++++++ sys/tools/syscalls/scripts/systrace_args.lua | 9 +++++++++ sys/tools/syscalls/tools/util.lua | 5 +++++ 7 files changed, 55 insertions(+), 8 deletions(-) diff --git a/sys/kern/syscalls.master b/sys/kern/syscalls.master index 52502fc662be..fbd2ffb60cae 100644 --- a/sys/kern/syscalls.master +++ b/sys/kern/syscalls.master @@ -107,9 +107,10 @@ ; timet_ Object contains a time_t and varies between i386 and other ; ABIs. -; #include's, #defines's, etc. may be included, and are copied to the output -; files. However, #ifdef, etc will be copied, but any lines that don't start -; with # will not. Caveat Emptor. +; #include's, #defines's, etc. may be included, and are copied to a +; limited set of output files. Before the first syscalls, #include lines will +; be copied and %%ABI_HEADERS%% expanded. Between system call entries, +; all lines beginning with # will be copied. Caveat Emptor. #include #include diff --git a/sys/tools/syscalls/core/freebsd-syscall.lua b/sys/tools/syscalls/core/freebsd-syscall.lua index 193b1e43563c..fdd3a9011b7a 100644 --- a/sys/tools/syscalls/core/freebsd-syscall.lua +++ b/sys/tools/syscalls/core/freebsd-syscall.lua @@ -48,7 +48,8 @@ function FreeBSDSyscall:parseSysfile() end local incs = "" - local defs = "" + local prolog = "" + local first = true local s for line in fh:lines() do line = line:gsub(commentExpr, "") -- Strip any comments. @@ -80,12 +81,18 @@ function FreeBSDSyscall:parseSysfile() if h ~= nil and h ~= "" then incs = incs .. h .. "\n" end - elseif line:match("^#%s*define") then - defs = defs .. line.. "\n" elseif line:match("^#") then - util.abort(1, "Unsupported cpp op " .. line) + prolog = prolog .. line .. "\n" else s = syscall:new() + if first then + self.prolog = prolog + s.prolog = "" + first = false + else + s.prolog = prolog + end + prolog = "" if s:add(line) then -- Append to system call list. for t in s:iter() do @@ -114,7 +121,13 @@ function FreeBSDSyscall:parseSysfile() assert(fh:close()) self.includes = incs - self.defines = defs + self.epilog = prolog + + if self.prolog ~= "" then + util.warn("non-include pre-processor directives in the " .. + "config prolog will not appear in generated output:\n" .. + self.prolog) + end end function FreeBSDSyscall:findStructs() diff --git a/sys/tools/syscalls/scripts/init_sysent.lua b/sys/tools/syscalls/scripts/init_sysent.lua index 66683250b482..a1f51b5f3152 100755 --- a/sys/tools/syscalls/scripts/init_sysent.lua +++ b/sys/tools/syscalls/scripts/init_sysent.lua @@ -66,6 +66,8 @@ struct sysent %s[] = { -- based on the type of system call. local comment = v.name + gen:write(v.prolog); + -- Handle non-compat: if v:native() then gen:write(string.format( @@ -163,6 +165,7 @@ struct sysent %s[] = { gen:write(string.format("\t/* %d = %s */\n", v.num, comment)) end + gen:write(tbl.epilog) -- End gen:write("};\n") diff --git a/sys/tools/syscalls/scripts/syscalls.lua b/sys/tools/syscalls/scripts/syscalls.lua index 38ed396a73ae..a40ebc137627 100755 --- a/sys/tools/syscalls/scripts/syscalls.lua +++ b/sys/tools/syscalls/scripts/syscalls.lua @@ -38,6 +38,9 @@ function syscalls.generate(tbl, config, fh) for _, v in pairs(s) do --print("num " .. v.num .. " name " .. v.name) local c = v:compatLevel() + + gen:write(v.prolog); + if v:native() then gen:write(string.format([[ "%s", /* %d = %s */ @@ -80,6 +83,7 @@ function syscalls.generate(tbl, config, fh) end end + gen:write(tbl.epilog) -- End gen:write("};\n") end diff --git a/sys/tools/syscalls/scripts/sysproto_h.lua b/sys/tools/syscalls/scripts/sysproto_h.lua index 6770e0548899..2b0b9293ea66 100755 --- a/sys/tools/syscalls/scripts/sysproto_h.lua +++ b/sys/tools/syscalls/scripts/sysproto_h.lua @@ -93,6 +93,12 @@ struct thread; -- intuitive). local audit_idx = 10000 -- this should do + gen:write(v.prolog) + gen:store(v.prolog, 1) + for _, w in pairs(config.compat_options) do + gen:store(v.prolog, w.compatlevel * 10) + end + -- Handle non-compat: if v:native() then -- All these negation conditions are because (in @@ -202,6 +208,12 @@ struct %s { end_idx) end + gen:write(tbl.epilog) + gen:store(tbl.epilog, 1) + for _, w in pairs(config.compat_options) do + gen:store(tbl.epilog, w.compatlevel * 10) + end + if gen.storage_levels ~= nil then gen:writeStorage() end diff --git a/sys/tools/syscalls/scripts/systrace_args.lua b/sys/tools/syscalls/scripts/systrace_args.lua index 88170b85e737..40dd072bc0c3 100755 --- a/sys/tools/syscalls/scripts/systrace_args.lua +++ b/sys/tools/syscalls/scripts/systrace_args.lua @@ -68,6 +68,10 @@ systrace_return_setargdesc(int sysnum, int ndx, char *desc, size_t descsz) for _, v in pairs(s) do + gen:write(v.prolog); + gen:store(v.prolog, 1); + gen:store(v.prolog, 2); + -- Handle non compat: if v:native() then gen:write(string.format([[ @@ -212,6 +216,7 @@ systrace_return_setargdesc(int sysnum, int ndx, char *desc, size_t descsz) end end + gen:write(tbl.epilog) gen:write([[ default: *n_args = 0; @@ -219,6 +224,8 @@ systrace_return_setargdesc(int sysnum, int ndx, char *desc, size_t descsz) }; } ]]) + + gen:store(tbl.epilog, 1) gen:store([[ default: break; @@ -227,6 +234,8 @@ systrace_return_setargdesc(int sysnum, int ndx, char *desc, size_t descsz) strlcpy(desc, p, descsz); } ]], 1) + + gen:store(tbl.epilog, 2) gen:store([[ default: break; diff --git a/sys/tools/syscalls/tools/util.lua b/sys/tools/syscalls/tools/util.lua index c9ff98dda786..41247e34a10e 100644 --- a/sys/tools/syscalls/tools/util.lua +++ b/sys/tools/syscalls/tools/util.lua @@ -35,6 +35,11 @@ function util.split(s, re) return t end +-- Prints a warning to stderr +function util.warn(msg) + assert(io.stderr:write("WARNING: " .. msg .. "\n")) +end + -- Aborts with a message and does a clean exit procedure. function util.abort(status, msg) assert(io.stderr:write(msg .. "\n"))