From owner-freebsd-arch@freebsd.org Mon Nov 2 22:13:34 2020 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 96FE045C7C6 for ; Mon, 2 Nov 2020 22:13:34 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CQ6cs4BWqz45Jv for ; Mon, 2 Nov 2020 22:13:33 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id 0A2MDUxO073577 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 2 Nov 2020 14:13:30 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id 0A2MDUQS073576; Mon, 2 Nov 2020 14:13:30 -0800 (PST) (envelope-from jmg) Date: Mon, 2 Nov 2020 14:13:30 -0800 From: John-Mark Gurney To: "Alexander V. Chernikov" Cc: freebsd-arch Subject: Re: Versioning support for kernel<>userland sysctl interface Message-ID: <20201102221330.GS31099@funkthat.com> Mail-Followup-To: "Alexander V. Chernikov" , freebsd-arch References: <356181604233241@mail.yandex.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <356181604233241@mail.yandex.ru> X-Operating-System: FreeBSD 11.3-STABLE amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Mon, 02 Nov 2020 14:13:31 -0800 (PST) X-Rspamd-Queue-Id: 4CQ6cs4BWqz45Jv X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of jmg@gold.funkthat.com has no SPF policy when checking 208.87.223.18) smtp.mailfrom=jmg@gold.funkthat.com X-Spamd-Result: default: False [0.62 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[jmg]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_MEDIUM(-0.11)[-0.106]; MID_RHS_MATCH_FROM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[funkthat.com]; AUTH_NA(1.00)[]; NEURAL_SPAM_SHORT(0.26)[0.255]; NEURAL_HAM_LONG(-0.73)[-0.725]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[jmg@funkthat.com,jmg@gold.funkthat.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:32354, ipnet:208.87.216.0/21, country:US]; FROM_NEQ_ENVFROM(0.00)[jmg@funkthat.com,jmg@gold.funkthat.com]; MAILMAN_DEST(0.00)[freebsd-arch]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Nov 2020 22:13:34 -0000 Alexander V. Chernikov wrote this message on Sun, Nov 01, 2020 at 12:47 +0000: > I would like to propose a change [1] that introduces versioning support for the data structures exposed to userland by sysctl interface. > > We have dozens of interfaces exposing various statistics and control data by filling in and exporting structures. > net.inet6.icmp6.stats or net.inet6.icmp6.nd6_prlist can be a good examples of such interaction. We also need to decide the policy on dealing w/ support for these data structures going forward... Because if we do the simple, default policy of all userland apps can handle all structures, and kernel can produce all structures, we now have an unbounded growth of complexity and testing... I do understand the desire to solve this problem, but IMO, this solution is too simple, and dangerous to unbounded growth above. While I do like it's simplicity, one idea that I've had, while being a bit more complex, has the ability to handle modification in a more compatible way. Since we have dtrace, one of the outputs of dtrace is ctf, which allows use to convey the type and structure information in a machine parseable format. The idea is that each sysctl oid (that supports this) would have the ability to fetch the ctf data for that oid. The userland would then be able to convert the members to the local members of a similar struct. A set of defaults could also be provided, allowing new fields to have sane initial values. As long as the name of a structure member is never reused for a different meaning, this will get us most of the way there, in a much cleaner method... I do realize that this isn't the easiest thing, but the tools to do this are in the tree, and would solve this problem, IMO, in a way that is a lot more maintainable, and long term than the current proposal. Other solution, use ctf data to produce nvlist generation/consumption code for a structure... The data transfered would be larger, but also more compatible... Overall, using bare structures is an ABI compatibility nightmare that should be fixed in a better method. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."