From owner-freebsd-arch@freebsd.org Mon Nov 9 22:29:08 2020 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3A4CF46EFC5 for ; Mon, 9 Nov 2020 22:29:08 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from forward501o.mail.yandex.net (forward501o.mail.yandex.net [IPv6:2a02:6b8:0:1a2d::611]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CVQdb0thWz4S8B for ; Mon, 9 Nov 2020 22:29:06 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from mxback30g.mail.yandex.net (mxback30g.mail.yandex.net [IPv6:2a02:6b8:0:1472:2741:0:8b7:330]) by forward501o.mail.yandex.net (Yandex) with ESMTP id 85B011E80171; Tue, 10 Nov 2020 01:28:57 +0300 (MSK) Received: from localhost (localhost [::1]) by mxback30g.mail.yandex.net (mxback/Yandex) with ESMTP id pbz1iOfF2Y-SuOWEFwW; Tue, 10 Nov 2020 01:28:56 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfw.ru; s=mail; t=1604960936; bh=rsTZAncfKQIfIwHjFDsU1wE4XHGvKMgJIj9yiGi1UBg=; h=Message-Id:Cc:Subject:In-Reply-To:Date:References:To:From; b=ayrw8H+or3mi7ZlWLHpcQmvbV4L/947vCs3YVmqcYxr+O6KGEr7recicup8kYRdMQ fHVCZ7cu0mFuItqGua09LyYR/2d3R6Q+hw+eUF0FUEj71hy717s7adN6bm1SY0s3AF JzATx9ft25SBZe+TkxHRB0Oj/TfT+EPJ2mHB9lpU= Received: by myt3-e9df8ad73dde.qloud-c.yandex.net with HTTP; Tue, 10 Nov 2020 01:28:56 +0300 From: Alexander V. Chernikov To: John-Mark Gurney Cc: freebsd-arch In-Reply-To: <20201102221330.GS31099@funkthat.com> References: <356181604233241@mail.yandex.ru> <20201102221330.GS31099@funkthat.com> Subject: Re: Versioning support for kernel<>userland sysctl interface MIME-Version: 1.0 X-Mailer: Yamail [ http://yandex.ru ] 5.0 Date: Mon, 09 Nov 2020 22:28:56 +0000 Message-Id: <428251604959994@mail.yandex.ru> Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=utf-8 X-Rspamd-Queue-Id: 4CVQdb0thWz4S8B X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ipfw.ru header.s=mail header.b=ayrw8H+o; dmarc=none; spf=pass (mx1.freebsd.org: domain of melifaro@ipfw.ru designates 2a02:6b8:0:1a2d::611 as permitted sender) smtp.mailfrom=melifaro@ipfw.ru X-Spamd-Result: default: False [-3.50 / 15.00]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a02:6b8:0:1a2d::611:from]; R_DKIM_ALLOW(-0.20)[ipfw.ru:s=mail]; FREEFALL_USER(0.00)[melifaro]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a02:6b8:0:1000::/52:c]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[ipfw.ru]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; SPAMHAUS_ZRD(0.00)[2a02:6b8:0:1a2d::611:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[ipfw.ru:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:13238, ipnet:2a02:6b8::/32, country:RU]; MAILMAN_DEST(0.00)[freebsd-arch] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Nov 2020 22:29:08 -0000 02.11.2020, 22:13, "John-Mark Gurney" : > Alexander V. Chernikov wrote this message on Sun, Nov 01, 2020 at 12:47 +0000: >>  I would like to propose a change [1] that introduces versioning support for the data structures exposed to userland by sysctl interface. >> >>  We have dozens of interfaces exposing various statistics and control data by filling in and exporting structures. >>  net.inet6.icmp6.stats or net.inet6.icmp6.nd6_prlist can be a good examples of such interaction. > > We also need to decide the policy on dealing w/ support for these > data structures going forward... Because if we do the simple, default > policy of all userland apps can handle all structures, and kernel can > produce all structures, we now have an unbounded growth of complexity > and testing... I totally agree. While backward compatibility is important, it should not impose notable technical debt. I had the following as my mental model: * the code should be organised to support output for the latest version. * There should be a separate, isolatable, piece of code that converts from latest to n-1 (which can be chained: from n-1 to n-2 and so on) * when introducing changes we should garden older versions by COMPAT_X defines. > I do understand the desire to solve this problem, but IMO, this solution > is too simple, and dangerous to unbounded growth above. > > While I do like it's simplicity, one idea that I've had, while being a > bit more complex, has the ability to handle modification in a more > compatible way. > > Since we have dtrace, one of the outputs of dtrace is ctf, which allows > use to convey the type and structure information in a machine parseable > format. The idea is that each sysctl oid (that supports this) would > have the ability to fetch the ctf data for that oid. The userland would > then be able to convert the members to the local members of a similar > struct. A set of defaults could also be provided, allowing new fields > to have sane initial values. > > As long as the name of a structure member is never reused for a different > meaning, this will get us most of the way there, in a much cleaner > method... > > I do realize that this isn't the easiest thing, but the tools to do this > are in the tree, and would solve this problem, IMO, in a way that is a > lot more maintainable, and long term than the current proposal. > > Other solution, use ctf data to produce nvlist generation/consumption > code for a structure... The data transfered would be larger, but also > more compatible... I do like idea on the self-documenting approach. It addresses append-only case nicely, but that's not always the case. For example, in the initially-discussed icmp6 stats we have 256 64-bit counters representing icmp6 protocol historgram, resulting in 4k frame being allocated on stack for the current kernel implementation. If in the future our icmp6 kernel implementation changes and we won't be able to provide this counters, eventually we would want to remove all these counters from the structure. I'm not sure how can this be addressed without some sort of versioning scheme. > Overall, using bare structures is an ABI compatibility nightmare that > should be fixed in a better method. > > -- >   John-Mark Gurney Voice: +1 415 225 5579 > >      "All that I will do, has been done, All that I have, has not." > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org"