From owner-freebsd-arch@freebsd.org Mon Jan 25 18:45:46 2021 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1CA4E4EC2DF for ; Mon, 25 Jan 2021 18:45:46 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from tor1-11.mx.scaleengine.net (tor1-11.mx.scaleengine.net [IPv6:2001:470:1:474::25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DPf2K5pRqz4qqQ; Mon, 25 Jan 2021 18:45:45 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by tor1-11.mx.scaleengine.net (Postfix) with ESMTPSA id 363E71AF49; Mon, 25 Jan 2021 18:45:39 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 tor1-11.mx.scaleengine.net 363E71AF49 Subject: Re: Should we enable KERN_TLS on amd64 for FreeBSD 13? To: Andrew Gallatin , freebsd-arch@FreeBSD.org, Ed Maste , John Baldwin References: <8eff83e5-49bc-d410-626e-603c03877b80@cs.duke.edu> From: Allan Jude Message-ID: Date: Mon, 25 Jan 2021 13:45:38 -0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: <8eff83e5-49bc-d410-626e-603c03877b80@cs.duke.edu> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4DPf2K5pRqz4qqQ X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jan 2021 18:45:46 -0000 On 2021-01-08 12:26, Andrew Gallatin wrote: > > Kernel TLS (KTLS) support was added roughly a year ago, and provides > an efficient software or hardware accelerated path to have the kernel > (or the NIC) handle TLS crypto.  This is quite useful for web and > NFS servers, and provides a huge (2x -> 5x) efficiency gain by > avoiding data copies into userspace for crypto, and potentially > offloading the crypto to hardware. > > > KTLS is well tested on amd64, having been used in production at Netflix > for nearly 4 years.   The vast majority of Netflix video has been served > via KTLS for the last few years.  Its what has allowed us to serve > 100Gb/s on Xeon 2697A cpus for years, and what allows us to serve > nearly 400Gb/s on AMD servers with NICs which support crypto offload. > > I have received a few requests to enable it by default in GENERIC, and > I'd like to get some opinions. > > There are essentially 3 options > > 1) Fully enable KTLS by adding 'options KERN_TLS' to GENERIC, and > flipping kern.ipc.tls.enable=1 > > The advantage of this is that it "just works" out of the box for users, > and for reviewers. > > The drawback is that new code is thrust on unsuspecting users, > potentially exposing them to bugs that we have not found in our > somewhat limited web serving workload. > > 2) Enable KTLS in GENERIC, but leave it turned off by default. > > This option allows users to enable ktls without a rebuild of GENERIC, > but does not enable it by default. So they can enable it if they > know about it, but are protected from bugs. > > The disadvantages of this are that it increases the kernel size > by ~20K, starts up one thread per core on every amd64 machine, > and it adds more required tuning to get good performance from FreeBSD. > > > 3) Continue along with KTLS disabled in GENERIC > > This is the lowest risk, but adds a higher bar for users wanting > to use ktls. > > > > Note that the discussion is focused on amd64 only, as KTLS will > only work on 64-bit platforms which use a direct map.  It has > not been tested at all on ppc64, and currently causes a > panic-at-boot on arm64 due to what are suspected to be problems > in the arm64 PCB setup. See: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247945 > > Drew > Just before this went in, Ed cleaned up the arm64 GENERIC to get it closer to the amd64 one. Can we enable KERN_TLS in arm64 GENERIC as well? -- Allan Jude From owner-freebsd-arch@freebsd.org Mon Jan 25 19:59:57 2021 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7EFD14EFD40 for ; Mon, 25 Jan 2021 19:59:57 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DPggx2jsJz3Fx2; Mon, 25 Jan 2021 19:59:57 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro.local (unknown [IPv6:2601:648:8681:1cb0:5c12:2e91:6a1e:30bf]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 9A8421E37; Mon, 25 Jan 2021 19:59:56 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Subject: Re: Should we enable KERN_TLS on amd64 for FreeBSD 13? To: Allan Jude , Andrew Gallatin , freebsd-arch@FreeBSD.org, Ed Maste References: <8eff83e5-49bc-d410-626e-603c03877b80@cs.duke.edu> From: John Baldwin Message-ID: Date: Mon, 25 Jan 2021 11:59:55 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jan 2021 19:59:57 -0000 On 1/25/21 10:45 AM, Allan Jude wrote: > On 2021-01-08 12:26, Andrew Gallatin wrote: >> >> Kernel TLS (KTLS) support was added roughly a year ago, and provides >> an efficient software or hardware accelerated path to have the kernel >> (or the NIC) handle TLS crypto.  This is quite useful for web and >> NFS servers, and provides a huge (2x -> 5x) efficiency gain by >> avoiding data copies into userspace for crypto, and potentially >> offloading the crypto to hardware. >> >> >> KTLS is well tested on amd64, having been used in production at Netflix >> for nearly 4 years.   The vast majority of Netflix video has been served >> via KTLS for the last few years.  Its what has allowed us to serve >> 100Gb/s on Xeon 2697A cpus for years, and what allows us to serve >> nearly 400Gb/s on AMD servers with NICs which support crypto offload. >> >> I have received a few requests to enable it by default in GENERIC, and >> I'd like to get some opinions. >> >> There are essentially 3 options >> >> 1) Fully enable KTLS by adding 'options KERN_TLS' to GENERIC, and >> flipping kern.ipc.tls.enable=1 >> >> The advantage of this is that it "just works" out of the box for users, >> and for reviewers. >> >> The drawback is that new code is thrust on unsuspecting users, >> potentially exposing them to bugs that we have not found in our >> somewhat limited web serving workload. >> >> 2) Enable KTLS in GENERIC, but leave it turned off by default. >> >> This option allows users to enable ktls without a rebuild of GENERIC, >> but does not enable it by default. So they can enable it if they >> know about it, but are protected from bugs. >> >> The disadvantages of this are that it increases the kernel size >> by ~20K, starts up one thread per core on every amd64 machine, >> and it adds more required tuning to get good performance from FreeBSD. >> >> >> 3) Continue along with KTLS disabled in GENERIC >> >> This is the lowest risk, but adds a higher bar for users wanting >> to use ktls. >> >> >> >> Note that the discussion is focused on amd64 only, as KTLS will >> only work on 64-bit platforms which use a direct map.  It has >> not been tested at all on ppc64, and currently causes a >> panic-at-boot on arm64 due to what are suspected to be problems >> in the arm64 PCB setup. See: >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247945 >> >> Drew >> > > Just before this went in, Ed cleaned up the arm64 GENERIC to get it > closer to the amd64 one. Can we enable KERN_TLS in arm64 GENERIC as well? Well, I also fixed a bug KERN_TLS exposed on arm64 that was gating for this (247945). I would not be opposed to enabling it on arm64, but I have not personally tested it on arm64. If someone can verify it works ok on arm64 I'd be happy for it to be enabled there. -- John Baldwin From owner-freebsd-arch@freebsd.org Mon Jan 25 20:05:04 2021 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0C7D34F01E3 for ; Mon, 25 Jan 2021 20:05:04 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x834.google.com (mail-qt1-x834.google.com [IPv6:2607:f8b0:4864:20::834]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DPgnq1ysqz3Gnb for ; Mon, 25 Jan 2021 20:05:02 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x834.google.com with SMTP id z6so10670877qtn.0 for ; Mon, 25 Jan 2021 12:05:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=247yrtCki+Zg1FRdPGB8qQu813vVoeOIC+5tbfTVRsE=; b=OwvokZnBOSJExPMVe8BvZJZ9n93E9uK105+mS7q9HBq3TukyB9xXR1nU0laPj6pcJO md8ypEkGOPNEPJcgdZ+9xxAsmsAgppeWdriHiKiQ3J0oA86+IS5QYvb707fJiylUxcaU TenQdWUehoaw4tBt+pYZo9/zVALYV/IT+1ZruQXpgVtt2M30vmfrYqr9ge4RbFiOomax nok7ibkWT64t6odxX6PsjR+YCjR7OeT2KndaDKpua95Hj1KyPu+3xOzvgGdceXLiHH37 G7Qjd31lQ9F5vmpc2oCDKTsIgH7XRH5lKkO9+9CYtuHOjNku4MEOhjdYmiQ7BNptzYzV zgsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=247yrtCki+Zg1FRdPGB8qQu813vVoeOIC+5tbfTVRsE=; b=ta2Ajz2ZRmYHdF2JehVgZadlkXY+/do1YW+4w1to0k/WeWrMOm++ZYSlo3GzXXWtvV hs187DzJ0b7cla2hwEnQUcYVrlhhjPr9mnOu+4fM/ABBmEKGX0oBx0O7nl62in5/uAai 1+2CmwikuaujDlKOeld5UYGnBHhH71YfWbrVcpt0paqLShewDK791xLBBGCxNTRdbI0L auc1vDtnKjD5HggaWoCakTps7BwsBbsmxYz9GTlcdX3jF7ktu1I98B/YdCmsSZ+A/iQI 1TqWZdJoktjzV8YwUWkKX5/s/57/bzQ5Kkh2jKo6oJ+EqF9Mc9a7Ox7K8YkAZ4iWUm5m 4YtQ== X-Gm-Message-State: AOAM531hykB1Ro5nseQ+gvzFWQ3f+7hsSaRPBG1vSLAmxspw7jLksE7Q gXcVXkCq08/7pnbLgp9SIApv/cN8XdC9Pkfh1SX9LwdzAdsnYo0c X-Google-Smtp-Source: ABdhPJyNJ7DEtNcEweWtth+s7nUkEx9TEw2L894o7hLxlPdEOwFkn6uFn8c6/kXQcee1MfJqzelLF/tB7mEGtIUO6fM= X-Received: by 2002:a05:622a:90:: with SMTP id o16mr2055237qtw.49.1611605101295; Mon, 25 Jan 2021 12:05:01 -0800 (PST) MIME-Version: 1.0 From: Warner Losh Date: Mon, 25 Jan 2021 13:04:50 -0700 Message-ID: Subject: Change in the uname To: "freebsd-arch@freebsd.org" X-Rspamd-Queue-Id: 4DPgnq1ysqz3Gnb X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=OwvokZnB; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::834) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-3.00 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arch@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::834:from:127.0.2.255]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::834:from]; TO_DN_EQ_ADDR_ALL(0.00)[]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::834:from]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; MAILMAN_DEST(0.00)[freebsd-arch]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jan 2021 20:05:04 -0000 Now that we have some experience with git, I'd like to propose we tweak the uname a little. Right now it is branch-cXXXXX-gHASH. First problem: cXXXX isn't linear. It suffers the same issues that git log does in that it includes all changes from merge commits back to a common ancestor. However, branch~1234 (1234 commits before the tip of branch) doesn't count those. Solution: change this to vXXXX and add --first-parent to the git rev-list command. This will give us something we can use with the branch~number notation to track down versions more easily and allow people moving forward 100 commits to see the v number increment by 100 commits. I'm changing c->v here for two reasons. (1) c is a valid hex character so cXXXXX looks like a hash and (2) we're changing the semantic meaning of XXXXX, so this will make things unambiguous. This will also make it easier to find the revision with the branch~123 notation should that ever become necessary (and the v number would decrement by 123 when people do this). Granted, this isn't a super-huge deal since we have the hash, but it will make things a bit more predictable and eliminate at least one source of confusion. Second problem: gHASH isn't cut-and-pastable. And it makes it look like a git describe token when it isn't. Solution: delete the 'g'. So the new uname will be branch-vXXXXX-HASH. I'd like to push this to stable/13 as well before the release so we're not stuck with two versions. See https://reviews.freebsd.org/D28338 for the proposed change. Comments? Warner From owner-freebsd-arch@freebsd.org Mon Jan 25 21:03:59 2021 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 750A74F1A11 for ; Mon, 25 Jan 2021 21:03:59 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound5a.ore.mailhop.org (outbound5a.ore.mailhop.org [44.233.67.66]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DPj5q06ggz3LYQ for ; Mon, 25 Jan 2021 21:03:58 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1611608631; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=CUsU0C7t5rmn6R2353TKMR6W4X00nthtqpUtU4Y9EsgJ9iTRqebwspAp3U/MuK9Em6W2Vl2CPOqcQ cp2exbvMOpQD8x1famynlirU+o2vM7sVycn5JqwNShRNj9jN4ktRZMaW3Nk2lJkLz90J6M2WlFE9PH 67AFpuiA2ToGFNQD0Yq7oLfokUrezA4XfWjbyZaliUCe+5nHbluJXUyfQ3cOMtoVBXOkMdpdPPAs0a FFvGyX72awlf3gcK/VViY2yirhDlyxmsOltaoXTvH5Htj304G/OyyyO2UNI+ccGvlD/YzE9m+eEoD4 wTTJQ52i4nQ7/T/s7hgSqV/g1mZMVfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:to:from:subject:message-id:dkim-signature:from; bh=YuAWh12qlCE4xVZWvIVAQg3KvG1sXlQ3Eo+sQE/mdhk=; b=nKSv9s+yQBsTl6jvYqkAj2tPI16/5gg255JJHAPdB/BqUEfEaKWMr0dk4RyyOJBnYb3olPZIqWePa 1GTQPEB0NoLdWtPW++m9cTrl3BLutnRsKujacZn8//1rbolJPqZdJarSYeBFGKQC0D/T1fgU8qgN8p 4VdPbz78cThu5j7wxauTzq8bxYT4+8Bo+Z5DQwaYA9xRt8CdLYmmVcYA3oHw5xEfVY38O9UwDtQ140 9FWWRacrTYh3vHNQGFQ3pFt0redUSRx9AzILPYm/ofeXBsn068i+z1kJXQEgWSBTgsL/z36lwg7cam fPs1MbphaNB+ihM0HP2PCJZ0IOlYBxg== ARC-Authentication-Results: i=1; outbound3.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:to:from:subject:message-id:from; bh=YuAWh12qlCE4xVZWvIVAQg3KvG1sXlQ3Eo+sQE/mdhk=; b=gAGxPuOlLh8zNHBspQJVsKBKx0ImV72xGM1ufBuecsZwjNPs/p6BYuE4+Sm3uqbVUtAYJRhr2O85a AcCE30mFUqFSXzCWDy0Bg1cT/+FtNdAVH1BA+DGuq3aErRCXILpWfBsEsoCDv8hahs0o/LqGzzFIA0 MW7MbtEzqQGpSUGyw2Fj53SlO1Y+/1tLIpNO/Ij52Yln9ZlnI2o3Jgg+2m8KBb4FgOdkIN7DvxjVmY 8eBn8yyxbvv8pTLDo/Ij8VPgEDcvP66An25mxvJaa63RfDEUoPklvd5/68jGDJC4r8qyXFoeaRZZr/ aL2PLOce6inGxPlFhJrHxvFpJ53qNNg== X-Originating-IP: 67.177.211.60 X-MHO-RoutePath: aGlwcGll X-MHO-User: d1e59d72-5f50-11eb-8ba0-614106969e8d X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (c-67-177-211-60.hsd1.co.comcast.net [67.177.211.60]) by outbound3.ore.mailhop.org (Halon) with ESMTPSA id d1e59d72-5f50-11eb-8ba0-614106969e8d; Mon, 25 Jan 2021 21:03:50 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id 10PL3mf7008909; Mon, 25 Jan 2021 14:03:48 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <10fec1da415cd8d3c080a2173b9245ff35fceac8.camel@freebsd.org> Subject: Re: Change in the uname From: Ian Lepore To: Warner Losh , "freebsd-arch@freebsd.org" Date: Mon, 25 Jan 2021 14:03:48 -0700 In-Reply-To: References: Content-Type: text/plain; charset="ASCII" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4DPj5q06ggz3LYQ X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:16509, ipnet:44.224.0.0/11, country:US] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jan 2021 21:03:59 -0000 On Mon, 2021-01-25 at 13:04 -0700, Warner Losh wrote: > Now that we have some experience with git, I'd like to propose we > tweak the > uname a little. > > Right now it is branch-cXXXXX-gHASH. > > First problem: cXXXX isn't linear. It suffers the same issues that > git log > does in that it includes all changes from merge commits back to a > common > ancestor. However, branch~1234 (1234 commits before the tip of > branch) > doesn't count those. > > Solution: change this to vXXXX and add --first-parent to the git rev- > list > command. This will give us something we can use with the > branch~number > notation to track down versions more easily and allow people moving > forward > 100 commits to see the v number increment by 100 commits. I'm > changing c->v > here for two reasons. (1) c is a valid hex character so cXXXXX looks > like a > hash and (2) we're changing the semantic meaning of XXXXX, so this > will > make things unambiguous. This will also make it easier to find the > revision > with the branch~123 notation should that ever become necessary (and > the v > number would decrement by 123 when people do this). > > Granted, this isn't a super-huge deal since we have the hash, but it > will > make things a bit more predictable and eliminate at least one source > of > confusion. > > Second problem: gHASH isn't cut-and-pastable. And it makes it look > like a > git describe token when it isn't. > > Solution: delete the 'g'. > > So the new uname will be branch-vXXXXX-HASH. > > I'd like to push this to stable/13 as well before the release so > we're not > stuck with two versions. > > See https://reviews.freebsd.org/D28338 for the proposed change. > > Comments? > > Warner That all sounds good, except the 'v' is a bit odd, normally it implies a "version" number, which isn't exactly what this is. How about 'n' or '#' since it's just a count? -- Ian From owner-freebsd-arch@freebsd.org Mon Jan 25 21:21:41 2021 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6DFCB4F23B9 for ; Mon, 25 Jan 2021 21:21:41 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from tor1-11.mx.scaleengine.net (tor1-11.mx.scaleengine.net [209.51.186.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DPjVF2HPCz3Mys; Mon, 25 Jan 2021 21:21:41 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by tor1-11.mx.scaleengine.net (Postfix) with ESMTPSA id BDC551C113; Mon, 25 Jan 2021 21:21:40 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 tor1-11.mx.scaleengine.net BDC551C113 Subject: Re: Should we enable KERN_TLS on amd64 for FreeBSD 13? To: Andrew Gallatin , John Baldwin , freebsd-arch@FreeBSD.org, Ed Maste References: <8eff83e5-49bc-d410-626e-603c03877b80@cs.duke.edu> <7c8f5dfa-3ae5-5620-2505-2324d41deaca@cs.duke.edu> From: Allan Jude Message-ID: <545e9227-a4a2-8c77-1400-c4371b654f36@freebsd.org> Date: Mon, 25 Jan 2021 16:21:40 -0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: <7c8f5dfa-3ae5-5620-2505-2324d41deaca@cs.duke.edu> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4DPjVF2HPCz3Mys X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jan 2021 21:21:41 -0000 On 2021-01-25 15:33, Andrew Gallatin wrote: > On 1/25/21 2:59 PM, John Baldwin wrote: >> On 1/25/21 10:45 AM, Allan Jude wrote: >>> On 2021-01-08 12:26, Andrew Gallatin wrote: >>>> >>>> Kernel TLS (KTLS) support was added roughly a year ago, and provides >>>> an efficient software or hardware accelerated path to have the kernel >>>> (or the NIC) handle TLS crypto.  This is quite useful for web and >>>> NFS servers, and provides a huge (2x -> 5x) efficiency gain by >>>> avoiding data copies into userspace for crypto, and potentially >>>> offloading the crypto to hardware. >>>> >>>> >>>> KTLS is well tested on amd64, having been used in production at Netflix >>>> for nearly 4 years.   The vast majority of Netflix video has been >>>> served >>>> via KTLS for the last few years.  Its what has allowed us to serve >>>> 100Gb/s on Xeon 2697A cpus for years, and what allows us to serve >>>> nearly 400Gb/s on AMD servers with NICs which support crypto offload. >>>> >>>> I have received a few requests to enable it by default in GENERIC, and >>>> I'd like to get some opinions. >>>> >>>> There are essentially 3 options >>>> >>>> 1) Fully enable KTLS by adding 'options KERN_TLS' to GENERIC, and >>>> flipping kern.ipc.tls.enable=1 >>>> >>>> The advantage of this is that it "just works" out of the box for users, >>>> and for reviewers. >>>> >>>> The drawback is that new code is thrust on unsuspecting users, >>>> potentially exposing them to bugs that we have not found in our >>>> somewhat limited web serving workload. >>>> >>>> 2) Enable KTLS in GENERIC, but leave it turned off by default. >>>> >>>> This option allows users to enable ktls without a rebuild of GENERIC, >>>> but does not enable it by default. So they can enable it if they >>>> know about it, but are protected from bugs. >>>> >>>> The disadvantages of this are that it increases the kernel size >>>> by ~20K, starts up one thread per core on every amd64 machine, >>>> and it adds more required tuning to get good performance from FreeBSD. >>>> >>>> >>>> 3) Continue along with KTLS disabled in GENERIC >>>> >>>> This is the lowest risk, but adds a higher bar for users wanting >>>> to use ktls. >>>> >>>> >>>> >>>> Note that the discussion is focused on amd64 only, as KTLS will >>>> only work on 64-bit platforms which use a direct map.  It has >>>> not been tested at all on ppc64, and currently causes a >>>> panic-at-boot on arm64 due to what are suspected to be problems >>>> in the arm64 PCB setup. See: >>>> https://urldefense.com/v3/__https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247945__;!!OToaGQ!7pQUcHPbxA12vEdKTCp5jkyVxDCqYEJ-BI38kgHqGgweT7yYYG1BVhbDek0_Jc7mqA$ >>>> >>>> Drew >>>> >>> >>> Just before this went in, Ed cleaned up the arm64 GENERIC to get it >>> closer to the amd64 one. Can we enable KERN_TLS in arm64 GENERIC as >>> well? >> >> Well, I also fixed a bug KERN_TLS exposed on arm64 that was gating for >> this (247945).  I would not be opposed to enabling it on arm64, but I >> have not personally tested it on arm64.  If someone can verify it works >> ok on arm64 I'd be happy for it to be enabled there. >> > > Yeah, that's the thing, I have much less confidence in ktls on arm64 > because we have not run it in production recently.  So I'm personally > much less confident in enabling it on arm64. > > Drew Klara has tested it on arm64 fairly heavily, and only found an issue with OpenSSL, but not found any issues with KERN_TLS itself. -- Allan Jude From owner-freebsd-arch@freebsd.org Mon Jan 25 22:03:39 2021 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 699E14F3949 for ; Mon, 25 Jan 2021 22:03:39 +0000 (UTC) (envelope-from lwhsu@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DPkQg2bYQz3h0d; Mon, 25 Jan 2021 22:03:39 +0000 (UTC) (envelope-from lwhsu@freebsd.org) Received: from mail-yb1-f182.google.com (mail-yb1-f182.google.com [209.85.219.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: lwhsu/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 47AFD2DF3; Mon, 25 Jan 2021 22:03:39 +0000 (UTC) (envelope-from lwhsu@freebsd.org) Received: by mail-yb1-f182.google.com with SMTP id 84so1871913yba.3; Mon, 25 Jan 2021 14:03:39 -0800 (PST) X-Gm-Message-State: AOAM5330+FEK0cludadscAHLpW5e/qiEpQr/lGyMfccXQ0w1tV+6sxR3 8GcsAYCvVMrseFuNBCsflGExdb7qfSmz1fz6cUc= X-Google-Smtp-Source: ABdhPJzNPwaDipSEQ0vRYx43EiVvYNMy/jbz3kuwPeDJq/5yk6M1K+gKYm00HBBzEDcy5T22LJPwbE9+BMZ4AQZLC2E= X-Received: by 2002:a5b:410:: with SMTP id m16mr3912580ybp.451.1611612218704; Mon, 25 Jan 2021 14:03:38 -0800 (PST) MIME-Version: 1.0 References: <10fec1da415cd8d3c080a2173b9245ff35fceac8.camel@freebsd.org> In-Reply-To: <10fec1da415cd8d3c080a2173b9245ff35fceac8.camel@freebsd.org> From: Li-Wen Hsu Date: Tue, 26 Jan 2021 06:03:27 +0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Change in the uname To: Ian Lepore Cc: Warner Losh , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jan 2021 22:03:39 -0000 On Tue, Jan 26, 2021 at 5:04 AM Ian Lepore wrote: > That all sounds good, except the 'v' is a bit odd, normally it implies > a "version" number, which isn't exactly what this is. How about 'n' or > '#' since it's just a count? Seconded, sorry for discussing the color of bikeshed, but 'v' is also not intuitive to me. Since it is still a counter, and we are removing the 'g' in the second part, cXXXXX-HASH is still distinguishable with the old cXXXXX-gHASH. Or, maybe call it gXXXXX-HASH which stands for git commit count XXXXX, with hash value HASH? Best, Li-Wen From owner-freebsd-arch@freebsd.org Tue Jan 26 01:39:00 2021 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 45F5D4FA251 for ; Tue, 26 Jan 2021 01:39:00 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DPqC72RM8z4TrD for ; Tue, 26 Jan 2021 01:38:58 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id 10Q1cucP009804 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 25 Jan 2021 17:38:57 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id 10Q1ctjd009802; Mon, 25 Jan 2021 17:38:55 -0800 (PST) (envelope-from jmg) Date: Mon, 25 Jan 2021 17:38:55 -0800 From: John-Mark Gurney To: "Alexander V. Chernikov" Cc: freebsd-arch Subject: Re: Versioning support for kernel<>userland sysctl interface Message-ID: <20210126013855.GN31099@funkthat.com> Mail-Followup-To: "Alexander V. Chernikov" , freebsd-arch References: <356181604233241@mail.yandex.ru> <20201102221330.GS31099@funkthat.com> <428251604959994@mail.yandex.ru> <20201109231529.GH31099@funkthat.com> <10301605217434@mail.yandex.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <10301605217434@mail.yandex.ru> X-Operating-System: FreeBSD 11.3-STABLE amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Mon, 25 Jan 2021 17:38:57 -0800 (PST) X-Rspamd-Queue-Id: 4DPqC72RM8z4TrD X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of jmg@gold.funkthat.com has no SPF policy when checking 208.87.223.18) smtp.mailfrom=jmg@gold.funkthat.com X-Spamd-Result: default: False [-1.80 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[jmg]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[208.87.223.18:from]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; MID_RHS_MATCH_FROM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[funkthat.com]; AUTH_NA(1.00)[]; SPAMHAUS_ZRD(0.00)[208.87.223.18:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-1.00)[-0.996]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[jmg@funkthat.com,jmg@gold.funkthat.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:32354, ipnet:208.87.216.0/21, country:US]; FROM_NEQ_ENVFROM(0.00)[jmg@funkthat.com,jmg@gold.funkthat.com]; MAILMAN_DEST(0.00)[freebsd-arch]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jan 2021 01:39:00 -0000 Alexander V. Chernikov wrote this message on Thu, Nov 12, 2020 at 22:15 +0000: > 09.11.2020, 23:15, "John-Mark Gurney" : > > Alexander V. Chernikov wrote this message on Mon, Nov 09, 2020 at 22:28 +0000: > >>  02.11.2020, 22:13, "John-Mark Gurney" : > >>  > Alexander V. Chernikov wrote this message on Sun, Nov 01, 2020 at 12:47 +0000: > >>  >>  I would like to propose a change [1] that introduces versioning support for the data structures exposed to userland by sysctl interface. > >>  >> > >>  >>  We have dozens of interfaces exposing various statistics and control data by filling in and exporting structures. > >>  >>  net.inet6.icmp6.stats or net.inet6.icmp6.nd6_prlist can be a good examples of such interaction. > >>  > > >>  > We also need to decide the policy on dealing w/ support for these > >>  > data structures going forward... Because if we do the simple, default > >>  > policy of all userland apps can handle all structures, and kernel can > >>  > produce all structures, we now have an unbounded growth of complexity > >>  > and testing... > >>  I totally agree. While backward compatibility is important, it should not impose notable technical debt. I had the following as my mental model: > >>  * the code should be organised to support output for the latest version. > >>  * There should be a separate, isolatable, piece of code that converts from latest to n-1 (which can be chained: from n-1 to n-2 and so on) > >>  * when introducing changes we should garden older versions by COMPAT_X defines. > > > > Yeah, if we restrict the code to COMPAT_x for the existing versions, > > and ensure that it doesn't change, it isn't TOO terrible, but still, > > the likelyhood of people writing tests and verifying that they work > > to make sure that the compat code works for all n-x versions isn't > > great... it's doable, but I dobut most people are going to put in the > > effort.. > > > >>  > I do understand the desire to solve this problem, but IMO, this solution > >>  > is too simple, and dangerous to unbounded growth above. > >>  > > >>  > While I do like it's simplicity, one idea that I've had, while being a > >>  > bit more complex, has the ability to handle modification in a more > >>  > compatible way. > >>  > > >>  > Since we have dtrace, one of the outputs of dtrace is ctf, which allows > >>  > use to convey the type and structure information in a machine parseable > >>  > format. The idea is that each sysctl oid (that supports this) would > >>  > have the ability to fetch the ctf data for that oid. The userland would > >>  > then be able to convert the members to the local members of a similar > >>  > struct. A set of defaults could also be provided, allowing new fields > >>  > to have sane initial values. > >>  > > >>  > As long as the name of a structure member is never reused for a different > >>  > meaning, this will get us most of the way there, in a much cleaner > >>  > method... > >>  > > >>  > I do realize that this isn't the easiest thing, but the tools to do this > >>  > are in the tree, and would solve this problem, IMO, in a way that is a > >>  > lot more maintainable, and long term than the current proposal. > >>  > > >>  > Other solution, use ctf data to produce nvlist generation/consumption > >>  > code for a structure... The data transfered would be larger, but also > >>  > more compatible... > >>  I do like idea on the self-documenting approach. It addresses append-only case nicely, but that's not always the case. > >>  For example, in the initially-discussed icmp6 stats we have 256 64-bit counters representing icmp6 protocol historgram, resulting in 4k frame being allocated on stack for the current kernel implementation. If in the future our icmp6 kernel implementation changes and we won't be able to provide this counters, eventually we would want to remove all these counters from the structure. I'm not sure how can this be addressed without some sort of versioning scheme. > > > > So the bit that gets the ctf data would also have an nvlist (or > > something) that contains the defaults for when fields are removed... > > > > So, initally: > > struct foo { > >         int x; > >         int y; > > }; > > > > Then it gets changed to: > > struct foo { > >         int x; > >         int y; > >         int z; > > }; > > > > This is easy, the z will be included, and transmitted, but be ignored > > by older code, then when it changes to: > > struct { > >         int y; > >         int z; > > }; > > > > The ctf data would be something like: > > , > > > > Where nvlist defaults is: > > x: -1 > > > > So, the consuming code would set the defaults from the nvlist first, > > then set the fetch data, so that x gets set to some default value. > > > > With a few simple rules like this, handling deletions is not a problem > > when the older code is expecting it. If a variable must always have > > a value that "must" be correct (and a default cannot be set), then > > another member needs to be added (I think ctf handles bit fields > > properly) that says if that member is valid... when it gets removed, > > that is valid flag gets set to zero, and then the old code knows not > > to handle it. > Yep, thank you for clarifying! It looks pretty close to thrift/protobuf definitions with optional fields and default values. > We get much better support for "common" operations like adds and some removals. > However, field renaming (or, wider, changing field path, if we're talking about nested structures) is still problematic. > For example, https://docs.microsoft.com/en-us/aspnet/core/grpc/versioning?view=aspnetcore-5.0 lists the common practices/problems associated with this approach. > > > >>  > Overall, using bare structures is an ABI compatibility nightmare that > >>  > should be fixed in a better method. > I don't disagree and I'd love to see some variations of the interface described above in base - it will make development much easier. > I'm trying to state 2 things: > 1) Versioning is mostly orthogonal to the encoding scheme. There are cases where we need some sort of versioning to keep ABI even with the variations of an above interface > 2) we need to move from current state of the things to _some_ newer interface, which, itself, is _currently_ a breaking change. > > So what I'm advocating for is the generic mechanism that allows to pass version from userland to kernel in sysctl. > Adopting it will allow us to gradually move to nvlist / ctf or whatever approach we prefer, with pace not bound to any release cycle. > It will also allow us to switch such mechanism to another one if so desired. > > This versioning scheme already exists in the code, is light-weight and can be easily adopted before FreeBSD13 release. So, this proof of concept doesn't address anywhere close to all the issues, but I finally got around to doing a little PoC of usint CTF data for this. It is available at: https://www.funkthat.com/gitea/jmg/freebsd/src/branch/kvm_ctf The test includes a demo of it working w/ struct kinfo_proc: https://www.funkthat.com/gitea/jmg/freebsd/src/branch/kvm_ctf/lib/libkvm/tests/kvm_ctf_test.c#L153 There is a LOT more work if this is to be explored more fully. It doesn't handle missing kernel members w/ default. It breaks if the struct contains sub-structs as libctf doesn't handle them properly. It doesn't handle endian differences currently. It doesn't handle bit fields either. The good thing about this though is that it will work w/ core dumps, allowing netstat and others so be more ABI agnostic, and continue to work on core dumps. One of the cons is that it does make CTF a required build option to use. There is a BSD licensed libctf[1], but I have not evaluated how difficult it would be to use this version instead of the CDDL version that is currently in the tree. Another con is that ctf doesn't have a way to export just a part of the type hierarchy either. This means that kvm will parse ALL the ctf data for the kernel, even if you need a small section of it. This work was spured by the fact that the newly introduced SIOCGIFDATA ioctl[2], where it attempted to do basic ABI detection, but ended up failing. [1] https://github.com/lovasko/libctf [2] https://reviews.freebsd.org/D26538 -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@freebsd.org Mon Jan 25 20:33:22 2021 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9532D4F0E53 for ; Mon, 25 Jan 2021 20:33:22 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DPhQV3TxQz3JsM; Mon, 25 Jan 2021 20:33:22 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from [192.168.1.2] (pool-74-110-137-7.rcmdva.fios.verizon.net [74.110.137.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: gallatin) by duke.cs.duke.edu (Postfix) with ESMTPSA id B334427008D5; Mon, 25 Jan 2021 15:33:20 -0500 (EST) DMARC-Filter: OpenDMARC Filter v1.3.1 duke.cs.duke.edu B334427008D5 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=cs.duke.edu; s=mail0816; t=1611606801; bh=B2qvrFzIVyj8rXi4oOaonoArCCgHKp25vEUOAzrlD2c=; h=Subject:To:From:Date:From; b=pgG6mAZK/U5mOGvmbl9nqULh8QIXILzGJetLDRYEvFtFJKVEbRlp7gfLztwK+4pxZ /6M2yNJEZryNDzsWxdfPl0FbBXpU+arWYcCKgyOf5ujava+vDVRXQi6cDBORVery76 M9ai/9OIIjMq/0BIcBwx1PR4vzLvcLjNODWMLFPpIGY1I41faonJPD8H3hX3GT0xf5 jOZLYSrmJJzGc4Ad2MH/exBbPdEkx3wFZ6RqanKT8MqDYXGk06AG4RsnFKuKL4bKyX oz1Rs07su5xPrHjP+h8phylmFGWyWhFWfjyM/pkmPfyESjKTsAkNv+X91i5XiHVYSD XhS5ZHLaP2Rng== Subject: Re: Should we enable KERN_TLS on amd64 for FreeBSD 13? To: John Baldwin , Allan Jude , freebsd-arch@FreeBSD.org, Ed Maste References: <8eff83e5-49bc-d410-626e-603c03877b80@cs.duke.edu> From: Andrew Gallatin Message-ID: <7c8f5dfa-3ae5-5620-2505-2324d41deaca@cs.duke.edu> Date: Mon, 25 Jan 2021 15:33:19 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4DPhQV3TxQz3JsM X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-Mailman-Approved-At: Tue, 26 Jan 2021 10:50:30 +0000 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jan 2021 20:33:22 -0000 On 1/25/21 2:59 PM, John Baldwin wrote: > On 1/25/21 10:45 AM, Allan Jude wrote: >> On 2021-01-08 12:26, Andrew Gallatin wrote: >>> >>> Kernel TLS (KTLS) support was added roughly a year ago, and provides >>> an efficient software or hardware accelerated path to have the kernel >>> (or the NIC) handle TLS crypto.  This is quite useful for web and >>> NFS servers, and provides a huge (2x -> 5x) efficiency gain by >>> avoiding data copies into userspace for crypto, and potentially >>> offloading the crypto to hardware. >>> >>> >>> KTLS is well tested on amd64, having been used in production at Netflix >>> for nearly 4 years.   The vast majority of Netflix video has been served >>> via KTLS for the last few years.  Its what has allowed us to serve >>> 100Gb/s on Xeon 2697A cpus for years, and what allows us to serve >>> nearly 400Gb/s on AMD servers with NICs which support crypto offload. >>> >>> I have received a few requests to enable it by default in GENERIC, and >>> I'd like to get some opinions. >>> >>> There are essentially 3 options >>> >>> 1) Fully enable KTLS by adding 'options KERN_TLS' to GENERIC, and >>> flipping kern.ipc.tls.enable=1 >>> >>> The advantage of this is that it "just works" out of the box for users, >>> and for reviewers. >>> >>> The drawback is that new code is thrust on unsuspecting users, >>> potentially exposing them to bugs that we have not found in our >>> somewhat limited web serving workload. >>> >>> 2) Enable KTLS in GENERIC, but leave it turned off by default. >>> >>> This option allows users to enable ktls without a rebuild of GENERIC, >>> but does not enable it by default. So they can enable it if they >>> know about it, but are protected from bugs. >>> >>> The disadvantages of this are that it increases the kernel size >>> by ~20K, starts up one thread per core on every amd64 machine, >>> and it adds more required tuning to get good performance from FreeBSD. >>> >>> >>> 3) Continue along with KTLS disabled in GENERIC >>> >>> This is the lowest risk, but adds a higher bar for users wanting >>> to use ktls. >>> >>> >>> >>> Note that the discussion is focused on amd64 only, as KTLS will >>> only work on 64-bit platforms which use a direct map.  It has >>> not been tested at all on ppc64, and currently causes a >>> panic-at-boot on arm64 due to what are suspected to be problems >>> in the arm64 PCB setup. See: >>> https://urldefense.com/v3/__https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247945__;!!OToaGQ!7pQUcHPbxA12vEdKTCp5jkyVxDCqYEJ-BI38kgHqGgweT7yYYG1BVhbDek0_Jc7mqA$ >>> >>> Drew >>> >> >> Just before this went in, Ed cleaned up the arm64 GENERIC to get it >> closer to the amd64 one. Can we enable KERN_TLS in arm64 GENERIC as well? > > Well, I also fixed a bug KERN_TLS exposed on arm64 that was gating for > this (247945).  I would not be opposed to enabling it on arm64, but I > have not personally tested it on arm64.  If someone can verify it works > ok on arm64 I'd be happy for it to be enabled there. > Yeah, that's the thing, I have much less confidence in ktls on arm64 because we have not run it in production recently. So I'm personally much less confident in enabling it on arm64. Drew From owner-freebsd-arch@freebsd.org Mon Jan 25 21:30:22 2021 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D3C044F20F7 for ; Mon, 25 Jan 2021 21:30:22 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DPjhF60xDz3N5s; Mon, 25 Jan 2021 21:30:21 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from [192.168.1.2] (pool-74-110-137-7.rcmdva.fios.verizon.net [74.110.137.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: gallatin) by duke.cs.duke.edu (Postfix) with ESMTPSA id EC6AE2700318; Mon, 25 Jan 2021 16:30:20 -0500 (EST) DMARC-Filter: OpenDMARC Filter v1.3.1 duke.cs.duke.edu EC6AE2700318 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=cs.duke.edu; s=mail0816; t=1611610221; bh=LOKOZat1H3CgMvlx/xHifqZOGHgQ+Nk5AmIvChronNM=; h=Subject:To:From:Date:From; b=n9rxTaRAS4RguKLrnwJ4bvaAprITCmsECvEFZqV5GCxNlv4DiKIOR1m1h0qakLrxv OfSDINl1Ih3Za0mPNp76KFHtzL6JFebEJj+6wONYHZhI64hvQM33Q8ihr4kOEqjjJn Re/Jao0JvZErAS4F0mbtKj+PWCealgZvDpaEZJOlCA/Ylj5NIak32HBBReKm8wzYTm yRBcd/tdVR+SA3SrCSmkAfNvIF72Z9lrtMbSLsPw1hEDT+RsMWarT25s7Fl4UeDPmW 3pBPRcrvwM1yYl90KsdXB+4F/8fjrBRfZ5cE8U9htXqKV0iPQxEwDAPE1gbIVn1ame 6dILrho/+ZxTA== Subject: Re: Should we enable KERN_TLS on amd64 for FreeBSD 13? To: Allan Jude , John Baldwin , freebsd-arch@FreeBSD.org, Ed Maste References: <8eff83e5-49bc-d410-626e-603c03877b80@cs.duke.edu> <7c8f5dfa-3ae5-5620-2505-2324d41deaca@cs.duke.edu> <545e9227-a4a2-8c77-1400-c4371b654f36@freebsd.org> From: Andrew Gallatin Message-ID: Date: Mon, 25 Jan 2021 16:30:20 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: <545e9227-a4a2-8c77-1400-c4371b654f36@freebsd.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4DPjhF60xDz3N5s X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=cs.duke.edu header.s=mail0816 header.b=n9rxTaRA; dmarc=pass (policy=none) header.from=cs.duke.edu; spf=pass (mx1.freebsd.org: domain of gallatin@cs.duke.edu designates 152.3.140.1 as permitted sender) smtp.mailfrom=gallatin@cs.duke.edu X-Spamd-Result: default: False [-4.10 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:152.3.140.0/23]; DKIM_TRACE(0.00)[cs.duke.edu:+]; DMARC_POLICY_ALLOW(-0.50)[cs.duke.edu,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RECEIVED_SPAMHAUS_PBL(0.00)[74.110.137.7:received]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[152.3.140.1:from]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:13371, ipnet:152.3.128.0/17, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[cs.duke.edu:s=mail0816]; FREEFALL_USER(0.00)[gallatin]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SUBJECT_ENDS_QUESTION(1.00)[]; SPAMHAUS_ZRD(0.00)[152.3.140.1:from:127.0.2.255]; RCVD_IN_DNSWL_LOW(-0.10)[152.3.140.1:from]; DWL_DNSWL_LOW(-1.00)[duke.edu:dkim]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-arch] X-Mailman-Approved-At: Tue, 26 Jan 2021 10:50:50 +0000 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jan 2021 21:30:22 -0000 On 1/25/21 4:21 PM, Allan Jude wrote: > On 2021-01-25 15:33, Andrew Gallatin wrote: >> On 1/25/21 2:59 PM, John Baldwin wrote: >>> On 1/25/21 10:45 AM, Allan Jude wrote: >>>> On 2021-01-08 12:26, Andrew Gallatin wrote: >>>>> >>>>> Kernel TLS (KTLS) support was added roughly a year ago, and provides >>>>> an efficient software or hardware accelerated path to have the kernel >>>>> (or the NIC) handle TLS crypto.  This is quite useful for web and >>>>> NFS servers, and provides a huge (2x -> 5x) efficiency gain by >>>>> avoiding data copies into userspace for crypto, and potentially >>>>> offloading the crypto to hardware. >>>>> >>>>> >>>>> KTLS is well tested on amd64, having been used in production at Netflix >>>>> for nearly 4 years.   The vast majority of Netflix video has been >>>>> served >>>>> via KTLS for the last few years.  Its what has allowed us to serve >>>>> 100Gb/s on Xeon 2697A cpus for years, and what allows us to serve >>>>> nearly 400Gb/s on AMD servers with NICs which support crypto offload. >>>>> >>>>> I have received a few requests to enable it by default in GENERIC, and >>>>> I'd like to get some opinions. >>>>> >>>>> There are essentially 3 options >>>>> >>>>> 1) Fully enable KTLS by adding 'options KERN_TLS' to GENERIC, and >>>>> flipping kern.ipc.tls.enable=1 >>>>> >>>>> The advantage of this is that it "just works" out of the box for users, >>>>> and for reviewers. >>>>> >>>>> The drawback is that new code is thrust on unsuspecting users, >>>>> potentially exposing them to bugs that we have not found in our >>>>> somewhat limited web serving workload. >>>>> >>>>> 2) Enable KTLS in GENERIC, but leave it turned off by default. >>>>> >>>>> This option allows users to enable ktls without a rebuild of GENERIC, >>>>> but does not enable it by default. So they can enable it if they >>>>> know about it, but are protected from bugs. >>>>> >>>>> The disadvantages of this are that it increases the kernel size >>>>> by ~20K, starts up one thread per core on every amd64 machine, >>>>> and it adds more required tuning to get good performance from FreeBSD. >>>>> >>>>> >>>>> 3) Continue along with KTLS disabled in GENERIC >>>>> >>>>> This is the lowest risk, but adds a higher bar for users wanting >>>>> to use ktls. >>>>> >>>>> >>>>> >>>>> Note that the discussion is focused on amd64 only, as KTLS will >>>>> only work on 64-bit platforms which use a direct map.  It has >>>>> not been tested at all on ppc64, and currently causes a >>>>> panic-at-boot on arm64 due to what are suspected to be problems >>>>> in the arm64 PCB setup. See: >>>>> https://urldefense.com/v3/__https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247945__;!!OToaGQ!7pQUcHPbxA12vEdKTCp5jkyVxDCqYEJ-BI38kgHqGgweT7yYYG1BVhbDek0_Jc7mqA$ >>>>> >>>>> Drew >>>>> >>>> >>>> Just before this went in, Ed cleaned up the arm64 GENERIC to get it >>>> closer to the amd64 one. Can we enable KERN_TLS in arm64 GENERIC as >>>> well? >>> >>> Well, I also fixed a bug KERN_TLS exposed on arm64 that was gating for >>> this (247945).  I would not be opposed to enabling it on arm64, but I >>> have not personally tested it on arm64.  If someone can verify it works >>> ok on arm64 I'd be happy for it to be enabled there. >>> >> >> Yeah, that's the thing, I have much less confidence in ktls on arm64 >> because we have not run it in production recently.  So I'm personally >> much less confident in enabling it on arm64. >> >> Drew > > Klara has tested it on arm64 fairly heavily, and only found an issue > with OpenSSL, but not found any issues with KERN_TLS itself. > I have no objection if you want to enable it. But since I don't currently have an arm64 test machine, I don't want to be the one to enable it, if that makes sense.. Drew From owner-freebsd-arch@freebsd.org Tue Jan 26 15:30:23 2021 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B56104F31A7 for ; Tue, 26 Jan 2021 15:30:23 +0000 (UTC) (envelope-from gbe@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DQ9fR4lplz4Xpj; Tue, 26 Jan 2021 15:30:23 +0000 (UTC) (envelope-from gbe@freebsd.org) Received: from localhost (p200300d5d70b55db8499059115e9bb88.dip0.t-ipconnect.de [IPv6:2003:d5:d70b:55db:8499:591:15e9:bb88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gbe) by smtp.freebsd.org (Postfix) with ESMTPSA id 3D02EABF1; Tue, 26 Jan 2021 15:30:23 +0000 (UTC) (envelope-from gbe@freebsd.org) Date: Tue, 26 Jan 2021 16:30:22 +0100 From: Gordon Bergling To: John Baldwin Cc: Allan Jude , Andrew Gallatin , freebsd-arch@freebsd.org, Ed Maste Subject: Re: Should we enable KERN_TLS on amd64 for FreeBSD 13? Message-ID: References: <8eff83e5-49bc-d410-626e-603c03877b80@cs.duke.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Url: X-Operating-System: FreeBSD 12.2-STABLE amd64 X-Host-Uptime: 4:18PM up 13 days, 4:43, 4 users, load averages: 0.48, 0.33, 0.27 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jan 2021 15:30:23 -0000 On Mon, Jan 25, 2021 at 11:59:55AM -0800, John Baldwin wrote: > On 1/25/21 10:45 AM, Allan Jude wrote: > > On 2021-01-08 12:26, Andrew Gallatin wrote: > >> > >> Kernel TLS (KTLS) support was added roughly a year ago, and provides > >> an efficient software or hardware accelerated path to have the kernel > >> (or the NIC) handle TLS crypto.  This is quite useful for web and > >> NFS servers, and provides a huge (2x -> 5x) efficiency gain by > >> avoiding data copies into userspace for crypto, and potentially > >> offloading the crypto to hardware. > >> > >> > >> KTLS is well tested on amd64, having been used in production at Netflix > >> for nearly 4 years.   The vast majority of Netflix video has been served > >> via KTLS for the last few years.  Its what has allowed us to serve > >> 100Gb/s on Xeon 2697A cpus for years, and what allows us to serve > >> nearly 400Gb/s on AMD servers with NICs which support crypto offload. > >> > >> I have received a few requests to enable it by default in GENERIC, and > >> I'd like to get some opinions. > >> > >> There are essentially 3 options > >> > >> 1) Fully enable KTLS by adding 'options KERN_TLS' to GENERIC, and > >> flipping kern.ipc.tls.enable=1 > >> > >> The advantage of this is that it "just works" out of the box for users, > >> and for reviewers. > >> > >> The drawback is that new code is thrust on unsuspecting users, > >> potentially exposing them to bugs that we have not found in our > >> somewhat limited web serving workload. > >> > >> 2) Enable KTLS in GENERIC, but leave it turned off by default. > >> > >> This option allows users to enable ktls without a rebuild of GENERIC, > >> but does not enable it by default. So they can enable it if they > >> know about it, but are protected from bugs. > >> > >> The disadvantages of this are that it increases the kernel size > >> by ~20K, starts up one thread per core on every amd64 machine, > >> and it adds more required tuning to get good performance from FreeBSD. > >> > >> > >> 3) Continue along with KTLS disabled in GENERIC > >> > >> This is the lowest risk, but adds a higher bar for users wanting > >> to use ktls. > >> > >> > >> > >> Note that the discussion is focused on amd64 only, as KTLS will > >> only work on 64-bit platforms which use a direct map.  It has > >> not been tested at all on ppc64, and currently causes a > >> panic-at-boot on arm64 due to what are suspected to be problems > >> in the arm64 PCB setup. See: > >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247945 > >> > >> Drew > >> > > > > Just before this went in, Ed cleaned up the arm64 GENERIC to get it > > closer to the amd64 one. Can we enable KERN_TLS in arm64 GENERIC as well? > > Well, I also fixed a bug KERN_TLS exposed on arm64 that was gating for > this (247945). I would not be opposed to enabling it on arm64, but I > have not personally tested it on arm64. If someone can verify it works > ok on arm64 I'd be happy for it to be enabled there. I am the author of the mentioned PR and have beeing running the respective patch since a few weeks on a RPi4B without seeing any problems. The KTLS thread is present and the sysctl 'kern.ipc.tls.enable' is set to 1. I haven't done any real workload tests using encrypted NFSv4 traffic or nginx based HTTPS traffic. Maybe [1] should be integrated first to enable ATF based tests with the shipped OpenSSL version. Besides of this two points, I think it should be safe to enable it for arm64, as it was okay to enable it for amd64. --Gordon [1] https://reviews.freebsd.org/D28273 From owner-freebsd-arch@freebsd.org Wed Jan 27 19:20:43 2021 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 003A74E6AE4 for ; Wed, 27 Jan 2021 19:20:43 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x72b.google.com (mail-qk1-x72b.google.com [IPv6:2607:f8b0:4864:20::72b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DQtjk0gj3z3F5L for ; Wed, 27 Jan 2021 19:20:41 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x72b.google.com with SMTP id 19so2902334qkh.3 for ; Wed, 27 Jan 2021 11:20:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=KYtZb+x2IA4Wg2HJlmyUQWGa7ACFB0R/ol7oMyscYUs=; b=sZOwzdEApRp4QJ+50+aiijVNXh82GS9SiexDUDVerXMLJJlLTA9YMCFuH2DZDvAFOq sRt5VB68JlaapjbGxmZIV1GGKq/xWD1gNNekINUeIwMR4S4LOJtuco+JojoJeP+zqNqv bFfxS6eCmL+oNOsoJvs/YHpn3dyqDewssiPpNL5q++rT3s7//+Ti7xDhxkSfERFtJuq3 6z8b4jQFFxYa56QXcrLePKPXE4fFxj5duRntsEXURrAEi1mXEOsbqxraKR61MnW/JGbD Hw/JAU5LHZ/D0LH4M/bIKdF3XBkn3EFtlIBxkntqvWjiDMCrbsUt3deBzB+w1jO51Tyz TQew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=KYtZb+x2IA4Wg2HJlmyUQWGa7ACFB0R/ol7oMyscYUs=; b=bPs0U83A0oS4/YEfxi3hxrinRAoHtq9uV+eoknBXTU+FGGnAYdO3EcTcsW6q0AgnYd VKCG3MnzZDRrzMEqx3sFMUBRi3shRQ1emXoRjDqYntoCJdMK73Emf1eTjmPJC2In1dOq j95y7HgfPkjPfm1i6CAXnoE/kcxwfgOd2YvhvEJ/U1nfVPS5M0/UnF2WCFAhekY/CPoQ ra4OUiTOq+QttXTd3FIqCCBUrpzDHEJUOMvnvCD+Q3aMhqMiNkedwiNqxNP6wCLFqM13 bJnNuBYh8lnP+4ze2ZaL7zwP6K+OKLSmpMIP18FxJRU9EFj9gm/7AjFVWBizROS1o1hn bg0w== X-Gm-Message-State: AOAM533EI8nl218LwcN0ckAfQJH65MgrctfYEoBDYnicdaxGSg4LJTet D1RnXHQhICiWNtL22jONkr+EtHDuPRI7X4Ho+Wnnw1DN9Ka7Og== X-Google-Smtp-Source: ABdhPJyDPfScYKUdGV+vVv1+EULa77oHMbQ+8mDteDBe7MZdue/qc0eX/gt3qRbMrZEBH1GdIawJ/cnMxNFr0ZFQRPY= X-Received: by 2002:a05:620a:9cf:: with SMTP id y15mr12307315qky.44.1611775241062; Wed, 27 Jan 2021 11:20:41 -0800 (PST) MIME-Version: 1.0 References: <10fec1da415cd8d3c080a2173b9245ff35fceac8.camel@freebsd.org> In-Reply-To: From: Warner Losh Date: Wed, 27 Jan 2021 12:20:30 -0700 Message-ID: Subject: Re: Change in the uname To: Li-Wen Hsu Cc: Ian Lepore , "freebsd-arch@freebsd.org" X-Rspamd-Queue-Id: 4DQtjk0gj3z3F5L X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=sZOwzdEA; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::72b) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-3.00 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arch@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::72b:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::72b:from]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::72b:from]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; MAILMAN_DEST(0.00)[freebsd-arch] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jan 2021 19:20:43 -0000 On Mon, Jan 25, 2021 at 3:03 PM Li-Wen Hsu wrote: > On Tue, Jan 26, 2021 at 5:04 AM Ian Lepore wrote: > > That all sounds good, except the 'v' is a bit odd, normally it implies > > a "version" number, which isn't exactly what this is. How about 'n' or > > '#' since it's just a count? > > Seconded, sorry for discussing the color of bikeshed, but 'v' is also > not intuitive to me. Since it is still a counter, and we are removing > the 'g' in the second part, cXXXXX-HASH is still distinguishable with > the old cXXXXX-gHASH. > > Or, maybe call it gXXXXX-HASH which stands for git commit count XXXXX, > with hash value HASH? > I can do 'n' instead of 'v'. I think that 'g' would be a bit confusing since the connection to git is less obvious. I happen to like the 'old school' versioning that would do X.Y(Z) where Z is a count of changes (whatever that means) (either since X.0 started or back to the start of the thing). That would be branch(version)-hash. I opted to not implement that, though. While I like it, I know I'm old school and this style has fallen out of fashion since the 80s and 90s when it was popular on the 'big iron' (TOPS-20, IBM, etc) as the 'big iron' culture has given way to other schools of thought. It's also harder to script parse and harder to implement. All in all, I think 'v' or 'n' is best and most clear. # I'd like to avoid because in the non-reproducible build case #x is the build number. Any last comments before I roll this in later today to hit the MFC window? Warner