From nobody Tue Jul 30 16:15:40 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WYL1w0kRJz5RV0f for ; Tue, 30 Jul 2024 16:16:04 +0000 (UTC) (envelope-from ararslan@comcast.net) Received: from resqmta-a2p-658780.sys.comcast.net (resqmta-a2p-658780.sys.comcast.net [IPv6:2001:558:fd01:2bb4::5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WYL1t41Qbz47BL for ; Tue, 30 Jul 2024 16:16:02 +0000 (UTC) (envelope-from ararslan@comcast.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=comcast.net header.s=20190202a header.b=2lk4nfL3; dmarc=pass (policy=quarantine) header.from=comcast.net; spf=pass (mx1.freebsd.org: domain of ararslan@comcast.net designates 2001:558:fd01:2bb4::5 as permitted sender) smtp.mailfrom=ararslan@comcast.net Received: from resomta-a2p-647652.sys.comcast.net ([96.103.145.226]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 256/256 bits) (Client did not present a certificate) by resqmta-a2p-658780.sys.comcast.net with ESMTPS id Ymsisa7rMVTmSYpVxsm3LR; Tue, 30 Jul 2024 16:15:53 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; s=20190202a; t=1722356153; bh=L2Ju1HV+lZZVEhAaUWoefLnU1SlQFJ/DvdK5WdU0EAc=; h=Received:Received:Content-Type:Mime-Version:Subject:From:Date: Message-Id:To:Xfinity-Spam-Result; b=2lk4nfL3sTCgYHGnTxcwQLwngoyvDdtpdFEC2IuUKYT33DtsCk6kQkk7VJPf8twD1 f4iWC64pmwoIWSI1gCphLaRVFWpWTmxQqIQ8gRABVp2Cia1lnwAOD7cdeq9dlQ6PZq txgfsYZUzaFFDLYcHs1RDrvMR/CzAJyRkqcz+EWJcgZvAscN57Xe7rzAg4ExF1ykXH 7xP8Mm4GR6yxMCAQ85A3E1sKfauvEOzRCuR/1th1/m4LpyC7YP6kUy7qiBBL35S1RI Mf1z08Qjc1bizN2PNkw9SBlWWIe5QdpVLBtsehKMghmHl/ABCW3guoRSmRIDrkGRig khwyXRLnPrXnQ== Received: from smtpclient.apple ([67.160.29.205]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 256/256 bits) (Client did not present a certificate) by resomta-a2p-647652.sys.comcast.net with ESMTPSA id YpVvsEnc8pd4XYpVwsYi1L; Tue, 30 Jul 2024 16:15:53 +0000 Content-Type: text/plain; charset=utf-8 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: Diagnosing virtual machine network issues From: Alex Arslan In-Reply-To: <4AB1C33B-DD93-4484-B63A-9FF8FE612B15@comcast.net> Date: Tue, 30 Jul 2024 09:15:40 -0700 Cc: freebsd-hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <4a5a177a-5356-453c-8a09-f1d63d5d2e16@sentex.net> <4AB1C33B-DD93-4484-B63A-9FF8FE612B15@comcast.net> To: mike tancsa X-Mailer: Apple Mail (2.3774.600.62) X-CMAE-Envelope: MS4xfAEh4InnfXxmteUqlLp+uVLpZA8l6gFlzwlItECO8Sr2+JDfFeLjnOBpVRW/k74w0BFkHRPnErs+gZ25RCV5UAAQrS3JNySpBCOOGMrffpRGo+GunIAb OX8DnMAnmnPsq7EJktJaArKCCaHSPIWS6tRWT62qNQRez/5rrbkPKguI4naM0pqyi/hd3c98wkfT/VFR5wtw2f+tmsYrHEidHlsYbkPo0m1yx0ChsFNxEAy5 X-Spamd-Bar: / X-Spamd-Result: default: False [-1.00 / 15.00]; HFILTER_HELO_5(3.00)[resqmta-a2p-658780.sys.comcast.net]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[comcast.net,quarantine]; R_SPF_ALLOW(-0.20)[+ip6:2001:558:fd01:2bb4::/64]; R_DKIM_ALLOW(-0.20)[comcast.net:s=20190202a]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; DWL_DNSWL_NONE(0.00)[comcast.net:dkim]; ARC_NA(0.00)[]; FREEMAIL_FROM(0.00)[comcast.net]; TO_DN_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; FREEMAIL_ENVFROM(0.00)[comcast.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[comcast.net:+]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:7922, ipnet:2001:558::/29, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[96.103.145.226:received] X-Rspamd-Queue-Id: 4WYL1t41Qbz47BL Some progress on this front... > On Jul 19, 2024, at 9:08=E2=80=AFAM, Alex Arslan = wrote: >=20 >> If it happens frequently and you think it might be the network, = perhaps try with the Intel em driver instead of the virtio network = driver ? I forgot to mention, we swapped the virtio network device to the Intel e1000 device but haven't observed any difference. I did manage to narrow down the issue to a more minimal reproducer, = which is just a simple request to the invalid domain using libcurl, but with a 30-second timeout. (The Julia code sets that timeout internally.) I = tried compiling the reproducer and running the executable wrapped with `time` separately in the FreeBSD VM and on the Linux host. In the VM, it = actually sits and waits out the 30 seconds then errors with a timeout, and `time` reports a real time of 30.289 seconds. On the host, it's instantaneous; the resolution failure is reported immediately. I then tried setting the timeout in the VM to 31 seconds, and it produces the expected resolution failure with a real time of 30.420 seconds. I don't understand what's going on here... From nobody Tue Jul 30 20:27:53 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WYRcY3xF1z5Rvvd for ; Tue, 30 Jul 2024 20:27:57 +0000 (UTC) (envelope-from bacon4000@gmail.com) Received: from mail-yb1-xb2e.google.com (mail-yb1-xb2e.google.com [IPv6:2607:f8b0:4864:20::b2e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WYRcY0vJmz4kts for ; Tue, 30 Jul 2024 20:27:57 +0000 (UTC) (envelope-from bacon4000@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-yb1-xb2e.google.com with SMTP id 3f1490d57ef6-e0874f138aeso3134465276.2 for ; Tue, 30 Jul 2024 13:27:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1722371275; x=1722976075; darn=freebsd.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=yENPUMpdeB0PgU1jmVVYrmmA7xPHK6KJo/h725MqWhU=; b=LdTvQZf+W9s4X1aWPAte7fgWOJJcNT8ALeFvU1lTjtdZRuTDq8V0ZuwfFgWwIu00jY JBdvtILABWWk+5HHBraG/Ilp208Le3J39Sn28vsi6+PoLB/Wt+5gO132U9rEECtn+LRe Wm28JmLD89KipBrLqY/9aKsON5235vv1zjRZh4idK/hAKizQ9wuNqrjSCMaDurVlPblj RP/CLmORM7H1WFzb5srhL5QSO52M0cgArjeMuINviaJEBfTKBfZ3VcxnntsSMJ+SULoQ yDvzvt1cw2tVR4nW3fS0SxhvLUFzb4D6MZ5YkHxwbc7Cd6Ph0L23hPrS/S3Nc4gb1UlZ kOoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722371275; x=1722976075; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=yENPUMpdeB0PgU1jmVVYrmmA7xPHK6KJo/h725MqWhU=; b=q9muYq94RKRTsiU+6SqQ37Q+z3Ex9Ef8gDS5JtEsQ/QBI7DzHIvAuwpL7rWolF1xSH FV7pw28Jz/ahHbBjjz3fXFvQPaZYToFnjUFE3AqWY1g4n0jxYUj5zfmVoxIdBw2p8rKo 1oYK0NMfia9gq7qmf5i5NURWbJ75tvzF8kCIaFKT6mm7oJJlurp7u4HuwugWW8jKANWK EZshF93sVCaKluaewdqqpaq68YiYBm+JXyFPe2fps8N81kRURxRm/+T4HeFwqGgjWvzw 2G7dMNXRFaB3jyQrNTI05MC7JJO5D2ZXDaqmqxKBS3qIwOeS6y1Yh+lkS6euZydWtm9d NypQ== X-Gm-Message-State: AOJu0YzWcwa/h7ZsnflblZZC4ySxJ+SxGLfXL8/sR/MVHo6RIVa3lFQi tvhlSXXqsykP0iOrhfb7+iE2FY66WaKJF8n0MlzkPLVLNhk+zvVY X-Google-Smtp-Source: AGHT+IFJ5sb4kQ/2nlQDDGcpGKuaz5oVJbe6jFIeO+s9fT4arYxOW6yDn/jcw/urtmmLhX+wjDSrmA== X-Received: by 2002:a05:6902:168b:b0:e05:fdbe:bbf7 with SMTP id 3f1490d57ef6-e0b546006c1mr11128162276.42.1722371275131; Tue, 30 Jul 2024 13:27:55 -0700 (PDT) Received: from [192.168.0.146] (108-255-3-0.lightspeed.milwwi.sbcglobal.net. [108.255.3.0]) by smtp.gmail.com with ESMTPSA id 3f1490d57ef6-e0b33f63e1fsm2256649276.39.2024.07.30.13.27.54 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 30 Jul 2024 13:27:54 -0700 (PDT) Message-ID: <799c7a15-52b8-4b44-bcbd-5ab6a3ef97a6@gmail.com> Date: Tue, 30 Jul 2024 15:27:53 -0500 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Diagnosing virtual machine network issues To: Alex Arslan , mike tancsa Cc: freebsd-hackers@freebsd.org References: <4a5a177a-5356-453c-8a09-f1d63d5d2e16@sentex.net> <4AB1C33B-DD93-4484-B63A-9FF8FE612B15@comcast.net> Content-Language: en-US From: Jason Bacon In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4WYRcY0vJmz4kts T24gNy8zMC8yNCAxMToxNSwgQWxleCBBcnNsYW4gd3JvdGU6DQo+IFNvbWUgcHJvZ3Jlc3Mg b24gdGhpcyBmcm9udC4uLg0KPiANCj4+IE9uIEp1bCAxOSwgMjAyNCwgYXQgOTowOOKAr0FN LCBBbGV4IEFyc2xhbiA8YXJhcnNsYW5AY29tY2FzdC5uZXQ+IHdyb3RlOg0KPj4NCj4+PiBJ ZiBpdCBoYXBwZW5zIGZyZXF1ZW50bHkgYW5kIHlvdSB0aGluayBpdCBtaWdodCBiZSB0aGUg bmV0d29yaywgcGVyaGFwcyB0cnkgd2l0aCB0aGUgSW50ZWwgZW0gZHJpdmVyIGluc3RlYWQg b2YgdGhlIHZpcnRpbyBuZXR3b3JrIGRyaXZlciA/DQo+IA0KPiBJIGZvcmdvdCB0byBtZW50 aW9uLCB3ZSBzd2FwcGVkIHRoZSB2aXJ0aW8gbmV0d29yayBkZXZpY2UgdG8gdGhlIEludGVs DQo+IGUxMDAwIGRldmljZSBidXQgaGF2ZW4ndCBvYnNlcnZlZCBhbnkgZGlmZmVyZW5jZS4N Cj4gDQo+IEkgZGlkIG1hbmFnZSB0byBuYXJyb3cgZG93biB0aGUgaXNzdWUgdG8gYSBtb3Jl IG1pbmltYWwgcmVwcm9kdWNlciwgd2hpY2gNCj4gaXMganVzdCBhIHNpbXBsZSByZXF1ZXN0 IHRvIHRoZSBpbnZhbGlkIGRvbWFpbiB1c2luZyBsaWJjdXJsLCBidXQgd2l0aCBhDQo+IDMw LXNlY29uZCB0aW1lb3V0LiAoVGhlIEp1bGlhIGNvZGUgc2V0cyB0aGF0IHRpbWVvdXQgaW50 ZXJuYWxseS4pIEkgdHJpZWQNCj4gY29tcGlsaW5nIHRoZSByZXByb2R1Y2VyIGFuZCBydW5u aW5nIHRoZSBleGVjdXRhYmxlIHdyYXBwZWQgd2l0aCBgdGltZWANCj4gc2VwYXJhdGVseSBp biB0aGUgRnJlZUJTRCBWTSBhbmQgb24gdGhlIExpbnV4IGhvc3QuIEluIHRoZSBWTSwgaXQg YWN0dWFsbHkNCj4gc2l0cyBhbmQgd2FpdHMgb3V0IHRoZSAzMCBzZWNvbmRzIHRoZW4gZXJy b3JzIHdpdGggYSB0aW1lb3V0LCBhbmQgYHRpbWVgDQo+IHJlcG9ydHMgYSByZWFsIHRpbWUg b2YgMzAuMjg5IHNlY29uZHMuIE9uIHRoZSBob3N0LCBpdCdzIGluc3RhbnRhbmVvdXM7DQo+ IHRoZSByZXNvbHV0aW9uIGZhaWx1cmUgaXMgcmVwb3J0ZWQgaW1tZWRpYXRlbHkuIEkgdGhl biB0cmllZCBzZXR0aW5nIHRoZQ0KPiB0aW1lb3V0IGluIHRoZSBWTSB0byAzMSBzZWNvbmRz LCBhbmQgaXQgcHJvZHVjZXMgdGhlIGV4cGVjdGVkIHJlc29sdXRpb24NCj4gZmFpbHVyZSB3 aXRoIGEgcmVhbCB0aW1lIG9mIDMwLjQyMCBzZWNvbmRzLiBJIGRvbid0IHVuZGVyc3RhbmQg d2hhdCdzDQo+IGdvaW5nIG9uIGhlcmUuLi4NCj4gDQoNCkNhbiB5b3UgcHJvdmlkZSBtb3Jl IGNvbnRleHQ/ICBJJ20gbm90IHNlZWluZyBlYXJsaWVyIG1lc3NhZ2VzIGFueXdoZXJlIA0K aW4gbXkgZW1haWwgZm9sZGVycy4gIElzIHRoaXMgYSBRZW11IGlzc3VlPw0KDQpDb2luY2lk ZW50YWxseSwgSSdtIGV4cGVyaW1lbnRpbmcgd2l0aCBGcmVlQlNEIHVuZGVyIFFlbXUgb24g bXkgTWFjIE1pbmkgDQpNMSBhbmQgc2VlaW5nIGFib3V0IDkzIG1iaXRzL3NlYyBpbiBpcGVy ZiwgcmVnYXJkbGVzcyBvZiB0aGUgTklDIA0KY29uZmlndXJlZC4gICggVk0gdG8gYmFyZSBt ZXRhbCBob3N0ICkgIEJhcmUgbWV0YWwgdG8gYmFyZSBtZXRhbCBzaG93cyANCjkzMCBtYml0 cy9zZWMuDQoNCi0tIA0KTGlmZSBpcyBhIGdhbWUuICBQbGF5IGhhcmQuICBQbGF5IGZhaXIu ICBIYXZlIGZ1bi4NCg0K From nobody Tue Jul 30 21:11:55 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WYScJ3drBz5S0WN for ; Tue, 30 Jul 2024 21:12:48 +0000 (UTC) (envelope-from ararslan@comcast.net) Received: from resqmta-a2p-640029.sys.comcast.net (resqmta-a2p-640029.sys.comcast.net [IPv6:2001:558:fd01:2bb4::b]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WYScJ1XlFz4qYK for ; Tue, 30 Jul 2024 21:12:48 +0000 (UTC) (envelope-from ararslan@comcast.net) Authentication-Results: mx1.freebsd.org; none Received: from resomta-a2p-646770.sys.comcast.net ([96.103.145.230]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 256/256 bits) (Client did not present a certificate) by resqmta-a2p-640029.sys.comcast.net with ESMTPS id YkHPsEtJPovMQYu8zsfrkM; Tue, 30 Jul 2024 21:12:29 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; s=20190202a; t=1722373949; bh=0NXD0JIj9rbEgWkUAtyq0p8p8qkcU/VABmFARtL6dhc=; h=Received:Received:Content-Type:Mime-Version:Subject:From:Date: Message-Id:To:Xfinity-Spam-Result; b=cggL86Q3yGP829+evVuI2lo4cxXDWuLr1zlLc2pzfjQVQzbVMQZavtGNNQ8Iw6qzv mnMJtpnx+NGfoEaaf/r68E85pmNStw1W7ahnpXfaCjDFVUEYfQVhKHltXuiOBA+PY9 i5vxvrv3GAN6+tVxmZjKMHQxeZAlcOLIbihvIc7yt4lfoxkNkZSkPQ2/MPTwxLWsOA Al6jIZVXqGb93fGLrm7sOPPEb4Bjk8pbzRZwfXgjUsukPPXwmu7Y/zQ7hbJzSL722n eyLrFko3HDPrfNejtCdxNz8lJoqTJEY+TYJS1/z1KVbQsso7LS1KvLcOLNpAO9W8cS 50knfcJOlFhlQ== Received: from smtpclient.apple ([67.160.29.205]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 256/256 bits) (Client did not present a certificate) by resomta-a2p-646770.sys.comcast.net with ESMTPSA id Yu8bsuOsYvtH6Yu8csg6xY; Tue, 30 Jul 2024 21:12:07 +0000 Content-Type: text/plain; charset=us-ascii List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: Diagnosing virtual machine network issues From: Alex Arslan In-Reply-To: <799c7a15-52b8-4b44-bcbd-5ab6a3ef97a6@gmail.com> Date: Tue, 30 Jul 2024 14:11:55 -0700 Cc: mike tancsa , freebsd-hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <0747ED5F-2ED6-461C-9C0B-CFD0EE480D82@comcast.net> References: <4a5a177a-5356-453c-8a09-f1d63d5d2e16@sentex.net> <4AB1C33B-DD93-4484-B63A-9FF8FE612B15@comcast.net> <799c7a15-52b8-4b44-bcbd-5ab6a3ef97a6@gmail.com> To: Jason Bacon X-Mailer: Apple Mail (2.3774.600.62) X-CMAE-Envelope: MS4xfGfAh/SC4nD6ApzwYp2omROyjKaVXb2tXF0JhsndxbEBbRcGE+hGqWevkmxP68zV02p9emxHHP1OYMWs5J8hip0EefFg8Mu+rTyTbmhv3sjei4yUf247 GHCqL2XziVPzHyoSLEVeDjF8HclreQlOORzPBz+t2gdMrHmVLqdzbRAJ956ufltbPkLXfeOeER3/YdLG++VYOqYNocALB9q588PVM4b4VUC/yiqfgAqO84Xy oyZ8bb/uhhyqBMAJqEsWqe7lXgEQAusKaQ12DAlvjto= X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7922, ipnet:2001:558::/29, country:US] X-Rspamd-Queue-Id: 4WYScJ1XlFz4qYK > Can you provide more context? I'm not seeing earlier messages = anywhere in my email folders. Is this a Qemu issue? The original message is from just over a month ago, archived here: https://lists.freebsd.org/archives/freebsd-hackers/2024-June/003378.html Basically, we have FreeBSD 13.2 VMs running under KVM on a Linux = machine. Some code is using libcurl to make a request to an invalid domain and is testing that the error is a resolution failure. This test passes on all platforms except specifically in these FreeBSD VMs; I can't reproduce locally on FreeBSD. That made me think that there's an issue with how = the VM was set up, prompting the original message and discussion. Then what I recently found was that we set a 30-second timeout for the libcurl request, which FreeBSD hits in the VM, as it evidently spends a full 30 seconds attempting to resolve the host, while e.g. Linux reports a resolution failure immediately. > Coincidentally, I'm experimenting with FreeBSD under Qemu on my Mac = Mini M1 and seeing about 93 mbits/sec in iperf, regardless of the NIC = configured. ( VM to bare metal host ) Bare metal to bare metal shows = 930 mbits/sec. That's interesting, can you show how you did that? I'm not familiar with iperf (or most things in the realm of networking). Do you know why it's so much slower?= From nobody Tue Jul 30 21:22:54 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WYSrG06C7z5S1R5 for ; Tue, 30 Jul 2024 21:23:10 +0000 (UTC) (envelope-from bakul@iitbombay.org) Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WYSrD3Srgz4sjb for ; Tue, 30 Jul 2024 21:23:08 +0000 (UTC) (envelope-from bakul@iitbombay.org) Authentication-Results: mx1.freebsd.org; none Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-70eae5896bcso4327846b3a.2 for ; Tue, 30 Jul 2024 14:23:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iitbombay-org.20230601.gappssmtp.com; s=20230601; t=1722374586; x=1722979386; darn=freebsd.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=SykG7BQPk1q+j0c5Wfzk5UoGAPatmGhcWwLqZG7BByQ=; b=N9WKtCqvfZIo1JDSd3rWxUPPL2wRR4uFELBSAG+6S/Ir80HwIbktDYhEOPyB/tZAzc 9blqg0Oux+FZwvY40hYWmIq6CdcfTP+ROw/Ej+xkj+3Au5euTNC6yNVg2euLLcaFm/oL g6GaLMT+p4yZK0lKF3KhBl5+wMQg7jycBb3O9CjMjBI9WoqMEgtWcdqVwtDcWxWhC8UC ZZLR4NcDg16+tQDFSg6eeS5Zy1wvYa8DoaT5tF2CwG2zZ9TrWtpT9tDWtjHIQFbNUqth bzobyAJlkC9fiqxTOFeVIcRvlgka35r80iS2h0Tok0gfcVMn0vdVZlnyvh97uxD9eKeW 2pDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722374586; x=1722979386; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SykG7BQPk1q+j0c5Wfzk5UoGAPatmGhcWwLqZG7BByQ=; b=NjE3cAJZTCzbq3euBq8CJB9CRf3hgarHaL98DFRRxLqLszKp+60oUQX6m2pPv7Drf4 gcUpPVRjZ9a6hpUm4yQG2vLlYBBExLQjjJHbYc5M+iVQZrzcHjMXHFOZS4ffOWm9elM7 Y18F4SGBqhIgqW2u/zznxqCnNdBLqbIbiexY6jEB5lE3YXLslu1IWSKB2n7zFytx2b2k 19w6CbNA5zNvtTHJJVAMmigXkuoYX5oEulOjKb9oWdTaip/eFJ9+ow0jW+rvMpUrnNOz o+2QyXC7Nc5m3YLmEmPRBU4+q5cFNpM+ZcapcZfNpjhk/b0hBaXDYhrtPFtsMY37kt7h JBQQ== X-Gm-Message-State: AOJu0YzjivaiwJwS/s3vKYC/cYgXM8Ko5j8Rq+UqWG3fgv+AatL2RP9U Tj/Uxl+KrKwdMgtH2F+LfkUVlljIyV1y1yYPcrEkfa6RJfnHdNvUXqOqYnQTKA== X-Google-Smtp-Source: AGHT+IE+QGjpMnjrYeTi90OBX0Z67vjLicrGyF9AFHG3riM3CEwSYmrGfCvW3mNUKELyZQpOFqY5Eg== X-Received: by 2002:a05:6a20:6a0b:b0:1c4:f209:f1e2 with SMTP id adf61e73a8af0-1c4f209f466mr561669637.13.1722374585929; Tue, 30 Jul 2024 14:23:05 -0700 (PDT) Received: from smtpclient.apple (107-215-223-229.lightspeed.sntcca.sbcglobal.net. [107.215.223.229]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7a9f9ec3d0esm9494248a12.70.2024.07.30.14.23.04 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 30 Jul 2024 14:23:05 -0700 (PDT) Content-Type: text/plain; charset=utf-8 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: Diagnosing virtual machine network issues From: Bakul Shah In-Reply-To: <0747ED5F-2ED6-461C-9C0B-CFD0EE480D82@comcast.net> Date: Tue, 30 Jul 2024 14:22:54 -0700 Cc: FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: <2B0A1E6F-7B89-4F7C-9ECE-ABA94E476D5A@iitbombay.org> References: <4a5a177a-5356-453c-8a09-f1d63d5d2e16@sentex.net> <4AB1C33B-DD93-4484-B63A-9FF8FE612B15@comcast.net> <799c7a15-52b8-4b44-bcbd-5ab6a3ef97a6@gmail.com> <0747ED5F-2ED6-461C-9C0B-CFD0EE480D82@comcast.net> To: Alex Arslan X-Mailer: Apple Mail (2.3774.600.62) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4WYSrD3Srgz4sjb > On Jul 30, 2024, at 2:11=E2=80=AFPM, Alex Arslan = wrote: >=20 >=20 >> Can you provide more context? I'm not seeing earlier messages = anywhere in my email folders. Is this a Qemu issue? >=20 > The original message is from just over a month ago, archived here: > = https://lists.freebsd.org/archives/freebsd-hackers/2024-June/003378.html > Basically, we have FreeBSD 13.2 VMs running under KVM on a Linux = machine. > Some code is using libcurl to make a request to an invalid domain and = is > testing that the error is a resolution failure. This test passes on = all > platforms except specifically in these FreeBSD VMs; I can't reproduce > locally on FreeBSD. That made me think that there's an issue with how = the > VM was set up, prompting the original message and discussion. Then = what > I recently found was that we set a 30-second timeout for the libcurl > request, which FreeBSD hits in the VM, as it evidently spends a full > 30 seconds attempting to resolve the host, while e.g. Linux reports a > resolution failure immediately. What does /etc/resolv.conf look like on the FreeBSD VM?= From nobody Tue Jul 30 21:53:28 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WYTWx1jClz5S3hY for ; Tue, 30 Jul 2024 21:54:05 +0000 (UTC) (envelope-from ararslan@comcast.net) Received: from resqmta-c2p-570110.sys.comcast.net (resqmta-c2p-570110.sys.comcast.net [IPv6:2001:558:fd00:56::b]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WYTWw18sdz3wsX for ; Tue, 30 Jul 2024 21:54:04 +0000 (UTC) (envelope-from ararslan@comcast.net) Authentication-Results: mx1.freebsd.org; none Received: from resomta-c2p-555954.sys.comcast.net ([96.102.18.234]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 256/256 bits) (Client did not present a certificate) by resqmta-c2p-570110.sys.comcast.net with ESMTPS id YtyKsAXzOghfMYunCsAZWc; Tue, 30 Jul 2024 21:54:02 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; s=20190202a; t=1722376442; bh=xPgUCHaepCFX4Xn/LoJ2aWQKAVXVbFCXJJO5sIZNkCE=; h=Received:Received:From:Message-Id:Content-Type:Mime-Version: Subject:Date:To:Xfinity-Spam-Result; b=WJzi7Z7R0PpaOvtj7s5frT78hsP3O9CHOxxl8b87igaobb4puH/6VlZQLCg77tlWM zZHvgkulZ8SScnVu+x3qRxbOCkHdghesdqStYXPUGmfNLwnAJ6/yFLORGs86aC22py Yp9JaVqQ2VnswE7m+FXyk189iTwNm2m7Mhzr1Ov2iowgrL7lSqQXrRRxrk0jJK3m0J qkkdt5B9VZQXgicaUH6jpOdfQ04CDSNhZk2KfKO7Kcfsf/sqHdpmfRD7ox6cdxKkkY l5nvx52huG+BOlpMOG0c/wnsexXwxouMkE+zXkOVH0Q+opAuKfsX4MtTWaI0I0v8a3 sAECIX8K2dsGg== Received: from smtpclient.apple ([67.160.29.205]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 256/256 bits) (Client did not present a certificate) by resomta-c2p-555954.sys.comcast.net with ESMTPSA id Yumosrx9bHLd3YumqsAYUj; Tue, 30 Jul 2024 21:53:40 +0000 From: Alex Arslan Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_ABE8E1E7-5527-4C1F-88D6-08A485B4B25D" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: Diagnosing virtual machine network issues Date: Tue, 30 Jul 2024 14:53:28 -0700 In-Reply-To: <2B0A1E6F-7B89-4F7C-9ECE-ABA94E476D5A@iitbombay.org> Cc: FreeBSD Hackers To: Bakul Shah References: <4a5a177a-5356-453c-8a09-f1d63d5d2e16@sentex.net> <4AB1C33B-DD93-4484-B63A-9FF8FE612B15@comcast.net> <799c7a15-52b8-4b44-bcbd-5ab6a3ef97a6@gmail.com> <0747ED5F-2ED6-461C-9C0B-CFD0EE480D82@comcast.net> <2B0A1E6F-7B89-4F7C-9ECE-ABA94E476D5A@iitbombay.org> X-Mailer: Apple Mail (2.3774.600.62) X-CMAE-Envelope: MS4xfIqEhx5ZZnJGGOjtqLp5hDJZEBT1OlmKy92e7ax3aa3LLLTE0QdQiFqu6wDXIbh2WFyT2ARSv8HEXBMHhZPOicDpC4sYNpwXdCKkmwkKVC2KBMnLmgSF 5WzgmblYZkau8p7PTJFjw+hDbBa8EinNek0zw2NbgLsHF00H25hjpTOfxWSvza/5HaaOnaPK7rZupeGIqdtHp0CgYbEL/erN+7Rz/GTvP/jrmPuDNXIo/2OO X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7922, ipnet:2001:558::/29, country:US] X-Rspamd-Queue-Id: 4WYTWw18sdz3wsX --Apple-Mail=_ABE8E1E7-5527-4C1F-88D6-08A485B4B25D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jul 30, 2024, at 2:22=E2=80=AFPM, Bakul Shah = wrote: >=20 >> On Jul 30, 2024, at 2:11=E2=80=AFPM, Alex Arslan = wrote: >>=20 >>> Can you provide more context? I'm not seeing earlier messages = anywhere in my email folders. Is this a Qemu issue? >>=20 >> The original message is from just over a month ago, archived here: >> = https://lists.freebsd.org/archives/freebsd-hackers/2024-June/003378.html >> Basically, we have FreeBSD 13.2 VMs running under KVM on a Linux = machine. >> Some code is using libcurl to make a request to an invalid domain and = is >> testing that the error is a resolution failure. This test passes on = all >> platforms except specifically in these FreeBSD VMs; I can't reproduce >> locally on FreeBSD. That made me think that there's an issue with how = the >> VM was set up, prompting the original message and discussion. Then = what >> I recently found was that we set a 30-second timeout for the libcurl >> request, which FreeBSD hits in the VM, as it evidently spends a full >> 30 seconds attempting to resolve the host, while e.g. Linux reports a >> resolution failure immediately. >=20 > What does /etc/resolv.conf look like on the FreeBSD VM? Just a comment and a name server line: $ cat /etc/resolv.conf # Generated by resolvconf nameserver 192.168.122.1= --Apple-Mail=_ABE8E1E7-5527-4C1F-88D6-08A485B4B25D Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
On Jul 30, = 2024, at 2:22=E2=80=AFPM, Bakul Shah <bakul@iitbombay.org> = wrote:

On Jul 30, 2024, at 2:11=E2=80=AFPM, Alex Arslan = <ararslan@comcast.net> wrote:

Can = you provide more context?  I'm not seeing earlier messages anywhere = in my email folders.  Is this a Qemu issue?

The = original message is from just over a month ago, archived = here:
https://lists.freebsd.org/archives/freebsd-hackers/2024-June/0033= 78.html
Basically, we have FreeBSD 13.2 VMs running under KVM on a = Linux machine.
Some code is using libcurl to make a request to an = invalid domain and is
testing that the error is a resolution failure. = This test passes on all
platforms except specifically in these = FreeBSD VMs; I can't reproduce
locally on FreeBSD. That made me think = that there's an issue with how the
VM was set up, prompting the = original message and discussion. Then what
I recently found was that = we set a 30-second timeout for the libcurl
request, which FreeBSD = hits in the VM, as it evidently spends a full
30 seconds attempting = to resolve the host, while e.g. Linux reports a
resolution failure = immediately.

What = does /etc/resolv.conf look like on the FreeBSD = VM?

Just a comment and a = name server line:

$ cat /etc/resolv.conf
# = Generated by resolvconf
nameserver = 192.168.122.1
= --Apple-Mail=_ABE8E1E7-5527-4C1F-88D6-08A485B4B25D-- From nobody Tue Jul 30 22:54:17 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WYVsR6nCGz5S8kF for ; Tue, 30 Jul 2024 22:54:19 +0000 (UTC) (envelope-from bacon4000@gmail.com) Received: from mail-qk1-x72c.google.com (mail-qk1-x72c.google.com [IPv6:2607:f8b0:4864:20::72c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WYVsR50t5z43gQ for ; Tue, 30 Jul 2024 22:54:19 +0000 (UTC) (envelope-from bacon4000@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-qk1-x72c.google.com with SMTP id af79cd13be357-7a1df0a93eeso286973785a.1 for ; Tue, 30 Jul 2024 15:54:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1722380059; x=1722984859; darn=freebsd.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=R/XcHkmziDOFbYwFtE9zgAQsY+7/3FLSK5sYRuEvzuA=; b=cLLofk3LbL4PrCGtOhtYqpMwP5thRxnX+huhPEBmQ7QKUh0RSbfmVnGWLPIIJjZizw /JJoDUstYVJ77XsAkeGiD7bvdbNAKNtODomxW/q/TkxTD5uZ7zenKN/GThxmrdnVuhUR NhMgZUSHseT07Vgz/xb+L6H8zwR7624EK8+0Sh1jVf2xMd5Dg2jlfdvi7FYTxdkANZXO Mr1tcWKNJ+wqP1jjMyI1YS7JvX81wwKRUKVgLnbSPAze2WQYMOWyg/oZW8klxo5j7KEC gSrTL5Mf67ULDGo+TGiIcGW9yS18MXWf/nrDeGx1w9h5cJRLuuzxpBLGj/0LlBqaJpEO Ey+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722380059; x=1722984859; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=R/XcHkmziDOFbYwFtE9zgAQsY+7/3FLSK5sYRuEvzuA=; b=IRof2oxyr74gylvGnQ+91UQco7Dqjp1UEBhLVhig55p6c8ggtfijd9T1qB1jYKWt27 JqbOwPKlsoK9wYKVmHEV743hg9MmBbsIsl65B/QdRzFTi81QQ07xsPzon0c6CCZIXefE 5KDfR7krYMrqNbLKeYBgx3ctTz2eUoYRQLtM+2KC/IJe6fPCNC0nOqGCnrbTCiOYIxfu Lne74ZJd4jA8nHb+VWrq8km6qEa3n1btBTxF/SdapbiggOzn0zephSTlY2wySiBDPLCI t5dkQCorCuoPLchr6GYdmrfK+1b9yzWBrBeRkRrbI3R8vmIgRdJIySDUh9f2QdWTUbxX cfgA== X-Forwarded-Encrypted: i=1; AJvYcCXhlnG0lxQg2wsbwzf4Hye5tR+V83smU+NbiD/3WIN6AdPbdV28F07TBViz4ngrQvpe137Ht0m4zryPpI2naB9wwK4gU0nCAGbCTbk= X-Gm-Message-State: AOJu0YygOtTfdQ/QY/9VfSMP4hl4wUyOxqVL1JXK9O8DkmvIbyvpn8Ly f/UjmQr2gPrOPPhB2tedFbcE027t26zfWvhGd1HybM33Qbhkbg7/eGGtoQ== X-Google-Smtp-Source: AGHT+IFp+jkTznPOovrAWZg+xVzfS1P1xsfcywmBMrsAhFF7S9FHIy5qGw9/LzmvZgv2DtmbfBraeQ== X-Received: by 2002:a05:620a:f11:b0:79f:190a:8ad7 with SMTP id af79cd13be357-7a1e526549bmr1113409685a.33.1722380058678; Tue, 30 Jul 2024 15:54:18 -0700 (PDT) Received: from [192.168.1.8] (syn-174-097-140-229.res.spectrum.com. [174.97.140.229]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7a1eec80b0esm351863985a.117.2024.07.30.15.54.17 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 30 Jul 2024 15:54:18 -0700 (PDT) Message-ID: Date: Tue, 30 Jul 2024 17:54:17 -0500 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Diagnosing virtual machine network issues To: Alex Arslan Cc: mike tancsa , freebsd-hackers@freebsd.org References: <4a5a177a-5356-453c-8a09-f1d63d5d2e16@sentex.net> <4AB1C33B-DD93-4484-B63A-9FF8FE612B15@comcast.net> <799c7a15-52b8-4b44-bcbd-5ab6a3ef97a6@gmail.com> <0747ED5F-2ED6-461C-9C0B-CFD0EE480D82@comcast.net> Content-Language: en-US From: Jason Bacon In-Reply-To: <0747ED5F-2ED6-461C-9C0B-CFD0EE480D82@comcast.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4WYVsR50t5z43gQ On 7/30/24 16:11, Alex Arslan wrote: > >> Can you provide more context? I'm not seeing earlier messages anywhere in my email folders. Is this a Qemu issue? > > The original message is from just over a month ago, archived here: > https://lists.freebsd.org/archives/freebsd-hackers/2024-June/003378.html > Basically, we have FreeBSD 13.2 VMs running under KVM on a Linux machine. > Some code is using libcurl to make a request to an invalid domain and is > testing that the error is a resolution failure. This test passes on all > platforms except specifically in these FreeBSD VMs; I can't reproduce > locally on FreeBSD. That made me think that there's an issue with how the > VM was set up, prompting the original message and discussion. Then what > I recently found was that we set a 30-second timeout for the libcurl > request, which FreeBSD hits in the VM, as it evidently spends a full > 30 seconds attempting to resolve the host, while e.g. Linux reports a > resolution failure immediately. > >> Coincidentally, I'm experimenting with FreeBSD under Qemu on my Mac Mini M1 and seeing about 93 mbits/sec in iperf, regardless of the NIC configured. ( VM to bare metal host ) Bare metal to bare metal shows 930 mbits/sec. > > > That's interesting, can you show how you did that? I'm not familiar with > iperf (or most things in the realm of networking). Do you know why it's > so much slower? Typical use is very simple: One one machine: iperf -s On the other: iperf -c hostname-of-1st-machine The iperf man page shows other options if you want to fine-tune. -- Life is a game. Play hard. Play fair. Have fun. From nobody Wed Jul 31 12:45:55 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WYsK84XmKz5RrgT for ; Wed, 31 Jul 2024 12:46:04 +0000 (UTC) (envelope-from bacon4000@gmail.com) Received: from mail-yw1-x1131.google.com (mail-yw1-x1131.google.com [IPv6:2607:f8b0:4864:20::1131]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WYsK73Sxwz4XpJ for ; Wed, 31 Jul 2024 12:46:03 +0000 (UTC) (envelope-from bacon4000@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=GRvVwDyz; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of bacon4000@gmail.com designates 2607:f8b0:4864:20::1131 as permitted sender) smtp.mailfrom=bacon4000@gmail.com Received: by mail-yw1-x1131.google.com with SMTP id 00721157ae682-67b709024bfso48222727b3.3 for ; Wed, 31 Jul 2024 05:46:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1722429962; x=1723034762; darn=freebsd.org; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:from:subject:user-agent:mime-version:date:message-id:from:to :cc:subject:date:message-id:reply-to; bh=uAOLz6x+4hYoaLDpCr2P0/eQKxqAY54OFXG1WMoecOc=; b=GRvVwDyzdS00RypEzpx9gAw16yzpJZ7d+HOb5W6fnhUCUsbsPRSkf4yhlTlR16ugxz dN+DsRHYvxhnJprgkN9LTR4OQWl9ZXiG0RHHaBXsqh+fATUfZmfrmmONKCVWAbxLEUEE 76w3bVDanmejoWs7b6met10/Iy06qdV/V4bmZDChg5kUKWl42JaXedHPnNBdhrkxQxnQ CX1dpd10zZwrf68cxbYh7KlGd0qYt/n8zLrA2E3DAkTah10dMZhpJX/ohWnXlaqw7Dhg MFetcXArYQV+7jtPNrw1FXWAYES2yepHH7N+sIBRPRpnl2G0JSu2mbDZo9+shOuY1+uh 80LQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722429962; x=1723034762; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:from:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=uAOLz6x+4hYoaLDpCr2P0/eQKxqAY54OFXG1WMoecOc=; b=WwNLBDqWAe6c1LR6BWBiWmWd8vfeNqhpegy6YDkYmUWADq72O2ymta9AVpyXvDKoN7 n4jMpGqMr1QsSFhKk+rkHPj2e7cyj8hjT68yINFhK0ygryspVQBeaWGXX0wIxDv3MY6c tcFxwX4HPqo17EtsAyS/wOzx+pruc6yb7PItsLiRGJtv9Q/URKUp3B9p4il1gCkovK93 t+Zuh5fBD5QpiZKpEkksVVMEaVb+F7RDRjkCnIt2nZP5q+sOrjN/ApcYqaWNeE1MKYSQ zvEw4KEDEqEEBtH0Fr5KVaMOGUizrlxMfVRVNA2Wa70bvpMV0JzffOx4oq2NvLP1XTVW 7/yw== X-Gm-Message-State: AOJu0YxFp4yj3vWuP2sx69XbyAAh/qGsKi3FG93ZKkICPpCwQsoMNKxP ipjAagRoF+2DLvc+LSJU1WTmpOLiksyn+u+5HFXm4go1jmJ0NmuuSt5aeA== X-Google-Smtp-Source: AGHT+IE63mjyWQu/r3DOSIBrh3QMQ/S0/P3hdVXHLFo5ZMhyrjvmf0wy0rfS4WBvcnC3e+r6c6m0Vg== X-Received: by 2002:a0d:ffc2:0:b0:65c:2536:bea2 with SMTP id 00721157ae682-67a072baa14mr161988657b3.19.1722429961971; Wed, 31 Jul 2024 05:46:01 -0700 (PDT) Received: from [192.168.0.146] (108-255-3-0.lightspeed.milwwi.sbcglobal.net. [108.255.3.0]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6756bab4cfbsm29475027b3.122.2024.07.31.05.46.01 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 31 Jul 2024 05:46:01 -0700 (PDT) Message-ID: Date: Wed, 31 Jul 2024 07:45:55 -0500 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Diagnosing virtual machine network issues From: Jason Bacon To: Alex Arslan , mike tancsa Cc: freebsd-hackers@freebsd.org References: <4a5a177a-5356-453c-8a09-f1d63d5d2e16@sentex.net> <4AB1C33B-DD93-4484-B63A-9FF8FE612B15@comcast.net> <799c7a15-52b8-4b44-bcbd-5ab6a3ef97a6@gmail.com> Content-Language: en-US In-Reply-To: <799c7a15-52b8-4b44-bcbd-5ab6a3ef97a6@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.15 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.26)[-0.258]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[text/plain]; MIME_BASE64_TEXT(0.10)[]; XM_UA_NO_VERSION(0.01)[]; FREEMAIL_TO(0.00)[comcast.net,sentex.net]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::1131:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-Rspamd-Queue-Id: 4WYsK73Sxwz4XpJ T24gNy8zMC8yNCAxNToyNywgSmFzb24gQmFjb24gd3JvdGU6DQo+IENvaW5jaWRlbnRhbGx5 LCBJJ20gZXhwZXJpbWVudGluZyB3aXRoIEZyZWVCU0QgdW5kZXIgUWVtdSBvbiBteSBNYWMg TWluaSANCj4gTTEgYW5kIHNlZWluZyBhYm91dCA5MyBtYml0cy9zZWMgaW4gaXBlcmYsIHJl Z2FyZGxlc3Mgb2YgdGhlIE5JQyANCj4gY29uZmlndXJlZC7CoCAoIFZNIHRvIGJhcmUgbWV0 YWwgaG9zdCApwqAgQmFyZSBtZXRhbCB0byBiYXJlIG1ldGFsIHNob3dzIA0KPiA5MzAgbWJp dHMvc2VjLg0KDQpOZXZlciBtaW5kLiAgVGhlcmUgd2FzIGFuIGludGVybWl0dGVudCBpc3N1 ZSBvZiBzb21lIHNvcnQsIHBvc3NpYmx5IGEgDQpsb29zZSBFdGhlcm5ldCBjYWJsZT8gIE1h Y09TIG1heSBoYXZlIGJlZW4gZmFsbGluZyBiYWNrIG9uIFdpRmkgZHVyaW5nIA0KbXkgVk0g dGVzdHMuICBJJ20gbm93IHNlZWluZyBuYXRpdmUgTklDIHBlcmZvcm1hbmNlIHVuZGVyIFFl bXUgd2l0aCAtbmljIA0KdXNlcixtb2RlbD12aXJ0aW86DQoNCjw8PFJPT1RAdGFycG9udm0u YWNhZGl4Pj4+IH4gMTYgIyBpcGVyZiAtYyBiYXJyYWN1ZGEgDQoNCi0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLSANCg0KQ2xp ZW50IGNvbm5lY3RpbmcgdG8gYmFycmFjdWRhLCBUQ1AgcG9ydCA1MDAxIA0KDQpUQ1Agd2lu ZG93IHNpemU6IDMyLjggS0J5dGUgKGRlZmF1bHQpIA0KDQotLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0gDQoNClsgIDFdIGxv Y2FsIDEwLjAuMi4xNSBwb3J0IDYwNjIzIGNvbm5lY3RlZCB3aXRoIDE5Mi4xNjguMC40OCBw b3J0IDUwMDEgDQoNClsgSURdIEludGVydmFsICAgICAgIFRyYW5zZmVyICAgICBCYW5kd2lk dGggDQoNClsgIDFdIDAuMDAtMTAuMDEgc2VjICAxLjA5IEdCeXRlcyAgIDkzOSBNYml0cy9z ZWMNCg0KU28gUWVtdSBpcyB0b3RhbGx5IHZpYWJsZSBmb3IgcnVubmluZyBhIGhpZ2gtcGVy Zm9ybWFuY2UgRnJlZUJTRCBWTSBvbiANCkFwcGxlIFNpbGljb24uDQoNCi0tIA0KTGlmZSBp cyBhIGdhbWUuICBQbGF5IGhhcmQuICBQbGF5IGZhaXIuICBIYXZlIGZ1bi4NCg0K From nobody Wed Jul 31 14:36:54 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WYvn66Z9Gz5S1DV for ; Wed, 31 Jul 2024 14:36:58 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-oi1-x22a.google.com (mail-oi1-x22a.google.com [IPv6:2607:f8b0:4864:20::22a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WYvn619bHz4jl4 for ; Wed, 31 Jul 2024 14:36:58 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=hardenedbsd.org header.s=google header.b="RP/owPUu"; dmarc=none; spf=pass (mx1.freebsd.org: domain of shawn.webb@hardenedbsd.org designates 2607:f8b0:4864:20::22a as permitted sender) smtp.mailfrom=shawn.webb@hardenedbsd.org Received: by mail-oi1-x22a.google.com with SMTP id 5614622812f47-3db14339fb0so3901170b6e.2 for ; Wed, 31 Jul 2024 07:36:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; t=1722436616; x=1723041416; darn=freebsd.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=KLFu5wt1mQgkqPG58Kxgr6FnHz4NlThe4Z/YR3efNl0=; b=RP/owPUuFn9f6GVkDr0BaFfBnvtXRZP3Bo12L5fBmOdw5cVVSy9SboGL0pccP+vbGc bP4xL/zblgrfvxk4RvWvIfKt4Z26DxWbjgj3kIP6veLi3jEe4R8U3IuLcWwg8Sa0SYcx lFfK5b1g9U0wZ9v50V4cwNWVF1m1Y25+XNbhsHURN3qVGFXnbW6dAs92NaB6c00Co3zT x9hQ8Tj4nq8ybqO+UJfkeBrixhQQECyCP9a94sWFgkVh68XhDCoC1BPeyHt6APO6gvlj M/IbYO3yLwcN8fQ15mVl9GpStpS1+rLlV0kCLySRONWujUA1nGYXWkyPazsEKurOULKf UbNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722436616; x=1723041416; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=KLFu5wt1mQgkqPG58Kxgr6FnHz4NlThe4Z/YR3efNl0=; b=au5izMRtzevPCqoIxmcgnwVoPrpYwZ6ZlXTHkPhrKztNqirP1CgmsYkby06pkis7cH lWRKXr7RatqzlhG7ePyY4cPDujH7QI+pCuAZAiWTguP6CFFTt17tcbJIQ/IiNMUv6FjH Kep3hnJj9E6o7NBDBhFLsaTG7U2XpfjBKAfI2XD1UizvZo1ofB7xXHGmNwnenLC2o+Ll ia74LwPX2IKuRtIXax/7ja3QJg1dlnPtlLXV40SS5j4d2d4btVRw8H8CwApAOMwsZap4 /FAlR8BqMq/FcGL1FPJumQ0PWYQOuwC8vr1cyY5P8VeY3Tl3BR1TBScKer2saE/Hgoqs U9XA== X-Gm-Message-State: AOJu0YwqBM74zOGcNhcpyXEDUjRv+5AMslnU5gjAsVqL57Bat89nR7WK bk7bJSxlZGJABGFKztmmgfOojAxSaSvssVHwcNj2rij+J46Oex7nBN/V3UWB8zc= X-Google-Smtp-Source: AGHT+IFqeFMlXAWojOkR77YthzMiQd12EzWzKKR5jF9mKoRG/M7Sy2DiK8YIZP4w+Wjq8+CK05flDA== X-Received: by 2002:a05:6808:1b22:b0:3d5:5fbe:b2fa with SMTP id 5614622812f47-3db23aa2aa6mr17312900b6e.35.1722436616382; Wed, 31 Jul 2024 07:36:56 -0700 (PDT) Received: from mutt-hbsd (174-24-87-135.clsp.qwest.net. [174.24.87.135]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3db420f8085sm690849b6e.50.2024.07.31.07.36.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jul 2024 07:36:55 -0700 (PDT) Date: Wed, 31 Jul 2024 14:36:54 +0000 From: Shawn Webb To: Alan Somers Cc: FreeBSD Hackers , Warner Losh , Scott Long , meka@tilda.center Subject: Re: The Case for Rust (in the base system) Message-ID: X-Operating-System: FreeBSD mutt-hbsd 15.0-CURRENT-HBSD FreeBSD 15.0-CURRENT-HBSD X-PGP-Key: https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/blob/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="mnp5en7dxtqafvrw" Content-Disposition: inline In-Reply-To: X-Spamd-Bar: ----- X-Spamd-Result: default: False [-5.10 / 15.00]; SIGNED_PGP(-2.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; R_DKIM_ALLOW(-0.20)[hardenedbsd.org:s=google]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; MISSING_XM_UA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::22a:from]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DMARC_NA(0.00)[hardenedbsd.org]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DKIM_TRACE(0.00)[hardenedbsd.org:+] X-Rspamd-Queue-Id: 4WYvn619bHz4jl4 --mnp5en7dxtqafvrw Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jan 20, 2024 at 09:51:25AM -0700, Alan Somers wrote: > In a recent thread on src-committers, we discussed the costs and > benefits of including Rust code in the FreeBSD base system. To > summarize, the cost is that it would double our build times. imp > suggested adding an additional step after buildworld for stuff that > requires an external toolchain. That would ease the build time pain. > The benefit is that some tools would become easier to write, or even > become possible. Here is a list of actual and potential Rust projects > that could benefit from being in-tree. If anybody else has items to > add, I suggest moving this into the project wiki: >=20 > Stuff that could only be written in Rust if it were in base > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > * ctl-exporter (I started this, but discovered that the CTL stats API is > unstable, so it can't live in ports. Instead, I had to do it in C). > https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a864= 01469181a67ec34 >=20 > * fusefs tests. Absolutely impossible to do in C. I considered Rust, bu= t went > with C++ so they could live in base. They are too closely coupled to > fusefs(5) to live out-of-tree. > https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs >=20 > * devd. Currently C++, but imp suggested a rewrite. > https://github.com/freebsd/freebsd-src/tree/main/sbin/devd >=20 > * zfsd. Currently C++, but I've long pondered a rewrite. Using Rust wou= ld > make it more testable. > https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd >=20 > * nscd. Currently C, but confusing and with no test coverage. I've > contemplated a rewrite myself, but I don't want to do it in C. > https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd >=20 > * The userland portion of the 802.11ac and Lightning stacks. scottl sugg= ested > that these were good candidates for Rust. >=20 > * freebsd-kpi-r14-0 . https://crates.io/crates/freebsd-kpi-r14-0 >=20 > Stuff that can live in ports, but would be nicer in base > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D >=20 > * gstat-rs https://crates.io/crates/gstat >=20 > * geom-exporter (I've started this, but haven't published it) >=20 > * nfs-exporter https://crates.io/crates/freebsd-nfs-exporter >=20 > * virtiofsd-rs . Nobody has yet tried to port it to FreeBSD. But if the > connection to bhyve(8) is too intimate, it might be hard to do in ports. > https://gitlab.com/virtio-fs/virtiofsd >=20 > * jail-exporter https://crates.io/crates/jail_exporter >=20 > * Various jail managers have been attempted in Rust. I think these are f= ine in > ports, but others like Goran Mekic have opined that they should be move= d to > base instead. >=20 > * musikid's pjdfstest rewrite. I think it would be great to start using = this > to test the base system's file systems. If the tests themselves lived = in > base, they would be easier to sync with file system development. > https://github.com/musikid/pjdfstest >=20 > * pf-rs. I suspect that the API isn't very stable. > https://crates.io/crates/pf-rs >=20 > * benchpmc. The pmc counter names changes between releases. > https://crates.io/crates/benchpmc >=20 > FreeBSD-related applications that are just fine in ports > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D >=20 > * fsx-rs. Unlike pjdfstest, this only tests datapath APIs. Those are us= ually > more stable than control path APIs, so I think there's little to be gai= ned by > moving this into base. https://crates.io/crates/fsx >=20 > * ztop. It uses ZFS's kstats sysctl interface, which is pretty stable. > https://crates.io/crates/ztop >=20 > * iocage-provision https://crates.io/crates/iocage-provision >=20 > * rsblk https://crates.io/crates/rsblk >=20 > * xfuse https://github.com/KhaledEmaraDev/xfuse >=20 > Other FreeBSD-related libraries in Rust > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > Just see the list at https://crates.io/keywords/freebsd >=20 One new data point: DARPA is looking to rewrite a significant amount of C code to Rust with their "Translating All C to Rust (TRACTOR)" project: https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A= 4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc --mnp5en7dxtqafvrw Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAmaqS/8ACgkQ/y5nonf4 4frHTQ/8DtRB+Lil25TF9JWKhFKchRyUuHGik05ju2w/Ve5ovX1lUNlmbu2PoVCN IY36s+YP49iwpTzrEu77QhHzfrSKJBy+D1m2688gnBv35T6f9oReRZwn9Px1j9EM l1ZZklK/YPZNIjrqghUA6Wk4mqHjWrb2RQImGiVXUfYEW/NhSN8N+rmqoddegcHc CFeuGYg0850q7OzAVumd7q4/vTiRrvP49ec6aJICMFf7XHiB3uR3FiXSmaThsKVi oKD2USR4uq9RHL7rpZZ2+mspQ+5L8+bwxVVyroIasvWYPXwITv3uRPLrYQ/jsaO0 b3K3jOjB8pZ+9VFyZBFrrAm4O/X2TFiIk3+hMaDnTv9WgkfsxEVPn4d+lMR++QIQ YvHOBv1kOsgi9Fsh6NlwXsZmLiLoZ9zTwrovO/4tHQnulVK+tmKI0RTbLoGG3Vtu lwRHgi9t8P4dCEXWMcYigbUIIR6M3JdUABQ5Ky2dQx/EWKBS/4K0ux9G6XWwT00e vkC1Y1uAUFzOQGhzQlpcADsmpS+3dC3ji3/wBgGS3UDURErkKHixEmcFGr9Zzl+j s6mtDJ5pu1HI5QU2PNCGDIF4ikrninGZg3csMArnMpoH00kGL/XtBFQSf6C1R33X eZrw55rdH1zF7ie5Ud+jpVmjU7/HG1a4x4PrLa1qei5zx+hbG98= =bsZA -----END PGP SIGNATURE----- --mnp5en7dxtqafvrw-- From nobody Wed Jul 31 15:40:31 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WYxBh5DP9z5S5nW for ; Wed, 31 Jul 2024 15:40:44 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ua1-f45.google.com (mail-ua1-f45.google.com [209.85.222.45]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WYxBh3KwDz3wtM; Wed, 31 Jul 2024 15:40:44 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ua1-f45.google.com with SMTP id a1e0cc1a2514c-8223f0614b6so1604872241.2; Wed, 31 Jul 2024 08:40:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722440443; x=1723045243; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=w5ZVUkhhJHEO3QIftB/BaFGIz6i4F6Y0KvxSE98eKmE=; b=YIm3wqVODzlW7aEpqQk1yGt7KJJb9ktcDwhM3clEvb36cPFS1K1Bz+iAi9zH+9rGLU C5ZDXfkvPk0OhJRRWyudWRZQK5EGDrKpxC1l6DV3jjPfqZonHYARcgeppcZA94OFRLu+ I7C3SQFlqtZp6Ui0NHAqlRHgRApvZ6l2KEu153ma0k1ZUaFJfHszzYw5BA2QZ2hN+1kk mp23/stP8PQGZx1z5KaQ+QRcn37FAZt6kuZTpk47HncSP+iriP9FTHJzbeyyIgmKkBcJ 1u3aiCsmcfUygNZnbg2gvT1rWyEqOysPHN7qhd1t9ZRot+bzV08XE5fpl5Z+ReTdZDWf HTeQ== X-Forwarded-Encrypted: i=1; AJvYcCXvZLQbtiEdNTg1UxbcLPugwW9sYFr5AqMex7I+Z96zo8YY6f8bphL1f/SOixlKl+AklfdTf6c=@freebsd.org X-Gm-Message-State: AOJu0YxI9NAvipMWslV9bfIymsmj3BdBTbwPst+eqhDC57VRXRgGAWWH VuEBSDNs1cqH7sm1A4oOhrrOy3os+NWVUfRSqNqiZpyix5SUDqvc0UXlaZA6vBy2jIYFDi7KOk8 YZ+Jzu9ngi/symSzp2b/q2sYi8Tc= X-Google-Smtp-Source: AGHT+IGBqiVCP7PNXB55UNDJ6GyT0FaqGAxMFaiUlrbJhh/ZwSkgeOWgX1NnRE9GJG49mOnqiSGQnevKwIL8YTV6hb8= X-Received: by 2002:a05:6102:3708:b0:492:a11f:a878 with SMTP id ada2fe7eead31-493fad1fbaemr17533495137.23.1722440443147; Wed, 31 Jul 2024 08:40:43 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Wed, 31 Jul 2024 09:40:31 -0600 Message-ID: Subject: Re: The Case for Rust (in the base system) To: Shawn Webb Cc: FreeBSD Hackers , Warner Losh , Scott Long , meka@tilda.center Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4WYxBh3KwDz3wtM On Wed, Jul 31, 2024 at 8:37=E2=80=AFAM Shawn Webb wrote: > > On Sat, Jan 20, 2024 at 09:51:25AM -0700, Alan Somers wrote: > > In a recent thread on src-committers, we discussed the costs and > > benefits of including Rust code in the FreeBSD base system. To > > summarize, the cost is that it would double our build times. imp > > suggested adding an additional step after buildworld for stuff that > > requires an external toolchain. That would ease the build time pain. > > The benefit is that some tools would become easier to write, or even > > become possible. Here is a list of actual and potential Rust projects > > that could benefit from being in-tree. If anybody else has items to > > add, I suggest moving this into the project wiki: > > > > Stuff that could only be written in Rust if it were in base > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > * ctl-exporter (I started this, but discovered that the CTL stats API i= s > > unstable, so it can't live in ports. Instead, I had to do it in C). > > https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a8= 6401469181a67ec34 > > > > * fusefs tests. Absolutely impossible to do in C. I considered Rust, = but went > > with C++ so they could live in base. They are too closely coupled to > > fusefs(5) to live out-of-tree. > > https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs > > > > * devd. Currently C++, but imp suggested a rewrite. > > https://github.com/freebsd/freebsd-src/tree/main/sbin/devd > > > > * zfsd. Currently C++, but I've long pondered a rewrite. Using Rust w= ould > > make it more testable. > > https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd > > > > * nscd. Currently C, but confusing and with no test coverage. I've > > contemplated a rewrite myself, but I don't want to do it in C. > > https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd > > > > * The userland portion of the 802.11ac and Lightning stacks. scottl su= ggested > > that these were good candidates for Rust. > > > > * freebsd-kpi-r14-0 . https://crates.io/crates/freebsd-kpi-r14-0 > > > > Stuff that can live in ports, but would be nicer in base > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D > > > > * gstat-rs https://crates.io/crates/gstat > > > > * geom-exporter (I've started this, but haven't published it) > > > > * nfs-exporter https://crates.io/crates/freebsd-nfs-exporter > > > > * virtiofsd-rs . Nobody has yet tried to port it to FreeBSD. But if t= he > > connection to bhyve(8) is too intimate, it might be hard to do in por= ts. > > https://gitlab.com/virtio-fs/virtiofsd > > > > * jail-exporter https://crates.io/crates/jail_exporter > > > > * Various jail managers have been attempted in Rust. I think these are= fine in > > ports, but others like Goran Mekic have opined that they should be mo= ved to > > base instead. > > > > * musikid's pjdfstest rewrite. I think it would be great to start usin= g this > > to test the base system's file systems. If the tests themselves live= d in > > base, they would be easier to sync with file system development. > > https://github.com/musikid/pjdfstest > > > > * pf-rs. I suspect that the API isn't very stable. > > https://crates.io/crates/pf-rs > > > > * benchpmc. The pmc counter names changes between releases. > > https://crates.io/crates/benchpmc > > > > FreeBSD-related applications that are just fine in ports > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > * fsx-rs. Unlike pjdfstest, this only tests datapath APIs. Those are = usually > > more stable than control path APIs, so I think there's little to be g= ained by > > moving this into base. https://crates.io/crates/fsx > > > > * ztop. It uses ZFS's kstats sysctl interface, which is pretty stable. > > https://crates.io/crates/ztop > > > > * iocage-provision https://crates.io/crates/iocage-provision > > > > * rsblk https://crates.io/crates/rsblk > > > > * xfuse https://github.com/KhaledEmaraDev/xfuse > > > > Other FreeBSD-related libraries in Rust > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > Just see the list at https://crates.io/keywords/freebsd > > > > One new data point: DARPA is looking to rewrite a significant amount > of C code to Rust with their "Translating All C to Rust (TRACTOR)" > project: > https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view Interesting. And since you bring it up, I have two new data points myself: * ctld: while working on some bugs in ctld, I had trouble understanding the config file parsing. So I rewrote that part in Rust, just to help my understanding. Later, I rewrote the XML parsing, too. Then I rewrote the LUN creation and deletion, just to see how hard it would be. All of those parts take about 5x fewer SLOC in Rust than in C, and they're less buggy, too. Config file parsing is more consistent, no memory leaks, etc. Alas, I'm not planning to finish this project, since the base system doesn't allow Rust and ctld is too tightly coupled to ctl to live in ports. * geom-exporter: I mentioned this project up above. It's finished now, and you can find it in ports at net-mgmt/geom-exporter . From nobody Wed Jul 31 15:46:27 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WYxKW5rbCz5S6Kc for ; Wed, 31 Jul 2024 15:46:39 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WYxKW4zzYz40QL; Wed, 31 Jul 2024 15:46:39 +0000 (UTC) (envelope-from theraven@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1722440799; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=PfpzM2k237HHnjXJDVKcAula2Nck2bewkpHICsHiegM=; b=G831ol982h6DcGQowlMVAzOyU/rIJ7dYc+Gnx4ybhTaLMlinVL1K3fgsER8eoONvkg7nSP fVUp/R9UjOO+aM2up9wmw2wVUrPjl8dJb7j51SdtSfQIfk+o606AFiCbY5epbOicrJ+Ps0 LQXPmjuiuwCDZIuCU/tHg2r31jNdkj3gw74tJG9xCNc3QEKkbf/efT6pzo0ZHVa9PsYibU VC1y1V3SJPO8Bz6DltgnT1cjIIPPjPltfrN8A9wI/pHxIJFyWwnt0R4aqwdgW4HvYXv5Fs khctq0FLBUxmniEIrfgNLQ4LNe6dfHBWRcPlCVmOh7vYw44GGZyUzooPWaI6dw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1722440799; a=rsa-sha256; cv=none; b=r3zPIP/mWEa+pGgyHFqttPYcHt7wnUZRXCXO6zOrB2oFAGI9opC2ZJxshx6HzwRl4jaJpq CcAiIkxzOfZ7Dp/AE+D4RgXZyCw63RM4detwWrTbG3qXE02GKbBUMvM+nzKy+llTR87w2G d2OVKYD9O6jr8QTys2CwiRheIl5UupyQsV0FzNLR4aGlEq97yN2oHT10l5ySsGotpVdVLW LSKImNBkfF1e5A+NNd63QlBZT69VTZnAio8xtqXsNfXO6Ta6kIkOuGEz9QzeOtPePGbyKs GPpt2z2nD1yXn8QhGq5LsX1ZbOBJ86nY9kOt1cg33riIDAQbJR1ry6bxoSN5sQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1722440799; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=PfpzM2k237HHnjXJDVKcAula2Nck2bewkpHICsHiegM=; b=F95wO0GUEbUUUBAo7ucI9covd6UUWiWmsVvQpY3KfadRuibGEIEWS52cBiStXpa4uO8zIR kG6NzoIXPOnhHxXBbdeIli0BR4CmHtNhiqe0wsN4Yd/C9gZUH9G7EBV/Zcwk9MnVuGnjpS tzO/exFmklB47c+CwLj/3pb4zErO+Rq3oimaG07OSpZrwcoSdi12oAiT+gyc4q2y3UYn7u +2hCAzjqTAwLSS66g9vnLe177J8LEH6DUmVegugFZTuOnQ+4C39aRA1PgHTUzg4R72rcsp +85qlNdnYdDBHFUD0ehWtwmHU3ifPxEBwdrhC0SDxZ9V1Of4GmoHB8pUJl/MVw== Received: from smtp.theravensnest.org (smtp.theravensnest.org [45.77.103.195]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: theraven) by smtp.freebsd.org (Postfix) with ESMTPSA id 4WYxKW4Qszz1CDV; Wed, 31 Jul 2024 15:46:39 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from smtpclient.apple (host86-138-165-11.range86-138.btcentralplus.com [86.138.165.11]) by smtp.theravensnest.org (Postfix) with ESMTPSA id 65F804737; Wed, 31 Jul 2024 16:46:38 +0100 (BST) From: David Chisnall Message-Id: <1D235928-2513-45C1-8439-C2A2B9C58E46@FreeBSD.org> Content-Type: multipart/alternative; boundary="Apple-Mail=_80CE49DB-FD58-4421-9220-3682165573DE" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: The Case for Rust (in the base system) Date: Wed, 31 Jul 2024 16:46:27 +0100 In-Reply-To: Cc: Shawn Webb , FreeBSD Hackers , Warner Losh , Scott Long , meka@tilda.center To: Alan Somers References: X-Mailer: Apple Mail (2.3774.600.62) --Apple-Mail=_80CE49DB-FD58-4421-9220-3682165573DE Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 On 31 Jul 2024, at 16:40, Alan Somers wrote: >=20 > * ctld: while working on some bugs in ctld, I had trouble > understanding the config file parsing. So I rewrote that part in > Rust, just to help my understanding. Later, I rewrote the XML > parsing, too. Then I rewrote the LUN creation and deletion, just to > see how hard it would be. All of those parts take about 5x fewer SLOC > in Rust than in C, and they're less buggy, too. Config file parsing > is more consistent, no memory leaks, etc. Alas, I'm not planning to > finish this project, since the base system doesn't allow Rust and ctld > is too tightly coupled to ctl to live in ports. C is absolutely terrible for parsing on any metric (even C++ lets you = write parsers in a fraction of the code and fewer bugs). It=E2=80=99s = one of the places where Rust provides some very easy wins: - Lifetimes are easy to reason about in parsers, they fit well into = Rust=E2=80=99s ownership model because the input is a stream and the = output is a tree. - Parsers, by definition, are part of your attack surface because = they=E2=80=99re taking data from outside. Replacing parsers with Rust (or something like EverParse) has a very = high security return relative to the investment of effort. David --Apple-Mail=_80CE49DB-FD58-4421-9220-3682165573DE Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 On 31 Jul = 2024, at 16:40, Alan Somers <asomers@FreeBSD.org> = wrote:

* ctld: while working on = some bugs in ctld, I had trouble
understanding the config file parsing.  So I rewrote = that part in
Rust, = just to help my understanding.  Later, I rewrote the XML
parsing, too.  Then I rewrote the LUN creation and = deletion, just to
see how = hard it would be.  All of those parts take about 5x fewer = SLOC
in Rust = than in C, and they're less buggy, too.  Config file = parsing
is more = consistent, no memory leaks, etc.  Alas, I'm not planning = to
finish = this project, since the base system doesn't allow Rust and = ctld
is too = tightly coupled to ctl to live in ports.

C is absolutely terrible for = parsing on any metric (even C++ lets you write parsers in a fraction of = the code and fewer bugs).  It=E2=80=99s one of the places where = Rust provides some very easy wins:

 - = Lifetimes are easy to reason about in parsers, they fit well into = Rust=E2=80=99s ownership model because the input is a stream and the = output is a tree.
 - Parsers, by definition, are part of = your attack surface because they=E2=80=99re taking data from = outside.

Replacing parsers with Rust (or = something like EverParse) has a very high security return relative to = the investment of = effort.

David

= --Apple-Mail=_80CE49DB-FD58-4421-9220-3682165573DE-- From nobody Wed Jul 31 16:52:47 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WYyp344xgz5SCfj for ; Wed, 31 Jul 2024 16:52:59 +0000 (UTC) (envelope-from lain@fair.moe) Received: from mail.076.ne.jp (mail.076.ne.jp [45.76.218.69]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WYyp11ww4z47xw for ; Wed, 31 Jul 2024 16:52:57 +0000 (UTC) (envelope-from lain@fair.moe) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=076.ne.jp header.s=dkim header.b=mw+sZMKU; dmarc=none; spf=none (mx1.freebsd.org: domain of lain@fair.moe has no SPF policy when checking 45.76.218.69) smtp.mailfrom=lain@fair.moe Received: from mail.076.ne.jp (localhost [127.0.0.1]) by mail.076.ne.jp (Postfix) with ESMTP id 4WYynr37fTzW0vw for ; Thu, 1 Aug 2024 01:52:48 +0900 (JST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=076.ne.jp; h= user-agent:in-reply-to:content-disposition:content-type :mime-version:references:message-id:subject:to:from:date; s= dkim; t=1722444767; x=1725036768; bh=gRJIbcQjawPx+zcOe6GioObOOjc hi9xwZZ75YAr/ZNk=; b=mw+sZMKUgTRjY9hgLhNW77FNMEuAoctgdvsh5NA3GMp sae1gpPRWdG+wr52kHFHoFop+Gp/4KeoJZWuscQkTQOLyShb8ev8VO/Jn+O5K/yr FhT4VonPr8t0dA9fhO9hFfU/5kEQeE7L3qVFISluNieeZiR92nDd67RlDqWf1q3l vC/2ixOU/aceO42/jE99XTpWkCYkiPUD9n/vH+ok24L4VJypKsleWpbt1RhAYmt1 uBP2K1ZP5LW1ufXjK8xLJ4Aj5t08bRyZLoHAcZ+A3MsoGigLFYSC5hX9fcgQMGex VLZQHSlOlw2G7GKDN9AHTa+nnyKIDj121sRBN/lhPXA== X-Virus-Scanned: Debian amavisd-new at guest.guest Received: from mail.076.ne.jp ([127.0.0.1]) by mail.076.ne.jp (mail.076.ne.jp [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id HP2xX4MrN--6 for ; Thu, 1 Aug 2024 01:52:47 +0900 (JST) Received: from mail.fair.moe (124.110.12.171.ap.gmobb-fix.jp [124.110.12.171]) by mail.076.ne.jp (Postfix) with ESMTPSA id 4WYynq46tfzW0bn for ; Thu, 1 Aug 2024 01:52:47 +0900 (JST) Date: Thu, 1 Aug 2024 01:52:47 +0900 From: "lain." To: freebsd-hackers@freebsd.org Subject: Re: The Case for Rust (in the base system) Message-ID: X-Location: =?utf-8?B?IkVhcnRoL+WcsOeQgyI=?= X-Operating-System: "OpenBSD" References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="ktm3zy4hkbihhatn" Content-Disposition: inline In-Reply-To: Autocrypt: addr=(null); prefer-encrypt=mutual; keydata= mQGNBGHVL30BDAC2g9WjfETVgMHoWqAQdNDzTFEnuIginZzZdx5CpkBwAeQZexW5Eo/9C1anl4F e0d3KeOFfLMBlaTohsfgvlNyOE8iVyWi5b4Op4cvSfUVm7vQvm+axVVDjXA0o6H4cOWp4etxKfb lD8/kO7WbvMxGeu2IyENoUXZR6/mr1Y6TOWkouTzbWFB1vOxMn68UMouuk4fYf6K3E7KavUMUqu ME3nqFjlKtyQBQmvpe4SnPVnjlOgIHTz9ffGV4l07j3QeCetR2h+CgOFgb1SNLdgxDCuDAdh0iF fGrPQP4jA7BNOcNBdUrkzuNAXDK4H8GB+Z3Pxc2+7jmtPWMPvmpCaZw2jPw2gaey3HPbhxvM1jX gPqLNnr3hKEFGkGPcXwpuxU9saoLOOArLACOkIy9G3GtaU26IJYLCMSi2N8M873oyFOShtcarNY lqetrJ/37tPxdVlOizE0ZB6VD+v6iFpUeHh9aGAiaTIYjM/tfMVUBHjtoQUrJfR8ONxdd1OuHZr e8AEQEAAbQh56We5bGx5byY5piOIDxrYW5heWFtYUAwNzYubmUuanA+iQHOBBMBCAA4AhsDBQsJ CAcCBhUKCQgLAgQWAgMBAh4BAheAFiEEXqMTFPwNvHspnw/iU5MGhC0WaM0FAmVQVRoACgkQU5M GhC0WaM0atAv7B/9RsNYsjeCbwDSYqN41A7qipucfxdPpabDeqQgeCYuFPzAaCKl1u+HpF2bb9/ euJlml2pOj4jOKbPr0XyzfDDJEQgHCM7H+mg7CgdpWKeMbOodIgJ4I/rg/a6cVUXZijtrCOXURM eAObcvQRTNTtNqePDgcJ20/3vEB8TPJLbO7Va0YQ0qH9gC36sgmQhksG5UMEpBlStnrY9St+9VX wSB2J3exLt6T2SCCmPM0IG3kAgbtSCzT08MfCHe3s3PL4fqW2SuJeW8p17EzrUFuOhTuQs1oOmJ +2vww4w+1TL5Km45KdxosvzcEzLOh0g+9ml7O2fnVtR3jvSzijFJJCcrn5+Di1EyjrhinEcQQ1+ Oh270wW/NtwWSBQ/peZb2iiuD7Ry/6zgYlLuASCoSXkOZWH76+zRDIEEx4XX3B0bey8qPgyEvXo OFIt9CHCOGwbDVKx0LgXxzYx6JzvJB4xlrQ9zMMmet2GIT6JUIALA25P9jgRep4elNMH9SxiYk0 uQGNBGHVL30BDACdui3F1uwOwgZ8y0zsL2c3Qw6kFVW4sp2ql7w9hz3IbSO7kTrRUqvZ7Wn97AB LRr4piml02S/ljtdlU4P3Hq1hrnRwReG3AWQjJByhZms4VU3QWauUbv/pZti5vuuB7BEmP2txPr npfBBJW5DdPnebc+BecnhsJRE7jegt8XpDWixxyAwmDdmz0hhjL/dYgGzAfx3RD3SZ5c0KhqEHX 5oTOND8/ncInK7hWB1zBq3oaPB2sfzDiUL5eBk+SPvsSoz8rBBsGGnrBX/BIGTIzQ6nB1AIqeze Rcz4R0j/g67/2yz2puwYzr+3QjjfkK4p4ZYuG7nd41CQUWxy3lgUz9kCnxWcR50AHAQhhQGPZKy hicGU1JyJUZMxqyTslSkPa6ziiC2FCOJ77hZV2Ow+4y9usWkTo6Xuce3gfAGV6xgDLarl/P2hN+ DCIV4INlBKj70WaQg2ZlaKatGgVcCrbY5X/PbI9nEFMVOpjo6nXXhf/WI1mRH3lXQJGuiawF8Lm PkAEQEAAYkBtgQYAQgAIAIbDBYhBF6jExT8Dbx7KZ8P4lOTBoQtFmjNBQJlUFUiAAoJEFOTBoQt FmjNfbsL/2jYau5JOYIE0+qjeXe/skuUJ6pRrthXGI/ap7id/XVi7P9IZSDrVEsetNzBvR+9fiu pP1nwAaNS9MaNTb7dwdKutRjrj/X2kFj1HCMJJPJIfmQVdrCaA7AnrBMx4YgA2eAg899LN4v/j5 Y1ljoBxxxJ7OVw30uGCysiMgfQKNFKKRiKMqcfyzF2SImhTO0xBvkjamTmupY0MZdgoK0LDI0bQ dTDOsQJa9D9d25DnH8oCNttapFx9NhVA3+1TG9bJF1JukRyYuHyn7m9GP21hpBjBbvgNtLsZT5a 772XAem0Ro5qLT1BUv+R1B+EtffjKYp8Rhy4VBuSUx3e8ELOdIe+ok1XhrnA5xeMVlPwEADO4jp R09BcYQzA6Fjjo9/yGx1n0TEeYBHfLCggBZlgC66J1XNDIjWc2rNiLUCZh/kZAmlGbG2+3tNFlR DgmMeOKxwPO73VbuJbcMwx7sBNu9TzH2DMVLg8OHzWD6KNg8pYrwVugk1xNjvOroSLqN96uA== User-Agent: NeoMutt/20240201 X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.89 / 15.00]; SIGNED_PGP(-2.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.991]; MID_RHS_NOT_FQDN(0.50)[]; R_DKIM_ALLOW(-0.20)[076.ne.jp:s=dkim]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; DMARC_NA(0.00)[fair.moe]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:20473, ipnet:45.76.192.0/19, country:US]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+,1:+,2:~]; R_SPF_NA(0.00)[no SPF record]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[076.ne.jp:+] X-Rspamd-Queue-Id: 4WYyp11ww4z47xw --ktm3zy4hkbihhatn Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2024=E5=B9=B407=E6=9C=8831=E6=97=A5 14:36, the silly Shawn Webb claimed = to have said: > One new data point: DARPA is looking to rewrite a significant amount > of C code to Rust with their "Translating All C to Rust (TRACTOR)" > project: > https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view Seeing a government endorse Rust over C makes me endorse C over Rust even more than I already did. --=20 lain. PGP public key: https://fair.moe/lain.asc --ktm3zy4hkbihhatn Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEABYKAB0WIQRjt7VGft6AEgroX7Nt5cCzjCJFhwUCZqpr2wAKCRBt5cCzjCJF h8vXAP9FrKXU46qQnioUnKex8MedHvnTD8XAFcGmiksxGRvdwwD+MtznV9IqbxGC ZaNF3l8iQw9P9z1TbsA1tOaC8JylvgY= =y4EB -----END PGP SIGNATURE----- --ktm3zy4hkbihhatn-- From nobody Wed Jul 31 17:01:17 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WYz1k0TC8z5SDW8 for ; Wed, 31 Jul 2024 17:03:06 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WYz1j5mVbz4FVM for ; Wed, 31 Jul 2024 17:03:05 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x1032.google.com with SMTP id 98e67ed59e1d1-2cb4c4de4cbso3962127a91.1 for ; Wed, 31 Jul 2024 10:03:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1722445384; x=1723050184; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=inTcaeSwBnSI8SKQHYaKN61i1wOqZffoACUbgd4Y+ME=; b=Nhn/SCdzGSdzPkanq9F991ZAh6TLmKqLv2mBV/y3yyQ+SyZC4XtuOEY7P30Us3vlpG QMv6hBbGTSJ3K/Az8ioNObgVyZXejXnzPaU3yPOOUwrjybrL9dpRptGJ/9T8vwJZV9wu 5pN2qHv8ws+LvOzTKRfaj83hpvWyLYR1hE+EjJL9J7yNm8WU4arJaUXhYlmq0tKXa8EH dyh/rGPSTD3IwqayCbbYPgpAWKCzOuIoytBEMsM0dce/mt6n7AOaOLUXicpzy56wpoTK L/lH9cHf/r4V1e9DE6T7iC9+5I8qRVtiPU5iq1DHSO1mCDPrpZrMRPhvSxXryAX4ptUN yhdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722445384; x=1723050184; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=inTcaeSwBnSI8SKQHYaKN61i1wOqZffoACUbgd4Y+ME=; b=KdngEVdRQHwDsTNowW1M1zEK0o8Xri8V6tMmUsb/K48Zye/4O8FMiMZOOZlA70HU/T ZERHG45pQFU1ZLyWVsMdkEgD7a0rJ06A3Elu1gNyaHGjM2LwddsNPR7eIMKUKVeXxnBt K9iY1waZXK0gqjaG5A/buBRaOAaJGRrsWCGTyk7RdG1FvHS0TQtzOtAKwaX1fchAb5e7 7dzErLvJu/g6156y6T8Yo+zvfsfATKt0bxax5cekCR1DOReXB5sQ9PjZwL3dY0fLMk2t Px4qjOhai+sdm47/G0BZZdBeFPByqVGfBbZc83Oi11enDfPz8doRXnTiKUvTUP3DuESL e+Lw== X-Forwarded-Encrypted: i=1; AJvYcCWZ/9q+a7PIpU8s3tQ7b35zZgVj4hF4h+RC1/ZUEFyVu3DvRl/izmyYUMD8/1YsAYg04efzjAAj9EwegVgjDhKowHFpcxCvHuxLbSE= X-Gm-Message-State: AOJu0YwVpcGgPw7AVDT2vS0pOeUH6sSJ6SQcR7itrlmDVUAUnJV1QHQn FBLiBu4uf46hV3sB3X1mb+PAkVsTT/q87Cy5CXQlXT0j/L5Z9O7ZNCVAQI6bVhgGPiLyQ/ETGqU voMGkGkK8iuMbYUnt/4qiRVOvVtfCXEGQo7GotLCtXwo2DZMH X-Google-Smtp-Source: AGHT+IETHIow8ZE20L0IY7SLDTy3Qlk7huf4Ea6LC0dh49rd1JqKoEUP+a45LK/BmrPbICot3XhqEnrqTgP9eB+OAoo= X-Received: by 2002:a17:90a:b397:b0:2c7:e46e:f8b7 with SMTP id 98e67ed59e1d1-2cf7e0976admr14319938a91.4.1722445384101; Wed, 31 Jul 2024 10:03:04 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Wed, 31 Jul 2024 11:01:17 -0600 Message-ID: Subject: Re: The Case for Rust (in the base system) To: Alan Somers Cc: Shawn Webb , FreeBSD Hackers , Scott Long , "Goran Meki??" Content-Type: multipart/alternative; boundary="00000000000043da1a061e8e11f5" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4WYz1j5mVbz4FVM --00000000000043da1a061e8e11f5 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, Jul 31, 2024, 9:40=E2=80=AFAM Alan Somers wro= te: > On Wed, Jul 31, 2024 at 8:37=E2=80=AFAM Shawn Webb > wrote: > > > > On Sat, Jan 20, 2024 at 09:51:25AM -0700, Alan Somers wrote: > > > In a recent thread on src-committers, we discussed the costs and > > > benefits of including Rust code in the FreeBSD base system. To > > > summarize, the cost is that it would double our build times. imp > > > suggested adding an additional step after buildworld for stuff that > > > requires an external toolchain. That would ease the build time pain. > > > The benefit is that some tools would become easier to write, or even > > > become possible. Here is a list of actual and potential Rust project= s > > > that could benefit from being in-tree. If anybody else has items to > > > add, I suggest moving this into the project wiki: > > > > > > Stuff that could only be written in Rust if it were in base > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > > > * ctl-exporter (I started this, but discovered that the CTL stats API > is > > > unstable, so it can't live in ports. Instead, I had to do it in C)= . > > > > https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a86401= 469181a67ec34 > > > > > > * fusefs tests. Absolutely impossible to do in C. I considered Rust= , > but went > > > with C++ so they could live in base. They are too closely coupled = to > > > fusefs(5) to live out-of-tree. > > > https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusef= s > > > > > > * devd. Currently C++, but imp suggested a rewrite. > > > https://github.com/freebsd/freebsd-src/tree/main/sbin/devd > > > > > > * zfsd. Currently C++, but I've long pondered a rewrite. Using Rust > would > > > make it more testable. > > > https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd > > > > > > * nscd. Currently C, but confusing and with no test coverage. I've > > > contemplated a rewrite myself, but I don't want to do it in C. > > > https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd > > > > > > * The userland portion of the 802.11ac and Lightning stacks. scottl > suggested > > > that these were good candidates for Rust. > > > > > > * freebsd-kpi-r14-0 . https://crates.io/crates/freebsd-kpi-r14-0 > > > > > > Stuff that can live in ports, but would be nicer in base > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D > > > > > > * gstat-rs https://crates.io/crates/gstat > > > > > > * geom-exporter (I've started this, but haven't published it) > > > > > > * nfs-exporter https://crates.io/crates/freebsd-nfs-exporter > > > > > > * virtiofsd-rs . Nobody has yet tried to port it to FreeBSD. But if > the > > > connection to bhyve(8) is too intimate, it might be hard to do in > ports. > > > https://gitlab.com/virtio-fs/virtiofsd > > > > > > * jail-exporter https://crates.io/crates/jail_exporter > > > > > > * Various jail managers have been attempted in Rust. I think these > are fine in > > > ports, but others like Goran Mekic have opined that they should be > moved to > > > base instead. > > > > > > * musikid's pjdfstest rewrite. I think it would be great to start > using this > > > to test the base system's file systems. If the tests themselves > lived in > > > base, they would be easier to sync with file system development. > > > https://github.com/musikid/pjdfstest > > > > > > * pf-rs. I suspect that the API isn't very stable. > > > https://crates.io/crates/pf-rs > > > > > > * benchpmc. The pmc counter names changes between releases. > > > https://crates.io/crates/benchpmc > > > > > > FreeBSD-related applications that are just fine in ports > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > > > * fsx-rs. Unlike pjdfstest, this only tests datapath APIs. Those ar= e > usually > > > more stable than control path APIs, so I think there's little to be > gained by > > > moving this into base. https://crates.io/crates/fsx > > > > > > * ztop. It uses ZFS's kstats sysctl interface, which is pretty stabl= e. > > > https://crates.io/crates/ztop > > > > > > * iocage-provision https://crates.io/crates/iocage-provision > > > > > > * rsblk https://crates.io/crates/rsblk > > > > > > * xfuse https://github.com/KhaledEmaraDev/xfuse > > > > > > Other FreeBSD-related libraries in Rust > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > Just see the list at https://crates.io/keywords/freebsd > > > > > > > One new data point: DARPA is looking to rewrite a significant amount > > of C code to Rust with their "Translating All C to Rust (TRACTOR)" > > project: > > https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view > > Interesting. And since you bring it up, I have two new data points mysel= f: > > * ctld: while working on some bugs in ctld, I had trouble > understanding the config file parsing. So I rewrote that part in > Rust, just to help my understanding. Later, I rewrote the XML > parsing, too. Then I rewrote the LUN creation and deletion, just to > see how hard it would be. All of those parts take about 5x fewer SLOC > in Rust than in C, and they're less buggy, too. Config file parsing > is more consistent, no memory leaks, etc. Alas, I'm not planning to > finish this project, since the base system doesn't allow Rust and ctld > is too tightly coupled to ctl to live in ports. > Cool. Still waiting for anybody to take me up on the offer to do build system integration. Since the Rust advocates can't get even this basic step done for review, it's going to be impossible to have Rust in the base. This isn't even integrate rust compiler like we do with llvm, but with external Rust toolchain. Until somebody steps up for this task, the status quo can't possibly change= . Warner * geom-exporter: I mentioned this project up above. It's finished > now, and you can find it in ports at net-mgmt/geom-exporter . > --00000000000043da1a061e8e11f5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Wed, Jul 31, 2024, 9:40=E2=80=AFAM Alan Somers <= asomers@freebsd.org> wrote:
On Wed, Jul 31, 2024 at 8:37=E2=80= =AFAM Shawn Webb <shawn.webb@hardenedbsd.org> wrote: >
> On Sat, Jan 20, 2024 at 09:51:25AM -0700, Alan Somers wrote:
> > In a recent thread on src-committers, we discussed the costs and<= br> > > benefits of including Rust code in the FreeBSD base system.=C2=A0= To
> > summarize, the cost is that it would double our build times.=C2= =A0 imp
> > suggested adding an additional step after buildworld for stuff th= at
> > requires an external toolchain.=C2=A0 That would ease the build t= ime pain.
> > The benefit is that some tools would become easier to write, or e= ven
> > become possible.=C2=A0 Here is a list of actual and potential Rus= t projects
> > that could benefit from being in-tree.=C2=A0 If anybody else has = items to
> > add, I suggest moving this into the project wiki:
> >
> > Stuff that could only be written in Rust if it were in base
> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> >
> > * ctl-exporter (I started this, but discovered that the CTL stats= API is
> >=C2=A0 =C2=A0unstable, so it can't live in ports.=C2=A0 Instea= d, I had to do it in C).
> >=C2=A0 =C2=A0https://github.com/freebsd/freebsd-src/commit/1a7f22d9c2= 11f504f6c48a86401469181a67ec34
> >
> > * fusefs tests.=C2=A0 Absolutely impossible to do in C.=C2=A0 I c= onsidered Rust, but went
> >=C2=A0 =C2=A0with C++ so they could live in base.=C2=A0 They are t= oo closely coupled to
> >=C2=A0 =C2=A0fusefs(5) to live out-of-tree.
> >=C2=A0 =C2=A0https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs > >
> > * devd.=C2=A0 Currently C++, but imp suggested a rewrite.
> >=C2=A0 =C2=A0https://g= ithub.com/freebsd/freebsd-src/tree/main/sbin/devd
> >
> > * zfsd.=C2=A0 Currently C++, but I've long pondered a rewrite= .=C2=A0 Using Rust would
> >=C2=A0 =C2=A0make it more testable.
> >=C2=A0 =C2=A0= https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd
> >
> > * nscd.=C2=A0 Currently C, but confusing and with no test coverag= e.=C2=A0 I've
> >=C2=A0 =C2=A0contemplated a rewrite myself, but I don't want t= o do it in C.
> >=C2=A0 =C2=A0https= ://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd
> >
> > * The userland portion of the 802.11ac and Lightning stacks.=C2= =A0 scottl suggested
> >=C2=A0 =C2=A0that these were good candidates for Rust.
> >
> > * freebsd-kpi-r14-0 .=C2=A0 https://c= rates.io/crates/freebsd-kpi-r14-0
> >
> > Stuff that can live in ports, but would be nicer in base
> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> >
> > * gstat-rs https://crates.io/crates/gstat
> >
> > * geom-exporter (I've started this, but haven't published= it)
> >
> > * nfs-exporter https://crates.io/c= rates/freebsd-nfs-exporter
> >
> > * virtiofsd-rs .=C2=A0 Nobody has yet tried to port it to FreeBSD= .=C2=A0 But if the
> >=C2=A0 =C2=A0connection to bhyve(8) is too intimate, it might be h= ard to do in ports.
> >=C2=A0 =C2=A0https://gitlab.com/virtio-fs/= virtiofsd
> >
> > * jail-exporter https://crates.io/crates/= jail_exporter
> >
> > * Various jail managers have been attempted in Rust.=C2=A0 I thin= k these are fine in
> >=C2=A0 =C2=A0ports, but others like Goran Mekic have opined that t= hey should be moved to
> >=C2=A0 =C2=A0base instead.
> >
> > * musikid's pjdfstest rewrite.=C2=A0 I think it would be grea= t to start using this
> >=C2=A0 =C2=A0to test the base system's file systems.=C2=A0 If = the tests themselves lived in
> >=C2=A0 =C2=A0base, they would be easier to sync with file system d= evelopment.
> >=C2=A0 =C2=A0https://github.com/musikid/pjd= fstest
> >
> > * pf-rs.=C2=A0 I suspect that the API isn't very stable.
> >=C2=A0 =C2=A0https://crates.io/crates/pf-rs > >
> > * benchpmc.=C2=A0 The pmc counter names changes between releases.=
> >=C2=A0 =C2=A0https://crates.io/crates/benchpmc<= /a>
> >
> > FreeBSD-related applications that are just fine in ports
> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> >
> > * fsx-rs.=C2=A0 Unlike pjdfstest, this only tests datapath APIs.= =C2=A0 Those are usually
> >=C2=A0 =C2=A0more stable than control path APIs, so I think there&= #39;s little to be gained by
> >=C2=A0 =C2=A0moving this into base. https://crates.i= o/crates/fsx
> >
> > * ztop.=C2=A0 It uses ZFS's kstats sysctl interface, which is= pretty stable.
> >=C2=A0 =C2=A0https://crates.io/crates/ztop
> >
> > * iocage-provision=C2=A0 https://crate= s.io/crates/iocage-provision
> >
> > * rsblk https://crates.io/crates/rsblk
> >
> > * xfuse=C2=A0 https://github.com/KhaledE= maraDev/xfuse
> >
> > Other FreeBSD-related libraries in Rust
> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> > Just see the list at https://crates.io/keywor= ds/freebsd
> >
>
> One new data point: DARPA is looking to rewrite a significant amount > of C code to Rust with their "Translating All C to Rust (TRACTOR)= "
> project:
> https://sam.gov/opp/1e45d64= 8886b4e9ca91890285af77eb7/view

Interesting.=C2=A0 And since you bring it up, I have two new data points my= self:

* ctld: while working on some bugs in ctld, I had trouble
understanding the config file parsing.=C2=A0 So I rewrote that part in
Rust, just to help my understanding.=C2=A0 Later, I rewrote the XML
parsing, too.=C2=A0 Then I rewrote the LUN creation and deletion, just to see how hard it would be.=C2=A0 All of those parts take about 5x fewer SLOC=
in Rust than in C, and they're less buggy, too.=C2=A0 Config file parsi= ng
is more consistent, no memory leaks, etc.=C2=A0 Alas, I'm not planning = to
finish this project, since the base system doesn't allow Rust and ctld<= br> is too tightly coupled to ctl to live in ports.

Cool. Still waiting for anyb= ody to take me up on the offer to do build system integration. Since the Ru= st advocates can't get even this basic step done for review, it's g= oing to be impossible to have Rust in the base. This isn't even integra= te rust compiler like we do with llvm, but with external Rust toolchain.

Until somebody steps up fo= r this task, the status quo can't possibly change.

Warner

* geom-exporter: I mentioned this project up above.=C2=A0 It's finished=
now, and you can find it in ports at net-mgmt/geom-exporter .
--00000000000043da1a061e8e11f5-- From nobody Wed Jul 31 17:49:52 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WZ03z661Bz5SHbM for ; Wed, 31 Jul 2024 17:50:07 +0000 (UTC) (envelope-from bakul@iitbombay.org) Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WZ03y6jFZz4KDS for ; Wed, 31 Jul 2024 17:50:06 +0000 (UTC) (envelope-from bakul@iitbombay.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=iitbombay-org.20230601.gappssmtp.com header.s=20230601 header.b=qeGPwRQO; dmarc=pass (policy=quarantine) header.from=iitbombay.org; spf=pass (mx1.freebsd.org: domain of bakul@iitbombay.org designates 2607:f8b0:4864:20::52c as permitted sender) smtp.mailfrom=bakul@iitbombay.org Received: by mail-pg1-x52c.google.com with SMTP id 41be03b00d2f7-7a130ae7126so4266396a12.0 for ; Wed, 31 Jul 2024 10:50:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iitbombay-org.20230601.gappssmtp.com; s=20230601; t=1722448204; x=1723053004; darn=freebsd.org; h=message-id:in-reply-to:to:references:date:subject:mime-version :content-transfer-encoding:from:from:to:cc:subject:date:message-id :reply-to; bh=sdKDeSLQWr27BjGmZwLIj8vZs3ttLrTTo6V8SNnPc8w=; b=qeGPwRQOnHmiQPbHzfL5IUen/X2lKbVHgoLmQrXqzKn/eDNxpMhdenTB+6XHw9Dp6b 5RDv9Zib9Vh5+q7sRX6MltNrAazlaM1wtzp8WztXQCLpOOv28jyByEY0EM1MnkceBwGu 8zs02hG+4iBYtAlkOSm7OdhJBuonmvKP9Vm+9Y2R45d7/TCRtRjAyKXnpHffXd5aJUHV VQbbe0ZlrrkbGKJRe+Wp7mzsHOoIMYOEk8K88xjN1EvpUvV/+xd+Sb36YrjWkSr+a0Ig WSjedcP5WzfWCsGNVXO0fJ7lOs0cFoSLVwUjfjVvQuz8ASR321DjodxL9Hy8GZlYRFPB ZKtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722448204; x=1723053004; h=message-id:in-reply-to:to:references:date:subject:mime-version :content-transfer-encoding:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sdKDeSLQWr27BjGmZwLIj8vZs3ttLrTTo6V8SNnPc8w=; b=OYMOVKjl+iwWBndn4owJje5ld89R0Po1UJWdX9YjZ95Xvtl9y/V6U+Ft+2g2MpZS5R jc6Lddxu0cug7VpsaHUjE3a9EiPoZ1GsoZD/cPyMKGfxl/NtTS1euUQcaGh4lhIwOpN6 +a5Ze7vBVGbHcfk8GmO5pV+J5e4PhvEHPsaj188tULhMXD/oDBcs+kaYe0kk2g67m+Hv T9d88hsSEXpcBfM8fDztd+fynjnnl77soTKIq+8e9m9mK532zzyLDMpVO/5ZeFKI4nP3 zoP3z03sP1Nvjjb2YzlMY1R/q5Rp4K92XE/ktPofTzbFnCS3Hr39fwDuiVkwGK/pDQDt SNtA== X-Gm-Message-State: AOJu0Yy/CHf+WA4yVbqV2Yu1CBlU5cwgcDJrht/75z75HO5f1AskKrI8 4p0x4lUoMZxVEJQHMcZz1Zyb4UorvjKPCDvzcRli1ESeHT+3q0uXTOa0mTrmP9G5yuewI7/emA8 = X-Google-Smtp-Source: AGHT+IH98frOH1kefbRsEPwy9XNQdYIippsvhjGDzTywIi4AtBBsv6mJIrITD6jYzYJ4NPjc5ryV8w== X-Received: by 2002:a05:6a21:6d91:b0:1c0:f1cb:c4b2 with SMTP id adf61e73a8af0-1c68cec7b21mr271971637.4.1722448204111; Wed, 31 Jul 2024 10:50:04 -0700 (PDT) Received: from smtpclient.apple (107-215-223-229.lightspeed.sntcca.sbcglobal.net. [107.215.223.229]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1fed7f1fb90sm122941565ad.182.2024.07.31.10.50.02 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 31 Jul 2024 10:50:03 -0700 (PDT) From: Bakul Shah Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: The Case for Rust (in the base system) Date: Wed, 31 Jul 2024 10:49:52 -0700 References: To: FreeBSD Hackers In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.3774.600.62) X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.99 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.992]; DMARC_POLICY_ALLOW(-0.50)[iitbombay.org,quarantine]; R_DKIM_ALLOW(-0.20)[iitbombay-org.20230601.gappssmtp.com:s=20230601]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[iitbombay-org.20230601.gappssmtp.com:+]; TO_DN_ALL(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MIME_TRACE(0.00)[0:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEFALL_USER(0.00)[bakul]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::52c:from] X-Rspamd-Queue-Id: 4WZ03y6jFZz4KDS All this talk won't lead to anything much so a suggestion to those who want Rust in the FreeBSD base: why not exec it in a fork? That'll force you to solve practical problems, give you good experience [may be even cure you of this strange desire ;-)] and If it looks like a success, more people will join. From nobody Wed Jul 31 18:51:39 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WZ1R257Xfz5R82l for ; Wed, 31 Jul 2024 18:51:42 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-ot1-x330.google.com (mail-ot1-x330.google.com [IPv6:2607:f8b0:4864:20::330]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WZ1R216JPz4Ptn for ; Wed, 31 Jul 2024 18:51:42 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Authentication-Results: mx1.freebsd.org; none Received: by mail-ot1-x330.google.com with SMTP id 46e09a7af769-709485aca4bso2542957a34.1 for ; Wed, 31 Jul 2024 11:51:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; t=1722451901; x=1723056701; darn=freebsd.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=nZzZx6jVzUCnA0taDbobiwaDjINrS3ymc0NvhAY4LZU=; b=HVzOiNOXxwmnlsZIzR02pi/ArPBh8lOPAUB1jqjn+wnCJt0E+ChEF0IJbZUhxLlWEL 0ZJw7UuquByzh62eWm7z9KIkYkbFlnzU1RXxhjDWuezbA0udTg/8WzW7IeDQ5NF6bwyO RNNkkkZ5Nn6ZFUYp/sdCZQ6cAcT99L2Z1Lc+/re+IvJtX7IK7fV9hbTlQ4IQw2OeRDDm SGToCgIeELP60ASrBZC7LB6EYLT9CERhaEc/5/FYCaMnZv/3UjPeljYa21wUu722lazX No438TzDrbYCuCGVyxi0yP0Vkag//QW5AaAxJqaUCmHXoNbOxL/THRnCIn429QAIeHJR em9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722451901; x=1723056701; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=nZzZx6jVzUCnA0taDbobiwaDjINrS3ymc0NvhAY4LZU=; b=XpjrYk/pr7eVg+KNaoJDUO5w4/n2FCk2f6GmYS9j8MD10GqB4Z6XATSCcE12Zc5gIT pSYDZh0IU2ndvNBPn1wP8bdWnuHeAQZGjxFldJ1OI/HRcu7uCgN+2LLnhHavFwQskR28 p/1FFq0AYbqrD0l2JYAlYZsvFKjrWE6/TXsY2IjmBJP2oeoafQk8qAamWGUbxyPVRWH2 Kuyfbskmalt5A0SKgusjyafCmQZXCbPhfRAhSsYFVdqvnynqjTVLsXvVZQ4CC2RmA2/F 8vPwOY92oY/1XGRpUhve2eSh9u+ViheQ3Yf5i6S7DhwEiiwfbEWEa1Xzv9a38I33ZHdD pZuw== X-Forwarded-Encrypted: i=1; AJvYcCW2wh9LBwUcYRrbkgnOqFO+dYBecvlmd/u+Z2bblaNKJrI/WqhLKchroQGMShf4YrHkck0gyvL44+hEkpYcikRiTnvRDUilv+oSjgw= X-Gm-Message-State: AOJu0Yy0qz/DDShJoUHGT5QKxYzHbLzXNkvPeKzG1YBImJJ5UNe9md5s tYE2gcKq0HWyvw1qinGYDUTLc8ZXXpl32bk4R2/Pb+WOzyEBsarXQrgLzgFYTlA= X-Google-Smtp-Source: AGHT+IEAz0/f3yD5w/czW6go8CJDc6rknfedbtI2Mc/4TwJd1sedY2GkYurCSRWJLbvYE3PNNfvvbg== X-Received: by 2002:a05:6830:2a11:b0:704:2748:5ab6 with SMTP id 46e09a7af769-7096b84669fmr51658a34.20.1722451900756; Wed, 31 Jul 2024 11:51:40 -0700 (PDT) Received: from mutt-hbsd (174-24-87-135.clsp.qwest.net. [174.24.87.135]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7095ac7ba70sm1014852a34.0.2024.07.31.11.51.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jul 2024 11:51:40 -0700 (PDT) Date: Wed, 31 Jul 2024 18:51:39 +0000 From: Shawn Webb To: Warner Losh Cc: Alan Somers , FreeBSD Hackers , Scott Long , Goran Meki?? Subject: Re: The Case for Rust (in the base system) Message-ID: X-Operating-System: FreeBSD mutt-hbsd 15.0-CURRENT-HBSD FreeBSD 15.0-CURRENT-HBSD X-PGP-Key: https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/blob/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="l2ox7s2au2w6utkz" Content-Disposition: inline In-Reply-To: X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4WZ1R216JPz4Ptn --l2ox7s2au2w6utkz Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jul 31, 2024 at 11:01:17AM -0600, Warner Losh wrote: > On Wed, Jul 31, 2024, 9:40=E2=80=AFAM Alan Somers w= rote: >=20 > > On Wed, Jul 31, 2024 at 8:37=E2=80=AFAM Shawn Webb > > wrote: > > > > > > On Sat, Jan 20, 2024 at 09:51:25AM -0700, Alan Somers wrote: > > > > In a recent thread on src-committers, we discussed the costs and > > > > benefits of including Rust code in the FreeBSD base system. To > > > > summarize, the cost is that it would double our build times. imp > > > > suggested adding an additional step after buildworld for stuff that > > > > requires an external toolchain. That would ease the build time pai= n. > > > > The benefit is that some tools would become easier to write, or even > > > > become possible. Here is a list of actual and potential Rust proje= cts > > > > that could benefit from being in-tree. If anybody else has items to > > > > add, I suggest moving this into the project wiki: > > > > > > > > Stuff that could only be written in Rust if it were in base > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > > > > > * ctl-exporter (I started this, but discovered that the CTL stats A= PI > > is > > > > unstable, so it can't live in ports. Instead, I had to do it in = C). > > > > > > https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a864= 01469181a67ec34 > > > > > > > > * fusefs tests. Absolutely impossible to do in C. I considered Ru= st, > > but went > > > > with C++ so they could live in base. They are too closely couple= d to > > > > fusefs(5) to live out-of-tree. > > > > https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fus= efs > > > > > > > > * devd. Currently C++, but imp suggested a rewrite. > > > > https://github.com/freebsd/freebsd-src/tree/main/sbin/devd > > > > > > > > * zfsd. Currently C++, but I've long pondered a rewrite. Using Ru= st > > would > > > > make it more testable. > > > > https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zf= sd > > > > > > > > * nscd. Currently C, but confusing and with no test coverage. I've > > > > contemplated a rewrite myself, but I don't want to do it in C. > > > > https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd > > > > > > > > * The userland portion of the 802.11ac and Lightning stacks. scottl > > suggested > > > > that these were good candidates for Rust. > > > > > > > > * freebsd-kpi-r14-0 . https://crates.io/crates/freebsd-kpi-r14-0 > > > > > > > > Stuff that can live in ports, but would be nicer in base > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > > > > > * gstat-rs https://crates.io/crates/gstat > > > > > > > > * geom-exporter (I've started this, but haven't published it) > > > > > > > > * nfs-exporter https://crates.io/crates/freebsd-nfs-exporter > > > > > > > > * virtiofsd-rs . Nobody has yet tried to port it to FreeBSD. But = if > > the > > > > connection to bhyve(8) is too intimate, it might be hard to do in > > ports. > > > > https://gitlab.com/virtio-fs/virtiofsd > > > > > > > > * jail-exporter https://crates.io/crates/jail_exporter > > > > > > > > * Various jail managers have been attempted in Rust. I think these > > are fine in > > > > ports, but others like Goran Mekic have opined that they should be > > moved to > > > > base instead. > > > > > > > > * musikid's pjdfstest rewrite. I think it would be great to start > > using this > > > > to test the base system's file systems. If the tests themselves > > lived in > > > > base, they would be easier to sync with file system development. > > > > https://github.com/musikid/pjdfstest > > > > > > > > * pf-rs. I suspect that the API isn't very stable. > > > > https://crates.io/crates/pf-rs > > > > > > > > * benchpmc. The pmc counter names changes between releases. > > > > https://crates.io/crates/benchpmc > > > > > > > > FreeBSD-related applications that are just fine in ports > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > > > > > * fsx-rs. Unlike pjdfstest, this only tests datapath APIs. Those = are > > usually > > > > more stable than control path APIs, so I think there's little to = be > > gained by > > > > moving this into base. https://crates.io/crates/fsx > > > > > > > > * ztop. It uses ZFS's kstats sysctl interface, which is pretty sta= ble. > > > > https://crates.io/crates/ztop > > > > > > > > * iocage-provision https://crates.io/crates/iocage-provision > > > > > > > > * rsblk https://crates.io/crates/rsblk > > > > > > > > * xfuse https://github.com/KhaledEmaraDev/xfuse > > > > > > > > Other FreeBSD-related libraries in Rust > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > Just see the list at https://crates.io/keywords/freebsd > > > > > > > > > > One new data point: DARPA is looking to rewrite a significant amount > > > of C code to Rust with their "Translating All C to Rust (TRACTOR)" > > > project: > > > https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view > > > > Interesting. And since you bring it up, I have two new data points mys= elf: > > > > * ctld: while working on some bugs in ctld, I had trouble > > understanding the config file parsing. So I rewrote that part in > > Rust, just to help my understanding. Later, I rewrote the XML > > parsing, too. Then I rewrote the LUN creation and deletion, just to > > see how hard it would be. All of those parts take about 5x fewer SLOC > > in Rust than in C, and they're less buggy, too. Config file parsing > > is more consistent, no memory leaks, etc. Alas, I'm not planning to > > finish this project, since the base system doesn't allow Rust and ctld > > is too tightly coupled to ctl to live in ports. > > >=20 > Cool. Still waiting for anybody to take me up on the offer to do build > system integration. Since the Rust advocates can't get even this basic st= ep > done for review, it's going to be impossible to have Rust in the base. Th= is > isn't even integrate rust compiler like we do with llvm, but with external > Rust toolchain. >=20 > Until somebody steps up for this task, the status quo can't possibly chan= ge. Back at the FreeBSD Developer Summit at this last BSDCan, there was interest in supporting optional external toolchains in the src build framework. You had mentioned you would be happy to mentor someone, but not do the nitty gritty yourself. I could carve off some time in September to be the primary developer, doing the nitty gritty work. Would you be comfortable answering my questions, should I have any? Also: what work (or research), if any, has been done on the concept of external toolchain support for optional components in the FreeBSD source tree? Am I starting afresh or building upon existing work? Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A= 4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc --l2ox7s2au2w6utkz Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAmaqh7MACgkQ/y5nonf4 4frCGg//fh37VVA0JqPk72OOOoPW3RmJ+AhJ2F5KHKMVScrgJGl6vrzmrbqfJe0P 0tp+6TCx/nSHZHfqq8RL+S5vimv7vq+BdZgyKoFSnprjsLODe28azIilvaj/z5tJ 2DIUMWVkIOyDHojx8uxe3B/EfSOrryVGWjx1ltwRn2AcG/lEEqedwBej999fd2jo 8zBbrM/Zi/B7A2u+Sp0WM+D4jvdQLr6GHK+hteZNkovAH1O5SxdGg7nZVFIWSP8h tQpCnt7hg8awyE4AT5FPNZmtJdHnMg9K9r8I2peOwUaFleuqiIJ/eW0FP3fZ6QVh ilda8XfPiZ5TYBM5rnhZR/3vsjz68zhKpo8AypAIeLaUqFNqDIHz4VuyYwhSPox0 HDjttwThDizqqx9Pg0j7YaStjVNH8mlIgLYT22hqH8eDFSnQfInLRX7n37HcZ8DU WsSW2Z0tfEwBG587T95xVdwxlM+jZjH2mdXCaRYhb5drsEp1jePY6AAfeK6iPjmo qXF1aOffPfAkA5wqspWlcklfbpUqMh8w2RqgzrWSw5vzE3mK7gU1dRYI1Z7s0Eii lz/3dxstAYJzcwoN/Rnavehitohj9QwNX0BH5Qy2OecZAjiav8IbZ0irfxnr+8Bo ii6yD7FJiWRC7EnGq3Rg82JHlh0/+FNtbLPGzqVhxf93arxRQJ8= =oSjC -----END PGP SIGNATURE----- --l2ox7s2au2w6utkz-- From nobody Fri Aug 2 00:50:37 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WZnLr144kz5RpGK for ; Fri, 02 Aug 2024 00:50:44 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qv1-xf2e.google.com (mail-qv1-xf2e.google.com [IPv6:2607:f8b0:4864:20::f2e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WZnLq15nHz413p; Fri, 2 Aug 2024 00:50:43 +0000 (UTC) (envelope-from markjdb@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=AAZ+oFkE; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=freebsd.org (policy=none); spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::f2e as permitted sender) smtp.mailfrom=markjdb@gmail.com Received: by mail-qv1-xf2e.google.com with SMTP id 6a1803df08f44-6b97097f7fdso47476456d6.0; Thu, 01 Aug 2024 17:50:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1722559841; x=1723164641; darn=freebsd.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:from:to:cc:subject:date:message-id :reply-to; bh=aVAyTsBPCnVnWVPf+qO9rchUG27iHn71Ar8jdRrvo9U=; b=AAZ+oFkE5V8YDCuZJLs4cLyCjvy5mK2nQr/rRNzoQ3dMtiW0iKQDJDrcdMdydwvPyh hOLOt+XkRD1Qos94jLf2wnMwR/tT6PMTpogTYHPLwLXGtgb2wE2Ok3PpTM6mg6V0/H+Y WnF6xKrAdhE2sXLngugfOn4Y0shamXNEMRAEgpwic90kVwYjyJR6zNQzAE7C7iMJImcy 9h+cq3Vsks9+lDO2xZEFYZdL63pwdWWOG+U/0VHXhy5zZL0NwXzb5HC26XFk3KDxmRme +HBvHXAMzyFD5OLvUTpP1HCrOYh3ZUVK4Ld1HtgGveFk9zEzz7trKS2anymYE9BJGMDw 1jiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722559841; x=1723164641; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aVAyTsBPCnVnWVPf+qO9rchUG27iHn71Ar8jdRrvo9U=; b=FpbE+7BgaQdwiulYqr4+q7Tv+eOMaF/xskLVknJ8jExEfUi5+nahMuEnU52IY30ogU 5r9G/wrCONjC/2TwAScWNZbcNHBqUc4D76OTkKqAgOFzXHMq3Z9Zuyf0F4XUtA+IUelw LL7Hv8HOdcPpIa12V1SUnoEg+gS/bBVZrLTO4T3Poci/V0ZdfScYN7t5EZswhlR9wDQR byRURIl6EcZ0FmLHkJ/DaFBIKtuj9rFpyHzsl8ZDWy8Q7Rjw1+k1MI5PPlq7hQHK2nw9 hAav59JRcnaXD4leFfihQ5AiJZGuTWTT+hb3MfbgTczjtptb/rlbMA2Un6Mn1Iyzabwg /3eg== X-Forwarded-Encrypted: i=1; AJvYcCWvJlvAHS1X8UzCvMAqX/qTMNq9hVE1g6NDuU5SXseode/hmNTuRE2WhcNSX8IyZXjzvBIFieSwEnZmu6zQZhK4vF5YuCxd9Csh8oUvNluSQA== X-Gm-Message-State: AOJu0YyNB6s+CgzxlaVTNK4u9ZMEDhqn54gc7PLhcIGYodLdh4ZR3Qzj brHwbySr7ITOdmXdoFWmgIeTt0j/nXrlk2MG8RwA7uRpxY4Vkd+AE3WFJOUz X-Google-Smtp-Source: AGHT+IFU0b1UD0j3exOECUMzyBsd3sjBR83rWxQykLOprTS6XSmZSTJyCpjIE/tU10NV0xu5RYUhJg== X-Received: by 2002:a05:6214:5405:b0:6b4:b179:8eea with SMTP id 6a1803df08f44-6bb98346b73mr27528756d6.3.1722559841467; Thu, 01 Aug 2024 17:50:41 -0700 (PDT) Received: from nuc (192-0-220-237.cpe.teksavvy.com. [192.0.220.237]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6bb9c864ee0sm1882236d6.119.2024.08.01.17.50.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Aug 2024 17:50:40 -0700 (PDT) Date: Thu, 1 Aug 2024 20:50:37 -0400 From: Mark Johnston To: obiwac Cc: freebsd-hackers , imp@freebsd.org, John Baldwin , "shawn.webb@hardenedbsd.org" Subject: Re: Questions about best practices w.r.t. writing a kernel module port which modified LinuxKPI (BATMAN) Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.60 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MID_RHS_NOT_FQDN(0.50)[]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : SPF not aligned (relaxed), DKIM not aligned (relaxed),none]; MIME_GOOD(-0.10)[text/plain]; TO_DN_EQ_ADDR_SOME(0.00)[]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::f2e:from]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; RCVD_VIA_SMTP_AUTH(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; MISSING_XM_UA(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com] X-Rspamd-Queue-Id: 4WZnLq15nHz413p On Sun, Jul 28, 2024 at 11:04:03PM +0200, obiwac wrote: > Hi! > > I worked on porting the batman_adv Linux kernel module to FreeBSD > using the LinuxKPI last year as part of a GSoC project and I now have > time to work on WiFi support for it (which is necessary for it to > actually be useful in practice). Before I do so though, I'd like to > create a port for it. I have a few questions about best practices > w.r.t. going about this which I was unable to answer by myself by > looking around at other ports (specifically drm-kmod, it's the only > other port that I know of that distributes a kernel module that > depends on the LinuxKPI). Here are the two main questions I have: > > 1. I have made changes to the LinuxKPI headers and other parts of the > kernel in order to accommodate batman_adv. Is it okay for me to > upstream those changes, or should I expect users to apply a patchset > on their kernel source and to recompile it? It's most likely better to try and upstream your changes. Doing so lowers the amount of effort required of users that wish to test your module, which makes it more likely that users will test it. > If I can upstream them, > what should I do about older versions than -CURRENT? Will I just have > to wait for those changes to go into the next -STABLE release? In general, yes. Depending on the nature of your linuxkpi modifications and/or extensions, it might be possible to bundle them in such a way that the module is portable to different versions of FreeBSD. > And if > so, will that mean that any updates that I make to the LinuxKPI > headers necessary for newer versions of batman_adv will either have to > wait until the next release or be distributed alongside the port? It's hard to say in general. It might be possible to bundle your own linuxkpi extensions, but it depends on what you're changing and adding. > 2. I have made changes to ifconfig to support the creation of BATMAN > soft interfaces. Should I upstream those changes and somehow disable > them when the kernel module is not loaded, or should I distribute a > patched version of ifconfig with my port? The typical behaviour for ifconfig is to load requisite kernel modules automatically when creating a new interface. See ifmaybeload() in sbin/ifconfig/ifconfig.c. So, it's generally fine for ifconfig to support functionality that requires a kernel module. > Or should I go with a > different solution entirely, and write and distribute a tool similar > to batctl (which from what I understand was the route taken when > distributing BATMAN on most Linux distros before iproute2 added > support for managing BATMAN interfaces)? It's hard to say without some pointers your code. As a general rule, integrating your code into FreeBSD reduces the amount of work needed to keep out-of-tree code up-to-date, but requires more work up front, so there's a tradeoff involved. Given that you said that some additional WiFi support is needed to make your module useful, I wonder if it's worth your time to create a port first - if I install it locally, would I be able to do anything with it? You might find that it's better to focus on the module in the near term, and maybe try to upstream linuxkpi changes in parallel so that it's easier to create a port later. > Thank you so much in advance for your answers & help! > > (Warner, John, I've CC'd you two as you were in the thread on the > possibility of upstreaming this to the source tree.) > From nobody Fri Aug 2 22:52:55 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WbLjF13SPz5RMGB for ; Fri, 02 Aug 2024 22:53:37 +0000 (UTC) (envelope-from ararslan@comcast.net) Received: from resqmta-c2p-569298.sys.comcast.net (resqmta-c2p-569298.sys.comcast.net [IPv6:2001:558:fd00:56::4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WbLjD0l2Sz4dr4 for ; Fri, 2 Aug 2024 22:53:36 +0000 (UTC) (envelope-from ararslan@comcast.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=comcast.net header.s=20190202a header.b=ahg7BmD2; dmarc=pass (policy=quarantine) header.from=comcast.net; spf=pass (mx1.freebsd.org: domain of ararslan@comcast.net designates 2001:558:fd00:56::4 as permitted sender) smtp.mailfrom=ararslan@comcast.net Received: from resomta-c2p-555662.sys.comcast.net ([96.102.18.235]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 256/256 bits) (Client did not present a certificate) by resqmta-c2p-569298.sys.comcast.net with ESMTPS id a0ZesF9cBaAKOa19Ns7Fc2; Fri, 02 Aug 2024 22:53:29 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; s=20190202a; t=1722639209; bh=uAUPCNTYQN4/hyXgCFkcTPDLr5AgxNj7MLitpnteAnE=; h=Received:Received:From:Message-Id:Content-Type:Mime-Version: Subject:Date:To:Xfinity-Spam-Result; b=ahg7BmD2vpfxkFS9MWUcZMTA3Had/dtKgftkxGDiVcK/3CgDCy9DXpMhaJ7eBPIdt TsA5WHn+fa4CedftmYOVrTyUn79HD+13iyv/Gh/ko8/J+aqlf6YAahTO8l763ixAZh KbNxBFE//XgDYywIAFJivhZZzDMAIMAz/wqcHAN2VqGMH0/qw+QTFUE5orFO11CNs6 1Iar9heJNTfGyV/lB2LNn/YszkdpIcjBUIqBzrmfjHPId8pOeC6+BXTTTjwsOxKB5b m8Q8ki1RHHQiuPsx5UW5rMViu8il3sTXhdeuejvLpo2zl6JmCyUtp/F64YLYzL36Py kWP8qkCEEqo7w== Received: from smtpclient.apple ([67.160.29.205]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 256/256 bits) (Client did not present a certificate) by resomta-c2p-555662.sys.comcast.net with ESMTPSA id a18zsvnM8khVga190syxId; Fri, 02 Aug 2024 22:53:07 +0000 From: Alex Arslan Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_6BB90C8F-19B1-41B0-94E0-8A904F8698C2" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: Diagnosing virtual machine network issues Date: Fri, 2 Aug 2024 15:52:55 -0700 In-Reply-To: Cc: FreeBSD Hackers To: Bakul Shah References: <4a5a177a-5356-453c-8a09-f1d63d5d2e16@sentex.net> <4AB1C33B-DD93-4484-B63A-9FF8FE612B15@comcast.net> <799c7a15-52b8-4b44-bcbd-5ab6a3ef97a6@gmail.com> <0747ED5F-2ED6-461C-9C0B-CFD0EE480D82@comcast.net> <2B0A1E6F-7B89-4F7C-9ECE-ABA94E476D5A@iitbombay.org> X-Mailer: Apple Mail (2.3774.600.62) X-CMAE-Envelope: MS4xfEi5xW9svWWQhtHtTCSkdvfJ/nDaW80+k6h5DZcSpKm+zrRNC0TxOepmESr83BeqxG8cGjh/ztFNgN8sZwia8Mmjg/gobE//fA925DZpJKbdvsrnGtR4 NVv3MWEasyKPCOzsxafQ1eTtN2l+nWJXB1C88CXYYb9k4Ne+WXl3VOmM1uxt5WALUYrG/qT9hDjhrv3RQfam+fOseIaxlBg8bfhGX51SV79KUlZrgd0eCMSS X-Spamd-Bar: / X-Spamd-Result: default: False [-1.00 / 15.00]; HFILTER_HELO_5(3.00)[resqmta-c2p-569298.sys.comcast.net]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[comcast.net,quarantine]; R_SPF_ALLOW(-0.20)[+ip6:2001:558:fd00:56::/64]; R_DKIM_ALLOW(-0.20)[comcast.net:s=20190202a]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ARC_NA(0.00)[]; FREEMAIL_FROM(0.00)[comcast.net]; RCVD_TLS_ALL(0.00)[]; ASN(0.00)[asn:7922, ipnet:2001:558::/29, country:US]; FREEMAIL_ENVFROM(0.00)[comcast.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[comcast.net:+]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DWL_DNSWL_NONE(0.00)[comcast.net:dkim] X-Rspamd-Queue-Id: 4WbLjD0l2Sz4dr4 --Apple-Mail=_6BB90C8F-19B1-41B0-94E0-8A904F8698C2 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jul 30, 2024, at 2:53=E2=80=AFPM, Alex Arslan = wrote: >=20 >=20 >> On Jul 30, 2024, at 2:22=E2=80=AFPM, Bakul Shah = wrote: >>=20 >>> On Jul 30, 2024, at 2:11=E2=80=AFPM, Alex Arslan = wrote: >>>=20 >>>> Can you provide more context? I'm not seeing earlier messages = anywhere in my email folders. Is this a Qemu issue? >>>=20 >>> The original message is from just over a month ago, archived here: >>> = https://lists.freebsd.org/archives/freebsd-hackers/2024-June/003378.html >>> Basically, we have FreeBSD 13.2 VMs running under KVM on a Linux = machine. >>> Some code is using libcurl to make a request to an invalid domain = and is >>> testing that the error is a resolution failure. This test passes on = all >>> platforms except specifically in these FreeBSD VMs; I can't = reproduce >>> locally on FreeBSD. That made me think that there's an issue with = how the >>> VM was set up, prompting the original message and discussion. Then = what >>> I recently found was that we set a 30-second timeout for the libcurl >>> request, which FreeBSD hits in the VM, as it evidently spends a full >>> 30 seconds attempting to resolve the host, while e.g. Linux reports = a >>> resolution failure immediately. >>=20 >> What does /etc/resolv.conf look like on the FreeBSD VM? >=20 >=20 > Just a comment and a name server line: >=20 > $ cat /etc/resolv.conf > # Generated by resolvconf > nameserver 192.168.122.1 I believe that is the host IP, so I guess the VM is using the host for = DNS resolution? Interestingly, if I add `nameserver 8.8.8.8` below the line with the host IP, it takes 10 seconds rather than 30 to reach the = expected domain resolution failure. If I put 8.8.8.8 above the host IP, the = domain resolution failure is instantaneous. Not a particularly satisfying conclusion to this saga as I don't = understand why it's happening but at least I have a workaround that should = hopefully do the job. I really appreciate everyone's help and input thus far! What's the best way to add `nameserver 8.8.8.8` to /etc/resolv.conf as part of the VM's configuration?= --Apple-Mail=_6BB90C8F-19B1-41B0-94E0-8A904F8698C2 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8

On Jul 30, 2024, at 2:53=E2=80=AFPM, Alex Arslan = <ararslan@comcast.net> wrote:


On Jul 30, = 2024, at 2:22=E2=80=AFPM, Bakul Shah <bakul@iitbombay.org> = wrote:

On Jul 30, 2024, at 2:11=E2=80=AFPM, Alex Arslan = <ararslan@comcast.net> wrote:

Can = you provide more context?  I'm not seeing earlier messages anywhere = in my email folders.  Is this a Qemu issue?

The = original message is from just over a month ago, archived = here:
https://lists.freebsd.org/archives/freebsd-hackers/2024-June/0033= 78.html
Basically, we have FreeBSD 13.2 VMs running under KVM on a = Linux machine.
Some code is using libcurl to make a request to an = invalid domain and is
testing that the error is a resolution failure. = This test passes on all
platforms except specifically in these = FreeBSD VMs; I can't reproduce
locally on FreeBSD. That made me think = that there's an issue with how the
VM was set up, prompting the = original message and discussion. Then what
I recently found was that = we set a 30-second timeout for the libcurl
request, which FreeBSD = hits in the VM, as it evidently spends a full
30 seconds attempting = to resolve the host, while e.g. Linux reports a
resolution failure = immediately.

What = does /etc/resolv.conf look like on the FreeBSD = VM?

Just a comment and a = name server line:

$ cat /etc/resolv.conf
# = Generated by resolvconf
nameserver = 192.168.122.1

I believe that is the host IP, so I guess the VM is using the host for = DNS
resolution? Interestingly, if I add `nameserver 8.8.8.8` = below the line
with the host IP, it takes 10 seconds rather = than 30 to reach the expected
domain resolution failure. If I = put 8.8.8.8 above the host IP, the domain
resolution failure = is instantaneous.

Not a particularly satisfying = conclusion to this saga as I don't understand
why it's = happening but at least I have a workaround that should = hopefully
do the job. I really appreciate everyone's help = and input thus far!

What's the best way to add = `nameserver 8.8.8.8` to /etc/resolv.conf as
part of the VM's = configuration?
= --Apple-Mail=_6BB90C8F-19B1-41B0-94E0-8A904F8698C2-- From nobody Fri Aug 2 23:58:13 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WbN842CV1z5RRmc for ; Fri, 02 Aug 2024 23:58:28 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-vk1-f174.google.com (mail-vk1-f174.google.com [209.85.221.174]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WbN830cgNz4kSy for ; Fri, 2 Aug 2024 23:58:27 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=freebsd.org (policy=none); spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.221.174 as permitted sender) smtp.mailfrom=asomers@gmail.com Received: by mail-vk1-f174.google.com with SMTP id 71dfb90a1353d-4f524fa193aso2045531e0c.0 for ; Fri, 02 Aug 2024 16:58:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722643105; x=1723247905; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=20mU2lwdAIqGs7yMFbseysQkl6IYMdffs4Udn8UU7JE=; b=hzQW0KwlOX+ky4WhqSrYefVW8BRB25BgLE2co0KclG4pK06nJyOsOBxVR9+dvpm1NN M07RS3ZnhhKpspAnmpaQp1FMf8x2JbyhZSCTx6bS+oviGBAVhwpShFgHxwuYeefjd7Uq iOGTgJNNC7lyxSXTfh8ciOws0IAfMwRISGnErddGP5mbhIde9msy/PpcHH/4nFMTam2S l9m5t0dtdQ2BlolipYlNTG3M2MgXFW/XO/xRLFy+nKmSa75QqgVR4zyHbgyZkXJrsoQK qa+JkSKRT+Yb+ncImpR1LouurKCfwEJ9VjqUeFPEDyrFFFjghiJxL7b1PRXRIvUDbhS7 GPxw== X-Gm-Message-State: AOJu0Yxws71dMCw4dB66THX9MbVJGHBf6UrEQObt8h4fRUevwMPTqb27 kRmZvud1snDOhihkm3gp2kAAMiryKeyMAcXIwjTfceOt7QoU0OCfcEhj1IRE6TU3J2KOHg6mH2l qvtrtXyxShftJoCf7yvbx4J86DU3H6w== X-Google-Smtp-Source: AGHT+IECdBdQBrU95k92lq7XFMqQjyI72b58uzgGs9QuUjzq9Eum5iRF2x0hrFIGqD6yALe82nT0RwOWTggJgVKc5n0= X-Received: by 2002:a05:6122:3288:b0:4f5:24e0:26d4 with SMTP id 71dfb90a1353d-4f89c06b348mr5481078e0c.1.1722643105144; Fri, 02 Aug 2024 16:58:25 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 From: Alan Somers Date: Fri, 2 Aug 2024 17:58:13 -0600 Message-ID: Subject: RFC: ACLs on fusefs To: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.44 / 15.00]; NEURAL_HAM_SHORT(-0.99)[-0.986]; NEURAL_HAM_LONG(-0.98)[-0.978]; NEURAL_HAM_MEDIUM(-0.58)[-0.577]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; MIME_GOOD(-0.10)[text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : SPF not aligned (relaxed), No valid DKIM,none]; TO_DN_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_ONE(0.00)[1]; MISSING_XM_UA(0.00)[]; FREEFALL_USER(0.00)[asomers]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; RCVD_COUNT_ONE(0.00)[1]; R_DKIM_NA(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.221.174:from]; TO_DOM_EQ_FROM_DOM(0.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; RCVD_IN_DNSWL_NONE(0.00)[209.85.221.174:from] X-Rspamd-Queue-Id: 4WbN830cgNz4kSy TLDR; how useful would it be if fusefs(4) could support ACLs? The current state of fusefs is that while it has full support for extended attributes, it lacks any support for ACLs. If a file system image contains files with ACL entries, the user can look them up with getextattr, but they'll just look like a binary blob. getfacl won't work at all. And the file system won't be able to enforce the ACLs during VOP_ACCESS. Fixing this situation for posix.1e ACLs would require three things: * A good test suite for posix.1e ACLs. pjdfstest has some tests, but it's incomplete. * An example file system to use for testing the kernel driver. It isn't sufficient for the example file system merely to support xattrs, because the file system server must also enforce inheritance of default ACLs. * The actual kernel support. Enforcing ACLs during VOP_ACCESS must be done within the kernel, not the server. The important parts are already in sub_acl_posix1e.c. The fusefs test suite would need a few more test cases for VOP_GETACL and VOP_SETACL, but wouldn't need to test any of the fancy stuff, like inheritance or enforcement during access. Fixing the situation for NFSv4 ACLs would require the above, and also a small extension to the fusefs protocol. All of the above might make a good GSoC project. But is it worth our time? How many real-world users would benefit? Alternatively, doing just the kernel support would be fairly easy. That would be too small for GSoC. But we could easily overlook important bugs if we don't do the other steps, too. So my question is: is this worthwhile? Does anybody know of a real-world workload that would benefit? From nobody Sat Aug 3 00:58:44 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WbPTv6f0bz5RXwB for ; Sat, 03 Aug 2024 00:58:59 +0000 (UTC) (envelope-from bakul@iitbombay.org) Received: from mail-oi1-x22a.google.com (mail-oi1-x22a.google.com [IPv6:2607:f8b0:4864:20::22a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WbPTv4rPKz4qhF for ; Sat, 3 Aug 2024 00:58:59 +0000 (UTC) (envelope-from bakul@iitbombay.org) Authentication-Results: mx1.freebsd.org; none Received: by mail-oi1-x22a.google.com with SMTP id 5614622812f47-3db23a60850so4702994b6e.0 for ; Fri, 02 Aug 2024 17:58:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iitbombay-org.20230601.gappssmtp.com; s=20230601; t=1722646736; x=1723251536; darn=freebsd.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=B5+v8XZ9nmJLWNoxBmLI0TTM9EAaKTX6l2YYUa52G7U=; b=OsN+AkMWnXpVa+KVNnR7IJMe9M3K3mJoc1oDi7I9JNPyVQEjutC4IUIDTnj5JQvWCZ EEI9l8/r5Ko1vMA07pXIknxsKLO4ubvyH6Q3CjftKyM3CLYX8AeEz36ZbfK2EC6F7TyV 2af3Ljh+q/F4kt/SIN+sQUTZLqbu2eWsB6oKqhH1xUXvUTS5paOJ0JAPkua7NowlJqe2 +tEeh79DL1yRDyuaPxRLqZKG/8AQVzUyEksirZWA3THa6oU+1+Xql1MP18URYDPFbhg3 JKd45N+d+81rTKcD//2PXT/YMN88tJQhKvsjPA+rzAunZ6du4bc1+PgCT6VvWuJwYtRr 4iwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722646736; x=1723251536; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=B5+v8XZ9nmJLWNoxBmLI0TTM9EAaKTX6l2YYUa52G7U=; b=nlIDbD14Ber4xeP0EA2LBBTacE6Znp7SHSjNK5rep7HcfXw0LOC4W88PKBUgNg7T39 fOy6H99QmgnVJqzzCBLm4FYJp9+e16rcmziucoINjDrcObd4lGeHLFAALB/NN4wnQ+Q8 8LmyXWPGu+jMsSJqzHmV5NSdt1BCwhOhIaPraL+M/J+bWrF62weTFrelU76JtoJ3uX2O CaEGRpdZ/J8hE7ayyRl82VfPLnV6I74UEtPOKCbNST0GTLpMFDtvyiWLoys0odG7iPGq paOQ8FCQhqvN0iVmxmx4QJy0pgX308k/9bLReW5a2fIg21y246TdCPvKDpNDGE622Pq3 a0pA== X-Gm-Message-State: AOJu0YwKwOJQrQzZ8ugzP/mvyRlgZFNQjiKtnpqExVHhLQaWVR655Zj4 /nRKEk0+dHEd1STsjkSC1lOYSz7gfjMoVV/QI5LxuAga8ICY59YVNbD3VIuMc2iqwwV4ukosxko = X-Google-Smtp-Source: AGHT+IG4Tm1vLEpoJVc9rWOw1RYdoIbPRMGTERpc4rRttBBDDj71qYdRUjJbiF275rp1JcqQmvOPVg== X-Received: by 2002:a05:6808:3027:b0:3db:1b29:f28f with SMTP id 5614622812f47-3db55815cfdmr5960780b6e.24.1722646736410; Fri, 02 Aug 2024 17:58:56 -0700 (PDT) Received: from smtpclient.apple (107-215-223-229.lightspeed.sntcca.sbcglobal.net. [107.215.223.229]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7b76346a9absm1568838a12.29.2024.08.02.17.58.55 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 02 Aug 2024 17:58:55 -0700 (PDT) Content-Type: text/plain; charset=utf-8 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: Diagnosing virtual machine network issues From: Bakul Shah In-Reply-To: Date: Fri, 2 Aug 2024 17:58:44 -0700 Cc: FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: <1532517A-7BD4-42F8-828D-5E1840A10853@iitbombay.org> References: <4a5a177a-5356-453c-8a09-f1d63d5d2e16@sentex.net> <4AB1C33B-DD93-4484-B63A-9FF8FE612B15@comcast.net> <799c7a15-52b8-4b44-bcbd-5ab6a3ef97a6@gmail.com> <0747ED5F-2ED6-461C-9C0B-CFD0EE480D82@comcast.net> <2B0A1E6F-7B89-4F7C-9ECE-ABA94E476D5A@iitbombay.org> To: Alex Arslan X-Mailer: Apple Mail (2.3774.600.62) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4WbPTv4rPKz4qhF On Aug 2, 2024, at 3:52=E2=80=AFPM, Alex Arslan = wrote: >=20 >> Just a comment and a name server line: >>=20 >> $ cat /etc/resolv.conf >> # Generated by resolvconf >> nameserver 192.168.122.1 >=20 > I believe that is the host IP, so I guess the VM is using the host for = DNS > resolution? Interestingly, if I add `nameserver 8.8.8.8` below the = line > with the host IP, it takes 10 seconds rather than 30 to reach the = expected > domain resolution failure. If I put 8.8.8.8 above the host IP, the = domain > resolution failure is instantaneous. What does your host use as a namesever? > Not a particularly satisfying conclusion to this saga as I don't = understand > why it's happening but at least I have a workaround that should = hopefully > do the job. I really appreciate everyone's help and input thus far! >=20 > What's the best way to add `nameserver 8.8.8.8` to /etc/resolv.conf as > part of the VM's configuration? You should diagnose the problem of the nameserver at 192.168.122.1 and fix it to act properly. I don't use vm (just bhyve) so can't help you with its config. From nobody Sat Aug 3 01:51:30 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WbQgK0p6wz5RfBx for ; Sat, 03 Aug 2024 01:52:13 +0000 (UTC) (envelope-from ararslan@comcast.net) Received: from resdmta-a2p-658371.sys.comcast.net (resdmta-a2p-658371.sys.comcast.net [IPv6:2001:558:fd01:2bb4::d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WbQgJ51NTz42tJ for ; Sat, 3 Aug 2024 01:52:12 +0000 (UTC) (envelope-from ararslan@comcast.net) Authentication-Results: mx1.freebsd.org; none Received: from resomta-a2p-647973.sys.comcast.net ([96.103.145.227]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 256/256 bits) (Client did not present a certificate) by resdmta-a2p-658371.sys.comcast.net with ESMTPS id a33cszUXEPxKoa3wDs8aqk; Sat, 03 Aug 2024 01:52:05 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; s=20190202a; t=1722649925; bh=4HZizdOtWECUdTlFfKxBTY3npmy3Asfp5oRd+Hmxv24=; h=Received:Received:Content-Type:Mime-Version:Subject:From:Date: Message-Id:To:Xfinity-Spam-Result; b=ptg108FITWyrORbMQlu3F3/iVzkED1hzbJRZ2AL7K9AG1k8rRMHgoBImgO7yzNQtn CHjd16uP4kbZMiTD4yqF33/wF2k0YZfNA750lsudSZ5xWyOb/pNSkhamQFZyp7AVf8 X99THTpTXBeVsj0avC/r8UYPgigtN10xcp9EQE45fk/E7fPZRL3WncwuDoW5J12cFV QodMCsHr/Xr4zrLRFiIplf7uVhlT7KJRQmia+9zd4WJf0rsadqbPnfSXU18ub4rFCP D3k1Ar7hLGXzDIVhClnfGeQXcCu6L871VjhGn2iHcAFO6+KTqYJJKXYpJhFtWbJc++ uUdbsywCVlgPQ== Received: from smtpclient.apple ([67.160.29.205]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 256/256 bits) (Client did not present a certificate) by resomta-a2p-647973.sys.comcast.net with ESMTPSA id a3vpsRWiOjB19a3vqsHbOH; Sat, 03 Aug 2024 01:51:43 +0000 Content-Type: text/plain; charset=utf-8 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: Diagnosing virtual machine network issues From: Alex Arslan In-Reply-To: <1532517A-7BD4-42F8-828D-5E1840A10853@iitbombay.org> Date: Fri, 2 Aug 2024 18:51:30 -0700 Cc: FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: <4129CE13-5B1A-4E2E-A9C4-7A0162D1E5CC@comcast.net> References: <4a5a177a-5356-453c-8a09-f1d63d5d2e16@sentex.net> <4AB1C33B-DD93-4484-B63A-9FF8FE612B15@comcast.net> <799c7a15-52b8-4b44-bcbd-5ab6a3ef97a6@gmail.com> <0747ED5F-2ED6-461C-9C0B-CFD0EE480D82@comcast.net> <2B0A1E6F-7B89-4F7C-9ECE-ABA94E476D5A@iitbombay.org> <1532517A-7BD4-42F8-828D-5E1840A10853@iitbombay.org> To: Bakul Shah X-Mailer: Apple Mail (2.3774.600.62) X-CMAE-Envelope: MS4xfI7y2lf5/tgVDt5LGvQhSKZrw/uUg6vvU7QKs6iNcb6F3H+TMa1GISz+6w6EjXPyrfgBeB0n7yvx3wcm56gotUxQv3vv5OOugx51ALgN79JgtrM8L5Qn jDRY/lQNupoBzhmZGwGcBc9RWp5JPPCnIy4UFLgj9kcdsyzmwnPs4C0Ip7CMRf7RSMIpLrhzAInmfTg2zQgwisKi1bM+exqHuFOhIE5iyi4YF5uoAuJJ2jXL X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7922, ipnet:2001:558::/29, country:US] X-Rspamd-Queue-Id: 4WbQgJ51NTz42tJ > On Aug 2, 2024, at 5:58=E2=80=AFPM, Bakul Shah = wrote: >=20 > On Aug 2, 2024, at 3:52=E2=80=AFPM, Alex Arslan = wrote: >>=20 >>> Just a comment and a name server line: >>>=20 >>> $ cat /etc/resolv.conf >>> # Generated by resolvconf >>> nameserver 192.168.122.1 >>=20 >> I believe that is the host IP, so I guess the VM is using the host = for DNS >> resolution? Interestingly, if I add `nameserver 8.8.8.8` below the = line >> with the host IP, it takes 10 seconds rather than 30 to reach the = expected >> domain resolution failure. If I put 8.8.8.8 above the host IP, the = domain >> resolution failure is instantaneous. >=20 > What does your host use as a namesever? The nameserver is 127.0.0.53. It sets options edns0 and trust-ad, and includes a search entry as well. >=20 >> Not a particularly satisfying conclusion to this saga as I don't = understand >> why it's happening but at least I have a workaround that should = hopefully >> do the job. I really appreciate everyone's help and input thus far! >>=20 >> What's the best way to add `nameserver 8.8.8.8` to /etc/resolv.conf = as >> part of the VM's configuration? >=20 > You should diagnose the problem of the nameserver at 192.168.122.1 > and fix it to act properly. I don't use vm (just bhyve) so can't help > you with its config. I do still plan to try to figure out what the actual issue is, but I = also now have a path forward in the meantime. :)= From nobody Sat Aug 3 02:34:11 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WbRc10lXPz5RjRt for ; Sat, 03 Aug 2024 02:34:25 +0000 (UTC) (envelope-from khng300@gmail.com) Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WbRc02YNCz48Wk; Sat, 3 Aug 2024 02:34:24 +0000 (UTC) (envelope-from khng300@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-1fb3b7d0d56so50233785ad.1; Fri, 02 Aug 2024 19:34:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1722652462; x=1723257262; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=HuXZAh9r2k6BiRXmdqa/NQSbCo6244Rb4rxDQLB2lw8=; b=SIUBc0gmSPmxNx86xCbkr3rYqq2CwlseFnwIANS9T4lM0bqr+Z66wF7wVO/fiqVxwr vrc32hASyShTVfBfQb6OI41a9nUv4QcbR3ZxM/KOaqRT3JCpZH3q8j/6akyLlO028cXJ IsGfkhKcGZ6NHWxGGQu5QwsdTm2RMC5nYv4ifUl2my4LWyAxwwEpxRxTPek05O/Vt25k JZhSDXdu/h+ZsrwZpe4gHMex/ZOPKllw2H75nf1FVfcxzh41If4KdWQ/mOUGzyCTseI8 qdL4l1BqJH5LGXXJf1Y7w9rHbdk/3ij8b6IfcibY6k5h9wWqK+n7niyPGtaG61kJWvu3 jOMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722652462; x=1723257262; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=HuXZAh9r2k6BiRXmdqa/NQSbCo6244Rb4rxDQLB2lw8=; b=Adx0wy55P9H5ciMMw8JVLjBGWgUP+aoRHXs+fFbxyBt2++DVedUvUaep+SPcuJOFyG sdxnyhpN460Hn3WFOAeq0UT5jy7z6HENNp02QH32Tj3AhlFTqZoxkO8vUbdkrF6sB13e 8/6aeRfJjqnrY+FtEJkJj/BeptovHPW10wMNVekNPYzGPmuGNqI8jJ+xctKTUFC+JSqx XkDWJZo9B9RjDmDdGQ4tiQ6YJLdmCtG99PFIb6y2g2atojqlbIc0LxtcFjo/TiZzqteY Jmhu1di5ffTIjcOLxDkyb1iuu/9lqiwILktXZfyU7Mkt2Oei07iamCAeuHgSi7I3Eeh/ CncQ== X-Gm-Message-State: AOJu0YxFBqtOJC7A01DH2WcFEOIJ+YR8Z+E7FpQPmnKGRRS/irk76sVs i0O2mhoN1/ZW6Yuq8X98oQV2+3PemvFblOyUKaQzWq9fQmIzU9WXiEi/EcnriRqHk9WJFg7OVvh 6GF+BmbzJ2Mfu7tOFApuAM7DSwt+Hfg== X-Google-Smtp-Source: AGHT+IEYddWVzR7bjsw22P6Bs21ha4IXG3eG7AqbP/KSp1cyxE3NXcv9jzUs9/lMQzra9LiUx5dQR5GG+XAFLB1PJuo= X-Received: by 2002:a17:903:2288:b0:1fd:7664:d870 with SMTP id d9443c01a7336-1ff574a9a5dmr62757285ad.51.1722652461912; Fri, 02 Aug 2024 19:34:21 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Ka Ho Ng Date: Fri, 2 Aug 2024 22:34:11 -0400 Message-ID: Subject: Re: RFC: ACLs on fusefs To: Alan Somers Cc: FreeBSD Hackers Content-Type: multipart/alternative; boundary="000000000000105b56061ebe48c3" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4WbRc02YNCz48Wk --000000000000105b56061ebe48c3 Content-Type: text/plain; charset="UTF-8" I would rather see the support of XATTR and NFSv4 ACL being two orthogonal things, just like how it's being worked out on ZFS. On Fri, Aug 2, 2024, 19:58 Alan Somers wrote: > TLDR; > how useful would it be if fusefs(4) could support ACLs? > > The current state of fusefs is that while it has full support for > extended attributes, it lacks any support for ACLs. If a file system > image contains files with ACL entries, the user can look them up with > getextattr, but they'll just look like a binary blob. getfacl won't > work at all. And the file system won't be able to enforce the ACLs > during VOP_ACCESS. > > Fixing this situation for posix.1e ACLs would require three things: > * A good test suite for posix.1e ACLs. pjdfstest has some tests, but > it's incomplete. > * An example file system to use for testing the kernel driver. It > isn't sufficient for the example file system merely to support xattrs, > because the file system server must also enforce inheritance of > default ACLs. > * The actual kernel support. Enforcing ACLs during VOP_ACCESS must be > done within the kernel, not the server. The important parts are > already in sub_acl_posix1e.c. The fusefs test suite would need a few > more test cases for VOP_GETACL and VOP_SETACL, but wouldn't need to > test any of the fancy stuff, like inheritance or enforcement during > access. > > Fixing the situation for NFSv4 ACLs would require the above, and also > a small extension to the fusefs protocol. > > All of the above might make a good GSoC project. But is it worth our > time? How many real-world users would benefit? Alternatively, doing > just the kernel support would be fairly easy. That would be too small > for GSoC. But we could easily overlook important bugs if we don't do > the other steps, too. > > So my question is: is this worthwhile? Does anybody know of a > real-world workload that would benefit? > > --000000000000105b56061ebe48c3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

I would rather see the support of XATTR and NFSv4 ACL being = two orthogonal things, just like how it's being worked out on ZFS.


On Fri= , Aug 2, 2024, 19:58 Alan Somers <asomers@freebsd.org> wrote:
TLDR;
how useful would it be if fusefs(4) could support ACLs?

The current state of fusefs is that while it has full support for
extended attributes, it lacks any support for ACLs.=C2=A0 If a file system<= br> image contains files with ACL entries, the user can look them up with
getextattr, but they'll just look like a binary blob.=C2=A0 getfacl won= 't
work at all.=C2=A0 And the file system won't be able to enforce the ACL= s
during VOP_ACCESS.

Fixing this situation for posix.1e ACLs would require three things:
* A good test suite for posix.1e ACLs.=C2=A0 pjdfstest has some tests, but<= br> it's incomplete.
* An example file system to use for testing the kernel driver.=C2=A0 It
isn't sufficient for the example file system merely to support xattrs,<= br> because the file system server must also enforce inheritance of
default ACLs.
* The actual kernel support.=C2=A0 Enforcing ACLs during VOP_ACCESS must be=
done within the kernel, not the server.=C2=A0 The important parts are
already in sub_acl_posix1e.c.=C2=A0 The fusefs test suite would need a few<= br> more test cases for VOP_GETACL and VOP_SETACL, but wouldn't need to
test=C2=A0 any of the fancy stuff, like inheritance or enforcement during access.

Fixing the situation for NFSv4 ACLs would require the above, and also
a small extension to the fusefs protocol.

All of the above might make a good GSoC project.=C2=A0 But is it worth our<= br> time?=C2=A0 How many real-world users would benefit?=C2=A0 Alternatively, d= oing
just the kernel support would be fairly easy.=C2=A0 That would be too small=
for GSoC.=C2=A0 But we could easily overlook important bugs if we don't= do
the other steps, too.

So my question is: is this worthwhile?=C2=A0 Does anybody know of a
real-world workload that would benefit?

--000000000000105b56061ebe48c3-- From nobody Sat Aug 3 02:53:05 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WbS1p2Mz7z5Rlvg for ; Sat, 03 Aug 2024 02:53:18 +0000 (UTC) (envelope-from khng300@gmail.com) Received: from mail-oi1-x22e.google.com (mail-oi1-x22e.google.com [IPv6:2607:f8b0:4864:20::22e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WbS1n1Wtlz4DNR; Sat, 3 Aug 2024 02:53:17 +0000 (UTC) (envelope-from khng300@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b="AdPgkxe/"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of khng300@gmail.com designates 2607:f8b0:4864:20::22e as permitted sender) smtp.mailfrom=khng300@gmail.com Received: by mail-oi1-x22e.google.com with SMTP id 5614622812f47-3db145c8010so5609036b6e.3; Fri, 02 Aug 2024 19:53:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1722653596; x=1723258396; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=aU8F/RvuJQgsElEIoR1780nYdJlkJnZ4QxSRWjVYZ1Y=; b=AdPgkxe/mnLfEq3SA74V9dIBBE8z5p6y0Pt3lv1dg3PYeDwCEgfEt9ZlXH1wz6QYlK Sjc4mUHJMdf2y4FdOitMvoR7RSLCwBzOihGIs686AnEqnm0mrwnhSGMZxpy5cplE/as1 if5YdoA1Vsj5n0EmMkbeUnnlDnrpBqlOA3Fp+e4t57bq3rrPulM2tohOXY2NS1U7Xshe JFrmdqG6PHzqeKVyE/5dQUEHMSzjnX/gYRco0pzsS8MaY3lhIp8PtaD4Bvy9I2S5nEJ5 QzpLN+EEtW2vgYJObx5boJF07ch1d31CNlFpIgX+H4XrHaOwOkFVZmFgXdzV32SCEYBy pxeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722653596; x=1723258396; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=aU8F/RvuJQgsElEIoR1780nYdJlkJnZ4QxSRWjVYZ1Y=; b=KqUBoRAFOuBjmETJu0yCXTNXZVLlhmY+t40YqxJQAHtLUohcTuFqX9cD9CuBFho7mx N4zSKEE2ht/R94jqSoejZE9IWhQGBNY/FdSxlaMSIqehHwzvq6OaZkYXQpP0aaH5n8MM MT0a9GDwbqON3hQ9GFTl18VAMV3hTykkUm5YrQrLaPy2RHAEpb8Si8v310mdtYs4KuR9 XrKZH8Fa70/mbhmnfHebAye7l0d5/3QbhX4+ocnTCgPZpAhITTpzD+qpTuKcyVKlig0Z Pmyj4OW0d9YCWORLBvcICjFyiTrGvfcWKSbQhEePctrGxyO9/wDNAG0KJFPQl9lvxHU9 k3zQ== X-Gm-Message-State: AOJu0Ywa5kYsa09Zr4Gs+4NC0OL220WQ7jD6tbVhu8L1qmVriqwzQVL0 v8DYukPP8SgEFz9z4CuruBhurxZgr1msDjs1shWF6YejzeZ+jRohJd69+8eTbxBXl6lMIwLXiAe Xt2ZDJSUfJp/wSD5qRDu3Rs3f7GMwEw== X-Google-Smtp-Source: AGHT+IGIvjA9AmXVPunRPI07k/4RqTXtwxehjPkhyAsCLFawYN6w38Uw4bqemnmLhva87CqoHkzwBZKcSiWPr36QSng= X-Received: by 2002:a05:6808:1244:b0:3d9:2751:a207 with SMTP id 5614622812f47-3db5583912amr5938781b6e.44.1722653595728; Fri, 02 Aug 2024 19:53:15 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Ka Ho Ng Date: Fri, 2 Aug 2024 22:53:05 -0400 Message-ID: Subject: Re: RFC: ACLs on fusefs To: Alan Somers Cc: FreeBSD Hackers Content-Type: multipart/alternative; boundary="000000000000a50756061ebe8ba6" X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.79 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.79)[-0.789]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MISSING_XM_UA(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; ARC_NA(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; RCVD_COUNT_ONE(0.00)[1]; FREEMAIL_ENVFROM(0.00)[gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; MIME_TRACE(0.00)[0:+,1:+,2:~]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::22e:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+] X-Rspamd-Queue-Id: 4WbS1n1Wtlz4DNR --000000000000a50756061ebe8ba6 Content-Type: text/plain; charset="UTF-8" Having said that, I am not sure if the FUSE protocol itself is extensible to accommodate the needs to directly implement the counterparts of our existing ACL syscalls. Otherwise XATTR tunneling for both NFSv4 and POSIX 1.e on FUSE might be the only way to go. May I know if there're any users of the XATTR approach besides the e2fsprogs/fuse2fs implementation of the EXT4 filesystem? Ka Ho On Fri, Aug 2, 2024, 22:34 Ka Ho Ng wrote: > I would rather see the support of XATTR and NFSv4 ACL being two orthogonal > things, just like how it's being worked out on ZFS. > > On Fri, Aug 2, 2024, 19:58 Alan Somers wrote: > >> TLDR; >> how useful would it be if fusefs(4) could support ACLs? >> >> The current state of fusefs is that while it has full support for >> extended attributes, it lacks any support for ACLs. If a file system >> image contains files with ACL entries, the user can look them up with >> getextattr, but they'll just look like a binary blob. getfacl won't >> work at all. And the file system won't be able to enforce the ACLs >> during VOP_ACCESS. >> >> Fixing this situation for posix.1e ACLs would require three things: >> * A good test suite for posix.1e ACLs. pjdfstest has some tests, but >> it's incomplete. >> * An example file system to use for testing the kernel driver. It >> isn't sufficient for the example file system merely to support xattrs, >> because the file system server must also enforce inheritance of >> default ACLs. >> * The actual kernel support. Enforcing ACLs during VOP_ACCESS must be >> done within the kernel, not the server. The important parts are >> already in sub_acl_posix1e.c. The fusefs test suite would need a few >> more test cases for VOP_GETACL and VOP_SETACL, but wouldn't need to >> test any of the fancy stuff, like inheritance or enforcement during >> access. >> >> Fixing the situation for NFSv4 ACLs would require the above, and also >> a small extension to the fusefs protocol. >> >> All of the above might make a good GSoC project. But is it worth our >> time? How many real-world users would benefit? Alternatively, doing >> just the kernel support would be fairly easy. That would be too small >> for GSoC. But we could easily overlook important bugs if we don't do >> the other steps, too. >> >> So my question is: is this worthwhile? Does anybody know of a >> real-world workload that would benefit? >> >> --000000000000a50756061ebe8ba6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Having said that, I am not sure if the FUSE protocol itself = is extensible to accommodate the needs to directly implement the counterpar= ts of our existing ACL syscalls. Otherwise XATTR tunneling for both NFSv4 a= nd POSIX 1.e on FUSE might be the only way to go.

May I know if there're any users of the XATTR approach b= esides the e2fsprogs/fuse2fs implementation of the EXT4 filesystem?

Ka Ho


On Fri= , Aug 2, 2024, 22:34 Ka Ho Ng <khng= 300@gmail.com> wrote:

I would rather see the support of XATTR and NFSv4 ACL being two or= thogonal things, just like how it's being worked out on ZFS.


On Fri= , Aug 2, 2024, 19:58 Alan Somers <asomers@freebsd.org> wrote:
TLDR;
how useful would it be if fusefs(4) could support ACLs?

The current state of fusefs is that while it has full support for
extended attributes, it lacks any support for ACLs.=C2=A0 If a file system<= br> image contains files with ACL entries, the user can look them up with
getextattr, but they'll just look like a binary blob.=C2=A0 getfacl won= 't
work at all.=C2=A0 And the file system won't be able to enforce the ACL= s
during VOP_ACCESS.

Fixing this situation for posix.1e ACLs would require three things:
* A good test suite for posix.1e ACLs.=C2=A0 pjdfstest has some tests, but<= br> it's incomplete.
* An example file system to use for testing the kernel driver.=C2=A0 It
isn't sufficient for the example file system merely to support xattrs,<= br> because the file system server must also enforce inheritance of
default ACLs.
* The actual kernel support.=C2=A0 Enforcing ACLs during VOP_ACCESS must be=
done within the kernel, not the server.=C2=A0 The important parts are
already in sub_acl_posix1e.c.=C2=A0 The fusefs test suite would need a few<= br> more test cases for VOP_GETACL and VOP_SETACL, but wouldn't need to
test=C2=A0 any of the fancy stuff, like inheritance or enforcement during access.

Fixing the situation for NFSv4 ACLs would require the above, and also
a small extension to the fusefs protocol.

All of the above might make a good GSoC project.=C2=A0 But is it worth our<= br> time?=C2=A0 How many real-world users would benefit?=C2=A0 Alternatively, d= oing
just the kernel support would be fairly easy.=C2=A0 That would be too small=
for GSoC.=C2=A0 But we could easily overlook important bugs if we don't= do
the other steps, too.

So my question is: is this worthwhile?=C2=A0 Does anybody know of a
real-world workload that would benefit?

--000000000000a50756061ebe8ba6-- From nobody Sat Aug 3 04:11:34 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WbTmD5HLpz5Rt3X for ; Sat, 03 Aug 2024 04:11:40 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WbTmC5DL0z4M60; Sat, 3 Aug 2024 04:11:39 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; none Received: from kalamity.joker.local (123-1-21-232.area1b.commufa.jp [123.1.21.232]) (authenticated bits=0) by www121.sakura.ne.jp (8.17.1/8.17.1/[SAKURA-WEB]/20201212) with ESMTPA id 4734BYjo099049; Sat, 3 Aug 2024 13:11:35 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=dec.sakura.ne.jp; s=s2405; t=1722658295; bh=D6fduBz/WL+hu1jwwR2Y7iomjUMcncpquJt8fHRV060=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=OklR3nnrOYY4DNNkdV8HCFYcBpQF2J2xubiB3jtVJjMm+D6zq77XVGBkpIUVJw33r 06+DxUP3gXKfQkBxFU0FjNyqzemJrKIbv8ipeXqpRum+qU3oYjdquShLcsdD4uFThb MtnvbWkt1Rt/AvQVScRj7bcRJRe0kvQf3RLmK0fo= Date: Sat, 3 Aug 2024 13:11:34 +0900 From: Tomoaki AOKI To: Ka Ho Ng Cc: Alan Somers , FreeBSD Hackers Subject: Re: RFC: ACLs on fusefs Message-Id: <20240803131134.20f61fe590d1929a1733abc3@dec.sakura.ne.jp> In-Reply-To: References: Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd14.1) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP] X-Rspamd-Queue-Id: 4WbTmC5DL0z4M60 I'm not understanding the implementation of fusefs, but possible problem to be considered is the levels of suuports for EAs and ACLs of each implemented filesystems. What I recall is the implementation of EAs on OS/2. IIRC, HPFS natively supported EAs and WPS/Presentation manager-specific attributes (like registry on recent Windoze). To support both on FAT, OS/2 used 2 hidden system files named EA_DATA.SF and WP_ROOT.SF respectively. What's needed to consider would be: a) Store EAs and ACLs on hidden system files with fixed name on the root directory of filesystems which don't support EAs and/or ACLs. b) Silently ignore unsupportd attributes/ACLs and store supported ones only without returning error. c) Error out if any of unsupported attributes/ACLs are specified. Maybe b) would be fatally problematic, as admins who don't understand / don't want to learn about implementations in deep would missingly believe every attribs/ACLs are completely and flawlessly stored even on filesystems which does not support them. a) would be best, if ALL FUSEFS GUYS ON OTHER OS'es agree with the specific filename and its internals, promissing to implement theirs as such, too. Unfortunately, maybe not realistic, unless ISO/IEC or POSIX mandates the support and standardize the name and formats of special file. On Fri, 2 Aug 2024 22:53:05 -0400 Ka Ho Ng wrote: > Having said that, I am not sure if the FUSE protocol itself is extensible > to accommodate the needs to directly implement the counterparts of our > existing ACL syscalls. Otherwise XATTR tunneling for both NFSv4 and POSIX > 1.e on FUSE might be the only way to go. > > May I know if there're any users of the XATTR approach besides the > e2fsprogs/fuse2fs implementation of the EXT4 filesystem? > > Ka Ho > > On Fri, Aug 2, 2024, 22:34 Ka Ho Ng wrote: > > > I would rather see the support of XATTR and NFSv4 ACL being two orthogonal > > things, just like how it's being worked out on ZFS. > > > > On Fri, Aug 2, 2024, 19:58 Alan Somers wrote: > > > >> TLDR; > >> how useful would it be if fusefs(4) could support ACLs? > >> > >> The current state of fusefs is that while it has full support for > >> extended attributes, it lacks any support for ACLs. If a file system > >> image contains files with ACL entries, the user can look them up with > >> getextattr, but they'll just look like a binary blob. getfacl won't > >> work at all. And the file system won't be able to enforce the ACLs > >> during VOP_ACCESS. > >> > >> Fixing this situation for posix.1e ACLs would require three things: > >> * A good test suite for posix.1e ACLs. pjdfstest has some tests, but > >> it's incomplete. > >> * An example file system to use for testing the kernel driver. It > >> isn't sufficient for the example file system merely to support xattrs, > >> because the file system server must also enforce inheritance of > >> default ACLs. > >> * The actual kernel support. Enforcing ACLs during VOP_ACCESS must be > >> done within the kernel, not the server. The important parts are > >> already in sub_acl_posix1e.c. The fusefs test suite would need a few > >> more test cases for VOP_GETACL and VOP_SETACL, but wouldn't need to > >> test any of the fancy stuff, like inheritance or enforcement during > >> access. > >> > >> Fixing the situation for NFSv4 ACLs would require the above, and also > >> a small extension to the fusefs protocol. > >> > >> All of the above might make a good GSoC project. But is it worth our > >> time? How many real-world users would benefit? Alternatively, doing > >> just the kernel support would be fairly easy. That would be too small > >> for GSoC. But we could easily overlook important bugs if we don't do > >> the other steps, too. > >> > >> So my question is: is this worthwhile? Does anybody know of a > >> real-world workload that would benefit? > >> > >> -- Tomoaki AOKI From nobody Sat Aug 3 04:13:05 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WbTp11xxTz5Rtlv for ; Sat, 03 Aug 2024 04:13:13 +0000 (UTC) (envelope-from jamie@catflap.org) Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [IPv6:2001:19f0:7400:8808:123::1]) by mx1.freebsd.org (Postfix) with ESMTP id 4WbTp04zymz4NDY; Sat, 3 Aug 2024 04:13:12 +0000 (UTC) (envelope-from jamie@catflap.org) Authentication-Results: mx1.freebsd.org; none X-Catflap-Envelope-From: X-Catflap-Envelope-To: asomers@FreeBSD.org Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [209.250.224.51]) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5) with ESMTP id 4734D5rA042999; Sat, 3 Aug 2024 05:13:05 +0100 (BST) (envelope-from jamie@donotpassgo.dyslexicfish.net) Received: (from jamie@localhost) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5/Submit) id 4734D5gd042998; Sat, 3 Aug 2024 05:13:05 +0100 (BST) (envelope-from jamie) From: Jamie Landeg-Jones Message-Id: <202408030413.4734D5gd042998@donotpassgo.dyslexicfish.net> Date: Sat, 03 Aug 2024 05:13:05 +0100 Organization: Dyslexic Fish To: freebsd-hackers@FreeBSD.org, asomers@FreeBSD.org Subject: Re: RFC: ACLs on fusefs References: In-Reply-To: User-Agent: Heirloom mailx 12.4 7/29/08 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (donotpassgo.dyslexicfish.net [209.250.224.51]); Sat, 03 Aug 2024 05:13:05 +0100 (BST) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:20473, ipnet:2001:19f0:7400::/38, country:US] X-Rspamd-Queue-Id: 4WbTp04zymz4NDY Alan Somers wrote: > TLDR; > how useful would it be if fusefs(4) could support ACLs? I, personally, don't use ACLs generally, so have not missed them on fusefs. However, I do make extensive use of XATTRs, so those are what I've really missed. I didn't know xatrs were now supported - is that a new thing, or maybe the client I use (borgs sshfs implementation) needs to be updated? Cheers, Jamie From nobody Sat Aug 3 11:53:45 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wbh1S6tw8z5SKKC for ; Sat, 03 Aug 2024 11:53:48 +0000 (UTC) (envelope-from bacon4000@gmail.com) Received: from mail-yw1-x1136.google.com (mail-yw1-x1136.google.com [IPv6:2607:f8b0:4864:20::1136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wbh1S4zZdz41Zj for ; Sat, 3 Aug 2024 11:53:48 +0000 (UTC) (envelope-from bacon4000@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-yw1-x1136.google.com with SMTP id 00721157ae682-6659e81bc68so82131227b3.0 for ; Sat, 03 Aug 2024 04:53:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1722686027; x=1723290827; darn=freebsd.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=gSHyYy8CVaYWRyVTVpo8Z+PjKk8ebHkJLXkjs3QxuIk=; b=Q+/8IypE5UjnOQNUdMTDZeMgZYTvokcjj6g8f/1qRZmw3evF5hH2TNA/hs0ftUVN3q G8xHlq9rNssOiyDWTQT35b162c0Gh47BIrXg24C56WKeweA/XbmL8SBdYWq7zoy98gT6 8E5d0XLoCtGLbEz0ubwcesysY8Z1h4oJEI+5u27BCGrnc4zn/t++ylWDbi4KJlyHmgpP CJ6bGtHd4s7k+exQa/yMf+OvMk9fINCOV2Mibt0YhHQM8B1ngO1Pu9hSd7dmoB/DCktr 2FosG7uw1nI2i9Hep4VWqkZrgwY1OtykGogyF96v5uNsHwmF7tKEPESlk7ffirwFbyXV cNOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722686027; x=1723290827; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=gSHyYy8CVaYWRyVTVpo8Z+PjKk8ebHkJLXkjs3QxuIk=; b=jtXEumUq4ZpPYCSxvWxKwDC+bT3RKLEx1+eBjBKoCDQsb7UepGtn93XLwjmMwrdilo IwUbWmzt3jg8B5EzRN/1FmT6MtiYAdH0+ekFnuBYsfBnvHwWX/PmNUwRq8eDrWRFr6sD EgRToYHTOzJcannFgUD9/nwnl90VvEzkBkxoyboP0+QNAx0RvfFWt2cCqjIRSjjk94bl FsNPoVz+gBuQbYEt7sUznk3gfz3LT+wJ/8j2tp50e7hpQteZ73zdP23cubILe/66lcru eqsF21L2N4EitL05G4oEUiLYkeEVfKURk+JnZJzn5v/cC2nIZ13gAxBa6BCp1GwUDNx+ x6GA== X-Gm-Message-State: AOJu0YzzT2/MffWWBdjaiJQpl+WlOURKXfeN1SbiGCEhZwKWtIDHvJL/ 7S2axXe89Dm1pjI4d23lyrw7FbkZ4V5NgC5EGfC1jXsKsAOe6vZK X-Google-Smtp-Source: AGHT+IEExv9cdL0V2+zdB0F/4sJyeAR/O/sM5z3xAi/cOZJK7EvaeBSCfwGHUvZZh0VysqO8xQsCPg== X-Received: by 2002:a81:c24e:0:b0:631:43e1:6b99 with SMTP id 00721157ae682-689601ab907mr75412017b3.12.1722686027246; Sat, 03 Aug 2024 04:53:47 -0700 (PDT) Received: from [192.168.0.3] (108-255-3-0.lightspeed.milwwi.sbcglobal.net. [108.255.3.0]) by smtp.gmail.com with ESMTPSA id 00721157ae682-68fcd17279esm404087b3.83.2024.08.03.04.53.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 03 Aug 2024 04:53:46 -0700 (PDT) Message-ID: Date: Sat, 3 Aug 2024 06:53:45 -0500 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Diagnosing virtual machine network issues To: Alex Arslan , Bakul Shah Cc: FreeBSD Hackers References: <4a5a177a-5356-453c-8a09-f1d63d5d2e16@sentex.net> <4AB1C33B-DD93-4484-B63A-9FF8FE612B15@comcast.net> <799c7a15-52b8-4b44-bcbd-5ab6a3ef97a6@gmail.com> <0747ED5F-2ED6-461C-9C0B-CFD0EE480D82@comcast.net> <2B0A1E6F-7B89-4F7C-9ECE-ABA94E476D5A@iitbombay.org> Content-Language: en-US From: Jason Bacon In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4Wbh1S4zZdz41Zj On 8/2/24 17:52, Alex Arslan wrote: > What's the best way to add `nameserver 8.8.8.8` to /etc/resolv.conf as > part of the VM's configuration? You might be looking for resolvconf.conf. Not enough information posted to be sure, but that's where you override name servers auto-configured by DHCP. -- Life is a game. Play hard. Play fair. Have fun. From nobody Sat Aug 3 13:52:25 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WbkfP5RR6z5STdc for ; Sat, 03 Aug 2024 13:52:29 +0000 (UTC) (envelope-from SRS0=oXap=PC=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WbkfN4KTZz4Cq7 for ; Sat, 3 Aug 2024 13:52:28 +0000 (UTC) (envelope-from SRS0=oXap=PC=quip.cz=000.fbsd@elsa.codelab.cz) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=quip.cz header.s=private header.b=EIeLCTBy; dkim=pass header.d=quip.cz header.s=private header.b=P3dKqpY+; dmarc=none; spf=none (mx1.freebsd.org: domain of "SRS0=oXap=PC=quip.cz=000.fbsd@elsa.codelab.cz" has no SPF policy when checking 94.124.105.4) smtp.mailfrom="SRS0=oXap=PC=quip.cz=000.fbsd@elsa.codelab.cz" Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 6EC82D788C for ; Sat, 3 Aug 2024 15:52:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quip.cz; s=private; t=1722693146; bh=1lwB9RvNL9o/PwS3SkiefmCvOuX2KlHS7DZ+IUj0Gx0=; h=Date:To:From:Subject; b=EIeLCTByqfSMT9m4qUB3y5jVpDxAhyZmYncjVF3iPoNyHusAtjcmk7xJkS7r/Gnte Yx9dLmSLjesRb7y0XVOCL+kIuTaZR+Rsn4k9VEYHtPJTbSXQ3Tb6KvhLu1jSxUZ5uK SHOxHwwodLU5nobK/W7/Pf8LF+6tp4BAdinGf3DY= Received: from [192.168.145.49] (ip-89-177-27-225.bb.vodafone.cz [89.177.27.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id A50E9D788A for ; Sat, 3 Aug 2024 15:52:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quip.cz; s=private; t=1722693145; bh=1lwB9RvNL9o/PwS3SkiefmCvOuX2KlHS7DZ+IUj0Gx0=; h=Date:To:From:Subject; b=P3dKqpY+lnhTj9Y4rUntk/gBWJMZNLH49Pt71MMMcroavq0S48ELjkmPCVOizso10 M8N7xEV8h0d4Do51My1UDq3vL3EVCPQo2Nbbibf6rAW4Pu9AS7Sdz/CAxv2yASqJ42 uG3Ds9dRXTYTLrdlsk1ALHjTOBuP4Lwcn+qhGJy0= Message-ID: Date: Sat, 3 Aug 2024 15:52:25 +0200 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: freebsd-hackers@FreeBSD.org Content-Language: en-US From: Miroslav Lachman <000.fbsd@quip.cz> Subject: auditd not logging file operations thru NFS Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.97 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.977]; FORGED_SENDER(0.30)[000.fbsd@quip.cz,SRS0=oXap=PC=quip.cz=000.fbsd@elsa.codelab.cz]; R_DKIM_ALLOW(-0.20)[quip.cz:s=private]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; R_SPF_NA(0.00)[no SPF record]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; DMARC_NA(0.00)[quip.cz]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:42000, ipnet:94.124.104.0/21, country:CZ]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@FreeBSD.org]; FROM_NEQ_ENVFROM(0.00)[000.fbsd@quip.cz,SRS0=oXap=PC=quip.cz=000.fbsd@elsa.codelab.cz]; FROM_HAS_DN(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DKIM_TRACE(0.00)[quip.cz:+] X-Rspamd-Queue-Id: 4WbkfN4KTZz4Cq7 I have auditd running on two machines with a configuration to monitor all changes in files on the filesystem. If I write to the file from the localhost (on machine A), everything works and the record appears in the logfile. However, if a directory is exported via NFS, mounted on another machine (machine B), and I write to the file on the machine B, then no record appears in the audit log on machine A. Is there a way to configure auditd to log these events too? /etc/security/audit_user is empty /etc/security/audit_event is default /etc/security/audit_class is default # cat /etc/security/audit_control # # $FreeBSD: releng/10.3/contrib/openbsm/etc/audit_control 293161 2016-01-04 16:32:21Z brueffer $ # dir:/var/audit dist:off flags:lo,aa,ad,fw,fm,fc,fd minfree:5 naflags:lo,aa,ad,fw,fm,fc,fd policy:cnt,argv filesz:50M expire-after:600s Kind regards Miroslav Lachman From nobody Sat Aug 3 14:53:05 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wbm0c17dRz5SYn5 for ; Sat, 03 Aug 2024 14:53:20 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ua1-f51.google.com (mail-ua1-f51.google.com [209.85.222.51]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wbm0b42Y0z4Kkf for ; Sat, 3 Aug 2024 14:53:19 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ua1-f51.google.com with SMTP id a1e0cc1a2514c-8225a1f4d5fso2251856241.0 for ; Sat, 03 Aug 2024 07:53:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722696798; x=1723301598; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BZwQWuLymhgfBw9kuzAtUwTc/JcmyEKrS3jWwVZ0TME=; b=gVmUa03o7fAz4tPRjz5vyBVs0/PR9D4GuLC0z4pK0rfLSLK/EfxfXYy7uL7d6H+Er9 hO5Hg34BuZpRKr1tDdsQAYLCFiqJrHMrNGrGzOtzCUbaoq0JeoLaS+ETfQ8c82W3gxhE TBMo3SOnydc8uI0yMoiBodtAza2Kp53YgohqdOyk8vaY/QR9ieaj5s9/0WqV2A7xsv/R j4GywDqYoxmv0FhJscPTAn+JgSoItUkDxdGpDNajqHrx71lVnMHzx9QuEhlDOkemmURK Y2KwFqAa6fgHG68mQZhRDZtg+hs6niNRmgAYbdlxtE7ZabyMRpKn6Od9osCcwV80m667 P/uw== X-Gm-Message-State: AOJu0YzvjsP5aFtNXL77BVdjSfD11j2XXibbQBTTsrNaVp0S7cT3A3ir G/qwOpxMrLfkPld3DnzlW+qw2EcxSTeZSGCAxyoN1pjdCTkg1d7xPnthqzexoo4+tDRoNemYfqD QfQhWfgVOpbh3JAbFhdngWFusuwQY7w== X-Google-Smtp-Source: AGHT+IH0uOTaNRzPcsWt+BswMjycd+Hn5Xa53kr2RyxaYD6dGrJ+xqSWtzKtwxLiTFDL37nm+Bhr40504UBqYAXiO+I= X-Received: by 2002:a05:6102:26d3:b0:493:bcbd:4633 with SMTP id ada2fe7eead31-4945bdb0381mr6377297137.3.1722696797748; Sat, 03 Aug 2024 07:53:17 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Sat, 3 Aug 2024 08:53:05 -0600 Message-ID: Subject: Re: RFC: ACLs on fusefs To: Ka Ho Ng Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4Wbm0b42Y0z4Kkf On Fri, Aug 2, 2024 at 8:53=E2=80=AFPM Ka Ho Ng wrote: > > Having said that, I am not sure if the FUSE protocol itself is extensible= to accommodate the needs to directly implement the counterparts of our exi= sting ACL syscalls. Otherwise XATTR tunneling for both NFSv4 and POSIX 1.e = on FUSE might be the only way to go. Not currently. There's no way to send an ACL to a fuse server except as an extended attribute. That's similar to how ACLs work on UFS. Some kind of FUSE_SETFACL operation could be added, but only if there's a file system that needs it. > > May I know if there're any users of the XATTR approach besides the e2fspr= ogs/fuse2fs implementation of the EXT4 filesystem? Actually, fusefs-ext2 doesn't make use of ACLs. Neither do sysutils/e2fsprogs-core or sysutils/fusefs-lkl. I don't know of any fuse file system that does, though I haven't grepped them all. > > Ka Ho > > > On Fri, Aug 2, 2024, 22:34 Ka Ho Ng wrote: >> >> I would rather see the support of XATTR and NFSv4 ACL being two orthogon= al things, just like how it's being worked out on ZFS. >> >> >> On Fri, Aug 2, 2024, 19:58 Alan Somers wrote: >>> >>> TLDR; >>> how useful would it be if fusefs(4) could support ACLs? >>> >>> The current state of fusefs is that while it has full support for >>> extended attributes, it lacks any support for ACLs. If a file system >>> image contains files with ACL entries, the user can look them up with >>> getextattr, but they'll just look like a binary blob. getfacl won't >>> work at all. And the file system won't be able to enforce the ACLs >>> during VOP_ACCESS. >>> >>> Fixing this situation for posix.1e ACLs would require three things: >>> * A good test suite for posix.1e ACLs. pjdfstest has some tests, but >>> it's incomplete. >>> * An example file system to use for testing the kernel driver. It >>> isn't sufficient for the example file system merely to support xattrs, >>> because the file system server must also enforce inheritance of >>> default ACLs. >>> * The actual kernel support. Enforcing ACLs during VOP_ACCESS must be >>> done within the kernel, not the server. The important parts are >>> already in sub_acl_posix1e.c. The fusefs test suite would need a few >>> more test cases for VOP_GETACL and VOP_SETACL, but wouldn't need to >>> test any of the fancy stuff, like inheritance or enforcement during >>> access. >>> >>> Fixing the situation for NFSv4 ACLs would require the above, and also >>> a small extension to the fusefs protocol. >>> >>> All of the above might make a good GSoC project. But is it worth our >>> time? How many real-world users would benefit? Alternatively, doing >>> just the kernel support would be fairly easy. That would be too small >>> for GSoC. But we could easily overlook important bugs if we don't do >>> the other steps, too. >>> >>> So my question is: is this worthwhile? Does anybody know of a >>> real-world workload that would benefit? >>> From nobody Sat Aug 3 14:59:15 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wbm7h4FWHz5SZ2n for ; Sat, 03 Aug 2024 14:59:28 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ua1-f54.google.com (mail-ua1-f54.google.com [209.85.222.54]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wbm7h2St1z4Lss for ; Sat, 3 Aug 2024 14:59:28 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ua1-f54.google.com with SMTP id a1e0cc1a2514c-825a23c5e4cso2109755241.3 for ; Sat, 03 Aug 2024 07:59:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722697167; x=1723301967; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BlmKVfuM9Bw/Si/uiawLEB/KWNGMW1VtVUQ7d3nILDU=; b=Fb3vpQ2vGhnycdhufq9R8CHI2+6EaIBWNvQGtXBkCUx7zw4iqGAoOOnCWowN6Rjeno SdDvspXPitp8jm3w1iwbHzIdJd198IxnieW0uHUNb9Y/1tS79hFxQbfMJdhss8TNyaEO W71P0kB6w1Z5zfTEpI7xrJtn1z+ovH33tyDHjmnarFtXTYdInG8GsTIZ8H1zKg7tQ0u+ fxjbn0uOCQDRGgmlC4Lqeoyp6g2xBYXm9PvxMbJAt/yxKf13aPaygs4pGu7ev4BAWfI3 ob+VUEgppCDt9c3D5K8kAM0i9MfJZnEtZkNwMELo7sF4kYqW7KR3sMQJW7+7hNor/B0z R7gg== X-Forwarded-Encrypted: i=1; AJvYcCVPsN+wZaK/ZGIpW1QOX0NLnQsEvNRdejMzU+b+XjZLhiJIJU9Qnemt0/Da3FF4kV6FUxzDPlIaFKLAH/eIpMhCqeX3ydltQwamOyE= X-Gm-Message-State: AOJu0Yz5pXEyHSGSutXmcFAUW5nIBPkf4icPkFdAyjJDWKRHmQ51Q+5h 2517Zny/T0niUi7x7QmKXTEaDxVFt1DeqHEuqRcZthXy6paaoDGL/1Ywv7FP5YE+0TQKS0kvw5m 0zgv35mwnFlQS8HhMwl/eV5diZDg= X-Google-Smtp-Source: AGHT+IGIupZ1EuuD7nuSVEsLMch+VcijqcalU5059Oi5WYkzqFpJol9b+nZD2m1w7f7l5pi3M/GhB9Eq1BgsYJZ5LH8= X-Received: by 2002:a05:6102:4408:b0:48f:9324:db08 with SMTP id ada2fe7eead31-4945bde6c6emr7019414137.4.1722697166933; Sat, 03 Aug 2024 07:59:26 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <20240803131134.20f61fe590d1929a1733abc3@dec.sakura.ne.jp> In-Reply-To: <20240803131134.20f61fe590d1929a1733abc3@dec.sakura.ne.jp> From: Alan Somers Date: Sat, 3 Aug 2024 08:59:15 -0600 Message-ID: Subject: Re: RFC: ACLs on fusefs To: Tomoaki AOKI Cc: Ka Ho Ng , FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4Wbm7h2St1z4Lss Storing ACLs in hidden files is beyond the scope of what I had in mind. That would be an entirely new challenge. And it wouldn't be possible to do it correctly, because the file system server is still responsible for the inheritance of default ACLs. For the present, all that I have in mind is file systems that can store ACLs natively. On Fri, Aug 2, 2024 at 10:11=E2=80=AFPM Tomoaki AOKI wrote: > > I'm not understanding the implementation of fusefs, but possible > problem to be considered is the levels of suuports for EAs and ACLs of > each implemented filesystems. > > What I recall is the implementation of EAs on OS/2. > IIRC, HPFS natively supported EAs and WPS/Presentation manager-specific > attributes (like registry on recent Windoze). > To support both on FAT, OS/2 used 2 hidden system files named > EA_DATA.SF and WP_ROOT.SF respectively. > > What's needed to consider would be: > a) Store EAs and ACLs on hidden system files with fixed name > on the root directory of filesystems which don't support > EAs and/or ACLs. > > b) Silently ignore unsupportd attributes/ACLs and store supported > ones only without returning error. > > c) Error out if any of unsupported attributes/ACLs are specified. > > Maybe b) would be fatally problematic, as admins who don't understand / > don't want to learn about implementations in deep would missingly > believe every attribs/ACLs are completely and flawlessly stored even on > filesystems which does not support them. > > a) would be best, if ALL FUSEFS GUYS ON OTHER OS'es agree with the > specific filename and its internals, promissing to implement theirs as > such, too. Unfortunately, maybe not realistic, unless ISO/IEC or POSIX > mandates the support and standardize the name and formats of special > file. > > > On Fri, 2 Aug 2024 22:53:05 -0400 > Ka Ho Ng wrote: > > > Having said that, I am not sure if the FUSE protocol itself is extensib= le > > to accommodate the needs to directly implement the counterparts of our > > existing ACL syscalls. Otherwise XATTR tunneling for both NFSv4 and POS= IX > > 1.e on FUSE might be the only way to go. > > > > May I know if there're any users of the XATTR approach besides the > > e2fsprogs/fuse2fs implementation of the EXT4 filesystem? > > > > Ka Ho > > > > On Fri, Aug 2, 2024, 22:34 Ka Ho Ng wrote: > > > > > I would rather see the support of XATTR and NFSv4 ACL being two ortho= gonal > > > things, just like how it's being worked out on ZFS. > > > > > > On Fri, Aug 2, 2024, 19:58 Alan Somers wrote: > > > > > >> TLDR; > > >> how useful would it be if fusefs(4) could support ACLs? > > >> > > >> The current state of fusefs is that while it has full support for > > >> extended attributes, it lacks any support for ACLs. If a file syste= m > > >> image contains files with ACL entries, the user can look them up wit= h > > >> getextattr, but they'll just look like a binary blob. getfacl won't > > >> work at all. And the file system won't be able to enforce the ACLs > > >> during VOP_ACCESS. > > >> > > >> Fixing this situation for posix.1e ACLs would require three things: > > >> * A good test suite for posix.1e ACLs. pjdfstest has some tests, bu= t > > >> it's incomplete. > > >> * An example file system to use for testing the kernel driver. It > > >> isn't sufficient for the example file system merely to support xattr= s, > > >> because the file system server must also enforce inheritance of > > >> default ACLs. > > >> * The actual kernel support. Enforcing ACLs during VOP_ACCESS must = be > > >> done within the kernel, not the server. The important parts are > > >> already in sub_acl_posix1e.c. The fusefs test suite would need a fe= w > > >> more test cases for VOP_GETACL and VOP_SETACL, but wouldn't need to > > >> test any of the fancy stuff, like inheritance or enforcement during > > >> access. > > >> > > >> Fixing the situation for NFSv4 ACLs would require the above, and als= o > > >> a small extension to the fusefs protocol. > > >> > > >> All of the above might make a good GSoC project. But is it worth ou= r > > >> time? How many real-world users would benefit? Alternatively, doin= g > > >> just the kernel support would be fairly easy. That would be too sma= ll > > >> for GSoC. But we could easily overlook important bugs if we don't d= o > > >> the other steps, too. > > >> > > >> So my question is: is this worthwhile? Does anybody know of a > > >> real-world workload that would benefit? > > >> > > >> > > > -- > Tomoaki AOKI > From nobody Sat Aug 3 15:03:38 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WbmDl6QsPz5SZbv for ; Sat, 03 Aug 2024 15:03:51 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f170.google.com (mail-oi1-f170.google.com [209.85.167.170]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WbmDl4g1dz4NrD for ; Sat, 3 Aug 2024 15:03:51 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oi1-f170.google.com with SMTP id 5614622812f47-3db1d0fca58so5716965b6e.3 for ; Sat, 03 Aug 2024 08:03:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722697430; x=1723302230; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2y0SlcuGw9006ZKTM/jdrrrOFK3c8HrAxRSA0U3qAyo=; b=iMCUVPIJoFOi22sO+rCSPI8EXHwgGIyPPElbQFHmqRC0dNzuSZTBC36bMTv8srYjvi Ru6GLLmV2GzAmPUpQFHn8H44YMfIRJlh56THIcrEbON9ZwjFs4OGQHXceGihyLUKd8Kl AgMN++rgkdmY5lzsdZfMvXGKn058ob+k14x2+N79apuQ1UgVn6+ifyhMFnQtxLhO6jKQ D7y83s/S1rxYjSKzyZKKZTa8EBBYYsFU4HrD3LVIH3UimikVz78jwhS0pi6akJhgKvEV Jfm05UCu+JcBK4l97GlIWR0c2rYo2Y95d6Wf0ZMsQLs93serKe7+eQ93F3SY2snbC2Mt qmiQ== X-Gm-Message-State: AOJu0YwkQM/81lrtRlufyh7cWVK7V+3HZNOVyMMbj01LiDdvt40fIDwC /zWbcAQAKpaJ+Kl/jBXmK51Rs4IrBRlnG3vTWFAEOPr6GWiCcgcRxe6Z6pvlgEVAa+AysE6PZB+ xgu3mz8/i6+9Dec/awvcfMDotLWtAsrNA X-Google-Smtp-Source: AGHT+IH8bjixZSHbjOsqB1ybyhl7Pwx28SwF0ZRjvNARVentDEosJTkvJZIYiIF1IedMkdFVPea/h7WxLv+soV0oJwo= X-Received: by 2002:a05:6808:130d:b0:3d5:5c77:fc2f with SMTP id 5614622812f47-3db55840a42mr6614835b6e.48.1722697430236; Sat, 03 Aug 2024 08:03:50 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <202408030413.4734D5gd042998@donotpassgo.dyslexicfish.net> In-Reply-To: <202408030413.4734D5gd042998@donotpassgo.dyslexicfish.net> From: Alan Somers Date: Sat, 3 Aug 2024 09:03:38 -0600 Message-ID: Subject: Re: RFC: ACLs on fusefs To: Jamie Landeg-Jones Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4WbmDl4g1dz4NrD On Fri, Aug 2, 2024 at 10:13=E2=80=AFPM Jamie Landeg-Jones wrote: > > Alan Somers wrote: > > > TLDR; > > how useful would it be if fusefs(4) could support ACLs? > > I, personally, don't use ACLs generally, so have not missed them on > fusefs. > > However, I do make extensive use of XATTRs, so those are what I've > really missed. > > I didn't know xatrs were now supported - is that a new thing, or maybe > the client I use (borgs sshfs implementation) needs to be updated? > > Cheers, Jamie Our fusefs has supported xattrs for a long time. But the specific fuse file system needs support too. Looking right now, I don't see any support in sysutils/fusefs-sshfs . From nobody Sat Aug 3 15:06:28 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WbmJ32rZNz5SZTZ for ; Sat, 03 Aug 2024 15:06:43 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oo1-f51.google.com (mail-oo1-f51.google.com [209.85.161.51]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WbmJ30dBmz4PmD for ; Sat, 3 Aug 2024 15:06:43 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oo1-f51.google.com with SMTP id 006d021491bc7-5d5d4d07babso4554500eaf.3 for ; Sat, 03 Aug 2024 08:06:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722697601; x=1723302401; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YsiQE8Aq0wf7X8Ii0LlRyKv+ACc+n8vcj7o0Kus3YPA=; b=IxCs4wFitVFXDi9ozs93NyNefOGHJgDPMlLMFxnU8+6d3rVUxh6ZXQydW/UVFwg6Sk dHtUA4GCYlKGimczxq+Ffj9NhArpDeNjFo7R3d33G1ItOwIt2GVvsD97rqIf7FkMazP1 U9L5c/X7NQm8ioPk5Dp5hAasb908xRAAuYErrtEmNtAp15KehnqaPj8jpvmS1gcQvrqh f0SRFZH/QqTZLx0TXiJxnKNz2TragSUsVAsGoyQsxA5XZwIbkQs7YHDiBj/biFcPRGGD qhtyYM82i6eqmRbUPqbjhwYX+Grse38Emv+C4L7FWEILmLkxNapG3GBKla1pR8AKrbCb oAVw== X-Gm-Message-State: AOJu0Yyrrnu8FDMnPVhPG++WUnkfo4epjIjjb00JEWaHpweGRUkj76wW fSR1rou7OVrmEFYLzKyCl0WyJ6uvIY8iaGzg4Ih1+9p4oha73ly1qYqRzGDx+jNRX8gi0Btq8Wj l66Zq/i1OAiU3HtPcR+TpumSbCok2kg== X-Google-Smtp-Source: AGHT+IGR458pgs3uD2ynmm8zDBlMX83cop3u7E6BlnmMDPyU4hAHXgBuR18x+I0PyhADm4lMY59JfHeTQHlkpC1uPAw= X-Received: by 2002:a05:6808:1897:b0:3d9:33d0:cc40 with SMTP id 5614622812f47-3db557fd0f5mr8089336b6e.9.1722697601634; Sat, 03 Aug 2024 08:06:41 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Sat, 3 Aug 2024 09:06:28 -0600 Message-ID: Subject: Re: auditd not logging file operations thru NFS To: Miroslav Lachman <000.fbsd@quip.cz> Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4WbmJ30dBmz4PmD On Sat, Aug 3, 2024 at 7:52=E2=80=AFAM Miroslav Lachman <000.fbsd@quip.cz> = wrote: > > I have auditd running on two machines with a configuration to monitor > all changes in files on the filesystem. If I write to the file from the > localhost (on machine A), everything works and the record appears in the > logfile. However, if a directory is exported via NFS, mounted on another > machine (machine B), and I write to the file on the machine B, then no > record appears in the audit log on machine A. > Is there a way to configure auditd to log these events too? > > /etc/security/audit_user is empty > /etc/security/audit_event is default > /etc/security/audit_class is default > > # cat /etc/security/audit_control > # > # $FreeBSD: releng/10.3/contrib/openbsm/etc/audit_control 293161 > 2016-01-04 16:32:21Z brueffer $ > # > dir:/var/audit > dist:off > flags:lo,aa,ad,fw,fm,fc,fd > minfree:5 > naflags:lo,aa,ad,fw,fm,fc,fd > policy:cnt,argv > filesz:50M > expire-after:600s > > Kind regards > Miroslav Lachman Nope. That's a known limitation of auditd. It works at a higher level than nfs. If you want to audit operations over NFS, currently you must run auditd on the NFS client. There was actually a GSoC project that tried to fix this a few years ago, but it ran into too many problems and was ultimately unsuccessful. From nobody Sat Aug 3 16:00:39 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WbnVL2v50z5SfDZ for ; Sat, 03 Aug 2024 16:00:42 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-io1-xd31.google.com (mail-io1-xd31.google.com [IPv6:2607:f8b0:4864:20::d31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WbnVL1Hhkz4TZ3 for ; Sat, 3 Aug 2024 16:00:42 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Authentication-Results: mx1.freebsd.org; none Received: by mail-io1-xd31.google.com with SMTP id ca18e2360f4ac-81fb419f77bso253290439f.2 for ; Sat, 03 Aug 2024 09:00:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; t=1722700841; x=1723305641; darn=freebsd.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=k3zaJ9DRmpXWHsaHBlbJGoNSUEkDA+VeXTfEgdF8pUo=; b=EKeu1/yWDmhkoLIS8y8xpy4nfuta4Zp1wheY4kuYDXfgoMhugXei5ZC01DFq00cTrU 65awKfhQduCD8mqktUgQR6n3Hu/fYgYCWMdcvn//N53Oj23p5LGSjyVDHQlEh+HvmU/N WxYBN0OJJzuSauj/LIAKm6PKz5VYgBQD3eBz7dwrOGCFOJtSnmTkeNIvzHPAvQR2oLEV d6+QgrjFo0wJsMvXTWloN1p4Xqxn1FYQuV/DcqoX2Ph1HYL0XoDFDsB1u+eBwd4SKCAt G08XhDCBQQA5iI8/ZctQSgBj9rOrNcHZCiSeq1RFArAu19ARyP2M4JzkEyLt0cQZBEtq 2T3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722700841; x=1723305641; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=k3zaJ9DRmpXWHsaHBlbJGoNSUEkDA+VeXTfEgdF8pUo=; b=MP/8yPPCY/WQ4VnPIrcmECMwc9/V0HSr8QWXVyUr1KhUwxakx/jjDbJYIBuixa3nx6 AxMHhZoUxNUSZuZv5gkarGK0l4o5HewvZr+rGInn+YudneWLU5VfDoDjfA8WR0lYl8y+ Ug7ViBhhotoBh885y+LKEiCkvGN763Om56wqlNfbKIRahkJ824pPyUlvU66vm7tVuQkO xHSsJKaxD7y+39vkvsJgf0nk+vL9MWeeLFoerwHMDO/G6g6thyDSSsdSUFFwZDtp/VnP uPJWszQBZBxc9GYoy/dxoa4CfzA4zapEdYhDH1xo+XfU7M3CZteIo8dL9LybqKBzzykd b0vw== X-Forwarded-Encrypted: i=1; AJvYcCXvUW9fULTxTVPK8Hc+DbLXw0mN0TYLSYhwjzI91nR4xUD8jyjBLBASwLpmkQU5mIJZIhy+a0rInWDTFXG2Epo2HI3uI+zhdeDmBqU= X-Gm-Message-State: AOJu0YytQkF601awDfORizdZazQlZeSusK0Hj8CNV+QQ1RHGbsqdiRfE doGBUWxguM9CVam8cjtNewGn4lNovUf83re5A+4b6vRYXqKuipbXqrQMI1GPXncyVW5llUympcX 4 X-Google-Smtp-Source: AGHT+IFa20bPXeSh4GJkH516fPJKuNcEpH7LCJ0pEN0DcGF/IlSawj9IpmEguiuQuJveoPnX/KAC2g== X-Received: by 2002:a05:6602:15c5:b0:806:f495:3b34 with SMTP id ca18e2360f4ac-81fd43509a9mr885419739f.2.1722700841026; Sat, 03 Aug 2024 09:00:41 -0700 (PDT) Received: from mutt-hbsd (174-24-73-190.clsp.qwest.net. [174.24.73.190]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-821ef0bbfb2sm19815539f.2.2024.08.03.09.00.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 03 Aug 2024 09:00:40 -0700 (PDT) Date: Sat, 3 Aug 2024 16:00:39 +0000 From: Shawn Webb To: Alan Somers Cc: Jamie Landeg-Jones , freebsd-hackers@freebsd.org Subject: Re: RFC: ACLs on fusefs Message-ID: X-Operating-System: FreeBSD mutt-hbsd 15.0-CURRENT-HBSD FreeBSD 15.0-CURRENT-HBSD X-PGP-Key: https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/blob/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc References: <202408030413.4734D5gd042998@donotpassgo.dyslexicfish.net> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="ren7enky4jnymnu3" Content-Disposition: inline In-Reply-To: X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4WbnVL1Hhkz4TZ3 --ren7enky4jnymnu3 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Aug 03, 2024 at 09:03:38AM -0600, Alan Somers wrote: > On Fri, Aug 2, 2024 at 10:13=E2=80=AFPM Jamie Landeg-Jones wrote: > > > > Alan Somers wrote: > > > > > TLDR; > > > how useful would it be if fusefs(4) could support ACLs? > > > > I, personally, don't use ACLs generally, so have not missed them on > > fusefs. > > > > However, I do make extensive use of XATTRs, so those are what I've > > really missed. > > > > I didn't know xatrs were now supported - is that a new thing, or maybe > > the client I use (borgs sshfs implementation) needs to be updated? > > > > Cheers, Jamie >=20 > Our fusefs has supported xattrs for a long time. But the specific > fuse file system needs support too. Looking right now, I don't see > any support in sysutils/fusefs-sshfs . In fact, I have a (significantly buggy) proof-of-concept fusefs server that stores file payload data as extended attributes. Since the tar file format supports extended attributes, this makes data exfiltration somewhat easier. Though, I suppose, since my proof-of-concept is buggy, using my solution would make data exfil somewhat more difficult. ;-) Hopefully someday, I'll have the time to finish the PoC and make it usable for production. PoC code: https://git.hardenedbsd.org/shawn.webb/altfs Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A= 4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc --ren7enky4jnymnu3 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAmauVCAACgkQ/y5nonf4 4fp8wxAAhIuxX9brcIWuwdSfg+tQZuViyrtWH3k6l32LWohknJLnPPMWO/vCNePu UO7G8WeWA3iJQwAxnXt3eOf9EpOEaTSUfTACr78NSK9XTzIvT4DSzNetT9VQ1TKx x7xOENH6jxXNR/x8/K8F2l+8DOmevl7FcP7A4TLDoiLYnibZw5xP+XVnEwEnFFXv gcpBNzNbBfkH1lNTRUiYRkx6gnUTsyAy4xj9auznQdFGsI+951j8lNTK2tu/Fmba 6hUcdUNrOZcejVIkT3Eu29tf0qE6mcODM17zeZ/ShY+ZNnH51aetGWFtur0PgSrI t/a69UJ6XzFjmjaAw2+NmgZuveIXNGdaDXIcDskGCpm87aZMLScXoym4kEgFjYnw VLx04BVG5q3Yjd/f70dhKO/coRYGudndkuDkYNE54ZelQBZALGuUTEq3VAevJN9g i+XX1hjYjEJxsFVKUdkzUaTdy5s+Wr8ODrbrAn15nClGp1UswoU+F1WDe59EJtgE DO4HfOkgt0JFPG29iPgvOapcTw0dOX4zBN1K/nAFT5ejg6M25XXHruGiqC1G1Fdx XjoY+GItzENkNXwXgDxmpHhKrBLb+KkIyGbnQFo4yK/1sJclambmiFlMAl0TG1p/ ZqEY5j7GsMkrrPA5BO+cKLIe+9eLQ8V3ss5c4+1LdjGcEUbm0E8= =HfKq -----END PGP SIGNATURE----- --ren7enky4jnymnu3-- From nobody Sat Aug 3 16:10:05 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WbnjR1QsKz5Sfqb for ; Sat, 03 Aug 2024 16:10:19 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oo1-f46.google.com (mail-oo1-f46.google.com [209.85.161.46]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WbnjQ4qy2z4WX4 for ; Sat, 3 Aug 2024 16:10:18 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oo1-f46.google.com with SMTP id 006d021491bc7-5d5e97b8adbso3811487eaf.1 for ; Sat, 03 Aug 2024 09:10:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722701417; x=1723306217; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wmYcJChwULjKxG+8OpmvGvGRfkVbM4VXLdC0Ya9fLK8=; b=Me730G3Y7zVqep7wRW665ttPK8iF+tnVAdmuJHgYTc4KMGTJR1lStDk+PrGioBrtVq 8xha16+i8N+kTq4xMC8vMGDzPtimWCTx8D5Gcao/JISeIHdV+gTEZ85EpAQY2sRQ+nuq unCXzB4FUGhl4If1vYWprbWNeDxjgUPIZA6KDqdUegFUEoXAxievck6IcsUcBZXoZ7Yh HEqx6HCu7Vl22EpSugIhH/PY8iR/i84Q99+2UGQIYYx8ucS8eK4c7nbymqD4DiD/R/+V fm2lHVglkQVG1s9QqMx6v4Aelp0aKGq6I1BR2xplNDCS/eiNkYQsgfq1/1xotIVt2JlH cZGA== X-Forwarded-Encrypted: i=1; AJvYcCX7ekGWI6QbfnVpyJ50TvPl5YI4O6VDNBSpDdekS47YhWnwGi+qJERVHtcxVvuy3DHMlLGgg4VpSUcOz2a/Ti799eWJRee0qXP+5JE= X-Gm-Message-State: AOJu0Yy90aFGj64jC8ELoBsuEBRnZGPJ1erQ0S2SD0AoFa+hpsE9ZeP8 kLif4tv0NJBHIpLLbv6JpEPRhZ2sgFpR7ONE2P9qVrUFUVLn/nd2C21O3NaiGuRMW8YGdzGrLKA x27WD4N9k5w+M1RB+l+NBiOY/tTvFeA== X-Google-Smtp-Source: AGHT+IE4kXP+nIYWw0LjgFbszs4RU5zEd13zSU8IF7JyEGW+/ZylxVPIkIrjwM2m5eE7mrrs41iiwhb6paTK2340O2I= X-Received: by 2002:a05:6358:6f97:b0:1a4:e539:26af with SMTP id e5c5f4694b2df-1af3bac1a68mr827152455d.27.1722701417231; Sat, 03 Aug 2024 09:10:17 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <202408030413.4734D5gd042998@donotpassgo.dyslexicfish.net> In-Reply-To: From: Alan Somers Date: Sat, 3 Aug 2024 10:10:05 -0600 Message-ID: Subject: Re: RFC: ACLs on fusefs To: Shawn Webb Cc: Jamie Landeg-Jones , freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4WbnjQ4qy2z4WX4 On Sat, Aug 3, 2024 at 10:00=E2=80=AFAM Shawn Webb wrote: > > On Sat, Aug 03, 2024 at 09:03:38AM -0600, Alan Somers wrote: > > On Fri, Aug 2, 2024 at 10:13=E2=80=AFPM Jamie Landeg-Jones wrote: > > > > > > Alan Somers wrote: > > > > > > > TLDR; > > > > how useful would it be if fusefs(4) could support ACLs? > > > > > > I, personally, don't use ACLs generally, so have not missed them on > > > fusefs. > > > > > > However, I do make extensive use of XATTRs, so those are what I've > > > really missed. > > > > > > I didn't know xatrs were now supported - is that a new thing, or mayb= e > > > the client I use (borgs sshfs implementation) needs to be updated? > > > > > > Cheers, Jamie > > > > Our fusefs has supported xattrs for a long time. But the specific > > fuse file system needs support too. Looking right now, I don't see > > any support in sysutils/fusefs-sshfs . > > In fact, I have a (significantly buggy) proof-of-concept fusefs server > that stores file payload data as extended attributes. Since the tar > file format supports extended attributes, this makes data exfiltration > somewhat easier. > > Though, I suppose, since my proof-of-concept is buggy, using my > solution would make data exfil somewhat more difficult. ;-) > > Hopefully someday, I'll have the time to finish the PoC and make it > usable for production. > > PoC code: https://git.hardenedbsd.org/shawn.webb/altfs That's interesting. It looks like the opposite of what Tomoaki was describing. What's the intended application? Is it like a sort of unionfs, used to place a second file system on-top of an existing one? From nobody Sat Aug 3 17:02:07 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WbpsH08T4z5SkMy for ; Sat, 03 Aug 2024 17:02:11 +0000 (UTC) (envelope-from jamie@catflap.org) Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [IPv6:2001:19f0:7400:8808:123::1]) by mx1.freebsd.org (Postfix) with ESMTP id 4WbpsG3JSVz4bWL; Sat, 3 Aug 2024 17:02:10 +0000 (UTC) (envelope-from jamie@catflap.org) Authentication-Results: mx1.freebsd.org; none X-Catflap-Envelope-From: Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [209.250.224.51]) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5) with ESMTP id 473H27cT072032; Sat, 3 Aug 2024 18:02:07 +0100 (BST) (envelope-from jamie@donotpassgo.dyslexicfish.net) Received: (from jamie@localhost) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5/Submit) id 473H27HM072031; Sat, 3 Aug 2024 18:02:07 +0100 (BST) (envelope-from jamie) From: Jamie Landeg-Jones Message-Id: <202408031702.473H27HM072031@donotpassgo.dyslexicfish.net> Date: Sat, 03 Aug 2024 18:02:07 +0100 Organization: Dyslexic Fish To: jamie@catflap.org, asomers@freebsd.org Cc: freebsd-hackers@freebsd.org Subject: Re: RFC: ACLs on fusefs References: <202408030413.4734D5gd042998@donotpassgo.dyslexicfish.net> In-Reply-To: User-Agent: Heirloom mailx 12.4 7/29/08 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (donotpassgo.dyslexicfish.net [209.250.224.51]); Sat, 03 Aug 2024 18:02:07 +0100 (BST) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:20473, ipnet:2001:19f0:7400::/38, country:US] X-Rspamd-Queue-Id: 4WbpsG3JSVz4bWL Alan Somers wrote: > Our fusefs has supported xattrs for a long time. But the specific > fuse file system needs support too. Looking right now, I don't see > any support in sysutils/fusefs-sshfs . Thanks for the clarifications. I note that borgbackup uses py-llfuse, which itself uses fusefs-libs version 2, so there are presumably issues with that too. Cheers, J From nobody Sat Aug 3 17:36:52 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wbqdb0CJYz5SmtR for ; Sat, 03 Aug 2024 17:37:07 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WbqdY4KzZz4f7d for ; Sat, 3 Aug 2024 17:37:05 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20230601.gappssmtp.com header.s=20230601 header.b=O9dkIxu1; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::62b) smtp.mailfrom=wlosh@bsdimp.com Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-1fc65329979so81212135ad.0 for ; Sat, 03 Aug 2024 10:37:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1722706624; x=1723311424; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=XqFfVLSqV8xdEeoiI1HrXZuwGKNreqe4roBCdmQRCHo=; b=O9dkIxu1yNAym/bMJlo06P2wHN7fzhcNItCLqxFWcBS6ROehA3ixI4KqIEX+ZLEVfG LSamBTj9daetyo/hwLMsRJ9Ui9XM/KF3pBJdr/gv1qDdn4ZJ8ZnVaB+r2+STqrHPVY6v 3aOZNWax5f/V+Q/woDz9PEljuP7VVOn+YZUw2ZQY99kp9Bu7SsYNJuheOBb68FVNwPaw BoD4ncxvp2GVNr6jdfBve4YGNEIcptfJs4ENec3DPC5L6nw5PCzxYbsrF85OlB+Dfcvl nsgxb6JTybxhddN5TpTA3PzHcHXo9ms94YJYFIub37hEPswPTxOFCVmmRGSOXU/gD4N7 WcLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722706624; x=1723311424; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=XqFfVLSqV8xdEeoiI1HrXZuwGKNreqe4roBCdmQRCHo=; b=YoyhjD/CNq7Mxo/l3Xg87ea8CLz5sT18kbcQNWWxgXDXD1B/h3jJZOovMapIHQ+ux/ kZMmAOP4NSSVVvAH4zZJzKBrYNs0QZIJhs24HHm+1CcUcqLtB2zQqA/tYPqWh58UOkS9 Tuv58wpFWny/ETHIoZBBPUF9wAbPVR3gNSOZRDK40lwIIDNpnYFS29u0ffUUOFIFTiW4 LXolGF6aI+VnKzVDsN35khX7sk4/SCmZALefvfeZ5BjI1vNT5tYi8Xp5ke27mNfAJuFP MN4ZwZKRmunjZbAbsomCsDsgnxPiFFCk+qO9Vjpm3qMJDlUWUfbFL/fIQ/VQrAoyQhDn aFcg== X-Forwarded-Encrypted: i=1; AJvYcCXBQE6NBF1Eom0IVhSFKDKRD+94/Vx/38Gx7IK6LSQZm4NgeOwMfbWlfMruR0aTJarBC4eS44PWyeZMx9ahF20aQK5TX0RPcqnD1lc= X-Gm-Message-State: AOJu0YwThUIKY4tGqAsL8fS1yUcy/ZweixMPBtGXZYCai59CM7MXMECL eDIEUpbHQwZ6NM/Jpa8leEDA3rwDiChw0kKgmsbgr0GINO9Ig5CkNtWSvLgS8bDu/z1bd9sOt1S sTT2efK6pbo6Za5JJsJ/bJKGzC6xhz5ovUHWTBA== X-Google-Smtp-Source: AGHT+IHZ9KGw4MfUFPVfv9ejPrn4ecskMJIqGSNLSfNsMYDMH93+wDmUwRz7fw1e91kcmHm0CteS+v5KJMBl+zCMOzo= X-Received: by 2002:a17:902:ec8a:b0:1fd:a1d2:c03b with SMTP id d9443c01a7336-1ff5748f3aamr86327135ad.59.1722706623885; Sat, 03 Aug 2024 10:37:03 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Sat, 3 Aug 2024 11:36:52 -0600 Message-ID: Subject: Re: The Case for Rust (in the base system) To: Shawn Webb Cc: Alan Somers , FreeBSD Hackers , Scott Long , "Goran Meki??" Content-Type: multipart/alternative; boundary="0000000000005e8b7a061ecae4d2" X-Spamd-Bar: -- X-Spamd-Result: default: False [-3.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20230601.gappssmtp.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ARC_NA(0.00)[]; MISSING_XM_UA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::62b:from]; R_SPF_NA(0.00)[no SPF record]; DMARC_NA(0.00)[bsdimp.com]; RCVD_TLS_LAST(0.00)[]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; DKIM_TRACE(0.00)[bsdimp-com.20230601.gappssmtp.com:+] X-Rspamd-Queue-Id: 4WbqdY4KzZz4f7d --0000000000005e8b7a061ecae4d2 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, Jul 31, 2024 at 12:51=E2=80=AFPM Shawn Webb wrote: > On Wed, Jul 31, 2024 at 11:01:17AM -0600, Warner Losh wrote: > > On Wed, Jul 31, 2024, 9:40=E2=80=AFAM Alan Somers = wrote: > > > > > On Wed, Jul 31, 2024 at 8:37=E2=80=AFAM Shawn Webb > > > > wrote: > > > > > > > > On Sat, Jan 20, 2024 at 09:51:25AM -0700, Alan Somers wrote: > > > > > In a recent thread on src-committers, we discussed the costs and > > > > > benefits of including Rust code in the FreeBSD base system. To > > > > > summarize, the cost is that it would double our build times. imp > > > > > suggested adding an additional step after buildworld for stuff th= at > > > > > requires an external toolchain. That would ease the build time > pain. > > > > > The benefit is that some tools would become easier to write, or > even > > > > > become possible. Here is a list of actual and potential Rust > projects > > > > > that could benefit from being in-tree. If anybody else has items > to > > > > > add, I suggest moving this into the project wiki: > > > > > > > > > > Stuff that could only be written in Rust if it were in base > > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > > > > > > > * ctl-exporter (I started this, but discovered that the CTL stats > API > > > is > > > > > unstable, so it can't live in ports. Instead, I had to do it i= n > C). > > > > > > > > > https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a86401= 469181a67ec34 > > > > > > > > > > * fusefs tests. Absolutely impossible to do in C. I considered > Rust, > > > but went > > > > > with C++ so they could live in base. They are too closely > coupled to > > > > > fusefs(5) to live out-of-tree. > > > > > > https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs > > > > > > > > > > * devd. Currently C++, but imp suggested a rewrite. > > > > > https://github.com/freebsd/freebsd-src/tree/main/sbin/devd > > > > > > > > > > * zfsd. Currently C++, but I've long pondered a rewrite. Using > Rust > > > would > > > > > make it more testable. > > > > > > https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd > > > > > > > > > > * nscd. Currently C, but confusing and with no test coverage. > I've > > > > > contemplated a rewrite myself, but I don't want to do it in C. > > > > > https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd > > > > > > > > > > * The userland portion of the 802.11ac and Lightning stacks. > scottl > > > suggested > > > > > that these were good candidates for Rust. > > > > > > > > > > * freebsd-kpi-r14-0 . https://crates.io/crates/freebsd-kpi-r14-0 > > > > > > > > > > Stuff that can live in ports, but would be nicer in base > > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > > > > > > > * gstat-rs https://crates.io/crates/gstat > > > > > > > > > > * geom-exporter (I've started this, but haven't published it) > > > > > > > > > > * nfs-exporter https://crates.io/crates/freebsd-nfs-exporter > > > > > > > > > > * virtiofsd-rs . Nobody has yet tried to port it to FreeBSD. Bu= t > if > > > the > > > > > connection to bhyve(8) is too intimate, it might be hard to do = in > > > ports. > > > > > https://gitlab.com/virtio-fs/virtiofsd > > > > > > > > > > * jail-exporter https://crates.io/crates/jail_exporter > > > > > > > > > > * Various jail managers have been attempted in Rust. I think the= se > > > are fine in > > > > > ports, but others like Goran Mekic have opined that they should > be > > > moved to > > > > > base instead. > > > > > > > > > > * musikid's pjdfstest rewrite. I think it would be great to star= t > > > using this > > > > > to test the base system's file systems. If the tests themselve= s > > > lived in > > > > > base, they would be easier to sync with file system development= . > > > > > https://github.com/musikid/pjdfstest > > > > > > > > > > * pf-rs. I suspect that the API isn't very stable. > > > > > https://crates.io/crates/pf-rs > > > > > > > > > > * benchpmc. The pmc counter names changes between releases. > > > > > https://crates.io/crates/benchpmc > > > > > > > > > > FreeBSD-related applications that are just fine in ports > > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > > > > > > > * fsx-rs. Unlike pjdfstest, this only tests datapath APIs. Thos= e > are > > > usually > > > > > more stable than control path APIs, so I think there's little t= o > be > > > gained by > > > > > moving this into base. https://crates.io/crates/fsx > > > > > > > > > > * ztop. It uses ZFS's kstats sysctl interface, which is pretty > stable. > > > > > https://crates.io/crates/ztop > > > > > > > > > > * iocage-provision https://crates.io/crates/iocage-provision > > > > > > > > > > * rsblk https://crates.io/crates/rsblk > > > > > > > > > > * xfuse https://github.com/KhaledEmaraDev/xfuse > > > > > > > > > > Other FreeBSD-related libraries in Rust > > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > > Just see the list at https://crates.io/keywords/freebsd > > > > > > > > > > > > > One new data point: DARPA is looking to rewrite a significant amoun= t > > > > of C code to Rust with their "Translating All C to Rust (TRACTOR)" > > > > project: > > > > https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view > > > > > > Interesting. And since you bring it up, I have two new data points > myself: > > > > > > * ctld: while working on some bugs in ctld, I had trouble > > > understanding the config file parsing. So I rewrote that part in > > > Rust, just to help my understanding. Later, I rewrote the XML > > > parsing, too. Then I rewrote the LUN creation and deletion, just to > > > see how hard it would be. All of those parts take about 5x fewer SLO= C > > > in Rust than in C, and they're less buggy, too. Config file parsing > > > is more consistent, no memory leaks, etc. Alas, I'm not planning to > > > finish this project, since the base system doesn't allow Rust and ctl= d > > > is too tightly coupled to ctl to live in ports. > > > > > > > Cool. Still waiting for anybody to take me up on the offer to do build > > system integration. Since the Rust advocates can't get even this basic > step > > done for review, it's going to be impossible to have Rust in the base. > This > > isn't even integrate rust compiler like we do with llvm, but with > external > > Rust toolchain. > > > > Until somebody steps up for this task, the status quo can't possibly > change. > > Back at the FreeBSD Developer Summit at this last BSDCan, there was > interest in supporting optional external toolchains in the src build > framework. You had mentioned you would be happy to mentor someone, but > not do the nitty gritty yourself. > Yes. I've made that offer half a dozen times now. > I could carve off some time in September to be the primary developer, > doing the nitty gritty work. Would you be comfortable answering my > questions, should I have any? > You bet. I'd love to see progress made on this front. But I'm in Ireland on vacation half the month, so there may be timing issues. Other than that, I'm happy. > Also: what work (or research), if any, has been done on the concept of > external toolchain support for optional components in the FreeBSD > source tree? Am I starting afresh or building upon existing work? > We already have clang and gcc external tool chains, so there's a proven mechanism for that. But there's not a good notion of the concept "I have a rust compiler" or "I depend on rust". And there's no concept of crates or similar that rust programs use, but that will be one thorny area that we'll have to design for. Do we just pull them in and junk any notion of a reproducible build for these components into the future (since any crate can go away), or do we have a way to build up our own set of crates in the tree that the optional components depend on. How do we do change management on that if we have multiple programs that depend on a crate that's updated? how do we keep things fresh while not having update cascades be too burdensome a task. How does this tie into pkgbase? These are the things to think about. We don't need to solve all of them, but the Rust ecosystem is quite a bit different than the C ecosystem in the details of a number of these points, so we have to address them if we want to use Rust in base with the same traits as all the other bits in base today (or we need to have a thoughtful discussion on paradigm shift and settle on that). To my thinking, pkgbase might be a good way to segregate crates that are build from the base tree and express dependencies on optional components that use it, and have the ultimate dependency be a pkg from ports. These questions and design points aren't hard and aren't designed to block anything, but a bare minimum of what we need to articulate is the vision for these components. Likely a design document that spells these out in some degree of detail (or that we punt in this phase) would be good as well. I can help with that as well. Warner > Thanks, > > -- > Shawn Webb > Cofounder / Security Engineer > HardenedBSD > > Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 > > https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/0= 3A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc > --0000000000005e8b7a061ecae4d2 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Wed, Jul 31, 2024 at 12:51=E2=80= =AFPM Shawn Webb <shawn.we= bb@hardenedbsd.org> wrote:
On Wed, Jul 31, 2024 at 11:01:17AM -0600, Warner Losh wro= te:
> On Wed, Jul 31, 2024, 9:40=E2=80=AFAM Alan Somers <asomers@freebsd.org> wrote:=
>
> > On Wed, Jul 31, 2024 at 8:37=E2=80=AFAM Shawn Webb <shawn.webb@hardenedbs= d.org>
> > wrote:
> > >
> > > On Sat, Jan 20, 2024 at 09:51:25AM -0700, Alan Somers wrote:=
> > > > In a recent thread on src-committers, we discussed the = costs and
> > > > benefits of including Rust code in the FreeBSD base sys= tem.=C2=A0 To
> > > > summarize, the cost is that it would double our build t= imes.=C2=A0 imp
> > > > suggested adding an additional step after buildworld fo= r stuff that
> > > > requires an external toolchain.=C2=A0 That would ease t= he build time pain.
> > > > The benefit is that some tools would become easier to w= rite, or even
> > > > become possible.=C2=A0 Here is a list of actual and pot= ential Rust projects
> > > > that could benefit from being in-tree.=C2=A0 If anybody= else has items to
> > > > add, I suggest moving this into the project wiki:
> > > >
> > > > Stuff that could only be written in Rust if it were in = base
> > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> > > >
> > > > * ctl-exporter (I started this, but discovered that the= CTL stats API
> > is
> > > >=C2=A0 =C2=A0unstable, so it can't live in ports.=C2= =A0 Instead, I had to do it in C).
> > > >
> > http= s://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a86401469181= a67ec34
> > > >
> > > > * fusefs tests.=C2=A0 Absolutely impossible to do in C.= =C2=A0 I considered Rust,
> > but went
> > > >=C2=A0 =C2=A0with C++ so they could live in base.=C2=A0 = They are too closely coupled to
> > > >=C2=A0 =C2=A0fusefs(5) to live out-of-tree.
> > > >=C2=A0 =C2=A0= https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs > > > >
> > > > * devd.=C2=A0 Currently C++, but imp suggested a rewrit= e.
> > > >=C2=A0 =C2=A0https://gi= thub.com/freebsd/freebsd-src/tree/main/sbin/devd
> > > >
> > > > * zfsd.=C2=A0 Currently C++, but I've long pondered= a rewrite.=C2=A0 Using Rust
> > would
> > > >=C2=A0 =C2=A0make it more testable.
> > > >=C2=A0 =C2=A0h= ttps://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd
> > > >
> > > > * nscd.=C2=A0 Currently C, but confusing and with no te= st coverage.=C2=A0 I've
> > > >=C2=A0 =C2=A0contemplated a rewrite myself, but I don= 9;t want to do it in C.
> > > >=C2=A0 =C2=A0https:= //github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd
> > > >
> > > > * The userland portion of the 802.11ac and Lightning st= acks.=C2=A0 scottl
> > suggested
> > > >=C2=A0 =C2=A0that these were good candidates for Rust. > > > >
> > > > * freebsd-kpi-r14-0 .=C2=A0 https://cr= ates.io/crates/freebsd-kpi-r14-0
> > > >
> > > > Stuff that can live in ports, but would be nicer in bas= e
> > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> > > >
> > > > * gstat-rs https://crates.io/crates/gstat
> > > >
> > > > * geom-exporter (I've started this, but haven't= published it)
> > > >
> > > > * nfs-exporter https://crates.io/cr= ates/freebsd-nfs-exporter
> > > >
> > > > * virtiofsd-rs .=C2=A0 Nobody has yet tried to port it = to FreeBSD.=C2=A0 But if
> > the
> > > >=C2=A0 =C2=A0connection to bhyve(8) is too intimate, it = might be hard to do in
> > ports.
> > > >=C2=A0 =C2=A0https://gitlab.com/virtio-fs/v= irtiofsd
> > > >
> > > > * jail-exporter https://crates.io/crates/j= ail_exporter
> > > >
> > > > * Various jail managers have been attempted in Rust.=C2= =A0 I think these
> > are fine in
> > > >=C2=A0 =C2=A0ports, but others like Goran Mekic have opi= ned that they should be
> > moved to
> > > >=C2=A0 =C2=A0base instead.
> > > >
> > > > * musikid's pjdfstest rewrite.=C2=A0 I think it wou= ld be great to start
> > using this
> > > >=C2=A0 =C2=A0to test the base system's file systems.= =C2=A0 If the tests themselves
> > lived in
> > > >=C2=A0 =C2=A0base, they would be easier to sync with fil= e system development.
> > > >=C2=A0 =C2=A0https://github.com/musikid/pjdfs= test
> > > >
> > > > * pf-rs.=C2=A0 I suspect that the API isn't very st= able.
> > > >=C2=A0 =C2=A0https://crates.io/crates/pf-rs
> > > >
> > > > * benchpmc.=C2=A0 The pmc counter names changes between= releases.
> > > >=C2=A0 =C2=A0https://crates.io/crates/benchpmc
> > > >
> > > > FreeBSD-related applications that are just fine in port= s
> > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> > > >
> > > > * fsx-rs.=C2=A0 Unlike pjdfstest, this only tests datap= ath APIs.=C2=A0 Those are
> > usually
> > > >=C2=A0 =C2=A0more stable than control path APIs, so I th= ink there's little to be
> > gained by
> > > >=C2=A0 =C2=A0moving this into base. https://crates.io= /crates/fsx
> > > >
> > > > * ztop.=C2=A0 It uses ZFS's kstats sysctl interface= , which is pretty stable.
> > > >=C2=A0 =C2=A0https://crates.io/crates/ztop
> > > >
> > > > * iocage-provision=C2=A0 https://crates= .io/crates/iocage-provision
> > > >
> > > > * rsblk https://crates.io/crates/rsblk
> > > >
> > > > * xfuse=C2=A0 https://github.com/KhaledEm= araDev/xfuse
> > > >
> > > > Other FreeBSD-related libraries in Rust
> > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> > > > Just see the list at https://crates.io/keyword= s/freebsd
> > > >
> > >
> > > One new data point: DARPA is looking to rewrite a significan= t amount
> > > of C code to Rust with their "Translating All C to Rust= (TRACTOR)"
> > > project:
> > > https://sam.gov/opp/1e45d648= 886b4e9ca91890285af77eb7/view
> >
> > Interesting.=C2=A0 And since you bring it up, I have two new data= points myself:
> >
> > * ctld: while working on some bugs in ctld, I had trouble
> > understanding the config file parsing.=C2=A0 So I rewrote that pa= rt in
> > Rust, just to help my understanding.=C2=A0 Later, I rewrote the X= ML
> > parsing, too.=C2=A0 Then I rewrote the LUN creation and deletion,= just to
> > see how hard it would be.=C2=A0 All of those parts take about 5x = fewer SLOC
> > in Rust than in C, and they're less buggy, too.=C2=A0 Config = file parsing
> > is more consistent, no memory leaks, etc.=C2=A0 Alas, I'm not= planning to
> > finish this project, since the base system doesn't allow Rust= and ctld
> > is too tightly coupled to ctl to live in ports.
> >
>
> Cool. Still waiting for anybody to take me up on the offer to do build=
> system integration. Since the Rust advocates can't get even this b= asic step
> done for review, it's going to be impossible to have Rust in the b= ase. This
> isn't even integrate rust compiler like we do with llvm, but with = external
> Rust toolchain.
>
> Until somebody steps up for this task, the status quo can't possib= ly change.

Back at the FreeBSD Developer Summit at this last BSDCan, there was
interest in supporting optional external toolchains in the src build
framework. You had mentioned you would be happy to mentor someone, but
not do the nitty gritty yourself.

Yes. = I've made that offer half a dozen times now.
=C2=A0
=
I could carve off some time in September to be the primary developer,
doing the nitty gritty work. Would you be comfortable answering my
questions, should I have any?

You bet. = I'd love to see progress made on this front. But I'm in Ireland
on vacation half the month, so there may be timing issues. Other tha= n that,
I'm happy.
=C2=A0
Also: what work (or research), if any, has been done on the concept of
external toolchain support for optional components in the FreeBSD
source tree? Am I starting afresh or building upon existing work?

We already have clang and gcc external tool chai= ns, so there's a proven
mechanism for that. But there's n= ot a good notion of the concept "I have
a rust compiler"= ; or "I depend on rust". And there's no concept of crates
or similar that rust programs use, but that will be one thorny area = that
we'll have to design for. Do we just pull them in and ju= nk any notion of
a reproducible build for these components into t= he future (since any crate
can go away), or do we have a way to b= uild up our own set of crates
in the tree that the optional compo= nents depend on. How do we do change
management on that if we hav= e multiple programs that depend on a crate
that's updated? ho= w do we keep things fresh while not having update
cascades be too= burdensome a task. How does this tie into pkgbase?

These are the things to think about. We don't need to solve all of
them, but the Rust ecosystem is quite a bit different than the C ec= osystem
in the details of a number of these points, so we have to= address them
if we want to use Rust in base with the same traits= as all the other bits
in base today (or we need to have a though= tful discussion on paradigm
shift and settle on that). To my thin= king, pkgbase might be a good way
to segregate crates that are bu= ild from the base tree and express dependencies
on optional compo= nents that use it, and have the ultimate dependency
be a pkg from= ports.

These questions and design points are= n't hard and aren't designed to
block anything, but a bar= e minimum of what we need to articulate is the
vision for these c= omponents. Likely a design document that spells these
out in some= degree of detail (or that we punt in this phase) would be good
a= s well. I can help with that as well.

Warner
=C2=A0
Thanks,

--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/m= aster/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
--0000000000005e8b7a061ecae4d2-- From nobody Sat Aug 3 17:44:46 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wbqpj3Qf4z5Sn1s for ; Sat, 03 Aug 2024 17:45:01 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-vs1-f47.google.com (mail-vs1-f47.google.com [209.85.217.47]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wbqpj1pczz4gHF; Sat, 3 Aug 2024 17:45:01 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-vs1-f47.google.com with SMTP id ada2fe7eead31-4929754aee9so1987098137.1; Sat, 03 Aug 2024 10:45:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722707100; x=1723311900; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=a3CYZlWlk6vS0UjPKStFZ/e4xLWxfUs9E+bD51nFIh8=; b=TVeBpuQTU9L+AcqY56P00JFRCO/AwfRp68r5rXe4/JOx02l1tNKKEjyd89kC11A9S7 ZeOBF/19wqpeFdJZVIGNi+WbzBBbvIZmr/eVQeMNx+e5mFwrJvl8Y2N1hTkdMfNp9cqc BesaaBmFgJZHqWCA/sSfOs1gdqk12gLyYs2wrldAMhVyIjd0K/4E0Y5nCBKLiEAaYVhx rQ3R0H+HbtUwSatMVMlXhn17L6J6Hm6/rgT7UeYmQpDRMBYAk2ktKO6c3++0yEXDLLRr K5dVnne+taUu3/JIyvKCs7ZuzAfc7qj5FRQF6Ly/6EATxCbevlz4ACOmwA701hGiS4oA Aciw== X-Forwarded-Encrypted: i=1; AJvYcCXyaOHtX2WJVIyKrVaVdVVjYnRIDLLFUmQFjmzaNuKC428GL1TbyVq5I1zk5yyMFwDTtidahSR2fJa+TVQSr7wkas+NYgJ8vmjZlJs= X-Gm-Message-State: AOJu0Yw+qmIl2cMhkV9E0s0uMhy2ThJivjwHxLHUngR0myvnUJOwkMGg aP4NKJFKbBqKuVeP5yqP4egR8CrYW6lSADlmI4Y7WF+xI9X5bjpW0oLnHOIjntZUxFcYR94nSWe Jm78BTGpf2rMdTFqOdQbosEGdEbwNpA== X-Google-Smtp-Source: AGHT+IEhFVJ8m4sLwbJRKBHgRUaJeIttpjGhqn0Z5tWXbDGVJrYxNyM8KBtRpMfQmYnUGTov/W4/2yKacM92XFA8rZg= X-Received: by 2002:a05:6102:3ec2:b0:48f:e913:7f5b with SMTP id ada2fe7eead31-4945bf0143fmr6559877137.19.1722707099819; Sat, 03 Aug 2024 10:44:59 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <202408030413.4734D5gd042998@donotpassgo.dyslexicfish.net> <202408031702.473H27HM072031@donotpassgo.dyslexicfish.net> In-Reply-To: <202408031702.473H27HM072031@donotpassgo.dyslexicfish.net> From: Alan Somers Date: Sat, 3 Aug 2024 11:44:46 -0600 Message-ID: Subject: Re: RFC: ACLs on fusefs To: Jamie Landeg-Jones Cc: Alan Somers , FreeBSD Hackers Content-Type: multipart/alternative; boundary="000000000000bca6e8061ecb0028" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4Wbqpj1pczz4gHF --000000000000bca6e8061ecb0028 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Aug 3, 2024, 11:02=E2=80=AFAM Jamie Landeg-Jones wrote: > Alan Somers wrote: > > > Our fusefs has supported xattrs for a long time. But the specific > > fuse file system needs support too. Looking right now, I don't see > > any support in sysutils/fusefs-sshfs . > > Thanks for the clarifications. I note that borgbackup uses py-llfuse, > which itself uses fusefs-libs version 2, so there are presumably issues > with that too. > > Cheers, J > Yeah, fusefs-libs 2 cannot do acls. > --000000000000bca6e8061ecb0028 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Sat, Aug 3, 2024, 11:02=E2=80=AFAM Jamie Landeg-Jon= es <jamie@catflap.org> wrote= :
Alan Somers &l= t;asomers@freebsd.org> wrote:

> Our fusefs has supported xattrs for a long time.=C2=A0 But the specifi= c
> fuse file system needs support too.=C2=A0 Looking right now, I don'= ;t see
> any support in sysutils/fusefs-sshfs .

Thanks for the clarifications. I note that borgbackup uses py-llfuse,
which itself uses fusefs-libs version 2, so there are presumably issues
with that too.

Cheers, J

Yeah,=C2=A0 fusefs-libs 2 cannot do acls.
=
; Sat, 03 Aug 2024 20:34:34 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-vk1-f172.google.com (mail-vk1-f172.google.com [209.85.221.172]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WbvZK4tFjz42HY for ; Sat, 3 Aug 2024 20:34:33 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-vk1-f172.google.com with SMTP id 71dfb90a1353d-4f51551695cso3368503e0c.3 for ; Sat, 03 Aug 2024 13:34:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722717272; x=1723322072; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AnkBGPLDoTNDYhqwPJpFV7OdxrHsrMWlGj92DjCtt3I=; b=L1ixsjwcdSHU406lUrkUqooMTcJ0u0UD1rV5ZWKhPUojxLpfZons0PvwbpigLaEMeE n8oBgjdW1r+A2o61C0kl/GEVsTtTiYPT5NcExvLOnTEg3byPkdiyY+wq28aSItb0PqO1 LSgVIEz4oAzC6LXTUwZx2e+Nta/cm0+k+hll1IvWmBc7ZjLt+8fPJuzrdicj4tBJ+AtX OVgTrkMAlsSkVSg08D4X7HWylPil19mlFpDzPwetLqjpYx4HCVqbpkqKAal7MISHi31G B7G/5V3RzWaYYFR9IikzLnmcJUoCYTP8WWisxCvpiAMaWOZS0MPBCS9EsVe+Ktvkvisl ckpQ== X-Gm-Message-State: AOJu0Yz1qT+Uyw9TfmFuRLyjWccdtwGwLmewx7MYHw1Q+vf07UIgPeS0 v06DPKj65xGAkW6LgKhDCMFsnxj4hPq8UtBI2gsauV3xLlsEkchdpuVGV8KmmT4CotxNomI9Yv2 B+JP7hITdR6jP1HntyS6BiM3BGEX+oQ== X-Google-Smtp-Source: AGHT+IGX9kbcLKCexw/SlTZd8OLI/RLadTp7MDCf6tzawW2HL1ugpsKzKfw1HWpdhFCR4hYHj76SS/uLFPcb9JIbM4A= X-Received: by 2002:a05:6122:c91:b0:4f2:a973:8ae with SMTP id 71dfb90a1353d-4f89ff6c5f2mr8709998e0c.5.1722717272075; Sat, 03 Aug 2024 13:34:32 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <98abd3b2-5d57-439b-aafb-9a497a08e712@quip.cz> In-Reply-To: <98abd3b2-5d57-439b-aafb-9a497a08e712@quip.cz> From: Alan Somers Date: Sat, 3 Aug 2024 14:34:20 -0600 Message-ID: Subject: Re: auditd not logging file operations thru NFS To: Miroslav Lachman <000.fbsd@quip.cz> Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4WbvZK4tFjz42HY On Sat, Aug 3, 2024 at 1:37=E2=80=AFPM Miroslav Lachman <000.fbsd@quip.cz> = wrote: > > On 03/08/2024 17:06, Alan Somers wrote: > > On Sat, Aug 3, 2024 at 7:52=E2=80=AFAM Miroslav Lachman <000.fbsd@quip.= cz> wrote: > >> > >> I have auditd running on two machines with a configuration to monitor > >> all changes in files on the filesystem. If I write to the file from th= e > >> localhost (on machine A), everything works and the record appears in t= he > >> logfile. However, if a directory is exported via NFS, mounted on anoth= er > >> machine (machine B), and I write to the file on the machine B, then no > >> record appears in the audit log on machine A. > >> Is there a way to configure auditd to log these events too? > >> > >> /etc/security/audit_user is empty > >> /etc/security/audit_event is default > >> /etc/security/audit_class is default > >> > >> # cat /etc/security/audit_control > >> # > >> # $FreeBSD: releng/10.3/contrib/openbsm/etc/audit_control 293161 > >> 2016-01-04 16:32:21Z brueffer $ > >> # > >> dir:/var/audit > >> dist:off > >> flags:lo,aa,ad,fw,fm,fc,fd > >> minfree:5 > >> naflags:lo,aa,ad,fw,fm,fc,fd > >> policy:cnt,argv > >> filesz:50M > >> expire-after:600s > >> > >> Kind regards > >> Miroslav Lachman > > > > Nope. That's a known limitation of auditd. It works at a higher > > level than nfs. If you want to audit operations over NFS, currently > > you must run auditd on the NFS client. There was actually a GSoC > > project that tried to fix this a few years ago, but it ran into too > > many problems and was ultimately unsuccessful. > > Thank you very much for the explanation. > I wouldn't have thought that auditd doesn't support it. From my point of > view, it's a pretty fundamental bug. If I'm deploying a system for > auditing access and changes, I would expect it to be able to record > really all accesses to files, but this way all it takes is "some daemon" > (NFS) and changes to files can take place without there being an audit > trail. > Of course, I don't understand these system issues at all and have no > idea how difficult it is to fix this deficiency, but I would be happy if > the fix could be sponsored by the FreeBSD Foundation. > And I would also like to see it mentioned in the manual and handbook. > Nowhere did I find mention that the inability to log events through NFS > is a long known problem. > > In this case, fortunately I have access to both machines - the NFS > server and the NFS client, so I can take audit logs from the client as > well, but in some other cases I am managing an NFS server for foreign > clients where I am not able to set up auditd on the client side. > > Kind regards > Miroslav Lachman Yep yep yep. It's definitely surprising. Too bad it isn't easier to fix. From nobody Sat Aug 3 22:53:39 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wbyg31HSJz5RVDf for ; Sat, 03 Aug 2024 22:53:51 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wbyg23jR1z4LY1; Sat, 3 Aug 2024 22:53:49 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; none Received: from kalamity.joker.local (123-1-21-232.area1b.commufa.jp [123.1.21.232]) (authenticated bits=0) by www121.sakura.ne.jp (8.17.1/8.17.1/[SAKURA-WEB]/20201212) with ESMTPA id 473Mrd9f042178; Sun, 4 Aug 2024 07:53:39 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=dec.sakura.ne.jp; s=s2405; t=1722725619; bh=d5Sof0pfDlzTA+ACZ4f47a183iqJB+F6FQyVapNR+1E=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=KrTtjxB+04k2K0/bHuGfvP/RSIiD+6RXDHz2ci6b1uOkSdcM+zK9jdIULgeM2e2GG NWZRdhXh1pp0Q5/Bu9QpO5WGFkLodtORdaqCcidF6EkmappIxDkxUtoXGmLqaCePzO 23q+GzGoTsV1CEgPeDx68ChirzcSporE2VnU2SLI= Date: Sun, 4 Aug 2024 07:53:39 +0900 From: Tomoaki AOKI To: freebsd-hackers@freebsd.org Cc: Alan Somers , Ka Ho Ng Subject: Re: RFC: ACLs on fusefs Message-Id: <20240804075339.740897064c00d731b102e1d9@dec.sakura.ne.jp> In-Reply-To: References: <20240803131134.20f61fe590d1929a1733abc3@dec.sakura.ne.jp> Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd14.1) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-2022-JP Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP] X-Rspamd-Queue-Id: 4Wbyg23jR1z4LY1 Resent, as this didn't sent from ML system for more than 4 hrs. To be sure, readded original recipients. Agreed. So options a) and b) I described should not be applicable at least for now. Although I've described (just for completeness) option b), it should not be accepted. Could cause severe and unrecoverable problems in the wild. (Not noticing about lost EAs/ACLs.) For option c), maybe we must decide what error to be returned when underlying filesystem does not support storing requested EAs/ACLs to notify that the user should use any kind of archivers which supports storing wanted EAs/ACLs, or copy/move to different filesystem that support them. Maybe any of 13 EACCESS, 22 EINVAL, 45-47 E*NOSUPPORT, 79 EFTYPE, 87 ENOATTR? 87 seems to fit for EAs, but not for ACLs. On Sat, 3 Aug 2024 08:59:15 -0600 Alan Somers wrote: > Storing ACLs in hidden files is beyond the scope of what I had in > mind. That would be an entirely new challenge. And it wouldn't be > possible to do it correctly, because the file system server is still > responsible for the inheritance of default ACLs. For the present, all > that I have in mind is file systems that can store ACLs natively. > > On Fri, Aug 2, 2024 at 10:11$B".(BPM Tomoaki AOKI wrote: > > > > I'm not understanding the implementation of fusefs, but possible > > problem to be considered is the levels of suuports for EAs and ACLs of > > each implemented filesystems. > > > > What I recall is the implementation of EAs on OS/2. > > IIRC, HPFS natively supported EAs and WPS/Presentation manager-specific > > attributes (like registry on recent Windoze). > > To support both on FAT, OS/2 used 2 hidden system files named > > EA_DATA.SF and WP_ROOT.SF respectively. > > > > What's needed to consider would be: > > a) Store EAs and ACLs on hidden system files with fixed name > > on the root directory of filesystems which don't support > > EAs and/or ACLs. > > > > b) Silently ignore unsupportd attributes/ACLs and store supported > > ones only without returning error. > > > > c) Error out if any of unsupported attributes/ACLs are specified. > > > > Maybe b) would be fatally problematic, as admins who don't understand / > > don't want to learn about implementations in deep would missingly > > believe every attribs/ACLs are completely and flawlessly stored even on > > filesystems which does not support them. > > > > a) would be best, if ALL FUSEFS GUYS ON OTHER OS'es agree with the > > specific filename and its internals, promissing to implement theirs as > > such, too. Unfortunately, maybe not realistic, unless ISO/IEC or POSIX > > mandates the support and standardize the name and formats of special > > file. > > > > > > On Fri, 2 Aug 2024 22:53:05 -0400 > > Ka Ho Ng wrote: > > > > > Having said that, I am not sure if the FUSE protocol itself is extensible > > > to accommodate the needs to directly implement the counterparts of our > > > existing ACL syscalls. Otherwise XATTR tunneling for both NFSv4 and POSIX > > > 1.e on FUSE might be the only way to go. > > > > > > May I know if there're any users of the XATTR approach besides the > > > e2fsprogs/fuse2fs implementation of the EXT4 filesystem? > > > > > > Ka Ho > > > > > > On Fri, Aug 2, 2024, 22:34 Ka Ho Ng wrote: > > > > > > > I would rather see the support of XATTR and NFSv4 ACL being two orthogonal > > > > things, just like how it's being worked out on ZFS. > > > > > > > > On Fri, Aug 2, 2024, 19:58 Alan Somers wrote: > > > > > > > >> TLDR; > > > >> how useful would it be if fusefs(4) could support ACLs? > > > >> > > > >> The current state of fusefs is that while it has full support for > > > >> extended attributes, it lacks any support for ACLs. If a file system > > > >> image contains files with ACL entries, the user can look them up with > > > >> getextattr, but they'll just look like a binary blob. getfacl won't > > > >> work at all. And the file system won't be able to enforce the ACLs > > > >> during VOP_ACCESS. > > > >> > > > >> Fixing this situation for posix.1e ACLs would require three things: > > > >> * A good test suite for posix.1e ACLs. pjdfstest has some tests, but > > > >> it's incomplete. > > > >> * An example file system to use for testing the kernel driver. It > > > >> isn't sufficient for the example file system merely to support xattrs, > > > >> because the file system server must also enforce inheritance of > > > >> default ACLs. > > > >> * The actual kernel support. Enforcing ACLs during VOP_ACCESS must be > > > >> done within the kernel, not the server. The important parts are > > > >> already in sub_acl_posix1e.c. The fusefs test suite would need a few > > > >> more test cases for VOP_GETACL and VOP_SETACL, but wouldn't need to > > > >> test any of the fancy stuff, like inheritance or enforcement during > > > >> access. > > > >> > > > >> Fixing the situation for NFSv4 ACLs would require the above, and also > > > >> a small extension to the fusefs protocol. > > > >> > > > >> All of the above might make a good GSoC project. But is it worth our > > > >> time? How many real-world users would benefit? Alternatively, doing > > > >> just the kernel support would be fairly easy. That would be too small > > > >> for GSoC. But we could easily overlook important bugs if we don't do > > > >> the other steps, too. > > > >> > > > >> So my question is: is this worthwhile? Does anybody know of a > > > >> real-world workload that would benefit? > > > >> > > > >> > > > > > > -- > > Tomoaki AOKI -- Tomoaki AOKI From nobody Sat Aug 3 23:31:15 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WbzVP6n7vz5RYDf for ; Sat, 03 Aug 2024 23:31:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WbzVP3VyWz4SGj; Sat, 3 Aug 2024 23:31:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: from tom.home (kib@localhost [127.0.0.1] (may be forged)) by kib.kiev.ua (8.18.1/8.18.1) with ESMTP id 473NVGIn037119; Sun, 4 Aug 2024 02:31:19 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 473NVGIn037119 Received: (from kostik@localhost) by tom.home (8.18.1/8.18.1/Submit) id 473NVFsM037117; Sun, 4 Aug 2024 02:31:15 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 4 Aug 2024 02:31:15 +0300 From: Konstantin Belousov To: Warner Losh Cc: Alan Somers , FreeBSD Hackers Subject: Re: The Case for Rust (in the base system) Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=4.0.1 X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on tom.home X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-Rspamd-Queue-Id: 4WbzVP3VyWz4SGj On Sat, Aug 03, 2024 at 11:36:52AM -0600, Warner Losh wrote: > We already have clang and gcc external tool chains, so there's a proven > mechanism for that. But there's not a good notion of the concept "I have > a rust compiler" or "I depend on rust". And there's no concept of crates > or similar that rust programs use, but that will be one thorny area that > we'll have to design for. Do we just pull them in and junk any notion of > a reproducible build for these components into the future (since any crate > can go away), or do we have a way to build up our own set of crates > in the tree that the optional components depend on. How do we do change > management on that if we have multiple programs that depend on a crate > that's updated? how do we keep things fresh while not having update > cascades be too burdensome a task. How does this tie into pkgbase? > > These are the things to think about. We don't need to solve all of > them, but the Rust ecosystem is quite a bit different than the C ecosystem > in the details of a number of these points, so we have to address them > if we want to use Rust in base with the same traits as all the other bits > in base today (or we need to have a thoughtful discussion on paradigm > shift and settle on that). To my thinking, pkgbase might be a good way > to segregate crates that are build from the base tree and express > dependencies > on optional components that use it, and have the ultimate dependency > be a pkg from ports. > > These questions and design points aren't hard and aren't designed to > block anything, but a bare minimum of what we need to articulate is the > vision for these components. Likely a design document that spells these > out in some degree of detail (or that we punt in this phase) would be good > as well. I can help with that as well. Lets take a look at 'Rust in FreeBSD base' from different angle. Instead of starting with integration into the build system, lets decide which useful things can we implement in Rust. Lets ignore the problems of integrating (not yet written) code into the build, the problem of crates that should be vendored, etc. What would be a task that needs to be solved, and which implementation in Rust would utilize the hyped memory safety, type system, and all other goodies of the language? It is quite desirable, IMO, for this to be a solution for a problem, instead of rewriting existing tool in Rust. We do not want to produce more exa/eza/fd/bat/gstat-rs etc (I am not claiming that these are even slightly wrong, just that we want new stuff and not reimplementations of existing stuff). It is clearly should be a userspace thing, because producing idiomatic Rust bindings for kernel is a huge task on its own. I once thought about re-writing (sigh) rtld in Rust, perhaps with utilizing existing rtld/libc/rust stdlib for the new rtld bootstrap, but this feels like April 1 joke instead of useful thing. And it is still a rewrite. From nobody Sun Aug 4 00:41:22 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wc14b0SWnz5Rg53 for ; Sun, 04 Aug 2024 00:42:39 +0000 (UTC) (envelope-from theron.tarigo@gmail.com) Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com [IPv6:2607:f8b0:4864:20::730]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wc14Z1W9Bz4Yhd for ; Sun, 4 Aug 2024 00:42:38 +0000 (UTC) (envelope-from theron.tarigo@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=mimrF4Ug; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of theron.tarigo@gmail.com designates 2607:f8b0:4864:20::730 as permitted sender) smtp.mailfrom=theron.tarigo@gmail.com Received: by mail-qk1-x730.google.com with SMTP id af79cd13be357-7a1d81dc0beso596712385a.2 for ; Sat, 03 Aug 2024 17:42:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1722732156; x=1723336956; darn=freebsd.org; h=in-reply-to:from:content-language:references:to:subject:user-agent :mime-version:date:message-id:sender:from:to:cc:subject:date :message-id:reply-to; bh=3YsizotCreUnHpWq155YO402V8FUSet2QOHH+YJ71MA=; b=mimrF4UgWPdKOVft/X3OwzrVRzprklhdW+HdEKLHLl70sNTY4SyrBVFcixwt9nO/xZ aibmHXF+cu5jPh3aqU5HCyE3sADPDUdhZCykIvuUnao46LIGrHiE3p0bWdLSvGCkaI1I wUjTahnIASNXlCk4v7QTyuOWFr+iz3nM5afNRttTa6AfufOKQlk5OFjfCz53+syMYu5B AlHGRa7U5mYzkwHv6JZySivLuqVjtZc26bmDTeBdyWgiGKhN6je9ung4/wz6UtKZ7m5+ YgWdwu5a/A4QHALKAaH6T7TaFRYrkuZ8PK8LqHFrL7PK4cf+9zzJyviLv9ttD+hgvRZx Z/eA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722732156; x=1723336956; h=in-reply-to:from:content-language:references:to:subject:user-agent :mime-version:date:message-id:sender:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3YsizotCreUnHpWq155YO402V8FUSet2QOHH+YJ71MA=; b=EwEWw8H1AkClfiBBI8uYJxYrNVw2t1Oo4n361sOY6fWd0REQjF1afliy7JuVtY8eHV eCuXjSeGrkssunI67C4J3H1XdGaPuucFMstINtWFspkSzwd88N3mFFigUnL5+2+vgY71 VM0+E3g33xQcwRcCxFwecmglvFI/zVTxdeFc7g9pXfs3qsdF9XWx6dWla1XEXz1xz/7K MMwNBIu8q5Bs2agr8XS9rFCfz1dGpUbawQwnhG+m+y5dOp4+SBcX7NUHfenq0eeJVDUT cIFazKkIgV9OMAb7mslsqPFXGpVljfb36/XROoE+lhXE/QNamSdI1gpmT1R9ldXMIgDp ohOA== X-Gm-Message-State: AOJu0YyP/z+nLJkWZ6DNy6+dXGM5u6RlFX2sLWVwYj9nx8Chd8Bq4Eyl mzDLaCk8BzRo6EMbjICxpt4YCH2U35Q7mGfn7aFUgbI81xxsF0KDtfx1UQ== X-Google-Smtp-Source: AGHT+IFNFfxH9YIknytJ9SIn0w/ZY0MNRG7INZK6cMaMMUbBT2roDV9p6weBapTXU/S/wkCIyITnLw== X-Received: by 2002:a05:620a:3712:b0:79e:ff31:5876 with SMTP id af79cd13be357-7a34ef47009mr1017493285a.35.1722732156130; Sat, 03 Aug 2024 17:42:36 -0700 (PDT) Received: from [192.168.2.2] ([71.173.77.29]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7a34f6dca50sm215308485a.16.2024.08.03.17.42.35 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 03 Aug 2024 17:42:35 -0700 (PDT) Content-Type: multipart/alternative; boundary="------------9V5AylmlYCLzngsaGUCpsxgx" Message-ID: Date: Sat, 3 Aug 2024 20:41:22 -0400 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: The Case for Rust (in the base system) To: freebsd-hackers@freebsd.org References: Content-Language: en-US From: Theron In-Reply-To: X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.16 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_SPAM_SHORT(0.83)[0.831]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; XM_UA_NO_VERSION(0.01)[]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TAGGED_FROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TO_DN_NONE(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::730:from] X-Rspamd-Queue-Id: 4Wc14Z1W9Bz4Yhd This is a multi-part message in MIME format. --------------9V5AylmlYCLzngsaGUCpsxgx Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 8/3/24 13:36, Warner Losh wrote: > We already have clang and gcc external tool chains, so there's a proven > mechanism for that. But there's not a good notion of the concept "I have > a rust compiler" or "I depend on rust". And there's no concept of crates > or similar that rust programs use, but that will be one thorny area that > we'll have to design for. Do we just pull them in and junk any notion of > a reproducible build for these components into the future (since any crate > can go away), or do we have a way to build up our own set of crates > in the tree that the optional components depend on. How do we do change > management on that if we have multiple programs that depend on a crate > that's updated? how do we keep things fresh while not having update > cascades be too burdensome a task. How does this tie into pkgbase? > > These are the things to think about. We don't need to solve all of > them, but the Rust ecosystem is quite a bit different than the C ecosystem > in the details of a number of these points, so we have to address them > if we want to use Rust in base with the same traits as all the other bits > in base today (or we need to have a thoughtful discussion on paradigm > shift and settle on that). To my thinking, pkgbase might be a good way > to segregate crates that are build from the base tree and express > dependencies > on optional components that use it, and have the ultimate dependency > be a pkg from ports. > > These questions and design points aren't hard and aren't designed to > block anything, but a bare minimum of what we need to articulate is the > vision for these components. Likely a design document that spells these > out in some degree of detail (or that we punt in this phase) would be good > as well. I can help with that as well. > > Warner Rust must be adapted to the established practice of keeping base dependencies in-tree, not the other way around.  Whatever shift of thinking is required within Rust for cooperating with this kind of stability within a project will be good for the Rust ecosystem as well. Theron --------------9V5AylmlYCLzngsaGUCpsxgx Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit On 8/3/24 13:36, Warner Losh wrote:
We already have clang and gcc external tool chains, so there's a proven
mechanism for that. But there's not a good notion of the concept "I have
a rust compiler" or "I depend on rust". And there's no concept of crates
or similar that rust programs use, but that will be one thorny area that
we'll have to design for. Do we just pull them in and junk any notion of
a reproducible build for these components into the future (since any crate
can go away), or do we have a way to build up our own set of crates
in the tree that the optional components depend on. How do we do change
management on that if we have multiple programs that depend on a crate
that's updated? how do we keep things fresh while not having update
cascades be too burdensome a task. How does this tie into pkgbase?

These are the things to think about. We don't need to solve all of
them, but the Rust ecosystem is quite a bit different than the C ecosystem
in the details of a number of these points, so we have to address them
if we want to use Rust in base with the same traits as all the other bits
in base today (or we need to have a thoughtful discussion on paradigm
shift and settle on that). To my thinking, pkgbase might be a good way
to segregate crates that are build from the base tree and express dependencies
on optional components that use it, and have the ultimate dependency
be a pkg from ports.

These questions and design points aren't hard and aren't designed to
block anything, but a bare minimum of what we need to articulate is the
vision for these components. Likely a design document that spells these
out in some degree of detail (or that we punt in this phase) would be good
as well. I can help with that as well.

Warner

Rust must be adapted to the established practice of keeping base dependencies in-tree, not the other way around.  Whatever shift of thinking is required within Rust for cooperating with this kind of stability within a project will be good for the Rust ecosystem as well.

Theron
--------------9V5AylmlYCLzngsaGUCpsxgx-- From nobody Sun Aug 4 00:54:47 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wc1Lv114Xz5Rh53 for ; Sun, 04 Aug 2024 00:55:03 +0000 (UTC) (envelope-from joesuf4@gmail.com) Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wc1Ls47HTz4b8C for ; Sun, 4 Aug 2024 00:55:01 +0000 (UTC) (envelope-from joesuf4@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=RUBu1ON6; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of joesuf4@gmail.com designates 2a00:1450:4864:20::332 as permitted sender) smtp.mailfrom=joesuf4@gmail.com Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-4280c55e488so25966525e9.0 for ; Sat, 03 Aug 2024 17:55:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1722732898; x=1723337698; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=WM/ONP++piTyJK4vIPpRyQj3BIYLbntT3spoCoOA1zw=; b=RUBu1ON60MJY74t41A1KKigmz2lgMI5CtI1E0g9nOIez7bFIQYTiHEKNAtQbbZuL4B riWgSVOgga0H8G9CrrXbjyR9OzAyUg2VLU2zO0D3eVxpvjEHUf6rtXd1Q+PCymYxbZf+ RTM9jp7qqJh96jIgBSGvS7DJryERkjzJHfRhs6VhOBsxlJuVT6rEqMroq7q0DzyFImjd mP32L5t8U+xVmyR/JTtDOqOyzL9F0KngW8ENb/mGbiQtjwOtZxgB5RlO5IFx1CTP6nEH E07aOHqBnAl/H20bTPB8ur45dRwS9mpvZacFZm/2LNYqtw2/+/s77rVoLAMZnY9Qbm7B 5wKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722732898; x=1723337698; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=WM/ONP++piTyJK4vIPpRyQj3BIYLbntT3spoCoOA1zw=; b=A8IEAHvYL4xwKGSnTuZgqCUDV3G/Z1WmcrQr1/53XB+NIknSfZCKbBThOTFsUAJde8 oMWwDrRd/oH/SFJGDS7WvcA4+1+bI6rloRZB0LQsjgKjGOPhA0ywznyPpMIBXtNnuW4+ Hzm0zhnCPGc6Xa2VX3YNSNBfuuovCW8ho73BxuHRaNFQOb1cNlnZ4rFJZ0OAwykxWHuQ a71J32nA+5YILaq7ikxM6SZyVlRrCkaOKjFQ4TI7T3AcgmJPVir0HDNtvabwWSvLoSPu /1xUMekAeuNq3gR5V7QIbVdh6hdRjLi3s3SOnaVV+3wUT1SBfLuaWrtAvuVRG4r9zl5n rP3g== X-Gm-Message-State: AOJu0YwQDbnyfVOA+MtXLxOfrbpLLSDZ+prPD+WG7V3nPkXWbrfC0LAU e6BgBz9ROuyicdUj79Hf6cE0itAEnqnhd/64Ocx1WzaYp/YhHT96zZZ3NChIyTheegMWBUQ/gBp K8vRlhZ2ervv2KOy8L785QQkDEtA= X-Google-Smtp-Source: AGHT+IEbJw6Dp8y0BJ06WTCrtn12uAg2l+N24U8JCvG6KwhGw2AAEZ0xYHUen6JxYHbgk0rWS3b/ASn26dXYaEzkC1k= X-Received: by 2002:a05:600c:3b21:b0:428:6ac:426e with SMTP id 5b1f17b1804b1-428e4713fe7mr71519065e9.5.1722732898336; Sat, 03 Aug 2024 17:54:58 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Joe Schaefer Date: Sat, 3 Aug 2024 20:54:47 -0400 Message-ID: Subject: Re: The Case for Rust (in the base system) To: Theron Cc: freebsd-hackers@freebsd.org Content-Type: multipart/alternative; boundary="00000000000072c4d2061ed10222" X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-0.999]; NEURAL_HAM_SHORT(-0.99)[-0.991]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; ARC_NA(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; FREEMAIL_FROM(0.00)[gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; FROM_HAS_DN(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MISSING_XM_UA(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; TAGGED_RCPT(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::332:from] X-Rspamd-Queue-Id: 4Wc1Ls47HTz4b8C --00000000000072c4d2061ed10222 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Have fun with the less than zero backwards compatibility every new quarter. On Sat, Aug 3, 2024 at 8:42=E2=80=AFPM Theron wro= te: > On 8/3/24 13:36, Warner Losh wrote: > > We already have clang and gcc external tool chains, so there's a proven > > mechanism for that. But there's not a good notion of the concept "I have > a rust compiler" or "I depend on rust". And there's no concept of crates > or similar that rust programs use, but that will be one thorny area that > we'll have to design for. Do we just pull them in and junk any notion of > a reproducible build for these components into the future (since any crat= e > can go away), or do we have a way to build up our own set of crates > in the tree that the optional components depend on. How do we do change > management on that if we have multiple programs that depend on a crate > that's updated? how do we keep things fresh while not having update > cascades be too burdensome a task. How does this tie into pkgbase? > > These are the things to think about. We don't need to solve all of > them, but the Rust ecosystem is quite a bit different than the C ecosyste= m > in the details of a number of these points, so we have to address them > if we want to use Rust in base with the same traits as all the other bits > in base today (or we need to have a thoughtful discussion on paradigm > shift and settle on that). To my thinking, pkgbase might be a good way > to segregate crates that are build from the base tree and express > dependencies > on optional components that use it, and have the ultimate dependency > be a pkg from ports. > > These questions and design points aren't hard and aren't designed to > block anything, but a bare minimum of what we need to articulate is the > vision for these components. Likely a design document that spells these > out in some degree of detail (or that we punt in this phase) would be goo= d > as well. I can help with that as well. > > Warner > > > Rust must be adapted to the established practice of keeping base > dependencies in-tree, not the other way around. Whatever shift of thinki= ng > is required within Rust for cooperating with this kind of stability withi= n > a project will be good for the Rust ecosystem as well. > > > Theron > --00000000000072c4d2061ed10222 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Have fun with the less than zero backwards compatibility = every new quarter.

On Sat, Aug 3, 2024 at 8:42=E2=80=AFPM Theron <theron.tarigo@gmail.com> wr= ote:
=20 =20 =20
On 8/3/24 13:36, Warner Losh wrote:
=20
We already have clang and gcc external tool chains, so there's a proven
mechanism for that. But there's not a good notion of the concept "I have
a rust compiler" or "I depend on rust". And t= here's no concept of crates
or similar that rust programs use, but that will be one thorny area that
we'll have to design for. Do we just pull them in and junk any notion of
a reproducible build for these components into the future (since any crate
can go away), or do we have a way to build up our own set of crates
in the tree that the optional components depend on. How do we do change
management on that if we have multiple programs that depend on a crate
that's updated? how do we keep things fresh while not having update
cascades be too burdensome a task. How does this tie into pkgbase?

These are the things to think about. We don't need to solve all of
them, but the Rust ecosystem is quite a bit different than the C ecosystem
in the details of a number of these points, so we have to address them
if we want to use Rust in base with the same traits as all the other bits
in base today (or we need to have a thoughtful discussion on paradigm
shift and settle on that). To my thinking, pkgbase might be a good way
to segregate crates that are build from the base tree and express dependencies
on optional components that use it, and have the ultimate dependency
be a pkg from ports.

These questions and design points aren't hard and aren&#= 39;t designed to
block anything, but a bare minimum of what we need to articulate is the
vision for these components. Likely a design document that spells these
out in some degree of detail (or that we punt in this phase) would be good
as well. I can help with that as well.

Warner

Rust must be adapted to the established practice of keeping base dependencies in-tree, not the other way around.=C2=A0 Whatever shift of thinking is required within Rust for cooperating with this kind of stability within a project will be good for the Rust ecosystem as well.


Theron
--00000000000072c4d2061ed10222-- From nobody Sun Aug 4 03:37:36 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wc4yd1Qgbz5S0KH for ; Sun, 04 Aug 2024 03:37:45 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wc4yZ0hRcz4s5H for ; Sun, 4 Aug 2024 03:37:41 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=dec.sakura.ne.jp header.s=s2405 header.b=G9z9Txgx; dmarc=pass (policy=none) header.from=dec.sakura.ne.jp; spf=pass (mx1.freebsd.org: domain of junchoon@dec.sakura.ne.jp designates 153.125.133.21 as permitted sender) smtp.mailfrom=junchoon@dec.sakura.ne.jp Received: from kalamity.joker.local (123-1-21-232.area1b.commufa.jp [123.1.21.232]) (authenticated bits=0) by www121.sakura.ne.jp (8.17.1/8.17.1/[SAKURA-WEB]/20201212) with ESMTPA id 4743baP2080746; Sun, 4 Aug 2024 12:37:37 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=dec.sakura.ne.jp; s=s2405; t=1722742657; bh=AmblL0NKxyRpUSQXNY+DcfhGlkNWMMBSddXBEye9me8=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=G9z9TxgxPEWCxOo/ge0ujhAzr72gSXxBTNe12ls+kg0YG461W6HQyKapCP3+h87AQ sfJ6Vk/J+dMEh3TGR18Dt4Jmc/0eBCIrz++/b/cwIj+PsqDeHeUg4odZKXk858mRwp 4nnK0t2qTgrmzh1pRdFsR8dR0M5MYA70ZQOHJrAU= Date: Sun, 4 Aug 2024 12:37:36 +0900 From: Tomoaki AOKI To: Theron Cc: freebsd-hackers@freebsd.org Subject: Re: The Case for Rust (in the base system) Message-Id: <20240804123736.3db9da477c91659775d3c530@dec.sakura.ne.jp> In-Reply-To: References: Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd14.1) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spamd-Bar: ++ X-Spamd-Result: default: False [2.20 / 15.00]; URIBL_RED(3.50)[dec.sakura.ne.jp:dkim]; SUSPICIOUS_URL_IN_SUSPICIOUS_MESSAGE(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; HAS_ANON_DOMAIN(0.10)[]; ONCE_RECEIVED(0.10)[]; BAD_REP_POLICIES(0.10)[]; HAS_ORG_HEADER(0.00)[]; DMARC_POLICY_ALLOW(0.00)[dec.sakura.ne.jp,none]; DKIM_TRACE(0.00)[dec.sakura.ne.jp:+]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(0.00)[dec.sakura.ne.jp:s=s2405]; RCPT_COUNT_TWO(0.00)[2]; GREYLIST(0.00)[pass,meta]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_SPF_ALLOW(0.00)[+ip4:153.125.133.16/28]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TAGGED_RCPT(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP]; TO_DN_SOME(0.00)[] X-Rspamd-Queue-Id: 4Wc4yZ0hRcz4s5H On Sat, 3 Aug 2024 20:41:22 -0400 Theron wrote: > On 8/3/24 13:36, Warner Losh wrote: > > We already have clang and gcc external tool chains, so there's a proven > > mechanism for that. But there's not a good notion of the concept "I have > > a rust compiler" or "I depend on rust". And there's no concept of crates > > or similar that rust programs use, but that will be one thorny area that > > we'll have to design for. Do we just pull them in and junk any notion of > > a reproducible build for these components into the future (since any crate > > can go away), or do we have a way to build up our own set of crates > > in the tree that the optional components depend on. How do we do change > > management on that if we have multiple programs that depend on a crate > > that's updated? how do we keep things fresh while not having update > > cascades be too burdensome a task. How does this tie into pkgbase? > > > > These are the things to think about. We don't need to solve all of > > them, but the Rust ecosystem is quite a bit different than the C ecosystem > > in the details of a number of these points, so we have to address them > > if we want to use Rust in base with the same traits as all the other bits > > in base today (or we need to have a thoughtful discussion on paradigm > > shift and settle on that). To my thinking, pkgbase might be a good way > > to segregate crates that are build from the base tree and express > > dependencies > > on optional components that use it, and have the ultimate dependency > > be a pkg from ports. > > > > These questions and design points aren't hard and aren't designed to > > block anything, but a bare minimum of what we need to articulate is the > > vision for these components. Likely a design document that spells these > > out in some degree of detail (or that we punt in this phase) would be good > > as well. I can help with that as well. > > > > Warner > > Rust must be adapted to the established practice of keeping base > dependencies in-tree, not the other way around.  Whatever shift of > thinking is required within Rust for cooperating with this kind of > stability within a project will be good for the Rust ecosystem as well. > > Theron Putting other problems (like managements of crates, moving goals) aside. At least, Rust components for base should be built with --crate-type=cdylib option [1] to allow dynamically linking with other components. Otherwise, base llvm/clang would be forced to generate Rust-style object format even for libc, I fear. [1] https://doc.rust-lang.org/reference/linkage.html -- Tomoaki AOKI From nobody Sun Aug 4 04:55:18 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wc6hb3mkHz5S5xH for ; Sun, 04 Aug 2024 04:55:43 +0000 (UTC) (envelope-from bsd-lists@bsdforge.com) Received: from udns.ultimatedns.net (udns.ultimatedns.net [24.113.41.81]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "ultimatedns.net", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wc6hb0Cpbz50VH; Sun, 4 Aug 2024 04:55:42 +0000 (UTC) (envelope-from bsd-lists@bsdforge.com) Authentication-Results: mx1.freebsd.org; none Received: from ultimatedns.net (localhost [127.0.0.1]) by udns.ultimatedns.net (8.16.1/8.16.1) with ESMTP id 4744tJce029712; Sat, 3 Aug 2024 21:55:25 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ultimatedns.net; s=mx99; t=1722747340; x=1722747940; r=y; bh=nvv8CEvpch99BYbK2lerADToZuiq0+Iy8MPa9N5O3BY=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=JjwYlC7A4TRyB3ImeLOZylA5ktJOBPvvDFjF7i045cMBHeg5CqJRvQAkHGWfKuhVs 3A2JrUmkW4TX/5OhPGllsETqix1hODbo2hHNcs48LJ+9CibUnF4BjpkilCMVd4oJxG huUhS2AKhpKWhP1QnqHkcT18vv/4banwZZ8QebKIRLp4RMPtqnHz8R+0u+Bxp0yFi9 f0fIxUifjkZG+t+FDXZSmS+B/x/dVwEsoZvYIZ8pNBnJYEmXk1EUifvPagjtK7ICDa 9RT05v1j+XXTYjq0pdur+/fdxHP8VkerlT/vheRk7JxE//hkfMzElwDKlN+nxSBR0s DWPvRIBGWFHbg== List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Date: Sat, 03 Aug 2024 21:55:18 -0700 From: Chris To: Warner Losh Cc: Shawn Webb , Alan Somers , FreeBSD Hackers , Scott Long , Goran Meki?? Subject: Re: The Case for Rust (in the base system) In-Reply-To: References: User-Agent: UDNSMS/17.0 Message-ID: <3a529dfbc17ff2c90796d0a6149d34b8@bsdforge.com> X-Sender: bsd-lists@bsdforge.com Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:11404, ipnet:24.113.0.0/16, country:US] X-Rspamd-Queue-Id: 4Wc6hb0Cpbz50VH On 2024-08-03 10:36, Warner Losh wrote: > On Wed, Jul 31, 2024 at 12:51 PM Shawn Webb > wrote: > >> On Wed, Jul 31, 2024 at 11:01:17AM -0600, Warner Losh wrote: >> > On Wed, Jul 31, 2024, 9:40 AM Alan Somers wrote: >> > >> > > On Wed, Jul 31, 2024 at 8:37 AM Shawn Webb > > >> > > wrote: >> > > > >> > > > On Sat, Jan 20, 2024 at 09:51:25AM -0700, Alan Somers wrote: >> > > > > In a recent thread on src-committers, we discussed the costs and >> > > > > benefits of including Rust code in the FreeBSD base system. To >> > > > > summarize, the cost is that it would double our build times. imp >> > > > > suggested adding an additional step after buildworld for stuff that >> > > > > requires an external toolchain. That would ease the build time >> pain. >> > > > > The benefit is that some tools would become easier to write, or >> even >> > > > > become possible. Here is a list of actual and potential Rust >> projects >> > > > > that could benefit from being in-tree. If anybody else has items >> to >> > > > > add, I suggest moving this into the project wiki: >> > > > > >> > > > > Stuff that could only be written in Rust if it were in base >> > > > > =========================================================== >> > > > > >> > > > > * ctl-exporter (I started this, but discovered that the CTL stats >> API >> > > is >> > > > > unstable, so it can't live in ports. Instead, I had to do it in >> C). >> > > > > >> > > >> https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a86401469181a67ec34 >> > > > > >> > > > > * fusefs tests. Absolutely impossible to do in C. I considered >> Rust, >> > > but went >> > > > > with C++ so they could live in base. They are too closely >> coupled to >> > > > > fusefs(5) to live out-of-tree. >> > > > > >> https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs >> > > > > >> > > > > * devd. Currently C++, but imp suggested a rewrite. >> > > > > https://github.com/freebsd/freebsd-src/tree/main/sbin/devd >> > > > > >> > > > > * zfsd. Currently C++, but I've long pondered a rewrite. Using >> Rust >> > > would >> > > > > make it more testable. >> > > > > >> https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd >> > > > > >> > > > > * nscd. Currently C, but confusing and with no test coverage. >> I've >> > > > > contemplated a rewrite myself, but I don't want to do it in C. >> > > > > https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd >> > > > > >> > > > > * The userland portion of the 802.11ac and Lightning stacks. >> scottl >> > > suggested >> > > > > that these were good candidates for Rust. >> > > > > >> > > > > * freebsd-kpi-r14-0 . https://crates.io/crates/freebsd-kpi-r14-0 >> > > > > >> > > > > Stuff that can live in ports, but would be nicer in base >> > > > > ======================================================== >> > > > > >> > > > > * gstat-rs https://crates.io/crates/gstat >> > > > > >> > > > > * geom-exporter (I've started this, but haven't published it) >> > > > > >> > > > > * nfs-exporter https://crates.io/crates/freebsd-nfs-exporter >> > > > > >> > > > > * virtiofsd-rs . Nobody has yet tried to port it to FreeBSD. But >> if >> > > the >> > > > > connection to bhyve(8) is too intimate, it might be hard to do in >> > > ports. >> > > > > https://gitlab.com/virtio-fs/virtiofsd >> > > > > >> > > > > * jail-exporter https://crates.io/crates/jail_exporter >> > > > > >> > > > > * Various jail managers have been attempted in Rust. I think these >> > > are fine in >> > > > > ports, but others like Goran Mekic have opined that they should >> be >> > > moved to >> > > > > base instead. >> > > > > >> > > > > * musikid's pjdfstest rewrite. I think it would be great to start >> > > using this >> > > > > to test the base system's file systems. If the tests themselves >> > > lived in >> > > > > base, they would be easier to sync with file system development. >> > > > > https://github.com/musikid/pjdfstest >> > > > > >> > > > > * pf-rs. I suspect that the API isn't very stable. >> > > > > https://crates.io/crates/pf-rs >> > > > > >> > > > > * benchpmc. The pmc counter names changes between releases. >> > > > > https://crates.io/crates/benchpmc >> > > > > >> > > > > FreeBSD-related applications that are just fine in ports >> > > > > ========================================================= >> > > > > >> > > > > * fsx-rs. Unlike pjdfstest, this only tests datapath APIs. Those >> are >> > > usually >> > > > > more stable than control path APIs, so I think there's little to >> be >> > > gained by >> > > > > moving this into base. https://crates.io/crates/fsx >> > > > > >> > > > > * ztop. It uses ZFS's kstats sysctl interface, which is pretty >> stable. >> > > > > https://crates.io/crates/ztop >> > > > > >> > > > > * iocage-provision https://crates.io/crates/iocage-provision >> > > > > >> > > > > * rsblk https://crates.io/crates/rsblk >> > > > > >> > > > > * xfuse https://github.com/KhaledEmaraDev/xfuse >> > > > > >> > > > > Other FreeBSD-related libraries in Rust >> > > > > ======================================= >> > > > > Just see the list at https://crates.io/keywords/freebsd >> > > > > >> > > > >> > > > One new data point: DARPA is looking to rewrite a significant amount >> > > > of C code to Rust with their "Translating All C to Rust (TRACTOR)" >> > > > project: >> > > > https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view >> > > >> > > Interesting. And since you bring it up, I have two new data points >> myself: >> > > >> > > * ctld: while working on some bugs in ctld, I had trouble >> > > understanding the config file parsing. So I rewrote that part in >> > > Rust, just to help my understanding. Later, I rewrote the XML >> > > parsing, too. Then I rewrote the LUN creation and deletion, just to >> > > see how hard it would be. All of those parts take about 5x fewer SLOC >> > > in Rust than in C, and they're less buggy, too. Config file parsing >> > > is more consistent, no memory leaks, etc. Alas, I'm not planning to >> > > finish this project, since the base system doesn't allow Rust and ctld >> > > is too tightly coupled to ctl to live in ports. >> > > >> > >> > Cool. Still waiting for anybody to take me up on the offer to do build >> > system integration. Since the Rust advocates can't get even this basic >> step >> > done for review, it's going to be impossible to have Rust in the base. >> This >> > isn't even integrate rust compiler like we do with llvm, but with >> external >> > Rust toolchain. >> > >> > Until somebody steps up for this task, the status quo can't possibly >> change. >> >> Back at the FreeBSD Developer Summit at this last BSDCan, there was >> interest in supporting optional external toolchains in the src build >> framework. You had mentioned you would be happy to mentor someone, but >> not do the nitty gritty yourself. >> > > Yes. I've made that offer half a dozen times now. > > >> I could carve off some time in September to be the primary developer, >> doing the nitty gritty work. Would you be comfortable answering my >> questions, should I have any? >> > > You bet. I'd love to see progress made on this front. But I'm in Ireland > on vacation half the month, so there may be timing issues. Other than that, > I'm happy. > > >> Also: what work (or research), if any, has been done on the concept of >> external toolchain support for optional components in the FreeBSD >> source tree? Am I starting afresh or building upon existing work? >> > > We already have clang and gcc external tool chains, so there's a proven > mechanism for that. But there's not a good notion of the concept "I have > a rust compiler" or "I depend on rust". And there's no concept of crates > or similar that rust programs use, but that will be one thorny area that > we'll have to design for. Do we just pull them in and junk any notion of > a reproducible build for these components into the future (since any crate > can go away), or do we have a way to build up our own set of crates > in the tree that the optional components depend on. How do we do change > management on that if we have multiple programs that depend on a crate > that's updated? how do we keep things fresh while not having update > cascades be too burdensome a task. How does this tie into pkgbase? > > These are the things to think about. We don't need to solve all of > them, but the Rust ecosystem is quite a bit different than the C ecosystem > in the details of a number of these points, so we have to address them > if we want to use Rust in base with the same traits as all the other bits > in base today (or we need to have a thoughtful discussion on paradigm > shift and settle on that). To my thinking, pkgbase might be a good way > to segregate crates that are build from the base tree and express > dependencies > on optional components that use it, and have the ultimate dependency > be a pkg from ports. > > These questions and design points aren't hard and aren't designed to > block anything, but a bare minimum of what we need to articulate is the > vision for these components. Likely a design document that spells these > out in some degree of detail (or that we punt in this phase) would be good > as well. I can help with that as well. > > Warner I've been thinking about this when it was first brought up. I've run across something that claims: This tool is able to translate most C modules into semantically equivalent Rust code. These modules are intended to be compiled in isolation in order to produce compatible object files. We are developing several tools that help transform the initial Rust sources into idiomatic Rust. The translator focuses on supporting the C99 standard. C source code is parsed and type checked using clang before being translated by our tool. If nothing else, it might at least provide some insights. see: https://c2rust.com/ https://github.com/immunant/c2rust I hope to give a spin this weekend. --Chris > > >> Thanks, >> >> -- >> Shawn Webb >> Cofounder / Security Engineer >> HardenedBSD >> >> Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 >> >> https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc >> From nobody Sun Aug 4 10:09:35 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WcFfx3PFLz5SXXt for ; Sun, 04 Aug 2024 10:09:45 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WcFfw4b5nz47Ml; Sun, 4 Aug 2024 10:09:44 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Authentication-Results: mx1.freebsd.org; none Received: from critter.freebsd.dk (unknown [192.168.55.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by phk.freebsd.dk (Postfix) with ESMTPS id 1A1D4892B6; Sun, 04 Aug 2024 10:09:36 +0000 (UTC) Received: (from phk@localhost) by critter.freebsd.dk (8.18.1/8.16.1/Submit) id 474A9ZXW048024; Sun, 4 Aug 2024 10:09:35 GMT (envelope-from phk) Message-Id: <202408041009.474A9ZXW048024@critter.freebsd.dk> To: Konstantin Belousov cc: Warner Losh , Alan Somers , FreeBSD Hackers Subject: Re: The Case for Rust (in the base system) In-reply-to: From: "Poul-Henning Kamp" References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <48022.1722766175.1@critter.freebsd.dk> Date: Sun, 04 Aug 2024 10:09:35 +0000 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:1835, ipnet:130.225.0.0/16, country:EU] X-Rspamd-Queue-Id: 4WcFfw4b5nz47Ml -------- Konstantin Belousov writes: > On Sat, Aug 03, 2024 at 11:36:52AM -0600, Warner Losh wrote: > > [...] > > These are the things to think about. We don't need to solve all of > > them, but the Rust ecosystem is quite a bit different than the C ecosystem > > in the details of a number of these points, so we have to address them > > if we want to use Rust in base with the same traits as all the other bits > > in base today (or we need to have a thoughtful discussion on paradigm > > shift and settle on that). [...] > > [...] > > Instead of starting with integration into the build system, lets decide > which useful things can we implement in Rust. Lets ignore the problems > of integrating (not yet written) code into the build, the problem of > crates that should be vendored, etc. Stop. Just Stop! Let me address kib@'s argument first: Until we recognize the difference between "a programming language" and "an ecosystem" these debates will lead nowhere. If we continue ignore that crucial difference, we will be repeating our prior mistakes instead of learning from them. Back in the mists of time we imported Perl into the base system, based on arguments /identical/ to what we hear now about Rust. Importing Perl almost instantly turned into a disaster. We had overlooked that Perl was not just a programming language, it was an ecosystem of a language /and/ an rapidly exploding number of Perl Modules from everybody and everywhere. None of the lofty "We can (re)write all this stuff in Perl" promises came to anything, once people realized that we only imported Perl the programming language and not Perl the ecosystem (later known as CPAN). Having Perl in the tree was a net loss, and a big loss, because it created a version gap between "real Perl" and "freebsd Perl", a gap which kept growing larger over time as enthusiam for Perl in the tree evaporated. Adding Rust to FreeBSD will be the /exact/ same thing! The many differences between the Perl and Rust languages does not matter, the only thing which matters is that they are both ecosystems. We can import only the Rust language, there is /no/ way we can import the Rust ecosystem. Any discussion and all promises about "useful things we can implement in Rust" must therefore respect the following explicit restriction: You can only "use std::" and nothing else. (And probably not even all of std:: ?) That then brings me to Warner's point: Because we cannot import the ecosystem, we have two options: A) FreeBSD-Rust is only intended for use in freebsd-src, where the Rust ecosystem will /never/ be available. B) Make FreeBSD-Rust (seamlessly and continuously!) play nice with both Rust in ports and the Rust ecosystem outside ports. I have no idea if B) is even possible or what it would take, but without a design, implementation /and/ continuous maintenance of B) we (eventually) get A). In-tree-Perl was not (well-)integrated with ports, which I recall not so much as caused by technical limitation as by the users wanting the latest and greatest Perl, rather than the out-of-date FreeBSD-Perl. I expect that outcome for /any/ programming language ecosystem[1]. Assuming B) is even possible, and attempted, it is almost guaranteed to be a waste of our time and energy, and it will fizzle back to A) in a couple of years. The alternative to spending time on A) for a modest likely benefit, is to work towards a pkg-based distribution of FreeBSD, where nobody notices, or cares, that some of the pkg's are built out of src and others are built out of ports, with the benefit of the Rust ecosystem. ... or built out of C++ modules. ... or written in Python. ... or ... Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From nobody Sun Aug 4 11:35:47 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WcHZZ547Jz5S93b for ; Sun, 04 Aug 2024 11:36:06 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WcHZZ0WzRz4L1r; Sun, 4 Aug 2024 11:36:04 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; none Received: from kalamity.joker.local (123-1-21-232.area1b.commufa.jp [123.1.21.232]) (authenticated bits=0) by www121.sakura.ne.jp (8.17.1/8.17.1/[SAKURA-WEB]/20201212) with ESMTPA id 474BZlfa043559; Sun, 4 Aug 2024 20:35:58 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=dec.sakura.ne.jp; s=s2405; t=1722771358; bh=FPomr0q2qfzOVXJAN2P9Duq5CjdWdetI6hKdM8UN/tc=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=Xy8gCedIxJoObxSnL5bcADu1ZVqehVFuMDD+gV6l0QgTaXR1R/BmEmBjubrbeFe6D h/S9GvmCvDfZkEtbX6gZV5e1jjsQuBlN2YXe1qTWR1CBuRr9H2K+mH5N7i9s0KS5n9 4BCg/8JJ/SxGoBP/72q4pYGDrE3bZZmKmg4pJEYs= Date: Sun, 4 Aug 2024 20:35:47 +0900 From: Tomoaki AOKI To: "Poul-Henning Kamp" Cc: Konstantin Belousov , Warner Losh , Alan Somers , FreeBSD Hackers Subject: Re: The Case for Rust (in the base system) Message-Id: <20240804203547.4f7aeeb9760fcfba3758b28a@dec.sakura.ne.jp> In-Reply-To: <202408041009.474A9ZXW048024@critter.freebsd.dk> References: <202408041009.474A9ZXW048024@critter.freebsd.dk> Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd14.1) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP] X-Rspamd-Queue-Id: 4WcHZZ0WzRz4L1r On Sun, 04 Aug 2024 10:09:35 +0000 "Poul-Henning Kamp" wrote: > -------- > Konstantin Belousov writes: > > On Sat, Aug 03, 2024 at 11:36:52AM -0600, Warner Losh wrote: > > > [...] > > > These are the things to think about. We don't need to solve all of > > > them, but the Rust ecosystem is quite a bit different than the C ecosystem > > > in the details of a number of these points, so we have to address them > > > if we want to use Rust in base with the same traits as all the other bits > > > in base today (or we need to have a thoughtful discussion on paradigm > > > shift and settle on that). [...] > > > > [...] > > > > Instead of starting with integration into the build system, lets decide > > which useful things can we implement in Rust. Lets ignore the problems > > of integrating (not yet written) code into the build, the problem of > > crates that should be vendored, etc. > > Stop. > > Just Stop! > > Let me address kib@'s argument first: > > Until we recognize the difference between "a programming language" and > "an ecosystem" these debates will lead nowhere. > > If we continue ignore that crucial difference, we will be repeating > our prior mistakes instead of learning from them. > > Back in the mists of time we imported Perl into the base system, > based on arguments /identical/ to what we hear now about Rust. > > Importing Perl almost instantly turned into a disaster. > > We had overlooked that Perl was not just a programming language, > it was an ecosystem of a language /and/ an rapidly exploding number > of Perl Modules from everybody and everywhere. > > None of the lofty "We can (re)write all this stuff in Perl" promises > came to anything, once people realized that we only imported Perl > the programming language and not Perl the ecosystem (later known > as CPAN). > > Having Perl in the tree was a net loss, and a big loss, because it > created a version gap between "real Perl" and "freebsd Perl", a gap > which kept growing larger over time as enthusiam for Perl in the > tree evaporated. > > Adding Rust to FreeBSD will be the /exact/ same thing! > > The many differences between the Perl and Rust languages does not > matter, the only thing which matters is that they are both ecosystems. Although I've not yet popped into FreeBSD community at the moment, but I'm old enough to see what happened with base perl in sync. So I can understand what you feel. Me, too. This is why I intentionally and explicitly put aside problems other than object format in my previous post. Introducing Rust ecosystem would force us (at last) to rewrite everything possible into Rust and build/link just as usual Rust programs does (build into single binary to maximize build time check and computations to make Rust codes to be what Rust is). I think it would be unacceptable. I think introducing Rust ecosystem also requires introducing Rust philosophy, which I "feel" in reverse of Unix philosophy. Ideally, ISO/IEC nnnnn:yyyy "Programming language Rust" should be proposed/discussed/approved and published before we actually start introducing Rust "language" into FreeBSD base. Frankly, I have no severe objection to introduce codes in Rust "language" into base, if it is assured to be sanely linked with existing codes and runs without harm. And more, I don't want to see commit messages like "Rewritten whole bunch of Rust codes because language spec of Rust is changed.". > We can import only the Rust language, there is /no/ way we can > import the Rust ecosystem. Agreed, at least on current state. > Any discussion and all promises about "useful things we can implement > in Rust" must therefore respect the following explicit restriction: > > You can only "use std::" and nothing else. > (And probably not even all of std:: ?) > > > That then brings me to Warner's point: > > Because we cannot import the ecosystem, we have two options: > > A) FreeBSD-Rust is only intended for use in freebsd-src, where > the Rust ecosystem will /never/ be available. > > B) Make FreeBSD-Rust (seamlessly and continuously!) play nice with > both Rust in ports and the Rust ecosystem outside ports. > > I have no idea if B) is even possible or what it would take, but > without a design, implementation /and/ continuous maintenance of > B) we (eventually) get A). > > In-tree-Perl was not (well-)integrated with ports, which I recall > not so much as caused by technical limitation as by the users wanting > the latest and greatest Perl, rather than the out-of-date FreeBSD-Perl. > > I expect that outcome for /any/ programming language ecosystem[1]. > > Assuming B) is even possible, and attempted, it is almost guaranteed > to be a waste of our time and energy, and it will fizzle back to > A) in a couple of years. > > The alternative to spending time on A) for a modest likely benefit, > is to work towards a pkg-based distribution of FreeBSD, where nobody > notices, or cares, that some of the pkg's are built out of src and > others are built out of ports, with the benefit of the Rust ecosystem. There could be option C). C) Keep Rust itself as external toolchain maintained in ports framework, regardless how important and mandatory the Rust codes in base are. But restrict in-base Rust codes not to use external crates and use standard and vendor-imported ones as contrib. Allow external (usual) crates only for ports. Ah, maybe one more restriction could be useful. Prohibit use of "UNSAFE" blocks in base Rust codes, and keep such memory-unsafe portions of codes in *.c and *.S then link with them. It would ease debugging memory-safety related problems, as (if Rust works as Rust guys state) we not at all required to look into *.rs codes. If "UNSAFE" blocks are not prohibited, we would be forced to look into *.rs codes, too, in such a situations. > > ... or built out of C++ modules. > > ... or written in Python. > > ... or ... > > Poul-Henning > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. -- Tomoaki AOKI From nobody Sun Aug 4 11:41:04 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WcHhc0vjsz5S9ql for ; Sun, 04 Aug 2024 11:41:20 +0000 (UTC) (envelope-from x9k@charlie.emu.st) Received: from f3.bushwire.net (f3.emu.st [IPv6:2403:580c:e522:0:203:0:120:11]) by mx1.freebsd.org (Postfix) with ESMTP id 4WcHhX5JKnz4MBm for ; Sun, 4 Aug 2024 11:41:15 +0000 (UTC) (envelope-from x9k@charlie.emu.st) Authentication-Results: mx1.freebsd.org; dkim=fail ("headers rsa verify failed") header.d=emu.st header.s=2019 header.b=KrTT1Thx; dmarc=none; spf=pass (mx1.freebsd.org: domain of x9k@charlie.emu.st designates 2403:580c:e522:0:203:0:120:11 as permitted sender) smtp.mailfrom=x9k@charlie.emu.st Received: by f3.bushwire.net (Postfix, from userid 1001) id 135D64E79F; Sun, 04 Aug 2024 21:41:04 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/simple; d=emu.st; s=2019; t=1722771664; bh=rADXJ4PTeRVr2RTqg8O15358MY8=; h=Comments:Received:From:Comments:Message-ID:Content-Type:Subject: References:Mime-Version:Content-Disposition:Date:To:In-Reply-To; b=KrTT1ThxF2Fv8xRMeXqmsua4E5K9MgAmohwAZP7wUi2SdfyoRuriVNSDJlAxj960G /LZFfSka7HHb/XcSGYbc8tjaRlCEAx/S+nrF5bmEm6Ozi5ijExbA/tQH7ffg3jpThr rm+QC206OBMnGBGSiYShJ71FmlwAkfPpBnzbXLRI=bXLRI= Comments: QMDA 0.3a Received: (qmail 14034 invoked by uid 1001); 4 Aug 2024 11:41:04 -0000 From: "Mark Delany" Comments: QMDASubmit submit() 0.2.0-final Message-ID: <0.2.0-final-1722771664.023-0x79f107@qmda.emu.st> Content-Type: text/plain; charset=utf-8 Subject: Re: The Case for Rust (in the base system) References: <202408041009.474A9ZXW048024@critter.freebsd.dk> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 Content-Disposition: inline Date: Sun, 4 Aug 2024 11:41:04 +0000 To: FreeBSD Hackers In-Reply-To: <202408041009.474A9ZXW048024@critter.freebsd.dk> X-Spamd-Bar: / X-Spamd-Result: default: False [-0.72 / 15.00]; R_DKIM_REJECT(1.00)[emu.st:s=2019]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-0.86)[-0.861]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-0.26)[-0.256]; R_SPF_ALLOW(-0.20)[+ip6:2403:580c:e522::0/48]; ONCE_RECEIVED(0.10)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_NO_TLS_LAST(0.10)[]; RCPT_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:4764, ipnet:2403:5800::/27, country:AU]; RCVD_COUNT_ONE(0.00)[1]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MISSING_XM_UA(0.00)[]; TO_DN_ALL(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[emu.st]; DKIM_TRACE(0.00)[emu.st:-] X-Rspamd-Queue-Id: 4WcHhX5JKnz4MBm On 04Aug24, Poul-Henning Kamp apparently wrote: > is to work towards a pkg-based distribution of FreeBSD, where nobody One presumes that in this world, kernel modules would also be delivered as packages. After all, isn't half the argument for Rust that you can write safer kernel code with it? So the end-result is a micro-kernel that mostly loads packages from ecosystems. No reason why that can't work. (As an aside, I'm curious as to what ABI works best in this scenario. Maybe a JSON message passing micro-kernel?) Yes, I'm being a tad snarky, but the ecosystem vs language distinction raises the question as to whether Rust is the one-and-only ecosystem to support or whether it is merely one amongst many future equals? PHK may not have meant it, but if a step-function above C is presumed critical, then maybe an ecosystem model is not such a bad outcome. Mark. From nobody Sun Aug 4 13:40:29 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WcLLP4dZBz5SMJ0 for ; Sun, 04 Aug 2024 13:40:45 +0000 (UTC) (envelope-from tomek@cedro.info) Received: from mail-yw1-x112a.google.com (mail-yw1-x112a.google.com [IPv6:2607:f8b0:4864:20::112a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WcLLP2HqRz4cP1 for ; Sun, 4 Aug 2024 13:40:45 +0000 (UTC) (envelope-from tomek@cedro.info) Authentication-Results: mx1.freebsd.org; none Received: by mail-yw1-x112a.google.com with SMTP id 00721157ae682-661d7e68e89so37798117b3.0 for ; Sun, 04 Aug 2024 06:40:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cedro.info; s=google; t=1722778844; x=1723383644; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=X/JvGn+07MDHd3vI3VLA8vlB7j7PzDEpYVSAVVSDtZY=; b=huC6Ub4E7LWAVIOJsVvUArZe3HYmaR6WitV2aM+CRmESzt0+7Dt8nd6GIgtmHpH+bv tPmDDL+L0ABxXfvB73qveY0a6z49dvZFdgY1InXCpx38/EGuIGY45k2LroeEtaoX5OoB lVpv0Zwjn07DwtUgMmbAOIxV4D4nU1DgdndrV6V6rmymPorFz/BcmHUIh+WBFWKKWevK GbO9dhVHPBrrfel1rLAP/hsBjSHslsg5o9J/nA0Ph14qszpOZMZitl1jEDS0QIuujqo5 pgAomV9BIRc/eAWDNfH/jmH80nNq4aByEvocUmNqGFzoHqbpMhBsbkSsey60t8kqGlXo mrcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722778844; x=1723383644; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=X/JvGn+07MDHd3vI3VLA8vlB7j7PzDEpYVSAVVSDtZY=; b=lUDbI+Px6dbPOdtc6n0Gs3gOb5mGEahsmyrQq03HRmHTZ5lL5CUv5iXN260EYI0f0r jK1kT2cECbQ7hit4opTXQCxtD6iosC1nhrwOrHXBv9ES5wxgOb8PnJ+/rIZM3cdnoJbE NzC1F3zPf7fe+bfkUgvasLLd8AqMLUSD+YInq/zlAFDzGQCl51/5PqHw0v39PZszVvAR jkoDukpS8Ubg+pO6V3PH8yGKWqSPPqqpVfXQnC0nKYdgDKk0/5ZGj/mx4zU/nWFMwOKa AgUo+9G/le95nZN3hJl3aHcCeBpLaGPs0zvGbkkK2YC4904SsmZweyuH/ayDGNOGbr7Z Q/vQ== X-Forwarded-Encrypted: i=1; AJvYcCX3nySi6+gHFfifZt2yi/eGNZ4P9XM9oStiZi8uWAZKsv+1YX2AaOhDfVSsfVNrVG1k0QI+MoVFzE4vP0+/pxdBxB5LxGMh7d6ULiQ= X-Gm-Message-State: AOJu0YwcSIQrFELxRmEFPT/6cc3VbmtM1J+mgTawgGCSwi+6B0sFYLsn Ro2FTrOv17baC6bPZeHq7nBAiCzhUSssrmzx7hB6VMnc98hipMdZJERLOS7QkQ== X-Google-Smtp-Source: AGHT+IGaYqcecV4rRKgHUOcSRwEaWPY3BotGhe6kDRbpfk46VJevn5+h4gduXJUx2nPCw+ICM/OvLw== X-Received: by 2002:a81:8186:0:b0:651:ee07:76c with SMTP id 00721157ae682-6885493fce5mr89850837b3.15.1722778843940; Sun, 04 Aug 2024 06:40:43 -0700 (PDT) Received: from mail-yw1-f171.google.com (mail-yw1-f171.google.com. [209.85.128.171]) by smtp.gmail.com with ESMTPSA id 00721157ae682-690b5f3fe74sm2604667b3.89.2024.08.04.06.40.42 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 04 Aug 2024 06:40:42 -0700 (PDT) Received: by mail-yw1-f171.google.com with SMTP id 00721157ae682-661d7e68e89so37798027b3.0; Sun, 04 Aug 2024 06:40:42 -0700 (PDT) X-Forwarded-Encrypted: i=1; AJvYcCXAlEHR/lzbBP+x/BK+rA8nAOXwUJ/J5YDH4rBfEW272dbbTUdAqJ9O7fkJWjmsw9KhNgVM1VxdZo4CM8oE43RwrZIDlTpzHhFYala2Iuy0U+PnLUkSgAV9W3Tr6TlWazM= X-Received: by 2002:a0d:e741:0:b0:65f:d6fe:5de4 with SMTP id 00721157ae682-688554df748mr85200577b3.21.1722778842460; Sun, 04 Aug 2024 06:40:42 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <202408041009.474A9ZXW048024@critter.freebsd.dk> In-Reply-To: <202408041009.474A9ZXW048024@critter.freebsd.dk> From: Tomek CEDRO Date: Sun, 4 Aug 2024 15:40:29 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: The Case for Rust (in the base system) To: Poul-Henning Kamp Cc: Konstantin Belousov , Warner Losh , Alan Somers , FreeBSD Hackers Content-Type: multipart/alternative; boundary="000000000000ee83f5061edbb448" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4WcLLP2HqRz4cP1 --000000000000ee83f5061edbb448 Content-Type: text/plain; charset="UTF-8" On Sun, Aug 4, 2024, 12:10 Poul-Henning Kamp wrote: > -------- > Konstantin Belousov writes: > > On Sat, Aug 03, 2024 at 11:36:52AM -0600, Warner Losh wrote: > > > [...] > > > These are the things to think about. We don't need to solve all of > > > them, but the Rust ecosystem is quite a bit different than the C > ecosystem > > > in the details of a number of these points, so we have to address them > > > if we want to use Rust in base with the same traits as all the other > bits > > > in base today (or we need to have a thoughtful discussion on paradigm > > > shift and settle on that). [...] > > > > [...] > > > > Instead of starting with integration into the build system, lets decide > > which useful things can we implement in Rust. Lets ignore the problems > > of integrating (not yet written) code into the build, the problem of > > crates that should be vendored, etc. > > Stop. > > Just Stop! > > Let me address kib@'s argument first: > > Until we recognize the difference between "a programming language" and > "an ecosystem" these debates will lead nowhere. > > If we continue ignore that crucial difference, we will be repeating > our prior mistakes instead of learning from them. > > Back in the mists of time we imported Perl into the base system, > based on arguments /identical/ to what we hear now about Rust. > > Importing Perl almost instantly turned into a disaster. > > We had overlooked that Perl was not just a programming language, > it was an ecosystem of a language /and/ an rapidly exploding number > of Perl Modules from everybody and everywhere. (..) +1 :-) -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info --000000000000ee83f5061edbb448 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Sun, Aug 4, 2024, 12:10 Poul-Henning= Kamp <phk@phk.freebsd.dk> = wrote:
--------
Konstantin Belousov writes:
> On Sat, Aug 03, 2024 at 11:36:52AM -0600, Warner Losh wrote:
> > [...]
> > These are the things to think about. We don't need to solve a= ll of
> > them, but the Rust ecosystem is quite a bit different than the C = ecosystem
> > in the details of a number of these points, so we have to address= them
> > if we want to use Rust in base with the same traits as all the ot= her bits
> > in base today (or we need to have a thoughtful discussion on para= digm
> > shift and settle on that). [...]
>
> [...]
>
> Instead of starting with integration into the build system, lets decid= e
> which useful things can we implement in Rust.=C2=A0 Lets ignore the pr= oblems
> of integrating (not yet written) code into the build, the problem of > crates that should be vendored, etc.

Stop.

Just Stop!

Let me address kib@'s argument first:

Until we recognize the difference between "a programming language"= ; and
"an ecosystem" these debates will lead nowhere.

If we continue ignore that crucial difference, we will be repeating
our prior mistakes instead of learning from them.

Back in the mists of time we imported Perl into the base system,
based on arguments /identical/ to what we hear now about Rust.

Importing Perl almost instantly turned into a disaster.

We had overlooked that Perl was not just a programming language,
it was an ecosystem of a language /and/ an rapidly exploding number
of Perl Modules from everybody and everywhere.
(..)

+1 :-)=

--
--000000000000ee83f5061edbb448-- From nobody Sun Aug 4 17:55:26 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WcS0Y63D5z5Sl9P for ; Sun, 04 Aug 2024 17:55:41 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f181.google.com (mail-oi1-f181.google.com [209.85.167.181]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WcS0X4Mm1z4GXN for ; Sun, 4 Aug 2024 17:55:40 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=freebsd.org (policy=none); spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.167.181 as permitted sender) smtp.mailfrom=asomers@gmail.com Received: by mail-oi1-f181.google.com with SMTP id 5614622812f47-3daf0e73a5bso6736003b6e.0 for ; Sun, 04 Aug 2024 10:55:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722794138; x=1723398938; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=KxSIRVRW/ZpoIf8iKWngQZtYPj8n0+B+0okMCK9WXMg=; b=GprXuOgNg7ofjb+2LyZI7zH+71ch8g3HcSSQqbwNW5YRTumFFAL2LhPD1DRTJEd1Aw 9MuTi+O4fEmx61H+8pXClGdvmnqxJbvYwG10LPSqg+Cq1QDDSr/X39vF08LX9QQM5fp7 MN2oQOuy+Wi79wsKe04fnZ6+a/fzJObsK7lO0NqRSjxrDf7wQJ/UGqWzaMuEDoKIPsGP SP7yF7NPNpfQAcMWGCs0Z8/ySz3dUIO+fxTVwtCEgxVWiTcFjhVR9wbHJN/Z9XDN4mgj JDgY1mpiMKmxG3gDSifXO5vXhRAJUTgNvm4all6KgqqDwU6PMjlb/Z04FsXPK26G7/tK GSzw== X-Gm-Message-State: AOJu0Yx4STT4IHxuLN0Ge8vPmK7+tYF/pRIfp1+Vlp44NXOvTb3xAg6z ujJNbUlE69e57bbOlnYnf+Z80nXxcpvt8i9bNUytIyR25YRty8CEpRmZMi829AdyK7K720ooWst M0V4ZayIsiNQCrKATK9G2Uv9J9gH1/eE4 X-Google-Smtp-Source: AGHT+IHqnlqfOEiAmxV1qq6r002h138qDojeqz+N15B6vWh7JnGcUn0TEn7CpGLKS9WAvMoE3ZVJk5mnQsVAlLqk6Rs= X-Received: by 2002:a05:6358:5283:b0:1a8:b83a:8e26 with SMTP id e5c5f4694b2df-1af3b946fc2mr1104706055d.0.1722794138129; Sun, 04 Aug 2024 10:55:38 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 From: Alan Somers Date: Sun, 4 Aug 2024 11:55:26 -0600 Message-ID: Subject: A Demo of rust-in-base To: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.86 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.96)[-0.962]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; MIME_GOOD(-0.10)[text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : SPF not aligned (relaxed), No valid DKIM,none]; TO_DN_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_ONE(0.00)[1]; MISSING_XM_UA(0.00)[]; FREEFALL_USER(0.00)[asomers]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; RCVD_COUNT_ONE(0.00)[1]; R_DKIM_NA(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.167.181:from]; TO_DOM_EQ_FROM_DOM(0.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; RCVD_IN_DNSWL_NONE(0.00)[209.85.167.181:from] X-Rspamd-Queue-Id: 4WcS0X4Mm1z4GXN Due to all of the recent discussion of using Rust for code in the FreeBSD base, I've put together a demo of what it might look like. It demonstrates: * Interspersing Rust crates through the tree (usr.bin/nfs-exporter, cddl/usr.bin/ztop, etc) rather than in some special directory. * Build integration for all Rust crates. You can build them all with a single "cargo build" command from the top level, and test them all with a single "cargo test". * Wholly new programs written from scratch in Rust (ztop plus three Prometheus exporters) * Old programs rewritten in Rust with substantial new features (gstat and fsx) * Libs (freebsd-libgeom and freebsd-libgeom-sys) * Commits that reconcile the dependencies of multiple crates, so as to minimize duplicate dependency versions (5764fb383d4 and 1edf2e19e50) * Vendoring all dependencies, direct and transitive, to ensure internet-independent and reproducible builds (37ef9ffb6a6). This process is automated and requires almost no manual effort. Note: don't panic if you look in the "vendor" directory and see a bunch of crates with "windows" in the name. They're all just empty stubs. * All Rust object files get stored in the "target" directory rather than /usr/obj. Today, if you want them to be stored in /usr/obj the best way is to use a symlink, though there's WIP to add MAKEOBJDIRPREFIX-like functionality to Cargo. It does NOT demonstrate: * Integrating the Rust build system with Make. Warner has some ideas about how to do that. * Pulling rustc into contrib. This tree requires an external Rust toolchain. * Building any cdylib libraries with Rust. That's useful if you want a C program to call a Rust library, but I don't have any good examples for it. * kernel modules. As already discussed, those are hard. * Any Rust crates that involve private APIs, like CTL stuff. Those are among the most tantalizing programs to move from ports to base, but nobody's written any yet, because Rust-in-base doesn't exist yet. Also, I want to address a question that's popped up a few times: backwards-compatibility. There is a fear that Rust code needs to be updated for each new toolchain release. But that's not true. It hasn't been true for most crates since Rust 1.0 was released about a decade ago. A few exotic crates required "nightly" features after that, but they are very few in number these days, and none of them are included in this branch's vendored sources. What Rust _does_ do is it releases a new toolchain about every six weeks. Each new release typically includes a few new features in the standard library and they often add more compiler warnings, too. Sometimes they include wholly new compiler features, but they are _always_ backwards compatible with existing syntax. Roughly every three years, Rust publishes a new "Edition". Rust Editions are very similar to C++ versions. i.e. Rust 2018 is to Rust 2021 as C++14 is to C++17. New editions can include backwards-incompatible syntax changes, but each crate always knows which Edition it uses. Crates of different Editions can be linked together in the same build. This branch, for example, contains crates using Editions 2015, 2018, and 2021. If you have any questions about what Rust in Base would look like, please examine this branch. And if you've never used Rust before, I highly encourage you to try it. It really is the best new systems-program language for decades. IMHO, it's the only one that's a compelling replacement for C++ in all new applications, and C in most. https://github.com/asomers/freebsd-src/tree/rust-in-base-demo From nobody Sun Aug 4 18:00:17 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WcS5x45c0z5SlbT for ; Sun, 04 Aug 2024 18:00:21 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WcS5x1m26z4H8l; Sun, 4 Aug 2024 18:00:20 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Authentication-Results: mx1.freebsd.org; none Received: from critter.freebsd.dk (unknown [192.168.55.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by phk.freebsd.dk (Postfix) with ESMTPS id 2DA7B892B6; Sun, 04 Aug 2024 18:00:18 +0000 (UTC) Received: (from phk@localhost) by critter.freebsd.dk (8.18.1/8.16.1/Submit) id 474I0HUM050473; Sun, 4 Aug 2024 18:00:17 GMT (envelope-from phk) Message-Id: <202408041800.474I0HUM050473@critter.freebsd.dk> To: Alan Somers cc: FreeBSD Hackers Subject: Re: A Demo of rust-in-base In-reply-to: From: "Poul-Henning Kamp" References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <50471.1722794417.1@critter.freebsd.dk> Date: Sun, 04 Aug 2024 18:00:17 +0000 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:1835, ipnet:130.225.0.0/16, country:EU] X-Rspamd-Queue-Id: 4WcS5x1m26z4H8l -------- Alan Somers writes: > Due to all of the recent discussion of using Rust for code in the > FreeBSD base, I've put together a demo of what it might look like. It > demonstrates: Awesome! But to be blunt: Is it worth the effort, relative to concentrating on a pkg-based distribution of FreeBSD, where these things could be built with the regular "COTS" Rust ecosystem, without having to do all this extra work and adding all this extra maintenance load ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From nobody Sun Aug 4 18:09:32 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WcSJp0YpVz5SmPm for ; Sun, 04 Aug 2024 18:09:46 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-vk1-f179.google.com (mail-vk1-f179.google.com [209.85.221.179]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WcSJn5WlLz4Jc8 for ; Sun, 4 Aug 2024 18:09:45 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-vk1-f179.google.com with SMTP id 71dfb90a1353d-4f50dd3eab9so3361864e0c.1 for ; Sun, 04 Aug 2024 11:09:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722794984; x=1723399784; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6L4NWeWj2Hhg7/mlnIJ+yski0Kq4sGqVIeVIUIaEMr8=; b=RQVoA8PlMnGfbY1rwdLUrQCu5u7iXD0r2TNn0KTeDHNrZYyfM/CBsV5NvHT5mhdsyU apxX33dTxXO9yjUJrZOkXgTWy+DZL5GSN3fAsiwvou7/cu/NxL1GKW7vpISXWAj2ecTL ZcrcFzTOHwtG5src23Bsok3eC+im2xAmrHY0WEpSc2EyPL44hHftREyJLqazDRNUC/Vz IiKCgmAB760Jiks6QxBr5s4EW1yN0q3YFm4bATpShdeqg5UYSOpV3g+2h+3XKrfdBqmN I0K+6d+J9E3c3mT9aMx9fENW1f5RW2iyl38odFfJKsZhBFVx1nQ5TugyMz5q+uwPDquu ovlQ== X-Gm-Message-State: AOJu0Yz6P0ii6RNrHKRyG7GCBOFPkeKS3lJIxJVpYYSS1lzDRs8zDOhJ zDc7CGy2hgtcR/JN0Da2mqyzEl2MGLRKo+tC6apMaOfRMxRBPXtagd8Qzezt6hKCod+TBOU7L78 1BBf9yzI5TeyL8dgjxNLrL9FRlS2FnQ== X-Google-Smtp-Source: AGHT+IEw3GYaroDLv9rM1aXkz8ucHnnVi3HD68hVUSY9AA+hi6Ol5OLy8AuO6MyLiNO+UwD556iSDS/5K3rStaQmkRo= X-Received: by 2002:a05:6122:2a56:b0:4ed:14e:9342 with SMTP id 71dfb90a1353d-4f89ff40db3mr9513487e0c.1.1722794984524; Sun, 04 Aug 2024 11:09:44 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <202408041800.474I0HUM050473@critter.freebsd.dk> In-Reply-To: <202408041800.474I0HUM050473@critter.freebsd.dk> From: Alan Somers Date: Sun, 4 Aug 2024 12:09:32 -0600 Message-ID: Subject: Re: A Demo of rust-in-base To: Poul-Henning Kamp Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4WcSJn5WlLz4Jc8 On Sun, Aug 4, 2024 at 12:00=E2=80=AFPM Poul-Henning Kamp wrote: > > -------- > Alan Somers writes: > > Due to all of the recent discussion of using Rust for code in the > > FreeBSD base, I've put together a demo of what it might look like. It > > demonstrates: > > Awesome! > > But to be blunt: Is it worth the effort, relative to concentrating > on a pkg-based distribution of FreeBSD, where these things could > be built with the regular "COTS" Rust ecosystem, without having to > do all this extra work and adding all this extra maintenance load ? If it weren't for my experience with CTL, I would say no. But CTL stuff _cannot_ exist in the ports tree, since the ioctl interface is unstable. Similarly, stuff like the fusefs test suite can't exist in the ports tree, either. It needs to be updated in lock-step with even minor kernel changes. If those are to use Rust, they need to reside in the same git repository as freebsd-src. From nobody Sun Aug 4 18:20:45 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WcSYY2pS8z5SmyM for ; Sun, 04 Aug 2024 18:20:49 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WcSYY1vplz4L99; Sun, 4 Aug 2024 18:20:47 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Authentication-Results: mx1.freebsd.org; none Received: from critter.freebsd.dk (unknown [192.168.55.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by phk.freebsd.dk (Postfix) with ESMTPS id 071D1892B6; Sun, 04 Aug 2024 18:20:45 +0000 (UTC) Received: (from phk@localhost) by critter.freebsd.dk (8.18.1/8.16.1/Submit) id 474IKjVV050602; Sun, 4 Aug 2024 18:20:45 GMT (envelope-from phk) Message-Id: <202408041820.474IKjVV050602@critter.freebsd.dk> To: Alan Somers cc: FreeBSD Hackers Subject: Re: A Demo of rust-in-base In-reply-to: From: "Poul-Henning Kamp" References: <202408041800.474I0HUM050473@critter.freebsd.dk> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <50600.1722795645.1@critter.freebsd.dk> Content-Transfer-Encoding: quoted-printable Date: Sun, 04 Aug 2024 18:20:45 +0000 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:1835, ipnet:130.225.0.0/16, country:EU] X-Rspamd-Queue-Id: 4WcSYY1vplz4L99 -------- Alan Somers writes: > On Sun, Aug 4, 2024 at 12:00=3DE2=3D80=3DAFPM Poul-Henning Kamp wrote: > If it weren't for my experience with CTL, I would say no. > But CTL stuff _cannot_ exist in the ports tree, since the ioctl interfac= e is unstable. > Similarly, stuff like the fusefs test suite can't exist in the ports tre= e, either. > It needs to be updated in lock-step with even minor kernel changes. How is that different from any other dependency management in ports ? -- = Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe = Never attribute to malice what can adequately be explained by incompetence= . From nobody Sun Aug 4 18:25:18 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WcSg24RQLz5SnDL for ; Sun, 04 Aug 2024 18:25:34 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ot1-x330.google.com (mail-ot1-x330.google.com [IPv6:2607:f8b0:4864:20::330]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WcSg22j7Gz4MJ6 for ; Sun, 4 Aug 2024 18:25:34 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ot1-x330.google.com with SMTP id 46e09a7af769-7093f6adc4aso3930523a34.3 for ; Sun, 04 Aug 2024 11:25:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1722795933; x=1723400733; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=+S+1wSig+UCb93IoY6kbUViZqA9EwWPSKBDtU/BSXSE=; b=SG0qwE13DHlh8iVU397E7cWGsXdQjyMjjKyFucogEhmyOO7bMbX69QmM1IRbXXgIQY sYbABA1SZDLYDR1yUBn/SOuPQoyV4MX3PCAPdpn6tXbKte9yeqv/8ORzzNdk2F7p1/DL MqT05+NQzakoxEdk31V8U4eeFKIILF/78xPi3jwGsgWMgdkROrT/c7nrSD9yOFdpkRW4 ybHAi10euvzJ5WMrOXzUQS0xmHzX1Ptoqtab5nU81ReXfaeH8wp3FyC335jaHl6LU5h7 DTnSx3r70U4vtGcSZAHHsZPDOXAS94w4VYPUqL+khtIk5B7WanCymXH59ToO+ZXnhVLL 2E2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722795933; x=1723400733; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=+S+1wSig+UCb93IoY6kbUViZqA9EwWPSKBDtU/BSXSE=; b=hteNw5tRSFtH2LGH1C8bvR/Pb0bjnHBYIRQE+l/5EB/P88PVXs+z/VxdOgWtiT1jxZ 4aqOdJFBEICA7vI7ztuj/xWiiv54vPnXUvwPCnROFxJksdmmpBwdrUNT5EtyofwrMpT/ JpxKhLlsJ0Y19lbEO7LjL/grPHtuWcdtKTJKrBJF2ldybGLhe8/EQRIthRkODkaxTbj3 lkxAcs6vrIaAu9VoBXo7MEE76Lp45aVlT1oDqP7NQREOWzaeYbZ1RNPwIe7TvBnnM0XX hJGI9kQtG7h1OM9KiQpbm2Y1ZYdwde5TH3lZmJpgzuT1aeKGRAFzKPK8O37ypYJrWjRS 4J5g== X-Forwarded-Encrypted: i=1; AJvYcCW0aMNJ5CdlmzmztW824lT+RQNYTq7sOnLy+vNi+4jnGJoAVT+H5ksx10w115uoX7zO4mn8voGkXDWjeclz3mxcat5V7LKRPIsZaQM= X-Gm-Message-State: AOJu0YwBg/1TWVfSBcjquopzuXpYZOdNtR2cAzU09seTU/rNUkFfkRn7 GgOgxefW3kzKIuuhDIDgIrele4EdahWJps+GBkB4QbKv2Rj9CEr4BZnh1pEAgusmQRVwckJsbis QZynOL4H0OpIbRzvlfVfMLa8FIWShaaMz0u912t1w/9XFlWxP X-Google-Smtp-Source: AGHT+IGp7Pv/AqwPqE/CHblpmLaEaN2Pzt2m6OUvCYkHdrTuJDnh4mUhVtqSc/YPFy8+Bq59Wiig/SD959XFlGD6QyI= X-Received: by 2002:a05:6830:3884:b0:703:6883:30be with SMTP id 46e09a7af769-709b3208a6fmr16254154a34.11.1722795932852; Sun, 04 Aug 2024 11:25:32 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <202408041800.474I0HUM050473@critter.freebsd.dk> <202408041820.474IKjVV050602@critter.freebsd.dk> In-Reply-To: <202408041820.474IKjVV050602@critter.freebsd.dk> From: Warner Losh Date: Sun, 4 Aug 2024 12:25:18 -0600 Message-ID: Subject: Re: A Demo of rust-in-base To: Poul-Henning Kamp Cc: Alan Somers , FreeBSD Hackers Content-Type: multipart/alternative; boundary="000000000000993e6c061edfaf2d" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4WcSg22j7Gz4MJ6 --000000000000993e6c061edfaf2d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sun, Aug 4, 2024, 12:21=E2=80=AFPM Poul-Henning Kamp wrote: > -------- > Alan Somers writes: > > On Sun, Aug 4, 2024 at 12:00=3DE2=3D80=3DAFPM Poul-Henning Kamp < > phk@phk.freebsd.dk> wrote: > > > If it weren't for my experience with CTL, I would say no. > > > But CTL stuff _cannot_ exist in the ports tree, since the ioctl > interface is unstable. > > Similarly, stuff like the fusefs test suite can't exist in the ports > tree, either. > > It needs to be updated in lock-step with even minor kernel changes. > > How is that different from any other dependency management in ports ? > Or any different that qemu's bsd-user perputually chasing things like struct user... Warner --=20 > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetenc= e. > > --000000000000993e6c061edfaf2d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Sun, Aug 4, 2024, 12:21=E2=80=AFPM Poul-Henning Kam= p <phk@phk.freebsd.dk> wrot= e:
--------
Alan Somers writes:
> On Sun, Aug 4, 2024 at 12:00=3DE2=3D80=3DAFPM Poul-Henning Kamp <ph= k@phk.freebsd.dk> wrote:

> If it weren't for my experience with CTL, I would say no.

> But CTL stuff _cannot_ exist in the ports tree, since the ioctl interf= ace is unstable.
> Similarly, stuff like the fusefs test suite can't exist in the por= ts tree, either.
> It needs to be updated in lock-step with even minor kernel changes.
How is that different from any other dependency management in ports ?

Or any= different that qemu's bsd-user perputually chasing things like struct = user...

Warner

--
Poul-Henning Kamp=C2=A0 =C2=A0 =C2=A0 =C2=A0| UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| TCP/IP since RFC 956
FreeBSD committer=C2=A0 =C2=A0 =C2=A0 =C2=A0| BSD since 4.3-tahoe=C2=A0 =C2= =A0
Never attribute to malice what can adequately be explained by incompetence.=

--000000000000993e6c061edfaf2d-- From nobody Sun Aug 4 18:36:01 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WcSvM43QCz5SpNX for ; Sun, 04 Aug 2024 18:36:15 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-vk1-f171.google.com (mail-vk1-f171.google.com [209.85.221.171]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WcSvL6vLVz4NYt for ; Sun, 4 Aug 2024 18:36:14 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-vk1-f171.google.com with SMTP id 71dfb90a1353d-4f6abcf0567so3799586e0c.0 for ; Sun, 04 Aug 2024 11:36:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722796573; x=1723401373; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5tJ37ZWvUqw7C8wcbhYZJawGFEdHnW5gF1KhkXgYBVw=; b=QUGb+/YuwJgOzKBUnRunt/S+BCUZsnAWEX+ofJ3SeaixJUAoKssKHS9BXIepgx3JbV O2X8PyMe5f7fIRcQbepT2xmlXuo3/DYOM5BXtejaW3QDWa2kv16QsbIFPDs8P1o/XQ2J lT0Vi9ftKcJHy7J4gT4uQ/SfJ+uAyjLN9fd0Fb2IGuS7PkP2b/32JOldyOEJWHu6o7Cm nR9fWTciwK+lH7Yf43esDDVHYUA9JAhyCE+MM3velNXYH64B3kkypbYe0ge/v1iLCHPM QkaNYcwPM5ppSMFFNDSzHGoDbTCHzUZSStmQnEa1W/ZKNOqocB9dkixm7XdLAOBNZZ8O eMqw== X-Gm-Message-State: AOJu0Yz0F8PTgKL9qnsI4wQBRqSYMdmkCT3zlvRkdCpAVUiBc80YG8fC 97Er1VFwXKFUYEzwuTmXBtQTw3yeQ05upgR8gd+triLWmtJYhg1yUMa0a0SFWUW3TYBiDbIrmZv kIkG+zhuZnPjIvcbo60AGgJ0jZPzHcw== X-Google-Smtp-Source: AGHT+IHbd/zsg7n0hrIYXs8OCMsZdoHFnnP6BBnbgGN5hTK8IEqAAjYL1gvfLQsZIMyotBwMhUPmgk+vOtG4nSIbyUk= X-Received: by 2002:a05:6122:4581:b0:4f5:23e4:b7c with SMTP id 71dfb90a1353d-4f89fe608fbmr11441169e0c.0.1722796573632; Sun, 04 Aug 2024 11:36:13 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <202408041800.474I0HUM050473@critter.freebsd.dk> <202408041820.474IKjVV050602@critter.freebsd.dk> In-Reply-To: <202408041820.474IKjVV050602@critter.freebsd.dk> From: Alan Somers Date: Sun, 4 Aug 2024 12:36:01 -0600 Message-ID: Subject: Re: A Demo of rust-in-base To: Poul-Henning Kamp Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4WcSvL6vLVz4NYt On Sun, Aug 4, 2024 at 12:20=E2=80=AFPM Poul-Henning Kamp wrote: > > -------- > Alan Somers writes: > > On Sun, Aug 4, 2024 at 12:00=3DE2=3D80=3DAFPM Poul-Henning Kamp wrote: > > > If it weren't for my experience with CTL, I would say no. > > > But CTL stuff _cannot_ exist in the ports tree, since the ioctl interfa= ce is unstable. > > Similarly, stuff like the fusefs test suite can't exist in the ports tr= ee, either. > > It needs to be updated in lock-step with even minor kernel changes. > > How is that different from any other dependency management in ports ? Because those two components need to be updated in lock-step with potentially any git commit to the base system. Not just official releases, even minor ones. From nobody Sun Aug 4 19:04:37 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WcTX91vGkz5SrCR for ; Sun, 04 Aug 2024 19:04:41 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WcTX90Kpmz4Qt4; Sun, 4 Aug 2024 19:04:40 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Authentication-Results: mx1.freebsd.org; none Received: from critter.freebsd.dk (unknown [192.168.55.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by phk.freebsd.dk (Postfix) with ESMTPS id E7853892B6; Sun, 04 Aug 2024 19:04:37 +0000 (UTC) Received: (from phk@localhost) by critter.freebsd.dk (8.18.1/8.16.1/Submit) id 474J4b9e050871; Sun, 4 Aug 2024 19:04:37 GMT (envelope-from phk) Message-Id: <202408041904.474J4b9e050871@critter.freebsd.dk> To: Alan Somers cc: FreeBSD Hackers Subject: Re: A Demo of rust-in-base In-reply-to: From: "Poul-Henning Kamp" References: <202408041800.474I0HUM050473@critter.freebsd.dk> <202408041820.474IKjVV050602@critter.freebsd.dk> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <50869.1722798277.1@critter.freebsd.dk> Date: Sun, 04 Aug 2024 19:04:37 +0000 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:1835, ipnet:130.225.0.0/16, country:EU] X-Rspamd-Queue-Id: 4WcTX90Kpmz4Qt4 -------- Alan Somers writes: > On Sun, Aug 4, 2024 at 12:20=E2=80=AFPM Poul-Henning Kamp dk> wrote: > > How is that different from any other dependency management in ports ? > > Because those two components need to be updated in lock-step with > potentially any git commit to the base system. Not just official > releases, even minor ones. I'm not trying to be glib here: I really want to make sure I understand any fine nuances you are trying to communicate. Isn't that precisely what drm-kmod already deals with ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From nobody Sun Aug 4 19:27:25 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WcV2f6Bglz5Ssvf for ; Sun, 04 Aug 2024 19:27:38 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pj1-x102c.google.com (mail-pj1-x102c.google.com [IPv6:2607:f8b0:4864:20::102c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WcV2f4Zkcz4TSW for ; Sun, 4 Aug 2024 19:27:38 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x102c.google.com with SMTP id 98e67ed59e1d1-2cfec641429so2174325a91.0 for ; Sun, 04 Aug 2024 12:27:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1722799657; x=1723404457; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=vqwnzLEfeClr8KWOWCGqSVFgE9PtNSgIF3YqppZc+jY=; b=WmXwEu9k4rTJzl3UC/3XMZNnzDQ5+92wVD80DDiJJHFplLUezahudovn9kNNMUYaUi cd4t53W2HO+uBYKAjVvtZ1z1HrTAn6EYlmLaXso2ZVA/P/3vwPCPj1BBEM+C4t846OJ9 3nR1VP0qFFKsK1sZqNX9iOTi39IstlF8xoXAUhO/O8ElUjNqeQawq5AK7FPbVEGRKWrb 6a0LRaI1nTRP/tefI33CCC7TzYea2g1xDxUn5Z0Yg30rHADIixgTJe7D+GQHQJuDDT68 lu+fq86PVnkjs7prbZA89zlrTZS2L6k82XfTFmgcv/wHXWgZNDNjS53zjNMvk6gTJ2A9 xYjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722799657; x=1723404457; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=vqwnzLEfeClr8KWOWCGqSVFgE9PtNSgIF3YqppZc+jY=; b=igyxuNT6SCF6MwaUiE6LXTFD2somVc3Psd8MtjSGNnG8MBhNHmqMwsSyFVTAQ3K8Lo HvzdLZePxyXWBdR0JyZEB7Swgc8sT3dtIo4LqQmaR+UdJLhOAQABI7uR0/NCCxCbKrwg l26Etx9Az1jGrMerGmJP9ZtVwxoi4ItEMk5a3xjuBFzARIsqBAz5qKITK9EuwopCdvVM 8lyzyGS2jLyyPbjzL6594Q47Whykt4r6k7/mvQcFHlfhpAHgitMz6Uce75zn4zZ9uW/U 0iCqaOkrne4tfb8azkWTnR9tRTwmr0ZbjRNEKNWlNvUKA7huPW2CHmHxuZYsrNrCwFHl cx8w== X-Forwarded-Encrypted: i=1; AJvYcCXCBGKDyt+zqLZ3WLjobe4gEKA8Ry1yUzxBFuo8iFF0xjUyVCpe9kwE2j/jX+mI45LljiXEkJiNiqEDd8VhF8O6lo/uyfQ26QwI/oQ= X-Gm-Message-State: AOJu0YxAWoVntY2moHp+Ae9SAW/uww4iOaC4BDFs8ncca82u2aVUUu9q qlbjTtznVAtyp8sS723ZA6mYfAdu0ZP/mEgd+f2vGRKkXM0nqGIDUBIRyNvlSdqnhPkQEj/CF5r /V84Lrbsio8c6XjnXp+NTui5LOy1urSyRnTwzfxgYUeTRMHB8 X-Google-Smtp-Source: AGHT+IGPUZuUVhQqtaf59cCMLUE+wP8e/nNJaMzXJ8MSCnQ2EoOwPXZrSObUkVRtmn4MJFg6cA4Wx39elfcAr/kYZdI= X-Received: by 2002:a17:90a:a00e:b0:2c9:7cb6:38b0 with SMTP id 98e67ed59e1d1-2cff9444f6bmr8350401a91.19.1722799657053; Sun, 04 Aug 2024 12:27:37 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <202408041800.474I0HUM050473@critter.freebsd.dk> <202408041820.474IKjVV050602@critter.freebsd.dk> <202408041904.474J4b9e050871@critter.freebsd.dk> In-Reply-To: <202408041904.474J4b9e050871@critter.freebsd.dk> From: Warner Losh Date: Sun, 4 Aug 2024 13:27:25 -0600 Message-ID: Subject: Re: A Demo of rust-in-base To: Poul-Henning Kamp Cc: Alan Somers , FreeBSD Hackers Content-Type: multipart/alternative; boundary="000000000000940d3e061ee08d14" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4WcV2f4Zkcz4TSW --000000000000940d3e061ee08d14 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sun, Aug 4, 2024, 1:05=E2=80=AFPM Poul-Henning Kamp = wrote: > -------- > Alan Somers writes: > > On Sun, Aug 4, 2024 at 12:20=3DE2=3D80=3DAFPM Poul-Henning Kamp > > dk> wrote: > > > > How is that different from any other dependency management in ports ? > > > > Because those two components need to be updated in lock-step with > > potentially any git commit to the base system. Not just official > > releases, even minor ones. > > I'm not trying to be glib here: I really want to make sure I understand > any fine nuances you are trying to communicate. > > Isn't that precisely what drm-kmod already deals with ? > There's two issues with drm-kmod. 1 is KPI and keeping up. That's pretty easy to manage in the grand scheme of things. The other is KBI and matching the kernel. The massive inlining in linuxkpi make a stable KBI basically impossible (I did it for much of 12.x, and that was a nightmare because it broke faster than I had time to fix it). Warner --=20 > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetenc= e. > > --000000000000940d3e061ee08d14 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Sun, Aug 4, 2024, 1:05=E2=80=AFPM Poul-Henning Kamp= <phk@phk.freebsd.dk> wrote= :
--------
Alan Somers writes:
> On Sun, Aug 4, 2024 at 12:20=3DE2=3D80=3DAFPM Poul-Henning Kamp <ph= k@phk.freebsd.=3D
> dk> wrote:

> > How is that different from any other dependency management in por= ts ?
>
> Because those two components need to be updated in lock-step with
> potentially any git commit to the base system.=C2=A0 Not just official=
> releases, even minor ones.

I'm not trying to be glib here: I really want to make sure I understand=
any fine nuances you are trying to communicate.

Isn't that precisely what drm-kmod already deals with ?

There's two = issues with drm-kmod. 1 is KPI and keeping up. That's pretty easy to ma= nage in the grand scheme of things.

The other is KBI and matching the kernel. The massive inlining = in linuxkpi make a stable KBI basically impossible (I did it for much of 12= .x, and that was a nightmare because it broke faster than I had time to fix= it).

Warner


--
Poul-Henning Kamp=C2=A0 =C2=A0 =C2=A0 =C2=A0| UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| TCP/IP since RFC 956
FreeBSD committer=C2=A0 =C2=A0 =C2=A0 =C2=A0| BSD since 4.3-tahoe=C2=A0 =C2= =A0
Never attribute to malice what can adequately be explained by incompetence.=

--000000000000940d3e061ee08d14-- From nobody Sun Aug 4 20:38:20 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WcWcH1JQJz5Sywj for ; Sun, 04 Aug 2024 20:38:23 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WcWcG65nqz4cFs; Sun, 4 Aug 2024 20:38:22 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Authentication-Results: mx1.freebsd.org; none Received: from critter.freebsd.dk (unknown [192.168.55.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by phk.freebsd.dk (Postfix) with ESMTPS id EE800892B6; Sun, 04 Aug 2024 20:38:20 +0000 (UTC) Received: (from phk@localhost) by critter.freebsd.dk (8.18.1/8.16.1/Submit) id 474KcKrD052069; Sun, 4 Aug 2024 20:38:20 GMT (envelope-from phk) Message-Id: <202408042038.474KcKrD052069@critter.freebsd.dk> To: Warner Losh cc: Alan Somers , FreeBSD Hackers Subject: Re: A Demo of rust-in-base In-reply-to: From: "Poul-Henning Kamp" References: <202408041800.474I0HUM050473@critter.freebsd.dk> <202408041820.474IKjVV050602@critter.freebsd.dk> <202408041904.474J4b9e050871@critter.freebsd.dk> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <52067.1722803900.1@critter.freebsd.dk> Date: Sun, 04 Aug 2024 20:38:20 +0000 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:1835, ipnet:130.225.0.0/16, country:EU] X-Rspamd-Queue-Id: 4WcWcG65nqz4cFs -------- Warner Losh writes: > > > Because those two components need to be updated in lock-step with > > > potentially any git commit to the base system. Not just official > > > releases, even minor ones. > > > > I'm not trying to be glib here: I really want to make sure I understand > > any fine nuances you are trying to communicate. > > > > Isn't that precisely what drm-kmod already deals with ? > > [...] > > The other is KBI and matching the kernel. The massive inlining in linuxkpi > make a stable KBI basically impossible [...] That was essentially my point: We already have things in ports which are totally fragile relative to the kernel, so I want to make sure there is not som new/other/worse/different aspects in the two cases Alan brought up. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From nobody Sun Aug 4 22:42:55 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WcZNF0dkwz5RyNJ for ; Sun, 04 Aug 2024 22:43:09 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ua1-f46.google.com (mail-ua1-f46.google.com [209.85.222.46]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WcZND5s8Pz42ym for ; Sun, 4 Aug 2024 22:43:08 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ua1-f46.google.com with SMTP id a1e0cc1a2514c-81ff6a80cbbso2970481241.1 for ; Sun, 04 Aug 2024 15:43:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722811387; x=1723416187; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=t7oF/nKVlwsNni18gO67iosH80T7P6bGbvEOtJvJvw4=; b=DHAQmdNy3er7c2iZs0KHt56GWkhY4e/5S1fmvK7kTWQ1aViB79ABkmPmPsiN1ZsRj7 WB5Hy1OTo7ywcK+/L8bypTlFacEwImmyseFwnu6iPxCDAKT5oxFr5GY3abD1iOqXl1qG 8le/f+LzBZkqVe1XwMNp3NvpVvZXxkqkr2R4IlCa0mU9iSdgcG85CvzGDys1AUt0vEX2 haGSYFWEtxQ5/BHfHRZYF7e1F831D/ixlFzVeaPV8RlDKR44cW2uTQ9lwFuAl/8v4JFn tzPVXhYRMQwcuTl1W8vh1auAgHkPErbzpaT3bbqMDj3pHKuggp0sLNVLb7TbYu3CyK1x jA7g== X-Forwarded-Encrypted: i=1; AJvYcCWcBtqZFo/Xr+lZm8uVLo4BHne3AY92qFovbUfow+nMC7BQh1b1aSunL+axWGmBjjzaGVw+pAoOmvROGSoqOC8y9bOt1xUIR6IKcHw= X-Gm-Message-State: AOJu0YyheqzRLdZY3AMq8zjj06HCVKCqe0w+RojIpor5EXJUY5fC0REz EERcIpzAiYVzW3IdC0KoTVmkzWOb7Qk6aTJr9b4D1qK0EYFTZO85cx1muW/7Jk1HjMRcq5N6D/h XdSKMMYWzV1wYylfpcHhXwyjZlOU4CZRT X-Google-Smtp-Source: AGHT+IGB/Dj8zXTX7t23KPwkmTkppB2Bu+mfn045+bl8tRP00qS0hifTmkp7qeWOqf/7pUbtIXG99Q1rHRypzucJl0U= X-Received: by 2002:a05:6102:3f01:b0:493:daea:615d with SMTP id ada2fe7eead31-4945bbf1a47mr13425454137.0.1722811387456; Sun, 04 Aug 2024 15:43:07 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: <202408041800.474I0HUM050473@critter.freebsd.dk> <202408041820.474IKjVV050602@critter.freebsd.dk> <202408041904.474J4b9e050871@critter.freebsd.dk> <202408042038.474KcKrD052069@critter.freebsd.dk> In-Reply-To: <202408042038.474KcKrD052069@critter.freebsd.dk> From: Alan Somers Date: Sun, 4 Aug 2024 16:42:55 -0600 Message-ID: Subject: Re: A Demo of rust-in-base To: Poul-Henning Kamp Cc: Warner Losh , FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4WcZND5s8Pz42ym On Sun, Aug 4, 2024 at 2:38=E2=80=AFPM Poul-Henning Kamp wrote: > > -------- > Warner Losh writes: > > > > > Because those two components need to be updated in lock-step with > > > > potentially any git commit to the base system. Not just official > > > > releases, even minor ones. > > > > > > I'm not trying to be glib here: I really want to make sure I understa= nd > > > any fine nuances you are trying to communicate. > > > > > > Isn't that precisely what drm-kmod already deals with ? > > > > [...] > > > > The other is KBI and matching the kernel. The massive inlining in linux= kpi > > make a stable KBI basically impossible [...] > > That was essentially my point: We already have things in ports which > are totally fragile relative to the kernel, so I want to make sure there > is not som new/other/worse/different aspects in the two cases Alan > brought up. Yeah, the problem is pretty similar to what drm-kmod faces. To quantify the scale: * Between 13.0-RELEASE and the latest stable/13 there were 61 changes to tests/sys/fs/fusefs. * 37 of those were paired with a kernel change, usually in the same commit. If the fusefs test suite were external, that's 37 changes that would need to be replicated in three places (freebsd-src, fusefs-tests, and freebsd-ports). Any user running the tests would need to build the port about once per month, if they track stable/13. Git archaeology would be a lot harder. And worst of all, MFCing changes from main to stable branches would be very confusing. Should the fusefs-tests repository have its own stable branches? Or would the stable branches just not be tested? Those are the reasons why I wrote the fusefs tests in C++ instead of Rust, even though it's a lower productivity language. -Alan