From nobody Tue Jun 9 17:02:25 2026 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4gZZwL6LMsz6gHv8 for ; Tue, 09 Jun 2026 17:02:42 +0000 (UTC) (envelope-from bakul@iitbombay.org) Received: from mail-dy1-x132c.google.com (mail-dy1-x132c.google.com [IPv6:2607:f8b0:4864:20::132c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4gZZwK54C5z4Lyw for ; Tue, 09 Jun 2026 17:02:41 +0000 (UTC) (envelope-from bakul@iitbombay.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=iitbombay.org header.s=google header.b=Wo+j776O; dmarc=pass (policy=quarantine) header.from=iitbombay.org; spf=pass (mx1.freebsd.org: domain of bakul@iitbombay.org designates 2607:f8b0:4864:20::132c as permitted sender) smtp.mailfrom=bakul@iitbombay.org Received: by mail-dy1-x132c.google.com with SMTP id 5a478bee46e88-304d3d3d8f8so384501eec.2 for ; Tue, 09 Jun 2026 10:02:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iitbombay.org; s=google; t=1781024559; x=1781629359; darn=freebsd.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=lMzmzawubs4v/ml3jS5U3BxWuz0RqbGnGMXBEmEKhFM=; b=Wo+j776Ox+gVKC3EyMrrivsTQl9my2U2nFq8hphvE49KWKgCIqAIR7bfaqat0IW0qS ZASAlNa8J15kLUf/1IOtL4vtRms6prYSwJwNRCcHF4KwxyFSxnijS4OZ+1gkv7MANlIU UrpRq8fZxiQiENevmAoRoHr3E8/fMLZSjZmDc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781024559; x=1781629359; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-gg:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=lMzmzawubs4v/ml3jS5U3BxWuz0RqbGnGMXBEmEKhFM=; b=OSqeB+b9a48hh17ymvyjbHwuE2jSTKvvSFm12Wz7lc8/LOKLJ7L7vF6zHR2uByedYf V+Vx+6dTPC5NgkGnGuwwE2uOlvz1W5LZ0gtas7ZBWz1Q4G8YnUdhtilxA8vAbH47ARDa FDiDtVL6uyqd1nykuDt+gc2ldLVNBgnDBfTL36YfZXBbk/LCFCrJ7uufMStxvtf30HOK GhH5SlE/ErtPkxPMEsuQhHy6KMch3HEzgAo8b377GSdYgg3d9CqCMnXPGFwKddJhjUwE 6TAmb/vbd8fi3rX+NVlkSHEWpMDG2kzb5CHyQxxB0ZW5E8ZlI+axt34T7Ylsc1tbCNpg 2Spg== X-Forwarded-Encrypted: i=1; AFNElJ9lP5w3La0Uq7Lqez0K8rc9Z1EXwFR4iRdR+q26bUV9t1Jq2Zv79/w8KDnrf2cY9544t2EpXIR1UgILptXJ0so=@freebsd.org X-Gm-Message-State: AOJu0YzSSVSFnrhpTAO6cMDAM8hQ8DVee13xKFSrs4Qyunh66iTQyBbL 4+e9vbLFPDYZmOjJefoGmOAyQ64aIZCRHwGdeZJO2GDjgWlRuf2yG0OfJKZLwOMJcQ== X-Gm-Gg: Acq92OGXxk5XyYOkLa+RyIBNf65ySFjQwQAomYdVpDhmwHhny6nBbnBIAXN4RaX4UAU Oqyt7lvcLAwp3qdA89L9hCzYOpYaI7OE9MoYQ5aDRF9eGPLajjxfT5KZqYNsgUwtLZ3vtGkXzx8 tWyaj5IZv2elPY7gtjjFgoXfvkpbKzJyj/J0V9kWeGzb41sh85IkZQD9wUTPC11q9MQz5hzqY95 jqnLtkU55JjubY4r8YAriG0Qsi1GYONR+AYhKPRR53V6zYpXCuRy2h1UoEBmPUpRMgGX5Xjj0Rp xsjK1Ui9ensCrs9I+C5CzB2Fv7lVoAB9x+cyx5VMoeUg6930MZA4L6QTNNeCJS3cfHyOhW1YGmx n5pETOWa34gQ0uc+8d5SCSccAggJGPcYqDRUYu7X7BnQLmO2Xjnr7UntCgtMLDJ9lSCV7UlNtjE HbQI9N9933gEmgQPBnVmAQCnuO5jVjMuEd3MnZnQWT525b7ZZ/Pm0no4hgbopenaxXIIx/gMVVq 6zRXk3l1vtFN2FIjka2ptSu6SjX6a7KiJpsv+iCuBZ0rRIBYw== X-Received: by 2002:a05:7300:72cc:b0:304:3c33:7afd with SMTP id 5a478bee46e88-3077b258b9emr6708882eec.1.1781024558794; Tue, 09 Jun 2026 10:02:38 -0700 (PDT) Received: from smtpclient.apple (107-215-223-229.lightspeed.sntcca.sbcglobal.net. [107.215.223.229]) by smtp.gmail.com with ESMTPSA id 5a478bee46e88-3074df75ff6sm22808805eec.26.2026.06.09.10.02.37 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Jun 2026 10:02:37 -0700 (PDT) Content-Type: text/plain; charset=utf-8 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3864.600.51.1.1\)) Subject: Re: BHYVE on NFSv4.2 filesystem? From: Bakul Shah In-Reply-To: Date: Tue, 9 Jun 2026 10:02:25 -0700 Cc: Dan Shelton , freebsd-hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <725FEEE6-D75E-4D75-B464-008B2678120B@iitbombay.org> References: To: Rick Macklem X-Mailer: Apple Mail (2.3864.600.51.1.1) X-Spamd-Result: default: False [-2.50 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[iitbombay.org,quarantine]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[iitbombay.org:s=google]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4864::/56]; MIME_GOOD(-0.10)[text/plain]; DKIM_TRACE(0.00)[iitbombay.org:+]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; RCPT_COUNT_THREE(0.00)[3]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEFALL_USER(0.00)[bakul]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; TO_DN_SOME(0.00)[]; TAGGED_RCPT(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::132c:from] X-Spamd-Bar: -- X-Rspamd-Queue-Id: 4gZZwK54C5z4Lyw On Jun 9, 2026, at 8:49=E2=80=AFAM, Rick Macklem = wrote: >=20 > On Mon, May 18, 2026 at 3:39=E2=80=AFPM Bakul Shah = wrote: >>=20 >> On May 18, 2026, at 2:26=E2=80=AFPM, Dan Shelton = wrote: >>>=20 >>> On Fri, 20 Feb 2026 at 01:21, Bakul Shah = wrote: >>>>=20 >>>> On Feb 19, 2026, at 11:45=E2=80=AFAM, Dan Shelton = wrote: >>>>>=20 >>>>> On Wed, 18 Feb 2026 at 22:45, Dan Shelton = wrote: >>>>>>=20 >>>>>> Hello, >>>>>>=20 >>>>>> Has anyone tried a BHYVE with a disk as file on a NFSv4.2 mount? >>>>=20 >>>> Yes. [I tried this on a 15.0-RELEASE-p3 host, nfsv4.2 mounting >>>> a filesystem from a 15.0-STABLE machine] >>>=20 >>> How about the performance? Is it better than iSCSI? >>=20 >> I don't know about iSCSI but comparing with p9fs: >>=20 >> Test1: >> dd bs=3D1m count=3D4000 > /dev/null < large-file >>=20 >> nfsV3: >> 32.3 >> 46.3 >> 51.3 >>=20 >> nfsV4: >> 129.1 >> 59.9 >> 48.8 >>=20 >> p9fs: >> 17.7 >> 17.5 >> 17.6 >>=20 >> Test2: >> find /usr/src/ > /dev/null >>=20 >> nfsV3: >> 60.0 >> 39.0 >> 30.9 >>=20 >> nfsV4: >> 54.0 >> 17.9 >> 35.8 >>=20 >> p9fs: >> 6.9 >> 6.5 >> 6.6 >>=20 >>=20 >> So slower in all cases. In addition the variability in nfs numbers is = concerning! >>=20 >> p9fs doesn't cache but nfs does, so anything cached is served much = faster. > At this time it is only a hunch, but I think this might help.. > https://datatracker.ietf.org/doc/draft-ietf-nfsv4-uncacheable-files/ Just to clarify, if data is cached, NFS is usually much faster; but the above tests didn't have any caching effect. In general caching can be a win but it is more that the 9p protocol *can't* do caching - no way to specify cache invalidation etc. > I plan on implementing this soon for FreeBSD. May be useful in some cases but won't help nfs numbers.... >=20 > rick