From nobody Wed Apr 3 14:13:56 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V8mvf0wgMz5Gscm for ; Wed, 3 Apr 2024 14:14:06 +0000 (UTC) (envelope-from igor.ostapenko@pm.me) Received: from mail-40131.protonmail.ch (mail-40131.protonmail.ch [185.70.40.131]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V8mvd1tzCz4Zj3 for ; Wed, 3 Apr 2024 14:14:05 +0000 (UTC) (envelope-from igor.ostapenko@pm.me) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=pm.me header.s=protonmail3 header.b=QmyF9ODX; dmarc=pass (policy=quarantine) header.from=pm.me; spf=pass (mx1.freebsd.org: domain of igor.ostapenko@pm.me designates 185.70.40.131 as permitted sender) smtp.mailfrom=igor.ostapenko@pm.me DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail3; t=1712153640; x=1712412840; bh=b6miHEyY4QID+K9kaFacfCIvrqCaLmx0ntbJ11ovLdo=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=QmyF9ODXXdcKrAOXT3bPZUIubu3vlrY1ZE8s9+O3DPTKNVMxXt9zIwQqBavaXL+yU JJysyCUM8lV1ZtHLWS9/CpwvCxif1EFeJ2GHsMVPS0n6mKT7kDV3sCEs5tjoa11/UV cdRMMSiN98c7hL4vk4jStC7D3rmz0+LVAVbp+HxF6tngZDnBEaRCWon5BDSHkXPFzH QeTNX6WLkpDCvhAomIp1PTT47YswgDn1y6aiOYjwks1JiTIF1Y2Dp1HmZSbcEWITQ8 rEoQXCfYzGXl9GRpdda6FOmM3wgoZ9T9xjFYSoibEjfqNd2WCiqCmOkYRbmjv58HmF w5ZH6IQBH5kEg== Date: Wed, 03 Apr 2024 14:13:56 +0000 To: freebsd-hackers@freebsd.org From: Igor Ostapenko Cc: "freebsd-testing@FreeBSD.org" , Kristof Provost Subject: Re: Add jail execution environment support to the FreeBSD test suite Message-ID: <8H_refsJfTozkldDQEkQFQY-RBauBRmKOJRWFfdSSZkxJ6KOmarYb8Fk97FjQ1yGgZhKGmkbZWgsgbaLgPhExtasz8OLJujk-54AO_erA5U=@pm.me> In-Reply-To: References: <2bjQNp1msrv-_AqyamMun6kY-SCqbgPm3Q7DqVQHAYlqvFkiE1i85svfIT-QQdUG1cg3cKippyTyv8Z-5nbLu4WaMutgZQ7KT-YYo_5Pbro=@pm.me> Feedback-ID: 8300135:user:proton List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.20 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[pm.me,quarantine]; RWL_MAILSPIKE_VERYGOOD(-0.20)[185.70.40.131:from]; R_DKIM_ALLOW(-0.20)[pm.me:s=protonmail3]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24:c]; MIME_GOOD(-0.10)[text/plain]; MISSING_XM_UA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; TO_DN_SOME(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[pm.me:+] X-Rspamd-Queue-Id: 4V8mvd1tzCz4Zj3 Hi, Thanks for testing. My current vision of the action points is as follows: The first phase: - [x] igoro: Separate FreeBSD specifics and fix existing Kyua tests broken = by the change - [x] igoro: Migrate some of the existing tests for the start, e.g. netpfil= /pf - [x] igoro: Cover Paul's use case mentioned in this email thread - [x] igoro: Cover Olivier's use case mentioned in this email thread - [x] igoro: Provide the respective documentation updates (manual pages) - [~] community: Review, testing, comments -- probably we want to change th= e design - [ ] committers: Help with the main commit -- it should hit freebsd/kyua GitHub fork first, then vendor branch, and merge to main = after The next phases: - [ ] igoro: Provide the PRs to add brand new tests of Kyua itself to cover the new feature - [ ] igoro: Help with the Handbook(s) update to cover the new concept for test authors The future phases: - [ ] igoro: Work on the related improvements and ideas like required_klds = etc If there is nothing to change or add at this stage then the next step could= be to merge the GitHub PR: https://github.com/freebsd/kyua/pull/224 Thanks the community for your time invested, I hope it will be eventually payed back with better test run time during development. Best regards, Igor. On Thursday, March 21st, 2024 at 4:59 AM, Kristof Provost = wrote: > >=20 > I=E2=80=99ve been toying with this patch. Adding only the following patch= I can get Kyua to run the pf tests in parallel: >=20 > diff --git a/tests/sys/netpfil/pf/Makefile b/tests/sys/netpfil/pf/Mak= efile > index 867b98e5f6c2..c2f0f15fa798 100644 > --- a/tests/sys/netpfil/pf/Makefile > +++ b/tests/sys/netpfil/pf/Makefile > @@ -51,8 +51,8 @@ ATF_TESTS_PYTEST+=3D frag6.py > ATF_TESTS_PYTEST+=3D nat66.py > ATF_TESTS_PYTEST+=3D sctp.py > =20 > -# Tests reuse jail names and so cannot run in parallel. > -TEST_METADATA+=3D is_exclusive=3Dtrue > +TEST_METADATA+=3D execenv=3D"jail" > +TEST_METADATA+=3D execenv_jail=3D"vnet allow.raw_sockets" > =20 > PROGS=3D divapp > =20 >=20 > That gets the test time, with parallelism=3D5, down from 22 minutes to 5m= 40s. > So I=E2=80=99m rather keen to see this work land. >=20 > My read from the reactions here is that people are generally okay with th= e approach, especially (I assume) given that the current version lets us tu= rn this on on a per-test basis. >=20 > Is there anything else anyone wants to raise before we land this? >=20 > Best regards, > Kristof >=20 > On 28 Feb 2024, at 2:32, Igor Ostapenko wrote: >=20 > > Hi, > >=20 > > The patch was updated after the recent discussion. > >=20 > > Currently, the patch provides the following new functionality, from bot= tom to > > top: > >=20 > > 1 ATF based tests > >=20 > > - The new "execenv" metadata property can be set to explicitly ask for = an > > execution environment: "host" or "jail". If it's not defined, as all > > existing tests do, then it implicitly means "host". > >=20 > > - The new "execenv.jail" metadata property can be optionally defined to= ask > > Kyua to use specific jail(8) parameters during creation of a temporary > > jail. An example is "vnet allow.raw_sockets". > >=20 > > 2 Kyuafile > >=20 > > - The same new metadata properties can be defined on Kyuafile level: > > "execenv" and "execenv_jail". > >=20 > > - Note that historically ATF uses dotted style of metadata naming, whil= e > > Kyua uses underscore style. Hence "execenv.jail" vs. "execenv_jail". > >=20 > > 3 kyua.conf, kyua CLI > >=20 > > - The new "execenv" engine configuration variable can be set to a list = of > > execution environments to run only tests designed for. Tests of not lis= ted > > environments are skipped. > >=20 > > - By default, this variable lists all execution environments supported = by a > > Kyua binary, e.g. execenv=3D"host jail". > >=20 > > - This variable can be changed via "kyua.conf" or via kyua CLI's "-v" > > parameter. For example, "kyua -v execenv=3Dhost test" will run only > > host-based tests and skip jail-based ones. > >=20 > > - Current value of this variable can be examined with "kyua config". > >=20 > > The patch is https://reviews.freebsd.org/D42350. > >=20 > > Any help with review and testing is welcome. Its test plan covers the > > details and refers to the demo patch of how existing tests could be > > converted to be run in a jail. > >=20 > > Best regards, Igor. > >=20 > > On Thursday, February 22nd, 2024 at 10:57 PM, igor.ostapenko@pm.me wrote: > >=20 > > > Hi FreeBSD developers, > > >=20 > > > There is a proposal to improve the FreeBSD test suite. > > >=20 > > > 1 The Problem > > >=20 > > > The FreeBSD test suite is based on the Kyua framework. The latter sup= ports > > > running tests in parallel. However, some tests cannot be run in paral= lel and > > > are marked with is_exclusive=3D"true" metadata, which makes Kyua run = such tests > > > in sequence. > > >=20 > > > Many tests are not meant to be exclusive conceptually, they are so fo= r very > > > simple technical reasons. For instance, some network related tests ar= e based > > > on jail and vnet usage. It's convenient for such tests and it provide= s a lot > > > of isolation already not to conflict with other tests. But they are s= till > > > marked as exclusive due to the shared space of jail names, routing, e= tc. > > >=20 > > > The project seeks more tests, and it's kind of a trend for new tests = like > > > jail/vnet based ones to be created as is_exclusive=3D"true" from the = very > > > beginning. It only piles up the suite with exclusive tests, e.g. new = tests > > > from my side faced a fair question from a reviewer whether they could= be > > > re-designed for a parallel run. [1] > > >=20 > > > If such tests were 100% isolated they would be able to run in paralle= l and > > > decrease the test time for CI runs and for the runs within the develo= pment > > > process. > > >=20 > > > And the problem is that trying to add more isolation by a test itself= looks to > > > be a doable task from a glance, but it would add a lot of complexity = to a test > > > code, or could be found as an impossible task in a specific case. > > >=20 > > > 2 The Idea > > >=20 > > > The idea is not new. A test could be running in a jail -- it provides= the > > > required isolation with minimum or zero effort from a test. > > >=20 > > > 3 The Implementation > > >=20 > > > There is a lot of work done already and the working patch passed the = initial > > > review (thanks to markj@ and ngie@). [2] > > >=20 > > > It adds a new concept to the Kyua framework -- an execution environme= nt. Two > > > new metadata were added for that: execenv and execenv_jail. > > >=20 > > > execenv is a switch to select an environment. If a test's metadata de= fines > > > execenv=3D"jail" then Kyua will create a temporary jail, run such tes= t within > > > it, and remove the jail. If execenv=3D"host" is provided or execenv m= etadata is > > > undefined then Kyua will run such test as it does today. > > >=20 > > > execenv_jail metadata takes effect only in case of execenv=3D"jail". = It allows a > > > test to request specific parameters for its jail. These parameters ar= e simply > > > arguments to jail(8), e.g. execenv_jail=3D"vnet allow.raw_sockets". > > >=20 > > > 4 The Adoption > > >=20 > > > ATF based tests can easily define this new metadata via Kyuafile or d= irectly, > > > e.g. for atf-sh based tests: > > >=20 > > > test_head() > > > { > > > atf_set descr "Test foo in case of bar" > > > atf_set require.user root > > > atf_set execenv jail > > > atf_set execenv.jail vnet allow.raw_sockets > > > } > > >=20 > > > Non-ATF based ones will do it via Kyuafile. Our test suite does it th= rough a > > > Makefile: > > >=20 > > > TEST_METADATA+=3D execenv=3D"jail" > > > TEST_METADATA+=3D execenv_jail=3D"vnet allow.raw_sockets" > > >=20 > > > The patch got some little evolution, I started with a single execenv_= jail > > > metadata, and during the patch discussion and review, I ended up with= two > > > knobs: execenv and execenv_jail. It turned out to be a cleaner and le= ss tricky > > > interface such way. The evolution reasoning can be found in the histo= ry of the > > > respective Differential. [2] > > >=20 > > > 5 MFC Concerns > > >=20 > > > For now, I see at least one issue from the usual project workflow per= spective. > > > Let's imagine that the Kyua framework got this execenv feature commit= ted to > > > 15-CURRENT, we started to convert existing tests and create new ones = to use > > > execenv=3D"jail". If some feature or a bug fix needs to be ported bac= k to > > > 14-STABLE or 13-STABLE, then "old" Kyua without execenv feature will = fail to > > > run such tests: > > >=20 > > > kyua: E: Load of 'Kyuafile' failed: Failed to load Lua file 'Kyuafile= ': Kyuafile:9: Unknown metadata property execenv. > > >=20 > > > From a combinatorics perspective, the first three options pop up to d= eal with > > > that: > > > a) Patch Kyua the same way for the supported STABLE branches so it wi= ll be > > > able to run back ported tests based on execenv=3D"jail" (it's not sys= tem ABI > > > change after all) > > > b) Exclusively patch Kyua framework for the supported STABLE branches= to > > > simply skip such tests (does not look to provide much benefit) > > > c) Do not back port tests, only the fix/feature itself (kind of a bad= idea) > > >=20 > > > 6 The Demo > > >=20 > > > My test environment showed promising run time numbers for almost the = whole > > > test suite (ZFS excluded). One of the tests yielded 36 min with test > > > parallelism improvement versus 1 h 25 min without. In my case with 8 = cores, > > > the suite runs about 2 times faster with the improvement. [3] > > >=20 > > > 7 Action Points > > >=20 > > > My current vision of the plan looks as follows: > > > - [ ] community: Review, testing, comments -- probably we want to cha= nge the > > > design > > > - [ ] committers: Help with the main commit -- it should hit freebsd/= kyua > > > GitHub fork first [4], then vendor branch, and merge to > > > main after > > > - [ ] igoro: Provide the subsequent PRs to separate FreeBSD specifics= and fix > > > existing Kyua tests > > > - [ ] igoro: Provide the PRs to add brand new tests of Kyua itself to= cover > > > the new feature > > > - [ ] igoro: Provide the respective documentation updates > > > - [ ] igoro: Migrate some of the existing tests for the start, e.g. n= etpfil/pf > > > - [ ] committers: Help with review and respective commits/merges > > >=20 > > > The plan is not strict, it depends on the discussion and interest of > > > volunteers. > > >=20 > > > I hope that this proposal is found valuable for the project. If so, a= ny help > > > is appreciated. > > >=20 > > > [1] New tests exclusivity concern: https://reviews.freebsd.org/D42314 > > > [2] The Kyua patch: https://reviews.freebsd.org/D42350 > > > [3] The whole test suite demo: https://reviews.freebsd.org/D42410 > > > [4] The respective PR to the fork: https://github.com/freebsd/kyua/pu= ll/224 >=20 > > > Best regards, Igor. From nobody Thu Apr 4 18:14:45 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V9VC72Z9tz5FkB1 for ; Thu, 4 Apr 2024 18:14:59 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ua1-f44.google.com (mail-ua1-f44.google.com [209.85.222.44]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V9VC63Gssz4pbL for ; Thu, 4 Apr 2024 18:14:58 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=freebsd.org (policy=none); spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.222.44 as permitted sender) smtp.mailfrom=asomers@gmail.com Received: by mail-ua1-f44.google.com with SMTP id a1e0cc1a2514c-7e389d74dcaso717227241.0 for ; Thu, 04 Apr 2024 11:14:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712254497; x=1712859297; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Xrx6WqUUlO4uJ1JrSOcgEiGvmQDmBLpmbULANJpqIaQ=; b=listoN7RfVUcEn1WcDQq6fEEYs2hgYNiK0e5zkYFGWhHHmXKWqDLoci4UlpbVZh/+r GN2xsdpcYyhhvGhJYkKy1oQCDDMNqONanoiqxpGD/FwXWJLJstxSr1GTbUtDKEd0xXmV GnLGCl+VmwDucE36BqFCcQkfISp2G696DsnYYS3cZTjQnlFPgpwe31WwxSH8AsejBGhR tKA+Fu6BcRJLaCjhzcPdCAXYKF4H9DFe+d5NwKoeqvg83+qgRjeWAIThjNY1Gc5GTDDK RNXtP5OmoiSqM0rZ/0XaPiLq5PVf5rjAPjmuhJASndrw7pxlusN3fsmtcNaNNjN7sz0V HgRA== X-Gm-Message-State: AOJu0YzIn9FKzPdNUt0WQCMZXGUZiB9aPYo41HuAfXc84bI8mtsSSgzo O2NdyOprnuC28Ya8npFPzFJ+l2cqmrulSE2fC3pZaDYfL0N1nDJns568mkRaajjoAcwyD4xFTcc 7znQ3m11PLQalRIcaXsHJkA3l6cY7+/ZhF/s= X-Google-Smtp-Source: AGHT+IFhk0OK163c0nvCPs9ytqPQkLLH83Kz/iQmda5ncv15XvrFuqdswHhQ/8IGkb7nNx7gXA5QZ/vZ8a3CN45oWME= X-Received: by 2002:a67:ebc5:0:b0:479:c133:a743 with SMTP id y5-20020a67ebc5000000b00479c133a743mr379937vso.9.1712254496868; Thu, 04 Apr 2024 11:14:56 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 From: Alan Somers Date: Thu, 4 Apr 2024 12:14:45 -0600 Message-ID: Subject: SEEK_HOLE at EOF To: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.52 / 15.00]; NEURAL_HAM_LONG(-0.99)[-0.987]; NEURAL_HAM_SHORT(-0.98)[-0.982]; NEURAL_HAM_MEDIUM(-0.65)[-0.648]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; MIME_GOOD(-0.10)[text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : SPF not aligned (relaxed), No valid DKIM,none]; TO_DN_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_ONE(0.00)[1]; MISSING_XM_UA(0.00)[]; FREEFALL_USER(0.00)[asomers]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; RCVD_COUNT_ONE(0.00)[1]; R_DKIM_NA(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.222.44:from]; TO_DOM_EQ_FROM_DOM(0.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; RCVD_IN_DNSWL_NONE(0.00)[209.85.222.44:from] X-Rspamd-Queue-Id: 4V9VC63Gssz4pbL tldr; there are two problems: 1) tmpfs handles SEEK_HOLE differently than other file systems 2) everything else handles SEEK_HOLE at EOF poorly, IMHO Details: According to lseek(2), SEEK_HOLE should return the start of the next hole greater than or equal to the supplied offset. Also, each file has a zero-sized virtual hole at the very end of the file. So I would expect that calling SEEK_HOLE at EOF would return the file's size. However, the man page also says that SEEK_HOLE will return ENXIO when the offset points to EOF. Those two statements seem contradictory to me. The first behavior seems more logical. I would expect SEEK_HOLE to work the same way both at EOF and at any other file offset. What does the spec say? There is no POSIX standard for this. It was invented by Solaris, Illumos's man page does not say clearly say what should happen at EOF. Linux's man page is clear: "whence is SEEK_DATA or SEEK_HOLE, and offset is beyond the end of the file". That would seem to indicate behavior 1: SEEK_HOLE should return the file's size at EOF. Only beyond EOF should it return ENXIO. But what do other implementations do? Contrary to its man page, Linux behaves mostly like FreeBSD. SEEK_HOLE returns ENXIO at EOF on most file systems. I tested a number of file systems on both FreeBSD and Linux. Most of them return ENXIO. The only two outliers are FreeBSD's tmpfs and Linux's NFS client. FreeBSD Linux ======= ========= ===== UFS ENXIO ZFS ENXIO tmpfs file size ENXIO msdosfs ENXIO ENXIO ext2fs ENXIO ENXIO xfs ENXIO tarfs ENXIO nfs ENXIO file size So what should we change? Clearly, it's bad for tmpfs to be inconsistent. My preference would be for everything to behave like tmpfs, but it's currently losing the popularity contest. Anybody else have thoughts? -Alan From nobody Thu Apr 4 20:56:31 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V9Ynm1C7Yz5G1sH for ; Thu, 4 Apr 2024 20:56:44 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-pg1-x52e.google.com (mail-pg1-x52e.google.com [IPv6:2607:f8b0:4864:20::52e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V9Ynl6LFJz48gT; Thu, 4 Apr 2024 20:56:43 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pg1-x52e.google.com with SMTP id 41be03b00d2f7-5e152c757a5so1063872a12.2; Thu, 04 Apr 2024 13:56:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712264202; x=1712869002; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=nlSBVHCs0YVCZ0uSZJLknAlnnOSyNQQNN0+5MKsXJj0=; b=LyZV3buwAYATpxL7N6Q1KfhKfpEkq1MF93w3L38NNyRyNZXTT8CYIghRfifivpcNdM KuFXJDX8CyK6BMex7oDxyPAL/xR80MCJzvmSUEEScQT0R9zfsO3aYdB+KbfOsy2iJSSt nn0i0833JTVD1XH+WoK/705A9IjkD6QF1m6ZAP6Fg4ut0huPfQxU2PdYPIsWjn40jv51 LwJFOPQLFtOcFvGrOgtRvLUDlWabsdxaMNbN9XTgNZqCSLIs+ehuTs8O6L8drxKI1ych Svl6rDk68TTleYKXKdq346KsYv2hnMmmi9p58v5hyDKXVAIGL8FJibum85N//odtY+RA pFRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712264202; x=1712869002; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nlSBVHCs0YVCZ0uSZJLknAlnnOSyNQQNN0+5MKsXJj0=; b=TpYmu1mKGC/3Ohw7KTf7/CEfvJkWy/eRS7Hf2E42mHkUUh2pxHIZngU1MWky5tjV7d Eq+6blEjRLev1BkZYDuiBdXKOCo5JmuxGrKtszPTwPyOkGJJA/l+MubgV9lh8tDWC8Lv 2KiNf4z01D1kG5K2c57Fl/rngK2GW7mMMKtvyi/u5KGBD1OXl/pKM7UhSZVBxyI/xM2q eqiPUtNZnxbyIhaEW6mL0x0UT6Mr5pLCF6PmC4zPq28iTofqnCoRh1yT3X8Bg7z0o4Av sGr1E1BrYmm1lwFkePssR/ofhWCuunq+XL0Wv8balM2fvIfhvZrjdVYEMtK/f48v3Q3f AJ2w== X-Gm-Message-State: AOJu0YyaHYLU1Pj+PF9MYCLKSdI3JA679/t5BsfL8cteGaC1PRbgShah D6NRpELez7GcG0pTLJpkJmGfK4CZ7D7FyoRfabaebLuDKgm4GHUjsM5K1vsjQzwmjHsM5kfP368 QHO+eFPaM1EkbN0F3xPLVEolLv1IY2+BOqQ== X-Google-Smtp-Source: AGHT+IH1V+VwAToujy/3QQ3S5iwlqyKw1Oowm+DTo05EkNaed3UlukfjF4qBEigyrx1Zw2VvQbWjhklYDLKaCPkmuQ0= X-Received: by 2002:a17:90b:33c7:b0:29c:7566:a1d6 with SMTP id lk7-20020a17090b33c700b0029c7566a1d6mr3457947pjb.25.1712264201868; Thu, 04 Apr 2024 13:56:41 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Rick Macklem Date: Thu, 4 Apr 2024 13:56:31 -0700 Message-ID: Subject: Re: SEEK_HOLE at EOF To: Alan Somers Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4V9Ynl6LFJz48gT On Thu, Apr 4, 2024 at 11:15=E2=80=AFAM Alan Somers w= rote: > > tldr; there are two problems: > 1) tmpfs handles SEEK_HOLE differently than other file systems > 2) everything else handles SEEK_HOLE at EOF poorly, IMHO > > Details: > > According to lseek(2), SEEK_HOLE should return the start of the next > hole greater than or equal to the supplied offset. Also, each file > has a zero-sized virtual hole at the very end of the file. So I would > expect that calling SEEK_HOLE at EOF would return the file's size. > However, the man page also says that SEEK_HOLE will return ENXIO when > the offset points to EOF. Those two statements seem contradictory to > me. The first behavior seems more logical. I would expect SEEK_HOLE > to work the same way both at EOF and at any other file offset. > > What does the spec say? > > There is no POSIX standard for this. It was invented by Solaris, > Illumos's man page does not say clearly say what should happen at EOF. > Linux's man page is clear: "whence is SEEK_DATA or SEEK_HOLE, and > offset is beyond the end of the file". That would seem to indicate > behavior 1: SEEK_HOLE should return the file's size at EOF. Only > beyond EOF should it return ENXIO. Well, there is the Austin Group stuff (never ratified by POSIX as I understand it). Here's what it says about SEEK_HOLE and offset: If whence is SEEK_HOLE, the file offset shall be set to the smallest location of a byte within a hole and not less than offset, except that if offset falls within the last hole, then the file offset may be set to the file size instead. It shall be an error if offset is greater or equal to the size of the file. I'd suggest we follow this, since it is the closest to a standard that ther= e is. rick > > But what do other implementations do? > > Contrary to its man page, Linux behaves mostly like FreeBSD. SEEK_HOLE > returns ENXIO at EOF on most file systems. I tested a number of file > systems on both FreeBSD and Linux. Most of them return ENXIO. The > only two outliers are FreeBSD's tmpfs and Linux's NFS client. > > FreeBSD Linux > =3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D > UFS ENXIO > ZFS ENXIO > tmpfs file size ENXIO > msdosfs ENXIO ENXIO > ext2fs ENXIO ENXIO > xfs ENXIO > tarfs ENXIO > nfs ENXIO file size > > So what should we change? Clearly, it's bad for tmpfs to be > inconsistent. My preference would be for everything to behave like > tmpfs, but it's currently losing the popularity contest. Anybody else > have thoughts? > > -Alan > From nobody Thu Apr 4 20:59:25 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V9Ys65RPDz5G1wR for ; Thu, 4 Apr 2024 20:59:38 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-vk1-f178.google.com (mail-vk1-f178.google.com [209.85.221.178]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V9Ys60cZfz4BLn for ; Thu, 4 Apr 2024 20:59:38 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=freebsd.org (policy=none); spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.221.178 as permitted sender) smtp.mailfrom=asomers@gmail.com Received: by mail-vk1-f178.google.com with SMTP id 71dfb90a1353d-4daa2b076beso328419e0c.1 for ; Thu, 04 Apr 2024 13:59:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712264377; x=1712869177; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=W47NbeJUTqELp97PtD11oAZ66jphlgvnF6pzh6Ji9/A=; b=kTyNgIvJCImpBfmEqYVZNynaqzBBiNSbC2PqmfPEbIY8N3TNBJNQt0kFBPHhQJdQpH Rte8SFK4sSfm3jvNj3m2KUCnTaXtrzreKRwSwzeZFQZdJgvGnlUUBou1mhTl+tvsAHAT dKPL3D8ZErzyLVSUwGjnO6FtkBGSPus0JfYXfq590p8tauLKOzDApHt44sAbb1RhjSTR xoRonQUE+bjPHMyztzijuqBV4NIMjdCNA+f0ddYw25w/5cZaM94/IoUSpgEq4Dtyu5tN 2u6KcTNl1WPepsclJVOJgSubkn5kpvFVa1INnccyybvulEmzWRiNz/ZnG08sPZchQJ8t RpjA== X-Gm-Message-State: AOJu0YyAZZvXTh8nOK/i804cyUu+yiHXlT2Qh1kNjNa+zGBNf6uIL/3+ zNIy2Cm4cmipfktwiVcsVBSVU10tJyS5R+l8lYcURQcNLjKgpwred7Co/5+y/FRpdhp+iVXMePe rpW5N8bvOrASht0yCqRoLwGfqZAwqxdMG X-Google-Smtp-Source: AGHT+IGGFnXFOwce/hOQcINwTdbOayw8whtiJN30+Ev2wkLWN47xlq2ogTCxjMwZ5gSN/Y3AnHDYu/aS77oquM0ya5U= X-Received: by 2002:a1f:f20f:0:b0:4d3:35ac:3553 with SMTP id q15-20020a1ff20f000000b004d335ac3553mr2901408vkh.10.1712264376935; Thu, 04 Apr 2024 13:59:36 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Thu, 4 Apr 2024 14:59:25 -0600 Message-ID: Subject: Re: SEEK_HOLE at EOF To: Rick Macklem Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: / X-Spamd-Result: default: False [-0.50 / 15.00]; NEURAL_HAM_LONG(-0.96)[-0.959]; NEURAL_SPAM_SHORT(0.65)[0.653]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; NEURAL_HAM_MEDIUM(-0.30)[-0.297]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; MIME_GOOD(-0.10)[text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : SPF not aligned (relaxed), No valid DKIM,none]; TO_DN_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; ARC_NA(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; MISSING_XM_UA(0.00)[]; FREEFALL_USER(0.00)[asomers]; FREEMAIL_ENVFROM(0.00)[gmail.com]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; RCVD_COUNT_ONE(0.00)[1]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.221.178:from]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; TAGGED_RCPT(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[209.85.221.178:from] X-Rspamd-Queue-Id: 4V9Ys60cZfz4BLn On Thu, Apr 4, 2024 at 2:56=E2=80=AFPM Rick Macklem wrote: > > On Thu, Apr 4, 2024 at 11:15=E2=80=AFAM Alan Somers = wrote: > > > > tldr; there are two problems: > > 1) tmpfs handles SEEK_HOLE differently than other file systems > > 2) everything else handles SEEK_HOLE at EOF poorly, IMHO > > > > Details: > > > > According to lseek(2), SEEK_HOLE should return the start of the next > > hole greater than or equal to the supplied offset. Also, each file > > has a zero-sized virtual hole at the very end of the file. So I would > > expect that calling SEEK_HOLE at EOF would return the file's size. > > However, the man page also says that SEEK_HOLE will return ENXIO when > > the offset points to EOF. Those two statements seem contradictory to > > me. The first behavior seems more logical. I would expect SEEK_HOLE > > to work the same way both at EOF and at any other file offset. > > > > What does the spec say? > > > > There is no POSIX standard for this. It was invented by Solaris, > > Illumos's man page does not say clearly say what should happen at EOF. > > Linux's man page is clear: "whence is SEEK_DATA or SEEK_HOLE, and > > offset is beyond the end of the file". That would seem to indicate > > behavior 1: SEEK_HOLE should return the file's size at EOF. Only > > beyond EOF should it return ENXIO. > Well, there is the Austin Group stuff (never ratified by POSIX as I > understand it). > > Here's what it says about SEEK_HOLE and offset: > If whence is SEEK_HOLE, the file offset shall be set to the smallest > location of a byte within a hole and not less than offset, except that > if offset falls within the last hole, then the file offset may be set > to the file size instead. It shall be an error if offset is greater > or equal to the size of the file. > > I'd suggest we follow this, since it is the closest to a standard that th= ere is. That sounds like behavior 2: return ENXIO at EOF. For reference, do you have a link to that somewhere? From nobody Thu Apr 4 21:38:43 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V9ZkS6B3zz5G5MM for ; Thu, 4 Apr 2024 21:38:56 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V9ZkR6BxDz4MyP; Thu, 4 Apr 2024 21:38:55 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x1032.google.com with SMTP id 98e67ed59e1d1-29f69710cbbso1081975a91.1; Thu, 04 Apr 2024 14:38:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712266734; x=1712871534; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=h5CI78Ayh7+CctyCA+wUNBTrxrUkQEU9ZS6nLy2+JNM=; b=c8yHw4cv3pe+YRtSbZsi6Yg9HxW278GOb/dZyGKV0h01t6wIghzFBTrWfABKw+KmAc nKUJwh4648kqOE3X6pK3Mz5FIuFAx+qpHsZiZ0SFBVxuFzrg2r9trWB057NyqL6X22El BL/iTzZcHhemNpg2uXj9YXZivPNE/8eyWt0YkinYTc5D9ZjKra7pQNg7zN5ZZpQOW7Kp rm87Qob7zy0rLiY7TlQXMjwE5wkejxQZPgg5NXtbhA8b+Y8eCSalxyuK5thqaAvtVc64 1v/OWP8WuLquV5ZMIWPxPogitD2Y9VSdIs4fbLVY+F5yY9nYu15qhJyTrCH958D/5YOB EDIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712266734; x=1712871534; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=h5CI78Ayh7+CctyCA+wUNBTrxrUkQEU9ZS6nLy2+JNM=; b=Y9fAEEcRevHFe4/S/yTNBNDQDt259TwACjEX1zd6cMLdzH9TdrYQpjp+p0pDeKVTbT BtNRpNsyv1lGLOfKjKhH33lkI2hiI5pH/NM67s1nalOd1hhEyCgF5jMRRcmuQjoYvURL XZ/Qm8mywjXWOl1vJRu+aVoKS75gDmfESVbJKuUmN1XnfrLuxykE4DYsNzZ1f9CG45wT Gm8I672iju87OVOfzlLdMklyq0rF0PN1Uwz+vBk2J/8kPUqyiV9xyU6x/XVQxQbHOLjN s2+Drq6UOT/lGIU3PNUitkNKS8v3cRTHMMoDSZy36iyarCNWhewQ3WgFqQnSkYC7GcQX Z+nA== X-Gm-Message-State: AOJu0YxJ+hD7CcqQIxDlX5kTFgb4KYemu1lnFeEvYGMX0yNoXfknw0w4 KQXgEGN5JFZQsJ+s2cL4h+P98gs6UavJWduHPIRjZz7CQn7dLkgnaaNMbLKogXrb8jDxQWALd2e viw0OSPbs1DLJ0xsnX4lU+BANvvsXlUVUcQ== X-Google-Smtp-Source: AGHT+IFQBcQGWwJMSAWWBZNp1YDL06ahHH4rkSUcFpHDKgl6XRUCYt4ympG/PaIrF//SmxUjqeD3WY/Lm7XT9gSZTAM= X-Received: by 2002:a17:90a:4cc5:b0:2a2:61a0:d1f4 with SMTP id k63-20020a17090a4cc500b002a261a0d1f4mr961033pjh.18.1712266734002; Thu, 04 Apr 2024 14:38:54 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Rick Macklem Date: Thu, 4 Apr 2024 14:38:43 -0700 Message-ID: Subject: Re: SEEK_HOLE at EOF To: Alan Somers Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4V9ZkR6BxDz4MyP On Thu, Apr 4, 2024 at 1:59=E2=80=AFPM Alan Somers wr= ote: > > On Thu, Apr 4, 2024 at 2:56=E2=80=AFPM Rick Macklem wrote: > > > > On Thu, Apr 4, 2024 at 11:15=E2=80=AFAM Alan Somers wrote: > > > > > > tldr; there are two problems: > > > 1) tmpfs handles SEEK_HOLE differently than other file systems > > > 2) everything else handles SEEK_HOLE at EOF poorly, IMHO > > > > > > Details: > > > > > > According to lseek(2), SEEK_HOLE should return the start of the next > > > hole greater than or equal to the supplied offset. Also, each file > > > has a zero-sized virtual hole at the very end of the file. So I woul= d > > > expect that calling SEEK_HOLE at EOF would return the file's size. > > > However, the man page also says that SEEK_HOLE will return ENXIO when > > > the offset points to EOF. Those two statements seem contradictory to > > > me. The first behavior seems more logical. I would expect SEEK_HOLE > > > to work the same way both at EOF and at any other file offset. > > > > > > What does the spec say? > > > > > > There is no POSIX standard for this. It was invented by Solaris, > > > Illumos's man page does not say clearly say what should happen at EOF= . > > > Linux's man page is clear: "whence is SEEK_DATA or SEEK_HOLE, and > > > offset is beyond the end of the file". That would seem to indicate > > > behavior 1: SEEK_HOLE should return the file's size at EOF. Only > > > beyond EOF should it return ENXIO. > > Well, there is the Austin Group stuff (never ratified by POSIX as I > > understand it). > > > > Here's what it says about SEEK_HOLE and offset: > > If whence is SEEK_HOLE, the file offset shall be set to the smallest > > location of a byte within a hole and not less than offset, except that > > if offset falls within the last hole, then the file offset may be set > > to the file size instead. It shall be an error if offset is greater > > or equal to the size of the file. > > > > I'd suggest we follow this, since it is the closest to a standard that = there is. > > That sounds like behavior 2: return ENXIO at EOF. For reference, do > you have a link to that somewhere? 0000415: add SEEK_HOLE, SEEK_DATA to lseek - Austin Group Defect Tracker (austingroupbugs.net) If this doesn't give you a link (gmail never shows the raw url for me) just google "SEEK_HOLE austin group". rick From nobody Thu Apr 4 22:16:29 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V9bZ01Ymqz5G8Gq for ; Thu, 4 Apr 2024 22:16:40 +0000 (UTC) (envelope-from steffen@sdaoden.eu) Received: from sdaoden.eu (sdaoden.eu [217.144.132.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4V9bYz2Qgyz4RD6; Thu, 4 Apr 2024 22:16:39 +0000 (UTC) (envelope-from steffen@sdaoden.eu) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=sdaoden.eu header.s=citron header.b=MkYJLjoJ; dkim=none ("invalid DKIM record") header.d=sdaoden.eu header.s=orange header.b=bgswyEAz; dmarc=none; spf=none (mx1.freebsd.org: domain of steffen@sdaoden.eu has no SPF policy when checking 217.144.132.164) smtp.mailfrom=steffen@sdaoden.eu DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sdaoden.eu; s=citron; t=1712268991; x=1712935657; h=date:author:from:to:cc:subject: message-id:in-reply-to:references:openpgp:blahblahblah:mime-version: content-type:content-transfer-encoding:author:from:subject:date:to:cc: resent-date:resent-from:resent-to:resent-cc:in-reply-to:references: mime-version:content-type:content-transfer-encoding:message-id: mail-followup-to:openpgp:blahblahblah; bh=FAaTl4ALiZW84dbQBsXm2AAuMmpJZMBCUNS/0vk3ktA=; b=MkYJLjoJbYtfl+e+VPLV0wpGdT7OJHjp2VbhKH5zXz6Wqcms1jlZ3m11NWD8Cww9cqqbkp6x QZPoq5VJqb/Ujdgh/tZNNuqiySl2FqeL6Q8y558+HKH9ZqfADKrV+UyRfgYzKaYWcLNG+V2OiH qO+O4fkjFuXGGTopwAS5swzbc4H1qCZsS/RrmDqzqMer2UNUzL86EUvprgcHHfbJmJjBcpcqVk qpJZlGd2IaT8tk6rC8a39ublgv/u6QY1h73Gv3bLvy33ocG/I530ed2XvFYIy/DllO+n+MOaiR zCen8CPg868MyXfZk3/zAOvPMYjJygNiKippmTs0/fzIhJzQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=sdaoden.eu; s=orange; t=1712268991; x=1712935657; h=date:author:from:to:cc:subject: message-id:in-reply-to:references:openpgp:blahblahblah:mime-version: content-type:content-transfer-encoding:author:from:subject:date:to:cc: resent-date:resent-from:resent-to:resent-cc:in-reply-to:references: mime-version:content-type:content-transfer-encoding:message-id: mail-followup-to:openpgp:blahblahblah; bh=FAaTl4ALiZW84dbQBsXm2AAuMmpJZMBCUNS/0vk3ktA=; b=bgswyEAzVbAcwrsjoZc959CSeufPTyn8nEF9aA5ITdcDcau2+wFP4XzElC5bbP4pXowgKogv Y27KK7HoAUjmBg== Date: Fri, 05 Apr 2024 00:16:29 +0200 Author: Steffen Nurpmeso From: Steffen Nurpmeso To: Rick Macklem Cc: Alan Somers , FreeBSD Hackers Subject: Re: SEEK_HOLE at EOF Message-ID: <20240404221629.pFpzUB3t@steffen%sdaoden.eu> In-Reply-To: References: User-Agent: s-nail v14.9.24-612-g7e3bfac540 OpenPGP: id=EE19E1C1F2F7054F8D3954D8308964B51883A0DD; url=https://ftp.sdaoden.eu/steffen.asc; preference=signencrypt BlahBlahBlah: Any stupid boy can crush a beetle. But all the professors in the world can make no bugs. List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.20 / 15.00]; AUTH_NA_OR_FAIL(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.90)[-0.902]; R_DKIM_ALLOW(-0.20)[sdaoden.eu:s=citron]; MIME_GOOD(-0.10)[text/plain]; R_DKIM_PERMFAIL(0.00)[sdaoden.eu:s=orange]; ASN(0.00)[asn:15987, ipnet:217.144.128.0/20, country:DE]; R_SPF_NA(0.00)[no SPF record]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[sdaoden.eu]; ARC_NA(0.00)[]; TAGGED_RCPT(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; DKIM_MIXED(0.00)[]; TO_DN_ALL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; DKIM_TRACE(0.00)[sdaoden.eu:+,sdaoden.eu:~] X-Rspamd-Queue-Id: 4V9bYz2Qgyz4RD6 Rick Macklem wrote in : |On Thu, Apr 4, 2024 at 1:59=E2=80=AFPM Alan Somers = wrote: |> |> On Thu, Apr 4, 2024 at 2:56=E2=80=AFPM Rick Macklem \ |> wrote: |>> |>> On Thu, Apr 4, 2024 at 11:15=E2=80=AFAM Alan Somers wrote: ... |>> Here's what it says about SEEK_HOLE and offset: |>> If whence is SEEK_HOLE, the file offset shall be set to the smallest |>> location of a byte within a hole and not less than offset, except that |>> if offset falls within the last hole, then the file offset may be set |>> to the file size instead. It shall be an error if offset is greater |>> or equal to the size of the file. |>> |>> I'd suggest we follow this, since it is the closest to a standard \ |>> that there is. |> |> That sounds like behavior 2: return ENXIO at EOF. For reference, do |> you have a link to that somewhere? |0000415: add SEEK_HOLE, SEEK_DATA to lseek - Austin Group Defect |Tracker (austingroupbugs.net) |If this doesn't give you a link (gmail never shows the raw url for me) |just google |"SEEK_HOLE austin group". just a few lines further below 46396 [ENXIO] The whence argument is SEEK_HO= LE or SEEK_DATA, and offset is greater 46397 than or equal to the file size= ; or the whence argument is SEEK_DATA and the 46398 offset falls beyond the last b= yte not within a hole. ... --End of --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From nobody Thu Apr 4 22:44:54 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V9cBv1Z87z5GC5C for ; Thu, 4 Apr 2024 22:45:11 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com [IPv6:2a00:1450:4864:20::12e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V9cBs5R6Bz4Tqb for ; Thu, 4 Apr 2024 22:45:09 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20230601.gappssmtp.com header.s=20230601 header.b=BTB1E7Zh; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2a00:1450:4864:20::12e) smtp.mailfrom=wlosh@bsdimp.com Received: by mail-lf1-x12e.google.com with SMTP id 2adb3069b0e04-516ab4b3251so1842312e87.0 for ; Thu, 04 Apr 2024 15:45:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1712270704; x=1712875504; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=EudZB0DoPEMlHnwHLGVrMw4HmrwRMUP9pboBN63XqDs=; b=BTB1E7Zhn00cASx48XWKeSEFal0fxg+pxH17bUkQHGon9nmmsAn7UHvrMZ37wEUPwc h0QKZuVJsKNsR/yVyLdyGdw8rJIK7oMTGmyjRvP/w+eY130ThJITmVLBeGxzaAyUGgoY 7MZiOJrLW5akvZ56ytKWk9zTRpxgbVapiclQv5sAcHAiGw0TlY4QjcQKsRi8kV1tUDaN CKiEm6QsV8lWAgCf1UaBBgi78tHF0HRUlV7CyMLSxNvZweAtz55Jp9GoSVMvbStZZzCE ltNwH+1mNDfhL4NDMoDDglge5Sjle4b+O6COiXNL4Zeb1fC64Bf3q/OHRGj/eIXvSEF4 Js8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712270704; x=1712875504; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=EudZB0DoPEMlHnwHLGVrMw4HmrwRMUP9pboBN63XqDs=; b=Pr0w5zzq36adzfKXLnd+z02icfgHD44BheOFgepV0w3x0ewSy1xfcOcYbdIKTGtp4b PExlFOM5oS5OLZj5lWZPCPLAvkQy4wVR1IfYC+XbxAXUDZ0M4NfFA9E+gf5uF1WP3zjM PzrIF/3Z1+Fivxufyber717C2kbvO32u3vA8NfLYs2FcjjDpuvMql+jwGxY/WLAKYrFH PhhzE9BAqHtwdiSMSCnHUetk+pQS2RpTnnbpZXVEN504wu2N2/j4KZ5k65BY+sF03v7U rY2skCTIsTtKDx3gzKjM9YF+Mz31V1KcSWhJec9mfgq3PsSjBHG6gPDqqxxonFYUsYxd 4XnA== X-Forwarded-Encrypted: i=1; AJvYcCUvvNVFYl5U4JObflS5AWEgLy1c4oxHdKIBqwR4xuL3BuO1W6Ysqxw+YSsdUHGG196BSFCS6igXxKugsnnG6A/Lvp/LIhRuBUWZB6U= X-Gm-Message-State: AOJu0YyI5hM7omKERlu87Se7mX2Df9Sx9FdcNPtJPt5nxa5fPyHpEuWJ D26IyZ0j/vqeiOxQSOqsvqMB8ZJDcokYZm6MQpCKni2oYElqNdiewC66bAW96MvV8k7zeVIcahc bXtMCfvn0Qo/ar62xUIyRArvJU8qARefU4Q4nKw== X-Google-Smtp-Source: AGHT+IFW5jZVagr5k6qXD9OuQklyGZvO+CQuNBck3UG/HEHv322bdxCIvnk1BijuYOe8paxlKkYerhUkiJBV5Od3vHg= X-Received: by 2002:a05:6512:554:b0:516:c41f:b0fc with SMTP id h20-20020a056512055400b00516c41fb0fcmr503853lfl.58.1712270704357; Thu, 04 Apr 2024 15:45:04 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Thu, 4 Apr 2024 16:44:54 -0600 Message-ID: Subject: Re: SEEK_HOLE at EOF To: Rick Macklem Cc: Alan Somers , FreeBSD Hackers Content-Type: multipart/alternative; boundary="00000000000017e07c06154d17b1" X-Spamd-Bar: -- X-Spamd-Result: default: False [-3.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20230601.gappssmtp.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:~]; MISSING_XM_UA(0.00)[]; DMARC_NA(0.00)[bsdimp.com]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::12e:from]; ARC_NA(0.00)[]; TAGGED_RCPT(0.00)[]; R_SPF_NA(0.00)[no SPF record]; FREEMAIL_TO(0.00)[gmail.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; TO_DN_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20230601.gappssmtp.com:+] X-Rspamd-Queue-Id: 4V9cBs5R6Bz4Tqb --00000000000017e07c06154d17b1 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Apr 4, 2024 at 3:39=E2=80=AFPM Rick Macklem wrote: > On Thu, Apr 4, 2024 at 1:59=E2=80=AFPM Alan Somers = wrote: > > > > On Thu, Apr 4, 2024 at 2:56=E2=80=AFPM Rick Macklem > wrote: > > > > > > On Thu, Apr 4, 2024 at 11:15=E2=80=AFAM Alan Somers > wrote: > > > > > > > > tldr; there are two problems: > > > > 1) tmpfs handles SEEK_HOLE differently than other file systems > > > > 2) everything else handles SEEK_HOLE at EOF poorly, IMHO > > > > > > > > Details: > > > > > > > > According to lseek(2), SEEK_HOLE should return the start of the nex= t > > > > hole greater than or equal to the supplied offset. Also, each file > > > > has a zero-sized virtual hole at the very end of the file. So I > would > > > > expect that calling SEEK_HOLE at EOF would return the file's size. > > > > However, the man page also says that SEEK_HOLE will return ENXIO wh= en > > > > the offset points to EOF. Those two statements seem contradictory = to > > > > me. The first behavior seems more logical. I would expect SEEK_HO= LE > > > > to work the same way both at EOF and at any other file offset. > > > > > > > > What does the spec say? > > > > > > > > There is no POSIX standard for this. It was invented by Solaris, > > > > Illumos's man page does not say clearly say what should happen at > EOF. > > > > Linux's man page is clear: "whence is SEEK_DATA or SEEK_HOLE, and > > > > offset is beyond the end of the file". That would seem to indicate > > > > behavior 1: SEEK_HOLE should return the file's size at EOF. Only > > > > beyond EOF should it return ENXIO. > > > Well, there is the Austin Group stuff (never ratified by POSIX as I > > > understand it). > > > > > > Here's what it says about SEEK_HOLE and offset: > > > If whence is SEEK_HOLE, the file offset shall be set to the smallest > > > location of a byte within a hole and not less than offset, except tha= t > > > if offset falls within the last hole, then the file offset may be set > > > to the file size instead. It shall be an error if offset is greater > > > or equal to the size of the file. > > > > > > I'd suggest we follow this, since it is the closest to a standard tha= t > there is. > > > > That sounds like behavior 2: return ENXIO at EOF. For reference, do > > you have a link to that somewhere? > 0000415: add SEEK_HOLE, SEEK_DATA to lseek - Austin Group Defect > Tracker (austingroupbugs.net) > If this doesn't give you a link (gmail never shows the raw url for me) > just google > "SEEK_HOLE austin group". > You have to join the mailing list to have access. It's easy to do. You can then download the latest draft (which I think is the ballot draft, so will be quite close to final, usually just 'typos' and such are corrected before the published standard).This will be the next POSIX.1 standard, likely this year. So it's kinda hard to give an exact link :(. Warner --00000000000017e07c06154d17b1 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Apr 4, 2024 at 3:39=E2=80=AFP= M Rick Macklem <rick.macklem@g= mail.com> wrote:
On Thu, Apr 4, 2024 at 1:59=E2=80=AFPM Alan Somers <asomers@freebsd.org> wr= ote:
>
> On Thu, Apr 4, 2024 at 2:56=E2=80=AFPM Rick Macklem <rick.macklem@gmail.com>= ; wrote:
> >
> > On Thu, Apr 4, 2024 at 11:15=E2=80=AFAM Alan Somers <asomers@freebsd.org&g= t; wrote:
> > >
> > > tldr; there are two problems:
> > > 1) tmpfs handles SEEK_HOLE differently than other file syste= ms
> > > 2) everything else handles SEEK_HOLE at EOF poorly, IMHO
> > >
> > > Details:
> > >
> > > According to lseek(2), SEEK_HOLE should return the start of = the next
> > > hole greater than or equal to the supplied offset.=C2=A0 Als= o, each file
> > > has a zero-sized virtual hole at the very end of the file.= =C2=A0 So I would
> > > expect that calling SEEK_HOLE at EOF would return the file&#= 39;s size.
> > > However, the man page also says that SEEK_HOLE will return E= NXIO when
> > > the offset points to EOF.=C2=A0 Those two statements seem co= ntradictory to
> > > me.=C2=A0 The first behavior seems more logical.=C2=A0 I wou= ld expect SEEK_HOLE
> > > to work the same way both at EOF and at any other file offse= t.
> > >
> > > What does the spec say?
> > >
> > > There is no POSIX standard for this.=C2=A0 It was invented b= y Solaris,
> > > Illumos's man page does not say clearly say what should = happen at EOF.
> > > Linux's man page is clear: "whence is SEEK_DATA or = SEEK_HOLE, and
> > > offset is beyond the end of the file".=C2=A0 That would= seem to indicate
> > > behavior 1: SEEK_HOLE should return the file's size at E= OF.=C2=A0 Only
> > > beyond EOF should it return ENXIO.
> > Well, there is the Austin Group stuff (never ratified by POSIX as= I
> > understand it).
> >
> > Here's what it says about SEEK_HOLE and offset:
> > If whence is SEEK_HOLE, the file offset shall be set to the small= est
> > location of a byte within a hole and not less than offset, except= that
> > if offset falls within the last hole, then the file offset may be= set
> > to the file size instead. It shall be an error if offset is great= er
> > or equal to the size of the file.
> >
> > I'd suggest we follow this, since it is the closest to a stan= dard that there is.
>
> That sounds like behavior 2: return ENXIO at EOF.=C2=A0 For reference,= do
> you have a link to that somewhere?
0000415: add SEEK_HOLE, SEEK_DATA to lseek - Austin Group Defect
Tracker (austingroupbugs.net)
If this doesn't give you a link (gmail never shows the raw url for me)<= br> just google
"SEEK_HOLE austin group".

You= have to join the mailing list to have access. It's easy to do. You can= then download the latest draft (which I think is the ballot draft, so will= be quite close to final, usually just 'typos' and such are correct= ed before the published standard).This will be the next POSIX.1 standard, l= ikely this year.=C2=A0

So it's kinda hard to g= ive an exact link :(.

Warner
--00000000000017e07c06154d17b1-- From nobody Fri Apr 5 05:43:13 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V9nTQ50Xhz5Fgfs for ; Fri, 5 Apr 2024 05:43:22 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4V9nTQ2YWDz4HJs; Fri, 5 Apr 2024 05:43:21 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Authentication-Results: mx1.freebsd.org; none Received: from critter.freebsd.dk (unknown [192.168.55.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by phk.freebsd.dk (Postfix) with ESMTPS id CE0F489293; Fri, 5 Apr 2024 05:43:13 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.18.1/8.16.1) with ESMTPS id 4355hDMC009861 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 5 Apr 2024 05:43:13 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.18.1/8.16.1/Submit) id 4355hDcS009860; Fri, 5 Apr 2024 05:43:13 GMT (envelope-from phk) Message-Id: <202404050543.4355hDcS009860@critter.freebsd.dk> To: Alan Somers cc: FreeBSD Hackers Subject: Re: SEEK_HOLE at EOF In-reply-to: From: "Poul-Henning Kamp" References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <9858.1712295793.1@critter.freebsd.dk> Date: Fri, 05 Apr 2024 05:43:13 +0000 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:1835, ipnet:130.225.0.0/16, country:EU] X-Rspamd-Queue-Id: 4V9nTQ2YWDz4HJs -------- Alan Somers writes: > Linux's man page is clear: "whence is SEEK_DATA or SEEK_HOLE, and > offset is beyond the end of the file". >[...] > Contrary to its man page, Linux behaves mostly like FreeBSD. SEEK_HOLE > returns ENXIO at EOF on most file systems. Just two minor quibbles: If the file position is EOF, then you /are/ "beyond the end of the file" because a read(2) would not be able to return any data. And returning ENXIO is more informative than returning the size of the file, since it atomically tells you that there are no more holes. If it returned the size of the file, you would have to make another syscall (opening a race) to check if what you got was EOF or a hole. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From nobody Fri Apr 5 11:25:39 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V9x4P5SRtz5GGDB for ; Fri, 5 Apr 2024 11:25:41 +0000 (UTC) (envelope-from des@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V9x4P4xhSz4mhk; Fri, 5 Apr 2024 11:25:41 +0000 (UTC) (envelope-from des@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1712316341; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=88iRtbgT/EsqmplkzMNgpiZcGm9M+5o6DTfJBgodRJ0=; b=oLdJrRk1jqJN5VccVQYBHDXxsXMoGQ6/H39OpkwWPAqtNB9Un6sLBc9HnjUYpeKkwQyUN8 rYPUDAJDHPsJeQ4AoSNKq/2MhAdmtQLFdv4o0qPH2COWd79v52reSOZ7HerFTUp4/b5vUn O8joxJ0UI55jc+xVSQnIt4Nmq3sioxrojbSQ46dngApA/enAH594UUXXaOMI7vJhxUN/83 vdCsnTVaCUjyxUR9aHb2wnXc6/QKfm0ccc20zWbr6L4yCAZ7exWUygmf2bii+ADZ2KrGkV zjNgUALlDjH9INZBi6OqmioDtxFxOlSgcUxMZ0ADWYdX6tCuI9q7d74ClAwExw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1712316341; a=rsa-sha256; cv=none; b=i/0sjW+JdQ7CeEII8ZL+yQen1yrxY9QIgwWgyBLPTKsOht+b//3yz5HOAoQIqjwApYL2zz MUjqbuluAgPD/2A5JRq4eC3XL5WTnylMZV5Jw3903XFgYOx5cKtPcoO8q9d3uEc8yHH9+G Heidd1oOt2uiGsAmLK2DG84opkR1FQwjy5qVEEKdvPvf2+REiZ9Pm2KgaUgqhpEcjXrR7G X2qALsW5wr5a0Itvn/IybL/ps6rWT/AoS8UJYmKGlpwoPiieGxL1DdW2RR3LNcwwNNo/qi G7jojFpiMd6JWenu3e+1JjaP/v3K5b+SBbpwSbUGP3QjvdMY4K2veLqft12YtQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1712316341; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=88iRtbgT/EsqmplkzMNgpiZcGm9M+5o6DTfJBgodRJ0=; b=hC3RxjJrWeAYjXNZSXxV87tOeo3PmXtLkpyWD0h41whWnQsuNujf3BHP9c86DfzxqEGCD5 EKlBJ6lI+c6usDTGWFcRL1zwBJfkVuoUtfnY9FqLHsHrs1UYt3/dX3Oumqt4ogXNJbEoj3 ZrSVD45E8XDFBIoj4HyFf+Nfjbavg2FTjT4ZptTWZjz6DaNIf1LWfvu9vw3t55TD6+beOV tUIBboceIFFBcGazLr8L6pOWNFtjcXLtP8TgKu1O6Z6Vzr7wQvVn6GhWFYqRCsWTDEg9LQ ISFyZ4o0/XgavewRlyrUmFoEWKlISVjjEjPVd89fW6DMPbXa8vt/+M006HN/Sg== Received: from ltc.des.dev (163.23.65.37.rev.sfr.net [37.65.23.163]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: des) by smtp.freebsd.org (Postfix) with ESMTPSA id 4V9x4P3m7nz173h; Fri, 5 Apr 2024 11:25:41 +0000 (UTC) (envelope-from des@freebsd.org) Received: by ltc.des.dev (Postfix, from userid 1001) id 361767A5; Fri, 05 Apr 2024 13:25:39 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Warner Losh Cc: Rick Macklem , Alan Somers , FreeBSD Hackers Subject: Re: SEEK_HOLE at EOF In-Reply-To: (Warner Losh's message of "Thu, 4 Apr 2024 16:44:54 -0600") References: User-Agent: Gnus/5.13 (Gnus v5.13) Date: Fri, 05 Apr 2024 13:25:39 +0200 Message-ID: <86jzlc6ogc.fsf@ltc.des.dev> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Warner Losh writes: > Rick Macklem writes: > > Alan Somers writes: > > > That sounds like behavior 2: return ENXIO at EOF.=C2=A0 For reference= , do > > > you have a link to that somewhere? > > 0000415: add SEEK_HOLE, SEEK_DATA to lseek - Austin Group Defect > > Tracker (austingroupbugs.net) > > If this doesn't give you a link (gmail never shows the raw url for me) > > just google "SEEK_HOLE austin group". > You have to join the mailing list to have access. To get access to the drafts, yes, but the defect tracker is open: https://austingroupbugs.net/view.php?id=3D415 DES --=20 Dag-Erling Sm=C3=B8rgrav - des@FreeBSD.org From nobody Fri Apr 5 13:41:31 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VB05N23yMz5GTnP for ; Fri, 5 Apr 2024 13:41:44 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ua1-f48.google.com (mail-ua1-f48.google.com [209.85.222.48]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VB05M6qq4z42xT for ; Fri, 5 Apr 2024 13:41:43 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ua1-f48.google.com with SMTP id a1e0cc1a2514c-7e3555015a8so759260241.3 for ; Fri, 05 Apr 2024 06:41:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712324503; x=1712929303; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5zwR0lJ1LPV998qvSsM1rOystsOoaVLHxJbH1O2ELY0=; b=OW19+ZeE8PF4qq4Epn2cb8CErrYn3fykdHDqCzf/v/D+4SB/zfGZfPSqP38tX1DVrk 45Gd1pM186PVzEcSvVqHSLJYVkEYc2SYWybfZTaL6zIWW29HZC/iJcEjsAj0UHCeK79u oAX53Q5+8DRz/AORj3CAmn+0oqP0V1wDF8W2AaEWu2ND4ExfzvmZD2YY0wR6GMt9o5mF 5BilO8T2OaNloUVoqCc9zRAXiX1FnNEOmjrne5CZKjKeKTUuzdkhnhpJBH0mxRnZbuYL /wlh9UlRd1UdOAqDsCm3kjKzlQga/ngNp2OqqRL3tvsGTXJQL11RL4vL8SMu51kWImlv 5mNg== X-Gm-Message-State: AOJu0Yw0eXXyGuz/dPDif6ttp7XGsSMHzjUzba8ptSPSgSs9e7TlI+gi erOXQkpfQaH8lqkjhHt5htQt4hPJhSj8hD6WYLGxD5xZ82uU1v6U9bgrCp7ihLTib0gGz5LvFpB +yXxjxSs8IvKIfbHPpGaMxH2Ybh7XpjAh X-Google-Smtp-Source: AGHT+IHyLJSw3smpIG0uyT89yOs5BjVZp7G96sbNAzexY62tToSXU3Ss+GgNgKY9JbMKALhg3+7rkr4tf7tWqvPKRSI= X-Received: by 2002:a05:6122:1792:b0:4d3:3846:73bb with SMTP id o18-20020a056122179200b004d3384673bbmr1833706vkf.7.1712324502904; Fri, 05 Apr 2024 06:41:42 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: <202404050543.4355hDcS009860@critter.freebsd.dk> In-Reply-To: <202404050543.4355hDcS009860@critter.freebsd.dk> From: Alan Somers Date: Fri, 5 Apr 2024 07:41:31 -0600 Message-ID: Subject: Re: SEEK_HOLE at EOF To: Poul-Henning Kamp Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4VB05M6qq4z42xT On Thu, Apr 4, 2024 at 11:43=E2=80=AFPM Poul-Henning Kamp wrote: > > -------- > Alan Somers writes: > > > Linux's man page is clear: "whence is SEEK_DATA or SEEK_HOLE, and > > offset is beyond the end of the file". > >[...] > > Contrary to its man page, Linux behaves mostly like FreeBSD. SEEK_HOLE > > returns ENXIO at EOF on most file systems. > > Just two minor quibbles: > > If the file position is EOF, then you /are/ "beyond the end of the file" > because a read(2) would not be able to return any data. Do you distinguish between "at EOF" and "beyond EOF"? And does it not trouble you that calling SEEK_HOLE from the beginning of the "virtual hole at EOF" will return ENXIO, even though calling SEEK_HOLE from the beginning of any real hole will return the current offset? > > And returning ENXIO is more informative than returning the size of the > file, since it atomically tells you that there are no more holes. Ahh, that's a good point. It's the first point I've heard in favor of this option. Are you aware of any applications that need to know that? > > If it returned the size of the file, you would have to make another > syscall (opening a race) to check if what you got was EOF or a hole. > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetenc= e. From nobody Fri Apr 5 13:54:01 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VB0Mc11j5z5GVsp for ; Fri, 5 Apr 2024 13:54:04 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4VB0Mb591Vz44nf; Fri, 5 Apr 2024 13:54:03 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Authentication-Results: mx1.freebsd.org; none Received: from critter.freebsd.dk (unknown [192.168.55.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by phk.freebsd.dk (Postfix) with ESMTPS id 0F7418928B; Fri, 5 Apr 2024 13:54:02 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.18.1/8.16.1) with ESMTPS id 435Ds1ID086244 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 5 Apr 2024 13:54:01 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.18.1/8.16.1/Submit) id 435Ds1KX086243; Fri, 5 Apr 2024 13:54:01 GMT (envelope-from phk) Message-Id: <202404051354.435Ds1KX086243@critter.freebsd.dk> To: Alan Somers cc: FreeBSD Hackers Subject: Re: SEEK_HOLE at EOF In-reply-to: From: "Poul-Henning Kamp" References: <202404050543.4355hDcS009860@critter.freebsd.dk> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <86241.1712325241.1@critter.freebsd.dk> Date: Fri, 05 Apr 2024 13:54:01 +0000 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:1835, ipnet:130.225.0.0/16, country:EU] X-Rspamd-Queue-Id: 4VB0Mb591Vz44nf -------- Alan Somers writes: > On Thu, Apr 4, 2024 at 11:43=E2=80=AFPM Poul-Henning Kamp dk> wrote: > > Just two minor quibbles: > > > > If the file position is EOF, then you /are/ "beyond the end of the file" > > because a read(2) would not be able to return any data. > > Do you distinguish between "at EOF" and "beyond EOF"? And does it not > trouble you that calling SEEK_HOLE from the beginning of the "virtual > hole at EOF" will return ENXIO, even though calling SEEK_HOLE from the > beginning of any real hole will return the current offset? EOF is where the file ends and there's no "hole" there, because there no more file on the other side of that "hole". When you stand on a cliff, the ocean is not "a hole in the landscape", it's where the landscape ends. > > And returning ENXIO is more informative than returning the size of the > > file, since it atomically tells you that there are no more holes. > > Ahh, that's a good point. It's the first point I've heard in favor of > this option. Are you aware of any applications that need to know > that? No, but that should not get in the way of good syscall architecture :-) It might be useful for archivers which try to be smart about sparse files. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From nobody Fri Apr 5 14:13:18 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VB0p3658Nz5GXgg for ; Fri, 5 Apr 2024 14:13:31 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ot1-x32d.google.com (mail-ot1-x32d.google.com [IPv6:2607:f8b0:4864:20::32d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VB0p33646z47DZ; Fri, 5 Apr 2024 14:13:31 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ot1-x32d.google.com with SMTP id 46e09a7af769-6e89b6daa1fso1398183a34.0; Fri, 05 Apr 2024 07:13:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712326410; x=1712931210; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=EMUrHZK0O+csdYb7ATktvE273n4pSUHgg/bOxwt/SBo=; b=jr5nPoopxURCdmkr5J0ia5n/LiLGoQBClu2Gv8obdVf//MiNOQlLntH6lQD9Hq0bWj 3yztzvwRsH4JlPsiwQdjPoS0vVcXBwbVDw7bZ1P5PyuJuB660beRK3TkXnILtbc/Qe7D c4c91jCnBpgwUn22DWqnREOWwIyKcXybJClkSeEx62ETLu4vYizklpX8PeuTE+xhC6Uo q+6One+rJEjz4wr3PoXZEOTGUtq9MJ5f1RbHIqcZmuLzLDzBrsGmXJM4DEqAjo0RQAb9 /+LiM+pFBEvv7GH/Ca40lkjYzUmonUKpVOg+pgqdiN0tvAPiAl+BhSQ1S6i46H18nObq I+wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712326410; x=1712931210; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EMUrHZK0O+csdYb7ATktvE273n4pSUHgg/bOxwt/SBo=; b=SZNJaAOpg5za7frLCki9T321osSVukNcTrtiSar7J/kBPO+/d4bMYukwRQJIJMDqxR 9cGtSnwTV/K+/ImIpHm5rGg/Me04CyWLrDY/R1I4iSKmT6iyDhVhMrcdkqNdAUuBkTQE lqAYSv3tCGscAH1XdqGPo/h7AELulIwn15JIX+SGbyGy1QGhaUvaIWeubTZp12bMlZzf Q8LQi04+Ylbbhvyh3R0oGYGQzrQMrHYoW9ElMFQ1sLKsNMLFjlp+aDTKeBczJhC2pE2D dolTe0EBwt7JAyRbiymjn2Yiw+ddfN1/qK/5+XutQ41mLHmTThKo7Zudj/O619hF8Nkr lFJg== X-Forwarded-Encrypted: i=1; AJvYcCWlSEJXtWirNKteqbTLvJQHchnl4bDstsPOvjAnUSrb6nNkLvKwkuQgqu3hkPPH2TvyVUdFJNbaWP36bohNNbF80FVrP8pJfX6q9PA= X-Gm-Message-State: AOJu0Yxgav7k3Kd72JNKP/oRgeujFY/WSS4zw/t7tQIlYtkKvVKMWHuo ZxwbUKQ7k2Jum63+5SvHwhOE59XXNuCn86IB414NZMcSNoQUi4zo+X64LHtfvonoXIKJpcMupbB iqjNkNKewOLhik9EzeGBm7Psw/oMpbjcyAGk= X-Google-Smtp-Source: AGHT+IHbo6Ur6/Ho6ctHrPlyzE8b8Lv1Ih6j8AiWUXjJn6vDK79iLxptyCpPMdyiVoxRrzyqPNZfda8RJ5Coxil9YIo= X-Received: by 2002:a05:6808:1806:b0:3c5:d2f7:cbaa with SMTP id bh6-20020a056808180600b003c5d2f7cbaamr1765183oib.13.1712326409985; Fri, 05 Apr 2024 07:13:29 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: <202404050543.4355hDcS009860@critter.freebsd.dk> <202404051354.435Ds1KX086243@critter.freebsd.dk> In-Reply-To: <202404051354.435Ds1KX086243@critter.freebsd.dk> From: alan somers Date: Fri, 5 Apr 2024 08:13:18 -0600 Message-ID: Subject: Re: SEEK_HOLE at EOF To: Poul-Henning Kamp Cc: Alan Somers , FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4VB0p33646z47DZ On Fri, Apr 5, 2024 at 7:54=E2=80=AFAM Poul-Henning Kamp wrote: > > -------- > Alan Somers writes: > > On Thu, Apr 4, 2024 at 11:43=3DE2=3D80=3DAFPM Poul-Henning Kamp > dk> wrote: > > > > Just two minor quibbles: > > > > > > If the file position is EOF, then you /are/ "beyond the end of the fi= le" > > > because a read(2) would not be able to return any data. > > > > Do you distinguish between "at EOF" and "beyond EOF"? And does it not > > trouble you that calling SEEK_HOLE from the beginning of the "virtual > > hole at EOF" will return ENXIO, even though calling SEEK_HOLE from the > > beginning of any real hole will return the current offset? > > EOF is where the file ends and there's no "hole" there, because there > no more file on the other side of that "hole". > > When you stand on a cliff, the ocean is not "a hole in the landscape", > it's where the landscape ends. Except there is a hole at EOF, a virtual hole. The draft spec specifically says "all seekable files shall have a virtual hole starting at the current size of the file". > > > > And returning ENXIO is more informative than returning the size of th= e > > > file, since it atomically tells you that there are no more holes. > > > > Ahh, that's a good point. It's the first point I've heard in favor of > > this option. Are you aware of any applications that need to know > > that? > > No, but that should not get in the way of good syscall architecture :-) > > It might be useful for archivers which try to be smart about sparse files= . I imagine that most archivers would work like this: ofs =3D 0 loop { let start =3D lseek(fd, ofs, SEEK_DATA); if ENXIO { // No more data regions break } let end =3D lseek(fd, ofs, SEEK_HOLE); assert!(!ENXIO) // thanks to the virtual hole, we should never have ENXIO here copy(fd, start, end - start, ...) ofs =3D end } truncate(output_file, fd.fsize) Since archivers really only care about data regions, not holes, I don't think that they would usually call SEEK_HOLE at EOF. > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetenc= e. From nobody Fri Apr 5 14:18:36 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VB0vz0yRlz5GY2P for ; Fri, 5 Apr 2024 14:18:39 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4VB0vy4y02z49Gj; Fri, 5 Apr 2024 14:18:38 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Authentication-Results: mx1.freebsd.org; none Received: from critter.freebsd.dk (unknown [192.168.55.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by phk.freebsd.dk (Postfix) with ESMTPS id 04D488928B; Fri, 5 Apr 2024 14:18:36 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.18.1/8.16.1) with ESMTPS id 435EIabf087361 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 5 Apr 2024 14:18:36 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.18.1/8.16.1/Submit) id 435EIalt087360; Fri, 5 Apr 2024 14:18:36 GMT (envelope-from phk) Message-Id: <202404051418.435EIalt087360@critter.freebsd.dk> To: alan somers cc: Alan Somers , FreeBSD Hackers Subject: Re: SEEK_HOLE at EOF In-reply-to: From: "Poul-Henning Kamp" References: <202404050543.4355hDcS009860@critter.freebsd.dk> <202404051354.435Ds1KX086243@critter.freebsd.dk> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <87358.1712326716.1@critter.freebsd.dk> Content-Transfer-Encoding: quoted-printable Date: Fri, 05 Apr 2024 14:18:36 +0000 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:1835, ipnet:130.225.0.0/16, country:EU] X-Rspamd-Queue-Id: 4VB0vy4y02z49Gj -------- alan somers writes: > The draft spec specifically says "all seekable files shall have a virtua= l hole > starting at the current size of the file". I have never subscribed to the notion that people standardizing C and UNIX= were particular competent, so that carries no water with me. -- = Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe = Never attribute to malice what can adequately be explained by incompetence= . From nobody Fri Apr 5 14:23:20 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VB11c5SNKz5GYhK for ; Fri, 5 Apr 2024 14:23:32 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VB11c2Zrwz4CQN; Fri, 5 Apr 2024 14:23:32 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-6e703e0e5deso1972948b3a.3; Fri, 05 Apr 2024 07:23:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712327011; x=1712931811; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=ZgqB9irm2zD89tqI5J1B2zI4O+2L3bPCs8gndeHHYnM=; b=ap6K6Bn4O7hPm8xICGxy/mjoCp7M7/K3iCgUyGxtRuvnQvLaxI/JdXCIA2ugQ6OOKt L0CFTdzJ7DzVbDa4vybFKvfoUL55qSby1FtzOhv5Raqpq5uBo63VXbgQApkxM7di0TOg AirAk1oN8ogLfeKyvkHK50LYKfB9jSV+HwmG9H6Pon/yCyeKxIp8gTY9zAjOL7RSVRkR YKryZD2A0t52klJo08UmqO4Y/aJuZ4WVuqxfE9YAFN9moFYgfdHqJbirrxIMl5EAKaai qR+Y2MhVmDoZniQF69KWZJpsx9la/9r4FVLhH51sbQhXxPH/gLb045y3W6IF+Aw18c2c ywWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712327011; x=1712931811; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZgqB9irm2zD89tqI5J1B2zI4O+2L3bPCs8gndeHHYnM=; b=J9lS7n50zGDtcK+D+I0w/hgIcKc4/z54nBhKyDWHwzmeUOZIecF8599iyb6QboDYlM ED64kKSHf/q+BITEC02G+3wqJ+LUWSQmTDdwiJgz6RBIKJcGUIPIAMN6XG+Cr01I0Dnu cXTKOypC5lfrleewv37naxxUM/T5eJX2I+/CuC1XEXbNryl/ZKKc1qLPg4JDHa5i9qKd 0X1NtE8reKega8H6wT+ygSpeP4AGf1niFWutHee5z4LFnDmvwo8WPbXxa58M58UxyVMq fXYsBGbKhc8+fm4vARvyn1kKGOYKVSaH75pfDvbGcjCPt/Yk4YRb7nVe+Hh1NKQjxMjK AajQ== X-Forwarded-Encrypted: i=1; AJvYcCWPZBXcr1XSEOEeTHbY9THdXhskYkvQrG3YqVPnTPFajA/cy5+XPAZogG4m3pV1fBIkFIs53stc+f4Atk81AUEJ3LIR/IlyZdF5+3tIVIt2OBUHDAEP5aglpyMajygwkBY= X-Gm-Message-State: AOJu0Yx6W8sEd3G+SWBM6XXiY6dFo03epGcVtaQThKE2l9VyyLsgAbwy T4qGiL1FBzuF+CbeOqRf+7ZDd94FFF5nEMvBIKiKsP13SIh4lKA9y92AINYPnDZlWbLEclncKh0 HEPnBDqBVFyrw2OGT2rlgz2BO6syiD0A= X-Google-Smtp-Source: AGHT+IF049EW95kZXCGh3XpKGESTw3k7os+YmJMBk7knMvvufo2Ro/pDHLZyN+NaR1nk4QDx65GptHX/XzdesO4CzDg= X-Received: by 2002:a05:6a21:33aa:b0:1a3:c2dd:f1cd with SMTP id yy42-20020a056a2133aa00b001a3c2ddf1cdmr2012276pzb.56.1712327010711; Fri, 05 Apr 2024 07:23:30 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: <202404050543.4355hDcS009860@critter.freebsd.dk> <202404051354.435Ds1KX086243@critter.freebsd.dk> In-Reply-To: From: Rick Macklem Date: Fri, 5 Apr 2024 07:23:20 -0700 Message-ID: Subject: Re: SEEK_HOLE at EOF To: alan somers Cc: Poul-Henning Kamp , Alan Somers , FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4VB11c2Zrwz4CQN On Fri, Apr 5, 2024 at 7:13=E2=80=AFAM alan somers wrot= e: > > On Fri, Apr 5, 2024 at 7:54=E2=80=AFAM Poul-Henning Kamp wrote: > > > > -------- > > Alan Somers writes: > > > On Thu, Apr 4, 2024 at 11:43=3DE2=3D80=3DAFPM Poul-Henning Kamp > > dk> wrote: > > > > > > Just two minor quibbles: > > > > > > > > If the file position is EOF, then you /are/ "beyond the end of the = file" > > > > because a read(2) would not be able to return any data. > > > > > > Do you distinguish between "at EOF" and "beyond EOF"? As a bit of an aside, NFSv4.2 does differentiate between "at EOF" and "beyond EOF" for its Seek operation. The fun part is that Linux did not implement what is in the RFC and shipped to many before the "bug" was noticed (and still do not conform to the RFC afaik). As such, there are now two ways to do it, The RFC way or the Linux way. Selecting between them is what the sysctl vfs.nfsd.linux42server does. > > > And does it not > > > trouble you that calling SEEK_HOLE from the beginning of the "virtual > > > hole at EOF" will return ENXIO, even though calling SEEK_HOLE from th= e > > > beginning of any real hole will return the current offset? > > > > EOF is where the file ends and there's no "hole" there, because there > > no more file on the other side of that "hole". > > > > When you stand on a cliff, the ocean is not "a hole in the landscape", > > it's where the landscape ends. > > Except there is a hole at EOF, a virtual hole. The draft spec > specifically says "all seekable files shall have a virtual hole > starting at the > current size of the file". I think that they used the term "virtual" to indicate this is not a real ho= le and I think it was a good idea, since it allows file systems that do not support holes to support SEEK_DATA. However, I still believe that conforming to the Austin Group draft is preferable. rick > > > > > > > And returning ENXIO is more informative than returning the size of = the > > > > file, since it atomically tells you that there are no more holes. > > > > > > Ahh, that's a good point. It's the first point I've heard in favor o= f > > > this option. Are you aware of any applications that need to know > > > that? > > > > No, but that should not get in the way of good syscall architecture :-) > > > > It might be useful for archivers which try to be smart about sparse fil= es. > > I imagine that most archivers would work like this: > ofs =3D 0 > loop { > let start =3D lseek(fd, ofs, SEEK_DATA); > if ENXIO { > // No more data regions > break > } > let end =3D lseek(fd, ofs, SEEK_HOLE); > assert!(!ENXIO) // thanks to the virtual hole, we should never > have ENXIO here > copy(fd, start, end - start, ...) > ofs =3D end > } > truncate(output_file, fd.fsize) > > Since archivers really only care about data regions, not holes, I > don't think that they would usually call SEEK_HOLE at EOF. > > > > > -- > > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > > phk@FreeBSD.ORG | TCP/IP since RFC 956 > > FreeBSD committer | BSD since 4.3-tahoe > > Never attribute to malice what can adequately be explained by incompete= nce. > From nobody Fri Apr 5 14:26:33 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VB15J5zzPz5GYYd for ; Fri, 5 Apr 2024 14:26:44 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-pf1-x432.google.com (mail-pf1-x432.google.com [IPv6:2607:f8b0:4864:20::432]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VB15J42P2z4DDS; Fri, 5 Apr 2024 14:26:44 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-6ecfd29f65dso892491b3a.0; Fri, 05 Apr 2024 07:26:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712327203; x=1712932003; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=GON+WPW6YwNqLnDTjt0M1/pmYIe7ejJREK7E+LTvc5o=; b=kVdi6XuZc//hsqLItf/BYk0JG9lbh9Ux/Hr3z/c4m7w4Nqc1eyp1hrBvzlttrEnCJi XEWTVkiPTLGJbodSgOPS5N0X6qmBqEy/AF7rLCBJrMrINV95FXZ/k9EO8Mqcdl0EYtyb ss6dvQGpUfuu7WxW/9Hpc5P9DTEr6HKVh4Kba+tGYcsAllaHrUibaC0FOyVzqT/xqbYG zVnrR4qkRHsc9BJaBmG21hX79g1kU8j8yAHAI69HBlPWlM8dwIpBByHGOtGs9FEShNTG oZCMK3rIaTs884fp/zfRVRDBatKtEXnHOe3MKppUOER0Jg8RzL8NrMIbD//avinXL7ks sQOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712327203; x=1712932003; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GON+WPW6YwNqLnDTjt0M1/pmYIe7ejJREK7E+LTvc5o=; b=jXzFujjS8UseiPlVk2OFI/mkuAGL4H+YqkIqzs/wQvDd38S73ynVYJlkSkg4oszr3d ruyEoJSbGzMymKzNgSI0CA1fvBUBYbhf3uZXGZom3kokaW7vVuWMavOb/PsJGm4lt2d2 Nn5pkaQmUME0xO21OhoXX+z5Xhz62lahm/cdV1jzyAa5sqG/cBCR8Fp/UhOfSqPi3cGd sAD1plfopE+6T+kywL6yB1HcicayGIdEYkxM8+8WqRvbj6I2YQFCdTnUcBDvY+aFZPRu vckA/2m0C+A3tBvfaPOTlohBM4xLpTjTDcDtnEV6NrTNR/9q8JYKYClZ7SPclUeumEcf IzHQ== X-Forwarded-Encrypted: i=1; AJvYcCX/zZ82g1sJvJzTulsQes/48Dv2JLrkVdpGnssWrNv2ns0NUSdMhS7W3PxqIEwWW3BPPmrkCPC9xg8EHBpdG3JlX0XjvwZMFUOoUALN6FJH3XW1bIrgxcALy0C776kp9cQ= X-Gm-Message-State: AOJu0YyR0CwYB/jAwyFWpGVmf9ahOCC5dxkjo6s8AOPslcHUDf30yCwv 9qjOyKziH+qk8fJgdNICxLeFlod3zvSvhP0Yt3h0FL9Bd2wXzCH1PeCG/oQHG4ZvPl7qVFuC14F smJhIJmG86nvmmcgTL9ScjwGosg== X-Google-Smtp-Source: AGHT+IEckNTu3Gxp3PuPmyVHDkUyx0DkhAjSkinHktlQg6IwrTFIZH3rTMg8G21I87kMnBuJla3nBOcs3PdhoO9T7hc= X-Received: by 2002:a05:6a20:244e:b0:1a7:e94:3d22 with SMTP id t14-20020a056a20244e00b001a70e943d22mr1900043pzc.0.1712327203249; Fri, 05 Apr 2024 07:26:43 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: <202404050543.4355hDcS009860@critter.freebsd.dk> <202404051354.435Ds1KX086243@critter.freebsd.dk> <202404051418.435EIalt087360@critter.freebsd.dk> In-Reply-To: <202404051418.435EIalt087360@critter.freebsd.dk> From: Rick Macklem Date: Fri, 5 Apr 2024 07:26:33 -0700 Message-ID: Subject: Re: SEEK_HOLE at EOF To: Poul-Henning Kamp Cc: alan somers , Alan Somers , FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4VB15J42P2z4DDS On Fri, Apr 5, 2024 at 7:18=E2=80=AFAM Poul-Henning Kamp wrote: > > -------- > alan somers writes: > > > The draft spec specifically says "all seekable files shall have a virtu= al hole > > starting at the current size of the file". > > I have never subscribed to the notion that people standardizing C and UNI= X were > particular competent, so that carries no water with me. Sure, but choosing to be non-conformant just creates portability problems. rick > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetenc= e. > From nobody Fri Apr 5 18:15:37 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VB69h6WY3z5GwDd for ; Fri, 5 Apr 2024 18:15:52 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4VB69f28Nyz4fQb; Fri, 5 Apr 2024 18:15:49 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of junchoon@dec.sakura.ne.jp designates 153.125.133.21 as permitted sender) smtp.mailfrom=junchoon@dec.sakura.ne.jp Received: from kalamity.joker.local (123-1-21-232.area1b.commufa.jp [123.1.21.232]) (authenticated bits=0) by www121.sakura.ne.jp (8.17.1/8.17.1/[SAKURA-WEB]/20201212) with ESMTPA id 435IFb1o023783; Sat, 6 Apr 2024 03:15:38 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Sat, 6 Apr 2024 03:15:37 +0900 From: Tomoaki AOKI To: Rick Macklem Cc: alan somers , Poul-Henning Kamp , Alan Somers , FreeBSD Hackers Subject: Re: SEEK_HOLE at EOF Message-Id: <20240406031537.69bc9e9fc6724171fe604fa8@dec.sakura.ne.jp> In-Reply-To: References: <202404050543.4355hDcS009860@critter.freebsd.dk> <202404051354.435Ds1KX086243@critter.freebsd.dk> Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spamd-Bar: - X-Spamd-Result: default: False [-1.12 / 15.00]; NEURAL_HAM_LONG(-1.00)[-0.996]; NEURAL_HAM_SHORT(-0.52)[-0.520]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:153.125.133.16/28]; ONCE_RECEIVED(0.10)[]; MIME_GOOD(-0.10)[text/plain]; NEURAL_SPAM_MEDIUM(0.09)[0.095]; HAS_ORG_HEADER(0.00)[]; TO_DN_ALL(0.00)[]; FREEMAIL_CC(0.00)[gmail.com,phk.freebsd.dk,freebsd.org]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[sakura.ne.jp]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP]; RCPT_COUNT_FIVE(0.00)[5]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; R_DKIM_NA(0.00)[]; TAGGED_RCPT(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[] X-Rspamd-Queue-Id: 4VB69f28Nyz4fQb On Fri, 5 Apr 2024 07:23:20 -0700 Rick Macklem wrote: > On Fri, Apr 5, 2024 at 7:13 AM alan somers wrote: > > > > On Fri, Apr 5, 2024 at 7:54 AM Poul-Henning Kamp wrote: > > > > > > -------- > > > Alan Somers writes: > > > > On Thu, Apr 4, 2024 at 11:43=E2=80=AFPM Poul-Henning Kamp > > > dk> wrote: > > > > > > > > Just two minor quibbles: > > > > > > > > > > If the file position is EOF, then you /are/ "beyond the end of the file" > > > > > because a read(2) would not be able to return any data. > > > > > > > > Do you distinguish between "at EOF" and "beyond EOF"? > As a bit of an aside, NFSv4.2 does differentiate between "at EOF" > and "beyond EOF" for its Seek operation. > The fun part is that Linux did not implement what is in the RFC and shipped > to many before the "bug" was noticed (and still do not conform to the RFC > afaik). As such, there are now two ways to do it, The RFC way or the Linux > way. Selecting between them is what the sysctl vfs.nfsd.linux42server does. > > > > > And does it not > > > > trouble you that calling SEEK_HOLE from the beginning of the "virtual > > > > hole at EOF" will return ENXIO, even though calling SEEK_HOLE from the > > > > beginning of any real hole will return the current offset? > > > > > > EOF is where the file ends and there's no "hole" there, because there > > > no more file on the other side of that "hole". > > > > > > When you stand on a cliff, the ocean is not "a hole in the landscape", > > > it's where the landscape ends. > > > > Except there is a hole at EOF, a virtual hole. The draft spec > > specifically says "all seekable files shall have a virtual hole > > starting at the > > current size of the file". > I think that they used the term "virtual" to indicate this is not a real hole > and I think it was a good idea, since it allows file systems that do not > support holes to support SEEK_DATA. > > However, I still believe that conforming to the Austin Group draft is > preferable. > > rick Not read the specs and codes, so this is just a point of view from an admin and a user. I think what admins/users want woulde be that: *File size with holes seen using `ls -l` should be ALWAYS the size when ALL existed holes are completely filled, including virtual hole at the EOF (classic wording of End Of File). *If the word EOF here means the End Of Filled part, it SHALL be called differently (for example, EOFP?) to clarify. *The virlal hole should be considered as an actual hole which ends at just at classic EOF and sized (file size - position of EOFP). So there cannot be any virtual holes but called so just for convenience. Otherwise, how admins/users manage capacities left on the FS/Quota which contains the sparse file? To be clearer, for text files having EOF character code (to be clear, call it as EOFC here) at the end, and said EOF means where the EOFC is at, yes, it can be the problem raised. But it had been often happening in the wild, usually to avoid errors on next append. (As non-intentional case, cluster gaps are look-alike, but different actually and causing newbies confused, "why free disk [partition] space is not equals to the size that is subtracting summed file sizes exists from whole disk space?".) Recently, more confusions exist with FS-level compressions like lz4 compression on ZFS datasets, though. Regards > > > > > And returning ENXIO is more informative than returning the size of the > > > > > file, since it atomically tells you that there are no more holes. > > > > > > > > Ahh, that's a good point. It's the first point I've heard in favor of > > > > this option. Are you aware of any applications that need to know > > > > that? > > > > > > No, but that should not get in the way of good syscall architecture :-) > > > > > > It might be useful for archivers which try to be smart about sparse files. > > > > I imagine that most archivers would work like this: > > ofs = 0 > > loop { > > let start = lseek(fd, ofs, SEEK_DATA); > > if ENXIO { > > // No more data regions > > break > > } > > let end = lseek(fd, ofs, SEEK_HOLE); > > assert!(!ENXIO) // thanks to the virtual hole, we should never > > have ENXIO here > > copy(fd, start, end - start, ...) > > ofs = end > > } > > truncate(output_file, fd.fsize) > > > > Since archivers really only care about data regions, not holes, I > > don't think that they would usually call SEEK_HOLE at EOF. > > > > > > > > -- > > > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > > > phk@FreeBSD.ORG | TCP/IP since RFC 956 > > > FreeBSD committer | BSD since 4.3-tahoe > > > Never attribute to malice what can adequately be explained by incompetence. -- Tomoaki AOKI From nobody Sat Apr 6 20:27:12 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VBn304q82z5HHff for ; Sat, 6 Apr 2024 20:27:24 +0000 (UTC) (envelope-from rockyhotas@tilde.team) Received: from tilde.team (tilde.team [IPv6:2607:5300:60:4f58::248]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4VBn2z5Rkzz4W77 for ; Sat, 6 Apr 2024 20:27:23 +0000 (UTC) (envelope-from rockyhotas@tilde.team) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=tilde.team header.s=mail header.b=Fb16vaNK; dmarc=pass (policy=reject) header.from=tilde.team; spf=pass (mx1.freebsd.org: domain of rockyhotas@tilde.team designates 2607:5300:60:4f58::248 as permitted sender) smtp.mailfrom=rockyhotas@tilde.team DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tilde.team; s=mail; t=1712435235; bh=HHjarQ3wPRlmnsowe2El1zG1k2y+ihf97tlRMr7bEtU=; h=Date:From:To:Subject:Reply-To:From; b=Fb16vaNKvPzLxyzScyBjsDYLvZEusen5P+4LSQtVu/1zlOjpmfaewaVV1aqd7JRJW bEwrlk7waTDnRsKxC4zKOaeTIRKhV4DMn4jrRZ/f75nDtgYsbiHGjFAmqhtZq4YLKR g/9sOL6R8V1EitIk9ijH2CqN1CnQfFhUaQAePYDQ= Received: from localhost (mob-5-91-202-243.net.vodafone.it [5.91.202.243]) by tilde.team (Postfix) with ESMTPSA id E6C0D4C1D88 for ; Sat, 6 Apr 2024 20:27:14 +0000 (UTC) Date: Sat, 6 Apr 2024 22:27:12 +0200 From: Rocky Hotas To: freebsd-hackers@freebsd.org Subject: Kernel module: return a number from a device Message-ID: Reply-To: Rocky Hotas List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.40 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[tilde.team,reject]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_ALLOW(-0.20)[tilde.team:s=mail]; ONCE_RECEIVED(0.10)[]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_ALL(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[tilde.team:+]; RCVD_COUNT_ONE(0.00)[1]; HAS_REPLYTO(0.00)[rockyhotas@tilde.team]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MISSING_XM_UA(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; ASN(0.00)[asn:16276, ipnet:2607:5300::/32, country:FR]; REPLYTO_EQ_FROM(0.00)[] X-Rspamd-Queue-Id: 4VBn2z5Rkzz4W77 Hello! I'm trying to write a simple kernel module, using as a model the example in I am a newbie. My module should be simpler than the one in the link: it should just create a read-only /dev/rolld file; each time it is read by the user (for example through `cat'), the file should provide a random number mod d_size. So, the "output" should always be 1 character. I modified the echo kernel module presented in the link. My module can successfully be loaded into the kernel and the device is created, but if I run as a user `cat /dev/rolld': $ cat /dev/rolld Opened device "rolld" successfully. and it hangs, giving no more output and without generating an error. May be this due to the fact that uiomove receives a pointer &random_out, which is a pointer to a uint32_t instead of for example a char? (And if this is the issue, how to convert a uint32_t to char inside the kernel?) Or is there some other error that I made? I paste my code below. Bye! Rocky #include #include #include #include #include #include #include #include #include static d_open_t rolld_open; static d_close_t rolld_close; static d_read_t rolld_read; static struct cdevsw rolld_cdevsw = { .d_version = D_VERSION, .d_open = rolld_open, .d_close = rolld_close, .d_read = rolld_read, .d_name = "rolld", }; /* vars */ static struct cdev *rolld_dev; static uint32_t d_size = 6; static int rolld_loader(struct module *m __unused, int what, void *arg __unused) { int error = 0; switch (what) { case MOD_LOAD: /* kldload */ error = make_dev_p(MAKEDEV_CHECKNAME | MAKEDEV_WAITOK, &rolld_dev, &rolld_cdevsw, 0, UID_ROOT, GID_WHEEL, 0444, "rolld"); if (error != 0) break; printf("Roll device loaded.\n"); break; case MOD_UNLOAD: destroy_dev(rolld_dev); printf("Roll device unloaded.\n"); break; default: error = EOPNOTSUPP; break; } return (error); } static int rolld_open(struct cdev *dev __unused, int oflags __unused, int devtype __unused, struct thread *td __unused) { int error = 0; uprintf("Opened device \"rolld\" successfully.\n"); return (error); } static int rolld_close(struct cdev *dev __unused, int fflag __unused, int devtype __unused, struct thread *td __unused) { uprintf("Closing device \"rolld\".\n"); return (0); } static int rolld_read(struct cdev *dev __unused, struct uio *uio, int ioflag __unused) { uint32_t random_out; uint32_t random_item; int error; random_item = arc4random(); random_out = random_item % d_size; if ((error = uiomove(&random_out, 1, uio)) != 0) uprintf("uiomove failed!\n"); return (error); } DEV_MODULE(rolld, rolld_loader, NULL); From nobody Sat Apr 6 20:39:26 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VBnSR0nKDz5HKDH for ; Sat, 6 Apr 2024 20:45:59 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VBnSQ3LXnz4Xqp for ; Sat, 6 Apr 2024 20:45:58 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-a51addddbd4so160684266b.0 for ; Sat, 06 Apr 2024 13:45:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1712436353; x=1713041153; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=WpnJbSX3L4dvFURo+NNQxR6rjlpAIaMYmExj0O6bLJk=; b=eO+DuNRMbgYxGo4JxKsiAEs498pDsKaY+xuYSHNOVMXXdgEiqijtFnhgGXk/yD4bzt tJc9LbQ6jL20/8fnLyImpv/EGeQ4gNYyYqIIBHdn0UkPxM5SCJfqgHUBlADzJ3+59vIl XQA4vLNWzv14M+b6Agi3e+nhKVJ4MK/l4MtJ07dAV0x6F96tKjrUpurG89eqJDhHFg3o p7174AtzWUSLDSleKcbgSOdM2jVQhKQ8T+ypKnh4bH26kPzCaEADEI6oomjLk5s7pVRo 7luEpLU5T+MXuLZ3STopSkZLpONiwaVWAmsbLajvT+YImgjJBaVTFQ83siVZMgYKp7JE E12g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712436353; x=1713041153; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=WpnJbSX3L4dvFURo+NNQxR6rjlpAIaMYmExj0O6bLJk=; b=BWxNexPILT1y1j/xyHg80YXi717WgAXbPmfOjWwYxdiwnaj04PvTEJfqtZMQjKYZEQ CfgEf1GAVk2l/l/mj4eP+S/DKR3D/GNGvWmu46/jT1ftTkri4xJIOHgJN8EzzY9M33IZ tZUBi4Z2tLSRLJJTln5XiGYn8jODqh8APrcpPWv4mCeplp6D0a6/UiOJKwwIkTok+32Y G9Da46Jtx6IajQXGqS08tF5vP1vGwBZLej0PoYRcxaND0bD7LiS4zfvWg3LyvhLcJrfd 2ivC/xgFGDcFqpf0SR0giZN+a5AgZ+1erp4mUEbBV+cXFaTKrnQeHW7OwebapOIPvw8V cofw== X-Gm-Message-State: AOJu0Yy9tpW8kyzkfaieMMtaMI0sB3pgeYlnz1I65evAG1aSbIYD8KxG PpzO21zjTOZ/54453mCIVeW5d4K1JLDikplYwWo8Twul0pTUFiJgnG1Kh+2rsX0gL14ag8Bjwu3 qc2S7nHAivgckfMqudfsPzWn9UAO/W/tHROmOWYSbfoyyjXAJ X-Google-Smtp-Source: AGHT+IGqgPU1fMsf6qVBBGqCRFzzDzpLuXCZSq2HOEgv6UMgpEX+S20cUDk35fd4Qw0xgzXMoclyjqUenfxCw9IThjk= X-Received: by 2002:a17:907:d92:b0:a51:c1e0:2ba9 with SMTP id go18-20020a1709070d9200b00a51c1e02ba9mr1217581ejc.11.1712435978743; Sat, 06 Apr 2024 13:39:38 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Sat, 6 Apr 2024 14:39:26 -0600 Message-ID: Subject: Re: Kernel module: return a number from a device To: Rocky Hotas Cc: FreeBSD Hackers Content-Type: multipart/alternative; boundary="00000000000036ce4a06157392a6" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4VBnSQ3LXnz4Xqp --00000000000036ce4a06157392a6 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable When this happens, hit ^t (control t). That will give a traceback of the call stack which may help you track down where it is hanging (most likely something is sleeping waiting for an event). Warner On Sat, Apr 6, 2024, 2:27=E2=80=AFPM Rocky Hotas wr= ote: > Hello! > I'm trying to write a simple kernel module, using as a model the example > in > > > > I am a newbie. My module should be simpler than the one in the link: it > should just create a read-only /dev/rolld file; each time it is read > by the user (for example through `cat'), the file should provide a > random number mod d_size. So, the "output" should always be 1 character. > > I modified the echo kernel module presented in the link. My module > can successfully be loaded into the kernel and the device is created, > but if I run as a user `cat /dev/rolld': > > $ cat /dev/rolld > Opened device "rolld" successfully. > > and it hangs, giving no more output and without generating an error. > > May be this due to the fact that uiomove receives a pointer &random_out, > which is a pointer to a uint32_t instead of for example a char? (And if > this is the issue, how to convert a uint32_t to char inside the kernel?) > > Or is there some other error that I made? > > I paste my code below. > > Bye! > > Rocky > > > > #include > #include > #include > #include > #include > #include > #include > #include > #include > > static d_open_t rolld_open; > static d_close_t rolld_close; > static d_read_t rolld_read; > > static struct cdevsw rolld_cdevsw =3D { > .d_version =3D D_VERSION, > .d_open =3D rolld_open, > .d_close =3D rolld_close, > .d_read =3D rolld_read, > .d_name =3D "rolld", > }; > > /* vars */ > static struct cdev *rolld_dev; > static uint32_t d_size =3D 6; > > static int > rolld_loader(struct module *m __unused, int what, void *arg __unused) > { > int error =3D 0; > > switch (what) { > case MOD_LOAD: /* kldload */ > error =3D make_dev_p(MAKEDEV_CHECKNAME | MAKEDEV_WAITOK, > &rolld_dev, > &rolld_cdevsw, > 0, > UID_ROOT, > GID_WHEEL, > 0444, > "rolld"); > if (error !=3D 0) > break; > > printf("Roll device loaded.\n"); > break; > case MOD_UNLOAD: > destroy_dev(rolld_dev); > printf("Roll device unloaded.\n"); > break; > default: > error =3D EOPNOTSUPP; > break; > } > return (error); > } > > static int > rolld_open(struct cdev *dev __unused, int oflags __unused, int devtype > __unused, > struct thread *td __unused) > { > int error =3D 0; > > uprintf("Opened device \"rolld\" successfully.\n"); > return (error); > } > > static int > rolld_close(struct cdev *dev __unused, int fflag __unused, int devtype > __unused, > struct thread *td __unused) > { > uprintf("Closing device \"rolld\".\n"); > return (0); > } > > static int > rolld_read(struct cdev *dev __unused, struct uio *uio, int ioflag __unuse= d) > { > uint32_t random_out; > uint32_t random_item; > int error; > > random_item =3D arc4random(); > random_out =3D random_item % d_size; > > if ((error =3D uiomove(&random_out, 1, uio)) !=3D 0) > uprintf("uiomove failed!\n"); > > return (error); > } > > DEV_MODULE(rolld, rolld_loader, NULL); > > --00000000000036ce4a06157392a6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
When this happens, hit ^t (control t). That will give a t= raceback of the call stack which may help you track down where it is hangin= g (most likely something is sleeping waiting for an event).

Warner=C2=A0

On Sat, Apr 6, 2024, 2:27= =E2=80=AFPM Rocky Hotas <rockyhotas@tilde.team> wrote:
Hello!
I'm trying to write a simple kernel module, using as a model the exampl= e
in

=C2=A0<https://docs.freeb= sd.org/en/books/arch-handbook/driverbasics/>

I am a newbie. My module should be simpler than the one in the link: it
should just create a read-only /dev/rolld file; each time it is read
by the user (for example through `cat'), the file should provide a
random number mod d_size. So, the "output" should always be 1 cha= racter.

I modified the echo kernel module presented in the link. My module
can successfully be loaded into the kernel and the device is created,
but if I run as a user `cat /dev/rolld':

$ cat /dev/rolld
Opened device "rolld" successfully.

and it hangs, giving no more output and without generating an error.

May be this due to the fact that uiomove receives a pointer &random_out= ,
which is a pointer to a uint32_t instead of for example a char? (And if
this is the issue, how to convert a uint32_t to char inside the kernel?)
Or is there some other error that I made?

I paste my code below.

Bye!

Rocky



#include <sys/types.h>
#include <sys/systm.h>
#include <sys/param.h>
#include <sys/module.h>
#include <sys/kernel.h>
#include <sys/conf.h>
#include <sys/uio.h>
#include <sys/malloc.h>
#include <sys/libkern.h>

static d_open_t=C2=A0 =C2=A0 =C2=A0 rolld_open;
static d_close_t=C2=A0 =C2=A0 =C2=A0rolld_close;
static d_read_t=C2=A0 =C2=A0 =C2=A0 rolld_read;

static struct cdevsw rolld_cdevsw =3D {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 .d_version =3D D_VERSION,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 .d_open =3D rolld_open,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 .d_close =3D rolld_close,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 .d_read =3D rolld_read,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 .d_name =3D "rolld",
};

/* vars */
static struct cdev *rolld_dev;
static uint32_t d_size =3D 6;

static int
rolld_loader(struct module *m __unused, int what, void *arg __unused)
{
=C2=A0 =C2=A0 =C2=A0 =C2=A0 int error =3D 0;

=C2=A0 =C2=A0 =C2=A0 =C2=A0 switch (what) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 case MOD_LOAD:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 /* kldload */
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 error =3D make_dev_= p(MAKEDEV_CHECKNAME | MAKEDEV_WAITOK,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 &= rolld_dev,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 &= rolld_cdevsw,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0, =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 UID_R= OOT,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 GID_W= HEEL,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0444,=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "= ;rolld");
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (error !=3D 0) =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 break;

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 printf("Roll d= evice loaded.\n");
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 break;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 case MOD_UNLOAD:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 destroy_dev(rolld_d= ev);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 printf("Roll d= evice unloaded.\n");
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 break;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 default:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 error =3D EOPNOTSUP= P;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 break;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
=C2=A0 =C2=A0 =C2=A0 =C2=A0 return (error);
}

static int
rolld_open(struct cdev *dev __unused, int oflags __unused, int devtype __un= used,
=C2=A0 =C2=A0 struct thread *td __unused)
{
=C2=A0 =C2=A0 =C2=A0 =C2=A0 int error =3D 0;

=C2=A0 =C2=A0 =C2=A0 =C2=A0 uprintf("Opened device \"rolld\"= successfully.\n");
=C2=A0 =C2=A0 =C2=A0 =C2=A0 return (error);
}

static int
rolld_close(struct cdev *dev __unused, int fflag __unused, int devtype __un= used,
=C2=A0 =C2=A0 struct thread *td __unused)
{
=C2=A0 =C2=A0 =C2=A0 =C2=A0 uprintf("Closing device \"rolld\"= ;.\n");
=C2=A0 =C2=A0 =C2=A0 =C2=A0 return (0);
}

static int
rolld_read(struct cdev *dev __unused, struct uio *uio, int ioflag __unused)=
{
=C2=A0 =C2=A0 =C2=A0 =C2=A0 uint32_t random_out;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 uint32_t random_item;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 int error;

=C2=A0 =C2=A0 =C2=A0 =C2=A0 random_item =3D arc4random();
=C2=A0 =C2=A0 =C2=A0 =C2=A0 random_out =3D random_item % d_size;

=C2=A0 =C2=A0 =C2=A0 =C2=A0 if ((error =3D uiomove(&random_out, 1, uio)= ) !=3D 0)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 uprintf("uiomo= ve failed!\n");

=C2=A0 =C2=A0 =C2=A0 =C2=A0 return (error);
}

DEV_MODULE(rolld, rolld_loader, NULL);

--00000000000036ce4a06157392a6-- From nobody Sat Apr 6 21:00:20 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VBnn356z1z5HKrX for ; Sat, 6 Apr 2024 21:00:23 +0000 (UTC) (envelope-from rockyhotas@tilde.team) Received: from tilde.team (tilde.team [IPv6:2607:5300:60:4f58::248]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4VBnn34d9Wz4bN1 for ; Sat, 6 Apr 2024 21:00:23 +0000 (UTC) (envelope-from rockyhotas@tilde.team) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tilde.team; s=mail; t=1712437222; bh=Jot0WVCXwJ5+qW21SFWudH+7QCjhehUkOpBcShLZFho=; h=Date:From:To:Cc:Subject:Reply-To:In-Reply-To:From; b=n01udT4+LwUomfh24Y3NyrQJq/KPHk5EXKXfl/ym44/wEyiIsYMyT7L4l5leIS2Ay RBgjFhNNkdcR1U/uwQCghcNCgUl8TWzINz3MgZn3cN6OE28E75AQGLG7y8yMXKdXZy g6DBrxl1fje81E8guDQmmTLIyh86zP5F/FZZFBBc= Received: from localhost (mob-5-91-202-243.net.vodafone.it [5.91.202.243]) by tilde.team (Postfix) with ESMTPSA id 07AAC4C1D30; Sat, 6 Apr 2024 21:00:21 +0000 (UTC) Date: Sat, 6 Apr 2024 23:00:20 +0200 From: Rocky Hotas To: freebsd-hackers@freebsd.org Cc: Warner Losh Subject: Re: Re: Kernel module: return a number from a device Message-ID: Reply-To: Rocky Hotas List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:16276, ipnet:2607:5300::/32, country:FR] X-Rspamd-Queue-Id: 4VBnn34d9Wz4bN1 On apr 06 14:39, Warner Losh wrote: > When this happens, hit ^t (control t). That will give a traceback of the > call stack which may help you track down where it is hanging (most likely > something is sleeping waiting for an event). Thanks! It seems that cat itself is hanging (so, uiomove can still be the culprit...): $ cat rolld Opened device "rolld" successfully. load: 0.44 cmd: cat 13392 [running] 7.67r 1.25u 6.39s 38% 1936k I also tried to modify rolld_read using only char variables: static int rolld_read(struct cdev *dev __unused, struct uio *uio, int ioflag __unused) { char random_out; char random_item; int error; random_item = (char) arc4random(); random_out = random_item % d_size; if ((error = uiomove(&random_out, 1, uio)) != 0) uprintf("uiomove failed!\n"); return (error); } But nothing changed with respect to the first version. Rocky From nobody Sun Apr 7 04:21:12 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VBzZ06y1jz5G7CG for ; Sun, 7 Apr 2024 04:21:28 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [IPv6:2a00:1450:4864:20::632]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VBzZ03cFmz4Kjc for ; Sun, 7 Apr 2024 04:21:28 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x632.google.com with SMTP id a640c23a62f3a-a44665605f3so377859366b.2 for ; Sat, 06 Apr 2024 21:21:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1712463685; x=1713068485; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=bd1XU7jJyUI4UCiG9SaTyKJXBirptqiypKKLjQiim08=; b=Vp//2XjiwERot+g4JZ5a0kIGPVR4gKiQ0kCtVKV179n2bvK7d+cAH6W761G9wkWGu7 QUm8ejAQ5XzqkNmawSw0RmDkBlI/7jOCZv8ikur59UAkoOyVP9QuCCq6mm2+1mWOWlYT O/86AJb5nvKkmt9CPH4HVBkkY5azbASPoLH1D86w/J6pbJ6mv2I17Tf1Jt2pdxSDuNUX ooJ3UH87rSEIjqeTqcjm3sfKZJNFPGkHTNM998Y9vW5+YcZ2frXlanR+ntGjxhp1/4I2 12I1BEIBaWBFVI1j3WEJgCT2bnfCICZd1wuj+BGDO2UQLFOJZ23it5yXkA+lFcSiWxkh Z9mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712463685; x=1713068485; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=bd1XU7jJyUI4UCiG9SaTyKJXBirptqiypKKLjQiim08=; b=hQANAiY1Yvf6hTTX8k3DNGk/P1b4Mkl5nWO+k9Ha/EnqKFtICKxa6qv5t3yLYQHmGA 90Si2P89V72FLh5gZfFsGnwc8wuhQd7dBwhPOUxaqtM0AwDsQ8KTQYbRLhA1QElUTygU jqBiopCDMJ7G9W4NwCHP9vTK4wB+6W+1AQg+d7RhzIF4A9xvPU/yMjT7Fr5XJKVgr/aj nqRh08IHdtJnYmCB4MTx+gaovkMWxfOilyOUCu5tZ2Qr5gLE2k09aXvI9s0wfbTxCvfl LLhHIjeo9p3GGzWu0ugmU6u2IpuQwm+rr6Y8NtdB60GWSbm7juvQUti7dBNI+Gvz3jWH P2VQ== X-Gm-Message-State: AOJu0Yyip2k9L2d82QJ0Ri9MNpVgcTvKq5Rr278Fzyveadpi3Ttvshb5 NCjV77NpOcUTk9dibuIv7LkqK8rxf6m/p1wcfjDZXLZoSaTUGJc1fDG1M7qPAhmqKiFu/OYwkfD 6zW6XQQOsE01X00Bx8/Rkal0XJiERoYYkIl4x1R8Wa++v79E01Us= X-Google-Smtp-Source: AGHT+IE/ldAPWyX70bRzES0EwNol7l2NGRefWtMOEEqvH6dGTjG1Vy8TB3wo2s9//u7Pv6mU7i12xu6dOAhSnl1bs6s= X-Received: by 2002:a17:906:bf47:b0:a51:cc20:9116 with SMTP id ps7-20020a170906bf4700b00a51cc209116mr483982ejb.68.1712463684959; Sat, 06 Apr 2024 21:21:24 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Sat, 6 Apr 2024 22:21:12 -0600 Message-ID: Subject: Re: Re: Kernel module: return a number from a device To: Rocky Hotas Cc: FreeBSD Hackers Content-Type: multipart/alternative; boundary="000000000000a21afa06157a0538" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4VBzZ03cFmz4Kjc --000000000000a21afa06157a0538 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Apr 6, 2024 at 3:00=E2=80=AFPM Rocky Hotas = wrote: > On apr 06 14:39, Warner Losh wrote: > > When this happens, hit ^t (control t). That will give a traceback of th= e > > call stack which may help you track down where it is hanging (most like= ly > > something is sleeping waiting for an event). > > Thanks! It seems that cat itself is hanging (so, uiomove can still be > the > culprit...): > > $ cat rolld > Opened device "rolld" successfully. > load: 0.44 cmd: cat 13392 [running] 7.67r 1.25u 6.39s 38% 1936k > running means there's a tight loop somewhere... uiomove doesn't do that. It is a bunch of ifs that go to a copyout. Arc4random shouldn't either. I'd add printf to see where. > I also tried to modify rolld_read using only char variables: > > static int > rolld_read(struct cdev *dev __unused, struct uio *uio, int ioflag > __unused) > { > char random_out; > char random_item; > int error; > > random_item =3D (char) arc4random(); > random_out =3D random_item % d_size; > > if ((error =3D uiomove(&random_out, 1, uio)) !=3D 0) > uprintf("uiomove failed!\n"); > > return (error); > } > > > But nothing changed with respect to the first version. > This should produce an infinite number of chars... maybe it is and d_size is 1 and they are all NULs. Try cat -v. Warner > Rocky > > --000000000000a21afa06157a0538 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Sat, Apr 6, 2024= at 3:00=E2=80=AFPM Rocky Hotas <rockyhotas@tilde.team> wrote:
On apr 06 14:39, Warne= r Losh wrote:
> When this happens, hit ^t (control t). That will give a traceback of t= he
> call stack which may help you track down where it is hanging (most lik= ely
> something is sleeping waiting for an event).

Thanks! It seems that cat itself is hanging (so, uiomove can still be
the
culprit...):

$ cat rolld
Opened device "rolld" successfully.
load: 0.44=C2=A0 cmd: cat 13392 [running] 7.67r 1.25u 6.39s 38% 1936k

running means there's a tight loop somew= here... uiomove doesn't do that. It is a bunch of ifs that go to a copy= out.=C2=A0 Arc4random shouldn't either. I'd add printf to see where= .


=C2=A0
I also tried to modify rolld_read using only char variables:

static int
rolld_read(struct cdev *dev __unused, struct uio *uio, int ioflag
__unused)
{
=C2=A0 =C2=A0 =C2=A0 =C2=A0 char random_out;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 char random_item;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 int error;

=C2=A0 =C2=A0 =C2=A0 =C2=A0 random_item =3D (char) arc4random();
=C2=A0 =C2=A0 =C2=A0 =C2=A0 random_out =3D random_item % d_size;

=C2=A0 =C2=A0 =C2=A0 =C2=A0 if ((error =3D uiomove(&random_out, 1, uio)= ) !=3D 0)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 uprintf("uiomo= ve failed!\n");

=C2=A0 =C2=A0 =C2=A0 =C2=A0 return (error);
}


But nothing changed with respect to the first version.

This should produce a= n infinite number of chars...=C2=A0 maybe it is and d_size is 1 and they ar= e all NULs. Try cat -v.

= Warner


Rocky

--000000000000a21afa06157a0538-- From nobody Sun Apr 7 10:50:00 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VC8BL4KqDz5GXvb for ; Sun, 7 Apr 2024 10:50:02 +0000 (UTC) (envelope-from des@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VC8BL3qlgz4yRq; Sun, 7 Apr 2024 10:50:02 +0000 (UTC) (envelope-from des@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1712487002; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=x/sRRRrpgfj/flzYtjRGDaxa0pcmZJWpWZscL22QJRI=; b=JsYf7RyQ2jH0UvqjdWyCq4kstEFvMzAo5L+87dVHXdlLNle/v2jxSldfQ4mik0QcSKwB+i pucvMkJyu4NzJRbpoKarziAuXEDem8WFE5WcwF7pRZ3C0VmvTjx8VL5YbiUDBLUWttF7DP Vn2yKChwO5xwAGPSQKfjgyV5kPUkj/wJKDi6zQLji7aJr/fdUSOFMcSAGzIYFbviPZ+R0R p6NFOrUbLKb87mN/ZYYvQv+ZCu6r3rccBkJM7yVzBFlNaOQpmIoIoz0Y48bZbrkYXDu068 8kN4wAbMSKzIaRbm7FDRot9FaA6GEG2dZxMkQJkhboOBbKLBfBIYgz8I4R+zew== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1712487002; a=rsa-sha256; cv=none; b=OUdqxJupJtWZ8ro/svzpKFMspPochjFAhMNP1G1z5E053MAySyXOMMbSyjzO8+74QpGZ2c fHb95iIR7zNwnvdakqIXIDPudeqL0eNIxG3Z+R/mhnHsodAZ2UFXZANeaYw5NgVDzW1zFa LUtuqqQzZI749Z8F5eDjkLPHh4DFXRlUOwevIMWeC16ep49Cwubmf0bNZiyHw2ywV+f7Dh xVNTmzWUc4lrLqGm9WbSH8d3PMml7CrIzTo2r7tRG7pgR194Wv0PtAICVGFGt9xbAJ/Fw3 vw+vRQQ9DM0xHpojDYkcaVEIMqWX+CAFOH11+tPhIADQv8OFpB0mxXNA2NyU4w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1712487002; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=x/sRRRrpgfj/flzYtjRGDaxa0pcmZJWpWZscL22QJRI=; b=MALh9ulri8mAacC2vQvJO5SvWqBhK9atORob5OBL3sQEy64/VsWfsEBHsfx7i9dIhPzeoj aPfSA+TQZD22mXX38cT9QRjx79iEX62N9xjYF6BsLD1BMsqj89Q04QmUpmHiqY9ETGCbFd FYLIUMK5ZNLpveRGcC7JkpcE6l9vkN8qz/RnkTk7xg/UfEZRgnhHzl0f1sIsq1KjyIHy87 ZIFowRtuNuGf484JnSNxPKzl9wDbkZ/xy99Z/W5qn3P9BP9Q9h6nCW7jAXhnOdHJ2vTTj1 8lC6QWE73z6rXt4DE/0phu878m01r4/BMnxCVzRAl6Mv2G8VjKWMJtmaGfHM5w== Received: from ltc.des.dev (163.23.65.37.rev.sfr.net [37.65.23.163]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: des) by smtp.freebsd.org (Postfix) with ESMTPSA id 4VC8BL2gtbz15q2; Sun, 7 Apr 2024 10:50:02 +0000 (UTC) (envelope-from des@freebsd.org) Received: by ltc.des.dev (Postfix, from userid 1001) id 0DC071E585; Sun, 07 Apr 2024 12:50:00 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Rocky Hotas Cc: freebsd-hackers@freebsd.org Subject: Re: Kernel module: return a number from a device In-Reply-To: (Rocky Hotas's message of "Sat, 6 Apr 2024 22:27:12 +0200") References: User-Agent: Gnus/5.13 (Gnus v5.13) Date: Sun, 07 Apr 2024 12:50:00 +0200 Message-ID: <86r0fh5twn.fsf@ltc.des.dev> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Rocky Hotas writes: > static int > rolld_read(struct cdev *dev __unused, struct uio *uio, int ioflag __unuse= d) > { > uint32_t random_out; > uint32_t random_item; > int error; > > random_item =3D arc4random(); > random_out =3D random_item % d_size; > > if ((error =3D uiomove(&random_out, 1, uio)) !=3D 0) > uprintf("uiomove failed!\n"); > > return (error); > } Using a uint32_t will work on little-endian systems (such as amd64) because the least-significant byte, which is the only non-zero byte, comes first. On big-endian systems, it would simply always return 0. Furthermore, this won't only return one byte; rather, it will return one byte _at a time_, very inefficiently. This is why cat appears to hang. To truly only return one byte, you need to look at uio->uio_offset and return 0 without calling uiomove(), signaling EOF, if it is non-zero. In summary, you should write rolld_read() as: uint8_t roll =3D arc4random() % d_size; if (uio->uio_offset > 0) return (0); return (uiomove(&roll, 1, uio)); You can also use uiomove_frombuf(), which will take care of that check for you. It's a bit overkill when you're only writing a single byte, but if you wanted to output text instead of binary, you could use this: char roll[2]; roll[0] =3D '0' + arc4random() % d_size; roll[1] =3D '\n'; return (uiomove_frombuf(roll, sizeof(roll), uio)); Obviously, this will only work for d_size <=3D 9. For larger values, you will want to use snprintf(): char roll[16]; int len =3D snprintf(roll, sizeof(roll), "%d\n", arc4random() % d_s= ize); return (uiomove_frombuf(roll, len, uio)); Have fun, DES --=20 Dag-Erling Sm=C3=B8rgrav - des@FreeBSD.org