From owner-svn-src-projects@freebsd.org Mon Jun 17 14:42:28 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3A7BA15BBADC for ; Mon, 17 Jun 2019 14:42:28 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C87706C3A1; Mon, 17 Jun 2019 14:42:27 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A43F4270C9; Mon, 17 Jun 2019 14:42:27 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5HEgRqv034118; Mon, 17 Jun 2019 14:42:27 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5HEgRea034117; Mon, 17 Jun 2019 14:42:27 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906171442.x5HEgRea034117@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Mon, 17 Jun 2019 14:42:27 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349136 - projects/fuse2/tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/tests/sys/fs/fusefs X-SVN-Commit-Revision: 349136 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: C87706C3A1 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.97)[-0.970,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Jun 2019 14:42:28 -0000 Author: asomers Date: Mon Jun 17 14:42:27 2019 New Revision: 349136 URL: https://svnweb.freebsd.org/changeset/base/349136 Log: fusefs: rename the ReadCacheable.default_readahead test The test didn't actually have anything to do with readahead. Rename it to "ReadCacheable.cache_block" Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/tests/sys/fs/fusefs/read.cc Modified: projects/fuse2/tests/sys/fs/fusefs/read.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/read.cc Mon Jun 17 11:46:37 2019 (r349135) +++ projects/fuse2/tests/sys/fs/fusefs/read.cc Mon Jun 17 14:42:27 2019 (r349136) @@ -628,8 +628,11 @@ TEST_F(Read_7_8, read) /* Deliberately leak fd. close(2) will be tested in release.cc */ } -/* If the filesystem allows it, the kernel should try to readahead */ -TEST_F(ReadCacheable, default_readahead) +/* + * If cacheing is enabled, the kernel should try to read an entire cache block + * at a time. + */ +TEST_F(ReadCacheable, cache_block) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; @@ -637,9 +640,7 @@ TEST_F(ReadCacheable, default_readahead) uint64_t ino = 42; int fd; ssize_t bufsize = 8; - /* hard-coded in fuse_internal.c */ - size_t default_maxreadahead = 65536; - ssize_t filesize = default_maxreadahead * 2; + ssize_t filesize = m_maxbcachebuf * 2; char *contents; char buf[bufsize]; const char *contents1 = CONTENTS0 + bufsize; @@ -650,7 +651,7 @@ TEST_F(ReadCacheable, default_readahead) expect_lookup(RELPATH, ino, filesize); expect_open(ino, 0, 1); - expect_read(ino, 0, default_maxreadahead, default_maxreadahead, + expect_read(ino, 0, m_maxbcachebuf, m_maxbcachebuf, contents); fd = open(FULLPATH, O_RDONLY); From owner-svn-src-projects@freebsd.org Mon Jun 17 16:56:53 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8EAD015BF4E3 for ; Mon, 17 Jun 2019 16:56:53 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 331F771F6C; Mon, 17 Jun 2019 16:56:53 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0EF6E829; Mon, 17 Jun 2019 16:56:53 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5HGuqXf003920; Mon, 17 Jun 2019 16:56:52 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5HGupbU003915; Mon, 17 Jun 2019 16:56:51 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906171656.x5HGupbU003915@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Mon, 17 Jun 2019 16:56:51 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349147 - in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Commit-Revision: 349147 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 331F771F6C X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.982,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Jun 2019 16:56:53 -0000 Author: asomers Date: Mon Jun 17 16:56:51 2019 New Revision: 349147 URL: https://svnweb.freebsd.org/changeset/base/349147 Log: fusefs: implement non-clustered readahead fusefs will now read ahead at most one cache block at a time (usually 64 KB). Clustered reads are still TODO. Individual file systems may disable read ahead by setting fuse_init_out.max_readahead=0 during initialization. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c projects/fuse2/sys/fs/fuse/fuse_io.c projects/fuse2/sys/fs/fuse/fuse_ipc.h projects/fuse2/tests/sys/fs/fusefs/read.cc projects/fuse2/tests/sys/fs/fusefs/utils.hh Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_internal.c Mon Jun 17 16:54:51 2019 (r349146) +++ projects/fuse2/sys/fs/fuse/fuse_internal.c Mon Jun 17 16:56:51 2019 (r349147) @@ -908,6 +908,7 @@ fuse_internal_init_callback(struct fuse_ticket *tick, if (fuse_libabi_geq(data, 7, 5)) { if (fticket_resp(tick)->len == sizeof(struct fuse_init_out)) { + data->max_readahead = fiio->max_readahead; data->max_write = fiio->max_write; if (fiio->flags & FUSE_ASYNC_READ) data->dataflags |= FSESS_ASYNC_READ; @@ -951,9 +952,8 @@ fuse_internal_send_init(struct fuse_data *data, struct fiii->major = FUSE_KERNEL_VERSION; fiii->minor = FUSE_KERNEL_MINOR_VERSION; /* - * fusefs currently doesn't do any readahead other than fetching whole - * buffer cache block sized regions at once. So the max readahead is - * the size of a buffer cache block. + * fusefs currently reads ahead no more than one cache block at a time. + * See fuse_read_biobackend */ fiii->max_readahead = maxbcachebuf; /* Modified: projects/fuse2/sys/fs/fuse/fuse_io.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_io.c Mon Jun 17 16:54:51 2019 (r349146) +++ projects/fuse2/sys/fs/fuse/fuse_io.c Mon Jun 17 16:56:51 2019 (r349147) @@ -271,16 +271,22 @@ fuse_read_biobackend(struct vnode *vp, struct uio *uio struct ucred *cred, struct fuse_filehandle *fufh, pid_t pid) { struct buf *bp; - daddr_t lbn; - int bcount; - int err, n = 0, on = 0; + struct mount *mp; + struct fuse_data *data; + daddr_t lbn, nextlbn; + int bcount, nextsize; + int err, n = 0, on = 0, seqcount; off_t filesize; const int biosize = fuse_iosize(vp); + mp = vnode_mount(vp); + data = fuse_get_mpdata(mp); if (uio->uio_offset < 0) return (EINVAL); + seqcount = ioflag >> IO_SEQSHIFT; + err = fuse_vnode_size(vp, &filesize, cred, curthread); if (err) return err; @@ -302,12 +308,25 @@ fuse_read_biobackend(struct vnode *vp, struct uio *uio } else { bcount = biosize; } + nextlbn = lbn + 1; + nextsize = MIN(biosize, filesize - nextlbn * biosize); SDT_PROBE4(fusefs, , io, read_bio_backend_start, biosize, (int)lbn, on, bcount); - /* TODO: readahead. See ext2_read for an example */ - err = bread(vp, lbn, bcount, NOCRED, &bp); + if (bcount < biosize) { + /* If near EOF, don't do readahead */ + err = bread(vp, lbn, bcount, NOCRED, &bp); + /* TODO: clustered read */ + } else if (seqcount > 1 && data->max_readahead >= nextsize) { + /* Try non-clustered readahead */ + err = breadn(vp, lbn, bcount, &nextlbn, &nextsize, 1, + NOCRED, &bp); + } else { + /* Just read what was requested */ + err = bread(vp, lbn, bcount, NOCRED, &bp); + } + if (err) { brelse(bp); bp = NULL; Modified: projects/fuse2/sys/fs/fuse/fuse_ipc.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_ipc.h Mon Jun 17 16:54:51 2019 (r349146) +++ projects/fuse2/sys/fs/fuse/fuse_ipc.h Mon Jun 17 16:56:51 2019 (r349147) @@ -197,6 +197,7 @@ struct fuse_data { uint32_t fuse_libabi_major; uint32_t fuse_libabi_minor; + uint32_t max_readahead; uint32_t max_write; uint32_t max_read; uint32_t subtype; Modified: projects/fuse2/tests/sys/fs/fusefs/read.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/read.cc Mon Jun 17 16:54:51 2019 (r349146) +++ projects/fuse2/tests/sys/fs/fusefs/read.cc Mon Jun 17 16:56:51 2019 (r349147) @@ -112,7 +112,7 @@ virtual void SetUp() { class ReadAhead: public ReadCacheable, public WithParamInterface { virtual void SetUp() { m_maxreadahead = GetParam(); - Read::SetUp(); + ReadCacheable::SetUp(); } }; @@ -747,37 +747,40 @@ TEST_F(ReadCacheable, DISABLED_sendfile_eio) } /* fuse(4) should honor the filesystem's requested m_readahead parameter */ -/* https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=236472 */ -TEST_P(ReadAhead, DISABLED_readahead) { +TEST_P(ReadAhead, readahead) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; - const char *CONTENTS0 = "abcdefghijklmnop"; uint64_t ino = 42; - int fd; - ssize_t bufsize = 8; - ssize_t filesize = m_maxbcachebuf * 2; - char *contents; - char buf[bufsize]; + int fd, i; + ssize_t bufsize = m_maxbcachebuf; + ssize_t filesize = m_maxbcachebuf * 4; + char *rbuf, *contents; - ASSERT_TRUE(GetParam() < (uint32_t)m_maxbcachebuf) - << "Test assumes that max_readahead < maxbcachebuf"; - - contents = (char*)calloc(1, filesize); + contents = (char*)malloc(filesize); ASSERT_NE(NULL, contents); - memmove(contents, CONTENTS0, strlen(CONTENTS0)); + memset(contents, 'X', filesize); + rbuf = (char*)calloc(1, bufsize); expect_lookup(RELPATH, ino, filesize); expect_open(ino, 0, 1); /* fuse(4) should only read ahead the allowed amount */ - expect_read(ino, 0, GetParam(), GetParam(), contents); + expect_read(ino, 0, m_maxbcachebuf, m_maxbcachebuf, contents); + for (i = 0; i < (int)GetParam() / m_maxbcachebuf; i++) { + off_t offs = (i + 1) * m_maxbcachebuf; + expect_read(ino, offs, m_maxbcachebuf, m_maxbcachebuf, + contents + offs); + } fd = open(FULLPATH, O_RDONLY); ASSERT_LE(0, fd) << strerror(errno); - ASSERT_EQ(bufsize, read(fd, buf, bufsize)) << strerror(errno); - ASSERT_EQ(0, memcmp(buf, CONTENTS0, bufsize)); + /* Set the internal readahead counter to a "large" value */ + ASSERT_EQ(0, fcntl(fd, F_READAHEAD, 1'000'000'000)) << strerror(errno); + ASSERT_EQ(bufsize, read(fd, rbuf, bufsize)) << strerror(errno); + ASSERT_EQ(0, memcmp(rbuf, contents, bufsize)); + /* Deliberately leak fd. close(2) will be tested in release.cc */ } -INSTANTIATE_TEST_CASE_P(RA, ReadAhead, ::testing::Values(0u, 2048u)); +INSTANTIATE_TEST_CASE_P(RA, ReadAhead, ::testing::Values(0u, 65536)); Modified: projects/fuse2/tests/sys/fs/fusefs/utils.hh ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/utils.hh Mon Jun 17 16:54:51 2019 (r349146) +++ projects/fuse2/tests/sys/fs/fusefs/utils.hh Mon Jun 17 16:56:51 2019 (r349147) @@ -61,11 +61,7 @@ class FuseTest : public ::testing::Test { int m_maxbcachebuf; FuseTest(): - /* - * libfuse's default max_readahead is UINT_MAX, though it can - * be lowered - */ - m_maxreadahead(UINT_MAX), + m_maxreadahead(0), m_maxwrite(default_max_write), m_init_flags(0), m_allow_other(false), From owner-svn-src-projects@freebsd.org Mon Jun 17 17:17:02 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3613F15BFED4 for ; Mon, 17 Jun 2019 17:17:02 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CF1BF72C4F; Mon, 17 Jun 2019 17:17:01 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A682DB89; Mon, 17 Jun 2019 17:17:01 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5HHH16s014714; Mon, 17 Jun 2019 17:17:01 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5HHH1j2014713; Mon, 17 Jun 2019 17:17:01 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906171717.x5HHH1j2014713@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Mon, 17 Jun 2019 17:17:01 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349149 - projects/fuse2/tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/tests/sys/fs/fusefs X-SVN-Commit-Revision: 349149 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: CF1BF72C4F X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.98)[-0.981,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Jun 2019 17:17:02 -0000 Author: asomers Date: Mon Jun 17 17:17:01 2019 New Revision: 349149 URL: https://svnweb.freebsd.org/changeset/base/349149 Log: fusefs: skip the Write.mmap test when mmap is not available fusefs doesn't not allow mmap when data caching is disabled. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/tests/sys/fs/fusefs/write.cc Modified: projects/fuse2/tests/sys/fs/fusefs/write.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/write.cc Mon Jun 17 16:57:44 2019 (r349148) +++ projects/fuse2/tests/sys/fs/fusefs/write.cc Mon Jun 17 17:17:01 2019 (r349149) @@ -92,6 +92,23 @@ void expect_write(uint64_t ino, uint64_t offset, uint6 }; +class WriteCacheable: public Write { +public: +virtual void SetUp() { + const char *node = "vfs.fusefs.data_cache_mode"; + int val = 0; + size_t size = sizeof(val); + + FuseTest::SetUp(); + + ASSERT_EQ(0, sysctlbyname(node, &val, &size, NULL, 0)) + << strerror(errno); + if (val == 0) + GTEST_SKIP() << + "fusefs data caching must be enabled for this test"; +} +}; + sig_atomic_t Write::s_sigxfsz = 0; class Write_7_8: public FuseTest { @@ -483,8 +500,7 @@ TEST_F(Write, rlimit_fsize) * write, then it must set the FUSE_WRITE_CACHE bit */ /* https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=236378 */ -// TODO: check vfs.fusefs.mmap_enable -TEST_F(Write, mmap) +TEST_F(WriteCacheable, mmap) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; From owner-svn-src-projects@freebsd.org Mon Jun 17 22:01:25 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 34DF315C5B23 for ; Mon, 17 Jun 2019 22:01:25 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CFA52849C5; Mon, 17 Jun 2019 22:01:24 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AA5943F0A; Mon, 17 Jun 2019 22:01:24 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5HM1OTH061030; Mon, 17 Jun 2019 22:01:24 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5HM1N9R061021; Mon, 17 Jun 2019 22:01:23 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906172201.x5HM1N9R061021@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Mon, 17 Jun 2019 22:01:23 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349158 - in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Commit-Revision: 349158 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: CFA52849C5 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.97)[-0.973,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Jun 2019 22:01:25 -0000 Author: asomers Date: Mon Jun 17 22:01:23 2019 New Revision: 349158 URL: https://svnweb.freebsd.org/changeset/base/349158 Log: fusefs: use cluster_read for more readahead fusefs will now use cluster_read. This allows readahead of more than one cache block. However, it won't yet actually cluster the reads because that requires VOP_BMAP, which fusefs does not yet implement. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_io.c projects/fuse2/tests/sys/fs/fusefs/mockfs.cc projects/fuse2/tests/sys/fs/fusefs/mockfs.hh projects/fuse2/tests/sys/fs/fusefs/read.cc projects/fuse2/tests/sys/fs/fusefs/utils.cc projects/fuse2/tests/sys/fs/fusefs/utils.hh Modified: projects/fuse2/sys/fs/fuse/fuse_io.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_io.c Mon Jun 17 21:46:13 2019 (r349157) +++ projects/fuse2/sys/fs/fuse/fuse_io.c Mon Jun 17 22:01:23 2019 (r349158) @@ -317,7 +317,13 @@ fuse_read_biobackend(struct vnode *vp, struct uio *uio if (bcount < biosize) { /* If near EOF, don't do readahead */ err = bread(vp, lbn, bcount, NOCRED, &bp); - /* TODO: clustered read */ + } else if ((vp->v_mount->mnt_flag & MNT_NOCLUSTERR) == 0) { + /* Try clustered read */ + long totread = uio->uio_resid + on; + seqcount = MIN(seqcount, + data->max_readahead / biosize + 1); + err = cluster_read(vp, filesize, lbn, bcount, NOCRED, + totread, seqcount, 0, &bp); } else if (seqcount > 1 && data->max_readahead >= nextsize) { /* Try non-clustered readahead */ err = breadn(vp, lbn, bcount, &nextlbn, &nextsize, 1, Modified: projects/fuse2/tests/sys/fs/fusefs/mockfs.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/mockfs.cc Mon Jun 17 21:46:13 2019 (r349157) +++ projects/fuse2/tests/sys/fs/fusefs/mockfs.cc Mon Jun 17 22:01:23 2019 (r349158) @@ -336,7 +336,8 @@ void MockFS::debug_response(const mockfs_buf_out &out) MockFS::MockFS(int max_readahead, bool allow_other, bool default_permissions, bool push_symlinks_in, bool ro, enum poll_method pm, uint32_t flags, - uint32_t kernel_minor_version, uint32_t max_write, bool async) + uint32_t kernel_minor_version, uint32_t max_write, bool async, + bool noclusterr) { struct sigaction sa; struct iovec *iov = NULL; @@ -408,6 +409,10 @@ MockFS::MockFS(int max_readahead, bool allow_other, bo if (async) { build_iovec(&iov, &iovlen, "async", __DECONST(void*, &trueval), sizeof(bool)); + } + if (noclusterr) { + build_iovec(&iov, &iovlen, "noclusterr", + __DECONST(void*, &trueval), sizeof(bool)); } if (nmount(iov, iovlen, 0)) throw(std::system_error(errno, std::system_category(), Modified: projects/fuse2/tests/sys/fs/fusefs/mockfs.hh ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/mockfs.hh Mon Jun 17 21:46:13 2019 (r349157) +++ projects/fuse2/tests/sys/fs/fusefs/mockfs.hh Mon Jun 17 22:01:23 2019 (r349158) @@ -303,7 +303,8 @@ class MockFS { MockFS(int max_readahead, bool allow_other, bool default_permissions, bool push_symlinks_in, bool ro, enum poll_method pm, uint32_t flags, - uint32_t kernel_minor_version, uint32_t max_write, bool async); + uint32_t kernel_minor_version, uint32_t max_write, bool async, + bool no_clusterr); virtual ~MockFS(); Modified: projects/fuse2/tests/sys/fs/fusefs/read.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/read.cc Mon Jun 17 21:46:13 2019 (r349157) +++ projects/fuse2/tests/sys/fs/fusefs/read.cc Mon Jun 17 22:01:23 2019 (r349158) @@ -109,9 +109,12 @@ virtual void SetUp() { } }; -class ReadAhead: public ReadCacheable, public WithParamInterface { +class ReadAhead: public ReadCacheable, + public WithParamInterface> +{ virtual void SetUp() { - m_maxreadahead = GetParam(); + m_maxreadahead = get<1>(GetParam()); + m_noclusterr = get<0>(GetParam()); ReadCacheable::SetUp(); } }; @@ -746,6 +749,48 @@ TEST_F(ReadCacheable, DISABLED_sendfile_eio) /* Deliberately leak fd. close(2) will be tested in release.cc */ } +/* Large reads should be clustered, even across cache block boundaries */ +/* + * Disabled because clustered reads requires VOP_BMAP, which fusefs does not + * yet support + */ +TEST_P(ReadAhead, DISABLED_cluster) { + const char FULLPATH[] = "mountpoint/some_file.txt"; + const char RELPATH[] = "some_file.txt"; + uint64_t ino = 42; + int fd, maxcontig; + ssize_t bufsize = 4 * m_maxbcachebuf; + ssize_t filesize = bufsize; + uint64_t len; + char *rbuf, *contents; + off_t offs; + + contents = (char*)malloc(filesize); + ASSERT_NE(NULL, contents); + memset(contents, 'X', filesize); + rbuf = (char*)calloc(1, bufsize); + + expect_lookup(RELPATH, ino, filesize); + expect_open(ino, 0, 1); + maxcontig = m_noclusterr ? m_maxbcachebuf : + m_maxbcachebuf + (int)get<1>(GetParam()); + for (offs = 0; offs < bufsize; offs += maxcontig) { + len = std::min((size_t)maxcontig, (size_t)(filesize - offs)); + expect_read(ino, offs, len, len, contents + offs); + } + + fd = open(FULLPATH, O_RDONLY); + ASSERT_LE(0, fd) << strerror(errno); + + /* Set the internal readahead counter to a "large" value */ + ASSERT_EQ(0, fcntl(fd, F_READAHEAD, 1'000'000'000)) << strerror(errno); + + ASSERT_EQ(bufsize, read(fd, rbuf, bufsize)) << strerror(errno); + ASSERT_EQ(0, memcmp(rbuf, contents, bufsize)); + + /* Deliberately leak fd. close(2) will be tested in release.cc */ +} + /* fuse(4) should honor the filesystem's requested m_readahead parameter */ TEST_P(ReadAhead, readahead) { const char FULLPATH[] = "mountpoint/some_file.txt"; @@ -753,7 +798,7 @@ TEST_P(ReadAhead, readahead) { uint64_t ino = 42; int fd, i; ssize_t bufsize = m_maxbcachebuf; - ssize_t filesize = m_maxbcachebuf * 4; + ssize_t filesize = m_maxbcachebuf * 6; char *rbuf, *contents; contents = (char*)malloc(filesize); @@ -765,7 +810,7 @@ TEST_P(ReadAhead, readahead) { expect_open(ino, 0, 1); /* fuse(4) should only read ahead the allowed amount */ expect_read(ino, 0, m_maxbcachebuf, m_maxbcachebuf, contents); - for (i = 0; i < (int)GetParam() / m_maxbcachebuf; i++) { + for (i = 0; i < (int)get<1>(GetParam()) / m_maxbcachebuf; i++) { off_t offs = (i + 1) * m_maxbcachebuf; expect_read(ino, offs, m_maxbcachebuf, m_maxbcachebuf, contents + offs); @@ -783,4 +828,10 @@ TEST_P(ReadAhead, readahead) { /* Deliberately leak fd. close(2) will be tested in release.cc */ } -INSTANTIATE_TEST_CASE_P(RA, ReadAhead, ::testing::Values(0u, 65536)); +INSTANTIATE_TEST_CASE_P(RA, ReadAhead, + Values(tuple(false, 0u), + tuple(false, 0x10000), + tuple(false, 0x20000), + tuple(false, 0x30000), + tuple(true, 0u), + tuple(true, 0x10000))); Modified: projects/fuse2/tests/sys/fs/fusefs/utils.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/utils.cc Mon Jun 17 21:46:13 2019 (r349157) +++ projects/fuse2/tests/sys/fs/fusefs/utils.cc Mon Jun 17 22:01:23 2019 (r349158) @@ -113,7 +113,7 @@ void FuseTest::SetUp() { m_mock = new MockFS(m_maxreadahead, m_allow_other, m_default_permissions, m_push_symlinks_in, m_ro, m_pm, m_init_flags, m_kernel_minor_version, - m_maxwrite, m_async); + m_maxwrite, m_async, m_noclusterr); /* * FUSE_ACCESS is called almost universally. Expecting it in * each test case would be super-annoying. Instead, set a Modified: projects/fuse2/tests/sys/fs/fusefs/utils.hh ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/utils.hh Mon Jun 17 21:46:13 2019 (r349157) +++ projects/fuse2/tests/sys/fs/fusefs/utils.hh Mon Jun 17 22:01:23 2019 (r349158) @@ -54,6 +54,7 @@ class FuseTest : public ::testing::Test { bool m_push_symlinks_in; bool m_ro; bool m_async; + bool m_noclusterr; MockFS *m_mock = NULL; const static uint64_t FH = 0xdeadbeef1a7ebabe; @@ -70,7 +71,8 @@ class FuseTest : public ::testing::Test { m_pm(BLOCKING), m_push_symlinks_in(false), m_ro(false), - m_async(false) + m_async(false), + m_noclusterr(false) {} virtual void SetUp(); From owner-svn-src-projects@freebsd.org Mon Jun 17 22:14:00 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A3E6315C5E4A for ; Mon, 17 Jun 2019 22:14:00 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 19EFD8509E; Mon, 17 Jun 2019 22:14:00 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E84B2415F; Mon, 17 Jun 2019 22:13:59 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5HMDxFr069315; Mon, 17 Jun 2019 22:13:59 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5HMDx2M069313; Mon, 17 Jun 2019 22:13:59 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906172213.x5HMDx2M069313@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Mon, 17 Jun 2019 22:13:59 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349159 - projects/fuse2/tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/tests/sys/fs/fusefs X-SVN-Commit-Revision: 349159 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 19EFD8509E X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.97)[-0.973,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Jun 2019 22:14:00 -0000 Author: asomers Date: Mon Jun 17 22:13:59 2019 New Revision: 349159 URL: https://svnweb.freebsd.org/changeset/base/349159 Log: fusefs: run the Io tests with various combinations of mount options Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/tests/sys/fs/fusefs/io.cc Modified: projects/fuse2/tests/sys/fs/fusefs/io.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/io.cc Mon Jun 17 22:01:23 2019 (r349158) +++ projects/fuse2/tests/sys/fs/fusefs/io.cc Mon Jun 17 22:13:59 2019 (r349159) @@ -72,7 +72,8 @@ static void compare(const void *tbuf, const void *cont } } -class Io: public FuseTest { +class Io: public FuseTest, + public WithParamInterface> { public: int m_backing_fd, m_control_fd, m_test_fd; @@ -87,6 +88,11 @@ void SetUp() if (m_control_fd < 0) FAIL() << strerror(errno); srandom(22'9'1982); // Seed with my birthday + + m_init_flags = get<0>(GetParam()); + m_maxwrite = get<1>(GetParam()); + m_async = get<2>(GetParam()); + FuseTest::SetUp(); if (IsSkipped()) return; @@ -221,7 +227,7 @@ void do_write(ssize_t size, off_t offs) * * fsx -WR -P /tmp -S8 -N3 fsx.bin */ -TEST_F(Io, extend_from_dirty_page) +TEST_P(Io, extend_from_dirty_page) { off_t wofs = 0x21a0; ssize_t wsize = 0xf0a8; @@ -241,7 +247,7 @@ TEST_F(Io, extend_from_dirty_page) * * fsx -WR -P /tmp -S642 -N3 fsx.bin */ -TEST_F(Io, last_page) +TEST_P(Io, last_page) { off_t wofs0 = 0x1134f; ssize_t wsize0 = 0xcc77; @@ -260,7 +266,7 @@ TEST_F(Io, last_page) * * fsx -WR -P /tmp -S55 fsx.bin */ -TEST_F(Io, read_hole_from_cached_block) +TEST_P(Io, read_hole_from_cached_block) { off_t wofs = 0x160c5; ssize_t wsize = 0xa996; @@ -277,7 +283,7 @@ TEST_F(Io, read_hole_from_cached_block) * * fsx -WR -P /tmp -S839 -d -N6 fsx.bin */ -TEST_F(Io, truncate_into_dirty_buffer) +TEST_P(Io, truncate_into_dirty_buffer) { off_t wofs0 = 0x3bad7; ssize_t wsize0 = 0x4529; @@ -304,7 +310,7 @@ TEST_F(Io, truncate_into_dirty_buffer) * Based on this command with a few steps removed: * fsx -WR -P /tmp -S677 -d -N8 fsx.bin */ -TEST_F(Io, truncate_into_dirty_buffer2) +TEST_P(Io, truncate_into_dirty_buffer2) { off_t truncsize0 = 0x344f3; off_t wofs = 0x2790c; @@ -353,7 +359,7 @@ TEST_F(Io, truncate_into_dirty_buffer2) * Based on: * fsx -WR -l 524388 -o 131072 -P /tmp -S6456 -q fsx.bin */ -TEST_F(Io, resize_a_valid_buffer_while_extending) +TEST_P(Io, resize_a_valid_buffer_while_extending) { do_write(0x14530, 0x36ee6); /* [0x36ee6, 0x4b415] */ do_write(0x1507c, 0x33256); /* [0x33256, 0x482d1] */ @@ -361,3 +367,8 @@ TEST_F(Io, resize_a_valid_buffer_while_extending) do_read(0xe277, 0x3599c); /* [0x3599c, 0x43c12] */ close(m_test_fd); } + +INSTANTIATE_TEST_CASE_P(Io, Io, + Combine(Values(0, FUSE_ASYNC_READ), /* m_init_flags */ + Values(0x1000, 0x10000, 0x20000), /* m_maxwrite */ + Bool())); /* m_async */ From owner-svn-src-projects@freebsd.org Mon Jun 17 23:34:13 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6D66D15C7478 for ; Mon, 17 Jun 2019 23:34:13 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0B8D78744F; Mon, 17 Jun 2019 23:34:13 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EEF5E4EC1; Mon, 17 Jun 2019 23:34:12 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5HNYC5s011508; Mon, 17 Jun 2019 23:34:12 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5HNYCZC011505; Mon, 17 Jun 2019 23:34:12 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906172334.x5HNYCZC011505@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Mon, 17 Jun 2019 23:34:12 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349162 - in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Commit-Revision: 349162 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 0B8D78744F X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.99 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.99)[-0.988,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Jun 2019 23:34:13 -0000 Author: asomers Date: Mon Jun 17 23:34:11 2019 New Revision: 349162 URL: https://svnweb.freebsd.org/changeset/base/349162 Log: fusefs: multiple fixes related to the write cache * Don't always write the last page synchronously. That's not actually required. It was probably just masking another bug that I fixed later, possibly in r349021. * Enable the NotifyWriteback tests now that Writeback cache is working. * Add a test to ensure that the write cache isn't flushed synchronously when in writeback mode. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_io.c projects/fuse2/tests/sys/fs/fusefs/notify.cc projects/fuse2/tests/sys/fs/fusefs/write.cc Modified: projects/fuse2/sys/fs/fuse/fuse_io.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_io.c Mon Jun 17 23:03:30 2019 (r349161) +++ projects/fuse2/sys/fs/fuse/fuse_io.c Mon Jun 17 23:34:11 2019 (r349162) @@ -789,23 +789,7 @@ again: vfs_bio_set_flags(bp, ioflag); - if (last_page) { - /* - * When writing the last page of a file we must write - * synchronously. If we didn't, then a subsequent - * operation could extend the file, making the last - * page of this buffer invalid because it would only be - * partially cached. - * - * As an optimization, it would be allowable to only - * write the last page synchronously. Or, it should be - * possible to synchronously flush the last - * already-written page whenever extending a file with - * ftruncate or another write. - */ - SDT_PROBE2(fusefs, , io, write_biobackend_issue, 1, bp); - err = bwrite(bp); - } else if (ioflag & IO_SYNC) { + if (ioflag & IO_SYNC) { SDT_PROBE2(fusefs, , io, write_biobackend_issue, 2, bp); err = bwrite(bp); } else if (vm_page_count_severe() || Modified: projects/fuse2/tests/sys/fs/fusefs/notify.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/notify.cc Mon Jun 17 23:03:30 2019 (r349161) +++ projects/fuse2/tests/sys/fs/fusefs/notify.cc Mon Jun 17 23:34:11 2019 (r349162) @@ -50,6 +50,18 @@ using namespace testing; class Notify: public FuseTest { public: +/* Ignore an optional FUSE_FSYNC */ +void maybe_expect_fsync(uint64_t ino) +{ + EXPECT_CALL(*m_mock, process( + ResultOf([=](auto in) { + return (in.header.opcode == FUSE_FSYNC && + in.header.nodeid == ino); + }, Eq(true)), + _) + ).WillOnce(Invoke(ReturnErrno(0))); +} + void expect_lookup(uint64_t parent, const char *relpath, uint64_t ino, off_t size, Sequence &seq) { @@ -76,6 +88,7 @@ virtual void SetUp() { int val = 0; size_t size = sizeof(val); + m_async = true; Notify::SetUp(); if (IsSkipped()) return; @@ -363,8 +376,7 @@ TEST_F(Notify, inval_inode_with_clean_cache) /* Deliberately leak fd. close(2) will be tested in release.cc */ } -/* https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238312 */ -TEST_F(NotifyWriteback, DISABLED_inval_inode_with_dirty_cache) +TEST_F(NotifyWriteback, inval_inode_with_dirty_cache) { const static char FULLPATH[] = "mountpoint/foo"; const static char RELPATH[] = "foo"; @@ -384,8 +396,14 @@ TEST_F(NotifyWriteback, DISABLED_inval_inode_with_dirt fd = open(FULLPATH, O_RDWR); ASSERT_EQ(bufsize, write(fd, CONTENTS, bufsize)) << strerror(errno); - /* Evict the data cache */ expect_write(ino, 0, bufsize, CONTENTS); + /* + * The FUSE protocol does not require an fsync here, but FreeBSD's + * bufobj_invalbuf sends it anyway + */ + maybe_expect_fsync(ino); + + /* Evict the data cache */ iia.mock = m_mock; iia.ino = ino; iia.off = 0; @@ -398,8 +416,7 @@ TEST_F(NotifyWriteback, DISABLED_inval_inode_with_dirt /* Deliberately leak fd. close(2) will be tested in release.cc */ } -/* https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238312 */ -TEST_F(NotifyWriteback, DISABLED_inval_inode_attrs_only) +TEST_F(NotifyWriteback, inval_inode_attrs_only) { const static char FULLPATH[] = "mountpoint/foo"; const static char RELPATH[] = "foo"; @@ -425,7 +442,7 @@ TEST_F(NotifyWriteback, DISABLED_inval_inode_attrs_onl EXPECT_CALL(*m_mock, process( ResultOf([=](auto in) { return (in.header.opcode == FUSE_GETATTR && - in.header.nodeid == FUSE_ROOT_ID); + in.header.nodeid == ino); }, Eq(true)), _) ).WillOnce(Invoke(ReturnImmediate([=](auto i __unused, auto& out) { Modified: projects/fuse2/tests/sys/fs/fusefs/write.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/write.cc Mon Jun 17 23:03:30 2019 (r349161) +++ projects/fuse2/tests/sys/fs/fusefs/write.cc Mon Jun 17 23:34:11 2019 (r349162) @@ -90,6 +90,31 @@ void expect_write(uint64_t ino, uint64_t offset, uint6 FuseTest::expect_write(ino, offset, isize, osize, 0, 0, contents); } +/* Expect a write that may or may not come, depending on the cache mode */ +void maybe_expect_write(uint64_t ino, uint64_t offset, uint64_t size, + const void *contents) +{ + EXPECT_CALL(*m_mock, process( + ResultOf([=](auto in) { + const char *buf = (const char*)in.body.bytes + + sizeof(struct fuse_write_in); + + return (in.header.opcode == FUSE_WRITE && + in.header.nodeid == ino && + in.body.write.offset == offset && + in.body.write.size == size && + 0 == bcmp(buf, contents, size)); + }, Eq(true)), + _) + ).Times(AtMost(1)) + .WillRepeatedly(Invoke( + ReturnImmediate([=](auto in __unused, auto& out) { + SET_OUT_HEADER_LEN(out, write); + out.body.write.size = size; + }) + )); +} + }; class WriteCacheable: public Write { @@ -196,6 +221,14 @@ void expect_write(uint64_t ino, uint64_t offset, uint6 } }; +class WriteBackAsync: public WriteBack { +public: +virtual void SetUp() { + m_async = true; + WriteBack::SetUp(); +} +}; + /* Tests for clustered writes with WriteBack cacheing */ class WriteCluster: public WriteBack { public: @@ -302,7 +335,7 @@ TEST_F(Write, append_to_cached) expect_lookup(RELPATH, ino, oldsize); expect_open(ino, 0, 1); expect_read(ino, 0, oldsize, oldsize, oldcontents); - expect_write(ino, oldsize, BUFSIZE, BUFSIZE, CONTENTS); + maybe_expect_write(ino, oldsize, BUFSIZE, CONTENTS); /* Must open O_RDWR or fuse(4) implicitly sets direct_io */ fd = open(FULLPATH, O_RDWR | O_APPEND); @@ -610,7 +643,7 @@ TEST_F(Write, write_large) expect_lookup(RELPATH, ino, 0); expect_open(ino, 0, 1); expect_write(ino, 0, halfbufsize, halfbufsize, contents); - expect_write(ino, halfbufsize, halfbufsize, halfbufsize, + maybe_expect_write(ino, halfbufsize, halfbufsize, &contents[halfbufsize / sizeof(int)]); fd = open(FULLPATH, O_WRONLY); @@ -662,7 +695,7 @@ TEST_F(Write_7_8, write) } /* In writeback mode, dirty data should be written on close */ -TEST_F(WriteBack, close) +TEST_F(WriteBackAsync, close) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; @@ -795,7 +828,7 @@ TEST_F(WriteBack, rmw) FuseTest::expect_lookup(RELPATH, ino, S_IFREG | 0644, fsize, 1); expect_open(ino, 0, 1); expect_read(ino, 0, fsize, fsize, INITIAL, O_WRONLY); - expect_write(ino, offset, bufsize, bufsize, CONTENTS); + maybe_expect_write(ino, offset, bufsize, CONTENTS); fd = open(FULLPATH, O_WRONLY); EXPECT_LE(0, fd) << strerror(errno); @@ -808,7 +841,7 @@ TEST_F(WriteBack, rmw) /* * Without direct_io, writes should be committed to cache */ -TEST_F(WriteBack, writeback) +TEST_F(WriteBack, cache) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; @@ -865,6 +898,36 @@ TEST_F(WriteBack, o_direct) ASSERT_EQ(0, fcntl(fd, F_SETFL, 0)) << strerror(errno); ASSERT_EQ(bufsize, read(fd, readbuf, bufsize)) << strerror(errno); /* Deliberately leak fd. close(2) will be tested in release.cc */ +} + +/* + * When mounted with -o async, the writeback cache mode should delay writes + */ +TEST_F(WriteBackAsync, delay) +{ + const char FULLPATH[] = "mountpoint/some_file.txt"; + const char RELPATH[] = "some_file.txt"; + const char *CONTENTS = "abcdefgh"; + uint64_t ino = 42; + int fd; + ssize_t bufsize = strlen(CONTENTS); + + expect_lookup(RELPATH, ino, 0); + expect_open(ino, 0, 1); + /* Write should be cached, but FUSE_WRITE shouldn't be sent */ + EXPECT_CALL(*m_mock, process( + ResultOf([=](auto in) { + return (in.header.opcode == FUSE_WRITE); + }, Eq(true)), + _) + ).Times(0); + + fd = open(FULLPATH, O_RDWR); + EXPECT_LE(0, fd) << strerror(errno); + + ASSERT_EQ(bufsize, write(fd, CONTENTS, bufsize)) << strerror(errno); + + /* Don't close the file because that would flush the cache */ } /* From owner-svn-src-projects@freebsd.org Thu Jun 20 15:56:15 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5633215BFE6A for ; Thu, 20 Jun 2019 15:56:15 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 052E8871B0; Thu, 20 Jun 2019 15:56:15 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B46E9E2FA; Thu, 20 Jun 2019 15:56:14 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5KFuEKj063728; Thu, 20 Jun 2019 15:56:14 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5KFu9Fm063704; Thu, 20 Jun 2019 15:56:09 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906201556.x5KFu9Fm063704@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Thu, 20 Jun 2019 15:56:09 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349236 - in projects/fuse2: . bin/csh bin/sh bin/stty cddl/usr.sbin/dtrace/tests/amd64 contrib/compiler-rt/lib/sanitizer_common contrib/elftoolchain/elfdump contrib/elftoolchain/readel... X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: . bin/csh bin/sh bin/stty cddl/usr.sbin/dtrace/tests/amd64 contrib/compiler-rt/lib/sanitizer_common contrib/elftoolchain/elfdump contrib/elftoolchain/readelf contrib/libarchive cont... X-SVN-Commit-Revision: 349236 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 052E8871B0 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.97)[-0.966,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2019 15:56:15 -0000 Author: asomers Date: Thu Jun 20 15:56:08 2019 New Revision: 349236 URL: https://svnweb.freebsd.org/changeset/base/349236 Log: MFHead @349234 Sponsored by: The FreeBSD Foundation Added: projects/fuse2/.clang-format - copied unchanged from r349234, head/.clang-format projects/fuse2/cddl/usr.sbin/dtrace/tests/amd64/Makefile - copied unchanged from r349234, head/cddl/usr.sbin/dtrace/tests/amd64/Makefile projects/fuse2/contrib/libarchive/libarchive/test/test_read_format_rar5_different_window_size.rar.uu - copied unchanged from r349234, head/contrib/libarchive/libarchive/test/test_read_format_rar5_different_window_size.rar.uu projects/fuse2/contrib/libarchive/libarchive/test/test_read_format_rar_ppmd_use_after_free2.rar.uu - copied unchanged from r349234, head/contrib/libarchive/libarchive/test/test_read_format_rar_ppmd_use_after_free2.rar.uu projects/fuse2/share/man/man4/pwmc.4 - copied unchanged from r349234, head/share/man/man4/pwmc.4 projects/fuse2/share/man/man9/VOP_BMAP.9 - copied unchanged from r349234, head/share/man/man9/VOP_BMAP.9 projects/fuse2/share/timedef/zh_TW.Big5.src - copied unchanged from r349234, head/share/timedef/zh_TW.Big5.src projects/fuse2/stand/efi/boot1/proto.c - copied unchanged from r349234, head/stand/efi/boot1/proto.c projects/fuse2/stand/efi/boot1/proto.h - copied unchanged from r349234, head/stand/efi/boot1/proto.h projects/fuse2/stand/efi/gptboot/ - copied from r349234, head/stand/efi/gptboot/ projects/fuse2/sys/arm/mv/a37x0_gpio.c - copied unchanged from r349234, head/sys/arm/mv/a37x0_gpio.c projects/fuse2/sys/conf/ldscript.set_padding - copied unchanged from r349234, head/sys/conf/ldscript.set_padding projects/fuse2/sys/dev/pwm/ofw_pwm.h - copied unchanged from r349234, head/sys/dev/pwm/ofw_pwm.h projects/fuse2/sys/dev/pwm/ofw_pwmbus.c - copied unchanged from r349234, head/sys/dev/pwm/ofw_pwmbus.c projects/fuse2/sys/dev/pwm/pwmc.h - copied unchanged from r349234, head/sys/dev/pwm/pwmc.h projects/fuse2/sys/dev/sdio/ - copied from r349234, head/sys/dev/sdio/ projects/fuse2/sys/dev/usb/usb_hub_acpi.c - copied unchanged from r349234, head/sys/dev/usb/usb_hub_acpi.c projects/fuse2/sys/dev/usb/usb_hub_private.h - copied unchanged from r349234, head/sys/dev/usb/usb_hub_private.h projects/fuse2/sys/libkern/gsb_crc32.c - copied unchanged from r349234, head/sys/libkern/gsb_crc32.c projects/fuse2/sys/modules/arm_ti/ - copied from r349234, head/sys/modules/arm_ti/ projects/fuse2/sys/modules/pwm/ - copied from r349234, head/sys/modules/pwm/ projects/fuse2/sys/modules/sdio/ - copied from r349234, head/sys/modules/sdio/ projects/fuse2/sys/sys/gsb_crc32.h - copied unchanged from r349234, head/sys/sys/gsb_crc32.h projects/fuse2/sys/tools/sdiodevs2h.awk - copied unchanged from r349234, head/sys/tools/sdiodevs2h.awk projects/fuse2/tests/sys/devrandom/ - copied from r349234, head/tests/sys/devrandom/ projects/fuse2/usr.sbin/bhyve/net_utils.c - copied unchanged from r349234, head/usr.sbin/bhyve/net_utils.c projects/fuse2/usr.sbin/bhyve/net_utils.h - copied unchanged from r349234, head/usr.sbin/bhyve/net_utils.h Replaced: projects/fuse2/sys/dev/pwm/pwmbus.h - copied unchanged from r349234, head/sys/dev/pwm/pwmbus.h Deleted: projects/fuse2/sys/dev/pwm/pwm_if.m projects/fuse2/sys/libkern/crc32.c projects/fuse2/sys/sys/capability.h projects/fuse2/sys/sys/pwm.h Modified: projects/fuse2/.gitattributes projects/fuse2/Makefile.inc1 projects/fuse2/ObsoleteFiles.inc projects/fuse2/UPDATING projects/fuse2/bin/csh/Makefile projects/fuse2/bin/sh/Makefile projects/fuse2/bin/stty/modes.c projects/fuse2/bin/stty/print.c projects/fuse2/bin/stty/stty.1 projects/fuse2/contrib/compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_netbsd.cc projects/fuse2/contrib/elftoolchain/elfdump/elfdump.c projects/fuse2/contrib/elftoolchain/readelf/readelf.c projects/fuse2/contrib/libarchive/NEWS projects/fuse2/contrib/libarchive/README.md projects/fuse2/contrib/libarchive/libarchive/archive.h projects/fuse2/contrib/libarchive/libarchive/archive_entry.h projects/fuse2/contrib/libarchive/libarchive/archive_read_support_format_rar.c projects/fuse2/contrib/libarchive/libarchive/archive_read_support_format_rar5.c projects/fuse2/contrib/libarchive/libarchive/archive_write_add_filter_b64encode.c projects/fuse2/contrib/libarchive/libarchive/archive_write_disk_posix.c projects/fuse2/contrib/libarchive/libarchive/test/test_read_format_rar.c projects/fuse2/contrib/libarchive/libarchive/test/test_read_format_rar5.c projects/fuse2/contrib/libarchive/libarchive_fe/line_reader.c projects/fuse2/contrib/libarchive/libarchive_fe/passphrase.c projects/fuse2/contrib/libarchive/tar/bsdtar.1 projects/fuse2/contrib/libunwind/src/UnwindRegistersRestore.S projects/fuse2/contrib/libunwind/src/UnwindRegistersSave.S projects/fuse2/contrib/libunwind/src/assembly.h projects/fuse2/contrib/llvm/lib/DebugInfo/DWARF/DWARFDebugFrame.cpp projects/fuse2/contrib/llvm/lib/MC/ELFObjectWriter.cpp projects/fuse2/contrib/llvm/lib/MC/MCWin64EH.cpp projects/fuse2/contrib/llvm/lib/MC/WasmObjectWriter.cpp projects/fuse2/contrib/llvm/lib/Object/COFFImportFile.cpp projects/fuse2/contrib/llvm/lib/Target/AArch64/AArch64SchedExynosM4.td projects/fuse2/contrib/llvm/lib/Target/AArch64/AArch64SchedPredExynos.td projects/fuse2/contrib/llvm/lib/Target/AArch64/AArch64SchedPredicates.td projects/fuse2/contrib/llvm/lib/Target/AMDGPU/SIFoldOperands.cpp projects/fuse2/contrib/llvm/lib/Target/AMDGPU/VOP2Instructions.td projects/fuse2/contrib/llvm/lib/Target/AVR/AVRISelLowering.cpp projects/fuse2/contrib/llvm/lib/Target/AVR/AVRISelLowering.h projects/fuse2/contrib/llvm/lib/Target/AVR/AVRSubtarget.cpp projects/fuse2/contrib/llvm/lib/Target/AVR/AVRSubtarget.h projects/fuse2/contrib/llvm/lib/Target/Mips/MCTargetDesc/MipsTargetStreamer.cpp projects/fuse2/contrib/llvm/lib/Target/Mips/MicroMips32r6InstrInfo.td projects/fuse2/contrib/llvm/lib/Target/Mips/MicroMipsInstrFPU.td projects/fuse2/contrib/llvm/lib/Target/Mips/MipsAsmPrinter.cpp projects/fuse2/contrib/llvm/lib/Target/Mips/MipsDSPInstrInfo.td projects/fuse2/contrib/llvm/lib/Target/Mips/MipsDelaySlotFiller.cpp projects/fuse2/contrib/llvm/lib/Target/Mips/MipsFastISel.cpp projects/fuse2/contrib/llvm/lib/Target/Mips/MipsSEInstrInfo.cpp projects/fuse2/contrib/llvm/lib/Target/PowerPC/Disassembler/PPCDisassembler.cpp projects/fuse2/contrib/llvm/lib/Target/PowerPC/InstPrinter/PPCInstPrinter.cpp projects/fuse2/contrib/llvm/lib/Target/PowerPC/MCTargetDesc/PPCMCTargetDesc.cpp projects/fuse2/contrib/llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp projects/fuse2/contrib/llvm/lib/Target/PowerPC/PPCInstrInfo.td projects/fuse2/contrib/llvm/lib/Target/PowerPC/PPCSubtarget.cpp projects/fuse2/contrib/llvm/lib/Target/Sparc/SparcRegisterInfo.cpp projects/fuse2/contrib/llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp projects/fuse2/contrib/llvm/lib/Target/X86/X86FastISel.cpp projects/fuse2/contrib/llvm/lib/Target/X86/X86TargetMachine.cpp projects/fuse2/contrib/llvm/tools/clang/lib/Basic/Version.cpp projects/fuse2/contrib/llvm/tools/clang/lib/CodeGen/CGStmtOpenMP.cpp projects/fuse2/contrib/llvm/tools/clang/lib/Driver/ToolChains/Clang.cpp projects/fuse2/contrib/llvm/tools/clang/lib/Driver/ToolChains/Linux.cpp projects/fuse2/contrib/llvm/tools/clang/lib/Sema/SemaOpenMP.cpp projects/fuse2/contrib/llvm/tools/lld/COFF/Writer.cpp projects/fuse2/contrib/llvm/tools/lld/ELF/Arch/PPC64.cpp projects/fuse2/contrib/llvm/tools/lld/ELF/InputSection.cpp projects/fuse2/contrib/llvm/tools/lld/ELF/SyntheticSections.cpp projects/fuse2/contrib/llvm/tools/lld/ELF/Writer.cpp projects/fuse2/contrib/llvm/tools/llvm-objdump/llvm-objdump.cpp projects/fuse2/contrib/traceroute/traceroute.8 projects/fuse2/etc/Makefile projects/fuse2/etc/mtree/BSD.debug.dist projects/fuse2/etc/mtree/BSD.include.dist projects/fuse2/etc/mtree/BSD.tests.dist projects/fuse2/etc/mtree/BSD.usr.dist projects/fuse2/etc/mtree/BSD.var.dist projects/fuse2/etc/sendmail/Makefile projects/fuse2/gnu/lib/csu/Makefile projects/fuse2/gnu/lib/libgcc/Makefile projects/fuse2/gnu/lib/libgcov/Makefile projects/fuse2/include/Makefile projects/fuse2/lib/clang/freebsd_cc_version.h projects/fuse2/lib/clang/headers/Makefile projects/fuse2/lib/clang/include/clang/Basic/Version.inc projects/fuse2/lib/clang/include/clang/Config/config.h projects/fuse2/lib/clang/include/lld/Common/Version.inc projects/fuse2/lib/clang/include/llvm/Config/config.h projects/fuse2/lib/clang/include/llvm/Config/llvm-config.h projects/fuse2/lib/clang/include/llvm/Support/VCSRevision.h projects/fuse2/lib/csu/Makefile.inc projects/fuse2/lib/libarchive/config_freebsd.h projects/fuse2/lib/libarchive/tests/Makefile projects/fuse2/lib/libc/sys/open.2 projects/fuse2/lib/libcasper/services/cap_fileargs/cap_fileargs.c projects/fuse2/lib/libcasper/services/cap_fileargs/cap_fileargs.h projects/fuse2/lib/libclang_rt/Makefile.inc projects/fuse2/lib/libmagic/Makefile projects/fuse2/lib/libufs/Makefile projects/fuse2/lib/libusb/libusb20.c projects/fuse2/lib/ncurses/ncurses/Makefile projects/fuse2/libexec/rc/rc.conf projects/fuse2/libexec/rc/rc.d/hostapd projects/fuse2/libexec/rc/rc.d/local projects/fuse2/release/release.sh projects/fuse2/release/tools/arm.subr projects/fuse2/sbin/camcontrol/camcontrol.c projects/fuse2/sbin/dump/dump.8 projects/fuse2/sbin/fsck_msdosfs/boot.c projects/fuse2/sbin/fsck_msdosfs/dir.c projects/fuse2/sbin/fsck_msdosfs/main.c projects/fuse2/share/examples/bhyve/vmrun.sh projects/fuse2/share/man/man3/Makefile projects/fuse2/share/man/man4/Makefile projects/fuse2/share/man/man4/filemon.4 projects/fuse2/share/man/man4/netmap.4 projects/fuse2/share/man/man4/pci.4 projects/fuse2/share/man/man4/termios.4 projects/fuse2/share/man/man9/Makefile projects/fuse2/share/man/man9/fail.9 projects/fuse2/share/man/man9/sleep.9 projects/fuse2/share/man/man9/sleepqueue.9 projects/fuse2/share/man/man9/sysctl.9 projects/fuse2/share/misc/bsd-family-tree projects/fuse2/share/mk/bsd.README projects/fuse2/share/mk/bsd.dep.mk projects/fuse2/share/mk/bsd.progs.mk projects/fuse2/share/mk/bsd.suffixes.mk projects/fuse2/share/mk/bsd.sys.mk projects/fuse2/share/mk/local.sys.mk projects/fuse2/share/syscons/scrnmaps/Makefile projects/fuse2/share/timedef/Makefile projects/fuse2/share/vt/fonts/Makefile projects/fuse2/stand/common/disk.c projects/fuse2/stand/efi/Makefile projects/fuse2/stand/efi/boot1/Makefile projects/fuse2/stand/efi/boot1/boot1.c projects/fuse2/stand/efi/boot1/boot_module.h projects/fuse2/stand/efi/boot1/ufs_module.c projects/fuse2/stand/efi/boot1/zfs_module.c projects/fuse2/stand/efi/libefi/efinet.c projects/fuse2/stand/libsa/crc32_libkern.c projects/fuse2/stand/libsa/zfs/zfsimpl.c projects/fuse2/stand/ofw/libofw/ofw_console.c projects/fuse2/sys/amd64/amd64/fpu.c projects/fuse2/sys/amd64/amd64/pmap.c projects/fuse2/sys/amd64/amd64/trap.c projects/fuse2/sys/amd64/include/pmap.h projects/fuse2/sys/amd64/vmm/intel/vtd.c projects/fuse2/sys/arm/allwinner/a10_timer.c projects/fuse2/sys/arm/allwinner/aw_mmc.c projects/fuse2/sys/arm/allwinner/aw_pwm.c projects/fuse2/sys/arm/arm/pmap-v6.c projects/fuse2/sys/arm/broadcom/bcm2835/bcm2835_sdhci.c projects/fuse2/sys/arm/broadcom/bcm2835/bcm2835_sdhost.c projects/fuse2/sys/arm/conf/GENERIC projects/fuse2/sys/arm/nvidia/drm2/tegra_bo.c projects/fuse2/sys/arm/ti/am335x/am335x_dmtpps.c projects/fuse2/sys/arm/ti/ti_i2c.c projects/fuse2/sys/arm64/arm64/pmap.c projects/fuse2/sys/arm64/conf/GENERIC projects/fuse2/sys/arm64/include/armreg.h projects/fuse2/sys/arm64/rockchip/if_dwc_rk.c projects/fuse2/sys/cam/cam_sim.c projects/fuse2/sys/cam/cam_sim.h projects/fuse2/sys/cam/nvme/nvme_xpt.c projects/fuse2/sys/cam/scsi/scsi_xpt.c projects/fuse2/sys/cddl/contrib/opensolaris/uts/common/dtrace/fasttrap.c projects/fuse2/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c projects/fuse2/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c projects/fuse2/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/multilist.h projects/fuse2/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c projects/fuse2/sys/compat/linuxkpi/common/include/linux/gfp.h projects/fuse2/sys/compat/linuxkpi/common/src/linux_page.c projects/fuse2/sys/conf/files projects/fuse2/sys/conf/files.arm64 projects/fuse2/sys/conf/kmod.mk projects/fuse2/sys/conf/ldscript.riscv projects/fuse2/sys/contrib/ipfilter/netinet/fil.c projects/fuse2/sys/contrib/ipfilter/netinet/ip_fil.h projects/fuse2/sys/contrib/ipfilter/netinet/ip_fil_freebsd.c projects/fuse2/sys/contrib/ipfilter/netinet/mlfk_ipl.c projects/fuse2/sys/dev/atkbdc/psm.c projects/fuse2/sys/dev/ciss/ciss.c projects/fuse2/sys/dev/cxgbe/t4_main.c projects/fuse2/sys/dev/cxgbe/tom/t4_tls.c projects/fuse2/sys/dev/drm2/ttm/ttm_page_alloc.c projects/fuse2/sys/dev/etherswitch/e6000sw/e6000sw.c projects/fuse2/sys/dev/etherswitch/e6000sw/e6000swreg.h projects/fuse2/sys/dev/gpio/gpioregulator.c projects/fuse2/sys/dev/ipmi/ipmi_kcs.c projects/fuse2/sys/dev/iscsi/icl_soft.c projects/fuse2/sys/dev/iscsi_initiator/isc_subr.c projects/fuse2/sys/dev/liquidio/lio_bsd.h projects/fuse2/sys/dev/nvdimm/nvdimm.c projects/fuse2/sys/dev/nvme/nvme_sim.c projects/fuse2/sys/dev/pwm/ofw_pwm.c projects/fuse2/sys/dev/pwm/pwmbus.c projects/fuse2/sys/dev/pwm/pwmbus_if.m projects/fuse2/sys/dev/pwm/pwmc.c projects/fuse2/sys/dev/random/fortuna.c projects/fuse2/sys/dev/random/fortuna.h projects/fuse2/sys/dev/random/hash.c projects/fuse2/sys/dev/random/hash.h projects/fuse2/sys/dev/random/other_algorithm.c projects/fuse2/sys/dev/random/randomdev.c projects/fuse2/sys/dev/random/randomdev.h projects/fuse2/sys/dev/random/uint128.h projects/fuse2/sys/dev/sdhci/sdhci.c projects/fuse2/sys/dev/sdhci/sdhci_xenon.c projects/fuse2/sys/dev/ti/if_ti.c projects/fuse2/sys/dev/uart/uart_tty.c projects/fuse2/sys/dev/usb/net/if_cdce.c projects/fuse2/sys/dev/usb/serial/umcs.c projects/fuse2/sys/dev/usb/serial/usb_serial.c projects/fuse2/sys/dev/usb/usb_hub.c projects/fuse2/sys/dev/xen/gntdev/gntdev.c projects/fuse2/sys/fs/ext2fs/ext2_csum.c projects/fuse2/sys/fs/nandfs/nandfs_segment.c projects/fuse2/sys/fs/nandfs/nandfs_subr.c projects/fuse2/sys/fs/nandfs/nandfs_vfsops.c projects/fuse2/sys/geom/concat/g_concat.c projects/fuse2/sys/geom/eli/g_eli.c projects/fuse2/sys/geom/eli/g_eli.h projects/fuse2/sys/geom/eli/g_eli_hmac.c projects/fuse2/sys/geom/eli/pkcs5v2.c projects/fuse2/sys/geom/eli/pkcs5v2.h projects/fuse2/sys/geom/geom_disk.c projects/fuse2/sys/geom/geom_dump.c projects/fuse2/sys/geom/geom_int.h projects/fuse2/sys/geom/geom_kern.c projects/fuse2/sys/geom/journal/g_journal.c projects/fuse2/sys/geom/mirror/g_mirror.c projects/fuse2/sys/geom/multipath/g_multipath.c projects/fuse2/sys/geom/part/g_part_apm.c projects/fuse2/sys/geom/part/g_part_bsd64.c projects/fuse2/sys/geom/part/g_part_gpt.c projects/fuse2/sys/geom/part/g_part_mbr.c projects/fuse2/sys/geom/raid/g_raid.c projects/fuse2/sys/geom/raid/md_ddf.c projects/fuse2/sys/geom/raid3/g_raid3.c projects/fuse2/sys/geom/stripe/g_stripe.c projects/fuse2/sys/i386/i386/pmap.c projects/fuse2/sys/i386/i386/trap.c projects/fuse2/sys/kern/imgact_elf.c projects/fuse2/sys/kern/kern_clock.c projects/fuse2/sys/kern/kern_sendfile.c projects/fuse2/sys/kern/kern_synch.c projects/fuse2/sys/kern/kern_sysctl.c projects/fuse2/sys/kern/kern_uuid.c projects/fuse2/sys/kern/link_elf.c projects/fuse2/sys/kern/subr_compressor.c projects/fuse2/sys/kern/subr_sbuf.c projects/fuse2/sys/kern/subr_sleepqueue.c projects/fuse2/sys/kern/subr_taskqueue.c projects/fuse2/sys/kern/tty.c projects/fuse2/sys/kern/vfs_default.c projects/fuse2/sys/kern/vfs_vnops.c projects/fuse2/sys/kgssapi/krb5/kcrypto.c projects/fuse2/sys/kgssapi/krb5/kcrypto.h projects/fuse2/sys/kgssapi/krb5/kcrypto_arcfour.c projects/fuse2/sys/kgssapi/krb5/kcrypto_des.c projects/fuse2/sys/kgssapi/krb5/kcrypto_des3.c projects/fuse2/sys/libkern/x86/crc32_sse42.c projects/fuse2/sys/mips/mips/pmap.c projects/fuse2/sys/modules/Makefile projects/fuse2/sys/modules/allwinner/aw_pwm/Makefile projects/fuse2/sys/modules/efirt/Makefile projects/fuse2/sys/modules/hwpmc/Makefile projects/fuse2/sys/modules/iwmfw/Makefile.inc projects/fuse2/sys/modules/iwnfw/Makefile.inc projects/fuse2/sys/modules/mwlfw/Makefile projects/fuse2/sys/modules/ocs_fc/Makefile projects/fuse2/sys/modules/ralfw/Makefile.inc projects/fuse2/sys/modules/rtwnfw/Makefile.inc projects/fuse2/sys/modules/ti/Makefile projects/fuse2/sys/modules/usb/rsufw/Makefile.inc projects/fuse2/sys/modules/usb/runfw/Makefile projects/fuse2/sys/modules/usb/usb/Makefile projects/fuse2/sys/net/if_lagg.c projects/fuse2/sys/net/iflib.c projects/fuse2/sys/net/iflib.h projects/fuse2/sys/net/iflib_private.h projects/fuse2/sys/net80211/ieee80211.c projects/fuse2/sys/netinet/ip_carp.c projects/fuse2/sys/netinet/ip_output.c projects/fuse2/sys/netinet/libalias/alias_sctp.c projects/fuse2/sys/netinet/sctp_crc32.c projects/fuse2/sys/netinet/tcp_stacks/rack.c projects/fuse2/sys/netinet/tcp_stacks/tcp_rack.h projects/fuse2/sys/netinet6/ip6_output.c projects/fuse2/sys/netipsec/ipsec.c projects/fuse2/sys/netipsec/ipsec.h projects/fuse2/sys/netipsec/xform_ah.c projects/fuse2/sys/netipsec/xform_esp.c projects/fuse2/sys/netpfil/pf/pf.c projects/fuse2/sys/opencrypto/cryptodev.c projects/fuse2/sys/powerpc/aim/mmu_oea.c projects/fuse2/sys/powerpc/aim/mmu_oea64.c projects/fuse2/sys/powerpc/aim/mmu_oea64.h projects/fuse2/sys/powerpc/aim/slb.c projects/fuse2/sys/powerpc/include/slb.h projects/fuse2/sys/powerpc/ofw/ofw_initrd.c projects/fuse2/sys/powerpc/ofw/ofw_pcibus.c projects/fuse2/sys/powerpc/powernv/opal_hmi.c projects/fuse2/sys/powerpc/powernv/opal_sensor.c projects/fuse2/sys/powerpc/powerpc/trap.c projects/fuse2/sys/powerpc/pseries/mmu_phyp.c projects/fuse2/sys/powerpc/pseries/xics.c projects/fuse2/sys/riscv/conf/GENERIC projects/fuse2/sys/riscv/include/asm.h projects/fuse2/sys/riscv/include/elf.h projects/fuse2/sys/riscv/include/md_var.h projects/fuse2/sys/riscv/include/pcpu.h projects/fuse2/sys/riscv/riscv/elf_machdep.c projects/fuse2/sys/riscv/riscv/exception.S projects/fuse2/sys/riscv/riscv/identcpu.c projects/fuse2/sys/riscv/riscv/locore.S projects/fuse2/sys/riscv/riscv/machdep.c projects/fuse2/sys/riscv/riscv/mp_machdep.c projects/fuse2/sys/riscv/riscv/pmap.c projects/fuse2/sys/riscv/riscv/swtch.S projects/fuse2/sys/riscv/riscv/vm_machdep.c projects/fuse2/sys/sys/_termios.h projects/fuse2/sys/sys/counter.h projects/fuse2/sys/sys/elf_common.h projects/fuse2/sys/sys/fail.h projects/fuse2/sys/sys/filio.h projects/fuse2/sys/sys/libkern.h projects/fuse2/sys/sys/mbuf.h projects/fuse2/sys/sys/param.h projects/fuse2/sys/sys/queue.h projects/fuse2/sys/sys/sbuf.h projects/fuse2/sys/sys/sleepqueue.h projects/fuse2/sys/sys/syscallsubr.h projects/fuse2/sys/sys/sysctl.h projects/fuse2/sys/sys/systm.h projects/fuse2/sys/sys/ucred.h projects/fuse2/sys/ufs/ffs/ffs_alloc.c projects/fuse2/sys/ufs/ffs/ffs_snapshot.c projects/fuse2/sys/ufs/ffs/ffs_subr.c projects/fuse2/sys/ufs/ffs/ffs_vfsops.c projects/fuse2/sys/ufs/ufs/ufs_bmap.c projects/fuse2/sys/vm/swap_pager.c projects/fuse2/sys/vm/uma_core.c projects/fuse2/sys/vm/vm_glue.c projects/fuse2/sys/vm/vm_kern.c projects/fuse2/sys/vm/vm_map.c projects/fuse2/sys/vm/vm_map.h projects/fuse2/sys/vm/vm_mmap.c projects/fuse2/sys/vm/vm_page.c projects/fuse2/sys/vm/vm_reserv.c projects/fuse2/sys/x86/x86/mca.c projects/fuse2/tests/sys/Makefile projects/fuse2/tests/sys/kern/Makefile projects/fuse2/tests/sys/kern/libkern_crc32.c projects/fuse2/tests/sys/net/if_clone_test.sh projects/fuse2/tools/build/Makefile projects/fuse2/tools/build/mk/OptionalObsoleteFiles.inc projects/fuse2/usr.bin/awk/Makefile projects/fuse2/usr.bin/bsdcat/Makefile projects/fuse2/usr.bin/calendar/calendars/calendar.birthday projects/fuse2/usr.bin/cpio/Makefile projects/fuse2/usr.bin/procstat/procstat_auxv.c projects/fuse2/usr.bin/tail/extern.h projects/fuse2/usr.bin/tail/forward.c projects/fuse2/usr.bin/tail/misc.c projects/fuse2/usr.bin/tail/read.c projects/fuse2/usr.bin/tail/reverse.c projects/fuse2/usr.bin/tail/tail.c projects/fuse2/usr.bin/tar/Makefile projects/fuse2/usr.bin/vi/catalog/Makefile projects/fuse2/usr.bin/vtfontcvt/vtfontcvt.c projects/fuse2/usr.sbin/bhyve/Makefile projects/fuse2/usr.sbin/bhyve/pci_e82545.c projects/fuse2/usr.sbin/bhyve/pci_emul.c projects/fuse2/usr.sbin/bhyve/pci_emul.h projects/fuse2/usr.sbin/bhyve/pci_nvme.c projects/fuse2/usr.sbin/bhyve/pci_passthru.c projects/fuse2/usr.sbin/bhyve/pci_virtio_console.c projects/fuse2/usr.sbin/bhyve/pci_virtio_net.c projects/fuse2/usr.sbin/bhyve/pci_virtio_scsi.c projects/fuse2/usr.sbin/bhyve/rfb.c (contents, props changed) projects/fuse2/usr.sbin/bhyve/virtio.c projects/fuse2/usr.sbin/bhyve/virtio.h projects/fuse2/usr.sbin/camdd/camdd.c projects/fuse2/usr.sbin/iostat/iostat.c projects/fuse2/usr.sbin/pwm/pwm.8 projects/fuse2/usr.sbin/pwm/pwm.c Directory Properties: projects/fuse2/ (props changed) projects/fuse2/cddl/ (props changed) projects/fuse2/contrib/compiler-rt/ (props changed) projects/fuse2/contrib/elftoolchain/ (props changed) projects/fuse2/contrib/elftoolchain/elfdump/ (props changed) projects/fuse2/contrib/libarchive/ (props changed) projects/fuse2/contrib/libc++/ (props changed) projects/fuse2/contrib/libunwind/ (props changed) projects/fuse2/contrib/llvm/ (props changed) projects/fuse2/contrib/llvm/tools/clang/ (props changed) projects/fuse2/contrib/llvm/tools/lld/ (props changed) projects/fuse2/contrib/llvm/tools/lldb/ (props changed) projects/fuse2/contrib/openmp/ (props changed) projects/fuse2/gnu/lib/ (props changed) projects/fuse2/sys/cddl/contrib/opensolaris/ (props changed) projects/fuse2/sys/contrib/ipfilter/ (props changed) Copied: projects/fuse2/.clang-format (from r349234, head/.clang-format) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ projects/fuse2/.clang-format Thu Jun 20 15:56:08 2019 (r349236, copy of r349234, head/.clang-format) @@ -0,0 +1,77 @@ +# $FreeBSD$ +# Basic .clang-format +--- +BasedOnStyle: WebKit +AlignAfterOpenBracket: DontAlign +AlignConsecutiveAssignments: false +AlignConsecutiveDeclarations: false +AlignEscapedNewlines: Left +AlignOperands: false +AlignTrailingComments: false +AllowAllParametersOfDeclarationOnNextLine: false +AllowShortBlocksOnASingleLine: false +AllowShortCaseLabelsOnASingleLine: false +AllowShortFunctionsOnASingleLine: InlineOnly +AllowShortIfStatementsOnASingleLine: false +AllowShortLoopsOnASingleLine: false +AlwaysBreakAfterReturnType: TopLevelDefinitions +AlwaysBreakBeforeMultilineStrings: false +AlwaysBreakTemplateDeclarations: MultiLine +BinPackArguments: true +BinPackParameters: true +BreakBeforeBinaryOperators: None +BreakBeforeBraces: WebKit +BreakBeforeTernaryOperators: false +# TODO: BreakStringLiterals can cause very strange formatting so turn it off? +BreakStringLiterals: false +PenaltyBreakBeforeFirstCallParameter: 1000 +CompactNamespaces: true +DerivePointerAlignment: false +DisableFormat: false +ForEachMacros: + - SLIST_FOREACH + - SLIST_FOREACH_SAFE + - LIST_FOREACH + - LIST_FOREACH_SAFE + - STAILQ_FOREACH + - STAILQ_FOREACH_SAFE + - TAILQ_FOREACH + - TAILQ_FOREACH_SAFE + - TAILQ_FOREACH_REVERSE + - TAILQ_FOREACH_REVERSE_SAFE + - RB_FOREACH + - RB_FOREACH_SAFE + - RB_FOREACH_FROM + - RB_FOREACH_REVERSE + - RB_FOREACH_REVERSE_FROM + - RB_FOREACH_REVERSE_SAFE + - FOREACH_THREAD_IN_PROC + - FOREACH_PROC_IN_SYSTEM + - FOREACH_PRISON_CHILD + - FOREACH_PRISON_DESCENDANT + - FOREACH_PRISON_DESCENDANT_LOCKED + - FOREACH_PRISON_DESCENDANT_LOCKED_LEVEL + - MNT_VNODE_FOREACH_ALL + - MNT_VNODE_FOREACH_ACTIVE +IndentCaseLabels: false +IndentPPDirectives: None +Language: Cpp +NamespaceIndentation: None +PointerAlignment: Right +ContinuationIndentWidth: 4 +IndentWidth: 8 +TabWidth: 8 +ColumnLimit: 80 +UseTab: Always +SpaceAfterCStyleCast: false +SortIncludes: false +KeepEmptyLinesAtTheStartOfBlocks: true +# The options below will only be supported starting with clang 9.0: +# TODO-CLANG-9: TypenameMacros: +# TODO-CLANG-9: - SLIST_HEAD +# TODO-CLANG-9: - SLIST_ENTRY +# TODO-CLANG-9: - TAILQ_ENTRY +# TODO-CLANG-9: - TAILQ_HEAD +# TODO-CLANG-9: - STAILQ_ENTRY +# TODO-CLANG-9: - STAILQ_HEAD +... Modified: projects/fuse2/.gitattributes ============================================================================== --- projects/fuse2/.gitattributes Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/.gitattributes Thu Jun 20 15:56:08 2019 (r349236) @@ -4,3 +4,4 @@ *.hpp diff=cpp *.py diff=python . svn-properties=svn:keywords=tools/build/options/WITHOUT_LOADER_ZFS +.clang-format svn-properties=svn:keywords=FreeBSD=%H Modified: projects/fuse2/Makefile.inc1 ============================================================================== --- projects/fuse2/Makefile.inc1 Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/Makefile.inc1 Thu Jun 20 15:56:08 2019 (r349236) @@ -553,16 +553,23 @@ VERSION= FreeBSD ${_REVISION}-${_BRANCH:C/-p[0-9]+$//} .endif .if !defined(PKG_VERSION) -.if ${_BRANCH:MSTABLE*} || ${_BRANCH:MCURRENT*} || ${_BRANCH:MALPHA*} +.if ${_BRANCH:MSTABLE*} || ${_BRANCH:MCURRENT*} TIMENOW= %Y%m%d%H%M%S EXTRA_REVISION= .s${TIMENOW:gmtime} -.endif -.if ${_BRANCH:M*-p*} +.elif ${_BRANCH:MALPHA*} +EXTRA_REVISION= _${_BRANCH:C/-ALPHA/.a/} +.elif ${_BRANCH:MBETA*} +EXTRA_REVISION= _${_BRANCH:C/-BETA/.b/} +.elif ${_BRANCH:MRC*} +EXTRA_REVISION= _${_BRANCH:C/-RC/.r/} +.elif ${_BRANCH:MPRERELEASE*} +EXTRA_REVISION= _${_BRANCH:C/-PRERELEASE/.p/} +.elif ${_BRANCH:M*-p*} EXTRA_REVISION= _${_BRANCH:C/.*-p([0-9]+$)/\1/} .endif PKG_VERSION= ${_REVISION}${EXTRA_REVISION} .endif -.endif # !defined(_MKSHOWCONFIG) +.endif # !defined(PKG_VERSION) .if !defined(_MKSHOWCONFIG) _CPUTYPE!= MAKEFLAGS= CPUTYPE=${_TARGET_CPUTYPE} ${MAKE} -f /dev/null \ @@ -2546,6 +2553,7 @@ NXBDIRS+= \ usr.bin/find \ usr.bin/grep \ usr.bin/gzip \ + usr.bin/head \ usr.bin/id \ usr.bin/lex \ usr.bin/limits \ Modified: projects/fuse2/ObsoleteFiles.inc ============================================================================== --- projects/fuse2/ObsoleteFiles.inc Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/ObsoleteFiles.inc Thu Jun 20 15:56:08 2019 (r349236) @@ -38,6 +38,161 @@ # xargs -n1 | sort | uniq -d; # done +# 20190618: sys/capability.h removed (sys/capsicum.h is the one to use) +OLD_FILES+=usr/include/sys/capability.h +# 20190615: sys/pwm.h renamed to dev/pwmc.h +OLD_FILES+=usr/include/sys/pwm.h +# 20190612: new clang import which bumps version from 8.0.0 to 8.0.1. +OLD_FILES+=usr/lib/clang/8.0.0/include/sanitizer/allocator_interface.h +OLD_FILES+=usr/lib/clang/8.0.0/include/sanitizer/asan_interface.h +OLD_FILES+=usr/lib/clang/8.0.0/include/sanitizer/common_interface_defs.h +OLD_FILES+=usr/lib/clang/8.0.0/include/sanitizer/coverage_interface.h +OLD_FILES+=usr/lib/clang/8.0.0/include/sanitizer/dfsan_interface.h +OLD_FILES+=usr/lib/clang/8.0.0/include/sanitizer/esan_interface.h +OLD_FILES+=usr/lib/clang/8.0.0/include/sanitizer/hwasan_interface.h +OLD_FILES+=usr/lib/clang/8.0.0/include/sanitizer/linux_syscall_hooks.h +OLD_FILES+=usr/lib/clang/8.0.0/include/sanitizer/lsan_interface.h +OLD_FILES+=usr/lib/clang/8.0.0/include/sanitizer/msan_interface.h +OLD_FILES+=usr/lib/clang/8.0.0/include/sanitizer/netbsd_syscall_hooks.h +OLD_FILES+=usr/lib/clang/8.0.0/include/sanitizer/scudo_interface.h +OLD_FILES+=usr/lib/clang/8.0.0/include/sanitizer/tsan_interface.h +OLD_FILES+=usr/lib/clang/8.0.0/include/sanitizer/tsan_interface_atomic.h +OLD_DIRS+=usr/lib/clang/8.0.0/include/sanitizer +OLD_FILES+=usr/lib/clang/8.0.0/include/__clang_cuda_builtin_vars.h +OLD_FILES+=usr/lib/clang/8.0.0/include/__clang_cuda_cmath.h +OLD_FILES+=usr/lib/clang/8.0.0/include/__clang_cuda_complex_builtins.h +OLD_FILES+=usr/lib/clang/8.0.0/include/__clang_cuda_device_functions.h +OLD_FILES+=usr/lib/clang/8.0.0/include/__clang_cuda_intrinsics.h +OLD_FILES+=usr/lib/clang/8.0.0/include/__clang_cuda_libdevice_declares.h +OLD_FILES+=usr/lib/clang/8.0.0/include/__clang_cuda_math_forward_declares.h +OLD_FILES+=usr/lib/clang/8.0.0/include/__clang_cuda_runtime_wrapper.h +OLD_FILES+=usr/lib/clang/8.0.0/include/__stddef_max_align_t.h +OLD_FILES+=usr/lib/clang/8.0.0/include/__wmmintrin_aes.h +OLD_FILES+=usr/lib/clang/8.0.0/include/__wmmintrin_pclmul.h +OLD_FILES+=usr/lib/clang/8.0.0/include/adxintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/altivec.h +OLD_FILES+=usr/lib/clang/8.0.0/include/ammintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/arm64intr.h +OLD_FILES+=usr/lib/clang/8.0.0/include/arm_acle.h +OLD_FILES+=usr/lib/clang/8.0.0/include/arm_fp16.h +OLD_FILES+=usr/lib/clang/8.0.0/include/arm_neon.h +OLD_FILES+=usr/lib/clang/8.0.0/include/armintr.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx2intrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512bitalgintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512bwintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512cdintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512dqintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512erintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512fintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512ifmaintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512ifmavlintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512pfintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512vbmi2intrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512vbmiintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512vbmivlintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512vlbitalgintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512vlbwintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512vlcdintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512vldqintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512vlintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512vlvbmi2intrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512vlvnniintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512vnniintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512vpopcntdqintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avx512vpopcntdqvlintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/avxintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/bmi2intrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/bmiintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/cetintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/cldemoteintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/clflushoptintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/clwbintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/clzerointrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/cpuid.h +OLD_FILES+=usr/lib/clang/8.0.0/include/emmintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/f16cintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/fma4intrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/fmaintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/fxsrintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/gfniintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/htmintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/htmxlintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/ia32intrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/immintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/invpcidintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/lwpintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/lzcntintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/mm3dnow.h +OLD_FILES+=usr/lib/clang/8.0.0/include/mm_malloc.h +OLD_FILES+=usr/lib/clang/8.0.0/include/mmintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/module.modulemap +OLD_FILES+=usr/lib/clang/8.0.0/include/movdirintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/msa.h +OLD_FILES+=usr/lib/clang/8.0.0/include/mwaitxintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/nmmintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/opencl-c.h +OLD_FILES+=usr/lib/clang/8.0.0/include/pconfigintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/pkuintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/pmmintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/popcntintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/prfchwintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/ptwriteintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/rdseedintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/rtmintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/s390intrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/sgxintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/shaintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/smmintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/tbmintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/tmmintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/vadefs.h +OLD_FILES+=usr/lib/clang/8.0.0/include/vaesintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/vecintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/vpclmulqdqintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/waitpkgintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/wbnoinvdintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/wmmintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/x86intrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/xmmintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/xopintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/xsavecintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/xsaveintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/xsaveoptintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/xsavesintrin.h +OLD_FILES+=usr/lib/clang/8.0.0/include/xtestintrin.h +OLD_DIRS+=usr/lib/clang/8.0.0/include +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.asan-i386.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.asan-i386.so +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.asan-preinit-i386.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.asan-preinit-x86_64.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.asan-x86_64.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.asan-x86_64.so +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.asan_cxx-i386.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.asan_cxx-x86_64.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.msan-x86_64.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.msan_cxx-x86_64.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.profile-arm.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.profile-armhf.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.profile-i386.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.profile-x86_64.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.safestack-i386.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.safestack-x86_64.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.stats-i386.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.stats-x86_64.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.stats_client-i386.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.stats_client-x86_64.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.tsan-x86_64.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.tsan_cxx-x86_64.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.ubsan_minimal-i386.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.ubsan_minimal-x86_64.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.ubsan_standalone-i386.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.ubsan_standalone-x86_64.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.ubsan_standalone_cxx-i386.a +OLD_FILES+=usr/lib/clang/8.0.0/lib/freebsd/libclang_rt.ubsan_standalone_cxx-x86_64.a +OLD_DIRS+=usr/lib/clang/8.0.0/lib/freebsd +OLD_DIRS+=usr/lib/clang/8.0.0/lib +OLD_DIRS+=usr/lib/clang/8.0.0 + # 20190523: Remove obsolete kgzip and support files OLD_FILES+=usr/sbin/kgzip OLD_FILES+=usr/lib/kgzldr.o @@ -1409,6 +1564,8 @@ OLD_DIRS+=usr/share/openssl/man/en.ISO8859-1/cat1 OLD_DIRS+=usr/share/openssl/man/en.ISO8859-1/cat3 # 20170802: ksyms(4) ioctl interface was removed OLD_FILES+=usr/include/sys/ksyms.h +# 20170729: the iicbus/pcf8563 driver is replaced with iicbus/nxprtc +OLD_FILES+=usr/include/dev/iicbus/pcf8563reg.h # 20170722: new clang import which bumps version from 4.0.0 to 5.0.0. OLD_FILES+=usr/lib/clang/4.0.0/include/sanitizer/allocator_interface.h OLD_FILES+=usr/lib/clang/4.0.0/include/sanitizer/asan_interface.h Modified: projects/fuse2/UPDATING ============================================================================== --- projects/fuse2/UPDATING Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/UPDATING Thu Jun 20 15:56:08 2019 (r349236) @@ -31,7 +31,7 @@ NOTE TO PEOPLE WHO THINK THAT FreeBSD 13.x IS SLOW: disable the most expensive debugging functionality run "ln -s 'abort:false,junk:false' /etc/malloc.conf".) -20190606: +20190620: The vfs.fusefs.sync_unmount and vfs.fusefs.init_backgrounded sysctls and the "-o sync_unmount" and "-o init_backgrounded" mount options have been removed from mount_fusefs(8). You can safely remove them from @@ -42,6 +42,17 @@ NOTE TO PEOPLE WHO THINK THAT FreeBSD 13.x IS SLOW: vfs.fusefs.data_cache_invalidate sysctls have been removed. If you felt the need to set any of them to a non-default value, please tell asomers@FreeBSD.org why. + +20190612: + Clang, llvm, lld, lldb, compiler-rt, libc++, libunwind and openmp have + been upgraded to 8.0.1. Please see the 20141231 entry below for + information about prerequisites and upgrading, if you are not already + using clang 3.5.0 or higher. + +20190608: + A fix was applied to i386 kernel modules to avoid panics with + dpcpu or vnet. Users need to recompile i386 kernel modules + having pcpu or vnet sections or they will refuse to load. 20190513: User-wired pages now have their own counter, Modified: projects/fuse2/bin/csh/Makefile ============================================================================== --- projects/fuse2/bin/csh/Makefile Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/bin/csh/Makefile Thu Jun 20 15:56:08 2019 (r349236) @@ -117,6 +117,7 @@ csh.1: tcsh.man build-tools: gethost +DEPENDOBJS+= gethost gethost: gethost.c sh.err.h tc.const.h sh.h ${BUILD_TOOLS_META} @rm -f ${.TARGET} ${CC:N${CCACHE_BIN}} -o gethost ${LDFLAGS} ${CFLAGS:C/-DHAVE_ICONV//} \ Modified: projects/fuse2/bin/sh/Makefile ============================================================================== --- projects/fuse2/bin/sh/Makefile Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/bin/sh/Makefile Thu Jun 20 15:56:08 2019 (r349236) @@ -48,6 +48,7 @@ builtins.h: .NOMETA builtins.c builtins.h: mkbuiltins builtins.def sh ${.CURDIR}/mkbuiltins ${.CURDIR} +DEPENDOBJS+= mknodes mksyntax mknodes mksyntax: ${BUILD_TOOLS_META} .ORDER: nodes.c nodes.h Modified: projects/fuse2/bin/stty/modes.c ============================================================================== --- projects/fuse2/bin/stty/modes.c Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/bin/stty/modes.c Thu Jun 20 15:56:08 2019 (r349236) @@ -91,6 +91,8 @@ static const struct modes cmodes[] = { { "-rtsflow", 0, CRTS_IFLOW }, { "mdmbuf", MDMBUF, 0 }, { "-mdmbuf", 0, MDMBUF }, + { "rtsdtr", 0, CNO_RTSDTR }, + { "-rtsdtr", CNO_RTSDTR, 0 }, { NULL, 0, 0 }, }; Modified: projects/fuse2/bin/stty/print.c ============================================================================== --- projects/fuse2/bin/stty/print.c Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/bin/stty/print.c Thu Jun 20 15:56:08 2019 (r349236) @@ -184,6 +184,12 @@ print(struct termios *tp, struct winsize *wp, int ldis put("-dsrflow", CDSR_OFLOW, 0); put("-dtrflow", CDTR_IFLOW, 0); put("-mdmbuf", MDMBUF, 0); /* XXX mdmbuf == dtrflow */ + if (on(CNO_RTSDTR)) + bput("-rtsdtr"); + else { + if (fmt >= BSD) + bput("rtsdtr"); + } /* special control characters */ cc = tp->c_cc; Modified: projects/fuse2/bin/stty/stty.1 ============================================================================== --- projects/fuse2/bin/stty/stty.1 Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/bin/stty/stty.1 Thu Jun 20 15:56:08 2019 (r349236) @@ -145,6 +145,8 @@ Assume a line without (with) modem control. .It Cm crtscts Pq Fl crtscts Enable (disable) RTS/CTS flow control. +.It Cm rtsdtr Pq Fl -rtsdtr +Enable (disable) asserting RTS/DTR on open. .El .Ss Input Modes: This corresponds to the c_iflag in the termios structure. Copied: projects/fuse2/cddl/usr.sbin/dtrace/tests/amd64/Makefile (from r349234, head/cddl/usr.sbin/dtrace/tests/amd64/Makefile) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ projects/fuse2/cddl/usr.sbin/dtrace/tests/amd64/Makefile Thu Jun 20 15:56:08 2019 (r349236, copy of r349234, head/cddl/usr.sbin/dtrace/tests/amd64/Makefile) @@ -0,0 +1,9 @@ +# $FreeBSD$ + +TESTSDIR= ${TESTSBASE}/cddl/usr.sbin/dtrace/amd64 +TESTS_SUBDIRS+= arrays + +.PATH: ${.CURDIR:H:H:H:H:H}/tests +KYUAFILE= YES + +.include Modified: projects/fuse2/contrib/compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_netbsd.cc ============================================================================== --- projects/fuse2/contrib/compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_netbsd.cc Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/contrib/compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_netbsd.cc Thu Jun 20 15:56:08 2019 (r349236) @@ -124,7 +124,9 @@ #include #include #include +#if 0 #include +#endif #include #include #include Modified: projects/fuse2/contrib/elftoolchain/elfdump/elfdump.c ============================================================================== --- projects/fuse2/contrib/elftoolchain/elfdump/elfdump.c Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/contrib/elftoolchain/elfdump/elfdump.c Thu Jun 20 15:56:08 2019 (r349236) @@ -343,17 +343,20 @@ elf_phdr_type_str(unsigned int type) static char s_type[32]; switch (type) { - case PT_NULL: return "PT_NULL"; - case PT_LOAD: return "PT_LOAD"; - case PT_DYNAMIC: return "PT_DYNAMIC"; - case PT_INTERP: return "PT_INTERP"; - case PT_NOTE: return "PT_NOTE"; - case PT_SHLIB: return "PT_SHLIB"; - case PT_PHDR: return "PT_PHDR"; - case PT_TLS: return "PT_TLS"; - case PT_GNU_EH_FRAME: return "PT_GNU_EH_FRAME"; - case PT_GNU_STACK: return "PT_GNU_STACK"; - case PT_GNU_RELRO: return "PT_GNU_RELRO"; + case PT_NULL: return "PT_NULL"; + case PT_LOAD: return "PT_LOAD"; + case PT_DYNAMIC: return "PT_DYNAMIC"; + case PT_INTERP: return "PT_INTERP"; + case PT_NOTE: return "PT_NOTE"; + case PT_SHLIB: return "PT_SHLIB"; + case PT_PHDR: return "PT_PHDR"; + case PT_TLS: return "PT_TLS"; + case PT_GNU_EH_FRAME: return "PT_GNU_EH_FRAME"; + case PT_GNU_STACK: return "PT_GNU_STACK"; + case PT_GNU_RELRO: return "PT_GNU_RELRO"; + case PT_OPENBSD_RANDOMIZE: return "PT_OPENBSD_RANDOMIZE"; + case PT_OPENBSD_WXNEEDED: return "PT_OPENBSD_WXNEEDED"; + case PT_OPENBSD_BOOTDATA: return "PT_OPENBSD_BOOTDATA"; } snprintf(s_type, sizeof(s_type), "", type); return (s_type); Modified: projects/fuse2/contrib/elftoolchain/readelf/readelf.c ============================================================================== --- projects/fuse2/contrib/elftoolchain/readelf/readelf.c Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/contrib/elftoolchain/readelf/readelf.c Thu Jun 20 15:56:08 2019 (r349236) @@ -674,6 +674,9 @@ phdr_type(unsigned int mach, unsigned int ptype) case PT_GNU_EH_FRAME: return "GNU_EH_FRAME"; case PT_GNU_STACK: return "GNU_STACK"; case PT_GNU_RELRO: return "GNU_RELRO"; + case PT_OPENBSD_RANDOMIZE: return "OPENBSD_RANDOMIZE"; + case PT_OPENBSD_WXNEEDED: return "OPENBSD_WXNEEDED"; + case PT_OPENBSD_BOOTDATA: return "OPENBSD_BOOTDATA"; default: if (ptype >= PT_LOOS && ptype <= PT_HIOS) snprintf(s_ptype, sizeof(s_ptype), "LOOS+%#x", Modified: projects/fuse2/contrib/libarchive/NEWS ============================================================================== --- projects/fuse2/contrib/libarchive/NEWS Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/contrib/libarchive/NEWS Thu Jun 20 15:56:08 2019 (r349236) @@ -1,3 +1,7 @@ +Jun 11, 2019: libarchive 3.4.0 released + +May 18, 2019: Fixes for reading Android APK and JAR archives + Apr 16, 2019: Support for non-recursive list and extract Apr 14, 2019: New tar option: --exclude-vcs @@ -6,7 +10,7 @@ Mar 27, 2019: Support for file and directory symlinks Mar 12, 2019: Important fixes for storing file attributes and flags -Jan 20, 2019: Support for xz, lzma, ppmd8 and bzip2 compression in zip archives +Jan 20, 2019: Support for xz, lzma, ppmd8 and bzip2 decompression in ZIP files Oct 06, 2018: RAR 5.0 reader Modified: projects/fuse2/contrib/libarchive/README.md ============================================================================== --- projects/fuse2/contrib/libarchive/README.md Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/contrib/libarchive/README.md Thu Jun 20 15:56:08 2019 (r349236) @@ -81,6 +81,7 @@ Currently, the library automatically detects and reads * Binary cpio (big-endian or little-endian) * ISO9660 CD-ROM images (with optional Rockridge or Joliet extensions) * ZIP archives (with uncompressed or "deflate" compressed entries, including support for encrypted Zip archives) + * ZIPX archives (with support for bzip2, ppmd8, lzma and xz compressed entries) * GNU and BSD 'ar' archives * 'mtree' format * 7-Zip archives Modified: projects/fuse2/contrib/libarchive/libarchive/archive.h ============================================================================== --- projects/fuse2/contrib/libarchive/libarchive/archive.h Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/contrib/libarchive/libarchive/archive.h Thu Jun 20 15:56:08 2019 (r349236) @@ -36,7 +36,7 @@ * assert that ARCHIVE_VERSION_NUMBER >= 2012108. */ /* Note: Compiler will complain if this does not match archive_entry.h! */ -#define ARCHIVE_VERSION_NUMBER 3003003 +#define ARCHIVE_VERSION_NUMBER 3004000 #include #include /* for wchar_t */ @@ -155,7 +155,7 @@ __LA_DECL int archive_version_number(void); /* * Textual name/version of the library, useful for version displays. */ -#define ARCHIVE_VERSION_ONLY_STRING "3.3.3" +#define ARCHIVE_VERSION_ONLY_STRING "3.4.0" #define ARCHIVE_VERSION_STRING "libarchive " ARCHIVE_VERSION_ONLY_STRING __LA_DECL const char * archive_version_string(void); Modified: projects/fuse2/contrib/libarchive/libarchive/archive_entry.h ============================================================================== --- projects/fuse2/contrib/libarchive/libarchive/archive_entry.h Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/contrib/libarchive/libarchive/archive_entry.h Thu Jun 20 15:56:08 2019 (r349236) @@ -30,7 +30,7 @@ #define ARCHIVE_ENTRY_H_INCLUDED /* Note: Compiler will complain if this does not match archive.h! */ -#define ARCHIVE_VERSION_NUMBER 3003003 +#define ARCHIVE_VERSION_NUMBER 3004000 /* * Note: archive_entry.h is for use outside of libarchive; the Modified: projects/fuse2/contrib/libarchive/libarchive/archive_read_support_format_rar.c ============================================================================== --- projects/fuse2/contrib/libarchive/libarchive/archive_read_support_format_rar.c Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/contrib/libarchive/libarchive/archive_read_support_format_rar.c Thu Jun 20 15:56:08 2019 (r349236) @@ -1027,6 +1027,7 @@ archive_read_format_rar_read_data(struct archive_read if (ret != ARCHIVE_OK && ret != ARCHIVE_WARN) { __archive_ppmd7_functions.Ppmd7_Free(&rar->ppmd7_context); rar->start_new_table = 1; + rar->ppmd_valid = 0; } break; Modified: projects/fuse2/contrib/libarchive/libarchive/archive_read_support_format_rar5.c ============================================================================== --- projects/fuse2/contrib/libarchive/libarchive/archive_read_support_format_rar5.c Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/contrib/libarchive/libarchive/archive_read_support_format_rar5.c Thu Jun 20 15:56:08 2019 (r349236) @@ -1125,6 +1125,13 @@ static void init_header(struct archive_read* a) { a->archive.archive_format_name = "RAR5"; } +static void init_window_mask(struct rar5* rar) { + if (rar->cstate.window_size) + rar->cstate.window_mask = rar->cstate.window_size - 1; + else + rar->cstate.window_mask = 0; +} + enum HEADER_FLAGS { HFL_EXTRA_DATA = 0x0001, HFL_DATA = 0x0002, @@ -1672,6 +1679,7 @@ static int process_head_file(struct archive_read* a, s /* Values up to 64M should fit into ssize_t on every * architecture. */ rar->cstate.window_size = (ssize_t) window_size; + init_window_mask(rar); rar->file.solid = (compression_info & SOLID) > 0; rar->file.service = 0; @@ -2235,10 +2243,7 @@ static int rar5_read_header(struct archive_read *a, static void init_unpack(struct rar5* rar) { rar->file.calculated_crc32 = 0; - if (rar->cstate.window_size) - rar->cstate.window_mask = rar->cstate.window_size - 1; - else - rar->cstate.window_mask = 0; + init_window_mask(rar); free(rar->cstate.window_buf); free(rar->cstate.filtered_buf); @@ -2851,7 +2856,7 @@ static int do_uncompress_block(struct archive_read* a, * - Values lower than 256 are just bytes. Those codes * can be stored in the output buffer directly. * - * - Code 256 defines a new filter, which is later used to + * - Code 256 defines a new filter, which is later used to * ransform the data block accordingly to the filter type. * The data block needs to be fully uncompressed first. * @@ -3906,7 +3911,7 @@ static int rar5_read_data_skip(struct archive_read *a) /* Turn off "skip mode". */ rar->skip_mode--; - if(ret < 0) { + if(ret < 0 || ret == ARCHIVE_EOF) { /* Propagate any potential error conditions * to the caller. */ return ret; Modified: projects/fuse2/contrib/libarchive/libarchive/archive_write_add_filter_b64encode.c ============================================================================== --- projects/fuse2/contrib/libarchive/libarchive/archive_write_add_filter_b64encode.c Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/contrib/libarchive/libarchive/archive_write_add_filter_b64encode.c Thu Jun 20 15:56:08 2019 (r349236) @@ -60,7 +60,7 @@ static int archive_filter_b64encode_write(struct archi const void *, size_t); static int archive_filter_b64encode_close(struct archive_write_filter *); static int archive_filter_b64encode_free(struct archive_write_filter *); -static void b64_encode(struct archive_string *, const unsigned char *, size_t); +static void la_b64_encode(struct archive_string *, const unsigned char *, size_t); static int64_t atol8(const char *, size_t); static const char base64[] = { @@ -180,7 +180,7 @@ archive_filter_b64encode_open(struct archive_write_fil } static void -b64_encode(struct archive_string *as, const unsigned char *p, size_t len) +la_b64_encode(struct archive_string *as, const unsigned char *p, size_t len) { int c; @@ -234,12 +234,12 @@ archive_filter_b64encode_write(struct archive_write_fi } if (state->hold_len < LBYTES) return (ret); - b64_encode(&state->encoded_buff, state->hold, LBYTES); + la_b64_encode(&state->encoded_buff, state->hold, LBYTES); state->hold_len = 0; } for (; length >= LBYTES; length -= LBYTES, p += LBYTES) - b64_encode(&state->encoded_buff, p, LBYTES); + la_b64_encode(&state->encoded_buff, p, LBYTES); /* Save remaining bytes. */ if (length > 0) { @@ -270,7 +270,7 @@ archive_filter_b64encode_close(struct archive_write_fi /* Flush remaining bytes. */ if (state->hold_len != 0) - b64_encode(&state->encoded_buff, state->hold, state->hold_len); + la_b64_encode(&state->encoded_buff, state->hold, state->hold_len); archive_string_sprintf(&state->encoded_buff, "====\n"); /* Write the last block */ archive_write_set_bytes_in_last_block(f->archive, 1); Modified: projects/fuse2/contrib/libarchive/libarchive/archive_write_disk_posix.c ============================================================================== --- projects/fuse2/contrib/libarchive/libarchive/archive_write_disk_posix.c Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/contrib/libarchive/libarchive/archive_write_disk_posix.c Thu Jun 20 15:56:08 2019 (r349236) @@ -165,6 +165,10 @@ __FBSDID("$FreeBSD$"); #define O_NOFOLLOW 0 #endif +#ifndef AT_FDCWD +#define AT_FDCWD -100 +#endif + struct fixup_entry { struct fixup_entry *next; struct archive_acl acl; @@ -348,6 +352,8 @@ struct archive_write_disk { #define HFS_BLOCKS(s) ((s) >> 12) + +static int la_opendirat(int, const char *); static void fsobj_error(int *, struct archive_string *, int, const char *, const char *); static int check_symlinks_fsobj(char *, int *, struct archive_string *, @@ -401,6 +407,37 @@ static ssize_t _archive_write_disk_data_block(struct a size_t, int64_t); static int +la_opendirat(int fd, const char *path) { + const int flags = O_CLOEXEC +#if defined(O_BINARY) + | O_BINARY +#endif +#if defined(O_DIRECTORY) + | O_DIRECTORY +#endif +#if defined(O_PATH) + | O_PATH +#elif defined(O_SEARCH) + | O_SEARCH +#elif defined(O_EXEC) + | O_EXEC +#else + | O_RDONLY +#endif + ; + +#if !defined(HAVE_OPENAT) + if (fd != AT_FDCWD) { + errno = ENOTSUP; + return (-1); + } else + return (open(fd, path, flags)); +#else + return (openat(fd, path, flags)); +#endif +} + +static int lazy_stat(struct archive_write_disk *a) { if (a->pst != NULL) { @@ -1909,7 +1946,7 @@ edit_deep_directories(struct archive_write_disk *a) return; /* Try to record our starting dir. */ - a->restore_pwd = open(".", O_RDONLY | O_BINARY | O_CLOEXEC); + a->restore_pwd = la_opendirat(AT_FDCWD, "."); __archive_ensure_cloexec_flag(a->restore_pwd); if (a->restore_pwd < 0) return; @@ -2342,7 +2379,7 @@ _archive_write_disk_close(struct archive *_a) { struct archive_write_disk *a = (struct archive_write_disk *)_a; struct fixup_entry *next, *p; - int ret; + int fd, ret; archive_check_magic(&a->archive, ARCHIVE_WRITE_DISK_MAGIC, ARCHIVE_STATE_HEADER | ARCHIVE_STATE_DATA, @@ -2353,21 +2390,33 @@ _archive_write_disk_close(struct archive *_a) p = sort_dir_list(a->fixup_list); while (p != NULL) { + fd = -1; a->pst = NULL; /* Mark stat cache as out-of-date. */ + if (p->fixup & + (TODO_TIMES | TODO_MODE_BASE | TODO_ACLS | TODO_FFLAGS)) { + fd = open(p->name, + O_WRONLY | O_BINARY | O_NOFOLLOW | O_CLOEXEC); + } if (p->fixup & TODO_TIMES) { - set_times(a, -1, p->mode, p->name, + set_times(a, fd, p->mode, p->name, p->atime, p->atime_nanos, p->birthtime, p->birthtime_nanos, p->mtime, p->mtime_nanos, p->ctime, p->ctime_nanos); } - if (p->fixup & TODO_MODE_BASE) + if (p->fixup & TODO_MODE_BASE) { +#ifdef HAVE_FCHMOD + if (fd >= 0) + fchmod(fd, p->mode); + else +#endif chmod(p->name, p->mode); + } if (p->fixup & TODO_ACLS) - archive_write_disk_set_acls(&a->archive, -1, p->name, - &p->acl, p->mode); + archive_write_disk_set_acls(&a->archive, fd, + p->name, &p->acl, p->mode); if (p->fixup & TODO_FFLAGS) - set_fflags_platform(a, -1, p->name, + set_fflags_platform(a, fd, p->name, p->mode, p->fflags_set, 0); if (p->fixup & TODO_MAC_METADATA) set_mac_metadata(a, p->name, p->mac_metadata, @@ -2376,6 +2425,8 @@ _archive_write_disk_close(struct archive *_a) archive_acl_clear(&p->acl); free(p->mac_metadata); free(p->name); + if (fd >= 0) + close(fd); free(p); p = next; } @@ -2540,8 +2591,6 @@ fsobj_error(int *a_eno, struct archive_string *a_estr, * scan the path and both can be optimized by comparing against other * recent paths. */ -/* TODO: Extend this to support symlinks on Windows Vista and later. */ - /* * Checks the given path to see if any elements along it are symlinks. Returns * ARCHIVE_OK if there are none, otherwise puts an error in errmsg. @@ -2550,7 +2599,8 @@ static int check_symlinks_fsobj(char *path, int *a_eno, struct archive_string *a_estr, int flags) { -#if !defined(HAVE_LSTAT) +#if !defined(HAVE_LSTAT) && \ + !(defined(HAVE_OPENAT) && defined(HAVE_FSTATAT) && defined(HAVE_UNLINKAT)) /* Platform doesn't have lstat, so we can't look for symlinks. */ (void)path; /* UNUSED */ (void)error_number; /* UNUSED */ @@ -2565,7 +2615,10 @@ check_symlinks_fsobj(char *path, int *a_eno, struct ar char c; int r; struct stat st; - int restore_pwd; + int chdir_fd; +#if defined(HAVE_OPENAT) && defined(HAVE_FSTATAT) && defined(HAVE_UNLINKAT) + int fd; +#endif /* Nothing to do here if name is empty */ if(path[0] == '\0') @@ -2586,9 +2639,9 @@ check_symlinks_fsobj(char *path, int *a_eno, struct ar * c holds what used to be in *tail * last is 1 if this is the last tail */ - restore_pwd = open(".", O_RDONLY | O_BINARY | O_CLOEXEC); - __archive_ensure_cloexec_flag(restore_pwd); - if (restore_pwd < 0) { + chdir_fd = la_opendirat(AT_FDCWD, "."); + __archive_ensure_cloexec_flag(chdir_fd); + if (chdir_fd < 0) { fsobj_error(a_eno, a_estr, errno, "Could not open ", path); return (ARCHIVE_FATAL); @@ -2621,7 +2674,11 @@ check_symlinks_fsobj(char *path, int *a_eno, struct ar c = tail[0]; tail[0] = '\0'; /* Check that we haven't hit a symlink. */ +#if defined(HAVE_OPENAT) && defined(HAVE_FSTATAT) && defined(HAVE_UNLINKAT) + r = fstatat(chdir_fd, head, &st, AT_SYMLINK_NOFOLLOW); +#else r = lstat(head, &st); +#endif if (r != 0) { tail[0] = c; /* We've hit a dir that doesn't exist; stop now. */ @@ -2647,7 +2704,19 @@ check_symlinks_fsobj(char *path, int *a_eno, struct ar } } else if (S_ISDIR(st.st_mode)) { if (!last) { - if (chdir(head) != 0) { +#if defined(HAVE_OPENAT) && defined(HAVE_FSTATAT) && defined(HAVE_UNLINKAT) + fd = la_opendirat(chdir_fd, head); + if (fd < 0) + r = -1; + else { + r = 0; + close(chdir_fd); + chdir_fd = fd; + } +#else + r = chdir(head); +#endif + if (r != 0) { tail[0] = c; fsobj_error(a_eno, a_estr, errno, "Could not chdir ", path); @@ -2664,7 +2733,12 @@ check_symlinks_fsobj(char *path, int *a_eno, struct ar * so we can overwrite it with the * item being extracted. */ - if (unlink(head)) { +#if defined(HAVE_OPENAT) && defined(HAVE_FSTATAT) && defined(HAVE_UNLINKAT) + r = unlinkat(chdir_fd, head, 0); +#else + r = unlink(head); +#endif + if (r != 0) { tail[0] = c; fsobj_error(a_eno, a_estr, errno, "Could not remove symlink ", @@ -2694,7 +2768,12 @@ check_symlinks_fsobj(char *path, int *a_eno, struct ar break; } else if (flags & ARCHIVE_EXTRACT_UNLINK) { /* User asked us to remove problems. */ - if (unlink(head) != 0) { +#if defined(HAVE_OPENAT) && defined(HAVE_FSTATAT) && defined(HAVE_UNLINKAT) + r = unlinkat(chdir_fd, head, 0); +#else + r = unlink(head); +#endif + if (r != 0) { tail[0] = c; fsobj_error(a_eno, a_estr, 0, "Cannot remove intervening " @@ -2712,7 +2791,11 @@ check_symlinks_fsobj(char *path, int *a_eno, struct ar * This is needed to extract hardlinks over * symlinks. */ +#if defined(HAVE_OPENAT) && defined(HAVE_FSTATAT) && defined(HAVE_UNLINKAT) + r = fstatat(chdir_fd, head, &st, 0); +#else r = la_stat(head, &st); +#endif if (r != 0) { tail[0] = c; if (errno == ENOENT) { @@ -2725,7 +2808,19 @@ check_symlinks_fsobj(char *path, int *a_eno, struct ar break; } } else if (S_ISDIR(st.st_mode)) { - if (chdir(head) != 0) { +#if defined(HAVE_OPENAT) && defined(HAVE_FSTATAT) && defined(HAVE_UNLINKAT) + fd = la_opendirat(chdir_fd, head); + if (fd < 0) + r = -1; + else { + r = 0; + close(chdir_fd); + chdir_fd = fd; + } +#else + r = chdir(head); +#endif + if (r != 0) { tail[0] = c; fsobj_error(a_eno, a_estr, errno, @@ -2761,16 +2856,21 @@ check_symlinks_fsobj(char *path, int *a_eno, struct ar } /* Catches loop exits via break */ tail[0] = c; -#ifdef HAVE_FCHDIR +#if defined(HAVE_OPENAT) && defined(HAVE_FSTATAT) && defined(HAVE_UNLINKAT) + /* If we operate with openat(), fstatat() and unlinkat() there was + * no chdir(), so just close the fd */ + if (chdir_fd >= 0) + close(chdir_fd); +#elif HAVE_FCHDIR /* If we changed directory above, restore it here. */ - if (restore_pwd >= 0) { - r = fchdir(restore_pwd); + if (chdir_fd >= 0) { + r = fchdir(chdir_fd); if (r != 0) { fsobj_error(a_eno, a_estr, errno, "chdir() failure", ""); } - close(restore_pwd); - restore_pwd = -1; + close(chdir_fd); + chdir_fd = -1; if (r != 0) { res = (ARCHIVE_FATAL); } @@ -3387,6 +3487,7 @@ static int set_mode(struct archive_write_disk *a, int mode) { int r = ARCHIVE_OK; + int r2; mode &= 07777; /* Strip off file type bits. */ if (a->todo & TODO_SGID_CHECK) { @@ -3480,21 +3581,19 @@ set_mode(struct archive_write_disk *a, int mode) * post-extract fixup, which is handled elsewhere. */ #ifdef HAVE_FCHMOD - if (a->fd >= 0) { - if (fchmod(a->fd, mode) != 0) { - archive_set_error(&a->archive, errno, - "Can't set permissions to 0%o", (int)mode); - r = ARCHIVE_WARN; - } - } else + if (a->fd >= 0) + r2 = fchmod(a->fd, mode); + else #endif - /* If this platform lacks fchmod(), then - * we'll just use chmod(). */ - if (chmod(a->name, mode) != 0) { - archive_set_error(&a->archive, errno, - "Can't set permissions to 0%o", (int)mode); - r = ARCHIVE_WARN; - } + /* If this platform lacks fchmod(), then + * we'll just use chmod(). */ + r2 = chmod(a->name, mode); + + if (r2 != 0) { + archive_set_error(&a->archive, errno, + "Can't set permissions to 0%o", (int)mode); + r = ARCHIVE_WARN; + } } return (r); } Modified: projects/fuse2/contrib/libarchive/libarchive/test/test_read_format_rar.c ============================================================================== --- projects/fuse2/contrib/libarchive/libarchive/test/test_read_format_rar.c Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/contrib/libarchive/libarchive/test/test_read_format_rar.c Thu Jun 20 15:56:08 2019 (r349236) @@ -3776,6 +3776,35 @@ DEFINE_TEST(test_read_format_rar_ppmd_use_after_free) assertA(ARCHIVE_OK == archive_read_next_header(a, &ae)); assertA(archive_read_data(a, buf, sizeof(buf)) <= 0); + /* Test EOF */ + assertA(1 == archive_read_next_header(a, &ae)); + + assertEqualIntA(a, ARCHIVE_OK, archive_read_close(a)); + assertEqualInt(ARCHIVE_OK, archive_read_free(a)); +} + +DEFINE_TEST(test_read_format_rar_ppmd_use_after_free2) +{ + uint8_t buf[16]; + const char* reffile = "test_read_format_rar_ppmd_use_after_free2.rar"; + + struct archive_entry *ae; + struct archive *a; + + extract_reference_file(reffile); + assert((a = archive_read_new()) != NULL); + assertA(0 == archive_read_support_filter_all(a)); + assertA(0 == archive_read_support_format_all(a)); + assertA(0 == archive_read_open_filename(a, reffile, 10240)); + + assertA(ARCHIVE_OK == archive_read_next_header(a, &ae)); + assertA(archive_read_data(a, buf, sizeof(buf)) <= 0); + assertA(ARCHIVE_OK == archive_read_next_header(a, &ae)); + assertA(archive_read_data(a, buf, sizeof(buf)) <= 0); + + /* Test EOF */ + assertA(1 == archive_read_next_header(a, &ae)); + assertEqualIntA(a, ARCHIVE_OK, archive_read_close(a)); assertEqualInt(ARCHIVE_OK, archive_read_free(a)); } Modified: projects/fuse2/contrib/libarchive/libarchive/test/test_read_format_rar5.c ============================================================================== --- projects/fuse2/contrib/libarchive/libarchive/test/test_read_format_rar5.c Thu Jun 20 15:44:43 2019 (r349235) +++ projects/fuse2/contrib/libarchive/libarchive/test/test_read_format_rar5.c Thu Jun 20 15:56:08 2019 (r349236) @@ -1194,3 +1194,24 @@ DEFINE_TEST(test_read_format_rar5_fileattr) EPILOGUE(); } + *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-projects@freebsd.org Thu Jun 20 17:08:25 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 15B8615C1D16 for ; Thu, 20 Jun 2019 17:08:25 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AC82E8A64E; Thu, 20 Jun 2019 17:08:24 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7D894EF54; Thu, 20 Jun 2019 17:08:24 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5KH8OQY002687; Thu, 20 Jun 2019 17:08:24 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5KH8M6e002674; Thu, 20 Jun 2019 17:08:22 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906201708.x5KH8M6e002674@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Thu, 20 Jun 2019 17:08:22 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349238 - in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Commit-Revision: 349238 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: AC82E8A64E X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_SHORT(-0.99)[-0.985,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2019 17:08:25 -0000 Author: asomers Date: Thu Jun 20 17:08:21 2019 New Revision: 349238 URL: https://svnweb.freebsd.org/changeset/base/349238 Log: fusefs: implement VOP_BMAP If the fuse daemon supports FUSE_BMAP, then use that for the block mapping. Otherwise, use the same technique used by vop_stdbmap. Report large values for runp and runb in order to maximize read clustering and minimize upcalls, even if we don't know the true layout. The major result of this change is that sequential reads to FUSE files will now usually happen 128KB at a time instead of 64KB. Sponsored by: The FreeBSD Foundation Added: projects/fuse2/tests/sys/fs/fusefs/bmap.cc (contents, props changed) Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c projects/fuse2/sys/fs/fuse/fuse_io.c projects/fuse2/sys/fs/fuse/fuse_ipc.c projects/fuse2/sys/fs/fuse/fuse_ipc.h projects/fuse2/sys/fs/fuse/fuse_vnops.c projects/fuse2/tests/sys/fs/fusefs/Makefile projects/fuse2/tests/sys/fs/fusefs/mockfs.cc projects/fuse2/tests/sys/fs/fusefs/mockfs.hh projects/fuse2/tests/sys/fs/fusefs/read.cc projects/fuse2/tests/sys/fs/fusefs/utils.cc Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_internal.c Thu Jun 20 16:36:20 2019 (r349237) +++ projects/fuse2/sys/fs/fuse/fuse_internal.c Thu Jun 20 17:08:21 2019 (r349238) @@ -908,7 +908,8 @@ fuse_internal_init_callback(struct fuse_ticket *tick, if (fuse_libabi_geq(data, 7, 5)) { if (fticket_resp(tick)->len == sizeof(struct fuse_init_out)) { - data->max_readahead = fiio->max_readahead; + data->max_readahead_blocks = fiio->max_readahead / + maxbcachebuf; data->max_write = fiio->max_write; if (fiio->flags & FUSE_ASYNC_READ) data->dataflags |= FSESS_ASYNC_READ; Modified: projects/fuse2/sys/fs/fuse/fuse_io.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_io.c Thu Jun 20 16:36:20 2019 (r349237) +++ projects/fuse2/sys/fs/fuse/fuse_io.c Thu Jun 20 17:08:21 2019 (r349238) @@ -321,10 +321,10 @@ fuse_read_biobackend(struct vnode *vp, struct uio *uio /* Try clustered read */ long totread = uio->uio_resid + on; seqcount = MIN(seqcount, - data->max_readahead / biosize + 1); + data->max_readahead_blocks + 1); err = cluster_read(vp, filesize, lbn, bcount, NOCRED, totread, seqcount, 0, &bp); - } else if (seqcount > 1 && data->max_readahead >= nextsize) { + } else if (seqcount > 1 && data->max_readahead_blocks >= 1) { /* Try non-clustered readahead */ err = breadn(vp, lbn, bcount, &nextlbn, &nextsize, 1, NOCRED, &bp); Modified: projects/fuse2/sys/fs/fuse/fuse_ipc.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_ipc.c Thu Jun 20 16:36:20 2019 (r349237) +++ projects/fuse2/sys/fs/fuse/fuse_ipc.c Thu Jun 20 17:08:21 2019 (r349238) @@ -711,6 +711,10 @@ fuse_body_audit(struct fuse_ticket *ftick, size_t blen opcode = fticket_opcode(ftick); switch (opcode) { + case FUSE_BMAP: + err = (blen == sizeof(struct fuse_bmap_out)) ? 0 : EINVAL; + break; + case FUSE_LINK: case FUSE_LOOKUP: case FUSE_MKDIR: Modified: projects/fuse2/sys/fs/fuse/fuse_ipc.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_ipc.h Thu Jun 20 16:36:20 2019 (r349237) +++ projects/fuse2/sys/fs/fuse/fuse_ipc.h Thu Jun 20 17:08:21 2019 (r349238) @@ -197,7 +197,7 @@ struct fuse_data { uint32_t fuse_libabi_major; uint32_t fuse_libabi_minor; - uint32_t max_readahead; + uint32_t max_readahead_blocks; uint32_t max_write; uint32_t max_read; uint32_t subtype; Modified: projects/fuse2/sys/fs/fuse/fuse_vnops.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_vnops.c Thu Jun 20 16:36:20 2019 (r349237) +++ projects/fuse2/sys/fs/fuse/fuse_vnops.c Thu Jun 20 17:08:21 2019 (r349238) @@ -120,6 +120,7 @@ SDT_PROBE_DEFINE2(fusefs, , vnops, trace, "int", "char /* vnode ops */ static vop_access_t fuse_vnop_access; static vop_advlock_t fuse_vnop_advlock; +static vop_bmap_t fuse_vnop_bmap; static vop_close_t fuse_fifo_close; static vop_close_t fuse_vnop_close; static vop_create_t fuse_vnop_create; @@ -174,6 +175,7 @@ struct vop_vector fuse_vnops = { .vop_default = &default_vnodeops, .vop_access = fuse_vnop_access, .vop_advlock = fuse_vnop_advlock, + .vop_bmap = fuse_vnop_bmap, .vop_close = fuse_vnop_close, .vop_create = fuse_vnop_create, .vop_deleteextattr = fuse_vnop_deleteextattr, @@ -464,6 +466,92 @@ fuse_vnop_advlock(struct vop_advlock_args *ap) } return err; +} + +/* { + struct vnode *a_vp; + daddr_t a_bn; + struct bufobj **a_bop; + daddr_t *a_bnp; + int *a_runp; + int *a_runb; +} */ +static int +fuse_vnop_bmap(struct vop_bmap_args *ap) +{ + struct vnode *vp = ap->a_vp; + struct bufobj **bo = ap->a_bop; + struct thread *td = curthread; + struct mount *mp; + struct fuse_dispatcher fdi; + struct fuse_bmap_in *fbi; + struct fuse_bmap_out *fbo; + struct fuse_data *data; + uint64_t biosize; + off_t filesize; + daddr_t lbn = ap->a_bn; + daddr_t *pbn = ap->a_bnp; + int *runp = ap->a_runp; + int *runb = ap->a_runb; + int error = 0; + int maxrun; + + if (fuse_isdeadfs(vp)) { + return ENXIO; + } + + mp = vnode_mount(vp); + data = fuse_get_mpdata(mp); + biosize = fuse_iosize(vp); + maxrun = MIN(vp->v_mount->mnt_iosize_max / biosize - 1, + data->max_readahead_blocks); + + if (bo != NULL) + *bo = &vp->v_bufobj; + + /* + * The FUSE_BMAP operation does not include the runp and runb + * variables, so we must guess. Report nonzero contiguous runs so + * cluster_read will combine adjacent reads. It's worthwhile to reduce + * upcalls even if we don't know the true physical layout of the file. + * + * FUSE file systems may opt out of read clustering in two ways: + * * mounting with -onoclusterr + * * Setting max_readahead <= maxbcachebuf during FUSE_INIT + */ + if (runb != NULL) + *runb = MIN(lbn, maxrun); + if (runp != NULL) { + error = fuse_vnode_size(vp, &filesize, td->td_ucred, td); + if (error == 0) + *runp = MIN(MAX(0, filesize / biosize - lbn - 1), + maxrun); + else + *runp = 0; + } + + if (fsess_isimpl(mp, FUSE_BMAP)) { + fdisp_init(&fdi, sizeof(*fbi)); + fdisp_make_vp(&fdi, FUSE_BMAP, vp, td, td->td_ucred); + fbi = fdi.indata; + fbi->block = lbn; + fbi->blocksize = biosize; + error = fdisp_wait_answ(&fdi); + if (error == ENOSYS) { + fsess_set_notimpl(mp, FUSE_BMAP); + error = 0; + } else { + fbo = fdi.answ; + if (error == 0 && pbn != NULL) + *pbn = fbo->block; + return error; + } + } + + /* If the daemon doesn't support BMAP, make up a sensible default */ + if (pbn != NULL) + *pbn = lbn * btodb(biosize); + return (error); } /* Modified: projects/fuse2/tests/sys/fs/fusefs/Makefile ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/Makefile Thu Jun 20 16:36:20 2019 (r349237) +++ projects/fuse2/tests/sys/fs/fusefs/Makefile Thu Jun 20 17:08:21 2019 (r349238) @@ -9,6 +9,7 @@ TESTSDIR= ${TESTSBASE}/sys/fs/fusefs # out, so we get more granular reporting. GTESTS+= access GTESTS+= allow_other +GTESTS+= bmap GTESTS+= create GTESTS+= default_permissions GTESTS+= default_permissions_privileged Added: projects/fuse2/tests/sys/fs/fusefs/bmap.cc ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ projects/fuse2/tests/sys/fs/fusefs/bmap.cc Thu Jun 20 17:08:21 2019 (r349238) @@ -0,0 +1,159 @@ +/*- + * SPDX-License-Identifier: BSD-2-Clause-FreeBSD + * + * Copyright (c) 2019 The FreeBSD Foundation + * + * This software was developed by BFF Storage Systems, LLC under sponsorship + * from the FreeBSD Foundation. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +extern "C" { +#include +#include +#include + +#include +} + +#include "mockfs.hh" +#include "utils.hh" + +using namespace testing; + +const static char FULLPATH[] = "mountpoint/foo"; +const static char RELPATH[] = "foo"; + +class Bmap: public FuseTest { +public: +virtual void SetUp() { + m_maxreadahead = UINT32_MAX; + FuseTest::SetUp(); +} +void expect_bmap(uint64_t ino, uint64_t lbn, uint32_t blocksize, uint64_t pbn) +{ + EXPECT_CALL(*m_mock, process( + ResultOf([=](auto in) { + return (in.header.opcode == FUSE_BMAP && + in.header.nodeid == ino && + in.body.bmap.block == lbn && + in.body.bmap.blocksize == blocksize); + }, Eq(true)), + _) + ).WillOnce(Invoke(ReturnImmediate([=](auto i __unused, auto& out) { + SET_OUT_HEADER_LEN(out, bmap); + out.body.bmap.block = pbn; + }))); +} + +void expect_lookup(const char *relpath, uint64_t ino, off_t size) +{ + FuseTest::expect_lookup(relpath, ino, S_IFREG | 0644, size, 1, + UINT64_MAX); +} +}; + +/* + * Test FUSE_BMAP + * XXX The FUSE protocol does not include the runp and runb variables, so those + * must be guessed in-kernel. + */ +TEST_F(Bmap, bmap) +{ + struct fiobmap2_arg arg; + const off_t filesize = 1 << 20; + const ino_t ino = 42; + int64_t lbn = 10; + int64_t pbn = 12345; + int fd; + + expect_lookup(RELPATH, 42, filesize); + expect_open(ino, 0, 1); + expect_bmap(ino, lbn, m_maxbcachebuf, pbn); + + fd = open(FULLPATH, O_RDWR); + ASSERT_LE(0, fd) << strerror(errno); + + arg.bn = lbn; + arg.runp = -1; + arg.runb = -1; + ASSERT_EQ(0, ioctl(fd, FIOBMAP2, &arg)) << strerror(errno); + EXPECT_EQ(arg.bn, pbn); + EXPECT_EQ(arg.runp, MAXPHYS / m_maxbcachebuf - 1); + EXPECT_EQ(arg.runb, MAXPHYS / m_maxbcachebuf - 1); +} + +/* + * If the daemon does not implement VOP_BMAP, fusefs should return sensible + * defaults. + */ +TEST_F(Bmap, default_) +{ + struct fiobmap2_arg arg; + const off_t filesize = 1 << 20; + const ino_t ino = 42; + int64_t lbn; + int fd; + + expect_lookup(RELPATH, 42, filesize); + expect_open(ino, 0, 1); + EXPECT_CALL(*m_mock, process( + ResultOf([=](auto in) { + return (in.header.opcode == FUSE_BMAP); + }, Eq(true)), + _) + ).WillOnce(Invoke(ReturnErrno(ENOSYS))); + + fd = open(FULLPATH, O_RDWR); + ASSERT_LE(0, fd) << strerror(errno); + + /* First block */ + lbn = 0; + arg.bn = lbn; + arg.runp = -1; + arg.runb = -1; + ASSERT_EQ(0, ioctl(fd, FIOBMAP2, &arg)) << strerror(errno); + EXPECT_EQ(arg.bn, 0); + EXPECT_EQ(arg.runp, MAXPHYS / m_maxbcachebuf - 1); + EXPECT_EQ(arg.runb, 0); + + /* In the middle */ + lbn = filesize / m_maxbcachebuf / 2; + arg.bn = lbn; + arg.runp = -1; + arg.runb = -1; + ASSERT_EQ(0, ioctl(fd, FIOBMAP2, &arg)) << strerror(errno); + EXPECT_EQ(arg.bn, lbn * m_maxbcachebuf / DEV_BSIZE); + EXPECT_EQ(arg.runp, MAXPHYS / m_maxbcachebuf - 1); + EXPECT_EQ(arg.runb, MAXPHYS / m_maxbcachebuf - 1); + + /* Last block */ + lbn = filesize / m_maxbcachebuf - 1; + arg.bn = lbn; + arg.runp = -1; + arg.runb = -1; + ASSERT_EQ(0, ioctl(fd, FIOBMAP2, &arg)) << strerror(errno); + EXPECT_EQ(arg.bn, lbn * m_maxbcachebuf / DEV_BSIZE); + EXPECT_EQ(arg.runp, 0); + EXPECT_EQ(arg.runb, MAXPHYS / m_maxbcachebuf - 1); +} Modified: projects/fuse2/tests/sys/fs/fusefs/mockfs.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/mockfs.cc Thu Jun 20 16:36:20 2019 (r349237) +++ projects/fuse2/tests/sys/fs/fusefs/mockfs.cc Thu Jun 20 17:08:21 2019 (r349238) @@ -168,6 +168,10 @@ void MockFS::debug_request(const mockfs_buf_in &in) case FUSE_ACCESS: printf(" mask=%#x", in.body.access.mask); break; + case FUSE_BMAP: + printf(" block=%#lx blocksize=%#x", in.body.bmap.block, + in.body.bmap.blocksize); + break; case FUSE_CREATE: if (m_kernel_minor_version >= 12) name = (const char*)in.body.bytes + Modified: projects/fuse2/tests/sys/fs/fusefs/mockfs.hh ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/mockfs.hh Thu Jun 20 16:36:20 2019 (r349237) +++ projects/fuse2/tests/sys/fs/fusefs/mockfs.hh Thu Jun 20 17:08:21 2019 (r349238) @@ -124,6 +124,7 @@ struct fuse_create_out_7_8 { union fuse_payloads_in { fuse_access_in access; + fuse_bmap_in bmap; /* value is from fuse_kern_chan.c in fusefs-libs */ uint8_t bytes[0x21000 - sizeof(struct fuse_in_header)]; fuse_create_in create; @@ -164,6 +165,7 @@ struct mockfs_buf_in { union fuse_payloads_out { fuse_attr_out attr; fuse_attr_out_7_8 attr_7_8; + fuse_bmap_out bmap; fuse_create_out create; fuse_create_out_7_8 create_7_8; /* Modified: projects/fuse2/tests/sys/fs/fusefs/read.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/read.cc Thu Jun 20 16:36:20 2019 (r349237) +++ projects/fuse2/tests/sys/fs/fusefs/read.cc Thu Jun 20 17:08:21 2019 (r349238) @@ -29,7 +29,7 @@ */ extern "C" { -#include +#include #include #include #include @@ -749,16 +749,15 @@ TEST_F(ReadCacheable, DISABLED_sendfile_eio) /* Deliberately leak fd. close(2) will be tested in release.cc */ } -/* Large reads should be clustered, even across cache block boundaries */ -/* - * Disabled because clustered reads requires VOP_BMAP, which fusefs does not - * yet support +/* + * Sequential reads should use readahead. And if allowed, large reads should + * be clustered. */ -TEST_P(ReadAhead, DISABLED_cluster) { +TEST_P(ReadAhead, readahead) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; uint64_t ino = 42; - int fd, maxcontig; + int fd, maxcontig, clustersize; ssize_t bufsize = 4 * m_maxbcachebuf; ssize_t filesize = bufsize; uint64_t len; @@ -774,8 +773,9 @@ TEST_P(ReadAhead, DISABLED_cluster) { expect_open(ino, 0, 1); maxcontig = m_noclusterr ? m_maxbcachebuf : m_maxbcachebuf + (int)get<1>(GetParam()); - for (offs = 0; offs < bufsize; offs += maxcontig) { - len = std::min((size_t)maxcontig, (size_t)(filesize - offs)); + clustersize = MIN(maxcontig, MAXPHYS); + for (offs = 0; offs < bufsize; offs += clustersize) { + len = std::min((size_t)clustersize, (size_t)(filesize - offs)); expect_read(ino, offs, len, len, contents + offs); } @@ -791,47 +791,11 @@ TEST_P(ReadAhead, DISABLED_cluster) { /* Deliberately leak fd. close(2) will be tested in release.cc */ } -/* fuse(4) should honor the filesystem's requested m_readahead parameter */ -TEST_P(ReadAhead, readahead) { - const char FULLPATH[] = "mountpoint/some_file.txt"; - const char RELPATH[] = "some_file.txt"; - uint64_t ino = 42; - int fd, i; - ssize_t bufsize = m_maxbcachebuf; - ssize_t filesize = m_maxbcachebuf * 6; - char *rbuf, *contents; - - contents = (char*)malloc(filesize); - ASSERT_NE(NULL, contents); - memset(contents, 'X', filesize); - rbuf = (char*)calloc(1, bufsize); - - expect_lookup(RELPATH, ino, filesize); - expect_open(ino, 0, 1); - /* fuse(4) should only read ahead the allowed amount */ - expect_read(ino, 0, m_maxbcachebuf, m_maxbcachebuf, contents); - for (i = 0; i < (int)get<1>(GetParam()) / m_maxbcachebuf; i++) { - off_t offs = (i + 1) * m_maxbcachebuf; - expect_read(ino, offs, m_maxbcachebuf, m_maxbcachebuf, - contents + offs); - } - - fd = open(FULLPATH, O_RDONLY); - ASSERT_LE(0, fd) << strerror(errno); - - /* Set the internal readahead counter to a "large" value */ - ASSERT_EQ(0, fcntl(fd, F_READAHEAD, 1'000'000'000)) << strerror(errno); - - ASSERT_EQ(bufsize, read(fd, rbuf, bufsize)) << strerror(errno); - ASSERT_EQ(0, memcmp(rbuf, contents, bufsize)); - - /* Deliberately leak fd. close(2) will be tested in release.cc */ -} - INSTANTIATE_TEST_CASE_P(RA, ReadAhead, Values(tuple(false, 0u), tuple(false, 0x10000), tuple(false, 0x20000), tuple(false, 0x30000), tuple(true, 0u), - tuple(true, 0x10000))); + tuple(true, 0x10000), + tuple(true, 0x20000))); Modified: projects/fuse2/tests/sys/fs/fusefs/utils.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/utils.cc Thu Jun 20 16:36:20 2019 (r349237) +++ projects/fuse2/tests/sys/fs/fusefs/utils.cc Thu Jun 20 17:08:21 2019 (r349238) @@ -129,6 +129,20 @@ void FuseTest::SetUp() { _) ).Times(AnyNumber()) .WillRepeatedly(Invoke(ReturnErrno(ENOSYS))); + /* + * FUSE_BMAP is called for most test cases that read data. Set + * a default expectation and return ENOSYS. + * + * Individual test cases can override this expectation since + * googlemock evaluates expectations in LIFO order. + */ + EXPECT_CALL(*m_mock, process( + ResultOf([=](auto in) { + return (in.header.opcode == FUSE_BMAP); + }, Eq(true)), + _) + ).Times(AnyNumber()) + .WillRepeatedly(Invoke(ReturnErrno(ENOSYS))); } catch (std::system_error err) { FAIL() << err.what(); } From owner-svn-src-projects@freebsd.org Thu Jun 20 18:11:58 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 343CD15C3DBE for ; Thu, 20 Jun 2019 18:11:58 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C68D08FACB; Thu, 20 Jun 2019 18:11:57 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 0AFFA21F8E; Thu, 20 Jun 2019 14:11:51 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Thu, 20 Jun 2019 14:11:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsco.org; h= content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=fm3; bh=V RL4Y4LSyjCUPHshtNVWz93eM1O7XAPfYLcxD4lk4xI=; b=CYGmS0U299suyAb3o bFpRzU+qIpjKGTPcZaCbBLZGpehN7ric4RUQIT9bITjcTiQBbyoj/ioVQLQlOfmN IOSdEGxbqCHgXMNdz9ZIG5VHw0TlzYKPVrN0xeL45FESt5l1PVw6RT8nhVx8kvBP uiBIX9ULZb6xlo0UBO6EYYzdFa2hP76DnRGiHYjaZ2J3cDHz/0W4fG9aCli8Gh5f pggfwxHtwmD4BK/+bZfdhxFJfPotfsS/KbBnCl/4MTD7M3czKoIf05bqy+x48LYS 8/OUvYvZ7PSYlclMoow9igbBHLORuwJiYR7TmoRTEft5qJ4wx2fr+N86jkqtvVX1 OBaaQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=VRL4Y4LSyjCUPHshtNVWz93eM1O7XAPfYLcxD4lk4 xI=; b=q8VdX0qsZMtX1/licDHoDPkWX0YzqELbNa78F0aV8/zWZQJxM75sKKY9G v+UAbL/QFNLFMQR50uB8nOJyaIjV8wBfQm/5qrTY53tnMlzwCszRjHNNDDbBw8zN SC7A1a0GQZMl2HwRU2fxaR8GZX8II9V8fwHX3d+0Y7+p/CwWKUV7mVJSFjEmBCck pq8cAK/qMpZwRN8Aq1MEr8xv9WwrbDojzCPF2hXAq/m+8wDyxzKO/ngUbgjfb7q3 n6MGOtAsATZl1nzUWlyZdock5blLeZT2FNag0gI1u2pVT88rw6gV9/tc9TZ9ApDA P76tUvQi1LO92HQzvvwN5Ulmuh8Zw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrtdeggdduvdeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurheptggguffhjgffgffkfhfvofesthhqmhdthhdtvdenucfhrhhomhepufgtohht thcunfhonhhguceoshgtohhtthhlsehsrghmshgtohdrohhrgheqnecuffhomhgrihhnpe hfrhgvvggsshgurdhorhhgnecukfhppeduledvrdehhedrheegrdehleenucfrrghrrghm pehmrghilhhfrhhomhepshgtohhtthhlsehsrghmshgtohdrohhrghenucevlhhushhtvg hrufhiiigvpedt X-ME-Proxy: Received: from [10.178.24.10] (unknown [192.55.54.59]) by mail.messagingengine.com (Postfix) with ESMTPA id 97FFE380079; Thu, 20 Jun 2019 14:11:49 -0400 (EDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: svn commit: r349238 - in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs From: Scott Long In-Reply-To: <201906201708.x5KH8M6e002674@repo.freebsd.org> Date: Thu, 20 Jun 2019 12:11:48 -0600 Cc: src-committers@freebsd.org, svn-src-projects@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <5E43BBD0-3308-4CDC-9E26-28AE496E8281@samsco.org> References: <201906201708.x5KH8M6e002674@repo.freebsd.org> To: Alan Somers X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: C68D08FACB X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.98)[-0.981,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2019 18:11:58 -0000 Great work! What happens if you set MAXPHYS to something big like 1MB? Scott > On Jun 20, 2019, at 11:08 AM, Alan Somers wrote: >=20 > Author: asomers > Date: Thu Jun 20 17:08:21 2019 > New Revision: 349238 > URL: https://svnweb.freebsd.org/changeset/base/349238 >=20 > Log: > fusefs: implement VOP_BMAP >=20 > If the fuse daemon supports FUSE_BMAP, then use that for the block = mapping. > Otherwise, use the same technique used by vop_stdbmap. Report large = values > for runp and runb in order to maximize read clustering and minimize = upcalls, > even if we don't know the true layout. >=20 > The major result of this change is that sequential reads to FUSE = files will > now usually happen 128KB at a time instead of 64KB. >=20 > Sponsored by: The FreeBSD Foundation >=20 > Added: > projects/fuse2/tests/sys/fs/fusefs/bmap.cc (contents, props = changed) > Modified: > projects/fuse2/sys/fs/fuse/fuse_internal.c > projects/fuse2/sys/fs/fuse/fuse_io.c > projects/fuse2/sys/fs/fuse/fuse_ipc.c > projects/fuse2/sys/fs/fuse/fuse_ipc.h > projects/fuse2/sys/fs/fuse/fuse_vnops.c > projects/fuse2/tests/sys/fs/fusefs/Makefile > projects/fuse2/tests/sys/fs/fusefs/mockfs.cc > projects/fuse2/tests/sys/fs/fusefs/mockfs.hh > projects/fuse2/tests/sys/fs/fusefs/read.cc > projects/fuse2/tests/sys/fs/fusefs/utils.cc >=20 > Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- projects/fuse2/sys/fs/fuse/fuse_internal.c Thu Jun 20 = 16:36:20 2019 (r349237) > +++ projects/fuse2/sys/fs/fuse/fuse_internal.c Thu Jun 20 = 17:08:21 2019 (r349238) > @@ -908,7 +908,8 @@ fuse_internal_init_callback(struct fuse_ticket = *tick,=20 >=20 > if (fuse_libabi_geq(data, 7, 5)) { > if (fticket_resp(tick)->len =3D=3D sizeof(struct = fuse_init_out)) { > - data->max_readahead =3D fiio->max_readahead; > + data->max_readahead_blocks =3D = fiio->max_readahead / > + maxbcachebuf; > data->max_write =3D fiio->max_write; > if (fiio->flags & FUSE_ASYNC_READ) > data->dataflags |=3D FSESS_ASYNC_READ; >=20 > Modified: projects/fuse2/sys/fs/fuse/fuse_io.c > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- projects/fuse2/sys/fs/fuse/fuse_io.c Thu Jun 20 16:36:20 2019 = (r349237) > +++ projects/fuse2/sys/fs/fuse/fuse_io.c Thu Jun 20 17:08:21 2019 = (r349238) > @@ -321,10 +321,10 @@ fuse_read_biobackend(struct vnode *vp, struct = uio *uio > /* Try clustered read */ > long totread =3D uio->uio_resid + on; > seqcount =3D MIN(seqcount, > - data->max_readahead / biosize + 1); > + data->max_readahead_blocks + 1); > err =3D cluster_read(vp, filesize, lbn, bcount, = NOCRED, > totread, seqcount, 0, &bp); > - } else if (seqcount > 1 && data->max_readahead >=3D = nextsize) { > + } else if (seqcount > 1 && data->max_readahead_blocks >=3D= 1) { > /* Try non-clustered readahead */ > err =3D breadn(vp, lbn, bcount, &nextlbn, = &nextsize, 1, > NOCRED, &bp); >=20 > Modified: projects/fuse2/sys/fs/fuse/fuse_ipc.c > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- projects/fuse2/sys/fs/fuse/fuse_ipc.c Thu Jun 20 16:36:20 2019 = (r349237) > +++ projects/fuse2/sys/fs/fuse/fuse_ipc.c Thu Jun 20 17:08:21 2019 = (r349238) > @@ -711,6 +711,10 @@ fuse_body_audit(struct fuse_ticket *ftick, size_t = blen > opcode =3D fticket_opcode(ftick); >=20 > switch (opcode) { > + case FUSE_BMAP: > + err =3D (blen =3D=3D sizeof(struct fuse_bmap_out)) ? 0 : = EINVAL; > + break; > + > case FUSE_LINK: > case FUSE_LOOKUP: > case FUSE_MKDIR: >=20 > Modified: projects/fuse2/sys/fs/fuse/fuse_ipc.h > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- projects/fuse2/sys/fs/fuse/fuse_ipc.h Thu Jun 20 16:36:20 2019 = (r349237) > +++ projects/fuse2/sys/fs/fuse/fuse_ipc.h Thu Jun 20 17:08:21 2019 = (r349238) > @@ -197,7 +197,7 @@ struct fuse_data { > uint32_t fuse_libabi_major; > uint32_t fuse_libabi_minor; >=20 > - uint32_t max_readahead; > + uint32_t max_readahead_blocks; > uint32_t max_write; > uint32_t max_read; > uint32_t subtype; >=20 > Modified: projects/fuse2/sys/fs/fuse/fuse_vnops.c > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- projects/fuse2/sys/fs/fuse/fuse_vnops.c Thu Jun 20 16:36:20 2019 = (r349237) > +++ projects/fuse2/sys/fs/fuse/fuse_vnops.c Thu Jun 20 17:08:21 2019 = (r349238) > @@ -120,6 +120,7 @@ SDT_PROBE_DEFINE2(fusefs, , vnops, trace, "int", = "char > /* vnode ops */ > static vop_access_t fuse_vnop_access; > static vop_advlock_t fuse_vnop_advlock; > +static vop_bmap_t fuse_vnop_bmap; > static vop_close_t fuse_fifo_close; > static vop_close_t fuse_vnop_close; > static vop_create_t fuse_vnop_create; > @@ -174,6 +175,7 @@ struct vop_vector fuse_vnops =3D { > .vop_default =3D &default_vnodeops, > .vop_access =3D fuse_vnop_access, > .vop_advlock =3D fuse_vnop_advlock, > + .vop_bmap =3D fuse_vnop_bmap, > .vop_close =3D fuse_vnop_close, > .vop_create =3D fuse_vnop_create, > .vop_deleteextattr =3D fuse_vnop_deleteextattr, > @@ -464,6 +466,92 @@ fuse_vnop_advlock(struct vop_advlock_args *ap) > } >=20 > return err; > +} > + > +/* { > + struct vnode *a_vp; > + daddr_t a_bn; > + struct bufobj **a_bop; > + daddr_t *a_bnp; > + int *a_runp; > + int *a_runb; > +} */ > +static int > +fuse_vnop_bmap(struct vop_bmap_args *ap) > +{ > + struct vnode *vp =3D ap->a_vp; > + struct bufobj **bo =3D ap->a_bop; > + struct thread *td =3D curthread; > + struct mount *mp; > + struct fuse_dispatcher fdi; > + struct fuse_bmap_in *fbi; > + struct fuse_bmap_out *fbo; > + struct fuse_data *data; > + uint64_t biosize; > + off_t filesize; > + daddr_t lbn =3D ap->a_bn; > + daddr_t *pbn =3D ap->a_bnp; > + int *runp =3D ap->a_runp; > + int *runb =3D ap->a_runb; > + int error =3D 0; > + int maxrun; > + > + if (fuse_isdeadfs(vp)) { > + return ENXIO; > + } > + > + mp =3D vnode_mount(vp); > + data =3D fuse_get_mpdata(mp); > + biosize =3D fuse_iosize(vp); > + maxrun =3D MIN(vp->v_mount->mnt_iosize_max / biosize - 1, > + data->max_readahead_blocks); > + > + if (bo !=3D NULL) > + *bo =3D &vp->v_bufobj; > + > + /* > + * The FUSE_BMAP operation does not include the runp and runb > + * variables, so we must guess. Report nonzero contiguous runs = so > + * cluster_read will combine adjacent reads. It's worthwhile to = reduce > + * upcalls even if we don't know the true physical layout of the = file. > + *=20 > + * FUSE file systems may opt out of read clustering in two ways: > + * * mounting with -onoclusterr > + * * Setting max_readahead <=3D maxbcachebuf during FUSE_INIT > + */ > + if (runb !=3D NULL) > + *runb =3D MIN(lbn, maxrun); > + if (runp !=3D NULL) { > + error =3D fuse_vnode_size(vp, &filesize, td->td_ucred, = td); > + if (error =3D=3D 0) > + *runp =3D MIN(MAX(0, filesize / biosize - lbn - = 1), > + maxrun); > + else > + *runp =3D 0; > + } > + > + if (fsess_isimpl(mp, FUSE_BMAP)) { > + fdisp_init(&fdi, sizeof(*fbi)); > + fdisp_make_vp(&fdi, FUSE_BMAP, vp, td, td->td_ucred); > + fbi =3D fdi.indata; > + fbi->block =3D lbn; > + fbi->blocksize =3D biosize; > + error =3D fdisp_wait_answ(&fdi); > + if (error =3D=3D ENOSYS) { > + fsess_set_notimpl(mp, FUSE_BMAP); > + error =3D 0; > + } else { > + fbo =3D fdi.answ; > + if (error =3D=3D 0 && pbn !=3D NULL) > + *pbn =3D fbo->block; > + return error; > + } > + } > + > + /* If the daemon doesn't support BMAP, make up a sensible = default */ > + if (pbn !=3D NULL) > + *pbn =3D lbn * btodb(biosize); > + return (error); > } >=20 > /* >=20 > Modified: projects/fuse2/tests/sys/fs/fusefs/Makefile > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- projects/fuse2/tests/sys/fs/fusefs/Makefile Thu Jun 20 = 16:36:20 2019 (r349237) > +++ projects/fuse2/tests/sys/fs/fusefs/Makefile Thu Jun 20 = 17:08:21 2019 (r349238) > @@ -9,6 +9,7 @@ TESTSDIR=3D ${TESTSBASE}/sys/fs/fusefs > # out, so we get more granular reporting. > GTESTS+=3D access > GTESTS+=3D allow_other > +GTESTS+=3D bmap > GTESTS+=3D create > GTESTS+=3D default_permissions > GTESTS+=3D default_permissions_privileged >=20 > Added: projects/fuse2/tests/sys/fs/fusefs/bmap.cc > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ projects/fuse2/tests/sys/fs/fusefs/bmap.cc Thu Jun 20 = 17:08:21 2019 (r349238) > @@ -0,0 +1,159 @@ > +/*- > + * SPDX-License-Identifier: BSD-2-Clause-FreeBSD > + * > + * Copyright (c) 2019 The FreeBSD Foundation > + * > + * This software was developed by BFF Storage Systems, LLC under = sponsorship > + * from the FreeBSD Foundation. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * 1. Redistributions of source code must retain the above copyright > + * notice, this list of conditions and the following disclaimer. > + * 2. Redistributions in binary form must reproduce the above = copyright > + * notice, this list of conditions and the following disclaimer in = the > + * documentation and/or other materials provided with the = distribution. > + * > + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' = AND > + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, = THE > + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR = PURPOSE > + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE = LIABLE > + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR = CONSEQUENTIAL > + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE = GOODS > + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS = INTERRUPTION) > + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN = CONTRACT, STRICT > + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN = ANY WAY > + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE = POSSIBILITY OF > + * SUCH DAMAGE. > + */ > + > +extern "C" { > +#include > +#include > +#include > + > +#include > +} > + > +#include "mockfs.hh" > +#include "utils.hh" > + > +using namespace testing; > + > +const static char FULLPATH[] =3D "mountpoint/foo"; > +const static char RELPATH[] =3D "foo"; > + > +class Bmap: public FuseTest { > +public: > +virtual void SetUp() { > + m_maxreadahead =3D UINT32_MAX; > + FuseTest::SetUp(); > +} > +void expect_bmap(uint64_t ino, uint64_t lbn, uint32_t blocksize, = uint64_t pbn) > +{ > + EXPECT_CALL(*m_mock, process( > + ResultOf([=3D](auto in) { > + return (in.header.opcode =3D=3D FUSE_BMAP && > + in.header.nodeid =3D=3D ino && > + in.body.bmap.block =3D=3D lbn && > + in.body.bmap.blocksize =3D=3D = blocksize); > + }, Eq(true)), > + _) > + ).WillOnce(Invoke(ReturnImmediate([=3D](auto i __unused, auto& = out) { > + SET_OUT_HEADER_LEN(out, bmap); > + out.body.bmap.block =3D pbn; > + }))); > +} > +=09 > +void expect_lookup(const char *relpath, uint64_t ino, off_t size) > +{ > + FuseTest::expect_lookup(relpath, ino, S_IFREG | 0644, size, 1, > + UINT64_MAX); > +} > +}; > + > +/* > + * Test FUSE_BMAP > + * XXX The FUSE protocol does not include the runp and runb = variables, so those > + * must be guessed in-kernel. > + */ > +TEST_F(Bmap, bmap) > +{ > + struct fiobmap2_arg arg; > + const off_t filesize =3D 1 << 20; > + const ino_t ino =3D 42; > + int64_t lbn =3D 10; > + int64_t pbn =3D 12345; > + int fd; > + > + expect_lookup(RELPATH, 42, filesize); > + expect_open(ino, 0, 1); > + expect_bmap(ino, lbn, m_maxbcachebuf, pbn); > + > + fd =3D open(FULLPATH, O_RDWR); > + ASSERT_LE(0, fd) << strerror(errno); > + > + arg.bn =3D lbn; > + arg.runp =3D -1; > + arg.runb =3D -1; > + ASSERT_EQ(0, ioctl(fd, FIOBMAP2, &arg)) << strerror(errno); > + EXPECT_EQ(arg.bn, pbn); > + EXPECT_EQ(arg.runp, MAXPHYS / m_maxbcachebuf - 1); > + EXPECT_EQ(arg.runb, MAXPHYS / m_maxbcachebuf - 1); > +} > + > +/*=20 > + * If the daemon does not implement VOP_BMAP, fusefs should return = sensible > + * defaults. > + */ > +TEST_F(Bmap, default_) > +{ > + struct fiobmap2_arg arg; > + const off_t filesize =3D 1 << 20; > + const ino_t ino =3D 42; > + int64_t lbn; > + int fd; > + > + expect_lookup(RELPATH, 42, filesize); > + expect_open(ino, 0, 1); > + EXPECT_CALL(*m_mock, process( > + ResultOf([=3D](auto in) { > + return (in.header.opcode =3D=3D FUSE_BMAP); > + }, Eq(true)), > + _) > + ).WillOnce(Invoke(ReturnErrno(ENOSYS))); > + > + fd =3D open(FULLPATH, O_RDWR); > + ASSERT_LE(0, fd) << strerror(errno); > + > + /* First block */ > + lbn =3D 0; > + arg.bn =3D lbn; > + arg.runp =3D -1; > + arg.runb =3D -1; > + ASSERT_EQ(0, ioctl(fd, FIOBMAP2, &arg)) << strerror(errno); > + EXPECT_EQ(arg.bn, 0); > + EXPECT_EQ(arg.runp, MAXPHYS / m_maxbcachebuf - 1); > + EXPECT_EQ(arg.runb, 0); > + > + /* In the middle */ > + lbn =3D filesize / m_maxbcachebuf / 2; > + arg.bn =3D lbn; > + arg.runp =3D -1; > + arg.runb =3D -1; > + ASSERT_EQ(0, ioctl(fd, FIOBMAP2, &arg)) << strerror(errno); > + EXPECT_EQ(arg.bn, lbn * m_maxbcachebuf / DEV_BSIZE); > + EXPECT_EQ(arg.runp, MAXPHYS / m_maxbcachebuf - 1); > + EXPECT_EQ(arg.runb, MAXPHYS / m_maxbcachebuf - 1); > + > + /* Last block */ > + lbn =3D filesize / m_maxbcachebuf - 1; > + arg.bn =3D lbn; > + arg.runp =3D -1; > + arg.runb =3D -1; > + ASSERT_EQ(0, ioctl(fd, FIOBMAP2, &arg)) << strerror(errno); > + EXPECT_EQ(arg.bn, lbn * m_maxbcachebuf / DEV_BSIZE); > + EXPECT_EQ(arg.runp, 0); > + EXPECT_EQ(arg.runb, MAXPHYS / m_maxbcachebuf - 1); > +} >=20 > Modified: projects/fuse2/tests/sys/fs/fusefs/mockfs.cc > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- projects/fuse2/tests/sys/fs/fusefs/mockfs.cc Thu Jun 20 = 16:36:20 2019 (r349237) > +++ projects/fuse2/tests/sys/fs/fusefs/mockfs.cc Thu Jun 20 = 17:08:21 2019 (r349238) > @@ -168,6 +168,10 @@ void MockFS::debug_request(const mockfs_buf_in = &in) > case FUSE_ACCESS: > printf(" mask=3D%#x", in.body.access.mask); > break; > + case FUSE_BMAP: > + printf(" block=3D%#lx blocksize=3D%#x", = in.body.bmap.block, > + in.body.bmap.blocksize); > + break; > case FUSE_CREATE: > if (m_kernel_minor_version >=3D 12) > name =3D (const char*)in.body.bytes + >=20 > Modified: projects/fuse2/tests/sys/fs/fusefs/mockfs.hh > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- projects/fuse2/tests/sys/fs/fusefs/mockfs.hh Thu Jun 20 = 16:36:20 2019 (r349237) > +++ projects/fuse2/tests/sys/fs/fusefs/mockfs.hh Thu Jun 20 = 17:08:21 2019 (r349238) > @@ -124,6 +124,7 @@ struct fuse_create_out_7_8 { >=20 > union fuse_payloads_in { > fuse_access_in access; > + fuse_bmap_in bmap; > /* value is from fuse_kern_chan.c in fusefs-libs */ > uint8_t bytes[0x21000 - sizeof(struct fuse_in_header)]; > fuse_create_in create; > @@ -164,6 +165,7 @@ struct mockfs_buf_in { > union fuse_payloads_out { > fuse_attr_out attr; > fuse_attr_out_7_8 attr_7_8; > + fuse_bmap_out bmap; > fuse_create_out create; > fuse_create_out_7_8 create_7_8; > /* >=20 > Modified: projects/fuse2/tests/sys/fs/fusefs/read.cc > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- projects/fuse2/tests/sys/fs/fusefs/read.cc Thu Jun 20 = 16:36:20 2019 (r349237) > +++ projects/fuse2/tests/sys/fs/fusefs/read.cc Thu Jun 20 = 17:08:21 2019 (r349238) > @@ -29,7 +29,7 @@ > */ >=20 > extern "C" { > -#include > +#include > #include > #include > #include > @@ -749,16 +749,15 @@ TEST_F(ReadCacheable, DISABLED_sendfile_eio) > /* Deliberately leak fd. close(2) will be tested in release.cc = */ > } >=20 > -/* Large reads should be clustered, even across cache block = boundaries */ > -/*=20 > - * Disabled because clustered reads requires VOP_BMAP, which fusefs = does not > - * yet support > +/* > + * Sequential reads should use readahead. And if allowed, large = reads should > + * be clustered. > */ > -TEST_P(ReadAhead, DISABLED_cluster) { > +TEST_P(ReadAhead, readahead) { > const char FULLPATH[] =3D "mountpoint/some_file.txt"; > const char RELPATH[] =3D "some_file.txt"; > uint64_t ino =3D 42; > - int fd, maxcontig; > + int fd, maxcontig, clustersize; > ssize_t bufsize =3D 4 * m_maxbcachebuf; > ssize_t filesize =3D bufsize; > uint64_t len; > @@ -774,8 +773,9 @@ TEST_P(ReadAhead, DISABLED_cluster) { > expect_open(ino, 0, 1); > maxcontig =3D m_noclusterr ? m_maxbcachebuf : > m_maxbcachebuf + = (int)get<1>(GetParam()); > - for (offs =3D 0; offs < bufsize; offs +=3D maxcontig) { > - len =3D std::min((size_t)maxcontig, (size_t)(filesize - = offs)); > + clustersize =3D MIN(maxcontig, MAXPHYS); > + for (offs =3D 0; offs < bufsize; offs +=3D clustersize) { > + len =3D std::min((size_t)clustersize, (size_t)(filesize = - offs)); > expect_read(ino, offs, len, len, contents + offs); > } >=20 > @@ -791,47 +791,11 @@ TEST_P(ReadAhead, DISABLED_cluster) { > /* Deliberately leak fd. close(2) will be tested in release.cc = */ > } >=20 > -/* fuse(4) should honor the filesystem's requested m_readahead = parameter */ > -TEST_P(ReadAhead, readahead) { > - const char FULLPATH[] =3D "mountpoint/some_file.txt"; > - const char RELPATH[] =3D "some_file.txt"; > - uint64_t ino =3D 42; > - int fd, i; > - ssize_t bufsize =3D m_maxbcachebuf; > - ssize_t filesize =3D m_maxbcachebuf * 6; > - char *rbuf, *contents; > - > - contents =3D (char*)malloc(filesize); > - ASSERT_NE(NULL, contents); > - memset(contents, 'X', filesize); > - rbuf =3D (char*)calloc(1, bufsize); > - > - expect_lookup(RELPATH, ino, filesize); > - expect_open(ino, 0, 1); > - /* fuse(4) should only read ahead the allowed amount */ > - expect_read(ino, 0, m_maxbcachebuf, m_maxbcachebuf, contents); > - for (i =3D 0; i < (int)get<1>(GetParam()) / m_maxbcachebuf; i++) = { > - off_t offs =3D (i + 1) * m_maxbcachebuf; > - expect_read(ino, offs, m_maxbcachebuf, m_maxbcachebuf, > - contents + offs); > - } > - > - fd =3D open(FULLPATH, O_RDONLY); > - ASSERT_LE(0, fd) << strerror(errno); > - > - /* Set the internal readahead counter to a "large" value */ > - ASSERT_EQ(0, fcntl(fd, F_READAHEAD, 1'000'000'000)) << = strerror(errno); > - > - ASSERT_EQ(bufsize, read(fd, rbuf, bufsize)) << strerror(errno); > - ASSERT_EQ(0, memcmp(rbuf, contents, bufsize)); > - > - /* Deliberately leak fd. close(2) will be tested in release.cc = */ > -} > - > INSTANTIATE_TEST_CASE_P(RA, ReadAhead, > Values(tuple(false, 0u), > tuple(false, 0x10000), > tuple(false, 0x20000), > tuple(false, 0x30000), > tuple(true, 0u), > - tuple(true, 0x10000))); > + tuple(true, 0x10000), > + tuple(true, 0x20000))); >=20 > Modified: projects/fuse2/tests/sys/fs/fusefs/utils.cc > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- projects/fuse2/tests/sys/fs/fusefs/utils.cc Thu Jun 20 = 16:36:20 2019 (r349237) > +++ projects/fuse2/tests/sys/fs/fusefs/utils.cc Thu Jun 20 = 17:08:21 2019 (r349238) > @@ -129,6 +129,20 @@ void FuseTest::SetUp() { > _) > ).Times(AnyNumber()) > .WillRepeatedly(Invoke(ReturnErrno(ENOSYS))); > + /* > + * FUSE_BMAP is called for most test cases that read = data. Set > + * a default expectation and return ENOSYS. > + * > + * Individual test cases can override this expectation = since > + * googlemock evaluates expectations in LIFO order. > + */ > + EXPECT_CALL(*m_mock, process( > + ResultOf([=3D](auto in) { > + return (in.header.opcode =3D=3D = FUSE_BMAP); > + }, Eq(true)), > + _) > + ).Times(AnyNumber()) > + .WillRepeatedly(Invoke(ReturnErrno(ENOSYS))); > } catch (std::system_error err) { > FAIL() << err.what(); > } >=20 From owner-svn-src-projects@freebsd.org Thu Jun 20 18:29:47 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C248115C4330 for ; Thu, 20 Jun 2019 18:29:46 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lj1-f175.google.com (mail-lj1-f175.google.com [209.85.208.175]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3C8EC90861; Thu, 20 Jun 2019 18:29:46 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lj1-f175.google.com with SMTP id x25so3643409ljh.2; Thu, 20 Jun 2019 11:29:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Fzqec5kKC8Py/MhV5c+/Xoag+mXQSPsr/s5fp4yQ9IM=; b=remFzR30c9b4ywlWcP4vM3O4td+rjFBy0CV43rRpCnbmNN9omGPP9UACQObA7kMTBE Ex50P8+UfGCZsCFVT61RzA504i64xDjOXNA+R4nF/UjJfmmUYbjnc/wcZe84WeoCfTha QTixlr2HucLAufyYFscjzJh4zE+Mblofb3TFOmnppAn6mrLJXlAvcC09Lo5UYyId2FWX 8zGqzctsRlfIDmLqKy1AfvZz2smZDTVOz9MK96d7fYkCiMlmbnwMCzHM4eOdb7LeNIvf 9K/rHxSFDLsALfSOXtRZzc6j67FwExnQag3qohlBMmvAAC85XioZqDhVEdHvmh5WwBcN Dcng== X-Gm-Message-State: APjAAAWMbDCGPAMW9tPWvI2RCGW2A9Rv2OYUL2J+3nAGV//ZI462ChbO oNbdKFynit/I9UNmShObYYDMfaotg29tVOf1vLw= X-Google-Smtp-Source: APXvYqxch6BH/rcYX5EKFZ1BD2KAflIibTWXbX3FANHa3+KSzvMxCUnIyaHYhiO6/XgZfQdd4ZmKOqMsIWOe5GVzREY= X-Received: by 2002:a2e:2c07:: with SMTP id s7mr30568533ljs.44.1561054887378; Thu, 20 Jun 2019 11:21:27 -0700 (PDT) MIME-Version: 1.0 References: <201906201708.x5KH8M6e002674@repo.freebsd.org> <5E43BBD0-3308-4CDC-9E26-28AE496E8281@samsco.org> In-Reply-To: <5E43BBD0-3308-4CDC-9E26-28AE496E8281@samsco.org> From: Alan Somers Date: Thu, 20 Jun 2019 12:21:14 -0600 Message-ID: Subject: Re: svn commit: r349238 - in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs To: Scott Long Cc: src-committers , svn-src-projects@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 3C8EC90861 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.96 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_SHORT(-0.96)[-0.960,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2019 18:29:47 -0000 Well, I haven't tried. But if maxbcachebuf remains 64KB and MAXPHYS is 1MB, then reading a file sequentially should start by issuing a 128KB read and progress to 1MB reads if the I/O remains sequential. One outstanding problem is that fusefs will greedily try to cluster reads whenever possible, to minimize upcalls. But if the file system is stored on rotating media, like for instance when using fusefs-ntfs on a single HDD, then seeks will probably be slower than upcalls. In such a case the greedy clustering could hurt performance. But I don't see a good alternative. The best would be for FUSE_BMAP to include runp and runb fields, but that would require a change to the FUSE protocol. -Alan On Thu, Jun 20, 2019 at 12:12 PM Scott Long wrote: > > Great work! What happens if you set MAXPHYS to something big like 1MB? > > Scott > > > > On Jun 20, 2019, at 11:08 AM, Alan Somers wrote: > > > > Author: asomers > > Date: Thu Jun 20 17:08:21 2019 > > New Revision: 349238 > > URL: https://svnweb.freebsd.org/changeset/base/349238 > > > > Log: > > fusefs: implement VOP_BMAP > > > > If the fuse daemon supports FUSE_BMAP, then use that for the block mapping. > > Otherwise, use the same technique used by vop_stdbmap. Report large values > > for runp and runb in order to maximize read clustering and minimize upcalls, > > even if we don't know the true layout. > > > > The major result of this change is that sequential reads to FUSE files will > > now usually happen 128KB at a time instead of 64KB. > > > > Sponsored by: The FreeBSD Foundation > > > > Added: > > projects/fuse2/tests/sys/fs/fusefs/bmap.cc (contents, props changed) > > Modified: > > projects/fuse2/sys/fs/fuse/fuse_internal.c > > projects/fuse2/sys/fs/fuse/fuse_io.c > > projects/fuse2/sys/fs/fuse/fuse_ipc.c > > projects/fuse2/sys/fs/fuse/fuse_ipc.h > > projects/fuse2/sys/fs/fuse/fuse_vnops.c > > projects/fuse2/tests/sys/fs/fusefs/Makefile > > projects/fuse2/tests/sys/fs/fusefs/mockfs.cc > > projects/fuse2/tests/sys/fs/fusefs/mockfs.hh > > projects/fuse2/tests/sys/fs/fusefs/read.cc > > projects/fuse2/tests/sys/fs/fusefs/utils.cc > > > > Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c > > ============================================================================== > > --- projects/fuse2/sys/fs/fuse/fuse_internal.c Thu Jun 20 16:36:20 2019 (r349237) > > +++ projects/fuse2/sys/fs/fuse/fuse_internal.c Thu Jun 20 17:08:21 2019 (r349238) > > @@ -908,7 +908,8 @@ fuse_internal_init_callback(struct fuse_ticket *tick, > > > > if (fuse_libabi_geq(data, 7, 5)) { > > if (fticket_resp(tick)->len == sizeof(struct fuse_init_out)) { > > - data->max_readahead = fiio->max_readahead; > > + data->max_readahead_blocks = fiio->max_readahead / > > + maxbcachebuf; > > data->max_write = fiio->max_write; > > if (fiio->flags & FUSE_ASYNC_READ) > > data->dataflags |= FSESS_ASYNC_READ; > > > > Modified: projects/fuse2/sys/fs/fuse/fuse_io.c > > ============================================================================== > > --- projects/fuse2/sys/fs/fuse/fuse_io.c Thu Jun 20 16:36:20 2019 (r349237) > > +++ projects/fuse2/sys/fs/fuse/fuse_io.c Thu Jun 20 17:08:21 2019 (r349238) > > @@ -321,10 +321,10 @@ fuse_read_biobackend(struct vnode *vp, struct uio *uio > > /* Try clustered read */ > > long totread = uio->uio_resid + on; > > seqcount = MIN(seqcount, > > - data->max_readahead / biosize + 1); > > + data->max_readahead_blocks + 1); > > err = cluster_read(vp, filesize, lbn, bcount, NOCRED, > > totread, seqcount, 0, &bp); > > - } else if (seqcount > 1 && data->max_readahead >= nextsize) { > > + } else if (seqcount > 1 && data->max_readahead_blocks >= 1) { > > /* Try non-clustered readahead */ > > err = breadn(vp, lbn, bcount, &nextlbn, &nextsize, 1, > > NOCRED, &bp); > > > > Modified: projects/fuse2/sys/fs/fuse/fuse_ipc.c > > ============================================================================== > > --- projects/fuse2/sys/fs/fuse/fuse_ipc.c Thu Jun 20 16:36:20 2019 (r349237) > > +++ projects/fuse2/sys/fs/fuse/fuse_ipc.c Thu Jun 20 17:08:21 2019 (r349238) > > @@ -711,6 +711,10 @@ fuse_body_audit(struct fuse_ticket *ftick, size_t blen > > opcode = fticket_opcode(ftick); > > > > switch (opcode) { > > + case FUSE_BMAP: > > + err = (blen == sizeof(struct fuse_bmap_out)) ? 0 : EINVAL; > > + break; > > + > > case FUSE_LINK: > > case FUSE_LOOKUP: > > case FUSE_MKDIR: > > > > Modified: projects/fuse2/sys/fs/fuse/fuse_ipc.h > > ============================================================================== > > --- projects/fuse2/sys/fs/fuse/fuse_ipc.h Thu Jun 20 16:36:20 2019 (r349237) > > +++ projects/fuse2/sys/fs/fuse/fuse_ipc.h Thu Jun 20 17:08:21 2019 (r349238) > > @@ -197,7 +197,7 @@ struct fuse_data { > > uint32_t fuse_libabi_major; > > uint32_t fuse_libabi_minor; > > > > - uint32_t max_readahead; > > + uint32_t max_readahead_blocks; > > uint32_t max_write; > > uint32_t max_read; > > uint32_t subtype; > > > > Modified: projects/fuse2/sys/fs/fuse/fuse_vnops.c > > ============================================================================== > > --- projects/fuse2/sys/fs/fuse/fuse_vnops.c Thu Jun 20 16:36:20 2019 (r349237) > > +++ projects/fuse2/sys/fs/fuse/fuse_vnops.c Thu Jun 20 17:08:21 2019 (r349238) > > @@ -120,6 +120,7 @@ SDT_PROBE_DEFINE2(fusefs, , vnops, trace, "int", "char > > /* vnode ops */ > > static vop_access_t fuse_vnop_access; > > static vop_advlock_t fuse_vnop_advlock; > > +static vop_bmap_t fuse_vnop_bmap; > > static vop_close_t fuse_fifo_close; > > static vop_close_t fuse_vnop_close; > > static vop_create_t fuse_vnop_create; > > @@ -174,6 +175,7 @@ struct vop_vector fuse_vnops = { > > .vop_default = &default_vnodeops, > > .vop_access = fuse_vnop_access, > > .vop_advlock = fuse_vnop_advlock, > > + .vop_bmap = fuse_vnop_bmap, > > .vop_close = fuse_vnop_close, > > .vop_create = fuse_vnop_create, > > .vop_deleteextattr = fuse_vnop_deleteextattr, > > @@ -464,6 +466,92 @@ fuse_vnop_advlock(struct vop_advlock_args *ap) > > } > > > > return err; > > +} > > + > > +/* { > > + struct vnode *a_vp; > > + daddr_t a_bn; > > + struct bufobj **a_bop; > > + daddr_t *a_bnp; > > + int *a_runp; > > + int *a_runb; > > +} */ > > +static int > > +fuse_vnop_bmap(struct vop_bmap_args *ap) > > +{ > > + struct vnode *vp = ap->a_vp; > > + struct bufobj **bo = ap->a_bop; > > + struct thread *td = curthread; > > + struct mount *mp; > > + struct fuse_dispatcher fdi; > > + struct fuse_bmap_in *fbi; > > + struct fuse_bmap_out *fbo; > > + struct fuse_data *data; > > + uint64_t biosize; > > + off_t filesize; > > + daddr_t lbn = ap->a_bn; > > + daddr_t *pbn = ap->a_bnp; > > + int *runp = ap->a_runp; > > + int *runb = ap->a_runb; > > + int error = 0; > > + int maxrun; > > + > > + if (fuse_isdeadfs(vp)) { > > + return ENXIO; > > + } > > + > > + mp = vnode_mount(vp); > > + data = fuse_get_mpdata(mp); > > + biosize = fuse_iosize(vp); > > + maxrun = MIN(vp->v_mount->mnt_iosize_max / biosize - 1, > > + data->max_readahead_blocks); > > + > > + if (bo != NULL) > > + *bo = &vp->v_bufobj; > > + > > + /* > > + * The FUSE_BMAP operation does not include the runp and runb > > + * variables, so we must guess. Report nonzero contiguous runs so > > + * cluster_read will combine adjacent reads. It's worthwhile to reduce > > + * upcalls even if we don't know the true physical layout of the file. > > + * > > + * FUSE file systems may opt out of read clustering in two ways: > > + * * mounting with -onoclusterr > > + * * Setting max_readahead <= maxbcachebuf during FUSE_INIT > > + */ > > + if (runb != NULL) > > + *runb = MIN(lbn, maxrun); > > + if (runp != NULL) { > > + error = fuse_vnode_size(vp, &filesize, td->td_ucred, td); > > + if (error == 0) > > + *runp = MIN(MAX(0, filesize / biosize - lbn - 1), > > + maxrun); > > + else > > + *runp = 0; > > + } > > + > > + if (fsess_isimpl(mp, FUSE_BMAP)) { > > + fdisp_init(&fdi, sizeof(*fbi)); > > + fdisp_make_vp(&fdi, FUSE_BMAP, vp, td, td->td_ucred); > > + fbi = fdi.indata; > > + fbi->block = lbn; > > + fbi->blocksize = biosize; > > + error = fdisp_wait_answ(&fdi); > > + if (error == ENOSYS) { > > + fsess_set_notimpl(mp, FUSE_BMAP); > > + error = 0; > > + } else { > > + fbo = fdi.answ; > > + if (error == 0 && pbn != NULL) > > + *pbn = fbo->block; > > + return error; > > + } > > + } > > + > > + /* If the daemon doesn't support BMAP, make up a sensible default */ > > + if (pbn != NULL) > > + *pbn = lbn * btodb(biosize); > > + return (error); > > } > > > > /* > > > > Modified: projects/fuse2/tests/sys/fs/fusefs/Makefile > > ============================================================================== > > --- projects/fuse2/tests/sys/fs/fusefs/Makefile Thu Jun 20 16:36:20 2019 (r349237) > > +++ projects/fuse2/tests/sys/fs/fusefs/Makefile Thu Jun 20 17:08:21 2019 (r349238) > > @@ -9,6 +9,7 @@ TESTSDIR= ${TESTSBASE}/sys/fs/fusefs > > # out, so we get more granular reporting. > > GTESTS+= access > > GTESTS+= allow_other > > +GTESTS+= bmap > > GTESTS+= create > > GTESTS+= default_permissions > > GTESTS+= default_permissions_privileged > > > > Added: projects/fuse2/tests/sys/fs/fusefs/bmap.cc > > ============================================================================== > > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > > +++ projects/fuse2/tests/sys/fs/fusefs/bmap.cc Thu Jun 20 17:08:21 2019 (r349238) > > @@ -0,0 +1,159 @@ > > +/*- > > + * SPDX-License-Identifier: BSD-2-Clause-FreeBSD > > + * > > + * Copyright (c) 2019 The FreeBSD Foundation > > + * > > + * This software was developed by BFF Storage Systems, LLC under sponsorship > > + * from the FreeBSD Foundation. > > + * > > + * Redistribution and use in source and binary forms, with or without > > + * modification, are permitted provided that the following conditions > > + * are met: > > + * 1. Redistributions of source code must retain the above copyright > > + * notice, this list of conditions and the following disclaimer. > > + * 2. Redistributions in binary form must reproduce the above copyright > > + * notice, this list of conditions and the following disclaimer in the > > + * documentation and/or other materials provided with the distribution. > > + * > > + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND > > + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > > + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE > > + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE > > + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL > > + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > > + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > > + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT > > + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY > > + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > > + * SUCH DAMAGE. > > + */ > > + > > +extern "C" { > > +#include > > +#include > > +#include > > + > > +#include > > +} > > + > > +#include "mockfs.hh" > > +#include "utils.hh" > > + > > +using namespace testing; > > + > > +const static char FULLPATH[] = "mountpoint/foo"; > > +const static char RELPATH[] = "foo"; > > + > > +class Bmap: public FuseTest { > > +public: > > +virtual void SetUp() { > > + m_maxreadahead = UINT32_MAX; > > + FuseTest::SetUp(); > > +} > > +void expect_bmap(uint64_t ino, uint64_t lbn, uint32_t blocksize, uint64_t pbn) > > +{ > > + EXPECT_CALL(*m_mock, process( > > + ResultOf([=](auto in) { > > + return (in.header.opcode == FUSE_BMAP && > > + in.header.nodeid == ino && > > + in.body.bmap.block == lbn && > > + in.body.bmap.blocksize == blocksize); > > + }, Eq(true)), > > + _) > > + ).WillOnce(Invoke(ReturnImmediate([=](auto i __unused, auto& out) { > > + SET_OUT_HEADER_LEN(out, bmap); > > + out.body.bmap.block = pbn; > > + }))); > > +} > > + > > +void expect_lookup(const char *relpath, uint64_t ino, off_t size) > > +{ > > + FuseTest::expect_lookup(relpath, ino, S_IFREG | 0644, size, 1, > > + UINT64_MAX); > > +} > > +}; > > + > > +/* > > + * Test FUSE_BMAP > > + * XXX The FUSE protocol does not include the runp and runb variables, so those > > + * must be guessed in-kernel. > > + */ > > +TEST_F(Bmap, bmap) > > +{ > > + struct fiobmap2_arg arg; > > + const off_t filesize = 1 << 20; > > + const ino_t ino = 42; > > + int64_t lbn = 10; > > + int64_t pbn = 12345; > > + int fd; > > + > > + expect_lookup(RELPATH, 42, filesize); > > + expect_open(ino, 0, 1); > > + expect_bmap(ino, lbn, m_maxbcachebuf, pbn); > > + > > + fd = open(FULLPATH, O_RDWR); > > + ASSERT_LE(0, fd) << strerror(errno); > > + > > + arg.bn = lbn; > > + arg.runp = -1; > > + arg.runb = -1; > > + ASSERT_EQ(0, ioctl(fd, FIOBMAP2, &arg)) << strerror(errno); > > + EXPECT_EQ(arg.bn, pbn); > > + EXPECT_EQ(arg.runp, MAXPHYS / m_maxbcachebuf - 1); > > + EXPECT_EQ(arg.runb, MAXPHYS / m_maxbcachebuf - 1); > > +} > > + > > +/* > > + * If the daemon does not implement VOP_BMAP, fusefs should return sensible > > + * defaults. > > + */ > > +TEST_F(Bmap, default_) > > +{ > > + struct fiobmap2_arg arg; > > + const off_t filesize = 1 << 20; > > + const ino_t ino = 42; > > + int64_t lbn; > > + int fd; > > + > > + expect_lookup(RELPATH, 42, filesize); > > + expect_open(ino, 0, 1); > > + EXPECT_CALL(*m_mock, process( > > + ResultOf([=](auto in) { > > + return (in.header.opcode == FUSE_BMAP); > > + }, Eq(true)), > > + _) > > + ).WillOnce(Invoke(ReturnErrno(ENOSYS))); > > + > > + fd = open(FULLPATH, O_RDWR); > > + ASSERT_LE(0, fd) << strerror(errno); > > + > > + /* First block */ > > + lbn = 0; > > + arg.bn = lbn; > > + arg.runp = -1; > > + arg.runb = -1; > > + ASSERT_EQ(0, ioctl(fd, FIOBMAP2, &arg)) << strerror(errno); > > + EXPECT_EQ(arg.bn, 0); > > + EXPECT_EQ(arg.runp, MAXPHYS / m_maxbcachebuf - 1); > > + EXPECT_EQ(arg.runb, 0); > > + > > + /* In the middle */ > > + lbn = filesize / m_maxbcachebuf / 2; > > + arg.bn = lbn; > > + arg.runp = -1; > > + arg.runb = -1; > > + ASSERT_EQ(0, ioctl(fd, FIOBMAP2, &arg)) << strerror(errno); > > + EXPECT_EQ(arg.bn, lbn * m_maxbcachebuf / DEV_BSIZE); > > + EXPECT_EQ(arg.runp, MAXPHYS / m_maxbcachebuf - 1); > > + EXPECT_EQ(arg.runb, MAXPHYS / m_maxbcachebuf - 1); > > + > > + /* Last block */ > > + lbn = filesize / m_maxbcachebuf - 1; > > + arg.bn = lbn; > > + arg.runp = -1; > > + arg.runb = -1; > > + ASSERT_EQ(0, ioctl(fd, FIOBMAP2, &arg)) << strerror(errno); > > + EXPECT_EQ(arg.bn, lbn * m_maxbcachebuf / DEV_BSIZE); > > + EXPECT_EQ(arg.runp, 0); > > + EXPECT_EQ(arg.runb, MAXPHYS / m_maxbcachebuf - 1); > > +} > > > > Modified: projects/fuse2/tests/sys/fs/fusefs/mockfs.cc > > ============================================================================== > > --- projects/fuse2/tests/sys/fs/fusefs/mockfs.cc Thu Jun 20 16:36:20 2019 (r349237) > > +++ projects/fuse2/tests/sys/fs/fusefs/mockfs.cc Thu Jun 20 17:08:21 2019 (r349238) > > @@ -168,6 +168,10 @@ void MockFS::debug_request(const mockfs_buf_in &in) > > case FUSE_ACCESS: > > printf(" mask=%#x", in.body.access.mask); > > break; > > + case FUSE_BMAP: > > + printf(" block=%#lx blocksize=%#x", in.body.bmap.block, > > + in.body.bmap.blocksize); > > + break; > > case FUSE_CREATE: > > if (m_kernel_minor_version >= 12) > > name = (const char*)in.body.bytes + > > > > Modified: projects/fuse2/tests/sys/fs/fusefs/mockfs.hh > > ============================================================================== > > --- projects/fuse2/tests/sys/fs/fusefs/mockfs.hh Thu Jun 20 16:36:20 2019 (r349237) > > +++ projects/fuse2/tests/sys/fs/fusefs/mockfs.hh Thu Jun 20 17:08:21 2019 (r349238) > > @@ -124,6 +124,7 @@ struct fuse_create_out_7_8 { > > > > union fuse_payloads_in { > > fuse_access_in access; > > + fuse_bmap_in bmap; > > /* value is from fuse_kern_chan.c in fusefs-libs */ > > uint8_t bytes[0x21000 - sizeof(struct fuse_in_header)]; > > fuse_create_in create; > > @@ -164,6 +165,7 @@ struct mockfs_buf_in { > > union fuse_payloads_out { > > fuse_attr_out attr; > > fuse_attr_out_7_8 attr_7_8; > > + fuse_bmap_out bmap; > > fuse_create_out create; > > fuse_create_out_7_8 create_7_8; > > /* > > > > Modified: projects/fuse2/tests/sys/fs/fusefs/read.cc > > ============================================================================== > > --- projects/fuse2/tests/sys/fs/fusefs/read.cc Thu Jun 20 16:36:20 2019 (r349237) > > +++ projects/fuse2/tests/sys/fs/fusefs/read.cc Thu Jun 20 17:08:21 2019 (r349238) > > @@ -29,7 +29,7 @@ > > */ > > > > extern "C" { > > -#include > > +#include > > #include > > #include > > #include > > @@ -749,16 +749,15 @@ TEST_F(ReadCacheable, DISABLED_sendfile_eio) > > /* Deliberately leak fd. close(2) will be tested in release.cc */ > > } > > > > -/* Large reads should be clustered, even across cache block boundaries */ > > -/* > > - * Disabled because clustered reads requires VOP_BMAP, which fusefs does not > > - * yet support > > +/* > > + * Sequential reads should use readahead. And if allowed, large reads should > > + * be clustered. > > */ > > -TEST_P(ReadAhead, DISABLED_cluster) { > > +TEST_P(ReadAhead, readahead) { > > const char FULLPATH[] = "mountpoint/some_file.txt"; > > const char RELPATH[] = "some_file.txt"; > > uint64_t ino = 42; > > - int fd, maxcontig; > > + int fd, maxcontig, clustersize; > > ssize_t bufsize = 4 * m_maxbcachebuf; > > ssize_t filesize = bufsize; > > uint64_t len; > > @@ -774,8 +773,9 @@ TEST_P(ReadAhead, DISABLED_cluster) { > > expect_open(ino, 0, 1); > > maxcontig = m_noclusterr ? m_maxbcachebuf : > > m_maxbcachebuf + (int)get<1>(GetParam()); > > - for (offs = 0; offs < bufsize; offs += maxcontig) { > > - len = std::min((size_t)maxcontig, (size_t)(filesize - offs)); > > + clustersize = MIN(maxcontig, MAXPHYS); > > + for (offs = 0; offs < bufsize; offs += clustersize) { > > + len = std::min((size_t)clustersize, (size_t)(filesize - offs)); > > expect_read(ino, offs, len, len, contents + offs); > > } > > > > @@ -791,47 +791,11 @@ TEST_P(ReadAhead, DISABLED_cluster) { > > /* Deliberately leak fd. close(2) will be tested in release.cc */ > > } > > > > -/* fuse(4) should honor the filesystem's requested m_readahead parameter */ > > -TEST_P(ReadAhead, readahead) { > > - const char FULLPATH[] = "mountpoint/some_file.txt"; > > - const char RELPATH[] = "some_file.txt"; > > - uint64_t ino = 42; > > - int fd, i; > > - ssize_t bufsize = m_maxbcachebuf; > > - ssize_t filesize = m_maxbcachebuf * 6; > > - char *rbuf, *contents; > > - > > - contents = (char*)malloc(filesize); > > - ASSERT_NE(NULL, contents); > > - memset(contents, 'X', filesize); > > - rbuf = (char*)calloc(1, bufsize); > > - > > - expect_lookup(RELPATH, ino, filesize); > > - expect_open(ino, 0, 1); > > - /* fuse(4) should only read ahead the allowed amount */ > > - expect_read(ino, 0, m_maxbcachebuf, m_maxbcachebuf, contents); > > - for (i = 0; i < (int)get<1>(GetParam()) / m_maxbcachebuf; i++) { > > - off_t offs = (i + 1) * m_maxbcachebuf; > > - expect_read(ino, offs, m_maxbcachebuf, m_maxbcachebuf, > > - contents + offs); > > - } > > - > > - fd = open(FULLPATH, O_RDONLY); > > - ASSERT_LE(0, fd) << strerror(errno); > > - > > - /* Set the internal readahead counter to a "large" value */ > > - ASSERT_EQ(0, fcntl(fd, F_READAHEAD, 1'000'000'000)) << strerror(errno); > > - > > - ASSERT_EQ(bufsize, read(fd, rbuf, bufsize)) << strerror(errno); > > - ASSERT_EQ(0, memcmp(rbuf, contents, bufsize)); > > - > > - /* Deliberately leak fd. close(2) will be tested in release.cc */ > > -} > > - > > INSTANTIATE_TEST_CASE_P(RA, ReadAhead, > > Values(tuple(false, 0u), > > tuple(false, 0x10000), > > tuple(false, 0x20000), > > tuple(false, 0x30000), > > tuple(true, 0u), > > - tuple(true, 0x10000))); > > + tuple(true, 0x10000), > > + tuple(true, 0x20000))); > > > > Modified: projects/fuse2/tests/sys/fs/fusefs/utils.cc > > ============================================================================== > > --- projects/fuse2/tests/sys/fs/fusefs/utils.cc Thu Jun 20 16:36:20 2019 (r349237) > > +++ projects/fuse2/tests/sys/fs/fusefs/utils.cc Thu Jun 20 17:08:21 2019 (r349238) > > @@ -129,6 +129,20 @@ void FuseTest::SetUp() { > > _) > > ).Times(AnyNumber()) > > .WillRepeatedly(Invoke(ReturnErrno(ENOSYS))); > > + /* > > + * FUSE_BMAP is called for most test cases that read data. Set > > + * a default expectation and return ENOSYS. > > + * > > + * Individual test cases can override this expectation since > > + * googlemock evaluates expectations in LIFO order. > > + */ > > + EXPECT_CALL(*m_mock, process( > > + ResultOf([=](auto in) { > > + return (in.header.opcode == FUSE_BMAP); > > + }, Eq(true)), > > + _) > > + ).Times(AnyNumber()) > > + .WillRepeatedly(Invoke(ReturnErrno(ENOSYS))); > > } catch (std::system_error err) { > > FAIL() << err.what(); > > } > > > From owner-svn-src-projects@freebsd.org Thu Jun 20 21:29:29 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7924815C830F for ; Thu, 20 Jun 2019 21:29:29 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 112D397253; Thu, 20 Jun 2019 21:29:29 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DBBE719BA7; Thu, 20 Jun 2019 21:29:28 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5KLTS1a048950; Thu, 20 Jun 2019 21:29:28 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5KLTS81048948; Thu, 20 Jun 2019 21:29:28 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906202129.x5KLTS81048948@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Thu, 20 Jun 2019 21:29:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349244 - projects/fuse2/sys/fs/fuse X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/sys/fs/fuse X-SVN-Commit-Revision: 349244 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 112D397253 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.96)[-0.961,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2019 21:29:29 -0000 Author: asomers Date: Thu Jun 20 21:29:28 2019 New Revision: 349244 URL: https://svnweb.freebsd.org/changeset/base/349244 Log: fusefs: raise protocol level to 7.13 This protocol version adds one new feature: the ability for the server to set the maximum number of background requests and a "congestion threshold" with ill-defined properties. I don't know of any fuse file systems in ports that use this feature, so I'm not implementing it. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c projects/fuse2/sys/fs/fuse/fuse_kernel.h Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_internal.c Thu Jun 20 20:29:42 2019 (r349243) +++ projects/fuse2/sys/fs/fuse/fuse_internal.c Thu Jun 20 21:29:28 2019 (r349244) @@ -921,6 +921,10 @@ fuse_internal_init_callback(struct fuse_ticket *tick, * Don't bother to check FUSE_BIG_WRITES, because it's * redundant with max_write */ + /* + * max_background and congestion_threshold are not + * implemented + */ } else { err = EINVAL; } Modified: projects/fuse2/sys/fs/fuse/fuse_kernel.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_kernel.h Thu Jun 20 20:29:42 2019 (r349243) +++ projects/fuse2/sys/fs/fuse/fuse_kernel.h Thu Jun 20 21:29:28 2019 (r349244) @@ -57,6 +57,10 @@ * - add umask flag to input argument of open, mknod and mkdir * - add notification messages for invalidation of inodes and * directory entries + * + * 7.13 + * - make max number of background requests and congestion threshold + * tunables */ #ifndef _FUSE_FUSE_KERNEL_H @@ -68,6 +72,7 @@ #define __s64 int64_t #define __u32 uint32_t #define __s32 int32_t +#define __u16 uint16_t #else #include #endif @@ -76,7 +81,7 @@ #define FUSE_KERNEL_VERSION 7 /** Minor version number of this interface */ -#define FUSE_KERNEL_MINOR_VERSION 12 +#define FUSE_KERNEL_MINOR_VERSION 13 /** The node ID of the root inode */ #define FUSE_ROOT_ID 1 @@ -476,7 +481,8 @@ struct fuse_init_out { __u32 minor; __u32 max_readahead; __u32 flags; - __u32 unused; + __u16 max_background; + __u16 congestion_threshold; __u32 max_write; }; From owner-svn-src-projects@freebsd.org Thu Jun 20 22:21:44 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1175115C9461 for ; Thu, 20 Jun 2019 22:21:44 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A9DE26A202; Thu, 20 Jun 2019 22:21:43 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6D6171A563; Thu, 20 Jun 2019 22:21:43 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5KMLhwC079724; Thu, 20 Jun 2019 22:21:43 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5KMLhm4079722; Thu, 20 Jun 2019 22:21:43 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906202221.x5KMLhm4079722@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Thu, 20 Jun 2019 22:21:43 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349247 - projects/fuse2/sys/fs/fuse X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/sys/fs/fuse X-SVN-Commit-Revision: 349247 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: A9DE26A202 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_SHORT(-0.97)[-0.971,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2019 22:21:44 -0000 Author: asomers Date: Thu Jun 20 22:21:42 2019 New Revision: 349247 URL: https://svnweb.freebsd.org/changeset/base/349247 Log: fusefs: attempt to support servers as old as protocol 7.4 Previously we allowed servers as old as 7.1 to connect (there never was a 7.0). However, we wrongly assumed a few things about protocols older than 7.8. This commit attempts to support servers as old as 7.4 but no older. I added no new tests because I'm not sure there actually _are_ any servers this old in the wild. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c projects/fuse2/sys/fs/fuse/fuse_vfsops.c Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_internal.c Thu Jun 20 22:20:30 2019 (r349246) +++ projects/fuse2/sys/fs/fuse/fuse_internal.c Thu Jun 20 22:21:42 2019 (r349247) @@ -896,20 +896,21 @@ fuse_internal_init_callback(struct fuse_ticket *tick, } fiio = fticket_resp(tick)->base; - /* XXX: Do we want to check anything further besides this? */ - if (fiio->major < 7) { + data->fuse_libabi_major = fiio->major; + data->fuse_libabi_minor = fiio->minor; + if (!fuse_libabi_geq(data, 7, 4)) { + /* + * With a little work we could support servers as old as 7.1. + * But there would be little payoff. + */ SDT_PROBE2(fusefs, , internal, trace, 1, "userpace version too low"); err = EPROTONOSUPPORT; goto out; } - data->fuse_libabi_major = fiio->major; - data->fuse_libabi_minor = fiio->minor; if (fuse_libabi_geq(data, 7, 5)) { if (fticket_resp(tick)->len == sizeof(struct fuse_init_out)) { - data->max_readahead_blocks = fiio->max_readahead / - maxbcachebuf; data->max_write = fiio->max_write; if (fiio->flags & FUSE_ASYNC_READ) data->dataflags |= FSESS_ASYNC_READ; @@ -929,8 +930,19 @@ fuse_internal_init_callback(struct fuse_ticket *tick, err = EINVAL; } } else { - /* Old fix values */ + /* Old fixed values */ data->max_write = 4096; + } + + if (fuse_libabi_geq(data, 7, 6)) + data->max_readahead_blocks = fiio->max_readahead / maxbcachebuf; + + if (!fuse_libabi_geq(data, 7, 7)) + fsess_set_notimpl(data->mp, FUSE_INTERRUPT); + + if (!fuse_libabi_geq(data, 7, 8)) { + fsess_set_notimpl(data->mp, FUSE_BMAP); + fsess_set_notimpl(data->mp, FUSE_DESTROY); } out: Modified: projects/fuse2/sys/fs/fuse/fuse_vfsops.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_vfsops.c Thu Jun 20 22:20:30 2019 (r349246) +++ projects/fuse2/sys/fs/fuse/fuse_vfsops.c Thu Jun 20 22:21:42 2019 (r349247) @@ -493,11 +493,13 @@ fuse_vfsop_unmount(struct mount *mp, int mntflags) if (fdata_get_dead(data)) { goto alreadydead; } - fdisp_init(&fdi, 0); - fdisp_make(&fdi, FUSE_DESTROY, mp, 0, td, NULL); + if (fsess_isimpl(mp, FUSE_DESTROY)) { + fdisp_init(&fdi, 0); + fdisp_make(&fdi, FUSE_DESTROY, mp, 0, td, NULL); - err = fdisp_wait_answ(&fdi); - fdisp_destroy(&fdi); + (void)fdisp_wait_answ(&fdi); + fdisp_destroy(&fdi); + } fdata_set_dead(data); From owner-svn-src-projects@freebsd.org Thu Jun 20 23:12:20 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A12FA15CA447 for ; Thu, 20 Jun 2019 23:12:20 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 433336BC50; Thu, 20 Jun 2019 23:12:20 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 20A0F1ADE9; Thu, 20 Jun 2019 23:12:20 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5KNCKVW006178; Thu, 20 Jun 2019 23:12:20 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5KNCKEL006177; Thu, 20 Jun 2019 23:12:20 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906202312.x5KNCKEL006177@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Thu, 20 Jun 2019 23:12:20 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349249 - projects/fuse2/sys/fs/fuse X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/sys/fs/fuse X-SVN-Commit-Revision: 349249 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 433336BC50 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_SHORT(-0.96)[-0.961,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2019 23:12:20 -0000 Author: asomers Date: Thu Jun 20 23:12:19 2019 New Revision: 349249 URL: https://svnweb.freebsd.org/changeset/base/349249 Log: fusefs: trivially raise protocol level to 7.14 The only new feature is splice(2) support on /dev/fuse, which FreeBSD can't support. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_kernel.h Modified: projects/fuse2/sys/fs/fuse/fuse_kernel.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_kernel.h Thu Jun 20 23:07:20 2019 (r349248) +++ projects/fuse2/sys/fs/fuse/fuse_kernel.h Thu Jun 20 23:12:19 2019 (r349249) @@ -61,6 +61,9 @@ * 7.13 * - make max number of background requests and congestion threshold * tunables + * + * 7.14 + * - add splice support to fuse device */ #ifndef _FUSE_FUSE_KERNEL_H @@ -81,7 +84,7 @@ #define FUSE_KERNEL_VERSION 7 /** Minor version number of this interface */ -#define FUSE_KERNEL_MINOR_VERSION 13 +#define FUSE_KERNEL_MINOR_VERSION 14 /** The node ID of the root inode */ #define FUSE_ROOT_ID 1 From owner-svn-src-projects@freebsd.org Thu Jun 20 23:32:27 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A440F15CAA20 for ; Thu, 20 Jun 2019 23:32:27 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 42AFA6C74E; Thu, 20 Jun 2019 23:32:27 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 18BCD1B151; Thu, 20 Jun 2019 23:32:27 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5KNWQAY017687; Thu, 20 Jun 2019 23:32:26 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5KNWQB2017682; Thu, 20 Jun 2019 23:32:26 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906202332.x5KNWQB2017682@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Thu, 20 Jun 2019 23:32:26 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349250 - in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Commit-Revision: 349250 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 42AFA6C74E X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.96)[-0.963,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2019 23:32:28 -0000 Author: asomers Date: Thu Jun 20 23:32:25 2019 New Revision: 349250 URL: https://svnweb.freebsd.org/changeset/base/349250 Log: fusefs: raise protocol level to 7.15 This protocol level adds two new features: the ability for the server to store or retrieve data into/from the client's cache. But the messages aren't defined soundly since they identify the file only by its inode, without the generation number. So it's possible for them to modify the wrong file's cache. Also, I don't know of any file systems in ports that use these messages. So I'm not implementing them. I did add a (disabled) test for the store message, however. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_device.c projects/fuse2/sys/fs/fuse/fuse_kernel.h projects/fuse2/tests/sys/fs/fusefs/mockfs.cc projects/fuse2/tests/sys/fs/fusefs/mockfs.hh projects/fuse2/tests/sys/fs/fusefs/notify.cc Modified: projects/fuse2/sys/fs/fuse/fuse_device.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_device.c Thu Jun 20 23:12:19 2019 (r349249) +++ projects/fuse2/sys/fs/fuse/fuse_device.c Thu Jun 20 23:32:25 2019 (r349250) @@ -518,7 +518,19 @@ fuse_device_write(struct cdev *dev, struct uio *uio, i case FUSE_NOTIFY_INVAL_INODE: err = fuse_internal_invalidate_inode(mp, uio); break; + case FUSE_NOTIFY_RETRIEVE: + case FUSE_NOTIFY_STORE: + /* + * Unimplemented. I don't know of any file systems + * that use them, and the protocol isn't sound anyway, + * since the notification messages don't include the + * inode's generation number. Without that, it's + * possible to manipulate the cache of the wrong vnode. + * Finally, it's not defined what this message should + * do for a file with dirty cache. + */ case FUSE_NOTIFY_POLL: + /* Unimplemented. See comments in fuse_vnops */ default: /* Not implemented */ err = ENOSYS; Modified: projects/fuse2/sys/fs/fuse/fuse_kernel.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_kernel.h Thu Jun 20 23:12:19 2019 (r349249) +++ projects/fuse2/sys/fs/fuse/fuse_kernel.h Thu Jun 20 23:32:25 2019 (r349250) @@ -64,6 +64,10 @@ * * 7.14 * - add splice support to fuse device + * + * 7.15 + * - add store notify + * - add retrieve notify */ #ifndef _FUSE_FUSE_KERNEL_H @@ -84,7 +88,7 @@ #define FUSE_KERNEL_VERSION 7 /** Minor version number of this interface */ -#define FUSE_KERNEL_MINOR_VERSION 14 +#define FUSE_KERNEL_MINOR_VERSION 15 /** The node ID of the root inode */ #define FUSE_ROOT_ID 1 @@ -269,6 +273,7 @@ enum fuse_opcode { FUSE_DESTROY = 38, FUSE_IOCTL = 39, FUSE_POLL = 40, + FUSE_NOTIFY_REPLY = 41, #ifdef linux /* CUSE specific operations */ @@ -280,6 +285,8 @@ enum fuse_notify_code { FUSE_NOTIFY_POLL = 1, FUSE_NOTIFY_INVAL_INODE = 2, FUSE_NOTIFY_INVAL_ENTRY = 3, + FUSE_NOTIFY_STORE = 4, + FUSE_NOTIFY_RETRIEVE = 5, FUSE_NOTIFY_CODE_MAX, }; @@ -556,6 +563,31 @@ struct fuse_poll_out { struct fuse_notify_poll_wakeup_out { __u64 kh; +}; + +struct fuse_notify_store_out { + __u64 nodeid; + __u64 offset; + __u32 size; + __u32 padding; +}; + +struct fuse_notify_retrieve_out { + __u64 notify_unique; + __u64 nodeid; + __u64 offset; + __u32 size; + __u32 padding; +}; + +/* Matches the size of fuse_write_in */ +struct fuse_notify_retrieve_in { + __u64 dummy1; + __u64 offset; + __u32 size; + __u32 dummy2; + __u64 dummy3; + __u64 dummy4; }; struct fuse_in_header { Modified: projects/fuse2/tests/sys/fs/fusefs/mockfs.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/mockfs.cc Thu Jun 20 23:12:19 2019 (r349249) +++ projects/fuse2/tests/sys/fs/fusefs/mockfs.cc Thu Jun 20 23:32:25 2019 (r349250) @@ -333,6 +333,13 @@ void MockFS::debug_response(const mockfs_buf_out &out) out.body.inval_inode.off, out.body.inval_inode.len); break; + case FUSE_NOTIFY_STORE: + printf("<- STORE ino=%" PRIu64 " off=%" PRIu64 + " size=%" PRIu32 "\n", + out.body.store.nodeid, + out.body.store.offset, + out.body.store.size); + break; default: break; } @@ -539,6 +546,22 @@ int MockFS::notify_inval_inode(ino_t ino, off_t off, s out->body.inval_inode.off = off; out->body.inval_inode.len = len; out->header.len = sizeof(out->header) + sizeof(out->body.inval_inode); + debug_response(*out); + write_response(*out); + return 0; +} + +int MockFS::notify_store(ino_t ino, off_t off, void* data, ssize_t size) +{ + std::unique_ptr out(new mockfs_buf_out); + + out->header.unique = 0; /* 0 means asynchronous notification */ + out->header.error = FUSE_NOTIFY_STORE; + out->body.store.nodeid = ino; + out->body.store.offset = off; + out->body.store.size = size; + bcopy(data, (char*)&out->body.bytes + sizeof(out->body.store), size); + out->header.len = sizeof(out->header) + sizeof(out->body.store) + size; debug_response(*out); write_response(*out); return 0; Modified: projects/fuse2/tests/sys/fs/fusefs/mockfs.hh ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/mockfs.hh Thu Jun 20 23:12:19 2019 (r349249) +++ projects/fuse2/tests/sys/fs/fusefs/mockfs.hh Thu Jun 20 23:32:25 2019 (r349250) @@ -181,6 +181,8 @@ union fuse_payloads_out { /* The inval_entry structure should be followed by the entry's name */ fuse_notify_inval_entry_out inval_entry; fuse_notify_inval_inode_out inval_inode; + /* The store structure should be followed by the data to store */ + fuse_notify_store_out store; fuse_listxattr_out listxattr; fuse_open_out open; fuse_statfs_out statfs; @@ -345,6 +347,20 @@ class MockFS { * to invalidate all cached data. */ int notify_inval_inode(ino_t ino, off_t off, ssize_t len); + + /* + * Send an asynchronous notification to store data directly into an + * inode's cache. Similar to libfuse's fuse_lowlevel_notify_store. + * + * This method will block until the client has responded, so it should + * generally be run in a separate thread from request processing. + * + * @param ino File's inode number + * @param off Offset at which to store data + * @param data Pointer to the data to cache + * @param len Size of data + */ + int notify_store(ino_t ino, off_t off, void* data, ssize_t size); /* * Request handler Modified: projects/fuse2/tests/sys/fs/fusefs/notify.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/notify.cc Thu Jun 20 23:12:19 2019 (r349249) +++ projects/fuse2/tests/sys/fs/fusefs/notify.cc Thu Jun 20 23:32:25 2019 (r349250) @@ -133,6 +133,14 @@ struct inval_inode_args { ssize_t len; }; +struct store_args { + MockFS *mock; + ino_t nodeid; + off_t offset; + ssize_t size; + void* data; +}; + static void* inval_inode(void* arg) { const struct inval_inode_args *iia = (struct inval_inode_args*)arg; ssize_t r; @@ -144,6 +152,17 @@ static void* inval_inode(void* arg) { return (void*)(intptr_t)errno; } +static void* store(void* arg) { + const struct store_args *sa = (struct store_args*)arg; + ssize_t r; + + r = sa->mock->notify_store(sa->nodeid, sa->offset, sa->data, sa->size); + if (r >= 0) + return 0; + else + return (void*)(intptr_t)errno; +} + /* Invalidate a nonexistent entry */ TEST_F(Notify, inval_entry_nonexistent) { @@ -370,6 +389,65 @@ TEST_F(Notify, inval_inode_with_clean_cache) /* This read should not be serviced by cache */ ASSERT_EQ(0, lseek(fd, 0, SEEK_SET)) << strerror(errno); + ASSERT_EQ(size1, read(fd, buf, size1)) << strerror(errno); + EXPECT_EQ(0, memcmp(buf, CONTENTS1, size1)); + + /* Deliberately leak fd. close(2) will be tested in release.cc */ +} + +/* FUSE_NOTIFY_STORE with a file that's not in the entry cache */ +/* disabled because FUSE_NOTIFY_STORE is not yet implemented */ +TEST_F(Notify, DISABLED_store_nonexistent) +{ + struct store_args sa; + ino_t ino = 42; + void *thr0_value; + pthread_t th0; + + sa.mock = m_mock; + sa.nodeid = ino; + sa.offset = 0; + sa.size = 0; + ASSERT_EQ(0, pthread_create(&th0, NULL, store, &sa)) << strerror(errno); + pthread_join(th0, &thr0_value); + /* It's not an error for a file to be unknown to the kernel */ + EXPECT_EQ(0, (intptr_t)thr0_value); +} + +/* Store data into for a file that does not yet have anything cached */ +/* disabled because FUSE_NOTIFY_STORE is not yet implemented */ +TEST_F(Notify, DISABLED_store_with_blank_cache) +{ + const static char FULLPATH[] = "mountpoint/foo"; + const static char RELPATH[] = "foo"; + const char CONTENTS1[] = "ijklmnopqrstuvwxyz"; + struct store_args sa; + ino_t ino = 42; + void *thr0_value; + Sequence seq; + pthread_t th0; + ssize_t size1 = sizeof(CONTENTS1); + char buf[80]; + int fd; + + expect_lookup(FUSE_ROOT_ID, RELPATH, ino, size1, seq); + expect_open(ino, 0, 1); + + /* Fill the data cache */ + fd = open(FULLPATH, O_RDWR); + ASSERT_LE(0, fd) << strerror(errno); + + /* Evict the data cache */ + sa.mock = m_mock; + sa.nodeid = ino; + sa.offset = 0; + sa.size = size1; + sa.data = (void*)CONTENTS1; + ASSERT_EQ(0, pthread_create(&th0, NULL, store, &sa)) << strerror(errno); + pthread_join(th0, &thr0_value); + EXPECT_EQ(0, (intptr_t)thr0_value); + + /* This read should be serviced by cache */ ASSERT_EQ(size1, read(fd, buf, size1)) << strerror(errno); EXPECT_EQ(0, memcmp(buf, CONTENTS1, size1)); From owner-svn-src-projects@freebsd.org Fri Jun 21 02:55:44 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 508F415CE4CA for ; Fri, 21 Jun 2019 02:55:44 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DF2017220C; Fri, 21 Jun 2019 02:55:43 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B613A1D304; Fri, 21 Jun 2019 02:55:43 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5L2thqW022949; Fri, 21 Jun 2019 02:55:43 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5L2thQl022948; Fri, 21 Jun 2019 02:55:43 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906210255.x5L2thQl022948@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 21 Jun 2019 02:55:43 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349258 - projects/fuse2/sys/fs/fuse X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/sys/fs/fuse X-SVN-Commit-Revision: 349258 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: DF2017220C X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.96)[-0.961,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jun 2019 02:55:44 -0000 Author: asomers Date: Fri Jun 21 02:55:43 2019 New Revision: 349258 URL: https://svnweb.freebsd.org/changeset/base/349258 Log: fusefs: diff reduction of fuse_kernel.h vs the upstream version fuse_kernel.h is based on Linux's fuse.h. In r349250 I modified fuse_kernel.h by generating a diff of two versions of Linux's fuse.h and applying it to our tree. patch succeeded, but it put one chunk in the wrong location. This commit fixes that. No functional changes. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_kernel.h Modified: projects/fuse2/sys/fs/fuse/fuse_kernel.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_kernel.h Fri Jun 21 02:49:36 2019 (r349257) +++ projects/fuse2/sys/fs/fuse/fuse_kernel.h Fri Jun 21 02:55:43 2019 (r349258) @@ -565,31 +565,6 @@ struct fuse_notify_poll_wakeup_out { __u64 kh; }; -struct fuse_notify_store_out { - __u64 nodeid; - __u64 offset; - __u32 size; - __u32 padding; -}; - -struct fuse_notify_retrieve_out { - __u64 notify_unique; - __u64 nodeid; - __u64 offset; - __u32 size; - __u32 padding; -}; - -/* Matches the size of fuse_write_in */ -struct fuse_notify_retrieve_in { - __u64 dummy1; - __u64 offset; - __u32 size; - __u32 dummy2; - __u64 dummy3; - __u64 dummy4; -}; - struct fuse_in_header { __u32 len; __u32 opcode; @@ -630,6 +605,31 @@ struct fuse_notify_inval_entry_out { __u64 parent; __u32 namelen; __u32 padding; +}; + +struct fuse_notify_store_out { + __u64 nodeid; + __u64 offset; + __u32 size; + __u32 padding; +}; + +struct fuse_notify_retrieve_out { + __u64 notify_unique; + __u64 nodeid; + __u64 offset; + __u32 size; + __u32 padding; +}; + +/* Matches the size of fuse_write_in */ +struct fuse_notify_retrieve_in { + __u64 dummy1; + __u64 offset; + __u32 size; + __u32 dummy2; + __u64 dummy3; + __u64 dummy4; }; #endif /* _FUSE_FUSE_KERNEL_H */ From owner-svn-src-projects@freebsd.org Fri Jun 21 03:04:57 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 964E815CE892 for ; Fri, 21 Jun 2019 03:04:57 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3B46B729AA; Fri, 21 Jun 2019 03:04:57 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 12B171D500; Fri, 21 Jun 2019 03:04:57 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5L34u1O029316; Fri, 21 Jun 2019 03:04:56 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5L34ulM029315; Fri, 21 Jun 2019 03:04:56 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906210304.x5L34ulM029315@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 21 Jun 2019 03:04:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349259 - projects/fuse2/sys/fs/fuse X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/sys/fs/fuse X-SVN-Commit-Revision: 349259 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 3B46B729AA X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.96)[-0.963,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jun 2019 03:04:57 -0000 Author: asomers Date: Fri Jun 21 03:04:56 2019 New Revision: 349259 URL: https://svnweb.freebsd.org/changeset/base/349259 Log: fusefs: raise the protocol level to 7.21 Jumping from protocol 7.15 to 7.21 adds several new features. While they're all potentially useful, they're also all optional, and I'm not implementing any right now because my highest priority lies in a later version. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_kernel.h Modified: projects/fuse2/sys/fs/fuse/fuse_kernel.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_kernel.h Fri Jun 21 02:55:43 2019 (r349258) +++ projects/fuse2/sys/fs/fuse/fuse_kernel.h Fri Jun 21 03:04:56 2019 (r349259) @@ -68,27 +68,49 @@ * 7.15 * - add store notify * - add retrieve notify + * + * 7.16 + * - add BATCH_FORGET request + * - FUSE_IOCTL_UNRESTRICTED shall now return with array of 'struct + * fuse_ioctl_iovec' instead of ambiguous 'struct iovec' + * - add FUSE_IOCTL_32BIT flag + * + * 7.17 + * - add FUSE_FLOCK_LOCKS and FUSE_RELEASE_FLOCK_UNLOCK + * + * 7.18 + * - add FUSE_IOCTL_DIR flag + * - add FUSE_NOTIFY_DELETE + * + * 7.19 + * - add FUSE_FALLOCATE + * + * 7.20 + * - add FUSE_AUTO_INVAL_DATA + * 7.21 + * - add FUSE_READDIRPLUS + * - send the requested events in POLL request */ #ifndef _FUSE_FUSE_KERNEL_H #define _FUSE_FUSE_KERNEL_H -#ifndef linux +#ifdef __linux__ +#include +#else #include #define __u64 uint64_t #define __s64 int64_t #define __u32 uint32_t #define __s32 int32_t #define __u16 uint16_t -#else -#include #endif /** Version number of this interface */ #define FUSE_KERNEL_VERSION 7 /** Minor version number of this interface */ -#define FUSE_KERNEL_MINOR_VERSION 15 +#define FUSE_KERNEL_MINOR_VERSION 21 /** The node ID of the root inode */ #define FUSE_ROOT_ID 1 @@ -163,8 +185,21 @@ struct fuse_file_lock { /** * INIT request/reply flags * + * FUSE_ASYNC_READ: asynchronous read requests + * FUSE_POSIX_LOCKS: remote locking for POSIX file locks + * FUSE_FILE_OPS: kernel sends file handle for fstat, etc... (not yet supported) + * FUSE_ATOMIC_O_TRUNC: handles the O_TRUNC open flag in the filesystem * FUSE_EXPORT_SUPPORT: filesystem handles lookups of "." and ".." + * FUSE_BIG_WRITES: filesystem can handle write size larger than 4kB * FUSE_DONT_MASK: don't apply umask to file mode on create operations + * FUSE_SPLICE_WRITE: kernel supports splice write on the device + * FUSE_SPLICE_MOVE: kernel supports splice move on the device + * FUSE_SPLICE_READ: kernel supports splice read on the device + * FUSE_FLOCK_LOCKS: remote locking for BSD style file locks + * FUSE_HAS_IOCTL_DIR: kernel supports ioctl on directories + * FUSE_AUTO_INVAL_DATA: automatically invalidate cached pages + * FUSE_DO_READDIRPLUS: do READDIRPLUS (READDIR+LOOKUP in one) + * FUSE_READDIRPLUS_AUTO: adaptive readdirplus */ #define FUSE_ASYNC_READ (1 << 0) #define FUSE_POSIX_LOCKS (1 << 1) @@ -173,6 +208,14 @@ struct fuse_file_lock { #define FUSE_EXPORT_SUPPORT (1 << 4) #define FUSE_BIG_WRITES (1 << 5) #define FUSE_DONT_MASK (1 << 6) +#define FUSE_SPLICE_WRITE (1 << 7) +#define FUSE_SPLICE_MOVE (1 << 8) +#define FUSE_SPLICE_READ (1 << 9) +#define FUSE_FLOCK_LOCKS (1 << 10) +#define FUSE_HAS_IOCTL_DIR (1 << 11) +#define FUSE_AUTO_INVAL_DATA (1 << 12) +#define FUSE_DO_READDIRPLUS (1 << 13) +#define FUSE_READDIRPLUS_AUTO (1 << 14) #ifdef linux /** @@ -187,6 +230,7 @@ struct fuse_file_lock { * Release flags */ #define FUSE_RELEASE_FLUSH (1 << 0) +#define FUSE_RELEASE_FLOCK_UNLOCK (1 << 1) /** * Getattr flags @@ -218,12 +262,16 @@ struct fuse_file_lock { * FUSE_IOCTL_COMPAT: 32bit compat ioctl on 64bit machine * FUSE_IOCTL_UNRESTRICTED: not restricted to well-formed ioctls, retry allowed * FUSE_IOCTL_RETRY: retry with new iovecs + * FUSE_IOCTL_32BIT: 32bit ioctl + * FUSE_IOCTL_DIR: is a directory * * FUSE_IOCTL_MAX_IOV: maximum of in_iovecs + out_iovecs */ #define FUSE_IOCTL_COMPAT (1 << 0) #define FUSE_IOCTL_UNRESTRICTED (1 << 1) #define FUSE_IOCTL_RETRY (1 << 2) +#define FUSE_IOCTL_32BIT (1 << 3) +#define FUSE_IOCTL_DIR (1 << 4) #define FUSE_IOCTL_MAX_IOV 256 @@ -274,6 +322,9 @@ enum fuse_opcode { FUSE_IOCTL = 39, FUSE_POLL = 40, FUSE_NOTIFY_REPLY = 41, + FUSE_BATCH_FORGET = 42, + FUSE_FALLOCATE = 43, + FUSE_READDIRPLUS = 44, #ifdef linux /* CUSE specific operations */ @@ -287,6 +338,7 @@ enum fuse_notify_code { FUSE_NOTIFY_INVAL_ENTRY = 3, FUSE_NOTIFY_STORE = 4, FUSE_NOTIFY_RETRIEVE = 5, + FUSE_NOTIFY_DELETE = 6, FUSE_NOTIFY_CODE_MAX, }; @@ -310,6 +362,16 @@ struct fuse_forget_in { __u64 nlookup; }; +struct fuse_forget_one { + __u64 nodeid; + __u64 nlookup; +}; + +struct fuse_batch_forget_in { + __u32 count; + __u32 dummy; +}; + struct fuse_getattr_in { __u32 getattr_flags; __u32 dummy; @@ -542,6 +604,11 @@ struct fuse_ioctl_in { __u32 out_size; }; +struct fuse_ioctl_iovec { + __u64 base; + __u64 len; +}; + struct fuse_ioctl_out { __s32 result; __u32 flags; @@ -553,7 +620,7 @@ struct fuse_poll_in { __u64 fh; __u64 kh; __u32 flags; - __u32 padding; + __u32 events; }; struct fuse_poll_out { @@ -565,6 +632,14 @@ struct fuse_notify_poll_wakeup_out { __u64 kh; }; +struct fuse_fallocate_in { + __u64 fh; + __u64 offset; + __u64 length; + __u32 mode; + __u32 padding; +}; + struct fuse_in_header { __u32 len; __u32 opcode; @@ -587,7 +662,7 @@ struct fuse_dirent { __u64 off; __u32 namelen; __u32 type; - char name[0]; + char name[]; }; #define FUSE_NAME_OFFSET offsetof(struct fuse_dirent, name) @@ -595,6 +670,16 @@ struct fuse_dirent { #define FUSE_DIRENT_SIZE(d) \ FUSE_DIRENT_ALIGN(FUSE_NAME_OFFSET + (d)->namelen) +struct fuse_direntplus { + struct fuse_entry_out entry_out; + struct fuse_dirent dirent; +}; + +#define FUSE_NAME_OFFSET_DIRENTPLUS \ + offsetof(struct fuse_direntplus, dirent.name) +#define FUSE_DIRENTPLUS_SIZE(d) \ + FUSE_DIRENT_ALIGN(FUSE_NAME_OFFSET_DIRENTPLUS + (d)->dirent.namelen) + struct fuse_notify_inval_inode_out { __u64 ino; __s64 off; @@ -603,6 +688,13 @@ struct fuse_notify_inval_inode_out { struct fuse_notify_inval_entry_out { __u64 parent; + __u32 namelen; + __u32 padding; +}; + +struct fuse_notify_delete_out { + __u64 parent; + __u64 child; __u32 namelen; __u32 padding; }; From owner-svn-src-projects@freebsd.org Fri Jun 21 03:17:28 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 831A815CEC05 for ; Fri, 21 Jun 2019 03:17:28 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2A9D273062; Fri, 21 Jun 2019 03:17:28 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 195511D6BB; Fri, 21 Jun 2019 03:17:28 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5L3HRBt034404; Fri, 21 Jun 2019 03:17:27 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5L3HRcm034403; Fri, 21 Jun 2019 03:17:27 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906210317.x5L3HRcm034403@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 21 Jun 2019 03:17:27 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349260 - projects/fuse2/sys/fs/fuse X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/sys/fs/fuse X-SVN-Commit-Revision: 349260 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 2A9D273062 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.96)[-0.964,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jun 2019 03:17:28 -0000 Author: asomers Date: Fri Jun 21 03:17:27 2019 New Revision: 349260 URL: https://svnweb.freebsd.org/changeset/base/349260 Log: fusefs: use standard integer types in fuse_kernel.h This is a merge of Linux revision 4c82456eeb4da081dd63dc69e91aa6deabd29e03. No functional change. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_kernel.h Modified: projects/fuse2/sys/fs/fuse/fuse_kernel.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_kernel.h Fri Jun 21 03:04:56 2019 (r349259) +++ projects/fuse2/sys/fs/fuse/fuse_kernel.h Fri Jun 21 03:17:27 2019 (r349260) @@ -99,11 +99,6 @@ #include #else #include -#define __u64 uint64_t -#define __s64 int64_t -#define __u32 uint32_t -#define __s32 int32_t -#define __u16 uint16_t #endif /** Version number of this interface */ @@ -119,42 +114,42 @@ userspace works under 64bit kernels */ struct fuse_attr { - __u64 ino; - __u64 size; - __u64 blocks; - __u64 atime; - __u64 mtime; - __u64 ctime; - __u32 atimensec; - __u32 mtimensec; - __u32 ctimensec; - __u32 mode; - __u32 nlink; - __u32 uid; - __u32 gid; - __u32 rdev; - __u32 blksize; - __u32 padding; + uint64_t ino; + uint64_t size; + uint64_t blocks; + uint64_t atime; + uint64_t mtime; + uint64_t ctime; + uint32_t atimensec; + uint32_t mtimensec; + uint32_t ctimensec; + uint32_t mode; + uint32_t nlink; + uint32_t uid; + uint32_t gid; + uint32_t rdev; + uint32_t blksize; + uint32_t padding; }; struct fuse_kstatfs { - __u64 blocks; - __u64 bfree; - __u64 bavail; - __u64 files; - __u64 ffree; - __u32 bsize; - __u32 namelen; - __u32 frsize; - __u32 padding; - __u32 spare[6]; + uint64_t blocks; + uint64_t bfree; + uint64_t bavail; + uint64_t files; + uint64_t ffree; + uint32_t bsize; + uint32_t namelen; + uint32_t frsize; + uint32_t padding; + uint32_t spare[6]; }; struct fuse_file_lock { - __u64 start; - __u64 end; - __u32 type; - __u32 pid; /* tgid */ + uint64_t start; + uint64_t end; + uint32_t type; + uint32_t pid; /* tgid */ }; /** @@ -348,143 +343,143 @@ enum fuse_notify_code { #define FUSE_COMPAT_ENTRY_OUT_SIZE 120 struct fuse_entry_out { - __u64 nodeid; /* Inode ID */ - __u64 generation; /* Inode generation: nodeid:gen must - be unique for the fs's lifetime */ - __u64 entry_valid; /* Cache timeout for the name */ - __u64 attr_valid; /* Cache timeout for the attributes */ - __u32 entry_valid_nsec; - __u32 attr_valid_nsec; + uint64_t nodeid; /* Inode ID */ + uint64_t generation; /* Inode generation: nodeid:gen must + be unique for the fs's lifetime */ + uint64_t entry_valid; /* Cache timeout for the name */ + uint64_t attr_valid; /* Cache timeout for the attributes */ + uint32_t entry_valid_nsec; + uint32_t attr_valid_nsec; struct fuse_attr attr; }; struct fuse_forget_in { - __u64 nlookup; + uint64_t nlookup; }; struct fuse_forget_one { - __u64 nodeid; - __u64 nlookup; + uint64_t nodeid; + uint64_t nlookup; }; struct fuse_batch_forget_in { - __u32 count; - __u32 dummy; + uint32_t count; + uint32_t dummy; }; struct fuse_getattr_in { - __u32 getattr_flags; - __u32 dummy; - __u64 fh; + uint32_t getattr_flags; + uint32_t dummy; + uint64_t fh; }; #define FUSE_COMPAT_ATTR_OUT_SIZE 96 struct fuse_attr_out { - __u64 attr_valid; /* Cache timeout for the attributes */ - __u32 attr_valid_nsec; - __u32 dummy; + uint64_t attr_valid; /* Cache timeout for the attributes */ + uint32_t attr_valid_nsec; + uint32_t dummy; struct fuse_attr attr; }; #define FUSE_COMPAT_MKNOD_IN_SIZE 8 struct fuse_mknod_in { - __u32 mode; - __u32 rdev; - __u32 umask; - __u32 padding; + uint32_t mode; + uint32_t rdev; + uint32_t umask; + uint32_t padding; }; struct fuse_mkdir_in { - __u32 mode; - __u32 umask; + uint32_t mode; + uint32_t umask; }; struct fuse_rename_in { - __u64 newdir; + uint64_t newdir; }; struct fuse_link_in { - __u64 oldnodeid; + uint64_t oldnodeid; }; struct fuse_setattr_in { - __u32 valid; - __u32 padding; - __u64 fh; - __u64 size; - __u64 lock_owner; - __u64 atime; - __u64 mtime; - __u64 unused2; - __u32 atimensec; - __u32 mtimensec; - __u32 unused3; - __u32 mode; - __u32 unused4; - __u32 uid; - __u32 gid; - __u32 unused5; + uint32_t valid; + uint32_t padding; + uint64_t fh; + uint64_t size; + uint64_t lock_owner; + uint64_t atime; + uint64_t mtime; + uint64_t unused2; + uint32_t atimensec; + uint32_t mtimensec; + uint32_t unused3; + uint32_t mode; + uint32_t unused4; + uint32_t uid; + uint32_t gid; + uint32_t unused5; }; struct fuse_open_in { - __u32 flags; - __u32 unused; + uint32_t flags; + uint32_t unused; }; struct fuse_create_in { - __u32 flags; - __u32 mode; - __u32 umask; - __u32 padding; + uint32_t flags; + uint32_t mode; + uint32_t umask; + uint32_t padding; }; struct fuse_open_out { - __u64 fh; - __u32 open_flags; - __u32 padding; + uint64_t fh; + uint32_t open_flags; + uint32_t padding; }; struct fuse_release_in { - __u64 fh; - __u32 flags; - __u32 release_flags; - __u64 lock_owner; + uint64_t fh; + uint32_t flags; + uint32_t release_flags; + uint64_t lock_owner; }; struct fuse_flush_in { - __u64 fh; - __u32 unused; - __u32 padding; - __u64 lock_owner; + uint64_t fh; + uint32_t unused; + uint32_t padding; + uint64_t lock_owner; }; struct fuse_read_in { - __u64 fh; - __u64 offset; - __u32 size; - __u32 read_flags; - __u64 lock_owner; - __u32 flags; - __u32 padding; + uint64_t fh; + uint64_t offset; + uint32_t size; + uint32_t read_flags; + uint64_t lock_owner; + uint32_t flags; + uint32_t padding; }; #define FUSE_COMPAT_WRITE_IN_SIZE 24 struct fuse_write_in { - __u64 fh; - __u64 offset; - __u32 size; - __u32 write_flags; - __u64 lock_owner; - __u32 flags; - __u32 padding; + uint64_t fh; + uint64_t offset; + uint32_t size; + uint32_t write_flags; + uint64_t lock_owner; + uint32_t flags; + uint32_t padding; }; struct fuse_write_out { - __u32 size; - __u32 padding; + uint32_t size; + uint32_t padding; }; #define FUSE_COMPAT_STATFS_SIZE 48 @@ -494,42 +489,42 @@ struct fuse_statfs_out { }; struct fuse_fsync_in { - __u64 fh; - __u32 fsync_flags; - __u32 padding; + uint64_t fh; + uint32_t fsync_flags; + uint32_t padding; }; struct fuse_setxattr_in { - __u32 size; - __u32 flags; + uint32_t size; + uint32_t flags; }; struct fuse_listxattr_in { - __u32 size; - __u32 padding; + uint32_t size; + uint32_t padding; }; struct fuse_listxattr_out { - __u32 size; - __u32 padding; + uint32_t size; + uint32_t padding; }; struct fuse_getxattr_in { - __u32 size; - __u32 padding; + uint32_t size; + uint32_t padding; }; struct fuse_getxattr_out { - __u32 size; - __u32 padding; + uint32_t size; + uint32_t padding; }; struct fuse_lk_in { - __u64 fh; - __u64 owner; + uint64_t fh; + uint64_t owner; struct fuse_file_lock lk; - __u32 lk_flags; - __u32 padding; + uint32_t lk_flags; + uint32_t padding; }; struct fuse_lk_out { @@ -537,136 +532,136 @@ struct fuse_lk_out { }; struct fuse_access_in { - __u32 mask; - __u32 padding; + uint32_t mask; + uint32_t padding; }; struct fuse_init_in { - __u32 major; - __u32 minor; - __u32 max_readahead; - __u32 flags; + uint32_t major; + uint32_t minor; + uint32_t max_readahead; + uint32_t flags; }; struct fuse_init_out { - __u32 major; - __u32 minor; - __u32 max_readahead; - __u32 flags; - __u16 max_background; - __u16 congestion_threshold; - __u32 max_write; + uint32_t major; + uint32_t minor; + uint32_t max_readahead; + uint32_t flags; + uint16_t max_background; + uint16_t congestion_threshold; + uint32_t max_write; }; #ifdef linux #define CUSE_INIT_INFO_MAX 4096 struct cuse_init_in { - __u32 major; - __u32 minor; - __u32 unused; - __u32 flags; + uint32_t major; + uint32_t minor; + uint32_t unused; + uint32_t flags; }; struct cuse_init_out { - __u32 major; - __u32 minor; - __u32 unused; - __u32 flags; - __u32 max_read; - __u32 max_write; - __u32 dev_major; /* chardev major */ - __u32 dev_minor; /* chardev minor */ - __u32 spare[10]; + uint32_t major; + uint32_t minor; + uint32_t unused; + uint32_t flags; + uint32_t max_read; + uint32_t max_write; + uint32_t dev_major; /* chardev major */ + uint32_t dev_minor; /* chardev minor */ + uint32_t spare[10]; }; #endif /* linux */ struct fuse_interrupt_in { - __u64 unique; + uint64_t unique; }; struct fuse_bmap_in { - __u64 block; - __u32 blocksize; - __u32 padding; + uint64_t block; + uint32_t blocksize; + uint32_t padding; }; struct fuse_bmap_out { - __u64 block; + uint64_t block; }; struct fuse_ioctl_in { - __u64 fh; - __u32 flags; - __u32 cmd; - __u64 arg; - __u32 in_size; - __u32 out_size; + uint64_t fh; + uint32_t flags; + uint32_t cmd; + uint64_t arg; + uint32_t in_size; + uint32_t out_size; }; struct fuse_ioctl_iovec { - __u64 base; - __u64 len; + uint64_t base; + uint64_t len; }; struct fuse_ioctl_out { - __s32 result; - __u32 flags; - __u32 in_iovs; - __u32 out_iovs; + int32_t result; + uint32_t flags; + uint32_t in_iovs; + uint32_t out_iovs; }; struct fuse_poll_in { - __u64 fh; - __u64 kh; - __u32 flags; - __u32 events; + uint64_t fh; + uint64_t kh; + uint32_t flags; + uint32_t events; }; struct fuse_poll_out { - __u32 revents; - __u32 padding; + uint32_t revents; + uint32_t padding; }; struct fuse_notify_poll_wakeup_out { - __u64 kh; + uint64_t kh; }; struct fuse_fallocate_in { - __u64 fh; - __u64 offset; - __u64 length; - __u32 mode; - __u32 padding; + uint64_t fh; + uint64_t offset; + uint64_t length; + uint32_t mode; + uint32_t padding; }; struct fuse_in_header { - __u32 len; - __u32 opcode; - __u64 unique; - __u64 nodeid; - __u32 uid; - __u32 gid; - __u32 pid; - __u32 padding; + uint32_t len; + uint32_t opcode; + uint64_t unique; + uint64_t nodeid; + uint32_t uid; + uint32_t gid; + uint32_t pid; + uint32_t padding; }; struct fuse_out_header { - __u32 len; - __s32 error; - __u64 unique; + uint32_t len; + int32_t error; + uint64_t unique; }; struct fuse_dirent { - __u64 ino; - __u64 off; - __u32 namelen; - __u32 type; + uint64_t ino; + uint64_t off; + uint32_t namelen; + uint32_t type; char name[]; }; #define FUSE_NAME_OFFSET offsetof(struct fuse_dirent, name) -#define FUSE_DIRENT_ALIGN(x) (((x) + sizeof(__u64) - 1) & ~(sizeof(__u64) - 1)) +#define FUSE_DIRENT_ALIGN(x) (((x) + sizeof(uint64_t) - 1) & ~(sizeof(uint64_t) - 1)) #define FUSE_DIRENT_SIZE(d) \ FUSE_DIRENT_ALIGN(FUSE_NAME_OFFSET + (d)->namelen) @@ -681,47 +676,47 @@ struct fuse_direntplus { FUSE_DIRENT_ALIGN(FUSE_NAME_OFFSET_DIRENTPLUS + (d)->dirent.namelen) struct fuse_notify_inval_inode_out { - __u64 ino; - __s64 off; - __s64 len; + uint64_t ino; + int64_t off; + int64_t len; }; struct fuse_notify_inval_entry_out { - __u64 parent; - __u32 namelen; - __u32 padding; + uint64_t parent; + uint32_t namelen; + uint32_t padding; }; struct fuse_notify_delete_out { - __u64 parent; - __u64 child; - __u32 namelen; - __u32 padding; + uint64_t parent; + uint64_t child; + uint32_t namelen; + uint32_t padding; }; struct fuse_notify_store_out { - __u64 nodeid; - __u64 offset; - __u32 size; - __u32 padding; + uint64_t nodeid; + uint64_t offset; + uint32_t size; + uint32_t padding; }; struct fuse_notify_retrieve_out { - __u64 notify_unique; - __u64 nodeid; - __u64 offset; - __u32 size; - __u32 padding; + uint64_t notify_unique; + uint64_t nodeid; + uint64_t offset; + uint32_t size; + uint32_t padding; }; /* Matches the size of fuse_write_in */ struct fuse_notify_retrieve_in { - __u64 dummy1; - __u64 offset; - __u32 size; - __u32 dummy2; - __u64 dummy3; - __u64 dummy4; + uint64_t dummy1; + uint64_t offset; + uint32_t size; + uint32_t dummy2; + uint64_t dummy3; + uint64_t dummy4; }; #endif /* _FUSE_FUSE_KERNEL_H */ From owner-svn-src-projects@freebsd.org Fri Jun 21 04:37:12 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1489D15D0169 for ; Fri, 21 Jun 2019 04:37:12 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AF3CE753D2; Fri, 21 Jun 2019 04:37:11 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 84EE71E43F; Fri, 21 Jun 2019 04:37:11 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5L4bB2T075803; Fri, 21 Jun 2019 04:37:11 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5L4bBJO075802; Fri, 21 Jun 2019 04:37:11 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906210437.x5L4bBJO075802@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 21 Jun 2019 04:37:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349262 - projects/fuse2/tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/tests/sys/fs/fusefs X-SVN-Commit-Revision: 349262 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: AF3CE753D2 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.977,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jun 2019 04:37:12 -0000 Author: asomers Date: Fri Jun 21 04:37:11 2019 New Revision: 349262 URL: https://svnweb.freebsd.org/changeset/base/349262 Log: fusefs: update tests after r349260 r349260 removed some Linuxisms from the FUSE protocol header file in favor of standard C99 types. This change follows suit in the tests. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/tests/sys/fs/fusefs/mockfs.hh Modified: projects/fuse2/tests/sys/fs/fusefs/mockfs.hh ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/mockfs.hh Fri Jun 21 03:49:36 2019 (r349261) +++ projects/fuse2/tests/sys/fs/fusefs/mockfs.hh Fri Jun 21 04:37:11 2019 (r349262) @@ -79,40 +79,40 @@ struct fuse_create_out { /* Protocol 7.8 version of struct fuse_attr */ struct fuse_attr_7_8 { - __u64 ino; - __u64 size; - __u64 blocks; - __u64 atime; - __u64 mtime; - __u64 ctime; - __u32 atimensec; - __u32 mtimensec; - __u32 ctimensec; - __u32 mode; - __u32 nlink; - __u32 uid; - __u32 gid; - __u32 rdev; + uint64_t ino; + uint64_t size; + uint64_t blocks; + uint64_t atime; + uint64_t mtime; + uint64_t ctime; + uint32_t atimensec; + uint32_t mtimensec; + uint32_t ctimensec; + uint32_t mode; + uint32_t nlink; + uint32_t uid; + uint32_t gid; + uint32_t rdev; }; /* Protocol 7.8 version of struct fuse_attr_out */ struct fuse_attr_out_7_8 { - __u64 attr_valid; - __u32 attr_valid_nsec; - __u32 dummy; + uint64_t attr_valid; + uint32_t attr_valid_nsec; + uint32_t dummy; struct fuse_attr_7_8 attr; }; /* Protocol 7.8 version of struct fuse_entry_out */ struct fuse_entry_out_7_8 { - __u64 nodeid; /* Inode ID */ - __u64 generation; /* Inode generation: nodeid:gen must + uint64_t nodeid; /* Inode ID */ + uint64_t generation; /* Inode generation: nodeid:gen must be unique for the fs's lifetime */ - __u64 entry_valid; /* Cache timeout for the name */ - __u64 attr_valid; /* Cache timeout for the attributes */ - __u32 entry_valid_nsec; - __u32 attr_valid_nsec; + uint64_t entry_valid; /* Cache timeout for the name */ + uint64_t attr_valid; /* Cache timeout for the attributes */ + uint32_t entry_valid_nsec; + uint32_t attr_valid_nsec; struct fuse_attr_7_8 attr; }; From owner-svn-src-projects@freebsd.org Fri Jun 21 04:57:25 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 49F6915D08B6 for ; Fri, 21 Jun 2019 04:57:25 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AEB107683A; Fri, 21 Jun 2019 04:57:24 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7D4D01E79C; Fri, 21 Jun 2019 04:57:24 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5L4vOXp086563; Fri, 21 Jun 2019 04:57:24 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5L4vNXq086556; Fri, 21 Jun 2019 04:57:23 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906210457.x5L4vNXq086556@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 21 Jun 2019 04:57:23 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349263 - in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Commit-Revision: 349263 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: AEB107683A X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.97)[-0.970,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jun 2019 04:57:25 -0000 Author: asomers Date: Fri Jun 21 04:57:23 2019 New Revision: 349263 URL: https://svnweb.freebsd.org/changeset/base/349263 Log: fusefs: raise protocol level to 7.23 None of the new features are implemented yet. This commit just adds the new protocol definitions and adds backwards-compatibility code for pre 7.23 servers. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c projects/fuse2/sys/fs/fuse/fuse_ipc.c projects/fuse2/sys/fs/fuse/fuse_kernel.h projects/fuse2/tests/sys/fs/fusefs/mockfs.cc projects/fuse2/tests/sys/fs/fusefs/mockfs.hh Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_internal.c Fri Jun 21 04:37:11 2019 (r349262) +++ projects/fuse2/sys/fs/fuse/fuse_internal.c Fri Jun 21 04:57:23 2019 (r349263) @@ -910,7 +910,8 @@ fuse_internal_init_callback(struct fuse_ticket *tick, } if (fuse_libabi_geq(data, 7, 5)) { - if (fticket_resp(tick)->len == sizeof(struct fuse_init_out)) { + if (fticket_resp(tick)->len == sizeof(struct fuse_init_out) || + fticket_resp(tick)->len == FUSE_COMPAT_22_INIT_OUT_SIZE) { data->max_write = fiio->max_write; if (fiio->flags & FUSE_ASYNC_READ) data->dataflags |= FSESS_ASYNC_READ; @@ -923,8 +924,8 @@ fuse_internal_init_callback(struct fuse_ticket *tick, * redundant with max_write */ /* - * max_background and congestion_threshold are not - * implemented + * max_background, congestion_threshold, and time_gran + * are not implemented */ } else { err = EINVAL; Modified: projects/fuse2/sys/fs/fuse/fuse_ipc.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_ipc.c Fri Jun 21 04:37:11 2019 (r349262) +++ projects/fuse2/sys/fs/fuse/fuse_ipc.c Fri Jun 21 04:57:23 2019 (r349263) @@ -812,7 +812,9 @@ fuse_body_audit(struct fuse_ticket *ftick, size_t blen break; case FUSE_INIT: - if (blen == sizeof(struct fuse_init_out) || blen == 8) { + if (blen == sizeof(struct fuse_init_out) || + blen == FUSE_COMPAT_INIT_OUT_SIZE || + blen == FUSE_COMPAT_22_INIT_OUT_SIZE) { err = 0; } else { err = EINVAL; Modified: projects/fuse2/sys/fs/fuse/fuse_kernel.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_kernel.h Fri Jun 21 04:37:11 2019 (r349262) +++ projects/fuse2/sys/fs/fuse/fuse_kernel.h Fri Jun 21 04:57:23 2019 (r349263) @@ -90,6 +90,18 @@ * 7.21 * - add FUSE_READDIRPLUS * - send the requested events in POLL request + * + * 7.22 + * - add FUSE_ASYNC_DIO + * + * 7.23 + * - add FUSE_WRITEBACK_CACHE + * - add time_gran to fuse_init_out + * - add reserved space to fuse_init_out + * - add FATTR_CTIME + * - add ctime and ctimensec to fuse_setattr_in + * - add FUSE_RENAME2 request + * - add FUSE_NO_OPEN_SUPPORT flag */ #ifndef _FUSE_FUSE_KERNEL_H @@ -105,7 +117,7 @@ #define FUSE_KERNEL_VERSION 7 /** Minor version number of this interface */ -#define FUSE_KERNEL_MINOR_VERSION 21 +#define FUSE_KERNEL_MINOR_VERSION 23 /** The node ID of the root inode */ #define FUSE_ROOT_ID 1 @@ -165,6 +177,7 @@ struct fuse_file_lock { #define FATTR_ATIME_NOW (1 << 7) #define FATTR_MTIME_NOW (1 << 8) #define FATTR_LOCKOWNER (1 << 9) +#define FATTR_CTIME (1 << 10) /** * Flags returned by the OPEN request @@ -195,6 +208,9 @@ struct fuse_file_lock { * FUSE_AUTO_INVAL_DATA: automatically invalidate cached pages * FUSE_DO_READDIRPLUS: do READDIRPLUS (READDIR+LOOKUP in one) * FUSE_READDIRPLUS_AUTO: adaptive readdirplus + * FUSE_ASYNC_DIO: asynchronous direct I/O submission + * FUSE_WRITEBACK_CACHE: use writeback cache for buffered writes + * FUSE_NO_OPEN_SUPPORT: kernel supports zero-message opens */ #define FUSE_ASYNC_READ (1 << 0) #define FUSE_POSIX_LOCKS (1 << 1) @@ -211,6 +227,9 @@ struct fuse_file_lock { #define FUSE_AUTO_INVAL_DATA (1 << 12) #define FUSE_DO_READDIRPLUS (1 << 13) #define FUSE_READDIRPLUS_AUTO (1 << 14) +#define FUSE_ASYNC_DIO (1 << 15) +#define FUSE_WRITEBACK_CACHE (1 << 16) +#define FUSE_NO_OPEN_SUPPORT (1 << 17) #ifdef linux /** @@ -320,6 +339,7 @@ enum fuse_opcode { FUSE_BATCH_FORGET = 42, FUSE_FALLOCATE = 43, FUSE_READDIRPLUS = 44, + FUSE_RENAME2 = 45, #ifdef linux /* CUSE specific operations */ @@ -400,6 +420,12 @@ struct fuse_rename_in { uint64_t newdir; }; +struct fuse_rename2_in { + uint64_t newdir; + uint32_t flags; + uint32_t padding; +}; + struct fuse_link_in { uint64_t oldnodeid; }; @@ -412,10 +438,10 @@ struct fuse_setattr_in { uint64_t lock_owner; uint64_t atime; uint64_t mtime; - uint64_t unused2; + uint64_t ctime; uint32_t atimensec; uint32_t mtimensec; - uint32_t unused3; + uint32_t ctimensec; uint32_t mode; uint32_t unused4; uint32_t uid; @@ -543,6 +569,9 @@ struct fuse_init_in { uint32_t flags; }; +#define FUSE_COMPAT_INIT_OUT_SIZE 8 +#define FUSE_COMPAT_22_INIT_OUT_SIZE 24 + struct fuse_init_out { uint32_t major; uint32_t minor; @@ -551,6 +580,8 @@ struct fuse_init_out { uint16_t max_background; uint16_t congestion_threshold; uint32_t max_write; + uint32_t time_gran; + uint32_t unused[9]; }; #ifdef linux @@ -615,7 +646,7 @@ struct fuse_poll_in { uint64_t fh; uint64_t kh; uint32_t flags; - uint32_t events; + uint32_t events; }; struct fuse_poll_out { @@ -661,7 +692,8 @@ struct fuse_dirent { }; #define FUSE_NAME_OFFSET offsetof(struct fuse_dirent, name) -#define FUSE_DIRENT_ALIGN(x) (((x) + sizeof(uint64_t) - 1) & ~(sizeof(uint64_t) - 1)) +#define FUSE_DIRENT_ALIGN(x) \ + (((x) + sizeof(uint64_t) - 1) & ~(sizeof(uint64_t) - 1)) #define FUSE_DIRENT_SIZE(d) \ FUSE_DIRENT_ALIGN(FUSE_NAME_OFFSET + (d)->namelen) Modified: projects/fuse2/tests/sys/fs/fusefs/mockfs.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/mockfs.cc Fri Jun 21 04:37:11 2019 (r349262) +++ projects/fuse2/tests/sys/fs/fusefs/mockfs.cc Fri Jun 21 04:57:23 2019 (r349263) @@ -469,11 +469,15 @@ void MockFS::init(uint32_t flags) { out->body.init.major = FUSE_KERNEL_VERSION; out->body.init.minor = m_kernel_minor_version;; out->body.init.flags = in->body.init.flags & flags; - out->body.init.max_write = m_maxwrite; - out->body.init.max_readahead = m_maxreadahead; - SET_OUT_HEADER_LEN(*out, init); + + if (m_kernel_minor_version < 23) { + SET_OUT_HEADER_LEN(*out, init_7_22); + } else { + SET_OUT_HEADER_LEN(*out, init); + } + write(m_fuse_fd, out.get(), out->header.len); } Modified: projects/fuse2/tests/sys/fs/fusefs/mockfs.hh ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/mockfs.hh Fri Jun 21 04:37:11 2019 (r349262) +++ projects/fuse2/tests/sys/fs/fusefs/mockfs.hh Fri Jun 21 04:57:23 2019 (r349263) @@ -122,6 +122,17 @@ struct fuse_create_out_7_8 { struct fuse_open_out open; }; +/* Output struct for FUSE_INIT for protocol 7.22 and earlier servers */ +struct fuse_init_out_7_22 { + uint32_t major; + uint32_t minor; + uint32_t max_readahead; + uint32_t flags; + uint16_t max_background; + uint16_t congestion_threshold; + uint32_t max_write; +}; + union fuse_payloads_in { fuse_access_in access; fuse_bmap_in bmap; @@ -178,6 +189,7 @@ union fuse_payloads_out { fuse_lk_out getlk; fuse_getxattr_out getxattr; fuse_init_out init; + fuse_init_out_7_22 init_7_22; /* The inval_entry structure should be followed by the entry's name */ fuse_notify_inval_entry_out inval_entry; fuse_notify_inval_inode_out inval_inode; From owner-svn-src-projects@freebsd.org Fri Jun 21 21:44:33 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 69F6C15C081C for ; Fri, 21 Jun 2019 21:44:33 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0CF0782770; Fri, 21 Jun 2019 21:44:33 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D788410F1; Fri, 21 Jun 2019 21:44:32 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5LLiW5o019576; Fri, 21 Jun 2019 21:44:32 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5LLiWYm019572; Fri, 21 Jun 2019 21:44:32 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906212144.x5LLiWYm019572@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 21 Jun 2019 21:44:32 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349279 - in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Commit-Revision: 349279 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 0CF0782770 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.97)[-0.974,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jun 2019 21:44:33 -0000 Author: asomers Date: Fri Jun 21 21:44:31 2019 New Revision: 349279 URL: https://svnweb.freebsd.org/changeset/base/349279 Log: fusefs: correctly handle short reads A fuse server may return a short read for three reasons: * The file is opened with FOPEN_DIRECT_IO. In this case, the short read should be returned directly to userland. We already handled this case correctly. * The file was truncated server-side, and the read hit EOF. In this case, the kernel should update the file size. Fixed in the case of VOP_READ. Fixing this for VOP_GETPAGES is TODO. * The file is opened in writeback mode, there are dirty buffers past what the server thinks is the file's EOF, and the read hit what the server thinks is the file's EOF. In this case, the client is trying to read a hole, and should zero-fill it. We already handled this case, and I added a test for it. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_io.c projects/fuse2/tests/sys/fs/fusefs/read.cc projects/fuse2/tests/sys/fs/fusefs/write.cc Modified: projects/fuse2/sys/fs/fuse/fuse_io.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_io.c Fri Jun 21 18:57:33 2019 (r349278) +++ projects/fuse2/sys/fs/fuse/fuse_io.c Fri Jun 21 21:44:31 2019 (r349279) @@ -348,8 +348,9 @@ fuse_read_biobackend(struct vnode *vp, struct uio *uio */ n = 0; - if (on < bcount) - n = MIN((unsigned)(bcount - on), uio->uio_resid); + if (on < bcount - bp->b_resid) + n = MIN((unsigned)(bcount - bp->b_resid - on), + uio->uio_resid); if (n > 0) { SDT_PROBE2(fusefs, , io, read_bio_backend_feed, n, bp); err = uiomove(bp->b_data + on, n, uio); @@ -357,6 +358,11 @@ fuse_read_biobackend(struct vnode *vp, struct uio *uio vfs_bio_brelse(bp, ioflag); SDT_PROBE4(fusefs, , io, read_bio_backend_end, err, uio->uio_resid, n, bp); + if (bp->b_resid > 0) { + /* Short read indicates EOF */ + (void)fuse_vnode_setsize(vp, uio->uio_offset); + break; + } } return (err); @@ -415,8 +421,13 @@ fuse_read_directbackend(struct vnode *vp, struct uio * if ((err = uiomove(fdi.answ, MIN(fri->size, fdi.iosize), uio))) break; - if (fdi.iosize < fri->size) + if (fdi.iosize < fri->size) { + /* + * Short read. Should only happen at EOF or with + * direct io. + */ break; + } } out: @@ -828,6 +839,7 @@ again: int fuse_io_strategy(struct vnode *vp, struct buf *bp) { + struct fuse_vnode_data *fvdat = VTOFUD(vp); struct fuse_filehandle *fufh; struct ucred *cred; struct uio *uiop; @@ -888,19 +900,35 @@ fuse_io_strategy(struct vnode *vp, struct buf *bp) if (!error && uiop->uio_resid) { /* - * If we had a short read with no error, we must have - * hit a file hole. We should zero-fill the remainder. - * This can also occur if the server hits the file EOF. - * - * Holes used to be able to occur due to pending - * writes, but that is not possible any longer. + * A short read with no error, when not using direct io, + * and when no writes are cached, indicates EOF. + * Update the file size accordingly. */ - int nread = bp->b_bcount - uiop->uio_resid; - int left = uiop->uio_resid; - - if (left > 0) + if (fuse_data_cache_mode != FUSE_CACHE_WB || + (fvdat->flag & FN_SIZECHANGE) == 0) { + SDT_PROBE2(fusefs, , io, trace, 1, + "Short read of a clean file"); + /* + * XXX To prevent lock order problems, we must + * truncate the file upstack + */ + } else { + /* + * If dirty writes _are_ cached beyond EOF, + * that indicates a newly created hole that the + * server doesn't know about. Fill it in. + * XXX: we don't currently track whether dirty + * writes are cached beyond EOF, before EOF, or + * both. + */ + SDT_PROBE2(fusefs, , io, trace, 1, + "Short read of a dirty file"); + int nread = bp->b_bcount - uiop->uio_resid; + int left = uiop->uio_resid; bzero((char *)bp->b_data + nread, left); - uiop->uio_resid = 0; + uiop->uio_resid = 0; + } + } if (error) { bp->b_ioflags |= BIO_ERROR; Modified: projects/fuse2/tests/sys/fs/fusefs/read.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/read.cc Fri Jun 21 18:57:33 2019 (r349278) +++ projects/fuse2/tests/sys/fs/fusefs/read.cc Fri Jun 21 21:44:31 2019 (r349279) @@ -412,6 +412,69 @@ TEST_F(Read, eio) } /* + * If the server returns a short read when direct io is not in use, that + * indicates EOF and we should update the file size. + */ +TEST_F(ReadCacheable, eof) +{ + const char FULLPATH[] = "mountpoint/some_file.txt"; + const char RELPATH[] = "some_file.txt"; + const char *CONTENTS = "abcdefghijklmnop"; + uint64_t ino = 42; + int fd; + uint64_t offset = 100; + ssize_t bufsize = strlen(CONTENTS); + ssize_t partbufsize = 3 * bufsize / 4; + char buf[bufsize]; + struct stat sb; + + expect_lookup(RELPATH, ino, offset + bufsize); + expect_open(ino, 0, 1); + expect_read(ino, 0, offset + bufsize, offset + partbufsize, CONTENTS); + + fd = open(FULLPATH, O_RDONLY); + ASSERT_LE(0, fd) << strerror(errno); + + ASSERT_EQ(partbufsize, pread(fd, buf, bufsize, offset)) + << strerror(errno); + ASSERT_EQ(0, fstat(fd, &sb)); + EXPECT_EQ((off_t)(offset + partbufsize), sb.st_size); + /* Deliberately leak fd. close(2) will be tested in release.cc */ +} + +/* Like ReadCacheable.eof, but causes an entire buffer to be invalidated */ +TEST_F(ReadCacheable, eof_of_whole_buffer) +{ + const char FULLPATH[] = "mountpoint/some_file.txt"; + const char RELPATH[] = "some_file.txt"; + const char *CONTENTS = "abcdefghijklmnop"; + uint64_t ino = 42; + int fd; + ssize_t bufsize = strlen(CONTENTS); + off_t old_filesize = m_maxbcachebuf * 2 + bufsize; + char buf[bufsize]; + struct stat sb; + + expect_lookup(RELPATH, ino, old_filesize); + expect_open(ino, 0, 1); + expect_read(ino, 2 * m_maxbcachebuf, bufsize, bufsize, CONTENTS); + expect_read(ino, m_maxbcachebuf, m_maxbcachebuf, 0, CONTENTS); + + fd = open(FULLPATH, O_RDONLY); + ASSERT_LE(0, fd) << strerror(errno); + + /* Cache the third block */ + ASSERT_EQ(bufsize, pread(fd, buf, bufsize, m_maxbcachebuf * 2)) + << strerror(errno); + /* Try to read the 2nd block, but it's past EOF */ + ASSERT_EQ(0, pread(fd, buf, bufsize, m_maxbcachebuf)) + << strerror(errno); + ASSERT_EQ(0, fstat(fd, &sb)); + EXPECT_EQ((off_t)(m_maxbcachebuf), sb.st_size); + /* Deliberately leak fd. close(2) will be tested in release.cc */ +} + +/* * With the keep_cache option, the kernel may keep its read cache across * multiple open(2)s. */ Modified: projects/fuse2/tests/sys/fs/fusefs/write.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/write.cc Fri Jun 21 18:57:33 2019 (r349278) +++ projects/fuse2/tests/sys/fs/fusefs/write.cc Fri Jun 21 21:44:31 2019 (r349279) @@ -931,6 +931,54 @@ TEST_F(WriteBackAsync, delay) } /* + * In WriteBack mode, writes may be cached beyond what the server thinks is the + * EOF. In this case, a short read at EOF should _not_ cause fusefs to update + * the file's size. + */ +TEST_F(WriteBackAsync, eof) +{ + const char FULLPATH[] = "mountpoint/some_file.txt"; + const char RELPATH[] = "some_file.txt"; + const char *CONTENTS0 = "abcdefgh"; + const char *CONTENTS1 = "ijklmnop"; + uint64_t ino = 42; + int fd; + off_t offset = m_maxbcachebuf; + ssize_t wbufsize = strlen(CONTENTS1); + off_t old_filesize = (off_t)strlen(CONTENTS0); + ssize_t rbufsize = 2 * old_filesize; + char readbuf[rbufsize]; + size_t holesize = rbufsize - old_filesize; + char hole[holesize]; + struct stat sb; + ssize_t r; + + expect_lookup(RELPATH, ino, 0); + expect_open(ino, 0, 1); + expect_read(ino, 0, m_maxbcachebuf, old_filesize, CONTENTS0); + + fd = open(FULLPATH, O_RDWR); + EXPECT_LE(0, fd) << strerror(errno); + + /* Write and cache data beyond EOF */ + ASSERT_EQ(wbufsize, pwrite(fd, CONTENTS1, wbufsize, offset)) + << strerror(errno); + + /* Read from the old EOF */ + r = pread(fd, readbuf, rbufsize, 0); + ASSERT_LE(0, r) << strerror(errno); + EXPECT_EQ(rbufsize, r) << "read should've synthesized a hole"; + EXPECT_EQ(0, memcmp(CONTENTS0, readbuf, old_filesize)); + bzero(hole, holesize); + EXPECT_EQ(0, memcmp(hole, readbuf + old_filesize, holesize)); + + /* The file's size should still be what was established by pwrite */ + ASSERT_EQ(0, fstat(fd, &sb)) << strerror(errno); + EXPECT_EQ(offset + wbufsize, sb.st_size); + /* Deliberately leak fd. close(2) will be tested in release.cc */ +} + +/* * Without direct_io, writes should be committed to cache */ TEST_F(WriteThrough, writethrough) From owner-svn-src-projects@freebsd.org Fri Jun 21 23:29:30 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8161015C3969 for ; Fri, 21 Jun 2019 23:29:30 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 21D5687225; Fri, 21 Jun 2019 23:29:30 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EAC8921D1; Fri, 21 Jun 2019 23:29:29 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5LNTTrh074640; Fri, 21 Jun 2019 23:29:29 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5LNTT4Q074639; Fri, 21 Jun 2019 23:29:29 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906212329.x5LNTT4Q074639@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 21 Jun 2019 23:29:29 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349282 - projects/fuse2/sys/fs/fuse X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/sys/fs/fuse X-SVN-Commit-Revision: 349282 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 21D5687225 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.95 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.95)[-0.955,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jun 2019 23:29:30 -0000 Author: asomers Date: Fri Jun 21 23:29:29 2019 New Revision: 349282 URL: https://svnweb.freebsd.org/changeset/base/349282 Log: fusefs: fix corruption on short reads caused by r349279 Even if a short read is caused by EOF, it's still necessary to bzero the remaining buffer, because that buffer could become valid as a result of a future ftruncate or pwrite operation. Reported by: fsx Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_io.c Modified: projects/fuse2/sys/fs/fuse/fuse_io.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_io.c Fri Jun 21 23:29:16 2019 (r349281) +++ projects/fuse2/sys/fs/fuse/fuse_io.c Fri Jun 21 23:29:29 2019 (r349282) @@ -899,33 +899,37 @@ fuse_io_strategy(struct vnode *vp, struct buf *bp) error = fuse_read_directbackend(vp, uiop, cred, fufh); if (!error && uiop->uio_resid) { - /* - * A short read with no error, when not using direct io, - * and when no writes are cached, indicates EOF. - * Update the file size accordingly. - */ + int nread = bp->b_bcount - uiop->uio_resid; + int left = uiop->uio_resid; + bzero((char *)bp->b_data + nread, left); + if (fuse_data_cache_mode != FUSE_CACHE_WB || (fvdat->flag & FN_SIZECHANGE) == 0) { - SDT_PROBE2(fusefs, , io, trace, 1, - "Short read of a clean file"); - /* + /* + * A short read with no error, when not using + * direct io, and when no writes are cached, + * indicates EOF. Update the file size + * accordingly. We must still bzero the + * remaining buffer so uninitialized data + * doesn't get exposed by a future truncate + * that extends the file. + * * XXX To prevent lock order problems, we must * truncate the file upstack */ + SDT_PROBE2(fusefs, , io, trace, 1, + "Short read of a clean file"); } else { /* * If dirty writes _are_ cached beyond EOF, * that indicates a newly created hole that the - * server doesn't know about. Fill it in. + * server doesn't know about. * XXX: we don't currently track whether dirty * writes are cached beyond EOF, before EOF, or * both. */ SDT_PROBE2(fusefs, , io, trace, 1, "Short read of a dirty file"); - int nread = bp->b_bcount - uiop->uio_resid; - int left = uiop->uio_resid; - bzero((char *)bp->b_data + nread, left); uiop->uio_resid = 0; }