From owner-svn-src-projects@freebsd.org Mon Jun 24 17:05:32 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4280F15D2DB7 for ; Mon, 24 Jun 2019 17:05:32 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DD1587679C; Mon, 24 Jun 2019 17:05:31 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C673B3FF4; Mon, 24 Jun 2019 17:05:31 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5OH5VXm046997; Mon, 24 Jun 2019 17:05:31 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5OH5Vo4046995; Mon, 24 Jun 2019 17:05:31 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906241705.x5OH5Vo4046995@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Mon, 24 Jun 2019 17:05:31 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349332 - in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Commit-Revision: 349332 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: DD1587679C X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.97)[-0.967,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jun 2019 17:05:32 -0000 Author: asomers Date: Mon Jun 24 17:05:31 2019 New Revision: 349332 URL: https://svnweb.freebsd.org/changeset/base/349332 Log: fusefs: improve the short read fix from r349279 VOP_GETPAGES intentionally tries to read beyond EOF, so fuse_read_biobackend can't rely on bp->b_resid > 0 indicating a short read. And adjusting bp->b_count after a short read seems to cause some sort of resource leak. Instead, store the shortfall in the bp->b_fsprivate1 field. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_io.c projects/fuse2/tests/sys/fs/fusefs/io.cc Modified: projects/fuse2/sys/fs/fuse/fuse_io.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_io.c Mon Jun 24 02:58:02 2019 (r349331) +++ projects/fuse2/sys/fs/fuse/fuse_io.c Mon Jun 24 17:05:31 2019 (r349332) @@ -348,8 +348,8 @@ fuse_read_biobackend(struct vnode *vp, struct uio *uio */ n = 0; - if (on < bcount - bp->b_resid) - n = MIN((unsigned)(bcount - bp->b_resid - on), + if (on < bcount - (intptr_t)bp->b_fsprivate1) + n = MIN((unsigned)(bcount - (intptr_t)bp->b_fsprivate1 - on), uio->uio_resid); if (n > 0) { SDT_PROBE2(fusefs, , io, read_bio_backend_feed, n, bp); @@ -358,9 +358,10 @@ fuse_read_biobackend(struct vnode *vp, struct uio *uio vfs_bio_brelse(bp, ioflag); SDT_PROBE4(fusefs, , io, read_bio_backend_end, err, uio->uio_resid, n, bp); - if (bp->b_resid > 0) { + if ((intptr_t)bp->b_fsprivate1 > 0) { /* Short read indicates EOF */ (void)fuse_vnode_setsize(vp, uio->uio_offset); + bp->b_fsprivate1 = (void*)0; break; } } @@ -891,16 +892,23 @@ fuse_io_strategy(struct vnode *vp, struct buf *bp) KASSERT(!(bp->b_flags & B_DONE), ("fuse_io_strategy: bp %p already marked done", bp)); if (bp->b_iocmd == BIO_READ) { + ssize_t left; + io.iov_len = uiop->uio_resid = bp->b_bcount; io.iov_base = bp->b_data; uiop->uio_rw = UIO_READ; uiop->uio_offset = ((off_t)bp->b_lblkno) * biosize; error = fuse_read_directbackend(vp, uiop, cred, fufh); + left = uiop->uio_resid; + /* + * Store the amount we failed to read in the buffer's private + * field, so callers can truncate the file if necessary' + */ + bp->b_fsprivate1 = (void*)(intptr_t)left; if (!error && uiop->uio_resid) { int nread = bp->b_bcount - uiop->uio_resid; - int left = uiop->uio_resid; bzero((char *)bp->b_data + nread, left); if (fuse_data_cache_mode != FUSE_CACHE_WB || @@ -914,11 +922,12 @@ fuse_io_strategy(struct vnode *vp, struct buf *bp) * doesn't get exposed by a future truncate * that extends the file. * - * XXX To prevent lock order problems, we must + * To prevent lock order problems, we must * truncate the file upstack */ SDT_PROBE2(fusefs, , io, trace, 1, "Short read of a clean file"); + uiop->uio_resid = 0; } else { /* * If dirty writes _are_ cached beyond EOF, Modified: projects/fuse2/tests/sys/fs/fusefs/io.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/io.cc Mon Jun 24 02:58:02 2019 (r349331) +++ projects/fuse2/tests/sys/fs/fusefs/io.cc Mon Jun 24 17:05:31 2019 (r349332) @@ -29,6 +29,8 @@ */ extern "C" { +#include + #include #include #include @@ -76,11 +78,13 @@ class Io: public FuseTest, public WithParamInterface> { public: int m_backing_fd, m_control_fd, m_test_fd; +off_t m_filesize; Io(): m_backing_fd(-1), m_control_fd(-1) {}; void SetUp() { + m_filesize = 0; m_backing_fd = open("backing_file", O_RDWR | O_CREAT | O_TRUNC, 0644); if (m_backing_fd < 0) FAIL() << strerror(errno); @@ -126,10 +130,11 @@ void SetUp() ssize_t isize = in.body.write.size; off_t iofs = in.body.write.offset; void *buf = out.body.bytes; + ssize_t osize; - ASSERT_LE(0, pread(m_backing_fd, buf, isize, iofs)) - << strerror(errno); - out.header.len = sizeof(struct fuse_out_header) + isize; + osize = pread(m_backing_fd, buf, isize, iofs); + ASSERT_LE(0, osize) << strerror(errno); + out.header.len = sizeof(struct fuse_out_header) + osize; }))); EXPECT_CALL(*m_mock, process( ResultOf([=](auto in) { @@ -182,21 +187,52 @@ void do_ftruncate(off_t offs) { ASSERT_EQ(0, ftruncate(m_test_fd, offs)) << strerror(errno); ASSERT_EQ(0, ftruncate(m_control_fd, offs)) << strerror(errno); + m_filesize = offs; } +void do_mapread(ssize_t size, off_t offs) +{ + void *control_buf, *p; + off_t pg_offset, page_mask; + size_t map_size; + + page_mask = getpagesize() - 1; + pg_offset = offs & page_mask; + map_size = pg_offset + size; + + p = mmap(NULL, map_size, PROT_READ, MAP_FILE | MAP_SHARED, m_test_fd, + offs - pg_offset); + ASSERT_NE(p, MAP_FAILED) << strerror(errno); + + control_buf = malloc(size); + ASSERT_NE(NULL, control_buf) << strerror(errno); + + ASSERT_EQ(size, pread(m_control_fd, control_buf, size, offs)) + << strerror(errno); + + compare((void*)((char*)p + pg_offset), control_buf, offs, size); + + ASSERT_EQ(0, munmap(p, map_size)) << strerror(errno); + free(control_buf); +} + void do_read(ssize_t size, off_t offs) { void *test_buf, *control_buf; + ssize_t r; test_buf = malloc(size); ASSERT_NE(NULL, test_buf) << strerror(errno); control_buf = malloc(size); ASSERT_NE(NULL, control_buf) << strerror(errno); - ASSERT_EQ(size, pread(m_test_fd, test_buf, size, offs)) - << strerror(errno); - ASSERT_EQ(size, pread(m_control_fd, control_buf, size, offs)) - << strerror(errno); + errno = 0; + r = pread(m_test_fd, test_buf, size, offs); + ASSERT_NE(-1, r) << strerror(errno); + ASSERT_EQ(size, r) << "unexpected short read"; + r = pread(m_control_fd, control_buf, size, offs); + ASSERT_NE(-1, r) << strerror(errno); + ASSERT_EQ(size, r) << "unexpected short read"; compare(test_buf, control_buf, offs, size); @@ -204,6 +240,43 @@ void do_read(ssize_t size, off_t offs) free(test_buf); } +void do_mapwrite(ssize_t size, off_t offs) +{ + char *buf; + void *p; + off_t pg_offset, page_mask; + size_t map_size; + long i; + + page_mask = getpagesize() - 1; + pg_offset = offs & page_mask; + map_size = pg_offset + size; + + buf = (char*)malloc(size); + ASSERT_NE(NULL, buf) << strerror(errno); + for (i=0; i < size; i++) + buf[i] = random(); + + if (offs + size > m_filesize) { + /* + * Must manually extend. vm_mmap_vnode will not implicitly + * extend a vnode + */ + do_ftruncate(offs + size); + } + + p = mmap(NULL, map_size, PROT_READ | PROT_WRITE, + MAP_FILE | MAP_SHARED, m_test_fd, offs - pg_offset); + ASSERT_NE(p, MAP_FAILED) << strerror(errno); + + bcopy(buf, (char*)p + pg_offset, size); + ASSERT_EQ(size, pwrite(m_control_fd, buf, size, offs)) + << strerror(errno); + + free(buf); + ASSERT_EQ(0, munmap(p, map_size)) << strerror(errno); +} + void do_write(ssize_t size, off_t offs) { char *buf; @@ -218,6 +291,9 @@ void do_write(ssize_t size, off_t offs) << strerror(errno); ASSERT_EQ(size, pwrite(m_control_fd, buf, size, offs)) << strerror(errno); + m_filesize = std::max(m_filesize, offs + size); + + free(buf); } }; @@ -241,6 +317,18 @@ TEST_P(Io, extend_from_dirty_page) } /* + * mapwrite into a newly extended part of a file. + * + * fsx -c 100 -i 100 -l 524288 -o 131072 -N5 -P /tmp -S19 fsx.bin + */ +TEST_P(Io, extend_by_mapwrite) +{ + do_mapwrite(0x849e, 0x29a3a); /* [0x29a3a, 0x31ed7] */ + do_mapwrite(0x3994, 0x3c7d8); /* [0x3c7d8, 0x4016b] */ + do_read(0xf556, 0x30c16); /* [0x30c16, 0x4016b] */ +} + +/* * When writing the last page of a file, it must be written synchronously. * Otherwise the cached page can become invalid by a subsequent extend * operation. @@ -249,16 +337,20 @@ TEST_P(Io, extend_from_dirty_page) */ TEST_P(Io, last_page) { - off_t wofs0 = 0x1134f; - ssize_t wsize0 = 0xcc77; - off_t wofs1 = 0x2096a; - ssize_t wsize1 = 0xdfa7; - off_t rofs = 0x1a3aa; - ssize_t rsize = 0xb5b7; + do_write(0xcc77, 0x1134f); /* [0x1134f, 0x1dfc5] */ + do_write(0xdfa7, 0x2096a); /* [0x2096a, 0x2e910] */ + do_read(0xb5b7, 0x1a3aa); /* [0x1a3aa, 0x25960] */ +} - do_write(wsize0, wofs0); - do_write(wsize1, wofs1); - do_read(rsize, rofs); +/* + * Read a hole using mmap + * + * fsx -c 100 -i 100 -l 524288 -o 131072 -N11 -P /tmp -S14 fsx.bin + */ +TEST_P(Io, mapread_hole) +{ + do_write(0x123b7, 0xf205); /* [0xf205, 0x215bb] */ + do_mapread(0xeeea, 0x2f4c); /* [0x2f4c, 0x11e35] */ } /* From owner-svn-src-projects@freebsd.org Mon Jun 24 20:08:29 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 96B3115D7FF1 for ; Mon, 24 Jun 2019 20:08:29 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 39F5886312; Mon, 24 Jun 2019 20:08:29 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 25CC25E96; Mon, 24 Jun 2019 20:08:29 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5OK8SaY043123; Mon, 24 Jun 2019 20:08:28 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5OK8Sq9043122; Mon, 24 Jun 2019 20:08:28 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906242008.x5OK8Sq9043122@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Mon, 24 Jun 2019 20:08:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349338 - projects/fuse2/sys/fs/fuse X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/sys/fs/fuse X-SVN-Commit-Revision: 349338 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 39F5886312 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_SHORT(-0.98)[-0.980,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jun 2019 20:08:29 -0000 Author: asomers Date: Mon Jun 24 20:08:28 2019 New Revision: 349338 URL: https://svnweb.freebsd.org/changeset/base/349338 Log: fusefs: refine the short read fix from r349332 b_fsprivate1 needs to be initialized even for write operations, probably because a buffer can be used to read, write, and read again with the final read serviced by cache. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_io.c Modified: projects/fuse2/sys/fs/fuse/fuse_io.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_io.c Mon Jun 24 19:52:08 2019 (r349337) +++ projects/fuse2/sys/fs/fuse/fuse_io.c Mon Jun 24 20:08:28 2019 (r349338) @@ -164,6 +164,8 @@ fuse_io_clear_suid_on_write(struct vnode *vp, struct u SDT_PROBE_DEFINE5(fusefs, , io, io_dispatch, "struct vnode*", "struct uio*", "int", "struct ucred*", "struct fuse_filehandle*"); +SDT_PROBE_DEFINE5(fusefs, , io, io_dispatch_filehandles_closed, "struct vnode*", + "struct uio*", "int", "bool", "struct ucred*"); int fuse_io_dispatch(struct vnode *vp, struct uio *uio, int ioflag, bool pages, struct ucred *cred, pid_t pid) @@ -186,6 +188,8 @@ fuse_io_dispatch(struct vnode *vp, struct uio *uio, in closefufh = true; } else if (err) { + SDT_PROBE5(fusefs, , io, io_dispatch_filehandles_closed, + vp, uio, ioflag, pages, cred); printf("FUSE: io dispatch: filehandles are closed\n"); return err; } @@ -860,6 +864,7 @@ fuse_io_strategy(struct vnode *vp, struct buf *bp) fflag = bp->b_iocmd == BIO_READ ? FREAD : FWRITE; cred = bp->b_iocmd == BIO_READ ? bp->b_rcred : bp->b_wcred; error = fuse_filehandle_getrw(vp, fflag, &fufh, cred, pid); + bp->b_fsprivate1 = (void*)(intptr_t)0; if (bp->b_iocmd == BIO_READ && error == EBADF) { /* * This may be a read-modify-write operation on a cached file From owner-svn-src-projects@freebsd.org Tue Jun 25 16:49:21 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9674C15D1364 for ; Tue, 25 Jun 2019 16:49:21 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2116F6BC58; Tue, 25 Jun 2019 16:49:21 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id F07D31B1AE; Tue, 25 Jun 2019 16:49:20 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5PGnK9G093586; Tue, 25 Jun 2019 16:49:20 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5PGnKos093585; Tue, 25 Jun 2019 16:49:20 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906251649.x5PGnKos093585@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Tue, 25 Jun 2019 16:49:20 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349375 - projects/fuse2/tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/tests/sys/fs/fusefs X-SVN-Commit-Revision: 349375 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 2116F6BC58 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.97)[-0.974,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jun 2019 16:49:21 -0000 Author: asomers Date: Tue Jun 25 16:49:20 2019 New Revision: 349375 URL: https://svnweb.freebsd.org/changeset/base/349375 Log: fusefs: fix multiple issues with the io tests * During TearDown, close the test file before the backing file. That way the backing file artifact will have the correct contents after the test completes. It doesn't matter when running in Kyua, but it may when running the test manually. * Add a closeopen operation that mimics what FSX does with the "-c" option. * Skip mmap-related tests when vfs.fusefs.data_cache_mode == 0 Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/tests/sys/fs/fusefs/io.cc Modified: projects/fuse2/tests/sys/fs/fusefs/io.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/io.cc Tue Jun 25 16:39:25 2019 (r349374) +++ projects/fuse2/tests/sys/fs/fusefs/io.cc Tue Jun 25 16:49:20 2019 (r349375) @@ -29,7 +29,9 @@ */ extern "C" { +#include #include +#include #include #include @@ -175,6 +177,8 @@ void SetUp() void TearDown() { + if (m_test_fd >= 0) + close(m_test_fd); if (m_backing_fd >= 0) close(m_backing_fd); if (m_control_fd >= 0) @@ -183,6 +187,17 @@ void TearDown() /* Deliberately leak test_fd */ } +void do_closeopen() +{ + ASSERT_EQ(0, close(m_test_fd)) << strerror(errno); + m_test_fd = open("backing_file", O_RDWR); + ASSERT_LE(0, m_test_fd) << strerror(errno); + + ASSERT_EQ(0, close(m_control_fd)) << strerror(errno); + m_control_fd = open("control", O_RDWR); + ASSERT_LE(0, m_control_fd) << strerror(errno); +} + void do_ftruncate(off_t offs) { ASSERT_EQ(0, ftruncate(m_test_fd, offs)) << strerror(errno); @@ -298,6 +313,22 @@ void do_write(ssize_t size, off_t offs) }; +class IoCacheable: public Io { +public: +virtual void SetUp() { + const char *node = "vfs.fusefs.data_cache_mode"; + int val = 0; + size_t size = sizeof(val); + + ASSERT_EQ(0, sysctlbyname(node, &val, &size, NULL, 0)) + << strerror(errno); + if (val == 0) + GTEST_SKIP() << + "fusefs data caching must be enabled for this test"; + Io::SetUp(); +} +}; + /* * Extend a file with dirty data in the last page of the last block. * @@ -321,7 +352,7 @@ TEST_P(Io, extend_from_dirty_page) * * fsx -c 100 -i 100 -l 524288 -o 131072 -N5 -P /tmp -S19 fsx.bin */ -TEST_P(Io, extend_by_mapwrite) +TEST_P(IoCacheable, extend_by_mapwrite) { do_mapwrite(0x849e, 0x29a3a); /* [0x29a3a, 0x31ed7] */ do_mapwrite(0x3994, 0x3c7d8); /* [0x3c7d8, 0x4016b] */ @@ -347,7 +378,7 @@ TEST_P(Io, last_page) * * fsx -c 100 -i 100 -l 524288 -o 131072 -N11 -P /tmp -S14 fsx.bin */ -TEST_P(Io, mapread_hole) +TEST_P(IoCacheable, mapread_hole) { do_write(0x123b7, 0xf205); /* [0xf205, 0x215bb] */ do_mapread(0xeeea, 0x2f4c); /* [0x2f4c, 0x11e35] */ @@ -461,6 +492,11 @@ TEST_P(Io, resize_a_valid_buffer_while_extending) } INSTANTIATE_TEST_CASE_P(Io, Io, + Combine(Values(0, FUSE_ASYNC_READ), /* m_init_flags */ + Values(0x1000, 0x10000, 0x20000), /* m_maxwrite */ + Bool())); /* m_async */ + +INSTANTIATE_TEST_CASE_P(Io, IoCacheable, Combine(Values(0, FUSE_ASYNC_READ), /* m_init_flags */ Values(0x1000, 0x10000, 0x20000), /* m_maxwrite */ Bool())); /* m_async */ From owner-svn-src-projects@freebsd.org Tue Jun 25 17:24:45 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 423A415D2283 for ; Tue, 25 Jun 2019 17:24:45 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E06626D5F5; Tue, 25 Jun 2019 17:24:44 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B80FC1B88D; Tue, 25 Jun 2019 17:24:44 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5PHOia8014884; Tue, 25 Jun 2019 17:24:44 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5PHOhN1014880; Tue, 25 Jun 2019 17:24:43 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906251724.x5PHOhN1014880@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Tue, 25 Jun 2019 17:24:43 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349378 - in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Commit-Revision: 349378 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: E06626D5F5 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.97)[-0.966,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jun 2019 17:24:45 -0000 Author: asomers Date: Tue Jun 25 17:24:43 2019 New Revision: 349378 URL: https://svnweb.freebsd.org/changeset/base/349378 Log: fusefs: rewrite vop_getpages and vop_putpages Use the standard facilities for getpages and putpages instead of bespoke implementations that don't work well with the writeback cache. This has several corollaries: * Change the way we handle short reads _again_. vfs_bio_getpages doesn't provide any way to handle unexpected short reads. Plus, I found some more lock-order problems. So now when the short read is detected we'll just clear the vnode's attribute cache, forcing the file size to be requeried the next time it's needed. VOP_GETPAGES doesn't have any way to indicate a short read to the "caller", so we just bzero the rest of the page whenever a short read happens. * Change the way we decide when to set the FUSE_WRITE_CACHE bit. We now set it for clustered writes even when the writeback cache is not in use. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_io.c projects/fuse2/sys/fs/fuse/fuse_io.h projects/fuse2/sys/fs/fuse/fuse_vnops.c projects/fuse2/tests/sys/fs/fusefs/read.cc projects/fuse2/tests/sys/fs/fusefs/write.cc Modified: projects/fuse2/sys/fs/fuse/fuse_io.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_io.c Tue Jun 25 17:15:44 2019 (r349377) +++ projects/fuse2/sys/fs/fuse/fuse_io.c Tue Jun 25 17:24:43 2019 (r349378) @@ -100,6 +100,12 @@ __FBSDID("$FreeBSD$"); #include "fuse_ipc.h" #include "fuse_io.h" +/* + * Set in a struct buf to indicate that the write came from the buffer cache + * and the originating cred and pid are no longer known. + */ +#define B_FUSEFS_WRITE_CACHE B_FS_FLAG1 + SDT_PROVIDER_DECLARE(fusefs); /* * Fuse trace probe: @@ -164,10 +170,10 @@ fuse_io_clear_suid_on_write(struct vnode *vp, struct u SDT_PROBE_DEFINE5(fusefs, , io, io_dispatch, "struct vnode*", "struct uio*", "int", "struct ucred*", "struct fuse_filehandle*"); -SDT_PROBE_DEFINE5(fusefs, , io, io_dispatch_filehandles_closed, "struct vnode*", - "struct uio*", "int", "bool", "struct ucred*"); +SDT_PROBE_DEFINE4(fusefs, , io, io_dispatch_filehandles_closed, "struct vnode*", + "struct uio*", "int", "struct ucred*"); int -fuse_io_dispatch(struct vnode *vp, struct uio *uio, int ioflag, bool pages, +fuse_io_dispatch(struct vnode *vp, struct uio *uio, int ioflag, struct ucred *cred, pid_t pid) { struct fuse_filehandle *fufh; @@ -188,8 +194,8 @@ fuse_io_dispatch(struct vnode *vp, struct uio *uio, in closefufh = true; } else if (err) { - SDT_PROBE5(fusefs, , io, io_dispatch_filehandles_closed, - vp, uio, ioflag, pages, cred); + SDT_PROBE4(fusefs, , io, io_dispatch_filehandles_closed, + vp, uio, ioflag, cred); printf("FUSE: io dispatch: filehandles are closed\n"); return err; } @@ -236,15 +242,13 @@ fuse_io_dispatch(struct vnode *vp, struct uio *uio, in start = uio->uio_offset; end = start + uio->uio_resid; - /* - * Invalidate the write cache unless we're coming from - * VOP_PUTPAGES, in which case we're writing _from_ the - * write cache - */ - if (!pages ) - v_inval_buf_range(vp, start, end, iosize); + KASSERT((ioflag & (IO_VMIO | IO_DIRECT)) != + (IO_VMIO | IO_DIRECT), + ("IO_DIRECT used for a cache flush?")); + /* Invalidate the write cache when writing directly */ + v_inval_buf_range(vp, start, end, iosize); err = fuse_write_directbackend(vp, uio, cred, fufh, - filesize, ioflag, pages); + filesize, ioflag, false); } else { SDT_PROBE2(fusefs, , io, trace, 1, "buffered write of vnode"); @@ -352,8 +356,8 @@ fuse_read_biobackend(struct vnode *vp, struct uio *uio */ n = 0; - if (on < bcount - (intptr_t)bp->b_fsprivate1) - n = MIN((unsigned)(bcount - (intptr_t)bp->b_fsprivate1 - on), + if (on < bcount - bp->b_resid) + n = MIN((unsigned)(bcount - bp->b_resid - on), uio->uio_resid); if (n > 0) { SDT_PROBE2(fusefs, , io, read_bio_backend_feed, n, bp); @@ -362,10 +366,8 @@ fuse_read_biobackend(struct vnode *vp, struct uio *uio vfs_bio_brelse(bp, ioflag); SDT_PROBE4(fusefs, , io, read_bio_backend_end, err, uio->uio_resid, n, bp); - if ((intptr_t)bp->b_fsprivate1 > 0) { + if (bp->b_resid > 0) { /* Short read indicates EOF */ - (void)fuse_vnode_setsize(vp, uio->uio_offset); - bp->b_fsprivate1 = (void*)0; break; } } @@ -725,6 +727,21 @@ again: brelse(bp); break; } + if (bp->b_resid > 0) { + /* + * Short read indicates EOF. Update file size + * from the server and try again. + */ + SDT_PROBE2(fusefs, , io, trace, 1, + "Short read during a RMW"); + brelse(bp); + err = fuse_vnode_size(vp, &filesize, cred, + curthread); + if (err) + break; + else + goto again; + } } if (bp->b_wcred == NOCRED) bp->b_wcred = crhold(cred); @@ -805,8 +822,11 @@ again: vfs_bio_set_flags(bp, ioflag); + bp->b_flags |= B_FUSEFS_WRITE_CACHE; if (ioflag & IO_SYNC) { SDT_PROBE2(fusefs, , io, write_biobackend_issue, 2, bp); + if (!(ioflag & IO_VMIO)) + bp->b_flags &= ~B_FUSEFS_WRITE_CACHE; err = bwrite(bp); } else if (vm_page_count_severe() || buf_dirty_count_severe() || @@ -864,7 +884,6 @@ fuse_io_strategy(struct vnode *vp, struct buf *bp) fflag = bp->b_iocmd == BIO_READ ? FREAD : FWRITE; cred = bp->b_iocmd == BIO_READ ? bp->b_rcred : bp->b_wcred; error = fuse_filehandle_getrw(vp, fflag, &fufh, cred, pid); - bp->b_fsprivate1 = (void*)(intptr_t)0; if (bp->b_iocmd == BIO_READ && error == EBADF) { /* * This may be a read-modify-write operation on a cached file @@ -905,39 +924,40 @@ fuse_io_strategy(struct vnode *vp, struct buf *bp) uiop->uio_offset = ((off_t)bp->b_lblkno) * biosize; error = fuse_read_directbackend(vp, uiop, cred, fufh); - left = uiop->uio_resid; /* * Store the amount we failed to read in the buffer's private * field, so callers can truncate the file if necessary' */ - bp->b_fsprivate1 = (void*)(intptr_t)left; if (!error && uiop->uio_resid) { int nread = bp->b_bcount - uiop->uio_resid; + left = uiop->uio_resid; bzero((char *)bp->b_data + nread, left); - if (fuse_data_cache_mode != FUSE_CACHE_WB || - (fvdat->flag & FN_SIZECHANGE) == 0) { + if ((fvdat->flag & FN_SIZECHANGE) == 0) { /* * A short read with no error, when not using * direct io, and when no writes are cached, - * indicates EOF. Update the file size - * accordingly. We must still bzero the - * remaining buffer so uninitialized data - * doesn't get exposed by a future truncate - * that extends the file. + * indicates EOF caused by a server-side + * truncation. Clear the attr cache so we'll + * pick up the new file size and timestamps. + * + * We must still bzero the remaining buffer so + * uninitialized data doesn't get exposed by a + * future truncate that extends the file. * * To prevent lock order problems, we must - * truncate the file upstack + * truncate the file upstack, not here. */ SDT_PROBE2(fusefs, , io, trace, 1, "Short read of a clean file"); - uiop->uio_resid = 0; + fuse_vnode_clear_attr_cache(vp); } else { /* * If dirty writes _are_ cached beyond EOF, * that indicates a newly created hole that the - * server doesn't know about. + * server doesn't know about. Those don't pose + * any problem. * XXX: we don't currently track whether dirty * writes are cached beyond EOF, before EOF, or * both. @@ -976,8 +996,9 @@ fuse_io_strategy(struct vnode *vp, struct buf *bp) io.iov_base = (char *)bp->b_data + bp->b_dirtyoff; uiop->uio_rw = UIO_WRITE; + bool pages = bp->b_flags & B_FUSEFS_WRITE_CACHE; error = fuse_write_directbackend(vp, uiop, cred, fufh, - filesize, 0, false); + filesize, 0, pages); if (error == EINTR || error == ETIMEDOUT) { bp->b_flags &= ~(B_INVAL | B_NOCACHE); Modified: projects/fuse2/sys/fs/fuse/fuse_io.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_io.h Tue Jun 25 17:15:44 2019 (r349377) +++ projects/fuse2/sys/fs/fuse/fuse_io.h Tue Jun 25 17:24:43 2019 (r349378) @@ -60,7 +60,7 @@ #ifndef _FUSE_IO_H_ #define _FUSE_IO_H_ -int fuse_io_dispatch(struct vnode *vp, struct uio *uio, int ioflag, bool pages, +int fuse_io_dispatch(struct vnode *vp, struct uio *uio, int ioflag, struct ucred *cred, pid_t pid); int fuse_io_strategy(struct vnode *vp, struct buf *bp); int fuse_io_flushbuf(struct vnode *vp, int waitfor, struct thread *td); Modified: projects/fuse2/sys/fs/fuse/fuse_vnops.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_vnops.c Tue Jun 25 17:15:44 2019 (r349377) +++ projects/fuse2/sys/fs/fuse/fuse_vnops.c Tue Jun 25 17:24:43 2019 (r349378) @@ -150,7 +150,6 @@ static vop_strategy_t fuse_vnop_strategy; static vop_symlink_t fuse_vnop_symlink; static vop_write_t fuse_vnop_write; static vop_getpages_t fuse_vnop_getpages; -static vop_putpages_t fuse_vnop_putpages; static vop_print_t fuse_vnop_print; static vop_vptofh_t fuse_vnop_vptofh; @@ -215,7 +214,6 @@ struct vop_vector fuse_vnops = { .vop_symlink = fuse_vnop_symlink, .vop_write = fuse_vnop_write, .vop_getpages = fuse_vnop_getpages, - .vop_putpages = fuse_vnop_putpages, .vop_print = fuse_vnop_print, .vop_vptofh = fuse_vnop_vptofh, }; @@ -1381,7 +1379,7 @@ fuse_vnop_read(struct vop_read_args *ap) ioflag |= IO_DIRECT; } - return fuse_io_dispatch(vp, uio, ioflag, false, cred, pid); + return fuse_io_dispatch(vp, uio, ioflag, cred, pid); } /* @@ -1960,229 +1958,60 @@ fuse_vnop_write(struct vop_write_args *ap) ioflag |= IO_DIRECT; } - return fuse_io_dispatch(vp, uio, ioflag, false, cred, pid); + return fuse_io_dispatch(vp, uio, ioflag, cred, pid); } -SDT_PROBE_DEFINE1(fusefs, , vnops, vnop_getpages_error, "int"); -/* - struct vnop_getpages_args { - struct vnode *a_vp; - vm_page_t *a_m; - int a_count; - int a_reqpage; - }; -*/ -static int -fuse_vnop_getpages(struct vop_getpages_args *ap) +static daddr_t +fuse_gbp_getblkno(struct vnode *vp, vm_ooffset_t off) { - int i, error, nextoff, size, toff, count, npages; - struct uio uio; - struct iovec iov; - vm_offset_t kva; - struct buf *bp; - struct vnode *vp; - struct thread *td; - struct ucred *cred; - vm_page_t *pages; - pid_t pid = curthread->td_proc->p_pid; + const int biosize = fuse_iosize(vp); - vp = ap->a_vp; - KASSERT(vp->v_object, ("objectless vp passed to getpages")); - td = curthread; /* XXX */ - cred = curthread->td_ucred; /* XXX */ - pages = ap->a_m; - npages = ap->a_count; + return (off / biosize); +} - if (!fsess_opt_mmap(vnode_mount(vp))) { - SDT_PROBE2(fusefs, , vnops, trace, 1, - "called on non-cacheable vnode??\n"); - return (VM_PAGER_ERROR); - } +static int +fuse_gbp_getblksz(struct vnode *vp, daddr_t lbn) +{ + off_t filesize; + int blksz, err; + const int biosize = fuse_iosize(vp); - /* - * If the last page is partially valid, just return it and allow - * the pager to zero-out the blanks. Partially valid pages can - * only occur at the file EOF. - * - * XXXGL: is that true for FUSE, which is a local filesystem, - * but still somewhat disconnected from the kernel? - */ - VM_OBJECT_WLOCK(vp->v_object); - if (pages[npages - 1]->valid != 0 && --npages == 0) - goto out; - VM_OBJECT_WUNLOCK(vp->v_object); + err = fuse_vnode_size(vp, &filesize, NULL, NULL); + KASSERT(err == 0, ("vfs_bio_getpages can't handle errors here")); + if (err) + return biosize; - /* - * We use only the kva address for the buffer, but this is extremely - * convenient and fast. - */ - bp = uma_zalloc(fuse_pbuf_zone, M_WAITOK); - - kva = (vm_offset_t)bp->b_data; - pmap_qenter(kva, pages, npages); - VM_CNT_INC(v_vnodein); - VM_CNT_ADD(v_vnodepgsin, npages); - - count = npages << PAGE_SHIFT; - iov.iov_base = (caddr_t)kva; - iov.iov_len = count; - uio.uio_iov = &iov; - uio.uio_iovcnt = 1; - uio.uio_offset = IDX_TO_OFF(pages[0]->pindex); - uio.uio_resid = count; - uio.uio_segflg = UIO_SYSSPACE; - uio.uio_rw = UIO_READ; - uio.uio_td = td; - - error = fuse_io_dispatch(vp, &uio, IO_DIRECT, true, cred, pid); - pmap_qremove(kva, npages); - - uma_zfree(fuse_pbuf_zone, bp); - - if (error && (uio.uio_resid == count)) { - SDT_PROBE1(fusefs, , vnops, vnop_getpages_error, error); - return VM_PAGER_ERROR; + if ((off_t)lbn * biosize >= filesize) { + blksz = 0; + } else if ((off_t)(lbn + 1) * biosize > filesize) { + blksz = filesize - (off_t)lbn *biosize; + } else { + blksz = biosize; } - /* - * Calculate the number of bytes read and validate only that number - * of bytes. Note that due to pending writes, size may be 0. This - * does not mean that the remaining data is invalid! - */ - - size = count - uio.uio_resid; - VM_OBJECT_WLOCK(vp->v_object); - fuse_vm_page_lock_queues(); - for (i = 0, toff = 0; i < npages; i++, toff = nextoff) { - vm_page_t m; - - nextoff = toff + PAGE_SIZE; - m = pages[i]; - - if (nextoff <= size) { - /* - * Read operation filled an entire page - */ - m->valid = VM_PAGE_BITS_ALL; - KASSERT(m->dirty == 0, - ("fuse_getpages: page %p is dirty", m)); - } else if (size > toff) { - /* - * Read operation filled a partial page. - */ - m->valid = 0; - vm_page_set_valid_range(m, 0, size - toff); - KASSERT(m->dirty == 0, - ("fuse_getpages: page %p is dirty", m)); - } else { - /* - * Read operation was short. If no error occurred - * we may have hit a zero-fill section. We simply - * leave valid set to 0. - */ - ; - } - } - fuse_vm_page_unlock_queues(); -out: - VM_OBJECT_WUNLOCK(vp->v_object); - if (ap->a_rbehind) - *ap->a_rbehind = 0; - if (ap->a_rahead) - *ap->a_rahead = 0; - return (VM_PAGER_OK); + return (blksz); } /* - struct vnop_putpages_args { + struct vnop_getpages_args { struct vnode *a_vp; vm_page_t *a_m; int a_count; - int a_sync; - int *a_rtvals; - vm_ooffset_t a_offset; + int a_reqpage; }; */ static int -fuse_vnop_putpages(struct vop_putpages_args *ap) +fuse_vnop_getpages(struct vop_getpages_args *ap) { - struct uio uio; - struct iovec iov; - vm_offset_t kva; - struct buf *bp; - int i, error, npages, count; - off_t offset; - int *rtvals; - struct vnode *vp; - struct thread *td; - struct ucred *cred; - vm_page_t *pages; - vm_ooffset_t fsize; - pid_t pid = curthread->td_proc->p_pid; + struct vnode *vp = ap->a_vp; - vp = ap->a_vp; - KASSERT(vp->v_object, ("objectless vp passed to putpages")); - fsize = vp->v_object->un_pager.vnp.vnp_size; - td = curthread; /* XXX */ - cred = curthread->td_ucred; /* XXX */ - pages = ap->a_m; - count = ap->a_count; - rtvals = ap->a_rtvals; - npages = btoc(count); - offset = IDX_TO_OFF(pages[0]->pindex); - if (!fsess_opt_mmap(vnode_mount(vp))) { SDT_PROBE2(fusefs, , vnops, trace, 1, "called on non-cacheable vnode??\n"); + return (VM_PAGER_ERROR); } - for (i = 0; i < npages; i++) - rtvals[i] = VM_PAGER_AGAIN; - /* - * When putting pages, do not extend file past EOF. - */ - - if (offset + count > fsize) { - count = fsize - offset; - if (count < 0) - count = 0; - } - /* - * We use only the kva address for the buffer, but this is extremely - * convenient and fast. - */ - bp = uma_zalloc(fuse_pbuf_zone, M_WAITOK); - - kva = (vm_offset_t)bp->b_data; - pmap_qenter(kva, pages, npages); - VM_CNT_INC(v_vnodeout); - VM_CNT_ADD(v_vnodepgsout, count); - - iov.iov_base = (caddr_t)kva; - iov.iov_len = count; - uio.uio_iov = &iov; - uio.uio_iovcnt = 1; - uio.uio_offset = offset; - uio.uio_resid = count; - uio.uio_segflg = UIO_SYSSPACE; - uio.uio_rw = UIO_WRITE; - uio.uio_td = td; - - error = fuse_io_dispatch(vp, &uio, IO_DIRECT, true, cred, pid); - - pmap_qremove(kva, npages); - uma_zfree(fuse_pbuf_zone, bp); - - if (!error) { - int nwritten = round_page(count - uio.uio_resid) / PAGE_SIZE; - - for (i = 0; i < nwritten; i++) { - rtvals[i] = VM_PAGER_OK; - VM_OBJECT_WLOCK(pages[i]->object); - vm_page_undirty(pages[i]); - VM_OBJECT_WUNLOCK(pages[i]->object); - } - } - return rtvals[0]; + return (vfs_bio_getpages(vp, ap->a_m, ap->a_count, ap->a_rbehind, + ap->a_rahead, fuse_gbp_getblkno, fuse_gbp_getblksz)); } static const char extattr_namespace_separator = '.'; Modified: projects/fuse2/tests/sys/fs/fusefs/read.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/read.cc Tue Jun 25 17:15:44 2019 (r349377) +++ projects/fuse2/tests/sys/fs/fusefs/read.cc Tue Jun 25 17:24:43 2019 (r349378) @@ -413,7 +413,8 @@ TEST_F(Read, eio) /* * If the server returns a short read when direct io is not in use, that - * indicates EOF and we should update the file size. + * indicates EOF, because of a server-side truncation. We should invalidate + * all cached attributes. We may update the file size, */ TEST_F(ReadCacheable, eof) { @@ -425,18 +426,21 @@ TEST_F(ReadCacheable, eof) uint64_t offset = 100; ssize_t bufsize = strlen(CONTENTS); ssize_t partbufsize = 3 * bufsize / 4; + ssize_t r; char buf[bufsize]; struct stat sb; expect_lookup(RELPATH, ino, offset + bufsize); expect_open(ino, 0, 1); expect_read(ino, 0, offset + bufsize, offset + partbufsize, CONTENTS); + expect_getattr(ino, offset + partbufsize); fd = open(FULLPATH, O_RDONLY); ASSERT_LE(0, fd) << strerror(errno); - ASSERT_EQ(partbufsize, pread(fd, buf, bufsize, offset)) - << strerror(errno); + r = pread(fd, buf, bufsize, offset); + ASSERT_LE(0, r) << strerror(errno); + EXPECT_EQ(partbufsize, r) << strerror(errno); ASSERT_EQ(0, fstat(fd, &sb)); EXPECT_EQ((off_t)(offset + partbufsize), sb.st_size); /* Deliberately leak fd. close(2) will be tested in release.cc */ @@ -459,6 +463,7 @@ TEST_F(ReadCacheable, eof_of_whole_buffer) expect_open(ino, 0, 1); expect_read(ino, 2 * m_maxbcachebuf, bufsize, bufsize, CONTENTS); expect_read(ino, m_maxbcachebuf, m_maxbcachebuf, 0, CONTENTS); + expect_getattr(ino, m_maxbcachebuf); fd = open(FULLPATH, O_RDONLY); ASSERT_LE(0, fd) << strerror(errno); @@ -581,6 +586,57 @@ TEST_F(ReadCacheable, mmap) ASSERT_NE(MAP_FAILED, p) << strerror(errno); ASSERT_EQ(0, memcmp(p, CONTENTS, bufsize)); + + ASSERT_EQ(0, munmap(p, len)) << strerror(errno); + /* Deliberately leak fd. close(2) will be tested in release.cc */ +} + +/* + * A read via mmap comes up short, indicating that the file was truncated + * server-side. + */ +TEST_F(ReadCacheable, mmap_eof) +{ + const char FULLPATH[] = "mountpoint/some_file.txt"; + const char RELPATH[] = "some_file.txt"; + const char *CONTENTS = "abcdefgh"; + uint64_t ino = 42; + int fd; + ssize_t len; + size_t bufsize = strlen(CONTENTS); + struct stat sb; + void *p; + + len = getpagesize(); + + expect_lookup(RELPATH, ino, 100000); + expect_open(ino, 0, 1); + /* mmap may legitimately try to read more data than is available */ + EXPECT_CALL(*m_mock, process( + ResultOf([=](auto in) { + return (in.header.opcode == FUSE_READ && + in.header.nodeid == ino && + in.body.read.fh == Read::FH && + in.body.read.offset == 0 && + in.body.read.size >= bufsize); + }, Eq(true)), + _) + ).WillOnce(Invoke(ReturnImmediate([=](auto in __unused, auto& out) { + out.header.len = sizeof(struct fuse_out_header) + bufsize; + memmove(out.body.bytes, CONTENTS, bufsize); + }))); + expect_getattr(ino, bufsize); + + fd = open(FULLPATH, O_RDONLY); + ASSERT_LE(0, fd) << strerror(errno); + + p = mmap(NULL, len, PROT_READ, MAP_SHARED, fd, 0); + ASSERT_NE(MAP_FAILED, p) << strerror(errno); + + /* The file size should be automatically truncated */ + ASSERT_EQ(0, memcmp(p, CONTENTS, bufsize)); + ASSERT_EQ(0, fstat(fd, &sb)) << strerror(errno); + EXPECT_EQ((off_t)bufsize, sb.st_size); ASSERT_EQ(0, munmap(p, len)) << strerror(errno); /* Deliberately leak fd. close(2) will be tested in release.cc */ Modified: projects/fuse2/tests/sys/fs/fusefs/write.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/write.cc Tue Jun 25 17:15:44 2019 (r349377) +++ projects/fuse2/tests/sys/fs/fusefs/write.cc Tue Jun 25 17:24:43 2019 (r349378) @@ -528,6 +528,39 @@ TEST_F(Write, rlimit_fsize) /* Deliberately leak fd. close(2) will be tested in release.cc */ } +/* + * A short read indicates EOF. Test that nothing bad happens if we get EOF + * during the R of a RMW operation. + */ +TEST_F(WriteCacheable, eof_during_rmw) +{ + const char FULLPATH[] = "mountpoint/some_file.txt"; + const char RELPATH[] = "some_file.txt"; + const char *CONTENTS = "abcdefgh"; + const char *INITIAL = "XXXXXXXXXX"; + uint64_t ino = 42; + uint64_t offset = 1; + ssize_t bufsize = strlen(CONTENTS); + off_t orig_fsize = 10; + off_t truncated_fsize = 5; + off_t final_fsize = bufsize; + int fd; + + FuseTest::expect_lookup(RELPATH, ino, S_IFREG | 0644, orig_fsize, 1); + expect_open(ino, 0, 1); + expect_read(ino, 0, orig_fsize, truncated_fsize, INITIAL, O_RDWR); + expect_getattr(ino, truncated_fsize); + expect_read(ino, 0, final_fsize, final_fsize, INITIAL, O_RDWR); + maybe_expect_write(ino, offset, bufsize, CONTENTS); + + fd = open(FULLPATH, O_RDWR); + EXPECT_LE(0, fd) << strerror(errno); + + ASSERT_EQ(bufsize, pwrite(fd, CONTENTS, bufsize, offset)) + << strerror(errno); + /* Deliberately leak fd. close(2) will be tested in release.cc */ +} + /* * If the kernel cannot be sure which uid, gid, or pid was responsible for a * write, then it must set the FUSE_WRITE_CACHE bit From owner-svn-src-projects@freebsd.org Tue Jun 25 18:36:12 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8811B15D3A00 for ; Tue, 25 Jun 2019 18:36:12 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2EF626FD0D; Tue, 25 Jun 2019 18:36:12 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 095701C4C1; Tue, 25 Jun 2019 18:36:12 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5PIaB88056546; Tue, 25 Jun 2019 18:36:11 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5PIaBFk056545; Tue, 25 Jun 2019 18:36:11 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906251836.x5PIaBFk056545@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Tue, 25 Jun 2019 18:36:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349382 - in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Commit-Revision: 349382 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 2EF626FD0D X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.96)[-0.959,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jun 2019 18:36:12 -0000 Author: asomers Date: Tue Jun 25 18:36:11 2019 New Revision: 349382 URL: https://svnweb.freebsd.org/changeset/base/349382 Log: fusefs: writes should update the file size, even when data_cache_mode=0 Writes that extend a file should update the file's size. r344185 restricted that behavior for fusefs to only happen when the data cache was enabled. That probably made sense at the time because the attribute cache wasn't fully baked yet. Now that it is, we should always update the cached file size during write. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_io.c projects/fuse2/tests/sys/fs/fusefs/write.cc Modified: projects/fuse2/sys/fs/fuse/fuse_io.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_io.c Tue Jun 25 18:35:23 2019 (r349381) +++ projects/fuse2/sys/fs/fuse/fuse_io.c Tue Jun 25 18:36:11 2019 (r349382) @@ -541,8 +541,7 @@ retry: diff = fwi->size - fwo->size; as_written_offset = uio->uio_offset - diff; - if (as_written_offset - diff > filesize && - fuse_data_cache_mode != FUSE_CACHE_UC) + if (as_written_offset - diff > filesize) fuse_vnode_setsize(vp, as_written_offset); if (as_written_offset - diff >= filesize) fvdat->flag &= ~FN_SIZECHANGE; Modified: projects/fuse2/tests/sys/fs/fusefs/write.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/write.cc Tue Jun 25 18:35:23 2019 (r349381) +++ projects/fuse2/tests/sys/fs/fusefs/write.cc Tue Jun 25 18:36:11 2019 (r349382) @@ -1041,8 +1041,8 @@ TEST_F(WriteThrough, writethrough) /* Deliberately leak fd. close(2) will be tested in release.cc */ } -/* With writethrough caching, writes update the cached file size */ -TEST_F(WriteThrough, update_file_size) +/* Writes that extend a file should update the cached file size */ +TEST_F(Write, update_file_size) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; From owner-svn-src-projects@freebsd.org Tue Jun 25 18:58:53 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3CCFF15D427F for ; Tue, 25 Jun 2019 18:58:53 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D8EC87098E; Tue, 25 Jun 2019 18:58:52 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 973A91C819; Tue, 25 Jun 2019 18:58:52 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5PIwqiH066991; Tue, 25 Jun 2019 18:58:52 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5PIwqk3066989; Tue, 25 Jun 2019 18:58:52 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906251858.x5PIwqk3066989@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Tue, 25 Jun 2019 18:58:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349384 - projects/fuse2/tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/tests/sys/fs/fusefs X-SVN-Commit-Revision: 349384 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: D8EC87098E X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.96)[-0.960,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jun 2019 18:58:53 -0000 Author: asomers Date: Tue Jun 25 18:58:51 2019 New Revision: 349384 URL: https://svnweb.freebsd.org/changeset/base/349384 Log: fusefs: fix the tests for nondefault values of vfs.maxbcachebuf Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/tests/sys/fs/fusefs/read.cc projects/fuse2/tests/sys/fs/fusefs/write.cc Modified: projects/fuse2/tests/sys/fs/fusefs/read.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/read.cc Tue Jun 25 18:47:40 2019 (r349383) +++ projects/fuse2/tests/sys/fs/fusefs/read.cc Tue Jun 25 18:58:51 2019 (r349384) @@ -163,7 +163,7 @@ TEST_F(AioRead, async_read_disabled) ssize_t bufsize = 50; char buf0[bufsize], buf1[bufsize]; off_t off0 = 0; - off_t off1 = 65536; + off_t off1 = m_maxbcachebuf; struct aiocb iocb0, iocb1; volatile sig_atomic_t read_count = 0; @@ -243,13 +243,14 @@ TEST_F(AsyncRead, async_read) ssize_t bufsize = 50; char buf0[bufsize], buf1[bufsize]; off_t off0 = 0; - off_t off1 = 65536; + off_t off1 = m_maxbcachebuf; + off_t fsize = 2 * m_maxbcachebuf; struct aiocb iocb0, iocb1; sem_t sem; ASSERT_EQ(0, sem_init(&sem, 0, 0)) << strerror(errno); - expect_lookup(RELPATH, ino, 131072); + expect_lookup(RELPATH, ino, fsize); expect_open(ino, 0, 1); EXPECT_CALL(*m_mock, process( ResultOf([=](auto in) { Modified: projects/fuse2/tests/sys/fs/fusefs/write.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/write.cc Tue Jun 25 18:47:40 2019 (r349383) +++ projects/fuse2/tests/sys/fs/fusefs/write.cc Tue Jun 25 18:58:51 2019 (r349384) @@ -235,10 +235,13 @@ class WriteCluster: public WriteBack { (public) virtual void SetUp() { if (MAXPHYS < 2 * DFLTPHYS) GTEST_SKIP() << "MAXPHYS must be at least twice DFLTPHYS" - << "for this test"; + << " for this test"; m_async = true; m_maxwrite = MAXPHYS; WriteBack::SetUp(); + if (MAXPHYS < 2 * m_maxbcachebuf) + GTEST_SKIP() << "MAXPHYS must be at least twice maxbcachebuf" + << " for this test"; } }; @@ -619,7 +622,7 @@ TEST_F(WriteThrough, pwrite) const char RELPATH[] = "some_file.txt"; const char *CONTENTS = "abcdefgh"; uint64_t ino = 42; - uint64_t offset = 65536; + uint64_t offset = m_maxbcachebuf; int fd; ssize_t bufsize = strlen(CONTENTS); @@ -767,8 +770,8 @@ TEST_F(WriteCluster, clustering) uint64_t ino = 42; int i, fd; void *wbuf, *wbuf2x; - ssize_t bufsize = 65536; - off_t filesize = 327680; + ssize_t bufsize = m_maxbcachebuf; + off_t filesize = 5 * bufsize; wbuf = malloc(bufsize); ASSERT_NE(NULL, wbuf) << strerror(errno); @@ -814,8 +817,8 @@ TEST_F(WriteCluster, DISABLED_cluster_write_err) uint64_t ino = 42; int i, fd; void *wbuf; - ssize_t bufsize = 65536; - off_t filesize = 262144; + ssize_t bufsize = m_maxbcachebuf; + off_t filesize = 4 * bufsize; wbuf = malloc(bufsize); ASSERT_NE(NULL, wbuf) << strerror(errno); From owner-svn-src-projects@freebsd.org Tue Jun 25 21:21:36 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 010AF15D7E32 for ; Tue, 25 Jun 2019 21:21:36 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9CE9A77757; Tue, 25 Jun 2019 21:21:35 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7959E1E11F; Tue, 25 Jun 2019 21:21:35 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5PLLZek044683; Tue, 25 Jun 2019 21:21:35 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5PLLYN5044678; Tue, 25 Jun 2019 21:21:34 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906252121.x5PLLYN5044678@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Tue, 25 Jun 2019 21:21:34 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349394 - projects/fuse2/tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/tests/sys/fs/fusefs X-SVN-Commit-Revision: 349394 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 9CE9A77757 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.980,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jun 2019 21:21:36 -0000 Author: asomers Date: Tue Jun 25 21:21:34 2019 New Revision: 349394 URL: https://svnweb.freebsd.org/changeset/base/349394 Log: fusefs: fix the tests for non-default values of MAXPHYS Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/tests/sys/fs/fusefs/bmap.cc projects/fuse2/tests/sys/fs/fusefs/read.cc projects/fuse2/tests/sys/fs/fusefs/utils.cc projects/fuse2/tests/sys/fs/fusefs/utils.hh projects/fuse2/tests/sys/fs/fusefs/write.cc Modified: projects/fuse2/tests/sys/fs/fusefs/bmap.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/bmap.cc Tue Jun 25 20:25:16 2019 (r349393) +++ projects/fuse2/tests/sys/fs/fusefs/bmap.cc Tue Jun 25 21:21:34 2019 (r349394) @@ -99,8 +99,8 @@ TEST_F(Bmap, bmap) arg.runb = -1; ASSERT_EQ(0, ioctl(fd, FIOBMAP2, &arg)) << strerror(errno); EXPECT_EQ(arg.bn, pbn); - EXPECT_EQ(arg.runp, MAXPHYS / m_maxbcachebuf - 1); - EXPECT_EQ(arg.runb, MAXPHYS / m_maxbcachebuf - 1); + EXPECT_EQ(arg.runp, m_maxphys / m_maxbcachebuf - 1); + EXPECT_EQ(arg.runb, m_maxphys / m_maxbcachebuf - 1); } /* @@ -134,7 +134,7 @@ TEST_F(Bmap, default_) arg.runb = -1; ASSERT_EQ(0, ioctl(fd, FIOBMAP2, &arg)) << strerror(errno); EXPECT_EQ(arg.bn, 0); - EXPECT_EQ(arg.runp, MAXPHYS / m_maxbcachebuf - 1); + EXPECT_EQ(arg.runp, m_maxphys / m_maxbcachebuf - 1); EXPECT_EQ(arg.runb, 0); /* In the middle */ @@ -144,8 +144,8 @@ TEST_F(Bmap, default_) arg.runb = -1; ASSERT_EQ(0, ioctl(fd, FIOBMAP2, &arg)) << strerror(errno); EXPECT_EQ(arg.bn, lbn * m_maxbcachebuf / DEV_BSIZE); - EXPECT_EQ(arg.runp, MAXPHYS / m_maxbcachebuf - 1); - EXPECT_EQ(arg.runb, MAXPHYS / m_maxbcachebuf - 1); + EXPECT_EQ(arg.runp, m_maxphys / m_maxbcachebuf - 1); + EXPECT_EQ(arg.runb, m_maxphys / m_maxbcachebuf - 1); /* Last block */ lbn = filesize / m_maxbcachebuf - 1; @@ -155,5 +155,5 @@ TEST_F(Bmap, default_) ASSERT_EQ(0, ioctl(fd, FIOBMAP2, &arg)) << strerror(errno); EXPECT_EQ(arg.bn, lbn * m_maxbcachebuf / DEV_BSIZE); EXPECT_EQ(arg.runp, 0); - EXPECT_EQ(arg.runb, MAXPHYS / m_maxbcachebuf - 1); + EXPECT_EQ(arg.runb, m_maxphys / m_maxbcachebuf - 1); } Modified: projects/fuse2/tests/sys/fs/fusefs/read.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/read.cc Tue Jun 25 20:25:16 2019 (r349393) +++ projects/fuse2/tests/sys/fs/fusefs/read.cc Tue Jun 25 21:21:34 2019 (r349394) @@ -110,10 +110,16 @@ virtual void SetUp() { }; class ReadAhead: public ReadCacheable, - public WithParamInterface> + public WithParamInterface> { virtual void SetUp() { - m_maxreadahead = get<1>(GetParam()); + int val; + const char *node = "vfs.maxbcachebuf"; + size_t size = sizeof(val); + ASSERT_EQ(0, sysctlbyname(node, &val, &size, NULL, 0)) + << strerror(errno); + + m_maxreadahead = val * get<1>(GetParam()); m_noclusterr = get<0>(GetParam()); ReadCacheable::SetUp(); } @@ -892,8 +898,8 @@ TEST_P(ReadAhead, readahead) { expect_lookup(RELPATH, ino, filesize); expect_open(ino, 0, 1); maxcontig = m_noclusterr ? m_maxbcachebuf : - m_maxbcachebuf + (int)get<1>(GetParam()); - clustersize = MIN(maxcontig, MAXPHYS); + m_maxbcachebuf + m_maxreadahead; + clustersize = MIN(maxcontig, m_maxphys); for (offs = 0; offs < bufsize; offs += clustersize) { len = std::min((size_t)clustersize, (size_t)(filesize - offs)); expect_read(ino, offs, len, len, contents + offs); @@ -912,10 +918,10 @@ TEST_P(ReadAhead, readahead) { } INSTANTIATE_TEST_CASE_P(RA, ReadAhead, - Values(tuple(false, 0u), - tuple(false, 0x10000), - tuple(false, 0x20000), - tuple(false, 0x30000), - tuple(true, 0u), - tuple(true, 0x10000), - tuple(true, 0x20000))); + Values(tuple(false, 0), + tuple(false, 1), + tuple(false, 2), + tuple(false, 3), + tuple(true, 0), + tuple(true, 1), + tuple(true, 2))); Modified: projects/fuse2/tests/sys/fs/fusefs/utils.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/utils.cc Tue Jun 25 20:25:16 2019 (r349393) +++ projects/fuse2/tests/sys/fs/fusefs/utils.cc Tue Jun 25 21:21:34 2019 (r349394) @@ -93,7 +93,8 @@ class FuseEnv: public Environment { }; void FuseTest::SetUp() { - const char *node = "vfs.maxbcachebuf"; + const char *maxbcachebuf_node = "vfs.maxbcachebuf"; + const char *maxphys_node = "kern.maxphys"; int val = 0; size_t size = sizeof(val); @@ -105,9 +106,12 @@ void FuseTest::SetUp() { if (IsSkipped()) return; - ASSERT_EQ(0, sysctlbyname(node, &val, &size, NULL, 0)) + ASSERT_EQ(0, sysctlbyname(maxbcachebuf_node, &val, &size, NULL, 0)) << strerror(errno); m_maxbcachebuf = val; + ASSERT_EQ(0, sysctlbyname(maxphys_node, &val, &size, NULL, 0)) + << strerror(errno); + m_maxphys = val; try { m_mock = new MockFS(m_maxreadahead, m_allow_other, Modified: projects/fuse2/tests/sys/fs/fusefs/utils.hh ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/utils.hh Tue Jun 25 20:25:16 2019 (r349393) +++ projects/fuse2/tests/sys/fs/fusefs/utils.hh Tue Jun 25 21:21:34 2019 (r349394) @@ -60,6 +60,7 @@ class FuseTest : public ::testing::Test { public: int m_maxbcachebuf; + int m_maxphys; FuseTest(): m_maxreadahead(0), Modified: projects/fuse2/tests/sys/fs/fusefs/write.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/write.cc Tue Jun 25 20:25:16 2019 (r349393) +++ projects/fuse2/tests/sys/fs/fusefs/write.cc Tue Jun 25 21:21:34 2019 (r349394) @@ -233,13 +233,13 @@ virtual void SetUp() { class WriteCluster: public WriteBack { public: virtual void SetUp() { - if (MAXPHYS < 2 * DFLTPHYS) + if (m_maxphys < 2 * DFLTPHYS) GTEST_SKIP() << "MAXPHYS must be at least twice DFLTPHYS" << " for this test"; m_async = true; - m_maxwrite = MAXPHYS; + m_maxwrite = m_maxphys; WriteBack::SetUp(); - if (MAXPHYS < 2 * m_maxbcachebuf) + if (m_maxphys < 2 * m_maxbcachebuf) GTEST_SKIP() << "MAXPHYS must be at least twice maxbcachebuf" << " for this test"; } @@ -678,7 +678,7 @@ TEST_F(Write, write_large) expect_lookup(RELPATH, ino, 0); expect_open(ino, 0, 1); - expect_write(ino, 0, halfbufsize, halfbufsize, contents); + maybe_expect_write(ino, 0, halfbufsize, contents); maybe_expect_write(ino, halfbufsize, halfbufsize, &contents[halfbufsize / sizeof(int)]); From owner-svn-src-projects@freebsd.org Tue Jun 25 23:40:20 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C771315D9C15 for ; Tue, 25 Jun 2019 23:40:20 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6DCC2834A4; Tue, 25 Jun 2019 23:40:20 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5CD7D1F705; Tue, 25 Jun 2019 23:40:20 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5PNeKbp017466; Tue, 25 Jun 2019 23:40:20 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5PNeJGJ017458; Tue, 25 Jun 2019 23:40:19 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906252340.x5PNeJGJ017458@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Tue, 25 Jun 2019 23:40:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349396 - in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Commit-Revision: 349396 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 6DCC2834A4 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.95 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.95)[-0.954,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jun 2019 23:40:21 -0000 Author: asomers Date: Tue Jun 25 23:40:18 2019 New Revision: 349396 URL: https://svnweb.freebsd.org/changeset/base/349396 Log: fusefs: automatically update mtime and ctime on write Writing should implicitly update a file's mtime and ctime. For fuse, the server is supposed to do that. But the client needs to do it too, because the FUSE_WRITE response does not include time attributes, and it's not desirable to issue a GETATTR after every WRITE. When using the writeback cache, there's another hitch: the kernel should ignore the mtime and ctime fields in any GETATTR response for files with a dirty write cache. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c projects/fuse2/sys/fs/fuse/fuse_io.c projects/fuse2/sys/fs/fuse/fuse_node.c projects/fuse2/sys/fs/fuse/fuse_node.h projects/fuse2/tests/sys/fs/fusefs/io.cc projects/fuse2/tests/sys/fs/fusefs/write.cc Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_internal.c Tue Jun 25 21:26:57 2019 (r349395) +++ projects/fuse2/sys/fs/fuse/fuse_internal.c Tue Jun 25 23:40:18 2019 (r349396) @@ -818,6 +818,8 @@ fuse_internal_do_getattr(struct vnode *vp, struct vatt struct fuse_getattr_in *fgai; struct fuse_attr_out *fao; off_t old_filesize = fvdat->cached_attrs.va_size; + struct timespec old_ctime = fvdat->cached_attrs.va_ctime; + struct timespec old_mtime = fvdat->cached_attrs.va_mtime; enum vtype vtyp; int err; @@ -840,6 +842,14 @@ fuse_internal_do_getattr(struct vnode *vp, struct vatt vtyp = IFTOVT(fao->attr.mode); if (fvdat->flag & FN_SIZECHANGE) fao->attr.size = old_filesize; + if (fvdat->flag & FN_CTIMECHANGE) { + fao->attr.ctime = old_ctime.tv_sec; + fao->attr.ctimensec = old_ctime.tv_nsec; + } + if (fvdat->flag & FN_MTIMECHANGE) { + fao->attr.mtime = old_mtime.tv_sec; + fao->attr.mtimensec = old_mtime.tv_nsec; + } fuse_internal_cache_attrs(vp, &fao->attr, fao->attr_valid, fao->attr_valid_nsec, vap); if (vtyp != vnode_vtype(vp)) { @@ -996,6 +1006,7 @@ fuse_internal_send_init(struct fuse_data *data, struct int fuse_internal_setattr(struct vnode *vp, struct vattr *vap, struct thread *td, struct ucred *cred) { + struct fuse_vnode_data *fvdat; struct fuse_dispatcher fdi; struct fuse_setattr_in *fsai; struct mount *mp; @@ -1008,6 +1019,7 @@ int fuse_internal_setattr(struct vnode *vp, struct vat uint64_t newsize = 0; mp = vnode_mount(vp); + fvdat = VTOFUD(vp); data = fuse_get_mpdata(mp); dataflags = data->dataflags; @@ -1057,6 +1069,10 @@ int fuse_internal_setattr(struct vnode *vp, struct vat fsai->valid |= FATTR_MTIME; if (vap->va_vaflags & VA_UTIMES_NULL) fsai->valid |= FATTR_MTIME_NOW; + } else if (fvdat->flag & FN_MTIMECHANGE) { + fsai->mtime = fvdat->cached_attrs.va_mtime.tv_sec; + fsai->mtimensec = fvdat->cached_attrs.va_mtime.tv_nsec; + fsai->valid |= FATTR_MTIME; } if (vap->va_mode != (mode_t)VNOVAL) { fsai->mode = vap->va_mode & ALLPERMS; @@ -1089,6 +1105,7 @@ int fuse_internal_setattr(struct vnode *vp, struct vat } if (err == 0) { struct fuse_attr_out *fao = (struct fuse_attr_out*)fdi.answ; + fuse_vnode_undirty_cached_timestamps(vp); fuse_internal_cache_attrs(vp, &fao->attr, fao->attr_valid, fao->attr_valid_nsec, NULL); } Modified: projects/fuse2/sys/fs/fuse/fuse_io.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_io.c Tue Jun 25 21:26:57 2019 (r349395) +++ projects/fuse2/sys/fs/fuse/fuse_io.c Tue Jun 25 23:40:18 2019 (r349396) @@ -229,6 +229,7 @@ fuse_io_dispatch(struct vnode *vp, struct uio *uio, in } break; case UIO_WRITE: + fuse_vnode_update(vp, FN_MTIMECHANGE | FN_CTIMECHANGE); if (directio) { const int iosize = fuse_iosize(vp); off_t start, end, filesize; @@ -458,6 +459,7 @@ fuse_write_directbackend(struct vnode *vp, struct uio int diff; int err = 0; bool direct_io = fufh->fuse_open_flags & FOPEN_DIRECT_IO; + bool wrote_anything = false; uint32_t write_flags; data = fuse_get_mpdata(vp->v_mount); @@ -533,6 +535,8 @@ retry: break; } else if (err) { break; + } else { + wrote_anything = true; } fwo = ((struct fuse_write_out *)fdi.answ); @@ -585,6 +589,9 @@ retry: } fdisp_destroy(&fdi); + + if (wrote_anything) + fuse_vnode_undirty_cached_timestamps(vp); return (err); } Modified: projects/fuse2/sys/fs/fuse/fuse_node.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_node.c Tue Jun 25 21:26:57 2019 (r349395) +++ projects/fuse2/sys/fs/fuse/fuse_node.c Tue Jun 25 23:40:18 2019 (r349396) @@ -441,3 +441,28 @@ fuse_vnode_size(struct vnode *vp, off_t *filesize, str return error; } + +void +fuse_vnode_undirty_cached_timestamps(struct vnode *vp) +{ + struct fuse_vnode_data *fvdat = VTOFUD(vp); + + fvdat->flag &= ~(FN_MTIMECHANGE | FN_CTIMECHANGE); +} + +/* Update a fuse file's cached timestamps */ +void +fuse_vnode_update(struct vnode *vp, int flags) +{ + struct fuse_vnode_data *fvdat = VTOFUD(vp); + struct timespec ts; + + vfs_timestamp(&ts); + + if (flags & FN_MTIMECHANGE) + fvdat->cached_attrs.va_mtime = ts; + if (flags & FN_CTIMECHANGE) + fvdat->cached_attrs.va_ctime = ts; + + fvdat->flag |= flags; +} Modified: projects/fuse2/sys/fs/fuse/fuse_node.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_node.h Tue Jun 25 21:26:57 2019 (r349395) +++ projects/fuse2/sys/fs/fuse/fuse_node.h Tue Jun 25 23:40:18 2019 (r349396) @@ -66,19 +66,27 @@ #include "fuse_file.h" -#define FN_REVOKED 0x00000020 -#define FN_FLUSHINPROG 0x00000040 -#define FN_FLUSHWANT 0x00000080 +#define FN_REVOKED 0x00000020 +#define FN_FLUSHINPROG 0x00000040 +#define FN_FLUSHWANT 0x00000080 /* * Indicates that the file's size is dirty; the kernel has changed it but not * yet send the change to the daemon. When this bit is set, the - * cache_attrs.va_size field does not time out + * cache_attrs.va_size field does not time out. */ -#define FN_SIZECHANGE 0x00000100 -#define FN_DIRECTIO 0x00000200 +#define FN_SIZECHANGE 0x00000100 +#define FN_DIRECTIO 0x00000200 /* Indicates that parent_nid is valid */ -#define FN_PARENT_NID 0x00000400 +#define FN_PARENT_NID 0x00000400 +/* + * Indicates that the file's cached timestamps are dirty. They will be flushed + * during the next SETATTR or WRITE. Until then, the cached fields will not + * time out. + */ +#define FN_MTIMECHANGE 0x00000800 +#define FN_CTIMECHANGE 0x00001000 + struct fuse_vnode_data { /** self **/ uint64_t nid; @@ -180,4 +188,7 @@ int fuse_vnode_savesize(struct vnode *vp, struct ucred int fuse_vnode_setsize(struct vnode *vp, off_t newsize); +void fuse_vnode_undirty_cached_timestamps(struct vnode *vp); + +void fuse_vnode_update(struct vnode *vp, int flags); #endif /* _FUSE_NODE_H_ */ Modified: projects/fuse2/tests/sys/fs/fusefs/io.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/io.cc Tue Jun 25 21:26:57 2019 (r349395) +++ projects/fuse2/tests/sys/fs/fusefs/io.cc Tue Jun 25 23:40:18 2019 (r349396) @@ -140,10 +140,10 @@ void SetUp() }))); EXPECT_CALL(*m_mock, process( ResultOf([=](auto in) { - uint32_t valid = FATTR_SIZE | FATTR_FH; return (in.header.opcode == FUSE_SETATTR && in.header.nodeid == ino && - in.body.setattr.valid == valid); + (in.body.setattr.valid & FATTR_SIZE)); + }, Eq(true)), _) ).WillRepeatedly(Invoke(ReturnImmediate([=](auto in, auto& out) { Modified: projects/fuse2/tests/sys/fs/fusefs/write.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/write.cc Tue Jun 25 21:26:57 2019 (r349395) +++ projects/fuse2/tests/sys/fs/fusefs/write.cc Tue Jun 25 23:40:18 2019 (r349396) @@ -638,6 +638,35 @@ TEST_F(WriteThrough, pwrite) /* Deliberately leak fd. close(2) will be tested in release.cc */ } +/* Writing a file should update its cached mtime and ctime */ +TEST_F(Write, timestamps) +{ + const char FULLPATH[] = "mountpoint/some_file.txt"; + const char RELPATH[] = "some_file.txt"; + const char *CONTENTS = "abcdefgh"; + ssize_t bufsize = strlen(CONTENTS); + uint64_t ino = 42; + struct stat sb0, sb1; + int fd; + + expect_lookup(RELPATH, ino, 0); + expect_open(ino, 0, 1); + maybe_expect_write(ino, 0, bufsize, CONTENTS); + + fd = open(FULLPATH, O_RDWR); + EXPECT_LE(0, fd) << strerror(errno); + ASSERT_EQ(0, fstat(fd, &sb0)) << strerror(errno); + ASSERT_EQ(bufsize, write(fd, CONTENTS, bufsize)) << strerror(errno); + + nap(); + + ASSERT_EQ(0, fstat(fd, &sb1)) << strerror(errno); + + EXPECT_EQ(sb0.st_atime, sb1.st_atime); + EXPECT_NE(sb0.st_mtime, sb1.st_mtime); + EXPECT_NE(sb0.st_ctime, sb1.st_ctime); +} + TEST_F(Write, write) { const char FULLPATH[] = "mountpoint/some_file.txt"; @@ -1012,6 +1041,99 @@ TEST_F(WriteBackAsync, eof) ASSERT_EQ(0, fstat(fd, &sb)) << strerror(errno); EXPECT_EQ(offset + wbufsize, sb.st_size); /* Deliberately leak fd. close(2) will be tested in release.cc */ +} + +/* + * When a file has dirty writes that haven't been flushed, the server's notion + * of its mtime and ctime will be wrong. The kernel should ignore those if it + * gets them from a FUSE_GETATTR before flushing. + */ +TEST_F(WriteBackAsync, timestamps) +{ + const char FULLPATH[] = "mountpoint/some_file.txt"; + const char RELPATH[] = "some_file.txt"; + const char *CONTENTS = "abcdefgh"; + ssize_t bufsize = strlen(CONTENTS); + uint64_t ino = 42; + uint64_t attr_valid = 0; + uint64_t attr_valid_nsec = 0; + uint64_t server_time = 12345; + mode_t mode = S_IFREG | 0644; + int fd; + + struct stat sb; + + EXPECT_LOOKUP(FUSE_ROOT_ID, RELPATH) + .WillRepeatedly(Invoke( + ReturnImmediate([=](auto in __unused, auto& out) { + SET_OUT_HEADER_LEN(out, entry); + out.body.entry.attr.mode = mode; + out.body.entry.nodeid = ino; + out.body.entry.attr.nlink = 1; + out.body.entry.attr_valid = attr_valid; + out.body.entry.attr_valid_nsec = attr_valid_nsec; + }))); + expect_open(ino, 0, 1); + EXPECT_CALL(*m_mock, process( + ResultOf([=](auto in) { + return (in.header.opcode == FUSE_GETATTR && + in.header.nodeid == ino); + }, Eq(true)), + _) + ).WillRepeatedly(Invoke( + ReturnImmediate([=](auto i __unused, auto& out) { + SET_OUT_HEADER_LEN(out, attr); + out.body.attr.attr.ino = ino; + out.body.attr.attr.mode = mode; + out.body.attr.attr_valid = attr_valid; + out.body.attr.attr_valid_nsec = attr_valid_nsec; + out.body.attr.attr.atime = server_time; + out.body.attr.attr.mtime = server_time; + out.body.attr.attr.ctime = server_time; + }))); + + fd = open(FULLPATH, O_RDWR); + EXPECT_LE(0, fd) << strerror(errno); + ASSERT_EQ(bufsize, write(fd, CONTENTS, bufsize)) << strerror(errno); + + ASSERT_EQ(0, fstat(fd, &sb)) << strerror(errno); + EXPECT_EQ((time_t)server_time, sb.st_atime); + EXPECT_NE((time_t)server_time, sb.st_mtime); + EXPECT_NE((time_t)server_time, sb.st_ctime); +} + +/* Any dirty timestamp fields should be flushed during a SETATTR */ +TEST_F(WriteBackAsync, timestamps_during_setattr) +{ + const char FULLPATH[] = "mountpoint/some_file.txt"; + const char RELPATH[] = "some_file.txt"; + const char *CONTENTS = "abcdefgh"; + ssize_t bufsize = strlen(CONTENTS); + uint64_t ino = 42; + const mode_t newmode = 0755; + int fd; + + expect_lookup(RELPATH, ino, 0); + expect_open(ino, 0, 1); + EXPECT_CALL(*m_mock, process( + ResultOf([=](auto in) { + /* In protocol 7.23, ctime will be changed too */ + uint32_t valid = FATTR_MODE | FATTR_MTIME; + return (in.header.opcode == FUSE_SETATTR && + in.header.nodeid == ino && + in.body.setattr.valid == valid); + }, Eq(true)), + _) + ).WillOnce(Invoke(ReturnImmediate([=](auto in __unused, auto& out) { + SET_OUT_HEADER_LEN(out, attr); + out.body.attr.attr.ino = ino; + out.body.attr.attr.mode = S_IFREG | newmode; + }))); + + fd = open(FULLPATH, O_RDWR); + EXPECT_LE(0, fd) << strerror(errno); + ASSERT_EQ(bufsize, write(fd, CONTENTS, bufsize)) << strerror(errno); + ASSERT_EQ(0, fchmod(fd, newmode)) << strerror(errno); } /* From owner-svn-src-projects@freebsd.org Wed Jun 26 00:03:38 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B7DC15DA784 for ; Wed, 26 Jun 2019 00:03:38 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E49AC842DF; Wed, 26 Jun 2019 00:03:37 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id BDC0B1FC05; Wed, 26 Jun 2019 00:03:37 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5Q03b5c032790; Wed, 26 Jun 2019 00:03:37 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5Q03bHh032788; Wed, 26 Jun 2019 00:03:37 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906260003.x5Q03bHh032788@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Wed, 26 Jun 2019 00:03:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349397 - in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Commit-Revision: 349397 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: E49AC842DF X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.94 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.94)[-0.938,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jun 2019 00:03:38 -0000 Author: asomers Date: Wed Jun 26 00:03:37 2019 New Revision: 349397 URL: https://svnweb.freebsd.org/changeset/base/349397 Log: fusefs: set ctime during FUSE_SETATTR following a write As of r349396 the kernel will internally update the mtime and ctime of files on write. It will also flush the mtime should a SETATTR happen before the data cache gets flushed. Now it will flush the ctime too, if the server is using protocol 7.23 or higher. This is the only case in which the kernel will explicitly set a file's ctime, since neither utimensat(2) nor any other user interfaces allow it. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c projects/fuse2/tests/sys/fs/fusefs/write.cc Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_internal.c Tue Jun 25 23:40:18 2019 (r349396) +++ projects/fuse2/sys/fs/fuse/fuse_internal.c Wed Jun 26 00:03:37 2019 (r349397) @@ -1074,6 +1074,11 @@ int fuse_internal_setattr(struct vnode *vp, struct vat fsai->mtimensec = fvdat->cached_attrs.va_mtime.tv_nsec; fsai->valid |= FATTR_MTIME; } + if (fuse_libabi_geq(data, 7, 23) && fvdat->flag & FN_CTIMECHANGE) { + fsai->ctime = fvdat->cached_attrs.va_ctime.tv_sec; + fsai->ctimensec = fvdat->cached_attrs.va_ctime.tv_nsec; + fsai->valid |= FATTR_CTIME; + } if (vap->va_mode != (mode_t)VNOVAL) { fsai->mode = vap->va_mode & ALLPERMS; fsai->valid |= FATTR_MODE; Modified: projects/fuse2/tests/sys/fs/fusefs/write.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/write.cc Tue Jun 25 23:40:18 2019 (r349396) +++ projects/fuse2/tests/sys/fs/fusefs/write.cc Wed Jun 26 00:03:37 2019 (r349397) @@ -1117,8 +1117,7 @@ TEST_F(WriteBackAsync, timestamps_during_setattr) expect_open(ino, 0, 1); EXPECT_CALL(*m_mock, process( ResultOf([=](auto in) { - /* In protocol 7.23, ctime will be changed too */ - uint32_t valid = FATTR_MODE | FATTR_MTIME; + uint32_t valid = FATTR_MODE | FATTR_MTIME | FATTR_CTIME; return (in.header.opcode == FUSE_SETATTR && in.header.nodeid == ino && in.body.setattr.valid == valid); From owner-svn-src-projects@freebsd.org Wed Jun 26 00:06:42 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3E27615DA835 for ; Wed, 26 Jun 2019 00:06:42 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DA0A48446C; Wed, 26 Jun 2019 00:06:41 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A60261FC06; Wed, 26 Jun 2019 00:06:41 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5Q06fIs032953; Wed, 26 Jun 2019 00:06:41 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5Q06fuQ032952; Wed, 26 Jun 2019 00:06:41 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906260006.x5Q06fuQ032952@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Wed, 26 Jun 2019 00:06:41 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349398 - projects/fuse2/tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/tests/sys/fs/fusefs X-SVN-Commit-Revision: 349398 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: DA0A48446C X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.94 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_SHORT(-0.94)[-0.941,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jun 2019 00:06:42 -0000 Author: asomers Date: Wed Jun 26 00:06:41 2019 New Revision: 349398 URL: https://svnweb.freebsd.org/changeset/base/349398 Log: fusefs: delete obsolete comments in the tests I originally thought that the kernel would be responsible for ctime in protocol 7.23. But now I realize that's not the case. The server is responsible for ctime. The kernel only sets it when there are dirty writes cached, because that's when the server can't. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/tests/sys/fs/fusefs/setattr.cc Modified: projects/fuse2/tests/sys/fs/fusefs/setattr.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/setattr.cc Wed Jun 26 00:03:37 2019 (r349397) +++ projects/fuse2/tests/sys/fs/fusefs/setattr.cc Wed Jun 26 00:06:41 2019 (r349398) @@ -123,7 +123,6 @@ TEST_F(Setattr, chmod) EXPECT_CALL(*m_mock, process( ResultOf([](auto in) { - /* In protocol 7.23, ctime will be changed too */ uint32_t valid = FATTR_MODE; return (in.header.opcode == FUSE_SETATTR && in.header.nodeid == ino && @@ -225,7 +224,6 @@ TEST_F(Setattr, chown) EXPECT_CALL(*m_mock, process( ResultOf([](auto in) { - /* In protocol 7.23, ctime will be changed too */ uint32_t valid = FATTR_GID | FATTR_UID; return (in.header.opcode == FUSE_SETATTR && in.header.nodeid == ino && @@ -308,7 +306,6 @@ TEST_F(Setattr, fchmod) EXPECT_CALL(*m_mock, process( ResultOf([=](auto in) { - /* In protocol 7.23, ctime will be changed too */ uint32_t valid = FATTR_MODE; return (in.header.opcode == FUSE_SETATTR && in.header.nodeid == ino && @@ -362,7 +359,6 @@ TEST_F(Setattr, ftruncate) EXPECT_CALL(*m_mock, process( ResultOf([=](auto in) { - /* In protocol 7.23, ctime will be changed too */ uint32_t valid = FATTR_SIZE | FATTR_FH; return (in.header.opcode == FUSE_SETATTR && in.header.nodeid == ino && @@ -401,7 +397,6 @@ TEST_F(Setattr, truncate) { EXPECT_CALL(*m_mock, process( ResultOf([](auto in) { - /* In protocol 7.23, ctime will be changed too */ uint32_t valid = FATTR_SIZE; return (in.header.opcode == FUSE_SETATTR && in.header.nodeid == ino && @@ -588,7 +583,6 @@ TEST_F(Setattr, utimensat) { EXPECT_CALL(*m_mock, process( ResultOf([=](auto in) { - /* In protocol 7.23, ctime will be changed too */ uint32_t valid = FATTR_ATIME | FATTR_MTIME; return (in.header.opcode == FUSE_SETATTR && in.header.nodeid == ino && @@ -642,7 +636,6 @@ TEST_F(Setattr, utimensat_mtime_only) { EXPECT_CALL(*m_mock, process( ResultOf([=](auto in) { - /* In protocol 7.23, ctime will be changed too */ uint32_t valid = FATTR_MTIME; return (in.header.opcode == FUSE_SETATTR && in.header.nodeid == ino && @@ -705,7 +698,6 @@ TEST_F(Setattr, utimensat_utime_now) { EXPECT_CALL(*m_mock, process( ResultOf([=](auto in) { - /* In protocol 7.23, ctime will be changed too */ uint32_t valid = FATTR_ATIME | FATTR_ATIME_NOW | FATTR_MTIME | FATTR_MTIME_NOW; return (in.header.opcode == FUSE_SETATTR && @@ -770,7 +762,6 @@ TEST_F(Setattr_7_8, chmod) EXPECT_CALL(*m_mock, process( ResultOf([](auto in) { - /* In protocol 7.23, ctime will be changed too */ uint32_t valid = FATTR_MODE; return (in.header.opcode == FUSE_SETATTR && in.header.nodeid == ino && From owner-svn-src-projects@freebsd.org Wed Jun 26 02:09:25 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8CDE815DCD64 for ; Wed, 26 Jun 2019 02:09:25 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 30FD488273; Wed, 26 Jun 2019 02:09:25 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 164EB20FE5; Wed, 26 Jun 2019 02:09:25 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5Q29OBd095907; Wed, 26 Jun 2019 02:09:24 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5Q29NDk095897; Wed, 26 Jun 2019 02:09:23 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906260209.x5Q29NDk095897@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Wed, 26 Jun 2019 02:09:23 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349403 - in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Commit-Revision: 349403 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 30FD488273 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.97)[-0.971,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jun 2019 02:09:25 -0000 Author: asomers Date: Wed Jun 26 02:09:22 2019 New Revision: 349403 URL: https://svnweb.freebsd.org/changeset/base/349403 Log: fusefs: implement the "time_gran" feature. If a server supports a timestamp granularity other than 1ns, it can tell the client this as of protocol 7.23. The client will use that granularity when updating its cached timestamps during write. This way the timestamps won't appear to change following flush. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c projects/fuse2/sys/fs/fuse/fuse_ipc.h projects/fuse2/sys/fs/fuse/fuse_node.c projects/fuse2/tests/sys/fs/fusefs/mockfs.cc projects/fuse2/tests/sys/fs/fusefs/mockfs.hh projects/fuse2/tests/sys/fs/fusefs/utils.cc projects/fuse2/tests/sys/fs/fusefs/utils.hh projects/fuse2/tests/sys/fs/fusefs/write.cc Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_internal.c Wed Jun 26 01:14:39 2019 (r349402) +++ projects/fuse2/sys/fs/fuse/fuse_internal.c Wed Jun 26 02:09:22 2019 (r349403) @@ -934,8 +934,8 @@ fuse_internal_init_callback(struct fuse_ticket *tick, * redundant with max_write */ /* - * max_background, congestion_threshold, and time_gran - * are not implemented + * max_background and congestion_threshold are not + * implemented */ } else { err = EINVAL; @@ -955,6 +955,12 @@ fuse_internal_init_callback(struct fuse_ticket *tick, fsess_set_notimpl(data->mp, FUSE_BMAP); fsess_set_notimpl(data->mp, FUSE_DESTROY); } + + if (fuse_libabi_geq(data, 7, 23) && fiio->time_gran >= 1 && + fiio->time_gran <= 1000000000) + data->time_gran = fiio->time_gran; + else + data->time_gran = 1; out: if (err) { Modified: projects/fuse2/sys/fs/fuse/fuse_ipc.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_ipc.h Wed Jun 26 01:14:39 2019 (r349402) +++ projects/fuse2/sys/fs/fuse/fuse_ipc.h Wed Jun 26 02:09:22 2019 (r349403) @@ -206,6 +206,7 @@ struct fuse_data { struct selinfo ks_rsel; int daemon_timeout; + unsigned time_gran; uint64_t notimpl; uint64_t mnt_flag; }; Modified: projects/fuse2/sys/fs/fuse/fuse_node.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_node.c Wed Jun 26 01:14:39 2019 (r349402) +++ projects/fuse2/sys/fs/fuse/fuse_node.c Wed Jun 26 02:09:22 2019 (r349403) @@ -455,9 +455,13 @@ void fuse_vnode_update(struct vnode *vp, int flags) { struct fuse_vnode_data *fvdat = VTOFUD(vp); + struct fuse_data *data = fuse_get_mpdata(vnode_mount(vp)); struct timespec ts; vfs_timestamp(&ts); + + if (data->time_gran > 1) + ts.tv_nsec = rounddown(ts.tv_nsec, data->time_gran); if (flags & FN_MTIMECHANGE) fvdat->cached_attrs.va_mtime = ts; Modified: projects/fuse2/tests/sys/fs/fusefs/mockfs.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/mockfs.cc Wed Jun 26 01:14:39 2019 (r349402) +++ projects/fuse2/tests/sys/fs/fusefs/mockfs.cc Wed Jun 26 02:09:22 2019 (r349403) @@ -348,7 +348,7 @@ void MockFS::debug_response(const mockfs_buf_out &out) MockFS::MockFS(int max_readahead, bool allow_other, bool default_permissions, bool push_symlinks_in, bool ro, enum poll_method pm, uint32_t flags, uint32_t kernel_minor_version, uint32_t max_write, bool async, - bool noclusterr) + bool noclusterr, unsigned time_gran) { struct sigaction sa; struct iovec *iov = NULL; @@ -362,6 +362,7 @@ MockFS::MockFS(int max_readahead, bool allow_other, bo m_maxwrite = max_write; m_nready = -1; m_pm = pm; + m_time_gran = time_gran; m_quit = false; if (m_pm == KQ) m_kq = kqueue(); @@ -475,6 +476,7 @@ void MockFS::init(uint32_t flags) { if (m_kernel_minor_version < 23) { SET_OUT_HEADER_LEN(*out, init_7_22); } else { + out->body.init.time_gran = m_time_gran; SET_OUT_HEADER_LEN(*out, init); } Modified: projects/fuse2/tests/sys/fs/fusefs/mockfs.hh ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/mockfs.hh Wed Jun 26 01:14:39 2019 (r349402) +++ projects/fuse2/tests/sys/fs/fusefs/mockfs.hh Wed Jun 26 02:09:22 2019 (r349403) @@ -277,6 +277,9 @@ class MockFS { /* Method the daemon should use for I/O to and from /dev/fuse */ enum poll_method m_pm; + /* Timestamp granularity in nanoseconds */ + unsigned m_time_gran; + void debug_request(const mockfs_buf_in&); void debug_response(const mockfs_buf_out&); @@ -320,7 +323,7 @@ class MockFS { bool default_permissions, bool push_symlinks_in, bool ro, enum poll_method pm, uint32_t flags, uint32_t kernel_minor_version, uint32_t max_write, bool async, - bool no_clusterr); + bool no_clusterr, unsigned time_gran); virtual ~MockFS(); Modified: projects/fuse2/tests/sys/fs/fusefs/utils.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/utils.cc Wed Jun 26 01:14:39 2019 (r349402) +++ projects/fuse2/tests/sys/fs/fusefs/utils.cc Wed Jun 26 02:09:22 2019 (r349403) @@ -117,7 +117,7 @@ void FuseTest::SetUp() { m_mock = new MockFS(m_maxreadahead, m_allow_other, m_default_permissions, m_push_symlinks_in, m_ro, m_pm, m_init_flags, m_kernel_minor_version, - m_maxwrite, m_async, m_noclusterr); + m_maxwrite, m_async, m_noclusterr, m_time_gran); /* * FUSE_ACCESS is called almost universally. Expecting it in * each test case would be super-annoying. Instead, set a Modified: projects/fuse2/tests/sys/fs/fusefs/utils.hh ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/utils.hh Wed Jun 26 01:14:39 2019 (r349402) +++ projects/fuse2/tests/sys/fs/fusefs/utils.hh Wed Jun 26 02:09:22 2019 (r349403) @@ -55,6 +55,7 @@ class FuseTest : public ::testing::Test { bool m_ro; bool m_async; bool m_noclusterr; + unsigned m_time_gran; MockFS *m_mock = NULL; const static uint64_t FH = 0xdeadbeef1a7ebabe; @@ -73,7 +74,8 @@ class FuseTest : public ::testing::Test { m_push_symlinks_in(false), m_ro(false), m_async(false), - m_noclusterr(false) + m_noclusterr(false), + m_time_gran(1) {} virtual void SetUp(); Modified: projects/fuse2/tests/sys/fs/fusefs/write.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/write.cc Wed Jun 26 01:14:39 2019 (r349402) +++ projects/fuse2/tests/sys/fs/fusefs/write.cc Wed Jun 26 02:09:22 2019 (r349403) @@ -229,6 +229,14 @@ virtual void SetUp() { } }; +class TimeGran: public WriteBackAsync, public WithParamInterface { +public: +virtual void SetUp() { + m_time_gran = 1 << GetParam(); + WriteBackAsync::SetUp(); +} +}; + /* Tests for clustered writes with WriteBack cacheing */ class WriteCluster: public WriteBack { public: @@ -1134,6 +1142,43 @@ TEST_F(WriteBackAsync, timestamps_during_setattr) ASSERT_EQ(bufsize, write(fd, CONTENTS, bufsize)) << strerror(errno); ASSERT_EQ(0, fchmod(fd, newmode)) << strerror(errno); } + +/* fuse_init_out.time_gran controls the granularity of timestamps */ +TEST_P(TimeGran, timestamps_during_setattr) +{ + const char FULLPATH[] = "mountpoint/some_file.txt"; + const char RELPATH[] = "some_file.txt"; + const char *CONTENTS = "abcdefgh"; + ssize_t bufsize = strlen(CONTENTS); + uint64_t ino = 42; + const mode_t newmode = 0755; + int fd; + + expect_lookup(RELPATH, ino, 0); + expect_open(ino, 0, 1); + EXPECT_CALL(*m_mock, process( + ResultOf([=](auto in) { + uint32_t valid = FATTR_MODE | FATTR_MTIME | FATTR_CTIME; + return (in.header.opcode == FUSE_SETATTR && + in.header.nodeid == ino && + in.body.setattr.valid == valid && + in.body.setattr.mtimensec % m_time_gran == 0 && + in.body.setattr.ctimensec % m_time_gran == 0); + }, Eq(true)), + _) + ).WillOnce(Invoke(ReturnImmediate([=](auto in __unused, auto& out) { + SET_OUT_HEADER_LEN(out, attr); + out.body.attr.attr.ino = ino; + out.body.attr.attr.mode = S_IFREG | newmode; + }))); + + fd = open(FULLPATH, O_RDWR); + EXPECT_LE(0, fd) << strerror(errno); + ASSERT_EQ(bufsize, write(fd, CONTENTS, bufsize)) << strerror(errno); + ASSERT_EQ(0, fchmod(fd, newmode)) << strerror(errno); +} + +INSTANTIATE_TEST_CASE_P(RA, TimeGran, Range(0u, 10u)); /* * Without direct_io, writes should be committed to cache From owner-svn-src-projects@freebsd.org Wed Jun 26 15:15:26 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 906F115CAC1A for ; Wed, 26 Jun 2019 15:15:26 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7BE6C71A29; Wed, 26 Jun 2019 15:15:25 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 484AB1666; Wed, 26 Jun 2019 15:15:25 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5QFFPRF009232; Wed, 26 Jun 2019 15:15:25 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5QFFOnE009231; Wed, 26 Jun 2019 15:15:24 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906261515.x5QFFOnE009231@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Wed, 26 Jun 2019 15:15:24 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349412 - projects/fuse2/sys/fs/fuse X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/sys/fs/fuse X-SVN-Commit-Revision: 349412 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 7BE6C71A29 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.97)[-0.967,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jun 2019 15:15:26 -0000 Author: asomers Date: Wed Jun 26 15:15:24 2019 New Revision: 349412 URL: https://svnweb.freebsd.org/changeset/base/349412 Log: fusefs: delete some unused mount options The fusefs kernel module allegedly supported no_attrcache, no_readahed, no_datacache, no_namecache, and no_mmap mount options, but the mount_fusefs binary never did. So there was no way to ever activate these options. Delete them. Some of them have alternatives: no_attrcache: set the attr_valid time to 0 in FUSE_LOOKUP and FUSE_GETATTR responses. no_readahed: set max_readahead to 0 in the FUSE_INIT response. no_datacache: set the vfs.fusefs.data_cache_mode sysctl to 0, or (coming soon) set the attr_valid time to 0 and set FUSE_AUTO_INVAL_DATA in the FUSE_INIT response. no_namecache: set entry_valid time to 0 in FUSE_LOOKUP and FUSE_GETATTR responses. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_ipc.h projects/fuse2/sys/fs/fuse/fuse_vfsops.c Modified: projects/fuse2/sys/fs/fuse/fuse_ipc.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_ipc.h Wed Jun 26 12:26:38 2019 (r349411) +++ projects/fuse2/sys/fs/fuse/fuse_ipc.h Wed Jun 26 15:15:24 2019 (r349412) @@ -212,25 +212,17 @@ struct fuse_data { }; #define FSESS_DEAD 0x0001 /* session is to be closed */ -#define FSESS_UNUSED0 0x0002 /* unused */ #define FSESS_INITED 0x0004 /* session has been inited */ #define FSESS_DAEMON_CAN_SPY 0x0010 /* let non-owners access this fs */ /* (and being observed by the daemon) */ #define FSESS_PUSH_SYMLINKS_IN 0x0020 /* prefix absolute symlinks with mp */ #define FSESS_DEFAULT_PERMISSIONS 0x0040 /* kernel does permission checking */ -#define FSESS_NO_ATTRCACHE 0x0080 /* no attribute caching */ -#define FSESS_NO_READAHEAD 0x0100 /* no readaheads */ -#define FSESS_NO_DATACACHE 0x0200 /* disable buffer cache */ -#define FSESS_NO_NAMECACHE 0x0400 /* disable name cache */ -#define FSESS_NO_MMAP 0x0800 /* disable mmap */ #define FSESS_ASYNC_READ 0x1000 /* allow multiple reads of some file */ #define FSESS_POSIX_LOCKS 0x2000 /* daemon supports POSIX locks */ #define FSESS_EXPORT_SUPPORT 0x10000 /* daemon supports NFS-style lookups */ #define FSESS_MNTOPTS_MASK ( \ FSESS_DAEMON_CAN_SPY | FSESS_PUSH_SYMLINKS_IN | \ - FSESS_DEFAULT_PERMISSIONS | FSESS_NO_ATTRCACHE | \ - FSESS_NO_READAHEAD | FSESS_NO_DATACACHE | \ - FSESS_NO_NAMECACHE | FSESS_NO_MMAP) + FSESS_DEFAULT_PERMISSIONS) enum fuse_data_cache_mode { FUSE_CACHE_UC, @@ -265,20 +257,13 @@ fsess_set_notimpl(struct mount *mp, int opcode) static inline bool fsess_opt_datacache(struct mount *mp) { - struct fuse_data *data = fuse_get_mpdata(mp); - - return (fuse_data_cache_mode != FUSE_CACHE_UC && - (data->dataflags & FSESS_NO_DATACACHE) == 0); + return (fuse_data_cache_mode != FUSE_CACHE_UC); } static inline bool fsess_opt_mmap(struct mount *mp) { - struct fuse_data *data = fuse_get_mpdata(mp); - - if (fuse_data_cache_mode == FUSE_CACHE_UC) - return (false); - return ((data->dataflags & (FSESS_NO_DATACACHE | FSESS_NO_MMAP)) == 0); + return (fuse_data_cache_mode != FUSE_CACHE_UC); } /* Insert a new upgoing message */ Modified: projects/fuse2/sys/fs/fuse/fuse_vfsops.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_vfsops.c Wed Jun 26 12:26:38 2019 (r349411) +++ projects/fuse2/sys/fs/fuse/fuse_vfsops.c Wed Jun 26 15:15:24 2019 (r349412) @@ -331,11 +331,6 @@ fuse_vfsop_mount(struct mount *mp) FUSE_FLAGOPT(allow_other, FSESS_DAEMON_CAN_SPY); FUSE_FLAGOPT(push_symlinks_in, FSESS_PUSH_SYMLINKS_IN); FUSE_FLAGOPT(default_permissions, FSESS_DEFAULT_PERMISSIONS); - FUSE_FLAGOPT(no_attrcache, FSESS_NO_ATTRCACHE); - FUSE_FLAGOPT(no_readahed, FSESS_NO_READAHEAD); - FUSE_FLAGOPT(no_datacache, FSESS_NO_DATACACHE); - FUSE_FLAGOPT(no_namecache, FSESS_NO_NAMECACHE); - FUSE_FLAGOPT(no_mmap, FSESS_NO_MMAP); (void)vfs_scanopt(opts, "max_read=", "%u", &max_read); if (vfs_scanopt(opts, "timeout=", "%u", &daemon_timeout) == 1) { From owner-svn-src-projects@freebsd.org Wed Jun 26 17:32:34 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7DD1315CDFBF for ; Wed, 26 Jun 2019 17:32:34 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 30F0C8010D; Wed, 26 Jun 2019 17:32:34 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0DD212EB2; Wed, 26 Jun 2019 17:32:34 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5QHWYBj084148; Wed, 26 Jun 2019 17:32:34 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5QHWWFS084138; Wed, 26 Jun 2019 17:32:32 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906261732.x5QHWWFS084138@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Wed, 26 Jun 2019 17:32:32 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349431 - in projects/fuse2: share/man/man5 sys/fs/fuse tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: share/man/man5 sys/fs/fuse tests/sys/fs/fusefs X-SVN-Commit-Revision: 349431 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 30F0C8010D X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.977,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jun 2019 17:32:34 -0000 Author: asomers Date: Wed Jun 26 17:32:31 2019 New Revision: 349431 URL: https://svnweb.freebsd.org/changeset/base/349431 Log: fusefs: implement protocol 7.23's FUSE_WRITEBACK_CACHE option As of protocol 7.23, fuse file systems can specify their cache behavior on a per-mountpoint basis. If they set FUSE_WRITEBACK_CACHE in fuse_init_out.flags, then they'll get the writeback cache. If not, then they'll get the writethrough cache. If they set FOPEN_DIRECT_IO in every FUSE_OPEN response, then they'll get no cache at all. The old vfs.fusefs.data_cache_mode sysctl is ignored for servers that use protocol 7.23 or later. However, it's retained for older servers, especially for those running in jails that lack access to the new protocol. This commit also fixes two other minor test bugs: * WriteCluster:SetUp was using an uninitialized variable. * Read.direct_io_pread wasn't verifying that the cache was actually bypassed. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/share/man/man5/fusefs.5 projects/fuse2/sys/fs/fuse/fuse_internal.c projects/fuse2/sys/fs/fuse/fuse_io.c projects/fuse2/sys/fs/fuse/fuse_ipc.h projects/fuse2/sys/fs/fuse/fuse_node.c projects/fuse2/tests/sys/fs/fusefs/io.cc projects/fuse2/tests/sys/fs/fusefs/notify.cc projects/fuse2/tests/sys/fs/fusefs/read.cc projects/fuse2/tests/sys/fs/fusefs/setattr.cc projects/fuse2/tests/sys/fs/fusefs/write.cc Modified: projects/fuse2/share/man/man5/fusefs.5 ============================================================================== --- projects/fuse2/share/man/man5/fusefs.5 Wed Jun 26 17:28:55 2019 (r349430) +++ projects/fuse2/share/man/man5/fusefs.5 Wed Jun 26 17:32:31 2019 (r349431) @@ -28,7 +28,7 @@ .\" SUCH DAMAGE. .\" .\" $FreeBSD$ -.Dd April 13, 2019 +.Dd June 26, 2019 .Dt FUSEFS 5 .Os .Sh NAME @@ -73,7 +73,7 @@ Minor version of the FUSE kernel ABI supported by this .It Va vfs.fusefs.data_cache_mode Controls how .Nm -will cache file data. +will cache file data for pre-7.23 file systems. A value of 0 will disable caching entirely. Every data access will be forwarded to the daemon. A value of 1 will select write-through caching. @@ -84,6 +84,9 @@ Reads and writes will both be cached, and writes will to the daemon by the page daemon. Write-back caching is usually unsafe, especially for FUSE file systems that require network access. +.Pp +FUSE file systems using protocol 7.23 or later specify their cache behavior +on a per-mountpoint basis, ignoring this sysctl. .It Va vfs.fusefs.lookup_cache_enable Controls whether .Nm Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_internal.c Wed Jun 26 17:28:55 2019 (r349430) +++ projects/fuse2/sys/fs/fuse/fuse_internal.c Wed Jun 26 17:32:31 2019 (r349431) @@ -962,6 +962,13 @@ fuse_internal_init_callback(struct fuse_ticket *tick, else data->time_gran = 1; + if (!fuse_libabi_geq(data, 7, 23)) + data->cache_mode = fuse_data_cache_mode; + else if (fiio->flags & FUSE_WRITEBACK_CACHE) + data->cache_mode = FUSE_CACHE_WB; + else + data->cache_mode = FUSE_CACHE_WT; + out: if (err) { fdata_set_dead(data); @@ -996,9 +1003,18 @@ fuse_internal_send_init(struct fuse_data *data, struct * FUSE_ATOMIC_O_TRUNC: our VFS cannot support it * FUSE_DONT_MASK: unlike Linux, FreeBSD always applies the umask, even * when default ACLs are in use. + * FUSE_SPLICE_WRITE, FUSE_SPLICE_MOVE, FUSE_SPLICE_READ: FreeBSD + * doesn't have splice(2). + * FUSE_FLOCK_LOCKS: not yet implemented + * FUSE_HAS_IOCTL_DIR: not yet implemented + * FUSE_AUTO_INVAL_DATA: not yet implemented + * FUSE_DO_READDIRPLUS: not yet implemented + * FUSE_READDIRPLUS_AUTO: not yet implemented + * FUSE_ASYNC_DIO: not yet implemented + * FUSE_NO_OPEN_SUPPORT: not yet implemented */ fiii->flags = FUSE_ASYNC_READ | FUSE_POSIX_LOCKS | FUSE_EXPORT_SUPPORT - | FUSE_BIG_WRITES; + | FUSE_BIG_WRITES | FUSE_WRITEBACK_CACHE; fuse_insert_callback(fdi.tick, fuse_internal_init_callback); fuse_insert_message(fdi.tick, false); Modified: projects/fuse2/sys/fs/fuse/fuse_io.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_io.c Wed Jun 26 17:28:55 2019 (r349430) +++ projects/fuse2/sys/fs/fuse/fuse_io.c Wed Jun 26 17:32:31 2019 (r349431) @@ -253,7 +253,7 @@ fuse_io_dispatch(struct vnode *vp, struct uio *uio, in } else { SDT_PROBE2(fusefs, , io, trace, 1, "buffered write of vnode"); - if (fuse_data_cache_mode == FUSE_CACHE_WT) + if (!fsess_opt_writeback(vnode_mount(vp))) ioflag |= IO_SYNC; err = fuse_write_biobackend(vp, uio, cred, fufh, ioflag, pid); @@ -481,7 +481,7 @@ fuse_write_directbackend(struct vnode *vp, struct uio write_flags = !pages && ( (ioflag & IO_DIRECT) || !fsess_opt_datacache(vnode_mount(vp)) || - fuse_data_cache_mode != FUSE_CACHE_WB) ? 0 : FUSE_WRITE_CACHE; + !fsess_opt_writeback(vnode_mount(vp))) ? 0 : FUSE_WRITE_CACHE; if (uio->uio_resid == 0) return (0); Modified: projects/fuse2/sys/fs/fuse/fuse_ipc.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_ipc.h Wed Jun 26 17:28:55 2019 (r349430) +++ projects/fuse2/sys/fs/fuse/fuse_ipc.h Wed Jun 26 17:32:31 2019 (r349431) @@ -63,6 +63,12 @@ #include #include +enum fuse_data_cache_mode { + FUSE_CACHE_UC, + FUSE_CACHE_WT, + FUSE_CACHE_WB, +}; + struct fuse_iov { void *base; size_t len; @@ -209,6 +215,7 @@ struct fuse_data { unsigned time_gran; uint64_t notimpl; uint64_t mnt_flag; + enum fuse_data_cache_mode cache_mode; }; #define FSESS_DEAD 0x0001 /* session is to be closed */ @@ -224,12 +231,6 @@ struct fuse_data { FSESS_DAEMON_CAN_SPY | FSESS_PUSH_SYMLINKS_IN | \ FSESS_DEFAULT_PERMISSIONS) -enum fuse_data_cache_mode { - FUSE_CACHE_UC, - FUSE_CACHE_WT, - FUSE_CACHE_WB, -}; - extern int fuse_data_cache_mode; static inline struct fuse_data * @@ -257,13 +258,23 @@ fsess_set_notimpl(struct mount *mp, int opcode) static inline bool fsess_opt_datacache(struct mount *mp) { - return (fuse_data_cache_mode != FUSE_CACHE_UC); + struct fuse_data *data = fuse_get_mpdata(mp); + + return (data->cache_mode != FUSE_CACHE_UC); } static inline bool fsess_opt_mmap(struct mount *mp) { - return (fuse_data_cache_mode != FUSE_CACHE_UC); + return (fsess_opt_datacache(mp)); +} + +static inline bool +fsess_opt_writeback(struct mount *mp) +{ + struct fuse_data *data = fuse_get_mpdata(mp); + + return (data->cache_mode == FUSE_CACHE_WB); } /* Insert a new upgoing message */ Modified: projects/fuse2/sys/fs/fuse/fuse_node.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_node.c Wed Jun 26 17:28:55 2019 (r349430) +++ projects/fuse2/sys/fs/fuse/fuse_node.c Wed Jun 26 17:32:31 2019 (r349431) @@ -108,6 +108,16 @@ SYSCTL_INT(_vfs_fusefs, OID_AUTO, node_count, CTLFLAG_ int fuse_data_cache_mode = FUSE_CACHE_WT; +/* + * DEPRECATED + * This sysctl is no longer needed as of fuse protocol 7.23. Individual + * servers can select the cache behavior they need for each mountpoint: + * - writethrough: the default + * - writeback: set FUSE_WRITEBACK_CACHE in fuse_init_out.flags + * - uncached: set FOPEN_DIRECT_IO for every file + * The sysctl is retained primarily for use by jails supporting older FUSE + * protocols. It may be removed entirely once FreeBSD 11.3 and 12.0 are EOL. + */ SYSCTL_PROC(_vfs_fusefs, OID_AUTO, data_cache_mode, CTLTYPE_INT|CTLFLAG_RW, &fuse_data_cache_mode, 0, sysctl_fuse_cache_mode, "I", "Zero: disable caching of FUSE file data; One: write-through caching " Modified: projects/fuse2/tests/sys/fs/fusefs/io.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/io.cc Wed Jun 26 17:28:55 2019 (r349430) +++ projects/fuse2/tests/sys/fs/fusefs/io.cc Wed Jun 26 17:32:31 2019 (r349431) @@ -77,7 +77,7 @@ static void compare(const void *tbuf, const void *cont } class Io: public FuseTest, - public WithParamInterface> { + public WithParamInterface> { public: int m_backing_fd, m_control_fd, m_test_fd; off_t m_filesize; @@ -95,9 +95,12 @@ void SetUp() FAIL() << strerror(errno); srandom(22'9'1982); // Seed with my birthday - m_init_flags = get<0>(GetParam()); + if (get<0>(GetParam())) + m_init_flags |= FUSE_ASYNC_READ; m_maxwrite = get<1>(GetParam()); - m_async = get<2>(GetParam()); + if (get<2>(GetParam())) + m_init_flags |= FUSE_WRITEBACK_CACHE; + m_async = get<3>(GetParam()); FuseTest::SetUp(); if (IsSkipped()) @@ -316,15 +319,6 @@ void do_write(ssize_t size, off_t offs) class IoCacheable: public Io { public: virtual void SetUp() { - const char *node = "vfs.fusefs.data_cache_mode"; - int val = 0; - size_t size = sizeof(val); - - ASSERT_EQ(0, sysctlbyname(node, &val, &size, NULL, 0)) - << strerror(errno); - if (val == 0) - GTEST_SKIP() << - "fusefs data caching must be enabled for this test"; Io::SetUp(); } }; @@ -492,11 +486,13 @@ TEST_P(Io, resize_a_valid_buffer_while_extending) } INSTANTIATE_TEST_CASE_P(Io, Io, - Combine(Values(0, FUSE_ASYNC_READ), /* m_init_flags */ + Combine(Bool(), /* async read */ Values(0x1000, 0x10000, 0x20000), /* m_maxwrite */ + Bool(), /* writeback cache */ Bool())); /* m_async */ INSTANTIATE_TEST_CASE_P(Io, IoCacheable, - Combine(Values(0, FUSE_ASYNC_READ), /* m_init_flags */ + Combine(Bool(), /* async read */ Values(0x1000, 0x10000, 0x20000), /* m_maxwrite */ + Bool(), /* writeback cache */ Bool())); /* m_async */ Modified: projects/fuse2/tests/sys/fs/fusefs/notify.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/notify.cc Wed Jun 26 17:28:55 2019 (r349430) +++ projects/fuse2/tests/sys/fs/fusefs/notify.cc Wed Jun 26 17:32:31 2019 (r349431) @@ -84,20 +84,11 @@ void expect_lookup(uint64_t parent, const char *relpat class NotifyWriteback: public Notify { public: virtual void SetUp() { - const char *node = "vfs.fusefs.data_cache_mode"; - int val = 0; - size_t size = sizeof(val); - + m_init_flags |= FUSE_WRITEBACK_CACHE; m_async = true; Notify::SetUp(); if (IsSkipped()) return; - - ASSERT_EQ(0, sysctlbyname(node, &val, &size, NULL, 0)) - << strerror(errno); - if (val != 2) - GTEST_SKIP() << "vfs.fusefs.data_cache_mode must be set to 2 " - "(writeback) for this test"; } void expect_write(uint64_t ino, uint64_t offset, uint64_t size, Modified: projects/fuse2/tests/sys/fs/fusefs/read.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/read.cc Wed Jun 26 17:28:55 2019 (r349430) +++ projects/fuse2/tests/sys/fs/fusefs/read.cc Wed Jun 26 17:32:31 2019 (r349431) @@ -92,24 +92,7 @@ class AsyncRead: public AioRead { } }; -class ReadCacheable: public Read { -public: -virtual void SetUp() { - const char *node = "vfs.fusefs.data_cache_mode"; - int val = 0; - size_t size = sizeof(val); - - FuseTest::SetUp(); - - ASSERT_EQ(0, sysctlbyname(node, &val, &size, NULL, 0)) - << strerror(errno); - if (val == 0) - GTEST_SKIP() << - "fusefs data caching must be enabled for this test"; -} -}; - -class ReadAhead: public ReadCacheable, +class ReadAhead: public Read, public WithParamInterface> { virtual void SetUp() { @@ -121,7 +104,7 @@ class ReadAhead: public ReadCacheable, m_maxreadahead = val * get<1>(GetParam()); m_noclusterr = get<0>(GetParam()); - ReadCacheable::SetUp(); + Read::SetUp(); } }; @@ -359,6 +342,12 @@ TEST_F(Read, direct_io_pread) ASSERT_EQ(bufsize, pread(fd, buf, bufsize, offset)) << strerror(errno); ASSERT_EQ(0, memcmp(buf, CONTENTS, bufsize)); + + // With FOPEN_DIRECT_IO, the cache should be bypassed. The server will + // get a 2nd read request. + expect_read(ino, offset, bufsize, bufsize, CONTENTS); + ASSERT_EQ(bufsize, pread(fd, buf, bufsize, offset)) << strerror(errno); + ASSERT_EQ(0, memcmp(buf, CONTENTS, bufsize)); /* Deliberately leak fd. close(2) will be tested in release.cc */ } @@ -423,7 +412,7 @@ TEST_F(Read, eio) * indicates EOF, because of a server-side truncation. We should invalidate * all cached attributes. We may update the file size, */ -TEST_F(ReadCacheable, eof) +TEST_F(Read, eof) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; @@ -453,8 +442,8 @@ TEST_F(ReadCacheable, eof) /* Deliberately leak fd. close(2) will be tested in release.cc */ } -/* Like ReadCacheable.eof, but causes an entire buffer to be invalidated */ -TEST_F(ReadCacheable, eof_of_whole_buffer) +/* Like Read.eof, but causes an entire buffer to be invalidated */ +TEST_F(Read, eof_of_whole_buffer) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; @@ -490,7 +479,7 @@ TEST_F(ReadCacheable, eof_of_whole_buffer) * With the keep_cache option, the kernel may keep its read cache across * multiple open(2)s. */ -TEST_F(ReadCacheable, keep_cache) +TEST_F(Read, keep_cache) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; @@ -556,7 +545,7 @@ TEST_F(Read, keep_cache_disabled) /* Deliberately leak fd0 and fd1. */ } -TEST_F(ReadCacheable, mmap) +TEST_F(Read, mmap) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; @@ -602,7 +591,7 @@ TEST_F(ReadCacheable, mmap) * A read via mmap comes up short, indicating that the file was truncated * server-side. */ -TEST_F(ReadCacheable, mmap_eof) +TEST_F(Read, mmap_eof) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; @@ -680,7 +669,7 @@ TEST_F(Read, o_direct) ASSERT_EQ(0, lseek(fd, 0, SEEK_SET)) << strerror(errno); ASSERT_EQ(bufsize, read(fd, buf, bufsize)) << strerror(errno); ASSERT_EQ(0, memcmp(buf, CONTENTS, bufsize)); - + /* Deliberately leak fd. close(2) will be tested in release.cc */ } @@ -761,7 +750,7 @@ TEST_F(Read_7_8, read) * If cacheing is enabled, the kernel should try to read an entire cache block * at a time. */ -TEST_F(ReadCacheable, cache_block) +TEST_F(Read, cache_block) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; @@ -796,7 +785,7 @@ TEST_F(ReadCacheable, cache_block) } /* Reading with sendfile should work (though it obviously won't be 0-copy) */ -TEST_F(ReadCacheable, sendfile) +TEST_F(Read, sendfile) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; @@ -843,7 +832,7 @@ TEST_F(ReadCacheable, sendfile) /* sendfile should fail gracefully if fuse declines the read */ /* https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=236466 */ -TEST_F(ReadCacheable, DISABLED_sendfile_eio) +TEST_F(Read, DISABLED_sendfile_eio) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; Modified: projects/fuse2/tests/sys/fs/fusefs/setattr.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/setattr.cc Wed Jun 26 17:28:55 2019 (r349430) +++ projects/fuse2/tests/sys/fs/fusefs/setattr.cc Wed Jun 26 17:32:31 2019 (r349431) @@ -415,8 +415,7 @@ TEST_F(Setattr, truncate) { /* * Truncating a file should discard cached data past the truncation point. - * This is a regression test for bug 233783. The bug only applies when - * vfs.fusefs.data_cache_mode=1 or 2, but the test should pass regardless. + * This is a regression test for bug 233783. * * There are two distinct failure modes. The first one is a failure to zero * the portion of the file's final buffer past EOF. It can be reproduced by @@ -476,11 +475,6 @@ TEST_F(Setattr, truncate_discards_cached_data) { out.body.attr.attr.mode = mode; out.body.attr.attr.size = cur_size; }))); - /* - * The exact pattern of FUSE_WRITE operations depends on the setting of - * vfs.fusefs.data_cache_mode. But it's not important for this test. - * Just set the mocks to accept anything - */ EXPECT_CALL(*m_mock, process( ResultOf([=](auto in) { return (in.header.opcode == FUSE_WRITE); @@ -510,7 +504,6 @@ TEST_F(Setattr, truncate_discards_cached_data) { cur_size = trunc_size; }))); - /* exact pattern of FUSE_READ depends on vfs.fusefs.data_cache_mode */ EXPECT_CALL(*m_mock, process( ResultOf([=](auto in) { return (in.header.opcode == FUSE_READ); @@ -534,7 +527,7 @@ TEST_F(Setattr, truncate_discards_cached_data) { ASSERT_EQ(static_cast(w0_size), pwrite(fd, w0buf, w0_size, w0_offset)); should_have_data = true; - /* Fill the cache, if data_cache_mode == 1 */ + /* Fill the cache */ ASSERT_EQ(static_cast(r0_size), pread(fd, r0buf, r0_size, r0_offset)); /* 1st truncate should discard cached data */ Modified: projects/fuse2/tests/sys/fs/fusefs/write.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/write.cc Wed Jun 26 17:28:55 2019 (r349430) +++ projects/fuse2/tests/sys/fs/fusefs/write.cc Wed Jun 26 17:32:31 2019 (r349431) @@ -117,23 +117,6 @@ void maybe_expect_write(uint64_t ino, uint64_t offset, }; -class WriteCacheable: public Write { -public: -virtual void SetUp() { - const char *node = "vfs.fusefs.data_cache_mode"; - int val = 0; - size_t size = sizeof(val); - - FuseTest::SetUp(); - - ASSERT_EQ(0, sysctlbyname(node, &val, &size, NULL, 0)) - << strerror(errno); - if (val == 0) - GTEST_SKIP() << - "fusefs data caching must be enabled for this test"; -} -}; - sig_atomic_t Write::s_sigxfsz = 0; class Write_7_8: public FuseTest { @@ -167,50 +150,14 @@ virtual void SetUp() { } }; -/* Tests for the write-through cache mode */ -class WriteThrough: public Write { -public: -virtual void SetUp() { - const char *cache_mode_node = "vfs.fusefs.data_cache_mode"; - int val = 0; - size_t size = sizeof(val); - - FuseTest::SetUp(); - if (IsSkipped()) - return; - - ASSERT_EQ(0, sysctlbyname(cache_mode_node, &val, &size, NULL, 0)) - << strerror(errno); - if (val != 1) - GTEST_SKIP() << "vfs.fusefs.data_cache_mode must be set to 1 " - "(writethrough) for this test"; -} - -void expect_write(uint64_t ino, uint64_t offset, uint64_t isize, - uint64_t osize, const void *contents) -{ - FuseTest::expect_write(ino, offset, isize, osize, 0, FUSE_WRITE_CACHE, - contents); -} -}; - /* Tests for the writeback cache mode */ class WriteBack: public Write { public: virtual void SetUp() { - const char *node = "vfs.fusefs.data_cache_mode"; - int val = 0; - size_t size = sizeof(val); - + m_init_flags |= FUSE_WRITEBACK_CACHE; FuseTest::SetUp(); if (IsSkipped()) return; - - ASSERT_EQ(0, sysctlbyname(node, &val, &size, NULL, 0)) - << strerror(errno); - if (val != 2) - GTEST_SKIP() << "vfs.fusefs.data_cache_mode must be set to 2 " - "(writeback) for this test"; } void expect_write(uint64_t ino, uint64_t offset, uint64_t isize, @@ -241,12 +188,12 @@ virtual void SetUp() { class WriteCluster: public WriteBack { public: virtual void SetUp() { - if (m_maxphys < 2 * DFLTPHYS) - GTEST_SKIP() << "MAXPHYS must be at least twice DFLTPHYS" - << " for this test"; m_async = true; m_maxwrite = m_maxphys; WriteBack::SetUp(); + if (m_maxphys < 2 * DFLTPHYS) + GTEST_SKIP() << "MAXPHYS must be at least twice DFLTPHYS" + << " for this test"; if (m_maxphys < 2 * m_maxbcachebuf) GTEST_SKIP() << "MAXPHYS must be at least twice maxbcachebuf" << " for this test"; @@ -543,7 +490,7 @@ TEST_F(Write, rlimit_fsize) * A short read indicates EOF. Test that nothing bad happens if we get EOF * during the R of a RMW operation. */ -TEST_F(WriteCacheable, eof_during_rmw) +TEST_F(Write, eof_during_rmw) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; @@ -577,7 +524,7 @@ TEST_F(WriteCacheable, eof_during_rmw) * write, then it must set the FUSE_WRITE_CACHE bit */ /* https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=236378 */ -TEST_F(WriteCacheable, mmap) +TEST_F(Write, mmap) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; @@ -624,7 +571,7 @@ TEST_F(WriteCacheable, mmap) free(zeros); } -TEST_F(WriteThrough, pwrite) +TEST_F(Write, pwrite) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; @@ -1183,7 +1130,7 @@ INSTANTIATE_TEST_CASE_P(RA, TimeGran, Range(0u, 10u)); /* * Without direct_io, writes should be committed to cache */ -TEST_F(WriteThrough, writethrough) +TEST_F(Write, writethrough) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; From owner-svn-src-projects@freebsd.org Wed Jun 26 19:10:40 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 042D915CFB2E for ; Wed, 26 Jun 2019 19:10:40 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 95E7C83D2B; Wed, 26 Jun 2019 19:10:39 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 72300428B; Wed, 26 Jun 2019 19:10:39 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5QJAda4031590; Wed, 26 Jun 2019 19:10:39 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5QJAd7b031589; Wed, 26 Jun 2019 19:10:39 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906261910.x5QJAd7b031589@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Wed, 26 Jun 2019 19:10:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349436 - projects/fuse2/tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/tests/sys/fs/fusefs X-SVN-Commit-Revision: 349436 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 95E7C83D2B X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.96)[-0.958,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jun 2019 19:10:40 -0000 Author: asomers Date: Wed Jun 26 19:10:39 2019 New Revision: 349436 URL: https://svnweb.freebsd.org/changeset/base/349436 Log: fusefs: run the io tests with direct io, too Now the io tests are run in all cache modes. The fusefs test suite can now get adequate coverage without changing the value of vfs.fusefs.data_cache_mode, which is only needed for legacy file systems now. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/tests/sys/fs/fusefs/io.cc Modified: projects/fuse2/tests/sys/fs/fusefs/io.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/io.cc Wed Jun 26 17:42:47 2019 (r349435) +++ projects/fuse2/tests/sys/fs/fusefs/io.cc Wed Jun 26 19:10:39 2019 (r349436) @@ -51,6 +51,28 @@ extern "C" { using namespace testing; +enum cache_mode { + Uncached, + Writethrough, + Writeback, + WritebackAsync +}; + +const char *cache_mode_to_s(enum cache_mode cm) { + switch (cm) { + case Uncached: + return "Uncached"; + case Writethrough: + return "Writethrough"; + case Writeback: + return "Writeback"; + case WritebackAsync: + return "WritebackAsync"; + default: + return "Unknown"; + } +} + const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; const uint64_t ino = 42; @@ -76,13 +98,15 @@ static void compare(const void *tbuf, const void *cont } } -class Io: public FuseTest, - public WithParamInterface> { +typedef tuple IoParam; + +class Io: public FuseTest, public WithParamInterface { public: int m_backing_fd, m_control_fd, m_test_fd; off_t m_filesize; +bool m_direct_io; -Io(): m_backing_fd(-1), m_control_fd(-1) {}; +Io(): m_backing_fd(-1), m_control_fd(-1), m_direct_io(false) {}; void SetUp() { @@ -98,16 +122,35 @@ void SetUp() if (get<0>(GetParam())) m_init_flags |= FUSE_ASYNC_READ; m_maxwrite = get<1>(GetParam()); - if (get<2>(GetParam())) - m_init_flags |= FUSE_WRITEBACK_CACHE; - m_async = get<3>(GetParam()); + switch (get<2>(GetParam())) { + case Uncached: + m_direct_io = true; + break; + case WritebackAsync: + m_async = true; + /* FALLTHROUGH */ + case Writeback: + m_init_flags |= FUSE_WRITEBACK_CACHE; + /* FALLTHROUGH */ + case Writethrough: + break; + default: + FAIL() << "Unknown cache mode"; + } FuseTest::SetUp(); if (IsSkipped()) return; + if (verbosity > 0) { + printf("Test Parameters: init_flags=%#x maxwrite=%#x " + "%sasync cache=%s\n", + m_init_flags, m_maxwrite, m_async? "" : "no", + cache_mode_to_s(get<2>(GetParam()))); + } + expect_lookup(RELPATH, ino, S_IFREG | 0644, 0, 1); - expect_open(ino, 0, 1); + expect_open(ino, m_direct_io ? FOPEN_DIRECT_IO : 0, 1); EXPECT_CALL(*m_mock, process( ResultOf([=](auto in) { return (in.header.opcode == FUSE_WRITE && @@ -488,11 +531,13 @@ TEST_P(Io, resize_a_valid_buffer_while_extending) INSTANTIATE_TEST_CASE_P(Io, Io, Combine(Bool(), /* async read */ Values(0x1000, 0x10000, 0x20000), /* m_maxwrite */ - Bool(), /* writeback cache */ - Bool())); /* m_async */ + Values(Uncached, Writethrough, Writeback, WritebackAsync) + ) +); INSTANTIATE_TEST_CASE_P(Io, IoCacheable, Combine(Bool(), /* async read */ Values(0x1000, 0x10000, 0x20000), /* m_maxwrite */ - Bool(), /* writeback cache */ - Bool())); /* m_async */ + Values(Writethrough, Writeback, WritebackAsync) + ) +); From owner-svn-src-projects@freebsd.org Wed Jun 26 20:07:17 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 520FA15D0DF3 for ; Wed, 26 Jun 2019 20:07:17 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E996A85D83; Wed, 26 Jun 2019 20:07:16 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C18984D53; Wed, 26 Jun 2019 20:07:16 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5QK7G8N062963; Wed, 26 Jun 2019 20:07:16 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5QK7GgO062962; Wed, 26 Jun 2019 20:07:16 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906262007.x5QK7GgO062962@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Wed, 26 Jun 2019 20:07:16 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349437 - projects/fuse2/share/man/man5 X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/share/man/man5 X-SVN-Commit-Revision: 349437 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: E996A85D83 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.97)[-0.972,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jun 2019 20:07:17 -0000 Author: asomers Date: Wed Jun 26 20:07:16 2019 New Revision: 349437 URL: https://svnweb.freebsd.org/changeset/base/349437 Log: fusefs.5: remove deleted sysctls from man page Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/share/man/man5/fusefs.5 Modified: projects/fuse2/share/man/man5/fusefs.5 ============================================================================== --- projects/fuse2/share/man/man5/fusefs.5 Wed Jun 26 19:10:39 2019 (r349436) +++ projects/fuse2/share/man/man5/fusefs.5 Wed Jun 26 20:07:16 2019 (r349437) @@ -87,13 +87,6 @@ require network access. .Pp FUSE file systems using protocol 7.23 or later specify their cache behavior on a per-mountpoint basis, ignoring this sysctl. -.It Va vfs.fusefs.lookup_cache_enable -Controls whether -.Nm -will cache lookup responses from the file system. -FUSE file systems indicate whether lookup responses should be cacheable, but -it may be useful to globally disable caching them if a file system is -misbehaving. .\" Undocumented sysctls .\" ==================== .\" Counters: I intend to rename to vfs.fusefs.stats.* for clarity @@ -102,15 +95,10 @@ misbehaving. .\" vfs.fusefs.ticker_count .\" vfs.fusefs.node_count .\" -.\" vfs.fusefs.version - useless since the driver moved in-tree .\" vfs.fusefs.reclaim_revoked: I don't understand it well-enough .\" vfs.fusefs.enforce_dev_perms: I don't understand it well enough. .\" vfs.fusefs.iov_credit: I don't understand it well enough .\" vfs.fusefs.iov_permanent_bufsize: I don't understand it well enough -.\" vfs.fusefs.fix_broken_io: I don't understand it well enough -.\" vfs.fusefs.refresh_size: probably useless? -.\" vfs.fusefs.mmap_enable: why is this optional? -.\" vfs.fusefs.data_cache_invalidate: what is this needed for? .Sh SEE ALSO .Xr mount_fusefs 8 .Sh HISTORY From owner-svn-src-projects@freebsd.org Wed Jun 26 20:26:01 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6639715D14F2 for ; Wed, 26 Jun 2019 20:26:01 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1A9E586AF9; Wed, 26 Jun 2019 20:26:01 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CEAEA50CA; Wed, 26 Jun 2019 20:26:00 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5QKQ03O073367; Wed, 26 Jun 2019 20:26:00 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5QKPvJ9073346; Wed, 26 Jun 2019 20:25:57 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906262025.x5QKPvJ9073346@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Wed, 26 Jun 2019 20:25:57 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349440 - projects/fuse2/tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/tests/sys/fs/fusefs X-SVN-Commit-Revision: 349440 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 1A9E586AF9 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.98)[-0.977,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jun 2019 20:26:01 -0000 Author: asomers Date: Wed Jun 26 20:25:57 2019 New Revision: 349440 URL: https://svnweb.freebsd.org/changeset/base/349440 Log: fusefs: annotate deliberate file descriptor leaks in the tests closing a file descriptor causes FUSE activity that is superfluous to the purpose of most tests, but would nonetheless require matching expectations. Rather than do that, most tests deliberately leak file descriptors instead. This commit moves the leakage from each test into two trivial functions: leak and leakdir. Hopefully Coverity will only complain about those functions and not all of their callers. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/tests/sys/fs/fusefs/allow_other.cc projects/fuse2/tests/sys/fs/fusefs/create.cc projects/fuse2/tests/sys/fs/fusefs/default_permissions.cc projects/fuse2/tests/sys/fs/fusefs/fifo.cc projects/fuse2/tests/sys/fs/fusefs/flush.cc projects/fuse2/tests/sys/fs/fusefs/fsync.cc projects/fuse2/tests/sys/fs/fusefs/fsyncdir.cc projects/fuse2/tests/sys/fs/fusefs/io.cc projects/fuse2/tests/sys/fs/fusefs/locks.cc projects/fuse2/tests/sys/fs/fusefs/nfs.cc projects/fuse2/tests/sys/fs/fusefs/notify.cc projects/fuse2/tests/sys/fs/fusefs/open.cc projects/fuse2/tests/sys/fs/fusefs/read.cc projects/fuse2/tests/sys/fs/fusefs/readdir.cc projects/fuse2/tests/sys/fs/fusefs/setattr.cc projects/fuse2/tests/sys/fs/fusefs/unlink.cc projects/fuse2/tests/sys/fs/fusefs/utils.hh projects/fuse2/tests/sys/fs/fusefs/write.cc Modified: projects/fuse2/tests/sys/fs/fusefs/allow_other.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/allow_other.cc Wed Jun 26 20:19:48 2019 (r349439) +++ projects/fuse2/tests/sys/fs/fusefs/allow_other.cc Wed Jun 26 20:25:57 2019 (r349440) @@ -182,11 +182,12 @@ TEST_F(AllowOther, privilege_escalation) strerror(errno)); return 1; } + leak(fd0); return 0; } ); ASSERT_EQ(0, WEXITSTATUS(status)); - /* Deliberately leak fd1. close(2) will be tested in release.cc */ + leak(fd1); } TEST_F(NoAllowOther, disallowed) Modified: projects/fuse2/tests/sys/fs/fusefs/create.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/create.cc Wed Jun 26 20:19:48 2019 (r349439) +++ projects/fuse2/tests/sys/fs/fusefs/create.cc Wed Jun 26 20:25:57 2019 (r349440) @@ -142,7 +142,7 @@ TEST_F(Create, attr_cache) fd = open(FULLPATH, O_CREAT | O_EXCL, mode); EXPECT_LE(0, fd) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* A successful CREATE operation should purge the parent dir's attr cache */ @@ -185,7 +185,7 @@ TEST_F(Create, clear_attr_cache) EXPECT_LE(0, fd) << strerror(errno); EXPECT_EQ(0, stat("mountpoint", &sb)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -253,7 +253,7 @@ TEST_F(Create, Enosys) fd = open(FULLPATH, O_CREAT | O_EXCL, mode); EXPECT_LE(0, fd) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -287,7 +287,7 @@ TEST_F(Create, entry_cache_negative) fd = open(FULLPATH, O_CREAT | O_EXCL, mode); ASSERT_LE(0, fd) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -323,7 +323,7 @@ TEST_F(Create, entry_cache_negative_purge) expect_lookup(RELPATH, ino, S_IFREG | mode, 0, 1); ASSERT_EQ(0, access(FULLPATH, F_OK)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -365,7 +365,7 @@ TEST_F(Create, ok) fd = open(FULLPATH, O_CREAT | O_EXCL, mode); EXPECT_LE(0, fd) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -397,7 +397,7 @@ TEST_F(Create, wronly_0444) fd = open(FULLPATH, O_CREAT | O_WRONLY, mode); EXPECT_LE(0, fd) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } TEST_F(Create_7_8, ok) @@ -421,7 +421,7 @@ TEST_F(Create_7_8, ok) fd = open(FULLPATH, O_CREAT | O_EXCL, mode); EXPECT_LE(0, fd) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } TEST_F(Create_7_11, ok) @@ -445,5 +445,5 @@ TEST_F(Create_7_11, ok) fd = open(FULLPATH, O_CREAT | O_EXCL, mode); EXPECT_LE(0, fd) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } Modified: projects/fuse2/tests/sys/fs/fusefs/default_permissions.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/default_permissions.cc Wed Jun 26 20:19:48 2019 (r349439) +++ projects/fuse2/tests/sys/fs/fusefs/default_permissions.cc Wed Jun 26 20:25:57 2019 (r349440) @@ -488,7 +488,7 @@ TEST_F(Create, ok) fd = open(FULLPATH, O_CREAT | O_EXCL, 0644); EXPECT_LE(0, fd) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } TEST_F(Create, eacces) @@ -763,7 +763,7 @@ TEST_F(Open, ok) fd = open(FULLPATH, O_RDONLY); EXPECT_LE(0, fd) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } TEST_F(Rename, eacces_on_srcdir) @@ -1024,7 +1024,7 @@ TEST_F(Setattr, ftruncate_of_newly_created_file) fd = open(FULLPATH, O_CREAT | O_RDWR, 0); ASSERT_LE(0, fd) << strerror(errno); ASSERT_EQ(0, ftruncate(fd, 100)) << strerror(errno); - /* Deliberately leak fd */ + leak(fd); } /* @@ -1242,7 +1242,7 @@ TEST_F(Write, clear_suid) ASSERT_EQ(1, write(fd, wbuf, sizeof(wbuf))) << strerror(errno); ASSERT_EQ(0, fstat(fd, &sb)) << strerror(errno); EXPECT_EQ(S_IFREG | newmode, sb.st_mode); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* A write by a non-owner should clear a file's SGID bit */ @@ -1268,7 +1268,7 @@ TEST_F(Write, clear_sgid) ASSERT_EQ(1, write(fd, wbuf, sizeof(wbuf))) << strerror(errno); ASSERT_EQ(0, fstat(fd, &sb)) << strerror(errno); EXPECT_EQ(S_IFREG | newmode, sb.st_mode); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* Regression test for a specific recurse-of-nonrecursive-lock panic @@ -1297,7 +1297,7 @@ TEST_F(Write, recursion_panic_while_clearing_suid) fd = open(FULLPATH, O_WRONLY); ASSERT_LE(0, fd) << strerror(errno); ASSERT_EQ(1, write(fd, wbuf, sizeof(wbuf))) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } Modified: projects/fuse2/tests/sys/fs/fusefs/fifo.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/fifo.cc Wed Jun 26 20:19:48 2019 (r349439) +++ projects/fuse2/tests/sys/fs/fusefs/fifo.cc Wed Jun 26 20:25:57 2019 (r349440) @@ -106,7 +106,7 @@ TEST_F(Fifo, read_write) } ASSERT_STREQ(message, MESSAGE); - /* Deliberately leak fd */ + leak(fd); } /* Writer thread */ @@ -203,5 +203,5 @@ TEST_F(Socket, read_write) } ASSERT_STREQ(message, MESSAGE); - /* Deliberately leak fd */ + leak(fd); } Modified: projects/fuse2/tests/sys/fs/fusefs/flush.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/flush.cc Wed Jun 26 20:19:48 2019 (r349439) +++ projects/fuse2/tests/sys/fs/fusefs/flush.cc Wed Jun 26 20:25:57 2019 (r349440) @@ -227,5 +227,6 @@ TEST_F(FlushWithLocks, unlock_on_close) fd2 = open(FULLPATH, O_WRONLY); ASSERT_LE(0, fd2) << strerror(errno); ASSERT_EQ(0, close(fd2)) << strerror(errno); - /* Deliberately leak fd */ + leak(fd); + leak(fd2); } Modified: projects/fuse2/tests/sys/fs/fusefs/fsync.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/fsync.cc Wed Jun 26 20:19:48 2019 (r349439) +++ projects/fuse2/tests/sys/fs/fusefs/fsync.cc Wed Jun 26 20:25:57 2019 (r349440) @@ -106,7 +106,7 @@ TEST_F(Fsync, aio_fsync) ASSERT_EQ(0, aio_fsync(O_SYNC, &iocb)) << strerror(errno); ASSERT_EQ(0, aio_waitcomplete(&piocb, NULL)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -171,7 +171,7 @@ TEST_F(Fsync, eio) ASSERT_NE(0, fdatasync(fd)); ASSERT_EQ(EIO, errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -200,7 +200,7 @@ TEST_F(Fsync, enosys) /* Subsequent calls shouldn't query the daemon*/ EXPECT_EQ(0, fdatasync(fd)); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } @@ -223,7 +223,7 @@ TEST_F(Fsync, fdatasync) ASSERT_EQ(bufsize, write(fd, CONTENTS, bufsize)) << strerror(errno); ASSERT_EQ(0, fdatasync(fd)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } TEST_F(Fsync, fsync) @@ -245,5 +245,5 @@ TEST_F(Fsync, fsync) ASSERT_EQ(bufsize, write(fd, CONTENTS, bufsize)) << strerror(errno); ASSERT_EQ(0, fsync(fd)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } Modified: projects/fuse2/tests/sys/fs/fusefs/fsyncdir.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/fsyncdir.cc Wed Jun 26 20:19:48 2019 (r349439) +++ projects/fuse2/tests/sys/fs/fusefs/fsyncdir.cc Wed Jun 26 20:25:57 2019 (r349440) @@ -97,7 +97,7 @@ TEST_F(FsyncDir, aio_fsync) ASSERT_EQ(0, aio_fsync(O_SYNC, &iocb)) << strerror(errno); ASSERT_EQ(0, aio_waitcomplete(&piocb, NULL)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } TEST_F(FsyncDir, eio) @@ -116,7 +116,7 @@ TEST_F(FsyncDir, eio) ASSERT_NE(0, fsync(fd)); ASSERT_EQ(EIO, errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -142,7 +142,7 @@ TEST_F(FsyncDir, enosys) /* Subsequent calls shouldn't query the daemon*/ EXPECT_EQ(0, fsync(fd)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } TEST_F(FsyncDir, fsyncdata) @@ -160,7 +160,7 @@ TEST_F(FsyncDir, fsyncdata) ASSERT_LE(0, fd) << strerror(errno); ASSERT_EQ(0, fdatasync(fd)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -182,5 +182,5 @@ TEST_F(FsyncDir, fsync) ASSERT_LE(0, fd) << strerror(errno); ASSERT_EQ(0, fsync(fd)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } Modified: projects/fuse2/tests/sys/fs/fusefs/io.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/io.cc Wed Jun 26 20:19:48 2019 (r349439) +++ projects/fuse2/tests/sys/fs/fusefs/io.cc Wed Jun 26 20:25:57 2019 (r349440) @@ -230,7 +230,7 @@ void TearDown() if (m_control_fd >= 0) close(m_control_fd); FuseTest::TearDown(); - /* Deliberately leak test_fd */ + leak(m_test_fd); } void do_closeopen() Modified: projects/fuse2/tests/sys/fs/fusefs/locks.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/locks.cc Wed Jun 26 20:19:48 2019 (r349439) +++ projects/fuse2/tests/sys/fs/fusefs/locks.cc Wed Jun 26 20:25:57 2019 (r349440) @@ -127,7 +127,7 @@ TEST_F(FlockFallback, local) fd = open(FULLPATH, O_RDWR); ASSERT_LE(0, fd) << strerror(errno); ASSERT_EQ(0, flock(fd, LOCK_EX)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -148,7 +148,7 @@ TEST_F(Flock, local) fd = open(FULLPATH, O_RDWR); ASSERT_LE(0, fd) << strerror(errno); ASSERT_EQ(0, flock(fd, LOCK_EX)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* Set a new flock lock with FUSE_SETLK */ @@ -167,7 +167,7 @@ TEST_F(Flock, DISABLED_set) fd = open(FULLPATH, O_RDWR); ASSERT_LE(0, fd) << strerror(errno); ASSERT_EQ(0, flock(fd, LOCK_EX)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* Fail to set a flock lock in non-blocking mode */ @@ -187,7 +187,7 @@ TEST_F(Flock, DISABLED_eagain) ASSERT_LE(0, fd) << strerror(errno); ASSERT_NE(0, flock(fd, LOCK_EX | LOCK_NB)); ASSERT_EQ(EAGAIN, errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -214,7 +214,7 @@ TEST_F(GetlkFallback, local) fl.l_whence = SEEK_SET; fl.l_sysid = 0; ASSERT_NE(-1, fcntl(fd, F_GETLK, &fl)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -260,7 +260,7 @@ TEST_F(Getlk, no_locks) fl.l_sysid = 0; ASSERT_NE(-1, fcntl(fd, F_GETLK, &fl)) << strerror(errno); ASSERT_EQ(F_UNLCK, fl.l_type); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* A different pid does have a lock */ @@ -311,7 +311,7 @@ TEST_F(Getlk, lock_exists) EXPECT_EQ(F_WRLCK, fl.l_type); EXPECT_EQ(SEEK_SET, fl.l_whence); EXPECT_EQ(0, fl.l_sysid); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -338,7 +338,7 @@ TEST_F(SetlkFallback, local) fl.l_whence = SEEK_SET; fl.l_sysid = 0; ASSERT_NE(-1, fcntl(fd, F_SETLK, &fl)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* Set a new lock with FUSE_SETLK */ @@ -364,7 +364,7 @@ TEST_F(Setlk, set) fl.l_whence = SEEK_SET; fl.l_sysid = 0; ASSERT_NE(-1, fcntl(fd, F_SETLK, &fl)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* l_len = 0 is a flag value that means to lock until EOF */ @@ -390,7 +390,7 @@ TEST_F(Setlk, set_eof) fl.l_whence = SEEK_SET; fl.l_sysid = 0; ASSERT_NE(-1, fcntl(fd, F_SETLK, &fl)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* Fail to set a new lock with FUSE_SETLK due to a conflict */ @@ -417,7 +417,7 @@ TEST_F(Setlk, eagain) fl.l_sysid = 0; ASSERT_EQ(-1, fcntl(fd, F_SETLK, &fl)); ASSERT_EQ(EAGAIN, errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -444,7 +444,7 @@ TEST_F(SetlkwFallback, local) fl.l_whence = SEEK_SET; fl.l_sysid = 0; ASSERT_NE(-1, fcntl(fd, F_SETLKW, &fl)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -474,5 +474,5 @@ TEST_F(Setlkw, set) fl.l_whence = SEEK_SET; fl.l_sysid = 0; ASSERT_NE(-1, fcntl(fd, F_SETLKW, &fl)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } Modified: projects/fuse2/tests/sys/fs/fusefs/nfs.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/nfs.cc Wed Jun 26 20:19:48 2019 (r349439) +++ projects/fuse2/tests/sys/fs/fusefs/nfs.cc Wed Jun 26 20:25:57 2019 (r349440) @@ -340,5 +340,5 @@ TEST_F(Readdir, getdirentries) r = getdirentries(fd, buf, sizeof(buf), 0); ASSERT_EQ(0, r) << strerror(errno); - /* Deliberately leak fd. RELEASEDIR will be tested separately */ + leak(fd); } Modified: projects/fuse2/tests/sys/fs/fusefs/notify.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/notify.cc Wed Jun 26 20:19:48 2019 (r349439) +++ projects/fuse2/tests/sys/fs/fusefs/notify.cc Wed Jun 26 20:25:57 2019 (r349440) @@ -383,7 +383,7 @@ TEST_F(Notify, inval_inode_with_clean_cache) ASSERT_EQ(size1, read(fd, buf, size1)) << strerror(errno); EXPECT_EQ(0, memcmp(buf, CONTENTS1, size1)); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* FUSE_NOTIFY_STORE with a file that's not in the entry cache */ @@ -442,7 +442,7 @@ TEST_F(Notify, DISABLED_store_with_blank_cache) ASSERT_EQ(size1, read(fd, buf, size1)) << strerror(errno); EXPECT_EQ(0, memcmp(buf, CONTENTS1, size1)); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } TEST_F(NotifyWriteback, inval_inode_with_dirty_cache) @@ -482,7 +482,7 @@ TEST_F(NotifyWriteback, inval_inode_with_dirty_cache) pthread_join(th0, &thr0_value); EXPECT_EQ(0, (intptr_t)thr0_value); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } TEST_F(NotifyWriteback, inval_inode_attrs_only) @@ -541,5 +541,5 @@ TEST_F(NotifyWriteback, inval_inode_attrs_only) EXPECT_EQ(uid, sb.st_uid); EXPECT_EQ(bufsize, sb.st_size); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } Modified: projects/fuse2/tests/sys/fs/fusefs/open.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/open.cc Wed Jun 26 20:19:48 2019 (r349439) +++ projects/fuse2/tests/sys/fs/fusefs/open.cc Wed Jun 26 20:25:57 2019 (r349440) @@ -64,7 +64,7 @@ void test_ok(int os_flags, int fuse_flags) { fd = open(FULLPATH, os_flags); EXPECT_LE(0, fd) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } }; Modified: projects/fuse2/tests/sys/fs/fusefs/read.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/read.cc Wed Jun 26 20:19:48 2019 (r349439) +++ projects/fuse2/tests/sys/fs/fusefs/read.cc Wed Jun 26 20:25:57 2019 (r349440) @@ -136,7 +136,8 @@ TEST_F(AioRead, aio_read) ASSERT_EQ(0, aio_read(&iocb)) << strerror(errno); ASSERT_EQ(bufsize, aio_waitcomplete(&piocb, NULL)) << strerror(errno); ASSERT_EQ(0, memcmp(buf, CONTENTS, bufsize)); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + + leak(fd); } /* @@ -216,7 +217,7 @@ TEST_F(AioRead, async_read_disabled) /* Wait for AIO activity to complete, but ignore errors */ (void)aio_waitcomplete(NULL, NULL); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -295,7 +296,7 @@ TEST_F(AsyncRead, async_read) /* Wait for AIO activity to complete, but ignore errors */ (void)aio_waitcomplete(NULL, NULL); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* 0-length reads shouldn't cause any confusion */ @@ -315,7 +316,7 @@ TEST_F(Read, direct_io_read_nothing) ASSERT_LE(0, fd) << strerror(errno); ASSERT_EQ(0, pread(fd, buf, 0, offset)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -348,7 +349,7 @@ TEST_F(Read, direct_io_pread) expect_read(ino, offset, bufsize, bufsize, CONTENTS); ASSERT_EQ(bufsize, pread(fd, buf, bufsize, offset)) << strerror(errno); ASSERT_EQ(0, memcmp(buf, CONTENTS, bufsize)); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -377,7 +378,7 @@ TEST_F(Read, direct_io_short_read) ASSERT_EQ(halfbufsize, pread(fd, buf, bufsize, offset)) << strerror(errno); ASSERT_EQ(0, memcmp(buf, CONTENTS, halfbufsize)); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } TEST_F(Read, eio) @@ -404,7 +405,7 @@ TEST_F(Read, eio) ASSERT_EQ(-1, read(fd, buf, bufsize)) << strerror(errno); ASSERT_EQ(EIO, errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -439,7 +440,7 @@ TEST_F(Read, eof) EXPECT_EQ(partbufsize, r) << strerror(errno); ASSERT_EQ(0, fstat(fd, &sb)); EXPECT_EQ((off_t)(offset + partbufsize), sb.st_size); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* Like Read.eof, but causes an entire buffer to be invalidated */ @@ -472,7 +473,7 @@ TEST_F(Read, eof_of_whole_buffer) << strerror(errno); ASSERT_EQ(0, fstat(fd, &sb)); EXPECT_EQ((off_t)(m_maxbcachebuf), sb.st_size); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -506,7 +507,8 @@ TEST_F(Read, keep_cache) */ ASSERT_EQ(bufsize, read(fd1, buf, bufsize)) << strerror(errno); - /* Deliberately leak fd0 and fd1. */ + leak(fd0); + leak(fd1); } /* @@ -542,7 +544,8 @@ TEST_F(Read, keep_cache_disabled) ASSERT_EQ(0, lseek(fd0, 0, SEEK_SET)) << strerror(errno); ASSERT_EQ(bufsize, read(fd0, buf, bufsize)) << strerror(errno); - /* Deliberately leak fd0 and fd1. */ + leak(fd0); + leak(fd1); } TEST_F(Read, mmap) @@ -584,7 +587,7 @@ TEST_F(Read, mmap) ASSERT_EQ(0, memcmp(p, CONTENTS, bufsize)); ASSERT_EQ(0, munmap(p, len)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -635,7 +638,7 @@ TEST_F(Read, mmap_eof) EXPECT_EQ((off_t)bufsize, sb.st_size); ASSERT_EQ(0, munmap(p, len)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -670,7 +673,7 @@ TEST_F(Read, o_direct) ASSERT_EQ(bufsize, read(fd, buf, bufsize)) << strerror(errno); ASSERT_EQ(0, memcmp(buf, CONTENTS, bufsize)); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } TEST_F(Read, pread) @@ -697,7 +700,7 @@ TEST_F(Read, pread) ASSERT_EQ(bufsize, pread(fd, buf, bufsize, offset)) << strerror(errno); ASSERT_EQ(0, memcmp(buf, CONTENTS, bufsize)); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } TEST_F(Read, read) @@ -720,7 +723,7 @@ TEST_F(Read, read) ASSERT_EQ(bufsize, read(fd, buf, bufsize)) << strerror(errno); ASSERT_EQ(0, memcmp(buf, CONTENTS, bufsize)); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } TEST_F(Read_7_8, read) @@ -743,7 +746,7 @@ TEST_F(Read_7_8, read) ASSERT_EQ(bufsize, read(fd, buf, bufsize)) << strerror(errno); ASSERT_EQ(0, memcmp(buf, CONTENTS, bufsize)); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -781,7 +784,7 @@ TEST_F(Read, cache_block) /* A subsequent read should be serviced by cache */ ASSERT_EQ(bufsize, read(fd, buf, bufsize)) << strerror(errno); ASSERT_EQ(0, memcmp(buf, contents1, bufsize)); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* Reading with sendfile should work (though it obviously won't be 0-copy) */ @@ -827,7 +830,7 @@ TEST_F(Read, sendfile) close(sp[1]); close(sp[0]); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* sendfile should fail gracefully if fuse declines the read */ @@ -861,7 +864,7 @@ TEST_F(Read, DISABLED_sendfile_eio) close(sp[1]); close(sp[0]); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -903,7 +906,7 @@ TEST_P(ReadAhead, readahead) { ASSERT_EQ(bufsize, read(fd, rbuf, bufsize)) << strerror(errno); ASSERT_EQ(0, memcmp(rbuf, contents, bufsize)); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } INSTANTIATE_TEST_CASE_P(RA, ReadAhead, Modified: projects/fuse2/tests/sys/fs/fusefs/readdir.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/readdir.cc Wed Jun 26 20:19:48 2019 (r349439) +++ projects/fuse2/tests/sys/fs/fusefs/readdir.cc Wed Jun 26 20:25:57 2019 (r349440) @@ -117,7 +117,7 @@ TEST_F(Readdir, dots) ASSERT_EQ(NULL, readdir(dir)); ASSERT_EQ(0, errno); - /* Deliberately leak dir. RELEASEDIR will be tested separately */ + leakdir(dir); } TEST_F(Readdir, eio) @@ -148,7 +148,7 @@ TEST_F(Readdir, eio) ASSERT_EQ(NULL, de); ASSERT_EQ(EIO, errno); - /* Deliberately leak dir. RELEASEDIR will be tested separately */ + leakdir(dir); } /* getdirentries(2) can use a larger buffer size than readdir(3) */ @@ -181,7 +181,7 @@ TEST_F(Readdir, getdirentries) r = getdirentries(fd, buf, sizeof(buf), 0); ASSERT_EQ(0, r) << strerror(errno); - /* Deliberately leak fd. RELEASEDIR will be tested separately */ + leak(fd); } /* @@ -228,7 +228,8 @@ TEST_F(Readdir, getdirentries_concurrent) r = getdirentries(fd1, buf, sizeof(buf), 0); ASSERT_EQ(0, r) << strerror(errno); - /* Deliberately leak fd1. */ + leak(fd0); + leak(fd1); } /* @@ -263,7 +264,7 @@ TEST_F(Readdir, nodots) ASSERT_EQ(NULL, readdir(dir)); ASSERT_EQ(0, errno); - /* Deliberately leak dir. RELEASEDIR will be tested separately */ + leakdir(dir); } /* telldir(3) and seekdir(3) should work with fuse */ @@ -339,7 +340,7 @@ TEST_F(Readdir, seekdir) ASSERT_NE(NULL, de) << strerror(errno); EXPECT_EQ(130ul, de->d_fileno); - /* Deliberately leak dir. RELEASEDIR will be tested separately */ + leakdir(dir); } TEST_F(Readdir_7_8, nodots) @@ -370,5 +371,5 @@ TEST_F(Readdir_7_8, nodots) ASSERT_EQ(NULL, readdir(dir)); ASSERT_EQ(0, errno); - /* Deliberately leak dir. RELEASEDIR will be tested separately */ + leakdir(dir); } Modified: projects/fuse2/tests/sys/fs/fusefs/setattr.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/setattr.cc Wed Jun 26 20:19:48 2019 (r349439) +++ projects/fuse2/tests/sys/fs/fusefs/setattr.cc Wed Jun 26 20:25:57 2019 (r349440) @@ -322,7 +322,7 @@ TEST_F(Setattr, fchmod) fd = open(FULLPATH, O_RDONLY); ASSERT_LE(0, fd) << strerror(errno); ASSERT_EQ(0, fchmod(fd, newmode)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* Change the size of an open file, by its file descriptor */ @@ -376,7 +376,7 @@ TEST_F(Setattr, ftruncate) fd = open(FULLPATH, O_RDWR); ASSERT_LE(0, fd) << strerror(errno); ASSERT_EQ(0, ftruncate(fd, newsize)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* Change the size of the file */ Modified: projects/fuse2/tests/sys/fs/fusefs/unlink.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/unlink.cc Wed Jun 26 20:19:48 2019 (r349439) +++ projects/fuse2/tests/sys/fs/fusefs/unlink.cc Wed Jun 26 20:25:57 2019 (r349440) @@ -135,5 +135,5 @@ TEST_F(Unlink, open_but_deleted) fd = open(FULLPATH, O_RDWR); ASSERT_LE(0, fd) << strerror(errno); ASSERT_EQ(0, unlink(FULLPATH)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } Modified: projects/fuse2/tests/sys/fs/fusefs/utils.hh ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/utils.hh Wed Jun 26 20:19:48 2019 (r349439) +++ projects/fuse2/tests/sys/fs/fusefs/utils.hh Wed Jun 26 20:25:57 2019 (r349440) @@ -30,6 +30,8 @@ struct _sem; typedef struct _sem sem_t; +struct _dirdesc; +typedef struct _dirdesc DIR; /* Nanoseconds to sleep, for tests that must */ #define NAP_NS (100'000'000) @@ -210,4 +212,23 @@ class FuseTest : public ::testing::Test { void fork(bool drop_privs, int *status, std::function parent_func, std::function child_func); + + /* + * Deliberately leak a file descriptor. + * + * Closing a file descriptor on fusefs would cause the server to + * receive FUSE_CLOSE and possibly FUSE_INACTIVE. Handling those + * operations would needlessly complicate most tests. So most tests + * deliberately leak the file descriptors instead. This method serves + * to document the leakage, and provide a single point of suppression + * for static analyzers. + */ + static void leak(int fd __unused) {} + + /* + * Deliberately leak a DIR* pointer + * + * See comments for FuseTest::leak + */ + static void leakdir(DIR* dirp __unused) {} }; Modified: projects/fuse2/tests/sys/fs/fusefs/write.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/write.cc Wed Jun 26 20:19:48 2019 (r349439) +++ projects/fuse2/tests/sys/fs/fusefs/write.cc Wed Jun 26 20:25:57 2019 (r349440) @@ -231,7 +231,7 @@ TEST_F(AioWrite, DISABLED_aio_write) iocb.aio_sigevent.sigev_notify = SIGEV_NONE; ASSERT_EQ(0, aio_write(&iocb)) << strerror(errno); ASSERT_EQ(bufsize, aio_waitcomplete(&piocb, NULL)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -267,7 +267,7 @@ TEST_F(Write, append) EXPECT_LE(0, fd) << strerror(errno); ASSERT_EQ(BUFSIZE, write(fd, CONTENTS, BUFSIZE)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* If a file is cached, then appending to the end should not cause a read */ @@ -305,7 +305,7 @@ TEST_F(Write, append_to_cached) /* Write the new data. There should be no more read operations */ ASSERT_EQ(BUFSIZE, write(fd, CONTENTS, BUFSIZE)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } TEST_F(Write, append_direct_io) @@ -326,7 +326,7 @@ TEST_F(Write, append_direct_io) EXPECT_LE(0, fd) << strerror(errno); ASSERT_EQ(BUFSIZE, write(fd, CONTENTS, BUFSIZE)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* A direct write should evict any overlapping cached data */ @@ -364,7 +364,7 @@ TEST_F(Write, direct_io_evicts_cache) ASSERT_EQ(bufsize, read(fd, readbuf, bufsize)) << strerror(errno); ASSERT_STREQ(readbuf, CONTENTS1); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -394,7 +394,7 @@ TEST_F(Write, indirect_io_short_write) EXPECT_LE(0, fd) << strerror(errno); ASSERT_EQ(bufsize, write(fd, CONTENTS, bufsize)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -419,7 +419,7 @@ TEST_F(Write, direct_io_short_write) EXPECT_LE(0, fd) << strerror(errno); ASSERT_EQ(halfbufsize, write(fd, CONTENTS, bufsize)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -453,7 +453,7 @@ TEST_F(Write, direct_io_short_write_iov) iov[1].iov_base = (void*)CONTENTS1; iov[1].iov_len = strlen(CONTENTS1); ASSERT_EQ(size0, writev(fd, iov, 2)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* fusefs should respect RLIMIT_FSIZE */ @@ -483,7 +483,7 @@ TEST_F(Write, rlimit_fsize) ASSERT_EQ(-1, pwrite(fd, CONTENTS, bufsize, offset)); EXPECT_EQ(EFBIG, errno); EXPECT_EQ(1, s_sigxfsz); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -516,7 +516,7 @@ TEST_F(Write, eof_during_rmw) ASSERT_EQ(bufsize, pwrite(fd, CONTENTS, bufsize, offset)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -590,7 +590,7 @@ TEST_F(Write, pwrite) ASSERT_EQ(bufsize, pwrite(fd, CONTENTS, bufsize, offset)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* Writing a file should update its cached mtime and ctime */ @@ -639,7 +639,7 @@ TEST_F(Write, write) EXPECT_LE(0, fd) << strerror(errno); ASSERT_EQ(bufsize, write(fd, CONTENTS, bufsize)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* fuse(4) should not issue writes of greater size than the daemon requests */ @@ -670,7 +670,7 @@ TEST_F(Write, write_large) EXPECT_LE(0, fd) << strerror(errno); ASSERT_EQ(bufsize, write(fd, contents, bufsize)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); free(contents); } @@ -691,7 +691,7 @@ TEST_F(Write, write_nothing) EXPECT_LE(0, fd) << strerror(errno); ASSERT_EQ(bufsize, write(fd, CONTENTS, bufsize)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } TEST_F(Write_7_8, write) @@ -711,7 +711,7 @@ TEST_F(Write_7_8, write) EXPECT_LE(0, fd) << strerror(errno); ASSERT_EQ(bufsize, write(fd, CONTENTS, bufsize)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* In writeback mode, dirty data should be written on close */ @@ -855,7 +855,7 @@ TEST_F(WriteBack, rmw) ASSERT_EQ(bufsize, pwrite(fd, CONTENTS, bufsize, offset)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -885,7 +885,7 @@ TEST_F(WriteBack, cache) */ ASSERT_EQ(0, lseek(fd, 0, SEEK_SET)) << strerror(errno); ASSERT_EQ(bufsize, read(fd, readbuf, bufsize)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -917,7 +917,7 @@ TEST_F(WriteBack, o_direct) ASSERT_EQ(0, lseek(fd, 0, SEEK_SET)) << strerror(errno); ASSERT_EQ(0, fcntl(fd, F_SETFL, 0)) << strerror(errno); ASSERT_EQ(bufsize, read(fd, readbuf, bufsize)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -995,7 +995,7 @@ TEST_F(WriteBackAsync, eof) /* The file's size should still be what was established by pwrite */ ASSERT_EQ(0, fstat(fd, &sb)) << strerror(errno); EXPECT_EQ(offset + wbufsize, sb.st_size); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* @@ -1154,7 +1154,7 @@ TEST_F(Write, writethrough) */ ASSERT_EQ(0, lseek(fd, 0, SEEK_SET)) << strerror(errno); ASSERT_EQ(bufsize, read(fd, readbuf, bufsize)) << strerror(errno); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } /* Writes that extend a file should update the cached file size */ @@ -1179,5 +1179,5 @@ TEST_F(Write, update_file_size) /* Get cached attributes */ ASSERT_EQ(0, fstat(fd, &sb)) << strerror(errno); ASSERT_EQ(bufsize, sb.st_size); - /* Deliberately leak fd. close(2) will be tested in release.cc */ + leak(fd); } From owner-svn-src-projects@freebsd.org Wed Jun 26 23:10:21 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 367EB15D466E for ; Wed, 26 Jun 2019 23:10:21 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C55168B8DD; Wed, 26 Jun 2019 23:10:20 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B49E56C58; Wed, 26 Jun 2019 23:10:20 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5QNAKDw056625; Wed, 26 Jun 2019 23:10:20 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5QNAKui056624; Wed, 26 Jun 2019 23:10:20 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906262310.x5QNAKui056624@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Wed, 26 Jun 2019 23:10:20 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349445 - projects/fuse2/tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/tests/sys/fs/fusefs X-SVN-Commit-Revision: 349445 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: C55168B8DD X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_SHORT(-0.96)[-0.959,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jun 2019 23:10:21 -0000 Author: asomers Date: Wed Jun 26 23:10:20 2019 New Revision: 349445 URL: https://svnweb.freebsd.org/changeset/base/349445 Log: fusefs: tighten expectations in mmap tests In r349378 I fixed mmap's habit of reading more data than was available. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/tests/sys/fs/fusefs/read.cc Modified: projects/fuse2/tests/sys/fs/fusefs/read.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/read.cc Wed Jun 26 22:06:40 2019 (r349444) +++ projects/fuse2/tests/sys/fs/fusefs/read.cc Wed Jun 26 23:10:20 2019 (r349445) @@ -563,14 +563,13 @@ TEST_F(Read, mmap) expect_lookup(RELPATH, ino, bufsize); expect_open(ino, 0, 1); - /* mmap may legitimately try to read more data than is available */ EXPECT_CALL(*m_mock, process( ResultOf([=](auto in) { return (in.header.opcode == FUSE_READ && in.header.nodeid == ino && in.body.read.fh == Read::FH && in.body.read.offset == 0 && - in.body.read.size >= bufsize); + in.body.read.size == bufsize); }, Eq(true)), _) ).WillOnce(Invoke(ReturnImmediate([=](auto in __unused, auto& out) { @@ -608,16 +607,15 @@ TEST_F(Read, mmap_eof) len = getpagesize(); - expect_lookup(RELPATH, ino, 100000); + expect_lookup(RELPATH, ino, m_maxbcachebuf); expect_open(ino, 0, 1); - /* mmap may legitimately try to read more data than is available */ EXPECT_CALL(*m_mock, process( ResultOf([=](auto in) { return (in.header.opcode == FUSE_READ && in.header.nodeid == ino && in.body.read.fh == Read::FH && in.body.read.offset == 0 && - in.body.read.size >= bufsize); + in.body.read.size == (uint32_t)m_maxbcachebuf); }, Eq(true)), _) ).WillOnce(Invoke(ReturnImmediate([=](auto in __unused, auto& out) { @@ -802,14 +800,13 @@ TEST_F(Read, sendfile) expect_lookup(RELPATH, ino, bufsize); expect_open(ino, 0, 1); - /* Like mmap, sendfile may request more data than is available */ EXPECT_CALL(*m_mock, process( ResultOf([=](auto in) { return (in.header.opcode == FUSE_READ && in.header.nodeid == ino && in.body.read.fh == Read::FH && in.body.read.offset == 0 && - in.body.read.size >= bufsize); + in.body.read.size == bufsize); }, Eq(true)), _) ).WillOnce(Invoke(ReturnImmediate([=](auto in __unused, auto& out) { From owner-svn-src-projects@freebsd.org Thu Jun 27 00:00:49 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B07515D56C2 for ; Thu, 27 Jun 2019 00:00:49 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E67148CD53; Thu, 27 Jun 2019 00:00:48 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C02DC75CD; Thu, 27 Jun 2019 00:00:48 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5R00mYn082619; Thu, 27 Jun 2019 00:00:48 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5R00mqn082618; Thu, 27 Jun 2019 00:00:48 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906270000.x5R00mqn082618@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Thu, 27 Jun 2019 00:00:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349447 - projects/fuse2/sys/fs/fuse X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/sys/fs/fuse X-SVN-Commit-Revision: 349447 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: E67148CD53 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.96)[-0.959,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jun 2019 00:00:49 -0000 Author: asomers Date: Thu Jun 27 00:00:48 2019 New Revision: 349447 URL: https://svnweb.freebsd.org/changeset/base/349447 Log: fusefs: fix some memory leaks Fix memory leaks relating to FUSE_BMAP and FUSE_CREATE. There are still leaks relating to FUSE_INTERRUPT, but they'll be harder to fix since the server is legally allowed to never respond to a FUSE_INTERRUPT operation. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_vnops.c Modified: projects/fuse2/sys/fs/fuse/fuse_vnops.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_vnops.c Wed Jun 26 23:33:32 2019 (r349446) +++ projects/fuse2/sys/fs/fuse/fuse_vnops.c Thu Jun 27 00:00:48 2019 (r349447) @@ -536,12 +536,14 @@ fuse_vnop_bmap(struct vop_bmap_args *ap) fbi->blocksize = biosize; error = fdisp_wait_answ(&fdi); if (error == ENOSYS) { + fdisp_destroy(&fdi); fsess_set_notimpl(mp, FUSE_BMAP); error = 0; } else { fbo = fdi.answ; if (error == 0 && pbn != NULL) *pbn = fbo->block; + fdisp_destroy(&fdi); return error; } } @@ -693,6 +695,7 @@ fuse_vnop_create(struct vop_create_args *ap) if (err) { if (err == ENOSYS && op == FUSE_CREATE) { fsess_set_notimpl(mp, FUSE_CREATE); + fdisp_destroy(fdip); fdisp_make_mknod_for_fallback(fdip, cnp, dvp, parentnid, td, cred, mode, &op); err = fdisp_wait_answ(fdip); From owner-svn-src-projects@freebsd.org Thu Jun 27 16:30:28 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38BC115C7523 for ; Thu, 27 Jun 2019 16:30:28 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CF1FE8D22B; Thu, 27 Jun 2019 16:30:27 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 989A119CFA; Thu, 27 Jun 2019 16:30:27 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5RGURi0012301; Thu, 27 Jun 2019 16:30:27 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5RGUPLA012288; Thu, 27 Jun 2019 16:30:25 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906271630.x5RGUPLA012288@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Thu, 27 Jun 2019 16:30:25 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349462 - in projects/fuse2: share/man/man5 sys/fs/fuse X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: share/man/man5 sys/fs/fuse X-SVN-Commit-Revision: 349462 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: CF1FE8D22B X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.982,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jun 2019 16:30:28 -0000 Author: asomers Date: Thu Jun 27 16:30:25 2019 New Revision: 349462 URL: https://svnweb.freebsd.org/changeset/base/349462 Log: fusefs: convert statistical sysctls to use counter(9) counter(9) is more performant than using atomic instructions to update sysctls that just report statistics to userland. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/share/man/man5/fusefs.5 projects/fuse2/sys/fs/fuse/fuse.h projects/fuse2/sys/fs/fuse/fuse_file.c projects/fuse2/sys/fs/fuse/fuse_file.h projects/fuse2/sys/fs/fuse/fuse_internal.c projects/fuse2/sys/fs/fuse/fuse_internal.h projects/fuse2/sys/fs/fuse/fuse_ipc.c projects/fuse2/sys/fs/fuse/fuse_main.c projects/fuse2/sys/fs/fuse/fuse_node.c projects/fuse2/sys/fs/fuse/fuse_node.h projects/fuse2/sys/fs/fuse/fuse_vnops.c Modified: projects/fuse2/share/man/man5/fusefs.5 ============================================================================== --- projects/fuse2/share/man/man5/fusefs.5 Thu Jun 27 15:51:50 2019 (r349461) +++ projects/fuse2/share/man/man5/fusefs.5 Thu Jun 27 16:30:25 2019 (r349462) @@ -28,7 +28,7 @@ .\" SUCH DAMAGE. .\" .\" $FreeBSD$ -.Dd June 26, 2019 +.Dd June 27, 2019 .Dt FUSEFS 5 .Os .Sh NAME @@ -60,11 +60,9 @@ Finally, the API is portable. Many daemons can run on multiple operating systems with minimal modifications. .Sh SYSCTL VARIABLES -The following variables are available as both +The following .Xr sysctl 8 -variables and -.Xr loader 8 -tunables: +variables are available: .Bl -tag -width indent .It Va vfs.fusefs.kernelabi_major Major version of the FUSE kernel ABI supported by this driver. @@ -87,14 +85,19 @@ require network access. .Pp FUSE file systems using protocol 7.23 or later specify their cache behavior on a per-mountpoint basis, ignoring this sysctl. +.It Va vfs.fusefs.stats.filehandle_count +Current number of open FUSE file handles. +.It Va vfs.fusefs.stats.lookup_cache_hits +Total number of lookup cache hits. +.It Va vfs.fusefs.stats.lookup_cache_misses +Total number of lookup cache misses. +.It Va vfs.fusefs.stats.node_count +Current number of allocated FUSE vnodes. +.It Va vfs.fusefs.stats.ticket_count +Current number of allocated FUSE tickets, which is roughly equal to the number +number of FUSE operations currently being processed by daemons. .\" Undocumented sysctls .\" ==================== -.\" Counters: I intend to rename to vfs.fusefs.stats.* for clarity -.\" vfs.fusefs.lookup_cache_{hits, misses} -.\" vfs.fusefs.filehandle_count -.\" vfs.fusefs.ticker_count -.\" vfs.fusefs.node_count -.\" .\" vfs.fusefs.reclaim_revoked: I don't understand it well-enough .\" vfs.fusefs.enforce_dev_perms: I don't understand it well enough. .\" vfs.fusefs.iov_credit: I don't understand it well enough Modified: projects/fuse2/sys/fs/fuse/fuse.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse.h Thu Jun 27 15:51:50 2019 (r349461) +++ projects/fuse2/sys/fs/fuse/fuse.h Thu Jun 27 16:30:25 2019 (r349462) @@ -66,6 +66,7 @@ /* misc */ SYSCTL_DECL(_vfs_fusefs); +SYSCTL_DECL(_vfs_fusefs_stats); /* Fuse locking */ Modified: projects/fuse2/sys/fs/fuse/fuse_file.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_file.c Thu Jun 27 15:51:50 2019 (r349461) +++ projects/fuse2/sys/fs/fuse/fuse_file.c Thu Jun 27 16:30:25 2019 (r349462) @@ -59,6 +59,7 @@ __FBSDID("$FreeBSD$"); #include +#include #include #include #include @@ -93,10 +94,10 @@ SDT_PROVIDER_DECLARE(fusefs); */ SDT_PROBE_DEFINE2(fusefs, , file, trace, "int", "char*"); -static int fuse_fh_count = 0; +static counter_u64_t fuse_fh_count = 0; -SYSCTL_INT(_vfs_fusefs, OID_AUTO, filehandle_count, CTLFLAG_RD, - &fuse_fh_count, 0, "number of open FUSE filehandles"); +SYSCTL_COUNTER_U64(_vfs_fusefs_stats, OID_AUTO, filehandle_count, CTLFLAG_RD, + &fuse_fh_count, "number of open FUSE filehandles"); /* Get the FUFH type for a particular access mode */ static inline fufh_type_t @@ -190,7 +191,7 @@ fuse_filehandle_close(struct vnode *vp, struct fuse_fi fdisp_destroy(&fdi); out: - atomic_subtract_acq_int(&fuse_fh_count, 1); + counter_u64_add(fuse_fh_count, -1); LIST_REMOVE(fufh, next); free(fufh, M_FUSE_FILEHANDLE); @@ -343,7 +344,7 @@ fuse_filehandle_init(struct vnode *vp, fufh_type_t fuf if (fufhp != NULL) *fufhp = fufh; - atomic_add_acq_int(&fuse_fh_count, 1); + counter_u64_add(fuse_fh_count, 1); if (foo->open_flags & FOPEN_DIRECT_IO) { ASSERT_VOP_ELOCKED(vp, __func__); @@ -355,4 +356,17 @@ fuse_filehandle_init(struct vnode *vp, fufh_type_t fuf VTOFUD(vp)->flag &= ~FN_DIRECTIO; } +} + +void +fuse_file_init(void) +{ + fuse_fh_count = counter_u64_alloc(M_WAITOK); + counter_u64_zero(fuse_fh_count); +} + +void +fuse_file_destroy(void) +{ + counter_u64_free(fuse_fh_count); } Modified: projects/fuse2/sys/fs/fuse/fuse_file.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_file.h Thu Jun 27 15:51:50 2019 (r349461) +++ projects/fuse2/sys/fs/fuse/fuse_file.h Thu Jun 27 16:30:25 2019 (r349462) @@ -213,4 +213,7 @@ int fuse_filehandle_open(struct vnode *vp, int mode, int fuse_filehandle_close(struct vnode *vp, struct fuse_filehandle *fufh, struct thread *td, struct ucred *cred); +void fuse_file_init(void); +void fuse_file_destroy(void); + #endif /* _FUSE_FILE_H_ */ Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_internal.c Thu Jun 27 15:51:50 2019 (r349461) +++ projects/fuse2/sys/fs/fuse/fuse_internal.c Thu Jun 27 16:30:25 2019 (r349462) @@ -59,6 +59,7 @@ __FBSDID("$FreeBSD$"); #include +#include #include #include #include @@ -107,6 +108,15 @@ static int isbzero(void *buf, size_t len); #endif +counter_u64_t fuse_lookup_cache_hits = 0; +counter_u64_t fuse_lookup_cache_misses = 0; + +SYSCTL_COUNTER_U64(_vfs_fusefs_stats, OID_AUTO, lookup_cache_hits, CTLFLAG_RD, + &fuse_lookup_cache_hits, "number of positive cache hits in lookup"); + +SYSCTL_COUNTER_U64(_vfs_fusefs_stats, OID_AUTO, lookup_cache_misses, CTLFLAG_RD, + &fuse_lookup_cache_misses, "number of cache misses in lookup"); + int fuse_internal_get_cached_vnode(struct mount* mp, ino_t ino, int flags, struct vnode **vpp) @@ -130,11 +140,11 @@ fuse_internal_get_cached_vnode(struct mount* mp, ino_t if (*vpp != NULL) { getbinuptime(&now); if (bintime_cmp(&(VTOFUD(*vpp)->entry_cache_timeout), &now, >)){ - atomic_add_acq_long(&fuse_lookup_cache_hits, 1); + counter_u64_add(fuse_lookup_cache_hits, 1); return 0; } else { /* Entry cache timeout */ - atomic_add_acq_long(&fuse_lookup_cache_misses, 1); + counter_u64_add(fuse_lookup_cache_misses, 1); cache_purge(*vpp); vput(*vpp); *vpp = NULL; @@ -1157,3 +1167,19 @@ isbzero(void *buf, size_t len) } #endif + +void +fuse_internal_init(void) +{ + fuse_lookup_cache_misses = counter_u64_alloc(M_WAITOK); + counter_u64_zero(fuse_lookup_cache_misses); + fuse_lookup_cache_hits = counter_u64_alloc(M_WAITOK); + counter_u64_zero(fuse_lookup_cache_hits); +} + +void +fuse_internal_destroy(void) +{ + counter_u64_free(fuse_lookup_cache_hits); + counter_u64_free(fuse_lookup_cache_misses); +} Modified: projects/fuse2/sys/fs/fuse/fuse_internal.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_internal.h Thu Jun 27 15:51:50 2019 (r349461) +++ projects/fuse2/sys/fs/fuse/fuse_internal.h Thu Jun 27 16:30:25 2019 (r349462) @@ -61,6 +61,7 @@ #define _FUSE_INTERNAL_H_ #include +#include #include #include #include @@ -68,8 +69,8 @@ #include "fuse_ipc.h" #include "fuse_node.h" -extern u_long fuse_lookup_cache_hits; -extern u_long fuse_lookup_cache_misses; +extern counter_u64_t fuse_lookup_cache_hits; +extern counter_u64_t fuse_lookup_cache_misses; static inline bool vfs_isrdonly(struct mount *mp) @@ -311,5 +312,9 @@ void fuse_internal_forget_send(struct mount *mp, struc int fuse_internal_init_callback(struct fuse_ticket *tick, struct uio *uio); void fuse_internal_send_init(struct fuse_data *data, struct thread *td); + +/* module load/unload */ +void fuse_internal_init(void); +void fuse_internal_destroy(void); #endif /* _FUSE_INTERNAL_H_ */ Modified: projects/fuse2/sys/fs/fuse/fuse_ipc.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_ipc.c Thu Jun 27 15:51:50 2019 (r349461) +++ projects/fuse2/sys/fs/fuse/fuse_ipc.c Thu Jun 27 16:30:25 2019 (r349462) @@ -61,6 +61,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -107,11 +108,10 @@ static int fuse_body_audit(struct fuse_ticket *ftick, static fuse_handler_t fuse_standard_handler; -SYSCTL_NODE(_vfs, OID_AUTO, fusefs, CTLFLAG_RW, 0, "FUSE tunables"); -static int fuse_ticket_count = 0; +static counter_u64_t fuse_ticket_count; +SYSCTL_COUNTER_U64(_vfs_fusefs_stats, OID_AUTO, ticket_count, CTLFLAG_RD, + &fuse_ticket_count, "Number of allocated tickets"); -SYSCTL_INT(_vfs_fusefs, OID_AUTO, ticket_count, CTLFLAG_RW, - &fuse_ticket_count, 0, "number of allocated tickets"); static long fuse_iov_permanent_bufsize = 1 << 19; SYSCTL_LONG(_vfs_fusefs, OID_AUTO, iov_permanent_bufsize, CTLFLAG_RW, @@ -326,7 +326,7 @@ fticket_ctor(void *mem, int size, void *arg, int flags ftick->irq_unique = 0; refcount_init(&ftick->tk_refcount, 1); - atomic_add_acq_int(&fuse_ticket_count, 1); + counter_u64_add(fuse_ticket_count, 1); return 0; } @@ -341,7 +341,7 @@ fticket_dtor(void *mem, int size, void *arg) FUSE_ASSERT_MS_DONE(ftick); FUSE_ASSERT_AW_DONE(ftick); - atomic_subtract_acq_int(&fuse_ticket_count, 1); + counter_u64_add(fuse_ticket_count, -1); } static int @@ -1075,10 +1075,13 @@ fuse_ipc_init(void) ticket_zone = uma_zcreate("fuse_ticket", sizeof(struct fuse_ticket), fticket_ctor, fticket_dtor, fticket_init, fticket_fini, UMA_ALIGN_PTR, 0); + fuse_ticket_count = counter_u64_alloc(M_WAITOK); + counter_u64_zero(fuse_ticket_count); } void fuse_ipc_destroy(void) { + counter_u64_free(fuse_ticket_count); uma_zdestroy(ticket_zone); } Modified: projects/fuse2/sys/fs/fuse/fuse_main.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_main.c Thu Jun 27 15:51:50 2019 (r349461) +++ projects/fuse2/sys/fs/fuse/fuse_main.c Thu Jun 27 16:30:25 2019 (r349462) @@ -77,6 +77,10 @@ __FBSDID("$FreeBSD$"); #include #include "fuse.h" +#include "fuse_file.h" +#include "fuse_ipc.h" +#include "fuse_internal.h" +#include "fuse_node.h" static void fuse_bringdown(eventhandler_tag eh_tag); static int fuse_loader(struct module *m, int what, void *arg); @@ -97,6 +101,8 @@ static struct vfsconf fuse_vfsconf = { .vfc_flags = VFCF_JAIL | VFCF_SYNTHETIC }; +SYSCTL_NODE(_vfs, OID_AUTO, fusefs, CTLFLAG_RW, 0, "FUSE tunables"); +SYSCTL_NODE(_vfs_fusefs, OID_AUTO, stats, CTLFLAG_RW, 0, "FUSE statistics"); SYSCTL_INT(_vfs_fusefs, OID_AUTO, kernelabi_major, CTLFLAG_RD, SYSCTL_NULL_INT_PTR, FUSE_KERNEL_VERSION, "FUSE kernel abi major version"); SYSCTL_INT(_vfs_fusefs, OID_AUTO, kernelabi_minor, CTLFLAG_RD, @@ -112,7 +118,9 @@ SDT_PROVIDER_DEFINE(fusefs); static void fuse_bringdown(eventhandler_tag eh_tag) { - + fuse_node_destroy(); + fuse_internal_destroy(); + fuse_file_destroy(); fuse_ipc_destroy(); fuse_device_destroy(); mtx_destroy(&fuse_mtx); @@ -133,6 +141,9 @@ fuse_loader(struct module *m, int what, void *arg) return (err); } fuse_ipc_init(); + fuse_file_init(); + fuse_internal_init(); + fuse_node_init(); fuse_pbuf_zone = pbuf_zsecond_create("fusepbuf", nswbuf / 2); /* vfs_modevent ignores its first arg */ Modified: projects/fuse2/sys/fs/fuse/fuse_node.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_node.c Thu Jun 27 15:51:50 2019 (r349461) +++ projects/fuse2/sys/fs/fuse/fuse_node.c Thu Jun 27 16:30:25 2019 (r349462) @@ -59,6 +59,7 @@ __FBSDID("$FreeBSD$"); #include +#include #include #include #include @@ -101,10 +102,10 @@ MALLOC_DEFINE(M_FUSEVN, "fuse_vnode", "fuse vnode priv static int sysctl_fuse_cache_mode(SYSCTL_HANDLER_ARGS); -static int fuse_node_count = 0; +static counter_u64_t fuse_node_count; -SYSCTL_INT(_vfs_fusefs, OID_AUTO, node_count, CTLFLAG_RD, - &fuse_node_count, 0, "Count of FUSE vnodes"); +SYSCTL_COUNTER_U64(_vfs_fusefs_stats, OID_AUTO, node_count, CTLFLAG_RD, + &fuse_node_count, "Count of FUSE vnodes"); int fuse_data_cache_mode = FUSE_CACHE_WT; @@ -158,7 +159,7 @@ fuse_vnode_init(struct vnode *vp, struct fuse_vnode_da vp->v_type = vtyp; vp->v_data = fvdat; - atomic_add_acq_int(&fuse_node_count, 1); + counter_u64_add(fuse_node_count, 1); } void @@ -171,7 +172,7 @@ fuse_vnode_destroy(struct vnode *vp) ("Destroying fuse vnode with open files!")); free(fvdat, M_FUSEVN); - atomic_subtract_acq_int(&fuse_node_count, 1); + counter_u64_add(fuse_node_count, -1); } int @@ -479,4 +480,17 @@ fuse_vnode_update(struct vnode *vp, int flags) fvdat->cached_attrs.va_ctime = ts; fvdat->flag |= flags; +} + +void +fuse_node_init(void) +{ + fuse_node_count = counter_u64_alloc(M_WAITOK); + counter_u64_zero(fuse_node_count); +} + +void +fuse_node_destroy(void) +{ + counter_u64_free(fuse_node_count); } Modified: projects/fuse2/sys/fs/fuse/fuse_node.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_node.h Thu Jun 27 15:51:50 2019 (r349461) +++ projects/fuse2/sys/fs/fuse/fuse_node.h Thu Jun 27 16:30:25 2019 (r349462) @@ -191,4 +191,7 @@ int fuse_vnode_setsize(struct vnode *vp, off_t newsize void fuse_vnode_undirty_cached_timestamps(struct vnode *vp); void fuse_vnode_update(struct vnode *vp, int flags); + +void fuse_node_init(void); +void fuse_node_destroy(void); #endif /* _FUSE_NODE_H_ */ Modified: projects/fuse2/sys/fs/fuse/fuse_vnops.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_vnops.c Thu Jun 27 15:51:50 2019 (r349461) +++ projects/fuse2/sys/fs/fuse/fuse_vnops.c Thu Jun 27 16:30:25 2019 (r349462) @@ -218,16 +218,6 @@ struct vop_vector fuse_vnops = { .vop_vptofh = fuse_vnop_vptofh, }; -u_long fuse_lookup_cache_hits = 0; - -SYSCTL_ULONG(_vfs_fusefs, OID_AUTO, lookup_cache_hits, CTLFLAG_RD, - &fuse_lookup_cache_hits, 0, "number of positive cache hits in lookup"); - -u_long fuse_lookup_cache_misses = 0; - -SYSCTL_ULONG(_vfs_fusefs, OID_AUTO, lookup_cache_misses, CTLFLAG_RD, - &fuse_lookup_cache_misses, 0, "number of cache misses in lookup"); - /* * XXX: This feature is highly experimental and can bring to instabilities, * needs revisiting before to be enabled by default. @@ -1048,11 +1038,10 @@ fuse_vnop_lookup(struct vop_lookup_args *ap) switch (err) { case -1: /* positive match */ if (timespeccmp(&timeout, &now, >)) { - atomic_add_acq_long(&fuse_lookup_cache_hits, 1); + counter_u64_add(fuse_lookup_cache_hits, 1); } else { /* Cache timeout */ - atomic_add_acq_long(&fuse_lookup_cache_misses, - 1); + counter_u64_add(fuse_lookup_cache_misses, 1); bintime_clear( &VTOFUD(*vpp)->entry_cache_timeout); cache_purge(*vpp); @@ -1066,7 +1055,7 @@ fuse_vnop_lookup(struct vop_lookup_args *ap) return 0; case 0: /* no match in cache */ - atomic_add_acq_long(&fuse_lookup_cache_misses, 1); + counter_u64_add(fuse_lookup_cache_misses, 1); break; case ENOENT: /* negative match */ From owner-svn-src-projects@freebsd.org Thu Jun 27 17:44:22 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4C8FC15C8B26 for ; Thu, 27 Jun 2019 17:44:22 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D5FA18F8AC; Thu, 27 Jun 2019 17:44:21 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C548C1AA8A; Thu, 27 Jun 2019 17:44:21 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5RHiLTM053614; Thu, 27 Jun 2019 17:44:21 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5RHiLO3053613; Thu, 27 Jun 2019 17:44:21 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906271744.x5RHiLO3053613@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Thu, 27 Jun 2019 17:44:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349464 - projects/fuse2/tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/tests/sys/fs/fusefs X-SVN-Commit-Revision: 349464 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: D5FA18F8AC X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.95 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.95)[-0.950,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jun 2019 17:44:22 -0000 Author: asomers Date: Thu Jun 27 17:44:21 2019 New Revision: 349464 URL: https://svnweb.freebsd.org/changeset/base/349464 Log: fusefs: fix a memory leak in the forget test Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/tests/sys/fs/fusefs/forget.cc Modified: projects/fuse2/tests/sys/fs/fusefs/forget.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/forget.cc Thu Jun 27 16:48:24 2019 (r349463) +++ projects/fuse2/tests/sys/fs/fusefs/forget.cc Thu Jun 27 17:44:21 2019 (r349464) @@ -96,6 +96,7 @@ TEST_F(Forget, ok) ASSERT_EQ(0, err) << strerror(errno); sem_wait(&sem); + sem_destroy(&sem); } /* From owner-svn-src-projects@freebsd.org Thu Jun 27 17:59:16 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B409315C8D9D for ; Thu, 27 Jun 2019 17:59:16 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4A8468FE00; Thu, 27 Jun 2019 17:59:16 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1BA361AC30; Thu, 27 Jun 2019 17:59:16 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5RHxF7A059252; Thu, 27 Jun 2019 17:59:15 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5RHxFlV059250; Thu, 27 Jun 2019 17:59:15 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906271759.x5RHxFlV059250@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Thu, 27 Jun 2019 17:59:15 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349465 - projects/fuse2/sys/fs/fuse X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/sys/fs/fuse X-SVN-Commit-Revision: 349465 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4A8468FE00 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.97)[-0.969,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jun 2019 17:59:16 -0000 Author: asomers Date: Thu Jun 27 17:59:15 2019 New Revision: 349465 URL: https://svnweb.freebsd.org/changeset/base/349465 Log: fusefs: counter(9) variables should not be statically initialized Reported by: rpokala Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_file.c projects/fuse2/sys/fs/fuse/fuse_internal.c Modified: projects/fuse2/sys/fs/fuse/fuse_file.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_file.c Thu Jun 27 17:44:21 2019 (r349464) +++ projects/fuse2/sys/fs/fuse/fuse_file.c Thu Jun 27 17:59:15 2019 (r349465) @@ -94,7 +94,7 @@ SDT_PROVIDER_DECLARE(fusefs); */ SDT_PROBE_DEFINE2(fusefs, , file, trace, "int", "char*"); -static counter_u64_t fuse_fh_count = 0; +static counter_u64_t fuse_fh_count; SYSCTL_COUNTER_U64(_vfs_fusefs_stats, OID_AUTO, filehandle_count, CTLFLAG_RD, &fuse_fh_count, "number of open FUSE filehandles"); Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_internal.c Thu Jun 27 17:44:21 2019 (r349464) +++ projects/fuse2/sys/fs/fuse/fuse_internal.c Thu Jun 27 17:59:15 2019 (r349465) @@ -108,8 +108,8 @@ static int isbzero(void *buf, size_t len); #endif -counter_u64_t fuse_lookup_cache_hits = 0; -counter_u64_t fuse_lookup_cache_misses = 0; +counter_u64_t fuse_lookup_cache_hits; +counter_u64_t fuse_lookup_cache_misses; SYSCTL_COUNTER_U64(_vfs_fusefs_stats, OID_AUTO, lookup_cache_hits, CTLFLAG_RD, &fuse_lookup_cache_hits, "number of positive cache hits in lookup"); From owner-svn-src-projects@freebsd.org Thu Jun 27 20:18:14 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B9D6515CCC32 for ; Thu, 27 Jun 2019 20:18:14 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 68C836F823; Thu, 27 Jun 2019 20:18:14 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3FB111C4F0; Thu, 27 Jun 2019 20:18:14 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5RKIEgV035784; Thu, 27 Jun 2019 20:18:14 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5RKIDtF035779; Thu, 27 Jun 2019 20:18:13 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906272018.x5RKIDtF035779@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Thu, 27 Jun 2019 20:18:13 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349468 - in projects/fuse2: . share/man/man5 sys/fs/fuse tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: . share/man/man5 sys/fs/fuse tests/sys/fs/fusefs X-SVN-Commit-Revision: 349468 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 68C836F823 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.97)[-0.967,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jun 2019 20:18:15 -0000 Author: asomers Date: Thu Jun 27 20:18:12 2019 New Revision: 349468 URL: https://svnweb.freebsd.org/changeset/base/349468 Log: fusefs: recycle vnodes after their last unlink Previously fusefs would never recycle vnodes. After VOP_INACTIVE, they'd linger around until unmount or the vnlru reclaimed them. This commit essentially actives and inlines the old reclaim_revoked sysctl, and fixes some issues dealing with the attribute cache and multiply linked files. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/UPDATING projects/fuse2/share/man/man5/fusefs.5 projects/fuse2/sys/fs/fuse/fuse_internal.c projects/fuse2/sys/fs/fuse/fuse_vnops.c projects/fuse2/tests/sys/fs/fusefs/rmdir.cc projects/fuse2/tests/sys/fs/fusefs/unlink.cc Modified: projects/fuse2/UPDATING ============================================================================== --- projects/fuse2/UPDATING Thu Jun 27 19:36:30 2019 (r349467) +++ projects/fuse2/UPDATING Thu Jun 27 20:18:12 2019 (r349468) @@ -31,17 +31,17 @@ NOTE TO PEOPLE WHO THINK THAT FreeBSD 13.x IS SLOW: disable the most expensive debugging functionality run "ln -s 'abort:false,junk:false' /etc/malloc.conf".) -20190620: +20190627: The vfs.fusefs.sync_unmount and vfs.fusefs.init_backgrounded sysctls and the "-o sync_unmount" and "-o init_backgrounded" mount options have been removed from mount_fusefs(8). You can safely remove them from your scripts, because they had no effect. The vfs.fusefs.fix_broken_io, vfs.fusefs.sync_resize, - vfs.fusefs.refresh_size, vfs.fusefs.mmap_enable, and - vfs.fusefs.data_cache_invalidate sysctls have been removed. If you - felt the need to set any of them to a non-default value, please tell - asomers@FreeBSD.org why. + vfs.fusefs.refresh_size, vfs.fusefs.mmap_enable, + vfs.fusefs.reclaim_revoked, and vfs.fusefs.data_cache_invalidate + sysctls have been removed. If you felt the need to set any of them to + a non-default value, please tell asomers@FreeBSD.org why. 20190612: Clang, llvm, lld, lldb, compiler-rt, libc++, libunwind and openmp have Modified: projects/fuse2/share/man/man5/fusefs.5 ============================================================================== --- projects/fuse2/share/man/man5/fusefs.5 Thu Jun 27 19:36:30 2019 (r349467) +++ projects/fuse2/share/man/man5/fusefs.5 Thu Jun 27 20:18:12 2019 (r349468) @@ -98,7 +98,6 @@ Current number of allocated FUSE tickets, which is rou number of FUSE operations currently being processed by daemons. .\" Undocumented sysctls .\" ==================== -.\" vfs.fusefs.reclaim_revoked: I don't understand it well-enough .\" vfs.fusefs.enforce_dev_perms: I don't understand it well enough. .\" vfs.fusefs.iov_credit: I don't understand it well enough .\" vfs.fusefs.iov_permanent_bufsize: I don't understand it well enough Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_internal.c Thu Jun 27 19:36:30 2019 (r349467) +++ projects/fuse2/sys/fs/fuse/fuse_internal.c Thu Jun 27 20:18:12 2019 (r349468) @@ -657,6 +657,7 @@ fuse_internal_remove(struct vnode *dvp, enum fuse_opcode op) { struct fuse_dispatcher fdi; + nlink_t nlink; int err = 0; fdisp_init(&fdi, cnp->cn_namelen + 1); @@ -667,6 +668,35 @@ fuse_internal_remove(struct vnode *dvp, err = fdisp_wait_answ(&fdi); fdisp_destroy(&fdi); + + if (err) + return (err); + + /* + * Access the cached nlink even if the attr cached has expired. If + * it's inaccurate, the worst that will happen is: + * 1) We'll recycle the vnode even though the file has another link we + * don't know about, costing a bit of cpu time, or + * 2) We won't recycle the vnode even though all of its links are gone. + * It will linger around until vnlru reclaims it, costing a bit of + * temporary memory. + */ + nlink = VTOFUD(vp)->cached_attrs.va_nlink--; + + /* + * Purge the parent's attribute cache because the daemon + * should've updated its mtime and ctime. + */ + fuse_vnode_clear_attr_cache(dvp); + + /* NB: nlink could be zero if it was never cached */ + if (nlink <= 1 || vnode_vtype(vp) == VDIR) { + fuse_internal_vnode_disappear(vp); + } else { + cache_purge(vp); + fuse_vnode_update(vp, FN_CTIMECHANGE); + } + return err; } @@ -894,8 +924,6 @@ fuse_internal_vnode_disappear(struct vnode *vp) ASSERT_VOP_ELOCKED(vp, "fuse_internal_vnode_disappear"); fvdat->flag |= FN_REVOKED; - bintime_clear(&fvdat->attr_cache_timeout); - bintime_clear(&fvdat->entry_cache_timeout); cache_purge(vp); } Modified: projects/fuse2/sys/fs/fuse/fuse_vnops.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_vnops.c Thu Jun 27 19:36:30 2019 (r349467) +++ projects/fuse2/sys/fs/fuse/fuse_vnops.c Thu Jun 27 20:18:12 2019 (r349468) @@ -218,15 +218,6 @@ struct vop_vector fuse_vnops = { .vop_vptofh = fuse_vnop_vptofh, }; -/* - * XXX: This feature is highly experimental and can bring to instabilities, - * needs revisiting before to be enabled by default. - */ -static int fuse_reclaim_revoked = 0; - -SYSCTL_INT(_vfs_fusefs, OID_AUTO, reclaim_revoked, CTLFLAG_RW, - &fuse_reclaim_revoked, 0, ""); - uma_zone_t fuse_pbuf_zone; #define fuse_vm_page_lock(m) vm_page_lock((m)); @@ -880,9 +871,9 @@ fuse_vnop_inactive(struct vop_inactive_args *ap) fuse_filehandle_close(vp, fufh, td, NULL); } - if ((fvdat->flag & FN_REVOKED) != 0 && fuse_reclaim_revoked) { + if ((fvdat->flag & FN_REVOKED) != 0) vrecycle(vp); - } + return 0; } @@ -1568,18 +1559,9 @@ fuse_vnop_remove(struct vop_remove_args *ap) if (vnode_isdir(vp)) { return EPERM; } - cache_purge(vp); err = fuse_internal_remove(dvp, vp, cnp, FUSE_UNLINK); - if (err == 0) { - fuse_internal_vnode_disappear(vp); - /* - * Purge the parent's attribute cache because the daemon - * should've updated its mtime and ctime - */ - fuse_vnode_clear_attr_cache(dvp); - } return err; } @@ -1691,14 +1673,6 @@ fuse_vnop_rmdir(struct vop_rmdir_args *ap) } err = fuse_internal_remove(dvp, vp, ap->a_cnp, FUSE_RMDIR); - if (err == 0) { - fuse_internal_vnode_disappear(vp); - /* - * Purge the parent's attribute cache because the daemon - * should've updated its mtime and ctime - */ - fuse_vnode_clear_attr_cache(dvp); - } return err; } Modified: projects/fuse2/tests/sys/fs/fusefs/rmdir.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/rmdir.cc Thu Jun 27 19:36:30 2019 (r349467) +++ projects/fuse2/tests/sys/fs/fusefs/rmdir.cc Thu Jun 27 20:18:12 2019 (r349468) @@ -30,6 +30,7 @@ extern "C" { #include +#include } #include "mockfs.hh" @@ -55,11 +56,13 @@ void expect_getattr(uint64_t ino, mode_t mode) }))); } -void expect_lookup(const char *relpath, uint64_t ino) +void expect_lookup(const char *relpath, uint64_t ino, int times=1) { EXPECT_LOOKUP(FUSE_ROOT_ID, relpath) - .WillOnce(Invoke(ReturnImmediate([=](auto in __unused, auto& out) { + .Times(times) + .WillRepeatedly(Invoke(ReturnImmediate([=](auto in __unused, auto& out) { SET_OUT_HEADER_LEN(out, entry); + out.body.entry.attr_valid = UINT64_MAX; out.body.entry.attr.mode = S_IFDIR | 0755; out.body.entry.nodeid = ino; out.body.entry.attr.nlink = 2; @@ -83,13 +86,16 @@ void expect_rmdir(uint64_t parent, const char *relpath * A successful rmdir should clear the parent directory's attribute cache, * because the fuse daemon should update its mtime and ctime */ -TEST_F(Rmdir, clear_attr_cache) +TEST_F(Rmdir, parent_attr_cache) { const char FULLPATH[] = "mountpoint/some_dir"; const char RELPATH[] = "some_dir"; struct stat sb; + sem_t sem; uint64_t ino = 42; + ASSERT_EQ(0, sem_init(&sem, 0, 0)) << strerror(errno); + EXPECT_CALL(*m_mock, process( ResultOf([=](auto in) { return (in.header.opcode == FUSE_GETATTR && @@ -105,9 +111,12 @@ TEST_F(Rmdir, clear_attr_cache) }))); expect_lookup(RELPATH, ino); expect_rmdir(FUSE_ROOT_ID, RELPATH, 0); + expect_forget(ino, 1, &sem); ASSERT_EQ(0, rmdir(FULLPATH)) << strerror(errno); EXPECT_EQ(0, stat("mountpoint", &sb)) << strerror(errno); + sem_wait(&sem); + sem_destroy(&sem); } TEST_F(Rmdir, enotempty) @@ -124,15 +133,40 @@ TEST_F(Rmdir, enotempty) ASSERT_EQ(ENOTEMPTY, errno); } +/* Removing a directory should expire its entry cache */ +TEST_F(Rmdir, entry_cache) +{ + const char FULLPATH[] = "mountpoint/some_dir"; + const char RELPATH[] = "some_dir"; + sem_t sem; + uint64_t ino = 42; + + expect_getattr(1, S_IFDIR | 0755); + expect_lookup(RELPATH, ino, 2); + expect_rmdir(FUSE_ROOT_ID, RELPATH, 0); + expect_forget(ino, 1, &sem); + + ASSERT_EQ(0, rmdir(FULLPATH)) << strerror(errno); + ASSERT_EQ(0, access(FULLPATH, F_OK)) << strerror(errno); + sem_wait(&sem); + sem_destroy(&sem); +} + TEST_F(Rmdir, ok) { const char FULLPATH[] = "mountpoint/some_dir"; const char RELPATH[] = "some_dir"; + sem_t sem; uint64_t ino = 42; + ASSERT_EQ(0, sem_init(&sem, 0, 0)) << strerror(errno); + expect_getattr(FUSE_ROOT_ID, S_IFDIR | 0755); expect_lookup(RELPATH, ino); expect_rmdir(FUSE_ROOT_ID, RELPATH, 0); + expect_forget(ino, 1, &sem); ASSERT_EQ(0, rmdir(FULLPATH)) << strerror(errno); + sem_wait(&sem); + sem_destroy(&sem); } Modified: projects/fuse2/tests/sys/fs/fusefs/unlink.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/unlink.cc Thu Jun 27 19:36:30 2019 (r349467) +++ projects/fuse2/tests/sys/fs/fusefs/unlink.cc Thu Jun 27 20:18:12 2019 (r349468) @@ -29,6 +29,7 @@ extern "C" { #include +#include } #include "mockfs.hh" @@ -54,18 +55,61 @@ void expect_getattr(uint64_t ino, mode_t mode) }))); } -void expect_lookup(const char *relpath, uint64_t ino, int times) +void expect_lookup(const char *relpath, uint64_t ino, int times, int nlink=1) { - FuseTest::expect_lookup(relpath, ino, S_IFREG | 0644, 0, times); + EXPECT_LOOKUP(FUSE_ROOT_ID, relpath) + .Times(times) + .WillRepeatedly(Invoke( + ReturnImmediate([=](auto in __unused, auto& out) { + SET_OUT_HEADER_LEN(out, entry); + out.body.entry.attr.mode = S_IFREG | 0644; + out.body.entry.nodeid = ino; + out.body.entry.attr.nlink = nlink; + out.body.entry.attr_valid = UINT64_MAX; + out.body.entry.attr.size = 0; + }))); } }; /* + * Unlinking a multiply linked file should update its ctime and nlink. This + * could be handled simply by invalidating the attributes, necessitating a new + * GETATTR, but we implement it in-kernel for efficiency's sake. + */ +TEST_F(Unlink, attr_cache) +{ + const char FULLPATH0[] = "mountpoint/some_file.txt"; + const char RELPATH0[] = "some_file.txt"; + const char FULLPATH1[] = "mountpoint/other_file.txt"; + const char RELPATH1[] = "other_file.txt"; + uint64_t ino = 42; + struct stat sb_old, sb_new; + int fd1; + + expect_getattr(1, S_IFDIR | 0755); + expect_lookup(RELPATH0, ino, 1, 2); + expect_lookup(RELPATH1, ino, 1, 2); + expect_open(ino, 0, 1); + expect_unlink(1, RELPATH0, 0); + + fd1 = open(FULLPATH1, O_RDONLY); + ASSERT_LE(0, fd1) << strerror(errno); + + ASSERT_EQ(0, fstat(fd1, &sb_old)) << strerror(errno); + ASSERT_EQ(0, unlink(FULLPATH0)) << strerror(errno); + ASSERT_EQ(0, fstat(fd1, &sb_new)) << strerror(errno); + EXPECT_NE(sb_old.st_ctime, sb_new.st_ctime); + EXPECT_EQ(1u, sb_new.st_nlink); + + leak(fd1); +} + +/* * A successful unlink should clear the parent directory's attribute cache, * because the fuse daemon should update its mtime and ctime */ -TEST_F(Unlink, clear_attr_cache) +TEST_F(Unlink, parent_attr_cache) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; @@ -85,7 +129,8 @@ TEST_F(Unlink, clear_attr_cache) out.body.attr.attr.mode = S_IFDIR | 0755; out.body.attr.attr_valid = UINT64_MAX; }))); - expect_lookup(RELPATH, ino, 1); + /* Use nlink=2 so we don't get a FUSE_FORGET */ + expect_lookup(RELPATH, ino, 1, 2); expect_unlink(1, RELPATH, 0); ASSERT_EQ(0, unlink(FULLPATH)) << strerror(errno); @@ -106,34 +151,101 @@ TEST_F(Unlink, eperm) ASSERT_EQ(EPERM, errno); } +/* + * Unlinking a file should expire its entry cache, even if it's multiply linked + */ +TEST_F(Unlink, entry_cache) +{ + const char FULLPATH[] = "mountpoint/some_file.txt"; + const char RELPATH[] = "some_file.txt"; + uint64_t ino = 42; + + expect_getattr(1, S_IFDIR | 0755); + expect_lookup(RELPATH, ino, 2, 2); + expect_unlink(1, RELPATH, 0); + + ASSERT_EQ(0, unlink(FULLPATH)) << strerror(errno); + ASSERT_EQ(0, access(FULLPATH, F_OK)) << strerror(errno); +} + +/* + * Unlink a multiply-linked file. There should be no FUSE_FORGET because the + * file is still linked. + */ +TEST_F(Unlink, multiply_linked) +{ + const char FULLPATH0[] = "mountpoint/some_file.txt"; + const char RELPATH0[] = "some_file.txt"; + const char FULLPATH1[] = "mountpoint/other_file.txt"; + const char RELPATH1[] = "other_file.txt"; + uint64_t ino = 42; + + expect_getattr(1, S_IFDIR | 0755); + expect_lookup(RELPATH0, ino, 1, 2); + expect_unlink(1, RELPATH0, 0); + EXPECT_CALL(*m_mock, process( + ResultOf([=](auto in) { + return (in.header.opcode == FUSE_FORGET && + in.header.nodeid == ino); + }, Eq(true)), + _) + ).Times(0); + expect_lookup(RELPATH1, ino, 1, 1); + + ASSERT_EQ(0, unlink(FULLPATH0)) << strerror(errno); + + /* + * The final syscall simply ensures that no FUSE_FORGET was ever sent, + * by scheduling an arbitrary different operation after a FUSE_FORGET + * would've been sent. + */ + ASSERT_EQ(0, access(FULLPATH1, F_OK)) << strerror(errno); +} + TEST_F(Unlink, ok) { const char FULLPATH[] = "mountpoint/some_file.txt"; const char RELPATH[] = "some_file.txt"; uint64_t ino = 42; + sem_t sem; + ASSERT_EQ(0, sem_init(&sem, 0, 0)) << strerror(errno); + expect_getattr(1, S_IFDIR | 0755); expect_lookup(RELPATH, ino, 1); expect_unlink(1, RELPATH, 0); + expect_forget(ino, 1, &sem); ASSERT_EQ(0, unlink(FULLPATH)) << strerror(errno); + sem_wait(&sem); + sem_destroy(&sem); } /* Unlink an open file */ TEST_F(Unlink, open_but_deleted) { - const char FULLPATH[] = "mountpoint/some_file.txt"; - const char RELPATH[] = "some_file.txt"; + const char FULLPATH0[] = "mountpoint/some_file.txt"; + const char RELPATH0[] = "some_file.txt"; + const char FULLPATH1[] = "mountpoint/other_file.txt"; + const char RELPATH1[] = "other_file.txt"; uint64_t ino = 42; int fd; expect_getattr(1, S_IFDIR | 0755); - expect_lookup(RELPATH, ino, 2); + expect_lookup(RELPATH0, ino, 2); expect_open(ino, 0, 1); - expect_unlink(1, RELPATH, 0); + expect_unlink(1, RELPATH0, 0); + expect_lookup(RELPATH1, ino, 1, 1); - fd = open(FULLPATH, O_RDWR); + fd = open(FULLPATH0, O_RDWR); ASSERT_LE(0, fd) << strerror(errno); - ASSERT_EQ(0, unlink(FULLPATH)) << strerror(errno); + ASSERT_EQ(0, unlink(FULLPATH0)) << strerror(errno); + + /* + * The final syscall simply ensures that no FUSE_FORGET was ever sent, + * by scheduling an arbitrary different operation after a FUSE_FORGET + * would've been sent. + */ + ASSERT_EQ(0, access(FULLPATH1, F_OK)) << strerror(errno); leak(fd); } From owner-svn-src-projects@freebsd.org Thu Jun 27 22:24:57 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CC21E15CFEEE for ; Thu, 27 Jun 2019 22:24:57 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6F4B774611; Thu, 27 Jun 2019 22:24:57 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4BF6C1DBCF; Thu, 27 Jun 2019 22:24:57 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5RMOvi5003932; Thu, 27 Jun 2019 22:24:57 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5RMOvCe003931; Thu, 27 Jun 2019 22:24:57 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906272224.x5RMOvCe003931@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Thu, 27 Jun 2019 22:24:57 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349473 - projects/fuse2/sys/fs/fuse X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: projects/fuse2/sys/fs/fuse X-SVN-Commit-Revision: 349473 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 6F4B774611 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.98)[-0.978,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jun 2019 22:24:58 -0000 Author: asomers Date: Thu Jun 27 22:24:56 2019 New Revision: 349473 URL: https://svnweb.freebsd.org/changeset/base/349473 Log: fusefs: fix a memory leak regarding FUSE_INTERRUPT We were leaking the fuse ticket if the original operation completed before the daemon received the INTERRUPT operation. Fixing this was easier than I expected. Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_device.c Modified: projects/fuse2/sys/fs/fuse/fuse_device.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_device.c Thu Jun 27 22:18:21 2019 (r349472) +++ projects/fuse2/sys/fs/fuse/fuse_device.c Thu Jun 27 22:24:56 2019 (r349473) @@ -473,6 +473,7 @@ fuse_device_write(struct cdev *dev, struct uio *uio, i x_tick) { if (itick->tk_unique == tick->irq_unique) { fuse_aw_remove(itick); + fuse_ticket_drop(itick); break; } } From owner-svn-src-projects@freebsd.org Thu Jun 27 23:51:04 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ABEFE15D20A3 for ; Thu, 27 Jun 2019 23:51:03 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 55AEC776BF; Thu, 27 Jun 2019 23:51:03 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 184621EA67; Thu, 27 Jun 2019 23:51:03 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5RNp26H046037; Thu, 27 Jun 2019 23:51:02 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5RNotjD045997; Thu, 27 Jun 2019 23:50:55 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906272350.x5RNotjD045997@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Thu, 27 Jun 2019 23:50:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349479 - in projects/fuse2: . contrib/elftoolchain/elfcopy contrib/elftoolchain/libdwarf contrib/elftoolchain/libelftc contrib/gcc/config/rs6000 contrib/ipfilter/man contrib/ipfilter/t... X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: . contrib/elftoolchain/elfcopy contrib/elftoolchain/libdwarf contrib/elftoolchain/libelftc contrib/gcc/config/rs6000 contrib/ipfilter/man contrib/ipfilter/tools contrib/llvm/lib/Tar... X-SVN-Commit-Revision: 349479 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 55AEC776BF X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.96)[-0.958,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jun 2019 23:51:04 -0000 Author: asomers Date: Thu Jun 27 23:50:54 2019 New Revision: 349479 URL: https://svnweb.freebsd.org/changeset/base/349479 Log: MFHead @349476 Sponsored by: The FreeBSD Foundation Added: projects/fuse2/stand/efi/include/Protocol/Http.h - copied unchanged from r349476, head/stand/efi/include/Protocol/Http.h projects/fuse2/stand/efi/include/Protocol/Ip4Config2.h - copied unchanged from r349476, head/stand/efi/include/Protocol/Ip4Config2.h projects/fuse2/stand/efi/include/Protocol/ServiceBinding.h - copied unchanged from r349476, head/stand/efi/include/Protocol/ServiceBinding.h projects/fuse2/stand/efi/libefi/efihttp.c - copied unchanged from r349476, head/stand/efi/libefi/efihttp.c projects/fuse2/sys/arm/conf/NOTES.armv5 - copied unchanged from r349476, head/sys/arm/conf/NOTES.armv5 projects/fuse2/sys/arm/conf/NOTES.armv7 - copied unchanged from r349476, head/sys/arm/conf/NOTES.armv7 projects/fuse2/usr.sbin/bhyve/audio.c - copied unchanged from r349476, head/usr.sbin/bhyve/audio.c projects/fuse2/usr.sbin/bhyve/audio.h - copied unchanged from r349476, head/usr.sbin/bhyve/audio.h projects/fuse2/usr.sbin/bhyve/hda_codec.c - copied unchanged from r349476, head/usr.sbin/bhyve/hda_codec.c projects/fuse2/usr.sbin/bhyve/hda_reg.h - copied unchanged from r349476, head/usr.sbin/bhyve/hda_reg.h projects/fuse2/usr.sbin/bhyve/hdac_reg.h - copied unchanged from r349476, head/usr.sbin/bhyve/hdac_reg.h projects/fuse2/usr.sbin/bhyve/pci_hda.c - copied unchanged from r349476, head/usr.sbin/bhyve/pci_hda.c projects/fuse2/usr.sbin/bhyve/pci_hda.h - copied unchanged from r349476, head/usr.sbin/bhyve/pci_hda.h Deleted: projects/fuse2/lib/libnandfs/Makefile projects/fuse2/lib/libnandfs/Makefile.depend projects/fuse2/lib/libnandfs/libnandfs.h projects/fuse2/lib/libnandfs/nandfs.c projects/fuse2/sbin/nandfs/Makefile projects/fuse2/sbin/nandfs/Makefile.depend projects/fuse2/sbin/nandfs/lssnap.c projects/fuse2/sbin/nandfs/mksnap.c projects/fuse2/sbin/nandfs/nandfs.8 projects/fuse2/sbin/nandfs/nandfs.c projects/fuse2/sbin/nandfs/nandfs.h projects/fuse2/sbin/nandfs/rmsnap.c projects/fuse2/sbin/newfs_nandfs/Makefile projects/fuse2/sbin/newfs_nandfs/Makefile.depend projects/fuse2/sbin/newfs_nandfs/newfs_nandfs.8 projects/fuse2/sbin/newfs_nandfs/newfs_nandfs.c projects/fuse2/share/man/man4/nand.4 projects/fuse2/share/man/man4/nandsim.4 projects/fuse2/share/man/man5/nandfs.5 projects/fuse2/share/man/man9/pwm.9 projects/fuse2/stand/libsa/nandfs.c projects/fuse2/sys/arm/freescale/vybrid/vf_nfc.c projects/fuse2/sys/dev/nand/nand.c projects/fuse2/sys/dev/nand/nand.h projects/fuse2/sys/dev/nand/nand_bbt.c projects/fuse2/sys/dev/nand/nand_cdev.c projects/fuse2/sys/dev/nand/nand_dev.h projects/fuse2/sys/dev/nand/nand_ecc_pos.h projects/fuse2/sys/dev/nand/nand_generic.c projects/fuse2/sys/dev/nand/nand_geom.c projects/fuse2/sys/dev/nand/nand_id.c projects/fuse2/sys/dev/nand/nand_if.m projects/fuse2/sys/dev/nand/nandbus.c projects/fuse2/sys/dev/nand/nandbus.h projects/fuse2/sys/dev/nand/nandbus_if.m projects/fuse2/sys/dev/nand/nandsim.c projects/fuse2/sys/dev/nand/nandsim.h projects/fuse2/sys/dev/nand/nandsim_chip.c projects/fuse2/sys/dev/nand/nandsim_chip.h projects/fuse2/sys/dev/nand/nandsim_ctrl.c projects/fuse2/sys/dev/nand/nandsim_log.c projects/fuse2/sys/dev/nand/nandsim_log.h projects/fuse2/sys/dev/nand/nandsim_swap.c projects/fuse2/sys/dev/nand/nandsim_swap.h projects/fuse2/sys/dev/nand/nfc_fsl.c projects/fuse2/sys/dev/nand/nfc_fsl.h projects/fuse2/sys/dev/nand/nfc_if.m projects/fuse2/sys/dev/nand/nfc_mv.c projects/fuse2/sys/dev/nand/nfc_rb.c projects/fuse2/sys/fs/nandfs/bmap.c projects/fuse2/sys/fs/nandfs/bmap.h projects/fuse2/sys/fs/nandfs/nandfs.h projects/fuse2/sys/fs/nandfs/nandfs_alloc.c projects/fuse2/sys/fs/nandfs/nandfs_bmap.c projects/fuse2/sys/fs/nandfs/nandfs_buffer.c projects/fuse2/sys/fs/nandfs/nandfs_cleaner.c projects/fuse2/sys/fs/nandfs/nandfs_cpfile.c projects/fuse2/sys/fs/nandfs/nandfs_dat.c projects/fuse2/sys/fs/nandfs/nandfs_dir.c projects/fuse2/sys/fs/nandfs/nandfs_fs.h projects/fuse2/sys/fs/nandfs/nandfs_ifile.c projects/fuse2/sys/fs/nandfs/nandfs_mount.h projects/fuse2/sys/fs/nandfs/nandfs_segment.c projects/fuse2/sys/fs/nandfs/nandfs_subr.c projects/fuse2/sys/fs/nandfs/nandfs_subr.h projects/fuse2/sys/fs/nandfs/nandfs_sufile.c projects/fuse2/sys/fs/nandfs/nandfs_vfsops.c projects/fuse2/sys/fs/nandfs/nandfs_vnops.c projects/fuse2/sys/modules/nand/Makefile projects/fuse2/sys/modules/nandfs/Makefile projects/fuse2/sys/modules/nandsim/Makefile projects/fuse2/usr.sbin/nandsim/Makefile projects/fuse2/usr.sbin/nandsim/Makefile.depend projects/fuse2/usr.sbin/nandsim/nandsim.8 projects/fuse2/usr.sbin/nandsim/nandsim.c projects/fuse2/usr.sbin/nandsim/nandsim_cfgparse.c projects/fuse2/usr.sbin/nandsim/nandsim_cfgparse.h projects/fuse2/usr.sbin/nandsim/nandsim_rcfile.c projects/fuse2/usr.sbin/nandsim/nandsim_rcfile.h projects/fuse2/usr.sbin/nandsim/sample.conf projects/fuse2/usr.sbin/nandtool/Makefile projects/fuse2/usr.sbin/nandtool/Makefile.depend projects/fuse2/usr.sbin/nandtool/nand_erase.c projects/fuse2/usr.sbin/nandtool/nand_info.c projects/fuse2/usr.sbin/nandtool/nand_read.c projects/fuse2/usr.sbin/nandtool/nand_readoob.c projects/fuse2/usr.sbin/nandtool/nand_write.c projects/fuse2/usr.sbin/nandtool/nand_writeoob.c projects/fuse2/usr.sbin/nandtool/nandtool.8 projects/fuse2/usr.sbin/nandtool/nandtool.c projects/fuse2/usr.sbin/nandtool/nandtool.h projects/fuse2/usr.sbin/nandtool/usage.h Modified: projects/fuse2/UPDATING projects/fuse2/contrib/elftoolchain/elfcopy/sections.c projects/fuse2/contrib/elftoolchain/libdwarf/libdwarf_attr.c projects/fuse2/contrib/elftoolchain/libelftc/elftc_string_table.c projects/fuse2/contrib/elftoolchain/libelftc/elftc_string_table_create.3 projects/fuse2/contrib/elftoolchain/libelftc/libelftc.h projects/fuse2/contrib/gcc/config/rs6000/tramp.asm projects/fuse2/contrib/ipfilter/man/ipmon.8 projects/fuse2/contrib/ipfilter/tools/ipmon.c projects/fuse2/contrib/llvm/lib/Target/PowerPC/PPCSubtarget.cpp projects/fuse2/contrib/llvm/tools/clang/lib/Driver/ToolChains/Arch/PPC.cpp projects/fuse2/etc/mtree/BSD.include.dist projects/fuse2/gnu/usr.bin/cc/cc_tools/Makefile.hdrs projects/fuse2/include/Makefile projects/fuse2/lib/Makefile projects/fuse2/lib/libbe/be_access.c projects/fuse2/lib/libc/gen/Symbol.map projects/fuse2/lib/libc/gen/_pthread_stubs.c projects/fuse2/lib/libc/gen/libc_dlopen.c projects/fuse2/lib/libc/gen/opendir.c projects/fuse2/lib/libc/gen/telldir.c projects/fuse2/lib/libc/include/libc_private.h projects/fuse2/lib/libc/powerpc/SYS.h projects/fuse2/lib/libc/powerpc/gen/_ctx_start.S projects/fuse2/lib/libc/powerpc/sys/cerror.S projects/fuse2/lib/libc/stdlib/realpath.c projects/fuse2/lib/libc/sys/mmap.2 projects/fuse2/lib/libc/sys/mprotect.2 projects/fuse2/lib/libjail/jail.c projects/fuse2/lib/libsecureboot/h/libsecureboot.h projects/fuse2/lib/libsecureboot/libsecureboot-priv.h projects/fuse2/lib/libsecureboot/local.trust.mk projects/fuse2/lib/libsecureboot/openpgp/opgp_key.c projects/fuse2/lib/libsecureboot/readfile.c projects/fuse2/lib/libsecureboot/verify_file.c projects/fuse2/lib/libsecureboot/vets.c projects/fuse2/lib/libthr/thread/thr_init.c projects/fuse2/lib/libusb/libusb10.h projects/fuse2/lib/libusb/libusb10_hotplug.c projects/fuse2/libexec/rc/rc.d/motd projects/fuse2/libexec/rtld-elf/debug.h projects/fuse2/libexec/rtld-elf/powerpc/reloc.c projects/fuse2/libexec/rtld-elf/powerpc/rtld_start.S projects/fuse2/libexec/rtld-elf/rtld.c projects/fuse2/libexec/rtld-elf/rtld.h projects/fuse2/libexec/rtld-elf/rtld_printf.h projects/fuse2/release/Makefile.mirrors projects/fuse2/release/picobsd/bridge/PICOBSD projects/fuse2/release/picobsd/build/picobsd projects/fuse2/release/picobsd/qemu/PICOBSD projects/fuse2/sbin/Makefile projects/fuse2/sbin/bectl/bectl.c projects/fuse2/sbin/bectl/tests/bectl_test.sh projects/fuse2/sbin/camcontrol/camcontrol.c projects/fuse2/sbin/dhclient/options.c projects/fuse2/sbin/dhclient/packet.c projects/fuse2/sbin/ipf/ipmon/Makefile projects/fuse2/sbin/ipfw/ipfw.8 projects/fuse2/sbin/ipfw/ipfw2.c projects/fuse2/sbin/ipfw/ipfw2.h projects/fuse2/sbin/swapon/swapon.8 projects/fuse2/sbin/swapon/swapon.c projects/fuse2/share/examples/etc/make.conf projects/fuse2/share/man/man4/Makefile projects/fuse2/share/man/man4/gpio.4 projects/fuse2/share/man/man4/owc.4 projects/fuse2/share/man/man4/random.4 projects/fuse2/share/man/man5/Makefile projects/fuse2/share/man/man5/fstab.5 projects/fuse2/share/man/man9/Makefile projects/fuse2/share/man/man9/VOP_REVOKE.9 projects/fuse2/share/man/man9/iflibdi.9 projects/fuse2/share/man/man9/pwmbus.9 projects/fuse2/share/man/man9/vm_map_protect.9 projects/fuse2/share/misc/bsd-family-tree projects/fuse2/share/mk/bsd.cpu.mk projects/fuse2/share/mk/bsd.libnames.mk projects/fuse2/share/mk/src.libnames.mk projects/fuse2/share/mk/src.opts.mk projects/fuse2/stand/arm/uboot/conf.c projects/fuse2/stand/arm/uboot/version projects/fuse2/stand/common/part.c projects/fuse2/stand/common/part.h projects/fuse2/stand/common/paths.h projects/fuse2/stand/efi/include/efidevp.h projects/fuse2/stand/efi/include/efilib.h projects/fuse2/stand/efi/libefi/Makefile projects/fuse2/stand/efi/loader/conf.c projects/fuse2/stand/efi/loader/copy.c projects/fuse2/stand/efi/loader/main.c projects/fuse2/stand/i386/loader/conf.c projects/fuse2/stand/libsa/Makefile projects/fuse2/stand/libsa/stand.h projects/fuse2/stand/loader.mk projects/fuse2/stand/mips/uboot/conf.c projects/fuse2/stand/mips/uboot/version projects/fuse2/sys/amd64/amd64/pmap.c projects/fuse2/sys/amd64/conf/GENERIC projects/fuse2/sys/amd64/conf/MINIMAL projects/fuse2/sys/amd64/sgx/sgx.c projects/fuse2/sys/amd64/vmm/vmm_instruction_emul.c projects/fuse2/sys/arm/allwinner/files.allwinner projects/fuse2/sys/arm/conf/ALPINE projects/fuse2/sys/arm/conf/ARMADA38X projects/fuse2/sys/arm/conf/ARMADAXP projects/fuse2/sys/arm/conf/DB-78XXX projects/fuse2/sys/arm/conf/DB-88F5XXX projects/fuse2/sys/arm/conf/DB-88F6XXX projects/fuse2/sys/arm/conf/DOCKSTAR projects/fuse2/sys/arm/conf/DREAMPLUG-1001 projects/fuse2/sys/arm/conf/EFIKA_MX projects/fuse2/sys/arm/conf/GENERIC projects/fuse2/sys/arm/conf/IMX53 projects/fuse2/sys/arm/conf/IMX6 projects/fuse2/sys/arm/conf/NOTES projects/fuse2/sys/arm/conf/RPI-B projects/fuse2/sys/arm/conf/RT1310 projects/fuse2/sys/arm/conf/SHEEVAPLUG projects/fuse2/sys/arm/conf/SOCFPGA projects/fuse2/sys/arm/conf/TEGRA124 projects/fuse2/sys/arm/conf/TS7800 projects/fuse2/sys/arm/conf/VERSATILEPB projects/fuse2/sys/arm/conf/VYBRID projects/fuse2/sys/arm/conf/ZEDBOARD projects/fuse2/sys/arm/mv/files.arm7 projects/fuse2/sys/arm/mv/files.mv projects/fuse2/sys/arm/ti/am335x/am335x_ehrpwm.c projects/fuse2/sys/arm64/acpica/acpi_iort.c projects/fuse2/sys/arm64/arm64/freebsd32_machdep.c projects/fuse2/sys/arm64/arm64/gic_v3.c projects/fuse2/sys/arm64/arm64/gic_v3_var.h projects/fuse2/sys/arm64/arm64/gicv3_its.c projects/fuse2/sys/arm64/arm64/machdep.c projects/fuse2/sys/arm64/arm64/pmap.c projects/fuse2/sys/arm64/conf/GENERIC projects/fuse2/sys/cam/ata/ata_all.c projects/fuse2/sys/cam/ata/ata_all.h projects/fuse2/sys/cam/ata/ata_xpt.c projects/fuse2/sys/cam/cam_xpt.c projects/fuse2/sys/cam/ctl/ctl_error.c projects/fuse2/sys/cam/scsi/scsi_all.c projects/fuse2/sys/cam/scsi/scsi_da.c projects/fuse2/sys/cam/scsi/scsi_enc.c projects/fuse2/sys/cam/scsi/scsi_enc.h projects/fuse2/sys/cam/scsi/scsi_enc_internal.h projects/fuse2/sys/cam/scsi/scsi_enc_safte.c projects/fuse2/sys/cam/scsi/scsi_enc_ses.c projects/fuse2/sys/cam/scsi/scsi_ses.h projects/fuse2/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c projects/fuse2/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/multilist.c projects/fuse2/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/multilist.h projects/fuse2/sys/compat/linuxkpi/common/include/asm/atomic-long.h projects/fuse2/sys/compat/linuxkpi/common/include/linux/rculist.h projects/fuse2/sys/conf/NOTES projects/fuse2/sys/conf/files projects/fuse2/sys/conf/files.amd64 projects/fuse2/sys/conf/files.arm projects/fuse2/sys/conf/files.arm64 projects/fuse2/sys/conf/files.powerpc projects/fuse2/sys/conf/kern.opts.mk projects/fuse2/sys/conf/makeLINT.mk projects/fuse2/sys/conf/options projects/fuse2/sys/contrib/ipfilter/netinet/fil.c projects/fuse2/sys/contrib/ipfilter/netinet/ip_compat.h projects/fuse2/sys/contrib/ipfilter/netinet/ip_fil.h projects/fuse2/sys/ddb/db_ps.c projects/fuse2/sys/dev/ahci/ahci.c projects/fuse2/sys/dev/ahci/ahci.h projects/fuse2/sys/dev/ahci/ahciem.c projects/fuse2/sys/dev/altera/msgdma/msgdma.c projects/fuse2/sys/dev/altera/msgdma/msgdma.h projects/fuse2/sys/dev/cxgbe/tom/t4_cpl_io.c projects/fuse2/sys/dev/cxgbe/tom/t4_ddp.c projects/fuse2/sys/dev/cxgbe/tom/t4_tom.h projects/fuse2/sys/dev/drm2/ttm/ttm_bo_vm.c projects/fuse2/sys/dev/gpio/gpiobus.c projects/fuse2/sys/dev/gpio/gpiobusvar.h projects/fuse2/sys/dev/gpio/ofw_gpiobus.c projects/fuse2/sys/dev/iicbus/ad7418.c projects/fuse2/sys/dev/ow/owc_gpiobus.c projects/fuse2/sys/dev/usb/usb_hub_acpi.c projects/fuse2/sys/dev/virtio/scsi/virtio_scsi.c projects/fuse2/sys/fs/cuse/cuse.c projects/fuse2/sys/fs/fifofs/fifo_vnops.c projects/fuse2/sys/fs/smbfs/smbfs_io.c projects/fuse2/sys/geom/geom_flashmap.c projects/fuse2/sys/i386/conf/GENERIC projects/fuse2/sys/i386/conf/MINIMAL projects/fuse2/sys/kern/Make.tags.inc projects/fuse2/sys/kern/kern_descrip.c projects/fuse2/sys/kern/kern_mib.c projects/fuse2/sys/kern/kern_rangelock.c projects/fuse2/sys/kern/kern_sig.c projects/fuse2/sys/kern/sys_pipe.c projects/fuse2/sys/kern/uipc_socket.c projects/fuse2/sys/kern/vfs_vnops.c projects/fuse2/sys/mips/conf/BCM projects/fuse2/sys/mips/conf/BERI_DE4_BASE projects/fuse2/sys/mips/conf/DIR-825B1 projects/fuse2/sys/mips/conf/ERL projects/fuse2/sys/mips/conf/JZ4780 projects/fuse2/sys/mips/conf/OCTEON1 projects/fuse2/sys/mips/conf/PB92 projects/fuse2/sys/mips/conf/PICOSTATION_M2HP projects/fuse2/sys/mips/conf/WZR-300HP projects/fuse2/sys/mips/conf/WZR-HPAG300H projects/fuse2/sys/mips/conf/X1000 projects/fuse2/sys/mips/conf/std.AR5312 projects/fuse2/sys/mips/conf/std.AR5315 projects/fuse2/sys/mips/conf/std.AR_MIPS_BASE projects/fuse2/sys/mips/conf/std.BERI projects/fuse2/sys/mips/conf/std.MALTA projects/fuse2/sys/mips/conf/std.XLP projects/fuse2/sys/mips/mediatek/std.mediatek projects/fuse2/sys/mips/mediatek/std.rt2880 projects/fuse2/sys/modules/Makefile projects/fuse2/sys/net/if_vxlan.c projects/fuse2/sys/net/iflib.c projects/fuse2/sys/net/vnet.h projects/fuse2/sys/netinet/in.h projects/fuse2/sys/netinet/in_mcast.c projects/fuse2/sys/netinet/in_pcb.c projects/fuse2/sys/netinet/in_var.h projects/fuse2/sys/netinet/ip_carp.c projects/fuse2/sys/netinet/ip_fw.h projects/fuse2/sys/netinet/ip_mroute.c projects/fuse2/sys/netinet/ip_output.c projects/fuse2/sys/netinet/ip_var.h projects/fuse2/sys/netinet/tcp_subr.c projects/fuse2/sys/netinet6/in6.h projects/fuse2/sys/netinet6/in6_ifattach.c projects/fuse2/sys/netinet6/in6_mcast.c projects/fuse2/sys/netinet6/in6_pcb.c projects/fuse2/sys/netinet6/in6_var.h projects/fuse2/sys/netinet6/ip6_var.h projects/fuse2/sys/netipsec/key.c projects/fuse2/sys/netpfil/ipfw/ip_fw2.c projects/fuse2/sys/netpfil/ipfw/ip_fw_sockopt.c projects/fuse2/sys/netpfil/pf/if_pfsync.c projects/fuse2/sys/powerpc/conf/GENERIC projects/fuse2/sys/powerpc/conf/GENERIC64 projects/fuse2/sys/powerpc/conf/MPC85XX projects/fuse2/sys/powerpc/conf/MPC85XXSPE projects/fuse2/sys/powerpc/conf/QORIQ64 projects/fuse2/sys/powerpc/conf/dpaa/DPAA projects/fuse2/sys/powerpc/include/trap.h projects/fuse2/sys/powerpc/powerpc/machdep.c projects/fuse2/sys/powerpc/powerpc/trap.c projects/fuse2/sys/riscv/conf/GENERIC projects/fuse2/sys/sparc64/conf/GENERIC projects/fuse2/sys/sys/_types.h projects/fuse2/sys/sys/boot.h projects/fuse2/sys/sys/file.h projects/fuse2/sys/sys/filio.h projects/fuse2/sys/sys/mman.h projects/fuse2/sys/sys/param.h projects/fuse2/sys/sys/pcpu.h projects/fuse2/sys/sys/pmckern.h projects/fuse2/sys/sys/random.h projects/fuse2/sys/sys/rangelock.h projects/fuse2/sys/sys/types.h projects/fuse2/sys/sys/vnode.h projects/fuse2/sys/ufs/ffs/ffs_softdep.c projects/fuse2/sys/vm/device_pager.c projects/fuse2/sys/vm/vm_fault.c projects/fuse2/sys/vm/vm_map.c projects/fuse2/sys/vm/vm_mmap.c projects/fuse2/sys/vm/vm_object.c projects/fuse2/sys/vm/vm_page.c projects/fuse2/sys/vm/vm_page.h projects/fuse2/targets/pseudo/userland/Makefile.depend projects/fuse2/targets/pseudo/userland/lib/Makefile.depend projects/fuse2/tests/sys/netinet/socket_afinet.c projects/fuse2/tools/build/mk/OptionalObsoleteFiles.inc projects/fuse2/tools/tools/nanobsd/defaults.sh projects/fuse2/tools/tools/nanobsd/pcengines/ALIX_DSK projects/fuse2/tools/tools/tinybsd/conf/bridge/TINYBSD projects/fuse2/tools/tools/tinybsd/conf/default/TINYBSD projects/fuse2/tools/tools/tinybsd/conf/firewall/TINYBSD projects/fuse2/tools/tools/tinybsd/conf/minimal/TINYBSD projects/fuse2/tools/tools/tinybsd/conf/vpn/TINYBSD projects/fuse2/tools/tools/tinybsd/conf/wireless/TINYBSD projects/fuse2/tools/tools/tinybsd/conf/wrap/TINYBSD projects/fuse2/usr.bin/calendar/calendars/hr_HR.ISO8859-2/calendar.praznici projects/fuse2/usr.bin/top/display.c projects/fuse2/usr.bin/top/machine.c projects/fuse2/usr.bin/uname/uname.1 projects/fuse2/usr.bin/uname/uname.c projects/fuse2/usr.bin/vtfontcvt/vtfontcvt.c projects/fuse2/usr.sbin/Makefile projects/fuse2/usr.sbin/bhyve/Makefile projects/fuse2/usr.sbin/bhyve/bhyve.8 projects/fuse2/usr.sbin/bhyve/net_utils.c projects/fuse2/usr.sbin/bhyve/pci_virtio_scsi.c projects/fuse2/usr.sbin/bsnmpd/modules/snmp_lm75/snmp_lm75.c projects/fuse2/usr.sbin/bsnmpd/modules/snmp_pf/pf_snmp.c projects/fuse2/usr.sbin/bsnmpd/modules/snmp_pf/pf_tree.def projects/fuse2/usr.sbin/kbdcontrol/kbdcontrol.c projects/fuse2/usr.sbin/mountd/mountd.c projects/fuse2/usr.sbin/usbdump/usbdump.c Directory Properties: projects/fuse2/ (props changed) projects/fuse2/contrib/elftoolchain/ (props changed) projects/fuse2/contrib/gcc/ (props changed) projects/fuse2/contrib/ipfilter/ (props changed) projects/fuse2/contrib/llvm/ (props changed) projects/fuse2/contrib/llvm/tools/clang/ (props changed) projects/fuse2/gnu/usr.bin/cc/cc_tools/ (props changed) projects/fuse2/sys/cddl/contrib/opensolaris/ (props changed) projects/fuse2/sys/contrib/ipfilter/ (props changed) Modified: projects/fuse2/UPDATING ============================================================================== --- projects/fuse2/UPDATING Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/UPDATING Thu Jun 27 23:50:54 2019 (r349479) @@ -43,6 +43,12 @@ NOTE TO PEOPLE WHO THINK THAT FreeBSD 13.x IS SLOW: sysctls have been removed. If you felt the need to set any of them to a non-default value, please tell asomers@FreeBSD.org why. +20190620: + Entropy collection and the /dev/random device are no longer optional + components. The "device random" option has been removed. + Implementations of distilling algorithms can still be made loadable + with "options RANDOM_LOADABLE" (e.g., random_fortuna.ko). + 20190612: Clang, llvm, lld, lldb, compiler-rt, libc++, libunwind and openmp have been upgraded to 8.0.1. Please see the 20141231 entry below for Modified: projects/fuse2/contrib/elftoolchain/elfcopy/sections.c ============================================================================== --- projects/fuse2/contrib/elftoolchain/elfcopy/sections.c Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/contrib/elftoolchain/elfcopy/sections.c Thu Jun 27 23:50:54 2019 (r349479) @@ -1398,8 +1398,24 @@ update_shdr(struct elfcopy *ecp, int update_link) void init_shstrtab(struct elfcopy *ecp) { + Elf_Scn *shstrtab; + GElf_Shdr shdr; struct section *s; + size_t indx, sizehint; + if (elf_getshstrndx(ecp->ein, &indx) != 0) { + shstrtab = elf_getscn(ecp->ein, indx); + if (shstrtab == NULL) + errx(EXIT_FAILURE, "elf_getscn failed: %s", + elf_errmsg(-1)); + if (gelf_getshdr(shstrtab, &shdr) != &shdr) + errx(EXIT_FAILURE, "gelf_getshdr failed: %s", + elf_errmsg(-1)); + sizehint = shdr.sh_size; + } else { + sizehint = 0; + } + if ((ecp->shstrtab = calloc(1, sizeof(*ecp->shstrtab))) == NULL) err(EXIT_FAILURE, "calloc failed"); s = ecp->shstrtab; @@ -1410,7 +1426,7 @@ init_shstrtab(struct elfcopy *ecp) s->loadable = 0; s->type = SHT_STRTAB; s->vma = 0; - s->strtab = elftc_string_table_create(0); + s->strtab = elftc_string_table_create(sizehint); add_to_shstrtab(ecp, ""); add_to_shstrtab(ecp, ".symtab"); Modified: projects/fuse2/contrib/elftoolchain/libdwarf/libdwarf_attr.c ============================================================================== --- projects/fuse2/contrib/elftoolchain/libdwarf/libdwarf_attr.c Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/contrib/elftoolchain/libdwarf/libdwarf_attr.c Thu Jun 27 23:50:54 2019 (r349479) @@ -100,7 +100,6 @@ _dwarf_attr_init(Dwarf_Debug dbg, Dwarf_Section *ds, u uint64_t form, int indirect, Dwarf_Error *error) { struct _Dwarf_Attribute atref; - Dwarf_Section *str; int ret; ret = DW_DLE_NONE; @@ -183,9 +182,7 @@ _dwarf_attr_init(Dwarf_Debug dbg, Dwarf_Section *ds, u break; case DW_FORM_strp: atref.u[0].u64 = dbg->read(ds->ds_data, offsetp, dwarf_size); - str = _dwarf_find_section(dbg, ".debug_str"); - assert(str != NULL); - atref.u[1].s = (char *) str->ds_data + atref.u[0].u64; + atref.u[1].s = _dwarf_strtab_get_table(dbg) + atref.u[0].u64; break; case DW_FORM_ref_sig8: atref.u[0].u64 = 8; Modified: projects/fuse2/contrib/elftoolchain/libelftc/elftc_string_table.c ============================================================================== --- projects/fuse2/contrib/elftoolchain/libelftc/elftc_string_table.c Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/contrib/elftoolchain/libelftc/elftc_string_table.c Thu Jun 27 23:50:54 2019 (r349479) @@ -44,7 +44,7 @@ ELFTC_VCSID("$Id: elftc_string_table.c 2869 2013-01-06 #define ELFTC_STRING_TABLE_POOL_SIZE_INCREMENT (4*1024) struct _Elftc_String_Table_Entry { - int ste_idx; + ssize_t ste_idx; SLIST_ENTRY(_Elftc_String_Table_Entry) ste_next; }; @@ -64,9 +64,9 @@ struct _Elftc_String_Table_Entry { } while (0) struct _Elftc_String_Table { - unsigned int st_len; /* length and flags */ + size_t st_len; /* length and flags */ int st_nbuckets; - int st_string_pool_size; + size_t st_string_pool_size; char *st_string_pool; SLIST_HEAD(_Elftc_String_Table_Bucket, _Elftc_String_Table_Entry) st_buckets[]; @@ -86,7 +86,7 @@ elftc_string_table_find_hash_entry(Elftc_String_Table *rhashindex = hashindex; SLIST_FOREACH(ste, &st->st_buckets[hashindex], ste_next) { - s = st->st_string_pool + abs(ste->ste_idx); + s = st->st_string_pool + labs(ste->ste_idx); assert(s > st->st_string_pool && s < st->st_string_pool + st->st_string_pool_size); @@ -102,7 +102,7 @@ static int elftc_string_table_add_to_pool(Elftc_String_Table *st, const char *string) { char *newpool; - int len, newsize, stlen; + size_t len, newsize, stlen; len = strlen(string) + 1; /* length, including the trailing NUL */ stlen = ELFTC_STRING_TABLE_LENGTH(st); @@ -119,17 +119,17 @@ elftc_string_table_add_to_pool(Elftc_String_Table *st, st->st_string_pool_size = newsize; } - strcpy(st->st_string_pool + stlen, string); + memcpy(st->st_string_pool + stlen, string, len); ELFTC_STRING_TABLE_UPDATE_LENGTH(st, stlen + len); return (stlen); } Elftc_String_Table * -elftc_string_table_create(int sizehint) +elftc_string_table_create(size_t sizehint) { - int n, nbuckets, tablesize; struct _Elftc_String_Table *st; + int n, nbuckets, tablesize; if (sizehint < ELFTC_STRING_TABLE_DEFAULT_SIZE) sizehint = ELFTC_STRING_TABLE_DEFAULT_SIZE; @@ -173,13 +173,13 @@ elftc_string_table_destroy(Elftc_String_Table *st) } Elftc_String_Table * -elftc_string_table_from_section(Elf_Scn *scn, int sizehint) +elftc_string_table_from_section(Elf_Scn *scn, size_t sizehint) { - int len; Elf_Data *d; GElf_Shdr sh; const char *s, *end; Elftc_String_Table *st; + size_t len; /* Verify the type of the section passed in. */ if (gelf_getshdr(scn, &sh) == NULL || @@ -235,7 +235,8 @@ elftc_string_table_image(Elftc_String_Table *st, size_ char *r, *s, *end; struct _Elftc_String_Table_Entry *ste; struct _Elftc_String_Table_Bucket *head; - int copied, hashindex, offset, length, newsize; + size_t copied, offset, length, newsize; + int hashindex; /* * For the common case of a string table has not seen @@ -303,8 +304,9 @@ elftc_string_table_image(Elftc_String_Table *st, size_ size_t elftc_string_table_insert(Elftc_String_Table *st, const char *string) { - int hashindex, idx; struct _Elftc_String_Table_Entry *ste; + ssize_t idx; + int hashindex; hashindex = 0; @@ -326,7 +328,7 @@ elftc_string_table_insert(Elftc_String_Table *st, cons idx = ste->ste_idx; if (idx < 0) /* Undelete. */ - ste->ste_idx = idx = (- idx); + ste->ste_idx = idx = -idx; return (idx); } @@ -334,8 +336,9 @@ elftc_string_table_insert(Elftc_String_Table *st, cons size_t elftc_string_table_lookup(Elftc_String_Table *st, const char *string) { - int hashindex, idx; struct _Elftc_String_Table_Entry *ste; + ssize_t idx; + int hashindex; ste = elftc_string_table_find_hash_entry(st, string, &hashindex); @@ -350,17 +353,17 @@ elftc_string_table_lookup(Elftc_String_Table *st, cons int elftc_string_table_remove(Elftc_String_Table *st, const char *string) { - int idx; struct _Elftc_String_Table_Entry *ste; + ssize_t idx; ste = elftc_string_table_find_hash_entry(st, string, NULL); if (ste == NULL || (idx = ste->ste_idx) < 0) return (ELFTC_FAILURE); - assert(idx > 0 && idx < (int) ELFTC_STRING_TABLE_LENGTH(st)); + assert(idx > 0 && (size_t)idx < ELFTC_STRING_TABLE_LENGTH(st)); - ste->ste_idx = (- idx); + ste->ste_idx = -idx; ELFTC_STRING_TABLE_SET_COMPACTION_FLAG(st); Modified: projects/fuse2/contrib/elftoolchain/libelftc/elftc_string_table_create.3 ============================================================================== --- projects/fuse2/contrib/elftoolchain/libelftc/elftc_string_table_create.3 Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/contrib/elftoolchain/libelftc/elftc_string_table_create.3 Thu Jun 27 23:50:54 2019 (r349479) @@ -24,7 +24,7 @@ .\" .\" $Id: elftc_string_table_create.3 3645 2018-10-15 20:17:14Z jkoshy $ .\" -.Dd January 5, 2013 +.Dd June 19, 2019 .Dt ELFTC_STRING_TABLE_CREATE 3 .Os .Sh NAME @@ -40,11 +40,11 @@ .Sh SYNOPSIS .In libelftc.h .Ft "Elftc_String_Table *" -.Fn elftc_string_table_create "int sizehint" -.Ft int +.Fn elftc_string_table_create "size_t sizehint" +.Ft void .Fn elftc_string_table_destroy "Elftc_String_Table *table" .Ft "Elftc_String_Table *" -.Fn elftc_string_table_from_section "Elf_Scn *scn" "int sizehint" +.Fn elftc_string_table_from_section "Elf_Scn *scn" "size_t sizehint" .Ft "const char *" .Fo elftc_string_table_image .Fa "Elftc_String_Table *table" Modified: projects/fuse2/contrib/elftoolchain/libelftc/libelftc.h ============================================================================== --- projects/fuse2/contrib/elftoolchain/libelftc/libelftc.h Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/contrib/elftoolchain/libelftc/libelftc.h Thu Jun 27 23:50:54 2019 (r349479) @@ -77,10 +77,10 @@ int elftc_demangle(const char *_mangledname, char *_b size_t _bufsize, unsigned int _flags); const char *elftc_reloc_type_str(unsigned int mach, unsigned int type); int elftc_set_timestamps(const char *_filename, struct stat *_sb); -Elftc_String_Table *elftc_string_table_create(int _hint); +Elftc_String_Table *elftc_string_table_create(size_t _sizehint); void elftc_string_table_destroy(Elftc_String_Table *_table); Elftc_String_Table *elftc_string_table_from_section(Elf_Scn *_scn, - int _hint); + size_t _sizehint); const char *elftc_string_table_image(Elftc_String_Table *_table, size_t *_sz); size_t elftc_string_table_insert(Elftc_String_Table *_table, Modified: projects/fuse2/contrib/gcc/config/rs6000/tramp.asm ============================================================================== --- projects/fuse2/contrib/gcc/config/rs6000/tramp.asm Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/contrib/gcc/config/rs6000/tramp.asm Thu Jun 27 23:50:54 2019 (r349479) @@ -38,6 +38,7 @@ .file "tramp.asm" .section ".text" #include "ppc-asm.h" + #include "auto-host.h" #ifndef __powerpc64__ .type trampoline_initial,@object @@ -105,7 +106,7 @@ FUNC_START(__trampoline_setup) blr .Labort: -#if defined SHARED && defined HAVE_AS_REL16 +#if (defined(__PIC__) || defined(__pic__)) && defined HAVE_AS_REL16 bcl 20,31,1f 1: mflr r30 addis r30,r30,_GLOBAL_OFFSET_TABLE_-1b@ha Modified: projects/fuse2/contrib/ipfilter/man/ipmon.8 ============================================================================== --- projects/fuse2/contrib/ipfilter/man/ipmon.8 Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/contrib/ipfilter/man/ipmon.8 Thu Jun 27 23:50:54 2019 (r349479) @@ -27,7 +27,7 @@ ipmon \- monitors /dev/ipl for logged packets .LP \fBipmon\fP opens \fB/dev/ipl\fP for reading and awaits data to be saved from the packet filter. The binary data read from the device is reprinted in -human readable for, however, IP#'s are not mapped back to hostnames, nor are +human readable form, however, IP#'s are not mapped back to hostnames, nor are ports mapped back to service names. The output goes to standard output by default or a filename, if given on the command line. Should the \fB\-s\fP option be used, output is instead sent to \fBsyslogd(8)\fP. Messages sent Modified: projects/fuse2/contrib/ipfilter/tools/ipmon.c ============================================================================== --- projects/fuse2/contrib/ipfilter/tools/ipmon.c Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/contrib/ipfilter/tools/ipmon.c Thu Jun 27 23:50:54 2019 (r349479) @@ -1438,7 +1438,10 @@ printipflog: static void usage(prog) char *prog; { - fprintf(stderr, "%s: [-NFhstvxX] [-f ]\n", prog); + fprintf(stderr, "Usage: %s [ -abDFhnpstvxX ] [ -B ] [ -C ]\n" + "\t[ -f ] [ -L ] [ -N ]\n" + "\t[ -o [NSI] ] [ -O [NSI] ] [ -P ] [ -S ]\n" + "\t[ ]\n", prog); exit(1); } Modified: projects/fuse2/contrib/llvm/lib/Target/PowerPC/PPCSubtarget.cpp ============================================================================== --- projects/fuse2/contrib/llvm/lib/Target/PowerPC/PPCSubtarget.cpp Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/contrib/llvm/lib/Target/PowerPC/PPCSubtarget.cpp Thu Jun 27 23:50:54 2019 (r349479) @@ -138,7 +138,8 @@ void PPCSubtarget::initSubtargetFeatures(StringRef CPU if (isDarwin()) HasLazyResolverStubs = true; - if (TargetTriple.isOSNetBSD() || TargetTriple.isOSOpenBSD()) + if ((TargetTriple.isOSFreeBSD() && TargetTriple.getOSMajorVersion() >= 13) + || TargetTriple.isOSNetBSD() || TargetTriple.isOSOpenBSD()) SecurePlt = true; if (HasSPE && IsPPC64) Modified: projects/fuse2/contrib/llvm/tools/clang/lib/Driver/ToolChains/Arch/PPC.cpp ============================================================================== --- projects/fuse2/contrib/llvm/tools/clang/lib/Driver/ToolChains/Arch/PPC.cpp Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/contrib/llvm/tools/clang/lib/Driver/ToolChains/Arch/PPC.cpp Thu Jun 27 23:50:54 2019 (r349479) @@ -116,7 +116,8 @@ ppc::ReadGOTPtrMode ppc::getPPCReadGOTPtrMode(const Dr const ArgList &Args) { if (Args.getLastArg(options::OPT_msecure_plt)) return ppc::ReadGOTPtrMode::SecurePlt; - if (Triple.isOSOpenBSD()) + if ((Triple.isOSFreeBSD() && Triple.getOSMajorVersion() >= 13) || + Triple.isOSOpenBSD()) return ppc::ReadGOTPtrMode::SecurePlt; else return ppc::ReadGOTPtrMode::Bss; Modified: projects/fuse2/etc/mtree/BSD.include.dist ============================================================================== --- projects/fuse2/etc/mtree/BSD.include.dist Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/etc/mtree/BSD.include.dist Thu Jun 27 23:50:54 2019 (r349479) @@ -138,8 +138,6 @@ mpilib .. .. - nand - .. nvme .. ofw @@ -183,8 +181,6 @@ fdescfs .. msdosfs - .. - nandfs .. nfs .. Modified: projects/fuse2/gnu/usr.bin/cc/cc_tools/Makefile.hdrs ============================================================================== --- projects/fuse2/gnu/usr.bin/cc/cc_tools/Makefile.hdrs Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/gnu/usr.bin/cc/cc_tools/Makefile.hdrs Thu Jun 27 23:50:54 2019 (r349479) @@ -21,6 +21,9 @@ TARGET_INC+= ${GCC_CPU}/${GCC_CPU}.h TARGET_INC+= ${GCC_CPU}/unix.h TARGET_INC+= ${GCC_CPU}/att.h .endif +.if ${TARGET_CPUARCH} == "powerpc" +TARGET_INC+= ${GCC_CPU}/secureplt.h +.endif TARGET_INC+= dbxelf.h TARGET_INC+= elfos-undef.h TARGET_INC+= elfos.h Modified: projects/fuse2/include/Makefile ============================================================================== --- projects/fuse2/include/Makefile Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/include/Makefile Thu Jun 27 23:50:54 2019 (r349479) @@ -48,7 +48,7 @@ LSUBDIRS= cam/ata cam/mmc cam/nvme cam/scsi \ dev/ic dev/iicbus dev/io dev/mfi dev/mmc dev/nvme \ dev/ofw dev/pbio dev/pci ${_dev_powermac_nvram} dev/ppbus dev/pwm \ dev/smbus dev/speaker dev/tcp_log dev/veriexec dev/vkbd dev/wi \ - fs/devfs fs/fdescfs fs/msdosfs fs/nandfs fs/nfs fs/nullfs \ + fs/devfs fs/fdescfs fs/msdosfs fs/nfs fs/nullfs \ fs/procfs fs/smbfs fs/udf fs/unionfs \ geom/cache geom/concat geom/eli geom/gate geom/journal geom/label \ geom/mirror geom/mountver geom/multipath geom/nop \ @@ -158,7 +158,7 @@ copies: .PHONY .META done; \ fi .endfor -.for i in ${LDIRS} ${LSUBDIRS:Ndev/agp:Ndev/acpica:Ndev/bktr:Ndev/evdev:Ndev/hyperv:Ndev/nand:Ndev/pci:Ndev/veriexec} ${LSUBSUBDIRS} +.for i in ${LDIRS} ${LSUBDIRS:Ndev/agp:Ndev/acpica:Ndev/bktr:Ndev/evdev:Ndev/hyperv:Ndev/pci:Ndev/veriexec} ${LSUBSUBDIRS} cd ${SRCTOP}/sys; \ ${INSTALL} -C ${TAG_ARGS} -o ${BINOWN} -g ${BINGRP} -m 444 $i/*.h \ ${SDESTDIR}${INCLUDEDIR}/$i @@ -174,13 +174,6 @@ copies: .PHONY .META cd ${SRCTOP}/sys/dev/bktr; \ ${INSTALL} -C ${TAG_ARGS} -o ${BINOWN} -g ${BINGRP} -m 444 ioctl_*.h \ ${SDESTDIR}${INCLUDEDIR}/dev/bktr -.if ${MK_NAND} != "no" - cd ${SRCTOP}/sys/dev/nand; \ - ${INSTALL} -C ${TAG_ARGS} -o ${BINOWN} -g ${BINGRP} -m 444 nandsim.h \ - ${SDESTDIR}${INCLUDEDIR}/dev/nand; \ - ${INSTALL} -C ${TAG_ARGS} -o ${BINOWN} -g ${BINGRP} -m 444 nand_dev.h \ - ${SDESTDIR}${INCLUDEDIR}/dev/nand -.endif cd ${SRCTOP}/sys/dev/evdev; \ ${INSTALL} -C -o ${BINOWN} -g ${BINGRP} -m 444 input.h \ ${SDESTDIR}${INCLUDEDIR}/dev/evdev; \ @@ -268,7 +261,7 @@ symlinks: .PHONY .META ${INSTALL_SYMLINK} ${TAG_ARGS} ../../../sys/$i/$$h ${SDESTDIR}${INCLUDEDIR}/$i; \ done .endfor -.for i in ${LSUBDIRS:Ndev/agp:Ndev/acpica:Ndev/bktr:Ndev/evdev:Ndev/hyperv:Ndev/nand:Ndev/pci:Ndev/veriexec} +.for i in ${LSUBDIRS:Ndev/agp:Ndev/acpica:Ndev/bktr:Ndev/evdev:Ndev/hyperv:Ndev/pci:Ndev/veriexec} cd ${SRCTOP}/sys/$i; \ for h in *.h; do \ ${INSTALL_SYMLINK} ${TAG_ARGS} ../../../../sys/$i/$$h ${SDESTDIR}${INCLUDEDIR}/$i; \ @@ -289,13 +282,6 @@ symlinks: .PHONY .META ${INSTALL_SYMLINK} ${TAG_ARGS} ../../../../sys/dev/bktr/$$h \ ${SDESTDIR}${INCLUDEDIR}/dev/bktr; \ done -.if ${MK_NAND} != "no" - cd ${SRCTOP}/sys/dev/nand; \ - for h in nandsim.h nand_dev.h; do \ - ${INSTALL_SYMLINK} ${TAG_ARGS} ../../../../sys/dev/nand/$$h \ - ${SDESTDIR}${INCLUDEDIR}/dev/nand; \ - done -.endif cd ${SRCTOP}/sys/dev/evdev; \ for h in input.h input-event-codes.h uinput.h; do \ ln -fs ../../../../sys/dev/evdev/$$h \ Modified: projects/fuse2/lib/Makefile ============================================================================== --- projects/fuse2/lib/Makefile Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/Makefile Thu Jun 27 23:50:54 2019 (r349479) @@ -174,7 +174,6 @@ SUBDIR.${MK_GOOGLETEST}+= googletest SUBDIR.${MK_LIBTHR}+= libthr SUBDIR.${MK_LLVM_LIBUNWIND}+= libgcc_eh SUBDIR.${MK_LLVM_LIBUNWIND}+= libgcc_s -SUBDIR.${MK_NAND}+= libnandfs SUBDIR.${MK_NETGRAPH}+= libnetgraph SUBDIR.${MK_NIS}+= libypclnt Modified: projects/fuse2/lib/libbe/be_access.c ============================================================================== --- projects/fuse2/lib/libbe/be_access.c Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/libbe/be_access.c Thu Jun 27 23:50:54 2019 (r349479) @@ -89,25 +89,31 @@ be_mount_iter(zfs_handle_t *zfs_hdl, void *data) return (0); } - if (zfs_prop_get_int(zfs_hdl, ZFS_PROP_CANMOUNT) == ZFS_CANMOUNT_OFF) - return (0); + /* + * canmount and mountpoint are both ignored for the BE dataset, because + * the rest of the system (kernel and loader) will effectively do the + * same. + */ + if (info->depth == 0) { + snprintf(tmp, BE_MAXPATHLEN, "%s", info->mountpoint); + } else { + if (zfs_prop_get_int(zfs_hdl, ZFS_PROP_CANMOUNT) == + ZFS_CANMOUNT_OFF) + return (0); - if (zfs_prop_get(zfs_hdl, ZFS_PROP_MOUNTPOINT, zfs_mnt, BE_MAXPATHLEN, - NULL, NULL, 0, 1)) - return (1); + if (zfs_prop_get(zfs_hdl, ZFS_PROP_MOUNTPOINT, zfs_mnt, + BE_MAXPATHLEN, NULL, NULL, 0, 1)) + return (1); - if (strcmp("none", zfs_mnt) == 0) { /* - * mountpoint=none; we'll mount it at info->mountpoint assuming - * we're at the root. If we're not at the root, we're likely - * at some intermediate dataset (e.g. zroot/var) that will have - * children that may need to be mounted. + * We've encountered mountpoint=none at some intermediate + * dataset (e.g. zroot/var) that will have children that may + * need to be mounted. Skip mounting it, but iterate through + * the children. */ - if (info->depth > 0) + if (strcmp("none", zfs_mnt) == 0) goto skipmount; - snprintf(tmp, BE_MAXPATHLEN, "%s", info->mountpoint); - } else { mountpoint = be_mountpoint_augmented(info->lbh, zfs_mnt); snprintf(tmp, BE_MAXPATHLEN, "%s%s", info->mountpoint, mountpoint); Modified: projects/fuse2/lib/libc/gen/Symbol.map ============================================================================== --- projects/fuse2/lib/libc/gen/Symbol.map Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/libc/gen/Symbol.map Thu Jun 27 23:50:54 2019 (r349479) @@ -338,6 +338,7 @@ FBSD_1.2 { getutxid; getutxline; getutxuser; + pthread_getthreadid_np; pututxline; sem_close; sem_destroy; Modified: projects/fuse2/lib/libc/gen/_pthread_stubs.c ============================================================================== --- projects/fuse2/lib/libc/gen/_pthread_stubs.c Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/libc/gen/_pthread_stubs.c Thu Jun 27 23:50:54 2019 (r349479) @@ -130,6 +130,7 @@ pthread_func_entry_t __thr_jtable[PJT_MAX] = { {PJT_DUAL_ENTRY(stub_zero)}, /* PJT_MUTEX_CONSISTENT */ {PJT_DUAL_ENTRY(stub_zero)}, /* PJT_MUTEXATTR_GETROBUST */ {PJT_DUAL_ENTRY(stub_zero)}, /* PJT_MUTEXATTR_SETROBUST */ + {PJT_DUAL_ENTRY(stub_zero)}, /* PJT_GETTHREADID_NP */ }; /* @@ -248,6 +249,7 @@ STUB_FUNC1(pthread_rwlock_trywrlock, PJT_RWLOCK_TRYWRL STUB_FUNC1(pthread_rwlock_unlock, PJT_RWLOCK_UNLOCK, int, void *) STUB_FUNC1(pthread_rwlock_wrlock, PJT_RWLOCK_WRLOCK, int, void *) STUB_FUNC(pthread_self, PJT_SELF, pthread_t) +STUB_FUNC(pthread_getthreadid_np, PJT_GETTHREADID_NP, int) STUB_FUNC2(pthread_setspecific, PJT_SETSPECIFIC, int, pthread_key_t, void *) STUB_FUNC3(pthread_sigmask, PJT_SIGMASK, int, int, void *, void *) STUB_FUNC3(pthread_atfork, PJT_ATFORK, int, void *, void *, void*) Modified: projects/fuse2/lib/libc/gen/libc_dlopen.c ============================================================================== --- projects/fuse2/lib/libc/gen/libc_dlopen.c Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/libc/gen/libc_dlopen.c Thu Jun 27 23:50:54 2019 (r349479) @@ -48,8 +48,8 @@ libc_dlopen(const char *path, int mode) if (__libc_restricted_mode) { _rtld_error("Service unavailable -- libc in restricted mode"); return (NULL); - } else - return (dlopen(path, mode)); + } + return (dlopen(path, mode)); } void @@ -57,6 +57,5 @@ __FreeBSD_libc_enter_restricted_mode(void) { __libc_restricted_mode = 1; - return; } Modified: projects/fuse2/lib/libc/gen/opendir.c ============================================================================== --- projects/fuse2/lib/libc/gen/opendir.c Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/libc/gen/opendir.c Thu Jun 27 23:50:54 2019 (r349479) @@ -99,8 +99,8 @@ static int opendir_compar(const void *p1, const void *p2) { - return (strcmp((*(const struct dirent **)p1)->d_name, - (*(const struct dirent **)p2)->d_name)); + return (strcmp((*(const struct dirent * const *)p1)->d_name, + (*(const struct dirent * const *)p2)->d_name)); } /* Modified: projects/fuse2/lib/libc/gen/telldir.c ============================================================================== --- projects/fuse2/lib/libc/gen/telldir.c Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/libc/gen/telldir.c Thu Jun 27 23:50:54 2019 (r349479) @@ -63,8 +63,8 @@ telldir(DIR *dirp) * 2) Otherwise, see if it's already been recorded in the linked list * 3) Otherwise, malloc a new one */ - if (dirp->dd_seek < (1ul << DD_SEEK_BITS) && - dirp->dd_loc < (1ul << DD_LOC_BITS)) { + if (dirp->dd_seek < (off_t)(1l << DD_SEEK_BITS) && + dirp->dd_loc < (1l << DD_LOC_BITS)) { ddloc.s.is_packed = 1; ddloc.s.loc = dirp->dd_loc; ddloc.s.seek = dirp->dd_seek; Modified: projects/fuse2/lib/libc/include/libc_private.h ============================================================================== --- projects/fuse2/lib/libc/include/libc_private.h Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/libc/include/libc_private.h Thu Jun 27 23:50:54 2019 (r349479) @@ -176,6 +176,7 @@ typedef enum { PJT_MUTEX_CONSISTENT, PJT_MUTEXATTR_GETROBUST, PJT_MUTEXATTR_SETROBUST, + PJT_GETTHREADID_NP, PJT_MAX } pjt_index_t; Modified: projects/fuse2/lib/libc/powerpc/SYS.h ============================================================================== --- projects/fuse2/lib/libc/powerpc/SYS.h Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/libc/powerpc/SYS.h Thu Jun 27 23:50:54 2019 (r349479) @@ -44,7 +44,7 @@ #define SYSCALL(name) \ .text; \ .align 2; \ -2: b PIC_PLT(CNAME(HIDENAME(cerror))); \ +2: b CNAME(HIDENAME(cerror)); \ ENTRY(__sys_##name); \ WEAK_REFERENCE(__sys_##name, name); \ WEAK_REFERENCE(__sys_##name, _##name); \ @@ -58,15 +58,14 @@ ENTRY(__sys_##name); \ WEAK_REFERENCE(__sys_##name, _##name); \ _SYSCALL(name); \ bnslr; \ - b PIC_PLT(CNAME(HIDENAME(cerror))) + b CNAME(HIDENAME(cerror)) #define RSYSCALL(name) \ .text; \ .align 2; \ -2: b PIC_PLT(CNAME(HIDENAME(cerror))); \ ENTRY(__sys_##name); \ WEAK_REFERENCE(__sys_##name, name); \ WEAK_REFERENCE(__sys_##name, _##name); \ _SYSCALL(name); \ bnslr; \ - b PIC_PLT(CNAME(HIDENAME(cerror))) + b CNAME(HIDENAME(cerror)) Modified: projects/fuse2/lib/libc/powerpc/gen/_ctx_start.S ============================================================================== --- projects/fuse2/lib/libc/powerpc/gen/_ctx_start.S Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/libc/powerpc/gen/_ctx_start.S Thu Jun 27 23:50:54 2019 (r349479) @@ -35,11 +35,18 @@ mtlr %r14 blrl /* branch to start function */ mr %r3,%r15 /* pass pointer to ucontext as argument */ - bl PIC_PLT(CNAME(_ctx_done)) /* branch to ctxt completion func */ + bl CNAME(_ctx_done) /* branch to ctxt completion func */ + /* * we should never return from the * above branch. */ + /* Don't bother saving off %r30, we're already in a bad state. */ + bcl 20,31,1f +1: mflr %r30 + mr %r3,%r30 # save for _DYNAMIC + addis %r30,%r30,_GLOBAL_OFFSET_TABLE_-1b@ha + addi %r30,%r30,_GLOBAL_OFFSET_TABLE_-1b@l bl PIC_PLT(CNAME(abort)) /* abort */ END(_cts_start) Modified: projects/fuse2/lib/libc/powerpc/sys/cerror.S ============================================================================== --- projects/fuse2/lib/libc/powerpc/sys/cerror.S Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/libc/powerpc/sys/cerror.S Thu Jun 27 23:50:54 2019 (r349479) @@ -40,16 +40,27 @@ __FBSDID("$FreeBSD$"); */ HIDENAME(cerror): mflr %r0 - stwu %r1,-16(%r1) /* allocate new stack frame */ - stw %r0,20(%r1) /* and save lr, r31 */ - stw %r31,8(%r1) + stwu %r1,-20(%r1) /* allocate new stack frame */ + stw %r0,24(%r1) /* and save lr, r31 */ + stw %r31,12(%r1) +#ifdef __PIC__ + stw %r30,8(%r1) + bcl 20,31,1f +1: + mflr %r30 + addis %r30,%r30,_GLOBAL_OFFSET_TABLE_-1b@ha + addi %r30,%r30,_GLOBAL_OFFSET_TABLE_-1b@l +#endif mr %r31,%r3 /* stash errval in callee-saved register */ bl PIC_PLT(CNAME(__error)) stw %r31,0(%r3) /* store errval into &errno */ - lwz %r0,20(%r1) - lwz %r31,8(%r1) + lwz %r0,24(%r1) + lwz %r31,12(%r1) +#ifdef __PIC__ + lwz %r30,8(%r1) +#endif mtlr %r0 - la %r1,16(%r1) + la %r1,20(%r1) li %r3,-1 li %r4,-1 blr /* return to callers caller */ Modified: projects/fuse2/lib/libc/stdlib/realpath.c ============================================================================== --- projects/fuse2/lib/libc/stdlib/realpath.c Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/libc/stdlib/realpath.c Thu Jun 27 23:50:54 2019 (r349479) @@ -91,7 +91,7 @@ realpath1(const char *path, char *resolved) */ p = strchr(left, '/'); - next_token_len = p != NULL ? p - left : left_len; + next_token_len = p != NULL ? (size_t)(p - left) : left_len; memcpy(next_token, left, next_token_len); next_token[next_token_len] = '\0'; @@ -146,7 +146,7 @@ realpath1(const char *path, char *resolved) return (NULL); } slen = readlink(resolved, symlink, sizeof(symlink)); - if (slen <= 0 || slen >= sizeof(symlink)) { + if (slen <= 0 || slen >= (ssize_t)sizeof(symlink)) { if (slen < 0) ; /* keep errno from readlink(2) call */ else if (slen == 0) @@ -173,7 +173,7 @@ realpath1(const char *path, char *resolved) */ if (p != NULL) { if (symlink[slen - 1] != '/') { - if (slen + 1 >= sizeof(symlink)) { + if (slen + 1 >= (ssize_t)sizeof(symlink)) { errno = ENAMETOOLONG; return (NULL); } Modified: projects/fuse2/lib/libc/sys/mmap.2 ============================================================================== --- projects/fuse2/lib/libc/sys/mmap.2 Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/libc/sys/mmap.2 Thu Jun 27 23:50:54 2019 (r349479) @@ -28,7 +28,7 @@ .\" @(#)mmap.2 8.4 (Berkeley) 5/11/95 .\" $FreeBSD$ .\" -.Dd June 22, 2017 +.Dd June 20, 2019 .Dt MMAP 2 .Os .Sh NAME @@ -113,6 +113,22 @@ Pages may be written. Pages may be executed. .El .Pp +In addition to these protection flags, +.Fx +provides the ability to set the maximum protection of a region allocated by +.Nm +and later altered by +.Xr mprotect 2 . +This is accomplished by +.Em or Ns 'ing +one or more +.Dv PROT_ +values wrapped in the +.Dv PROT_MAX() +macro into the +.Fa prot +argument. +.Pp The .Fa flags argument specifies the type of the mapped object, mapping options and @@ -416,6 +432,11 @@ An invalid value was passed in the .Fa prot argument. .It Bq Er EINVAL +The +.Fa prot +argument contains permissions which are not a subset of the specified +maximum permissions. +.It Bq Er EINVAL An undefined option was set in the .Fa flags argument. @@ -521,3 +542,16 @@ was specified and insufficient memory was available. .Xr munmap 2 , .Xr getpagesize 3 , .Xr getpagesizes 3 +.Sh HISTORY +The +.Nm +system call was first documented in +.Bx 4.2 +and implemented in +.Bx 4.4 . +.\" XXX: lots of missing history of FreeBSD additions. +.Pp +The +.Dv PROT_MAX +functionality was introduced in +.Fx 13 . Modified: projects/fuse2/lib/libc/sys/mprotect.2 ============================================================================== --- projects/fuse2/lib/libc/sys/mprotect.2 Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/libc/sys/mprotect.2 Thu Jun 27 23:50:54 2019 (r349479) @@ -28,7 +28,7 @@ .\" @(#)mprotect.2 8.1 (Berkeley) 6/9/93 .\" $FreeBSD$ .\" -.Dd August 3, 2016 +.Dd June 20, 2019 .Dt MPROTECT 2 .Os .Sh NAME @@ -65,6 +65,22 @@ The pages can be written. .It Dv PROT_EXEC The pages can be executed. .El +.Pp +In addition to these protection flags, +.Fx +provides the ability to set the maximum protection of a region +(which prevents +.Nm +from upgrading the permissions). +This is accomplished by +.Em or Ns 'ing +one or more +.Dv PROT_ +values wrapped in the +.Dv PROT_MAX() +macro into the +.Fa prot +argument. .Sh RETURN VALUES .Rv -std mprotect .Sh ERRORS @@ -78,6 +94,15 @@ The virtual address range specified by the and .Fa len arguments is not valid. +.It Bq Er EINVAL +The +.Fa prot +argument contains unhandled bits. +.It Bq Er EINVAL +The +.Fa prot +argument contains permissions which are not a subset of the specified +maximum permissions. .It Bq Er EACCES The calling process was not allowed to change the protection to the value specified by @@ -93,5 +118,12 @@ argument. .Sh HISTORY The .Fn mprotect -system call first appeared in +system call was first documented in +.Bx 4.2 +and first appeared in .Bx 4.4 . +.Pp +The +.Dv PROT_MAX +functionality was introduced in +.Fx 13 . Modified: projects/fuse2/lib/libjail/jail.c ============================================================================== --- projects/fuse2/lib/libjail/jail.c Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/libjail/jail.c Thu Jun 27 23:50:54 2019 (r349479) @@ -30,7 +30,6 @@ __FBSDID("$FreeBSD$"); #include -#include #include #include #include Modified: projects/fuse2/lib/libsecureboot/h/libsecureboot.h ============================================================================== --- projects/fuse2/lib/libsecureboot/h/libsecureboot.h Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/libsecureboot/h/libsecureboot.h Thu Jun 27 23:50:54 2019 (r349479) @@ -42,6 +42,7 @@ #include +unsigned char * read_fd(int, size_t); #ifndef NEED_BRSSL_H unsigned char * read_file(const char *, size_t *); #endif @@ -51,8 +52,12 @@ extern int DebugVe; #define DEBUG_PRINTF(n, x) if (DebugVe >= n) printf x int ve_trust_init(void); +size_t ve_trust_anchors_add_buf(unsigned char *, size_t); +size_t ve_trust_anchors_revoke(unsigned char *, size_t); int ve_trust_add(const char *); void ve_debug_set(int); +void ve_anchor_verbose_set(int); +int ve_anchor_verbose_get(void); void ve_utc_set(time_t utc); char *ve_error_get(void); int ve_error_set(const char *, ...) __printflike(1,2); Modified: projects/fuse2/lib/libsecureboot/libsecureboot-priv.h ============================================================================== --- projects/fuse2/lib/libsecureboot/libsecureboot-priv.h Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/libsecureboot/libsecureboot-priv.h Thu Jun 27 23:50:54 2019 (r349479) @@ -56,6 +56,8 @@ int is_verified(struct stat *stp); void add_verify_status(struct stat *stp, int status); int openpgp_trust_init(void); +int openpgp_trust_add_buf(unsigned char *, size_t); +int openpgp_trust_revoke(const char *); int openpgp_self_tests(void); int efi_secure_boot_enabled(void); Modified: projects/fuse2/lib/libsecureboot/local.trust.mk ============================================================================== --- projects/fuse2/lib/libsecureboot/local.trust.mk Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/libsecureboot/local.trust.mk Thu Jun 27 23:50:54 2019 (r349479) @@ -33,6 +33,10 @@ VE_SIGNATURE_EXT_LIST+= \ sig .endif +# add OpenPGP support - possibly dormant +VE_SIGNATURE_LIST+= OPENPGP +VE_SIGNATURE_EXT_LIST+= asc + SIGNER ?= ${SB_TOOLS_PATH:U/volume/buildtools/bin}/sign.py .if exists(${SIGNER}) @@ -42,7 +46,12 @@ SIGN_ECDSA= ${PYTHON} ${SIGNER} -u ${SIGN_HOST}:${ECDS RSA2_PORT:= ${163%y:L:gmtime} SIGN_RSA2= ${PYTHON} ${SIGNER} -u ${SIGN_HOST}:${RSA2_PORT} -h sha256 +# deal with quirk of our .esig format +XCFLAGS.vets+= -DVE_ECDSA_HASH_AGAIN + .if !empty(OPENPGP_SIGN_URL) +XCFLAGS.opgp_key+= -DHAVE_TA_ASC_H + VE_SIGNATURE_LIST+= OPENPGP VE_SIGNATURE_EXT_LIST+= asc Modified: projects/fuse2/lib/libsecureboot/openpgp/opgp_key.c ============================================================================== --- projects/fuse2/lib/libsecureboot/openpgp/opgp_key.c Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/libsecureboot/openpgp/opgp_key.c Thu Jun 27 23:50:54 2019 (r349479) @@ -209,13 +209,54 @@ openpgp_trust_add(OpenPGP_key *key) LIST_INIT(&trust_list); } - if (key) { - DEBUG_PRINTF(2, ("openpgp_trust_add(%s)\n", key->id)); + if (key && openpgp_trust_get(key->id) == NULL) { + if (ve_anchor_verbose_get()) + printf("openpgp_trust_add(%s)\n", key->id); LIST_INSERT_HEAD(&trust_list, key, entries); } } /** + * @brief add trust anchor from buf + */ +int +openpgp_trust_add_buf(unsigned char *buf, size_t nbytes) +{ + OpenPGP_key *key; + + if ((key = load_key_buf(buf, nbytes))) { + openpgp_trust_add(key); + } + return (key != NULL); +} + + +/** + * @brief if keyID is in our list clobber it + * + * @return true if keyID removed + */ +int +openpgp_trust_revoke(const char *keyID) +{ + OpenPGP_key *key, *tkey; + + openpgp_trust_add(NULL); /* initialize if needed */ + + LIST_FOREACH(key, &trust_list, entries) { + if (strcmp(key->id, keyID) == 0) { + tkey = key; + LIST_REMOVE(tkey, entries); + printf("openpgp_trust_revoke(%s)\n", key->id); + memset(key, 0, sizeof(OpenPGP_key)); + free(key); + return (1); + } + } + return (0); +} + +/** * @brief if keyID is in our list return the key * * @return key or NULL @@ -251,7 +292,9 @@ load_key_file(const char *kfile) return (key); } +#ifdef HAVE_TA_ASC_H #include +#endif #ifndef _STANDALONE /* we can lookup keyID in filesystem */ @@ -330,8 +373,8 @@ openpgp_trust_init(void) } } } - } #endif + } return (once); } Modified: projects/fuse2/lib/libsecureboot/readfile.c ============================================================================== --- projects/fuse2/lib/libsecureboot/readfile.c Thu Jun 27 23:39:06 2019 (r349478) +++ projects/fuse2/lib/libsecureboot/readfile.c Thu Jun 27 23:50:54 2019 (r349479) @@ -28,21 +28,13 @@ __FBSDID("$FreeBSD$"); #include unsigned char * -read_file(const char *path, size_t *len) +read_fd(int fd, size_t len) { - int fd, m, n, x; - struct stat st; + int m, n, x; unsigned char *buf; - if (len) - *len = 0; - if ((fd = open(path, O_RDONLY)) < 0) - return (NULL); - fstat(fd, &st); - if (len) - *len = st.st_size; - buf = malloc(st.st_size + 1); - for (x = 0, m = st.st_size; m > 0; ) { + buf = malloc(len + 1); + for (x = 0, m = len; m > 0; ) { n = read(fd, &buf[x], m); if (n < 0) break; @@ -51,11 +43,30 @@ read_file(const char *path, size_t *len) x += n; } } - close(fd); if (m == 0) { - buf[st.st_size] = '\0'; + buf[len] = '\0'; return (buf); } free(buf); return (NULL); } + +unsigned char * +read_file(const char *path, size_t *len) +{ + struct stat st; + unsigned char *ucp; + int fd; + + if (len) + *len = 0; + if ((fd = open(path, O_RDONLY)) < 0) + return (NULL); + fstat(fd, &st); + ucp = read_fd(fd, st.st_size); + close(fd); + if (len != NULL && ucp != NULL) *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-projects@freebsd.org Fri Jun 28 04:18:15 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06CFA15D7646 for ; Fri, 28 Jun 2019 04:18:15 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A29E888275; Fri, 28 Jun 2019 04:18:14 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 78C5E21836; Fri, 28 Jun 2019 04:18:14 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5S4IEKG089282; Fri, 28 Jun 2019 04:18:14 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5S4IAqZ089260; Fri, 28 Jun 2019 04:18:10 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906280418.x5S4IAqZ089260@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 28 Jun 2019 04:18:10 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349502 - in projects/fuse2: sbin/mount_fusefs share/man/man5 share/man/man9 sys/fs/fuse X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: sbin/mount_fusefs share/man/man5 share/man/man9 sys/fs/fuse X-SVN-Commit-Revision: 349502 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: A29E888275 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.97)[-0.973,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jun 2019 04:18:15 -0000 Author: asomers Date: Fri Jun 28 04:18:10 2019 New Revision: 349502 URL: https://svnweb.freebsd.org/changeset/base/349502 Log: [skip ci] update copyright headers in fusefs files Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sbin/mount_fusefs/mount_fusefs.8 projects/fuse2/sbin/mount_fusefs/mount_fusefs.c projects/fuse2/share/man/man5/fusefs.5 projects/fuse2/share/man/man9/VOP_FSYNC.9 projects/fuse2/sys/fs/fuse/fuse.h projects/fuse2/sys/fs/fuse/fuse_device.c projects/fuse2/sys/fs/fuse/fuse_file.c projects/fuse2/sys/fs/fuse/fuse_file.h projects/fuse2/sys/fs/fuse/fuse_internal.c projects/fuse2/sys/fs/fuse/fuse_internal.h projects/fuse2/sys/fs/fuse/fuse_io.c projects/fuse2/sys/fs/fuse/fuse_io.h projects/fuse2/sys/fs/fuse/fuse_ipc.c projects/fuse2/sys/fs/fuse/fuse_ipc.h projects/fuse2/sys/fs/fuse/fuse_main.c projects/fuse2/sys/fs/fuse/fuse_node.c projects/fuse2/sys/fs/fuse/fuse_node.h projects/fuse2/sys/fs/fuse/fuse_vfsops.c projects/fuse2/sys/fs/fuse/fuse_vnops.c Modified: projects/fuse2/sbin/mount_fusefs/mount_fusefs.8 ============================================================================== --- projects/fuse2/sbin/mount_fusefs/mount_fusefs.8 Fri Jun 28 04:02:56 2019 (r349501) +++ projects/fuse2/sbin/mount_fusefs/mount_fusefs.8 Fri Jun 28 04:18:10 2019 (r349502) @@ -3,6 +3,11 @@ .\" Copyright (c) 2005, 2006 Csaba Henk .\" All rights reserved. .\" +.\" Copyright (c) 2019 The FreeBSD Foundation +.\" +.\" Portions of this documentation were written by BFF Storage Systems under +.\" sponsorship from the FreeBSD Foundation. +.\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: Modified: projects/fuse2/sbin/mount_fusefs/mount_fusefs.c ============================================================================== --- projects/fuse2/sbin/mount_fusefs/mount_fusefs.c Fri Jun 28 04:02:56 2019 (r349501) +++ projects/fuse2/sbin/mount_fusefs/mount_fusefs.c Fri Jun 28 04:18:10 2019 (r349502) @@ -5,6 +5,11 @@ * Copyright (c) 2005 Csaba Henk * All rights reserved. * + * Copyright (c) 2019 The FreeBSD Foundation + * + * Portions of this software were developed by BFF Storage Systems under + * sponsorship from the FreeBSD Foundation. + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: Modified: projects/fuse2/share/man/man5/fusefs.5 ============================================================================== --- projects/fuse2/share/man/man5/fusefs.5 Fri Jun 28 04:02:56 2019 (r349501) +++ projects/fuse2/share/man/man5/fusefs.5 Fri Jun 28 04:18:10 2019 (r349502) @@ -3,8 +3,8 @@ .\" .\" Copyright (c) 2019 The FreeBSD Foundation .\" -.\" This software was developed by BFF Storage Systems, LLC under sponsorship -.\" from the FreeBSD Foundation. +.\" This documentation was written by BFF Storage Systems, LLC under +.\" sponsorship from the FreeBSD Foundation. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions Modified: projects/fuse2/share/man/man9/VOP_FSYNC.9 ============================================================================== --- projects/fuse2/share/man/man9/VOP_FSYNC.9 Fri Jun 28 04:02:56 2019 (r349501) +++ projects/fuse2/share/man/man9/VOP_FSYNC.9 Fri Jun 28 04:18:10 2019 (r349502) @@ -4,6 +4,11 @@ .\" .\" All rights reserved. .\" +.\" Copyright (c) 2019 The FreeBSD Foundation +.\" +.\" Portions of this documentation were written by BFF Storage Systems under +.\" sponsorship from the FreeBSD Foundation. +.\" .\" This program is free software. .\" .\" Redistribution and use in source and binary forms, with or without Modified: projects/fuse2/sys/fs/fuse/fuse.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse.h Fri Jun 28 04:02:56 2019 (r349501) +++ projects/fuse2/sys/fs/fuse/fuse.h Fri Jun 28 04:18:10 2019 (r349502) @@ -32,6 +32,11 @@ * * Copyright (C) 2005 Csaba Henk. * All rights reserved. + * + * Copyright (c) 2019 The FreeBSD Foundation + * + * Portions of this software were developed by BFF Storage Systems, LLC under + * sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: projects/fuse2/sys/fs/fuse/fuse_device.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_device.c Fri Jun 28 04:02:56 2019 (r349501) +++ projects/fuse2/sys/fs/fuse/fuse_device.c Fri Jun 28 04:18:10 2019 (r349502) @@ -33,6 +33,11 @@ * Copyright (C) 2005 Csaba Henk. * All rights reserved. * + * Copyright (c) 2019 The FreeBSD Foundation + * + * Portions of this software were developed by BFF Storage Systems, LLC under + * sponsorship from the FreeBSD Foundation. + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: Modified: projects/fuse2/sys/fs/fuse/fuse_file.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_file.c Fri Jun 28 04:02:56 2019 (r349501) +++ projects/fuse2/sys/fs/fuse/fuse_file.c Fri Jun 28 04:18:10 2019 (r349502) @@ -33,6 +33,11 @@ * Copyright (C) 2005 Csaba Henk. * All rights reserved. * + * Copyright (c) 2019 The FreeBSD Foundation + * + * Portions of this software were developed by BFF Storage Systems, LLC under + * sponsorship from the FreeBSD Foundation. + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: Modified: projects/fuse2/sys/fs/fuse/fuse_file.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_file.h Fri Jun 28 04:02:56 2019 (r349501) +++ projects/fuse2/sys/fs/fuse/fuse_file.h Fri Jun 28 04:18:10 2019 (r349502) @@ -32,6 +32,11 @@ * * Copyright (C) 2005 Csaba Henk. * All rights reserved. + * + * Copyright (c) 2019 The FreeBSD Foundation + * + * Portions of this software were developed by BFF Storage Systems, LLC under + * sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: projects/fuse2/sys/fs/fuse/fuse_internal.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_internal.c Fri Jun 28 04:02:56 2019 (r349501) +++ projects/fuse2/sys/fs/fuse/fuse_internal.c Fri Jun 28 04:18:10 2019 (r349502) @@ -33,6 +33,11 @@ * Copyright (C) 2005 Csaba Henk. * All rights reserved. * + * Copyright (c) 2019 The FreeBSD Foundation + * + * Portions of this software were developed by BFF Storage Systems, LLC under + * sponsorship from the FreeBSD Foundation. + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: Modified: projects/fuse2/sys/fs/fuse/fuse_internal.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_internal.h Fri Jun 28 04:02:56 2019 (r349501) +++ projects/fuse2/sys/fs/fuse/fuse_internal.h Fri Jun 28 04:18:10 2019 (r349502) @@ -32,6 +32,11 @@ * * Copyright (C) 2005 Csaba Henk. * All rights reserved. + * + * Copyright (c) 2019 The FreeBSD Foundation + * + * Portions of this software were developed by BFF Storage Systems, LLC under + * sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: projects/fuse2/sys/fs/fuse/fuse_io.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_io.c Fri Jun 28 04:02:56 2019 (r349501) +++ projects/fuse2/sys/fs/fuse/fuse_io.c Fri Jun 28 04:18:10 2019 (r349502) @@ -33,6 +33,11 @@ * Copyright (C) 2005 Csaba Henk. * All rights reserved. * + * Copyright (c) 2019 The FreeBSD Foundation + * + * Portions of this software were developed by BFF Storage Systems, LLC under + * sponsorship from the FreeBSD Foundation. + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: Modified: projects/fuse2/sys/fs/fuse/fuse_io.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_io.h Fri Jun 28 04:02:56 2019 (r349501) +++ projects/fuse2/sys/fs/fuse/fuse_io.h Fri Jun 28 04:18:10 2019 (r349502) @@ -32,6 +32,11 @@ * * Copyright (C) 2005 Csaba Henk. * All rights reserved. + * + * Copyright (c) 2019 The FreeBSD Foundation + * + * Portions of this software were developed by BFF Storage Systems, LLC under + * sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: projects/fuse2/sys/fs/fuse/fuse_ipc.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_ipc.c Fri Jun 28 04:02:56 2019 (r349501) +++ projects/fuse2/sys/fs/fuse/fuse_ipc.c Fri Jun 28 04:18:10 2019 (r349502) @@ -33,6 +33,11 @@ * Copyright (C) 2005 Csaba Henk. * All rights reserved. * + * Copyright (c) 2019 The FreeBSD Foundation + * + * Portions of this software were developed by BFF Storage Systems, LLC under + * sponsorship from the FreeBSD Foundation. + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: Modified: projects/fuse2/sys/fs/fuse/fuse_ipc.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_ipc.h Fri Jun 28 04:02:56 2019 (r349501) +++ projects/fuse2/sys/fs/fuse/fuse_ipc.h Fri Jun 28 04:18:10 2019 (r349502) @@ -32,6 +32,11 @@ * * Copyright (C) 2005 Csaba Henk. * All rights reserved. + * + * Copyright (c) 2019 The FreeBSD Foundation + * + * Portions of this software were developed by BFF Storage Systems, LLC under + * sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: projects/fuse2/sys/fs/fuse/fuse_main.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_main.c Fri Jun 28 04:02:56 2019 (r349501) +++ projects/fuse2/sys/fs/fuse/fuse_main.c Fri Jun 28 04:18:10 2019 (r349502) @@ -33,6 +33,11 @@ * Copyright (C) 2005 Csaba Henk. * All rights reserved. * + * Copyright (c) 2019 The FreeBSD Foundation + * + * Portions of this software were developed by BFF Storage Systems, LLC under + * sponsorship from the FreeBSD Foundation. + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: Modified: projects/fuse2/sys/fs/fuse/fuse_node.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_node.c Fri Jun 28 04:02:56 2019 (r349501) +++ projects/fuse2/sys/fs/fuse/fuse_node.c Fri Jun 28 04:18:10 2019 (r349502) @@ -33,6 +33,11 @@ * Copyright (C) 2005 Csaba Henk. * All rights reserved. * + * Copyright (c) 2019 The FreeBSD Foundation + * + * Portions of this software were developed by BFF Storage Systems, LLC under + * sponsorship from the FreeBSD Foundation. + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: Modified: projects/fuse2/sys/fs/fuse/fuse_node.h ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_node.h Fri Jun 28 04:02:56 2019 (r349501) +++ projects/fuse2/sys/fs/fuse/fuse_node.h Fri Jun 28 04:18:10 2019 (r349502) @@ -32,6 +32,11 @@ * * Copyright (C) 2005 Csaba Henk. * All rights reserved. + * + * Copyright (c) 2019 The FreeBSD Foundation + * + * Portions of this software were developed by BFF Storage Systems, LLC under + * sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: projects/fuse2/sys/fs/fuse/fuse_vfsops.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_vfsops.c Fri Jun 28 04:02:56 2019 (r349501) +++ projects/fuse2/sys/fs/fuse/fuse_vfsops.c Fri Jun 28 04:18:10 2019 (r349502) @@ -33,6 +33,11 @@ * Copyright (C) 2005 Csaba Henk. * All rights reserved. * + * Copyright (c) 2019 The FreeBSD Foundation + * + * Portions of this software were developed by BFF Storage Systems, LLC under + * sponsorship from the FreeBSD Foundation. + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: Modified: projects/fuse2/sys/fs/fuse/fuse_vnops.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_vnops.c Fri Jun 28 04:02:56 2019 (r349501) +++ projects/fuse2/sys/fs/fuse/fuse_vnops.c Fri Jun 28 04:18:10 2019 (r349502) @@ -33,6 +33,11 @@ * Copyright (C) 2005 Csaba Henk. * All rights reserved. * + * Copyright (c) 2019 The FreeBSD Foundation + * + * Portions of this software were developed by BFF Storage Systems, LLC under + * sponsorship from the FreeBSD Foundation. + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: From owner-svn-src-projects@freebsd.org Fri Jun 28 18:48:03 2019 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 35BF515C8E25 for ; Fri, 28 Jun 2019 18:48:03 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CD1CB842B5; Fri, 28 Jun 2019 18:48:02 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A77702CC9; Fri, 28 Jun 2019 18:48:02 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x5SIm2sv043792; Fri, 28 Jun 2019 18:48:02 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x5SIm2Mo043791; Fri, 28 Jun 2019 18:48:02 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201906281848.x5SIm2Mo043791@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 28 Jun 2019 18:48:02 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r349513 - in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Group: projects X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in projects/fuse2: sys/fs/fuse tests/sys/fs/fusefs X-SVN-Commit-Revision: 349513 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: CD1CB842B5 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.981,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jun 2019 18:48:03 -0000 Author: asomers Date: Fri Jun 28 18:48:02 2019 New Revision: 349513 URL: https://svnweb.freebsd.org/changeset/base/349513 Log: fusefs: don't leak memory of unsent operations on unmount Sponsored by: The FreeBSD Foundation Modified: projects/fuse2/sys/fs/fuse/fuse_device.c projects/fuse2/tests/sys/fs/fusefs/destroy.cc Modified: projects/fuse2/sys/fs/fuse/fuse_device.c ============================================================================== --- projects/fuse2/sys/fs/fuse/fuse_device.c Fri Jun 28 18:03:13 2019 (r349512) +++ projects/fuse2/sys/fs/fuse/fuse_device.c Fri Jun 28 18:48:02 2019 (r349513) @@ -157,6 +157,13 @@ fdata_dtor(void *arg) fuse_ticket_drop(tick); } fuse_lck_mtx_unlock(fdata->aw_mtx); + + /* Cleanup unsent operations */ + fuse_lck_mtx_lock(fdata->ms_mtx); + while ((tick = fuse_ms_pop(fdata))) { + fuse_ticket_drop(tick); + } + fuse_lck_mtx_unlock(fdata->ms_mtx); FUSE_UNLOCK(); fdata_trydestroy(fdata); Modified: projects/fuse2/tests/sys/fs/fusefs/destroy.cc ============================================================================== --- projects/fuse2/tests/sys/fs/fusefs/destroy.cc Fri Jun 28 18:03:13 2019 (r349512) +++ projects/fuse2/tests/sys/fs/fusefs/destroy.cc Fri Jun 28 18:48:02 2019 (r349513) @@ -28,12 +28,105 @@ * SUCH DAMAGE. */ +extern "C" { +#include +#include +#include +} + #include "mockfs.hh" #include "utils.hh" using namespace testing; +/* Tests for orderly unmounts */ class Destroy: public FuseTest {}; + +/* Tests for unexpected deaths of the server */ +class Death: public FuseTest{}; + +static void* open_th(void* arg) { + int fd; + const char *path = (const char*)arg; + + fd = open(path, O_RDONLY); + EXPECT_EQ(-1, fd); + EXPECT_EQ(ENOTCONN, errno); + return 0; +} + +/* + * The server dies with unsent operations still on the message queue. + * Check for any memory leaks like this: + * 1) kldunload fusefs, if necessary + * 2) kldload fusefs + * 3) ./destroy --gtest_filter=Destroy.unsent_operations + * 4) kldunload fusefs + * 5) check /var/log/messages for anything like this: +Freed UMA keg (fuse_ticket) was not empty (31 items). Lost 2 pages of memory. +Warning: memory type fuse_msgbuf leaked memory on destroy (68 allocations, 428800 bytes leaked). + */ +TEST_F(Death, unsent_operations) +{ + const char FULLPATH0[] = "mountpoint/some_file.txt"; + const char FULLPATH1[] = "mountpoint/other_file.txt"; + const char RELPATH0[] = "some_file.txt"; + const char RELPATH1[] = "other_file.txt"; + pthread_t th0, th1; + ino_t ino0 = 42, ino1 = 43; + sem_t sem; + mode_t mode = S_IFREG | 0644; + + sem_init(&sem, 0, 0); + + EXPECT_LOOKUP(FUSE_ROOT_ID, RELPATH0) + .WillRepeatedly(Invoke( + ReturnImmediate([=](auto in __unused, auto& out) { + SET_OUT_HEADER_LEN(out, entry); + out.body.entry.attr.mode = mode; + out.body.entry.nodeid = ino0; + out.body.entry.attr.nlink = 1; + }))); + EXPECT_LOOKUP(FUSE_ROOT_ID, RELPATH1) + .WillRepeatedly(Invoke( + ReturnImmediate([=](auto in __unused, auto& out) { + SET_OUT_HEADER_LEN(out, entry); + out.body.entry.attr.mode = mode; + out.body.entry.nodeid = ino1; + out.body.entry.attr.nlink = 1; + }))); + + EXPECT_CALL(*m_mock, process( + ResultOf([&](auto in) { + return (in.header.opcode == FUSE_OPEN); + }, Eq(true)), + _) + ).WillOnce(Invoke([&](auto in __unused, auto &out __unused) { + sem_post(&sem); + pause(); + })); + + /* + * One thread's operation will be sent to the daemon and block, and the + * other's will be stuck in the message queue. + */ + ASSERT_EQ(0, pthread_create(&th0, NULL, open_th, (void*)FULLPATH0)) + << strerror(errno); + ASSERT_EQ(0, pthread_create(&th1, NULL, open_th, (void*)FULLPATH1)) + << strerror(errno); + + /* Wait for the first thread to block */ + sem_wait(&sem); + /* Give the second thread time to block */ + nap(); + + m_mock->kill_daemon(); + + pthread_join(th0, NULL); + pthread_join(th1, NULL); + + sem_destroy(&sem); +} /* * On unmount the kernel should send a FUSE_DESTROY operation. It should also