From owner-freebsd-fs@freebsd.org Sun Jan 3 03:01:23 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C799A5F864 for ; Sun, 3 Jan 2016 03:01:23 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from vms173005pub.verizon.net (vms173005pub.verizon.net [206.46.173.5]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4350415FB for ; Sun, 3 Jan 2016 03:01:22 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.narawntapu ([100.1.236.52]) by vms173005.mailsrvcs.net (Oracle Communications Messaging Server 7.0.5.32.0 64bit (built Jul 16 2014)) with ESMTPA id <0O0C00AWXTLFD230@vms173005.mailsrvcs.net> for freebsd-fs@FreeBSD.org; Sat, 02 Jan 2016 20:00:57 -0600 (CST) X-CMAE-Score: 0 X-CMAE-Analysis: v=2.1 cv=btqxfxui c=1 sm=1 tr=0 a=UorMnhrCY2jH/mPejITChw==:117 a=LaogzpLLAAAA:8 a=oR5dmqMzAAAA:8 a=7aQ_Q-yQQ-AA:10 a=r77TgQKjGQsHNAKrUKIA:9 a=FnL9iEPV6B8Ad-8pvCMA:9 a=QEXdDO2ut3YA:10 a=oRl7WY1OJqzf9bcHoSsA:9 a=_W_S_7VecoQA:10 From: "Mikhail T." Subject: NFS reads vs. writes To: freebsd-fs@FreeBSD.org Message-id: <568880D3.3010402@aldan.algebra.com> Date: Sat, 02 Jan 2016 21:00:51 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 03:01:23 -0000 I have two similar 10.2-STABLE machines mounting each other's filesystems via NFS: a:/a and b:/b When moving some large files around today, I found a huge discrepancy between writing and reading over NFS. That is, machine a copying files from its own /a to NFS-mounted b:/b was pushing measly 2-3Mb/s over the Ethernet. When I cancelled that, logged-in to machine b and proceed to copy from a:/a to the now-local /b, I got 56Mb/s. I tried changing the wsize and rsize options, but it did not seem to make a difference... Any ideas, what is happening here? Why are NFS-writes some 25 times slower here, than reads? Both filesystems are zfs-backed (though with different options), both systems are plugged into the same switch, both use mtu of 9000. Thanks! Yours, -mi From owner-freebsd-fs@freebsd.org Sun Jan 3 04:35:00 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AC3E7A53625 for ; Sun, 3 Jan 2016 04:35:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 90C9710D9 for ; Sun, 3 Jan 2016 04:35:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u034Z0ur009289 for ; Sun, 3 Jan 2016 04:35:00 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205816] [ext2fs] [patch] EXT4 sparse blocks unsupported, contain garbage when read Date: Sun, 03 Jan 2016 04:35:00 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: damjan.jov@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status keywords bug_severity priority component assigned_to reporter cc attachments.created Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 04:35:00 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205816 Bug ID: 205816 Summary: [ext2fs] [patch] EXT4 sparse blocks unsupported, contain garbage when read Product: Base System Version: 11.0-CURRENT Hardware: Any OS: Any Status: New Keywords: patch Severity: Affects Many People Priority: --- Component: kern Assignee: freebsd-bugs@FreeBSD.org Reporter: damjan.jov@gmail.com CC: freebsd-fs@FreeBSD.org Created attachment 164979 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D164979&action= =3Dedit preliminary patch to implement support for EXT4 sparse files The ext2fs module does not support sparse files on EXT4 filesystems, yet doesn't fail. When attempting to read the sparse blocks from a sparse file, instead of zeroes, they contain garbage data read from the wrong disk block= s. Here's an example sparse file in debugfs: debugfs: dump_extents /home/user/Documents/file.iso Level Entries Logical Physical Length Flags 0/ 1 1/ 1 0 - 1122599 1082775 1122600 1/ 1 1/ 53 0 - 5 1372160 - 1372165 6 1/ 1 2/ 53 8 - 11 1372168 - 1372171 4 1/ 1 3/ 53 16 - 19 1372176 - 1372179 4 1/ 1 4/ 53 24 - 27 1372184 - 1372187 4 1/ 1 5/ 53 32 - 33 1372192 - 1372193 2 ... Note how block ranges 0-5, 8-11, 16-19 are allocated, but the in between bl= ocks are sparse. The problem starts in the ext4_ext_binsearch() function in ext2_extents.c w= hich expects the block ranges in the extents to be continuous, and doesn't check whether the lbn parameter even lies inside the found extent's block range. = It blindly sets: path->ep_ext =3D l - 1; which will set the pointer to invalid memory (a part of struct ext2_extent_header) when lbn=3D0 and the first block of a file is sparse. A= lso in the above example, lbn=3D6 will use the 0-5 extent, reading block 0 instead. My preliminary patch improves ext4_ext_binsearch() to check for out of range blocks, marks the path as being sparse, and stores the range of extents starting at the lbn that is sparse. Sparse extents are cached in ext4_ext_read() for future reuse. Actual reading of sparse blocks is implemented by copying from a static array of zeroes (is there a better way= of doing it?) and read() definitely works, but mmap() doesn't seem to (system instantly reboots when reading even 1 byte from mapped region, but this also happens for some dense files, so it's not (just?) my patch). If testing against other EXT4 implementations, note that ext4fuse also has = this bug. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Sun Jan 3 04:52:01 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ED313A53A69 for ; Sun, 3 Jan 2016 04:52:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DDE8418D4 for ; Sun, 3 Jan 2016 04:52:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u034q1QE042432 for ; Sun, 3 Jan 2016 04:52:01 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205668] [ext2fs] [patch] cannot mount EXT4 filesystems which weren't cleanly unmounted Date: Sun, 03 Jan 2016 04:52:02 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: damjan.jov@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 04:52:02 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205668 --- Comment #4 from Damjan Jovanovic --- Hi Pedro. Thank you, your patch gets the filesystem to mount. However there= is no warning that it hasn't been cleanly unmounted. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Sun Jan 3 07:31:18 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E86E3A5F2BE for ; Sun, 3 Jan 2016 07:31:18 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from Exch2-2.slu.se (imap.slu.se [77.235.224.122]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "webmail.slu.se", Issuer "TERENA SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8635617DF for ; Sun, 3 Jan 2016 07:31:17 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (77.235.224.124) by Exch2-2.slu.se (77.235.224.122) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Sun, 3 Jan 2016 08:16:04 +0100 Received: from exch2-4.slu.se ([::1]) by exch2-4.slu.se ([fe80::a44b:9cc6:beb0:b0f2%22]) with mapi id 15.00.1104.000; Sun, 3 Jan 2016 08:16:04 +0100 From: =?utf-8?B?S2FybGkgU2rDtmJlcmc=?= To: Mikhail T. CC: "freebsd-fs@FreeBSD.org" Subject: Re: NFS reads vs. writes Thread-Topic: NFS reads vs. writes Thread-Index: AQHRRfab4CZbQ4v5Eka/+DKD/zFriA== Date: Sun, 3 Jan 2016 07:16:04 +0000 Message-ID: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> Accept-Language: sv-SE, en-US Content-Language: sv-SE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 07:31:19 -0000 DQpEZW4gMyBqYW4uIDIwMTYgNDowMSBmbSBza3JldiAiTWlraGFpbCBULiIgPG1pK3RodW5AYWxk YW4uYWxnZWJyYS5jb20+Og0KPg0KPiBJIGhhdmUgdHdvIHNpbWlsYXIgMTAuMi1TVEFCTEUgbWFj aGluZXMgbW91bnRpbmcgZWFjaCBvdGhlcidzDQo+IGZpbGVzeXN0ZW1zIHZpYSBORlM6IGE6L2Eg YW5kIGI6L2INCj4NCj4gV2hlbiBtb3Zpbmcgc29tZSBsYXJnZSBmaWxlcyBhcm91bmQgdG9kYXks IEkgZm91bmQgYSBodWdlIGRpc2NyZXBhbmN5DQo+IGJldHdlZW4gd3JpdGluZyBhbmQgcmVhZGlu ZyBvdmVyIE5GUy4gVGhhdCBpcywgbWFjaGluZSBhIGNvcHlpbmcgZmlsZXMNCj4gZnJvbSBpdHMg b3duIC9hIHRvIE5GUy1tb3VudGVkIGI6L2Igd2FzIHB1c2hpbmcgbWVhc2x5IDItM01iL3Mgb3Zl ciB0aGUNCj4gRXRoZXJuZXQuIFdoZW4gSSBjYW5jZWxsZWQgdGhhdCwgbG9nZ2VkLWluIHRvIG1h Y2hpbmUgYiBhbmQgcHJvY2VlZCB0bw0KPiBjb3B5IGZyb20gYTovYSB0byB0aGUgbm93LWxvY2Fs IC9iLCBJIGdvdCA1Nk1iL3MuIEkgdHJpZWQgY2hhbmdpbmcgdGhlDQo+IHdzaXplIGFuZCByc2l6 ZSBvcHRpb25zLCBidXQgaXQgZGlkIG5vdCBzZWVtIHRvIG1ha2UgYSBkaWZmZXJlbmNlLi4uDQo+ DQo+IEFueSBpZGVhcywgd2hhdCBpcyBoYXBwZW5pbmcgaGVyZT8gV2h5IGFyZSBORlMtd3JpdGVz IHNvbWUgMjUgdGltZXMNCj4gc2xvd2VyIGhlcmUsIHRoYW4gcmVhZHM/IEJvdGggZmlsZXN5c3Rl bXMgYXJlIHpmcy1iYWNrZWQgKHRob3VnaCB3aXRoDQo+IGRpZmZlcmVudCBvcHRpb25zKSwgYm90 aCBzeXN0ZW1zIGFyZSBwbHVnZ2VkIGludG8gdGhlIHNhbWUgc3dpdGNoLCBib3RoDQo+IHVzZSBt dHUgb2YgOTAwMC4NCg0KVGhlIGRpZmZlcmVuY2UgYmV0d2VlbiAibW91bnQiIGFuZCAibW91bnQg LW8gYXN5bmMiIHNob3VsZCB0ZWxsIHlvdSBpZiB5b3UnZCBiZW5lZml0IGZyb20gYSBzZXBhcmF0 ZSBsb2cgZGV2aWNlIGluIHRoZSBwb29sLg0KDQovSw0KDQo+DQo+IFRoYW5rcyEgWW91cnMsDQo+ DQo+ICAgICAtbWkNCj4NCj4gX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX18NCj4gZnJlZWJzZC1mc0BmcmVlYnNkLm9yZyBtYWlsaW5nIGxpc3QNCj4gaHR0cHM6 Ly9saXN0cy5mcmVlYnNkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2ZyZWVic2QtZnMNCj4gVG8gdW5z dWJzY3JpYmUsIHNlbmQgYW55IG1haWwgdG8gImZyZWVic2QtZnMtdW5zdWJzY3JpYmVAZnJlZWJz ZC5vcmciDQo= From owner-freebsd-fs@freebsd.org Sun Jan 3 07:55:02 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EB386A5F9BE for ; Sun, 3 Jan 2016 07:55:02 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from vms173005pub.verizon.net (vms173005pub.verizon.net [206.46.173.5]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D0EC21547 for ; Sun, 3 Jan 2016 07:55:02 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) MIME-version: 1.0 Received: from aldan.narawntapu ([100.1.236.52]) by vms173005.mailsrvcs.net (Oracle Communications Messaging Server 7.0.5.32.0 64bit (built Jul 16 2014)) with ESMTPA id <0O0D00D7S9Z5M240@vms173005.mailsrvcs.net> for freebsd-fs@FreeBSD.org; Sun, 03 Jan 2016 01:54:42 -0600 (CST) X-CMAE-Score: 0 X-CMAE-Analysis: v=2.1 cv=btqxfxui c=1 sm=1 tr=0 a=UorMnhrCY2jH/mPejITChw==:117 a=LaogzpLLAAAA:8 a=oR5dmqMzAAAA:8 a=7aQ_Q-yQQ-AA:10 a=r77TgQKjGQsHNAKrUKIA:9 a=woJa6gbRWdhDhg-VFfoA:9 a=QEXdDO2ut3YA:10 a=n8i27M1mAAAA:8 a=EepKwpTd6zwdDYeGpzoA:9 a=_W_S_7VecoQA:10 Subject: Re: NFS reads vs. writes To: =?UTF-8?Q?Karli_Sj=c3=b6berg?= References: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> Cc: "freebsd-fs@FreeBSD.org" From: "Mikhail T." Message-id: <5688D3C1.90301@aldan.algebra.com> Date: Sun, 03 Jan 2016 02:54:41 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 In-reply-to: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 07:55:03 -0000 On 03.01.2016 02:16, Karli Sjöberg wrote: > > The difference between "mount" and "mount -o async" should tell you if > you'd benefit from a separate log device in the pool. > This is not a ZFS problem. The same filesystem is being read in both cases. The same data is being read from and written to the same filesystems. For some reason, it is much faster to read via NFS than to write to it, however. And finally, just to put the matter to rest, both ZFS-pools already have a separate zil-device (on an SSD). -mi From owner-freebsd-fs@freebsd.org Sun Jan 3 08:57:15 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 40EF7A60DCB for ; Sun, 3 Jan 2016 08:57:15 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from EXCH2-1.slu.se (pop.slu.se [77.235.224.121]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "webmail.slu.se", Issuer "TERENA SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CF92219DE for ; Sun, 3 Jan 2016 08:57:14 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (77.235.224.124) by EXCH2-1.slu.se (77.235.224.121) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Sun, 3 Jan 2016 09:42:00 +0100 Received: from exch2-4.slu.se ([::1]) by exch2-4.slu.se ([fe80::a44b:9cc6:beb0:b0f2%22]) with mapi id 15.00.1104.000; Sun, 3 Jan 2016 09:42:00 +0100 From: =?utf-8?B?S2FybGkgU2rDtmJlcmc=?= To: Mikhail T. CC: "freebsd-fs@FreeBSD.org" Subject: Re: NFS reads vs. writes Thread-Topic: NFS reads vs. writes Thread-Index: AQHRRgKc4CZbQ4v5Eka/+DKD/zFriA== Date: Sun, 3 Jan 2016 08:41:59 +0000 Message-ID: <332f85ba-a440-45fd-acd9-5b9e2184131b@email.android.com> Accept-Language: sv-SE, en-US Content-Language: sv-SE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 08:57:15 -0000 DQpEZW4gMyBqYW4uIDIwMTYgODo1NCBmbSBza3JldiAiTWlraGFpbCBULiIgPG1pK3RodW5AYWxk YW4uYWxnZWJyYS5jb20+Og0KPg0KPiBPbiAwMy4wMS4yMDE2IDAyOjE2LCBLYXJsaSBTasO2YmVy ZyB3cm90ZToNCj4+DQo+PiBUaGUgZGlmZmVyZW5jZSBiZXR3ZWVuICJtb3VudCIgYW5kICJtb3Vu dCAtbyBhc3luYyIgc2hvdWxkIHRlbGwgeW91IGlmIHlvdSdkIGJlbmVmaXQgZnJvbSBhIHNlcGFy YXRlIGxvZyBkZXZpY2UgaW4gdGhlIHBvb2wuDQo+DQo+IFRoaXMgaXMgbm90IGEgWkZTIHByb2Js ZW0uIFRoZSBzYW1lIGZpbGVzeXN0ZW0gaXMgYmVpbmcgcmVhZCBpbiBib3RoIGNhc2VzLiBUaGUg c2FtZSBkYXRhIGlzIGJlaW5nIHJlYWQgZnJvbSBhbmQgd3JpdHRlbiB0byB0aGUgc2FtZSBmaWxl c3lzdGVtcy4gRm9yIHNvbWUgcmVhc29uLCBpdCBpcyBtdWNoIGZhc3RlciB0byByZWFkIHZpYSBO RlMgdGhhbiB0byB3cml0ZSB0byBpdCwgaG93ZXZlci4NCj4NCj4gQW5kIGZpbmFsbHksIGp1c3Qg dG8gcHV0IHRoZSBtYXR0ZXIgdG8gcmVzdCwgYm90aCBaRlMtcG9vbHMgYWxyZWFkeSBoYXZlIGEg c2VwYXJhdGUgemlsLWRldmljZSAob24gYW4gU1NEKS4NCj4+DQo+PiAtbWkNCg0KT2ssIHNvIHdo YXQgYWJvdXQgdGhlIGxvY2FsIHBlcmZvcm1hbmNlPyBIYXZlIHlvdSBydW4gZS5nLiBCb25uaWUr KyBpbiB0aGUgc2FtZSBkaXJlY3Rvcnk/DQoNCi9LDQo= From owner-freebsd-fs@freebsd.org Sun Jan 3 14:36:07 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F10C2A605C1 for ; Sun, 3 Jan 2016 14:36:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E200C1BA7 for ; Sun, 3 Jan 2016 14:36:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u03Ea7LP060898 for ; Sun, 3 Jan 2016 14:36:07 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205668] [ext2fs] [patch] cannot mount EXT4 filesystems which weren't cleanly unmounted Date: Sun, 03 Jan 2016 14:36:08 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 14:36:08 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205668 --- Comment #5 from Pedro F. Giffuni --- (In reply to Damjan Jovanovic from comment #4) Hi Damjan; Yes, this is consistent with us ignoring the feature: read-only operation should not touch the disk and warning only when unmounting would give the impression that *we* somehow corrupted the information. Let the rw implementation (in linux) deal with it. An alternative would have been to simply not mount the disk and let the user deal with ext4 fsck (I think that was what was happening originally). --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Sun Jan 3 14:42:17 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F531A60947 for ; Sun, 3 Jan 2016 14:42:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 105BC1EC5 for ; Sun, 3 Jan 2016 14:42:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u03EgGfP072928 for ; Sun, 3 Jan 2016 14:42:16 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205816] [ext2fs] [patch] EXT4 sparse blocks unsupported, contain garbage when read Date: Sun, 03 Jan 2016 14:42:16 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to cc bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 14:42:17 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205816 Pedro F. Giffuni changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |pfg@FreeBSD.org CC| |flz@FreeBSD.org, | |pfg@FreeBSD.org Status|New |In Progress --- Comment #1 from Pedro F. Giffuni --- Thank you Damjan: I am CC'ing Zheng Liu as he did the original ext4 implementation. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Sun Jan 3 14:43:15 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38C35A60A5D for ; Sun, 3 Jan 2016 14:43:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 29E7A12AF for ; Sun, 3 Jan 2016 14:43:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u03EhEi4075847 for ; Sun, 3 Jan 2016 14:43:15 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205816] [ext2fs] [patch] EXT4 sparse blocks unsupported, contain garbage when read Date: Sun, 03 Jan 2016 14:43:15 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 14:43:15 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205816 Pedro F. Giffuni changed: What |Removed |Added ---------------------------------------------------------------------------- CC|flz@FreeBSD.org |gnehzuil@gmail.com --- Comment #2 from Pedro F. Giffuni --- drop flz@ ... I meant lz@ --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Sun Jan 3 15:05:31 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C7453A5F1A2 for ; Sun, 3 Jan 2016 15:05:31 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from vms173015pub.verizon.net (vms173015pub.verizon.net [206.46.173.15]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ABEDE1EF0 for ; Sun, 3 Jan 2016 15:05:31 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) MIME-version: 1.0 Received: from aldan.narawntapu ([100.1.236.52]) by vms173015.mailsrvcs.net (Oracle Communications Messaging Server 7.0.5.32.0 64bit (built Jul 16 2014)) with ESMTPA id <0O0D00BOTR40IM40@vms173015.mailsrvcs.net> for freebsd-fs@FreeBSD.org; Sun, 03 Jan 2016 08:04:53 -0600 (CST) X-CMAE-Score: 0 X-CMAE-Analysis: v=2.1 cv=J+9Xl1TS c=1 sm=1 tr=0 a=UorMnhrCY2jH/mPejITChw==:117 a=LaogzpLLAAAA:8 a=oR5dmqMzAAAA:8 a=7aQ_Q-yQQ-AA:10 a=r77TgQKjGQsHNAKrUKIA:9 a=BUiEDfTYXKncft4odhMA:9 a=QEXdDO2ut3YA:10 a=n8i27M1mAAAA:8 a=zjlkUf9so2vQcvGYABIA:9 a=_W_S_7VecoQA:10 Subject: Re: NFS reads vs. writes To: =?UTF-8?Q?Karli_Sj=c3=b6berg?= References: <332f85ba-a440-45fd-acd9-5b9e2184131b@email.android.com> Cc: "freebsd-fs@FreeBSD.org" From: "Mikhail T." Message-id: <56892A80.2000207@aldan.algebra.com> Date: Sun, 03 Jan 2016 09:04:48 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 In-reply-to: <332f85ba-a440-45fd-acd9-5b9e2184131b@email.android.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 15:05:31 -0000 On 03.01.2016 03:41, Karli Sjöberg wrote: > > Ok, so what about the local performance? Have you run e.g. Bonnie++ in > the same directory? > Let me try to explain the problem one more time. As I stated already, the data is read from and written to the same filesystems: a:/a --- NFS ---> b:/b The performance difference is between machine a /writing/ over NFS (2Mb/s) vs. machine b /reading/ over NFS (56Mb/s). If it is still unclear, I give up. -mi From owner-freebsd-fs@freebsd.org Sun Jan 3 15:21:13 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 17198A5F84D for ; Sun, 3 Jan 2016 15:21:13 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from EXCH2-1.slu.se (imap.slu.se [77.235.224.121]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "webmail.slu.se", Issuer "TERENA SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A7E0119EA for ; Sun, 3 Jan 2016 15:21:11 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (77.235.224.124) by EXCH2-1.slu.se (77.235.224.121) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Sun, 3 Jan 2016 16:21:08 +0100 Received: from exch2-4.slu.se ([::1]) by exch2-4.slu.se ([fe80::a44b:9cc6:beb0:b0f2%22]) with mapi id 15.00.1104.000; Sun, 3 Jan 2016 16:21:08 +0100 From: =?utf-8?B?S2FybGkgU2rDtmJlcmc=?= To: Mikhail T. CC: "freebsd-fs@FreeBSD.org" Subject: Re: NFS reads vs. writes Thread-Topic: NFS reads vs. writes Thread-Index: AQHRRjpe4CZbQ4v5Eka/+DKD/zFriA== Date: Sun, 3 Jan 2016 15:21:07 +0000 Message-ID: <8636f137-a586-4bf8-97b8-51e4c8ba50e5@email.android.com> Accept-Language: sv-SE, en-US Content-Language: sv-SE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 15:21:13 -0000 DQpEZW4gMyBqYW4uIDIwMTYgMzowNSBlbSBza3JldiAiTWlraGFpbCBULiIgPG1pK3RodW5AYWxk YW4uYWxnZWJyYS5jb20+Og0KPg0KPiBPbiAwMy4wMS4yMDE2IDAzOjQxLCBLYXJsaSBTasO2YmVy ZyB3cm90ZToNCj4+DQo+PiBPaywgc28gd2hhdCBhYm91dCB0aGUgbG9jYWwgcGVyZm9ybWFuY2U/ IEhhdmUgeW91IHJ1biBlLmcuIEJvbm5pZSsrIGluIHRoZSBzYW1lIGRpcmVjdG9yeT8NCj4NCj4g TGV0IG1lIHRyeSB0byBleHBsYWluIHRoZSBwcm9ibGVtIG9uZSBtb3JlIHRpbWUuIEFzIEkgc3Rh dGVkIGFscmVhZHksIHRoZSBkYXRhIGlzIHJlYWQgZnJvbSBhbmQgd3JpdHRlbiB0byB0aGUgc2Ft ZSBmaWxlc3lzdGVtczoNCj4NCj4gICAgIGE6L2EgLS0tIE5GUyAtLS0+IGI6L2INCj4NCj4gVGhl IHBlcmZvcm1hbmNlIGRpZmZlcmVuY2UgaXMgYmV0d2VlbiBtYWNoaW5lIGEgd3JpdGluZyBvdmVy IE5GUyAoMk1iL3MpIHZzLiBtYWNoaW5lIGIgcmVhZGluZyBvdmVyIE5GUyAoNTZNYi9zKS4gSWYg aXQgaXMgc3RpbGwgdW5jbGVhciwgSSBnaXZlIHVwLg0KDQpJIGRvbid0IHRoaW5rIGJlaW5nIHJ1 ZGUgaGFzIGV2ZXIgaGVscGVkIHNvbWVvbmUsIGNlcnRhaW5seSBkb2Vzbid0IG1ha2UgbWUgd2Fu dCB0by4uLg0KDQpCZWZvcmUgSSBsZWF2ZSB5b3UgdG8geW91ciBwcm9ibGVtIGFsb25lLCBJIHdv dWxkIHN1Z2dlc3QgeW91IHRyeSBhbmQgY29weSBmcm9tIG1hY2hpbmUgYSB0byBiIG92ZXIgYSBk aWZmZXJlbnQgcHJvdG9jb2wsIGxpa2UgU01CLCBTRlRQIChwcmVmZXJyZWJseSBvbmx5IGEgc3Ry ZWFtIHdpdGggTk9ORSBjaXBoZXIgZW5hYmxlZCksIEhUVFAgb3IgRlRQIGJlZm9yZSBibGFtaW5n IE5GUy4NCg0KL0sNCg0KPj4NCj4+IC1taQ0K From owner-freebsd-fs@freebsd.org Sun Jan 3 15:38:41 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EA436A5FD92 for ; Sun, 3 Jan 2016 15:38:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D6CE31316 for ; Sun, 3 Jan 2016 15:38:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u03FcfoJ053290 for ; Sun, 3 Jan 2016 15:38:41 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205816] [ext2fs] [patch] EXT4 sparse blocks unsupported, contain garbage when read Date: Sun, 03 Jan 2016 15:38:42 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 15:38:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205816 --- Comment #3 from Pedro F. Giffuni --- Off the top of my head ... We do have some support for sparse files, based on the generic support for UFS: https://svnweb.freebsd.org/base/head/sys/fs/ext2fs/ext2_vnops.c?r1=3D252956= &r2=3D252955&pathrev=3D252956 I don't think we have any filesystem-specific code to deal with it (which would be ideal). --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Sun Jan 3 16:13:15 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A26DA607B3 for ; Sun, 3 Jan 2016 16:13:15 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5F4C01299 for ; Sun, 3 Jan 2016 16:13:14 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from freddy.simplesystems.org (freddy.simplesystems.org [65.66.246.65]) by blade.simplesystems.org (8.14.4+Sun/8.14.4) with ESMTP id u03G9CO4016169; Sun, 3 Jan 2016 10:09:12 -0600 (CST) Date: Sun, 3 Jan 2016 10:09:12 -0600 (CST) From: Bob Friesenhahn X-X-Sender: bfriesen@freddy.simplesystems.org To: "Mikhail T." cc: freebsd-fs@freebsd.org Subject: Re: NFS reads vs. writes In-Reply-To: <568880D3.3010402@aldan.algebra.com> Message-ID: References: <568880D3.3010402@aldan.algebra.com> User-Agent: Alpine 2.01 (GSO 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (blade.simplesystems.org [65.66.246.90]); Sun, 03 Jan 2016 10:09:12 -0600 (CST) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 16:13:15 -0000 On Sat, 2 Jan 2016, Mikhail T. wrote: > I have two similar 10.2-STABLE machines mounting each other's > filesystems via NFS: a:/a and b:/b > > When moving some large files around today, I found a huge discrepancy > between writing and reading over NFS. That is, machine a copying files > from its own /a to NFS-mounted b:/b was pushing measly 2-3Mb/s over the > Ethernet. When I cancelled that, logged-in to machine b and proceed to > copy from a:/a to the now-local /b, I got 56Mb/s. I tried changing the > wsize and rsize options, but it did not seem to make a difference... > > Any ideas, what is happening here? Why are NFS-writes some 25 times > slower here, than reads? Both filesystems are zfs-backed (though with > different options), both systems are plugged into the same switch, both > use mtu of 9000. The most likely issue is a latency problem with synchronous writes on 'b'. The main pool disks seem to be working ok. Make sure that the SSD you are using for slog is working fine. Maybe it is abnormally slow. Check I/O latencies for your disks. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From owner-freebsd-fs@freebsd.org Sun Jan 3 18:46:03 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E7BDFA5F216 for ; Sun, 3 Jan 2016 18:46:02 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from vms173007pub.verizon.net (vms173007pub.verizon.net [206.46.173.7]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CADD61DAA for ; Sun, 3 Jan 2016 18:46:02 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) MIME-version: 1.0 Received: from aldan.narawntapu ([100.1.236.52]) by vms173007.mailsrvcs.net (Oracle Communications Messaging Server 7.0.5.32.0 64bit (built Jul 16 2014)) with ESMTPA id <0O0E00HBJ1BJFW00@vms173007.mailsrvcs.net> for freebsd-fs@freebsd.org; Sun, 03 Jan 2016 11:45:23 -0600 (CST) X-CMAE-Score: 0 X-CMAE-Analysis: v=2.1 cv=Nc0brD34 c=1 sm=1 tr=0 a=UorMnhrCY2jH/mPejITChw==:117 a=LaogzpLLAAAA:8 a=oR5dmqMzAAAA:8 a=7aQ_Q-yQQ-AA:10 a=r77TgQKjGQsHNAKrUKIA:9 a=DeA27PXj1JP4lalNxJwA:9 a=pILNOxqGKmIA:10 a=pGLkceISAAAA:8 a=RLQEIbXJAAAA:8 a=RhaCqEVCSXuYShOpXacA:9 a=3hQuPBN62KDJHp1x:21 a=_W_S_7VecoQA:10 Subject: Re: NFS reads vs. writes To: Bob Friesenhahn , Tom Curry References: <568880D3.3010402@aldan.algebra.com> From: "Mikhail T." X-Enigmail-Draft-Status: N1110 Cc: freebsd-fs@freebsd.org Message-id: <56895E2E.8060405@aldan.algebra.com> Date: Sun, 03 Jan 2016 12:45:18 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 In-reply-to: Content-Type: text/plain; CHARSET=US-ASCII Content-Transfer-Encoding: 7BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 18:46:03 -0000 On 03.01.2016 10:32, Tom Curry wrote: > What does disk activity (gstat or iostat) look like when this is going on? I use systat for such observations. Here is a typical snapshot of machine a, when it reads its own /a and writes over NFS to b:/b: 3,6%Sys 0,0%Intr 15,6%User 0,0%Nice 80,8%Idle ozfod 2 ata0 14 | | | | | | | | | | %ozfod 69 uhci0 ehci ==>>>>>>>> daefr 205 uhci1 ahci 13 dtbuf prcfr 2577 hpet0 20 Namei Name-cache Dir-cache 248362 desvn 2374 totfr 1277 em0 256 Calls hits % hits % 95980 numvn react hdac0 257 1575 1508 96 62091 frevn pdwak 94 hdac1 258 288 pdpgs vgapci0 Disks md0 ada0 ada1 ada2 ada3 da0 da1 intrn KB/t 0,00 119 26,42 19,10 21,72 0,00 0,00 6720528 wire tps 0 47 72 64 69 0 0 721552 act MB/s 0,00 5,42 1,87 1,19 1,45 0,00 0,00 2331396 inact %busy 0 4 19 11 11 0 0 88 cache 403384 free 1056992 buf The ada0 is the ssd hosting both read cache and zil devices, ada{1,2,3} are the three disks comprising a RAID5 zpool. Meanwhile on the b-side the following is going on: 4,2%Sys 0,0%Intr 0,0%User 0,0%Nice 95,8%Idle ozfod hdac0 18 | | | | | | | | | | %ozfod fwohci0 19 == daefr 429 hpet0 uhci 22 dtbuf prcfr 50 uhci0 uhci Namei Name-cache Dir-cache 282383 desvn totfr 598 atapci1 23 Calls hits % hits % 107825 numvn react 141 mpt0 257 18 17 94 70416 frevn pdwak 1025 bce1 258 50 pdpgs Disks md0 ada0 da0 da1 da2 da3 da4 intrn KB/t 0,00 6,50 0,00 80,21 16,00 79,59 68,42 4794972 wire tps 0 594 0 53 2 55 39 130060 act MB/s 0,00 3,77 0,00 4,18 0,03 4,29 2,63 7153984 inact %busy 0 95 0 10 1 14 8 131100 cache Here too the ada0 hosts the log-device and appears to be the bottleneck. There is no read-cache on b, and the zpool consists of da1, da3, and da4 simply striped together (no redundancy). When, instead of /pushing/ data out of a, I begin /pulling/ it (a different file from the same directory) from b, things change drastically. a looks like this: Disks md0 ada0 ada1 ada2 ada3 da0 da1 intrn KB/t 0,00 83,00 64,00 64,00 64,00 0,00 0,00 6547524 wire tps 0 27 469 456 472 0 0 744768 act MB/s 0,00 2,16 29,32 28,49 29,50 0,00 0,00 2722100 inact %busy 0 1 13 13 13 0 0 108 cache and b like this: Disks md0 ada0 da0 da1 da2 da3 da4 intrn KB/t 0,00 15,46 0,00 114 0,00 116 112 4627944 wire tps 0 45 0 189 0 192 160 130376 act MB/s 0,00 0,68 0,00 20,98 0,00 21,74 17,45 7308284 inact %busy 0 81 0 19 0 37 28 145200 cache ada0 is no longer the bottleneck and the copy is over almost instantly. > What is the average latency between the two machines? ping-ing b from a: round-trip min/avg/max/stddev = 0.137/0.156/0.178/0.015 ms ping-ing a from b: round-trip min/avg/max/stddev = 0.114/0.169/0.220/0.036 ms On 03.01.2016 11:09, Bob Friesenhahn wrote: > The most likely issue is a latency problem with synchronous writes on > 'b'. The main pool disks seem to be working ok. Make sure that the > SSD you are using for slog is working fine. Maybe it is abnormally slow. Why would the same ZFS -- with the same slog -- be working faster, when written to locally, than when over NFS? -mi From owner-freebsd-fs@freebsd.org Sun Jan 3 19:20:52 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 66244A5FB15 for ; Sun, 3 Jan 2016 19:20:52 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2A69B1B24 for ; Sun, 3 Jan 2016 19:20:51 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from freddy.simplesystems.org (freddy.simplesystems.org [65.66.246.65]) by blade.simplesystems.org (8.14.4+Sun/8.14.4) with ESMTP id u03JKcLb023682; Sun, 3 Jan 2016 13:20:38 -0600 (CST) Date: Sun, 3 Jan 2016 13:20:38 -0600 (CST) From: Bob Friesenhahn X-X-Sender: bfriesen@freddy.simplesystems.org To: "Mikhail T." cc: freebsd-fs@freebsd.org Subject: Re: NFS reads vs. writes In-Reply-To: <56895E2E.8060405@aldan.algebra.com> Message-ID: References: <568880D3.3010402@aldan.algebra.com> <56895E2E.8060405@aldan.algebra.com> User-Agent: Alpine 2.01 (GSO 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (blade.simplesystems.org [65.66.246.90]); Sun, 03 Jan 2016 13:20:38 -0600 (CST) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 19:20:52 -0000 On Sun, 3 Jan 2016, Mikhail T. wrote: > > Why would the same ZFS -- with the same slog -- be working faster, when written to locally, than when over NFS? Yes (it depends). Normal local writes are async writes so they do not use the slog at all. NFS writes are usually synchronous writes so they hit the slog hard. You would need software which intentionally issues synchronous writes to local disks in order to test synchronous local writes. Write sizes of 128k and above are written directly to pool disks and don't go to the slog, even if they are sync writes. This is done to make sure that the slog does not limit pool performance due to bandwidth limitations (vs IOPS). Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From owner-freebsd-fs@freebsd.org Sun Jan 3 21:00:10 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E28E4A5FBFA for ; Sun, 3 Jan 2016 21:00:10 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C0182181A for ; Sun, 3 Jan 2016 21:00:10 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u03L01h4078656 for ; Sun, 3 Jan 2016 21:00:10 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201601032100.u03L01h4078656@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-fs@FreeBSD.org Subject: Problem reports for freebsd-fs@FreeBSD.org that need special attention Date: Sun, 03 Jan 2016 21:00:10 +0000 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 21:00:11 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 203492 | mount_unionfs -o below causes panic Open | 136470 | [nfs] Cannot mount / in read-only, over NFS Open | 139651 | [nfs] mount(8): read-only remount of NFS volume d Open | 144447 | [zfs] sharenfs fsunshare() & fsshare_main() non f 4 problems total for which you should take action. From owner-freebsd-fs@freebsd.org Sun Jan 3 21:08:38 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B9E30A6027E for ; Sun, 3 Jan 2016 21:08:38 +0000 (UTC) (envelope-from feld@FreeBSD.org) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8B6E51003 for ; Sun, 3 Jan 2016 21:08:38 +0000 (UTC) (envelope-from feld@FreeBSD.org) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id 74CA420493 for ; Sun, 3 Jan 2016 16:08:37 -0500 (EST) Received: from web3 ([10.202.2.213]) by compute3.internal (MEProxy); Sun, 03 Jan 2016 16:08:37 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-sasl-enc:x-sasl-enc; s=smtpout; bh=omzDUqN1AzlbHUg ZOijNkC1r0uM=; b=H+SqUA1ZaHdLLDk7dTEq/rzVnlXkmYzTAuqhN4FqZ6HmB22 fLTyEtdS4NY2Tv7QltMAS9O+Ueu4oMjyoazTGFcTjTNZt7y09crPCYU5ov9gGZYx Vjy8YHlZx137XW0aMccfGVeBKVSeQBZuuLhYuKSAzKvL40JxfcNXumiQCDb4= Received: by web3.nyi.internal (Postfix, from userid 99) id 4015A105C30; Sun, 3 Jan 2016 16:08:37 -0500 (EST) Message-Id: <1451855317.3739776.481776018.64F6C4D9@webmail.messagingengine.com> X-Sasl-Enc: AFcM3jcWGpOnnhEHU5pi0eRQXSZUMSQTk+FsI9Kr7C5h 1451855317 From: Mark Felder To: Chris Stankevitz , FreeBSD Filesystems MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Mailer: MessagingEngine.com Webmail Interface - ajax-42cc5299 Subject: Re: Monitoring FS changes Date: Sun, 03 Jan 2016 15:08:37 -0600 In-Reply-To: <5684D810.6070700@stankevitz.com> References: <5684D810.6070700@stankevitz.com> X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 21:08:38 -0000 On Thu, Dec 31, 2015, at 01:24, Chris Stankevitz wrote: > Hi, > > I have a directory /foo that recursively contain ~250,000 > files/directories. > > I would like my application to know when a file is added, removed, or > modified under /foo. Is there a way to do that with FreeBSD? > > I believe on linux a facility called iNotify accomplishes this. > > On OSX a facility called FSEvents accomplishes this. > > kqueue apparently requires me to open every file and/or directory in my > tree... which won't work because I have so many. > > Is there any other option? Perhaps > > i=0 > while (true) > { > zfs snapshot pool/foo@${i} > zfs diff pool/foo@${i-1} pool/foo@${i} > ++i > } > Yes, Linux has inotify (just be aware it doesn't actually work on inodes like it indicates: changes to alternative hard links are ignored if they're not in the file path you're monitoring), OSX has fsevents, Solaris derivatives have File Events Notification, and we're stuck with kqueue which doesn't scale. I'm not aware of anything else being available for us. If someone, anyone out there is capable of bringing us something that does scale it would be greatly appreciated. Lots of nice Linux software uses this, but when they do port to FreeBSD we have to do full filesystem scans. It's such a waste. -- Mark Felder ports-secteam member feld@FreeBSD.org From owner-freebsd-fs@freebsd.org Sun Jan 3 21:36:53 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EBE24A60B6C for ; Sun, 3 Jan 2016 21:36:52 +0000 (UTC) (envelope-from jordanhubbard@icloud.com) Received: from pv33p03im-asmtp002.me.com (pv33p03im-asmtp002.me.com [17.143.180.11]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D1E561453; Sun, 3 Jan 2016 21:36:52 +0000 (UTC) (envelope-from jordanhubbard@icloud.com) Received: from [10.11.111.197] (50-250-239-90-static.hfc.comcastbusiness.net [50.250.239.90]) by pv33p03im-asmtp002.me.com (Oracle Communications Messaging Server 7.0.5.36.0 64bit (built Sep 8 2015)) with ESMTPSA id <0O0E00EJDC10C830@pv33p03im-asmtp002.me.com>; Sun, 03 Jan 2016 21:36:37 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-01-03_14:,, signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 clxscore=1011 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1510270003 definitions=main-1601030412 Content-type: text/plain; charset=utf-8 MIME-version: 1.0 (Mac OS X Mail 9.2 \(3112\)) Subject: Re: Monitoring FS changes From: Jordan Hubbard In-reply-to: <1451855317.3739776.481776018.64F6C4D9@webmail.messagingengine.com> Date: Sun, 03 Jan 2016 13:36:36 -0800 Cc: Chris Stankevitz , FreeBSD Filesystems Content-transfer-encoding: quoted-printable Message-id: <046580B0-BF8E-4A83-B1BB-28C632B1B9B5@icloud.com> References: <5684D810.6070700@stankevitz.com> <1451855317.3739776.481776018.64F6C4D9@webmail.messagingengine.com> To: Mark Felder X-Mailer: Apple Mail (2.3112) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 21:36:53 -0000 > On Jan 3, 2016, at 1:08 PM, Mark Felder wrote: >=20 > If someone, anyone out there is capable of bringing us something that > does scale it would be greatly appreciated. Lots of nice Linux = software > uses this, but when they do port to FreeBSD we have to do full > filesystem scans. It's such a waste. I=E2=80=99ve been pondering this for awhile since a lot of interesting = enterprise features require a working filesystem change notification = mechanism that scales to thousands or even millions of files (how did we = bump into the 32 bit NFS file handle problem at iXsystems? Somebody = tried to share more than 4 billion files over NFS - Enterprise folks do = some crazy s**t!). The big question is less whether it=E2=80=99s possible and more what = kind of mechanism people will find palatable. The OS X FSEvents = mechanism works reasonably well and is used constantly to trigger things = like spotlight search indexing and such, and I was by no means involved = in its creation at Apple so I can only speak peripherally to the = implementation, but it seems like it took a fairly long time for it to = become =E2=80=9Clight weight=E2=80=9D enough to use without the overhead = being punitive. Any similar mechanism in FreeBSD would also have to go = through some evolutionary performance iterations - do people want it = badly enough to invest in it long-term? I don=E2=80=99t know, but I do = know that a long-term investment would be necessary to really make it = work well and provide all of the appropriate APIs for talking to it. I think we can probably all agree that Linux inotify wouldn=E2=80=99t be = worth the trouble. =46rom the wikipedia page: =E2=80=A2 Inotify does not support recursively watching = directories, meaning that a separate inotify watch must be created for = every subdirectory.[4] =E2=80=A2 Inotify does report some but not all events in sysfs = and procfs. =E2=80=A2 Notification via inotify requires the kernel to be = aware of all relevant filesystem events, which is not always possible = for networked filesystems such as NFS where changes made by one client = are not immediately broadcast to other clients. =E2=80=A2 Rename events are not handled directly; i.e., inotify = issues two separate events that must be examined and matched in a = context of potential race conditions. I think the first issue alone is a deal killer. Having to walk the = filesystem tree posting notifications on every [new] directory just to = watch a filesystem in its entirety would be pretty onerous and = failure-prone to boot. By contrast: = https://en.wikipedia.org/wiki/FSEvents This is also not to say that I would expect anything in FreeBSD to be = API-compatible (though the upstream clients would probably grumble at = yet another notification mechanism API to #ifdef into their code), = simply that there are only so many design patterns to follow. A = filesystem change is a filesystem change. Everything beyond that is = just a glorified pub/sub mechanism. Assuming there=E2=80=99s interest, I could potentially see throwing some = engineering effort into this. - Jordan From owner-freebsd-fs@freebsd.org Sun Jan 3 21:47:33 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D20B4A60FCE for ; Sun, 3 Jan 2016 21:47:33 +0000 (UTC) (envelope-from feld@FreeBSD.org) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AA8851A1B for ; Sun, 3 Jan 2016 21:47:33 +0000 (UTC) (envelope-from feld@FreeBSD.org) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 47E8220400 for ; Sun, 3 Jan 2016 16:47:32 -0500 (EST) Received: from web3 ([10.202.2.213]) by compute4.internal (MEProxy); Sun, 03 Jan 2016 16:47:32 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-sasl-enc:x-sasl-enc; s=smtpout; bh=OVNlGRiFSbzvsHI qDVbTXkmn1V0=; b=ayInsFH8bsN1ZU0zqs+LS1LzUoipXbwaVCeg1mu46bQHHoj xhRL5tqsyZwlyOmAs9VuQWnwwbCufcC/q2tXs8A+81xV8pXOUhcIZ0zKfxzslDcw WUlUB1Thks99TUS953v4szOabrCRg3ZVKAjQfHgfdzf52tG3qiMrh/LOEcL4= Received: by web3.nyi.internal (Postfix, from userid 99) id 1C4C2105F46; Sun, 3 Jan 2016 16:47:32 -0500 (EST) Message-Id: <1451857652.3746528.481792578.0312DB48@webmail.messagingengine.com> X-Sasl-Enc: CnYoSefStSy0gwfY6mN9ySVQSAE5dEbPJRml0WfihlDR 1451857652 From: Mark Felder To: Jordan Hubbard Cc: Chris Stankevitz , FreeBSD Filesystems MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" X-Mailer: MessagingEngine.com Webmail Interface - ajax-42cc5299 Subject: Re: Monitoring FS changes Date: Sun, 03 Jan 2016 15:47:32 -0600 In-Reply-To: <046580B0-BF8E-4A83-B1BB-28C632B1B9B5@icloud.com> References: <5684D810.6070700@stankevitz.com> <1451855317.3739776.481776018.64F6C4D9@webmail.messagingengine.com> <046580B0-BF8E-4A83-B1BB-28C632B1B9B5@icloud.com> X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 21:47:33 -0000 On Sun, Jan 3, 2016, at 15:36, Jordan Hubbard wrote: >=20 > I think we can probably all agree that Linux inotify wouldn=E2=80=99t be = worth > the trouble. From the wikipedia page: Just talk to Bryan Cantrill if you want to know why we should avoid inotify at all costs. He had to work on mapping it to FEN on SmartOS and he discovered a world of hurt in the process. They're allegedly stuck with the broken implementation of inotify now because Linus doesn't want KBI breakage. Not to say we couldn't provide a compatibility shim so inotify things can compile on FreeBSD, but it might be wise to have something else that works better. Not sure if we really should reinvent the wheel, but I have zero clue how FSEvents or FEN scale. >=20 > Assuming there=E2=80=99s interest, I could potentially see throwing some > engineering effort into this. >=20 > - Jordan >=20 I would love to see this happen in the near future. It is *the* reason Dropbox hasn't released a FreeBSD-native client last I checked. I know that Plex would use it if it was available. There's a lot of cool things ripe for porting if we only had a mechanism... --=20 Mark Felder ports-secteam member feld@FreeBSD.org From owner-freebsd-fs@freebsd.org Mon Jan 4 01:37:37 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3B93AA60039 for ; Mon, 4 Jan 2016 01:37:37 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id E00DF19F5 for ; Mon, 4 Jan 2016 01:37:36 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:4hdGVRUc/69CWkNCn2z5Uo6IP4TV8LGtZVwlr6E/grcLSJyIuqrYZhOCt8tkgFKBZ4jH8fUM07OQ6PC+HzRYqb+681k8M7V0HycfjssXmwFySOWkMmbcaMDQUiohAc5ZX0Vk9XzoeWJcGcL5ekGA6ibqtW1aJBzzOEJPK/jvHcaK1oLsh770o8WbSj4LrQT+SIs6FA+xowTVu5teqqpZAYF19CH0pGBVcf9d32JiKAHbtR/94sCt4MwrqHwI6LoJvvRNWqTifqk+UacQTHF/azh0t4XXskyJaAqM5nIdVi0q1FAAVw3Erw36Q5HZuy/2v+w70S2VMMfsRPY/XjH0vIlxTxq9siYMNHYc+WrUjsF1xPZBpRuqpBhyxqbJZ46IOf5mfuXWdIVJFiJ6Qs9NWnkZUcuHZIwVAr9EZL4Aog== X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DQAQCIzIlW/61jaINehAxtBohTs3EBDYFkGAqFI0oCgUwUAQEBAQEBAQGBCYItggcBAQEDAQEBASAEJyALBQsCAQgYAgINGQICJwEJJgIECAcEARwEiAYIDq5YkHwBAQEBAQEBAQIBAQEBAQEBGASBAYVVhH+EMAcBAQUCFYMegUkFjjCIVoVAhSSESYduhTGKR4NxAiABAUKCERyBeyA0B4M+AQgXI4EIAQEB X-IronPort-AV: E=Sophos;i="5.20,518,1444708800"; d="scan'208";a="259468336" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 03 Jan 2016 20:37:13 -0500 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 0460715F55D; Sun, 3 Jan 2016 20:37:14 -0500 (EST) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id uVmf4gnFGM5d; Sun, 3 Jan 2016 20:37:13 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 6F9BF15F565; Sun, 3 Jan 2016 20:37:13 -0500 (EST) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 4s9y7xGCsDvv; Sun, 3 Jan 2016 20:37:13 -0500 (EST) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 4404715F55D; Sun, 3 Jan 2016 20:37:13 -0500 (EST) Date: Sun, 3 Jan 2016 20:37:13 -0500 (EST) From: Rick Macklem To: "Mikhail T." Cc: Karli =?utf-8?Q?Sj=C3=B6berg?= , freebsd-fs@FreeBSD.org Message-ID: <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <5688D3C1.90301@aldan.algebra.com> References: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> <5688D3C1.90301@aldan.algebra.com> Subject: Re: NFS reads vs. writes MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF43 (Win)/8.0.9_GA_6191) Thread-Topic: NFS reads vs. writes Thread-Index: wOzouO72fuKBfLd+9lmyD/JPSvSqJA== X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2016 01:37:37 -0000 Mikhail T. wrote: > On 03.01.2016 02:16, Karli Sj=C3=B6berg wrote: > > > > The difference between "mount" and "mount -o async" should tell you if > > you'd benefit from a separate log device in the pool. > > > This is not a ZFS problem. The same filesystem is being read in both > cases. The same data is being read from and written to the same > filesystems. For some reason, it is much faster to read via NFS than to > write to it, however. >=20 This issue isn't new. It showed up when Sun introduced NFS in 1985. NFSv3 did change things a little, by allowing UNSTABLE writes. Here's what an NFSv3 or NFSv4 client does when writing: - Issues some # of UNSTABLE writes. The server need only have these is serv= er RAM before replying NFS_OK. - Then the client does a Commit. At this point the NFS server is required t= o store all the data written in the above writes and related metadata on st= able storage before replying NFS_OK. --> This is where the "sync" vs "async" is a big issue. If you use "sync= =3Ddisabled" (I'm not a ZFS guy, but I think that is what the ZFS option looks lik= es) you *break* the NFS protocol (ie. violate the RFC) and put your data at s= ome risk, but you will typically get better (often much better) write performan= ce. OR You put a ZIL on a dedicated device with fast write performance, so t= he data can go there to satisfy the stable storage requirement. (I know nothi= ng about them, but SSDs have dramatically different write performance, s= o an SSD to be used for a ZIL must be carefully selected to ensure good write = performance.) How many writes are in "some #" is up to the client. For FreeBSD clients, t= he "wcommitsize" mount option can be used to adjust this. Recently the default tuning of thi= s changed significantly, but you didn't mention how recent your system(s) are, so man= ual tuning of it may be useful. (See "man mount_nfs" for more on this.) Also, the NFS server was recently tweaked so that it could handle 128K rsiz= e/wsize, but the FreeBSD client is limited to MAXBSIZE and this has not been increas= ed beyond 64K. To do so, you have to change the value of this in the kernel so= urces and rebuild your kernel. (The problem is that increasing MAXBSIZE makes the= kernel use more KVM for the buffer cache and if a system isn't doing significant c= lient side NFS, this is wasted.) Someday, I should see if MAXBSIZE can be made a TUNABLE, but I haven't done= that. --> As such, unless you use a Linux NFS client, the reads/writes will be 64= K, whereas 128K would work better for ZFS. Some NAS hardware vendors solve this problem by using non-volatile RAM, but= that isn't available in generic hardware. > And finally, just to put the matter to rest, both ZFS-pools already have > a separate zil-device (on an SSD). >=20 If this SSD is dedicated to the ZIL and is one known to have good write per= formance, it should help, but in your case the SSD seems to be the bottleneck. rick > -mi >=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Mon Jan 4 06:35:40 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3465FA5F12F for ; Mon, 4 Jan 2016 06:35:40 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from vms173013pub.verizon.net (vms173013pub.verizon.net [206.46.173.13]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 177C41589 for ; Mon, 4 Jan 2016 06:35:39 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from aldan.narawntapu ([100.1.236.52]) by vms173013.mailsrvcs.net (Oracle Communications Messaging Server 7.0.5.32.0 64bit (built Jul 16 2014)) with ESMTPA id <0O0E00J9PY637U40@vms173013.mailsrvcs.net> for freebsd-fs@FreeBSD.org; Sun, 03 Jan 2016 23:34:56 -0600 (CST) X-CMAE-Score: 0 X-CMAE-Analysis: v=2.1 cv=btqxfxui c=1 sm=1 tr=0 a=UorMnhrCY2jH/mPejITChw==:117 a=LaogzpLLAAAA:8 a=oR5dmqMzAAAA:8 a=7aQ_Q-yQQ-AA:10 a=r77TgQKjGQsHNAKrUKIA:9 a=x5da3wTigIZacOizYuAA:9 a=QEXdDO2ut3YA:10 a=enqpX9APRVFHdJLyDM4A:9 a=sVClx91riHHoF91-:21 a=_W_S_7VecoQA:10 Subject: Re: NFS reads vs. writes To: Rick Macklem References: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> <5688D3C1.90301@aldan.algebra.com> <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> Cc: =?UTF-8?Q?Karli_Sj=c3=b6berg?= , freebsd-fs@FreeBSD.org From: "Mikhail T." X-Enigmail-Draft-Status: N1110 Message-id: <568A047B.1010000@aldan.algebra.com> Date: Mon, 04 Jan 2016 00:34:51 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-version: 1.0 In-reply-to: <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2016 06:35:40 -0000 On 03.01.2016 20:37, Rick Macklem wrote: > This issue isn't new. It showed up when Sun introduced NFS in 1985. > NFSv3 did change things a little, by allowing UNSTABLE writes. Thank you very much, Rick, for the detailed explanation. > If you use "sync=disabled" > (I'm not a ZFS guy, but I think that is what the ZFS option looks likes) you > *break* the NFS protocol (ie. violate the RFC) and put your data at some risk, > but you will typically get better (often much better) write performance. Yes, indeed. Disabling sync got the writing throughput all the way up to about 86Mb/s... I still don't fully understand, why local writes are able to achieve this speed without async and without being considered dangerous. > Also, the NFS server was recently tweaked so that it could handle 128K rsize/wsize, > but the FreeBSD client is limited to MAXBSIZE and this has not been increased > beyond 64K. I just tried lowering ZFS' recordsize to 64k to match MAXBSIZE, but that didn't help NFS-writing (unless sync is disabled, that is). > If this SSD is dedicated to the ZIL and is one known to have good write performance, > it should help, but in your case the SSD seems to be the bottleneck. It is a chunk of an older SSD, that also houses the OS. But it is usually idle, because executables and libraries are cached in the abundant RAM. I've seen it do 90+Mb/s (sequential)... I just tried removing ZIL from the receiving pool -- to force direct writes -- but it didn't help the case, where the writes go over NFS. However, the local writes -- with reads from NFS -- went from the 56Mb/s I was seeing earlier to 90Mb/s!.. There is got to be a better way to do this -- preferably, some self-tuning smarts... Thanks again. Yours, -mi From owner-freebsd-fs@freebsd.org Mon Jan 4 08:30:20 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1BB8A61CE7 for ; Mon, 4 Jan 2016 08:30:20 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail109.syd.optusnet.com.au (mail109.syd.optusnet.com.au [211.29.132.80]) by mx1.freebsd.org (Postfix) with ESMTP id 70FCC1301 for ; Mon, 4 Jan 2016 08:30:20 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c211-30-166-197.carlnfd1.nsw.optusnet.com.au (c211-30-166-197.carlnfd1.nsw.optusnet.com.au [211.30.166.197]) by mail109.syd.optusnet.com.au (Postfix) with ESMTPS id AAE06D67F09; Mon, 4 Jan 2016 19:30:09 +1100 (AEDT) Date: Mon, 4 Jan 2016 19:30:08 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Rick Macklem cc: "Mikhail T." , freebsd-fs@freebsd.org Subject: Re: NFS reads vs. writes In-Reply-To: <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> Message-ID: <20160104181759.E1028@besplex.bde.org> References: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> <5688D3C1.90301@aldan.algebra.com> <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> MIME-Version: 1.0 X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=R4L+YolX c=1 sm=1 tr=0 a=KA6XNC2GZCFrdESI5ZmdjQ==:117 a=PO7r1zJSAAAA:8 a=JzwRw_2MAAAA:8 a=nlC_4_pT8q9DhB4Ho9EA:9 a=jQRj7DYTnsFuIglMeNMA:9 a=45ClL6m2LaAA:10 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2016 08:30:20 -0000 On Sun, 3 Jan 2016, Rick Macklem wrote: > Mikhail T. wrote: >> On 03.01.2016 02:16, Karli Sj=C3=B6berg wrote: >>> >>> The difference between "mount" and "mount -o async" should tell you if >>> you'd benefit from a separate log device in the pool. >>> >> This is not a ZFS problem. The same filesystem is being read in both >> cases. The same data is being read from and written to the same >> filesystems. For some reason, it is much faster to read via NFS than to >> write to it, however. >> > This issue isn't new. It showed up when Sun introduced NFS in 1985. nfs writes are slightly faster than reads in most configurations for me. This is because writes are easier to stream and most or all configurations don't do a very good just of trying to stream reads. > NFSv3 did change things a little, by allowing UNSTABLE writes. Of course I use async mounts (and ffs) if I want writes to be fast. Both the server and the client fs should be mounted async. This is most importa= nt for the client. > Here's what an NFSv3 or NFSv4 client does when writing: nfs also has a badly designed sysctl vfs.nfsd.async which does something more hackish for nfsv2 and might have undesirable side effects for nfsv3+. Part of its bad design is that it is global. It affects all clients. This might be a feature if the clients don't support async mounts. I never use this. > - Issues some # of UNSTABLE writes. The server need only have these is se= rver > RAM before replying NFS_OK. > - Then the client does a Commit. At this point the NFS server is required= to > store all the data written in the above writes and related metadata on s= table > storage before replying NFS_OK. async mounts in the FreeBSD client are implemented by 2 lines of code (and "async" in the list of supported options) that seem to work by pretending that UNSTABLE writes are FILESYNC so the Commit step is null. Thus everything except possibly metadata is async and unstable but the client doesn't know this. If the server fs is mounted with inconsistent async flags or the async flags give inconsistent policies, some async writes may turn into sync and vice versa. The worst inconsistencies are with a default (delayed Commit) client and an async (non-soft updates) server. Then async breaks the Commits by writing sync data but still writing async metadata. My version has partial fixes (it syncs inodes but not directories in fsync() for async mounts). > --> This is where the "sync" vs "async" is a big issue. If you use "sync= =3Ddisabled" > (I'm not a ZFS guy, but I think that is what the ZFS option looks li= kes) you > *break* the NFS protocol (ie. violate the RFC) and put your data at = some risk, > but you will typically get better (often much better) write performa= nce. Is zfs really as broken as ffs with async mounts? It takes ignoring FSYNC/ IO_SYNC flags when mounted async to get full brokenness. async for ffs was originally a hack to do something like that. I think it now honors the sync flags for everything except inodes and directories. Sync everything is too slow to use for everything, but the delayed Commit should make it usable, depending on how long the delay is. Perhaps it can interract badly with the server fs's delays. Something like a pipeline stall on a CPU -- to satisfy a synchronization request for 1 file, it might be necessary to wait for many MB of i/o for other files first. > Also, the NFS server was recently tweaked so that it could handle 128K rs= ize/wsize, > but the FreeBSD client is limited to MAXBSIZE and this has not been incre= ased > beyond 64K. To do so, you have to change the value of this in the kernel = sources Larger i/o sizes give negative benefits for me. Changes in the default siz= es give confusing peformance differences with larger sizes mostly worse, but there are too many combinations to test and I never figured out the details= , so I now force small sizes at mount time. This depends on having a fast network. With a really slow network, the i/o sizes must be very large or the streaming must be good. > and rebuild your kernel. (The problem is that increasing MAXBSIZE makes t= he kernel > use more KVM for the buffer cache and if a system isn't doing significant= client > side NFS, this is wasted.) > Someday, I should see if MAXBSIZE can be made a TUNABLE, but I haven't do= ne that. > --> As such, unless you use a Linux NFS client, the reads/writes will be = 64K, whereas > 128K would work better for ZFS. Not for ffs with 16K-blocks. Clustering usually turns these into 128K-bloc= ks but nfs client see little difference and may even work better with 8K-block= s. Bruce From owner-freebsd-fs@freebsd.org Mon Jan 4 09:02:11 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9E565A60D91 for ; Mon, 4 Jan 2016 09:02:11 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail108.syd.optusnet.com.au (mail108.syd.optusnet.com.au [211.29.132.59]) by mx1.freebsd.org (Postfix) with ESMTP id 68A731757 for ; Mon, 4 Jan 2016 09:02:11 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c211-30-166-197.carlnfd1.nsw.optusnet.com.au (c211-30-166-197.carlnfd1.nsw.optusnet.com.au [211.30.166.197]) by mail108.syd.optusnet.com.au (Postfix) with ESMTPS id B970A1A5E6C; Mon, 4 Jan 2016 20:02:02 +1100 (AEDT) Date: Mon, 4 Jan 2016 20:02:02 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: "Mikhail T." cc: Rick Macklem , freebsd-fs@freebsd.org Subject: Re: NFS reads vs. writes In-Reply-To: <568A047B.1010000@aldan.algebra.com> Message-ID: <20160104193054.E1028@besplex.bde.org> References: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> <5688D3C1.90301@aldan.algebra.com> <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> <568A047B.1010000@aldan.algebra.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=PfoC/XVd c=1 sm=1 tr=0 a=KA6XNC2GZCFrdESI5ZmdjQ==:117 a=PO7r1zJSAAAA:8 a=JzwRw_2MAAAA:8 a=kj9zAlcOel0A:10 a=jxJqPt9aaO_jOatjG30A:9 a=CjuIK1q_8ugA:10 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2016 09:02:11 -0000 On Mon, 4 Jan 2016, Mikhail T. wrote: > On 03.01.2016 20:37, Rick Macklem wrote: >> This issue isn't new. It showed up when Sun introduced NFS in 1985. >> NFSv3 did change things a little, by allowing UNSTABLE writes. > Thank you very much, Rick, for the detailed explanation. >> If you use "sync=disabled" >> (I'm not a ZFS guy, but I think that is what the ZFS option looks likes) you >> *break* the NFS protocol (ie. violate the RFC) and put your data at some risk, >> but you will typically get better (often much better) write performance. > Yes, indeed. Disabling sync got the writing throughput all the way up to > about 86Mb/s... I still don't fully understand, why local writes are > able to achieve this speed without async and without being considered > dangerous. 86 Mbits/S is still slow. Do you mean Mbytes/S? Try fsync() to make the local writes slow too. There is considerable confusiion between sync, async and neither. "neither" used to mean to writes using the bawrite() ("async" write) function. "async" means to write _not_ using bawrite(), but using the bdwrite() ("delayed" write) function. Soft updates obfuscate this more. "neither" with them means to write with more order than bawrite() and with less delay than with bdwrite(), so that writes are more robust and also faster than with simple bawrite(). "neither" writes are dangerous in ffs with soft updates only if the system crashes so that the delayed writes are never done. In zfs they are supposed to be safe by writing the delayed writes to small fast storage. I forget what this is named. This is supposed to work without any async hacks too. Apparently it doesn't. Maybe the nfs Commits are too large. Bruce From owner-freebsd-fs@freebsd.org Mon Jan 4 09:02:23 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C0DCCA60DCF for ; Mon, 4 Jan 2016 09:02:23 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 526EF17E7; Mon, 4 Jan 2016 09:02:23 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u0492H5P012128 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 4 Jan 2016 11:02:18 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u0492H5P012128 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u0492HpN012127; Mon, 4 Jan 2016 11:02:17 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 4 Jan 2016 11:02:17 +0200 From: Konstantin Belousov To: Jordan Hubbard Cc: Mark Felder , FreeBSD Filesystems Subject: Re: Monitoring FS changes Message-ID: <20160104090217.GT3625@kib.kiev.ua> References: <5684D810.6070700@stankevitz.com> <1451855317.3739776.481776018.64F6C4D9@webmail.messagingengine.com> <046580B0-BF8E-4A83-B1BB-28C632B1B9B5@icloud.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <046580B0-BF8E-4A83-B1BB-28C632B1B9B5@icloud.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2016 09:02:23 -0000 On Sun, Jan 03, 2016 at 01:36:36PM -0800, Jordan Hubbard wrote: > > > On Jan 3, 2016, at 1:08 PM, Mark Felder wrote: > > > > If someone, anyone out there is capable of bringing us something that > > does scale it would be greatly appreciated. Lots of nice Linux software > > uses this, but when they do port to FreeBSD we have to do full > > filesystem scans. It's such a waste. > > I???ve been pondering this for awhile since a lot of interesting enterprise features require a working filesystem change notification mechanism that scales to thousands or even millions of files (how did we bump into the 32 bit NFS file handle problem at iXsystems? Somebody tried to share more than 4 billion files over NFS - Enterprise folks do some crazy s**t!). > > The big question is less whether it???s possible and more what kind of mechanism people will find palatable. The OS X FSEvents mechanism works reasonably well and is used constantly to trigger things like spotlight search indexing and such, and I was by no means involved in its creation at Apple so I can only speak peripherally to the implementation, but it seems like it took a fairly long time for it to become ???light weight??? enough to use without the overhead being punitive. Any similar mechanism in FreeBSD would also have to go through some evolutionary performance iterations - do people want it badly enough to invest in it long-term? I don???t know, but I do know that a long-term investment would be necessary to really make it work well and provide all of the appropriate APIs for talking to it. > > I think we can probably all agree that Linux inotify wouldn???t be worth the trouble. From the wikipedia page: > > ??? Inotify does not support recursively watching directories, meaning that a separate inotify watch must be created for every subdirectory.[4] > ??? Inotify does report some but not all events in sysfs and procfs. > ??? Notification via inotify requires the kernel to be aware of all relevant filesystem events, which is not always possible for networked filesystems such as NFS where changes made by one client are not immediately broadcast to other clients. > ??? Rename events are not handled directly; i.e., inotify issues two separate events that must be examined and matched in a context of potential race conditions. > > I think the first issue alone is a deal killer. Having to walk the filesystem tree posting notifications on every [new] directory just to watch a filesystem in its entirety would be pretty onerous and failure-prone to boot. By contrast: https://en.wikipedia.org/wiki/FSEvents > > This is also not to say that I would expect anything in FreeBSD to be API-compatible (though the upstream clients would probably grumble at yet another notification mechanism API to #ifdef into their code), simply that there are only so many design patterns to follow. A filesystem change is a filesystem change. Everything beyond that is just a glorified pub/sub mechanism. > > Assuming there???s interest, I could potentially see throwing some engineering effort into this. > There are many people that claim to have very good ideas. This case seems to be an opportunity for such people to contribute something useful to the FreeBSD. I mean, agree upon and provide the precise enough technical specification for the API you want. It does not need to be exact in all details, you can assume a gleam of intelligence in the coders which would implement it, but the spec must be feasible to implement and satisfy the core requirements for consumers. E.g., you need a recursive notification, but there is no way to find all dirents pointing to the given inode, as you noted above. NFS client should not expect to (reliably) get a notification when other client updates a directory and so on. Ideally, this should be a man-page like text and several programs to illustrate the intended use. Programs should be complete, but cannot be tested (for understandable reason). After that, I promise that the spec will be implemented. From owner-freebsd-fs@freebsd.org Mon Jan 4 13:43:30 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DEB82A61946 for ; Mon, 4 Jan 2016 13:43:30 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 38D5E1A96 for ; Mon, 4 Jan 2016 13:43:29 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:NdPZHBbKch+tifd7CoELW3T/LSx+4OfEezUN459isYplN5qZpcu/bnLW6fgltlLVR4KTs6sC0LqI9fi4EUU7or+/81k6OKRWUBEEjchE1ycBO+WiTXPBEfjxciYhF95DXlI2t1uyMExSBdqsLwaK+i760zceF13FOBZvIaytQ8iJ35rxj7j60qaQSjsLrQL1Wal1IhSyoFeZnegtqqwmFJwMzADUqGBDYeVcyDAgD1uSmxHh+pX4p8Y7oGx48sgs/M9YUKj8Y79wDfkBVGxnYCgJ45ihkBjATQKO4jMgFC9exh9JQBTF8RfSV5P9uy28v+5y1SOANIv9SrViChq46KI+ch7ji28iPjU69GzSwphqiatQoxasojRixIHJbYWNNLx1d/WOLpshWWNdU5MJBGR6CYSmYt5KVrJZMA== X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DOAQC0dopW/61jaINehH+IU7N1AQ2BZIYPAoFTFAEBAQEBAQEBgQmCLYIHAQEBAwEjVhACAQgYAgINGQICVwIEiDoIr06RGAEBAQEBAQEDAQEBAQEBARyBAYVVhH+EMA4CgzOBSQWHV4ZZiFaPLY0fhW6EWYNxAiABAUKCERyBeyCDeQEfI4EIAQEB X-IronPort-AV: E=Sophos;i="5.20,520,1444708800"; d="scan'208";a="261049595" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 04 Jan 2016 08:43:01 -0500 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id EB50515F565; Mon, 4 Jan 2016 08:43:01 -0500 (EST) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 46UgQWVcv2NO; Mon, 4 Jan 2016 08:43:01 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 232EC15F56D; Mon, 4 Jan 2016 08:43:01 -0500 (EST) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id lycrEBXUuGyT; Mon, 4 Jan 2016 08:43:01 -0500 (EST) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 053D415F55D; Mon, 4 Jan 2016 08:43:01 -0500 (EST) Date: Mon, 4 Jan 2016 08:43:00 -0500 (EST) From: Rick Macklem To: "Mikhail T." Cc: Karli =?utf-8?Q?Sj=C3=B6berg?= , freebsd-fs@FreeBSD.org Message-ID: <2004192357.147781707.1451914980953.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <568A047B.1010000@aldan.algebra.com> References: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> <5688D3C1.90301@aldan.algebra.com> <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> <568A047B.1010000@aldan.algebra.com> Subject: Re: NFS reads vs. writes MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF43 (Win)/8.0.9_GA_6191) Thread-Topic: NFS reads vs. writes Thread-Index: wK5nCLuI/9YT3Q3m4MrPZq6cLlurZQ== X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2016 13:43:31 -0000 Mikhail T. wrote: > On 03.01.2016 20:37, Rick Macklem wrote: > > This issue isn't new. It showed up when Sun introduced NFS in 1985. > > NFSv3 did change things a little, by allowing UNSTABLE writes. > Thank you very much, Rick, for the detailed explanation. > > If you use "sync=disabled" > > (I'm not a ZFS guy, but I think that is what the ZFS option looks > > likes) you > > *break* the NFS protocol (ie. violate the RFC) and put your data at > > some risk, > > but you will typically get better (often much better) write > > performance. > Yes, indeed. Disabling sync got the writing throughput all the way up to > about 86Mb/s... I still don't fully understand, why local writes are > able to achieve this speed without async and without being considered > dangerous. The risk of data loss when using "sync=disabled" goes like this: - Client does a series of writes, followed by Commit. - Server replies OK to Commit, but hasn't yet actually committed the data to stable storage. - Server crashes/reboots just after sending the Commit reply and before getting the data on stable storage, loosing some of the recently written data. - Client flushes cache when OK reply to Commit received, because it assumes (per RFC) that the data is "safely stored". (It never crashes/reboots.) After this, it might be days/weeks/months before someone notices that the file data is stale/corrupted. (The problem is that the client has no reason to think that the data might be corrupt. It didn't crash/reboot and it doesn't know that the server crashed/rebooted. All it saw was a period of slow response when the server crashed/rebooted. For a local write, the process will be killed by the crash/reboot (or at least it will be known that the machine crashed just as the process was exiting). For POSIX type systems data loss is expected for this case. However, as you can see, the risk of data loss only lasts for a short period of time right after the server replies to a Commit and if you have a reliable server with good UPS etc, the risk may be acceptable. (To me the important part is that people be aware of the risk so that they can make a judgement call.) Btw, a trivial test I did here (don't take this as a benchmark) where I had two partitions on the same drive (one UFS and one a zpool) with everything else the same, I see a write rate for ZFS of about 25% of what I see for UFS. Others with experience using ZFS can maybe chime in w.r.t. how they handle NFS write performance for ZFS? rick ps: For 10.2 and later, what I said w.r.t. MAXBSIZE wasn't accurate. The constants you have to change for 10.2 and later are MAXBCACHESIZE and BKVASIZE. > > Also, the NFS server was recently tweaked so that it could handle 128K > > rsize/wsize, > > but the FreeBSD client is limited to MAXBSIZE and this has not been > > increased > > beyond 64K. > I just tried lowering ZFS' recordsize to 64k to match MAXBSIZE, but that > didn't help NFS-writing (unless sync is disabled, that is). > > If this SSD is dedicated to the ZIL and is one known to have good write > > performance, > > it should help, but in your case the SSD seems to be the bottleneck. > It is a chunk of an older SSD, that also houses the OS. But it is > usually idle, because executables and libraries are cached in the > abundant RAM. I've seen it do 90+Mb/s (sequential)... > > I just tried removing ZIL from the receiving pool -- to force direct > writes -- but it didn't help the case, where the writes go over NFS. > However, the local writes -- with reads from NFS -- went from the 56Mb/s > I was seeing earlier to 90Mb/s!.. > > There is got to be a better way to do this -- preferably, some > self-tuning smarts... Thanks again. Yours, > > -mi > > From owner-freebsd-fs@freebsd.org Mon Jan 4 15:03:41 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C0504A60BF6 for ; Mon, 4 Jan 2016 15:03:41 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 981941C85 for ; Mon, 4 Jan 2016 15:03:41 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (ppp121-45-250-125.lns20.per4.internode.on.net [121.45.250.125]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id u04F3Zb8064021 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Mon, 4 Jan 2016 07:03:39 -0800 (PST) (envelope-from julian@freebsd.org) Subject: Re: Monitoring FS changes To: freebsd-fs@freebsd.org References: <5684D810.6070700@stankevitz.com> <1451855317.3739776.481776018.64F6C4D9@webmail.messagingengine.com> <046580B0-BF8E-4A83-B1BB-28C632B1B9B5@icloud.com> <20160104090217.GT3625@kib.kiev.ua> From: Julian Elischer Message-ID: <568A89C2.3010804@freebsd.org> Date: Mon, 4 Jan 2016 23:03:30 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 In-Reply-To: <20160104090217.GT3625@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2016 15:03:41 -0000 On 4/01/2016 5:02 PM, Konstantin Belousov wrote: > On Sun, Jan 03, 2016 at 01:36:36PM -0800, Jordan Hubbard wrote: >>> On Jan 3, 2016, at 1:08 PM, Mark Felder wrote: >>> >>> If someone, anyone out there is capable of bringing us something that >>> does scale it would be greatly appreciated. Lots of nice Linux software >>> uses this, but when they do port to FreeBSD we have to do full >>> filesystem scans. It's such a waste. >> I???ve been pondering this for awhile since a lot of interesting enterprise features require a working filesystem change notification mechanism that scales to thousands or even millions of files (how did we bump into the 32 bit NFS file handle problem at iXsystems? Somebody tried to share more than 4 billion files over NFS - Enterprise folks do some crazy s**t!). >> >> The big question is less whether it???s possible and more what kind of mechanism people will find palatable. The OS X FSEvents mechanism works reasonably well and is used constantly to trigger things like spotlight search indexing and such, and I was by no means involved in its creation at Apple so I can only speak peripherally to the implementation, but it seems like it took a fairly long time for it to become ???light weight??? enough to use without the overhead being punitive. Any similar mechanism in FreeBSD would also have to go through some evolutionary performance iterations - do people want it badly enough to invest in it long-term? I don???t know, but I do know that a long-term investment would be necessary to really make it work well and provide all of the appropriate APIs for talking to it. >> >> I think we can probably all agree that Linux inotify wouldn???t be worth the trouble. From the wikipedia page: >> >> ??? Inotify does not support recursively watching directories, meaning that a separate inotify watch must be created for every subdirectory.[4] >> ??? Inotify does report some but not all events in sysfs and procfs. >> ??? Notification via inotify requires the kernel to be aware of all relevant filesystem events, which is not always possible for networked filesystems such as NFS where changes made by one client are not immediately broadcast to other clients. >> ??? Rename events are not handled directly; i.e., inotify issues two separate events that must be examined and matched in a context of potential race conditions. >> >> I think the first issue alone is a deal killer. Having to walk the filesystem tree posting notifications on every [new] directory just to watch a filesystem in its entirety would be pretty onerous and failure-prone to boot. By contrast: https://en.wikipedia.org/wiki/FSEvents >> >> This is also not to say that I would expect anything in FreeBSD to be API-compatible (though the upstream clients would probably grumble at yet another notification mechanism API to #ifdef into their code), simply that there are only so many design patterns to follow. A filesystem change is a filesystem change. Everything beyond that is just a glorified pub/sub mechanism. >> >> Assuming there???s interest, I could potentially see throwing some engineering effort into this. >> > There are many people that claim to have very good ideas. This case > seems to be an opportunity for such people to contribute something > useful to the FreeBSD. > > I mean, agree upon and provide the precise enough technical > specification for the API you want. It does not need to be exact in all > details, you can assume a gleam of intelligence in the coders which > would implement it, but the spec must be feasible to implement and > satisfy the core requirements for consumers. > > E.g., you need a recursive notification, but there is no way to find > all dirents pointing to the given inode, as you noted above. NFS client > should not expect to (reliably) get a notification when other client > updates a directory and so on. > > Ideally, this should be a man-page like text and several programs to > illustrate the intended use. Programs should be complete, but cannot > be tested (for understandable reason). > > After that, I promise that the spec will be implemented. I think the point is that setting a notify onto a directory vnode would force all directory vnodes above it (away from root) to stay in memory. That gives a path to traverse in memory to look for notify hooks when a change is made.. This has been discussed many times before. many yearsa ago it came down to resources usage.. I'm not sure that is so important with 128GB machines. (but it needs to handle runaway resource usage). The exact syntax of what is required needs to be spelled out well (as kib says). > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Mon Jan 4 15:13:42 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EFD69A6124E for ; Mon, 4 Jan 2016 15:13:42 +0000 (UTC) (envelope-from artem@artem.ru) Received: from fallback3.mail.ru (fallback3.mail.ru [94.100.181.189]) by mx1.freebsd.org (Postfix) with ESMTP id 9B96A132E for ; Mon, 4 Jan 2016 15:13:42 +0000 (UTC) (envelope-from artem@artem.ru) Received: from smtp33.i.mail.ru (smtp33.i.mail.ru [94.100.177.93]) by fallback3.mail.ru (mPOP.Fallback_MX) with ESMTP id 2D7611686C879 for ; Mon, 4 Jan 2016 18:13:33 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mail.ru; s=mail2; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Date:Message-ID:To:Subject:From; bh=oq4TkuC5AsCyg5sN5Tb0G8mYotp7lDPsjtS29LrpBSE=; b=FnkHQRFmHneAR7iE7/LatNy1GLAbSFAln8E08Vnt2W0QNrzh1SHOrqtIgrGDE1cbMdjo8fzzAI6o63EbNcESrwg8uC6g1KAVe1FQwJIYgsVYTYFy9J2+ofPPJnRSjsNZO2Yj1g8tYou+JHK8Frs1YNbxRILW1BLspxTNGwcq3dQ=; Received: from 79-172-96-94.dyn.broadband.iskratelecom.ru ([79.172.96.94]:54057 helo=[192.168.0.160]) by smtp33.i.mail.ru with esmtpa (envelope-from ) id 1aG6p2-0003pJ-BN for freebsd-fs@freebsd.org; Mon, 04 Jan 2016 18:13:24 +0300 From: Artem Kuchin Subject: SUJ+ async, speed, reliability To: freebsd-fs@freebsd.org Message-ID: <568A8C09.3080902@artem.ru> Date: Mon, 4 Jan 2016 18:13:13 +0300 User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Mras: Ok X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2016 15:13:43 -0000 Hello! For a long time i have run SU+J + mount async. I already do not remember, why i have async. Now i read some article found via google that async is ignored when SoftUpdates is used. Is it true? Is async also ignored when SU+J used? Artem From owner-freebsd-fs@freebsd.org Mon Jan 4 15:33:59 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 038D8A61A16 for ; Mon, 4 Jan 2016 15:33:59 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 77C4210DD; Mon, 4 Jan 2016 15:33:58 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u04FXrxJ012044 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 4 Jan 2016 17:33:53 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u04FXrxJ012044 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u04FXrmv012043; Mon, 4 Jan 2016 17:33:53 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 4 Jan 2016 17:33:53 +0200 From: Konstantin Belousov To: Julian Elischer Cc: freebsd-fs@freebsd.org Subject: Re: Monitoring FS changes Message-ID: <20160104153352.GZ3625@kib.kiev.ua> References: <5684D810.6070700@stankevitz.com> <1451855317.3739776.481776018.64F6C4D9@webmail.messagingengine.com> <046580B0-BF8E-4A83-B1BB-28C632B1B9B5@icloud.com> <20160104090217.GT3625@kib.kiev.ua> <568A89C2.3010804@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <568A89C2.3010804@freebsd.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2016 15:33:59 -0000 On Mon, Jan 04, 2016 at 11:03:30PM +0800, Julian Elischer wrote: > I think the point is that setting a notify onto a directory vnode > would force all directory vnodes above it (away from root) > to stay in memory. That gives a path to traverse in memory to look for > notify hooks when a change is made.. No, it is not. On lookup, you can mark instantiated vnodes, which are children of the monitored vnode, specially. This does not change the existing lifecycle of the vnodes, does not require to make all childs unreclamaible, and allows the caching to work. The tracking might be done with either modest struct vnode grow or even without, by placing several strategic hooks into the vnode lifecycle code. The generic problem we have there is quite different. Assume that we establish a new monitor on a directory, and assume there exists previously open file, which vnode should be now monitored by the 'children' rule. How can we learn that the vnode must be included in the watching set, i.e. marked ? Same issue occurs for fhopen() and for NFS handles. AFAIU, Linux solves it by making the name cache reliable, so you always know if the used name for the vnode is the child of the monitor root. This also explains why they cannot detect hard links. > This has been discussed many times before. many yearsa ago it came > down to resources usage.. I'm not sure that is so important with 128GB > machines. > (but it needs to handle runaway resource usage). The exact syntax of > what is required needs to be spelled out well (as kib says). > As explained above, this is not the issue. Whether the problem I noted is important for the requested API, can be only understood after the API requirements are provided. Even if it is important, might be some significant group of consumers still do not care or can accept such problem. From owner-freebsd-fs@freebsd.org Mon Jan 4 17:19:14 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 94D7CA61480 for ; Mon, 4 Jan 2016 17:19:14 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 734AE12CC for ; Mon, 4 Jan 2016 17:19:14 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (ppp121-45-250-125.lns20.per4.internode.on.net [121.45.250.125]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id u04HJAqU064511 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Mon, 4 Jan 2016 09:19:13 -0800 (PST) (envelope-from julian@freebsd.org) Subject: Re: NFS reads vs. writes To: freebsd-fs@freebsd.org References: <568880D3.3010402@aldan.algebra.com> From: Julian Elischer Message-ID: <568AA988.50903@freebsd.org> Date: Tue, 5 Jan 2016 01:19:04 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 In-Reply-To: <568880D3.3010402@aldan.algebra.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2016 17:19:14 -0000 On 3/01/2016 10:00 AM, Mikhail T. wrote: > I have two similar 10.2-STABLE machines mounting each other's > filesystems via NFS: a:/a and b:/b > > When moving some large files around today, I found a huge discrepancy > between writing and reading over NFS. That is, machine a copying files > from its own /a to NFS-mounted b:/b was pushing measly 2-3Mb/s over the > Ethernet. When I cancelled that, logged-in to machine b and proceed to > copy from a:/a to the now-local /b, I got 56Mb/s. I tried changing the > wsize and rsize options, but it did not seem to make a difference... > > Any ideas, what is happening here? Why are NFS-writes some 25 times > slower here, than reads? Both filesystems are zfs-backed (though with > different options), both systems are plugged into the same switch, both > use mtu of 9000. look at the work from the point of view of each machine: If A is writing to B's filesystem, it needs to wait synchronously for each write to complete before doing the next. (that's what NFS does by default). If B is reading from A's filesystem it can read the next bloc as soon as it has read the previous, and in fact given that there is read-ahead logic at several layers, it's probably not even waiting that long, but queuing up (at one end or the other) requests to disk faster than they can be satisfied. Basically the reads are parallelised and the writes are async in the B case. I see Rick has replied too. > > Thanks! Yours, > > -mi > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Mon Jan 4 20:08:41 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 48216A62257 for ; Mon, 4 Jan 2016 20:08:41 +0000 (UTC) (envelope-from sef@kithrup.com) Received: from kithrup.com (Kithrup.COM [64.142.31.202]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 237BB1DF7 for ; Mon, 4 Jan 2016 20:08:40 +0000 (UTC) (envelope-from sef@kithrup.com) Received: from kithrup.com (localhost [127.0.0.1]) by kithrup.com (8.14.4/8.14.4) with ESMTP id u04JxWSN053361 for ; Mon, 4 Jan 2016 11:59:32 -0800 (PST) (envelope-from sef@kithrup.com) Received: (from sef@localhost) by kithrup.com (8.14.4/8.14.4/Submit) id u04JxWGn053360; Mon, 4 Jan 2016 11:59:32 -0800 (PST) (envelope-from sef) Date: Mon, 4 Jan 2016 11:59:32 -0800 (PST) From: Sean Eric Fagan Message-Id: <201601041959.u04JxWGn053360@kithrup.com> To: freebsd-fs@freebsd.org Subject: Re: Monitoring FS changes X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2016 20:08:41 -0000 >The generic problem we have there is quite different. Assume that >we establish a new monitor on a directory, and assume there exists >previously open file, which vnode should be now monitored by the >'children' rule. How can we learn that the vnode must be included in the >watching set, i.e. marked ? Same issue occurs for fhopen() and for NFS >handles. xnu solved that by putting a parent pointer in each vnode (obviously, not set for non-fs objects). Once they did that, this kept a reference for each vnode, and voila, always there. They also keep a reference cache of names; this makes a lot more sense on a Mac OS system since so many directories and files have the same name (there are 9400 instances of "Info.plist" on my laptop at the moment, for example). The memory footprint for each of these was not too large. But, then, Apple wasn't supporting systems with less than 1gbytes of ram at the time 8-). From owner-freebsd-fs@freebsd.org Mon Jan 4 21:10:43 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9790AA61CF6 for ; Mon, 4 Jan 2016 21:10:43 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 67DFA1154 for ; Mon, 4 Jan 2016 21:10:42 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from freddy.simplesystems.org (freddy.simplesystems.org [65.66.246.65]) by blade.simplesystems.org (8.14.4+Sun/8.14.4) with ESMTP id u04LAM4H015797; Mon, 4 Jan 2016 15:10:23 -0600 (CST) Date: Mon, 4 Jan 2016 15:10:22 -0600 (CST) From: Bob Friesenhahn X-X-Sender: bfriesen@freddy.simplesystems.org To: "Mikhail T." cc: freebsd-fs@freebsd.org Subject: Re: NFS reads vs. writes In-Reply-To: <568A047B.1010000@aldan.algebra.com> Message-ID: References: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> <5688D3C1.90301@aldan.algebra.com> <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> <568A047B.1010000@aldan.algebra.com> User-Agent: Alpine 2.01 (GSO 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (blade.simplesystems.org [65.66.246.90]); Mon, 04 Jan 2016 15:10:23 -0600 (CST) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2016 21:10:43 -0000 On Mon, 4 Jan 2016, Mikhail T. wrote: > Yes, indeed. Disabling sync got the writing throughput all the way up to > about 86Mb/s... I still don't fully understand, why local writes are > able to achieve this speed without async and without being considered > dangerous. Local writes are buffered to RAM and the current set of changes (many writes may have been obviated by overwrites) are written at all once as part of the next ZFS transaction group, which can take up to 5 seconds to occur. Each transaction group completes after all disks have positively acknowledged a cache flush. Using this approach, the on-disk data is coherent but it is possible to lose up to 5 seconds of data (back to the previous commited transaction group). The zfs intent log (slog) remembers the pending synchronous writes (which are still written into RAM!) and marks them as committed when the transaction group completes. If the server loses power or spontaneously reboots, the pending writes from the intent log are written to pool disks when the server comes up. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From owner-freebsd-fs@freebsd.org Mon Jan 4 23:58:17 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DA7CBA613D4 for ; Mon, 4 Jan 2016 23:58:17 +0000 (UTC) (envelope-from thomasrcurry@gmail.com) Received: from mail-io0-x232.google.com (mail-io0-x232.google.com [IPv6:2607:f8b0:4001:c06::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AA64B1BDA for ; Mon, 4 Jan 2016 23:58:17 +0000 (UTC) (envelope-from thomasrcurry@gmail.com) Received: by mail-io0-x232.google.com with SMTP id q21so170650354iod.0 for ; Mon, 04 Jan 2016 15:58:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=2YMVRdpFH4dgLS76FSii0aKhpfdcu40xh/ux0otdyLE=; b=xl8ImqX9Z7JhqKUJDUwJHhT/aQe8hWqVBOVFk48Bim8Ryae9Kb+Wz6LGq0+KjIMdTj pKCXNjocOlUWgbBlr5cHkVLL2FeBoR9/OQNCTQKCeR00AeMg3qsTAzIaHFi8wFrkLgxJ YxwDclHU9tT5D4ZoYb13GwdfZtyVy5FiMIBHxwLN8qiOXmllG4ZKvXsb3ceL764wiMr6 EAf7F07RUFMghYGYgOaYmjdUHNeA7ASpc9KIxQwtufFv1WruByQncBJtUEdCjEKmo+Wx xYmv/0TS4vgOfohe8hta2b9UKk69fbkvvoH96xoUhrhSZ5TZJ+8Ty5lSKxLqQk66SO0T W0bg== MIME-Version: 1.0 X-Received: by 10.107.3.154 with SMTP id e26mr76692123ioi.99.1451951897049; Mon, 04 Jan 2016 15:58:17 -0800 (PST) Received: by 10.107.4.71 with HTTP; Mon, 4 Jan 2016 15:58:16 -0800 (PST) In-Reply-To: <568A047B.1010000@aldan.algebra.com> References: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> <5688D3C1.90301@aldan.algebra.com> <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> <568A047B.1010000@aldan.algebra.com> Date: Mon, 4 Jan 2016 18:58:16 -0500 Message-ID: Subject: Re: NFS reads vs. writes From: Tom Curry To: "Mikhail T." Cc: Rick Macklem , freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2016 23:58:17 -0000 On Mon, Jan 4, 2016 at 12:34 AM, Mikhail T. wrote: > On 03.01.2016 20:37, Rick Macklem wrote: > > This issue isn't new. It showed up when Sun introduced NFS in 1985. > > NFSv3 did change things a little, by allowing UNSTABLE writes. > Thank you very much, Rick, for the detailed explanation. > > If you use "sync=disabled" > > (I'm not a ZFS guy, but I think that is what the ZFS option looks > likes) you > > *break* the NFS protocol (ie. violate the RFC) and put your data > at some risk, > > but you will typically get better (often much better) write > performance. > Yes, indeed. Disabling sync got the writing throughput all the way up to > about 86Mb/s... I still don't fully understand, why local writes are > able to achieve this speed without async and without being considered > dangerous. > > Also, the NFS server was recently tweaked so that it could handle 128K > rsize/wsize, > > but the FreeBSD client is limited to MAXBSIZE and this has not been > increased > > beyond 64K. > I just tried lowering ZFS' recordsize to 64k to match MAXBSIZE, but that > didn't help NFS-writing (unless sync is disabled, that is). > > If this SSD is dedicated to the ZIL and is one known to have good write > performance, > > it should help, but in your case the SSD seems to be the bottleneck. > It is a chunk of an older SSD, that also houses the OS. But it is > usually idle, because executables and libraries are cached in the > abundant RAM. I've seen it do 90+Mb/s (sequential)... > > I just tried removing ZIL from the receiving pool -- to force direct > writes -- but it didn't help the case, where the writes go over NFS. > I assume you mean you removed the SLOG from the pool, in which case you most definitely still have a ZIL, its now located on the pool itself. Assuming you still have sync=standard I would venture a guess that writes over NFS would now be measured in KB/s. > However, the local writes -- with reads from NFS -- went from the 56Mb/s > I was seeing earlier to 90Mb/s!.. > > There is got to be a better way to do this -- preferably, some > self-tuning smarts... Thanks again. Yours, > > There is no getting around the performance impact of a synchronous operation, whether its NFS or a database log. If you don't believe me hop on your favorite Windows box, bring up the device manager and disable the write cache on its drive then run some benchmark supporting sync writes. One way to lessen the performance impact is to decrease the latency of writes, which is why SSD SLOGs help so much. Which brings me to my next point.. SSDs are so fast for three main reasons: low latency, large dram buffers, and parallel workloads. Only one of these is of any benefit (latency) as a SLOG. Unfortunately that particular metric is not usually advertised in consumer SSDs where the benchmarks they use to tout 90,000 random write iops consist of massively concurrent, highly compressible, short lived bursts of data. Add that drive as a SLOG and the onboard dram may as well not even exist, and queue depths count for nothing. It will be lucky to pull 2,000 IOPS. Once you start adding in ZFS features like checksums and compression, or network latency in the case of NFS that 2,000 number starts to drop even more. If you fancy a deeper understanding, this excellent article will take you as far down the rabbit hole as you wish to go. It's very informative. http://dtrace.org/blogs/brendan/2009/06/26/slog-screenshots/ > -mi > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Tue Jan 5 02:22:37 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0B4A6A61FAD for ; Tue, 5 Jan 2016 02:22:37 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id C2CB31C54 for ; Tue, 5 Jan 2016 02:22:36 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:K42XgxL40hDSkdSOWdmcpTZWNBhigK39O0sv0rFitYgULPTxwZ3uMQTl6Ol3ixeRBMOAu6wC07KempujcFJDyK7JiGoFfp1IWk1NouQttCtkPvS4D1bmJuXhdS0wEZcKflZk+3amLRodQ56mNBXsq3G/pQQfBg/4fVIsYL+lRMiK14ye7KObxd76W01wnj2zYLd/fl2djD76kY0ou7ZkMbs70RDTo3FFKKx8zGJsIk+PzV6nvp/jtLYqySlbuuog+shcSu26Ov1gFf0LRAghZls47cujmAPCRkOh73QVVC1CiRdGKyvE8BHgQ4+3mQzf4LlTwi6faPf3RrN8fD2p7KNmTVe8kiIOPD09/WT/l8t/ka9fuBLnrBUpkN2cW52cKPcrJvCVRtgdX2cUG58JDyE= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CvBACMKItW/61jaINehAxtBohTtWwYCoUjSgKBWBEBAQEBAQEBAYEJgi2CBwEBAQMBAQEBICsgCwULAgEIGAICDRkCAicBCSYCBAgHBAEcBIgGCA6wNJB8AQEBAQEBAQEBAQEBAQEBAQEBAQEVBIEBhVWEf4Q3AQGDOoFJBY05d4hWhUGFJIY8iyuOPAI4LIIRHIF7IDQHg0c6gQgBAQE X-IronPort-AV: E=Sophos;i="5.20,523,1444708800"; d="scan'208";a="259613851" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 04 Jan 2016 21:22:19 -0500 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 05D9D15F55D; Mon, 4 Jan 2016 21:22:20 -0500 (EST) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id OgTprwzb2rcG; Mon, 4 Jan 2016 21:22:19 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 8845815F565; Mon, 4 Jan 2016 21:22:19 -0500 (EST) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 9PjiFSH2C-Em; Mon, 4 Jan 2016 21:22:19 -0500 (EST) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 6EDC915F55D; Mon, 4 Jan 2016 21:22:19 -0500 (EST) Date: Mon, 4 Jan 2016 21:22:19 -0500 (EST) From: Rick Macklem To: Sean Eric Fagan Cc: freebsd-fs@freebsd.org Message-ID: <416747176.149128068.1451960539354.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <201601041959.u04JxWGn053360@kithrup.com> References: <201601041959.u04JxWGn053360@kithrup.com> Subject: Re: Monitoring FS changes MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.12] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF43 (Win)/8.0.9_GA_6191) Thread-Topic: Monitoring FS changes Thread-Index: tlJhMqxLAd4sv4+tuocU2sb3hHS3DA== X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2016 02:22:37 -0000 Sean Eric Fagan wrote: > >The generic problem we have there is quite different. Assume that > >we establish a new monitor on a directory, and assume there exists > >previously open file, which vnode should be now monitored by the > >'children' rule. How can we learn that the vnode must be included in the > >watching set, i.e. marked ? Same issue occurs for fhopen() and for NFS > >handles. > > xnu solved that by putting a parent pointer in each vnode (obviously, not set > for non-fs objects). Once they did that, this kept a reference for each > vnode, and voila, always there. > Just wondering how they handle the case of multiple hard links in different directories? rick > They also keep a reference cache of names; this makes a lot more sense on a > Mac OS system since so many directories and files have the same name > (there are 9400 instances of "Info.plist" on my laptop at the moment, for > example). > > The memory footprint for each of these was not too large. But, then, Apple > wasn't supporting systems with less than 1gbytes of ram at the time 8-). > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Tue Jan 5 02:24:05 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8342BA62026 for ; Tue, 5 Jan 2016 02:24:05 +0000 (UTC) (envelope-from sef@kithrup.com) Received: from kithrup.com (Kithrup.COM [64.142.31.202]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48F4E1CE8 for ; Tue, 5 Jan 2016 02:24:04 +0000 (UTC) (envelope-from sef@kithrup.com) Received: from kithrup.com (localhost [127.0.0.1]) by kithrup.com (8.14.4/8.14.4) with ESMTP id u052O349097374; Mon, 4 Jan 2016 18:24:03 -0800 (PST) (envelope-from sef@kithrup.com) Received: (from sef@localhost) by kithrup.com (8.14.4/8.14.4/Submit) id u052O3jo097373; Mon, 4 Jan 2016 18:24:03 -0800 (PST) (envelope-from sef) Date: Mon, 4 Jan 2016 18:24:03 -0800 (PST) From: Sean Eric Fagan Message-Id: <201601050224.u052O3jo097373@kithrup.com> To: rmacklem@uoguelph.ca Subject: Re: Monitoring FS changes Cc: freebsd-fs@freebsd.org In-Reply-To: <416747176.149128068.1451960539354.JavaMail.zimbra@uoguelph.ca> X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2016 02:24:05 -0000 >Just wondering how they handle the case of multiple hard links in different directories? You get back _a_ name, not necessarily _the_ name. And I believe (although I'd have to check the code) an error if the file is open-unlinked. Although then xnu has support for getting the next hard link (this is pretty HFS+ specific, mind you). From owner-freebsd-fs@freebsd.org Tue Jan 5 03:06:27 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 811D9A62A4A for ; Tue, 5 Jan 2016 03:06:27 +0000 (UTC) (envelope-from paul@kraus-haus.org) Received: from mail-qg0-x22d.google.com (mail-qg0-x22d.google.com [IPv6:2607:f8b0:400d:c04::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 42EFB1DFB for ; Tue, 5 Jan 2016 03:06:27 +0000 (UTC) (envelope-from paul@kraus-haus.org) Received: by mail-qg0-x22d.google.com with SMTP id e32so179272711qgf.3 for ; Mon, 04 Jan 2016 19:06:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kraus-haus-org.20150623.gappssmtp.com; s=20150623; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=ZyIEq6AwB3xpwkIFMLlixgpv9YjrblUYiA6g09QxuWs=; b=BRhmMZn/FAesDAgZoFvADkK5Jvmk0QTfX0+vxqychGGtNJFOXXWSWOKAkpAVaVw/0y Ctp79G5X+AdQZmb8gr1WMI6cyzWseWy0qFNWIx6p/6QwKVB4ljDZW+h26UG/B0zXFkjU eM2bzCj9D9Ok91IfbnfNRj8FaUt+42+wzOJh/sqzNsFvBb0EYKHOdQphTmtyVWNKYbJh N+6JiDJMvELGwhlVTB7w2hMpU/nM/u/SHrPPw0XlKJ9SlhxnQ5JDJOtxpkak40bZhK0o maLyoUy7CwA5whUqDWZJQgk1S2U4niB6pzVplCA8R+m/tXUtfieFFEQyCLwVwwybxDks c7og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:mime-version:content-type:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=ZyIEq6AwB3xpwkIFMLlixgpv9YjrblUYiA6g09QxuWs=; b=VEoSEyRlRqis2bIZQkCmHIsdq83lPsK9dRmw+f2qfkaiU6HsSVqQBhvAE2YmP8En3Z WWAOvZcyNEyY503RT1Eru8kU5S4wm1+LHP/YVbmv1vqL6TlieBnC0OpVPNA0ASUKLrRj 7ftrd1AMuFi6dCiX+SSVxhaEGrSvow+3BKE3I7xmZLBpQfdQAG9imN7o392QmOdF7EiQ rbxn0/jubdu6EOG+sc55UDHmmsqyOJYlBfFXG/HamR+ZUcNvxZg8Ws6YyTbzWyzWmsLQ K8OXfIoLOSCmwIaDQX9M8ZEP9a0mI4+XA/xPh3CP9+xdWoBc/wiQbC/cajUV95kaoFhW nz6w== X-Gm-Message-State: ALoCoQkaImwo13V1wyIcK1hIViXY5zhY3OxiGbafmTbATXiyuoR/2eHFsysEPoAoOCBlb0Ve5/pe8A27tCMjLr5ykag4Fy8Tog== X-Received: by 10.140.102.11 with SMTP id v11mr114679256qge.39.1451963186102; Mon, 04 Jan 2016 19:06:26 -0800 (PST) Received: from [192.168.2.138] (pool-100-4-199-226.albyny.fios.verizon.net. [100.4.199.226]) by smtp.gmail.com with ESMTPSA id z65sm41026343qhc.27.2016.01.04.19.06.24 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 04 Jan 2016 19:06:25 -0800 (PST) Subject: Re: NFS reads vs. writes Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Content-Type: text/plain; charset=windows-1252 From: Paul Kraus In-Reply-To: Date: Mon, 4 Jan 2016 22:06:22 -0500 Cc: "Mikhail T." , Tom Curry Content-Transfer-Encoding: quoted-printable Message-Id: References: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> <5688D3C1.90301@aldan.algebra.com> <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> <568A047B.1010000@aldan.algebra.com> To: FreeBSD Filesystems X-Mailer: Apple Mail (2.1878.6) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2016 03:06:27 -0000 On Jan 4, 2016, at 18:58, Tom Curry wrote: > SSDs are so fast for three main reasons: low latency, large dram = buffers, > and parallel workloads. Only one of these is of any benefit (latency) = as a > SLOG. Unfortunately that particular metric is not usually advertised = in > consumer SSDs where the benchmarks they use to tout 90,000 random = write > iops consist of massively concurrent, highly compressible, short lived > bursts of data. Add that drive as a SLOG and the onboard dram may as = well > not even exist, and queue depths count for nothing. It will be lucky = to > pull 2,000 IOPS. Once you start adding in ZFS features like checksums = and > compression, or network latency in the case of NFS that 2,000 number = starts > to drop even more. I have a file server that I am going through the task of optimizing for = NFS traffic (to store VM images). My first attempt, because I knew about = the need for an SSD based SLOG for the ZIL was using a pair of Intel 535 = series SSD=92s. The performance with the SLOG/ZIL on the SSD was = _worse_. Turns out that those SSD=92s have poor small block (8 KB) = random write performance (not well advertised). So I asked for advice = for choosing a _fast_ SSD on the OpenZFS list and had a number of people = recommend the Intel DC-Sxxxx series of SSDs. Based on the very thorough data sheets, I am going with a pair of = DC-S3710 200 GB SSDs. Once I get them in and configured I=92ll post = results. Note that my zpool consists of 5 top level vdevs each made up of a 3-way = mirror. So I am striping writes across 5 columns. I am using 500 GB WD = RE series drives leaving the ZIL on the primary vdevs was _faster_ than = adding the consumer SSD as SLOG for NFS writes. -- Paul Kraus paul@kraus-haus.org From owner-freebsd-fs@freebsd.org Tue Jan 5 03:40:23 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 991FDA616C6 for ; Tue, 5 Jan 2016 03:40:23 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: from mail-wm0-x22b.google.com (mail-wm0-x22b.google.com [IPv6:2a00:1450:400c:c09::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 36F001E0E for ; Tue, 5 Jan 2016 03:40:23 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: by mail-wm0-x22b.google.com with SMTP id f206so8022830wmf.0 for ; Mon, 04 Jan 2016 19:40:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=x4PPxEAwYSAQPQIf5KWB/4KJZNyYruMcWQoT/LLn9cM=; b=0gC9Dez+qO6394XyIlrtOaCslZwQPbpxU5zJOtewa2tYLWIs3rfUfkNDcurfqT5ZBW nzbeEXt/lKTAgdFNbtOvbi8mtqujU3O1/WkKcy0VhoLXdDFAzkE1FNegX+OePq7eko9D 18LgtWGDbRlTnsKktvdInI1EXLFuIWhWQnDhZKOSsstPSd6tZkX0TOiJrVgzzUgJgAgp 0VduffYvl+og+V3ffPelydcsgzzk7YhTsgyshMuyP//AoJFxeaU5/pgLb4qQAFBYG+lw aNRJj6mWOorBC2x9oc52LG47FPG4PKfVZOItHcPf0P81Mnanr3kDDi5HlnZfhKxroxVQ d24g== MIME-Version: 1.0 X-Received: by 10.28.63.200 with SMTP id m191mr1710581wma.67.1451965220653; Mon, 04 Jan 2016 19:40:20 -0800 (PST) Received: by 10.194.192.33 with HTTP; Mon, 4 Jan 2016 19:40:20 -0800 (PST) In-Reply-To: References: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> <5688D3C1.90301@aldan.algebra.com> <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> <568A047B.1010000@aldan.algebra.com> Date: Mon, 4 Jan 2016 21:40:20 -0600 Message-ID: Subject: Re: NFS reads vs. writes From: Adam Vande More To: Paul Kraus Cc: FreeBSD Filesystems , "Mikhail T." Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2016 03:40:23 -0000 On Mon, Jan 4, 2016 at 9:06 PM, Paul Kraus wrote: > I have a file server that I am going through the task of optimizing for > NFS traffic (to store VM images). My first attempt, because I knew about > the need for an SSD based SLOG for the ZIL was using a pair of Intel 535 > series SSD=E2=80=99s. The performance with the SLOG/ZIL on the SSD was _w= orse_. > Turns out that those SSD=E2=80=99s have poor small block (8 KB) random wr= ite > performance (not well advertised). So I asked for advice for choosing a > _fast_ SSD on the OpenZFS list and had a number of people recommend the > Intel DC-Sxxxx series of SSDs. > > Based on the very thorough data sheets, I am going with a pair of DC-S371= 0 > 200 GB SSDs. Once I get them in and configured I=E2=80=99ll post results. > > Note that my zpool consists of 5 top level vdevs each made up of a 3-way > mirror. So I am striping writes across 5 columns. I am using 500 GB WD RE > series drives leaving the ZIL on the primary vdevs was _faster_ than addi= ng > the consumer SSD as SLOG for NFS writes Something like zeusram or ddrdrive is probably going to yield the most preformant solution for zfs sync writes. --=20 Adam From owner-freebsd-fs@freebsd.org Tue Jan 5 03:42:27 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E8A6EA61851 for ; Tue, 5 Jan 2016 03:42:26 +0000 (UTC) (envelope-from paul@kraus-haus.org) Received: from mail-qg0-x233.google.com (mail-qg0-x233.google.com [IPv6:2607:f8b0:400d:c04::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A013D10A9 for ; Tue, 5 Jan 2016 03:42:26 +0000 (UTC) (envelope-from paul@kraus-haus.org) Received: by mail-qg0-x233.google.com with SMTP id 6so191813710qgy.1 for ; Mon, 04 Jan 2016 19:42:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kraus-haus-org.20150623.gappssmtp.com; s=20150623; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=pVLHt0Bbuy6gH9A8uj+KQuITcGRIUmwMjZ6WnlkxYOc=; b=tmtYnHet5o1f2m/KZBoqEC8xqPgORw6SK+3s9Xmnn478MEXPNJvwYKNP25FjLhv5zs ne/Z6V/5uu+MG5Su/UIcO415YlYPk7B/+/dKsdVYgsKuXMa6dn1/Cc/yDwJysl8CshWi lRfECk0MWTAqWeoq5QWjKt4/5ZvZiigThfPjVr/sCVHrcTlfoS03nW1TPJiRYH33laDW mYicIdmQHGbO0tCngfvIKYmmHegUyYz7jXnZN5dL85DE4rLNL631WQzwU4EEKg2ff8YK eUxZW9E/B3IZVDaXqQKJSVJvQpLnjWsOc5LIyTivovMXH7C1VIWea06C+xBlQuhLx7km J7Nw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:mime-version:content-type:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=pVLHt0Bbuy6gH9A8uj+KQuITcGRIUmwMjZ6WnlkxYOc=; b=GIymIw03NxW292ib+58yz7oNBxiROP4jNp3uD1L5mL3wbNgSD28BCRBUqappGzoKI5 SIixhefPjnDJjtFSNJU6Garl3GpSy4zybAwsFEiFWLM4ApaSeL0vbXTYHH/b0T2iigGb jzRqXd7wvJuOh0HrGw3bzYuzct6ygYsudZ640NUjCUppm3J3+snoeM7bNE0xBokREiWY YwC9YlKfwlwMY1nBvAIS8wTrlZRHo8guYf3N9sQNrpP2AsQGqCJxTDwT82qKhR5fgN/h tWVkEvthqdZiaBWOqAktwOdEoQFuWRaAe931I6xB3y8OKrZZkidLRZkAi4UgYpIPjrGQ XrAQ== X-Gm-Message-State: ALoCoQmo3Zaf8FU9awb5/2F/uL4W4a/srUo9ovxYKRW1rSuOb8/2lCuD2KBylYQPwStTlQK1wOISIX8EzUCT9Y6RNcqfNXObWg== X-Received: by 10.141.5.136 with SMTP id h130mr125358014qhd.47.1451965345725; Mon, 04 Jan 2016 19:42:25 -0800 (PST) Received: from [192.168.2.138] (pool-100-4-199-226.albyny.fios.verizon.net. [100.4.199.226]) by smtp.gmail.com with ESMTPSA id m127sm31648480qhm.43.2016.01.04.19.42.24 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 04 Jan 2016 19:42:24 -0800 (PST) Subject: Re: NFS reads vs. writes Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Content-Type: text/plain; charset=windows-1252 From: Paul Kraus In-Reply-To: Date: Mon, 4 Jan 2016 22:42:21 -0500 Cc: FreeBSD Filesystems , "Mikhail T." Content-Transfer-Encoding: quoted-printable Message-Id: <38A28FE6-5EE9-48B0-BC7C-6C02EB2C0700@kraus-haus.org> References: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> <5688D3C1.90301@aldan.algebra.com> <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> <568A047B.1010000@aldan.algebra.com> To: Adam Vande More X-Mailer: Apple Mail (2.1878.6) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2016 03:42:27 -0000 On Jan 4, 2016, at 22:40, Adam Vande More wrote: > On Mon, Jan 4, 2016 at 9:06 PM, Paul Kraus = wrote: >=20 >> I have a file server that I am going through the task of optimizing = for >> NFS traffic (to store VM images). My first attempt, because I knew = about >> the need for an SSD based SLOG for the ZIL was using a pair of Intel = 535 >> series SSD=92s. The performance with the SLOG/ZIL on the SSD was = _worse_. >> Turns out that those SSD=92s have poor small block (8 KB) random = write >> performance (not well advertised). So I asked for advice for choosing = a >> _fast_ SSD on the OpenZFS list and had a number of people recommend = the >> Intel DC-Sxxxx series of SSDs. > Something like zeusram or ddrdrive is probably going to yield the most > preformant solution for zfs sync writes. Agreed, but I don=92t have the budget for that :-( -- Paul Kraus paul@kraus-haus.org From owner-freebsd-fs@freebsd.org Tue Jan 5 03:53:08 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3B7BDA61BB4 for ; Tue, 5 Jan 2016 03:53:08 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (ns1.bitblocks.com [173.228.5.8]) by mx1.freebsd.org (Postfix) with ESMTP id 2716314FF for ; Tue, 5 Jan 2016 03:53:07 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (localhost [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id B650FB827 for ; Mon, 4 Jan 2016 19:43:27 -0800 (PST) To: freebsd-fs@freebsd.org Subject: Re: Monitoring FS changes In-reply-to: Your message of "Mon, 04 Jan 2016 21:22:19 EST." <416747176.149128068.1451960539354.JavaMail.zimbra@uoguelph.ca> References: <201601041959.u04JxWGn053360@kithrup.com> <416747176.149128068.1451960539354.JavaMail.zimbra@uoguelph.ca> Comments: In-reply-to Rick Macklem message dated "Mon, 04 Jan 2016 21:22:19 -0500." Date: Mon, 04 Jan 2016 19:43:27 -0800 From: Bakul Shah Message-Id: <20160105034327.B650FB827@mail.bitblocks.com> X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2016 03:53:08 -0000 Why not do this (at least at first) in a user mode program? Intercept FS system calls and write relevant info to a user program's memory. You still need to add a syscall to watch files/dirs for various events and to validate requests. This will allow you to experiment with various implementations before commiting to a complicated new mechanism in the kernel. Something like: For the client: fd = new_watcher(); watch(fd, path, flags); // can add multiple watches count = read(fd, buf, sizeof buf); flag = 0 => remove watchpoint. flag = recursive => watch everything underneath a tree (on the same fs) other flag decide the events you want to watch. read returns one or more events. For the watcher: fd = register_watcher(buf, length); fd1 = get_watcher(fd, &path, &flag); The watcher mmaps a bunch of space and the kernel uses it as a circular buffer. The watcher can create a map of inode-id to subscribers: a list of or a ptr to a list. The indirection should make recursion cheaper to handle: objects will point to parent dir in a recursive watch. You'd still need to spend something like 8 bytes per file or dir (on a 4GB addr space system). flag = 0 => unsubscribe fd from watching the path. close the fd when the number of watches is 0. On a client disappearing path is NULL. On a watcher disappearing all clients get one final event. Hard links are not a problem: such a file will just have N items on its list, one per watched dir. Multiple watchers, each handling a different set of clients, should be possible with a little extra cost. Basically what we are doing is outsourcing most of the actual work to the watcher but the kernel still have to tell all the watchers when something changes. Very likely the same mechanism can be used to report process deaths or dead network connections etc. Undoubtedly I have missed a bunch of things here! From owner-freebsd-fs@freebsd.org Tue Jan 5 05:19:36 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D0FFFA619B3 for ; Tue, 5 Jan 2016 05:19:36 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail109.syd.optusnet.com.au (mail109.syd.optusnet.com.au [211.29.132.80]) by mx1.freebsd.org (Postfix) with ESMTP id 8177C179C for ; Tue, 5 Jan 2016 05:19:36 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c211-30-166-197.carlnfd1.nsw.optusnet.com.au (c211-30-166-197.carlnfd1.nsw.optusnet.com.au [211.30.166.197]) by mail109.syd.optusnet.com.au (Postfix) with ESMTPS id 5C76FD61F8C; Tue, 5 Jan 2016 16:19:32 +1100 (AEDT) Date: Tue, 5 Jan 2016 16:19:30 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Tom Curry cc: "Mikhail T." , freebsd-fs@freebsd.org Subject: Re: NFS reads vs. writes In-Reply-To: Message-ID: <20160105143542.X1191@besplex.bde.org> References: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> <5688D3C1.90301@aldan.algebra.com> <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> <568A047B.1010000@aldan.algebra.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=cK4dyQqN c=1 sm=1 tr=0 a=KA6XNC2GZCFrdESI5ZmdjQ==:117 a=PO7r1zJSAAAA:8 a=L9H7d07YOLsA:10 a=9cW_t1CCXrUA:10 a=s5jvgZ67dGcA:10 a=JzwRw_2MAAAA:8 a=kj9zAlcOel0A:10 a=LaogzpLLAAAA:8 a=kX-zrYm7HQl9N4iR4CcA:9 a=CjuIK1q_8ugA:10 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2016 05:19:36 -0000 On Mon, 4 Jan 2016, Tom Curry wrote: > On Mon, Jan 4, 2016 at 12:34 AM, Mikhail T. > wrote: > >> On 03.01.2016 20:37, Rick Macklem wrote: >> ... >> I just tried lowering ZFS' recordsize to 64k to match MAXBSIZE, but that >> didn't help NFS-writing (unless sync is disabled, that is). >>> If this SSD is dedicated to the ZIL and is one known to have good write >> performance, >>> it should help, but in your case the SSD seems to be the bottleneck. >> It is a chunk of an older SSD, that also houses the OS. But it is >> usually idle, because executables and libraries are cached in the >> abundant RAM. I've seen it do 90+Mb/s (sequential)... Please be more careful with units (but don't use MiB's; I should killfile that). 90 Mbits/s is still slow. >> I just tried removing ZIL from the receiving pool -- to force direct >> writes -- but it didn't help the case, where the writes go over NFS. > > I assume you mean you removed the SLOG from the pool, in which case you > most definitely still have a ZIL, its now located on the pool itself. > Assuming you still have sync=standard I would venture a guess that writes > over NFS would now be measured in KB/s. > >> However, the local writes -- with reads from NFS -- went from the 56Mb/s >> I was seeing earlier to 90Mb/s!.. 56 to 90 is not a large difference. I think you mentioned factor of 10 differences earlier. >> There is got to be a better way to do this -- preferably, some >> self-tuning smarts... Thanks again. Yours, >> > There is no getting around the performance impact of a synchronous > operation, whether its NFS or a database log. If you don't believe me hop > on your favorite Windows box, bring up the device manager and disable the > write cache on its drive then run some benchmark supporting sync writes. > One way to lessen the performance impact is to decrease the latency of > writes, which is why SSD SLOGs help so much. Which brings me to my next > point.. But nfs doesn't do sync writes. As pointed out earlier in this threads, it does cached writes that is not very different from what other file systems do. It writes up to wcommitsize bytes per file and then commits them. The default value for wcommitsize is undocumented but according to the source code it is sqrt(hibufspace) * 256. This gives about 2.5MB on i386 with 1GB RAM and 17MB on amd64 with 24GB RAM. This is not very large, unless it is actually per-file and there is a backlog of many files with this much uncommitted data -- then it is too large. In most file systems, the corresponding limit is per-fs or per-system. On freefall, vfs.zfs.dirty_data_max is 2.5GB and vfs.hidirtybuffers is 26502. 2.5GB seems too high to me. It would take 25 seconds to drain if it is for a single disk that can do 100MB/s. 26502 is too high. It is 1.6GB with the maximum block size of 64K, and it can easily be for a single disk that is much slower than 100MB/s. I often see buffer cache delays of several seconds for backlogs of just a few MB on a slow (DVD) disk. When nfs commits the data, it has to do a sync write. Since wcommitsize is large, this shouldn't be very slow the file is small so it never reaches anywhere near size wcommitsize. (nfs apparently suffers from the same design errors as the buffer cache. Everthing is per-file or per-vnode, so there is no way to combine reads or writes even if reads are ahead and writes are long delayed. Caching in drives makes this problem not as large as it was 20-30 years ago, but it takes extra i/o's for the small i/o's and some drives haave too low an i/o's for their caching to help much). The implementation might still be fairly stupid and wait for the sync write to complete. This is what seems to happen with ffs for the server fs. With most mistunings, I get about half of the server speed for nfs (25MB/s). The timing with wcommitsize = 25MB might be: accumulate 25MB and send it to the server at line rate. My network can only do about 70MB/sec so this takes 0.35 seconds. Then wait for the server to do a sync write. My server can only do about 47MB/s so this takes 0.53 seconds. Stall writes on the client waiting for the server to confirm the commit. Total time 0.88 seconds or 28MB/s. Observed throughput more like 25MB/s. With everything async, I get 39MB/s today and 44MB/s with slightly diffenty configurations on other days. 2 interesting points turned up or were confirmed in my tests today: - async on the server makes little difference for large files. It was slightly slower if anything. This is because the only i/o that I tested today was a case that I am ususally not interested in -- large writes to a single file. In this case, almost all of the writes are sync for the commit. The possible reasons for async being slightly slower for committing are: - a larger backlog - bugs in vfs clustering -- some of its async conditions seem to be backwards. - when the server is mounted fully sync, writing on the client is faster than on the server, even with the small application buffer size of 512 on the client and a larger but not maximal buffer size on the server! This is because writes on the client are basically cached. They are combined on the server up to a big wcommitsize and done with a big sync write, while on the sever if the application writes 512 at a time it gets sync writes 512 at a time (plus pre-reads of the fs block size at a time, but only 1 of these per multiple 512-writes). It is easy to have a stupider implementation. E.g., when nfs commits, on the server don't give this any priority and get around to it 5-30 seconds later. Or give it some priority but put it behind the local backlog of 2.5GB or so. Give this priority too, but it still takes a long time since it is so large. Don't tell the client about the progress you are making (I think the nfs protocol doesnt have any partially-committed states). Maybe zfs is too smart about caching and it interacts badly with nfs, and ffs interacts better because it is not so smart. (I don't even use ffs with soft updates because they are too smart.) It is not so easy to have a better implementation, though protocols like zmodem and tcp have had one for 30-40 years. Just stream small writes to the server as fast as you can and let it nack them as fast as it prefers to commit them to stable storage (never ack, but negative ack for a problem). Then if you want to commit a file, tell the server to give the blocks for that file priority but don't wait for it to finish before writing more. Give some priority hints to minimize backlogs. Changing wcommitsize between 8K and 200MB for testing with a 128MB file made suprisingly little difference here. > SSDs are so fast for three main reasons: low latency, large dram buffers, > and parallel workloads. Only one of these is of any benefit (latency) as a > SLOG. Unfortunately that particular metric is not usually advertised in > consumer SSDs where the benchmarks they use to tout 90,000 random write > iops consist of massively concurrent, highly compressible, short lived > bursts of data. Add that drive as a SLOG and the onboard dram may as well > not even exist, and queue depths count for nothing. It will be lucky to > pull 2,000 IOPS. Once you start adding in ZFS features like checksums and > compression, or network latency in the case of NFS that 2,000 number starts > to drop even more. Latency seems to be unimportant for a big commit. It is important for lots of smaller commits if the client (kernel or application) needs to wait for just one of them. Bruce From owner-freebsd-fs@freebsd.org Tue Jan 5 05:33:53 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 13F03A61FA5 for ; Tue, 5 Jan 2016 05:33:53 +0000 (UTC) (envelope-from jordanhubbard@icloud.com) Received: from pv33p03im-asmtp002.me.com (pv33p03im-asmtp002.me.com [17.143.180.11]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EDC7112A0 for ; Tue, 5 Jan 2016 05:33:52 +0000 (UTC) (envelope-from jordanhubbard@icloud.com) Received: from [10.11.111.236] (50-250-239-90-static.hfc.comcastbusiness.net [50.250.239.90]) by pv33p03im-asmtp002.me.com (Oracle Communications Messaging Server 7.0.5.36.0 64bit (built Sep 8 2015)) with ESMTPSA id <0O0G001GYSS7EL50@pv33p03im-asmtp002.me.com> for freebsd-fs@freebsd.org; Tue, 05 Jan 2016 05:33:45 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-01-04_11:,, signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 clxscore=1011 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1510270003 definitions=main-1601050106 Content-type: text/plain; charset=utf-8 MIME-version: 1.0 (Mac OS X Mail 9.2 \(3112\)) Subject: Re: Monitoring FS changes From: Jordan Hubbard In-reply-to: <20160105034327.B650FB827@mail.bitblocks.com> Date: Mon, 04 Jan 2016 21:33:43 -0800 Cc: freebsd-fs@freebsd.org Content-transfer-encoding: quoted-printable Message-id: <1CAAFE08-D24E-49AA-8EC5-4397A199DA3E@icloud.com> References: <201601041959.u04JxWGn053360@kithrup.com> <416747176.149128068.1451960539354.JavaMail.zimbra@uoguelph.ca> <20160105034327.B650FB827@mail.bitblocks.com> To: Bakul Shah X-Mailer: Apple Mail (2.3112) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2016 05:33:53 -0000 > On Jan 4, 2016, at 7:43 PM, Bakul Shah wrote: >=20 > Why not do this (at least at first) in a user mode program? > Intercept FS system calls and write relevant info to a user > program's memory. You still need to add a syscall to watch > files/dirs for various events and to validate requests. This > will allow you to experiment with various implementations > before commiting to a complicated new mechanism in the kernel. That=E2=80=99s basically how FSEvents work. There=E2=80=99s a fairly = straight-forward (Mach IPC based) kernel upcall mechanism for = communicating the filesystem change events (and control inputs for what = to watch) to a daemon, fseventsd, and it=E2=80=99s the userland daemon = which subscribers talk to and it figures out how many events to cache, = when all subscribers have received the events (or timed out) and it can = re-use the memory, and so on. The kernel reporting mechanism can be relatively light-weight if you = proxy all the subscription and memory management details through a = userland daemon, which is why I certainly wouldn=E2=80=99t suggest doing = it any other way=E2=80=A6 - Jordan From owner-freebsd-fs@freebsd.org Tue Jan 5 05:40:38 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4E4B8A6233C for ; Tue, 5 Jan 2016 05:40:38 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from vms173015pub.verizon.net (vms173015pub.verizon.net [206.46.173.15]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2694B1767 for ; Tue, 5 Jan 2016 05:40:37 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) MIME-version: 1.0 Received: from aldan.narawntapu ([100.1.236.52]) by vms173015.mailsrvcs.net (Oracle Communications Messaging Server 7.0.5.32.0 64bit (built Jul 16 2014)) with ESMTPA id <0O0G00DDXT3EYP70@vms173015.mailsrvcs.net> for freebsd-fs@freebsd.org; Mon, 04 Jan 2016 23:40:31 -0600 (CST) X-CMAE-Score: 0 X-CMAE-Analysis: v=2.1 cv=btqxfxui c=1 sm=1 tr=0 a=UorMnhrCY2jH/mPejITChw==:117 a=LaogzpLLAAAA:8 a=oR5dmqMzAAAA:8 a=7aQ_Q-yQQ-AA:10 a=r77TgQKjGQsHNAKrUKIA:9 a=A8KfzhJwOMCciS3YhZgA:9 a=pILNOxqGKmIA:10 a=JzwRw_2MAAAA:8 a=-uXMOEYdzANfLp-DCMMA:9 a=Wwoz_WDnPw1EjHLx:21 a=_W_S_7VecoQA:10 Subject: Re: NFS reads vs. writes To: Bruce Evans References: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> <5688D3C1.90301@aldan.algebra.com> <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> <568A047B.1010000@aldan.algebra.com> <20160105143542.X1191@besplex.bde.org> Cc: freebsd-fs From: "Mikhail T." Message-id: <568B574A.7010603@aldan.algebra.com> Date: Tue, 05 Jan 2016 00:40:26 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 In-reply-to: <20160105143542.X1191@besplex.bde.org> Content-Type: text/plain; CHARSET=US-ASCII Content-Transfer-Encoding: 7BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2016 05:40:38 -0000 On 05.01.2016 00:19, Bruce Evans wrote: >>> It is a chunk of an older SSD, that also houses the OS. But it is >>> usually idle, because executables and libraries are cached in the >>> abundant RAM. I've seen it do 90+Mb/s (sequential)... > > Please be more careful with units (but don't use MiB's; I should killfile > that). 90 Mbits/s is still slow. Mb is megabyte in my book. This is the unit used by `systat -vm', which I mentioned earlier. 90 megabytes/s is still not at the limit of Gigabit Ethernet, however, but it is a lot closer to it. Yours, -mi From owner-freebsd-fs@freebsd.org Tue Jan 5 10:49:54 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE374A6181B for ; Tue, 5 Jan 2016 10:49:54 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6253D1322 for ; Tue, 5 Jan 2016 10:49:54 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u05Ann0C099365 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 5 Jan 2016 12:49:49 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u05Ann0C099365 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u05AnmsV099364; Tue, 5 Jan 2016 12:49:48 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 5 Jan 2016 12:49:48 +0200 From: Konstantin Belousov To: Sean Eric Fagan Cc: freebsd-fs@freebsd.org Subject: Re: Monitoring FS changes Message-ID: <20160105104948.GG3625@kib.kiev.ua> References: <201601041959.u04JxWGn053360@kithrup.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201601041959.u04JxWGn053360@kithrup.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2016 10:49:55 -0000 On Mon, Jan 04, 2016 at 11:59:32AM -0800, Sean Eric Fagan wrote: > >The generic problem we have there is quite different. Assume that > >we establish a new monitor on a directory, and assume there exists > >previously open file, which vnode should be now monitored by the > >'children' rule. How can we learn that the vnode must be included in the > >watching set, i.e. marked ? Same issue occurs for fhopen() and for NFS > >handles. > > xnu solved that by putting a parent pointer in each vnode (obviously, not set > for non-fs objects). Once they did that, this kept a reference for each > vnode, and voila, always there. The voila part is somewhat problematic. You must ensure the liveness of the reference to the parent vnode, which immediately raises the question of parent vnode being recycled without updating the children pointers. So you must track all children, to not leave stale parent pointers around. This is because vnodes only represent a cache of the on-disk structures. We do have a machinery to track the parent/children relations, it is called the name cache. But it is a cache and can be dropped at any time. This is why e.g. vn_fullpath() or executable names in /proc or lsof output are lost sometime. Anyway, having great ideas does not make this stuff implemented even a bit closer. If anybody with his/her great ideas does not bother even to formulate the wanted API and explicitely document desired behaviour, the thing will never happen. > > They also keep a reference cache of names; this makes a lot more sense on a > Mac OS system since so many directories and files have the same name > (there are 9400 instances of "Info.plist" on my laptop at the moment, for > example). > > The memory footprint for each of these was not too large. But, then, Apple > wasn't supporting systems with less than 1gbytes of ram at the time 8-). From owner-freebsd-fs@freebsd.org Tue Jan 5 11:08:14 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 53F59A62038 for ; Tue, 5 Jan 2016 11:08:14 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-outbound.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DFC2B1DA2 for ; Tue, 5 Jan 2016 11:08:12 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) by smtp-outbound.userve.net (8.15.1/8.15.1) with ESMTPS id u05AjX70023624 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=FAIL); Tue, 5 Jan 2016 10:45:39 GMT (envelope-from matt.churchyard@userve.net) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1451990746; bh=YI2qt/kZmVQpq19+PEvWY5e4E614LvIK3tzRe+xitZY=; h=From:To:CC:Subject:Date:References:In-Reply-To; b=mdwSA9xH9SPGNuIWiqivZ/j2yMpQZnfKx6wZux/KwBSD/vw3qO4bXmlbYPB7inDXT U2SBzMJopjf6SUSfZ53eb6flbL7gaJe5Zwq1aMu6XLr8jwk6vriFLBAkG2Jx6epS8p PUy/4z5CBDhe4B7YJzFTr5f/DW2VS5N1B3ecBYb0= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Tue, 5 Jan 2016 10:45:28 +0000 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Tue, 5 Jan 2016 10:45:28 +0000 From: Matt Churchyard To: "Mikhail T." , Bruce Evans CC: freebsd-fs Subject: RE: NFS reads vs. writes Thread-Topic: NFS reads vs. writes Thread-Index: AQHRRfabDWUMRFTBukiGQgVGFsctUJ7pbBOAgAEo3oCAAEJlgIABNEsAgABZwACAAAXZAIAAUDUw Date: Tue, 5 Jan 2016 10:45:27 +0000 Message-ID: References: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> <5688D3C1.90301@aldan.algebra.com> <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> <568A047B.1010000@aldan.algebra.com> <20160105143542.X1191@besplex.bde.org> <568B574A.7010603@aldan.algebra.com> In-Reply-To: <568B574A.7010603@aldan.algebra.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2016 11:08:14 -0000 >On 05.01.2016 00:19, Bruce Evans wrote: >>>> It is a chunk of an older SSD, that also houses the OS. But it is=20 >>>> usually idle, because executables and libraries are cached in the=20 >>>> abundant RAM. I've seen it do 90+Mb/s (sequential)... >> >> Please be more careful with units (but don't use MiB's; I should=20 >> killfile that). 90 Mbits/s is still slow. >Mb is megabyte in my book. This is the unit used by `systat -vm', which I = mentioned earlier. 90 megabytes/s is still not at the limit of Gigabit >Eth= ernet, however, but it is a lot closer to it. You have a different book to most then. I've always understood 'b' to mean bits, and 'B' to mean bytes. While every= one on here will understand what you were trying to say in context, given p= urely that number, I would expect most people to interpret it as 90+Mb/s = =3D ~11+MB/s. That is not what you meant, hence why Bruce said be careful w= ith the units. I just had a quick look in 'systat -vm' and it seems to use a capital B eve= rywhere from what I can see. Matt >Yours, > -mi From owner-freebsd-fs@freebsd.org Tue Jan 5 11:51:10 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B9142A62383 for ; Tue, 5 Jan 2016 11:51:10 +0000 (UTC) (envelope-from thomasrcurry@gmail.com) Received: from mail-ig0-x22b.google.com (mail-ig0-x22b.google.com [IPv6:2607:f8b0:4001:c05::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8863D1528 for ; Tue, 5 Jan 2016 11:51:10 +0000 (UTC) (envelope-from thomasrcurry@gmail.com) Received: by mail-ig0-x22b.google.com with SMTP id ik10so10547145igb.1 for ; Tue, 05 Jan 2016 03:51:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=dE9mh7AqyC57Yhk6meGwqjMtchTYwYzFWh/Bgmg+ynw=; b=U/1lXbakm/5JYsWJM4S+Mm25W9cbA+yhNridrFSdhdsczQRbQpx6rBRKrHPr5ZsBwo tllXQ4a3HoymCKl2SiZNmDnRVyx1gLfzdvGo0PRcv8U9vvqDmGSPM4Vdp0yzfxYCEm6f nsu4ojfdNTNv4/AjZPdh6wtS4N1fC591FsujfHdUvEUTE0WnIJetw2uvk/iTOzG1EjrO M2/4PX79A06AgWi+ZiFgXPU48jjw/NG8pBSoZHTCibcUX4Eq60gzjMcPdO403wIlQU+w O/SvXtLMlqjttjoHsLulIOvP9GhGpfdWewpXVhvI+4jR2N78V8IwYDAqELy+EDWWsYTi IcrA== MIME-Version: 1.0 X-Received: by 10.50.50.238 with SMTP id f14mr3409194igo.37.1451994669929; Tue, 05 Jan 2016 03:51:09 -0800 (PST) Received: by 10.107.4.71 with HTTP; Tue, 5 Jan 2016 03:51:09 -0800 (PST) In-Reply-To: References: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> <5688D3C1.90301@aldan.algebra.com> <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> <568A047B.1010000@aldan.algebra.com> Date: Tue, 5 Jan 2016 06:51:09 -0500 Message-ID: Subject: Re: NFS reads vs. writes From: Tom Curry To: Paul Kraus Cc: FreeBSD Filesystems , "Mikhail T." Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2016 11:51:10 -0000 On Mon, Jan 4, 2016 at 10:06 PM, Paul Kraus wrote: > On Jan 4, 2016, at 18:58, Tom Curry wrote: > > > SSDs are so fast for three main reasons: low latency, large dram buffer= s, > > and parallel workloads. Only one of these is of any benefit (latency) a= s > a > > SLOG. Unfortunately that particular metric is not usually advertised in > > consumer SSDs where the benchmarks they use to tout 90,000 random write > > iops consist of massively concurrent, highly compressible, short lived > > bursts of data. Add that drive as a SLOG and the onboard dram may as we= ll > > not even exist, and queue depths count for nothing. It will be lucky to > > pull 2,000 IOPS. Once you start adding in ZFS features like checksums a= nd > > compression, or network latency in the case of NFS that 2,000 number > starts > > to drop even more. > > I have a file server that I am going through the task of optimizing for > NFS traffic (to store VM images). My first attempt, because I knew about > the need for an SSD based SLOG for the ZIL was using a pair of Intel 535 > series SSD=E2=80=99s. The performance with the SLOG/ZIL on the SSD was _w= orse_. > Turns out that those SSD=E2=80=99s have poor small block (8 KB) random wr= ite > performance (not well advertised). So I asked for advice for choosing a > _fast_ SSD on the OpenZFS list and had a number of people recommend the > Intel DC-Sxxxx series of SSDs. > > Based on the very thorough data sheets, I am going with a pair of DC-S3710 > 200 GB SSDs. Once I get them in and configured I=E2=80=99ll post results. > > That is an excellent choice, I'm looking forward to the results. They have capacitors for power loss protection which means its safe to allow them to use their on-board dram cache. Wouldn't it be great if we could tell ZFS to not flush the cache on log devices via sysctl or some other means? Change nothing else about how sync operations occur. Unless I'm missing something this would provide the same level of protection but allow for greatly improved performance of the SSD. > Note that my zpool consists of 5 top level vdevs each made up of a 3-way > mirror. So I am striping writes across 5 columns. I am using 500 GB WD RE > series drives leaving the ZIL on the primary vdevs was _faster_ than addi= ng > the consumer SSD as SLOG for NFS writes. > > -- > Paul Kraus > paul@kraus-haus.org > > From owner-freebsd-fs@freebsd.org Tue Jan 5 17:21:05 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 31C16A6250A for ; Tue, 5 Jan 2016 17:21:05 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) Received: from vms173007pub.verizon.net (vms173007pub.verizon.net [206.46.173.7]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 128FC18D5 for ; Tue, 5 Jan 2016 17:21:04 +0000 (UTC) (envelope-from mi+thun@aldan.algebra.com) MIME-version: 1.0 Received: from aldan.narawntapu ([100.1.236.52]) by vms173007.mailsrvcs.net (Oracle Communications Messaging Server 7.0.5.32.0 64bit (built Jul 16 2014)) with ESMTPA id <0O0H008E5PIB8Z80@vms173007.mailsrvcs.net> for freebsd-fs@freebsd.org; Tue, 05 Jan 2016 11:20:40 -0600 (CST) X-CMAE-Score: 0 X-CMAE-Analysis: v=2.1 cv=WpDWSorv c=1 sm=1 tr=0 a=UorMnhrCY2jH/mPejITChw==:117 a=LaogzpLLAAAA:8 a=oR5dmqMzAAAA:8 a=7aQ_Q-yQQ-AA:10 a=r77TgQKjGQsHNAKrUKIA:9 a=-jmxLMdozlTs3IuwV8oA:9 a=bVxKfy92Q2wjxBX3:21 a=X7g00tsEmLOS7x0K:21 a=pILNOxqGKmIA:10 a=7a6K4RaKAAAA:8 a=LpYUZpimDiURvTF5xuIA:9 a=oXt2lh2RHotDM8qC:21 a=_W_S_7VecoQA:10 Subject: Mb vs. MB (Re: NFS reads vs. writes) To: Matt Churchyard , Bruce Evans References: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> <5688D3C1.90301@aldan.algebra.com> <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> <568A047B.1010000@aldan.algebra.com> <20160105143542.X1191@besplex.bde.org> <568B574A.7010603@aldan.algebra.com> Cc: freebsd-fs From: "Mikhail T." X-Enigmail-Draft-Status: N1110 Message-id: <568BFB63.5090203@aldan.algebra.com> Date: Tue, 05 Jan 2016 12:20:35 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 In-reply-to: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2016 17:21:05 -0000 On 05.01.2016 05:45, Matt Churchyard wrote: > You have a different book to most then. > I've always understood 'b' to mean bits, and 'B' to mean bytes. While everyone on here will understand what you were trying to say in context, given purely that number, I would expect most people to interpret it as 90+Mb/s = ~11+MB/s. That is not what you meant, hence why Bruce said be careful with the units. I've always found "megabit" to be a useless unit, owning its entire existence to marketing liars trying to make their wares have more impressive numbers. Unless the conversation is about information theory, or CPU-registers and bitfields, any mention of "bit" is useless and misleading. And then, of course, there is the contention -- by the same liars -- that megabyte is one million bytes. My book ignores them too -- and so does yours, I'm sure. I wish, we agreed on "megabits" being just as useless as, say, a kilomile (kM?). The only good reason -- in my tattered book -- to capitalize the name of a measurement unit is when it is derived from a person's name: Hertz, Ampre, Rntgen, etc. Yours, -mi From owner-freebsd-fs@freebsd.org Tue Jan 5 17:52:32 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E12BCA631EE for ; Tue, 5 Jan 2016 17:52:32 +0000 (UTC) (envelope-from eborisch@gmail.com) Received: from mail-qk0-x244.google.com (mail-qk0-x244.google.com [IPv6:2607:f8b0:400d:c09::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A297015D9 for ; Tue, 5 Jan 2016 17:52:32 +0000 (UTC) (envelope-from eborisch@gmail.com) Received: by mail-qk0-x244.google.com with SMTP id n135so16421844qka.0 for ; Tue, 05 Jan 2016 09:52:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=vUasHaQHTA8wnY0JIqO8eehHs61lTZfF3CHeeLiWskE=; b=T1PfWDUQ5pvzvG5fkKe4r1iWxTGsMuEeetB/4vyLeBK12M21553A8yb1jx+2mmYB2i 5tfHBClXqwINb1/FkHrijs0NHIj99zy9nogG9ypIEeNju8CrMKcxCVqsmsi/D+VATn71 bBloNDjwD2H/2WcLtKe3+6FkGyscqPc6C8ga5yQGnTOi0efycL2nbdF6ItYfsaSQx6X/ pXKJpyelqMsGrBc7b5i3eUURbJwPC6xUtkiCNi88PKA2NQKVlLr0PrpnJPvMj8W5oEB4 Z9lwzPz4DdnkRdEQ1ewMZ8APvZaZHwBpq7e/7inwWTcl/FsWTIBaIrozCVPDAYsFFe1C B78Q== MIME-Version: 1.0 X-Received: by 10.55.73.6 with SMTP id w6mr49342004qka.82.1452016351817; Tue, 05 Jan 2016 09:52:31 -0800 (PST) Received: by 10.55.22.75 with HTTP; Tue, 5 Jan 2016 09:52:31 -0800 (PST) In-Reply-To: <568BFB63.5090203@aldan.algebra.com> References: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> <5688D3C1.90301@aldan.algebra.com> <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> <568A047B.1010000@aldan.algebra.com> <20160105143542.X1191@besplex.bde.org> <568B574A.7010603@aldan.algebra.com> <568BFB63.5090203@aldan.algebra.com> Date: Tue, 5 Jan 2016 11:52:31 -0600 Message-ID: Subject: Re: Mb vs. MB (Re: NFS reads vs. writes) From: "Eric A. Borisch" To: "Mikhail T." Cc: Matt Churchyard , Bruce Evans , freebsd-fs Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2016 17:52:33 -0000 On Tue, Jan 5, 2016 at 11:20 AM, Mikhail T. wro= te: > On 05.01.2016 05:45, Matt Churchyard wrote: >> You have a different book to most then. >> I've always understood 'b' to mean bits, and 'B' to mean bytes. While ev= eryone on here will understand what you were trying to say in context, give= n purely that number, I would expect most people to interpret it as 90+Mb/s= =3D ~11+MB/s. That is not what you meant, hence why Bruce said be careful = with the units. > I've always found "megabit" to be a useless unit, owning its entire > existence to marketing liars trying to make their wares have more > impressive numbers. Unless the conversation is about information theory, > or CPU-registers and bitfields, any mention of "bit" is useless and > misleading. To (hopefully) end this off-topic discussion, there is a standard, and b =3D=3D bits, and B =3D=3D bytes (usually 8 * b). https://en.wikipedia.org/wiki/IEEE_1541-2002 [...] bit (symbol 'b'), a binary digit; byte (symbol 'B'), a set of adjacent bits (usually, but not necessarily, eight) operated on as a group; [...] It is terribly important ( =3D=3D not useless ) in digital communications, where bits are the basic unit of transferred data, distinct from the signaling rate (bd), but I digress... - Eric From owner-freebsd-fs@freebsd.org Tue Jan 5 23:53:26 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C0CB3A638B9 for ; Tue, 5 Jan 2016 23:53:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 970611D28 for ; Tue, 5 Jan 2016 23:53:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u05NrQA5027026 for ; Tue, 5 Jan 2016 23:53:26 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205938] [ext2fs][patch][panic] EXT4: reading mmaped file causes panic because struct buf leaks Date: Tue, 05 Jan 2016 23:53:26 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: crash, patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: damjan.jov@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status keywords bug_severity priority component assigned_to reporter cc attachments.created Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2016 23:53:26 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205938 Bug ID: 205938 Summary: [ext2fs][patch][panic] EXT4: reading mmaped file causes panic because struct buf leaks Product: Base System Version: 11.0-CURRENT Hardware: Any OS: Any Status: New Keywords: crash, patch Severity: Affects Many People Priority: --- Component: kern Assignee: freebsd-bugs@FreeBSD.org Reporter: damjan.jov@gmail.com CC: freebsd-fs@FreeBSD.org Created attachment 165127 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D165127&action= =3Dedit Fix a kernel panic when reading mmaped files from EXT4 Calling mmap() on any sizeable file on an EXT4 filesystem, and then attempt= ing to read that memory (can be easily tested using the "cmp file file" tool), causes a reproducible kernel panic: userret: returning with the following locks held: exclusive lockmgr bufwait (bufwait) r =3D 0 (0xfffffe001d90c220) locked @ /usr/src/sys/kern/vfs_bio.c:1454 panic: witness_warn cpuid =3D 0 KDB: stack backtrace: db_trace_self_wrapper() at db_trace-self_wrapper+0x2b/frame 0xfffffe002b7e6= 7f0 vpanic() at vpanic+0x182/frame 0xfffffe002b7e6870 kassert_panic() at kassert_panic+0x126/frame 0xfffffe002b7e68e0 witness_warn() at witness_warn+0x3c6/frame 0xfffffe002b7e69b0 userret() at userret+0x98/frame 0xfffffe002b7e69e0 trap() at trap+0x3f4/frame 0xfffffe002b7e6bf0 calltrap() at calltrap+0x8/frame 0xfffffe002b7e6bf0 --- trap 0xc, rip =3D 0x4019c0, rsp =3D 0x7fffffffe940, rbp =3D 0x7ffffffff= eea30 --- KDB: enter: panic [ thread pid 909 tid 100082 ] Stopped at kdb_enter+0x3b: movq $0,kdb_why The problem comes from ext4_bmapext() in sys/fs/ext2fs/ext2_bmap.c never calling brelse(), meaning the "struct buf" returned in path.ep_bp from ext4_ext_find_extent() is never released/unlocked, something userret() catc= hes later and panics from. The attached patch always calls brelse(path.ep_bp), fixing reading EXT4 fil= es using mmap(). This affects all versions of FreeBSD. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Wed Jan 6 16:10:01 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44964A62141 for ; Wed, 6 Jan 2016 16:10:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 34F701A9D for ; Wed, 6 Jan 2016 16:10:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u06GA0Yt025573 for ; Wed, 6 Jan 2016 16:10:01 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205938] [ext2fs][patch][panic] EXT4: reading mmaped file causes panic because struct buf leaks Date: Wed, 06 Jan 2016 16:10:01 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: crash, patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status assigned_to cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jan 2016 16:10:01 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205938 Pedro F. Giffuni changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |In Progress Assignee|freebsd-bugs@FreeBSD.org |pfg@FreeBSD.org CC| |gnehzuil@gmail.com, | |pfg@FreeBSD.org --- Comment #1 from Pedro F. Giffuni --- Patch looks good. Add Zheng Liu (author of original code) for review. Take. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Wed Jan 6 16:25:16 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D5A6AA629AF for ; Wed, 6 Jan 2016 16:25:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C651E18AF for ; Wed, 6 Jan 2016 16:25:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u06GPGlr060135 for ; Wed, 6 Jan 2016 16:25:16 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205668] [ext2fs] [patch] cannot mount EXT4 filesystems which weren't cleanly unmounted Date: Wed, 06 Jan 2016 16:25:16 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jan 2016 16:25:16 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205668 --- Comment #6 from commit-hook@freebsd.org --- A commit references this bug: Author: pfg Date: Wed Jan 6 16:25:01 UTC 2016 New revision: 293236 URL: https://svnweb.freebsd.org/changeset/base/293236 Log: MFC r292872: ext2: recognize ext4 INCOMPAT_RECOVER flag This is a flag specific for journalling in ext4. Add it to the list of ext4 features we ignore for read-only purposes. PR: 205668 Changes: _U stable/10/ stable/10/sys/fs/ext2fs/ext2fs.h --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Wed Jan 6 16:28:18 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 913DDA62B13 for ; Wed, 6 Jan 2016 16:28:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 81FD91AF2 for ; Wed, 6 Jan 2016 16:28:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u06GSIBQ064348 for ; Wed, 6 Jan 2016 16:28:18 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205668] [ext2fs] [patch] cannot mount EXT4 filesystems which weren't cleanly unmounted Date: Wed, 06 Jan 2016 16:28:18 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jan 2016 16:28:18 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205668 --- Comment #7 from commit-hook@freebsd.org --- A commit references this bug: Author: pfg Date: Wed Jan 6 16:27:18 UTC 2016 New revision: 293237 URL: https://svnweb.freebsd.org/changeset/base/293237 Log: MFC r292872: ext2: recognize ext4 INCOMPAT_RECOVER flag This is a flag specific for journalling in ext4. Add it to the list of ext4 features we ignore for read-only purposes. PR: 205668 Changes: _U stable/9/sys/ _U stable/9/sys/fs/ stable/9/sys/fs/ext2fs/ext2fs.h --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Wed Jan 6 17:51:07 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1CC8AA64969 for ; Wed, 6 Jan 2016 17:51:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0DA111605 for ; Wed, 6 Jan 2016 17:51:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u06Hp65A052911 for ; Wed, 6 Jan 2016 17:51:06 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205668] [ext2fs] [patch] cannot mount EXT4 filesystems which weren't cleanly unmounted Date: Wed, 06 Jan 2016 17:51:07 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jan 2016 17:51:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205668 Pedro F. Giffuni changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|In Progress |Closed --- Comment #8 from Pedro F. Giffuni --- This should be solved in all supported FreeBSD versions now. Thanks! --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Wed Jan 6 23:53:23 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A4B7A6441C for ; Wed, 6 Jan 2016 23:53:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8B63113D7 for ; Wed, 6 Jan 2016 23:53:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u06NrNXH075262 for ; Wed, 6 Jan 2016 23:53:23 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205974] can't detach devices from zpool Date: Wed, 06 Jan 2016 23:53:23 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jan 2016 23:53:23 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205974 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Jan 6 23:59:31 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E1CE4A64886 for ; Wed, 6 Jan 2016 23:59:31 +0000 (UTC) (envelope-from john@potato.growveg.org) Received: from potato.growveg.org (potato.growveg.org [62.49.247.163]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AAC8E164A for ; Wed, 6 Jan 2016 23:59:31 +0000 (UTC) (envelope-from john@potato.growveg.org) Received: from john by potato.growveg.org with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1aGxcT-000CsO-QG for freebsd-fs@freebsd.org; Wed, 06 Jan 2016 23:35:57 +0000 Date: Wed, 6 Jan 2016 23:35:57 +0000 From: John To: freebsd-fs@freebsd.org Subject: zfs send compression query Message-ID: <20160106233557.GA6185@potato.growveg.org> Reply-To: freebsd-fs@freebsd.org Mail-Followup-To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline User-Agent: Mutt/1.5.24 (2015-08-30) Sender: john X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jan 2016 23:59:32 -0000 Hello list, I have a selection of zfs datasets that I'm snapshotting to a remote freebsd server. Thing is, I want to use less bandwidth. Here is an example incantation, from the server: $ zfs send -R vms/132@2016-01-06.2122 | ssh backup@[remote_ip] \ zfs recv -Fdvu storage/snapshots How do/would I use xz -9 to compress the stream over the wire? The uncompressed snapshot is 57GB. 7zip (using lzma2 like xz) will compress it to 12GB. thanks -- John From owner-freebsd-fs@freebsd.org Thu Jan 7 01:07:59 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 71066A66104 for ; Thu, 7 Jan 2016 01:07:59 +0000 (UTC) (envelope-from sean@chittenden.org) Received: from mail01.lax1.stackjet.com (mon01.lax1.stackjet.com [174.136.104.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5901C1161 for ; Thu, 7 Jan 2016 01:07:57 +0000 (UTC) (envelope-from sean@chittenden.org) Received: from [22.24.238.88] (unknown [172.58.16.75]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: sean@chittenden.org) by mail01.lax1.stackjet.com (Postfix) with ESMTPSA id E076945B8C6 for ; Wed, 6 Jan 2016 17:01:45 -0800 (PST) From: Sean Chittenden Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (1.0) Subject: Re: zfs send compression query Message-Id: Date: Wed, 6 Jan 2016 17:01:44 -0800 References: <20160106233557.GA6185@potato.growveg.org> In-Reply-To: <20160106233557.GA6185@potato.growveg.org> To: freebsd-fs@freebsd.org X-Mailer: iPhone Mail (13C75) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jan 2016 01:07:59 -0000 -- Sean Chittenden > On Jan 6, 2016, at 15:35, John wrote: > > $ zfs send -R vms/132@2016-01-06.2122 | pigz -9c | ssh backup@[remote_ip] \ > pigz -dc | zfs recv -Fdvu storage/snapshots Cheers. -sc From owner-freebsd-fs@freebsd.org Thu Jan 7 01:45:19 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9D672A66B6C for ; Thu, 7 Jan 2016 01:45:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 840D0102C for ; Thu, 7 Jan 2016 01:45:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u071jJbt027870 for ; Thu, 7 Jan 2016 01:45:19 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 204358] zfs loader zfs_probe_args secsz is too small, causing memory corruption Date: Thu, 07 Jan 2016 01:45:19 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: allanjude@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jan 2016 01:45:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204358 Allan Jude changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |allanjude@FreeBSD.org Status|New |In Progress --- Comment #1 from Allan Jude --- https://reviews.freebsd.org/D4811 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Jan 7 02:43:58 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C7A29A662D1 for ; Thu, 7 Jan 2016 02:43:58 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 824351DE3 for ; Thu, 7 Jan 2016 02:43:58 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from freddy.simplesystems.org (freddy.simplesystems.org [65.66.246.65]) by blade.simplesystems.org (8.14.4+Sun/8.14.4) with ESMTP id u072hlbV026957; Wed, 6 Jan 2016 20:43:48 -0600 (CST) Date: Wed, 6 Jan 2016 20:43:47 -0600 (CST) From: Bob Friesenhahn X-X-Sender: bfriesen@freddy.simplesystems.org To: Sean Chittenden cc: freebsd-fs@freebsd.org Subject: Re: zfs send compression query In-Reply-To: Message-ID: References: <20160106233557.GA6185@potato.growveg.org> User-Agent: Alpine 2.01 (GSO 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (blade.simplesystems.org [65.66.246.90]); Wed, 06 Jan 2016 20:43:49 -0600 (CST) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jan 2016 02:43:58 -0000 Using 'lzop' rather than 'pigz' is a better choice for CPU efficiency. It offers more speed with less CPU use. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From owner-freebsd-fs@freebsd.org Thu Jan 7 07:36:15 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 12C89A66012 for ; Thu, 7 Jan 2016 07:36:15 +0000 (UTC) (envelope-from john@potato.growveg.org) Received: from potato.growveg.org (potato.growveg.org [62.49.247.163]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D052D1CDF for ; Thu, 7 Jan 2016 07:36:14 +0000 (UTC) (envelope-from john@potato.growveg.org) Received: from john by potato.growveg.org with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1aH57D-000JBz-AQ for freebsd-fs@freebsd.org; Thu, 07 Jan 2016 07:36:11 +0000 Date: Thu, 7 Jan 2016 07:36:11 +0000 From: John To: freebsd-fs@freebsd.org Subject: Re: zfs send compression query Message-ID: <20160107073611.GA50011@potato.growveg.org> Reply-To: freebsd-fs@freebsd.org Mail-Followup-To: freebsd-fs@freebsd.org References: <20160106233557.GA6185@potato.growveg.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) Sender: john X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jan 2016 07:36:15 -0000 On Wed, Jan 06, 2016 at 05:01:44PM -0800, Sean Chittenden wrote: > > >-- >Sean Chittenden > > >> On Jan 6, 2016, at 15:35, John wrote: >> >> $ zfs send -R vms/132@2016-01-06.2122 | pigz -9c | ssh backup@[remote_ip] \ >> pigz -dc | zfs recv -Fdvu storage/snapshots The problem I'm getting with that compressor (and others) is that it bails like this: send from @ to vms/133@2016-01-07.0648 estimated size is 32.1G total estimated size is 32.1G cannot receive: specified fs (storage/snapshots) does not exist Pseudo-terminal will not be allocated because stdin is not a terminal. TIME SENT SNAPSHOT -bash: line 1: syntax error near unexpected token |' [then get a load of binary garbled output and then a broken pipe error] zfs send works fine if I don't try to compress the stream: $ zfs send -Rv vms/133@2016-01-07.0648 | ssh backup@[redacted] zfs recv -Fdvu storage/snapshots send from @ to vms/133@2016-01-07.0648 estimated size is 32.1G total estimated size is 32.1G TIME SENT SNAPSHOT receiving full stream of vms/133@2016-01-07.0648 into storage/snapshots/133@2016-01-07.0648 07:26:44 300K vms/133@2016-01-07.0648 07:26:45 1.92M vms/133@2016-01-07.0648 07:26:46 3.42M vms/133@2016-01-07.0648 I'm guessing I'm doing something wrong with the pipes, but not sure how to solve it... -- John From owner-freebsd-fs@freebsd.org Thu Jan 7 07:59:37 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 48115A665BF for ; Thu, 7 Jan 2016 07:59:37 +0000 (UTC) (envelope-from john@potato.growveg.org) Received: from potato.growveg.org (potato.growveg.org [62.49.247.163]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0E8C11489 for ; Thu, 7 Jan 2016 07:59:37 +0000 (UTC) (envelope-from john@potato.growveg.org) Received: from john by potato.growveg.org with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1aH5Tr-000JDg-68 for freebsd-fs@freebsd.org; Thu, 07 Jan 2016 07:59:35 +0000 Date: Thu, 7 Jan 2016 07:59:35 +0000 From: John To: freebsd-fs@freebsd.org Subject: Re: zfs send compression query Message-ID: <20160107075935.GB50011@potato.growveg.org> Reply-To: freebsd-fs@freebsd.org Mail-Followup-To: freebsd-fs@freebsd.org References: <20160106233557.GA6185@potato.growveg.org> <20160107073611.GA50011@potato.growveg.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <20160107073611.GA50011@potato.growveg.org> User-Agent: Mutt/1.5.24 (2015-08-30) Sender: john X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jan 2016 07:59:37 -0000 On Thu, Jan 07, 2016 at 07:36:11AM +0000, John wrote: >I'm guessing I'm doing something wrong with the pipes, but not sure >how to solve it... sorted it, now that I have had enough caffeine. I was missing quotation marks and misplaced a pipe! $ zfs send -Rv vms/133@2016-01-07.0648 | pigz -9c | ssh backup@[redacted] "pigz -dc | zfs recv -Fdvu storage/snapshots" send from @ to vms/133@2016-01-07.0648 estimated size is 32.1G total estimated size is 32.1G TIME SENT SNAPSHOT receiving full stream of vms/133@2016-01-07.0648 into storage/snapshots/133@2016-01-07.0648 07:50:11 16.7M vms/133@2016-01-07.0648 07:50:12 63.2M vms/133@2016-01-07.0648 07:50:13 143M vms/133@2016-01-07.0648 07:50:14 153M vms/133@2016-01-07.0648 07:50:15 164M vms/133@2016-01-07.0648 *happy* -- John From owner-freebsd-fs@freebsd.org Thu Jan 7 08:05:40 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B364DA669EA for ; Thu, 7 Jan 2016 08:05:40 +0000 (UTC) (envelope-from sean@chittenden.org) Received: from mail01.lax1.stackjet.com (mon01.lax1.stackjet.com [174.136.104.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A409419A2 for ; Thu, 7 Jan 2016 08:05:39 +0000 (UTC) (envelope-from sean@chittenden.org) Received: from hormesis.local (173-228-13-241.dsl.static.fusionbroadband.com [173.228.13.241]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: sean@chittenden.org) by mail01.lax1.stackjet.com (Postfix) with ESMTPSA id 762D145B8AF; Thu, 7 Jan 2016 00:05:37 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\)) Subject: Re: zfs send compression query From: Sean Chittenden In-Reply-To: <20160107075935.GB50011@potato.growveg.org> Date: Thu, 7 Jan 2016 00:05:36 -0800 Cc: freebsd-fs@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <7C9E16F4-AFE3-49E5-9686-B33D0E0FDB37@chittenden.org> References: <20160106233557.GA6185@potato.growveg.org> <20160107073611.GA50011@potato.growveg.org> <20160107075935.GB50011@potato.growveg.org> To: freebsd-lists@potato.growveg.org X-Mailer: Apple Mail (2.3112) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jan 2016 08:05:40 -0000 Bonus points: $ SNAP_SIZE=3D`zfs send -nvP "vms/133@2016-01-07.0648" | tail -n 1 | awk = '{print $2}'` $ zfs send -Rv vms/133@2016-01-07.0648 | pv -r -a -b -t -e -s = "${SNAP_SIZE}" -B 512m | pigz -9c | ssh backup@[redacted] "pigz -dc | = zfs recv -Fdvu storage/snapshots" Useful for getting an ETA. -sc -- Sean Chittenden sean@chittenden.org > On Jan 6, 2016, at 23:59, John = wrote: >=20 > On Thu, Jan 07, 2016 at 07:36:11AM +0000, John wrote: >=20 >> I'm guessing I'm doing something wrong with the pipes, but not sure >> how to solve it... >=20 > sorted it, now that I have had enough caffeine. I was missing > quotation marks and misplaced a pipe! >=20 > $ zfs send -Rv vms/133@2016-01-07.0648 | pigz -9c | > ssh backup@[redacted] "pigz -dc | zfs recv -Fdvu > storage/snapshots" > send from @ to vms/133@2016-01-07.0648 estimated size is 32.1G > total estimated size is 32.1G > TIME SENT SNAPSHOT > receiving full stream of vms/133@2016-01-07.0648 into = storage/snapshots/133@2016-01-07.0648 > 07:50:11 16.7M vms/133@2016-01-07.0648 > 07:50:12 63.2M vms/133@2016-01-07.0648 > 07:50:13 143M vms/133@2016-01-07.0648 > 07:50:14 153M vms/133@2016-01-07.0648 > 07:50:15 164M vms/133@2016-01-07.0648 >=20 > *happy* > --=20 > John _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Thu Jan 7 12:18:37 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47244A64592 for ; Thu, 7 Jan 2016 12:18:37 +0000 (UTC) (envelope-from john@potato.growveg.org) Received: from potato.growveg.org (potato.growveg.org [62.49.247.163]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1022B1E0D for ; Thu, 7 Jan 2016 12:18:36 +0000 (UTC) (envelope-from john@potato.growveg.org) Received: from john by potato.growveg.org with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1aH9WU-000JgX-Qc for freebsd-fs@freebsd.org; Thu, 07 Jan 2016 12:18:34 +0000 Date: Thu, 7 Jan 2016 12:18:34 +0000 From: John To: freebsd-fs@freebsd.org Subject: Re: zfs send compression query Message-ID: <20160107121834.GC50011@potato.growveg.org> Reply-To: freebsd-fs@freebsd.org Mail-Followup-To: freebsd-fs@freebsd.org References: <20160106233557.GA6185@potato.growveg.org> <20160107073611.GA50011@potato.growveg.org> <20160107075935.GB50011@potato.growveg.org> <7C9E16F4-AFE3-49E5-9686-B33D0E0FDB37@chittenden.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <7C9E16F4-AFE3-49E5-9686-B33D0E0FDB37@chittenden.org> User-Agent: Mutt/1.5.24 (2015-08-30) Sender: john X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jan 2016 12:18:37 -0000 On Thu, Jan 07, 2016 at 12:05:36AM -0800, Sean Chittenden wrote: >Bonus points: > >$ SNAP_SIZE=`zfs send -nvP "vms/133@2016-01-07.0648" | tail -n 1 | awk '{print $2}'` >$ zfs send -Rv vms/133@2016-01-07.0648 | pv -r -a -b -t -e -s "${SNAP_SIZE}" -B 512m | pigz -9c | ssh backup@[redacted] "pigz -dc | zfs recv -Fdvu storage/snapshots" > >Useful for getting an ETA. -sc Thanks for that. What do you think of other streaming compressors like xz / lzrzip / pbzip2 / bzip2 / zpaq / pixz ? -- John From owner-freebsd-fs@freebsd.org Thu Jan 7 15:19:22 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CB95AA660A6 for ; Thu, 7 Jan 2016 15:19:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BCA3B1FFA for ; Thu, 7 Jan 2016 15:19:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u07FJM1h000559 for ; Thu, 7 Jan 2016 15:19:22 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205974] can't detach devices from zpool Date: Thu, 07 Jan 2016 15:19:23 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: lifanov@mail.lifanov.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jan 2016 15:19:22 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205974 --- Comment #3 from Nikolai Lifanov --- I was able to take a disk out using "zpool split" and add disks from former pool to it. I'm in a middle of a resilver right now, but I'll try to reproc= ude the problem and provide the info when I'm done. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Jan 7 16:54:21 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 34177A665D3 for ; Thu, 7 Jan 2016 16:54:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 254C01CA3 for ; Thu, 7 Jan 2016 16:54:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u07GsKpL023544 for ; Thu, 7 Jan 2016 16:54:21 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205974] can't detach devices from zpool Date: Thu, 07 Jan 2016 16:54:21 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: lifanov@mail.lifanov.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jan 2016 16:54:21 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205974 --- Comment #4 from Nikolai Lifanov --- $ zpool status pool: data state: ONLINE scan: scrub repaired 75K in 6h34m with 0 errors on Sun Dec 20 12:43:26 20= 15 config: NAME STATE READ WRITE CKSUM data ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 ada0 ONLINE 0 0 0 ada1 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 ada2 ONLINE 0 0 0 ada3 ONLINE 0 0 0 logs mirror-2 ONLINE 0 0 0 ada6p1 ONLINE 0 0 0 ada7p1 ONLINE 0 0 0 errors: No known data errors pool: tank state: ONLINE scan: resilvered 583G in 1h56m with 0 errors on Thu Jan 7 11:03:58 2016 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 ada4p2 ONLINE 0 0 0 ada5p2 ONLINE 0 0 0 logs mirror-1 ONLINE 0 0 0 ada6p2 ONLINE 0 0 0 ada7p2 ONLINE 0 0 0 errors: No known data errors $ uname -a FreeBSD lifanovbsd0 11.0-CURRENT FreeBSD 11.0-CURRENT #21 r293241M: Wed Jan= 6 13:21:02 EST 2016 root@lifanovbsd0:/usr/obj/usr/src/sys/GENERIC-NODEBUG= =20 amd64 $ zdb -C data: version: 5000 name: 'data' state: 0 txg: 12863548 pool_guid: 2244539855095666375 hostid: 657389954 hostname: 'lifanovbsd0' vdev_children: 3 vdev_tree: type: 'root' id: 0 guid: 2244539855095666375 children[0]: type: 'mirror' id: 0 guid: 9136418650500458576 metaslab_array: 37 metaslab_shift: 32 ashift: 9 asize: 500103118848 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 820204606399244410 path: '/dev/ada0' phys_path: '/dev/ada0' whole_disk: 1 DTL: 481 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 7129954888383586426 path: '/dev/ada1' phys_path: '/dev/ada1' whole_disk: 1 DTL: 480 create_txg: 4 children[1]: type: 'mirror' id: 1 guid: 22857558361533772 metaslab_array: 34 metaslab_shift: 32 ashift: 9 asize: 500103118848 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 3541792554089895143 path: '/dev/ada2' phys_path: '/dev/ada2' whole_disk: 1 DTL: 483 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 4600530648211267864 path: '/dev/ada3' phys_path: '/dev/ada3' whole_disk: 1 DTL: 482 create_txg: 4 children[2]: type: 'mirror' id: 2 guid: 10125232546100156377 metaslab_array: 8299 metaslab_shift: 27 ashift: 12 asize: 17175150592 is_log: 1 create_txg: 9083333 children[0]: type: 'disk' id: 0 guid: 9706229731141400493 path: '/dev/ada6p1' phys_path: '/dev/ada6p1' whole_disk: 1 DTL: 2682 create_txg: 9083333 children[1]: type: 'disk' id: 1 guid: 4287673589667345344 path: '/dev/ada7p1' phys_path: '/dev/ada7p1' whole_disk: 1 DTL: 2681 create_txg: 9083333 features_for_read: com.delphix:hole_birth com.delphix:embedded_data tank: version: 5000 name: 'tank' state: 0 txg: 13891 pool_guid: 1331125925218803251 hostid: 657389954 hostname: 'lifanovbsd0' vdev_children: 2 vdev_tree: type: 'root' id: 0 guid: 1331125925218803251 create_txg: 4 children[0]: type: 'mirror' id: 0 guid: 9985890284685236925 whole_disk: 0 metaslab_array: 34 metaslab_shift: 34 ashift: 12 asize: 2748774350848 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 18200602675754198144 path: '/dev/ada4p2' phys_path: '/dev/ada4p2' whole_disk: 1 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 9057482964507241068 path: '/dev/ada5p2' phys_path: '/dev/ada5p2' whole_disk: 1 DTL: 40 create_txg: 4 resilver_txg: 13092 children[1]: type: 'mirror' id: 1 guid: 1504801081987587436 metaslab_array: 68 metaslab_shift: 27 ashift: 12 asize: 17175150592 is_log: 1 create_txg: 13887 children[0]: type: 'disk' id: 0 guid: 14079142470155627886 path: '/dev/ada6p2' phys_path: '/dev/ada6p2' whole_disk: 1 create_txg: 13887 children[1]: type: 'disk' id: 1 guid: 2344469679409857908 path: '/dev/ada7p2' phys_path: '/dev/ada7p2' whole_disk: 1 create_txg: 13887 features_for_read: com.delphix:hole_birth com.delphix:embedded_data --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Jan 7 16:56:03 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 11277A666E5 for ; Thu, 7 Jan 2016 16:56:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 021F01E38 for ; Thu, 7 Jan 2016 16:56:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u07Gu2VJ025946 for ; Thu, 7 Jan 2016 16:56:02 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205974] can't detach devices from zpool Date: Thu, 07 Jan 2016 16:56:03 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: lifanov@mail.lifanov.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jan 2016 16:56:03 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205974 --- Comment #5 from Nikolai Lifanov --- The problem pool was "tank", but I can't reproduce the problem anymore after doing a "zpool split" and reconnecting the rest of the old pool devices to = it. I was able to successfully detach ada5p2 and then reattach it. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Jan 7 18:15:47 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9E3CCA66AD6 for ; Thu, 7 Jan 2016 18:15:47 +0000 (UTC) (envelope-from ndevos@redhat.com) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8623115C2 for ; Thu, 7 Jan 2016 18:15:47 +0000 (UTC) (envelope-from ndevos@redhat.com) Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) by mx1.redhat.com (Postfix) with ESMTPS id EEA4991EA3; Thu, 7 Jan 2016 18:10:23 +0000 (UTC) Received: from localhost (vpn1-6-42.ams2.redhat.com [10.36.6.42]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u07IALMg002946 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 7 Jan 2016 13:10:23 -0500 Date: Thu, 7 Jan 2016 19:10:21 +0100 From: Niels de Vos To: Rick Macklem Cc: gluster-devel@gluster.org, freebsd-fs , Shyam Ranganathan Subject: Re: [Gluster-devel] FreeBSD port of GlusterFS racks up a lot of CPU usage Message-ID: <20160107181021.GK27495@ndevos-x240.usersys.redhat.com> References: <571237035.145690509.1451437960464.JavaMail.zimbra@uoguelph.ca> <20151230103152.GS13942@ndevos-x240.usersys.redhat.com> <923007690.145828058.1451484032304.JavaMail.zimbra@uoguelph.ca> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="9q+4pEgVd7t11q9L" Content-Disposition: inline In-Reply-To: <923007690.145828058.1451484032304.JavaMail.zimbra@uoguelph.ca> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jan 2016 18:15:47 -0000 --9q+4pEgVd7t11q9L Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Dec 30, 2015 at 09:00:32AM -0500, Rick Macklem wrote: > Niels de Vos wrote: > > On Tue, Dec 29, 2015 at 08:12:40PM -0500, Rick Macklem wrote: > > > Hi, > > >=20 > > > I'm been playing with the FreeBSD port of GlusterFS and it seems > > > to be working ok. I do notice that the daemons use a lot of CPU, > > > even when there is nothing to do (no volumes started, etc). > > > When I ktrace the daemon, I see a small number of nanosleep() and > > > select() syscalls and lots of poll() syscalls (close to 1000/sec). > > >=20 > > > Looking at libglusterfs/src/event-poll.c, I find: > > > ret =3D poll(ufds, size, 1); > > > in a loop. The only thing the code seems to do when poll() times > > > out is a call to event_dispatch_poll_resize(). > > >=20 > > > So, is it necessary to call event_dispatch_poll_resize() 1000 times > > > per second? > > > Or is there a way to make event_dispatch_poll_resize() return quickly > > > when there is nothing to do? > >=20 > > I do not think this is critical. A longer timeout should be well > > acceptable. > >=20 > > > I'm guessing that Linux uses the event-epoll stuff instead of event-p= oll, > > > so it wouldn't exhibit this. Is that correct? > >=20 > > Well, both. most (if not all) Linux builds will use event-poll. But, > > that calls epoll_wait() with a timeout of 1 millisecond as well. > >=20 > Actually, when I look at the 3.7.6 sources in libglusterfs/src/event-epol= l.c > I only find one epoll_wait() at line#668: > ret =3D epoll_wait (event_pool->fd, &event, 1, -1); > so the timeout never happens in this code. > (It does have code after the call to handle the timeout case.) > All it seems to do (if it were to timeout) is adjust the # of threads in = the > event-epoll case, so my hunch is that a timeout isn't needed? Oh, yes, indeed, my mistake. > For the event-poll.c case, it calls event_dispatch_poll_size() which look= s like > it might add new file descriptors, so someone more familiar with this cod= e would > need to decide if the timeout can be disabled (my hunch is no, but I'm no= t familiar > with the code). Maybe Shyam (on CC) knows more how event-poll works. He is definitely one of the developers that could explain epoll, and I guess he looked into poll too. Niels > > > Thanks for any information on this, rick > > > ps: I am tempted to just crank the timeout of 1msec up to 10 or 20mse= c. > >=20 > > Yes, that is probably what I would do too. And have both poll functions > > use the same timeout, have it defined in libglusterfs/src/event.h. We > > could make it a configurable option too, but I do not think it is very > > useful to have. > >=20 > > Could you file a bug and/or send a patch for this? > >=20 > I will try bumping the timeout up in event-poll.c and if it seems to redu= ce > CPU usage and not cause any obvious grief, I will file a bug report. >=20 > Thanks for your help with this, rick >=20 > > Thanks, > > Niels > >=20 --9q+4pEgVd7t11q9L Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJWjqoNAAoJECXo5AApwsWzRlsP/iWp6VZReXrr/rix+kasiF1g YnMjvA4zS6AFSklXvcOifkvmpjr8VjHAqsoxWHOinZDLikk7m3Cbu5z+2wSLzUHh fQkY0n/s9fl6Ji4orX11CesetZAfMsNUZgn9qMqxu2zJf2zte1m5FbyvLNHfBdoe PdlAYJQGYEP6uD2hr8tNx6s37w5ypAwNRDuVK6nH0sZwYiQiY2uIxODaz0rf5zci gOKwvy2RDgKvUvDW5Jfq+sCCsm/KENkthENfQf8IWglNJhNwjPK5i/1CHPLKZk5e bvD9THZSsPoj5joCUuh1OfVsTj7AZMyasL3C5mL/NfWtOYBfIk0qFwzrvf9LZoWC d3P540PT0a2SogvrBa+NSsZeXCz1xT6A8M/wukCoX8t0WJ7gAvJukG6djbhQ75ZB xFUfACApwby5Z/JP1LYKJ5Npvd21Dse5HVxdyy0+mME89KJOjBGrbl7gry1Pks4R w4OgUuTqRPVN8XfMzXtPRZEXzLTNr33B3dSKFIoChPY60Tl6w2NgXHtFL/lbYNqb FxivkegF7Sq+fTb8hmmITCAVWYtyrm0So0LdFQCzBDgBQix5D5q/+ioj9Ralhk4d SbcLs5euksBzGeUfSgOf7MGAmXlkBRimn4j5HeE8TZ5zK90tVuP+Ohm5irAQnBvs g4YPOuWMzMqBXjlZ+EkT =3307 -----END PGP SIGNATURE----- --9q+4pEgVd7t11q9L-- From owner-freebsd-fs@freebsd.org Thu Jan 7 21:32:04 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F173A67998 for ; Thu, 7 Jan 2016 21:32:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8E2A31BAE for ; Thu, 7 Jan 2016 21:32:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u07LW4er013386 for ; Thu, 7 Jan 2016 21:32:04 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205938] [ext2fs][patch][panic] EXT4: reading mmaped file causes panic because struct buf leaks Date: Thu, 07 Jan 2016 21:32:04 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: crash, patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.isobsolete attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jan 2016 21:32:04 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205938 Pedro F. Giffuni changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #165127|0 |1 is obsolete| | --- Comment #2 from Pedro F. Giffuni --- Created attachment 165231 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D165231&action= =3Dedit Fix reading mmaped files from EXT4 Small change: it doesn't seem like defining "ret" in necessary or useful. Please review. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Thu Jan 7 21:34:59 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 32511A67A8D for ; Thu, 7 Jan 2016 21:34:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 211ED1CEE for ; Thu, 7 Jan 2016 21:34:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u07LYw95018868 for ; Thu, 7 Jan 2016 21:34:59 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205938] [ext2fs][patch][panic] EXT4: reading mmaped file causes panic because struct buf leaks Date: Thu, 07 Jan 2016 21:34:59 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: crash, patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.isobsolete Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jan 2016 21:34:59 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205938 Pedro F. Giffuni changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #165127|1 |0 is obsolete| | --- Comment #3 from Pedro F. Giffuni --- Comment on attachment 165127 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D165127 Fix a kernel panic when reading mmaped files from EXT4 Nevermind... I got that wrong. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Thu Jan 7 21:35:30 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 10205A67ADB for ; Thu, 7 Jan 2016 21:35:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F3E141D67 for ; Thu, 7 Jan 2016 21:35:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u07LZTkb019623 for ; Thu, 7 Jan 2016 21:35:29 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205938] [ext2fs][patch][panic] EXT4: reading mmaped file causes panic because struct buf leaks Date: Thu, 07 Jan 2016 21:35:30 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: crash, patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.isobsolete Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jan 2016 21:35:30 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205938 Pedro F. Giffuni changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #165231|0 |1 is obsolete| | --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Thu Jan 7 21:44:05 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2240A67EA4 for ; Thu, 7 Jan 2016 21:44:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A19E6194B for ; Thu, 7 Jan 2016 21:44:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u07Li5rB036472 for ; Thu, 7 Jan 2016 21:44:05 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205938] [ext2fs][patch][panic] EXT4: reading mmaped file causes panic because struct buf leaks Date: Thu, 07 Jan 2016 21:44:05 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: crash, patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jan 2016 21:44:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205938 --- Comment #4 from commit-hook@freebsd.org --- A commit references this bug: Author: pfg Date: Thu Jan 7 21:43:43 UTC 2016 New revision: 293370 URL: https://svnweb.freebsd.org/changeset/base/293370 Log: ext2fs: reading mmaped file in Ext4 causes panic Always call brelse(path.ep_bp), fixing reading EXT4 files using mmap(). Patch by Damjan Jovanovic. PR: 205938 MFC after: 1 week Changes: head/sys/fs/ext2fs/ext2_bmap.c --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Thu Jan 7 22:30:36 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 735C7A67498 for ; Thu, 7 Jan 2016 22:30:36 +0000 (UTC) (envelope-from case@SDF.ORG) Received: from sdf.lonestar.org (mx.sdf.org [192.94.73.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx.sdf.org", Issuer "SDF.ORG" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 425121B7F for ; Thu, 7 Jan 2016 22:30:35 +0000 (UTC) (envelope-from case@SDF.ORG) Received: from otaku.freeshell.org (IDENT:case@otaku.freeshell.org [192.94.73.9]) by sdf.lonestar.org (8.15.2/8.14.5) with ESMTPS id u07MS7Dg017875 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256 bits) verified NO) for ; Thu, 7 Jan 2016 22:28:14 GMT Date: Thu, 7 Jan 2016 22:28:07 +0000 (UTC) From: John Case X-X-Sender: case@faeroes.freeshell.org To: freebsd-fs@freebsd.org Subject: Solaris 11 ZFS send to FreeBSD 10.2 ... Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jan 2016 22:30:36 -0000 I would like to ZFS send from Solaris 11 to FreeBSD 10.2. The FreeBSD 10.2 system is: ZFS filesystem version: 5 ZFS storage pool version: features support (5000) It looks like Solaris 11 systems run ZFS version 6 ... I *think* that this is not supposed to work, but I wonder if there is some way to workaround the version differences and make it work ? Thanks. From owner-freebsd-fs@freebsd.org Fri Jan 8 00:23:30 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BD6EBA652EC for ; Fri, 8 Jan 2016 00:23:30 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-io0-x22b.google.com (mail-io0-x22b.google.com [IPv6:2607:f8b0:4001:c06::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8FE711D30 for ; Fri, 8 Jan 2016 00:23:30 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by mail-io0-x22b.google.com with SMTP id q21so270708290iod.0 for ; Thu, 07 Jan 2016 16:23:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=q0NKlRkxTeXfkhfCdwzTcIikDqQ2+rDZU6SQMjNJwI4=; b=rPhfFKXLQObQtmdziB2lSepqH/zefpNlSrD7xTHLcFs4W8SBpa50lm2pqWOtQXgzJy 9j1xP6RF/37Pde85023g8uesmyXiejspSJfzLkL3C3Kpg8kjzH5oYu44ay36LsyRIh4h HkVYcZg97WZ6tjRg8CSgaDIbwSYSUkct8A4nuzDBvolBMfSenb9jKB26cEYDl/iAncBx OaAVsEQR2ebgPuic3XWfoqtwMFXbBavftTxT5KWvvgpaHCg6R6iF8lRK2H6LQes5cOHE ErD/ye9b0nfNhHPq+m72UZ3fPvunN0nvNYV5KIFCacdBXPDChxD3V2lYEtUjb52hKYnx 6Msg== MIME-Version: 1.0 X-Received: by 10.107.18.219 with SMTP id 88mr94941079ios.130.1452212610061; Thu, 07 Jan 2016 16:23:30 -0800 (PST) Received: by 10.107.35.69 with HTTP; Thu, 7 Jan 2016 16:23:30 -0800 (PST) In-Reply-To: References: Date: Thu, 7 Jan 2016 16:23:30 -0800 Message-ID: Subject: Re: Solaris 11 ZFS send to FreeBSD 10.2 ... From: Freddie Cash To: John Case Cc: FreeBSD Filesystems Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2016 00:23:30 -0000 On Thu, Jan 7, 2016 at 2:28 PM, John Case wrote: > > I would like to ZFS send from Solaris 11 to FreeBSD 10.2. > > The FreeBSD 10.2 system is: > > ZFS filesystem version: 5 > ZFS storage pool version: features support (5000) > > It looks like Solaris 11 systems run ZFS version 6 ... > > I *think* that this is not supposed to work, but I wonder if there is som= e > way to workaround the version differences and make it work ? > > Thanks. > =E2=80=8BWhen you create the pool on Solaris, pass along the version flag y= ou want to use (ZFSv28 is the last version in common with non-Solaris systems). When you create the pool on FreeBSD, pass along the version flag you want to use (ZFSv28). That way, the pools are synced in their versions, and will support the same featureset, and you will be able to send/recv between them. =E2=80=8B(And you will constantly get nagged in "zpool status" output about= your pool version not being the latest and greatest. Would be nice to shut that up somehow!)=E2=80=8B --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@freebsd.org Fri Jan 8 02:34:40 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F0668A6629E for ; Fri, 8 Jan 2016 02:34:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E227A1C59 for ; Fri, 8 Jan 2016 02:34:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u082YeDH022218 for ; Fri, 8 Jan 2016 02:34:40 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205938] [ext2fs][patch][panic] EXT4: reading mmaped file causes panic because struct buf leaks Date: Fri, 08 Jan 2016 02:34:40 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: crash, patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: see_also Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2016 02:34:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205938 Pedro F. Giffuni changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.freebsd.org/bu | |gzilla/show_bug.cgi?id=3D2= 059 | |32 --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Fri Jan 8 03:52:57 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1CA5EA67E6B for ; Fri, 8 Jan 2016 03:52:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 067F01B03 for ; Fri, 8 Jan 2016 03:52:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u083qupR013722 for ; Fri, 8 Jan 2016 03:52:56 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205816] [ext2fs] [patch] EXT4 sparse blocks unsupported, contain garbage when read Date: Fri, 08 Jan 2016 03:52:55 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: damjan.jov@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.isobsolete attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2016 03:52:57 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205816 Damjan Jovanovic changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #164979|0 |1 is obsolete| | --- Comment #4 from Damjan Jovanovic --- Created attachment 165234 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D165234&action= =3Dedit preliminary patch to implement support for EXT4 sparse files, version 2 New version of the patch, rebased against the latest CURRENT. Should work, = but could be more efficient, eg. this case will be slow: path->ep_sparse_ext.e_len =3D 1; // FIXME: where does it end? --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Fri Jan 8 04:42:32 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A91E7A6604E for ; Fri, 8 Jan 2016 04:42:32 +0000 (UTC) (envelope-from raghavendra.hg@gmail.com) Received: from mail-pf0-x22a.google.com (mail-pf0-x22a.google.com [IPv6:2607:f8b0:400e:c00::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7EFD512EF for ; Fri, 8 Jan 2016 04:42:32 +0000 (UTC) (envelope-from raghavendra.hg@gmail.com) Received: by mail-pf0-x22a.google.com with SMTP id 65so3943568pff.2 for ; Thu, 07 Jan 2016 20:42:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=kPqpF9SRtW8WW3/pVLEFqOuZ/7tAsy0DyRQSYlHqMZc=; b=T5kSnrCxN3NiMN6Z5oHMKhXHWkHBIgpNvh2tfMzQn+QpTY+8jQc/ufcjGMPP8VjkZO /ODPRY+5zoq+R9DdjvqxFeEbiTJIOmHrlkMXK0p1XrG3OZoRDTUuzyoMpdtbZ23lWqkD mAuOlIKlKqIJeFMnZ0oCfUoA384iBUNE4cFNmViMFhzELUmyYaqhC6iffOHOvEuOokEF 2Z6I4+ZjlARJ7+weqe7kgaENs6iPIBJjJY4wgGqBgo5msJhwSkQKKOkGm5Jd8Gapvmn+ iWLMmP1JcaVgq75i5WJkVXMT24cvDKnIUnDNbMoT4ZLgSwfOuEjjkaT3S5m1VqzQtTs+ nbCg== MIME-Version: 1.0 X-Received: by 10.98.67.212 with SMTP id l81mr1586380pfi.90.1452228152089; Thu, 07 Jan 2016 20:42:32 -0800 (PST) Sender: raghavendra.hg@gmail.com Received: by 10.66.76.229 with HTTP; Thu, 7 Jan 2016 20:42:32 -0800 (PST) In-Reply-To: <1083933309.146084334.1451517977647.JavaMail.zimbra@uoguelph.ca> References: <571237035.145690509.1451437960464.JavaMail.zimbra@uoguelph.ca> <20151230103152.GS13942@ndevos-x240.usersys.redhat.com> <2D8C2729-D556-479B-B4E2-66E1BB222F41@ixsystems.com> <1083933309.146084334.1451517977647.JavaMail.zimbra@uoguelph.ca> Date: Fri, 8 Jan 2016 10:12:32 +0530 X-Google-Sender-Auth: 952Gb87wSa3jndUZyont4yAojH4 Message-ID: Subject: Re: [Gluster-devel] FreeBSD port of GlusterFS racks up a lot of CPU usage From: Raghavendra G To: Rick Macklem Cc: Hubbard Jordan , freebsd-fs , Gluster Devel Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2016 04:42:32 -0000 Sorry for the delayed reply. Had missed out this mail. Please find my comments inlined. On Thu, Dec 31, 2015 at 4:56 AM, Rick Macklem wrote: > Jordan Hubbard wrote: > > > > > On Dec 30, 2015, at 2:31 AM, Niels de Vos wrote: > > > > > >> I'm guessing that Linux uses the event-epoll stuff instead of > event-poll, > > >> so it wouldn't exhibit this. Is that correct? > > > > > > Well, both. most (if not all) Linux builds will use event-poll. But, > > > that calls epoll_wait() with a timeout of 1 millisecond as well. > > > > > >> Thanks for any information on this, rick > > >> ps: I am tempted to just crank the timeout of 1msec up to 10 or > 20msec. > > > > > > Yes, that is probably what I would do too. And have both poll functio= ns > > > use the same timeout, have it defined in libglusterfs/src/event.h. We > > > could make it a configurable option too, but I do not think it is ver= y > > > useful to have. > > > > I guess this begs the question - what=E2=80=99s the actual purpose of p= olling > for an > > event with a 1 millisecond timeout? If it was some sort of heartbeat > check, > > one might imagine that would be better served by a timer with nothing > close > > to 1 millisecond as an interval (that would be one seriously aggressive > > heartbeat) and if filesystem events are incoming that glusterfs needs t= o > > respond to, why timeout at all? > > > If I understand the code (I probably don't) the timeout allows the loop > to call a function that may add new fd's to be polled. (If I'm right, > the new ones might not get serviced.) > Yes, that's correct. Since in poll we pass the fds to be polled in an array as an argument, the only place where we can add/remove fds to be polled is at the time we call poll sycall. To make adding/removing fds from polling to be more responsive, poll timeouts "frequently enough". The trade-off we are considering here is between: 1. Number of calls to poll vs 2. Responsiveness of adding/removing a new fd from polling. For clients, there is not much change of the list of fds that are polled. However, for bricks/server this list can vary frequently as new clients are connected/disconnected. Since epoll provides a way to add new fds for polling while an epoll_wait is in progress (unlike poll), the timeout of epoll_wait is infinite. Also note that on systems where both epoll and poll are available, epoll is preferred over poll. > I'll post once I've tried a longer timeout and if it seems ok, I will > put it in the Redhat bugs database (as mentioned in the last post). > In its current form, it's fine for testing. > > > I also have a broader question to go with the specific one: We (at > > iXsystems) were attempting to engage with some of the Red Hat folks bac= k > > when the FreeBSD port was first done, in the hope of getting it more > > =E2=80=9Cofficially supported=E2=80=9D for FreeBSD and perhaps even don= ating some more > > serious stress-testing and integration work for it, but when those Red > Hat > > folks moved on we lost continuity and the effort stalled. Who at Red H= at > > would / could we work with in getting this back on track? We=E2=80=99d= like to > > integrate glusterfs with FreeNAS 10, and in fact have already done so b= ut > > it=E2=80=99s still early days and we=E2=80=99re not even really sure wh= at we have yet. > > > Just fyi..sofar, working with FreeBSD11/head and the port of 3.7.6 (the > port tarball > is in FreeBSD PR#194409), the only GlusterFS problem I've encountered is > the above one. I'm not sure why this isn't in /usr/ports, but that would = be > nice as it might get more people trying it. (I'm a src comitter, but not = a > ports one.) > > However, I have several patches for the FreeBSD fuse interface and for > a mount_glusterfs mount to work ok you need a couple of them. > 1 - When an open decides to do DIRECT_IO after the file has done buffer > cache I/O the buffer cache needs to be invalidated so you don't get > stale cached data. > 2 - For a WRONLY write, you need to force DIRECT_IO (or do a read/write > open). > If you don't do this, the buffer cache code will get stuck when tryin= g > to read a block in before writing a partial block. (I think this is > what FreeBSD PR#194293 is caused by.) > > Because I won't be able to do svn until April, these patches won't make i= t > into head for a while, but they will both be in PR#194293 within hours. > > The others add features like extended attributes, advisory byte range > locking > and the changes needed to export the fuse/glusterfs mount via the FreeBSD > kernel nfsd. If anyone wants/needs these patches, email and I can send yo= u > them. > > A bit off your topic, but until you have the fixes for FreeBSD fuse, you > probably can't do a lot of serious testing. > (I don't know, but I'd guess that FreeNAS has about the same fuse module > code as FreeBSD's head, since it hasn't been changed much in head > recently.) > > Thanks everyone for your help with this, rick > > > Thanks, > > > > - Jordan > > > > > _______________________________________________ > Gluster-devel mailing list > Gluster-devel@gluster.org > http://www.gluster.org/mailman/listinfo/gluster-devel > --=20 Raghavendra G From owner-freebsd-fs@freebsd.org Fri Jan 8 08:11:46 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 437E8A67697 for ; Fri, 8 Jan 2016 08:11:46 +0000 (UTC) (envelope-from xhernandez@datalab.es) Received: from dlbex1.datalab.es (dlbex1.datalab.es [192.146.172.55]) by mx1.freebsd.org (Postfix) with ESMTP id C2B0B12E0 for ; Fri, 8 Jan 2016 08:11:44 +0000 (UTC) (envelope-from xhernandez@datalab.es) Received: from xavih.datalab.es (unknown [192.168.200.206]) by dlbex1.datalab.es (Postfix) with ESMTP id 4E65C404AB; Fri, 8 Jan 2016 09:02:16 +0100 (CET) Subject: Re: [Gluster-devel] FreeBSD port of GlusterFS racks up a lot of CPU usage To: Raghavendra G , Rick Macklem References: <571237035.145690509.1451437960464.JavaMail.zimbra@uoguelph.ca> <20151230103152.GS13942@ndevos-x240.usersys.redhat.com> <2D8C2729-D556-479B-B4E2-66E1BB222F41@ixsystems.com> <1083933309.146084334.1451517977647.JavaMail.zimbra@uoguelph.ca> Cc: freebsd-fs , Gluster Devel , Hubbard Jordan From: Xavier Hernandez Message-ID: <568F6D07.6070500@datalab.es> Date: Fri, 8 Jan 2016 09:02:15 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2016 08:11:46 -0000 On 08/01/16 05:42, Raghavendra G wrote: > Sorry for the delayed reply. Had missed out this mail. Please find my > comments inlined. > > On Thu, Dec 31, 2015 at 4:56 AM, Rick Macklem > wrote: > > Jordan Hubbard wrote: > > > > > On Dec 30, 2015, at 2:31 AM, Niels de Vos > wrote: > > > > > >> I'm guessing that Linux uses the event-epoll stuff instead of event-poll, > > >> so it wouldn't exhibit this. Is that correct? > > > > > > Well, both. most (if not all) Linux builds will use event-poll. But, > > > that calls epoll_wait() with a timeout of 1 millisecond as well. > > > > > >> Thanks for any information on this, rick > > >> ps: I am tempted to just crank the timeout of 1msec up to 10 or 20msec. > > > > > > Yes, that is probably what I would do too. And have both poll functions > > > use the same timeout, have it defined in libglusterfs/src/event.h. We > > > could make it a configurable option too, but I do not think it is very > > > useful to have. > > > > I guess this begs the question - what’s the actual purpose of polling for an > > event with a 1 millisecond timeout? If it was some sort of heartbeat check, > > one might imagine that would be better served by a timer with nothing close > > to 1 millisecond as an interval (that would be one seriously aggressive > > heartbeat) and if filesystem events are incoming that glusterfs needs to > > respond to, why timeout at all? > > > If I understand the code (I probably don't) the timeout allows the loop > to call a function that may add new fd's to be polled. (If I'm right, > the new ones might not get serviced.) > > > Yes, that's correct. Since in poll we pass the fds to be polled in an > array as an argument, the only place where we can add/remove fds to be > polled is at the time we call poll sycall. To make adding/removing fds > from polling to be more responsive, poll timeouts "frequently enough". > The trade-off we are considering here is between: > > 1. Number of calls to poll > vs > 2. Responsiveness of adding/removing a new fd from polling. > > For clients, there is not much change of the list of fds that are > polled. However, for bricks/server this list can vary frequently as new > clients are connected/disconnected. > > Since epoll provides a way to add new fds for polling while an > epoll_wait is in progress (unlike poll), the timeout of epoll_wait is > infinite. Also note that on systems where both epoll and poll are > available, epoll is preferred over poll. I don't know anything about gluster's poll implementation so I may be totally wrong, but would it be possible to use an eventfd (or a pipe if eventfd is not supported) to signal the need to add more file descriptors to the poll call ? The poll call should listen on this new fd. When we need to change the fd list, we should simply write to the eventfd or pipe from another thread. This will cause the poll call to return and we will be able to change the fd list without having a short timeout nor having to decide on any trade-off. Just an idea... Xavi > > > I'll post once I've tried a longer timeout and if it seems ok, I will > put it in the Redhat bugs database (as mentioned in the last post). > In its current form, it's fine for testing. > > > I also have a broader question to go with the specific one: We (at > > iXsystems) were attempting to engage with some of the Red Hat folks back > > when the FreeBSD port was first done, in the hope of getting it more > > “officially supported” for FreeBSD and perhaps even donating some more > > serious stress-testing and integration work for it, but when those Red Hat > > folks moved on we lost continuity and the effort stalled. Who at Red Hat > > would / could we work with in getting this back on track? We’d like to > > integrate glusterfs with FreeNAS 10, and in fact have already done so but > > it’s still early days and we’re not even really sure what we have yet. > > > Just fyi..sofar, working with FreeBSD11/head and the port of 3.7.6 > (the port tarball > is in FreeBSD PR#194409), the only GlusterFS problem I've encountered is > the above one. I'm not sure why this isn't in /usr/ports, but that > would be > nice as it might get more people trying it. (I'm a src comitter, but > not a > ports one.) > > However, I have several patches for the FreeBSD fuse interface and for > a mount_glusterfs mount to work ok you need a couple of them. > 1 - When an open decides to do DIRECT_IO after the file has done buffer > cache I/O the buffer cache needs to be invalidated so you don't get > stale cached data. > 2 - For a WRONLY write, you need to force DIRECT_IO (or do a > read/write open). > If you don't do this, the buffer cache code will get stuck when > trying > to read a block in before writing a partial block. (I think this is > what FreeBSD PR#194293 is caused by.) > > Because I won't be able to do svn until April, these patches won't > make it > into head for a while, but they will both be in PR#194293 within hours. > > The others add features like extended attributes, advisory byte > range locking > and the changes needed to export the fuse/glusterfs mount via the > FreeBSD > kernel nfsd. If anyone wants/needs these patches, email and I can > send you > them. > > A bit off your topic, but until you have the fixes for FreeBSD fuse, you > probably can't do a lot of serious testing. > (I don't know, but I'd guess that FreeNAS has about the same fuse module > code as FreeBSD's head, since it hasn't been changed much in head > recently.) > > Thanks everyone for your help with this, rick > > > Thanks, > > > > - Jordan > > > > > _______________________________________________ > Gluster-devel mailing list > Gluster-devel@gluster.org > http://www.gluster.org/mailman/listinfo/gluster-devel > > > > > -- > Raghavendra G > > > _______________________________________________ > Gluster-devel mailing list > Gluster-devel@gluster.org > http://www.gluster.org/mailman/listinfo/gluster-devel > From owner-freebsd-fs@freebsd.org Fri Jan 8 08:22:09 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C0E8BA67A40 for ; Fri, 8 Jan 2016 08:22:09 +0000 (UTC) (envelope-from raghavendra.hg@gmail.com) Received: from mail-pa0-x22d.google.com (mail-pa0-x22d.google.com [IPv6:2607:f8b0:400e:c03::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9EC1119F0 for ; Fri, 8 Jan 2016 08:22:09 +0000 (UTC) (envelope-from raghavendra.hg@gmail.com) Received: by mail-pa0-x22d.google.com with SMTP id ho8so17803045pac.2 for ; Fri, 08 Jan 2016 00:22:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Uw5UiKy00yLqqWCkOm+zdzTikhjrzX2Xudw1dcsbaTo=; b=P8Nzx7EfK/DhFzJZiDwCELOyIK4xud//jehd3N5f53qsvJbXAvdQF2ep6MB4qfNeJV 8HUjGtJS111VrSz2foTMiRNVnRMKXAJ2oVRUmf/qGyuT07f+F+8gbd+QbqvXPJkgldQ9 3LH0hK6iBqVpCFpyG8oG465CpK9TdP0Jjc8qqk16CBDzDgltCUQor4F4HGk28Lj+pqv8 v95tdd6HpQ6YPS8vyUTKGCqvsAGdyqrQEXrkx4SX8hL/pdgd4NP4Vkq/+zdnTN68n48V 1nNXYKmx/IsWLNM6Z94gTLNVK6xnckYsgXHd3xn9Eb0qkRK97Q2eylMZ8lxc+16xBTQp 7MFQ== MIME-Version: 1.0 X-Received: by 10.66.218.225 with SMTP id pj1mr153155860pac.40.1452241329099; Fri, 08 Jan 2016 00:22:09 -0800 (PST) Sender: raghavendra.hg@gmail.com Received: by 10.66.76.229 with HTTP; Fri, 8 Jan 2016 00:22:09 -0800 (PST) In-Reply-To: <568F6D07.6070500@datalab.es> References: <571237035.145690509.1451437960464.JavaMail.zimbra@uoguelph.ca> <20151230103152.GS13942@ndevos-x240.usersys.redhat.com> <2D8C2729-D556-479B-B4E2-66E1BB222F41@ixsystems.com> <1083933309.146084334.1451517977647.JavaMail.zimbra@uoguelph.ca> <568F6D07.6070500@datalab.es> Date: Fri, 8 Jan 2016 13:52:09 +0530 X-Google-Sender-Auth: pFn_1I4NhtXCAWaBxhJIR3W34jw Message-ID: Subject: Re: [Gluster-devel] FreeBSD port of GlusterFS racks up a lot of CPU usage From: Raghavendra G To: Xavier Hernandez Cc: Rick Macklem , freebsd-fs , Hubbard Jordan , Gluster Devel Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2016 08:22:09 -0000 On Fri, Jan 8, 2016 at 1:32 PM, Xavier Hernandez wrote: > > On 08/01/16 05:42, Raghavendra G wrote: > >> Sorry for the delayed reply. Had missed out this mail. Please find my >> comments inlined. >> >> On Thu, Dec 31, 2015 at 4:56 AM, Rick Macklem > > wrote: >> >> Jordan Hubbard wrote: >> > >> > > On Dec 30, 2015, at 2:31 AM, Niels de Vos > > wrote: >> > > >> > >> I'm guessing that Linux uses the event-epoll stuff instead of >> event-poll, >> > >> so it wouldn't exhibit this. Is that correct? >> > > >> > > Well, both. most (if not all) Linux builds will use event-poll. >> But, >> > > that calls epoll_wait() with a timeout of 1 millisecond as well. >> > > >> > >> Thanks for any information on this, rick >> > >> ps: I am tempted to just crank the timeout of 1msec up to 10 or >> 20msec. >> > > >> > > Yes, that is probably what I would do too. And have both poll >> functions >> > > use the same timeout, have it defined in >> libglusterfs/src/event.h. We >> > > could make it a configurable option too, but I do not think it i= s >> very >> > > useful to have. >> > >> > I guess this begs the question - what=E2=80=99s the actual purpose= of >> polling for an >> > event with a 1 millisecond timeout? If it was some sort of >> heartbeat check, >> > one might imagine that would be better served by a timer with >> nothing close >> > to 1 millisecond as an interval (that would be one seriously >> aggressive >> > heartbeat) and if filesystem events are incoming that glusterfs >> needs to >> > respond to, why timeout at all? >> > >> If I understand the code (I probably don't) the timeout allows the >> loop >> to call a function that may add new fd's to be polled. (If I'm right= , >> the new ones might not get serviced.) >> >> >> Yes, that's correct. Since in poll we pass the fds to be polled in an >> array as an argument, the only place where we can add/remove fds to be >> polled is at the time we call poll sycall. To make adding/removing fds >> from polling to be more responsive, poll timeouts "frequently enough". >> The trade-off we are considering here is between: >> >> 1. Number of calls to poll >> vs >> 2. Responsiveness of adding/removing a new fd from polling. >> >> For clients, there is not much change of the list of fds that are >> polled. However, for bricks/server this list can vary frequently as new >> clients are connected/disconnected. >> >> Since epoll provides a way to add new fds for polling while an >> epoll_wait is in progress (unlike poll), the timeout of epoll_wait is >> infinite. Also note that on systems where both epoll and poll are >> available, epoll is preferred over poll. >> > > I don't know anything about gluster's poll implementation so I may be > totally wrong, but would it be possible to use an eventfd (or a pipe if > eventfd is not supported) to signal the need to add more file descriptors > to the poll call ? > > The poll call should listen on this new fd. When we need to change the fd > list, we should simply write to the eventfd or pipe from another thread. > This will cause the poll call to return and we will be able to change the > fd list without having a short timeout nor having to decide on any > trade-off. > Thats a nice idea. Based on my understanding of why timeouts are being used, this approach can work. > > Just an idea... > > Xavi > > >> >> I'll post once I've tried a longer timeout and if it seems ok, I wil= l >> put it in the Redhat bugs database (as mentioned in the last post). >> In its current form, it's fine for testing. >> >> > I also have a broader question to go with the specific one: We (a= t >> > iXsystems) were attempting to engage with some of the Red Hat folk= s >> back >> > when the FreeBSD port was first done, in the hope of getting it mo= re >> > =E2=80=9Cofficially supported=E2=80=9D for FreeBSD and perhaps eve= n donating some >> more >> > serious stress-testing and integration work for it, but when those >> Red Hat >> > folks moved on we lost continuity and the effort stalled. Who at >> Red Hat >> > would / could we work with in getting this back on track? We=E2= =80=99d >> like to >> > integrate glusterfs with FreeNAS 10, and in fact have already done >> so but >> > it=E2=80=99s still early days and we=E2=80=99re not even really su= re what we have >> yet. >> > >> Just fyi..sofar, working with FreeBSD11/head and the port of 3.7.6 >> (the port tarball >> is in FreeBSD PR#194409), the only GlusterFS problem I've encountere= d >> is >> the above one. I'm not sure why this isn't in /usr/ports, but that >> would be >> nice as it might get more people trying it. (I'm a src comitter, but >> not a >> ports one.) >> >> However, I have several patches for the FreeBSD fuse interface and f= or >> a mount_glusterfs mount to work ok you need a couple of them. >> 1 - When an open decides to do DIRECT_IO after the file has done >> buffer >> cache I/O the buffer cache needs to be invalidated so you don't >> get >> stale cached data. >> 2 - For a WRONLY write, you need to force DIRECT_IO (or do a >> read/write open). >> If you don't do this, the buffer cache code will get stuck when >> trying >> to read a block in before writing a partial block. (I think thi= s >> is >> what FreeBSD PR#194293 is caused by.) >> >> Because I won't be able to do svn until April, these patches won't >> make it >> into head for a while, but they will both be in PR#194293 within >> hours. >> >> The others add features like extended attributes, advisory byte >> range locking >> and the changes needed to export the fuse/glusterfs mount via the >> FreeBSD >> kernel nfsd. If anyone wants/needs these patches, email and I can >> send you >> them. >> >> A bit off your topic, but until you have the fixes for FreeBSD fuse, >> you >> probably can't do a lot of serious testing. >> (I don't know, but I'd guess that FreeNAS has about the same fuse >> module >> code as FreeBSD's head, since it hasn't been changed much in head >> recently.) >> >> Thanks everyone for your help with this, rick >> >> > Thanks, >> > >> > - Jordan >> > >> > >> _______________________________________________ >> Gluster-devel mailing list >> Gluster-devel@gluster.org >> http://www.gluster.org/mailman/listinfo/gluster-devel >> >> >> >> >> -- >> Raghavendra G >> >> >> _______________________________________________ >> Gluster-devel mailing list >> Gluster-devel@gluster.org >> http://www.gluster.org/mailman/listinfo/gluster-devel >> >> _______________________________________________ > Gluster-devel mailing list > Gluster-devel@gluster.org > http://www.gluster.org/mailman/listinfo/gluster-devel > --=20 Raghavendra G From owner-freebsd-fs@freebsd.org Fri Jan 8 10:17:32 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4886A675FD for ; Fri, 8 Jan 2016 10:17:32 +0000 (UTC) (envelope-from jdarcy@redhat.com) Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mx1.redhat.com", Issuer "DigiCert SHA2 Extended Validation Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A78A51FE0 for ; Fri, 8 Jan 2016 10:17:32 +0000 (UTC) (envelope-from jdarcy@redhat.com) Received: from zmail12.collab.prod.int.phx2.redhat.com (zmail12.collab.prod.int.phx2.redhat.com [10.5.83.14]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id u08AHU5O024635; Fri, 8 Jan 2016 05:17:30 -0500 Date: Fri, 8 Jan 2016 05:17:30 -0500 (EST) From: Jeff Darcy To: Raghavendra G Cc: Xavier Hernandez , freebsd-fs , Gluster Devel , Hubbard Jordan Message-ID: <1924941590.6473225.1452248249994.JavaMail.zimbra@redhat.com> In-Reply-To: References: <571237035.145690509.1451437960464.JavaMail.zimbra@uoguelph.ca> <20151230103152.GS13942@ndevos-x240.usersys.redhat.com> <2D8C2729-D556-479B-B4E2-66E1BB222F41@ixsystems.com> <1083933309.146084334.1451517977647.JavaMail.zimbra@uoguelph.ca> <568F6D07.6070500@datalab.es> Subject: Re: [Gluster-devel] FreeBSD port of GlusterFS racks up a lot of CPU usage MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.3.113.99] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - GC47 ([unknown])/8.0.6_GA_5922) Thread-Topic: FreeBSD port of GlusterFS racks up a lot of CPU usage Thread-Index: nzgvBLPPgcBXuRsf6GlZU17foVfxCA== X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2016 10:17:32 -0000 > > I don't know anything about gluster's poll implementation so I may > > be totally wrong, but would it be possible to use an eventfd (or a > > pipe if eventfd is not supported) to signal the need to add more > > file descriptors to the poll call ? > > > > > > The poll call should listen on this new fd. When we need to change > > the fd list, we should simply write to the eventfd or pipe from > > another thread. This will cause the poll call to return and we will > > be able to change the fd list without having a short timeout nor > > having to decide on any trade-off. > > > Thats a nice idea. Based on my understanding of why timeouts are being > used, this approach can work. The own-thread code which preceded the current poll implementation did something similar, using a pipe fd to be woken up for new *outgoing* messages. That code still exists, and might provide some insight into how to do this for the current poll code. From owner-freebsd-fs@freebsd.org Fri Jan 8 11:47:12 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B57EFA671B9 for ; Fri, 8 Jan 2016 11:47:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8C6B519C5 for ; Fri, 8 Jan 2016 11:47:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u08BlCAC070234 for ; Fri, 8 Jan 2016 11:47:12 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 204764] Filesystem deadlock, process in vodead state Date: Fri, 08 Jan 2016 11:47:12 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: daniel@blodan.se X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2016 11:47:12 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204764 --- Comment #1 from daniel@blodan.se --- I'm actually having the exact same problem, it started when we upgraded to = 10.2 (we were running 10.1 before and it never happened there) and it happens ab= out once a month and after a reboot it's good for about another month. It seems that its the periodic find that always gets stuck and after that a= lot of other i/o procs get stuck Here's the periodic find: 23293 100188 find - mi_switch+0xe1 sleepq_wait+0= x3a _sleep+0x287 vnode_create_vobject+0x100 ufs_lookup_ino+0xa0 VOP_CACHEDLOOKUP_APV+0xa1 vfs_cache_lookup+0xd6 VOP_LOOKUP_APV+0xa1 lookup+0x5a1 namei+0x4d4 kern_statat_vnhook+0xae sys_fstatat+0x2c amd64_syscall+0x357 Xfast_syscall+0xfb And here's a rsync call thats stuck: 83577 100644 rsync - mi_switch+0xe1 sleepq_wait+0= x3a _sleep+0x287 vnode_create_vobject+0x100 ufs_open+0x6d VOP_OPEN_APV+0xa1 vn_open_vnode+0x234 vn_open_cred+0x33e kern_openat+0x26f amd64_syscall+0x357 Xfast_syscall+0xfb One difference in setup is that I'm running a mfi raid and not zfs [root@backup-se ~]# mfiutil show volumes mfi0 Volumes: Id Size Level Stripe State Cache Name mfid0 ( 12T) RAID-5 64K OPTIMAL Disabled mfid1 ( 931G) RAID-0 64K OPTIMAL Disabled Filesystem Size Used Avail Capacity Mounted on /dev/mfid1p2 898G 4.8G 821G 1% / /dev/mfid0p1 12T 8.9T 1.8T 83% /backups devfs 1.0K 1.0K 0B 100% /dev fdescfs 1.0K 1.0K 0B 100% /dev/fd --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Jan 8 12:56:19 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EB9D9A68CFE for ; Fri, 8 Jan 2016 12:56:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DC55E1CFB for ; Fri, 8 Jan 2016 12:56:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u08CuJ2p038365 for ; Fri, 8 Jan 2016 12:56:19 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205974] can't detach devices from zpool Date: Fri, 08 Jan 2016 12:56:19 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: smh@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Unable to Reproduce X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2016 12:56:20 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205974 Steven Hartland changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |Unable to Reproduce Status|New |Closed --- Comment #6 from Steven Hartland --- Thanks Nikolai, closing as no reproducible. If anyone has the same issue again please feel free to re-open. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Jan 8 13:58:48 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6D678A67277 for ; Fri, 8 Jan 2016 13:58:48 +0000 (UTC) (envelope-from kkeithle@redhat.com) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 57C481B1B for ; Fri, 8 Jan 2016 13:58:48 +0000 (UTC) (envelope-from kkeithle@redhat.com) Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (Postfix) with ESMTPS id 5A6AD693D3; Fri, 8 Jan 2016 13:58:47 +0000 (UTC) Received: from kkeithle.usersys.redhat.com (dhcp-41-176.bos.redhat.com [10.18.41.176]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u08DwkxW027498; Fri, 8 Jan 2016 08:58:46 -0500 Subject: Re: [Gluster-devel] FreeBSD port of GlusterFS racks up a lot of CPU usage To: Hubbard Jordan References: <571237035.145690509.1451437960464.JavaMail.zimbra@uoguelph.ca> <20151230103152.GS13942@ndevos-x240.usersys.redhat.com> <2D8C2729-D556-479B-B4E2-66E1BB222F41@ixsystems.com> Cc: Niels de Vos , freebsd-fs , gluster-devel@gluster.org, Amye Scavarda From: "Kaleb S. KEITHLEY" X-Enigmail-Draft-Status: N1110 Message-ID: <568FC096.3080308@redhat.com> Date: Fri, 8 Jan 2016 08:58:46 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <2D8C2729-D556-479B-B4E2-66E1BB222F41@ixsystems.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2016 13:58:48 -0000 On 12/30/2015 01:22 PM, Hubbard Jordan wrote: > I also have a broader question to go with the specific one: We > (at iXsystems) were attempting to engage with some of the Red Hat > folks back when the FreeBSD port was first done, in the hope of > getting it more “officially supported” for FreeBSD and perhaps even > donating some more serious stress-testing and integration work for > it, but when those Red Hat folks moved on we lost continuity and > the effort stalled. Who at Red Hat would / could we work with in > getting this back on track? We’d like to integrate glusterfs with > FreeNAS 10, and in fact have already done so but it’s still early > days and we’re not even really sure what we have yet. > Hi, To me, from a community standpoint, to be "officially supported" I'd venture to say that what it takes is being visibly involved in the project. That can take many forms, e.g., do regular builds on your platform, submit bug reports (to our bugzilla) and associated fixes (to our gerrit), implement and contribute new features, review other people's patches in gerrit, build packages for your platform, evangelize GlusterFS, answer questions in IRC and the mailing lists, etc., etc. Everything that goes on the community is done by volunteers. There are no Red Hat employees whose sole responsibility is to work on Community GlusterFS. (Excepting our Community Manager, Amye.) The Red Hat mantra is "upstream first" so every feature and every bug fix that Red Hat employees work on does indeed go into Community GlusterFS first; a lot does get done as a side effect of that policy, but nobody should take it for granted that _everything_ (or anything) will just get done. Nobody would say no to having serious stress testing and integration work. If it plugs into our current gerrit and jenkins infrastructure, so much the better. If there are people in your community who can help maintain and/or grow our infrastructure, we could use a lot of help there. With that level of involvement, I could imagine eventually FreeBSD having more of a, I don't know, for lack of a better word, 'standing' in the GlusterFS community. We do compile every patch on FreeBSD to ensure that we don't break that level of portability, but that's the extent of it. Maybe elevated to running regressions, as we do for NetBSD, which has a bit of a legacy standing in the community due to Emmanuel Dreyfus' long time participation. Anyway, that's my opinion. (Emphasis on my and opinion. Perhaps others will weigh in with their opinions.) I look forward to your involvement in the community. Look for us at FOSDEM, a couple of us will be there. -- Kaleb From owner-freebsd-fs@freebsd.org Fri Jan 8 14:32:39 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7B6E9A68077 for ; Fri, 8 Jan 2016 14:32:39 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4AE7D163D for ; Fri, 8 Jan 2016 14:32:39 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from freddy.simplesystems.org (freddy.simplesystems.org [65.66.246.65]) by blade.simplesystems.org (8.14.4+Sun/8.14.4) with ESMTP id u08EWVWO003591; Fri, 8 Jan 2016 08:32:31 -0600 (CST) Date: Fri, 8 Jan 2016 08:32:31 -0600 (CST) From: Bob Friesenhahn X-X-Sender: bfriesen@freddy.simplesystems.org To: Freddie Cash cc: John Case , FreeBSD Filesystems Subject: Re: Solaris 11 ZFS send to FreeBSD 10.2 ... In-Reply-To: Message-ID: References: User-Agent: Alpine 2.01 (GSO 1266 2009-07-14) MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (blade.simplesystems.org [65.66.246.90]); Fri, 08 Jan 2016 08:32:32 -0600 (CST) Content-Type: TEXT/PLAIN; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2016 14:32:39 -0000 On Thu, 7 Jan 2016, Freddie Cash wrote: > ​When you create the pool on Solaris, pass along the version flag you want > to use (ZFSv28 is the last version in common with non-Solaris systems). > > When you create the pool on FreeBSD, pass along the version flag you want > to use (ZFSv28). I have read that the filesystem version is most important and that although both claim to support version 5, version 4 or earlier may be necessary on the Solaris sending side due to a send stream change for version 5 in Solaris after the fork. The filesystem version can be specified when it is created. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From owner-freebsd-fs@freebsd.org Fri Jan 8 14:40:16 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C53BEA682DE for ; Fri, 8 Jan 2016 14:40:16 +0000 (UTC) (envelope-from kkeithle@redhat.com) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9E9751CDC for ; Fri, 8 Jan 2016 14:40:16 +0000 (UTC) (envelope-from kkeithle@redhat.com) Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) by mx1.redhat.com (Postfix) with ESMTPS id 52065A35A5; Fri, 8 Jan 2016 14:40:16 +0000 (UTC) Received: from kkeithle.usersys.redhat.com (dhcp-41-176.bos.redhat.com [10.18.41.176]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u08EeFGu027047; Fri, 8 Jan 2016 09:40:15 -0500 Subject: Re: [Gluster-devel] FreeBSD port of GlusterFS racks up a lot of CPU usage To: Hubbard Jordan References: <571237035.145690509.1451437960464.JavaMail.zimbra@uoguelph.ca> <20151230103152.GS13942@ndevos-x240.usersys.redhat.com> <2D8C2729-D556-479B-B4E2-66E1BB222F41@ixsystems.com> <568FC096.3080308@redhat.com> Cc: freebsd-fs , gluster-devel@gluster.org From: "Kaleb S. KEITHLEY" Message-ID: <568FCA4F.2060804@redhat.com> Date: Fri, 8 Jan 2016 09:40:15 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <568FC096.3080308@redhat.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2016 14:40:17 -0000 On 01/08/2016 08:58 AM, Kaleb S. KEITHLEY wrote: > On 12/30/2015 01:22 PM, Hubbard Jordan wrote: > >> I also have a broader question to go with the specific one: We >> (at iXsystems) were attempting to engage with some of the Red Hat >> folks back when the FreeBSD port was first done, in the hope of >> getting it more “officially supported” for FreeBSD and perhaps even >> donating some more serious stress-testing and integration work for >> it, but when those Red Hat folks moved on we lost continuity and >> the effort stalled. Who at Red Hat would / could we work with in >> getting this back on track? We’d like to integrate glusterfs with >> FreeNAS 10, and in fact have already done so but it’s still early >> days and we’re not even really sure what we have yet. >> > > Hi, > > To me, from a community standpoint, to be "officially supported" I'd > venture to say that what it takes is being visibly involved in the > project. That can take many forms, e.g., do regular builds on your > platform, submit bug reports (to our bugzilla) and associated fixes (to > our gerrit), implement and contribute new features, review other > people's patches in gerrit, build packages for your platform, evangelize > GlusterFS, answer questions in IRC and the mailing lists, etc., etc. > > Everything that goes on the community is done by volunteers. There are > no Red Hat employees whose sole responsibility is to work on Community > GlusterFS. (Excepting our Community Manager, Amye.) The Red Hat mantra > is "upstream first" so every feature and every bug fix that Red Hat > employees work on does indeed go into Community GlusterFS first; a lot > does get done as a side effect of that policy, but nobody should take it > for granted that _everything_ (or anything) will just get done. > > Nobody would say no to having serious stress testing and integration > work. If it plugs into our current gerrit and jenkins infrastructure, so > much the better. If there are people in your community who can help > maintain and/or grow our infrastructure, we could use a lot of help there. Just to be clear, by "our infrastructure" I mean Community GlusterFS infrastructure. > > With that level of involvement, I could imagine eventually FreeBSD > having more of a, I don't know, for lack of a better word, 'standing' in > the GlusterFS community. We do compile every patch on FreeBSD to ensure > that we don't break that level of portability, but that's the extent of > it. Maybe elevated to running regressions, as we do for NetBSD, which > has a bit of a legacy standing in the community due to Emmanuel Dreyfus' > long time participation. > > Anyway, that's my opinion. (Emphasis on my and opinion. Perhaps others > will weigh in with their opinions.) I look forward to your involvement > in the community. Look for us at FOSDEM, a couple of us will be there. > -- Kaleb From owner-freebsd-fs@freebsd.org Fri Jan 8 19:33:06 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9092DA68CBA for ; Fri, 8 Jan 2016 19:33:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 80D3D1232 for ; Fri, 8 Jan 2016 19:33:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u08JX6k8020365 for ; Fri, 8 Jan 2016 19:33:06 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 205816] [ext2fs] [patch] EXT4 sparse blocks unsupported, contain garbage when read Date: Fri, 08 Jan 2016 19:33:06 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: pfg@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pfg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2016 19:33:06 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205816 --- Comment #5 from Pedro F. Giffuni --- (In reply to Damjan Jovanovic from comment #4) I like this patch. The only change I'd do is dropping the (int) cast from the second argument in uiomove, but that comes from the original code. I will try that and hold a bit more for some code review from Zheng but I will probably commit it over the weekend if there is silence. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Sat Jan 9 02:00:07 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1FEFDA67E29 for ; Sat, 9 Jan 2016 02:00:07 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id AF7B315E3 for ; Sat, 9 Jan 2016 02:00:06 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:D2KIhRWK+QvKr+JgPuwVClqbM+HV8LGtZVwlr6E/grcLSJyIuqrYZhKCt8tkgFKBZ4jH8fUM07OQ6PC+HzRYqb+681k8M7V0HycfjssXmwFySOWkMmbcaMDQUiohAc5ZX0Vk9XzoeWJcGcL5ekGA6ibqtW1aJBzzOEJPK/jvHcaK1oLsh770o8WbSj4LrQT+SIs6FA+xowTVu5teqqpZAYF19CH0pGBVcf9d32JiKAHbtR/94sCt4MwrqHwI6LoJvvRNWqTifqk+UacQTHF/azh0t4XXskzhQAWBrlcVSG4H2k5KDwHf5wDSRJr9siLm8OF63X/JE9fxSOUOWD+hp4JiQxzshSJPYyQ8+WrUjsF1pL9crw+sowR/hYXdNtLGfMFid7/QKItJDVFKWdxcAmkYWtux X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CtBAA7aZBW/61jaINehAxtBohTs1mBZBgKhSNKAoFWEgEBAQEBAQEBgQmCLYIIAQEEAQEBIAQnIAsFCwIBCA4KERkCAgIlAQkmAgQIBwQBHASIDg6xNJA2AQEBAQEBAQEBAQEBAQEBAQEBEgUEgi+EJ4R/hDcBARsZgwaBSQWOMIhdgnSCT4UphEiHbYUyjkwCKQQ3ghEcgXsgNAeEGDqBCAEBAQ X-IronPort-AV: E=Sophos;i="5.20,541,1444708800"; d="scan'208";a="260364885" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 08 Jan 2016 21:00:00 -0500 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id E4DF615F55D; Fri, 8 Jan 2016 20:59:59 -0500 (EST) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 7fOr1uN6_XPd; Fri, 8 Jan 2016 20:59:59 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 5557615F565; Fri, 8 Jan 2016 20:59:59 -0500 (EST) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id HHb_4WZqdWC7; Fri, 8 Jan 2016 20:59:59 -0500 (EST) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 342EA15F55D; Fri, 8 Jan 2016 20:59:59 -0500 (EST) Date: Fri, 8 Jan 2016 20:59:59 -0500 (EST) From: Rick Macklem To: Jeff Darcy Cc: Raghavendra G , freebsd-fs , Hubbard Jordan , Xavier Hernandez , Gluster Devel Message-ID: <981529129.154244852.1452304799182.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <1924941590.6473225.1452248249994.JavaMail.zimbra@redhat.com> References: <571237035.145690509.1451437960464.JavaMail.zimbra@uoguelph.ca> <20151230103152.GS13942@ndevos-x240.usersys.redhat.com> <2D8C2729-D556-479B-B4E2-66E1BB222F41@ixsystems.com> <1083933309.146084334.1451517977647.JavaMail.zimbra@uoguelph.ca> <568F6D07.6070500@datalab.es> <1924941590.6473225.1452248249994.JavaMail.zimbra@redhat.com> Subject: Re: [Gluster-devel] FreeBSD port of GlusterFS racks up a lot of CPU usage MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_154244850_659138303.1452304799181" X-Originating-IP: [172.17.95.10] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF43 (Win)/8.0.9_GA_6191) Thread-Topic: FreeBSD port of GlusterFS racks up a lot of CPU usage Thread-Index: nzgvBLPPgcBXuRsf6GlZU17foVfxCE+M3nAj X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jan 2016 02:00:07 -0000 ------=_Part_154244850_659138303.1452304799181 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Jeff Darcy wrote: > > > I don't know anything about gluster's poll implementation so I may > > > be totally wrong, but would it be possible to use an eventfd (or a > > > pipe if eventfd is not supported) to signal the need to add more > > > file descriptors to the poll call ? > > > > > > > > > The poll call should listen on this new fd. When we need to change > > > the fd list, we should simply write to the eventfd or pipe from > > > another thread. This will cause the poll call to return and we will > > > be able to change the fd list without having a short timeout nor > > > having to decide on any trade-off. > > > > > > Thats a nice idea. Based on my understanding of why timeouts are being > > used, this approach can work. > > The own-thread code which preceded the current poll implementation did > something similar, using a pipe fd to be woken up for new *outgoing* > messages. That code still exists, and might provide some insight into > how to do this for the current poll code. I took a look at event-poll.c and found something interesting... - A pipe called "breaker" is already set up by event_pool_new_poll() and closed by event_pool_destroy_poll(), however it never gets used for anything. So, I added a few lines of code that writes a byte to it whenever the list of file descriptors is changed and read when poll() returns, if its revents is set. I also changed the timeout to -1 (infinity) and it seems to work for a trivial test. --> Btw, I also noticed the "changed" variable gets set to 1 on a change, but never reset to 0. I didn't change this, since it looks "racey". (ie. I think you could easily get a race between a thread that clears it and one that adds a new fd.) A slightly safer version of the patch would set a long (100msec ??) timeout instead of -1. Anyhow, I've attached the patch in case anyone would like to try it and will create a bug report for this after I've had more time to test it. (I only use a couple of laptops, so my testing will be minimal.) Thanks for all the help, rick > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > ------=_Part_154244850_659138303.1452304799181 Content-Type: text/x-patch; name=gluster-poll.patch Content-Disposition: attachment; filename=gluster-poll.patch Content-Transfer-Encoding: base64 LS0tIGdsdXN0ZXJmcy0zLjcuNi9saWJnbHVzdGVyZnMvc3JjL2V2ZW50LXBvbGwuYy5zYXYJMjAx Ni0wMS0wNiAxNTo1ODowMy41MjIyODYwMDAgLTA1MDAKKysrIGdsdXN0ZXJmcy0zLjcuNi9saWJn bHVzdGVyZnMvc3JjL2V2ZW50LXBvbGwuYwkyMDE2LTAxLTA4IDE3OjQ1OjI4LjA3ODAyNDAwMCAt MDUwMApAQCAtMTgwLDYgKzE4MCwxNiBAQCBldmVudF9wb29sX25ld19wb2xsIChpbnQgY291bnQs IGludCBldmVuCiAgICAgICAgIHJldHVybiBldmVudF9wb29sOwogfQogCitzdGF0aWMgdm9pZAor ZXZlbnRfcG9vbF9jaGFuZ2VkIChzdHJ1Y3QgZXZlbnRfcG9vbCAqZXZlbnRfcG9vbCkKK3sKKwor ICAgICAgICBldmVudF9wb29sLT5jaGFuZ2VkID0gMTsKKyAgICAgICAgLyogV3JpdGUgYSBieXRl IGludG8gdGhlIGJyZWFrZXIgcGlwZSB0byB3YWtlIHVwIHBvbGwoKS4gKi8KKyAgICAgICAgaWYg KGV2ZW50X3Bvb2wtPmJyZWFrZXJbMV0gPj0gMCkKKyAgICAgICAgICAgICAgICB3cml0ZShldmVu dF9wb29sLT5icmVha2VyWzFdLCAiWCIsIDEpOworfQorCiAKIHN0YXRpYyBpbnQKIGV2ZW50X3Jl Z2lzdGVyX3BvbGwgKHN0cnVjdCBldmVudF9wb29sICpldmVudF9wb29sLCBpbnQgZmQsCkBAIC0y NDQsNyArMjU0LDcgQEAgZXZlbnRfcmVnaXN0ZXJfcG9sbCAoc3RydWN0IGV2ZW50X3Bvb2wgKgog ICAgICAgICAgICAgICAgICAgICAgICAgYnJlYWs7CiAgICAgICAgICAgICAgICAgfQogCi0gICAg ICAgICAgICAgICAgZXZlbnRfcG9vbC0+Y2hhbmdlZCA9IDE7CisgICAgICAgICAgICAgICAgZXZl bnRfcG9vbF9jaGFuZ2VkKGV2ZW50X3Bvb2wpOwogCiAgICAgICAgIH0KIHVubG9jazoKQEAgLTI3 NSw3ICsyODUsNyBAQCBldmVudF91bnJlZ2lzdGVyX3BvbGwgKHN0cnVjdCBldmVudF9wb29sCiAg ICAgICAgICAgICAgICAgfQogCiAgICAgICAgICAgICAgICAgZXZlbnRfcG9vbC0+cmVnW2lkeF0g PSAgZXZlbnRfcG9vbC0+cmVnWy0tZXZlbnRfcG9vbC0+dXNlZF07Ci0gICAgICAgICAgICAgICAg ZXZlbnRfcG9vbC0+Y2hhbmdlZCA9IDE7CisgICAgICAgICAgICAgICAgZXZlbnRfcG9vbF9jaGFu Z2VkKGV2ZW50X3Bvb2wpOwogICAgICAgICB9CiB1bmxvY2s6CiAgICAgICAgIHB0aHJlYWRfbXV0 ZXhfdW5sb2NrICgmZXZlbnRfcG9vbC0+bXV0ZXgpOwpAQCAtMzUwLDcgKzM2MCw3IEBAIGV2ZW50 X3NlbGVjdF9vbl9wb2xsIChzdHJ1Y3QgZXZlbnRfcG9vbCAKICAgICAgICAgICAgICAgICB9CiAK ICAgICAgICAgICAgICAgICBpZiAocG9sbF9pbiArIHBvbGxfb3V0ID4gLTIpCi0gICAgICAgICAg ICAgICAgICAgICAgICBldmVudF9wb29sLT5jaGFuZ2VkID0gMTsKKyAgICAgICAgICAgICAgICAg ICAgICAgIGV2ZW50X3Bvb2xfY2hhbmdlZChldmVudF9wb29sKTsKICAgICAgICAgfQogdW5sb2Nr OgogICAgICAgICBwdGhyZWFkX211dGV4X3VubG9jayAoJmV2ZW50X3Bvb2wtPm11dGV4KTsKQEAg LTQ0OCw2ICs0NTgsNyBAQCBldmVudF9kaXNwYXRjaF9wb2xsIChzdHJ1Y3QgZXZlbnRfcG9vbCAq CiAgICAgICAgIGludCAgICAgICAgICAgICAgc2l6ZSA9IDA7CiAgICAgICAgIGludCAgICAgICAg ICAgICAgaSA9IDA7CiAgICAgICAgIGludCAgICAgICAgICAgICAgcmV0ID0gLTE7CisgICAgICAg IGNoYXIgICAgICAgICAgICAgeDsKIAogICAgICAgICBHRl9WQUxJREFURV9PUl9HT1RPICgiZXZl bnQiLCBldmVudF9wb29sLCBvdXQpOwogCkBAIC00NzIsNyArNDgzLDcgQEAgZXZlbnRfZGlzcGF0 Y2hfcG9sbCAoc3RydWN0IGV2ZW50X3Bvb2wgKgogICAgICAgICAgICAgICAgIHNpemUgPSBldmVu dF9kaXNwYXRjaF9wb2xsX3Jlc2l6ZSAoZXZlbnRfcG9vbCwgdWZkcywgc2l6ZSk7CiAgICAgICAg ICAgICAgICAgdWZkcyA9IGV2ZW50X3Bvb2wtPmV2Y2FjaGU7CiAKLSAgICAgICAgICAgICAgICBy ZXQgPSBwb2xsICh1ZmRzLCBzaXplLCAxKTsKKyAgICAgICAgICAgICAgICByZXQgPSBwb2xsICh1 ZmRzLCBzaXplLCAtMSk7CiAKICAgICAgICAgICAgICAgICBpZiAocmV0ID09IDApCiAgICAgICAg ICAgICAgICAgICAgICAgICAvKiB0aW1lb3V0ICovCkBAIC00ODIsNyArNDkzLDEzIEBAIGV2ZW50 X2Rpc3BhdGNoX3BvbGwgKHN0cnVjdCBldmVudF9wb29sICoKICAgICAgICAgICAgICAgICAgICAg ICAgIC8qIHN5cyBjYWxsICovCiAgICAgICAgICAgICAgICAgICAgICAgICBjb250aW51ZTsKIAot ICAgICAgICAgICAgICAgIGZvciAoaSA9IDA7IGkgPCBzaXplOyBpKyspIHsKKyAgICAgICAgICAg ICAgICBpZiAodWZkc1swXS5yZXZlbnRzICE9IDAgJiYgZXZlbnRfcG9vbC0+YnJlYWtlclswXSA+ PSAwKSB7CisgICAgICAgICAgICAgICAgICAgICAgICAvKiBKdXN0IHJlYWQgYWxsIHRoZSBqdW5r IGluIHRoZSBicmVha2VyIHBpcGUuICovCisgICAgICAgICAgICAgICAgICAgICAgICB3aGlsZSAo cmVhZChldmVudF9wb29sLT5icmVha2VyWzBdLCAmeCwgMSkgPiAwKQorICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICA7CisgICAgICAgICAgICAgICAgfQorCisgICAgICAgICAgICAgICAg Zm9yIChpID0gMTsgaSA8IHNpemU7IGkrKykgewogICAgICAgICAgICAgICAgICAgICAgICAgaWYg KCF1ZmRzW2ldLnJldmVudHMpCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnRp bnVlOwogCg== ------=_Part_154244850_659138303.1452304799181-- From owner-freebsd-fs@freebsd.org Sat Jan 9 02:41:25 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4A94FA68BB2 for ; Sat, 9 Jan 2016 02:41:25 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id DA83815D7 for ; Sat, 9 Jan 2016 02:41:24 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:sBBklxKSxjKlN7v50dmcpTZWNBhigK39O0sv0rFitYgULPnxwZ3uMQTl6Ol3ixeRBMOAu6wC07KempujcFJDyK7JiGoFfp1IWk1NouQttCtkPvS4D1bmJuXhdS0wEZcKflZk+3amLRodQ56mNBXsq3G/pQQfBg/4fVIsYL+lRMiK14ye7KObxd76W01wnj2zYLd/fl2djD76kY0ou7ZkMbs70RDTo3FFKKx8zGJsIk+PzV6nvp/jtLYqySlbuuog+shcSu26Ov1gFf0LRAghZkI46sOjmRDZRhrHsnkQW38dgzJSDgTF5Q28VZD05HjUrO14jRObNs6+aLk/WjCv6u8/UhrhgyQDOjsR7WbYl8F0lKIdqxv39E83+JLdfIzAbKk2RajaZ95PADMZBss= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CuBABYcpBW/61jaINdhAxtBohTtT0YCoUjSgKBVxEBAQEBAQEBAYEJgi2CCAEBBAEBASAEJyALDAQCAQgOGxkCAgIlAQkmAgQIBwQBHASIDg6xM5AyAQEBAQEBAQEBAQEBAQEBAQEBEgUEgi+EJ4R/hDcBARsZgwaBSQWOMIhdgnSCT4UphEiHbYUyjkwCOCyCERyBeyA0B4QYOoEIAQEB X-IronPort-AV: E=Sophos;i="5.20,541,1444708800"; d="scan'208";a="261928249" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 08 Jan 2016 21:40:16 -0500 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 1CCA315F55D; Fri, 8 Jan 2016 21:40:16 -0500 (EST) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id WWiq5ZaflpHb; Fri, 8 Jan 2016 21:40:15 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 53C1715F565; Fri, 8 Jan 2016 21:40:15 -0500 (EST) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 67obn-TwXJgx; Fri, 8 Jan 2016 21:40:15 -0500 (EST) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 2E87715F55D; Fri, 8 Jan 2016 21:40:15 -0500 (EST) Date: Fri, 8 Jan 2016 21:40:15 -0500 (EST) From: Rick Macklem To: Jeff Darcy Cc: Raghavendra G , freebsd-fs , Hubbard Jordan , Xavier Hernandez , Gluster Devel Message-ID: <2013962695.154259810.1452307215168.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <1924941590.6473225.1452248249994.JavaMail.zimbra@redhat.com> References: <571237035.145690509.1451437960464.JavaMail.zimbra@uoguelph.ca> <20151230103152.GS13942@ndevos-x240.usersys.redhat.com> <2D8C2729-D556-479B-B4E2-66E1BB222F41@ixsystems.com> <1083933309.146084334.1451517977647.JavaMail.zimbra@uoguelph.ca> <568F6D07.6070500@datalab.es> <1924941590.6473225.1452248249994.JavaMail.zimbra@redhat.com> Subject: Re: [Gluster-devel] FreeBSD port of GlusterFS racks up a lot of CPU usage MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_154259808_2047258858.1452307215166" X-Originating-IP: [172.17.95.12] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF43 (Win)/8.0.9_GA_6191) Thread-Topic: FreeBSD port of GlusterFS racks up a lot of CPU usage Thread-Index: nzgvBLPPgcBXuRsf6GlZU17foVfxCMhA9xbs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jan 2016 02:41:25 -0000 ------=_Part_154259808_2047258858.1452307215166 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Oops, I realized the last patch did a write(2) while holding a pthread_mutex. I've never used pthread_mutexes, but I suspect this isn't allowed. The attached updated patch delays the write() until after the pthread_mutex_unlock(). Sorry about the confusion, rick ----- Original Message ----- > > > I don't know anything about gluster's poll implementation so I may > > > be totally wrong, but would it be possible to use an eventfd (or a > > > pipe if eventfd is not supported) to signal the need to add more > > > file descriptors to the poll call ? > > > > > > > > > The poll call should listen on this new fd. When we need to change > > > the fd list, we should simply write to the eventfd or pipe from > > > another thread. This will cause the poll call to return and we will > > > be able to change the fd list without having a short timeout nor > > > having to decide on any trade-off. > > > > > > Thats a nice idea. Based on my understanding of why timeouts are being > > used, this approach can work. > > The own-thread code which preceded the current poll implementation did > something similar, using a pipe fd to be woken up for new *outgoing* > messages. That code still exists, and might provide some insight into > how to do this for the current poll code. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > ------=_Part_154259808_2047258858.1452307215166 Content-Type: text/x-patch; name=gluster-poll.patch Content-Disposition: attachment; filename=gluster-poll.patch Content-Transfer-Encoding: base64 LS0tIGdsdXN0ZXJmcy0zLjcuNi9saWJnbHVzdGVyZnMvc3JjL2V2ZW50LXBvbGwuYy5zYXYJMjAx Ni0wMS0wNiAxNTo1ODowMy41MjIyODYwMDAgLTA1MDAKKysrIGdsdXN0ZXJmcy0zLjcuNi9saWJn bHVzdGVyZnMvc3JjL2V2ZW50LXBvbGwuYwkyMDE2LTAxLTA4IDE4OjE0OjU3LjY1ODY1MjAwMCAt MDUwMApAQCAtMTgwLDYgKzE4MCwxNSBAQCBldmVudF9wb29sX25ld19wb2xsIChpbnQgY291bnQs IGludCBldmVuCiAgICAgICAgIHJldHVybiBldmVudF9wb29sOwogfQogCitzdGF0aWMgdm9pZAor ZXZlbnRfcG9vbF9jaGFuZ2VkIChzdHJ1Y3QgZXZlbnRfcG9vbCAqZXZlbnRfcG9vbCkKK3sKKwor ICAgICAgICAvKiBXcml0ZSBhIGJ5dGUgaW50byB0aGUgYnJlYWtlciBwaXBlIHRvIHdha2UgdXAg cG9sbCgpLiAqLworICAgICAgICBpZiAoZXZlbnRfcG9vbC0+YnJlYWtlclsxXSA+PSAwKQorICAg ICAgICAgICAgICAgIHdyaXRlKGV2ZW50X3Bvb2wtPmJyZWFrZXJbMV0sICJYIiwgMSk7Cit9CisK IAogc3RhdGljIGludAogZXZlbnRfcmVnaXN0ZXJfcG9sbCAoc3RydWN0IGV2ZW50X3Bvb2wgKmV2 ZW50X3Bvb2wsIGludCBmZCwKQEAgLTE4Nyw2ICsxOTYsNyBAQCBldmVudF9yZWdpc3Rlcl9wb2xs IChzdHJ1Y3QgZXZlbnRfcG9vbCAqCiAgICAgICAgICAgICAgICAgICAgICB2b2lkICpkYXRhLCBp bnQgcG9sbF9pbiwgaW50IHBvbGxfb3V0KQogewogICAgICAgICBpbnQgaWR4ID0gLTE7CisgICAg ICAgIGludCBjaGFuZ2VkID0gMDsKIAogICAgICAgICBHRl9WQUxJREFURV9PUl9HT1RPICgiZXZl bnQiLCBldmVudF9wb29sLCBvdXQpOwogCkBAIC0yNDUsMTAgKzI1NSwxMyBAQCBldmVudF9yZWdp c3Rlcl9wb2xsIChzdHJ1Y3QgZXZlbnRfcG9vbCAqCiAgICAgICAgICAgICAgICAgfQogCiAgICAg ICAgICAgICAgICAgZXZlbnRfcG9vbC0+Y2hhbmdlZCA9IDE7CisgICAgICAgICAgICAgICAgY2hh bmdlZCA9IDE7CiAKICAgICAgICAgfQogdW5sb2NrOgogICAgICAgICBwdGhyZWFkX211dGV4X3Vu bG9jayAoJmV2ZW50X3Bvb2wtPm11dGV4KTsKKyAgICAgICAgaWYgKGNoYW5nZWQgIT0gMCkKKyAg ICAgICAgICAgICAgICBldmVudF9wb29sX2NoYW5nZWQoZXZlbnRfcG9vbCk7CiAKIG91dDoKICAg ICAgICAgcmV0dXJuIGlkeDsKQEAgLTI1OSw2ICsyNzIsNyBAQCBzdGF0aWMgaW50CiBldmVudF91 bnJlZ2lzdGVyX3BvbGwgKHN0cnVjdCBldmVudF9wb29sICpldmVudF9wb29sLCBpbnQgZmQsIGlu dCBpZHhfaGludCkKIHsKICAgICAgICAgaW50IGlkeCA9IC0xOworICAgICAgICBpbnQgY2hhbmdl ZCA9IDA7CiAKICAgICAgICAgR0ZfVkFMSURBVEVfT1JfR09UTyAoImV2ZW50IiwgZXZlbnRfcG9v bCwgb3V0KTsKIApAQCAtMjc2LDkgKzI5MCwxMiBAQCBldmVudF91bnJlZ2lzdGVyX3BvbGwgKHN0 cnVjdCBldmVudF9wb29sCiAKICAgICAgICAgICAgICAgICBldmVudF9wb29sLT5yZWdbaWR4XSA9 ICBldmVudF9wb29sLT5yZWdbLS1ldmVudF9wb29sLT51c2VkXTsKICAgICAgICAgICAgICAgICBl dmVudF9wb29sLT5jaGFuZ2VkID0gMTsKKyAgICAgICAgICAgICAgICBjaGFuZ2VkID0gMTsKICAg ICAgICAgfQogdW5sb2NrOgogICAgICAgICBwdGhyZWFkX211dGV4X3VubG9jayAoJmV2ZW50X3Bv b2wtPm11dGV4KTsKKyAgICAgICAgaWYgKGNoYW5nZWQgIT0gMCkKKyAgICAgICAgICAgICAgICBl dmVudF9wb29sX2NoYW5nZWQoZXZlbnRfcG9vbCk7CiAKIG91dDoKICAgICAgICAgcmV0dXJuIGlk eDsKQEAgLTMwNCw2ICszMjEsNyBAQCBldmVudF9zZWxlY3Rfb25fcG9sbCAoc3RydWN0IGV2ZW50 X3Bvb2wgCiAgICAgICAgICAgICAgICAgICAgICAgaW50IHBvbGxfaW4sIGludCBwb2xsX291dCkK IHsKICAgICAgICAgaW50IGlkeCA9IC0xOworICAgICAgICBpbnQgY2hhbmdlZCA9IDA7CiAKICAg ICAgICAgR0ZfVkFMSURBVEVfT1JfR09UTyAoImV2ZW50IiwgZXZlbnRfcG9vbCwgb3V0KTsKIApA QCAtMzQ5LDExICszNjcsMTUgQEAgZXZlbnRfc2VsZWN0X29uX3BvbGwgKHN0cnVjdCBldmVudF9w b29sIAogICAgICAgICAgICAgICAgICAgICAgICAgYnJlYWs7CiAgICAgICAgICAgICAgICAgfQog Ci0gICAgICAgICAgICAgICAgaWYgKHBvbGxfaW4gKyBwb2xsX291dCA+IC0yKQorICAgICAgICAg ICAgICAgIGlmIChwb2xsX2luICsgcG9sbF9vdXQgPiAtMikgewogICAgICAgICAgICAgICAgICAg ICAgICAgZXZlbnRfcG9vbC0+Y2hhbmdlZCA9IDE7CisgICAgICAgICAgICAgICAgICAgICAgICBj aGFuZ2VkID0gMTsKKyAgICAgICAgICAgICAgICB9CiAgICAgICAgIH0KIHVubG9jazoKICAgICAg ICAgcHRocmVhZF9tdXRleF91bmxvY2sgKCZldmVudF9wb29sLT5tdXRleCk7CisgICAgICAgIGlm IChjaGFuZ2VkICE9IDApCisgICAgICAgICAgICAgICAgZXZlbnRfcG9vbF9jaGFuZ2VkKGV2ZW50 X3Bvb2wpOwogCiBvdXQ6CiAgICAgICAgIHJldHVybiBpZHg7CkBAIC00NDgsNiArNDcwLDcgQEAg ZXZlbnRfZGlzcGF0Y2hfcG9sbCAoc3RydWN0IGV2ZW50X3Bvb2wgKgogICAgICAgICBpbnQgICAg ICAgICAgICAgIHNpemUgPSAwOwogICAgICAgICBpbnQgICAgICAgICAgICAgIGkgPSAwOwogICAg ICAgICBpbnQgICAgICAgICAgICAgIHJldCA9IC0xOworICAgICAgICBjaGFyICAgICAgICAgICAg IHg7CiAKICAgICAgICAgR0ZfVkFMSURBVEVfT1JfR09UTyAoImV2ZW50IiwgZXZlbnRfcG9vbCwg b3V0KTsKIApAQCAtNDcyLDcgKzQ5NSw3IEBAIGV2ZW50X2Rpc3BhdGNoX3BvbGwgKHN0cnVjdCBl dmVudF9wb29sICoKICAgICAgICAgICAgICAgICBzaXplID0gZXZlbnRfZGlzcGF0Y2hfcG9sbF9y ZXNpemUgKGV2ZW50X3Bvb2wsIHVmZHMsIHNpemUpOwogICAgICAgICAgICAgICAgIHVmZHMgPSBl dmVudF9wb29sLT5ldmNhY2hlOwogCi0gICAgICAgICAgICAgICAgcmV0ID0gcG9sbCAodWZkcywg c2l6ZSwgMSk7CisgICAgICAgICAgICAgICAgcmV0ID0gcG9sbCAodWZkcywgc2l6ZSwgLTEpOwog CiAgICAgICAgICAgICAgICAgaWYgKHJldCA9PSAwKQogICAgICAgICAgICAgICAgICAgICAgICAg LyogdGltZW91dCAqLwpAQCAtNDgyLDcgKzUwNSwxMyBAQCBldmVudF9kaXNwYXRjaF9wb2xsIChz dHJ1Y3QgZXZlbnRfcG9vbCAqCiAgICAgICAgICAgICAgICAgICAgICAgICAvKiBzeXMgY2FsbCAq LwogICAgICAgICAgICAgICAgICAgICAgICAgY29udGludWU7CiAKLSAgICAgICAgICAgICAgICBm b3IgKGkgPSAwOyBpIDwgc2l6ZTsgaSsrKSB7CisgICAgICAgICAgICAgICAgaWYgKHVmZHNbMF0u cmV2ZW50cyAhPSAwICYmIGV2ZW50X3Bvb2wtPmJyZWFrZXJbMF0gPj0gMCkgeworICAgICAgICAg ICAgICAgICAgICAgICAgLyogSnVzdCByZWFkIGFsbCB0aGUganVuayBpbiB0aGUgYnJlYWtlciBw aXBlLiAqLworICAgICAgICAgICAgICAgICAgICAgICAgd2hpbGUgKHJlYWQoZXZlbnRfcG9vbC0+ YnJlYWtlclswXSwgJngsIDEpID4gMCkKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg OworICAgICAgICAgICAgICAgIH0KKworICAgICAgICAgICAgICAgIGZvciAoaSA9IDE7IGkgPCBz aXplOyBpKyspIHsKICAgICAgICAgICAgICAgICAgICAgICAgIGlmICghdWZkc1tpXS5yZXZlbnRz KQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb250aW51ZTsKIAo= ------=_Part_154259808_2047258858.1452307215166-- From owner-freebsd-fs@freebsd.org Sat Jan 9 03:34:56 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 78306A67D47 for ; Sat, 9 Jan 2016 03:34:56 +0000 (UTC) (envelope-from rgowdapp@redhat.com) Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mx1.redhat.com", Issuer "DigiCert SHA2 Extended Validation Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5E2F61613 for ; Sat, 9 Jan 2016 03:34:56 +0000 (UTC) (envelope-from rgowdapp@redhat.com) Received: from zmail13.collab.prod.int.phx2.redhat.com (zmail13.collab.prod.int.phx2.redhat.com [10.5.83.15]) by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id u093Yq3U027687; Fri, 8 Jan 2016 22:34:52 -0500 Date: Fri, 8 Jan 2016 22:34:50 -0500 (EST) From: Raghavendra Gowdappa To: Rick Macklem Cc: Jeff Darcy , Raghavendra G , freebsd-fs , Hubbard Jordan , Xavier Hernandez , Gluster Devel Message-ID: <1256214214.7158114.1452310490692.JavaMail.zimbra@redhat.com> In-Reply-To: <981529129.154244852.1452304799182.JavaMail.zimbra@uoguelph.ca> References: <571237035.145690509.1451437960464.JavaMail.zimbra@uoguelph.ca> <2D8C2729-D556-479B-B4E2-66E1BB222F41@ixsystems.com> <1083933309.146084334.1451517977647.JavaMail.zimbra@uoguelph.ca> <568F6D07.6070500@datalab.es> <1924941590.6473225.1452248249994.JavaMail.zimbra@redhat.com> <981529129.154244852.1452304799182.JavaMail.zimbra@uoguelph.ca> Subject: Re: [Gluster-devel] FreeBSD port of GlusterFS racks up a lot of CPU usage MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.10.63.66] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - GC25 (Linux)/8.0.6_GA_5922) Thread-Topic: FreeBSD port of GlusterFS racks up a lot of CPU usage Thread-Index: nzgvBLPPgcBXuRsf6GlZU17foVfxCE+M3nAjcp2ixGo= X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jan 2016 03:34:56 -0000 ----- Original Message ----- > From: "Rick Macklem" > To: "Jeff Darcy" > Cc: "Raghavendra G" , "freebsd-fs" , "Hubbard Jordan" > , "Xavier Hernandez" , "Gluster Devel" > Sent: Saturday, January 9, 2016 7:29:59 AM > Subject: Re: [Gluster-devel] FreeBSD port of GlusterFS racks up a lot of CPU usage > > Jeff Darcy wrote: > > > > I don't know anything about gluster's poll implementation so I may > > > > be totally wrong, but would it be possible to use an eventfd (or a > > > > pipe if eventfd is not supported) to signal the need to add more > > > > file descriptors to the poll call ? > > > > > > > > > > > > The poll call should listen on this new fd. When we need to change > > > > the fd list, we should simply write to the eventfd or pipe from > > > > another thread. This will cause the poll call to return and we will > > > > be able to change the fd list without having a short timeout nor > > > > having to decide on any trade-off. > > > > > > > > > Thats a nice idea. Based on my understanding of why timeouts are being > > > used, this approach can work. > > > > The own-thread code which preceded the current poll implementation did > > something similar, using a pipe fd to be woken up for new *outgoing* > > messages. That code still exists, and might provide some insight into > > how to do this for the current poll code. > I took a look at event-poll.c and found something interesting... > - A pipe called "breaker" is already set up by event_pool_new_poll() and > closed by event_pool_destroy_poll(), however it never gets used for > anything. I did a check on history, but couldn't find any information on why it was removed. Can you send this patch to http://review.gluster.org ? We can review and merge the patch over there. If you are not aware, development work flow can be found at: http://www.gluster.org/community/documentation/index.php/Developers > > So, I added a few lines of code that writes a byte to it whenever the list of > file descriptors is changed and read when poll() returns, if its revents is > set. > I also changed the timeout to -1 (infinity) and it seems to work for a > trivial > test. > --> Btw, I also noticed the "changed" variable gets set to 1 on a change, but > never reset to 0. I didn't change this, since it looks "racey". (ie. I > think you could easily get a race between a thread that clears it and one > that adds a new fd.) > > A slightly safer version of the patch would set a long (100msec ??) timeout > instead > of -1. > > Anyhow, I've attached the patch in case anyone would like to try it and will > create a bug report for this after I've had more time to test it. > (I only use a couple of laptops, so my testing will be minimal.) > > Thanks for all the help, rick > > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > > From owner-freebsd-fs@freebsd.org Sat Jan 9 05:16:33 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EF34CA69C80 for ; Sat, 9 Jan 2016 05:16:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DFAB31C52 for ; Sat, 9 Jan 2016 05:16:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u095GWMX038190 for ; Sat, 9 Jan 2016 05:16:32 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206056] [ext2fs][patch][panic] EXT4: mount panic from freeing invalid pointers Date: Sat, 09 Jan 2016 05:16:32 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: damjan.jov@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status keywords bug_severity priority component assigned_to reporter cc attachments.created Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jan 2016 05:16:33 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206056 Bug ID: 206056 Summary: [ext2fs][patch][panic] EXT4: mount panic from freeing invalid pointers Product: Base System Version: 11.0-CURRENT Hardware: Any OS: Any Status: New Keywords: patch Severity: Affects Some People Priority: --- Component: kern Assignee: freebsd-bugs@FreeBSD.org Reporter: damjan.jov@gmail.com CC: freebsd-fs@FreeBSD.org Keywords: patch Created attachment 165290 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D165290&action= =3Dedit Preventing a panic when pointers from struct ext2mount's um_e2fs are freed On Linux I made a 500MB EXT4 filesystem for testing, and when I tried to mo= unt it in FreeBSD with: mdconfig -a /path/to/filesystem mount -t ext2fs -o ro /dev/md0 /path/to/mountpoint the following error got printed out, followed immediately by a panic: ext2fs: no space for extra inode timestamps Fatal trap 12: page fault while in kernel mode cpuid =3D 0; apicid =3D 00 fault_virtual_address =3D 0x4 fault code =3D supervisor read, page not present instruction pointer =3D 0x20:0xc0b1f1cc stack pointer =3D 0x28:0xcebee898 frame pointer =3D 0x28:0xcebee8c0 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, def32 1, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 777 (mount) [ thread pid 777 tid 100065 ] Stopped at free+0x5c: movl 0x4(%eax),%eax db> bt Tracing pid 777 tid 100065 td 0xc4e0c620 free(aa,c54ab298,2a3,2a1,0,...) at free+0x5c/frame 0xcebee8c0 ext2_mount(c4e16a80,c54ab208,c5374380,c4e10800,c4c40a70,...) at ext2_mount+0x1604/frame 0xceebe9e8 vfs_donmount(c4e4c620,1,0,c4c11b00,c4c11b00,...) at vfs_donmount+0xdc6/frame 0xceebebf0 sys_nmount(c4e0c620,ceebeca8,c506890c,c4e0c620,c506890c,...) at sys_nmount+0x78/frame 0xceebec18 syscall(ceebece8) at syscall+0x4a6/frame 0xceebecdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xceebecdc --- syscall (378, FreeBSD ELF32, sys_nmount), eip =3D 0x280e013b, esp =3D 0xbfbfdd20, ebp =3D 0xbfbfe278 The "ext2fs: no space for extra inode timestamps" message comes from compute_sb_data() in ext2_vfsops.c, which returns EINVAL after printing it, never reaching the subsequent lines that initialize fs->e2fs_gd and fs->e2fs_contigdirs. When ext2_mountfs() calls compute_sb_data(), it does a "goto out" on error, and in "out" it attempts to free() those 2 fields. Sin= ce the memory for the struct those fields are in wasn't initialized when it was allocated, free() is being passed invalid pointers, resulting in a panic. The attached patch initializes the struct with those fields to zeroes on allocation, preventing the panic. I'll investigate the original error that caused this buggy error path to be taken in a separate issue. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Sat Jan 9 06:12:33 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C0AE4A67F9A for ; Sat, 9 Jan 2016 06:12:33 +0000 (UTC) (envelope-from mail@ozzmosis.com) Received: from homiemail-a78.g.dreamhost.com (sub5.mail.dreamhost.com [208.113.200.129]) by mx1.freebsd.org (Postfix) with ESMTP id AC6981297 for ; Sat, 9 Jan 2016 06:12:33 +0000 (UTC) (envelope-from mail@ozzmosis.com) Received: from homiemail-a78.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a78.g.dreamhost.com (Postfix) with ESMTP id 53FB020006450; Fri, 8 Jan 2016 22:12:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=ozzmosis.com; h=date:from :to:cc:subject:message-id:references:mime-version:content-type :in-reply-to:content-transfer-encoding; s=ozzmosis.com; bh=Kvbl+ 8FdgWvCc6fEClIaOmRrL38=; b=s21f2WA8OMQ/QnKWMFyC3fzR2480trTe0zlTh DCnfVvGgHVG855eq56D91QInciRZ2G9ttDGuHVrfGx9n72mtgNecAANmnS9e7Wc+ IvRc8PhO3h12bCOfRTBwqg1hy8lW7DGSWlTddLomV7V47ms6uWDU0kLiP2PxIIvw 8m3Wi4= Received: from blizzard.ozzmosis.com (202-161-17-94.dyn.iinet.net.au [202.161.17.94]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: relay@ozzmosis.com) by homiemail-a78.g.dreamhost.com (Postfix) with ESMTPSA id 12C3F20003333; Fri, 8 Jan 2016 22:12:27 -0800 (PST) Received: by blizzard.ozzmosis.com (Postfix, from userid 1001) id 430ADF7F; Sat, 9 Jan 2016 17:12:24 +1100 (AEDT) Date: Sat, 9 Jan 2016 17:12:24 +1100 From: andrew clarke To: Freddie Cash Cc: FreeBSD Filesystems Subject: Re: Solaris 11 ZFS send to FreeBSD 10.2 ... Message-ID: <20160109061224.GA65980@ozzmosis.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jan 2016 06:12:33 -0000 On Thu 2016-01-07 16:23:30 UTC-0800, Freddie Cash (fjwcash@gmail.com) wro= te: > =E2=80=8B(And you will constantly get nagged in "zpool status" output a= bout your > pool version not being the latest and greatest. Would be nice to shut = that > up somehow!)=E2=80=8B Agreed. Likewise the "One or more devices are configured to use a non-native bloc= k size. Expect reduced performance." From owner-freebsd-fs@freebsd.org Sat Jan 9 07:51:22 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9B71CA68D8E for ; Sat, 9 Jan 2016 07:51:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8CD7C158C for ; Sat, 9 Jan 2016 07:51:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u097pMOR050390 for ; Sat, 9 Jan 2016 07:51:22 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206059] [ext2fs][patch] EXT4: cannot mount filesystems < 512 MiB in size: "ext2fs: no space for extra inode timestamps" Date: Sat, 09 Jan 2016 07:51:22 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: damjan.jov@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status keywords bug_severity priority component assigned_to reporter cc attachments.created Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jan 2016 07:51:22 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206059 Bug ID: 206059 Summary: [ext2fs][patch] EXT4: cannot mount filesystems < 512 MiB in size: "ext2fs: no space for extra inode timestamps" Product: Base System Version: 11.0-CURRENT Hardware: Any OS: Any Status: New Keywords: patch Severity: Affects Some People Priority: --- Component: kern Assignee: freebsd-bugs@FreeBSD.org Reporter: damjan.jov@gmail.com CC: freebsd-fs@FreeBSD.org Created attachment 165293 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D165293&action= =3Dedit Allow inode size < sizeof(struct ext2fs_dinode) when EXT2F_ROCOMPAT_EXTRA_I= SIZE is set When making the EXT4 filesystem, mkfs.ext4 uses different inode sizes depen= ding on the filesystem file. For a large filesystem, "dumpe2fs" returns these relevant fields: Filesystem features: has_journal ext_attr resize_inode dir_index filet= ype extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize Inode size: 256 Required extra isize: 28 Desired extra isize: 28 For a small filesystem, "dumpe2fs" returns this instead, and "Required extra isize" and "Desired extra isize" are missing: Filesystem features: has_journal ext_attr resize_inode dir_index filet= ype extent flex_bg sparse_super huge_file uninit_bg dir_nlink extra_isize Inode size: 128 Bisection searching with different filesystem sizes ("dd if=3D/dev/zero of=3D/tmp/filesystem bs=3D1M count=3D..." + "mkfs.ext4 /tmp/filesystem" + "= dumpe2fs -h /tmp/filesystem") shows inode size 256 is used for filesystems >=3D 512 = MiB, and smaller filesystems use inode size 128. Attemping to mount the small filesystems fails with this error (and unless you've patched ext2fs with the patch from bug 206056, also panics the kernel!!): ext2fs: no space for extra inode timestamps That message comes from compute_sb_data() in ext2_vfsops.c, which does /* Check for extra isize in big inodes. */ if (EXT2_HAS_RO_COMPAT_FEATURE(fs, EXT2F_ROCOMPAT_EXTRA_ISIZE) && EXT2_INODE_SIZE(fs) < sizeof(struct ext2fs_dinode)) { printf("ext2fs: no space for extra inode timestamps\n"); return (EINVAL); } which must be wrong, because even small filesystems have the extra_isize feature set, yet the inode size is 128 which is smaller than sizeof(struct ext2fs_dinode). Since EXT2F_ROCOMPAT_EXTRA_ISIZE isn't used anywhere else in the ext2fs mod= ule, I've made a patch that deletes that entire section, and it gets the small filesystems to mount and work. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Sat Jan 9 15:06:33 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B6295A68019 for ; Sat, 9 Jan 2016 15:06:33 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-io0-x232.google.com (mail-io0-x232.google.com [IPv6:2607:f8b0:4001:c06::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9085510E3 for ; Sat, 9 Jan 2016 15:06:33 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-io0-x232.google.com with SMTP id 1so265731204ion.1 for ; Sat, 09 Jan 2016 07:06:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=C/in4lTrFCaW/MLSUvwVqD3OmCCBbB0U4qrax2H++20=; b=vXwM0/bOEC1j6fW8lu2xDMlv45vS9uu/dKL2AaqQd4mw3bO8CZkbr+2Cftst0072x2 McdowCu1pOFXSFjc6ciHkkQduV1ztlaKvCGps+VzeHHoVBYWK0KZeLpJinfdlut8t0Lp MrKhqurlSAS34oqRqQOJ8AhnuZL4uzzWPSOClJnQZXcv9oXfmDOLb5AmcqAoo5H4SdWj yYl1FpiUkVziMRsTnvvxpBQ8TjRWZkzJpw5ESJWx3MQucK3CWzfUBjpAE15HcYo72Uhs zeGtKOkQ8ZsmlgiU6OJaNSZTwLhV0JYrnHj/KCueW9lhZ+x3Uq4RRdnGZioPHFPeMCVq PWNQ== MIME-Version: 1.0 X-Received: by 10.107.162.146 with SMTP id l140mr49751890ioe.123.1452351992946; Sat, 09 Jan 2016 07:06:32 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.36.121.202 with HTTP; Sat, 9 Jan 2016 07:06:32 -0800 (PST) Date: Sat, 9 Jan 2016 07:06:32 -0800 X-Google-Sender-Auth: WWrMZNz9sYHop2ozYOeZ_iSWr90 Message-ID: Subject: LOR on 10.2-REL-p8 z/ ZFS + NFS? From: Adrian Chadd To: FreeBSD Filesystems Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jan 2016 15:06:33 -0000 Hiya, Someone asked me about this. It's happening on a 10.2-REL-p8 box serving ZFS via NFS. lock order reversal: 1st 0xfffff801ed92ca28 zfs (zfs) @ /usr/src/sys/fs/nfsserver/nfs_nfsdsocket.c:967 2nd 0xfffff8015f98a068 ufs (ufs) @ /usr/src/sys/kern/vfs_vnops.c:534 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0467b8bb30 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe0467b8bbe0 witness_checkorder() at witness_checkorder+0xe24/frame 0xfffffe0467b8bc70 __lockmgr_args() at __lockmgr_args+0x9d9/frame 0xfffffe0467b8bdb0 ffs_lock() at ffs_lock+0x92/frame 0xfffffe0467b8be00 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xfc/frame 0xfffffe0467b8be30 _vn_lock() at _vn_lock+0xd2/frame 0xfffffe0467b8bea0 vn_rdwr() at vn_rdwr+0x1c1/frame 0xfffffe0467b8bf80 nfsrv_writestable() at nfsrv_writestable+0xbd/frame 0xfffffe0467b8bff0 nfsrv_openupdate() at nfsrv_openupdate+0x557/frame 0xfffffe0467b8c480 nfsrvd_openconfirm() at nfsrvd_openconfirm+0x175/frame 0xfffffe0467b8c560 nfsrvd_dorpc() at nfsrvd_dorpc+0xf66/frame 0xfffffe0467b8c720 nfssvc_program() at nfssvc_program+0x4e6/frame 0xfffffe0467b8c8d0 svc_run_internal() at svc_run_internal+0xbb7/frame 0xfffffe0467b8ca60 svc_thread_start() at svc_thread_start+0xb/frame 0xfffffe0467b8ca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0467b8cab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0467b8cab0 --- trap 0xc, rip = 0x80089242a, rsp = 0x7fffffffe5d8, rbp = 0x7fffffffe880 --- Any ideas? -a From owner-freebsd-fs@freebsd.org Sat Jan 9 15:30:54 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9D55CA68E19 for ; Sat, 9 Jan 2016 15:30:54 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 4094D1FC6; Sat, 9 Jan 2016 15:30:53 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:2Kcf3RDBXtbKMG4xUtEXUyQJP3N1i/DPJgcQr6AfoPdwSP79r8bcNUDSrc9gkEXOFd2CrakU1ayO6+jJYi8p39WoiDg6aptCVhsI2409vjcLJ4q7M3D9N+PgdCcgHc5PBxdP9nC/NlVJSo6lPwWB6kO74TNaIBjjLw09fr2zQd6MyZzvn8mJuLTtICxwzAKnZr1zKBjk5S7wjeIxxbVYF6Aq1xHSqWFJcekFjUlhJFaUggqurpzopM0r221qtvkg789NV7nhN+R9FOQATWduD2dgzcHxtBDFBSCP73cQGjEfngBJCg7t4gv3U53qvm39rOUriweAOsijd7E/WnyH5qxoTBLtwHMdMjcy82Xaj+Rti61GrRa5p1p0ytiHM8muKPNic/aFLpshTm1bU5MUDnQZDw== X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DQAQAXJpFW/61jaINehAxtBohTs04BDYFmGAqFI0oCgU8UAQEBAQEBAQGBCYItgggBAQQBAQEgBCcgCxACAQgYAgINGQICJwEJJgEBBAgHBAEcBIgNDrA7kAcBAQEBAQEBAwEBAQEBAQEBFwSBAYVVhH+ENwEBBYM1gUkFjjaIXYVDgnOCN5Fpjk8CIAEBQoQoIDQHhBg6gQgBAQE X-IronPort-AV: E=Sophos;i="5.20,544,1444708800"; d="scan'208";a="260428638" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 09 Jan 2016 10:30:52 -0500 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 9A0B915F55D; Sat, 9 Jan 2016 10:30:52 -0500 (EST) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id v7k1wSdIoW01; Sat, 9 Jan 2016 10:30:52 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id EE44215F565; Sat, 9 Jan 2016 10:30:51 -0500 (EST) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ne0y4JofCE8o; Sat, 9 Jan 2016 10:30:51 -0500 (EST) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id D1B8715F55D; Sat, 9 Jan 2016 10:30:51 -0500 (EST) Date: Sat, 9 Jan 2016 10:30:51 -0500 (EST) From: Rick Macklem To: Adrian Chadd Cc: FreeBSD Filesystems Message-ID: <2114482125.154452234.1452353451676.JavaMail.zimbra@uoguelph.ca> In-Reply-To: References: Subject: Re: LOR on 10.2-REL-p8 z/ ZFS + NFS? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.10] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF43 (Win)/8.0.9_GA_6191) Thread-Topic: LOR on 10.2-REL-p8 z/ ZFS + NFS? Thread-Index: A9idCivPcAZ6HQOPtPcXNxn3jLY/ZA== X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jan 2016 15:30:54 -0000 Adrian Chadd wrote: > Hiya, > > Someone asked me about this. It's happening on a 10.2-REL-p8 box > serving ZFS via NFS. > > lock order reversal: > 1st 0xfffff801ed92ca28 zfs (zfs) @ > /usr/src/sys/fs/nfsserver/nfs_nfsdsocket.c:967 > 2nd 0xfffff8015f98a068 ufs (ufs) @ /usr/src/sys/kern/vfs_vnops.c:534 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > 0xfffffe0467b8bb30 > kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe0467b8bbe0 > witness_checkorder() at witness_checkorder+0xe24/frame 0xfffffe0467b8bc70 > __lockmgr_args() at __lockmgr_args+0x9d9/frame 0xfffffe0467b8bdb0 > ffs_lock() at ffs_lock+0x92/frame 0xfffffe0467b8be00 > VOP_LOCK1_APV() at VOP_LOCK1_APV+0xfc/frame 0xfffffe0467b8be30 > _vn_lock() at _vn_lock+0xd2/frame 0xfffffe0467b8bea0 > vn_rdwr() at vn_rdwr+0x1c1/frame 0xfffffe0467b8bf80 > nfsrv_writestable() at nfsrv_writestable+0xbd/frame 0xfffffe0467b8bff0 > nfsrv_openupdate() at nfsrv_openupdate+0x557/frame 0xfffffe0467b8c480 > nfsrvd_openconfirm() at nfsrvd_openconfirm+0x175/frame 0xfffffe0467b8c560 > nfsrvd_dorpc() at nfsrvd_dorpc+0xf66/frame 0xfffffe0467b8c720 > nfssvc_program() at nfssvc_program+0x4e6/frame 0xfffffe0467b8c8d0 > svc_run_internal() at svc_run_internal+0xbb7/frame 0xfffffe0467b8ca60 > svc_thread_start() at svc_thread_start+0xb/frame 0xfffffe0467b8ca70 > fork_exit() at fork_exit+0x84/frame 0xfffffe0467b8cab0 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0467b8cab0 > --- trap 0xc, rip = 0x80089242a, rsp = 0x7fffffffe5d8, rbp = 0x7fffffffe880 > --- > > Any ideas? > This shouldn't cause a deadlock in practice. The second one is locking a vnode for a single file used exclusively by the nfs server for NFSv4 called "stablerestart". It shouldn't normally be on an exported volume and shouldn't be accessed by anything other than the nfsd. (It is only updated when a new NFSv4 client creates a clientid, which basically means "once per client" or "once per client mount" depending on the client.) Also, for the above, it exists on a UFS volume, so there is zero chance of a deadlock against a vnode on ZFS. I suppose if "stablerestart" existed on an exported volume and was accessed erroneously by a client as the first file opened after mounting, a deadlock is conceivable. --> I'll take a look and maybe the code can check to see if the nfsd thread already has a lock on the vnode to avoid this unlikely scenario. In summary, I may commit a fix for this someday, but I don't think it will ever cause a deadlock in practice. Thanks for reporting it, rick > > > -a > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Sat Jan 9 18:09:15 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CBED5A69A12 for ; Sat, 9 Jan 2016 18:09:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B522C1EEF for ; Sat, 9 Jan 2016 18:09:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u09I9ESK025081 for ; Sat, 9 Jan 2016 18:09:15 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 204764] Filesystem deadlock, process in vodead state Date: Sat, 09 Jan 2016 18:09:15 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: kib@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jan 2016 18:09:15 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204764 Konstantin Belousov changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kib@FreeBSD.org --- Comment #2 from Konstantin Belousov --- (In reply to daniel from comment #1) Recompile the kernel with INVARIANTS etc, see the guide at https://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/kerne= ldebug-deadlocks.html When the situation occurs, execute 'kgdb /kernel.debug /dev/mem'. Af= ter that, switch to the thread which hung, with the kgdb command 'thread '.= =20 Backtrace 'bt' would should you the arguments, find the vnode address which caused the hang (most likely it is shown as vp), and do 'p *vp', 'p *(vp->v_bufobj.bo_object)'. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Jan 9 22:00:30 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E45DDA697B7 for ; Sat, 9 Jan 2016 22:00:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D468B1B23 for ; Sat, 9 Jan 2016 22:00:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u09M0Up1073031 for ; Sat, 9 Jan 2016 22:00:30 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206059] [ext2fs][patch] EXT4: cannot mount filesystems < 512 MiB in size: "ext2fs: no space for extra inode timestamps" Date: Sat, 09 Jan 2016 22:00:30 +0000 X-Bugzilla-Reason: CC AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jan 2016 22:00:31 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206059 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org --=20 You are receiving this mail because: You are on the CC list for the bug. You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Jan 9 22:00:41 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 55F28A697E4 for ; Sat, 9 Jan 2016 22:00:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 467EF1C00 for ; Sat, 9 Jan 2016 22:00:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u09M0fhC076737 for ; Sat, 9 Jan 2016 22:00:41 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 206056] [ext2fs][patch][panic] EXT4: mount panic from freeing invalid pointers Date: Sat, 09 Jan 2016 22:00:41 +0000 X-Bugzilla-Reason: AssignedTo CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jan 2016 22:00:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206056 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Sat Jan 9 23:38:22 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A833AA698F8 for ; Sat, 9 Jan 2016 23:38:22 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [46.4.40.135]) by mx1.freebsd.org (Postfix) with ESMTP id 73DCA1D72 for ; Sat, 9 Jan 2016 23:38:22 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:88ce:dbff:dc03:12da]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPSA id 69F517231 for ; Sun, 10 Jan 2016 02:38:20 +0300 (MSK) Date: Sun, 10 Jan 2016 02:38:12 +0300 From: Lev Serebryakov Reply-To: lev@FreeBSD.org Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <133976260.20160110023807@serebryakov.spb.ru> To: FreeBSD Filesystems Subject: Restore only several files from ZFS snapshot without creating copy of them? MIME-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg="pgp-sha512"; boundary="----------0E91DF09008489B3C" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jan 2016 23:38:22 -0000 ------------0E91DF09008489B3C Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Hello FreeBSD, I have a ZFS filesystem, which have daily snapshots. User removed several multi-gigabyte files by accident and need to get them back. These files are present in old daily snapshots, allright. But as far as I understnad, if I do cp /fs/.zfs/snapshots/old-snapshot/file /fs/file I got TWO copies of this file (one in live filesystem and future snapsho= ts and other one in old snapshots). As files in question are multi-gigabyte (about 100G alltogether) I don't want such duplication (dedup is turned off, as it is very memory-consuming). Is it possible to restore these files without such data duplication? I could not rollback whole FS to old snapshot, as here was other changes to it after files removal. --=20 Best regards, Lev mailto:lev@FreeBSD.org ------------0E91DF09008489B3C Content-Type: application/pgp-signature -----BEGIN PGP MESSAGE----- Version: GnuPG v2.0.22 (MingW32) iQJ8BAEBCgBmBQJWkZnkXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRGOTZEMUNBMEI1RjQzMThCNjc0QjMzMEFF QUIwM0M1OEJGREM0NzhGAAoJEOqwPFi/3EePmrkP/1YUqUkqFX5twZ0x0Oej5ekr X/n31zHx31VNbJrMQjFZKMEJh/wDsGq8TTVKgdb1IlhuX+SgwNFpGgdnbLELyBZJ HWyOtBsNjZvRePCAZsQUEPO+uMLEUIkMYoZY/KlwaW8QGhTMr8lPl8oTA0cR0DiN Rp0lfyCKlK9rYMgqivDrVh8iX1Cn2VHLGXPU8B8erYTIFJ9ogTnZJkcBuT0qp468 rMNN28e5qXRmajS7gepKoq3O2n5uoAGYWOgumsyowM3FebBtue8hX1yLvItCEn8d q38ISw+iUQ6i54lgFU1sOmiMg03r9/5sq01R+s9Rjqdze7YNHHOuM22B9fSh0Ll7 aPwNe7wMZEUgWiUa7Oa61G6gmlpUhmAB8lTK2+kt/h/Ets1oP/f+DnC7SDXoE5yj xcSUFkyUf6FRbRBDI8TDBYZ49gU7iHJXJMGSab524bVSUz3itCOkYysGwCrfGaXu Ouk/OZyuB8GuHIr8cvkc6EbfNX5Et14Myq5bQMIClzRn3rNBxOXT7EKpkQwMyC8o bWU+RX1pTRTQkiyubkFQ80UPZM91+nJe+LfZIJaKtS4nEOKnmHI8H6VwBFntxg2M Ehz9cjc3XZKOuPdkXew20AClLbH0ZKkwZOVtqiLROaTzlEwAbhA9A0Jv7G3mgXM8 uEoJTrJEsR6d46xx0ztO =+3Un -----END PGP MESSAGE----- ------------0E91DF09008489B3C--