Date: Tue, 21 Feb 2017 13:45:37 +0500 From: "Eugene M. Zheganin" <emz@norma.perm.ru> To: freebsd-fs@freebsd.org Subject: zfs raidz overhead Message-ID: <1b54a2fe35407a95edca1f992fa08a71@norman-vivat.ru>
next in thread | raw e-mail | index | archive | help
Hi. There's an interesting case described here: http://serverfault.com/questions/512018/strange-zfs-disk-space-usage-report-for-a-zvol [1] It's a user story who encountered that under some situations zfs on raidz could use up to 200% of the space for a zvol. I have also seen this. For instance: [root@san1:~]# zfs get volsize gamestop/reference1 NAME PROPERTY VALUE SOURCE gamestop/reference1 volsize 2,50T local [root@san1:~]# zfs get all gamestop/reference1 NAME PROPERTY VALUE SOURCE gamestop/reference1 type volume - gamestop/reference1 creation чт нояб. 24 9:09 2016 - gamestop/reference1 used 4,38T - gamestop/reference1 available 1,33T - gamestop/reference1 referenced 4,01T - gamestop/reference1 compressratio 1.00x - gamestop/reference1 reservation none default gamestop/reference1 volsize 2,50T local gamestop/reference1 volblocksize 8K - gamestop/reference1 checksum on default gamestop/reference1 compression off default gamestop/reference1 readonly off default gamestop/reference1 copies 1 default gamestop/reference1 refreservation none received gamestop/reference1 primarycache all default gamestop/reference1 secondarycache all default gamestop/reference1 usedbysnapshots 378G - gamestop/reference1 usedbydataset 4,01T - gamestop/reference1 usedbychildren 0 - gamestop/reference1 usedbyrefreservation 0 - gamestop/reference1 logbias latency default gamestop/reference1 dedup off default gamestop/reference1 mlslabel - gamestop/reference1 sync standard default gamestop/reference1 refcompressratio 1.00x - gamestop/reference1 written 4,89G - gamestop/reference1 logicalused 2,72T - gamestop/reference1 logicalreferenced 2,49T - gamestop/reference1 volmode default default gamestop/reference1 snapshot_limit none default gamestop/reference1 snapshot_count none default gamestop/reference1 redundant_metadata all default [root@san1:~]# zpool status gamestop pool: gamestop state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM gamestop ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 da6 ONLINE 0 0 0 da7 ONLINE 0 0 0 da8 ONLINE 0 0 0 da9 ONLINE 0 0 0 da11 ONLINE 0 0 0 errors: No known data errors or, another server (overhead in this case isn't that big, but still considerable): [root@san01:~]# zfs get all data/reference1 NAME PROPERTY VALUE SOURCE data/reference1 type volume - data/reference1 creation Fri Jan 6 11:23 2017 - data/reference1 used 3.82T - data/reference1 available 13.0T - data/reference1 referenced 3.22T - data/reference1 compressratio 1.00x - data/reference1 reservation none default data/reference1 volsize 2T local data/reference1 volblocksize 8K - data/reference1 checksum on default data/reference1 compression off default data/reference1 readonly off default data/reference1 copies 1 default data/reference1 refreservation none received data/reference1 primarycache all default data/reference1 secondarycache all default data/reference1 usedbysnapshots 612G - data/reference1 usedbydataset 3.22T - data/reference1 usedbychildren 0 - data/reference1 usedbyrefreservation 0 - data/reference1 logbias latency default data/reference1 dedup off default data/reference1 mlslabel - data/reference1 sync standard default data/reference1 refcompressratio 1.00x - data/reference1 written 498K - data/reference1 logicalused 2.37T - data/reference1 logicalreferenced 2.00T - data/reference1 volmode default default data/reference1 snapshot_limit none default data/reference1 snapshot_count none default data/reference1 redundant_metadata all default [root@san01:~]# zpool status gamestop pool: data state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM data ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 da3 ONLINE 0 0 0 da4 ONLINE 0 0 0 da5 ONLINE 0 0 0 da6 ONLINE 0 0 0 da7 ONLINE 0 0 0 raidz1-1 ONLINE 0 0 0 da8 ONLINE 0 0 0 da9 ONLINE 0 0 0 da10 ONLINE 0 0 0 da11 ONLINE 0 0 0 da12 ONLINE 0 0 0 raidz1-2 ONLINE 0 0 0 da13 ONLINE 0 0 0 da14 ONLINE 0 0 0 da15 ONLINE 0 0 0 da16 ONLINE 0 0 0 da17 ONLINE 0 0 0 errors: No known data errors So my question is - how to avoid it ? Right now I'm experimenting with the volblocksize, making it around 64k. I'm also suspecting that such overhead may be the subsequence of the various resizing operations, like extening the volsize of the volume or adding new disks into the pool, because I have a couple of servers with raidz where the initial disk/volsize configuration didn't change, and the referenced/volsize numbers are pretty much close to each other. Eugene. Links: ------ [1] http://serverfault.com/questions/512018/strange-zfs-disk-space-usage-report-for-a-zvol From owner-freebsd-fs@freebsd.org Tue Feb 21 09:25:06 2017 Return-Path: <owner-freebsd-fs@freebsd.org> Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7937BCE60D5 for <freebsd-fs@mailman.ysv.freebsd.org>; Tue, 21 Feb 2017 09:25:06 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (imap.slu.se [77.235.224.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "webmail.slu.se", Issuer "TERENA SSL CA 3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 135FB1F0C for <freebsd-fs@freebsd.org>; Tue, 21 Feb 2017 09:25:05 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (77.235.224.124) by exch2-4.slu.se (77.235.224.124) with Microsoft SMTP Server (TLS) id 15.0.1236.3; Tue, 21 Feb 2017 10:09:52 +0100 Received: from exch2-4.slu.se ([fe80::e006:857e:1307:5005]) by exch2-4.slu.se ([fe80::e006:857e:1307:5005%22]) with mapi id 15.00.1236.000; Tue, 21 Feb 2017 10:09:52 +0100 From: =?utf-8?B?S2FybGkgU2rDtmJlcmc=?= <karli.sjoberg@slu.se> To: "Eugene M. Zheganin" <emz@norma.perm.ru> CC: "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org> Subject: Re: zfs raidz overhead Thread-Topic: zfs raidz overhead Thread-Index: AQHSjCJCwmvqtn7xL0CKHdy5p0XKYg== Date: Tue, 21 Feb 2017 09:09:52 +0000 Message-ID: <09aeee171ef34c1ba8fb7b4c684a7224@exch2-4.slu.se> Accept-Language: sv-SE, en-US Content-Language: sv-SE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted Content-Type: text/plain; charset="utf-8" Content-ID: <A6F29B6436C4154D850646FD536D52F8@ad.slu.se> Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems <freebsd-fs.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-fs>, <mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs/>; List-Post: <mailto:freebsd-fs@freebsd.org> List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-fs>, <mailto:freebsd-fs-request@freebsd.org?subject=subscribe> X-List-Received-Date: Tue, 21 Feb 2017 09:25:06 -0000 CkRlbiAyMSBmZWIuIDIwMTcgMDk6NDYgc2tyZXYgIkV1Z2VuZSBNLiBaaGVnYW5pbiIgPGVtekBu b3JtYS5wZXJtLnJ1PjoKPgo+Cj4KPiBIaS4gCj4KPiBUaGVyZSdzIGFuIGludGVyZXN0aW5nIGNh c2UgZGVzY3JpYmVkIGhlcmU6Cj4gaHR0cDovL3NlcnZlcmZhdWx0LmNvbS9xdWVzdGlvbnMvNTEy MDE4L3N0cmFuZ2UtemZzLWRpc2stc3BhY2UtdXNhZ2UtcmVwb3J0LWZvci1hLXp2b2wKPiBbMV0g Cj4KPiBJdCdzIGEgdXNlciBzdG9yeSB3aG8gZW5jb3VudGVyZWQgdGhhdCB1bmRlciBzb21lIHNp dHVhdGlvbnMgemZzIG9uCj4gcmFpZHogY291bGQgdXNlIHVwIHRvIDIwMCUgb2YgdGhlIHNwYWNl IGZvciBhIHp2b2wuIAo+Cj4gSSBoYXZlIGFsc28gc2VlbiB0aGlzLiBGb3IgaW5zdGFuY2U6IAo+ Cj4gW3Jvb3RAc2FuMTp+XSMgemZzIGdldCB2b2xzaXplIGdhbWVzdG9wL3JlZmVyZW5jZTEKPiDC oE5BTUUgUFJPUEVSVFkgVkFMVUUgU09VUkNFCj4gwqBnYW1lc3RvcC9yZWZlcmVuY2UxIHZvbHNp emUgMiw1MFQgbG9jYWwKPiDCoFtyb290QHNhbjE6fl0jIHpmcyBnZXQgYWxsIGdhbWVzdG9wL3Jl ZmVyZW5jZTEgCj4gwqBOQU1FIFBST1BFUlRZIFZBTFVFIFNPVVJDRQo+IMKgZ2FtZXN0b3AvcmVm ZXJlbmNlMSB0eXBlIHZvbHVtZSAtCj4gwqBnYW1lc3RvcC9yZWZlcmVuY2UxIGNyZWF0aW9uINGH 0YIg0L3QvtGP0LEuIDI0IDk6MDkgMjAxNiAtCj4gwqBnYW1lc3RvcC9yZWZlcmVuY2UxIHVzZWQg NCwzOFQgLQo+IMKgZ2FtZXN0b3AvcmVmZXJlbmNlMSBhdmFpbGFibGUgMSwzM1QgLQo+IMKgZ2Ft ZXN0b3AvcmVmZXJlbmNlMSByZWZlcmVuY2VkIDQsMDFUIC0KPiDCoGdhbWVzdG9wL3JlZmVyZW5j ZTEgY29tcHJlc3NyYXRpbyAxLjAweCAtCj4gwqBnYW1lc3RvcC9yZWZlcmVuY2UxIHJlc2VydmF0 aW9uIG5vbmUgZGVmYXVsdAo+IMKgZ2FtZXN0b3AvcmVmZXJlbmNlMSB2b2xzaXplIDIsNTBUIGxv Y2FsCj4gwqBnYW1lc3RvcC9yZWZlcmVuY2UxIHZvbGJsb2Nrc2l6ZSA4SyAtCj4gwqBnYW1lc3Rv cC9yZWZlcmVuY2UxIGNoZWNrc3VtIG9uIGRlZmF1bHQKPiDCoGdhbWVzdG9wL3JlZmVyZW5jZTEg Y29tcHJlc3Npb24gb2ZmIGRlZmF1bHQKPiDCoGdhbWVzdG9wL3JlZmVyZW5jZTEgcmVhZG9ubHkg b2ZmIGRlZmF1bHQKPiDCoGdhbWVzdG9wL3JlZmVyZW5jZTEgY29waWVzIDEgZGVmYXVsdAo+IMKg Z2FtZXN0b3AvcmVmZXJlbmNlMSByZWZyZXNlcnZhdGlvbiBub25lIHJlY2VpdmVkCj4gwqBnYW1l c3RvcC9yZWZlcmVuY2UxIHByaW1hcnljYWNoZSBhbGwgZGVmYXVsdAo+IMKgZ2FtZXN0b3AvcmVm ZXJlbmNlMSBzZWNvbmRhcnljYWNoZSBhbGwgZGVmYXVsdAo+IMKgZ2FtZXN0b3AvcmVmZXJlbmNl MSB1c2VkYnlzbmFwc2hvdHMgMzc4RyAtCj4gwqBnYW1lc3RvcC9yZWZlcmVuY2UxIHVzZWRieWRh dGFzZXQgNCwwMVQgLQo+IMKgZ2FtZXN0b3AvcmVmZXJlbmNlMSB1c2VkYnljaGlsZHJlbiAwIC0K PiDCoGdhbWVzdG9wL3JlZmVyZW5jZTEgdXNlZGJ5cmVmcmVzZXJ2YXRpb24gMCAtCj4gwqBnYW1l c3RvcC9yZWZlcmVuY2UxIGxvZ2JpYXMgbGF0ZW5jeSBkZWZhdWx0Cj4gwqBnYW1lc3RvcC9yZWZl cmVuY2UxIGRlZHVwIG9mZiBkZWZhdWx0Cj4gwqBnYW1lc3RvcC9yZWZlcmVuY2UxIG1sc2xhYmVs IC0KPiDCoGdhbWVzdG9wL3JlZmVyZW5jZTEgc3luYyBzdGFuZGFyZCBkZWZhdWx0Cj4gwqBnYW1l c3RvcC9yZWZlcmVuY2UxIHJlZmNvbXByZXNzcmF0aW8gMS4wMHggLQo+IMKgZ2FtZXN0b3AvcmVm ZXJlbmNlMSB3cml0dGVuIDQsODlHIC0KPiDCoGdhbWVzdG9wL3JlZmVyZW5jZTEgbG9naWNhbHVz ZWQgMiw3MlQgLQo+IMKgZ2FtZXN0b3AvcmVmZXJlbmNlMSBsb2dpY2FscmVmZXJlbmNlZCAyLDQ5 VCAtCj4gwqBnYW1lc3RvcC9yZWZlcmVuY2UxIHZvbG1vZGUgZGVmYXVsdCBkZWZhdWx0Cj4gwqBn YW1lc3RvcC9yZWZlcmVuY2UxIHNuYXBzaG90X2xpbWl0IG5vbmUgZGVmYXVsdAo+IMKgZ2FtZXN0 b3AvcmVmZXJlbmNlMSBzbmFwc2hvdF9jb3VudCBub25lIGRlZmF1bHQKPiDCoGdhbWVzdG9wL3Jl ZmVyZW5jZTEgcmVkdW5kYW50X21ldGFkYXRhIGFsbCBkZWZhdWx0IAo+Cj4gW3Jvb3RAc2FuMTp+ XSMgenBvb2wgc3RhdHVzIGdhbWVzdG9wCj4gwqBwb29sOiBnYW1lc3RvcAo+IMKgc3RhdGU6IE9O TElORQo+IMKgc2Nhbjogbm9uZSByZXF1ZXN0ZWQKPiDCoGNvbmZpZzoKPgo+IMKgTkFNRSBTVEFU RSBSRUFEIFdSSVRFIENLU1VNCj4gwqBnYW1lc3RvcCBPTkxJTkUgMCAwIDAKPiDCoHJhaWR6MS0w IE9OTElORSAwIDAgMAo+IMKgZGE2IE9OTElORSAwIDAgMAo+IMKgZGE3IE9OTElORSAwIDAgMAo+ IMKgZGE4IE9OTElORSAwIDAgMAo+IMKgZGE5IE9OTElORSAwIDAgMAo+IMKgZGExMSBPTkxJTkUg MCAwIDAKPgo+IMKgZXJyb3JzOiBObyBrbm93biBkYXRhIGVycm9ycyAKPgo+IG9yLCBhbm90aGVy IHNlcnZlciAob3ZlcmhlYWQgaW4gdGhpcyBjYXNlIGlzbid0IHRoYXQgYmlnLCBidXQgc3RpbGwK PiBjb25zaWRlcmFibGUpOiAKPgo+IFtyb290QHNhbjAxOn5dIyB6ZnMgZ2V0IGFsbCBkYXRhL3Jl ZmVyZW5jZTEKPiDCoE5BTUUgUFJPUEVSVFkgVkFMVUUgU09VUkNFCj4gwqBkYXRhL3JlZmVyZW5j ZTEgdHlwZSB2b2x1bWUgLQo+IMKgZGF0YS9yZWZlcmVuY2UxIGNyZWF0aW9uIEZyaSBKYW4gNiAx MToyMyAyMDE3IC0KPiDCoGRhdGEvcmVmZXJlbmNlMSB1c2VkIDMuODJUIC0KPiDCoGRhdGEvcmVm ZXJlbmNlMSBhdmFpbGFibGUgMTMuMFQgLQo+IMKgZGF0YS9yZWZlcmVuY2UxIHJlZmVyZW5jZWQg My4yMlQgLQo+IMKgZGF0YS9yZWZlcmVuY2UxIGNvbXByZXNzcmF0aW8gMS4wMHggLQo+IMKgZGF0 YS9yZWZlcmVuY2UxIHJlc2VydmF0aW9uIG5vbmUgZGVmYXVsdAo+IMKgZGF0YS9yZWZlcmVuY2Ux IHZvbHNpemUgMlQgbG9jYWwKPiDCoGRhdGEvcmVmZXJlbmNlMSB2b2xibG9ja3NpemUgOEsgLQo+ IMKgZGF0YS9yZWZlcmVuY2UxIGNoZWNrc3VtIG9uIGRlZmF1bHQKPiDCoGRhdGEvcmVmZXJlbmNl MSBjb21wcmVzc2lvbiBvZmYgZGVmYXVsdAo+IMKgZGF0YS9yZWZlcmVuY2UxIHJlYWRvbmx5IG9m ZiBkZWZhdWx0Cj4gwqBkYXRhL3JlZmVyZW5jZTEgY29waWVzIDEgZGVmYXVsdAo+IMKgZGF0YS9y ZWZlcmVuY2UxIHJlZnJlc2VydmF0aW9uIG5vbmUgcmVjZWl2ZWQKPiDCoGRhdGEvcmVmZXJlbmNl MSBwcmltYXJ5Y2FjaGUgYWxsIGRlZmF1bHQKPiDCoGRhdGEvcmVmZXJlbmNlMSBzZWNvbmRhcnlj YWNoZSBhbGwgZGVmYXVsdAo+IMKgZGF0YS9yZWZlcmVuY2UxIHVzZWRieXNuYXBzaG90cyA2MTJH IC0KPiDCoGRhdGEvcmVmZXJlbmNlMSB1c2VkYnlkYXRhc2V0IDMuMjJUIC0KPiDCoGRhdGEvcmVm ZXJlbmNlMSB1c2VkYnljaGlsZHJlbiAwIC0KPiDCoGRhdGEvcmVmZXJlbmNlMSB1c2VkYnlyZWZy ZXNlcnZhdGlvbiAwIC0KPiDCoGRhdGEvcmVmZXJlbmNlMSBsb2diaWFzIGxhdGVuY3kgZGVmYXVs dAo+IMKgZGF0YS9yZWZlcmVuY2UxIGRlZHVwIG9mZiBkZWZhdWx0Cj4gwqBkYXRhL3JlZmVyZW5j ZTEgbWxzbGFiZWwgLQo+IMKgZGF0YS9yZWZlcmVuY2UxIHN5bmMgc3RhbmRhcmQgZGVmYXVsdAo+ IMKgZGF0YS9yZWZlcmVuY2UxIHJlZmNvbXByZXNzcmF0aW8gMS4wMHggLQo+IMKgZGF0YS9yZWZl cmVuY2UxIHdyaXR0ZW4gNDk4SyAtCj4gwqBkYXRhL3JlZmVyZW5jZTEgbG9naWNhbHVzZWQgMi4z N1QgLQo+IMKgZGF0YS9yZWZlcmVuY2UxIGxvZ2ljYWxyZWZlcmVuY2VkIDIuMDBUIC0KPiDCoGRh dGEvcmVmZXJlbmNlMSB2b2xtb2RlIGRlZmF1bHQgZGVmYXVsdAo+IMKgZGF0YS9yZWZlcmVuY2Ux IHNuYXBzaG90X2xpbWl0IG5vbmUgZGVmYXVsdAo+IMKgZGF0YS9yZWZlcmVuY2UxIHNuYXBzaG90 X2NvdW50IG5vbmUgZGVmYXVsdAo+IMKgZGF0YS9yZWZlcmVuY2UxIHJlZHVuZGFudF9tZXRhZGF0 YSBhbGwgZGVmYXVsdAo+IMKgW3Jvb3RAc2FuMDE6fl0jIHpwb29sIHN0YXR1cyBnYW1lc3RvcAo+ IMKgcG9vbDogZGF0YQo+IMKgc3RhdGU6IE9OTElORQo+IMKgc2Nhbjogbm9uZSByZXF1ZXN0ZWQK PiDCoGNvbmZpZzoKPgo+IMKgTkFNRSBTVEFURSBSRUFEIFdSSVRFIENLU1VNCj4gwqBkYXRhIE9O TElORSAwIDAgMAo+IMKgcmFpZHoxLTAgT05MSU5FIDAgMCAwCj4gwqBkYTMgT05MSU5FIDAgMCAw Cj4gwqBkYTQgT05MSU5FIDAgMCAwCj4gwqBkYTUgT05MSU5FIDAgMCAwCj4gwqBkYTYgT05MSU5F IDAgMCAwCj4gwqBkYTcgT05MSU5FIDAgMCAwCj4gwqByYWlkejEtMSBPTkxJTkUgMCAwIDAKPiDC oGRhOCBPTkxJTkUgMCAwIDAKPiDCoGRhOSBPTkxJTkUgMCAwIDAKPiDCoGRhMTAgT05MSU5FIDAg MCAwCj4gwqBkYTExIE9OTElORSAwIDAgMAo+IMKgZGExMiBPTkxJTkUgMCAwIDAKPiDCoHJhaWR6 MS0yIE9OTElORSAwIDAgMAo+IMKgZGExMyBPTkxJTkUgMCAwIDAKPiDCoGRhMTQgT05MSU5FIDAg MCAwCj4gwqBkYTE1IE9OTElORSAwIDAgMAo+IMKgZGExNiBPTkxJTkUgMCAwIDAKPiDCoGRhMTcg T05MSU5FIDAgMCAwCj4KPiDCoGVycm9yczogTm8ga25vd24gZGF0YSBlcnJvcnMgCj4KPiBTbyBt eSBxdWVzdGlvbiBpcyAtIGhvdyB0byBhdm9pZCBpdCA/IFJpZ2h0IG5vdyBJJ20gZXhwZXJpbWVu dGluZyB3aXRoCj4gdGhlIHZvbGJsb2Nrc2l6ZSwgbWFraW5nIGl0IGFyb3VuZCA2NGsuIEknbSBh bHNvIHN1c3BlY3RpbmcgdGhhdCBzdWNoCj4gb3ZlcmhlYWQgbWF5IGJlIHRoZSBzdWJzZXF1ZW5j ZSBvZiB0aGUgdmFyaW91cyByZXNpemluZyBvcGVyYXRpb25zLCBsaWtlCj4gZXh0ZW5pbmcgdGhl IHZvbHNpemUgb2YgdGhlIHZvbHVtZSBvciBhZGRpbmcgbmV3IGRpc2tzIGludG8gdGhlIHBvb2ws Cj4gYmVjYXVzZSBJIGhhdmUgYSBjb3VwbGUgb2Ygc2VydmVycyB3aXRoIHJhaWR6IHdoZXJlIHRo ZSBpbml0aWFsCj4gZGlzay92b2xzaXplIGNvbmZpZ3VyYXRpb24gZGlkbid0IGNoYW5nZSwgYW5k IHRoZSByZWZlcmVuY2VkL3ZvbHNpemUKPiBudW1iZXJzIGFyZSBwcmV0dHkgbXVjaCBjbG9zZSB0 byBlYWNoIG90aGVyLiAKPgo+IEV1Z2VuZS4gCj4KPiBMaW5rczoKPiAtLS0tLS0KPiBbMV0KPiBo dHRwOi8vc2VydmVyZmF1bHQuY29tL3F1ZXN0aW9ucy81MTIwMTgvc3RyYW5nZS16ZnMtZGlzay1z cGFjZS11c2FnZS1yZXBvcnQtZm9yLWEtenZvbAo+IF9fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fCj4gZnJlZWJzZC1mc0BmcmVlYnNkLm9yZyBtYWlsaW5nIGxp c3QKPiBodHRwczovL2xpc3RzLmZyZWVic2Qub3JnL21haWxtYW4vbGlzdGluZm8vZnJlZWJzZC1m cwo+IFRvIHVuc3Vic2NyaWJlLCBzZW5kIGFueSBtYWlsIHRvICJmcmVlYnNkLWZzLXVuc3Vic2Ny aWJlQGZyZWVic2Qub3JnIgoKSGVsbG8hCgpUaGlzIHByb2JsZW0gaXMgZmFyIGZyb20gbmV3LCBJ IGludmVzdGlnYXRlZCB0aGlzIGJhY2sgaW4gMjAxMyBhbmQgcG9zdGVkIG15IGZpbmRpbmdzIGhl cmU6Cmh0dHBzOi8vZm9ydW1zLmZyZWVic2Qub3JnL3RocmVhZHMvMzczNjUvCgovSw==
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1b54a2fe35407a95edca1f992fa08a71>