Date: Tue, 04 Mar 2014 14:44:51 -0600 From: Karl Denninger <karl@denninger.net> To: freebsd-fs@freebsd.org Subject: Re: Is LZ4 compression of the ZFS L2ARC available in any RELEASE/STABLE? Message-ID: <53163B43.7010009@denninger.net> In-Reply-To: <CAJ7kQyFf19Un_TS=kW=T21HT%2BoabhsUhJij5oixQ2_uh0LvHRA@mail.gmail.com> References: <CAJ7kQyGTOuynOoLukXbP2E6GPKRiBWx8_mLEchk90WDKO%2Bo-SA@mail.gmail.com> <53157CC2.8080107@FreeBSD.org> <CAJ7kQyGQjf_WbY64bLVX=YfmJUfAd8i22kVbVhZhEWPMg7bbQw@mail.gmail.com> <5315D446.3040701@freebsd.org> <CAJ7kQyFf19Un_TS=kW=T21HT%2BoabhsUhJij5oixQ2_uh0LvHRA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a cryptographically signed message in MIME format. --------------ms080201060205040708070300 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable There's all sorts of issues here and as someone who uses Postgres on ZFS/FreeBSD in a heavy mixed-I/O environment I'm pretty aware of them. Getting the L2ARC cache OFF the spindles where the data set is will make a huge difference in performance in any mixed I/O (read and write) environment. The interleaving of that data on the base data set is murder on spinning rust due to the requirement to move the heads. I strongly recommend starting there. UFS *may* be faster, but don't count on it as the small I/O issue still is a serious factor and seek times becomes the overwhelming latency problem very quickly with small I/Os. Remember too that you still need data protection (e.g. RAID, Gmirror, etc.) Benchmark for your workload and see. If that's not enough going to SSDs erases the head-movement penalty completely. You may well see improvements in net I/O under mixed database loads as high as 20 or more *times* (not percent) what you get from rotating media for this reason, especially with the better SSD devices. I have clocked (under synthetic conditions) improvements in I/O latency on "first accesses" for data not in-RAMcache as high as *one hundred times* if the workload includes interleaved writes (e.g. large numbers of clients who both need to read and write at once.) On 3/4/2014 2:33 PM, Olav Gjerde wrote: > I managed to mess up who I replied to and Matthew replied back with a g= ood > answer which I think didn't reach the mailing list. > > I actually have a problem with query performance in one of my databases= > related to running PostgreSQL on ZFS. Which is why I'm so interested in= > compression for the L2ARC Cache. The problem is random IO read were > creating a report were I aggregate 75000 rows takes 30 minutes!!! The t= able > that I query has 400 million rows though. > The dataset easily fit in memory, so if I run the same query again it t= akes > less than a second. > > I'm going to test UFS with my dataset, it may be a lot faster as you sa= id. > Currently I've only tested ZFS with gzip, lz4 and no compression. Gzip = and > no compression has about the same performance and LZ4 is about 20% > faster(for both read and write). LZ4 has a compressratio about 2.5 and > gzip9 has a compressratio that is about 4.5 > > Steven Hartland, thank you for your suggestion. I will try the 10-STABL= E > then instead of a RELEASE. > > > On Tue, Mar 4, 2014 at 2:25 PM, Matthew Seaman <matthew@freebsd.org> wr= ote: > >> On 03/04/14 12:17, Olav Gjerde wrote: >>> This is really great, I wonder how well it plays together with Postgr= eSQL >>> and a SSD. >> You probably *don't* want to turn on any sort of compression for a >> Postgresql cluster's data area (ie. /usr/local/pgsql) -- and there are= a >> bunch of other tuning things to make ZFS and Pg play well together, li= ke >> adjusting the ZFS block size. The sort of small random IOs that RDBMS= es >> do are hard work for any filesystem, but particularly difficult for ZF= S >> due to the copy-on-write semantics it uses. It's a lot easier to get >> good performance on a UFS partition. >> >> On the other hand, ZFS has recently grown TRIM support, which makes it= a >> much happier prospect on SSDs. >> >> Cheers, >> >> Matthew >> >> >> >> > --=20 -- Karl karl@denninger.net --------------ms080201060205040708070300 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFTzCC BUswggQzoAMCAQICAQgwDQYJKoZIhvcNAQEFBQAwgZ0xCzAJBgNVBAYTAlVTMRAwDgYDVQQI EwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkqhkiG9w0BCQEWIGN1c3Rv bWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0MB4XDTEzMDgyNDE5MDM0NFoXDTE4MDgyMzE5 MDM0NFowWzELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExFzAVBgNVBAMTDkthcmwg RGVubmluZ2VyMSEwHwYJKoZIhvcNAQkBFhJrYXJsQGRlbm5pbmdlci5uZXQwggIiMA0GCSqG SIb3DQEBAQUAA4ICDwAwggIKAoICAQC5n2KBrBmG22nVntVdvgKCB9UcnapNThrW1L+dq6th d9l4mj+qYMUpJ+8I0rTbY1dn21IXQBoBQmy8t1doKwmTdQ59F0FwZEPt/fGbRgBKVt3Quf6W 6n7kRk9MG6gdD7V9vPpFV41e+5MWYtqGWY3ScDP8SyYLjL/Xgr+5KFKkDfuubK8DeNqdLniV jHo/vqmIgO+6NgzPGPgmbutzFQXlxUqjiNAAKzF2+Tkddi+WKABrcc/EqnBb0X8GdqcIamO5 SyVmuM+7Zdns7D9pcV16zMMQ8LfNFQCDvbCuuQKMDg2F22x5ekYXpwjqTyfjcHBkWC8vFNoY 5aFMdyiN/Kkz0/kduP2ekYOgkRqcShfLEcG9SQ4LQZgqjMpTjSOGzBr3tOvVn5LkSJSHW2Z8 Q0dxSkvFG2/lsOWFbwQeeZSaBi5vRZCYCOf5tRd1+E93FyQfpt4vsrXshIAk7IK7f0qXvxP4 GDli5PKIEubD2Bn+gp3vB/DkfKySh5NBHVB+OPCoXRUWBkQxme65wBO02OZZt0k8Iq0i4Rci WV6z+lQHqDKtaVGgMsHn6PoeYhjf5Al5SP+U3imTjF2aCca1iDB5JOccX04MNljvifXgcbJN nkMgrzmm1ZgJ1PLur/ADWPlnz45quOhHg1TfUCLfI/DzgG7Z6u+oy4siQuFr9QT0MQIDAQAB o4HWMIHTMAkGA1UdEwQCMAAwEQYJYIZIAYb4QgEBBAQDAgWgMAsGA1UdDwQEAwIF4DAsBglg hkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFHw4 +LnuALyLA5Cgy7T5ZAX1WzKPMB8GA1UdIwQYMBaAFF3U3hpBZq40HB5VM7B44/gmXiI0MDgG CWCGSAGG+EIBAwQrFilodHRwczovL2N1ZGFzeXN0ZW1zLm5ldDoxMTQ0My9yZXZva2VkLmNy bDANBgkqhkiG9w0BAQUFAAOCAQEAZ0L4tQbBd0hd4wuw/YVqEBDDXJ54q2AoqQAmsOlnoxLO 31ehM/LvrTIP4yK2u1VmXtUumQ4Ao15JFM+xmwqtEGsh70RRrfVBAGd7KOZ3GB39FP2TgN/c L5fJKVxOqvEnW6cL9QtvUlcM3hXg8kDv60OB+LIcSE/P3/s+0tEpWPjxm3LHVE7JmPbZIcJ1 YMoZvHh0NSjY5D0HZlwtbDO7pDz9sZf1QEOgjH828fhtborkaHaUI46pmrMjiBnY6ujXMcWD pxtikki0zY22nrxfTs5xDWGxyrc/cmucjxClJF6+OYVUSaZhiiHfa9Pr+41okLgsRB0AmNwE f6ItY3TI8DGCBQowggUGAgEBMIGjMIGdMQswCQYDVQQGEwJVUzEQMA4GA1UECBMHRmxvcmlk YTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExDMRwwGgYD VQQDExNDdWRhIFN5c3RlbXMgTExDIENBMS8wLQYJKoZIhvcNAQkBFiBjdXN0b21lci1zZXJ2 aWNlQGN1ZGFzeXN0ZW1zLm5ldAIBCDAJBgUrDgMCGgUAoIICOzAYBgkqhkiG9w0BCQMxCwYJ KoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNDAzMDQyMDQ0NTFaMCMGCSqGSIb3DQEJBDEW BBQ6uNq6mmv/pu3qrculPHmmiynbOjBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjAL BglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFA MAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIG0BgkrBgEEAYI3EAQxgaYwgaMwgZ0xCzAJBgNV BAYTAlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoT EEN1ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkq hkiG9w0BCQEWIGN1c3RvbWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0AgEIMIG2BgsqhkiG 9w0BCRACCzGBpqCBozCBnTELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExEjAQBgNV BAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1zIExMQzEcMBoGA1UEAxMTQ3Vk YSBTeXN0ZW1zIExMQyBDQTEvMC0GCSqGSIb3DQEJARYgY3VzdG9tZXItc2VydmljZUBjdWRh c3lzdGVtcy5uZXQCAQgwDQYJKoZIhvcNAQEBBQAEggIAPLdba8uy2Mo6JAf9Vr7rxK1mxMCz 5rjc/TG5QlnLsRshQRXe1B1rTsJzMm3I5Fwg6OV+dSq77cpNqlxkKs8Wgc/Z1OlAUczxmbtJ eH11ZFlIakDoSR+j+T3MOEwKFwLI5GfX87NSvvn9xoHusVqeTH2237beV2tnufwUjujHDtpj 604gFYLQYP+5QzYoOPykCn/lNlGkIx0K0CY5smjTzD1mKErZ7e3fPXdwqB+EzAR0bK4T7Xy6 G7mW530IRcx9OYQj6K8ZAAYcApN51jfBRLhwQfqLydRukX3muyYL6c/QsI7kSAZ2Jt1tlUm0 oerM8qi9UQi+CaS37EEMgiM50dx/f2onb3VQ4WiV5MGkgxSYnlPJOTNPNzPAdiO1wed/w6T0 khquwywWg0SFkHZq+55elBkiXdVMPjqz4tuVXHuLZCIWKB51pcLzAPe1ImXd76xI5ERrkKE6 Ly6vuFbFRyIQACDVGVQv7lR2QXCDkD/ly1fquCjfX4acp2670vNn8OQwzEvxtbQ5daZaqp6X wYNxPqk5HWq7RBSM+LX1XM+tbmGWiCQupKBcQvdxi2kHHAsrncWk6+chxkMSATCJfaj1lJLc RlHVkmxW4m4IQcExXQUAlitOfjuGfsSptAbfGdhUp7fgR7i1PNR2QwBBC5EYbcLkwzBDfAup EpVCMe0AAAAAAAA= --------------ms080201060205040708070300--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?53163B43.7010009>