Date: Tue, 20 Jun 2017 12:58:21 -0500 From: Karl Denninger <karl@denninger.net> To: "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org> Subject: Re: FreeBSD 11.1 Beta 2 ZFS performance degradation on SSDs Message-ID: <4276a57b-5974-0d4b-c535-ea11c491b46f@denninger.net> In-Reply-To: <2d8a525e5355406aa7b8bf6690101008@DM2PR58MB013.032d.mgd.msft.net> References: <2d8a525e5355406aa7b8bf6690101008@DM2PR58MB013.032d.mgd.msft.net>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a cryptographically signed message in MIME format. --------------ms080006030407040800070301 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 6/20/2017 12:29, Caza, Aaron wrote: >> -----Original Message----- >> From: Karl Denninger [mailto:karl@denninger.net] >> Sent: Monday, June 19, 2017 7:28 PM >> To: freebsd-fs@freebsd.org >> Subject: Re: FreeBSD 11.1 Beta 2 ZFS performance degradation on SSDs >> >> Just one note below... >> >> On 6/19/2017 19:57, Caza, Aaron wrote: >>> Note that file /testdb/test is 16GB, twice the size of ram available = in this system. The /testdb directory is a ZFS file system with recordsi= ze=3D8k, chosen as ultimately it's intended to host a PostgreSQL database= which uses an 8k page size. >> Do not make this assumption blindly. Yes, I know the docs say to set >> recordsize=3D8k but this is something you need to benchmark against yo= ur >> actual working data set. >> >> MANY Postgres workloads are MUCH faster (2x or more!) if you use a >> default page size and lz4 compression -- including one I have in >> production and have extensively benchmarked. The difference is NOT sm= all.. >> .... >> >> zroot/ticker compressratio 1.53x - >> zroot/ticker mounted yes - >> zroot/ticker quota none defa= ult >> zroot/ticker reservation none defa= ult >> zroot/ticker recordsize 128K defa= ult >> zroot/ticker mountpoint /usr/local/pgsql/data-ticker loca= l >> zroot/ticker sharenfs off defa= ult >> zroot/ticker checksum fletcher4 >> inherited from zroot >> zroot/ticker compression lz4 >> inherited from zroot >> zroot/ticker atime off >> inherited from zroot >> >> You may also want to consider setting logbias=3Dthroughput. In some c= ases >> the improvement there can be quite material as well -- depending on th= e >> insert/update traffic to the database in question. >> >> -- >> Karl Denninger >> karl@denninger.net <mailto:karl@denninger.net> >> /The Market Ticker/ >> /[S/MIME encrypted email preferred]/ > Thanks for the suggestions Karl. I'll investigate further after I reso= lve this performance degradation issue I'm experiencing. I recently read= another FreeBSD+ZFS+PostgreSQL user's Scale15x presentation, PostgreZFS,= Sean Chittenden if I recall correctly, who also advised lz4 compression = + 16K page size rather than 8K with PostgreZFS. > > With regards to my performance woes, I was originally using PostgreSQL = in my posts to freebsd-hackers@freebsd.org but started using 'dd' to remo= ve it as a point of contention. In attempting to resolve this issue, I t= ried using your patch to PR 187594 (https://bugs.freebsd.org/bugzilla/sho= w_bug.cgi?id=3D187594). Took a bit of effort to find a revision of FreeB= SD 10 Stable to which your FreeBSD10 patch would both apply and compile c= leanly; however, it didn't resolve the issue I'm experiencing. I would not have expected my PR to impact this issue. I suspicious of a drive firmware interaction with your I/O pattern; SSDs are somewhat-notorious for having that come up under certain workloads that involve a lot of writes. --=20 Karl Denninger karl@denninger.net <mailto:karl@denninger.net> /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms080006030407040800070301 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC BlwwggZYMIIEQKADAgECAgE9MA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE2MTIxODE5NDUzNVoXDTIxMTIxNzE5NDUzNVowVzEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxGzAZBgNVBAMUEmthcmxAZGVubmluZ2VyLm5ldDCCAiIwDQYJKoZIhvcNAQEBBQADggIP ADCCAgoCggIBAM2N5maxs7NkoY9g5NMxFWll0TYiO7gXrGZTo3q25ZJgNdPMwrntLz/5ewE9 07TEbwJ3ah/Ep9BfZm7JF9vTtE1HkgKtXNKi0pawNGm1Yn26Dz5AbUr1byby6dFtDJr14E07 trzDCtRRvTkOVSBj6PQPal0fAnDtkIYQBVcuMkXkuMCtyfE95pjm8g4K9l7lAcKii3T1/3rE hCc1o2nBnb7EN1/XwBeCDGB+I2SN/ftZDbKQqGAF5q9dUn+iXU7Z/CVSfUWmhVh6cVZA4Ftv TglUqj410OuPx+cUQch3h1kFgsuhQR63HiJc3HbRJllHsV0rihvL1CjeARQkhnA6uY9NLFST p5I/PfzBzW2MSmtN/tGZvmfKKnmtbfUNgkzbIR1K3lsum+yEL71kB93Xtz/4f1demEx5c8TJ RBIniDHjDeLGK1aoBu8nfnvXAvgthFNTWBOEoR49AHEPjC3kZj0l8JQml1Y8bTQD5gtC5txl klO60WV0EufU7Hy9CmynMuFtjiA2v71pm097rXeCdrAKgisdYeEESB+SFrlY65rLiLv4n8o1 PX7DqRfqKkOYIakZ0ug/yHVKcq2EM3RiJxwzls5gT70CoOBlKbrC98O8TA6teON0Jq30M06t NTI2HhvNbJDLbBH+Awf4h1UKB+0ufENwjVvF5Jfz8Ww/FaSDAgMBAAGjgfQwgfEwNwYIKwYB BQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgwCQYD VR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIBDQQf Fh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUpfAI3y+751pp9A0w 6vJHx8RoR/MwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYwFIES a2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBiB6MlugxYJdccD8boZ/u8 d8VxmLkJCtbfyYHRjYdyoABLW5hE3k3xSpYCM9L7vzWyV/UWwDYKi4ZzxHo4g+jG/GQZfKhx v38BQjL2G9xD0Hn2d+cygOq3UPjVYlbbfQoew6JbyCFXrrZ7/0jvRMLAN2+bRC7ynaFUixPH Whnj9JSH7ieYdzak8KN+G2coIC2t2iyfXVKehzi5gdNQ0vJ7+ypbGsRm4gE8Mdo9N/WgFPvZ HPFqR9Dwas7Z+aHwOabpk5r/336SyjOaZsn3MqKJQZL6GqDKusVOCWt+9uFAD8kadg7FetZe atIoD9I+zbp59oVoMnkMDMx7Hi85faU03csusqMGsjSsAzWSI1N8PJytZlchLiykokLKc3OL G87QKlErotlou7cfPX2BbEAH5wmkj9oiqZhxIL/wwAUA+PkiTbEmksKBNompSjUq/6UsR8EA s74gnu17lmijv8mrg2qMlwRirE7qG8pnE8egLtCDxcjd0Of9WMi2NJskn0/ovC7P+J60Napl m3ZIgPJst1piYSE0Zc1FIat4fFphMfK5v4iLblo1tFSlkdx1UNDGdg/U+LaXkNVXlMp8fyPm R80V6cIrCAlEWnBJNxG1UyfbbsvNMCCZBM4faGGsR/hhQOiydlruxhjL6P8J2WV8p11DdeGx KymWoil2s1J5WTGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMHRmxv cmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExDMRww GgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5c3Rl bXMgTExDIENBAgE9MA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZIhvcN AQcBMBwGCSqGSIb3DQEJBTEPFw0xNzA2MjAxNzU4MjFaME8GCSqGSIb3DQEJBDFCBECUyv9M wX6pcPihyfH7yS/VQif4zKEwwBc7+3W0Y3J/TEfuI6aEAkJxg00lchib+uT3uL45nltSjVCS fChiCMVJMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAKBggq hkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYIKoZI hvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNVBAgT B0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1zIExM QzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3VkYSBT eXN0ZW1zIExMQyBDQQIBPTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYTAlVT MRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEg U3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG9w0B CQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECAT0wDQYJKoZIhvcNAQEBBQAEggIABu52BojYATqQ 39/cDBA1Ve2tUHd64HEkGmsO53xYdwgyE2FsK3fN5c7TqsbkivzUEw0EfIiXDIXtdCsOwGlF eOBUd9IBjaeanz5nzC/szpwD0aXLOXtT2KQYR0Hvq6M7Wh2b8HChzORInNq7grDIUFdO2KSl PlmSuGc4yTI5wowAZDj2hc6V0A944dobWbQ7Ur5TXrth1Mba81FAPM64Lp/jYc+rZ0K2obpM wMiyuNezl8H28aKxzE77pi97MOn6WvPl3UlZZgsllwPcNYmpOUW1oybEl7gu/n0RmnSRI3yY htIP0kkW/v33wBC1TAv3sCjxCj3Quyg3S+NZc0fTg3ElitgSa7aUK7C1Y3iJMfxWroqSsrB1 vdO9zraYEk6RVFv+oaGsNjRdBO5REU059z2yxJLfYPrllAKroG6oIOOaeVj+r5XXJtNs869H V3nFGCQHl0kYUDodEJ0YsLULgsBdyW2+1apYq2vfZfQBfo1hQaxkYO6apURLdMxyQZ21VIva qylRwNRWl9YPAEFbnPe4URO96R0unPLi61KEmpLBYpyEgPx2JvOuazkIub+QqOKF9M2ViDPJ s/0/WiZc+N6i99uDNZBkEG2JqXWTyhgUgh+uFa9AjtMK4dXAU+4F/vIO2IjXY3vv+VK+frEF xPJXN9gf3TxqiqHaGUdrEYoAAAAAAAA= --------------ms080006030407040800070301--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4276a57b-5974-0d4b-c535-ea11c491b46f>