Date: Tue, 18 Mar 2014 08:50:29 -0500 From: Karl Denninger <karl@denninger.net> To: freebsd-fs@freebsd.org Subject: Re: Reoccurring ZFS performance problems [RESOLVED] Message-ID: <53284F25.9070007@denninger.net> In-Reply-To: <53284973.8010203@netlabs.org> References: <531E2406.8010301@denninger.net> <5320A0E8.2070406@denninger.net> <5322E64E.8020009@denninger.net> <CA%2BD9QhtguPKD9zQ35246LkMt5gTU6MJ%2BZBigoznk7FHQ4R0nhA@mail.gmail.com> <CA%2BD9Qht_7dW6gahTFKz7B9%2BpgJemkFoPLLcASmPvCfGytxF8cQ@mail.gmail.com> <53236BF3.9060500@denninger.net> <CA%2BD9QhstDyetoA5HdwyMA1BOyafMJ%2BbrxDEvdNHUXFPa3YGPtg@mail.gmail.com> <c5a61ac31b121de10f9b065967fe1ae3@mail.mikej.com> <53284973.8010203@netlabs.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a cryptographically signed message in MIME format. --------------ms000907020706050007040602 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable On 3/18/2014 8:26 AM, Adrian Gschwend wrote: > On 18.03.14 11:26, mikej wrote: > >> I am a little surprised this thread has been so quiet. I have been >> running with this patch and my desktop is more pleasant when memory >> demands are great - no more swapping - and wired no longer grows >> uncontrollable. >> >> Is more review coming the silence is deffining. > same here, works very nice so far and growth of memory looks much more > controlled now. Before within no time my server had all 16GB of RAM > wired, now it's growing only slowly. > > It's too early to say if my performance degradation is gone now but it > surely looks very good so far. > > Thanks again to Karl for the patch! Hope others test it and integrate i= t > soon. > Watch zfs-stats -A; you will see what the system has adapted to as=20 opposed to the hard limits in arc_max and arc_min. Changes upward in reservation percentage will be almost-instantly=20 reflected in reduced allocation, where changes downward will grow slowly = (there's a timed lockdown in the cache code that prevents it from=20 grabbing more space immediately when it was previously throttled back,=20 and the ARC cache in general only grows when I/O that is not in the=20 cache occurs, and thus new data becomes available to cache for later=20 re-use.) The nice thing about the way it behaves now is that it will release=20 memory immediately when required by other demands on the system but if=20 your active and inactive page count shrinks as images release RAM back=20 through the cache and then to the free list it will also be allowed to=20 expand as I/O demand diversity warrants. That was clearly the original design intent but it was being badly=20 frustrated by the former cache memory allocation behavior. There is an argument for not including cache pages in the "used" bucket=20 (that is, counting them as "free" instead); the way I coded it is a bit=20 more conservative than going the other way. Given the design of the VM=20 subsystem either is arguably acceptable since a cache page can be freed=20 when RAM is demanded. I decided not to do for two reasons -- first, a=20 page that is in the cache bucket could be reactivated and if it is then=20 you are going to have to release that ARC cache memory -- economy of=20 action suggests that you not do something you might quickly have to=20 undo. Second, my experience with the VM system over roughly a decade of = use of FreeBSD supports an argument that the VM implementation is=20 arguably the greatest strength that FreeBSD has, especially under=20 stress, and by allowing it to do its job rather than trying to "push"=20 the VM system to do a particular thing the philosophy of trusting that=20 which is believed to know what it's up to is maintained. --=20 -- Karl karl@denninger.net --------------ms000907020706050007040602 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFTzCC BUswggQzoAMCAQICAQgwDQYJKoZIhvcNAQEFBQAwgZ0xCzAJBgNVBAYTAlVTMRAwDgYDVQQI EwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkqhkiG9w0BCQEWIGN1c3Rv bWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0MB4XDTEzMDgyNDE5MDM0NFoXDTE4MDgyMzE5 MDM0NFowWzELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExFzAVBgNVBAMTDkthcmwg RGVubmluZ2VyMSEwHwYJKoZIhvcNAQkBFhJrYXJsQGRlbm5pbmdlci5uZXQwggIiMA0GCSqG SIb3DQEBAQUAA4ICDwAwggIKAoICAQC5n2KBrBmG22nVntVdvgKCB9UcnapNThrW1L+dq6th d9l4mj+qYMUpJ+8I0rTbY1dn21IXQBoBQmy8t1doKwmTdQ59F0FwZEPt/fGbRgBKVt3Quf6W 6n7kRk9MG6gdD7V9vPpFV41e+5MWYtqGWY3ScDP8SyYLjL/Xgr+5KFKkDfuubK8DeNqdLniV jHo/vqmIgO+6NgzPGPgmbutzFQXlxUqjiNAAKzF2+Tkddi+WKABrcc/EqnBb0X8GdqcIamO5 SyVmuM+7Zdns7D9pcV16zMMQ8LfNFQCDvbCuuQKMDg2F22x5ekYXpwjqTyfjcHBkWC8vFNoY 5aFMdyiN/Kkz0/kduP2ekYOgkRqcShfLEcG9SQ4LQZgqjMpTjSOGzBr3tOvVn5LkSJSHW2Z8 Q0dxSkvFG2/lsOWFbwQeeZSaBi5vRZCYCOf5tRd1+E93FyQfpt4vsrXshIAk7IK7f0qXvxP4 GDli5PKIEubD2Bn+gp3vB/DkfKySh5NBHVB+OPCoXRUWBkQxme65wBO02OZZt0k8Iq0i4Rci WV6z+lQHqDKtaVGgMsHn6PoeYhjf5Al5SP+U3imTjF2aCca1iDB5JOccX04MNljvifXgcbJN nkMgrzmm1ZgJ1PLur/ADWPlnz45quOhHg1TfUCLfI/DzgG7Z6u+oy4siQuFr9QT0MQIDAQAB o4HWMIHTMAkGA1UdEwQCMAAwEQYJYIZIAYb4QgEBBAQDAgWgMAsGA1UdDwQEAwIF4DAsBglg hkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFHw4 +LnuALyLA5Cgy7T5ZAX1WzKPMB8GA1UdIwQYMBaAFF3U3hpBZq40HB5VM7B44/gmXiI0MDgG CWCGSAGG+EIBAwQrFilodHRwczovL2N1ZGFzeXN0ZW1zLm5ldDoxMTQ0My9yZXZva2VkLmNy bDANBgkqhkiG9w0BAQUFAAOCAQEAZ0L4tQbBd0hd4wuw/YVqEBDDXJ54q2AoqQAmsOlnoxLO 31ehM/LvrTIP4yK2u1VmXtUumQ4Ao15JFM+xmwqtEGsh70RRrfVBAGd7KOZ3GB39FP2TgN/c L5fJKVxOqvEnW6cL9QtvUlcM3hXg8kDv60OB+LIcSE/P3/s+0tEpWPjxm3LHVE7JmPbZIcJ1 YMoZvHh0NSjY5D0HZlwtbDO7pDz9sZf1QEOgjH828fhtborkaHaUI46pmrMjiBnY6ujXMcWD pxtikki0zY22nrxfTs5xDWGxyrc/cmucjxClJF6+OYVUSaZhiiHfa9Pr+41okLgsRB0AmNwE f6ItY3TI8DGCBQowggUGAgEBMIGjMIGdMQswCQYDVQQGEwJVUzEQMA4GA1UECBMHRmxvcmlk YTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExDMRwwGgYD VQQDExNDdWRhIFN5c3RlbXMgTExDIENBMS8wLQYJKoZIhvcNAQkBFiBjdXN0b21lci1zZXJ2 aWNlQGN1ZGFzeXN0ZW1zLm5ldAIBCDAJBgUrDgMCGgUAoIICOzAYBgkqhkiG9w0BCQMxCwYJ KoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNDAzMTgxMzUwMjlaMCMGCSqGSIb3DQEJBDEW BBRx7pWv+R4AwATQb1Zp9ftapezq+DBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjAL BglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFA MAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIG0BgkrBgEEAYI3EAQxgaYwgaMwgZ0xCzAJBgNV BAYTAlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoT EEN1ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkq hkiG9w0BCQEWIGN1c3RvbWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0AgEIMIG2BgsqhkiG 9w0BCRACCzGBpqCBozCBnTELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExEjAQBgNV BAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1zIExMQzEcMBoGA1UEAxMTQ3Vk YSBTeXN0ZW1zIExMQyBDQTEvMC0GCSqGSIb3DQEJARYgY3VzdG9tZXItc2VydmljZUBjdWRh c3lzdGVtcy5uZXQCAQgwDQYJKoZIhvcNAQEBBQAEggIAmERrBrm5ROP7dMbMrWoyfE/nGjWF spmhk0LEm6WLzqUh3d0N5C15mvSgqjxEdAzD/ZF8mb1wGLJpt5EgqUUtARJ1SPcQ6ce0WoVg vfdDb6Q5oSo9WQ/+wbMDBooDElbPvt1pZ3eXpzvuuGNOgycXJDpXHsQ6D0K8kVz5QKpWrx0q jkmpHvpnDZFWbscnq9c1F4JEV/2qkQ19omW1iwDsNVAAQVBDz/0hqu8aytEPeB95ppzk2qFd bywVG0QpNQZpn1sSZ8OHtYwf77uAF+Pbj+5aXrmyHBYTWzAd+k1xfMvsBH7RmGpjoujfK+UH 9u9sQVLp0WlXU6vkRIqdZHZnFc5oQamyf/GHBzqRuybaeHv/7B672oWRFkVJ2y5iM1XV66tW PcSb6GB4NLQi8lpq0CD6tg6RtKXid9sDquG0l94zBUlGl91H/mF6ldSKjTx4WyOfMqMd+MQe 7CjBmvBdHnNOkWK3xrk+sj3tAA8WI227Y0IB1KAhYZywtNQvsfqneDtQ2dw/A8kAjpHy1EHJ 4xUS08WB70pCFCxY1/E7jEbjmzS5ExD5I0vH5zglDf1OaPltdN0eGoE9w7/IlhFfrPOxBLId hVJCOZULAKsjoOiCBndLV39iPWOiRImu6ONhYT7Ki39utsQkWLGCDijdPOk5swEawP7EhnrK rwRKEU4AAAAAAAA= --------------ms000907020706050007040602--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?53284F25.9070007>