Date: Wed, 29 Jul 2015 16:37:00 -0500 From: Karl Denninger <karl@denninger.net> To: freebsd-fs@freebsd.org Subject: Re: pw operations slow under zfs load Message-ID: <55B9477C.2050302@denninger.net> In-Reply-To: <CAP1HOmTXTau5nALRCmpgQRoS%2B=sRMhKPd8L-t_us1_xG26XeTg@mail.gmail.com> References: <CAP1HOmTuxi7HL0_-SLJ8pd9s=3ob2D_ZU0pu%2BMSZfuxMPDydWw@mail.gmail.com> <CAP1HOmTXTau5nALRCmpgQRoS%2B=sRMhKPd8L-t_us1_xG26XeTg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a cryptographically signed message in MIME format. --------------ms040103040106040808090908 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 7/29/2015 16:13, javocado wrote: > Sorry, the boot disk (SSD) is plain old ufs. > > On Wed, Jul 29, 2015 at 2:11 PM, javocado <javocado@gmail.com> wrote: > >> Hi, >> >> We have a pretty busy ZFS pool running on an 8.3 AMD system. We are >> noticing that when the pool is busy pw-related operations seem to tak= e a >> long time to complete: >> >> # time pw unlock 1000 >> 0.007u 0.036s 0:39.72 0.0% 45+1953k 0+113io 0pf+0w >> >> # time pw lock 1000 >> 0.032u 0.022s 1:09.63 0.0% 24+1132k 0+114io 0pf+0w >> >> Wile the command is running, we note that the process is locked in the= D >> state: >> >> root 85051 0.0 0.0 5832 960 0 D+ 1:53PM 0:00.02 >> /usr/sbin/pwd_mkdb -u 1000 /etc/master.passwd >> >> We also note that there is next to 0 disk activity on the boot volume:= >> >> # gstat -f ad >> >> dT: 1.005s w: 1.000s filter: ad >> L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name >> 0 0 0 0 0.0 0 0 0.0 0.0| ad6 >> 0 0 0 0 0.0 0 0 0.0 0.0| ad8 >> 0 0 0 0 0.0 0 0 0.0 0.0| ad10 >> 0 0 0 0 0.0 0 0 0.0 0.0| ad12 >> >> And plenty of free mem: >> >> Mem: 400M Active, 3391M Inact, 128G Wired, 1935M Cache, 14G Buf, 6055M= Free >> >> So, what's going on here? How does a busy pool with it's own set of dr= ives >> (which operate off an HBA) affect the speed of operations involving th= e >> boot volume (an SSD connected to the mobo)? >> >> >> A very common manifestation of the way ZFS and the VM system interact, unfortunately. It's better in 9.x and 10.x, and the patch I have against them makes it even better, but it still isn't completely resolved. --=20 Karl Denninger karl@denninger.net <mailto:karl@denninger.net> /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms040103040106040808090908 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGXzCC BlswggRDoAMCAQICASkwDQYJKoZIhvcNAQELBQAwgZAxCzAJBgNVBAYTAlVTMRAwDgYDVQQI EwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG9w0BCQEWE0N1ZGEg U3lzdGVtcyBMTEMgQ0EwHhcNMTUwNDIxMDIyMTU5WhcNMjAwNDE5MDIyMTU5WjBaMQswCQYD VQQGEwJVUzEQMA4GA1UECBMHRmxvcmlkYTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1zIExMQzEe MBwGA1UEAxMVS2FybCBEZW5uaW5nZXIgKE9DU1ApMIICIjANBgkqhkiG9w0BAQEFAAOCAg8A MIICCgKCAgEAuYRY+EB2mGtZ3grlVO8TmnEvduVFA/IYXcCmNSOC1q+pTVjylsjcHKBcOPb9 TP1KLxdWP+Q1soSORGHlKw2/HcVzShDW5WPIKrvML+Ry0XvIvNBu9adTiCsA9nci4Cnf98XE hVpenER0qbJkBUOGT1rP4iAcfjet0lEgzPEnm+pAxv6fYSNp1WqIY9u0b1pkQiaWrt8hgNOc rJOiLbc8CeQ/DBP6rUiQjYNO9/aPNauEtHkNNfR9RgLSfGUdZuOCmJqnIla1HsrZhA5p69Bv /e832BKiNPaH5wF6btAiPpTr2sRhwQO8/IIxcRX1Vxd1yZbjYtJGw+9lwEcWRYAmoxkzKLPi S6Zo/6z5wgNpeK1H+zOioMoZIczgI8BlX1iHxqy/FAvm4PHPnC8s+BLnJLwr+jvMNHm82QwL J9hC5Ho8AnFU6TkCuq+P2V8/clJVqnBuvTUKhYMGSm4mUp+lAgR4L+lwIEqSeWVsxirIcE7Z OKkvI7k5x3WeE3+c6w74L6PfWVAd84xFlo9DKRdU9YbkFuFZPu21fi/LmE5brImB5P+jdqnK eWnVwRq+RBFLy4kehCzMXooitAwgP8l/JJa9VDiSyd/PAHaVGiat2vCdDh4b8cFL7SV6jPA4 k0MgGUA/6Et7wDmhZmCigggr9K6VQCx8jpKB3x1NlNNiaWECAwEAAaOB9DCB8TA3BggrBgEF BQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNV HRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIFoDALBgNVHQ8EBAMCBeAwLAYJYIZIAYb4QgENBB8W HU9wZW5TU0wgR2VuZXJhdGVkIENlcnRpZmljYXRlMB0GA1UdDgQWBBTFHJQt6cloXBdG1Pv1 o2YgH+7lWTAfBgNVHSMEGDAWgBQkcZudhX383d29sMqSlAOh+tNtNTAdBgNVHREEFjAUgRJr YXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcNAQELBQADggIBAE9/dxi2YqjCYYhiybp4GKcm 7tBVa/GLW+qcHPcoT4dqmqghlLz8+iUH+HCJjRQATVGyMEnvISOKFVHC6aZIG+Sg7J8bfS4+ fjKDi9smRH2VPPx3bV8+yFYRNroMGHaPHZB/Xctmmvc+PZ9O2W7rExgrODtxIOB3Zs6wkYf+ ty+9r1KmTHlV+rRHI6timH1uiyFE3cPi1taAEBxf0851cJV8k40PGF8G48ewnq8SY9sCf5cv liXbpdgU+I4ND5BuTjg63WS32zuhLd1VSuH3ZC/QbcncMX5W3oLXmcQP5/5uTiBJy74kdPtG MSZ9rXwZPwNxP/8PXMSR7ViaFvjUkf4bJlyENFa2PGxLk4EUzOuO7t3brjMlQW1fuInfG+ko 3tVxko20Hp0tKGPe/9cOxBVBZeZH/VgpZn3cLculGzZjmdh2fqAQ6kv9Z9AVOG1+dq0c1zt8 2zm+Oi1pikGXkfz5UJq60psY6zbX25BuEZkthO/qiS4pxjxb7gQkS0rTEHTy+qv0l3QVL0wa NAT74Zaj7l5DEW3qdQQ0dtVieyvptg9CxkfQJE3JyBMb0zBj9Qhc5/hbTfhSlHzZMEbUuIyx h9vxqFAmGzfB1/WfOKkiNHChkpPW8ZeH9yPeDBKvrgZ96dREHFoVkDk7Vpw5lSM+tFOfdyLg xxhb/RZVUDeUMYIE4zCCBN8CAQEwgZYwgZAxCzAJBgNVBAYTAlVTMRAwDgYDVQQIEwdGbG9y aWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBMTEMxHDAa BgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG9w0BCQEWE0N1ZGEgU3lzdGVt cyBMTEMgQ0ECASkwCQYFKw4DAhoFAKCCAiEwGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAc BgkqhkiG9w0BCQUxDxcNMTUwNzI5MjEzNzAwWjAjBgkqhkiG9w0BCQQxFgQULQ5jn4+UoMwa LdEkrxNsgSXY11wwbAYJKoZIhvcNAQkPMV8wXTALBglghkgBZQMEASowCwYJYIZIAWUDBAEC MAoGCCqGSIb3DQMHMA4GCCqGSIb3DQMCAgIAgDANBggqhkiG9w0DAgIBQDAHBgUrDgMCBzAN BggqhkiG9w0DAgIBKDCBpwYJKwYBBAGCNxAEMYGZMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBAgEpMIGpBgsqhkiG9w0BCRACCzGBmaCBljCBkDELMAkGA1UE BhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQ Q3VkYSBTeXN0ZW1zIExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqG SIb3DQEJARYTQ3VkYSBTeXN0ZW1zIExMQyBDQQIBKTANBgkqhkiG9w0BAQEFAASCAgBCAOqW Ir4L188HeZJKC5ohsASybhP2DyhbXBogFpCzSsaalfnA+98a64RhmdFVEYJa27JiTD5yZXZ6 ZfaPnffaei6OCWny0fjBE5o7vrmDUclMM42TSGTax75mMR5Ptvl1Nth5g6f1Enbts2SVd3Gy 0zvl9qyVShOv7DrBup/XfGr/P7XU3b65wnIfoJm+BRYoGVYrdnzHPsw3TQxcm6GAeCxxCCuz m/4oxqS8jq3IZWl5FAmC+oMK3uI3dH3BYdF/5ZZgjAJW+pkIi3AlU0/UiT1K3JAbI8g1z/Qx 294j5KeRYtAC7okjJna7ywzzUCwtIBLE2nZ8psFVQK/AmX0OY0ayEWL+NbGuo7LWHWylz8mo qH3fvV5ZpjxwjW2BfvnJ4dSXH+AqUhZcK2QsyXCbUOTNvxounGAILkWTH4qZPYXvy8CgEOZw f2c0Xfz6TRvKh4f8ko6NlUMTADDxeKfj0hFwcEP8E0QEoCPBhrHMJj0UT4g4J9OyZJ6306tX lCDsUYoBExxc7l5UDLKtCoS4nAyTXdyZY4zFiasVzzMzeBasMc5Rr4IuDHCHzmz0wcI7B80P ncbn39WoL4j1EVTDTv0kPnrH7Au/NS5QxiDA/HyklrHqeR6ciRD5fkNZAS5Rd5MFPkYd4JVB ArAa9H1gx9W/QH6w9dhVyM+0+yr/9AAAAAAAAA== --------------ms040103040106040808090908--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?55B9477C.2050302>