Date: Tue, 10 Feb 2009 15:22:30 +0100 From: Sebastiaan van Erk <sebster@sebster.com> To: freebsd-performance@freebsd.org Subject: Re: FreeBSD 7.1 disk performance issue on ESXi 3.5 Message-ID: <49918DA6.4020608@sebster.com> In-Reply-To: <gms1i4$3vq$1@ger.gmane.org> References: <499165F3.6050803@sebster.com> <gms1i4$3vq$1@ger.gmane.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a cryptographically signed message in MIME format. --------------ms020902030302070808040208 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Ivan Voras wrote: Hi, Thanks for the reply. > Sebastiaan van Erk wrote: >> Hi, [snip] >> <n> 1 2 4 >> freebsd 12.0009 13.6348 12.9402 (MB/s) >> linux 376.145 651.314 634.649 (MB/s) >> >> Both virtual machines run dbench 3.04 and the results are extremely >> stable over repeated runs. > VMWare has many optimizations for Linux that are not used with FreeBSD. > VMI, for example, makes the Linux guest paravirtualized, and then there > are special drivers for networking, its vmotion driver (this one > probably doesn't contribute to performance much), etc. and Linux is in > any case much better tested and supported. VMI/paravirtualization is not enabled for this Linux host. Neither is VMotion. Networking is performing extremely well (see also below). > If VMWare allows, you may try changing the type of the controller (I > don't know about ESXi but VMWare Server supports LSI or Buslogic SCSI > emulation) or switch to ATA emulation and try again. I tried this, and it has no significant effect. Just for completeness, here's the relevant output of dmesg: bt0: <Buslogic Multi-Master SCSI Host Adapter> port 0x1060-0x107f mem 0xf4810000-0xf481001f irq 17 at device 16.0 on pci0 bt0: BT-958 FW Rev. 5.07B Ultra Wide SCSI Host Adapter, SCSI ID 7, 192 CCBs bt0: [GIANT-LOCKED] bt0: [ITHREAD] da0 at bt0 bus 0 target 0 lun 0 da0: <VMware Virtual disk 1.0> Fixed Direct Access SCSI-2 device da0: 40.000MB/s transfers (20.000MHz DT, offset 15, 16bit) da0: 8192MB (16777216 512 byte sectors: 255H 63S/T 1044C) The transfer rate for dbench 1 is 15.0118 MB/s. > A generic optimization is to reduce kern.hz to something like 50 but it > probably won't help your disk performance. I already had this (not 50, but 100), but this doesn't do anything for the disk performance. > As for unixbench, you need to examine and compare each microbenchmark > result individually before drawing a conclusion. Yes, I realize that. However the dbench result is my first priority, and when (if) that is fixed, I'll run the unixbench again and see what my next priority is. (However, just to give you an idea I attached the basic 5.1.2 unixbench outputs (the CPU info for FreeBSD is "fake", since unixbench does a cat /proc/cpuinfo, so I removed the /proc/ part and copied the output under linux to the "procinfo" file.) Finally, I also ran some network benchmarks such as netio, and tested VM to VM communication on *different* ESXi machines connected via Gigabit ethernet, and it achieved more than 100MB/s throughput. Since CPU speed + Network IO are doing just fine, I'm guessing this is a pure disk (driver?) related issue. However, to go into production with FreeBSD I *must* be able to fix it. Note also the discrepency: 12 MB/s vs 350 MB/s on disk access! My lousy home machine (FreeBSD) is even 5 times faster at 60 MB/s. This machine has extremely fast disks in a RAID10 configuration. Any ideas are welcome! Regards, Sebastiaan --------------ms020902030302070808040208 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJUTCC AwMwggJsoAMCAQICEFN8DarMNuuKJDEtfs0UaqUwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UE BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA4MDYzMDEzNTE1N1oX DTA5MDYzMDEzNTE1N1owaDEQMA4GA1UEBBMHdmFuIEVyazETMBEGA1UEKhMKU2ViYXN0aWFh bjEbMBkGA1UEAxMSU2ViYXN0aWFhbiB2YW4gRXJrMSIwIAYJKoZIhvcNAQkBFhNzZWJzdGVy QHNlYnN0ZXIuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsJDDAeYHVmH/ GVxi+bhFx27dmg++9BdhPJfk8k041sqEqq7oXnR2GT54quY3Ac7A1BuOM2JvoICraGmjud4y b3EanRnqGIK6iH+VAhhTlV/Owrb2Qm1e13DLxwLp1SocSQl4IrEbF9Y5H3ASdIrE0iFqkpju nPiiHeNhz3LaI5ipjiluKYoH+F6gPx8njHoaDxPePCkSLg4r0IA0afLM74LVZxCRBZEfyRZS J6VVUJefKlz91dWSzR/3xSw/rO4u9Ds/Zh7VBUKy3K+YFryHxRpUek0gSepE1b70Q39L9Sqd M/NZqMvFpwrqgW2Zh2Nh8nqRge90maR4ypBzz3GzLwIDAQABozAwLjAeBgNVHREEFzAVgRNz ZWJzdGVyQHNlYnN0ZXIuY29tMAwGA1UdEwEB/wQCMAAwDQYJKoZIhvcNAQEFBQADgYEAS1Sk NMgDVzb0ktO9tPPacV0KdKhTYOHcICVmuDEe2sFHOkjLAI1iAKp640pqJEVqvRnfRcCFJ9hK koPjjVZ+ui2rVmJWBG6FSloLRS/YYED4tUAw6DQhK61UOpjkpQxjCdm+5bHG/2ZgJAda1j0x uiN822+xFkcaW/5PQgxSRxcwggMDMIICbKADAgECAhBTfA2qzDbriiQxLX7NFGqlMA0GCSqG SIb3DQEBBQUAMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAo UHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBD QTAeFw0wODA2MzAxMzUxNTdaFw0wOTA2MzAxMzUxNTdaMGgxEDAOBgNVBAQTB3ZhbiBFcmsx EzARBgNVBCoTClNlYmFzdGlhYW4xGzAZBgNVBAMTElNlYmFzdGlhYW4gdmFuIEVyazEiMCAG CSqGSIb3DQEJARYTc2Vic3RlckBzZWJzdGVyLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEP ADCCAQoCggEBALCQwwHmB1Zh/xlcYvm4Rcdu3ZoPvvQXYTyX5PJNONbKhKqu6F50dhk+eKrm NwHOwNQbjjNib6CAq2hpo7neMm9xGp0Z6hiCuoh/lQIYU5VfzsK29kJtXtdwy8cC6dUqHEkJ eCKxGxfWOR9wEnSKxNIhapKY7pz4oh3jYc9y2iOYqY4pbimKB/heoD8fJ4x6Gg8T3jwpEi4O K9CANGnyzO+C1WcQkQWRH8kWUielVVCXnypc/dXVks0f98UsP6zuLvQ7P2Ye1QVCstyvmBa8 h8UaVHpNIEnqRNW+9EN/S/UqnTPzWajLxacK6oFtmYdjYfJ6kYHvdJmkeMqQc89xsy8CAwEA AaMwMC4wHgYDVR0RBBcwFYETc2Vic3RlckBzZWJzdGVyLmNvbTAMBgNVHRMBAf8EAjAAMA0G CSqGSIb3DQEBBQUAA4GBAEtUpDTIA1c29JLTvbTz2nFdCnSoU2Dh3CAlZrgxHtrBRzpIywCN YgCqeuNKaiRFar0Z30XAhSfYSpKD441Wfrotq1ZiVgRuhUpaC0Uv2GBA+LVAMOg0ISutVDqY 5KUMYwnZvuWxxv9mYCQHWtY9MbojfNtvsRZHGlv+T0IMUkcXMIIDPzCCAqigAwIBAgIBDTAN BgkqhkiG9w0BAQUFADCB0TELMAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTES MBAGA1UEBxMJQ2FwZSBUb3duMRowGAYDVQQKExFUaGF3dGUgQ29uc3VsdGluZzEoMCYGA1UE CxMfQ2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjEkMCIGA1UEAxMbVGhhd3RlIFBl cnNvbmFsIEZyZWVtYWlsIENBMSswKQYJKoZIhvcNAQkBFhxwZXJzb25hbC1mcmVlbWFpbEB0 aGF3dGUuY29tMB4XDTAzMDcxNzAwMDAwMFoXDTEzMDcxNjIzNTk1OVowYjELMAkGA1UEBhMC WkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1Ro YXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMIGfMA0GCSqGSIb3DQEBAQUAA4GN ADCBiQKBgQDEpjxVc1X7TrnKmVoeaMB1BHCd3+n/ox7svc31W/Iadr1/DDph8r9RzgHU5VAK MNcCY1osiRVwjt3J8CuFWqo/cVbLrzwLB+fxH5E2JCoTzyvV84J3PQO+K/67GD4Hv0CAAmTX p6a7n2XRxSpUhQ9IBH+nttE8YQRAHmQZcmC3+wIDAQABo4GUMIGRMBIGA1UdEwEB/wQIMAYB Af8CAQAwQwYDVR0fBDwwOjA4oDagNIYyaHR0cDovL2NybC50aGF3dGUuY29tL1RoYXd0ZVBl cnNvbmFsRnJlZW1haWxDQS5jcmwwCwYDVR0PBAQDAgEGMCkGA1UdEQQiMCCkHjAcMRowGAYD VQQDExFQcml2YXRlTGFiZWwyLTEzODANBgkqhkiG9w0BAQUFAAOBgQBIjNFQg+oLLswNo2as Zw9/r6y+whehQ5aUnX9MIbj4Nh+qLZ82L8D0HFAgk3A8/a3hYWLD2ToZfoSxmRsAxRoLgnSe JVCUYsfbJ3FXJY3dqZw5jowgT2Vfldr394fWxghOrvbqNOUQGls1TXfjViF4gtwhGTXeJLHT HUb/XV9lTzGCA3EwggNtAgEBMHYwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBD b25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFp bCBJc3N1aW5nIENBAhBTfA2qzDbriiQxLX7NFGqlMAkGBSsOAwIaBQCgggHQMBgGCSqGSIb3 DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTA5MDIxMDE0MjIzMFowIwYJKoZI hvcNAQkEMRYEFM2GHFUoO3rYCVUucuTgOTCuq59JMF8GCSqGSIb3DQEJDzFSMFAwCwYJYIZI AWUDBAECMAoGCCqGSIb3DQMHMA4GCCqGSIb3DQMCAgIAgDANBggqhkiG9w0DAgIBQDAHBgUr DgMCBzANBggqhkiG9w0DAgIBKDCBhQYJKwYBBAGCNxAEMXgwdjBiMQswCQYDVQQGEwJaQTEl MCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3Rl IFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEFN8DarMNuuKJDEtfs0UaqUwgYcGCyqG SIb3DQEJEAILMXigdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRp bmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3Vp bmcgQ0ECEFN8DarMNuuKJDEtfs0UaqUwDQYJKoZIhvcNAQEBBQAEggEATk58hXtWjV8H4R8Y YTUMFtR1X8QiSv8jrN7hNE9pd9Sg6dlEZ2SbK9uiWFbAronwVNQ/0IhEcMEJm34ekWHGI1w5 QVzgOCVr5a1awulx7/Ns3LRsad7MXD05z7r8ifLRRhSQuUTXa/W22iwCiBcTiiXaUp5tr4z5 ztFSQMOJCVstXmxDec2ZKwnY2hS4BIEApG6RzWyAee9tk6KW3AUMQgnyguUuvi30OZxqG0cJ b9KDk3Bcbh4vWnBrbkijn62vrg3Tz2TZi+vMM+GCcq11KbT+Z5mAeKC7NOctEmq/7vt/gSsh w23tEH4CeaymVKKbrW42BZAmZvcChCQbXrukhAAAAAAAAA== --------------ms020902030302070808040208--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?49918DA6.4020608>