Date: Thu, 5 May 2011 01:33:49 -0400 (GMT-04:00) From: John <aqqa11@earthlink.net> To: freebsd-emulation@freebsd.org Subject: Re: virtualbox I/O 3 times slower than KVM? Message-ID: <17339409.1304573629442.JavaMail.root@elwamui-royal.atl.sa.earthlink.net>
next in thread | raw e-mail | index | archive | help
-----Original Message----- >From: Ivan Voras <ivoras@freebsd.org> >Sent: May 4, 2011 7:36 AM >To: freebsd-emulation@freebsd.org >Subject: Re: virtualbox I/O 3 times slower than KVM? > >On 03/05/2011 20:25, John wrote: > >> What I do is to cat the 330MB binary file (XP service pack from Microsoft) 20 times into a single 6.6GB file, "date" before and afterwards, and after the second date finishes, immediately Force power shut down. There are two observations: >> >> 1. the time to complete copying into this 6.6GB file were 72s, 44s, 79s in three runs, presumably because there is another production VM on the same host. The average is 65s, so it's about 100MB/s. >> 2. After immediately power down, I do found the resulting file was less than 6.6GB. So indeed the VM claimed the completion of the copying before it actually did. >> >> I then did the same thing on the virtualbox, since I don't want the above premature I/O, I made sure the "Use Host I/O cache" is unchecked for the VM storage. >> >> 1. the time to complete copying into this 6.6GB file was 119s and 92s, the average is 105s, so the speed is 62MB/s. >> 2. after immediately "Reset" the machine, I couldn't boot. Both times it asked me to do fsck for that partition (GPT 2.2T). But after finally powering up, I found the file was also less than 6.6GB both times as well. >> >> So looks like virtualbox also suffers caching problem? Or did I do anything wrong? > >Aaargh. Please don't invent benchmarks. The ones which are already >around have enough troubles by themselves. > >Instead, if you really want to do a proper comparison of the systems, do >this (in the order I've written): > >1) add these lines to /etc/sysctl.conf: > >vfs.hirunningspace=8388608 >vfs.lorunningspace=6291456 >vfs.read_max=128 > >2) configure the FreeBSD system to use AHCI > >http://ivoras.net/blog/tree/2009-11-17.trying-ahci-in-8.0.html > >3) use benchmarks/bonnie++ from ports for benchmarking. Note that you >need to run *the same version of the benchmark* on all systems (i.e. >also on Linux) to be able do do comparison. > >Any do-it-yourself benchmarks you do are very likely meaningless and you >cannot conclude anything from them. Even with this, there could be >*huge* differences in the results simply as a consequence of rotational >positions of virtual drives on the physical one. See this results (note >the transfer rate results). > >lara:~# diskinfo -vt /dev/ada0 >/dev/ada0 > 512 # sectorsize > 500107862016 # mediasize in bytes (466G) > 976773168 # mediasize in sectors > 0 # stripesize > 0 # stripeoffset > 969021 # Cylinders according to firmware. > 16 # Heads according to firmware. > 63 # Sectors according to firmware. > 6QG3Z026 # Disk ident. > >Seek times: > Full stroke: 250 iter in 5.671675 sec = 22.687 msec > Half stroke: 250 iter in 4.294470 sec = 17.178 msec > Quarter stroke: 500 iter in 6.793844 sec = 13.588 msec > Short forward: 400 iter in 2.637252 sec = 6.593 msec > Short backward: 400 iter in 2.320116 sec = 5.800 msec > Seq outer: 2048 iter in 0.217796 sec = 0.106 msec > Seq inner: 2048 iter in 0.218317 sec = 0.107 msec >Transfer rates: > outside: 102400 kbytes in 1.221390 sec = 83839 kbytes/sec > middle: 102400 kbytes in 1.445180 sec = 70856 kbytes/sec > inside: 102400 kbytes in 2.451970 sec = 41762 kbytes/sec > >Benchmarking file systems and drive IO is complicated :) > Well this quickly becomes a debate on what is the most meaningful (or meaningless) I/O benchmark, and clearly people even may not agree on existing I/O benchmarking tools, so I'm not sure how meaningful more test down the road will be. Some later discussions were quite informative, although let's not forget the issues related to virtualbox or/and its implementation of FreeBSD reflected from these tests. It's suffice to say, in the default configuration of FreeBSD (sync mount), at least in one situation (copying a few GB files) the guest FreeBSD of virtualbox has much slower I/O than the host FreeBSD (and the FreeBSD guest on FreeBSD virtualbox is clearly slower than CentOS guest on KVM CentOS, even though the FreeBSD host is slightly faster than CentOS host), AND the guest FreeBSD is fooled into prematurely declaring completion of writing and renders itself unable to boot after reset. Which, I believe, defeated the proud sync mount default of FreeBSD. Whether this can be a typical usage pattern, or whether there are other situation similar (or not similar) to this is another issue. Clearly this is related to virtualbox and its implementation on FreeBSD. I just wish it can improve over time, since virtualbox is the best virtualization solution on *BSD.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?17339409.1304573629442.JavaMail.root>