From owner-freebsd-performance@FreeBSD.ORG Sun Jan 23 02:42:27 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EDCEB16A4CE for ; Sun, 23 Jan 2005 02:42:27 +0000 (GMT) Received: from web53908.mail.yahoo.com (web53908.mail.yahoo.com [206.190.36.218]) by mx1.FreeBSD.org (Postfix) with SMTP id B3A1D43D31 for ; Sun, 23 Jan 2005 02:42:26 +0000 (GMT) (envelope-from stheg_olloydson@yahoo.com) Received: (qmail 15692 invoked by uid 60001); 23 Jan 2005 02:42:26 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=Ihu9Xl1Ylpdcm0SnP2CPpa7QXyhzdMK2Yl7mgEUYjipBv73TkFB8v5ZvKjRzR4yx8hlGASnCiyAtMLi0RftInToUPcs5azxigdSxRU6eCqb9Cv62REVdoFFEsihHICQl4pNANsGaLsGOEa50ljV3MIx5zgej2ttkgjesAPvYvL0= ; Message-ID: <20050123024226.15690.qmail@web53908.mail.yahoo.com> Received: from [68.18.53.121] by web53908.mail.yahoo.com via HTTP; Sat, 22 Jan 2005 18:42:26 PST Date: Sat, 22 Jan 2005 18:42:26 -0800 (PST) From: stheg olloydson To: linicks@gmail.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii cc: freebsd-performance@freebsd.org cc: freebsd-questions@freebsd.org Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | More Info X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2005 02:42:28 -0000 it was said: >All, > This post is not about BSD VS. Linux and should not be taken that >way. I think that "Flame Wars/Engineer Wars" are waste of time and >energy. I was surprised by my test results and didn't want to take >FBSD out of the loop just yet. There may be flaws in my testing that >have led me to inaccurate results. I didn't share the testing details >in the original mail because of time constraints, and the notes are >fairly lengthy. I will add my notes to this mail so that there is a >better understanding of what tests I performed, and their results. >It's important to note that I did not tweak any of the default >settings of the OS or DB. The notes should be generally self >explanatory, but will be more that happy to clarify any questions that >you have. As a side note, I chose the email address linicks because >by name is Nick, and thought it was a fun play on words. I appreciate >all of your feedback, so that I can better understand the differences >in these great operating systems and communities. > >Thanks Again! >--Nick Pavlica Hello, I'm glad you weren't trolling. I, too, think the OS wars are a load of cark. Each OS has it strengths and weaknesses. Time is better spent increasing the strengths and fixing the weaknesses than arguing about whose are _better_. I can say right off that FBSD's out-of-the-box state is intended for stability rather than performance. The real question is what they do after tuning. Here I would expect FBSD to do somewhat better, especially on the uni-processor machine. Running the tests on the SC400 hardware won't be a problem for me, but I have no spare SMP or SCSI equipment to do the PE2400 tests - which I think would be the more interesting. Perhaps someone else on the list can do this? Not to provide a head-to-head showdown but to see if something is actually wrong that isn't already being looked at. (Everyone knows threading has problems that are being dealt with. That's why I'm not so sure FBSD will out-perform Fedora at this time on an SMP box.) To be on the safe side, I'll cc this to the performance list, as well. Maybe someone has already done something similar and has quick answers. Thus, I'm including unquoted the rest of your email below. Finally, the addy thing was just me getting in a shot at you if you had turned out to be trolling. Regards, stheg The tests and results below here. I don't know how the formatting is going to turn out. If its too mangled, see the original post on questions@ "OK, The testing notes already" :) --------------------------------------------------------------------------------------------------------------- Hardware Configs: Dell PE 2400 - Dual PIII 500Mhz - 512Mb Ram - Perc 2si controller - (2) 10k ultra160 drives in a raid 1 configuration. Dell SC400 - P4 2.4 Ghz (not hyperthreaded) - 512Mb Ram - Stock 40Gb IDE 7200RPM ---------------------------------------------------------------------------------------- Postgresql Test Scripts: CREATE TABLE test1 ( thedate TIMESTAMP, astring VARCHAR(200), anumber INTEGER ); CREATE FUNCTION build_data() RETURNS integer AS ' DECLARE i INTEGER DEFAULT 0; curtime TIMESTAMP; BEGIN FOR i IN 1..1000000 LOOP curtime := ''now''; INSERT INTO test1 VALUES (curtime, ''test string'', i); END LOOP; RETURN 1; END; ' LANGUAGE 'plpgsql'; SELECT build_data(); Then the following script is run under the time program to ascertain how long it takes to run: CREATE TABLE test2 ( thedate TIMESTAMP, astring VARCHAR(200), anumber INTEGER ); CREATE TABLE test3 AS SELECT * FROM test1; INSERT INTO test2 SELECT * FROM test1 WHERE ((anumber % 2) = 0); DELETE FROM test3 WHERE ((anumber % 2) = 0); DELETE FROM test3 WHERE ((anumber % 13) = 0); CREATE TABLE test4 AS SELECT test1.thedate AS t1date, test2.thedate AS t2date, test1.astring AS t1string, test2.astring AS t2string, test1.anumber AS t1number, test2.anumber AS t2number FROM test1 JOIN test2 ON test1.anumber=test2.anumber; UPDATE test3 SET thedate='now' WHERE ((anumber % 5) = 0); DROP TABLE test4; CREATE TABLE test4 AS SELECT * FROM test1; DELETE FROM test4 WHERE ((anumber % 27) = 0); VACUUM ANALYZE; VACUUM FULL; DROP TABLE test4; DROP TABLE test3; DROP TABLE test2; VACUUM FULL; ----------------------------------------------------------------------------------------- sc400 freeBSD5: $ time dd bs=1024 if=/dev/zero of=tstfile count=1M 1048576+0 records in 1048576+0 records out 1073741824 bytes transferred in 71.807645 secs (14953029 bytes/sec) 71.82real 0.68 user 8.83 sys 71.82 / 60 = 1.197 -- 517 nick.pavlica -16 0 1212K 588K wdrain 0:02 12.35% 5.91% dd 517 nick.pavlica -16 0 1212K 588K wdrain 0:13 12.48% 12.35% dd $ time dd bs=1024 if=/dev/zero of=tstfile count=2M 2097152+0 records in 2097152+0 records out 2147483648 bytes transferred in 136.815925 secs (15696153 bytes/sec) 136.85 real 1.29 user 17.49 sys 136.85 / 60 = 2.280833333 -- 542 nick.pavlica -16 0 1212K 588K wdrain 0:19 13.35% 13.33% dd 542 nick.pavlica -16 0 1212K 588K wdrain 0:24 12.99% 12.99% dd $ time dd bs=1024 if=/dev/zero of=tstfile count=3M 3145728+0 records in 3145728+0 records out 3221225472 bytes transferred in 205.722425 secs (15658115 bytes/sec) 205.72 real 1.82 user 27.39 sys 205.72 / 60 = 3.428666667 copy test: 558 nick.pavlica -4 0 1272K 680K getblk 0:01 2.30% 1.32% cp 558 nick.pavlica -4 0 1272K 680K getblk 0:02 1.80% 1.71% cp 558 nick.pavlica -4 0 1272K 680K getblk 0:03 1.87% 1.86% cp $ time cp tstfile tstfile2 579.31 real 0.03 user 14.61 sys 579.31 / 60 = 9.655166667 (FreeBSD 5.3+ on SC400) b test 1: 535 nick.pavlica -4 0 2380K 1216K getblk 0:17 2.84% 2.83% bonnie++ 568 nick.pavlica 105 0 2380K 1196K RUN 0:09 92.99% 36.62% bonnie++ 568 nick.pavlica -16 0 2380K 1192K wdrain 0:14 12.35% 11.23% bonnie++ $ bonnie++ -s 1024 -r 512 -n 5 Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.93c ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP sc400.e--station 1G 257 99 15135 5 10033 3 508 99 45572 10 124.3 5 Latency 33802us 161ms 467ms 32560us 17190us 537ms Version 1.93c ------Sequential Create------ --------Random Create-------- sc400.e--station.co -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 5 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ Latency 14891us 149us 226us 2671us 133us 1270us 1.93c,1.93c,sc400.e--station.com,1,1106248340,1G,,257,99,15135,5,10033,3,508,99, 45572,10,124.3,5,5,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+ ++,33802us,161ms,467ms,32560us,17190us,537ms,14891us,149us,226us,2671us,133us,12 70us B Test 2: 712 nick.pavlica -16 0 2380K 1212K wdrain 0:12 53.78% 33.01% bonnie++ 712 nick.pavlica -16 0 2380K 1208K wdrain 0:17 4.90% 4.88% bonnie++ $ bonnie++ -s 2048 -r 512 -n 5 Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.93c ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP sc400.e--station 2G 260 99 14394 5 9512 3 505 99 42976 9 94.7 4 Latency 32809us 284ms 470ms 30153us 15074us 3564ms Version 1.93c ------Sequential Create------ --------Random Create-------- sc400.e--station.co -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 5 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ Latency 6772us 1452us 175us 3964us 114us 46821us 1.93c,1.93c,sc400.e—station.com,1,1106252670,2G,,260,99,14394,5,9512,3,505,99,42976,9,94.7,4,5,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,32809us,284ms,470ms,30153us,15074us,3564ms,6772us,1452us,175us,3964us,114us,46821us Test 3: 754 nick.pavlica -16 0 2380K 1208K wdrain 0:13 26.89% 20.90% bonnie++ $ bonnie++ -s 3072 -r 512 -n 5 Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.93c ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP sc400.e--station 3G 259 99 14484 5 9671 3 507 99 40714 9 77.7 3 Latency 33254us 569ms 591ms 36025us 22245us 936ms Version 1.93c ------Sequential Create------ --------Random Create-------- sc400.e--station.co -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 5 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ Latency 13993us 154us 120us 20243us 555us 72us 1.93c,1.93c,sc400.e—station.com,1,1106253537,3G,,259,99,14484,5,9671,3,507,99,40714,9,77.7,3,5,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,33254us,569ms,591ms,36025us,22245us,936ms,13993us,154us,120us,20243us,555us,72us (PG Performance Test) 2039 pgsql -4 0 16328K 11984K getblk 0:04 19.92% 12.60% postgres 2039 pgsql -8 0 16344K 12024K biord 0:11 18.51% 17.14% postgres $ time psql -d perftst -f ptst.sql CREATE TABLE SELECT INSERT 0 500000 DELETE 500000 DELETE 38462 SELECT UPDATE 92308 DROP TABLE SELECT DELETE 37037 VACUUM VACUUM DROP TABLE DROP TABLE DROP TABLE VACUUM 304.00 real 0.00 user 0.00 sys 304.00/60 = 5.066666667 2077 pgsql -8 0 28168K 15640K biord 0:38 12.01% 12.01% postgres 2077 pgsql -4 0 28168K 15812K semwai 0:40 10.16% 10.16% postgres $ time psql -d perftst -f ptst.sql CREATE TABLE SELECT INSERT 0 500000 DELETE 500000 DELETE 38462 SELECT UPDATE 92308 DROP TABLE SELECT DELETE 37037 VACUUM VACUUM DROP TABLE DROP TABLE DROP TABLE VACUUM 307.97 real 0.00 user 0.00 sys 307.97/60=5.132833333 2113 pgsql -4 0 19960K 15296K ufs 0:30 15.97% 15.97% postgres 2113 pgsql -16 0 28168K 19280K wdrain 0:37 12.70% 12.70% postgres $ time psql -d perftst -f ptst.sql CREATE TABLE SELECT INSERT 0 500000 DELETE 500000 DELETE 38462 SELECT UPDATE 92308 DROP TABLE SELECT DELETE 37037 VACUUM VACUUM DROP TABLE DROP TABLE DROP TABLE VACUUM 304.89 real 0.00 user 0.00 sys 304.89/60=5.0815 ----------------------------------------------------------------------------------- Fedora C3 EXT3 [root@localhost ~]# time dd bs=1024 if=/dev/zero of=tstfile count=1M 1048576+0 records in 1048576+0 records out real 0m24.270s user 0m0.736s sys 0m5.528s [root@localhost ~]# time dd bs=1024 if=/dev/zero of=tstfile count=2M 2097152+0 records in 2097152+0 records out real 0m53.607s user 0m1.432s sys 0m11.306s [root@localhost ~]# time dd bs=1024 if=/dev/zero of=tstfile count=3M 3145728+0 records in 3145728+0 records out real 1m16.598s user 0m2.133s sys 0m16.794s Copy Test: [root@localhost ~]# time cp tstfile tstfile2 real 2m32.022s user 0m0.528s sys 0m12.417s Bonnie Tests: bonnie++ -u root -s 1024 -r 512 -n 5 -d /tmp Using uid:0, gid:0. Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP localhost.locald 1G 24983 72 49013 14 21798 5 21618 58 52864 6 137.5 0 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 5 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ localhost.localdomain,1G,24983,72,49013,14,21798,5,21618,58,52864,6,137.5,0,5,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++ bonnie++ -u root -s 2048 -r 512 -n 5 -d /tmp Using uid:0, gid:0. Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP localhost.locald 2G 30392 88 35467 10 16794 4 21350 57 40749 4 92.0 0 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 5 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ localhost.localdomain,2G,30392,88,35467,10,16794,4,21350,57,40749,4,92.0,0,5,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++ [root@localhost ~]# bonnie++ -u root -s 3072 -r 512 -n 5 -d /tmp Using uid:0, gid:0. Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP localhost.locald 3G 30940 90 36972 10 18177 4 22040 60 44367 4 85.7 0 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 5 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ localhost.localdomain,3G,30940,90,36972,10,18177,4,22040,60,44367,4,85.7,0,5,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++ 5513 postgres 18 0 20448 11m 10m D 13.0 4.6 0:01.37 postmaster 5513 postgres 19 0 20448 11m 10m D 13.3 4.6 0:02.97 postmaster 5513 postgres 18 0 20448 11m 10m D 35.6 4.7 0:08.26 postmaster 5513 postgres 18 0 21900 13m 11m D 39.0 5.3 0:20.90 postmaster bash-3.00$ time psql -d perftst -f tst1.sql CREATE TABLE SELECT INSERT 0 500000 DELETE 500000 DELETE 38462 SELECT UPDATE 92308 DROP TABLE SELECT DELETE 37037 VACUUM VACUUM DROP TABLE DROP TABLE DROP TABLE VACUUM real 4m20.061s user 0m0.007s sys 0m0.003s bash-3.00$ time psql -d perftst -f tst1.sql CREATE TABLE SELECT INSERT 0 500000 DELETE 500000 DELETE 38462 SELECT UPDATE 92308 DROP TABLE SELECT DELETE 37037 VACUUM VACUUM DROP TABLE DROP TABLE DROP TABLE VACUUM real 2m59.877s user 0m0.004s sys 0m0.007s 26307 postgres 18 0 21684 14m 11m R 8.0 5.7 0:39.30 postmaster 26307 postgres 18 0 21684 14m 11m R 20.3 5.7 0:43.37 postmaster bash-3.00$ time psql -d perftst -f tst1.sql CREATE TABLE SELECT INSERT 0 500000 DELETE 500000 DELETE 38462 SELECT UPDATE 92308 DROP TABLE SELECT DELETE 37037 VACUUM VACUUM DROP TABLE DROP TABLE DROP TABLE VACUUM real 2m29.036s user 0m0.002s sys 0m0.012s 26317 postgres 18 0 20472 10m 9.8m D 45.3 4.3 0:10.53 postmaster 26317 postgres 18 0 22500 12m 9.9m D 28.3 5.1 0:24.92 postmaster 26317 postgres 18 0 25580 16m 11m R 28.3 6.6 0:42.54 postmaster bash-3.00$ time psql -d perftst -f tst1.sql CREATE TABLE SELECT INSERT 0 500000 DELETE 500000 DELETE 38462 SELECT UPDATE 92308 DROP TABLE SELECT DELETE 37037 VACUUM VACUUM DROP TABLE DROP TABLE DROP TABLE VACUUM real 2m30.825s user 0m0.001s sys 0m0.009s 26323 postgres 18 0 20448 10m 10m R 46.9 4.4 0:36.98 postmaster 26323 postgres 16 0 31100 21m 10m R 48.9 8.6 3:41.24 postmaster 26323 postgres 16 0 29088 19m 10m R 26.6 7.9 5:19.27 postmaster bash-3.00$ time psql -d perftst -f tst1.sql CREATE TABLE SELECT INSERT 0 3000000 DELETE 3000000 DELETE 230772 SELECT UPDATE 553848 DROP TABLE SELECT DELETE 222222 VACUUM VACUUM DROP TABLE DROP TABLE DROP TABLE VACUUM real 24m55.547s user 0m0.008s sys 0m0.005s bash-3.00$ time psql -d perftst -f tst.sql thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 22:26:39.057576 | test string | 894455 2005-01-20 22:44:37.486004 | test string | 894455 2005-01-20 22:45:12.67617 | test string | 894455 2005-01-20 22:47:07.218733 | test string | 894455 2005-01-20 22:48:31.5945 | test string | 894455 2005-01-20 22:50:42.391373 | test string | 894455 (6 rows) thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 22:26:39.057576 | test string | 75455 2005-01-20 22:44:37.486004 | test string | 75455 2005-01-20 22:45:12.67617 | test string | 75455 2005-01-20 22:47:07.218733 | test string | 75455 2005-01-20 22:48:31.5945 | test string | 75455 2005-01-20 22:50:42.391373 | test string | 75455 (6 rows) thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 22:26:39.057576 | test string | 444455 2005-01-20 22:44:37.486004 | test string | 444455 2005-01-20 22:45:12.67617 | test string | 444455 2005-01-20 22:47:07.218733 | test string | 444455 2005-01-20 22:48:31.5945 | test string | 444455 2005-01-20 22:50:42.391373 | test string | 444455 (6 rows) thedate | astring | anumber ---------+---------+--------- (0 rows) thedate | astring | anumber ---------+---------+--------- (0 rows) thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 22:26:39.057576 | test string | 7555 2005-01-20 22:44:37.486004 | test string | 7555 2005-01-20 22:45:12.67617 | test string | 7555 2005-01-20 22:47:07.218733 | test string | 7555 2005-01-20 22:48:31.5945 | test string | 7555 2005-01-20 22:50:42.391373 | test string | 7555 (6 rows) thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 22:26:39.057576 | test string | 5 2005-01-20 22:44:37.486004 | test string | 5 2005-01-20 22:45:12.67617 | test string | 5 2005-01-20 22:47:07.218733 | test string | 5 2005-01-20 22:48:31.5945 | test string | 5 2005-01-20 22:50:42.391373 | test string | 5 (6 rows) thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 22:26:39.057576 | test string | 4455 2005-01-20 22:44:37.486004 | test string | 4455 2005-01-20 22:45:12.67617 | test string | 4455 2005-01-20 22:47:07.218733 | test string | 4455 2005-01-20 22:48:31.5945 | test string | 4455 2005-01-20 22:50:42.391373 | test string | 4455 (6 rows) thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 22:26:39.057576 | test string | 111155 2005-01-20 22:44:37.486004 | test string | 111155 2005-01-20 22:45:12.67617 | test string | 111155 2005-01-20 22:47:07.218733 | test string | 111155 2005-01-20 22:48:31.5945 | test string | 111155 2005-01-20 22:50:42.391373 | test string | 111155 (6 rows) thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 22:26:39.057576 | test string | 33355 2005-01-20 22:44:37.486004 | test string | 33355 2005-01-20 22:45:12.67617 | test string | 33355 2005-01-20 22:47:07.218733 | test string | 33355 2005-01-20 22:48:31.5945 | test string | 33355 2005-01-20 22:50:42.391373 | test string | 33355 (6 rows) thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 22:26:39.057576 | test string | 998455 2005-01-20 22:44:37.486004 | test string | 998455 2005-01-20 22:45:12.67617 | test string | 998455 2005-01-20 22:47:07.218733 | test string | 998455 2005-01-20 22:48:31.5945 | test string | 998455 2005-01-20 22:50:42.391373 | test string | 998455 (6 rows) thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 22:26:39.057576 | test string | 8455 2005-01-20 22:44:37.486004 | test string | 8455 2005-01-20 22:45:12.67617 | test string | 8455 2005-01-20 22:47:07.218733 | test string | 8455 2005-01-20 22:48:31.5945 | test string | 8455 2005-01-20 22:50:42.391373 | test string | 8455 (6 rows) thedate | astring | anumber ---------+---------+--------- (0 rows) real 3m13.732s user 0m0.005s sys 0m0.004s ------------------------------------------------------------------------------------------------- SC400 FC3 With XFS: 4665 root 18 0 4380 468 404 D 21.6 0.2 0:02.43 dd [root@localhost tmp]# time dd bs=1024 if=/dev/zero of=tstfile count=1M 1048576+0 records in 1048576+0 records out real 0m22.676s user 0m0.663s sys 0m3.494s 4667 root 19 0 3672 468 404 R 8.7 0.2 0:00.26 dd [root@localhost tmp]# time dd bs=1024 if=/dev/zero of=tstfile count=2M 2097152+0 records in 2097152+0 records out real 0m46.277s user 0m1.214s sys 0m7.141s 4674 root 25 0 4532 468 404 R 13.6 0.2 0:02.34 dd 4674 root 18 0 4532 468 404 D 17.6 0.2 0:05.52 dd [root@localhost tmp]# time dd bs=1024 if=/dev/zero of=tstfile count=3M 3145728+0 records in 3145728+0 records out real 1m12.727s user 0m1.884s sys 0m10.573s copy tests: 4744 root 18 0 3940 620 540 D 9.3 0.2 0:00.73 cp 4744 root 18 0 3940 620 540 R 9.3 0.2 0:01.78 cp 4744 root 18 0 3940 620 540 R 4.0 0.2 0:06.62 cp [root@localhost tmp]# time cp tstfile tstfile2 real 2m22.101s user 0m0.500s sys 0m8.178s bonnie tests: [root@localhost tmp]# bonnie++ -u root -s 1024 -r 512 -n 5 -d /tmp Using uid:0, gid:0. Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP localhost.locald 1G 33376 97 48548 7 20451 4 25233 69 47488 5 154.7 0 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 5 3724 17 +++++ +++ 3828 17 2957 14 +++++ +++ 2290 10 localhost.localdomain,1G,33376,97,48548,7,20451,4,25233,69,47488,5,154.7,0,5,3724,17,+++++,+++,3828,17,2957,14,+++++,+++,2290,10 4747 root 23 0 3004 976 840 R 88.9 0.4 1:56.32 bonnie++ 4747 root 18 0 2996 968 840 R 7.0 0.4 2:09.53 bonnie++ 4751 root 17 0 2996 980 844 D 0.3 0.4 0:00.02 bonnie++ [root@localhost tmp]# bonnie++ -u root -s 2048 -r 512 -n 5 -d /tmp Using uid:0, gid:0. Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP localhost.locald 2G 32988 97 47779 6 21777 5 26410 74 49463 6 130.0 0 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 5 3145 15 +++++ +++ 3641 15 3299 17 +++++ +++ 2037 9 localhost.localdomain,2G,32988,97,47779,6,21777,5,26410,74,49463,6,130.0,0,5,3145,15,+++++,+++,3641,15,3299,17,+++++,+++,2037,9 4788 root 25 0 3120 968 840 R 97.9 0.4 0:19.11 bonnie++ 4788 root 25 0 3124 972 840 R 97.6 0.4 0:42.54 bonnie++ 4788 root 25 0 3116 968 840 D 6.7 0.4 1:37.18 bonnie++ [root@localhost tmp]# bonnie++ -u root -s 3072 -r 512 -n 5 -d /tmp Using uid:0, gid:0. Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP localhost.locald 3G 32331 97 47200 6 21836 5 26740 75 49292 6 125.9 0 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 5 3215 14 +++++ +++ 3360 14 3036 15 +++++ +++ 1856 8 localhost.localdomain,3G,32331,97,47200,6,21836,5,26740,75,49292,6,125.9,0,5,3215,14,+++++,+++,3360,14,3036,15,+++++,+++,1856,8 5024 postgres 18 0 20116 11m 10m R 68.6 4.6 0:05.75 postmaster 5024 postgres 18 0 21288 12m 10m D 45.3 5.1 0:13.88 postmaster 4950 postgres 15 0 19572 10m 9828 D 1.0 4.0 0:00.13 postmaster 024 postgres 18 0 21568 12m 10m R 60.9 5.1 0:21.99 postmaster 5024 postgres 19 0 20176 11m 10m R 51.9 4.7 0:34.02 postmaster -bash-3.00$ time psql -d perftst -f tst1.sql CREATE TABLE SELECT INSERT 0 500000 DELETE 500000 DELETE 38462 SELECT UPDATE 92308 DROP TABLE SELECT DELETE 37037 VACUUM VACUUM DROP TABLE DROP TABLE DROP TABLE VACUUM real 2m9.088s user 0m0.007s sys 0m0.005s 5026 postgres 18 0 21392 13m 11m D 45.9 5.3 0:23.92 postmaster 5026 postgres 18 0 21392 13m 11m D 44.3 5.3 0:28.45 postmaster 5026 postgres 18 0 37780 17m 11m R 10.7 7.1 0:34.22 postmaster 5026 postgres 18 0 21392 14m 11m D 13.0 5.7 0:37.30 postmaster -bash-3.00$ time psql -d perftst -f tst1.sql CREATE TABLE SELECT INSERT 0 500000 DELETE 500000 DELETE 38462 SELECT UPDATE 92308 DROP TABLE SELECT DELETE 37037 VACUUM VACUUM DROP TABLE DROP TABLE DROP TABLE VACUUM real 3m22.001s user 0m0.007s sys 0m0.009s -bash-3.00$ time psql -d perftst -f tst1.sql CREATE TABLE SELECT INSERT 0 500000 DELETE 500000 DELETE 38462 SELECT UPDATE 92308 DROP TABLE SELECT DELETE 37037 VACUUM VACUUM DROP TABLE DROP TABLE DROP TABLE VACUUM real 2m17.877s user 0m0.003s sys 0m0.006s 26432 postgres 18 0 20140 10m 9764 R 46.6 4.2 0:03.92 postmaster bash-3.00$ time psql -d perftst -f tst1.sql CREATE TABLE SELECT INSERT 0 500000 DELETE 500000 DELETE 38462 SELECT UPDATE 92308 DROP TABLE SELECT DELETE 37037 VACUUM VACUUM DROP TABLE DROP TABLE DROP TABLE VACUUM real 2m17.666s user 0m0.003s sys 0m0.006s 26440 postgres 18 0 22160 13m 10m D 50.3 5.2 0:23.60 postmaster 26440 postgres 18 0 37820 17m 11m R 9.3 6.9 0:34.46 postmaster -bash-3.00$ time psql -d perftst -f tst1.sql CREATE TABLE SELECT INSERT 0 500000 DELETE 500000 DELETE 38462 SELECT UPDATE 92308 DROP TABLE SELECT DELETE 37037 VACUUM VACUUM DROP TABLE DROP TABLE DROP TABLE VACUUM real 2m19.578s user 0m0.005s sys 0m0.010s (6 Million Rows) 26445 postgres 19 0 20116 10m 10m R 58.9 4.3 0:03.32 postmaster 26445 postgres 19 0 20116 10m 10m R 51.6 4.4 0:18.37 postmaster 26445 postgres 18 0 31764 22m 10m D 9.3 8.9 1:33.57 postmaster 26445 postgres 18 0 45076 36m 11m R 8.3 14.7 5:21.49 postmaster -bash-3.00$ time psql -d perftst -f tst1.sql CREATE TABLE SELECT INSERT 0 3000000 DELETE 3000000 DELETE 230772 SELECT UPDATE 553848 DROP TABLE SELECT DELETE 222222 VACUUM VACUUM DROP TABLE DROP TABLE DROP TABLE VACUUM real 21m15.074s user 0m0.003s sys 0m0.007s (tst.sql) select * from test1 where anumber=894455; select * from test1 where anumber=75455; select * from test1 where anumber=444455; select * from test1 where anumber=7594455; select * from test1 where anumber=7511455; select * from test1 where anumber=7555; select * from test1 where anumber=5; select * from test1 where anumber=4455; select * from test1 where anumber=111155; select * from test1 where anumber=33355; select * from test1 where anumber=998455; select * from test1 where anumber=8455; select * from test1 where anumber=22754455; ------------------------------------------------------------------------ -bash-3.00$ time psql -d perftst -f tst.sql thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 18:45:25.058853 | test string | 894455 2005-01-20 19:10:26.609873 | test string | 894455 2005-01-20 19:12:06.125588 | test string | 894455 2005-01-20 19:12:47.447711 | test string | 894455 2005-01-20 19:15:21.224914 | test string | 894455 2005-01-20 19:16:12.809133 | test string | 894455 (6 rows) thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 18:45:25.058853 | test string | 75455 2005-01-20 19:10:26.609873 | test string | 75455 2005-01-20 19:12:06.125588 | test string | 75455 2005-01-20 19:12:47.447711 | test string | 75455 2005-01-20 19:15:21.224914 | test string | 75455 2005-01-20 19:16:12.809133 | test string | 75455 (6 rows) thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 18:45:25.058853 | test string | 444455 2005-01-20 19:10:26.609873 | test string | 444455 2005-01-20 19:12:06.125588 | test string | 444455 2005-01-20 19:12:47.447711 | test string | 444455 2005-01-20 19:15:21.224914 | test string | 444455 2005-01-20 19:16:12.809133 | test string | 444455 (6 rows) thedate | astring | anumber ---------+---------+--------- (0 rows) thedate | astring | anumber ---------+---------+--------- (0 rows) thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 18:45:25.058853 | test string | 7555 2005-01-20 19:10:26.609873 | test string | 7555 2005-01-20 19:12:06.125588 | test string | 7555 2005-01-20 19:12:47.447711 | test string | 7555 2005-01-20 19:15:21.224914 | test string | 7555 2005-01-20 19:16:12.809133 | test string | 7555 (6 rows) thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 18:45:25.058853 | test string | 5 2005-01-20 19:10:26.609873 | test string | 5 2005-01-20 19:12:06.125588 | test string | 5 2005-01-20 19:12:47.447711 | test string | 5 2005-01-20 19:15:21.224914 | test string | 5 2005-01-20 19:16:12.809133 | test string | 5 (6 rows) thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 18:45:25.058853 | test string | 4455 2005-01-20 19:10:26.609873 | test string | 4455 2005-01-20 19:12:06.125588 | test string | 4455 2005-01-20 19:12:47.447711 | test string | 4455 2005-01-20 19:15:21.224914 | test string | 4455 2005-01-20 19:16:12.809133 | test string | 4455 (6 rows) thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 18:45:25.058853 | test string | 111155 2005-01-20 19:10:26.609873 | test string | 111155 2005-01-20 19:12:06.125588 | test string | 111155 2005-01-20 19:12:47.447711 | test string | 111155 2005-01-20 19:15:21.224914 | test string | 111155 2005-01-20 19:16:12.809133 | test string | 111155 (6 rows) thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 18:45:25.058853 | test string | 33355 2005-01-20 19:10:26.609873 | test string | 33355 2005-01-20 19:12:06.125588 | test string | 33355 2005-01-20 19:12:47.447711 | test string | 33355 2005-01-20 19:15:21.224914 | test string | 33355 2005-01-20 19:16:12.809133 | test string | 33355 (6 rows) thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 18:45:25.058853 | test string | 998455 2005-01-20 19:10:26.609873 | test string | 998455 2005-01-20 19:12:06.125588 | test string | 998455 2005-01-20 19:12:47.447711 | test string | 998455 2005-01-20 19:15:21.224914 | test string | 998455 2005-01-20 19:16:12.809133 | test string | 998455 (6 rows) thedate | astring | anumber ----------------------------+-------------+--------- 2005-01-20 18:45:25.058853 | test string | 8455 2005-01-20 19:10:26.609873 | test string | 8455 2005-01-20 19:12:06.125588 | test string | 8455 2005-01-20 19:12:47.447711 | test string | 8455 2005-01-20 19:15:21.224914 | test string | 8455 2005-01-20 19:16:12.809133 | test string | 8455 (6 rows) thedate | astring | anumber ---------+---------+--------- (0 rows) real 2m31.883s user 0m0.002s sys 0m0.005s --------------------------------------------------------------------------------------------------------- Various Notes: --------------------------------------------------------------------------------------------------------- 1/18/05 PE 2400 bonnie++ Test with ext3: Top Output: 3911 root 25 0 3072 964 840 R 98.4 0.2 1:21.21 bonnie++ Bonnie++ Test1: [root@es1 npavlica]# bonnie++ -u root -s 1024 -r 512 -n 5 -d /tmp Using uid:0, gid:0. Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP es1.estation.com 1G 7469 91 9881 15 4846 6 9073 90 21034 15 362.4 2 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 5 7962 90 +++++ +++ +++++ +++ 8874 99 +++++ +++ 8803 69 es1.estation.com,1G,7469,91,9881,15,4846,6,9073,90,21034,15,362.4,2,5,7962,90,+++++,+++,+++++,+++,8874,99,+++++,+++,8803,69 Bonnie++ Test 2: [root@es1 npavlica]# bonnie++ -u root -s 2048 -r 512 -n 5 -d /tmp Using uid:0, gid:0. Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP es1.estation.com 2G 7208 88 8042 12 4652 6 8916 96 19611 14 252.9 2 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 5 8075 92 +++++ +++ +++++ +++ 6731 75 +++++ +++ +++++ +++ es1.estation.com,2G,7208,88,8042,12,4652,6,8916,96,19611,14,252.9,2,5,8075,92,+++++,+++,+++++,+++,6731,75,+++++,+++,+++++,+++ Bonnie++ Test 3: [root@es1 npavlica]# bonnie++ -u root -s 3072 -r 512 -n 5 -d /tmp Using uid:0, gid:0. Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP es1.estation.com 3G 6995 85 7921 12 4723 6 9728 96 20919 15 218.5 1 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 5 7840 90 +++++ +++ +++++ +++ 8852 100 +++++ +++ 9220 72 es1.estation.com,3G,6995,85,7921,12,4723,6,9728,96,20919,15,218.5,1,5,7840,90,+++++,+++,+++++,+++,8852,100,+++++,+++,9220,72 DD Test 1: [root@es1 npavlica]# time dd bs=1024 if=/dev/zero of=tstfile count=1M 1048576+0 records in 1048576+0 records out real 2m14.152s user 0m1.649s sys 0m29.622s DD Test 2: [root@es1 npavlica]# time dd bs=1024 if=/dev/zero of=tstfile count=1M 1048576+0 records in 1048576+0 records out real 2m14.152s user 0m1.649s sys 0m29.622s [root@es1 npavlica]# rm tstfile rm: remove regular file `tstfile'? y [root@es1 npavlica]# time dd bs=1024 if=/dev/zero of=tstfile count=2M 2097152+0 records in 2097152+0 records out real 4m7.145s user 0m3.152s sys 1m0.303s DD Test 3: [root@es1 npavlica]# time dd bs=1024 if=/dev/zero of=tstfile count=3M 3145728+0 records in 3145728+0 records out real 6m40.592s user 0m4.751s sys 1m31.968s FreeBSD5.3 I/O Tests: Partition Layout / Utilization: $ df -h Files ystem Size Used Avail Capacity Mounted on /dev/aacd0s1a 248M 35M 193M 15% / devfs 1.0K 1.0K 0B 100% /dev /dev/aacd0s1e 248M 6.0K 228M 0% /tmp /dev/aacd0s1f 31G 1.8G 27G 6% /usr /dev/aacd0s1d 248M 2.6M 225M 1% /var Bonnie++ Test 1: 715 nick.pavlica 97 0 2380K 1324K RUN 0:53 21.12% 20.85% bonnie++ $ bonnie++ -s 1024 -r 512 -n 5 Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done.535 nick.pavlica -4 0 2380K 1216K getblk 0:17 2.84% 2.83% bonnie++ Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.93c ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP vader.e--station 1G 60 99 6351 7 4169 5 170 99 20904 13 200.7 19 Latency 190ms 6458ms 10221ms 57902us 33802us 423ms Version 1.93c ------Sequential Create------ --------Random Create-------- vader.e--station.co -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 5 5647 59 +++++ +++ 9412 74 5757 56 +++++ +++ 5291 45 Latency 9741us 917us 315us 225ms 875us 11165us 1.93c,1.93c,vader.e—station.com,1,1106164339,1G,,60,99,6351,7,4169,5,170,99,20904,13,200.7,19,5,,,,,5647,59,+++++,+++,9412,74,5757,56,+++++,+++,5291,45,190ms,6458ms,10221ms,57902us,33802us,423ms,9741us,917us,315us,225ms,875us,11165us Bonnie++ Test 2: 745 nick.pavlica -16 0 2380K 1200K wdrain 0:58 7.03% 7.03% bonnie++ 745 nick.pavlica -4 0 2380K 1200K getblk 1:21 5.57% 5.57% bonnie++ $ bonnie++ -s 2048 -r 512 -n 5 Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.93c ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP vader.e--station 2G 60 99 5483 6 4448 5 170 99 19045 12 129.8 13 Latency 152ms 9726ms 7113ms 55161us 42746us 3598ms Version 1.93c ------Sequential Create------ --------Random Create-------- vader.e--station.co -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 5 7036 72 +++++ +++ 9521 75 4877 50 +++++ +++ 9029 76 Latency 5716us 955us 3648us 10997us 400us 320us 1.93c,1.93c,vader.e—station.com,1,1106164107,2G,,60,99,5483,6,4448,5,170,99,19045,12,129.8,13,5,,,,,7036,72,+++++,+++,9521,75,4877,50,+++++,+++,9029,76,152ms,9726ms,7113ms,55161us,42746us,3598ms,5716us,955us,3648us,10997us,400us,320us Bonnie++ Test 3: 789 nick.pavlica -4 0 2380K 1208K getblk 1:52 4.64% 4.64% bonnie++ 789 nick.pavlica -4 0 2380K 1208K getblk 1:52 2.49% 2.49% bonnie++ $ bonnie++ -s 3072 -r 512 -n 5 Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.93c ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP vader.e--station 3G 60 99 6184 7 3907 4 167 98 20993 14 99.3 11 Latency 177ms 9312ms 15369ms 72183us 70690us 608ms Version 1.93c ------Sequential Create------ --------Random Create-------- vader.e--station.co -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 5 7032 72 +++++ +++ 9667 77 7445 74 +++++ +++ 4197 36 Latency 5855us 1682us 359us 8565us 11109us 11436us 1.93c,1.93c,vader.e--station.com,1,1106166552,3G,,60,99,6184,7,3907,4,167,98,20993,14,99.3,11,5,,,,,7032,72,+++++,+++,9667,77,7445,74,+++++,+++,4197,36,177ms,9312ms,15369ms,72183us,70690us,608ms,5855us,1682us,359us,8565us,11109us,11436us DD Test1: $ time dd bs=1024 if=/dev/zero of=tstfile count=1M 1048576+0 records in 1048576+0 records out 1073741824 bytes transferred in 168.394218 secs (6376358 bytes/sec) 168.41 real 1.34 user 33.60 sys 168.41 / 60 = 2.806833333 DD Test 2: Top was: - 575 nick.pavlica -16 0 1212K 588K wdrain 0:13 17.30% 17.04% dd - 575 nick.pavlica -16 0 1212K 588K wdrain 0:32 20.85% 20.85% dd $ time dd bs=1024 if=/dev/zero of=tstfile count=2M 2097152+0 records in 2097152+0 records out 2147483648 bytes transferred in 359.389723 secs (5975362 bytes/sec) 359.59 real 2.57 user 68.14 sys 359.59 / 60 = 5.993166667 DD Test 3: 661 nick.pavlica -16 0 1212K 588K wdrain 0:06 20.86% 15.19% dd $ time dd bs=1024 if=/dev/zero of=tstfile count=3M 3145728+0 records in 3145728+0 records out 3221225472 bytes transferred in 520.399250 secs (6189912 bytes/sec) 520.60 real 4.06 user 102.48 sys 520.6 / 60 = 8.676666667 Fedora C3 With XFS: (minimal install) [root@localhost ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/sda3 34G 737M 33G 3% / /dev/sda1 92M 6.0M 86M 7% /boot none 252M 0 252M 0% /dev/shm Bonnie++ Test 1: [root@localhost tmp]# bonnie++ -u root -s 1024 -r 512 -n 5 -d /tmp Using uid:0, gid:0. Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP localhost.locald 1G 8206 96 11876 13 5982 10 9375 95 24909 21 538.8 4 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 5 652 13 +++++ +++ 600 14 622 14 +++++ +++ 365 7 localhost.localdomain,1G,8206,96,11876,13,5982,10,9375,95,24909,21,538.8,4,5,652,13,+++++,+++,600,14,622,14,+++++,+++,365,7 Bonnie++ Test 2: [root@localhost tmp]# bonnie++ -u root -s 2048 -r 512 -n 5 -d /tmp Using uid:0, gid:0. Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP localhost.locald 2G 7981 94 8794 9 6026 10 9535 97 25025 21 392.1 3 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 5 635 13 +++++ +++ 587 14 574 13 +++++ +++ 388 8 localhost.localdomain,2G,7981,94,8794,9,6026,10,9535,97,25025,21,392.1,3,5,635,13,+++++,+++,587,14,574,13,+++++,+++,388,8 Bonnie++ Test 3: 2745 root 25 0 4176 976 2300 R 99.9 0.2 3:34.49 bonnie++ [root@localhost tmp]# bonnie++ -u root -s 3072 -r 512 -n 5 -d /tmp Using uid:0, gid:0. Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03 ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP localhost.locald 3G 7909 93 8053 8 3498 5 9328 95 24895 21 351.9 3 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 5 559 12 +++++ +++ 528 13 594 13 +++++ +++ 414 9 localhost.localdomain,3G,7909,93,8053,8,3498,5,9328,95,24895,21,351.9,3,5,559,12,+++++,+++,528,13,594,13,+++++,+++,414,9 DD Test 1: 2629 root 18 0 4632 472 3344 D 24.5 0.1 0:20.46 dd [root@localhost ~]# time dd bs=1024 if=/dev/zero of=tstfile count=1M 1048576+0 records in 1048576+0 records out real 1m39.842s user 0m4.404s sys 0m22.647s DD Test 2: 2669 root 18 0 4596 472 3344 D 20.5 0.1 0:07.93 dd [root@localhost ~]# time dd bs=1024 if=/dev/zero of=tstfile count=2M 2097152+0 records in 2097152+0 records out real 4m11.713s user 0m9.131s sys 0m46.951s DD Test 3: 2707 root 18 0 3832 472 3344 R 23.2 0.1 0:30.84 dd [root@localhost ~]# time dd bs=1024 if=/dev/zero of=tstfile count=3M 3145728+0 records in 3145728+0 records out real 6m52.182s user 0m13.450s sys 1m10.167s ------------------------------------------------------------------------------------------------------------ SC 400 Tests fc3 with xfs: [root@localhost ~]# time dd bs=1024 if=/dev/zero of=tstfile count=1M 1048576+0 records in 1048576+0 records out real 0m27.491s user 0m1.436s sys 0m6.888s [root@localhost ~]# time dd bs=1024 if=/dev/zero of=tstfile count=2M 2097152+0 records in 2097152+0 records out real 1m3.194s user 0m2.847s sys 0m13.766s [root@localhost ~]# time dd bs=1024 if=/dev/zero of=tstfile count=3M 3145728+0 records in 3145728+0 records out real 1m59.668s user 0m4.304s sys 0m20.673s [root@localhost ~]# time dd bs=1024 if=/dev/zero of=tstfile count=7M 7340032+0 records in 7340032+0 records out real 4m6.371s user 0m9.877s sys 0m49.454s [root@localhost ~]# time dd bs=1024 if=/dev/zero of=tstfile count=3M 3145728+0 records in 3145728+0 records out real 1m35.411s user 0m4.495s sys 0m21.006s [root@localhost ~]# ls anaconda-ks.cfg install.log install.log.syslog tstfile copy test: (sc400) [root@localhost ~]# time cp tstfile tstfile2 real 3m22.202s user 0m1.320s sys 0m12.605s (pe2400) [root@localhost ~]# time cp tstfile tstfile2 real 11m6.046s user 0m1.091s sys 0m39.096s __________________________________ Do you Yahoo!? Yahoo! Mail - now with 250MB free storage. Learn more. http://info.mail.yahoo.com/mail_250 From owner-freebsd-performance@FreeBSD.ORG Sun Jan 23 03:05:45 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 443EE16A4CF for ; Sun, 23 Jan 2005 03:05:45 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.202]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1619E43D5C for ; Sun, 23 Jan 2005 03:05:44 +0000 (GMT) (envelope-from linicks@gmail.com) Received: by rproxy.gmail.com with SMTP id 40so41501rnz for ; Sat, 22 Jan 2005 19:05:43 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=n5mF3d3LJ/jRgGKtuet5K89i97QRfpm2XAXebDH6CHSvy++wyWGLQWDfiZNFDY7lciNz7yal1yLwwQDsRaM0e3Ft4W2e8sENu3uC5joZhC5QU+3/ctk97xDJenJy8kAkBhFUFk0p90ZEWIxKWGgReM/00kqpaDZc9tLXEaCFlbg= Received: by 10.38.8.13 with SMTP id 13mr57001rnh; Sat, 22 Jan 2005 19:05:43 -0800 (PST) Received: by 10.38.8.43 with HTTP; Sat, 22 Jan 2005 19:05:43 -0800 (PST) Message-ID: Date: Sat, 22 Jan 2005 20:05:43 -0700 From: Nick Pavlica To: stheg olloydson In-Reply-To: <20050123024226.15690.qmail@web53908.mail.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable References: <20050123024226.15690.qmail@web53908.mail.yahoo.com> X-Mailman-Approved-At: Sun, 23 Jan 2005 09:41:33 +0000 cc: freebsd-performance@freebsd.org cc: freebsd-questions@freebsd.org Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | More Info X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Nick Pavlica List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2005 03:05:45 -0000 Thanks for the help. I appreciate it! Having a good community around a project is sometimes as important as the technology itself. I'm excited to find out if others attain the same results. Thanks! --Nick On Sat, 22 Jan 2005 18:42:26 -0800 (PST), stheg olloydson wrote: > it was said: >=20 > >All, > > This post is not about BSD VS. Linux and should not be taken that > >way. I think that "Flame Wars/Engineer Wars" are waste of time and > >energy. I was surprised by my test results and didn't want to take > >FBSD out of the loop just yet. There may be flaws in my testing that > >have led me to inaccurate results. I didn't share the testing details > >in the original mail because of time constraints, and the notes are > >fairly lengthy. I will add my notes to this mail so that there is a > >better understanding of what tests I performed, and their results. > >It's important to note that I did not tweak any of the default > >settings of the OS or DB. The notes should be generally self > >explanatory, but will be more that happy to clarify any questions that > >you have. As a side note, I chose the email address linicks because > >by name is Nick, and thought it was a fun play on words. I appreciate > >all of your feedback, so that I can better understand the differences > >in these great operating systems and communities. > > > >Thanks Again! > >--Nick Pavlica >=20 > Hello, >=20 > I'm glad you weren't trolling. I, too, think the OS wars are a load of > cark. Each OS has it strengths and weaknesses. Time is better spent > increasing the strengths and fixing the weaknesses than arguing about > whose are _better_. > I can say right off that FBSD's out-of-the-box state is intended for > stability rather than performance. The real question is what they do > after tuning. Here I would expect FBSD to do somewhat better, > especially on the uni-processor machine. Running the tests on the SC400 > hardware won't be a problem for me, but I have no spare SMP or SCSI > equipment to do the PE2400 tests - which I think would be the more > interesting. Perhaps someone else on the list can do this? Not to > provide a head-to-head showdown but to see if something is actually > wrong that isn't already being looked at. (Everyone knows threading has > problems that are being dealt with. That's why I'm not so sure FBSD > will out-perform Fedora at this time on an SMP box.) > To be on the safe side, I'll cc this to the performance list, as well. > Maybe someone has already done something similar and has quick answers. > Thus, I'm including unquoted the rest of your email below. > Finally, the addy thing was just me getting in a shot at you if you had > turned out to be trolling. >=20 > Regards, >=20 > stheg >=20 > The tests and results below here. I don't know how the formatting is > going to turn out. If its too mangled, see the original post on > questions@ >=20 > "OK, The testing notes already" :) > -------------------------------------------------------------------------= -------------------------------------- > Hardware Configs: > Dell PE 2400 > - Dual PIII 500Mhz > - 512Mb Ram > - Perc 2si controller > - (2) 10k ultra160 drives in a raid 1 configuration. >=20 > Dell SC400 > - P4 2.4 Ghz (not hyperthreaded) > - 512Mb Ram > - Stock 40Gb IDE 7200RPM > -------------------------------------------------------------------------= --------------- > Postgresql Test Scripts: > CREATE TABLE test1 ( > thedate TIMESTAMP, > astring VARCHAR(200), > anumber INTEGER > ); >=20 > CREATE FUNCTION build_data() RETURNS integer AS ' > DECLARE > i INTEGER DEFAULT 0; > curtime TIMESTAMP; > BEGIN > FOR i IN 1..1000000 LOOP > curtime :=3D ''now''; > INSERT INTO test1 VALUES (curtime, ''test string'', i); > END LOOP; > RETURN 1; > END; > ' LANGUAGE 'plpgsql'; >=20 > SELECT build_data(); >=20 > Then the following script is run under the time program to ascertain > how long it takes to run: >=20 > CREATE TABLE test2 ( > thedate TIMESTAMP, > astring VARCHAR(200), > anumber INTEGER > ); > CREATE TABLE test3 AS SELECT * FROM test1; > INSERT INTO test2 SELECT * FROM test1 WHERE ((anumber % 2) =3D 0); > DELETE FROM test3 WHERE ((anumber % 2) =3D 0); > DELETE FROM test3 WHERE ((anumber % 13) =3D 0); > CREATE TABLE test4 AS > SELECT test1.thedate AS t1date, > test2.thedate AS t2date, > test1.astring AS t1string, > test2.astring AS t2string, > test1.anumber AS t1number, > test2.anumber AS t2number > FROM test1 JOIN test2 ON test1.anumber=3Dtest2.anumber; > UPDATE test3 SET thedate=3D'now' WHERE ((anumber % 5) =3D 0); > DROP TABLE test4; > CREATE TABLE test4 AS SELECT * FROM test1; > DELETE FROM test4 WHERE ((anumber % 27) =3D 0); > VACUUM ANALYZE; > VACUUM FULL; > DROP TABLE test4; > DROP TABLE test3; > DROP TABLE test2; > VACUUM FULL; >=20 > -------------------------------------------------------------------------= ---------------- > sc400 freeBSD5: > $ time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count=3D1M > 1048576+0 records in > 1048576+0 records out > 1073741824 bytes transferred in 71.807645 secs (14953029 bytes/sec) > 71.82real 0.68 user 8.83 sys > 71.82 / 60 =3D 1.197 >=20 > -- > 517 nick.pavlica -16 0 1212K 588K wdrain 0:02 12.35% 5.91% dd > 517 nick.pavlica -16 0 1212K 588K wdrain 0:13 12.48% 12.35% dd >=20 > $ time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count=3D2M > 2097152+0 records in > 2097152+0 records out > 2147483648 bytes transferred in 136.815925 secs (15696153 bytes/sec) > 136.85 real 1.29 user 17.49 sys >=20 > 136.85 / 60 =3D 2.280833333 > -- > 542 nick.pavlica -16 0 1212K 588K wdrain 0:19 13.35% 13.33% dd > 542 nick.pavlica -16 0 1212K 588K wdrain 0:24 12.99% 12.99% dd >=20 > $ time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count=3D3M > 3145728+0 records in > 3145728+0 records out > 3221225472 bytes transferred in 205.722425 secs (15658115 bytes/sec) > 205.72 real 1.82 user 27.39 sys >=20 > 205.72 / 60 =3D 3.428666667 >=20 > copy test: >=20 > 558 nick.pavlica -4 0 1272K 680K getblk 0:01 2.30% 1.32% cp > 558 nick.pavlica -4 0 1272K 680K getblk 0:02 1.80% 1.71% cp > 558 nick.pavlica -4 0 1272K 680K getblk 0:03 1.87% 1.86% cp >=20 > $ time cp tstfile tstfile2 > 579.31 real 0.03 user 14.61 sys > 579.31 / 60 =3D 9.655166667 >=20 > (FreeBSD 5.3+ on SC400) >=20 > b test 1: > 535 nick.pavlica -4 0 2380K 1216K getblk 0:17 2.84% 2.83% > bonnie++ > 568 nick.pavlica 105 0 2380K 1196K RUN 0:09 92.99% 36.62% > bonnie++ > 568 nick.pavlica -16 0 2380K 1192K wdrain 0:14 12.35% 11.23% > bonnie++ >=20 > $ bonnie++ -s 1024 -r 512 -n 5 > Writing a byte at a time...done > Writing intelligently...done > Rewriting...done > Reading a byte at a time...done > Reading intelligently...done > start 'em...done...done...done...done...done... > Create files in sequential order...done. > Stat files in sequential order...done. > Delete files in sequential order...done. > Create files in random order...done. > Stat files in random order...done. > Delete files in random order...done. > Version 1.93c ------Sequential Output------ --Sequential Input- > --Random- > Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP > /sec %CP > sc400.e--station 1G 257 99 15135 5 10033 3 508 99 45572 10 > 124.3 5 > Latency 33802us 161ms 467ms 32560us 17190us > 537ms > Version 1.93c ------Sequential Create------ --------Random > Create-------- > sc400.e--station.co -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > 5 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ > +++++ +++ > Latency 14891us 149us 226us 2671us 133us > 1270us > 1.93c,1.93c,sc400.e--station.com,1,1106248340,1G,,257,99,15135,5,10033,3,= 508,99, > 45572,10,124.3,5,5,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,= +++++,+ > ++,33802us,161ms,467ms,32560us,17190us,537ms,14891us,149us,226us,2671us,1= 33us,12 > 70us >=20 > B Test 2: >=20 > 712 nick.pavlica -16 0 2380K 1212K wdrain 0:12 53.78% 33.01% > bonnie++ > 712 nick.pavlica -16 0 2380K 1208K wdrain 0:17 4.90% 4.88% > bonnie++ >=20 > $ bonnie++ -s 2048 -r 512 -n 5 > Writing a byte at a time...done > Writing intelligently...done > Rewriting...done > Reading a byte at a time...done > Reading intelligently...done > start 'em...done...done...done...done...done... > Create files in sequential order...done. > Stat files in sequential order...done. > Delete files in sequential order...done. > Create files in random order...done. > Stat files in random order...done. > Delete files in random order...done. > Version 1.93c ------Sequential Output------ --Sequential Input- > --Random- > Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP > /sec %CP > sc400.e--station 2G 260 99 14394 5 9512 3 505 99 42976 9 > 94.7 4 > Latency 32809us 284ms 470ms 30153us 15074us > 3564ms > Version 1.93c ------Sequential Create------ --------Random > Create-------- > sc400.e--station.co -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > 5 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ > +++++ +++ > Latency 6772us 1452us 175us 3964us 114us > 46821us > 1.93c,1.93c,sc400.e=E2=80=94station.com,1,1106252670,2G,,260,99,14394,5,9= 512,3,505,99,42976,9,94.7,4,5,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+= ++++,+++,+++++,+++,32809us,284ms,470ms,30153us,15074us,3564ms,6772us,1452us= ,175us,3964us,114us,46821us >=20 > Test 3: > 754 nick.pavlica -16 0 2380K 1208K wdrain 0:13 26.89% 20.90% > bonnie++ >=20 > $ bonnie++ -s 3072 -r 512 -n 5 > Writing a byte at a time...done > Writing intelligently...done > Rewriting...done > Reading a byte at a time...done > Reading intelligently...done > start 'em...done...done...done...done...done... > Create files in sequential order...done. > Stat files in sequential order...done. > Delete files in sequential order...done. > Create files in random order...done. > Stat files in random order...done. > Delete files in random order...done. > Version 1.93c ------Sequential Output------ --Sequential Input- > --Random- > Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP > /sec %CP > sc400.e--station 3G 259 99 14484 5 9671 3 507 99 40714 9 > 77.7 3 > Latency 33254us 569ms 591ms 36025us 22245us > 936ms > Version 1.93c ------Sequential Create------ --------Random > Create-------- > sc400.e--station.co -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > 5 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ > +++++ +++ > Latency 13993us 154us 120us 20243us 555us > 72us > 1.93c,1.93c,sc400.e=E2=80=94station.com,1,1106253537,3G,,259,99,14484,5,9= 671,3,507,99,40714,9,77.7,3,5,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+= ++++,+++,+++++,+++,33254us,569ms,591ms,36025us,22245us,936ms,13993us,154us,= 120us,20243us,555us,72us >=20 > (PG Performance Test) > 2039 pgsql -4 0 16328K 11984K getblk 0:04 19.92% 12.60% > postgres > 2039 pgsql -8 0 16344K 12024K biord 0:11 18.51% 17.14% > postgres >=20 > $ time psql -d perftst -f ptst.sql > CREATE TABLE > SELECT > INSERT 0 500000 > DELETE 500000 > DELETE 38462 > SELECT > UPDATE 92308 > DROP TABLE > SELECT > DELETE 37037 > VACUUM > VACUUM > DROP TABLE > DROP TABLE > DROP TABLE > VACUUM > 304.00 real 0.00 user 0.00 sys >=20 > 304.00/60 =3D 5.066666667 >=20 > 2077 pgsql -8 0 28168K 15640K biord 0:38 12.01% 12.01% > postgres > 2077 pgsql -4 0 28168K 15812K semwai 0:40 10.16% 10.16% > postgres > $ time psql -d perftst -f ptst.sql > CREATE TABLE > SELECT > INSERT 0 500000 > DELETE 500000 > DELETE 38462 > SELECT > UPDATE 92308 > DROP TABLE > SELECT > DELETE 37037 > VACUUM > VACUUM > DROP TABLE > DROP TABLE > DROP TABLE > VACUUM > 307.97 real 0.00 user 0.00 sys > 307.97/60=3D5.132833333 >=20 > 2113 pgsql -4 0 19960K 15296K ufs 0:30 15.97% 15.97% > postgres > 2113 pgsql -16 0 28168K 19280K wdrain 0:37 12.70% 12.70% > postgres > $ time psql -d perftst -f ptst.sql > CREATE TABLE > SELECT > INSERT 0 500000 > DELETE 500000 > DELETE 38462 > SELECT > UPDATE 92308 > DROP TABLE > SELECT > DELETE 37037 > VACUUM > VACUUM > DROP TABLE > DROP TABLE > DROP TABLE > VACUUM > 304.89 real 0.00 user 0.00 sys >=20 > 304.89/60=3D5.0815 >=20 > -------------------------------------------------------------------------= ---------- > Fedora C3 EXT3 > [root@localhost ~]# time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count= =3D1M > 1048576+0 records in > 1048576+0 records out >=20 > real 0m24.270s > user 0m0.736s > sys 0m5.528s >=20 > [root@localhost ~]# time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count= =3D2M > 2097152+0 records in > 2097152+0 records out >=20 > real 0m53.607s > user 0m1.432s > sys 0m11.306s >=20 > [root@localhost ~]# time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count= =3D3M > 3145728+0 records in > 3145728+0 records out >=20 > real 1m16.598s > user 0m2.133s > sys 0m16.794s >=20 > Copy Test: > [root@localhost ~]# time cp tstfile tstfile2 >=20 > real 2m32.022s > user 0m0.528s > sys 0m12.417s >=20 > Bonnie Tests: >=20 > bonnie++ -u root -s 1024 -r 512 -n 5 -d /tmp > Using uid:0, gid:0. > Writing with putc()...done > Writing intelligently...done > Rewriting...done > Reading with getc()...done > Reading intelligently...done > start 'em...done...done...done... > Create files in sequential order...done. > Stat files in sequential order...done. > Delete files in sequential order...done. > Create files in random order...done. > Stat files in random order...done. > Delete files in random order...done. > Version 1.03 ------Sequential Output------ --Sequential Input- > --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP > /sec %CP > localhost.locald 1G 24983 72 49013 14 21798 5 21618 58 52864 6 > 137.5 0 > ------Sequential Create------ --------Random > Create-------- > -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > 5 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ > +++++ +++ > localhost.localdomain,1G,24983,72,49013,14,21798,5,21618,58,52864,6,137.5= ,0,5,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++ >=20 > bonnie++ -u root -s 2048 -r 512 -n 5 -d /tmp > Using uid:0, gid:0. > Writing with putc()...done > Writing intelligently...done > Rewriting...done > Reading with getc()...done > Reading intelligently...done > start 'em...done...done...done... > Create files in sequential order...done. > Stat files in sequential order...done. > Delete files in sequential order...done. > Create files in random order...done. > Stat files in random order...done. > Delete files in random order...done. > Version 1.03 ------Sequential Output------ --Sequential Input- > --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP > /sec %CP > localhost.locald 2G 30392 88 35467 10 16794 4 21350 57 40749 4 > 92.0 0 > ------Sequential Create------ --------Random > Create-------- > -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > 5 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ > +++++ +++ > localhost.localdomain,2G,30392,88,35467,10,16794,4,21350,57,40749,4,92.0,= 0,5,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++ >=20 > [root@localhost ~]# bonnie++ -u root -s 3072 -r 512 -n 5 -d /tmp > Using uid:0, gid:0. > Writing with putc()...done > Writing intelligently...done > Rewriting...done > Reading with getc()...done > Reading intelligently...done > start 'em...done...done...done... > Create files in sequential order...done. > Stat files in sequential order...done. > Delete files in sequential order...done. > Create files in random order...done. > Stat files in random order...done. > Delete files in random order...done. > Version 1.03 ------Sequential Output------ --Sequential Input- > --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP > /sec %CP > localhost.locald 3G 30940 90 36972 10 18177 4 22040 60 44367 4 > 85.7 0 > ------Sequential Create------ --------Random > Create-------- > -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > 5 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ > +++++ +++ > localhost.localdomain,3G,30940,90,36972,10,18177,4,22040,60,44367,4,85.7,= 0,5,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++ >=20 > 5513 postgres 18 0 20448 11m 10m D 13.0 4.6 0:01.37 postmaster > 5513 postgres 19 0 20448 11m 10m D 13.3 4.6 0:02.97 postmaster > 5513 postgres 18 0 20448 11m 10m D 35.6 4.7 0:08.26 postmaster > 5513 postgres 18 0 21900 13m 11m D 39.0 5.3 0:20.90 postmaster >=20 > bash-3.00$ time psql -d perftst -f tst1.sql > CREATE TABLE > SELECT > INSERT 0 500000 > DELETE 500000 > DELETE 38462 > SELECT > UPDATE 92308 > DROP TABLE > SELECT > DELETE 37037 > VACUUM > VACUUM > DROP TABLE > DROP TABLE > DROP TABLE > VACUUM >=20 > real 4m20.061s > user 0m0.007s > sys 0m0.003s >=20 > bash-3.00$ time psql -d perftst -f tst1.sql > CREATE TABLE > SELECT > INSERT 0 500000 > DELETE 500000 > DELETE 38462 > SELECT > UPDATE 92308 > DROP TABLE > SELECT > DELETE 37037 > VACUUM > VACUUM > DROP TABLE > DROP TABLE > DROP TABLE > VACUUM >=20 > real 2m59.877s > user 0m0.004s > sys 0m0.007s >=20 > 26307 postgres 18 0 21684 14m 11m R 8.0 5.7 0:39.30 postmaster > 26307 postgres 18 0 21684 14m 11m R 20.3 5.7 0:43.37 postmaster >=20 > bash-3.00$ time psql -d perftst -f tst1.sql > CREATE TABLE > SELECT > INSERT 0 500000 > DELETE 500000 > DELETE 38462 > SELECT > UPDATE 92308 > DROP TABLE > SELECT > DELETE 37037 > VACUUM > VACUUM > DROP TABLE > DROP TABLE > DROP TABLE > VACUUM >=20 > real 2m29.036s > user 0m0.002s > sys 0m0.012s >=20 > 26317 postgres 18 0 20472 10m 9.8m D 45.3 4.3 0:10.53 postmaster > 26317 postgres 18 0 22500 12m 9.9m D 28.3 5.1 0:24.92 postmaster > 26317 postgres 18 0 25580 16m 11m R 28.3 6.6 0:42.54 postmaster > bash-3.00$ time psql -d perftst -f tst1.sql > CREATE TABLE > SELECT > INSERT 0 500000 > DELETE 500000 > DELETE 38462 > SELECT > UPDATE 92308 > DROP TABLE > SELECT > DELETE 37037 > VACUUM > VACUUM > DROP TABLE > DROP TABLE > DROP TABLE > VACUUM >=20 > real 2m30.825s > user 0m0.001s > sys 0m0.009s >=20 > 26323 postgres 18 0 20448 10m 10m R 46.9 4.4 0:36.98 postmaster > 26323 postgres 16 0 31100 21m 10m R 48.9 8.6 3:41.24 postmaster > 26323 postgres 16 0 29088 19m 10m R 26.6 7.9 5:19.27 postmaster >=20 > bash-3.00$ time psql -d perftst -f tst1.sql > CREATE TABLE > SELECT > INSERT 0 3000000 > DELETE 3000000 > DELETE 230772 > SELECT > UPDATE 553848 > DROP TABLE > SELECT > DELETE 222222 > VACUUM > VACUUM > DROP TABLE > DROP TABLE > DROP TABLE > VACUUM >=20 > real 24m55.547s > user 0m0.008s > sys 0m0.005s >=20 > bash-3.00$ time psql -d perftst -f tst.sql > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 22:26:39.057576 | test string | 894455 > 2005-01-20 22:44:37.486004 | test string | 894455 > 2005-01-20 22:45:12.67617 | test string | 894455 > 2005-01-20 22:47:07.218733 | test string | 894455 > 2005-01-20 22:48:31.5945 | test string | 894455 > 2005-01-20 22:50:42.391373 | test string | 894455 > (6 rows) >=20 > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 22:26:39.057576 | test string | 75455 > 2005-01-20 22:44:37.486004 | test string | 75455 > 2005-01-20 22:45:12.67617 | test string | 75455 > 2005-01-20 22:47:07.218733 | test string | 75455 > 2005-01-20 22:48:31.5945 | test string | 75455 > 2005-01-20 22:50:42.391373 | test string | 75455 > (6 rows) >=20 > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 22:26:39.057576 | test string | 444455 > 2005-01-20 22:44:37.486004 | test string | 444455 > 2005-01-20 22:45:12.67617 | test string | 444455 > 2005-01-20 22:47:07.218733 | test string | 444455 > 2005-01-20 22:48:31.5945 | test string | 444455 > 2005-01-20 22:50:42.391373 | test string | 444455 > (6 rows) >=20 > thedate | astring | anumber > ---------+---------+--------- > (0 rows) >=20 > thedate | astring | anumber > ---------+---------+--------- > (0 rows) >=20 > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 22:26:39.057576 | test string | 7555 > 2005-01-20 22:44:37.486004 | test string | 7555 > 2005-01-20 22:45:12.67617 | test string | 7555 > 2005-01-20 22:47:07.218733 | test string | 7555 > 2005-01-20 22:48:31.5945 | test string | 7555 > 2005-01-20 22:50:42.391373 | test string | 7555 > (6 rows) >=20 > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 22:26:39.057576 | test string | 5 > 2005-01-20 22:44:37.486004 | test string | 5 > 2005-01-20 22:45:12.67617 | test string | 5 > 2005-01-20 22:47:07.218733 | test string | 5 > 2005-01-20 22:48:31.5945 | test string | 5 > 2005-01-20 22:50:42.391373 | test string | 5 > (6 rows) >=20 > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 22:26:39.057576 | test string | 4455 > 2005-01-20 22:44:37.486004 | test string | 4455 > 2005-01-20 22:45:12.67617 | test string | 4455 > 2005-01-20 22:47:07.218733 | test string | 4455 > 2005-01-20 22:48:31.5945 | test string | 4455 > 2005-01-20 22:50:42.391373 | test string | 4455 > (6 rows) >=20 > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 22:26:39.057576 | test string | 111155 > 2005-01-20 22:44:37.486004 | test string | 111155 > 2005-01-20 22:45:12.67617 | test string | 111155 > 2005-01-20 22:47:07.218733 | test string | 111155 > 2005-01-20 22:48:31.5945 | test string | 111155 > 2005-01-20 22:50:42.391373 | test string | 111155 > (6 rows) >=20 > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 22:26:39.057576 | test string | 33355 > 2005-01-20 22:44:37.486004 | test string | 33355 > 2005-01-20 22:45:12.67617 | test string | 33355 > 2005-01-20 22:47:07.218733 | test string | 33355 > 2005-01-20 22:48:31.5945 | test string | 33355 > 2005-01-20 22:50:42.391373 | test string | 33355 > (6 rows) >=20 > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 22:26:39.057576 | test string | 998455 > 2005-01-20 22:44:37.486004 | test string | 998455 > 2005-01-20 22:45:12.67617 | test string | 998455 > 2005-01-20 22:47:07.218733 | test string | 998455 > 2005-01-20 22:48:31.5945 | test string | 998455 > 2005-01-20 22:50:42.391373 | test string | 998455 > (6 rows) >=20 > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 22:26:39.057576 | test string | 8455 > 2005-01-20 22:44:37.486004 | test string | 8455 > 2005-01-20 22:45:12.67617 | test string | 8455 > 2005-01-20 22:47:07.218733 | test string | 8455 > 2005-01-20 22:48:31.5945 | test string | 8455 > 2005-01-20 22:50:42.391373 | test string | 8455 > (6 rows) >=20 > thedate | astring | anumber > ---------+---------+--------- > (0 rows) >=20 > real 3m13.732s > user 0m0.005s > sys 0m0.004s >=20 > -------------------------------------------------------------------------= ------------------------ > SC400 FC3 With XFS: >=20 > 4665 root 18 0 4380 468 404 D 21.6 0.2 0:02.43 dd >=20 > [root@localhost tmp]# time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count= =3D1M > 1048576+0 records in > 1048576+0 records out >=20 > real 0m22.676s > user 0m0.663s > sys 0m3.494s >=20 > 4667 root 19 0 3672 468 404 R 8.7 0.2 0:00.26 dd > [root@localhost tmp]# time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count= =3D2M > 2097152+0 records in > 2097152+0 records out >=20 > real 0m46.277s > user 0m1.214s > sys 0m7.141s >=20 > 4674 root 25 0 4532 468 404 R 13.6 0.2 0:02.34 dd > 4674 root 18 0 4532 468 404 D 17.6 0.2 0:05.52 dd >=20 > [root@localhost tmp]# time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count= =3D3M > 3145728+0 records in > 3145728+0 records out >=20 > real 1m12.727s > user 0m1.884s > sys 0m10.573s >=20 > copy tests: > 4744 root 18 0 3940 620 540 D 9.3 0.2 0:00.73 cp > 4744 root 18 0 3940 620 540 R 9.3 0.2 0:01.78 cp > 4744 root 18 0 3940 620 540 R 4.0 0.2 0:06.62 cp >=20 > [root@localhost tmp]# time cp tstfile tstfile2 >=20 > real 2m22.101s > user 0m0.500s > sys 0m8.178s >=20 > bonnie tests: >=20 > [root@localhost tmp]# bonnie++ -u root -s 1024 -r 512 -n 5 -d /tmp > Using uid:0, gid:0. > Writing with putc()...done > Writing intelligently...done > Rewriting...done > Reading with getc()...done > Reading intelligently...done > start 'em...done...done...done... > Create files in sequential order...done. > Stat files in sequential order...done. > Delete files in sequential order...done. > Create files in random order...done. > Stat files in random order...done. > Delete files in random order...done. > Version 1.03 ------Sequential Output------ --Sequential Input- > --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP > /sec %CP > localhost.locald 1G 33376 97 48548 7 20451 4 25233 69 47488 5 > 154.7 0 > ------Sequential Create------ --------Random > Create-------- > -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > 5 3724 17 +++++ +++ 3828 17 2957 14 +++++ +++ > 2290 10 > localhost.localdomain,1G,33376,97,48548,7,20451,4,25233,69,47488,5,154.7,= 0,5,3724,17,+++++,+++,3828,17,2957,14,+++++,+++,2290,10 >=20 > 4747 root 23 0 3004 976 840 R 88.9 0.4 1:56.32 bonnie++ > 4747 root 18 0 2996 968 840 R 7.0 0.4 2:09.53 bonnie++ > 4751 root 17 0 2996 980 844 D 0.3 0.4 0:00.02 bonnie++ >=20 > [root@localhost tmp]# bonnie++ -u root -s 2048 -r 512 -n 5 -d /tmp > Using uid:0, gid:0. > Writing with putc()...done > Writing intelligently...done > Rewriting...done > Reading with getc()...done > Reading intelligently...done > start 'em...done...done...done... > Create files in sequential order...done. > Stat files in sequential order...done. > Delete files in sequential order...done. > Create files in random order...done. > Stat files in random order...done. > Delete files in random order...done. > Version 1.03 ------Sequential Output------ --Sequential Input- > --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP > /sec %CP > localhost.locald 2G 32988 97 47779 6 21777 5 26410 74 49463 6 > 130.0 0 > ------Sequential Create------ --------Random > Create-------- > -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > 5 3145 15 +++++ +++ 3641 15 3299 17 +++++ +++ > 2037 9 > localhost.localdomain,2G,32988,97,47779,6,21777,5,26410,74,49463,6,130.0,= 0,5,3145,15,+++++,+++,3641,15,3299,17,+++++,+++,2037,9 >=20 > 4788 root 25 0 3120 968 840 R 97.9 0.4 0:19.11 bonnie++ > 4788 root 25 0 3124 972 840 R 97.6 0.4 0:42.54 bonnie++ > 4788 root 25 0 3116 968 840 D 6.7 0.4 1:37.18 bonnie++ >=20 > [root@localhost tmp]# bonnie++ -u root -s 3072 -r 512 -n 5 -d /tmp > Using uid:0, gid:0. > Writing with putc()...done > Writing intelligently...done > Rewriting...done > Reading with getc()...done > Reading intelligently...done > start 'em...done...done...done... > Create files in sequential order...done. > Stat files in sequential order...done. > Delete files in sequential order...done. > Create files in random order...done. > Stat files in random order...done. > Delete files in random order...done. > Version 1.03 ------Sequential Output------ --Sequential Input- > --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP > /sec %CP > localhost.locald 3G 32331 97 47200 6 21836 5 26740 75 49292 6 > 125.9 0 > ------Sequential Create------ --------Random > Create-------- > -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > 5 3215 14 +++++ +++ 3360 14 3036 15 +++++ +++ > 1856 8 > localhost.localdomain,3G,32331,97,47200,6,21836,5,26740,75,49292,6,125.9,= 0,5,3215,14,+++++,+++,3360,14,3036,15,+++++,+++,1856,8 >=20 > 5024 postgres 18 0 20116 11m 10m R 68.6 4.6 0:05.75 postmaster > 5024 postgres 18 0 21288 12m 10m D 45.3 5.1 0:13.88 postmaster > 4950 postgres 15 0 19572 10m 9828 D 1.0 4.0 0:00.13 postmaster > 024 postgres 18 0 21568 12m 10m R 60.9 5.1 0:21.99 postmaster > 5024 postgres 19 0 20176 11m 10m R 51.9 4.7 0:34.02 postmaster >=20 > -bash-3.00$ time psql -d perftst -f tst1.sql > CREATE TABLE > SELECT > INSERT 0 500000 > DELETE 500000 > DELETE 38462 > SELECT > UPDATE 92308 > DROP TABLE > SELECT > DELETE 37037 > VACUUM > VACUUM > DROP TABLE > DROP TABLE > DROP TABLE > VACUUM >=20 > real 2m9.088s > user 0m0.007s > sys 0m0.005s >=20 > 5026 postgres 18 0 21392 13m 11m D 45.9 5.3 0:23.92 postmaster > 5026 postgres 18 0 21392 13m 11m D 44.3 5.3 0:28.45 postmaster > 5026 postgres 18 0 37780 17m 11m R 10.7 7.1 0:34.22 postmaster > 5026 postgres 18 0 21392 14m 11m D 13.0 5.7 0:37.30 postmaster >=20 > -bash-3.00$ time psql -d perftst -f tst1.sql > CREATE TABLE > SELECT > INSERT 0 500000 > DELETE 500000 > DELETE 38462 > SELECT > UPDATE 92308 > DROP TABLE > SELECT > DELETE 37037 > VACUUM > VACUUM > DROP TABLE > DROP TABLE > DROP TABLE > VACUUM >=20 > real 3m22.001s > user 0m0.007s > sys 0m0.009s > -bash-3.00$ time psql -d perftst -f tst1.sql > CREATE TABLE > SELECT > INSERT 0 500000 > DELETE 500000 > DELETE 38462 > SELECT > UPDATE 92308 > DROP TABLE > SELECT > DELETE 37037 > VACUUM > VACUUM > DROP TABLE > DROP TABLE > DROP TABLE > VACUUM >=20 > real 2m17.877s > user 0m0.003s > sys 0m0.006s >=20 > 26432 postgres 18 0 20140 10m 9764 R 46.6 4.2 0:03.92 postmaster >=20 > bash-3.00$ time psql -d perftst -f tst1.sql > CREATE TABLE > SELECT > INSERT 0 500000 > DELETE 500000 > DELETE 38462 > SELECT > UPDATE 92308 > DROP TABLE > SELECT > DELETE 37037 > VACUUM > VACUUM > DROP TABLE > DROP TABLE > DROP TABLE > VACUUM >=20 > real 2m17.666s > user 0m0.003s > sys 0m0.006s >=20 > 26440 postgres 18 0 22160 13m 10m D 50.3 5.2 0:23.60 postmaster > 26440 postgres 18 0 37820 17m 11m R 9.3 6.9 0:34.46 postmaster >=20 > -bash-3.00$ time psql -d perftst -f tst1.sql > CREATE TABLE > SELECT > INSERT 0 500000 > DELETE 500000 > DELETE 38462 > SELECT > UPDATE 92308 > DROP TABLE > SELECT > DELETE 37037 > VACUUM > VACUUM > DROP TABLE > DROP TABLE > DROP TABLE > VACUUM >=20 > real 2m19.578s > user 0m0.005s > sys 0m0.010s >=20 > (6 Million Rows) > 26445 postgres 19 0 20116 10m 10m R 58.9 4.3 0:03.32 postmaster > 26445 postgres 19 0 20116 10m 10m R 51.6 4.4 0:18.37 postmaster > 26445 postgres 18 0 31764 22m 10m D 9.3 8.9 1:33.57 postmaster > 26445 postgres 18 0 45076 36m 11m R 8.3 14.7 5:21.49 postmaster >=20 > -bash-3.00$ time psql -d perftst -f tst1.sql > CREATE TABLE > SELECT > INSERT 0 3000000 > DELETE 3000000 > DELETE 230772 > SELECT > UPDATE 553848 > DROP TABLE > SELECT > DELETE 222222 > VACUUM > VACUUM > DROP TABLE > DROP TABLE > DROP TABLE > VACUUM >=20 > real 21m15.074s > user 0m0.003s > sys 0m0.007s >=20 > (tst.sql) > select * from test1 where anumber=3D894455; > select * from test1 where anumber=3D75455; > select * from test1 where anumber=3D444455; > select * from test1 where anumber=3D7594455; > select * from test1 where anumber=3D7511455; > select * from test1 where anumber=3D7555; > select * from test1 where anumber=3D5; > select * from test1 where anumber=3D4455; > select * from test1 where anumber=3D111155; > select * from test1 where anumber=3D33355; > select * from test1 where anumber=3D998455; > select * from test1 where anumber=3D8455; > select * from test1 where anumber=3D22754455; > ------------------------------------------------------------------------ >=20 > -bash-3.00$ time psql -d perftst -f tst.sql > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 18:45:25.058853 | test string | 894455 > 2005-01-20 19:10:26.609873 | test string | 894455 > 2005-01-20 19:12:06.125588 | test string | 894455 > 2005-01-20 19:12:47.447711 | test string | 894455 > 2005-01-20 19:15:21.224914 | test string | 894455 > 2005-01-20 19:16:12.809133 | test string | 894455 > (6 rows) >=20 > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 18:45:25.058853 | test string | 75455 > 2005-01-20 19:10:26.609873 | test string | 75455 > 2005-01-20 19:12:06.125588 | test string | 75455 > 2005-01-20 19:12:47.447711 | test string | 75455 > 2005-01-20 19:15:21.224914 | test string | 75455 > 2005-01-20 19:16:12.809133 | test string | 75455 > (6 rows) >=20 > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 18:45:25.058853 | test string | 444455 > 2005-01-20 19:10:26.609873 | test string | 444455 > 2005-01-20 19:12:06.125588 | test string | 444455 > 2005-01-20 19:12:47.447711 | test string | 444455 > 2005-01-20 19:15:21.224914 | test string | 444455 > 2005-01-20 19:16:12.809133 | test string | 444455 > (6 rows) >=20 > thedate | astring | anumber > ---------+---------+--------- > (0 rows) >=20 > thedate | astring | anumber > ---------+---------+--------- > (0 rows) >=20 > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 18:45:25.058853 | test string | 7555 > 2005-01-20 19:10:26.609873 | test string | 7555 > 2005-01-20 19:12:06.125588 | test string | 7555 > 2005-01-20 19:12:47.447711 | test string | 7555 > 2005-01-20 19:15:21.224914 | test string | 7555 > 2005-01-20 19:16:12.809133 | test string | 7555 > (6 rows) >=20 > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 18:45:25.058853 | test string | 5 > 2005-01-20 19:10:26.609873 | test string | 5 > 2005-01-20 19:12:06.125588 | test string | 5 > 2005-01-20 19:12:47.447711 | test string | 5 > 2005-01-20 19:15:21.224914 | test string | 5 > 2005-01-20 19:16:12.809133 | test string | 5 > (6 rows) >=20 > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 18:45:25.058853 | test string | 4455 > 2005-01-20 19:10:26.609873 | test string | 4455 > 2005-01-20 19:12:06.125588 | test string | 4455 > 2005-01-20 19:12:47.447711 | test string | 4455 > 2005-01-20 19:15:21.224914 | test string | 4455 > 2005-01-20 19:16:12.809133 | test string | 4455 > (6 rows) >=20 > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 18:45:25.058853 | test string | 111155 > 2005-01-20 19:10:26.609873 | test string | 111155 > 2005-01-20 19:12:06.125588 | test string | 111155 > 2005-01-20 19:12:47.447711 | test string | 111155 > 2005-01-20 19:15:21.224914 | test string | 111155 > 2005-01-20 19:16:12.809133 | test string | 111155 > (6 rows) >=20 > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 18:45:25.058853 | test string | 33355 > 2005-01-20 19:10:26.609873 | test string | 33355 > 2005-01-20 19:12:06.125588 | test string | 33355 > 2005-01-20 19:12:47.447711 | test string | 33355 > 2005-01-20 19:15:21.224914 | test string | 33355 > 2005-01-20 19:16:12.809133 | test string | 33355 > (6 rows) >=20 > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 18:45:25.058853 | test string | 998455 > 2005-01-20 19:10:26.609873 | test string | 998455 > 2005-01-20 19:12:06.125588 | test string | 998455 > 2005-01-20 19:12:47.447711 | test string | 998455 > 2005-01-20 19:15:21.224914 | test string | 998455 > 2005-01-20 19:16:12.809133 | test string | 998455 > (6 rows) >=20 > thedate | astring | anumber > ----------------------------+-------------+--------- > 2005-01-20 18:45:25.058853 | test string | 8455 > 2005-01-20 19:10:26.609873 | test string | 8455 > 2005-01-20 19:12:06.125588 | test string | 8455 > 2005-01-20 19:12:47.447711 | test string | 8455 > 2005-01-20 19:15:21.224914 | test string | 8455 > 2005-01-20 19:16:12.809133 | test string | 8455 > (6 rows) >=20 > thedate | astring | anumber > ---------+---------+--------- > (0 rows) >=20 > real 2m31.883s > user 0m0.002s > sys 0m0.005s >=20 > -------------------------------------------------------------------------= -------------------------------- > Various Notes: > -------------------------------------------------------------------------= -------------------------------- > 1/18/05 PE 2400 bonnie++ Test with ext3: >=20 > Top Output: > 3911 root 25 0 3072 964 840 R 98.4 0.2 1:21.21 bonnie++ >=20 > Bonnie++ Test1: > [root@es1 npavlica]# bonnie++ -u root -s 1024 -r 512 -n 5 -d /tmp > Using uid:0, gid:0. > Writing with putc()...done > Writing intelligently...done > Rewriting...done > Reading with getc()...done > Reading intelligently...done > start 'em...done...done...done... > Create files in sequential order...done. > Stat files in sequential order...done. > Delete files in sequential order...done. > Create files in random order...done. > Stat files in random order...done. > Delete files in random order...done. > Version 1.03 ------Sequential Output------ --Sequential Input- > --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP > /sec %CP > es1.estation.com 1G 7469 91 9881 15 4846 6 9073 90 21034 15 > 362.4 2 > ------Sequential Create------ --------Random > Create-------- > -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > 5 7962 90 +++++ +++ +++++ +++ 8874 99 +++++ +++ > 8803 69 > es1.estation.com,1G,7469,91,9881,15,4846,6,9073,90,21034,15,362.4,2,5,796= 2,90,+++++,+++,+++++,+++,8874,99,+++++,+++,8803,69 >=20 > Bonnie++ Test 2: >=20 > [root@es1 npavlica]# bonnie++ -u root -s 2048 -r 512 -n 5 -d /tmp > Using uid:0, gid:0. > Writing with putc()...done > Writing intelligently...done > Rewriting...done > Reading with getc()...done > Reading intelligently...done > start 'em...done...done...done... > Create files in sequential order...done. > Stat files in sequential order...done. > Delete files in sequential order...done. > Create files in random order...done. > Stat files in random order...done. > Delete files in random order...done. > Version 1.03 ------Sequential Output------ --Sequential Input- > --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP > /sec %CP > es1.estation.com 2G 7208 88 8042 12 4652 6 8916 96 19611 14 > 252.9 2 > ------Sequential Create------ --------Random > Create-------- > -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > 5 8075 92 +++++ +++ +++++ +++ 6731 75 +++++ +++ > +++++ +++ > es1.estation.com,2G,7208,88,8042,12,4652,6,8916,96,19611,14,252.9,2,5,807= 5,92,+++++,+++,+++++,+++,6731,75,+++++,+++,+++++,+++ >=20 > Bonnie++ Test 3: >=20 > [root@es1 npavlica]# bonnie++ -u root -s 3072 -r 512 -n 5 -d /tmp > Using uid:0, gid:0. > Writing with putc()...done > Writing intelligently...done > Rewriting...done > Reading with getc()...done > Reading intelligently...done > start 'em...done...done...done... > Create files in sequential order...done. > Stat files in sequential order...done. > Delete files in sequential order...done. > Create files in random order...done. > Stat files in random order...done. > Delete files in random order...done. > Version 1.03 ------Sequential Output------ --Sequential Input- > --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP > /sec %CP > es1.estation.com 3G 6995 85 7921 12 4723 6 9728 96 20919 15 > 218.5 1 > ------Sequential Create------ --------Random > Create-------- > -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > 5 7840 90 +++++ +++ +++++ +++ 8852 100 +++++ +++ > 9220 72 > es1.estation.com,3G,6995,85,7921,12,4723,6,9728,96,20919,15,218.5,1,5,784= 0,90,+++++,+++,+++++,+++,8852,100,+++++,+++,9220,72 >=20 > DD Test 1: > [root@es1 npavlica]# time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count= =3D1M > 1048576+0 records in > 1048576+0 records out >=20 > real 2m14.152s > user 0m1.649s > sys 0m29.622s >=20 > DD Test 2: > [root@es1 npavlica]# time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count= =3D1M > 1048576+0 records in > 1048576+0 records out >=20 > real 2m14.152s > user 0m1.649s > sys 0m29.622s > [root@es1 npavlica]# rm tstfile > rm: remove regular file `tstfile'? y > [root@es1 npavlica]# time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count= =3D2M > 2097152+0 records in > 2097152+0 records out >=20 > real 4m7.145s > user 0m3.152s > sys 1m0.303s >=20 > DD Test 3: > [root@es1 npavlica]# time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count= =3D3M > 3145728+0 records in > 3145728+0 records out >=20 > real 6m40.592s > user 0m4.751s > sys 1m31.968s >=20 > FreeBSD5.3 I/O Tests: > Partition Layout / Utilization: > $ df -h > Files ystem Size Used Avail Capacity Mounted on > /dev/aacd0s1a 248M 35M 193M 15% / > devfs 1.0K 1.0K 0B 100% /dev > /dev/aacd0s1e 248M 6.0K 228M 0% /tmp > /dev/aacd0s1f 31G 1.8G 27G 6% /usr > /dev/aacd0s1d 248M 2.6M 225M 1% /var >=20 > Bonnie++ Test 1: > 715 nick.pavlica 97 0 2380K 1324K RUN 0:53 21.12% 20.85% > bonnie++ >=20 > $ bonnie++ -s 1024 -r 512 -n 5 > Writing a byte at a time...done > Writing intelligently...done > Rewriting...done > Reading a byte at a time...done > Reading intelligently...done > start 'em...done...done...done...done...done... > Create files in sequential order...done. > Stat files in sequential order...done. > Delete files in sequential order...done.535 nick.pavlica -4 0 > 2380K 1216K getblk 0:17 2.84% 2.83% bonnie++ > Create files in random order...done. > Stat files in random order...done. > Delete files in random order...done. > Version 1.93c ------Sequential Output------ --Sequential Input- > --Random- > Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP > /sec %CP > vader.e--station 1G 60 99 6351 7 4169 5 170 99 20904 13 > 200.7 19 > Latency 190ms 6458ms 10221ms 57902us 33802us > 423ms > Version 1.93c ------Sequential Create------ --------Random > Create-------- > vader.e--station.co -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > 5 5647 59 +++++ +++ 9412 74 5757 56 +++++ +++ > 5291 45 > Latency 9741us 917us 315us 225ms 875us > 11165us > 1.93c,1.93c,vader.e=E2=80=94station.com,1,1106164339,1G,,60,99,6351,7,416= 9,5,170,99,20904,13,200.7,19,5,,,,,5647,59,+++++,+++,9412,74,5757,56,+++++,= +++,5291,45,190ms,6458ms,10221ms,57902us,33802us,423ms,9741us,917us,315us,2= 25ms,875us,11165us >=20 > Bonnie++ Test 2: > 745 nick.pavlica -16 0 2380K 1200K wdrain 0:58 7.03% 7.03% > bonnie++ > 745 nick.pavlica -4 0 2380K 1200K getblk 1:21 5.57% 5.57% > bonnie++ >=20 > $ bonnie++ -s 2048 -r 512 -n 5 > Writing a byte at a time...done > Writing intelligently...done > Rewriting...done > Reading a byte at a time...done > Reading intelligently...done > start 'em...done...done...done...done...done... > Create files in sequential order...done. > Stat files in sequential order...done. > Delete files in sequential order...done. > Create files in random order...done. > Stat files in random order...done. > Delete files in random order...done. > Version 1.93c ------Sequential Output------ --Sequential Input- > --Random- > Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP > /sec %CP > vader.e--station 2G 60 99 5483 6 4448 5 170 99 19045 12 > 129.8 13 > Latency 152ms 9726ms 7113ms 55161us 42746us > 3598ms > Version 1.93c ------Sequential Create------ --------Random > Create-------- > vader.e--station.co -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > 5 7036 72 +++++ +++ 9521 75 4877 50 +++++ +++ > 9029 76 > Latency 5716us 955us 3648us 10997us 400us > 320us > 1.93c,1.93c,vader.e=E2=80=94station.com,1,1106164107,2G,,60,99,5483,6,444= 8,5,170,99,19045,12,129.8,13,5,,,,,7036,72,+++++,+++,9521,75,4877,50,+++++,= +++,9029,76,152ms,9726ms,7113ms,55161us,42746us,3598ms,5716us,955us,3648us,= 10997us,400us,320us >=20 > Bonnie++ Test 3: > 789 nick.pavlica -4 0 2380K 1208K getblk 1:52 4.64% 4.64% > bonnie++ > 789 nick.pavlica -4 0 2380K 1208K getblk 1:52 2.49% 2.49% > bonnie++ >=20 > $ bonnie++ -s 3072 -r 512 -n 5 > Writing a byte at a time...done > Writing intelligently...done > Rewriting...done > Reading a byte at a time...done > Reading intelligently...done > start 'em...done...done...done...done...done... > Create files in sequential order...done. > Stat files in sequential order...done. > Delete files in sequential order...done. > Create files in random order...done. > Stat files in random order...done. > Delete files in random order...done. > Version 1.93c ------Sequential Output------ --Sequential Input- > --Random- > Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP > /sec %CP > vader.e--station 3G 60 99 6184 7 3907 4 167 98 20993 14 > 99.3 11 > Latency 177ms 9312ms 15369ms 72183us 70690us > 608ms > Version 1.93c ------Sequential Create------ --------Random > Create-------- > vader.e--station.co -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > 5 7032 72 +++++ +++ 9667 77 7445 74 +++++ +++ > 4197 36 > Latency 5855us 1682us 359us 8565us 11109us > 11436us > 1.93c,1.93c,vader.e--station.com,1,1106166552,3G,,60,99,6184,7,3907,4,167= ,98,20993,14,99.3,11,5,,,,,7032,72,+++++,+++,9667,77,7445,74,+++++,+++,4197= ,36,177ms,9312ms,15369ms,72183us,70690us,608ms,5855us,1682us,359us,8565us,1= 1109us,11436us >=20 > DD Test1: > $ time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count=3D1M > 1048576+0 records in > 1048576+0 records out > 1073741824 bytes transferred in 168.394218 secs (6376358 bytes/sec) > 168.41 real 1.34 user 33.60 sys >=20 > 168.41 / 60 =3D 2.806833333 >=20 > DD Test 2: >=20 > Top was: > - 575 nick.pavlica -16 0 1212K 588K wdrain 0:13 17.30% 17.04% > dd > - 575 nick.pavlica -16 0 1212K 588K wdrain 0:32 20.85% 20.85% > dd >=20 > $ time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count=3D2M > 2097152+0 records in > 2097152+0 records out > 2147483648 bytes transferred in 359.389723 secs (5975362 bytes/sec) > 359.59 real 2.57 user 68.14 sys >=20 > 359.59 / 60 =3D 5.993166667 >=20 > DD Test 3: > 661 nick.pavlica -16 0 1212K 588K wdrain 0:06 20.86% 15.19% dd >=20 > $ time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count=3D3M > 3145728+0 records in > 3145728+0 records out > 3221225472 bytes transferred in 520.399250 secs (6189912 bytes/sec) > 520.60 real 4.06 user 102.48 sys >=20 > 520.6 / 60 =3D 8.676666667 >=20 > Fedora C3 With XFS: > (minimal install) > [root@localhost ~]# df -h > Filesystem Size Used Avail Use% Mounted on > /dev/sda3 34G 737M 33G 3% / > /dev/sda1 92M 6.0M 86M 7% /boot > none 252M 0 252M 0% /dev/shm >=20 > Bonnie++ Test 1: > [root@localhost tmp]# bonnie++ -u root -s 1024 -r 512 -n 5 -d /tmp > Using uid:0, gid:0. > Writing with putc()...done > Writing intelligently...done > Rewriting...done > Reading with getc()...done > Reading intelligently...done > start 'em...done...done...done... > Create files in sequential order...done. > Stat files in sequential order...done. > Delete files in sequential order...done. > Create files in random order...done. > Stat files in random order...done. > Delete files in random order...done. > Version 1.03 ------Sequential Output------ --Sequential Input- > --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP > /sec %CP > localhost.locald 1G 8206 96 11876 13 5982 10 9375 95 24909 21 > 538.8 4 > ------Sequential Create------ --------Random > Create-------- > -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > 5 652 13 +++++ +++ 600 14 622 14 +++++ +++ > 365 7 > localhost.localdomain,1G,8206,96,11876,13,5982,10,9375,95,24909,21,538.8,= 4,5,652,13,+++++,+++,600,14,622,14,+++++,+++,365,7 >=20 > Bonnie++ Test 2: >=20 > [root@localhost tmp]# bonnie++ -u root -s 2048 -r 512 -n 5 -d /tmp > Using uid:0, gid:0. > Writing with putc()...done > Writing intelligently...done > Rewriting...done > Reading with getc()...done > Reading intelligently...done > start 'em...done...done...done... > Create files in sequential order...done. > Stat files in sequential order...done. > Delete files in sequential order...done. > Create files in random order...done. > Stat files in random order...done. > Delete files in random order...done. > Version 1.03 ------Sequential Output------ --Sequential Input- > --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP > /sec %CP > localhost.locald 2G 7981 94 8794 9 6026 10 9535 97 25025 21 > 392.1 3 > ------Sequential Create------ --------Random > Create-------- > -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > 5 635 13 +++++ +++ 587 14 574 13 +++++ +++ > 388 8 > localhost.localdomain,2G,7981,94,8794,9,6026,10,9535,97,25025,21,392.1,3,= 5,635,13,+++++,+++,587,14,574,13,+++++,+++,388,8 >=20 > Bonnie++ Test 3: > 2745 root 25 0 4176 976 2300 R 99.9 0.2 3:34.49 bonnie++ >=20 > [root@localhost tmp]# bonnie++ -u root -s 3072 -r 512 -n 5 -d /tmp > Using uid:0, gid:0. > Writing with putc()...done > Writing intelligently...done > Rewriting...done > Reading with getc()...done > Reading intelligently...done > start 'em...done...done...done... > Create files in sequential order...done. > Stat files in sequential order...done. > Delete files in sequential order...done. > Create files in random order...done. > Stat files in random order...done. > Delete files in random order...done. > Version 1.03 ------Sequential Output------ --Sequential Input- > --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP > /sec %CP > localhost.locald 3G 7909 93 8053 8 3498 5 9328 95 24895 21 > 351.9 3 > ------Sequential Create------ --------Random > Create-------- > -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > 5 559 12 +++++ +++ 528 13 594 13 +++++ +++ > 414 9 > localhost.localdomain,3G,7909,93,8053,8,3498,5,9328,95,24895,21,351.9,3,5= ,559,12,+++++,+++,528,13,594,13,+++++,+++,414,9 >=20 > DD Test 1: > 2629 root 18 0 4632 472 3344 D 24.5 0.1 0:20.46 dd >=20 > [root@localhost ~]# time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count= =3D1M > 1048576+0 records in > 1048576+0 records out >=20 > real 1m39.842s > user 0m4.404s > sys 0m22.647s >=20 > DD Test 2: > 2669 root 18 0 4596 472 3344 D 20.5 0.1 0:07.93 dd >=20 > [root@localhost ~]# time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count= =3D2M > 2097152+0 records in > 2097152+0 records out >=20 > real 4m11.713s > user 0m9.131s > sys 0m46.951s >=20 > DD Test 3: > 2707 root 18 0 3832 472 3344 R 23.2 0.1 0:30.84 dd >=20 > [root@localhost ~]# time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count= =3D3M > 3145728+0 records in > 3145728+0 records out >=20 > real 6m52.182s > user 0m13.450s > sys 1m10.167s >=20 > -------------------------------------------------------------------------= ----------------------------------- > SC 400 Tests > fc3 with xfs: >=20 > [root@localhost ~]# time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count= =3D1M > 1048576+0 records in > 1048576+0 records out >=20 > real 0m27.491s > user 0m1.436s > sys 0m6.888s >=20 > [root@localhost ~]# time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count= =3D2M > 2097152+0 records in > 2097152+0 records out >=20 > real 1m3.194s > user 0m2.847s > sys 0m13.766s > [root@localhost ~]# time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count= =3D3M > 3145728+0 records in > 3145728+0 records out >=20 > real 1m59.668s > user 0m4.304s > sys 0m20.673s >=20 > [root@localhost ~]# time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count= =3D7M > 7340032+0 records in > 7340032+0 records out >=20 > real 4m6.371s > user 0m9.877s > sys 0m49.454s >=20 > [root@localhost ~]# time dd bs=3D1024 if=3D/dev/zero of=3Dtstfile count= =3D3M > 3145728+0 records in > 3145728+0 records out >=20 > real 1m35.411s > user 0m4.495s > sys 0m21.006s > [root@localhost ~]# ls > anaconda-ks.cfg install.log install.log.syslog tstfile >=20 > copy test: >=20 > (sc400) > [root@localhost ~]# time cp tstfile tstfile2 >=20 > real 3m22.202s > user 0m1.320s > sys 0m12.605s >=20 > (pe2400) > [root@localhost ~]# time cp tstfile tstfile2 >=20 > real 11m6.046s > user 0m1.091s > sys 0m39.096s >=20 > =20 > __________________________________ > Do you Yahoo!? > Yahoo! Mail - now with 250MB free storage. Learn more. > http://info.mail.yahoo.com/mail_250 > From owner-freebsd-performance@FreeBSD.ORG Sun Jan 23 21:09:10 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 393EF16A4CE for ; Sun, 23 Jan 2005 21:09:10 +0000 (GMT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id D0CE043D3F for ; Sun, 23 Jan 2005 21:09:09 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.13.1/8.13.1) with ESMTP id j0NL8kk7053402; Sun, 23 Jan 2005 16:08:46 -0500 (EST) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)j0NL8kxS053399; Sun, 23 Jan 2005 21:08:46 GMT (envelope-from robert@fledge.watson.org) Date: Sun, 23 Jan 2005 21:08:45 +0000 (GMT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Eric Anderson In-Reply-To: <41EFEBFE.8050505@centtech.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-performance@freebsd.org cc: Mike Tancsa Subject: Re: NFS and SAMBA on RELENG_5 vs RELENG_4 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2005 21:09:10 -0000 On Thu, 20 Jan 2005, Eric Anderson wrote: > I can tell you this - you must increase the number of nfsd threads to a > high number, if you plan on really hammering the machine with nfs and > lots of clients. I recompiled the nfsd binary with it tweaked to allow > 256 threads, and that still isn't quite enough. You need something on > the order of: 1 per active machine using nfs * 1.10. The hard part is > finding out how many active machines you have. I usually start with > about 20% of my total machines mounted to the server, and then watch the > nfsd threads cpu time. If the lowest thread is using more than about > 3-4% of the time of the 10-15th top nfsd process, then you need to bump > up the number. That may be confusing.. Hmm. So it sounds like it would make sense for us to do that in the src tree. Is it sufficient to simply redefine MAXNFSDCNT from 20 to 256, or do other things also need tweaking? Robert N M Watson From owner-freebsd-performance@FreeBSD.ORG Sun Jan 23 21:29:08 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ABD3316A4CE; Sun, 23 Jan 2005 21:29:08 +0000 (GMT) Received: from otter3.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id F057043D49; Sun, 23 Jan 2005 21:29:07 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [192.168.42.24] (andersonbox4.centtech.com [192.168.42.24]) by otter3.centtech.com (8.12.3/8.12.3) with ESMTP id j0NLT5OJ090453; Sun, 23 Jan 2005 15:29:05 -0600 (CST) (envelope-from anderson@centtech.com) Message-ID: <41F41711.4020907@centtech.com> Date: Sun, 23 Jan 2005 15:28:49 -0600 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.3) Gecko/20041110 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Robert Watson References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-performance@freebsd.org cc: Mike Tancsa Subject: Re: NFS and SAMBA on RELENG_5 vs RELENG_4 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2005 21:29:08 -0000 Robert Watson wrote: > On Thu, 20 Jan 2005, Eric Anderson wrote: > > >>I can tell you this - you must increase the number of nfsd threads to a >>high number, if you plan on really hammering the machine with nfs and >>lots of clients. I recompiled the nfsd binary with it tweaked to allow >>256 threads, and that still isn't quite enough. You need something on >>the order of: 1 per active machine using nfs * 1.10. The hard part is >>finding out how many active machines you have. I usually start with >>about 20% of my total machines mounted to the server, and then watch the >>nfsd threads cpu time. If the lowest thread is using more than about >>3-4% of the time of the 10-15th top nfsd process, then you need to bump >>up the number. That may be confusing.. > > > Hmm. So it sounds like it would make sense for us to do that in the src > tree. Is it sufficient to simply redefine MAXNFSDCNT from 20 to 256, or > do other things also need tweaking? I think it would make sense. Yes, that is the only knob to turn.. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology I have seen the future and it is just like the present, only longer. ------------------------------------------------------------------------ From owner-freebsd-performance@FreeBSD.ORG Mon Jan 24 16:27:00 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7127D16A4CE; Mon, 24 Jan 2005 16:27:00 +0000 (GMT) Received: from stewie.obfuscated.net (stewie.obfuscated.net [66.118.188.125]) by mx1.FreeBSD.org (Postfix) with ESMTP id DD92D43D53; Mon, 24 Jan 2005 16:26:59 +0000 (GMT) (envelope-from m@obmail.net) Received: from [192.168.1.100] (653259hfc120.tampabay.rr.com [65.32.59.120]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by stewie.obfuscated.net (Postfix) with ESMTP id EDEC56104; Mon, 24 Jan 2005 11:26:58 -0500 (EST) In-Reply-To: References: Mime-Version: 1.0 (Apple Message framework v619) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Michael Conlen Date: Mon, 24 Jan 2005 11:26:16 -0500 To: Robert Watson X-Mailer: Apple Mail (2.619) cc: freebsd-performance@freebsd.org Subject: Re: NFS and SAMBA on RELENG_5 vs RELENG_4 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2005 16:27:00 -0000 On Jan 23, 2005, at 4:08 PM, Robert Watson wrote: > On Thu, 20 Jan 2005, Eric Anderson wrote: > >> I can tell you this - you must increase the number of nfsd threads to >> a >> high number, if you plan on really hammering the machine with nfs and >> lots of clients. I recompiled the nfsd binary with it tweaked to >> allow >> 256 threads, and that still isn't quite enough. You need something on >> the order of: 1 per active machine using nfs * 1.10. The hard part is >> finding out how many active machines you have. I usually start with >> about 20% of my total machines mounted to the server, and then watch >> the >> nfsd threads cpu time. If the lowest thread is using more than about >> 3-4% of the time of the 10-15th top nfsd process, then you need to >> bump >> up the number. That may be confusing.. > > Hmm. So it sounds like it would make sense for us to do that in the > src > tree. Is it sufficient to simply redefine MAXNFSDCNT from 20 to 256, > or > do other things also need tweaking? > > Robert, I had submitted some patches about six months ago to remove the constant but my need for paid work caught up with me and I didn't have time to follow up with the commiter. The problem is that some functions (interrupt handlers for example) need to know the length of the array when they get called. My solution was to allocate the array of nfsd processes then call the and use varargs pass the value of -d and the location of the array to them. Doesn't really make sense to me to have a statically compiled value at all. Since I'm dealing with NFS performance issues again I can dig this stuff up and resubmit it if your interested. From owner-freebsd-performance@FreeBSD.ORG Mon Jan 24 21:59:59 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 903E116A4CE for ; Mon, 24 Jan 2005 21:59:59 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.206]) by mx1.FreeBSD.org (Postfix) with ESMTP id DA1D843D3F for ; Mon, 24 Jan 2005 21:59:58 +0000 (GMT) (envelope-from linicks@gmail.com) Received: by rproxy.gmail.com with SMTP id 40so151139rnz for ; Mon, 24 Jan 2005 13:59:55 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding; b=ES0dVHEujONZVt2/8EQwXmhIt91lXmo+CtAqPpCtQ57d72DR5SzI5SPckWNBfm0tvf4hYUpGu4accQPqi4zSqCPUtOcrDAQq+4u65MZ4VzWZ1kj+2x31rj1n7DO0CK+4vXxuxR1dyNxQ+IIXA1MQiXhbDdB9XHBWLe4oQIMvaBc= Received: by 10.38.26.40 with SMTP id 40mr600402rnz; Mon, 24 Jan 2005 13:59:55 -0800 (PST) Received: by 10.38.8.43 with HTTP; Mon, 24 Jan 2005 13:59:55 -0800 (PST) Message-ID: Date: Mon, 24 Jan 2005 14:59:55 -0700 From: Nick Pavlica To: freebsd-questions@freebsd.org, freebsd-performance@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Nick Pavlica List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2005 21:59:59 -0000 All, I would like to start addressing some of the feedback that I have been given. I started this discussion because I felt that it was important to share the information I discovered in my testing. I also want to reiterate my earlier statement that this is not an X vs. X discussion, but an attempt to better understand the results, and hopefully look at ways of improving the results I had with FreeBSD 5.x. I'm also looking forward to seeing the improvements to the 5.x branch as it matures. I want to make it very clear that this is NOT A "Religious/Engineering War", please don't try to turn it into one. That said, lets move on to something more productive. I installed both operating systems using as many default options as possible and updated them with all of the latest patches. I was logged in via SSH from my workstation while running the tests. I didn't have X, running on any of the installations because it wasn't need. CPU and RAM utilization wasn't an issue during any of the tests, but the disk I/O performance was dramatically different. Please keep in mind that I ran these tests over and over to see if I had consistent results. I even did the same tests on other pieces of equipment not listed in my notes that yielded the same results time and time again. Some have confirmed that they have had similar results in there testing using other testing tools and methods. This makes me wounder why the gap is so large, and how it can be improved? I think that it would be beneficial to have others in this group do similar testing and post there results. This may help those that are working on the OS itself to find trouble areas, and ways to improve them. It may also help clarify many of the response questions because you will be able to completely control the testing environment. I look forward to seeing the testing results, and any good feedback that helps identify specific tuning options, or bugs that need to be addressed. Thanks! --Nick Pavlica --Laramie, WY From owner-freebsd-performance@FreeBSD.ORG Mon Jan 24 22:08:54 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E182516A4CE; Mon, 24 Jan 2005 22:08:54 +0000 (GMT) Received: from silver.he.iki.fi (helenius.fi [193.64.42.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7BCDB43D2F; Mon, 24 Jan 2005 22:08:53 +0000 (GMT) (envelope-from pete@he.iki.fi) Received: from [193.64.42.134] (h86.vuokselantie10.fi [193.64.42.134]) by silver.he.iki.fi (8.13.1/8.11.4) with ESMTP id j0OM8ofY091650; Tue, 25 Jan 2005 00:08:51 +0200 (EET) (envelope-from pete@he.iki.fi) Message-ID: <41F571F4.1090504@he.iki.fi> Date: Tue, 25 Jan 2005 00:08:52 +0200 From: Petri Helenius User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Nick Pavlica References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-performance@freebsd.org cc: freebsd-questions@freebsd.org Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2005 22:08:55 -0000 Are you sure you aren't comparing filesystems with different mount options? Async comes to mind first. Pete Nick Pavlica wrote: >All, > I would like to start addressing some of the feedback that I have >been given. I started this discussion because I felt that it was >important to share the information I discovered in my testing. I also >want to reiterate my earlier statement that this is not an X vs. X >discussion, but an attempt to better understand the results, and >hopefully look at ways of improving the results I had with FreeBSD >5.x. I'm also looking forward to seeing the improvements to the 5.x >branch as it matures. I want to make it very clear that this is NOT A >"Religious/Engineering War", please don't try to turn it into one. > >That said, lets move on to something more productive. I installed >both operating systems using as many default options as possible and >updated them with all of the latest patches. I was logged in via SSH >from my workstation while running the tests. I didn't have X, running >on any of the installations because it wasn't need. CPU and RAM >utilization wasn't an issue during any of the tests, but the disk I/O >performance was dramatically different. Please keep in mind that I >ran these tests over and over to see if I had consistent results. I >even did the same tests on other pieces of equipment not listed in my >notes that yielded the same results time and time again. Some have >confirmed that they have had similar results in there testing using >other testing tools and methods. This makes me wounder why the gap is >so large, and how it can be improved? > >I think that it would be beneficial to have others in this group do >similar testing and post there results. This may help those that are >working on the OS itself to find trouble areas, and ways to improve >them. It may also help clarify many of the response questions because >you will be able to completely control the testing environment. I >look forward to seeing the testing results, and any good feedback that >helps identify specific tuning options, or bugs that need to be >addressed. > >Thanks! >--Nick Pavlica >--Laramie, WY >_______________________________________________ >freebsd-performance@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-performance >To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" > > From owner-freebsd-performance@FreeBSD.ORG Mon Jan 24 22:16:23 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EF06216A4CE; Mon, 24 Jan 2005 22:16:23 +0000 (GMT) Received: from a.mx.ict1.everquick.net (67-67-61-21.ded.swbell.net [67.67.61.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5F6D943D31; Mon, 24 Jan 2005 22:16:23 +0000 (GMT) (envelope-from eddy+public+spam@noc.everquick.net) Received: from a.mx.ict1.everquick.net (localhost [127.0.0.1]) j0OMGUDM014666; Mon, 24 Jan 2005 22:16:30 GMT X-Everquick-No-Abuse-1: Report any email abuse to or X-Everquick-No-Abuse-2: call +1 (785) 865-5885. Please be sure to reference X-Everquick-No-Abuse-3: the Message-Id and include GMT timestamps. Received: from localhost (eddy@localhost)j0OMGTHY014651; Mon, 24 Jan 2005 22:16:29 GMT X-Authentication-Warning: a.mx.ict1.everquick.net: eddy owned process doing -bs Date: Mon, 24 Jan 2005 22:16:29 +0000 (GMT) From: "Edward B. Dreger" X-X-Sender: eddy@a.mx.ict1.everquick.net To: Petri Helenius In-Reply-To: <41F571F4.1090504@he.iki.fi> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Nick Pavlica cc: freebsd-performance@freebsd.org cc: freebsd-questions@freebsd.org Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2005 22:16:24 -0000 PH> Date: Tue, 25 Jan 2005 00:08:52 +0200 PH> From: Petri Helenius PH> To: Nick Pavlica PH> Are you sure you aren't comparing filesystems with different mount PH> options? Async comes to mind first. He _did_ say "as many default options as possible"... does Linux still mount async by default? Eddy -- Everquick Internet - http://www.everquick.net/ A division of Brotsman & Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita ________________________________________________________________________ DO NOT send mail to the following addresses: davidc@brics.com -*- jfconmaapaq@intc.net -*- sam@everquick.net Sending mail to spambait addresses is a great way to get blocked. From owner-freebsd-performance@FreeBSD.ORG Mon Jan 24 23:11:29 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 169E116A4CF for ; Mon, 24 Jan 2005 23:11:29 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5E79D43D4C for ; Mon, 24 Jan 2005 23:11:28 +0000 (GMT) (envelope-from linicks@gmail.com) Received: by rproxy.gmail.com with SMTP id 40so156016rnz for ; Mon, 24 Jan 2005 15:11:27 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=bVmt2iCYmA9d/Zz3fPNz3fma3AV6PXfukUHaWD6tf2r1H0/e1LkdKMawD11cKNYuFrpSsnXjIBRQaE9aXbe7maOeyuyVNPUzkF0w4d//j3A1fuPn/5shAN7yPu8ILH14AhdGl102hk/zrgX4/DCqlN83y5QeI4eFWPBYIcZXvB8= Received: by 10.38.8.52 with SMTP id 52mr540341rnh; Mon, 24 Jan 2005 15:11:27 -0800 (PST) Received: by 10.38.8.43 with HTTP; Mon, 24 Jan 2005 15:11:27 -0800 (PST) Message-ID: Date: Mon, 24 Jan 2005 16:11:27 -0700 From: Nick Pavlica To: Petri Helenius In-Reply-To: <41F571F4.1090504@he.iki.fi> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <41F571F4.1090504@he.iki.fi> cc: freebsd-performance@freebsd.org cc: freebsd-questions@freebsd.org Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Nick Pavlica List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2005 23:11:29 -0000 I didn't change any of the default mount options on either OS. ################################################################ FreeBSD: ################################################################ # cat /etc/fstab # Device Mountpoint FStype Options Dump Pass# /dev/ad0s1b none swap sw 0 0 /dev/ad0s1a / ufs rw 1 1 /dev/ad0s1e /tmp ufs rw 2 2 /dev/ad0s1f /usr ufs rw 2 2 /dev/ad0s1d /var ufs rw 2 2 /dev/acd0 /cdrom cd9660 ro,noauto 0 0 ---------------------------------------------------------------------------------------------------------------- # mount /dev/ad0s1a on / (ufs, local) devfs on /dev (devfs, local) /dev/ad0s1e on /tmp (ufs, local, soft-updates) /dev/ad0s1f on /usr (ufs, local, soft-updates) /dev/ad0s1d on /var (ufs, local, soft-updates) ################################################################ Linux: ################################################################ # cat /etc/fstab # This file is edited by fstab-sync - see 'man fstab-sync' for details LABEL=/1 / xfs defaults 1 1 LABEL=/boot1 /boot xfs defaults 1 2 none /dev/pts devpts gid=5,mode=620 0 0 none /dev/shm tmpfs defaults 0 0 none /proc proc defaults 0 0 none /sys sysfs defaults 0 0 LABEL=SWAP-sda2 swap swap defaults 0 0 /dev/scd0 /media/cdrom auto pamconsole,exec,noauto,managed 0 0 /dev/fd0 /media/floppy auto pamconsole,exec,noauto,managed 0 0 ---------------------------------------------------------------------------------------------------------------- # mount /dev/sda3 on / type xfs (rw) none on /proc type proc (rw) none on /sys type sysfs (rw) none on /dev/pts type devpts (rw,gid=5,mode=620) usbfs on /proc/bus/usb type usbfs (rw) /dev/sda1 on /boot type xfs (rw) none on /dev/shm type tmpfs (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) --------------------------------------------------------------------------------------------------------------- --Nick On Tue, 25 Jan 2005 00:08:52 +0200, Petri Helenius wrote: > > Are you sure you aren't comparing filesystems with different mount > options? Async comes to mind first. > > Pete > > > Nick Pavlica wrote: > > >All, > > I would like to start addressing some of the feedback that I have > >been given. I started this discussion because I felt that it was > >important to share the information I discovered in my testing. I also > >want to reiterate my earlier statement that this is not an X vs. X > >discussion, but an attempt to better understand the results, and > >hopefully look at ways of improving the results I had with FreeBSD > >5.x. I'm also looking forward to seeing the improvements to the 5.x > >branch as it matures. I want to make it very clear that this is NOT A > >"Religious/Engineering War", please don't try to turn it into one. > > > >That said, lets move on to something more productive. I installed > >both operating systems using as many default options as possible and > >updated them with all of the latest patches. I was logged in via SSH > >from my workstation while running the tests. I didn't have X, running > >on any of the installations because it wasn't need. CPU and RAM > >utilization wasn't an issue during any of the tests, but the disk I/O > >performance was dramatically different. Please keep in mind that I > >ran these tests over and over to see if I had consistent results. I > >even did the same tests on other pieces of equipment not listed in my > >notes that yielded the same results time and time again. Some have > >confirmed that they have had similar results in there testing using > >other testing tools and methods. This makes me wounder why the gap is > >so large, and how it can be improved? > > > >I think that it would be beneficial to have others in this group do > >similar testing and post there results. This may help those that are > >working on the OS itself to find trouble areas, and ways to improve > >them. It may also help clarify many of the response questions because > >you will be able to completely control the testing environment. I > >look forward to seeing the testing results, and any good feedback that > >helps identify specific tuning options, or bugs that need to be > >addressed. > > > >Thanks! > >--Nick Pavlica > >--Laramie, WY > >_______________________________________________ > >freebsd-performance@freebsd.org mailing list > >http://lists.freebsd.org/mailman/listinfo/freebsd-performance > >To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" > > > > > > From owner-freebsd-performance@FreeBSD.ORG Tue Jan 25 10:10:17 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AAA0016A4CF for ; Tue, 25 Jan 2005 10:10:17 +0000 (GMT) Received: from mongers.org (miracle.mongers.org [193.162.142.71]) by mx1.FreeBSD.org (Postfix) with SMTP id 7D01543D4C for ; Tue, 25 Jan 2005 10:10:16 +0000 (GMT) (envelope-from jlouis@mongers.org) Received: (qmail 18127 invoked by uid 1030); 25 Jan 2005 10:10:14 -0000 From: "Jesper Louis Andersen" Date: Tue, 25 Jan 2005 11:10:14 +0100 To: Nick Pavlica Message-ID: <20050125101014.GA15439@miracle.mongers.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2i cc: freebsd-performance@freebsd.org cc: freebsd-questions@freebsd.org Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2005 10:10:17 -0000 Quoting Nick Pavlica (linicks@gmail.com): > I would like to start addressing some of the feedback that I have > been given. I started this discussion because I felt that it was > important to share the information I discovered in my testing. I also > want to reiterate my earlier statement that this is not an X vs. X > discussion, but an attempt to better understand the results, and > hopefully look at ways of improving the results I had with FreeBSD > 5.x. I'm also looking forward to seeing the improvements to the 5.x > branch as it matures. I want to make it very clear that this is NOT A > "Religious/Engineering War", please don't try to turn it into one. Well, I apologize if I came about that way. The fact seems to be that linux outperforms freebsd in your tests. The question, obviously, is why? To be able to answer, we need to find the places where the 2 systems are different. I suggest creating a webpage, possibly as pure .txt, where all findings are posted. It makes it easier to process with graphical plotting tools and it lowers the bandwidth we all need to transfer. If I were you, I would drop the measurements of raw performance for a bit as we wouldn't gain anything from that. Instead, I would begin to probe the system while the tests are executing. For instance, what does ``vmstat 1'', ``iostat 1'' and (if applicable ``gstat'') report when the test is running in the respective operating systems? What about open filedescriptors (is the limit reached). Does ``systat -vmstat'' show anything odd on FreeBSD while running the tests, etc? I am sure people can fill in more interesting probes to try. Using the probes might alter the outcome of the test, but as we are not testing for performance, this doesn't matter. There is a fair chance that something odd show up. On the other hand, if nothing shows up, we have ruled a lot of possible stuff out. -- jlouis From owner-freebsd-performance@FreeBSD.ORG Tue Jan 25 20:19:47 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4603916A4D6 for ; Tue, 25 Jan 2005 20:19:47 +0000 (GMT) Received: from web41215.mail.yahoo.com (web41215.mail.yahoo.com [66.218.93.48]) by mx1.FreeBSD.org (Postfix) with SMTP id 0ACA843D45 for ; Tue, 25 Jan 2005 20:19:47 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 39333 invoked by uid 60001); 25 Jan 2005 20:19:46 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=o0i1VerJgdis5DlosMC0w6VxjgFcWHZ0y/w1waZZ3kwIUKRuRnofsLUD05Hd7V1YXYxItnw9QkF/wl+dPo0UcaIqkCICAs4I7BBmHznhvyO7NkxL2YdxZScDjBqUabRiCneToKcR3VzxYY8AmaSq24+wyWIjSw/QLgxWTMG/CDw= ; Message-ID: <20050125201946.39331.qmail@web41215.mail.yahoo.com> Received: from [83.129.210.78] by web41215.mail.yahoo.com via HTTP; Tue, 25 Jan 2005 12:19:46 PST Date: Tue, 25 Jan 2005 12:19:46 -0800 (PST) From: Arne "Wörner" To: freebsd-performance@freebsd.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii cc: Nick Pavlica Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2005 20:19:47 -0000 I did some tests with 1. Knoppix V3.7 2004-12-08 (kernel version 2.4.27) and 2. FreeBSD R5.3. I used the same harddisc slice for the tests (just the *ad1.sh scripts uses another slice, because I was curious; I do not know the results). hw.ata.wc was enabled. My findings are: 1. Linux is much faster with "async" mount (somebody in this thread already mentioned that yesterday). 2. FreeBSD with soft updates is much faster than Linux with sync mount. See http://home.tiscali.de/cmdr_faako/ for further details (until 2005-01-27 00:00 Zulu Time (GMT+00)). Further research is necessary, if I wanted to find out, what async/sync mount means (especially in the case of enabled write cache?), or why async mount is much faster... -Arne __________________________________ Do you Yahoo!? Take Yahoo! Mail with you! Get it on your mobile phone. http://mobile.yahoo.com/maildemo From owner-freebsd-performance@FreeBSD.ORG Tue Jan 25 16:55:22 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A2CB416A4CE; Tue, 25 Jan 2005 16:55:22 +0000 (GMT) Received: from luzifer.incubus.de (incubus.de [80.237.207.83]) by mx1.FreeBSD.org (Postfix) with ESMTP id 35EFB43D2D; Tue, 25 Jan 2005 16:55:22 +0000 (GMT) (envelope-from mkb@incubus.de) Received: from [192.168.2.10] (pD9542FCF.dip.t-dialin.net [217.84.47.207]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by luzifer.incubus.de (Postfix) with ESMTP id 59D8A30B4F; Tue, 25 Jan 2005 17:56:41 +0100 (CET) Message-ID: <41F679FF.5090809@incubus.de> Date: Tue, 25 Jan 2005 17:55:27 +0100 From: Matthias Buelow User-Agent: Mozilla Thunderbird 0.9 (X11/20041124) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Petri Helenius References: <41F571F4.1090504@he.iki.fi> In-Reply-To: <41F571F4.1090504@he.iki.fi> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Tue, 25 Jan 2005 23:22:47 +0000 cc: freebsd-performance@freebsd.org cc: freebsd-questions@freebsd.org Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2005 16:55:22 -0000 Petri Helenius wrote: > Are you sure you aren't comparing filesystems with different mount > options? Async comes to mind first. a) ext3 and xfs are logging filesystems, so the problem with asynchronous metadata updates possibly corrupting the filesystem on a crash doesn't arise. b) asynchronous metadata updates wouldn't have any performance benefit on a dd if=/dev/zero of=tstfile. c) please cut down your quotes, and write your answers below or between the quoted text, instead of the outlook text-above-fullquote style. thanks. mkb. From owner-freebsd-performance@FreeBSD.ORG Tue Jan 25 23:39:47 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8391E16A4CE; Tue, 25 Jan 2005 23:39:47 +0000 (GMT) Received: from silver.he.iki.fi (helenius.fi [193.64.42.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2369943D49; Tue, 25 Jan 2005 23:39:46 +0000 (GMT) (envelope-from pete@he.iki.fi) Received: from [193.64.42.134] (h86.vuokselantie10.fi [193.64.42.134]) by silver.he.iki.fi (8.13.1/8.11.4) with ESMTP id j0PNdhHc070434; Wed, 26 Jan 2005 01:39:44 +0200 (EET) (envelope-from pete@he.iki.fi) Message-ID: <41F6D8C1.2040907@he.iki.fi> Date: Wed, 26 Jan 2005 01:39:45 +0200 From: Petri Helenius User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Matthias Buelow References: <41F571F4.1090504@he.iki.fi> <41F679FF.5090809@incubus.de> In-Reply-To: <41F679FF.5090809@incubus.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-performance@freebsd.org cc: freebsd-questions@freebsd.org Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2005 23:39:47 -0000 Matthias Buelow wrote: > Petri Helenius wrote: > >> Are you sure you aren't comparing filesystems with different mount >> options? Async comes to mind first. > > > a) ext3 and xfs are logging filesystems, so the problem with > asynchronous metadata updates possibly corrupting the filesystem on a > crash doesn't arise. No, they have a different, though unrelated issues. I didn't notice which filesystem and which options were used for the benchmarks, that's why I was asking about it. > b) asynchronous metadata updates wouldn't have any performance benefit > on a dd if=/dev/zero of=tstfile. I was not aware that the tests were this simple. > c) please cut down your quotes, and write your answers below or > between the quoted text, instead of the outlook text-above-fullquote > style. thanks. I usually do, however in this case it was intentional. Pete From owner-freebsd-performance@FreeBSD.ORG Wed Jan 26 18:47:04 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 09E9916A4CE for ; Wed, 26 Jan 2005 18:47:04 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 456DF43D39 for ; Wed, 26 Jan 2005 18:47:03 +0000 (GMT) (envelope-from linicks@gmail.com) Received: by rproxy.gmail.com with SMTP id 40so158827rnz for ; Wed, 26 Jan 2005 10:47:02 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=TM2iA9oCgZRXXbjRW9GUzE0fAP3cHmpsHXNyTqwSey+0khpr1RDA+V00fWVlM+WwVS1lOaB0gxXT7oLOxbGygPahIWY4BoJzWUhWj0omFP75W+C4ZPcGh7ccejeO560Ij2UHMVCU9C778VnCwCVhiIbi/qz+qwtVSndw63wNQXA= Received: by 10.38.206.29 with SMTP id d29mr356710rng; Wed, 26 Jan 2005 10:47:02 -0800 (PST) Received: by 10.38.8.43 with HTTP; Wed, 26 Jan 2005 10:47:02 -0800 (PST) Message-ID: Date: Wed, 26 Jan 2005 11:47:02 -0700 From: Nick Pavlica To: freebsd-questions@freebsd.org, freebsd-performance@freebsd.org In-Reply-To: Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Nick Pavlica List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2005 18:47:05 -0000 All, With the recent release of 4.11 I thought that I would give it a spin and com pair my results with my previous testing. I was blown away by the performance difference between 4.11 and 5.3. Iostat showed a difference of over 30Mb/s difference between the two. In fact, it kept up or out performed fedora Core 3 with XFS in my testing. This seems to indicate that the 5.x branch may still needs allot of performance work. One of the interesting observations was that 4.11 utilized much more of the processor than 5.3. I hope that the changes in 5.4 will help close this gap considerably. Is there any specific components of the 5.3 that have been identified to cause this performance difference? Your feedback/thoughts on this are appreciated! --Nick On Mon, 24 Jan 2005 14:59:55 -0700, Nick Pavlica wrote: > All, > I would like to start addressing some of the feedback that I have > been given. I started this discussion because I felt that it was > important to share the information I discovered in my testing. I also > want to reiterate my earlier statement that this is not an X vs. X > discussion, but an attempt to better understand the results, and > hopefully look at ways of improving the results I had with FreeBSD > 5.x. I'm also looking forward to seeing the improvements to the 5.x > branch as it matures. I want to make it very clear that this is NOT A > "Religious/Engineering War", please don't try to turn it into one. > > That said, lets move on to something more productive. I installed > both operating systems using as many default options as possible and > updated them with all of the latest patches. I was logged in via SSH > from my workstation while running the tests. I didn't have X, running > on any of the installations because it wasn't need. CPU and RAM > utilization wasn't an issue during any of the tests, but the disk I/O > performance was dramatically different. Please keep in mind that I > ran these tests over and over to see if I had consistent results. I > even did the same tests on other pieces of equipment not listed in my > notes that yielded the same results time and time again. Some have > confirmed that they have had similar results in there testing using > other testing tools and methods. This makes me wounder why the gap is > so large, and how it can be improved? > > I think that it would be beneficial to have others in this group do > similar testing and post there results. This may help those that are > working on the OS itself to find trouble areas, and ways to improve > them. It may also help clarify many of the response questions because > you will be able to completely control the testing environment. I > look forward to seeing the testing results, and any good feedback that > helps identify specific tuning options, or bugs that need to be > addressed. > > Thanks! > --Nick Pavlica > --Laramie, WY > From owner-freebsd-performance@FreeBSD.ORG Wed Jan 26 20:26:32 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C279516A4CF; Wed, 26 Jan 2005 20:26:32 +0000 (GMT) Received: from avscan1.sentex.ca (avscan1.sentex.ca [199.212.134.11]) by mx1.FreeBSD.org (Postfix) with ESMTP id CD5E743D45; Wed, 26 Jan 2005 20:26:31 +0000 (GMT) (envelope-from mike@sentex.net) Received: from localhost (localhost.sentex.ca [127.0.0.1]) by avscan1.sentex.ca (8.12.11/8.12.11) with ESMTP id j0QKQUZ6068329; Wed, 26 Jan 2005 15:26:30 -0500 (EST) (envelope-from mike@sentex.net) Received: from avscan1.sentex.ca ([127.0.0.1]) by localhost (avscan1.sentex.ca [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 67656-09; Wed, 26 Jan 2005 15:26:30 -0500 (EST) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by avscan1.sentex.ca (8.12.11/8.12.11) with ESMTP id j0QKQU1B068292; Wed, 26 Jan 2005 15:26:30 -0500 (EST) (envelope-from mike@sentex.net) Received: from simian.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.12.11/8.12.11) with ESMTP id j0QKQNfc068506; Wed, 26 Jan 2005 15:26:23 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <6.2.0.14.0.20050126150959.0466f4b8@64.7.153.2> X-Mailer: QUALCOMM Windows Eudora Version 6.2.0.14 Date: Wed, 26 Jan 2005 15:27:30 -0500 To: Nick Pavlica , freebsd-questions@freebsd.org, freebsd-performance@freebsd.org From: Mike Tancsa In-Reply-To: References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: by amavisd-new X-Virus-Scanned: by amavisd-new at avscan1b Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2005 20:26:33 -0000 At 01:47 PM 26/01/2005, Nick Pavlica wrote: >All, > With the recent release of 4.11 I thought that I would give it a Yes, I found the same thing basically. My test box is a P4 3Ghz with 2G of RAM on a 3ware 8605 controller with 4 drives in RAID5. Virtually every test I did with iozone* showed a difference anywhere from 10-40% in favor of RELENG_4. Note, this is a 2G RAM machine hence the odd result for the 1.5G test -------Sequential Output-------- ---Sequential Input-- --Random-- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--- Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU 4 1500 37673 23.7 37848 6.6 40784 7.7 97064 99.8 1174906 99.4 89867.4 99.6 4 3000 38492 24.6 38753 7.0 18396 4.1 80355 86.0 92051 9.9 605.1 1.0 5 1500 31226 23.0 34529 7.9 36444 8.9 110295 99.8 983156 92.5 27388.8 99.6 5 3000 33820 26.1 34309 8.3 13339 3.7 59807 56.8 68059 9.8 330.8 0.9 And a local postmark test. RELENG_4 and RELENG_5 pm>set size 300 100000 pm>set location /card0-a pm>set transactions 400000 pm>run Creating files...Done Performing transactions..........Done Deleting files...Done Time: 1219 seconds total 1219 seconds of transactions (328 per second) Files: 200107 created (164 per second) Creation alone: 500 files (500 per second) Mixed with transactions: 199607 files (163 per second) 199905 read (163 per second) 199384 appended (163 per second) 200107 deleted (164 per second) Deletion alone: 889 files (889 per second) Mixed with transactions: 199218 files (163 per second) Data: 12715.55 megabytes read (10.43 megabytes per second) 12728.92 megabytes written (10.44 megabytes per second) pm> pm>set size 300 100000 pm>set location /card0-a pm>set transactions 400000 pm>run Creating files...Done Performing transactions..........Done Deleting files...Done Time: 2824 seconds total 2822 seconds of transactions (141 per second) Files: 200107 created (70 per second) Creation alone: 500 files (500 per second) Mixed with transactions: 199607 files (70 per second) 199905 read (70 per second) 199384 appended (70 per second) 200107 deleted (70 per second) Deletion alone: 889 files (889 per second) Mixed with transactions: 199218 files (70 per second) Data: 12715.55 megabytes read (4.50 megabytes per second) 12728.92 megabytes written (4.51 megabytes per second) pm> *I have the iozone results in 2 .xls files if anyone wants to see them at http://www.tancsa.com/iozone-r5vsr4.zip From owner-freebsd-performance@FreeBSD.ORG Thu Jan 27 00:36:04 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B5F6B16A4CE; Thu, 27 Jan 2005 00:36:04 +0000 (GMT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 49A1643D45; Thu, 27 Jan 2005 00:36:04 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.13.1/8.13.1) with ESMTP id j0R0ZatD041258; Wed, 26 Jan 2005 19:35:36 -0500 (EST) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)j0R0ZaAq041255; Thu, 27 Jan 2005 00:35:36 GMT (envelope-from robert@fledge.watson.org) Date: Thu, 27 Jan 2005 00:35:35 +0000 (GMT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Mike Tancsa In-Reply-To: <6.2.0.14.0.20050126150959.0466f4b8@64.7.153.2> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Nick Pavlica cc: freebsd-performance@freebsd.org cc: freebsd-questions@freebsd.org Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2005 00:36:04 -0000 On Wed, 26 Jan 2005, Mike Tancsa wrote: > At 01:47 PM 26/01/2005, Nick Pavlica wrote: > >All, > > With the recent release of 4.11 I thought that I would give it a > > Yes, I found the same thing basically. My test box is a P4 3Ghz with 2G > of RAM on a 3ware 8605 controller with 4 drives in RAID5. Virtually > every test I did with iozone* showed a difference anywhere from 10-40% > in favor of RELENG_4. > > Note, this is a 2G RAM machine hence the odd result for the 1.5G test While it's not for the feint of heart, it might be interesting to see how results compare in 6-CURRENT + debugging of various sorts (including malloc) turned off, and debug.mpsafevfs turned on. One possible issue with the twe/twa drivers is that they are currently MPSAFE, so may see substantial contention (and hence additional latency). The move to an MPSAFE VFS will help with that a lot, I should think. Also, while on face value this may seem odd, could you try the following additional variables: - Layer the test UFS partition directly over ad0 instead of ad0s1a - UFS1 vs UFS2 Also please make sure that background fsck is not running during the tests, and that no snapshots are currently defined on the test file system. Finally, in as much as is possible, make sure that the layout of the disks is approximately the same -- as countless benchmarking papers show, there are substantial differences (10%+) in I/O throughput depending on where on the disk surface operations occur. That's one of the reasons to try UFS1 for the test partition, although not the only one. Robert N M Watson > > -------Sequential Output-------- ---Sequential Input-- > --Random-- > -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- > --Seeks--- > Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec > %CPU > 4 1500 37673 23.7 37848 6.6 40784 7.7 97064 99.8 1174906 99.4 > 89867.4 99.6 > 4 3000 38492 24.6 38753 7.0 18396 4.1 80355 > 86.0 92051 9.9 605.1 1.0 > 5 1500 31226 23.0 34529 7.9 36444 8.9 110295 99.8 983156 92.5 > 27388.8 99.6 > 5 3000 33820 26.1 34309 8.3 13339 3.7 59807 > 56.8 68059 9.8 330.8 0.9 > > > And a local postmark test. RELENG_4 and RELENG_5 > > pm>set size 300 100000 > pm>set location /card0-a > pm>set transactions 400000 > pm>run > Creating files...Done > Performing transactions..........Done > Deleting files...Done > Time: > 1219 seconds total > 1219 seconds of transactions (328 per second) > > Files: > 200107 created (164 per second) > Creation alone: 500 files (500 per second) > Mixed with transactions: 199607 files (163 per second) > 199905 read (163 per second) > 199384 appended (163 per second) > 200107 deleted (164 per second) > Deletion alone: 889 files (889 per second) > Mixed with transactions: 199218 files (163 per second) > > Data: > 12715.55 megabytes read (10.43 megabytes per second) > 12728.92 megabytes written (10.44 megabytes per second) > pm> > > > pm>set size 300 100000 > pm>set location /card0-a > pm>set transactions 400000 > pm>run > Creating files...Done > Performing transactions..........Done > Deleting files...Done > Time: > 2824 seconds total > 2822 seconds of transactions (141 per second) > > Files: > 200107 created (70 per second) > Creation alone: 500 files (500 per second) > Mixed with transactions: 199607 files (70 per second) > 199905 read (70 per second) > 199384 appended (70 per second) > 200107 deleted (70 per second) > Deletion alone: 889 files (889 per second) > Mixed with transactions: 199218 files (70 per second) > > Data: > 12715.55 megabytes read (4.50 megabytes per second) > 12728.92 megabytes written (4.51 megabytes per second) > pm> > > > *I have the iozone results in 2 .xls files if anyone wants to see them at > > http://www.tancsa.com/iozone-r5vsr4.zip > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" > From owner-freebsd-performance@FreeBSD.ORG Thu Jan 27 00:40:37 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3CF7716A4CF; Thu, 27 Jan 2005 00:40:37 +0000 (GMT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 740AD43D49; Thu, 27 Jan 2005 00:40:36 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.13.1/8.13.1) with ESMTP id j0R0e8Y8041303; Wed, 26 Jan 2005 19:40:08 -0500 (EST) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)j0R0e8pY041300; Thu, 27 Jan 2005 00:40:08 GMT (envelope-from robert@fledge.watson.org) Date: Thu, 27 Jan 2005 00:40:08 +0000 (GMT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Mike Tancsa In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Nick Pavlica cc: freebsd-performance@freebsd.org cc: freebsd-questions@freebsd.org Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2005 00:40:37 -0000 On Thu, 27 Jan 2005, Robert Watson wrote: > While it's not for the feint of heart, it might be interesting to see > how results compare in 6-CURRENT + debugging of various sorts (including > malloc) turned off, and debug.mpsafevfs turned on. One possible issue > with the twe/twa drivers is that they are currently MPSAFE, so may see > substantial contention (and hence additional latency). The move to an > MPSAFE VFS will help with that a lot, I should think. And, if you're in the mood for hacking code, and promise not to use snapshots, try making vfs_subr.c:vn_start_write(), vfs_subr.c:vn_write_suspend_wait(), vfs_subr.c:vn_finished_write(), vfs_subr.c:vfs_write_suspend(), and vfs_subr.c:vfs_write_resume() into noop's. These calls are used to avoid some deadlock scenarios associated with snapshot generation, but they also introduce a small but non-trivial amount of overhead to a number of operations. Since you're set up to do some testing, knowing how much of that cost is from these operations should be quite interesting. Robert N M Watson From owner-freebsd-performance@FreeBSD.ORG Fri Jan 28 01:04:33 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8CD9B16A4CE for ; Fri, 28 Jan 2005 01:04:33 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.202]) by mx1.FreeBSD.org (Postfix) with ESMTP id AE24943D4C for ; Fri, 28 Jan 2005 01:04:31 +0000 (GMT) (envelope-from linicks@gmail.com) Received: by rproxy.gmail.com with SMTP id 40so279244rnz for ; Thu, 27 Jan 2005 17:04:31 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=rUTuN+HHQe/zShltRufJHCkAz3sjYr+Wg7+Sr8W/R+XmQcu7xc68qu8vNGj8GVGtCB0ez1KxfZxU+23qJpLKCSKwsec5FwdjbTPTOmDg8ideP25leuysc8g4A8G0ej7RBWBLMsjclMFZQIJK5CDORPygTFGThKIg5j1gQ4sa168= Received: by 10.38.206.57 with SMTP id d57mr56805rng; Thu, 27 Jan 2005 17:04:31 -0800 (PST) Received: by 10.38.8.43 with HTTP; Thu, 27 Jan 2005 17:04:31 -0800 (PST) Message-ID: Date: Thu, 27 Jan 2005 18:04:31 -0700 From: Nick Pavlica To: Robert Watson In-Reply-To: Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <6.2.0.14.0.20050126150959.0466f4b8@64.7.153.2> cc: freebsd-performance@freebsd.org cc: freebsd-questions@freebsd.org cc: Mike Tancsa Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Nick Pavlica List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2005 01:04:33 -0000 > The move to an MPSAFE VFS will help with that a lot, I should think. Do you know if this will find it's way to 5.x in the near future? > > Also, while on face value this may seem odd, could you try the following > additional variables: > > - Layer the test UFS partition directly over ad0 instead of ad0s1a > - UFS1 vs UFS2 I just tested with UFS1 and had almost the exact same results. > > Finally, in as much as is possible, make sure that the layout of the disks > is approximately the same -- as countless benchmarking papers show, there > are substantial differences (10%+) in I/O throughput depending on where on > the disk surface operations occur. That's one of the reasons to try UFS1 > for the test partition, although not the only one. My tests use the exact same disk layout, and hardware. However, I have had consistent results on all 4 boxes that I have tested on. At this point I'm making the assumption that the poor disk I/O performance on 5.3 isn't a file system issue, but is tied to a larger issue with the Kernel (I know never make assumptions ... :)). In all my testing, I have noticed that 5.3 doesn't appear to release cpu resources even if there isn't any other demand for resources. I would compare it to driveling a car with a governor on it. When I tested with 4.11, it allocated considerably more resources. I do hope that the 5.x issues are resolved soon so that I can deploy may production servers on it rather than starting on 4 and them making the big switch. I will probably test 6 for the fun of it. Thanks! --Nick Pavlica From owner-freebsd-performance@FreeBSD.ORG Fri Jan 28 01:15:25 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 851B516A4CE; Fri, 28 Jan 2005 01:15:25 +0000 (GMT) Received: from cyrus.watson.org (cyrus.watson.org [204.156.12.53]) by mx1.FreeBSD.org (Postfix) with ESMTP id 339FC43D3F; Fri, 28 Jan 2005 01:15:25 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by cyrus.watson.org (Postfix) with SMTP id 5DCF246B82; Thu, 27 Jan 2005 20:15:24 -0500 (EST) Date: Fri, 28 Jan 2005 01:14:54 +0000 (GMT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Nick Pavlica In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-performance@freebsd.org cc: freebsd-questions@freebsd.org cc: Mike Tancsa Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2005 01:15:25 -0000 On Thu, 27 Jan 2005, Nick Pavlica wrote: > > The move to an MPSAFE VFS will help with that a lot, I should think. > > Do you know if this will find it's way to 5.x in the near future? Hopefully not too quickly, it's fairly experimental. I know there's interest in getting it into 5.x however. Perhaps once it's settled for a few months and we've confirmed that in the "off" state it's quite un-harmful, it can be merged. > > Also, while on face value this may seem odd, could you try the following > > additional variables: > > > > - Layer the test UFS partition directly over ad0 instead of ad0s1a > > - UFS1 vs UFS2 > > I just tested with UFS1 and had almost the exact same results. OK, thanks. > > Finally, in as much as is possible, make sure that the layout of the disks > > is approximately the same -- as countless benchmarking papers show, there > > are substantial differences (10%+) in I/O throughput depending on where on > > the disk surface operations occur. That's one of the reasons to try UFS1 > > for the test partition, although not the only one. > > My tests use the exact same disk layout, and hardware. However, I have > had consistent results on all 4 boxes that I have tested on. > > At this point I'm making the assumption that the poor disk I/O > performance on 5.3 isn't a file system issue, but is tied to a larger > issue with the Kernel (I know never make assumptions ... :)). In all my > testing, I have noticed that 5.3 doesn't appear to release cpu resources > even if there isn't any other demand for resources. I would compare it > to driveling a car with a governor on it. When I tested with 4.11, it > allocated considerably more resources. I do hope that the 5.x issues > are resolved soon so that I can deploy may production servers on it > rather than starting on 4 and them making the big switch. I will > probably test 6 for the fun of it. Forgive me if this was in previous e-mails and I missed it, but -- how does I/O directly on /dev/[diskdevice] differ as compared to the file system I/O? In particular, it's interesting to compare both large block I/O (reads, writes at fairly large multiples of the sector size -- 512k is a good number) and small I/O size (512 bytes is good). This will help identify the source along two dimmensions: are we looking at a basic storage I/O problem that's present even without the file system, or can we conclude that some of the additional extra cost is in the file system code or the hand off to it. Also, with the large and small I/O size, we can perhaps draw some conclusions about to what extent the source is a per-transaction overhead. Finally -- I figure you've done this already, but it's worth asking -- can you confirm that your hardware is negotiating the same basic parameters under 5.x and 4.x? In particular, the ATA code has changed substantially, so if using ATA hardware you want to confirm that the same DMA mode is negotiated. Robert N M Watson From owner-freebsd-performance@FreeBSD.ORG Fri Jan 28 02:45:28 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A136E16A4CE; Fri, 28 Jan 2005 02:45:28 +0000 (GMT) Received: from avscan2.sentex.ca (avscan2.sentex.ca [199.212.134.19]) by mx1.FreeBSD.org (Postfix) with ESMTP id EA58143D41; Fri, 28 Jan 2005 02:45:27 +0000 (GMT) (envelope-from mike@sentex.net) Received: from localhost (localhost.sentex.ca [127.0.0.1]) by avscan2.sentex.ca (8.12.11/8.12.11) with ESMTP id j0S2jTl1041775; Thu, 27 Jan 2005 21:45:29 -0500 (EST) (envelope-from mike@sentex.net) Received: from avscan2.sentex.ca ([127.0.0.1]) by localhost (avscan2.sentex.ca [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 41635-02; Thu, 27 Jan 2005 21:45:28 -0500 (EST) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by avscan2.sentex.ca (8.12.11/8.12.11) with ESMTP id j0S2jSe6041767; Thu, 27 Jan 2005 21:45:28 -0500 (EST) (envelope-from mike@sentex.net) Received: from simian.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.12.11/8.12.11) with ESMTP id j0S2jKE5073641; Thu, 27 Jan 2005 21:45:20 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <6.2.0.14.0.20050127213817.02f19220@64.7.153.2> X-Mailer: QUALCOMM Windows Eudora Version 6.2.0.14 Date: Thu, 27 Jan 2005 21:47:04 -0500 To: Robert Watson From: Mike Tancsa In-Reply-To: References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: by amavisd-new X-Virus-Scanned: by amavisd-new at avscan2b cc: freebsd-performance@FreeBSD.org cc: freebsd-questions@FreeBSD.org Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2005 02:45:28 -0000 At 08:14 PM 27/01/2005, Robert Watson wrote: > > > > My tests use the exact same disk layout, and hardware. However, I have > > had consistent results on all 4 boxes that I have tested on. I am redoing mine so that I boot from a different drive and just test on one large RAID5 partition so that the layout is as consistent as possible >I/O (reads, writes at fairly large multiples of the sector size -- 512k is >a good number) and small I/O size (512 bytes is good). This will help >identify the source along two dimmensions: are we looking at a basic >storage I/O problem that's present even without the file system, or can we >conclude that some of the additional extra cost is in the file system code >or the hand off to it. Also, with the large and small I/O size, we can >perhaps draw some conclusions about to what extent the source is a >per-transaction overhead. Apart from postmark and iozone (directly to disk and over nfs), are there any particular tests you would like to see done ? Also, anyone know of a decent benchmark to run on windows ? I want to test samba's performance on the 2 platforms as seen from a couple of Windows clients. ---Mike From owner-freebsd-performance@FreeBSD.ORG Fri Jan 28 09:53:12 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6C74A16A4CE; Fri, 28 Jan 2005 09:53:12 +0000 (GMT) Received: from cyrus.watson.org (cyrus.watson.org [204.156.12.53]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3C5FE43D53; Fri, 28 Jan 2005 09:53:12 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by cyrus.watson.org (Postfix) with SMTP id 4D04C46BAD; Fri, 28 Jan 2005 04:53:09 -0500 (EST) Date: Fri, 28 Jan 2005 09:52:38 +0000 (GMT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Mike Tancsa In-Reply-To: <6.2.0.14.0.20050127213817.02f19220@64.7.153.2> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-performance@FreeBSD.org cc: freebsd-questions@FreeBSD.org Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2005 09:53:12 -0000 On Thu, 27 Jan 2005, Mike Tancsa wrote: > >I/O (reads, writes at fairly large multiples of the sector size -- 512k is > >a good number) and small I/O size (512 bytes is good). This will help > >identify the source along two dimmensions: are we looking at a basic > >storage I/O problem that's present even without the file system, or can we > >conclude that some of the additional extra cost is in the file system code > >or the hand off to it. Also, with the large and small I/O size, we can > >perhaps draw some conclusions about to what extent the source is a > >per-transaction overhead. > > Apart from postmark and iozone (directly to disk and over nfs), are > there any particular tests you would like to see done ? Just to get started, using dd to read and write at various block sizes is probably a decent start. Take a few samples, make sure there's a decent sample size, etc, and don't count the first couple of runs. Robert N M Watson From owner-freebsd-performance@FreeBSD.ORG Fri Jan 28 17:19:00 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4E77716A4CE; Fri, 28 Jan 2005 17:19:00 +0000 (GMT) Received: from mail02.solnet.ch (mail02.solnet.ch [212.101.4.136]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5FDD543D54; Fri, 28 Jan 2005 17:18:59 +0000 (GMT) (envelope-from freebsdlists@bsdunix.ch) Received: from mail02.solnet.ch ([127.0.0.1]) by localhost (mail02.solnet.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 90950-01-48; Fri, 28 Jan 2005 17:18:46 +0000 (GMT) Received: from bert.mlan.solnet.ch (bert.mlan.solnet.ch [212.101.1.83]) by mail02.solnet.ch (Postfix) with ESMTP id 3C0725C315; Fri, 28 Jan 2005 17:18:46 +0000 (GMT) From: Thomas Vogt To: freebsd-performance@freebsd.org Content-Type: text/plain Date: Fri, 28 Jan 2005 18:18:19 +0100 Message-Id: <1106932700.48903.58.camel@bert.mlan.solnet.ch> Mime-Version: 1.0 X-Mailer: Evolution 2.0.3 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at mail02.solnet.ch cc: freebsd-net@freebsd.org Subject: freebsd router project. Problems with polling? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2005 17:19:00 -0000 Hello Overview: ATM, I've a 4.10 router (xeon 2.4ghz UP, dual on board em gigE interfaces) in a productive enviroment. The routing software is quagga. It's doing quite well. It has an average of 200k pakets per seconds with the full routing table. The System has polling enabled and interrupt is almost idle and cpu too. New one: Now I'm building a FreeBSD 5.3 or -Stable based router system on CF cards too. Intel whitepaper shows a max pps for the em cards at 1.4mpps. I doubt that I can reach more than ~600k pps with the fully routing table loaded! But this would be ok great. My system: Xeon 2.4Ghz UP, 2x onboard em GigE interface, 5.3-RELEASE-p4 with if_em.c,v 1.44.2.4. my configs: sysctl.conf: vm.swap_enabled=0 kern.ipc.somaxconn=1024 kern.polling.enable=1 kern.random.sys.harvest.ethernet=0 kern.random.sys.harvest.interrupt=0 net.inet.udp.recvspace=65536 net.inet.tcp.sendspace=65536 net.inet.tcp.delayed_ack=0 net.inet.tcp.msl=5000 net.inet.ip.maxfragpackets=40 net.inet.ip.maxfragsperpacket=4 net.inet.ip.fw.one_pass=0 net.inet.ip.fw.dyn_max=8192 net.inet.ip.fw.dyn_udp_lifetime=15 net.inet.ip.fastforwarding=1 loader.conf: kern.ipc.nmbclusters=32768 Kernel config: http://www.bsdunix.ch/public/ROUTER5 Setup: I've 3 similar machines. I use my own udpsend.c and udprec.c to send packages and to count the pakages. It can send up to 300-330k pps (udp size 64 bytes) ATM, the full routing table is not loaded. It's a very basic setup. The goal is to find the maximum pps throughput for the router with small pakets. But atm I've problems with device polling. Graphic: ------------------ |10.0.1.2 udp send | ------------------ | | -------em0------ |freebsd router | -------em1------ | | ----------------------- | 192.168.1.2 udp recv | ------------------------ netstat -w 1 (polling disabled) input (Total) output packets errs bytes packets errs bytes colls 300531 0 23441438 300585 0 23444792 0 300939 0 25872898 300895 0 25870184 0 300738 0 23457584 300768 0 23460626 0 300304 10 25826466 300304 0 25826876 0 ... Interrupt load is about 70% with net.inet.ip.fastforwarding enabled. If I disable this the system becomes unusable. The system hasn't reach the limit yet. But the interrupt is much to high. It's not worthy to add a second udp sender machine, at the moment. netstat -w 1 (polling enabled) input (Total) output packets errs bytes packets errs bytes colls 150151 47647 12910330 150150 0 12911806 0 150151 0 11711798 150152 0 11711876 0 150151 47665 12910986 150151 0 12911810 0 150151 0 11711798 150151 0 11711798 0 ... Interrupt load is about 10%. CPU is about 60% and with kern.polling.idle_poll enabled it goes to 100% (as expected). As you see the speed is droping down to 50% with polling enable and and I got a lot of errors. kern.polling.lost_polls: 188748 and kern.polling.suspect: 186919 are also very high. I don't know why polling is so bad on this machine. All SMP option are disabled in the kernel and bios. I tried to do as much as in http://lists.freebsd.org/pipermail/freebsd-questions/2004-November/064427.html described. I will prepare others tests with -STABLE and -CURRENT in the next few days. At the mean time, are they some other magic things config option I can try? Perhaps increase the HZ to 2000 in the kernel or remove polling and try smp machine? I doubt that I can run the machine without polling. If you see 70% interrupt load with 300k pps without polling. regards Thomas Vogt From owner-freebsd-performance@FreeBSD.ORG Fri Jan 28 20:01:27 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B091316A4CE; Fri, 28 Jan 2005 20:01:27 +0000 (GMT) Received: from avscan2.sentex.ca (avscan2.sentex.ca [199.212.134.19]) by mx1.FreeBSD.org (Postfix) with ESMTP id 48A0B43D1F; Fri, 28 Jan 2005 20:01:27 +0000 (GMT) (envelope-from mike@sentex.net) Received: from localhost (localhost.sentex.ca [127.0.0.1]) by avscan2.sentex.ca (8.12.11/8.12.11) with ESMTP id j0SK1RHl085828; Fri, 28 Jan 2005 15:01:27 -0500 (EST) (envelope-from mike@sentex.net) Received: from avscan2.sentex.ca ([127.0.0.1]) by localhost (avscan2.sentex.ca [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 85673-02; Fri, 28 Jan 2005 15:01:27 -0500 (EST) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by avscan2.sentex.ca (8.12.11/8.12.11) with ESMTP id j0SK1QvO085816; Fri, 28 Jan 2005 15:01:26 -0500 (EST) (envelope-from mike@sentex.net) Received: from simian.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.12.11/8.12.11) with ESMTP id j0SK1IIX076739; Fri, 28 Jan 2005 15:01:18 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <6.2.0.14.0.20050128145435.098e3d48@64.7.153.2> X-Mailer: QUALCOMM Windows Eudora Version 6.2.0.14 Date: Fri, 28 Jan 2005 15:02:23 -0500 To: Robert Watson From: Mike Tancsa In-Reply-To: References: <6.2.0.14.0.20050127213817.02f19220@64.7.153.2> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: by amavisd-new X-Virus-Scanned: by amavisd-new at avscan2b cc: freebsd-performance@freebsd.org Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2005 20:01:27 -0000 At 04:52 AM 28/01/2005, Robert Watson wrote: >Just to get started, using dd to read and write at various block sizes is >probably a decent start. Take a few samples, make sure there's a decent >sample size, etc, and don't count the first couple of runs. I started over and installed the OS on a separate drive, so that the RAID5 partition was one big partition. Bonnie seems fairly equal on the 2 platforms. I am just putting together all the IOZONE tests as well which I will post later tonight / over the weekend. -------Sequential Output-------- ---Sequential Input-- --Random-- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--- Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU F 2000 33789 25.8 34065 8.3 14038 3.8 47484 44.6 67316 9.5 2873.7 5.7 L 2000 28073 62.0 22391 4.3 12189 2.0 17577 35.5 102132 5.5 1853.7 2.5 F=RELENG_5 L=2.6.10 LINUX However, on the postmark tests, there seems to be quite a bit more difference. RELENG_5 #2 [nfs]# time dd if=/dev/zero of=/mnt/test bs=16k count=2000 32768000 bytes transferred in 0.724865 secs (45205661 bytes/sec) 0.000u 0.066s 0:00.73 8.2% 33+281k 0+250io 0pf+0w [nfs]# time dd if=/dev/zero of=/mnt/test bs=32k count=2000 65536000 bytes transferred in 1.731888 secs (37840788 bytes/sec) 0.000u 0.122s 0:01.74 6.8% 28+283k 0+500io 0pf+0w [nfs]# time dd if=/dev/zero of=/mnt/test bs=64k count=2000 131072000 bytes transferred in 3.131403 secs (41857273 bytes/sec) 0.000u 0.245s 0:03.14 7.6% 25+335k 0+1000io 0pf+0w [nfs]# time dd if=/dev/zero of=/mnt/test bs=128k count=2000 262144000 bytes transferred in 6.973014 secs (37594073 bytes/sec) 0.000u 0.509s 0:07.00 7.1% 26+510k 0+2000io 0pf+0w [nfs]# time dd if=/dev/zero of=/mnt/test bs=256k count=2000 524288000 bytes transferred in 14.253915 secs (36782035 bytes/sec) 0.000u 1.003s 0:14.30 6.9% 25+784k 0+4000io 0pf+0w [nfs]# time dd if=/dev/zero of=/mnt/test bs=512k count=2000 1048576000 bytes transferred in 28.135716 secs (37268502 bytes/sec) 0.008u 2.165s 0:28.21 7.6% 24+1386k 0+8000io 0pf+0w [nfs]# LINUX 2.6.10 pm>set size 300 100000 pm>set location /mnt pm>set transactions 400000 pm>run Creating files...Done Performing transactions..........Done Deleting files...Done Time: 1504 seconds total 1503 seconds of transactions (266 per second) Files: 200107 created (133 per second) Creation alone: 500 files (500 per second) Mixed with transactions: 199607 files (132 per second) 199905 read (133 per second) 199384 appended (132 per second) 200107 deleted (133 per second) Deletion alone: 889 files (889 per second) Mixed with transactions: 199218 files (132 per second) Data: 12715.55 megabytes read (8.45 megabytes per second) 12728.92 megabytes written (8.46 megabytes per second) pm> RELENG_5. PostMark v1.5 : 3/27/01 pm>set location /mnt pm>size 300 100000 Eh? pm>set size 300 100000 pm>set location /mnt pm>set transactions 400000 pm>run Creating files...Done Performing transactions..........Done Deleting files...Done Time: 2490 seconds total 2487 seconds of transactions (160 per second) Files: 200107 created (80 per second) Creation alone: 500 files (500 per second) Mixed with transactions: 199607 files (80 per second) 199905 read (80 per second) 199384 appended (80 per second) 200107 deleted (80 per second) Deletion alone: 889 files (444 per second) Mixed with transactions: 199218 files (80 per second) Data: 12715.55 megabytes read (5.11 megabytes per second) 12728.92 megabytes written (5.11 megabytes per second) pm> RELENG_4 (Note, this was on a different partition on the RAID5 card pm>set size 300 100000 pm>set location /card0-a pm>set transactions 400000 pm>run Creating files...Done Performing transactions..........Done Deleting files...Done Time: 1219 seconds total 1219 seconds of transactions (328 per second) Files: 200107 created (164 per second) Creation alone: 500 files (500 per second) Mixed with transactions: 199607 files (163 per second) 199905 read (163 per second) 199384 appended (163 per second) 200107 deleted (164 per second) Deletion alone: 889 files (889 per second) Mixed with transactions: 199218 files (163 per second) Data: 12715.55 megabytes read (10.43 megabytes per second) 12728.92 megabytes written (10.44 megabytes per second) pm> From owner-freebsd-performance@FreeBSD.ORG Sat Jan 29 00:50:25 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5D5E716A4CE; Sat, 29 Jan 2005 00:50:25 +0000 (GMT) Received: from cyrus.watson.org (cyrus.watson.org [204.156.12.53]) by mx1.FreeBSD.org (Postfix) with ESMTP id C80C543D4C; Sat, 29 Jan 2005 00:50:24 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by cyrus.watson.org (Postfix) with SMTP id E9FB046B9A; Fri, 28 Jan 2005 19:50:23 -0500 (EST) Date: Sat, 29 Jan 2005 00:49:51 +0000 (GMT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Mike Tancsa In-Reply-To: <6.2.0.14.0.20050128145435.098e3d48@64.7.153.2> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-performance@freebsd.org cc: jeff@FreeBSD.org Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2005 00:50:25 -0000 On Fri, 28 Jan 2005, Mike Tancsa wrote: > At 04:52 AM 28/01/2005, Robert Watson wrote: > > >Just to get started, using dd to read and write at various block sizes is > >probably a decent start. Take a few samples, make sure there's a decent > >sample size, etc, and don't count the first couple of runs. > > > I started over and installed the OS on a separate drive, so that the > RAID5 partition was one big partition. Bonnie seems fairly equal on the > 2 platforms. I am just putting together all the IOZONE tests as well > which I will post later tonight / over the weekend. The basic I/O results are encouraging. Could you remind me which storage controller you're using? I'm guessing some of what you're seeing is Giant content and a lack of preemptibility to let the disk controller interrupt the file system code -- this would add a lot of latency to turning around I/O's, lowering the throughput. Perhaps combined with issues in soft updates and/or UFS generally. I've CC'd Jeff Roberson since he's working in several of these areas. For postmark, could you try running the test against a partition with the async flag set, but no soft updates turned on? Also, could you compare with the sync flag on both Linux and FreeBSD (again, no soft updates)? Robert N M Watson > > > > -------Sequential Output-------- ---Sequential Input-- > --Random-- > -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- > --Seeks--- > Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU > F 2000 33789 25.8 34065 8.3 14038 3.8 47484 44.6 67316 9.5 > 2873.7 5.7 > L 2000 28073 62.0 22391 4.3 12189 2.0 17577 35.5 102132 5.5 > 1853.7 2.5 > > F=RELENG_5 > L=2.6.10 LINUX > > However, on the postmark tests, there seems to be quite a bit more difference. > > RELENG_5 #2 > [nfs]# time dd if=/dev/zero of=/mnt/test bs=16k count=2000 > 32768000 bytes transferred in 0.724865 secs (45205661 bytes/sec) > 0.000u 0.066s 0:00.73 8.2% 33+281k 0+250io 0pf+0w > [nfs]# time dd if=/dev/zero of=/mnt/test bs=32k count=2000 > 65536000 bytes transferred in 1.731888 secs (37840788 bytes/sec) > 0.000u 0.122s 0:01.74 6.8% 28+283k 0+500io 0pf+0w > [nfs]# time dd if=/dev/zero of=/mnt/test bs=64k count=2000 > 131072000 bytes transferred in 3.131403 secs (41857273 bytes/sec) > 0.000u 0.245s 0:03.14 7.6% 25+335k 0+1000io 0pf+0w > [nfs]# time dd if=/dev/zero of=/mnt/test bs=128k count=2000 > 262144000 bytes transferred in 6.973014 secs (37594073 bytes/sec) > 0.000u 0.509s 0:07.00 7.1% 26+510k 0+2000io 0pf+0w > [nfs]# time dd if=/dev/zero of=/mnt/test bs=256k count=2000 > 524288000 bytes transferred in 14.253915 secs (36782035 bytes/sec) > 0.000u 1.003s 0:14.30 6.9% 25+784k 0+4000io 0pf+0w > [nfs]# time dd if=/dev/zero of=/mnt/test bs=512k count=2000 > 1048576000 bytes transferred in 28.135716 secs (37268502 bytes/sec) > 0.008u 2.165s 0:28.21 7.6% 24+1386k 0+8000io 0pf+0w > [nfs]# > > > LINUX 2.6.10 > > pm>set size 300 100000 > pm>set location /mnt > pm>set transactions 400000 > pm>run > Creating files...Done > Performing transactions..........Done > Deleting files...Done > Time: > 1504 seconds total > 1503 seconds of transactions (266 per second) > > Files: > 200107 created (133 per second) > Creation alone: 500 files (500 per second) > Mixed with transactions: 199607 files (132 per second) > 199905 read (133 per second) > 199384 appended (132 per second) > 200107 deleted (133 per second) > Deletion alone: 889 files (889 per second) > Mixed with transactions: 199218 files (132 per second) > > Data: > 12715.55 megabytes read (8.45 megabytes per second) > 12728.92 megabytes written (8.46 megabytes per second) > pm> > > > RELENG_5. > PostMark v1.5 : 3/27/01 > pm>set location /mnt > pm>size 300 100000 > Eh? > pm>set size 300 100000 > pm>set location /mnt > pm>set transactions 400000 > pm>run > Creating files...Done > Performing transactions..........Done > Deleting files...Done > Time: > 2490 seconds total > 2487 seconds of transactions (160 per second) > > Files: > 200107 created (80 per second) > Creation alone: 500 files (500 per second) > Mixed with transactions: 199607 files (80 per second) > 199905 read (80 per second) > 199384 appended (80 per second) > 200107 deleted (80 per second) > Deletion alone: 889 files (444 per second) > Mixed with transactions: 199218 files (80 per second) > > Data: > 12715.55 megabytes read (5.11 megabytes per second) > 12728.92 megabytes written (5.11 megabytes per second) > pm> > > > > RELENG_4 (Note, this was on a different partition on the RAID5 card > > pm>set size 300 100000 > pm>set location /card0-a > pm>set transactions 400000 > pm>run > Creating files...Done > Performing transactions..........Done > Deleting files...Done > Time: > 1219 seconds total > 1219 seconds of transactions (328 per second) > > Files: > 200107 created (164 per second) > Creation alone: 500 files (500 per second) > Mixed with transactions: 199607 files (163 per second) > 199905 read (163 per second) > 199384 appended (163 per second) > 200107 deleted (164 per second) > Deletion alone: 889 files (889 per second) > Mixed with transactions: 199218 files (163 per second) > > Data: > 12715.55 megabytes read (10.43 megabytes per second) > 12728.92 megabytes written (10.44 megabytes per second) > pm> > > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" > From owner-freebsd-performance@FreeBSD.ORG Sat Jan 29 05:20:16 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6B85816A4CE; Sat, 29 Jan 2005 05:20:16 +0000 (GMT) Received: from avscan1.sentex.ca (avscan1.sentex.ca [199.212.134.11]) by mx1.FreeBSD.org (Postfix) with ESMTP id D5F3B43D46; Sat, 29 Jan 2005 05:20:15 +0000 (GMT) (envelope-from mike@sentex.net) Received: from localhost (localhost.sentex.ca [127.0.0.1]) by avscan1.sentex.ca (8.12.11/8.12.11) with ESMTP id j0T5KE97049011; Sat, 29 Jan 2005 00:20:14 -0500 (EST) (envelope-from mike@sentex.net) Received: from avscan1.sentex.ca ([127.0.0.1]) by localhost (avscan1.sentex.ca [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 48727-04; Sat, 29 Jan 2005 00:20:14 -0500 (EST) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by avscan1.sentex.ca (8.12.11/8.12.11) with ESMTP id j0T5KEMj049005; Sat, 29 Jan 2005 00:20:14 -0500 (EST) (envelope-from mike@sentex.net) Received: from simian.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.12.11/8.12.11) with ESMTP id j0T5K6Ji078297; Sat, 29 Jan 2005 00:20:06 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <6.2.0.14.0.20050128200200.0860ddf8@64.7.153.2> X-Mailer: QUALCOMM Windows Eudora Version 6.2.0.14 Date: Sat, 29 Jan 2005 00:22:14 -0500 To: Robert Watson From: Mike Tancsa In-Reply-To: References: <6.2.0.14.0.20050128145435.098e3d48@64.7.153.2> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: by amavisd-new X-Virus-Scanned: by amavisd-new at avscan1b cc: freebsd-performance@FreeBSD.org cc: jeff@FreeBSD.org Subject: Re: FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2005 05:20:16 -0000 At 07:49 PM 28/01/2005, Robert Watson wrote: >On Fri, 28 Jan 2005, Mike Tancsa wrote: > > > At 04:52 AM 28/01/2005, Robert Watson wrote: > > > > >Just to get started, using dd to read and write at various block sizes is > > >probably a decent start. Take a few samples, make sure there's a decent > > >sample size, etc, and don't count the first couple of runs. > > > > > > I started over and installed the OS on a separate drive, so that the > > RAID5 partition was one big partition. Bonnie seems fairly equal on the > > 2 platforms. I am just putting together all the IOZONE tests as well > > which I will post later tonight / over the weekend. > >The basic I/O results are encouraging. Could you remind me which storage >controller you're using? Hi, 3ware 8605 with 4 SATA drives in RAID5. >For postmark, could you try running the test against a partition with the >async flag set, but no soft updates turned on? OK [nfs]# newfs /dev/twed0s1d /dev/twed0s1d: 715418.0MB (1465176132 sectors) block size 16384, fragment size 2048 using 3894 cylinder groups of 183.77MB, 11761 blks, 23552 inodes. super-block backups (for fsck -b #) at: [nfs]# mount -o async /dev/twed0s1d /mnt [nfs]# mount /dev/ad0s1a on / (ufs, local) devfs on /dev (devfs, local) /dev/ad0s1e on /tmp (ufs, local, soft-updates) /dev/ad0s1d on /usr (ufs, local, soft-updates) /dev/ad0s1f on /var (ufs, local, soft-updates) /dev/twed0s1d on /mnt (ufs, asynchronous, NFS exported, local) [nfs]# postmark PostMark v1.5 : 3/27/01 pm>set location /mnt pm>set transactions 400000 pm>set size 300 100000 pm>run Creating files...Done Performing transactions..........Done Deleting files...Done Time: 2427 seconds total 2419 seconds of transactions (165 per second) Files: 200107 created (82 per second) Creation alone: 500 files (250 per second) Mixed with transactions: 199607 files (82 per second) 199905 read (82 per second) 199384 appended (82 per second) 200107 deleted (82 per second) Deletion alone: 889 files (148 per second) Mixed with transactions: 199218 files (82 per second) Data: 12715.55 megabytes read (5.24 megabytes per second) 12728.92 megabytes written (5.24 megabytes per second) pm> No difference it would seem. > Also, could you compare >with the sync flag on both Linux and FreeBSD (again, no soft updates)? Linux was with on ext3, so no such option. ---Mike From owner-freebsd-performance@FreeBSD.ORG Sat Jan 29 16:30:10 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4460A16A4CE; Sat, 29 Jan 2005 16:30:10 +0000 (GMT) Received: from relay.bestcom.ru (relay.bestcom.ru [217.72.144.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3FFBE43D1F; Sat, 29 Jan 2005 16:30:09 +0000 (GMT) (envelope-from glebius@freebsd.org) Received: from cell.sick.ru (root@cell.sick.ru [217.72.144.68]) by relay.bestcom.ru (8.13.1/8.12.9) with ESMTP id j0TGTnnL042539 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Sat, 29 Jan 2005 19:29:50 +0300 (MSK) (envelope-from glebius@freebsd.org) Received: from cell.sick.ru (glebius@localhost [127.0.0.1]) by cell.sick.ru (8.12.11/8.12.8) with ESMTP id j0TGTnaX000129 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 29 Jan 2005 19:29:49 +0300 (MSK) (envelope-from glebius@freebsd.org) Received: (from glebius@localhost) by cell.sick.ru (8.12.11/8.12.11/Submit) id j0TGTnTs000128; Sat, 29 Jan 2005 19:29:49 +0300 (MSK) (envelope-from glebius@freebsd.org) X-Authentication-Warning: cell.sick.ru: glebius set sender to glebius@freebsd.org using -f Date: Sat, 29 Jan 2005 19:29:48 +0300 From: Gleb Smirnoff To: Thomas Vogt Message-ID: <20050129162948.GA99968@cell.sick.ru> Mail-Followup-To: Gleb Smirnoff , Thomas Vogt , freebsd-performance@freebsd.org, freebsd-net@freebsd.org References: <1106932700.48903.58.camel@bert.mlan.solnet.ch> Mime-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <1106932700.48903.58.camel@bert.mlan.solnet.ch> User-Agent: Mutt/1.5.6i X-Virus-Scanned: ClamAV version devel-20050125, clamav-milter version 0.80ff on relay.bestcom.ru X-Virus-Status: Clean cc: freebsd-net@freebsd.org cc: freebsd-performance@freebsd.org Subject: Re: freebsd router project. Problems with polling? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2005 16:30:10 -0000 Thomas, can you try if_em driver from HEAD and check whether this help. There were some work done during 5.3-RELEASE. On Fri, Jan 28, 2005 at 06:18:19PM +0100, Thomas Vogt wrote: T> netstat -w 1 (polling disabled) T> input (Total) output T> packets errs bytes packets errs bytes colls T> 300531 0 23441438 300585 0 23444792 0 T> 300939 0 25872898 300895 0 25870184 0 T> 300738 0 23457584 300768 0 23460626 0 T> 300304 10 25826466 300304 0 25826876 0 T> ... T> T> Interrupt load is about 70% with net.inet.ip.fastforwarding enabled. If T> I disable this the system becomes unusable. T> The system hasn't reach the limit yet. But the interrupt is much to T> high. It's not worthy to add a second udp sender machine, at the moment. T> T> T> netstat -w 1 (polling enabled) T> input (Total) output T> packets errs bytes packets errs bytes colls T> 150151 47647 12910330 150150 0 12911806 0 T> 150151 0 11711798 150152 0 11711876 0 T> 150151 47665 12910986 150151 0 12911810 0 T> 150151 0 11711798 150151 0 11711798 0 T> ... T> T> Interrupt load is about 10%. CPU is about 60% and with T> kern.polling.idle_poll enabled it goes to 100% (as expected). T> T> As you see the speed is droping down to 50% with polling enable and and T> I got a lot of errors. T> kern.polling.lost_polls: 188748 and kern.polling.suspect: 186919 are T> also very high. I don't know why polling is so bad on this machine. All T> SMP option are disabled in the kernel and bios. T> I tried to do as much as in T> http://lists.freebsd.org/pipermail/freebsd-questions/2004-November/064427.html described. T> T> I will prepare others tests with -STABLE and -CURRENT in the next few T> days. At the mean time, are they some other magic things config option I T> can try? Perhaps increase the HZ to 2000 in the kernel or remove polling T> and try smp machine? I doubt that I can run the machine without polling. T> If you see 70% interrupt load with 300k pps without polling. T> T> regards T> Thomas Vogt T> T> _______________________________________________ T> freebsd-net@freebsd.org mailing list T> http://lists.freebsd.org/mailman/listinfo/freebsd-net T> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" -- Totus tuus, Glebius. GLEBIUS-RIPN GLEB-RIPE