From owner-freebsd-performance@FreeBSD.ORG Wed Dec 27 07:00:11 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 26FC116A403; Wed, 27 Dec 2006 07:00:11 +0000 (UTC) (envelope-from markir@paradise.net.nz) Received: from smtp5.clear.net.nz (smtp5.clear.net.nz [203.97.33.68]) by mx1.freebsd.org (Postfix) with ESMTP id BF65C13C46F; Wed, 27 Dec 2006 07:00:10 +0000 (UTC) (envelope-from markir@paradise.net.nz) Received: from [192.168.1.11] (121-72-68-249.dsl.telstraclear.net [121.72.68.249]) by smtp5.clear.net.nz (CLEAR Net Mail) with ESMTP id <0JAX00L0V7G8WV00@smtp5.clear.net.nz>; Wed, 27 Dec 2006 20:00:09 +1300 (NZDT) Date: Wed, 27 Dec 2006 20:00:07 +1300 From: Mark Kirkwood In-reply-to: <458B3E0C.6090104@freebsd.org> To: David Xu Message-id: <459219F7.7080104@paradise.net.nz> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7bit References: <458B3651.8090601@paradise.net.nz> <458B3E0C.6090104@freebsd.org> User-Agent: Thunderbird 1.5.0.9 (X11/20061227) Cc: freebsd-performance@freebsd.org, bde@zeta.org.au Subject: Re: Cached file read performance X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2006 07:00:11 -0000 David Xu wrote: > Mark Kirkwood wrote: . >> >> (snippage) I used the attached program to read a cached >> 781MB file sequentially and randomly with a specified block size (see >> below). The conclusion I came to was that our (i.e FreeBSD) cached >> read performance (particularly for smaller block sizes) could perhaps >> be improved... > > I suspect in such a test, memory copying speed will be a key factor, > I don't have number to back up my idea, but I think Linux has lots > of tweaks, such as using MMX instruction to copy data. > > Thought it would be good to test this too: I used the small program (see below) to test memcpy'ing 50000 8192 byte chunks on both systems: Gentoo (2.6.18): $ ./memcpytest 8192 50000 50000 malloc in 0.5367s 50000 memcpy of 8192 byte blocks in 1.0887s FreeBSD (6.2-PRE Nov 27): $ ./memcpytest 8192 50000 50000 malloc in 0.1469s 50000 memcpy of 8192 byte blocks in 1.3599s So we are a little slower (factor of 1.24) on the memcpy, this I guess contributes a *little* to us being slower with the cached reads, but is consistent with Bruce's findings of the cache design itself being the major factor! One nice point to notice is that our malloc is substantially faster (~3.6 times for this test). These results were readily repeatable with minimal variation. Cheers Mark P.s: I've included the program in-line below, as the attachment stripper for this list seems to be real aggressive :-) ---------------------------------------------------------------------- /* * memcpytest.c: Attempt to measure performance of memcpy. */ #include #include #include #include #include #include int main(int argc, char **argv) { int blocksz; /* The block size to copy. */ int numblocks; /* How many copies to do. */ char *buf; /* Input buffer. */ typedef struct _BlockArray { char *blockentry; } BlockArray; BlockArray *blockarray; /* Array of block entry ptrs. */ int i; struct timeval starttp, endtp, elapsedtp; double elapsed; if (argc != 3) { printf("usage %s blocksize num_blocks\n", argv[0]); exit(1); } else { blocksz = atoi(argv[1]); numblocks = atoi(argv[2]); } /* Start timing setup. */ gettimeofday(&starttp, NULL); /* Allocate source buffer and initialize to something trivial. */ buf = (char *) malloc(blocksz); if (buf == NULL) { printf("out of memory initializing source buffer!\n"); exit(2); } memset(buf, 1, blocksz); /* Allocate destination array of buffer pointers. */ blockarray = malloc(numblocks * sizeof(blockarray)); if (blockarray == NULL) { printf("out of memory initializing destination array!\n"); exit(2); } /* Allocate a block entry for each element of blockarray. */ for (i = 0; i < numblocks; i++) { blockarray[i].blockentry = malloc(blocksz); if (blockarray[i].blockentry == NULL) { printf("out of memory initializing destination array contents!\n"); exit(2); } } gettimeofday(&endtp, NULL); timersub(&endtp, &starttp, &elapsedtp); elapsed = (double)elapsedtp.tv_sec + (double)elapsedtp.tv_usec/1000000.0; printf("%d malloc in %.4fs\n", numblocks, elapsed); /* Start timing copy now. */ gettimeofday(&starttp, NULL); /* Perform the copy. */ for (i = 0; i < numblocks; i++) { memcpy((void *)blockarray[i].blockentry, (void *)buf, blocksz); } gettimeofday(&endtp, NULL); timersub(&endtp, &starttp, &elapsedtp); elapsed = (double)elapsedtp.tv_sec + (double)elapsedtp.tv_usec/1000000.0; printf("%d memcpy of %d byte blocks in %.4fs\n", numblocks, blocksz, elapsed); free(buf); for (i = 0; i < numblocks; i++) { free(blockarray[i].blockentry); } free(blockarray); exit(0); } From owner-freebsd-performance@FreeBSD.ORG Wed Dec 27 07:10:49 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6940C16A412 for ; Wed, 27 Dec 2006 07:10:49 +0000 (UTC) (envelope-from markir@paradise.net.nz) Received: from smtp4.clear.net.nz (smtp4.clear.net.nz [203.97.37.64]) by mx1.freebsd.org (Postfix) with ESMTP id 2F47D13C46F for ; Wed, 27 Dec 2006 07:10:49 +0000 (UTC) (envelope-from markir@paradise.net.nz) Received: from [192.168.1.11] (121-72-68-249.dsl.telstraclear.net [121.72.68.249]) by smtp4.clear.net.nz (CLEAR Net Mail) with ESMTP id <0JAX003JI7XZQK30@smtp4.clear.net.nz> for freebsd-performance@freebsd.org; Wed, 27 Dec 2006 20:10:47 +1300 (NZDT) Date: Wed, 27 Dec 2006 20:10:46 +1300 From: Mark Kirkwood In-reply-to: <458B3651.8090601@paradise.net.nz> To: Mark Kirkwood Message-id: <45921C76.8020200@paradise.net.nz> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7bit References: <458B3651.8090601@paradise.net.nz> User-Agent: Thunderbird 1.5.0.9 (X11/20061227) Cc: freebsd-performance@freebsd.org Subject: Re: Cached file read performance X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2006 07:10:49 -0000 Mark Kirkwood wrote: > I used the attached program to read a cached > 781MB file sequentially and randomly with a specified block size (see > below). In the interest of making it easy for anyone to re-test this later, I'll in-line the program source here (I did post a link to my web space, but that could get cleaned up by accident...) ------------------------------------------------------------------------------------------- /* * readtest.c: read a file sequentially or randomly */ #include #include #include #include #include #include #include #include #include #include void usage(char *prog) { printf("usage %s filename blocksize 1|0 (0 - random, 1 - sequential)\n", prog); return; } int main(int argc, char **argv) { int fd; char *file; struct stat *fileinfo; char *buf; int blocksz; int numblocks, i; off_t offset, offsetmin, offsetmax; double offsetsum, offsetsumx2; int seq; int stats = 0; struct timeval starttp, endtp, elapsedtp; double elapsed; double iorate; if (argc != 4 && argc != 5) { usage(argv[0]); exit(1); } else { if ((file = (char *) malloc(strlen(argv[1]))) == NULL) { printf("out of memory!\n"); exit(2); } strcpy(file, argv[1]); blocksz = atoi(argv[2]); seq = atoi(argv[3]); if (argc == 5) stats = 1; } /* Start timing. */ gettimeofday(&starttp, NULL); if ((fd = open(file, O_RDONLY)) == -1) { /* Can't open the file. */ perror("cannot open"); exit(1); } else { /* How many random sequential access calls are needed? */ fileinfo = (struct stat*) malloc(sizeof(struct stat)); fstat(fd,fileinfo); numblocks = (fileinfo->st_size)/blocksz; free(fileinfo); } /* Allocate buffer. */ buf = (char *) malloc(blocksz); if (buf == NULL) { printf("out of memory!\n"); exit(2); } /* If we are random, initialize. */ if (seq != 1) srandom(2006122111); /* * Read the file sequentially or randomly. * If random then calculate the offset to seek to using the formula: * * offet in blocks = random() % (numblocks - 1) * */ for (i = 0; i < numblocks; i++) { if (seq == 1) { offset = (off_t)i * blocksz; /* only used for stats */ if (read(fd, buf, blocksz) != blocksz) { perror("read failed"); exit(1); } } else { offset = (off_t) (random() % (numblocks - 1)) * blocksz; if (lseek(fd, offset, SEEK_SET) == -1) { perror("seek failed"); exit(1); } if (read(fd, buf, blocksz) != blocksz) { perror("read failed"); exit(1); } } /* If we are collecting stats...*/ if (stats){ if (i == 0) { offsetmin = offsetmax = offset; offsetsum = (double)offset; offsetsumx2 = (double)offset*(double)offset; } else { if (offset < offsetmin) offsetmin = offset; if(offset > offsetmax) offsetmax = offset; offsetsum += (double)offset; offsetsumx2 += (double)offset*(double)offset; } } } free(buf); /* Close file now we are finished. */ close(fd); gettimeofday(&endtp, NULL); timersub(&endtp, &starttp, &elapsedtp); elapsed = (double)elapsedtp.tv_sec + (double)elapsedtp.tv_usec/1000000.0; iorate = (double)((double)numblocks*(double)blocksz)/(double)elapsed; printf("%s reads: %d of: %d bytes elapsed: %.4fs io rate: %.0f bytes/s\n", (seq == 1) ? "sequential" : "random", numblocks, blocksz,elapsed, iorate); if (stats) { printf("max offset: %d min offset: %d\n", offsetmax, offsetmin); printf("avg offset: %.0f ", offsetsum / (double)numblocks); printf("stddev offset: %.0f\n", sqrt(offsetsumx2 / (double)numblocks - (offsetsum / (double)numblocks) * (offsetsum / (double)numblocks))); } free(file); exit(0); } From owner-freebsd-performance@FreeBSD.ORG Thu Dec 28 15:12:02 2006 Return-Path: X-Original-To: freebsd-performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3A07E16A416 for ; Thu, 28 Dec 2006 15:12:02 +0000 (UTC) (envelope-from steve@localhost.lu) Received: from linion.ion.lu (linion.ion.lu [80.90.47.168]) by mx1.freebsd.org (Postfix) with ESMTP id 8211B13C475 for ; Thu, 28 Dec 2006 15:12:01 +0000 (UTC) (envelope-from steve@localhost.lu) Received: (qmail 5438 invoked by uid 89); 28 Dec 2006 15:58:54 +0100 Received: from localhost (HELO ?127.0.0.1?) (steve@localhost.lu@127.0.0.1) by linion.ion.lu with SMTP; 28 Dec 2006 15:58:54 +0100 Message-ID: <4593DB6A.7030208@localhost.lu> Date: Thu, 28 Dec 2006 15:57:46 +0100 From: Steve Clement User-Agent: Thunderbird 1.5.0.9 (X11/20061222) MIME-Version: 1.0 To: freebsd-hackers@FreeBSD.org, freebsd-performance@FreeBSD.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Thu, 28 Dec 2006 16:54:13 +0000 Cc: sos@FreeBSD.org Subject: ICH7 IO Performance Issues X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Dec 2006 15:12:02 -0000 Hi list, I recently bought a New Asus W5F - 12" Laptop and am experiencing some issues with: If "Heavy" disk access occurs I experience overall system hangs Say I just compiled OpenOffice.Org and am happy all is fine so I can do a make clean. Make clean generates quite a lot of disk IO and everything begins lagging like: ls or playing an MP3 file. So I tried to reproduce this by doing the following: cat /dev/random > /tmp/junk_file didn't have the same effect, but I think that's because the source was not the local disk but fast memory a cp of a big file (1GB) didn't do anything BUT an rsync of that very same big file did give me hangs in my music at least so I catted big file attached it to now even bigger file and had some trouble too. The more I spawned IO Processes the badder it got. Is that a known issue? With 1.5Gb Ram there is for sure no way I ran out of Ram or any other memory issue (3Gig Swap) Also whilst rsyncing locally from disc to disc I noticed only between 10MB/s - 15MB/s throughput on average it did spike on 24MB during a few seconds. I seem to notice this issue quite often on FreeBSD, I had major RAID performance problems with a Rocket Raid adapter (ok I know it ain't the best but for what it's worth it should be more or less stable) Are there disk I/O issues on a larger scale maybe? Sincerely yours and willing to test/help, Steve Clement From owner-freebsd-performance@FreeBSD.ORG Thu Dec 28 19:05:06 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1DF6B16A412 for ; Thu, 28 Dec 2006 19:05:06 +0000 (UTC) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id D140513C47C for ; Thu, 28 Dec 2006 19:05:05 +0000 (UTC) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from root by ciao.gmane.org with local (Exim 4.43) id 1H00Ac-0003mp-I4 for freebsd-performance@freebsd.org; Thu, 28 Dec 2006 19:40:02 +0100 Received: from 89-172-61-42.adsl.net.t-com.hr ([89.172.61.42]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 28 Dec 2006 19:40:02 +0100 Received: from ivoras by 89-172-61-42.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 28 Dec 2006 19:40:02 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-performance@freebsd.org From: Ivan Voras Date: Thu, 28 Dec 2006 19:38:02 +0100 Lines: 33 Message-ID: References: <4593DB6A.7030208@localhost.lu> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigA6C184B0C1034AFC428FBDA8" X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-61-42.adsl.net.t-com.hr User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) In-Reply-To: <4593DB6A.7030208@localhost.lu> X-Enigmail-Version: 0.94.1.2 Sender: news Subject: Re: ICH7 IO Performance Issues X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Dec 2006 19:05:06 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigA6C184B0C1034AFC428FBDA8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Steve Clement wrote: > Make clean generates quite a lot of disk IO and everything begins laggi= ng > like: ls or playing an MP3 file. What version and platform of FreeBSD (uname -a)? Can you run the 'gstat' tool when the "lag" occurs and report what it says about disk throughput and %busy? Also, run 'vmstat 1' during that time and report what it says. --------------enigA6C184B0C1034AFC428FBDA8 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFlA8RldnAQVacBcgRAvH/AKCraBZUpuQ9GxxS8T9rCmZa1EZzygCcCFqZ QgRJi9zrzxBVsDQW9lN5r7c= =O8hJ -----END PGP SIGNATURE----- --------------enigA6C184B0C1034AFC428FBDA8-- From owner-freebsd-performance@FreeBSD.ORG Thu Dec 28 21:43:43 2006 Return-Path: X-Original-To: freebsd-performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 594CB16A47C for ; Thu, 28 Dec 2006 21:43:43 +0000 (UTC) (envelope-from steve@localhost.lu) Received: from linion.ion.lu (linion.ion.lu [80.90.47.168]) by mx1.freebsd.org (Postfix) with ESMTP id 4A76B13C481 for ; Thu, 28 Dec 2006 21:43:42 +0000 (UTC) (envelope-from steve@localhost.lu) Received: (qmail 26208 invoked by uid 89); 28 Dec 2006 22:44:37 +0100 Received: from localhost (HELO ?127.0.0.1?) (steve@localhost.lu@127.0.0.1) by linion.ion.lu with SMTP; 28 Dec 2006 22:44:37 +0100 Message-ID: <45943A7B.3090804@localhost.lu> Date: Thu, 28 Dec 2006 22:43:23 +0100 From: Steve Clement User-Agent: Thunderbird 1.5.0.9 (X11/20061222) MIME-Version: 1.0 To: Ivan Voras , freebsd-performance@FreeBSD.org References: <4593DB6A.7030208@localhost.lu> In-Reply-To: Content-Type: multipart/mixed; boundary="------------020602000807070106020209" X-Mailman-Approved-At: Thu, 28 Dec 2006 23:11:11 +0000 Cc: Subject: Re: ICH7 IO Performance Issues X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Dec 2006 21:43:43 -0000 This is a multi-part message in MIME format. --------------020602000807070106020209 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Ivan Voras wrote: > Steve Clement wrote: > > >> Make clean generates quite a lot of disk IO and everything begins lagging >> like: ls or playing an MP3 file. >> > > What version and platform of FreeBSD (uname -a)? > > FreeBSD laptop-steve.localhost.lu 6.2-RC2 FreeBSD 6.2-RC2 #1: Sun Dec 24 19:31:12 CET 2006 root@laptop-steve.localhost.lu:/usr/obj/usr/src/sys/LAPTOP-STEVE i386 > Can you run the 'gstat' tool when the "lag" occurs and report what it > says about disk throughput and %busy? Also, run 'vmstat 1' during that > time and report what it says. > > I ran gstat and my ad4 is constantly on >90% whereas my da0 will stay in the 20% region... But I think thats normal. So see attached my different log files, they seem to be normal. the vmstat is difficult to read but I attached it anyways... cheers a lot., Steve --------------020602000807070106020209 Content-Type: text/plain; name="vmstat.log" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="vmstat.log" procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad4 da0 in sy cs us sy id 3 9 0 26853364 61640 1446 579 28 9 2005 817 0 0 4515 8177 9357 9 12 80 3 7 0 26639956 57016 371 214 63 0 1738 0 159 27 12351 22366 23248 9 24 67 2 9 0 26136540 53360 463 211 50 11 1357 0 120 18 12027 18762 22189 9 26 64 0 9 0 26136540 47992 366 187 74 11 1966 0 179 37 14183 19214 26051 6 31 62 2 12 0 26853372 61564 409 169 45 21 1734 4303 127 20 11012 16167 20698 3 22 75 0 10 0 26864184 76428 2280 188 53 11 3199 6963 85 38 13191 20268 24876 7 26 68 0 9 0 26862104 73480 4825 136 60 46 4990 0 96 24 12171 22961 23317 7 29 63 1 11 0 26863736 68528 2093 51 12 120 2552 0 132 19 9125 19959 17975 9 30 61 2 9 1 26865440 64084 2402 79 29 48 3062 0 96 23 11435 17831 22453 3 27 70 0 15 1 26147392 63252 361 10 9 45 673 0 125 8 6032 9671 12478 6 19 75 0 10 0 26147348 58532 951 56 16 0 1989 0 150 42 10876 17083 21738 3 25 72 0 10 0 26864456 53620 753 135 47 2 1851 0 75 28 11401 16168 22302 5 22 73 3 8 0 26147348 81372 2471 109 40 33 3349 8971 99 27 12118 19019 23554 7 31 62 1 7 0 26143472 77804 5942 165 67 6 6491 0 81 27 13570 23371 25875 13 29 59 1 12 0 26171160 73508 1866 122 50 0 2480 0 69 19 11492 15790 21810 8 24 68 0 15 0 26173672 69144 803 173 57 0 1702 0 70 21 10694 12257 19917 9 24 67 0 14 0 26162372 65920 2064 148 63 0 2696 0 74 18 10418 12568 19407 6 24 71 4 10 0 26134804 61280 8481 184 61 0 8364 0 73 23 12040 21977 22550 11 36 53 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad4 da0 in sy cs us sy id 0 6 0 26135096 57388 1919 283 67 0 2759 0 73 25 12689 14820 22858 4 29 67 1 8 1 26498192 54768 328 173 63 0 1223 0 125 14 9423 13784 17837 5 18 77 0 8 0 26851928 51332 627 183 68 0 1505 0 69 16 11421 15236 21066 6 21 73 0 6 0 26851928 47704 404 183 70 0 1465 0 72 22 11966 13796 22084 6 28 67 0 8 0 26851928 80644 466 182 58 32 1488 10135 96 22 10737 11808 19821 4 20 76 3 8 0 26797988 75716 447 195 61 0 1775 0 77 27 13031 17409 24260 4 27 69 0 11 0 26135096 71424 514 175 57 14 1627 0 75 25 12888 16612 24002 6 23 71 1 10 0 26136320 65780 383 226 64 0 1468 0 63 24 11866 14304 21896 7 20 73 1 10 0 26135096 63060 2771 162 74 0 3059 0 75 23 10956 16078 22404 7 26 66 0 12 0 26851804 58464 2147 139 65 0 2999 0 73 17 10348 16422 21333 13 25 62 2 10 0 26638388 49192 2675 87 37 0 4108 0 71 23 11167 18903 22470 16 29 54 4 12 0 26851148 85324 12737 200 65 32 11563 13510 104 14 9074 22969 19560 26 35 40 0 9 1 26132788 83568 6916 93 30 0 5591 0 110 7 6765 12519 13891 9 20 70 0 12 0 26801224 82104 2916 92 30 0 2987 0 99 25 6844 12738 14173 3 19 78 0 13 0 26138896 77168 807 162 77 0 1763 0 78 21 10832 14577 20775 5 25 69 0 11 0 26502756 73120 479 197 81 0 1457 0 81 17 10968 12807 20546 10 23 67 2 11 0 26139748 70276 614 155 84 0 1345 0 92 12 9589 10935 17978 7 22 71 1 11 0 26139748 68040 241 139 59 16 899 0 87 13 8466 9637 16017 4 17 79 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad4 da0 in sy cs us sy id 1 10 0 26497144 66128 631 178 76 0 1434 0 79 21 10028 14165 19009 8 19 72 0 131 0 26851172 62388 314 183 66 0 1294 0 84 20 9984 11929 19064 8 22 70 0 10 2 26802600 58112 5976 173 57 0 6458 0 72 26 8740 14105 16939 15 24 61 2 12 0 26502956 54872 321 140 38 66 1071 0 89 15 9092 10350 17308 4 25 71 0 11 0 26504468 51896 413 42 11 110 1015 0 129 14 7280 9514 14273 4 21 75 2 11 0 26141648 81660 348 74 19 72 971 9458 113 14 7062 9504 14072 2 18 80 0 11 0 26141288 79084 399 61 23 83 958 0 106 9 6886 8107 13348 6 19 75 1 14 0 26141316 77600 30 36 6 1 432 0 140 10 4760 5827 9551 3 11 86 6 10 2 26141304 75672 260 11 4 0 668 0 384 9 6034 10173 13117 5 14 82 2 6 0 26084828 72968 5748 175 40 0 6428 0 75 29 11502 19276 22946 13 32 54 1 10 2 26803876 66456 6077 140 44 4 6738 0 81 24 11518 18322 22498 14 33 53 4 8 0 26806696 60480 14838 207 68 0 14630 0 76 24 10577 21937 21788 39 32 29 2 9 0 26079936 60808 9734 199 58 0 10111 0 72 20 11437 21693 23237 26 37 37 2 7 0 26080336 57504 29375 216 68 0 25751 0 74 19 10342 32781 22727 23 49 28 1 10 0 26451352 50636 9968 200 61 0 9754 0 70 26 12557 24324 24452 21 37 42 2 9 0 26806320 46368 792 175 52 0 1718 0 76 19 10909 13929 21661 14 26 60 5 9 1 26804276 81140 5512 177 61 23 6757 9628 83 20 10602 16796 21014 39 29 32 3 9 1 26806148 76260 6754 213 62 0 7234 0 73 16 10080 16129 19469 49 26 24 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad4 da0 in sy cs us sy id 1 9 1 26087612 75468 6123 312 77 0 7384 0 77 13 10317 13855 19320 51 24 25 3 9 0 26089400 70004 4836 229 73 0 5114 0 73 15 11236 15928 21447 45 30 25 3 8 0 26583356 67796 5335 248 61 12 6625 0 76 20 11196 15423 21106 45 33 22 1 7 0 26088140 63616 7560 233 52 14 8022 0 80 16 9485 15631 18068 49 29 23 3 9 0 26087844 60928 8367 252 68 0 8826 0 68 18 10688 18511 20817 45 30 25 2 9 0 26088456 58404 6474 264 66 0 7193 0 67 13 10937 16972 20874 47 29 24 3 9 0 26804248 56544 6703 224 67 0 7543 0 75 16 10975 18015 20800 41 32 26 1 8 0 26443876 56916 2210 156 54 0 3592 0 82 13 8715 12802 17129 19 19 62 2 10 0 26443876 55136 2063 9 5 0 1736 0 144 14 7193 12081 15017 7 15 78 3 10 1 26796924 74932 9282 193 61 10 9173 8135 81 11 7889 20303 18675 31 33 36 2 6 1 26797288 84068 6882 487 64 1 7384 3711 75 23 10993 19740 21577 36 33 30 2 9 0 26799860 76804 7572 276 65 0 8493 0 69 42 14295 25171 26790 42 36 22 5 7 0 27167756 70576 7728 260 76 0 9031 0 87 31 14114 22326 26008 40 36 24 3 7 0 26606600 65440 7418 211 53 21 8369 0 81 26 12062 20864 22468 46 26 28 4 8 0 26235180 63664 7437 272 72 0 9497 0 77 29 12811 21612 23960 41 33 25 3 8 0 26236956 58068 7194 229 69 0 8430 0 77 50 13018 20620 23890 45 33 22 1 9 2 26748044 49356 7031 271 76 0 7695 0 82 32 13854 20500 25169 41 37 23 2 9 1 25527676 45840 7185 176 69 13 8194 0 79 20 11097 19636 21668 44 32 24 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad4 da0 in sy cs us sy id 3 9 0 25529304 85408 6263 178 60 47 7331 55584 78 20 10783 20026 21312 41 38 21 2 9 0 26236644 84668 6325 69 15 96 7452 0 125 28 11029 20157 21500 33 41 26 0 11 0 25517784 81416 4546 108 26 82 5804 0 108 23 12240 19340 23655 16 31 53 2 12 0 25879644 80760 322 35 11 28 354 0 117 10 6627 8633 13422 4 17 79 0 12 0 25880460 80348 1223 4 1 0 940 0 234 0 2846 5565 6549 4 8 88 4 10 0 25518540 73708 1390 91 28 0 2455 0 129 43 13065 20392 25622 7 29 64 0 10 1 25518540 69704 362 143 43 0 1378 0 69 27 10368 14058 20207 4 21 75 3 10 0 25518540 64596 435 149 45 2 1870 0 73 33 13517 18054 25701 5 28 68 2 10 0 25518540 60180 253 106 33 0 1456 0 78 29 12090 14866 22860 4 24 71 2 8 1 25518540 55660 411 160 43 0 1706 0 68 28 11378 14644 21720 9 25 65 0 10 0 25518540 51692 348 190 49 0 1438 0 77 23 11246 13056 21076 6 21 73 2 10 0 25518540 80364 424 185 47 0 1571 9179 75 22 11565 15728 22308 3 24 73 0 10 0 25518540 76520 236 150 44 0 1391 0 73 23 10689 12780 20576 7 20 73 2 10 0 26021956 73636 461 135 43 0 1285 0 72 17 9163 10368 17489 3 17 80 2 10 0 25517016 71108 258 136 45 0 1064 0 74 16 8113 9795 15739 3 17 80 1 10 0 25880112 69816 412 140 42 0 849 0 70 4 5865 6165 11479 6 11 84 0 8 1 25880112 68852 147 116 42 0 670 0 84 2 4810 5926 9732 5 9 86 0 10 1 25880112 68028 515 125 44 0 832 0 71 0 3672 5302 7805 6 8 86 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad4 da0 in sy cs us sy id 3 10 0 25880112 66960 204 113 41 0 550 0 74 0 5167 4550 10133 3 9 88 3 10 0 25880112 65856 219 30 9 0 838 0 116 3 4896 5638 9810 3 8 89 1 10 0 25880112 63928 387 137 33 5 737 0 81 10 6205 8251 12494 6 10 84 2 8 0 26383528 62196 470 117 41 0 975 0 69 7 7054 7556 13654 4 14 81 0 10 0 26383528 60520 318 132 48 0 811 0 74 7 7091 8353 13747 4 13 82 1 8 0 26383528 58940 447 143 42 0 910 0 68 6 6814 6897 13079 5 13 82 2 10 0 26383528 57620 267 138 51 0 723 0 79 21 7457 8452 14512 9 14 77 3 10 0 26383528 56944 463 149 45 0 723 0 68 0 6123 7226 12348 6 15 79 1 10 1 26383528 55888 199 109 40 0 517 0 62 3 5629 6642 11112 5 13 82 0 10 0 26383528 54940 424 106 36 18 767 0 77 13 5511 5925 11163 1 14 85 0 10 0 26385052 53800 472 108 46 23 687 0 78 1 5526 6239 10845 4 12 84 0 10 1 26385052 52328 522 143 43 7 952 0 76 8 6772 7144 12906 3 15 82 0 10 1 26385052 50956 407 117 42 0 768 0 70 5 7996 9068 15058 4 21 75 0 10 0 26385052 49076 574 145 45 0 979 0 73 8 8971 8959 16753 4 18 78 0 10 0 26385052 82844 156 83 21 86 527 12030 105 4 5539 6004 11030 3 11 86 0 10 0 26385052 80952 333 52 10 80 682 0 96 8 5974 6788 11859 3 20 77 1 10 0 26385052 78772 190 91 16 35 576 0 83 5 5639 6628 11262 4 12 84 0 11 0 26385052 77780 261 67 12 33 666 0 129 5 4761 5049 9600 3 15 81 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad4 da0 in sy cs us sy id 4 10 1 26385052 77236 4 4 2 0 339 0 132 8 4087 5815 8613 5 9 86 0 10 2 26383528 75144 744 146 44 0 1068 0 75 7 6313 7306 12498 3 25 72 1 11 3 26383944 73776 1302 154 40 0 1962 0 124 4 5486 7001 11092 3 15 82 0 10 0 26383528 72868 4373 152 43 4 3856 0 73 0 6270 11492 14533 13 25 62 2 10 0 26383528 71240 482 181 51 0 499 0 82 4 6681 6290 12766 4 10 86 0 10 0 26383528 70064 233 166 42 0 452 0 69 0 6140 6950 11950 5 12 84 1 10 0 26383528 69508 337 127 42 0 729 0 67 0 5792 6074 11227 6 13 81 1 8 0 26383528 67960 294 153 44 0 647 0 78 5 5693 7229 11368 6 12 83 0 10 0 26383528 66136 449 164 43 0 967 0 79 4 6896 8426 13483 4 14 81 1 10 0 26383528 64552 256 147 40 0 659 0 74 4 7254 8270 14150 5 17 78 0 10 1 26383528 62088 1611 123 38 0 2619 0 70 13 9516 14447 19110 13 19 68 2 10 0 26383528 59392 303 155 46 0 738 0 75 14 8662 11238 16915 5 19 75 0 10 1 26383528 57408 432 154 43 0 1068 0 76 8 7362 8581 14259 2 14 84 0 11 1 26383528 55480 185 80 25 0 807 0 89 6 5697 6105 11312 4 12 83 1 10 0 26383528 53712 545 127 42 9 950 0 86 6 7612 9599 15312 3 12 84 1 10 0 26383528 52612 256 147 46 0 694 0 70 7 6682 6709 13003 4 14 83 2 10 1 26383528 50604 449 124 46 0 1027 0 74 7 6777 8425 13216 4 14 81 0 9 1 26383528 48600 323 118 49 0 986 0 75 10 8623 9766 16444 5 16 78 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad4 da0 in sy cs us sy id 5 10 0 26383528 47048 435 132 47 0 1002 0 81 7 6726 7842 13180 6 14 80 0 10 0 26383528 85800 2189 116 42 6 3380 11106 70 10 6424 8276 12917 8 15 77 2 12 1 26385284 82116 11021 138 46 0 8315 0 73 21 6846 16115 18153 25 31 43 1 10 0 26383528 80804 893 155 47 15 1374 0 89 20 8176 11298 16680 9 15 75 3 8 0 26383528 78224 405 131 45 0 1088 0 74 8 7792 8388 14899 6 16 78 2 9 0 26382464 77268 2322 211 56 0 2580 0 84 4 6644 12522 14003 12 17 71 0 9 0 25719572 76112 509 229 62 3 923 0 111 0 6655 6498 12979 4 17 79 1 9 0 25719572 74368 461 178 49 0 988 0 84 12 7468 11860 14458 4 15 81 0 9 0 25719564 72560 682 265 55 0 1276 0 91 8 7850 15612 15215 3 20 77 1 9 0 25721096 70240 479 229 65 0 1172 0 111 5 7447 9187 14347 5 19 76 1 9 0 25721096 67592 300 60 15 149 924 0 131 12 7162 8665 14099 3 22 75 2 9 1 25721096 65092 175 48 12 70 689 0 99 10 5696 6630 11428 4 15 80 1 11 0 25721096 64880 226 12 3 26 463 0 134 2 4306 6236 9249 4 9 87 2 9 0 25721096 62440 2561 86 30 0 3180 0 182 16 7569 14569 17055 14 17 69 2 9 0 25721096 60632 605 229 60 0 965 0 110 4 7318 10002 14801 11 22 67 2 9 0 25721096 58536 365 217 62 0 1135 0 100 8 8589 11930 16906 10 17 73 1 9 0 25721096 56432 1051 201 61 0 1898 0 107 12 8376 9816 16335 5 17 78 0 9 1 25721096 54416 339 190 60 5 1065 0 124 8 8452 10729 17054 8 17 75 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad4 da0 in sy cs us sy id 0 10 0 25723152 61760 7799 163 48 24 7460 2962 98 7 8363 18304 17139 15 26 59 0 9 0 25720592 82216 3728 240 58 8 4209 6191 77 13 8563 17037 17958 16 25 58 2 10 0 25721124 79588 541 154 69 0 818 0 71 8 7915 11312 16889 13 20 67 1 10 0 25721604 78144 282 236 74 0 914 0 76 1 6190 8363 13682 7 16 77 5 9 0 25721584 76176 1154 184 65 0 1703 0 79 7 6742 10163 14710 7 15 78 1 13 0 25749036 72600 2250 146 58 0 2645 0 71 9 7815 11774 16545 9 19 72 2 11 0 25721944 72564 2412 107 38 0 2567 0 102 2 6435 14250 14267 12 20 68 0 10 0 25722008 70356 1868 94 36 0 2289 0 100 16 8028 15928 18343 14 20 66 1 8 2 25719236 69152 7489 161 56 0 6484 0 73 9 8253 19246 18478 9 31 60 0 8 1 25718800 68640 8592 239 68 0 7925 0 72 4 6247 14853 14566 11 26 63 4 10 0 25719008 67012 2103 140 51 11 2542 0 64 13 6291 12068 14367 7 20 73 2 12 0 26066036 63012 1728 156 65 0 2082 0 81 8 5691 8970 12842 6 12 82 0 10 2 26060340 62648 526 179 92 0 1222 0 92 7 5952 9090 13531 9 13 79 0 9 1 26063472 61704 6361 234 75 5 6568 0 122 2 5475 12318 12750 19 19 62 2 9 0 26060568 60456 4780 226 68 0 4985 0 104 11 6290 11387 13807 17 19 63 3 8 2 25711896 60552 3784 174 61 8 4378 0 100 16 6092 12530 13710 14 24 62 1 8 0 25208316 58228 2511 331 76 0 3243 0 84 9 7850 12274 16520 16 21 62 0 8 1 25208484 56936 6520 232 75 0 6212 0 109 4 8878 16337 18557 15 28 57 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad4 da0 in sy cs us sy id 0 30 0 25213836 52872 8113 163 52 0 7457 0 78 13 8567 17014 18247 11 29 60 0 13 0 25220656 50572 5418 91 54 13 5327 57 80 7 6691 14020 15124 14 22 64 0 13 0 25222800 78268 864 143 67 19 1449 8588 111 10 7555 10495 16122 11 18 71 5 8 0 25223284 74860 4410 144 49 8 4991 0 81 11 6847 12317 14522 35 24 42 2 11 0 25223568 72892 6461 145 50 0 7035 0 79 18 7210 14366 15157 41 20 39 0 9 0 25216844 73184 4686 100 29 120 5705 0 120 9 6392 12401 13487 30 21 49 2 10 0 25214324 72920 311 27 16 85 588 0 106 7 5469 11190 13047 9 21 71 2 12 0 25713908 73320 956 49 19 16 1219 0 122 6 4331 9071 10661 9 17 74 2 8 0 25713908 71832 1239 16 4 0 1296 0 237 12 5758 11801 14485 5 19 76 3 9 0 25214928 67492 7640 177 56 0 6718 0 72 6 6811 19564 15952 19 25 56 1 12 2 25241208 65580 4208 157 54 0 4151 0 68 9 6424 41976 14975 16 32 51 3 9 0 25212836 65676 11800 150 52 0 10959 0 54 6 6635 38273 15115 28 36 36 5 8 0 25730184 63528 8683 263 59 0 7659 0 82 1 6696 18965 15108 33 26 41 2 9 0 25228584 60036 4037 206 57 0 3739 0 72 11 8992 16235 18842 22 23 55 3 9 0 25228520 58764 3000 162 49 6 3059 0 76 5 7092 13239 15261 15 17 68 1 8 2 25226388 57792 3680 219 61 0 4166 0 82 12 8257 15418 17555 14 25 61 0 9 2 25223124 58620 10696 198 61 0 9797 0 67 5 7193 20847 15404 31 25 43 1 8 1 25194452 82604 898 189 49 0 7196 0 75 14 8724 13229 18634 12 18 70 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad4 da0 in sy cs us sy id 0 13 1 25194452 82192 26 37 12 0 151 0 135 3 5566 7290 11478 7 14 80 0 8 2 25194452 83896 854 130 43 0 1542 0 102 14 7043 13266 15291 16 16 69 3 8 1 25194452 82252 1084 266 75 0 679 0 76 5 7349 11485 15445 12 19 70 2 8 0 25194452 80072 495 197 53 0 235 0 61 13 9218 11894 18518 6 23 71 0 8 1 25194452 78536 286 232 61 0 34 0 60 5 8676 9791 16858 9 14 76 2 8 0 25697868 76992 481 215 59 0 171 0 76 4 7945 10209 16028 10 17 72 0 8 1 25194452 75828 344 261 67 11 33 0 83 3 7988 9321 15740 7 15 78 0 8 1 25194452 73968 1581 283 63 0 1227 0 64 9 9584 13793 19259 9 20 71 0 8 0 25697868 71764 394 284 66 29 34 0 81 14 8893 10868 17285 5 17 79 0 8 0 25194452 69744 493 284 66 0 712 0 67 10 8635 9820 16677 6 17 77 0 8 0 25194448 67896 315 237 59 0 939 0 60 10 8664 10863 16784 7 17 75 2 8 0 25194448 66116 496 241 64 0 1083 0 65 14 8099 8512 15491 7 14 78 0 8 0 25194448 64272 359 270 62 0 1031 0 64 12 8907 10100 16916 3 17 80 0 8 1 25194448 61972 535 289 71 0 1301 0 72 11 9991 10720 18693 6 16 78 0 6 1 25189664 62140 378 262 65 0 1055 0 68 8 8044 7830 15252 4 15 81 0 8 0 25189664 61004 531 304 78 0 801 0 76 2 7881 8103 14989 7 18 75 0 8 0 25189664 58600 339 229 79 0 1135 0 78 7 9612 10001 17908 7 18 75 0 8 0 25230624 56608 473 217 64 12 1252 0 81 10 9152 9395 17032 4 15 81 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad4 da0 in sy cs us sy id 1 8 0 25230624 54700 343 197 60 72 1039 0 126 7 8686 9314 16631 6 19 75 1 8 0 25230624 52300 248 30 14 71 801 0 108 10 8254 11032 16185 5 19 76 4 11 0 25230624 50996 162 57 23 93 699 114 113 8 7197 8819 14227 6 19 75 1 8 1 25230624 80664 441 57 16 43 851 8169 92 10 5653 7592 11352 5 18 77 1 8 2 25230624 79916 7 12 3 0 316 0 164 7 4693 6577 9633 4 12 84 3 8 0 25230624 78404 647 72 39 0 893 0 84 6 4755 6117 9572 6 9 84 0 8 1 25230624 76240 376 166 61 0 1009 0 74 8 7816 12224 15687 4 21 75 0 8 1 25230624 74772 584 255 69 0 1090 0 69 4 8509 9434 16541 4 18 77 0 8 0 25230624 73576 310 263 73 0 901 0 72 6 8471 9996 16416 6 17 78 1 8 0 25230624 72912 511 265 61 0 772 0 62 1 7586 10289 15073 8 20 72 3 8 0 25230624 70952 2452 261 63 5 3131 0 67 4 7571 9835 15358 6 18 76 1 10 1 25230624 69800 553 221 68 0 994 0 66 3 8305 8796 16201 5 19 75 0 10 0 25230624 68812 155 90 29 0 727 0 98 5 6959 7213 13501 5 12 83 0 8 0 25230624 66288 613 170 43 0 1229 0 103 14 7794 10428 15650 6 16 78 2 8 0 25230624 63016 4822 216 67 0 5546 0 70 17 10204 13583 20321 10 30 60 2 8 0 25230624 61124 2591 257 65 0 3359 0 71 6 8388 12661 16996 11 25 64 4 6 0 25230624 59664 2852 222 64 0 3086 0 64 7 7802 10666 16037 6 22 72 0 8 1 25230624 57304 526 202 59 0 1326 0 64 15 9941 13560 19274 7 20 74 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad4 da0 in sy cs us sy id 2 8 0 25230624 55420 358 301 68 11 1020 0 76 9 8487 9014 16506 5 14 81 1 8 0 25230624 53472 490 225 62 8 1193 0 65 11 8311 11069 16236 5 16 79 2 8 0 25230624 51940 383 212 59 0 914 0 60 17 8195 7399 15219 6 14 80 0 8 0 25230628 49532 529 215 71 5 1269 0 76 14 7634 9039 14731 9 16 74 2 8 0 25230628 47916 517 199 73 0 1120 0 75 3 8113 7688 14988 4 16 79 0 8 0 25230628 46140 713 198 74 0 1269 0 74 3 9137 10612 17012 6 16 78 0 6 0 25230628 44548 464 155 64 0 962 0 79 4 8663 8755 16216 6 15 79 0 8 0 25230628 82372 445 119 48 32 1014 13551 78 25 7522 8604 14650 2 15 83 1 8 1 25737304 80356 339 203 65 0 1142 0 66 11 8693 9590 16389 4 16 80 0 8 0 25233888 78772 499 271 76 0 1114 0 76 7 7886 8309 15100 4 17 79 0 8 1 25233888 76400 341 198 65 0 1086 0 68 8 8497 9216 16130 5 13 82 0 8 0 25233888 74940 521 230 72 0 1130 0 72 5 7789 8861 14979 6 17 77 0 9 0 25234256 72320 439 204 61 0 1315 0 69 15 8990 9422 16938 4 15 81 1 9 0 25234256 65656 2698 136 35 0 4284 0 71 29 8020 13820 17371 11 17 72 1 9 0 25234256 60180 1396 90 29 12 2754 0 70 22 6777 12026 14244 9 16 76 0 10 0 25234256 57756 277 36 13 177 859 0 142 5 4376 7882 9268 3 19 78 1 10 1 25235076 54196 305 44 13 53 1028 0 92 9 4841 11820 10252 7 22 71 1 10 1 25234256 51600 314 36 14 67 1243 43 93 11 4359 7449 9277 4 16 81 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad4 da0 in sy cs us sy id 0 11 1 25234256 49952 211 11 3 16 684 15 126 10 5928 9018 12273 2 16 82 0 9 0 25234256 80204 205 28 8 15 599 10565 149 6 3966 5808 8502 3 13 84 1 9 0 25234256 73640 717 175 33 0 2204 0 73 26 5572 7769 11328 6 11 83 1 9 0 25234256 67516 111 103 30 0 1737 0 86 29 6323 9872 13107 6 16 78 1 9 0 25234256 61320 430 112 33 0 2059 0 80 26 5861 9337 12118 7 11 81 4 9 0 25234256 54656 347 78 34 0 1889 0 83 24 6101 7568 12067 3 12 85 1 9 0 25234256 48628 466 123 35 0 2109 0 76 25 7079 9364 13936 4 20 77 2 9 1 25234256 45500 80 34 15 0 1085 0 99 17 5425 7158 10992 4 13 83 2 9 1 25234256 81000 306 28 22 41 1397 15330 80 19 4970 7485 10319 3 12 85 1 9 0 25234256 73928 371 105 25 0 1978 0 73 27 6054 9968 12467 6 18 76 1 9 0 25234256 67940 450 88 32 0 2026 0 69 24 6118 8072 12286 4 12 84 0 9 0 25234256 62356 172 98 32 0 1674 0 68 23 5114 6448 10340 4 11 85 0 9 0 25234256 56280 485 108 31 0 1938 0 72 23 6376 8336 12820 5 13 82 1 9 0 24764076 49804 270 118 32 11 1893 0 85 27 6814 8461 13485 4 16 79 1 12 0 25193296 44452 311 51 28 11 1890 74 72 24 6255 8026 12384 4 14 82 2 9 0 25193296 76000 283 105 37 21 2161 15701 99 44 8184 12156 16602 4 22 74 2 9 1 25193296 69936 503 147 44 0 2092 0 74 24 7386 10707 14645 7 17 76 1 9 1 25193296 62324 202 108 35 0 2185 0 77 32 8924 12047 17441 6 16 77 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad4 da0 in sy cs us sy id 0 9 1 25193296 55556 6411 95 36 10 7424 0 83 28 7393 14816 17262 15 25 60 1 9 0 25193296 48968 5071 112 34 0 6013 0 78 30 7767 14593 17269 13 20 67 0 9 0 24764076 81944 501 97 38 32 2267 11709 112 29 6900 11429 14481 6 17 77 2 9 1 25193296 74428 267 125 36 0 2275 0 84 41 7343 10118 14816 5 20 74 1 9 0 25193296 67200 501 120 39 0 2307 0 78 20 6088 6626 12061 27 12 61 0 9 0 25193296 60960 344 114 37 0 1875 0 75 20 6873 8341 13467 5 16 79 1 9 0 25193296 53220 397 128 44 0 2480 0 93 35 6817 11389 14186 6 14 80 3 9 0 25267492 45772 339 93 44 0 2178 0 88 29 6983 9956 14121 3 18 79 0 9 0 25193296 79388 423 119 35 32 1969 13563 107 28 7015 9564 14209 5 15 80 2 9 0 25193296 72728 251 108 35 0 2054 0 73 25 6991 9185 14056 4 20 76 1 9 3 25193296 65744 354 88 34 0 2123 0 75 26 7119 10295 14275 5 16 80 0 9 0 25193296 60524 204 94 30 30 1528 0 89 19 5126 7263 10847 2 15 83 0 9 0 25267492 56680 311 38 17 75 1186 0 118 13 5637 8471 11699 4 14 82 0 9 0 25267492 52076 78 40 16 62 1166 0 97 17 4673 7155 10014 1 16 82 0 11 0 25193296 79964 518 60 13 66 1213 11752 102 13 5585 8333 11657 3 14 83 0 11 0 25193296 69560 188 62 30 53 1324 0 92 11 6285 6548 12783 3 12 85 3 14 0 25193296 67628 504 65 42 46 2457 0 125 13 8604 8382 17263 2 11 88 0 16 1 25193752 64156 4635 51 25 0 5395 0 99 12 3552 6948 8279 7 9 84 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad4 da0 in sy cs us sy id 0 13 0 25193296 60688 491 75 47 0 1158 0 83 10 3359 3730 7024 0 8 92 1 14 0 24764176 56592 2510 82 50 0 3671 0 86 14 4564 5836 9596 3 14 83 1 15 0 24764176 52172 2594 59 48 3 3398 0 83 15 3852 6738 8724 9 11 80 0 15 1 24764176 49936 3989 18 19 0 4883 0 97 12 4092 7510 9339 8 9 83 0 15 1 24764076 44912 145 70 50 4 871 29 85 16 3665 5369 7792 6 9 86 1 13 0 24764076 77912 1331 19 25 28 2740 10113 77 13 4620 7883 10652 5 8 87 1 11 0 24764076 67420 127 73 40 2 2804 0 122 18 3551 7732 7325 11 8 81 2 10 0 24764076 66828 751 102 42 0 2894 0 90 21 6048 12128 12365 16 20 64 2 12 0 24752312 58100 2362 109 40 0 2924 0 100 23 5819 9244 12052 4 13 83 0 15 0 24752236 54064 6426 94 46 0 7155 0 81 23 6555 14855 15806 18 22 60 0 11 0 24750620 82152 611 67 35 27 1819 8116 105 29 6279 10377 13254 5 16 80 0 9 0 25105824 74404 473 113 31 0 2196 0 73 25 6471 9234 13045 4 20 76 --------------020602000807070106020209 Content-Type: text/plain; name="gstat.log" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="gstat.log" L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 0 0 0 0.0 0 0 0.0 0.0| acd0 10 102 88 3912 51.7 14 415 66.3 102.7| ad4 10 102 88 3912 51.7 14 415 66.3 102.7| ad4s1 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1a 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1b 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1c 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1d 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1e 10 102 88 3912 51.8 14 415 66.3 102.7| ad4s1f 0 22 0 0 0.0 22 2810 5.4 11.9| da0 0 22 0 0 0.0 22 2810 5.5 12.0| da0s1 0 0 0 0 0.0 0 0 0.0 0.0| da0s1c 0 22 0 0 0.0 22 2810 5.5 12.1| da0s1d L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 0 0 0 0.0 0 0 0.0 0.0| acd0 10 102 88 3912 51.7 14 415 66.3 102.7| ad4 10 102 88 3912 51.7 14 415 66.3 102.7| ad4s1 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1a 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1b 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1c 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1d 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1e 10 102 88 3912 51.8 14 415 66.3 102.7| ad4s1f 0 22 0 0 0.0 22 2810 5.4 11.9| da0 0 22 0 0 0.0 22 2810 5.5 12.0| da0s1 0 0 0 0 0.0 0 0 0.0 0.0| da0s1c 0 22 0 0 0.0 22 2810 5.5 12.1| da0s1d w: 0.500s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 0 0 0 0.0 0 0 0.0 0.0| acd0 2 66 66 2898 28.0 0 0 0.0 100.2| ad4 2 66 66 2898 28.0 0 0 0.0 100.2| ad4s1 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1a 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1b 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1c 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1d 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1e 2 66 66 2898 28.1 0 0 0.0 100.2| ad4s1f 0 16 0 0 0.0 16 2044 5.5 8.9| da0 0 16 0 0 0.0 16 2044 5.6 8.9| da0s1 0 0 0 0 0.0 0 0 0.0 0.0| da0s1c 0 16 0 0 0.0 16 2044 5.7 9.1| da0s1d dT: 0.501s w: 0.500s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 0 0 0 0.0 0 0 0.0 0.0| acd0 1 68 68 2227 20.2 0 0 0.0 96.6| ad4 1 68 68 2227 20.2 0 0 0.0 96.7| ad4s1 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1a 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1b 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1c 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1d 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1e 1 68 68 2227 20.2 0 0 0.0 96.8| ad4s1f 1 6 0 0 0.0 6 766 5.5 3.3| da0 1 6 0 0 0.0 6 766 5.6 3.4| da0s1 0 0 0 0 0.0 0 0 0.0 0.0| da0s1c 1 6 0 0 0.0 6 766 5.7 3.4| da0s1d dT: 0.502s w: 0.500s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 0 0 0 0.0 0 0 0.0 0.0| acd0 1 68 68 3203 24.1 0 0 0.0 103.0| ad4 1 68 68 3203 24.2 0 0 0.0 103.1| ad4s1 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1a 0 2 2 40 40.6 0 0 0.0 8.1| ad4s1b 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1c 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1d 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1e 1 66 66 3163 23.7 0 0 0.0 103.0| ad4s1f 0 6 0 0 0.0 6 765 5.6 3.4| da0 0 6 0 0 0.0 6 765 5.7 3.4| da0s1 0 0 0 0 0.0 0 0 0.0 0.0| da0s1c 0 6 0 0 0.0 6 765 5.8 3.5| da0s1d dT: 0.502s w: 0.500s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 0 0 0 0.0 0 0 0.0 0.0| acd0 2 70 70 2717 26.6 0 0 0.0 99.3| ad4 2 70 70 2717 26.6 0 0 0.0 99.3| ad4s1 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1a 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1b 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1c 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1d 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1e 2 70 70 2717 26.7 0 0 0.0 99.3| ad4s1f 1 4 0 0 0.0 4 510 5.6 2.2| da0 1 4 0 0 0.0 4 510 5.6 2.2| da0s1 0 0 0 0 0.0 0 0 0.0 0.0| da0s1c 1 4 0 0 0.0 4 510 5.7 2.3| da0s1d dT: 0.501s w: 0.500s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 0 0 0 0.0 0 0 0.0 0.0| acd0 2 78 78 3353 21.4 0 0 0.0 97.7| ad4 2 78 78 3353 21.5 0 0 0.0 97.8| ad4s1 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1a 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1b 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1c 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1d 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1e 2 78 78 3353 21.5 0 0 0.0 97.8| ad4s1f 0 12 0 0 0.0 12 1533 5.5 6.6| da0 0 12 0 0 0.0 12 1533 5.6 6.7| da0s1 0 0 0 0 0.0 0 0 0.0 0.0| da0s1c 0 12 0 0 0.0 12 1533 5.6 6.8| da0s1d dT: 0.501s w: 0.500s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 0 0 0 0.0 0 0 0.0 0.0| acd0 40 86 18 527 136.9 68 1820 152.2 99.7| ad4 40 86 18 527 137.0 68 1820 152.3 99.7| ad4s1 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1a 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1b 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1c 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1d 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1e 40 86 18 527 137.0 68 1820 152.4 99.7| ad4s1f 0 8 0 0 0.0 8 1022 5.5 4.4| da0 0 8 0 0 0.0 8 1022 5.6 4.4| da0s1 0 0 0 0 0.0 0 0 0.0 0.0| da0s1c 0 8 0 0 0.0 8 1022 5.7 4.5| da0s1d dT: 0.501s w: 0.500s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 0 0 0 0.0 0 0 0.0 0.0| acd0 2 76 76 3010 19.9 0 0 0.0 97.2| ad4 2 76 76 3010 20.0 0 0 0.0 97.3| ad4s1 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1a 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1b 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1c 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1d 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1e 2 76 76 3010 20.0 0 0 0.0 97.4| ad4s1f 0 22 0 0 0.0 22 2810 6.0 12.2| da0 0 22 0 0 0.0 22 2810 6.1 12.3| da0s1 0 0 0 0 0.0 0 0 0.0 0.0| da0s1c 0 22 0 0 0.0 22 2810 6.1 12.5| da0s1d dT: 0.502s w: 0.500s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 0 0 0 0.0 0 0 0.0 0.0| acd0 2 84 80 3459 22.4 4 8 40.2 99.5| ad4 2 84 80 3459 22.4 4 8 40.3 99.5| ad4s1 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1a 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1b 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1c 0 4 0 0 0.0 4 8 40.3 9.3| ad4s1d 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1e 2 80 80 3459 22.5 0 0 0.0 99.5| ad4s1f 0 2 0 0 0.0 2 255 5.7 1.1| da0 0 2 0 0 0.0 2 255 5.8 1.2| da0s1 0 0 0 0 0.0 0 0 0.0 0.0| da0s1c 0 2 0 0 0.0 2 255 5.8 1.2| da0s1d dT: 0.502s w: 0.500s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 0 0 0 0.0 0 0 0.0 0.0| acd0 2 72 68 2667 26.5 4 8 22.1 101.3| ad4 2 72 68 2667 26.6 4 8 22.2 101.3| ad4s1 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1a 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1b 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1c 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1d 0 4 0 0 0.0 4 8 22.2 7.1| ad4s1e 2 68 68 2667 26.6 0 0 0.0 101.3| ad4s1f 0 0 0 0 0.0 0 0 0.0 0.0| da0 0 0 0 0 0.0 0 0 0.0 0.0| da0s1 0 0 0 0 0.0 0 0 0.0 0.0| da0s1c 0 0 0 0 0.0 0 0 0.0 0.0| da0s1d dT: 0.501s w: 0.500s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 0 0 0 0.0 0 0 0.0 0.0| acd0 2 70 70 2739 24.6 0 0 0.0 99.5| ad4 2 70 70 2739 24.6 0 0 0.0 99.5| ad4s1 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1a 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1b 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1c 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1d 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1e 2 70 70 2739 24.7 0 0 0.0 99.5| ad4s1f 0 16 0 0 0.0 16 2044 5.5 8.9| da0 0 16 0 0 0.0 16 2044 5.6 9.0| da0s1 0 0 0 0 0.0 0 0 0.0 0.0| da0s1c 0 16 0 0 0.0 16 2044 5.7 9.1| da0s1d FreeBSD laptop-steve.localhost.lu 6.2-RC2 FreeBSD 6.2-RC2 #1: Sun Dec 24 19:31:12 CET 2006 root@laptop-steve.localhost.lu:/usr/obj/usr/src/sys/LAPTOP-STEVE i386 --------------020602000807070106020209-- From owner-freebsd-performance@FreeBSD.ORG Fri Dec 29 13:28:23 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9E06416A403 for ; Fri, 29 Dec 2006 13:28:23 +0000 (UTC) (envelope-from freebsdworld@gmail.com) Received: from wx-out-0506.google.com (wx-out-0506.google.com [66.249.82.226]) by mx1.freebsd.org (Postfix) with ESMTP id 642DB13C465 for ; Fri, 29 Dec 2006 13:28:23 +0000 (UTC) (envelope-from freebsdworld@gmail.com) Received: by wx-out-0506.google.com with SMTP id s18so4620914wxc for ; Fri, 29 Dec 2006 05:28:22 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:subject:from:to:content-type:date:message-id:mime-version:x-mailer:content-transfer-encoding; b=p7ALmhJkPbqBCMz7J3SH8RHlbXnABUMP8G9sRt2BJPaGBrk5sfFEb7K3yUSwl5zCCXtPxG4JDPtpVyQqEuYGNDfKqzGEctiJWUGEa7y+/ZSBNaUTsEu6qtH1DoH1JL/7Kee75kOwtyesH/XH/TFE8r5Z4o2xllMu7uvLmmHlgr8= Received: by 10.70.66.18 with SMTP id o18mr13385569wxa.1167360702905; Thu, 28 Dec 2006 18:51:42 -0800 (PST) Received: from ?24.213.219.146? ( [24.213.219.146]) by mx.google.com with ESMTP id h13sm23017360wxd.2006.12.28.18.51.41; Thu, 28 Dec 2006 18:51:42 -0800 (PST) From: Benjamin D Adams To: freebsd-performance@FreeBSD.org Content-Type: text/plain Date: Fri, 29 Dec 2006 00:01:39 -0500 Message-Id: <1167368499.641.3.camel@tard.freebsdworld.net> Mime-Version: 1.0 X-Mailer: Evolution 2.8.2.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: Subject: Best RAID setup X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2006 13:28:23 -0000 I have three SATA drives (750 GB Barracuda 7200) I'm trying to figureout the best raid configuration for this. I'm looking for great preformance but I also want to so if one of the drives dies. I can remove it and the other two will work fine. Then when I buy a new drive I can just add it to the RAID setup and it will work again like nothing happen. Thanks for any advice. -- Benjamin D Adams --http://www.FreeBSDWorld.NET From owner-freebsd-performance@FreeBSD.ORG Fri Dec 29 14:02:19 2006 Return-Path: X-Original-To: freebsd-performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3F45916A407 for ; Fri, 29 Dec 2006 14:02:19 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30313.mail.mud.yahoo.com (web30313.mail.mud.yahoo.com [209.191.69.75]) by mx1.freebsd.org (Postfix) with SMTP id E0A4813C44C for ; Fri, 29 Dec 2006 14:02:18 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 76993 invoked by uid 60001); 29 Dec 2006 13:35:37 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=wMvU3oO1rEEubHKvGR193lnKPa1x7v2eUZ5+Vv3dG71JNV8LsRgSP+fjuYoag7vBNQMfUSLvfRHtLp9q31EdiJMNe7FNv+VYf9ISHf/+goue3WI9+aDJ8+6YLcEU5slhzu2dMsJ78wi0RaMlLj1rpEFmE6x1YfF0QCkNEnUzdkc=; X-YMail-OSG: bap25K4VM1kIUjxQU7W0MMUMaRRxYxG8lkhe.E6d03AisMAg2S3T6nryfJbOrc4q8s879PmR3bCdeQb3xIEg3YLqcWAZYemKbgGMBfsjAWLEHFhYQfip8Y25IhmUfjicPU2RgrQk24Dev5k- Received: from [85.212.8.201] by web30313.mail.mud.yahoo.com via HTTP; Fri, 29 Dec 2006 05:35:37 PST Date: Fri, 29 Dec 2006 05:35:37 -0800 (PST) From: "R. B. Riddick" To: Benjamin D Adams , freebsd-performance@FreeBSD.org In-Reply-To: <1167368499.641.3.camel@tard.freebsdworld.net> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <497895.75266.qm@web30313.mail.mud.yahoo.com> Cc: Subject: Re: Best RAID setup X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2006 14:02:19 -0000 --- Benjamin D Adams wrote: > I have three SATA drives (750 GB Barracuda 7200) I'm trying to > figureout the best raid configuration for this. > I'm looking for great preformance but I also want to so if one of the > drives dies. I can remove it and the other two will work fine. Then > when I buy a new drive I can just add it to the RAID setup and it will > work again like nothing happen. > Sounds like u want to try graid3 (without gcache, since u just have 3 disks), which can only use [2^n +1]-disks (3, 5, 9, ...), and which introduces a higher sector size (1024 (3 disks), 2048, 4096, ...), or the more experimental graid5 ( http://home.tiscali.de/cmdr_faako/geom_raid5.tbz ), which might handle some special cases more gracefully, and which serves me well since months (no real stress on my box - just a PVR with max. 2 input channels of each about 500KB/sec; if u need install instructions, give me a note). -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-performance@FreeBSD.ORG Fri Dec 29 17:01:45 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1D94616A407 for ; Fri, 29 Dec 2006 17:01:45 +0000 (UTC) (envelope-from aaron.pratt@mytriggers.com) Received: from mail.mytriggers.com (mail.mytriggers.com [66.194.131.37]) by mx1.freebsd.org (Postfix) with ESMTP id D3B0F13C442 for ; Fri, 29 Dec 2006 17:01:44 +0000 (UTC) (envelope-from aaron.pratt@mytriggers.com) Received: (qmail 55128 invoked by uid 89); 29 Dec 2006 16:35:03 -0000 Received: from unknown (HELO ?10.15.20.85?) (aaron.pratt@mytriggers.com@10.15.20.85) by mail.mytriggers.com with ESMTPA; 29 Dec 2006 16:35:03 -0000 Message-ID: <459543AD.3090809@mytriggers.com> Date: Fri, 29 Dec 2006 11:34:53 -0500 From: Aaron Pratt User-Agent: Thunderbird 1.5.0.9 (Macintosh/20061207) MIME-Version: 1.0 To: freebsd-performance@freebsd.org X-Enigmail-Version: 0.94.1.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Timing/Interrupt issues under load on Dual Xeon Supermicro board X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2006 17:01:45 -0000 My company recently purchased several Silicon Mechanics servers with the Supermicro X7DBR-i+ motherboards. They run the intel 5000P chipset. Two dual-core 3.0ghz processors in each. 3ware SATA RAID controllers (twa). I installed 6-STABLE (6.2-PRERELEASE). Under very heavy CPU load on one or more of the cores (the load is generated by heavy parsing by perl scripts), the system will start throwing a combination of RAID timeouts and calcru negative runtime errors. Most of the servers, which have little-moderate load, have no problems. Here are some examples of the errors: Dec 29 10:20:45 butcher-p13 kernel: twa0: ERROR: (0x05: 0x210B): Request timed out!: request = 0xffffffff881f7610 Dec 29 10:20:45 butcher-p13 kernel: twa0: INFO: (0x16: 0x1108): Resetting controller...: Dec 29 10:20:45 butcher-p13 kernel: twa0: INFO: (0x04: 0x0001): Controller reset occurred: resets=21 Dec 29 10:20:45 butcher-p13 kernel: twa0: INFO: (0x16: 0x1107): Controller reset done!: Dec 29 10:20:45 butcher-p13 kernel: twa0: ERROR: (0x05: 0x210B): Request timed out!: request = 0xffffffff881f36d0 Dec 29 10:20:45 butcher-p13 kernel: twa0: INFO: (0x16: 0x1108): Resetting controller...: Dec 29 10:20:45 butcher-p13 kernel: twa0: INFO: (0x04: 0x0001): Controller reset occurred: resets=22 Dec 29 10:20:45 butcher-p13 kernel: twa0: INFO: (0x16: 0x1107): Controller reset done!: Dec 29 10:20:45 butcher-p13 kernel: twa0: ERROR: (0x05: 0x210B): Request timed out!: request = 0xffffffff881f7400 Dec 29 10:20:45 butcher-p13 kernel: twa0: INFO: (0x16: 0x1108): Resetting controller...: Dec 29 10:20:45 butcher-p13 kernel: twa0: INFO: (0x04: 0x0001): Controller reset occurred: resets=23 Dec 29 10:20:45 butcher-p13 kernel: twa0: INFO: (0x16: 0x1107): Controller reset done!: Dec 29 10:20:45 butcher-p13 kernel: twa0: ERROR: (0x05: 0x210B): Request timed out!: request = 0xffffffff881f59e0 Dec 29 10:20:45 butcher-p13 kernel: twa0: INFO: (0x16: 0x1108): Resetting controller...: Dec 29 10:20:45 butcher-p13 kernel: twa0: INFO: (0x04: 0x0001): Controller reset occurred: resets=24 Dec 29 10:20:45 butcher-p13 kernel: twa0: INFO: (0x16: 0x1107): Controller reset done!: --- Dec 29 09:44:15 butcher-p13 kernel: calcru: runtime went backwards from 244314 usec to 236341 usec for pid 24 (irq19: em1 uhci1) Dec 29 09:44:17 butcher-p13 kernel: calcru: negative runtime of -3744 usec for pid 852 (sshd) Dec 29 09:44:17 butcher-p13 kernel: calcru: runtime went backwards from 244314 usec to 241130 usec for pid 24 (irq19: em1 uhci1) Dec 29 09:44:17 butcher-p13 kernel: calcru: negative runtime of -3744 usec for pid 852 (sshd) Dec 29 09:44:17 butcher-p13 kernel: calcru: runtime went backwards from 244314 usec to 241130 usec for pid 24 (irq19: em1 uhci1) Dec 29 09:44:19 butcher-p13 kernel: calcru: negative runtime of -3583 usec for pid 852 (sshd) Dec 29 09:44:19 butcher-p13 kernel: calcru: runtime went backwards from 244314 usec to 241611 usec for pid 24 (irq19: em1 uhci1) Dec 29 09:44:19 butcher-p13 kernel: calcru: negative runtime of -3583 usec for pid 852 (sshd) Dec 29 09:44:19 butcher-p13 kernel: calcru: runtime went backwards from 244314 usec to 241611 usec for pid 24 (irq19: em1 uhci1) Dec 29 09:44:21 butcher-p13 kernel: calcru: negative runtime of -3486 usec for pid 852 (sshd) Dec 29 09:44:21 butcher-p13 kernel: calcru: negative runtime of -3486 usec for pid 852 (sshd) Dec 29 09:44:23 butcher-p13 kernel: calcru: negative runtime of -3436 usec for pid 852 (sshd) Dec 29 09:44:23 butcher-p13 kernel: calcru: negative runtime of -3436 usec for pid 852 (sshd) Notably, the RAID set suffers no data integrity issues. The timeouts, at least as far as I can tell, have to do an interrupt timing issue. I'm led to believe this because of the accompanying calcru errors and the fact that the processors are under heavy load. Basically anything that's trying to communicate while the CPUs are loaded up suffers. It'll drop SSH sessions, etc. I looked around the forums and list archives and saw several recommendations to check the kern.timecounter.hardware setting. It's running on ACPI-fast. Just for the sake of saying that I tried, I ran the system with both TSC and i8254. The problem persisted. After this, I called Silicon Mechanics who suggested enabling the Multimedia Timer (HPET) in the BIOS. I did this, only to see that 6.2 didn't yet support it. I fired up 7.0-current with the HPET support to no avail. Same issue. After trying this, I went to Supermicro's site and upgraded the BIOS from 1.1c to 1.2a. This wonderful upgrade caused the kernel (7.0) to detect an interrupt storm on em0 and then crash spectacularly (hang). On 6.2, it would just hang within 5 minutes of booting. I was able to downgrade to 1.1b and am back to the baseline. The system doesn't crash, but it pitches the above errors. Just now I tried increasing the kern.hz loader tunable to 2000 and 5000, but the problem still occurred. I figured that it was worth a shot. I haven't been able to test other operating systems with the same load, so I'm not sure if it's something inherent to the board or if FreeBSD just isn't playing nicely. Anything that might lead me in the right direction toward a solution is most appreciated. Thank you, Aaron Pratt myTriggers.com From owner-freebsd-performance@FreeBSD.ORG Fri Dec 29 21:56:59 2006 Return-Path: X-Original-To: freebsd-performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CAEAD16A407 for ; Fri, 29 Dec 2006 21:56:59 +0000 (UTC) (envelope-from dom@helenmarks.co.uk) Received: from mail.goodforbusiness.co.uk (mail.goodforbusiness.co.uk [81.19.179.90]) by mx1.freebsd.org (Postfix) with ESMTP id 8A4EE13C441 for ; Fri, 29 Dec 2006 21:56:59 +0000 (UTC) (envelope-from dom@helenmarks.co.uk) Received: from localhost (localhost [127.0.0.1]) by mail.goodforbusiness.co.uk (Postfix) with ESMTP id BC24811540; Fri, 29 Dec 2006 21:32:32 +0000 (GMT) X-Virus-Scanned: mail.goodforbusiness.co.uk Received: from mail.goodforbusiness.co.uk ([127.0.0.1]) by localhost (mail.goodforbusiness.co.uk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7ZdHnDsGSY8g; Fri, 29 Dec 2006 21:32:32 +0000 (GMT) Received: from mail.helenmarks.co.uk (unknown [192.168.100.1]) by mail.goodforbusiness.co.uk (Postfix) with ESMTP id 0C46311544; Fri, 29 Dec 2006 21:32:32 +0000 (GMT) Received: from localhost (localhost [127.0.0.1]) by mail.helenmarks.co.uk (Postfix) with ESMTP id B24B71795F; Fri, 29 Dec 2006 21:31:47 +0000 (GMT) X-Virus-Scanned: amavisd-new at helenmarks.co.uk Received: from mail.helenmarks.co.uk ([127.0.0.1]) by localhost (mail.helenmarks.co.uk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id notYbLr3lvf7; Fri, 29 Dec 2006 21:31:46 +0000 (GMT) Received: from [127.0.0.1] (unknown [192.168.15.7]) by mail.helenmarks.co.uk (Postfix) with ESMTP id 714031704A; Fri, 29 Dec 2006 21:31:46 +0000 (GMT) Message-ID: <4595893D.6060901@helenmarks.co.uk> Date: Fri, 29 Dec 2006 21:31:41 +0000 From: Dominic Marks User-Agent: Thunderbird 2.0b1 (Windows/20061206) MIME-Version: 1.0 To: "R. B. Riddick" References: <497895.75266.qm@web30313.mail.mud.yahoo.com> In-Reply-To: <497895.75266.qm@web30313.mail.mud.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Benjamin D Adams , freebsd-performance@FreeBSD.org Subject: Re: Best RAID setup X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2006 21:56:59 -0000 Benjamin, Consider that you don't have to raid the entire disc, so you could create make one volume for performance which is stripped over all three discs and have another area for security is a mirror, raid3, raid5, etc. Dom