From owner-freebsd-arch Thu Nov 11 21:18:11 1999 Delivered-To: freebsd-arch@freebsd.org Received: from ns1.yes.no (ns1.yes.no [195.204.136.10]) by hub.freebsd.org (Postfix) with ESMTP id 69B9F14FED for ; Thu, 11 Nov 1999 21:18:08 -0800 (PST) (envelope-from eivind@bitbox.follo.net) Received: from bitbox.follo.net (bitbox.follo.net [195.204.143.218]) by ns1.yes.no (8.9.3/8.9.3) with ESMTP id GAA09945 for ; Fri, 12 Nov 1999 06:18:07 +0100 (CET) Received: (from eivind@localhost) by bitbox.follo.net (8.8.8/8.8.6) id GAA16039 for freebsd-arch@freebsd.org; Fri, 12 Nov 1999 06:18:07 +0100 (MET) Received: from nomis.simon-shapiro.org (nomis.simon-shapiro.org [209.86.126.163]) by hub.freebsd.org (Postfix) with SMTP id 2E3FB14FED for ; Thu, 11 Nov 1999 21:17:58 -0800 (PST) (envelope-from shimon@simon-shapiro.org) Received: (qmail 43261 invoked from network); 12 Nov 1999 05:17:56 -0000 Received: from localhost.simon-shapiro.org (HELO simon-shapiro.org) (127.0.0.1) by localhost.simon-shapiro.org with SMTP; 12 Nov 1999 05:17:56 -0000 Message-ID: <382BA304.EE2F0D66@simon-shapiro.org> Date: Fri, 12 Nov 1999 00:17:56 -0500 From: Simon Shapiro Organization: Simon's Garage X-Mailer: Mozilla 4.6 [en] (X11; I; FreeBSD 3.3-STABLE i386) X-Accept-Language: en-US MIME-Version: 1.0 To: "Kenneth D. Merry" Cc: Randell Jesup , freebsd-arch@freebsd.org Subject: Re: I/O Evaluation Questions (Long but interesting!) References: <199911120444.VAA32051@panzer.kdm.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG "Kenneth D. Merry" wrote: > > Simon Shapiro wrote... > > "Kenneth D. Merry" wrote: > > > > > > Simon Shapiro wrote... > > > > "Kenneth D. Merry" wrote: > > > > > > > > > > [ Simon: the "charset = " (i.e. nothing) line your mail makes my mailer > > > > > barf. You may want to adjust your character set. ] > > > > [ Am using Netscape Messenger. Know not how to do that > > > > (no relevant preference found :-( ] > > > > > > My best guess is, go to: > > > > > > Edit -> Preferences -> Navigator -> Languages > > > > > > And make sure you at least have English defined there. Also, go to: > > > > > > View -> Character Set > > > > > > And make sure you've got Western (ISO-8559-1) defined. > > > > How's that? (sorry for the spam...) > > Much better, thanks. > > > > > > How can you get speeds like that with just a 32-bit PCI bus? The specs for > > > > > the PowerEdge 1300 say it has 5 32-bit PCI slots: > > > > > > > > These numbers are for block devices. The kernel obviously > > > > caches some of this. I should look next time at emory usage; > > > > The machine has 1GB of memory. The dataset is about 15GB per > > > > array. > > > > > > Is that for random or sequential I/O? With sequential I/O, you would > > > probably blow away any caching effects. With random I/O, though, you might > > > get significant help from the cache, especially with that much RAM. > > > > Random, of course. > > Okay, that fits the results. > > > To stay architectually minded, please consider these thoughts: > > > > Increasing the workers load in this test increases measured > > throughput (which is to be expected). However, past about > > 400 concurrent workers, performance declines rapidly. > > At about 600 the system simply goes nuts. Processes exit > > or hang solidly without any warnings. > > Must be some resources to be increased. How is the > > ftp.cdrom.com kernel configured? This may help me. > > wcarchive's configuration might help somewhat (whatever it is), but it is > operating with a very different load than the one you're using. It has > ~~5000 users, and pushes out I think somewhere in the neighborhood of > 100-150Mbits/sec of data. (DG would know for sure.) And it's almost all > reads. > > You're pushing 130-170Mbytes/sec of data, which is about 8 times more, with > a fraction of the processes. > > You may be running into context switch overhead, or who knows what else. > The hangs, though, are not good. > > > > > Raw disks perfromance is totally throttled by physics; > > > > We are running at about 200% of Seagate specs. > > > > > > How can you run at 200% of the spec? Most of the time disk manufacturers > > > are even a little optimistic about their high end performance. > > > > I suspect caching on the disk. I also know the DPT > > firmware, while claiming not to do READ caching, does some > > very interesting things with sorting, queuing, tagging, etc. > > This is worth the difference. More or less. > > > > BTW, I am not looking at claimed benchmarks from the mfgs. > > I am looking at what tends to be accurately reported; > > Sek times, internal transfer rates, data sheets timing > > specs, etc. > > I've found that the transfer rates are sometimes accurate. For instance, > I've got an IBM Ultrastar 9ZX, which IBM claims can do 10-17MB/sec: > > http://www.storage.ibm.com/techsup/hddtech/fedspd.pdf > > That's about right, from what I've seen. The low end may even be a little > lower than the actual performance. > > Another thing you can do is benchmark one disk, and then compare that with > the throughput you get from the array. Done that. It is exactly on the nose with the specs for sequential and the quoted 200% or less for random. > It could be that the combination of the DPT controller's 256MB cache and > fancy queueing, and your 1GB of RAM is causing the amazingly fast disk speeds. These DPTs seem to be optimal for RAID-5, very good at RAID-0 and nothing exciting for single disks. I have some FC-AL gear on order. What worries me is not the perfromance, but the corruption of the stack that I see. For example, I can run the same 400 processes against the raw device all day and all night without a hitch. Run them against a block device and something bizzare happens; A filesystem get corrupted, the Adaptec driver times out, tsleep segfaults, something. At times I can get the error in the driver, but then it makes no sense either. There are tons of self-checks and state verifications in the code. None trip, or when they do they are as illogical as the null pointer inside tsleep. > Ken > -- > Kenneth Merry > ken@kdm.org > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message -- Sincerely Yours, Shimon@Simon-Shapiro.ORG 404.664.6401 Simon Shapiro Unwritten code has no bugs and executes at twice the speed of mouth To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message