Date: Thu, 17 Feb 2000 13:22:55 +0100 From: Brad Knowles <blk@skynet.be> To: freebsd-stable@freebsd.org Subject: Initial performance testing w/ postmark & softupdates... Message-ID: <v0422080eb4d1975ec780@[195.238.1.121]>
next in thread | raw e-mail | index | archive | help
Folks, There have been some discussions going on over on the postfix-users mailing list about how much performance is really hurt by synchronous meta-data operations, and therefore how much additional performance you could expect to get out of a system that was designed to try to eliminate them. Well, I've got a Dell 1300 that I've recently installed with FreeBSD 3.4-STABLE, and I thought I'd run some benchmarks. Please keep in mind that these numbers are *very* preliminary, and that they measure only one aspect of the performance of this system. Nevertheless, I find them quite enlightening, and thought that they were good enough that I should share them with you here. Without further ado, here is the message as I sent it to postfix-users: Folks, Some of you were wondering just how much of a difference eliminating synchronous meta-data operations really could make. Well, I just got back my first set of preliminary testing data with the "postmark" benchmark program from Network Appliance, and I thought I'd share it with you. I do not pretend that what I've done so far is an accurate representation of the kind of speedup you should see, or that this is an accurate simulation of how postfix would actually run on this machine. This is just an example (and a preliminary one at that) of the kind of speedup that you might potentially see, and perhaps give you some kind of idea how much of a difference this really can make under some circumstances. If you go to <http://www.shub-internet.org/brad/FreeBSD/postmark.html>, you'll see some data I've collected so far, including both the data from original paper from NetApp, plus some of my own testing. This has not (yet) been updated to incorporate my latest results, which are being summarized here for the first time. The machine I tested on is a Dell 1300 PowerEdge dual-CPU machine with Pentium III processors @ 450Mhz and 1MB L2 cache each, 1GB of ECC RAM, and running FreeBSD 3.4-STABLE (I cvsup'ed on Tuesday, so it's pretty recent ;-). Hardware-wise, the base machine is identical to the "2xP3/FreeBSD" entry that is already in the results at the page I listed above, although that machine was a production server at the time and this machine was tested while otherwise totally idle. The disk drive I've tested with so far is also different -- instead of an IBM model, it's a Quantum Atlas IV 9GB WLS 0707 (80MB/s max), connected to the internal AIC-7890 controller. I tested both with and without softupdates. In the postmark results reported by NetApp, there are three tests that are run, with varying numbers of files to be used and transactions to be performed. The first test uses 1000 files (in the same directory) and 50,000 transactions, the second test uses 20,000 files (in the same directory) and 50,000 transactions, the third test uses 20,000 files (in the same directory) and 100,000 transactions, and at the suggestion of Matt Dillon, I also do a fourth test with 20,000 files and 100,000 transactions, and I also use 100 subdirectories. In this summary that I am posting to this list, I will provide the UFS, TMPFS, and NFS/F630 results, in addition to the ones I just generated. This will help you get a better feel for the kinds of performance we're talking about. The last two colums will be the new results, and I will list the non-softupdates data first ("2xP3/FreeBSD") and the softupdates-enabled data second ("+SOFT"): Test 1: 1000 files & 50,000 transactions UFS TMPFS NFS/F630 2xP3/FreeBSD +SOFT ------ ----- -------- ------------ ------- TPS 36 2000 253 54 458 Read 115.67 4880 799.91 171.77 1495.04 Write 118.27 7330 817.89 175.64 1525.76 Test 2: 20,000 files & 50,000 transactions UFS TMPFS NFS/F630 2xP3/FreeBSD +SOFT ------ ----- -------- ------------ ------- TPS 15 438 176 35 142 Read 29.93 663.64 383.41 67.30 318.81 Write 54.22 1530 694.58 121.91 577.55 Test 3: 20,000 files & 100,000 transactions UFS TMPFS NFS/F630 2xP3/FreeBSD +SOFT ------ ----- -------- ------------ ------- TPS 335 169 35 139 Read 613.03 446.69 85.86 379.79 Write 1160 617.45 118.69 524.98 Test 4: 20,000 files/100 subdirectories & 100,000 transactions UFS TMPFS NFS/F630 2xP3/FreeBSD +SOFT ------ ----- -------- ------------ ------- TPS 25 86 Read 60.92 237.10 Write 84.21 327.75 TPS = Transactions Per Second Read = Data read per second (in kilobytes) Write = Data written per second (in kilobytes) Now, if there is any doubt left in anyone's mind, I'd like to hear it. Please note that I came very close to the performance of the previous generation high-end (and extremely expensive) NFS server box on a *SINGLE* lower-speed commodity hard drive, when I enabled softupdates. I don't guarantee that you'll all see 3-5x performance increases if you enable softupdates on your /var/spool/postfix directories, but this should at least give you some kind of idea of how much of a difference optimizing synchronous meta-data operations can make. Now, just think what you could do if you could re-write the application to avoid the synchronous meta-data operations altogether, instead of depending on the filesystem and the OS to try to optimize them out of existance? I'm going to be doing my own memory-based filesystem checks, to get an idea of what the real upper bound of performance is on this hardware. I'll let you know more as I do it myself. I've also got an extremely expensive mainframe-style drive array that is going to be attached to this machine next week, and I'll run some benchmarks on it as well. I'd be willing to bet that I can beat the snot out of NFS on a softupdates-enabled filesystem that is on a software-striped vinum volume, and I could do it at *much* lower cost, too. In fact, I'd be willing to bet that I could put together an NFS server based on that kind of configuration and that I could beat the snot out of the best that NetApp has to give. This is pretty good stuff. I think I should post this to the freebsd-stable list as well. In fact, I think I will. -- These are my opinions and should not be taken as official Skynet policy _________________________________________________________________________ |o| Brad Knowles, <blk@skynet.be> Belgacom Skynet NV/SA |o| |o| Systems Architect, Mail/News/FTP/Proxy Admin Rue Col. Bourg, 124 |o| |o| Phone/Fax: +32-2-706.13.11/726.93.11 B-1140 Brussels |o| |o| http://www.skynet.be Belgium |o| \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/ Unix is like a wigwam -- no Gates, no Windows, and an Apache inside. Unix is very user-friendly. It's just picky who its friends are. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?v0422080eb4d1975ec780>