From owner-freebsd-current@FreeBSD.ORG Fri Jan 2 08:05:23 2009 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 37CEE106566C for ; Fri, 2 Jan 2009 08:05:23 +0000 (UTC) (envelope-from jjacobson@panasas.com) Received: from laguna.int.panasas.com (gw-ca.panasas.com [66.104.249.162]) by mx1.freebsd.org (Postfix) with ESMTP id 19E668FC1E for ; Fri, 2 Jan 2009 08:05:22 +0000 (UTC) (envelope-from jjacobson@panasas.com) Received: from [172.17.133.160] ([172.17.133.160]) by laguna.int.panasas.com with Microsoft SMTPSVC(6.0.3790.3959); Fri, 2 Jan 2009 00:05:06 -0800 Message-Id: <4B586604-517A-4945-9EED-DC93DD751FB3@panasas.com> From: Joel Jacobson To: Maxim Sobolev In-Reply-To: <495DB269.5030803@FreeBSD.org> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v929.2) Date: Fri, 2 Jan 2009 00:05:09 -0800 References: <3D76E927-AD4F-4B6E-83E8-44379814FD98@panasas.com> <495DB269.5030803@FreeBSD.org> X-Mailer: Apple Mail (2.929.2) X-OriginalArrivalTime: 02 Jan 2009 08:05:06.0542 (UTC) FILETIME=[D2EAD8E0:01C96CB0] Cc: freebsd-current@FreeBSD.org Subject: Re: gstripe performance oddity X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2009 08:05:23 -0000 im already doing what you suggest. here's a simple sample of experiments which probably better describe what im seeing: --------------------------- ca-sbox-2# foreach i (0 1) foreach? dd if=/dev/zero of=/dev/da$i bs=512k count=1024 & foreach? end [1] 5402 [2] 5403 ca-sbox-2# 1024+0 records in 1024+0 records out 536870912 bytes transferred in 4.262723 secs (125945532 bytes/sec) 1024+0 records in 1024+0 records out 536870912 bytes transferred in 4.272499 secs (125657357 bytes/sec) ca-sbox-2# gstripe create -s 262144 d0 /dev/da{0,1} ca-sbox-2# dd if=/dev/zero of=/dev/stripe/d0 bs=512k count=4096 4096+0 records in 4096+0 records out 2147483648 bytes transferred in 34.124683 secs (62930508 bytes/sec) ca-sbox-2# newfs /dev/stripe/d0 > /dev/null ca-sbox-2# mount /dev/stripe/d0 /mnt ca-sbox-2# dd if=/dev/zero of=/mnt/bigfile bs=512k count=4096 && /usr/ bin/time sync 4096+0 records in 4096+0 records out 2147483648 bytes transferred in 11.081184 secs (193795502 bytes/sec) 0.06 real 0.00 user 0.04 sys # sysctl kern.geom kern.geom.collectstats: 1 kern.geom.debugflags: 0 kern.geom.label.debug: 0 kern.geom.stripe.fast_failed: 0 kern.geom.stripe.maxmem: 13107200 kern.geom.stripe.fast: 1 kern.geom.stripe.debug: 0 ---------------------- - j On Jan 1, 2009, at 10:21 PM, Maxim Sobolev wrote: > Joel Jacobson wrote: >> i tried sending this to freebsd-geom but got no response, so i'll >> try a higher traffic superset list... >> i have a bit of a weird issue, which i suspect is a configuration >> problem, and was looking for a little advice. i have an LSI JBOD >> box with a bunch of SAS drives that i would like to gstripe >> together. each drive individually seems to be able to do about 80 >> MB/sec streaming write, and doing parallel dd's gives me the 160 MB/ >> sec i would expect. if i gstripe them together with a 256k stripe >> width, i only see 80 MB/sec, though. >> if, however, i newfs/mount it as ufs and then dd myself a big file, >> that gets me about 120-130 MB/sec. >> why does mounting matter? > > You will see dd write speed increase on stripe only if you are using > block size (bs=XX) larger than stripe size, so that you should try > dd bs=512k or something and see if it helps. > > -Maxim