From owner-freebsd-fs@FreeBSD.ORG Sun Jan 14 17:25:40 2007 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 592A916A40F; Sun, 14 Jan 2007 17:25:40 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.freebsd.org (Postfix) with ESMTP id DAFA013C441; Sun, 14 Jan 2007 17:25:37 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id DC40545CDA; Sun, 14 Jan 2007 18:25:34 +0100 (CET) Received: from localhost (154.81.datacomsa.pl [195.34.81.154]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 7407C4569A; Sun, 14 Jan 2007 18:25:29 +0100 (CET) Date: Sun, 14 Jan 2007 18:24:45 +0100 From: Pawel Jakub Dawidek To: Norikatsu Shigemura Message-ID: <20070114172445.GL90718@garage.freebsd.pl> References: <20061223111738.6292605a.nork@FreeBSD.org> <20061223142332.GA99259@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="5VuzLDXibKSJvVYD" Content-Disposition: inline In-Reply-To: <20061223142332.GA99259@garage.freebsd.pl> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-2.6 required=3.0 tests=BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@FreeBSD.org Subject: Re: [REPORT] Some panics on ZFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Jan 2007 17:25:40 -0000 --5VuzLDXibKSJvVYD Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Dec 23, 2006 at 03:23:32PM +0100, Pawel Jakub Dawidek wrote: > On Sat, Dec 23, 2006 at 11:17:38AM +0900, Norikatsu Shigemura wrote: > > Other case, but I think that this is current problem:-). > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -= - - > > panic: mi_switch: switch in a critical section > > cpuid =3D 0 > > KDB: enter: panic > > [thread pid 2262 tid 100131 ] > > Stopped at kdb_enter+0x30: leave > > db>=20 > > db>=20 > > db> bt > > Tracing pid 2262 tid 100131 td 0x84f95510 > > kdb_enter(806b5d1c,0,806b66d6,f9fe3b7c,84f95510,...) at kdb_enter+0x30 > > panic(806b66d6,9,806b6699,15a,84f8d510,...) at panic+0x14e > > mi_switch(1,0,806b98ac,285,80730ed4,...) at mi_switch+0xdb > > turnstile_wait(8072ab08,84f8d510,0,166,84f8d512,...) at turnstile_wait+= 0x4c5 > > _mtx_lock_sleep(8072ab08,84f95510,0,806b653d,171,...) at _mtx_lock_slee= p+0x18a > > _mtx_lock_flags(8072ab08,0,806b653d,171,93,...) at _mtx_lock_flags+0xc8 > > _sx_assert(85198cac,4,84f41a97,36,0,...) at _sx_assert+0x10b > > _sx_xunlock(85198cac,84f41a97,36,93,85198cac,...) at _sx_xunlock+0x28 > > cv_timedwait(85198d64,85198cac,1388,5f,0,...) at cv_timedwait+0xd2 > > txg_thread_wait(85198d64,5,85198d5c,85198d54,85198d64,...) at txg_threa= d_wait+0x3f > > txg_timelimit_thread(85198c00,f9fe3d38,806b2c0a,328,84f99900,...) at tx= g_timelimit_thread+0x75 > > fork_exit(84f083c0,85198c00,f9fe3d38) at fork_exit+0xd1 > > fork_trampoline() at fork_trampoline+0x8 > > --- trap 0x1, eip =3D 0, esp =3D 0xf9fe3d6c, ebp =3D 0 --- > > db>=20 > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -= - - >=20 > Not necessarily, it could be a bug in my Solaris compatible convar(9) > based on sx(9) locks. This was my bug and should be fixed now. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --5VuzLDXibKSJvVYD Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFFqmddForvXbEpPzQRAiT/AJ49nv2u20UTrPbK/BrQ2WO7hLnuKQCg4gsf HEBPzk9Aqe5eWXQJV7FPMPk= =1VvR -----END PGP SIGNATURE----- --5VuzLDXibKSJvVYD-- From owner-freebsd-fs@FreeBSD.ORG Mon Jan 15 11:50:47 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3B69616A407 for ; Mon, 15 Jan 2007 11:50:47 +0000 (UTC) (envelope-from joe@tao.org.uk) Received: from mailhost.tao.org.uk (transwarp.tao.org.uk [87.74.4.34]) by mx1.freebsd.org (Postfix) with ESMTP id F109613C467 for ; Mon, 15 Jan 2007 11:50:46 +0000 (UTC) (envelope-from joe@tao.org.uk) Received: from genius.tao.org.uk (wireless58.dhcp.tao.org.uk [87.74.4.58]) by mailhost.tao.org.uk (Postfix) with ESMTP id 7C0645FE5; Mon, 15 Jan 2007 11:21:09 +0000 (GMT) Received: by genius.tao.org.uk (Postfix, from userid 1000) id 0005C40C6; Mon, 15 Jan 2007 11:21:06 +0000 (GMT) Date: Mon, 15 Jan 2007 11:21:06 +0000 From: Josef Karthauser To: stable@freebsd.org, fs@freebsd.org Message-ID: <20070115112106.GA2304@genius.tao.org.uk> Mail-Followup-To: Josef Karthauser , stable@freebsd.org, fs@freebsd.org Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="y0ulUmNC+osPPQO6" Content-Disposition: inline User-Agent: Mutt/1.5.11 Cc: Subject: Dell hardware raid 0 (sas5ir) or gmirror? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jan 2007 11:50:47 -0000 --y0ulUmNC+osPPQO6 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I'm purchasing a new server, and was wondering what anyone thought about whether to pay extra for the SAS5IR card so I can RAID0 the two drives, or whether to just rely on gmirror. My worry about the former is that I can't seem to find management tools for controlling the hardware controller. What if one of the drives fails? How would I know? Joe --=20 =3D=3D=3D Josef Karthauser (joe@tao.org.uk) =3D=3D=3D http://x2obuilder.com= /tao =3D=3D=3D --y0ulUmNC+osPPQO6 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (FreeBSD) iEYEARECAAYFAkWrY6IACgkQXVIcjOaxUBY1pQCeJ2g4Np9/o0mueo1dvmII0AYa 7XkAnR1KrwJKAVdY5IXHekwc2mmpyz2W =CiEN -----END PGP SIGNATURE----- --y0ulUmNC+osPPQO6-- From owner-freebsd-fs@FreeBSD.ORG Mon Jan 15 11:56:54 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 164D716A55B; Mon, 15 Jan 2007 11:56:54 +0000 (UTC) (envelope-from joe@tao.org.uk) Received: from mailhost.tao.org.uk (transwarp.tao.org.uk [87.74.4.34]) by mx1.freebsd.org (Postfix) with ESMTP id C6D4F13C45A; Mon, 15 Jan 2007 11:56:53 +0000 (UTC) (envelope-from joe@tao.org.uk) Received: from genius.tao.org.uk (wireless58.dhcp.tao.org.uk [87.74.4.58]) by mailhost.tao.org.uk (Postfix) with ESMTP id F0EBE5F26; Mon, 15 Jan 2007 11:56:52 +0000 (GMT) Received: by genius.tao.org.uk (Postfix, from userid 1000) id 77E1D40C6; Mon, 15 Jan 2007 11:56:50 +0000 (GMT) Date: Mon, 15 Jan 2007 11:56:50 +0000 From: Josef Karthauser To: stable@freebsd.org, fs@freebsd.org Message-ID: <20070115115650.GB2304@genius.tao.org.uk> Mail-Followup-To: Josef Karthauser , stable@freebsd.org, fs@freebsd.org References: <20070115112106.GA2304@genius.tao.org.uk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="H1spWtNR+x+ondvy" Content-Disposition: inline In-Reply-To: <20070115112106.GA2304@genius.tao.org.uk> User-Agent: Mutt/1.5.11 Cc: Subject: Re: Dell hardware raid 0 (sas5ir) or gmirror? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jan 2007 11:56:54 -0000 --H1spWtNR+x+ondvy Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jan 15, 2007 at 11:21:06AM +0000, Josef Karthauser wrote: > I'm purchasing a new server, and was wondering what anyone thought about > whether to pay extra for the SAS5IR card so I can RAID0 the two drives, > or whether to just rely on gmirror. My worry about the former is that I > can't seem to find management tools for controlling the hardware > controller. What if one of the drives fails? How would I know? >=20 Of course I mean RAID1! J. --=20 =3D=3D=3D Josef Karthauser (joe@tao.org.uk) =3D=3D=3D http://x2obuilder.com= /tao =3D=3D=3D --H1spWtNR+x+ondvy Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (FreeBSD) iEYEARECAAYFAkWrbAIACgkQXVIcjOaxUBa1KQCgyM6zi+c0POXl/wkmzFMo3AaA gIcAn1JaMcGHqX7z0xSty3GUERWfIwNX =g8mj -----END PGP SIGNATURE----- --H1spWtNR+x+ondvy-- From owner-freebsd-fs@FreeBSD.ORG Mon Jan 15 14:27:32 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F1DD816A47E for ; Mon, 15 Jan 2007 14:27:32 +0000 (UTC) (envelope-from infofarmer@gmail.com) Received: from hu-out-0506.google.com (hu-out-0506.google.com [72.14.214.227]) by mx1.freebsd.org (Postfix) with ESMTP id 80D0F13C45E for ; Mon, 15 Jan 2007 14:27:30 +0000 (UTC) (envelope-from infofarmer@gmail.com) Received: by hu-out-0506.google.com with SMTP id 38so788678huc for ; Mon, 15 Jan 2007 06:27:28 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=p6lwtWVRHICTBuPIOruwIjJgk0K30spRhOc4awUclOYBfKk/UYKKmSZI8+vc9rYHZFQ+R+zqM9ERReROCpx9RNLoFBxfAy0aHIph2tVO9T5LATBc9uAFjsZmCHP+JS5EIjssGtdSBjRRO8YYW5BMkti8x95HLo4N4tlp+U2dzwQ= Received: by 10.78.170.17 with SMTP id s17mr2549960hue.1168869644742; Mon, 15 Jan 2007 06:00:44 -0800 (PST) Received: by 10.78.164.20 with HTTP; Mon, 15 Jan 2007 06:00:36 -0800 (PST) Message-ID: Date: Mon, 15 Jan 2007 17:00:36 +0300 From: "Andrew Pantyukhin" Sender: infofarmer@gmail.com To: "Josef Karthauser" , stable@freebsd.org, fs@freebsd.org In-Reply-To: <20070115112106.GA2304@genius.tao.org.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20070115112106.GA2304@genius.tao.org.uk> X-Google-Sender-Auth: 3a2a045d402ba0ea Cc: Subject: Re: Dell hardware raid 0 (sas5ir) or gmirror? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jan 2007 14:27:33 -0000 On 1/15/07, Josef Karthauser wrote: > I'm purchasing a new server, and was wondering what anyone thought about > whether to pay extra for the SAS5IR card so I can RAID0 the two drives, > or whether to just rely on gmirror. My worry about the former is that I > can't seem to find management tools for controlling the hardware > controller. What if one of the drives fails? How would I know? By all means I would go the gmirror way, and I always do even when a hardware raid controller is already present. From owner-freebsd-fs@FreeBSD.ORG Mon Jan 15 15:49:33 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AD8E016A4E1; Mon, 15 Jan 2007 15:49:33 +0000 (UTC) (envelope-from joe@osoft.us) Received: from mail.osoft.us (osoft.us [67.14.192.59]) by mx1.freebsd.org (Postfix) with ESMTP id 4D5EC13C459; Mon, 15 Jan 2007 15:49:33 +0000 (UTC) (envelope-from joe@osoft.us) Received: from [10.100.1.173] (mail.cteh.com [207.218.77.3]) by mail.osoft.us (Postfix) with ESMTP id 2007A33C6E; Mon, 15 Jan 2007 09:21:05 -0600 (CST) Message-ID: <45AB9BE4.1030606@osoft.us> Date: Mon, 15 Jan 2007 09:21:08 -0600 From: Joe Koberg User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) MIME-Version: 1.0 To: Josef Karthauser , stable@freebsd.org, fs@freebsd.org References: <20070115112106.GA2304@genius.tao.org.uk> <20070115115650.GB2304@genius.tao.org.uk> In-Reply-To: <20070115115650.GB2304@genius.tao.org.uk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: Dell hardware raid 0 (sas5ir) or gmirror? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jan 2007 15:49:33 -0000 Josef Karthauser wrote: > On Mon, Jan 15, 2007 at 11:21:06AM +0000, Josef Karthauser wrote: >> I'm purchasing a new server, and was wondering what anyone thought >> about whether to pay extra for the SAS5IR card so I can RAID0 the >> two drives, or whether to just rely on gmirror. My worry about the >> former is that I can't seem to find management tools for >> controlling the hardware controller. What if one of the drives >> fails? How would I know? > > Of course I mean RAID1! > > J. I just bought two Dell PE-1950's to use as routers. They have LSI Logic PERC/5i's attached to 80GB SATA drives. I am pretty sure this is the same card used for SAS. One thing is for sure, the mfi(4) card and driver aren't shy! See below for examples of the kernel messages I get regularly. I am sure drive failure would be well noted. As to rebuilding on-line, I suspect a drive (hot-)swap will initiate the rebuild. There is a CLI tool (megarc) in ports/sysutils/megarc but I haven't tried to use it yet. All in all I would recommend the PERC/5i. Joe Koberg joe at osoft dot us Jan 7 06:41:41 fw2 kernel: mfi0: 316 (4278190093s/0x0008/0) - Battery Present Jan 7 06:41:41 fw2 kernel: mfi0: 317 (4278190110s/0x0004/0) - PD 08(e1/s255) event: Enclosure (SES) discovered on PD 08(e1/s255) Jan 7 06:41:41 fw2 kernel: mfi0: 318 (4278190110s/0x0002/0) - PD 08(e1/s255) event: Inserted: PD 08(e1/s255) Jan 7 06:41:41 fw2 kernel: mfi0: 319 (4278190110s/0x0002/0) - Type 29: Inserted: PD 08(e1/s255) Info: enclPd=08, scsiType=d, portMap=00, sasAddr=500180b04375a600,0000000000000000 Jan 7 06:41:41 fw2 kernel: mfi0: 320 (4278190110s/0x0002/0) - PD 00(e1/s0) event: Inserted: PD 00(e1/s0) Jan 7 06:41:41 fw2 kernel: mfi0: 321 (4278190110s/0x0002/0) - Type 29: Inserted: PD 00(e1/s0) Info: enclPd=08, scsiType=0, portMap=01, sasAddr=1221000000000000,0000000000000000 Jan 7 06:41:41 fw2 kernel: mfi0: 322 (4278190110s/0x0002/0) - PD 01(e1/s1) event: Inserted: PD 01(e1/s1) Jan 7 06:41:41 fw2 kernel: mfi0: 323 (4278190110s/0x0002/0) - Type 29: Inserted: PD 01(e1/s1) Info: enclPd=08, scsiType=0, portMap=02, sasAddr=1221000001000000,0000000000000000 Jan 7 06:41:41 fw2 kernel: mfi0: 325 (4278190110s/0x0001/0) - VD 00/0 event: Background Initialization started on VD 00/0 Jan 7 06:41:41 fw2 kernel: mfi0: 326 (221488579s/0x0020/0) - Adapter ticks 221488579 elapsed 30s: Time established as 01/07/07 12:36:19; (30 seconds since power on) Jan 7 06:41:41 fw2 kernel: mfi0: 330 (4278190080s/0x0020/0) - PCI 0x041028 0x0415 0x041028 0x041f03: Firmware initialization started (PCI ID 0015/1028/1f03/1028) Jan 7 06:41:41 fw2 kernel: mfi0: 331 (4278190080s/0x0020/0) - Type 18: Firmware version 1.00.02-0157 Jan 7 06:41:41 fw2 kernel: mfi0: 332 (4278190096s/0x0008/0) - Battery Present Jan 7 06:41:41 fw2 kernel: mfi0: 333 (4278190113s/0x0004/0) - PD 08(e1/s255) event: Enclosure (SES) discovered on PD 08(e1/s255) Jan 7 06:41:41 fw2 kernel: mfi0: 334 (4278190113s/0x0002/0) - PD 08(e1/s255) event: Inserted: PD 08(e1/s255) Jan 7 06:41:41 fw2 kernel: mfi0: 335 (4278190113s/0x0002/0) - Type 29: Inserted: PD 08(e1/s255) Info: enclPd=08, scsiType=d, portMap=00, sasAddr=500180b04375a600,0000000000000000 Jan 7 06:41:41 fw2 kernel: mfi0: 336 (4278190113s/0x0002/0) - PD 00(e1/s0) event: Inserted: PD 00(e1/s0) Jan 7 06:41:41 fw2 kernel: mfi0: 337 (4278190113s/0x0002/0) - Type 29: Inserted: PD 00(e1/s0) Info: enclPd=08, scsiType=0, portMap=01, sasAddr=1221000000000000,0000000000000000 Jan 7 06:41:41 fw2 kernel: mfi0: 338 (4278190113s/0x0002/0) - PD 01(e1/s1) event: Inserted: PD 01(e1/s1) Jan 7 06:41:41 fw2 kernel: mfi0: 339 (4278190113s/0x0002/0) - Type 29: Inserted: PD 01(e1/s1) Info: enclPd=08, scsiType=0, portMap=02, sasAddr=1221000001000000,0000000000000000 Jan 7 06:41:41 fw2 kernel: mfi0: 341 (4278190113s/0x0001/0) - VD 00/0 event: Background Initialization started on VD 00/0 Jan 7 06:41:41 fw2 kernel: mfi0: 342 (221488658s/0x0020/0) - Adapter ticks 221488658 elapsed 33s: Time established as 01/07/07 12:37:38; (33 seconds since power on) Jan 7 06:41:41 fw2 kernel: mfi0: 347 (221488716s/0x0008/0) - Battery temperature is normal Jan 7 06:41:41 fw2 kernel: mfi0: 348 (221488716s/0x0008/0) - Battery started charging Jan 7 06:41:41 fw2 kernel: mfi0: 349 (221488716s/0x0008/0) - Current capacity of the battery is above threshold Jan 7 06:41:41 fw2 kernel: mfi0: 350 (4278190080s/0x0020/0) - PCI 0x041028 0x0415 0x041028 0x041f03: Firmware initialization started (PCI ID 0015/1028/1f03/1028) Jan 7 06:41:41 fw2 kernel: mfi0: 351 (4278190080s/0x0020/0) - Type 18: Firmware version 1.00.02-0157 Jan 7 06:41:41 fw2 kernel: mfi0: 352 (4278190096s/0x0008/0) - Battery Present Jan 7 06:41:41 fw2 kernel: mfi0: 353 (4278190113s/0x0004/0) - PD 08(e1/s255) event: Enclosure (SES) discovered on PD 08(e1/s255) Jan 7 06:41:41 fw2 kernel: mfi0: 354 (4278190113s/0x0002/0) - PD 08(e1/s255) event: Inserted: PD 08(e1/s255) Jan 7 06:41:41 fw2 kernel: mfi0: 355 (4278190113s/0x0002/0) - Type 29: Inserted: PD 08(e1/s255) Info: enclPd=08, scsiType=d, portMap=00, sasAddr=500180b04375a600,0000000000000000 Jan 7 06:41:41 fw2 kernel: mfi0: 356 (4278190113s/0x0002/0) - PD 00(e1/s0) event: Inserted: PD 00(e1/s0) Jan 7 06:41:41 fw2 kernel: mfi0: 357 (4278190113s/0x0002/0) - Type 29: Inserted: PD 00(e1/s0) Info: enclPd=08, scsiType=0, portMap=01, sasAddr=1221000000000000,0000000000000000 Jan 7 06:41:41 fw2 kernel: mfi0: 358 (4278190113s/0x0002/0) - PD 01(e1/s1) event: Inserted: PD 01(e1/s1) Jan 7 06:41:41 fw2 kernel: mfi0: 359 (4278190113s/0x0002/0) - Type 29: Inserted: PD 01(e1/s1) Info: enclPd=08, scsiType=0, portMap=02, sasAddr=1221000001000000,0000000000000000 Jan 7 06:41:41 fw2 kernel: mfi0: 361 (4278190114s/0x0001/0) - VD 00/0 event: Background Initialization started on VD 00/0 Jan 7 06:41:41 fw2 kernel: mfi0: 362 (221488767s/0x0020/0) - Adapter ticks 221488767 elapsed 34s: Time established as 01/07/07 12:39:27; (34 seconds since power on) Jan 7 06:41:41 fw2 kernel: mfi0: 366 (4278190080s/0x0020/0) - PCI 0x041028 0x0415 0x041028 0x041f03: Firmware initialization started (PCI ID 0015/1028/1f03/1028) Jan 7 06:41:41 fw2 kernel: mfi0: 367 (4278190080s/0x0020/0) - Type 18: Firmware version 1.00.02-0157 Jan 7 06:41:41 fw2 kernel: mfi0: 368 (4278190095s/0x0008/0) - Battery Present Jan 7 06:41:41 fw2 kernel: mfi0: 369 (4278190113s/0x0004/0) - PD 08(e1/s255) event: Enclosure (SES) discovered on PD 08(e1/s255) Jan 7 06:41:41 fw2 kernel: mfi0: 370 (4278190113s/0x0002/0) - PD 08(e1/s255) event: Inserted: PD 08(e1/s255) Jan 7 06:41:41 fw2 kernel: mfi0: 371 (4278190113s/0x0002/0) - Type 29: Inserted: PD 08(e1/s255) Info: enclPd=08, scsiType=d, portMap=00, sasAddr=500180b04375a600,0000000000000000 Jan 7 06:41:41 fw2 kernel: mfi0: 372 (4278190113s/0x0002/0) - PD 00(e1/s0) event: Inserted: PD 00(e1/s0) Jan 7 06:41:41 fw2 kernel: mfi0: 373 (4278190113s/0x0002/0) - Type 29: Inserted: PD 00(e1/s0) Info: enclPd=08, scsiType=0, portMap=01, sasAddr=1221000000000000,0000000000000000 Jan 7 06:41:41 fw2 kernel: mfi0: 374 (4278190113s/0x0002/0) - PD 01(e1/s1) event: Inserted: PD 01(e1/s1) Jan 7 06:41:41 fw2 kernel: mfi0: 375 (4278190113s/0x0002/0) - Type 29: Inserted: PD 01(e1/s1) Info: enclPd=08, scsiType=0, portMap=02, sasAddr=1221000001000000,0000000000000000 Jan 7 06:41:41 fw2 kernel: mfi0: 377 (4278190114s/0x0001/0) - VD 00/0 event: Background Initialization started on VD 00/0 Jan 7 06:41:41 fw2 kernel: mfi0: 378 (221488853s/0x0020/0) - Adapter ticks 221488853 elapsed 34s: Time established as 01/07/07 12:40:53; (34 seconds since power on) Jan 7 06:41:41 fw2 kernel: mfid0: on mfi0 Jan 7 06:41:42 fw2 kernel: mfi0: 382 (221488903s/0x0001/-1) - VD 00/0 progress 85% in 50s: Background Initialization progress on VD 00/0 is 85.81%(50s) Jan 7 06:41:48 fw2 kernel: mfi0: 383 (221488909s/0x0008/0) - Battery temperature is normal Jan 7 06:41:48 fw2 kernel: mfi0: 384 (221488909s/0x0008/0) - Battery started charging Jan 7 06:41:48 fw2 kernel: mfi0: 385 (221488909s/0x0008/0) - Current capacity of the battery is above threshold Jan 7 06:41:58 fw2 kernel: mfi0: 386 (221488919s/0x0001/-1) - VD 00/0 progress 86% in 66s: Background Initialization progress on VD 00/0 is 86.28%(66s) Jan 7 06:42:14 fw2 kernel: mfi0: 387 (221488935s/0x0001/-1) - VD 00/0 progress 86% in 82s: Background Initialization progress on VD 00/0 is 86.66%(82s) Jan 7 06:42:30 fw2 kernel: mfi0: 388 (221488951s/0x0001/-1) - VD 00/0 progress 87% in 98s: Background Initialization progress on VD 00/0 is 87.21%(98s) Jan 7 06:42:46 fw2 kernel: mfi0: 389 (221488967s/0x0001/-1) - VD 00/0 progress 87% in 114s: Background Initialization progress on VD 00/0 is 87.65%(114s) Jan 7 06:43:02 fw2 kernel: mfi0: 390 (221488983s/0x0001/-1) - VD 00/0 progress 88% in 130s: Background Initialization progress on VD 00/0 is 88.27%(130s) Jan 7 06:43:18 fw2 kernel: mfi0: 391 (221488999s/0x0001/-1) - VD 00/0 progress 88% in 146s: Background Initialization progress on VD 00/0 is 88.86%(146s) Jan 7 06:43:34 fw2 kernel: mfi0: 392 (221489015s/0x0001/-1) - VD 00/0 progress 89% in 162s: Background Initialization progress on VD 00/0 is 89.48%(162s) Jan 7 06:43:50 fw2 kernel: mfi0: 393 (221489031s/0x0001/-1) - VD 00/0 progress 90% in 178s: Background Initialization progress on VD 00/0 is 90.10%(178s) Jan 7 06:44:06 fw2 kernel: mfi0: 394 (221489047s/0x0001/-1) - VD 00/0 progress 90% in 194s: Background Initialization progress on VD 00/0 is 90.65%(194s) Jan 7 06:44:22 fw2 kernel: mfi0: 395 (221489063s/0x0001/-1) - VD 00/0 progress 91% in 210s: Background Initialization progress on VD 00/0 is 91.30%(210s) Jan 7 06:44:38 fw2 kernel: mfi0: 396 (221489079s/0x0001/-1) - VD 00/0 progress 91% in 226s: Background Initialization progress on VD 00/0 is 91.83%(226s) Jan 7 06:44:54 fw2 kernel: mfi0: 397 (221489095s/0x0001/-1) - VD 00/0 progress 92% in 242s: Background Initialization progress on VD 00/0 is 92.37%(242s) Jan 7 06:45:10 fw2 kernel: mfi0: 398 (221489111s/0x0001/-1) - VD 00/0 progress 92% in 258s: Background Initialization progress on VD 00/0 is 92.91%(258s) Jan 7 06:45:26 fw2 kernel: mfi0: 399 (221489127s/0x0001/-1) - VD 00/0 progress 93% in 274s: Background Initialization progress on VD 00/0 is 93.42%(274s) Jan 7 06:45:42 fw2 kernel: mfi0: 400 (221489143s/0x0001/-1) - VD 00/0 progress 93% in 290s: Background Initialization progress on VD 00/0 is 93.93%(290s) Jan 7 06:45:58 fw2 kernel: mfi0: 401 (221489159s/0x0001/-1) - VD 00/0 progress 94% in 306s: Background Initialization progress on VD 00/0 is 94.38%(306s) Jan 7 06:46:14 fw2 kernel: mfi0: 402 (221489175s/0x0001/-1) - VD 00/0 progress 94% in 322s: Background Initialization progress on VD 00/0 is 94.84%(322s) Jan 7 06:46:30 fw2 kernel: mfi0: 403 (221489191s/0x0001/-1) - VD 00/0 progress 95% in 338s: Background Initialization progress on VD 00/0 is 95.41%(338s) Jan 7 06:46:46 fw2 kernel: mfi0: 404 (221489207s/0x0001/-1) - VD 00/0 progress 96% in 354s: Background Initialization progress on VD 00/0 is 95.96%(354s) Jan 7 06:47:02 fw2 kernel: mfi0: 405 (221489223s/0x0001/-1) - VD 00/0 progress 96% in 370s: Background Initialization progress on VD 00/0 is 96.43%(370s) Jan 7 06:47:18 fw2 kernel: mfi0: 406 (221489239s/0x0001/-1) - VD 00/0 progress 97% in 386s: Background Initialization progress on VD 00/0 is 96.95%(386s) Jan 7 06:47:34 fw2 kernel: mfi0: 407 (221489255s/0x0001/-1) - VD 00/0 progress 97% in 402s: Background Initialization progress on VD 00/0 is 97.38%(402s) Jan 7 06:47:51 fw2 kernel: mfi0: 408 (221489271s/0x0001/-1) - VD 00/0 progress 97% in 418s: Background Initialization progress on VD 00/0 is 97.86%(418s) Jan 7 06:48:06 fw2 kernel: mfi0: 409 (221489287s/0x0001/-1) - VD 00/0 progress 98% in 434s: Background Initialization progress on VD 00/0 is 98.26%(434s) Jan 7 06:48:22 fw2 kernel: mfi0: 410 (221489303s/0x0001/-1) - VD 00/0 progress 98% in 450s: Background Initialization progress on VD 00/0 is 98.68%(450s) Jan 7 06:48:38 fw2 kernel: mfi0: 411 (221489319s/0x0001/-1) - VD 00/0 progress 99% in 466s: Background Initialization progress on VD 00/0 is 98.98%(466s) Jan 7 06:48:55 fw2 kernel: mfi0: 412 (221489335s/0x0001/-1) - VD 00/0 progress 99% in 482s: Background Initialization progress on VD 00/0 is 99.29%(482s) Jan 7 06:49:11 fw2 kernel: mfi0: 413 (221489352s/0x0001/-1) - VD 00/0 progress 99% in 499s: Background Initialization progress on VD 00/0 is 99.48%(499s) Jan 7 06:49:27 fw2 kernel: mfi0: 414 (221489368s/0x0001/-1) - VD 00/0 progress 99% in 515s: Background Initialization progress on VD 00/0 is 99.80%(515s) Jan 7 06:49:35 fw2 kernel: mfi0: 415 (221489376s/0x0001/0) - VD 00/0 event: Background Initialization completed on VD 00/0 Jan 7 13:08:00 fw2 kernel: mfi0: mem 0xd80f0000-0xd80fffff,0xfc6e0000-0xfc6fffff irq 78 at device 14.0 on pci2 Jan 7 13:08:00 fw2 kernel: mfi0: 416 (221490360s/0x0020/0) - Shutdown command received from host Jan 7 13:08:00 fw2 kernel: mfi0: 417 (4278190080s/0x0020/0) - PCI 0x041028 0x0415 0x041028 0x041f03: Firmware initialization started (PCI ID 0015/1028/1f03/1028) Jan 7 13:08:00 fw2 kernel: mfi0: 418 (4278190080s/0x0020/0) - Type 18: Firmware version 1.00.02-0157 Jan 7 13:08:00 fw2 kernel: mfi0: 419 (4278190093s/0x0008/0) - Battery Present Jan 7 13:08:00 fw2 kernel: mfi0: 420 (4278190110s/0x0004/0) - PD 08(e1/s255) event: Enclosure (SES) discovered on PD 08(e1/s255) Jan 7 13:08:00 fw2 kernel: mfi0: 421 (4278190110s/0x0002/0) - PD 08(e1/s255) event: Inserted: PD 08(e1/s255) Jan 7 13:08:00 fw2 kernel: mfi0: 422 (4278190110s/0x0002/0) - Type 29: Inserted: PD 08(e1/s255) Info: enclPd=08, scsiType=d, portMap=00, sasAddr=500180b04375a600,0000000000000000 Jan 7 13:08:00 fw2 kernel: mfi0: 423 (4278190110s/0x0002/0) - PD 00(e1/s0) event: Inserted: PD 00(e1/s0) Jan 7 13:08:00 fw2 kernel: mfi0: 424 (4278190110s/0x0002/0) - Type 29: Inserted: PD 00(e1/s0) Info: enclPd=08, scsiType=0, portMap=01, sasAddr=1221000000000000,0000000000000000 Jan 7 13:08:00 fw2 kernel: mfi0: 425 (4278190110s/0x0002/0) - PD 01(e1/s1) event: Inserted: PD 01(e1/s1) Jan 7 13:08:00 fw2 kernel: mfi0: 426 (4278190110s/0x0002/0) - Type 29: Inserted: PD 01(e1/s1) Info: enclPd=08, scsiType=0, portMap=02, sasAddr=1221000001000000,0000000000000000 Jan 7 13:08:00 fw2 kernel: mfi0: 427 (221512017s/0x0020/0) - Adapter ticks 221512017 elapsed 30s: Time established as 01/07/07 19:06:57; (30 seconds since power on) Jan 7 13:08:00 fw2 kernel: mfi0: 428 (221512075s/0x0008/0) - Battery temperature is normal Jan 7 13:08:00 fw2 kernel: mfi0: 429 (221512075s/0x0008/0) - Battery started charging Jan 7 13:08:00 fw2 kernel: mfi0: 430 (221512075s/0x0008/0) - Current capacity of the battery is above threshold Jan 7 13:08:00 fw2 kernel: mfid0: on mfi0 Jan 7 13:14:33 fw2 kernel: mfi0: mem 0xd80f0000-0xd80fffff,0xfc6e0000-0xfc6fffff irq 78 at device 14.0 on pci2 Jan 7 13:14:33 fw2 kernel: mfi0: 431 (221512357s/0x0020/0) - Shutdown command received from host Jan 7 13:14:33 fw2 kernel: mfi0: 432 (4278190080s/0x0020/0) - PCI 0x041028 0x0415 0x041028 0x041f03: Firmware initialization started (PCI ID 0015/1028/1f03/1028) Jan 7 13:14:33 fw2 kernel: mfi0: 433 (4278190080s/0x0020/0) - Type 18: Firmware version 1.00.02-0157 Jan 7 13:14:33 fw2 kernel: mfi0: 434 (4278190092s/0x0008/0) - Battery Present Jan 7 13:14:33 fw2 kernel: mfi0: 435 (4278190110s/0x0004/0) - PD 08(e1/s255) event: Enclosure (SES) discovered on PD 08(e1/s255) Jan 7 13:14:33 fw2 kernel: mfi0: 436 (4278190111s/0x0002/0) - PD 08(e1/s255) event: Inserted: PD 08(e1/s255) Jan 7 13:14:33 fw2 kernel: mfi0: 437 (4278190111s/0x0002/0) - Type 29: Inserted: PD 08(e1/s255) Info: enclPd=08, scsiType=d, portMap=00, sasAddr=500180b04375a600,0000000000000000 Jan 7 13:14:33 fw2 kernel: mfi0: 438 (4278190111s/0x0002/0) - PD 00(e1/s0) event: Inserted: PD 00(e1/s0) Jan 7 13:14:33 fw2 kernel: mfi0: 439 (4278190111s/0x0002/0) - Type 29: Inserted: PD 00(e1/s0) Info: enclPd=08, scsiType=0, portMap=01, sasAddr=1221000000000000,0000000000000000 Jan 7 13:14:33 fw2 kernel: mfi0: 440 (4278190111s/0x0002/0) - PD 01(e1/s1) event: Inserted: PD 01(e1/s1) Jan 7 13:14:33 fw2 kernel: mfi0: 441 (4278190111s/0x0002/0) - Type 29: Inserted: PD 01(e1/s1) Info: enclPd=08, scsiType=0, portMap=02, sasAddr=1221000001000000,0000000000000000 Jan 7 13:14:33 fw2 kernel: mfi0: 442 (221512408s/0x0020/0) - Adapter ticks 221512408 elapsed 31s: Time established as 01/07/07 19:13:28; (31 seconds since power on) Jan 7 13:14:33 fw2 kernel: mfi0: 443 (221512464s/0x0008/0) - Battery temperature is normal Jan 7 13:14:33 fw2 kernel: mfi0: 444 (221512464s/0x0008/0) - Battery started charging Jan 7 13:14:33 fw2 kernel: mfi0: 445 (221512464s/0x0008/0) - Current capacity of the battery is above threshold Jan 7 13:14:33 fw2 kernel: mfid0: on mfi0 Jan 7 15:41:56 fw2 kernel: mfi0: 446 (221521304s/0x0008/0) - Battery charge complete Jan 12 21:11:38 fw2 kernel: mfi0: 447 (221972400s/0x0020/0) - Patrol Read started Jan 12 21:14:00 fw2 kernel: mfi0: 448 (221972542s/0x0002/-1) - PD 01(e1/s1) progress 10% seconds 142s: Patrol Read progress on PD 01(e1/s1) is 10.02%(142s) Jan 12 21:14:00 fw2 kernel: mfi0: 449 (221972542s/0x0002/-1) - PD 00(e1/s0) progress 10% seconds 142s: Patrol Read progress on PD 00(e1/s0) is 10.02%(142s) Jan 12 21:16:24 fw2 kernel: mfi0: 450 (221972685s/0x0002/-1) - PD 01(e1/s1) progress 20% seconds 285s: Patrol Read progress on PD 01(e1/s1) is 20.00%(285s) Jan 12 21:16:26 fw2 kernel: mfi0: 451 (221972688s/0x0002/-1) - PD 00(e1/s0) progress 20% seconds 288s: Patrol Read progress on PD 00(e1/s0) is 20.00%(288s) Jan 12 21:18:49 fw2 kernel: mfi0: 452 (221972830s/0x0002/-1) - PD 01(e1/s1) progress 30% seconds 430s: Patrol Read progress on PD 01(e1/s1) is 30.03%(430s) Jan 12 21:18:52 fw2 kernel: mfi0: 453 (221972833s/0x0002/-1) - PD 00(e1/s0) progress 30% seconds 433s: Patrol Read progress on PD 00(e1/s0) is 30.03%(433s) Jan 12 21:21:20 fw2 kernel: mfi0: 454 (221972981s/0x0002/-1) - PD 01(e1/s1) progress 40% seconds 581s: Patrol Read progress on PD 01(e1/s1) is 40.01%(581s) Jan 12 21:21:23 fw2 kernel: mfi0: 455 (221972984s/0x0002/-1) - PD 00(e1/s0) progress 40% seconds 584s: Patrol Read progress on PD 00(e1/s0) is 40.01%(584s) Jan 12 21:23:59 fw2 kernel: mfi0: 456 (221973140s/0x0002/-1) - PD 01(e1/s1) progress 50% seconds 740s: Patrol Read progress on PD 01(e1/s1) is 49.99%(740s) Jan 12 21:24:02 fw2 kernel: mfi0: 457 (221973143s/0x0002/-1) - PD 00(e1/s0) progress 50% seconds 743s: Patrol Read progress on PD 00(e1/s0) is 49.99%(743s) Jan 12 21:26:43 fw2 kernel: mfi0: 458 (221973304s/0x0002/-1) - PD 01(e1/s1) progress 60% seconds 904s: Patrol Read progress on PD 01(e1/s1) is 60.02%(904s) Jan 12 21:26:47 fw2 kernel: mfi0: 459 (221973307s/0x0002/-1) - PD 00(e1/s0) progress 60% seconds 907s: Patrol Read progress on PD 00(e1/s0) is 60.02%(907s) Jan 12 21:29:30 fw2 kernel: mfi0: 460 (221973470s/0x0002/-1) - PD 01(e1/s1) progress 70% seconds 1070s: Patrol Read progress on PD 01(e1/s1) is 70.00%(1070s) Jan 12 21:29:33 fw2 kernel: mfi0: 461 (221973473s/0x0002/-1) - PD 00(e1/s0) progress 70% seconds 1073s: Patrol Read progress on PD 00(e1/s0) is 70.00%(1073s) Jan 12 21:32:37 fw2 kernel: mfi0: 462 (221973657s/0x0002/-1) - PD 01(e1/s1) progress 80% seconds 1257s: Patrol Read progress on PD 01(e1/s1) is 80.03%(1257s) Jan 12 21:32:40 fw2 kernel: mfi0: 463 (221973660s/0x0002/-1) - PD 00(e1/s0) progress 80% seconds 1260s: Patrol Read progress on PD 00(e1/s0) is 80.03%(1260s) Jan 12 21:36:16 fw2 kernel: mfi0: 464 (221973875s/0x0002/-1) - PD 01(e1/s1) progress 90% seconds 1475s: Patrol Read progress on PD 01(e1/s1) is 90.01%(1475s) Jan 12 21:36:19 fw2 kernel: mfi0: 465 (221973878s/0x0002/-1) - PD 00(e1/s0) progress 90% seconds 1478s: Patrol Read progress on PD 00(e1/s0) is 90.01%(1478s) Jan 12 21:40:21 fw2 kernel: mfi0: 466 (221974120s/0x0002/-1) - PD 01(e1/s1) progress 100% seconds 1720s: Patrol Read progress on PD 01(e1/s1) is 100.00%(1720s) Jan 12 21:40:24 fw2 kernel: mfi0: 467 (221974123s/0x0002/-1) - PD 00(e1/s0) progress 100% seconds 1723s: Patrol Read progress on PD 00(e1/s0) is 100.00%(1723s) Jan 12 21:40:24 fw2 kernel: mfi0: 468 (221974123s/0x0020/0) - Patrol Read complete Jan 15 08:47:14 fw2 kernel: mfi0: mem 0xd80f0000-0xd80fffff,0xfc6e0000-0xfc6fffff irq 78 at device 14.0 on pci2 Jan 15 08:47:14 fw2 kernel: mfi0: 483 (222187518s/0x0020/0) - Shutdown command received from host Jan 15 08:47:14 fw2 kernel: mfi0: 484 (4278190080s/0x0020/0) - PCI 0x041028 0x0415 0x041028 0x041f03: Firmware initialization started (PCI ID 0015/1028/1f03/1028) Jan 15 08:47:14 fw2 kernel: mfi0: 485 (4278190080s/0x0020/0) - Type 18: Firmware version 1.00.02-0157 Jan 15 08:47:14 fw2 kernel: mfi0: 486 (4278190092s/0x0008/0) - Battery Present Jan 15 08:47:14 fw2 kernel: mfi0: 487 (4278190109s/0x0004/0) - PD 08(e1/s255) event: Enclosure (SES) discovered on PD 08(e1/s255) Jan 15 08:47:14 fw2 kernel: mfi0: 488 (4278190109s/0x0002/0) - PD 08(e1/s255) event: Inserted: PD 08(e1/s255) Jan 15 08:47:14 fw2 kernel: mfi0: 489 (4278190109s/0x0002/0) - Type 29: Inserted: PD 08(e1/s255) Info: enclPd=08, scsiType=d, portMap=00, sasAddr=500180b04375a600,0000000000000000 Jan 15 08:47:14 fw2 kernel: mfi0: 490 (4278190109s/0x0002/0) - PD 00(e1/s0) event: Inserted: PD 00(e1/s0) Jan 15 08:47:14 fw2 kernel: mfi0: 491 (4278190109s/0x0002/0) - Type 29: Inserted: PD 00(e1/s0) Info: enclPd=08, scsiType=0, portMap=01, sasAddr=1221000000000000,0000000000000000 Jan 15 08:47:14 fw2 kernel: mfi0: 492 (4278190109s/0x0002/0) - PD 01(e1/s1) event: Inserted: PD 01(e1/s1) Jan 15 08:47:14 fw2 kernel: mfi0: 493 (4278190109s/0x0002/0) - Type 29: Inserted: PD 01(e1/s1) Info: enclPd=08, scsiType=0, portMap=02, sasAddr=1221000001000000,0000000000000000 Jan 15 08:47:14 fw2 kernel: mfi0: 494 (222187559s/0x0020/0) - Adapter ticks 222187559 elapsed 30s: Time established as 01/15/07 14:45:59; (30 seconds since power on) Jan 15 08:47:14 fw2 kernel: mfi0: 495 (222187616s/0x0008/0) - Battery temperature is normal Jan 15 08:47:14 fw2 kernel: mfi0: 496 (222187616s/0x0008/0) - Current capacity of the battery is above threshold From owner-freebsd-fs@FreeBSD.ORG Mon Jan 15 19:28:10 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AC9C516A40F; Mon, 15 Jan 2007 19:28:10 +0000 (UTC) (envelope-from rees@umich.edu) Received: from citi.umich.edu (citi.umich.edu [141.211.133.111]) by mx1.freebsd.org (Postfix) with ESMTP id 8D3D113C465; Mon, 15 Jan 2007 19:28:10 +0000 (UTC) (envelope-from rees@umich.edu) Received: from localhost (dsl093-001-248.det1.dsl.speakeasy.net [66.93.1.248]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by citi.umich.edu (Postfix) with ESMTP id 92A6139081; Mon, 15 Jan 2007 13:54:26 -0500 (EST) Date: Mon, 15 Jan 2007 13:54:26 -0500 From: Jim Rees To: stable@freebsd.org, fs@freebsd.org Message-ID: <20070115185425.GB23427@citi.umich.edu> References: <20070115112106.GA2304@genius.tao.org.uk> <20070115115650.GB2304@genius.tao.org.uk> <45AB9BE4.1030606@osoft.us> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <45AB9BE4.1030606@osoft.us> Cc: Subject: Re: Dell hardware raid 0 (sas5ir) or gmirror? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jan 2007 19:28:10 -0000 We've had nothing but trouble with various models of Adaptec RAID hardware, and now only use software RAID even where hardware is available. But we only run a handful of servers. From owner-freebsd-fs@FreeBSD.ORG Mon Jan 15 20:10:58 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DFD5A16A417 for ; Mon, 15 Jan 2007 20:10:58 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id A3D2C13C428 for ; Mon, 15 Jan 2007 20:10:58 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.4/8.13.4) with ESMTP id l0FJXWka055853; Mon, 15 Jan 2007 12:33:37 -0700 (MST) (envelope-from scottl@samsco.org) Message-ID: <45ABD709.90809@samsco.org> Date: Mon, 15 Jan 2007 12:33:29 -0700 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.2pre) Gecko/20070111 SeaMonkey/1.1 MIME-Version: 1.0 To: Jim Rees References: <20070115112106.GA2304@genius.tao.org.uk> <20070115115650.GB2304@genius.tao.org.uk> <45AB9BE4.1030606@osoft.us> <20070115185425.GB23427@citi.umich.edu> In-Reply-To: <20070115185425.GB23427@citi.umich.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (pooker.samsco.org [168.103.85.57]); Mon, 15 Jan 2007 12:33:37 -0700 (MST) X-Spam-Status: No, score=-1.4 required=3.8 tests=ALL_TRUSTED autolearn=failed version=3.1.1 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on pooker.samsco.org Cc: stable@freebsd.org, fs@freebsd.org Subject: Re: Dell hardware raid 0 (sas5ir) or gmirror? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jan 2007 20:10:59 -0000 Jim Rees wrote: > We've had nothing but trouble with various models of Adaptec RAID hardware, > and now only use software RAID even where hardware is available. But we > only run a handful of servers. Adaptec RAID doesn't apply to this discussion. Scott From owner-freebsd-fs@FreeBSD.ORG Mon Jan 15 21:43:38 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 32BB716A47C for ; Mon, 15 Jan 2007 21:43:38 +0000 (UTC) (envelope-from rodrigc@crodrigues.org) Received: from rwcrmhc15.comcast.net (rwcrmhc15.comcast.net [216.148.227.155]) by mx1.freebsd.org (Postfix) with ESMTP id 1D2B313C471 for ; Mon, 15 Jan 2007 21:43:38 +0000 (UTC) (envelope-from rodrigc@crodrigues.org) Received: from c-66-31-35-94.hsd1.ma.comcast.net ([66.31.35.94]) by comcast.net (rwcrmhc15) with ESMTP id <20070115213254m15005s9gbe>; Mon, 15 Jan 2007 21:32:54 +0000 Received: from c-66-31-35-94.hsd1.ma.comcast.net (localhost.crodrigues.org [127.0.0.1]) by c-66-31-35-94.hsd1.ma.comcast.net (8.13.8/8.13.8) with ESMTP id l0FLX19m060130; Mon, 15 Jan 2007 16:33:01 -0500 (EST) (envelope-from rodrigc@c-66-31-35-94.hsd1.ma.comcast.net) Received: (from rodrigc@localhost) by c-66-31-35-94.hsd1.ma.comcast.net (8.13.8/8.13.8/Submit) id l0FLX10b060129; Mon, 15 Jan 2007 16:33:01 -0500 (EST) (envelope-from rodrigc) Date: Mon, 15 Jan 2007 16:33:01 -0500 From: Craig Rodrigues To: Adam K Kirchhoff Message-ID: <20070115213301.GA60097@crodrigues.org> References: <1168363312.1450.9.camel@memory.visualtech.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1168363312.1450.9.camel@memory.visualtech.com> User-Agent: Mutt/1.4.2.1i Cc: freebsd-fs@FreeBSD.org Subject: Re: Mounting an xfs drive X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jan 2007 21:43:38 -0000 On Tue, Jan 09, 2007 at 12:21:52PM -0500, Adam K Kirchhoff wrote: > So I was hoping to see if the same is try for the FreeBSD version of > XFS. Again, FreeBSD sees the drive as /dev/da0, but no partitions or > slices show up. Much as I expected, trying to mount /dev/da0 directly > fails with "Operating not permitted". Before I continue fighting with What is the output of: ls /dev/da0* file - < /dev/da0 mount -t xfs /dev/da0 /mnt -- Craig Rodrigues rodrigc@crodrigues.org From owner-freebsd-fs@FreeBSD.ORG Mon Jan 15 22:30:04 2007 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 267C316A4CE for ; Mon, 15 Jan 2007 22:30:04 +0000 (UTC) (envelope-from adamk@voicenet.com) Received: from b.mx.visualtech.com (b.mx.visualtech.com [208.16.19.9]) by mx1.freebsd.org (Postfix) with ESMTP id BEAC513C457 for ; Mon, 15 Jan 2007 22:30:03 +0000 (UTC) (envelope-from adamk@voicenet.com) Received: from [192.168.0.101] (c-68-45-180-45.hsd1.nj.comcast.net [68.45.180.45]) by b.mx.visualtech.com (Postfix) with ESMTP id 033ED4839D; Mon, 15 Jan 2007 16:53:32 -0500 (EST) Message-ID: <45ABF7DF.6070508@voicenet.com> Date: Mon, 15 Jan 2007 16:53:35 -0500 From: Adam K Kirchhoff User-Agent: Thunderbird 1.5.0.9 (X11/20070103) MIME-Version: 1.0 To: Craig Rodrigues References: <1168363312.1450.9.camel@memory.visualtech.com> <20070115213301.GA60097@crodrigues.org> In-Reply-To: <20070115213301.GA60097@crodrigues.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org Subject: Re: Mounting an xfs drive X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jan 2007 22:30:04 -0000 Craig Rodrigues wrote: > On Tue, Jan 09, 2007 at 12:21:52PM -0500, Adam K Kirchhoff wrote: > >> So I was hoping to see if the same is try for the FreeBSD version of >> XFS. Again, FreeBSD sees the drive as /dev/da0, but no partitions or >> slices show up. Much as I expected, trying to mount /dev/da0 directly >> fails with "Operating not permitted". Before I continue fighting with >> > > What is the output of: > ls /dev/da0* > file - < /dev/da0 > mount -t xfs /dev/da0 /mnt > > > Craig, We've pretty much put getting the drives connected on hold for the time being. My focus now is on getting our StorageTek 9714 connected to another server so I can grab data off of our backup tapes :-) If I have the time, in a bit, I'll try hooking up one of those drives again. Thanks! Adam From owner-freebsd-fs@FreeBSD.ORG Tue Jan 16 09:31:37 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5AC0A16A407; Tue, 16 Jan 2007 09:31:37 +0000 (UTC) (envelope-from stefan.lambrev@sun-fish.com) Received: from sun-fish.com (blah.sun-fish.com [217.18.249.150]) by mx1.freebsd.org (Postfix) with ESMTP id D30B013C448; Tue, 16 Jan 2007 09:31:36 +0000 (UTC) (envelope-from stefan.lambrev@sun-fish.com) Received: from sun-fish.com (localhost.cmotd.com [127.0.0.1]) by sun-fish.com (Postfix) with ESMTP id 2DA95385BC; Tue, 16 Jan 2007 10:00:06 +0100 (CET) Received: from [192.168.3.125] (cheffo-mobile.cmotd.com [192.168.3.125]) by sun-fish.com (Postfix) with ESMTP id EABFA3859F; Tue, 16 Jan 2007 10:00:05 +0100 (CET) Message-ID: <45AC9418.60802@sun-fish.com> Date: Tue, 16 Jan 2007 11:00:08 +0200 From: Stefan Lambrev User-Agent: Thunderbird 1.5.0.9 (X11/20061230) MIME-Version: 1.0 To: Andrew Pantyukhin References: <20070115112106.GA2304@genius.tao.org.uk> In-Reply-To: Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: 7bit X-AV-Checked: ClamAV Cc: Josef Karthauser , stable@freebsd.org, fs@freebsd.org Subject: Re: Dell hardware raid 0 (sas5ir) or gmirror? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 09:31:37 -0000 Andrew Pantyukhin wrote: > On 1/15/07, Josef Karthauser wrote: >> I'm purchasing a new server, and was wondering what anyone thought about >> whether to pay extra for the SAS5IR card so I can RAID0 the two drives, >> or whether to just rely on gmirror. My worry about the former is that I >> can't seem to find management tools for controlling the hardware >> controller. What if one of the drives fails? How would I know? > > By all means I would go the gmirror way, and I always do > even when a hardware raid controller is already present. I really do not understand this. :) When you say something like this it will be good to explain why you think so. I have few servers with good hw raid controllers and I'm very happy with them, I also use gmirror on my desktop pc, but it is not as good as hw raid on servers for sure. Also it is harder to support it (during OS updates and etc). Also it (gmirror) will put some load on the CPU and hw raid have it's own CPU/memory for this. LSI have a nice tool to monitor/config RAID arrays that just works under fbsd in my case so I'm happy with it. There are a lot of reasons to use hw raids on mission critical servers... > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" -- Best Wishes, Stefan Lambrev ICQ# 24134177 From owner-freebsd-fs@FreeBSD.ORG Tue Jan 16 10:01:18 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 79E6816A415 for ; Tue, 16 Jan 2007 10:01:18 +0000 (UTC) (envelope-from infofarmer@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.173]) by mx1.freebsd.org (Postfix) with ESMTP id DD3B213C4BA for ; Tue, 16 Jan 2007 10:01:17 +0000 (UTC) (envelope-from infofarmer@gmail.com) Received: by ug-out-1314.google.com with SMTP id o2so1477729uge for ; Tue, 16 Jan 2007 02:01:16 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=tk8b8boDwZDZDlNyZVj5u3VORCXzFwEYkkyHGFom3yclwgt881r2ilGqqWyWIY+nrLj3IH65TiAZmrUTlOP7Ld6G926mSPOTvuX63fX5UZROdQFxBwzfd/yJhckUuqdlBBvXKmOFxTNh7oQl5DHLIROhtGbxpmi8xSJlFKgC1JA= Received: by 10.78.204.1 with SMTP id b1mr3318778hug.1168941675713; Tue, 16 Jan 2007 02:01:15 -0800 (PST) Received: by 10.78.164.20 with HTTP; Tue, 16 Jan 2007 02:01:15 -0800 (PST) Message-ID: Date: Tue, 16 Jan 2007 13:01:15 +0300 From: "Andrew Pantyukhin" Sender: infofarmer@gmail.com To: "Stefan Lambrev" In-Reply-To: <45AC9418.60802@sun-fish.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20070115112106.GA2304@genius.tao.org.uk> <45AC9418.60802@sun-fish.com> X-Google-Sender-Auth: 8a0c701bb50b143c Cc: Josef Karthauser , stable@freebsd.org, fs@freebsd.org Subject: Re: Dell hardware raid 0 (sas5ir) or gmirror? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 10:01:18 -0000 On 1/16/07, Stefan Lambrev wrote: > Andrew Pantyukhin wrote: > > On 1/15/07, Josef Karthauser wrote: > >> I'm purchasing a new server, and was wondering what anyone thought about > >> whether to pay extra for the SAS5IR card so I can RAID0 the two drives, > >> or whether to just rely on gmirror. My worry about the former is that I > >> can't seem to find management tools for controlling the hardware > >> controller. What if one of the drives fails? How would I know? > > > > By all means I would go the gmirror way, and I always do > > even when a hardware raid controller is already present. > > I really do not understand this. :) > When you say something like this it will be good to explain why you > think so. > I have few servers with good hw raid controllers and I'm very happy with > them, I also use > gmirror on my desktop pc, but it is not as good as hw raid on servers > for sure. > Also it is harder to support it (during OS updates and etc). > Also it (gmirror) will put some load on the CPU and hw raid have it's > own CPU/memory for this. > LSI have a nice tool to monitor/config RAID arrays that just works under > fbsd in my case so I'm happy with it. > There are a lot of reasons to use hw raids on mission critical servers... As a matter of fact Jonathan was also surprised by my answer. Here's a part of my response to him: ===================================================== raid3, raid5 and other computation-hungry configu- rations are a cpu hog, that's why people prefer hardware controllers for that. I'm quite sure at some point FreeBSD will gain an ability to use crypto/XOR hardware for the benefit of software raid performance and maybe then software raid5 will become a popular solution. As for raid0/raid1 - there's no cpu penalty at all. gmirror/gstripe in FreeBSD might need further tweaks and optimizations, but benchmarks show that with 2-4 drives performance almost equals the theoretical limits. Reliability of OS-integrated software raid is expected to be even higher than that of hardware one, because there's no hardware to fail and software bugs might be found in all solutions. What I really like about software raid is very high flexibility and manageability. There's no issue of having the right driver or the right userland tool, it just works. And it's a snap to setup. And you are free to experiment with virtual (file-based, for one) file systems before you implement a solution. As always, there are more than one correct opinions. I just expressed my own and I hope my explanation answers some of your interest. ===================================================== What I would add to answer some of your claims, Stefan, is that there's no single correct solution here. I would argue that money spent on main CPU/RAM are a better investment compared to a hardware raid 0/1 solution, OS buffers are there anyway, so why not make them larger/faster if you need that. As for CPU load, I'd argue it's negligible, at least in 2 SATA HDD situations (which are popular in all markets). For larger configurations, consisting of 5 drives and more I would advise against raid 0/1 and therefore against software raid. If you do continue to use gmirror/gstripe, I would expect some tweaks to be needed, but in general such systems should scale very well, especially on SMP systems, as FreeBSD 6.x brought mpsafe file system access to the table. All in all there are reasons to use hw raids and there are some not to use them. For some reasons I hold our homegrown (FreeBSD) solutions closer to my heart and choose them in favor of 3d-party ones. Cheers! From owner-freebsd-fs@FreeBSD.ORG Tue Jan 16 20:21:50 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6297F16A412; Tue, 16 Jan 2007 20:21:50 +0000 (UTC) (envelope-from ambrisko@ambrisko.com) Received: from mail.ambrisko.com (mail.ambrisko.com [64.174.51.43]) by mx1.freebsd.org (Postfix) with ESMTP id 361EF13C471; Tue, 16 Jan 2007 20:21:50 +0000 (UTC) (envelope-from ambrisko@ambrisko.com) Received: from server2.ambrisko.com (HELO www.ambrisko.com) ([192.168.1.2]) by mail.ambrisko.com with ESMTP; 16 Jan 2007 11:49:57 -0800 Received: from ambrisko.com (localhost [127.0.0.1]) by www.ambrisko.com (8.13.1/8.12.11) with ESMTP id l0GJr8Vi058179; Tue, 16 Jan 2007 11:53:08 -0800 (PST) (envelope-from ambrisko@ambrisko.com) Received: (from ambrisko@localhost) by ambrisko.com (8.13.1/8.13.1/Submit) id l0GJr77J058178; Tue, 16 Jan 2007 11:53:07 -0800 (PST) (envelope-from ambrisko) From: Doug Ambrisko Message-Id: <200701161953.l0GJr77J058178@ambrisko.com> In-Reply-To: <45AB9BE4.1030606@osoft.us> To: Joe Koberg Date: Tue, 16 Jan 2007 11:53:07 -0800 (PST) X-Mailer: ELM [version 2.4ME+ PL94b (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Cc: Josef Karthauser , stable@freebsd.org, fs@freebsd.org Subject: Re: Dell hardware raid 0 (sas5ir) or gmirror? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2007 20:21:50 -0000 Joe Koberg writes: | Josef Karthauser wrote: | > On Mon, Jan 15, 2007 at 11:21:06AM +0000, Josef Karthauser wrote: | >> I'm purchasing a new server, and was wondering what anyone thought | >> about whether to pay extra for the SAS5IR card so I can RAID0 the | >> two drives, or whether to just rely on gmirror. My worry about the | >> former is that I can't seem to find management tools for | >> controlling the hardware controller. What if one of the drives | >> fails? How would I know? | > | > Of course I mean RAID1! | | I just bought two Dell PE-1950's to use as routers. They have LSI Logic | PERC/5i's attached to 80GB SATA drives. I am pretty sure this is the | same card used for SAS. | | One thing is for sure, the mfi(4) card and driver aren't shy! See below | for examples of the kernel messages I get regularly. I am sure drive | failure would be well noted. FYI, you can silence it to your level of comfort via: hw.mfi.event_class in /boot/loader.conf. The values being: MFI_EVT_CLASS_DEBUG = -2, MFI_EVT_CLASS_PROGRESS = -1, MFI_EVT_CLASS_INFO = 0, MFI_EVT_CLASS_WARNING = 1, MFI_EVT_CLASS_CRITICAL = 2, MFI_EVT_CLASS_FATAL = 3, MFI_EVT_CLASS_DEAD = 4 The new default is info. so it's a little quieter. I'd suggest some care in going over info since a drive that failed will come through but when it is now okay will not. So if you are waiting for that you won't know. Here, we like the debug and progress stuff put into /var/log/messages. It makes support a lot easier. Doug A. From owner-freebsd-fs@FreeBSD.ORG Wed Jan 17 10:39:41 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0A0F416A415; Wed, 17 Jan 2007 10:39:41 +0000 (UTC) (envelope-from joe@tao.org.uk) Received: from mailhost.tao.org.uk (transwarp.tao.org.uk [87.74.4.34]) by mx1.freebsd.org (Postfix) with ESMTP id BBE9013C44C; Wed, 17 Jan 2007 10:39:40 +0000 (UTC) (envelope-from joe@tao.org.uk) Received: from genius.tao.org.uk (wireless58.dhcp.tao.org.uk [87.74.4.58]) by mailhost.tao.org.uk (Postfix) with ESMTP id C9E1F5E7B; Wed, 17 Jan 2007 10:39:39 +0000 (GMT) Received: by genius.tao.org.uk (Postfix, from userid 1000) id 20E7740D4; Wed, 17 Jan 2007 10:39:36 +0000 (GMT) Date: Wed, 17 Jan 2007 10:39:36 +0000 From: Josef Karthauser To: stable@freebsd.org, fs@freebsd.org Message-ID: <20070117103935.GC4018@genius.tao.org.uk> Mail-Followup-To: Josef Karthauser , stable@freebsd.org, fs@freebsd.org Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="UPT3ojh+0CqEDtpF" Content-Disposition: inline User-Agent: Mutt/1.5.11 Cc: Subject: gmirror disks vs partitions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2007 10:39:41 -0000 --UPT3ojh+0CqEDtpF Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable A poll for opinions if I may? I've got a few gmirrors running on various machines, all of which pair up two drives at the physical level (i.e. mirror /dev/ad0s1 with /dev/ad1s1). Of course there are other ways of doing it to, like mirroring at the partition level, ie pairing /dev/ad0s1a with /dev/ad1s1a, /dev/ad0s1e with /dev/ad0s1e, etc. Apart from potentially avoiding a whole disk from being copied during a resync after a crash, are there any other advantages to using partition level mirroring instead of drive level mirroring? Joe --=20 =3D=3D=3D Josef Karthauser (joe@tao.org.uk) =3D=3D=3D http://x2obuilder.com= /tao =3D=3D=3D --UPT3ojh+0CqEDtpF Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (FreeBSD) iEYEARECAAYFAkWt/OcACgkQXVIcjOaxUBYz9QCfeHRf+ThY17gJP8ZK9jLyenWL vA8AoIICM4nRF/P/Ldqtpe7TPvdKss5p =edWw -----END PGP SIGNATURE----- --UPT3ojh+0CqEDtpF-- From owner-freebsd-fs@FreeBSD.ORG Wed Jan 17 11:05:08 2007 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 08F7116A412 for ; Wed, 17 Jan 2007 11:05:07 +0000 (UTC) (envelope-from trhodes@FreeBSD.org) Received: from chipmunk.ai.net (axe.ai.net [205.134.161.26]) by mx1.freebsd.org (Postfix) with ESMTP id AF1BC13C480 for ; Wed, 17 Jan 2007 11:05:07 +0000 (UTC) (envelope-from trhodes@FreeBSD.org) Received: from localhost (net-ix.gw.ai.net [205.134.160.6] (may be forged)) by chipmunk.ai.net (8.13.4/8.13.4) with SMTP id l0HAlsVN084486 for ; Wed, 17 Jan 2007 05:47:54 -0500 (EST) (envelope-from trhodes@FreeBSD.org) Date: Wed, 17 Jan 2007 05:47:43 -0500 From: Tom Rhodes To: freebsd-fs@FreeBSD.org Message-Id: <20070117054743.1922dd6e.trhodes@FreeBSD.org> In-Reply-To: <20070117103935.GC4018@genius.tao.org.uk> References: <20070117103935.GC4018@genius.tao.org.uk> Organization: The FreeBSD Project X-Mailer: Sylpheed version 1.0.6 (GTK+ 1.2.10; i386-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Subject: Re: gmirror disks vs partitions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2007 11:05:08 -0000 On Wed, 17 Jan 2007 10:39:36 +0000 Josef Karthauser wrote: > A poll for opinions if I may? > > I've got a few gmirrors running on various machines, all of which > pair up two drives at the physical level (i.e. mirror /dev/ad0s1 > with /dev/ad1s1). Of course there are other ways of doing it to, > like mirroring at the partition level, ie pairing /dev/ad0s1a with > /dev/ad1s1a, /dev/ad0s1e with /dev/ad0s1e, etc. > > Apart from potentially avoiding a whole disk from being copied > during a resync after a crash, are there any other advantages to > using partition level mirroring instead of drive level mirroring? Would you replace the disk anyway? -- Tom Rhodes From owner-freebsd-fs@FreeBSD.ORG Wed Jan 17 11:29:36 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AF3A216A415 for ; Wed, 17 Jan 2007 11:29:36 +0000 (UTC) (envelope-from infofarmer@gmail.com) Received: from wr-out-0506.google.com (wr-out-0506.google.com [64.233.184.238]) by mx1.freebsd.org (Postfix) with ESMTP id 33BB213C45E for ; Wed, 17 Jan 2007 11:29:36 +0000 (UTC) (envelope-from infofarmer@gmail.com) Received: by wr-out-0506.google.com with SMTP id 71so415878wri for ; Wed, 17 Jan 2007 03:29:35 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=PurYKBumgiT3rJnhvCOL4RR0/Rf7FBR3pzcTwSwMlVU+rD/7T+ow3PhlcQuDXhtYiglu43qJE2XPK2jWjzXSGXdTGcI6UAvXgjvv7rwR29rQwxxQ6Pe1NisAlnKQJpRU5jjaI99mSuEanq3vVJc93k3hQED1qOcEbfwGa2ibTuE= Received: by 10.78.18.3 with SMTP id 3mr1599076hur.1169033374082; Wed, 17 Jan 2007 03:29:34 -0800 (PST) Received: by 10.78.164.20 with HTTP; Wed, 17 Jan 2007 03:29:33 -0800 (PST) Message-ID: Date: Wed, 17 Jan 2007 14:29:33 +0300 From: "Andrew Pantyukhin" Sender: infofarmer@gmail.com To: "Josef Karthauser" , stable@freebsd.org, fs@freebsd.org In-Reply-To: <20070117103935.GC4018@genius.tao.org.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20070117103935.GC4018@genius.tao.org.uk> X-Google-Sender-Auth: 7cc1e77383c11a74 Cc: Subject: Re: gmirror disks vs partitions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2007 11:29:36 -0000 On 1/17/07, Josef Karthauser wrote: > A poll for opinions if I may? > > I've got a few gmirrors running on various machines, all of which > pair up two drives at the physical level (i.e. mirror /dev/ad0s1 > with /dev/ad1s1). Of course there are other ways of doing it to, > like mirroring at the partition level, ie pairing /dev/ad0s1a with > /dev/ad1s1a, /dev/ad0s1e with /dev/ad0s1e, etc. > > Apart from potentially avoiding a whole disk from being copied > during a resync after a crash, are there any other advantages to > using partition level mirroring instead of drive level mirroring? I can imagine people using partition-level raid to implement a popular configuration: You divide a couple of identical drives proportionally in two partitions each, place a couple of the first partitions into gmirror and a couple of the second ones into gstripe. This way you get both reliable and fast storage with just two drives. Some strings are attached. From owner-freebsd-fs@FreeBSD.ORG Wed Jan 17 13:27:57 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3526516A40F; Wed, 17 Jan 2007 13:27:57 +0000 (UTC) (envelope-from xenophon@irtnog.org) Received: from mx1.irtnog.org (24-123-13-51.irtnog.org [24.123.13.51]) by mx1.freebsd.org (Postfix) with ESMTP id 08CDB13C441; Wed, 17 Jan 2007 13:27:57 +0000 (UTC) (envelope-from xenophon@irtnog.org) Received: from localhost (unknown [127.0.0.1]) by mx1.irtnog.org (Postfix) with ESMTP id 92FF9114C2; Wed, 17 Jan 2007 08:08:10 -0500 (EST) X-Virus-Scanned: amavisd-new at irtnog.org Received: from mx1.irtnog.org ([127.0.0.1]) by localhost (cinep010bsdmx.irtnog.org [127.0.0.1]) (amavisd-new, port 10024) with LMTP id MiXwham2Kyfk; Wed, 17 Jan 2007 08:08:09 -0500 (EST) Received: from irtnog.org (svr1.irtnog.org [10.63.0.100]) by mx1.irtnog.org (Postfix) with ESMTP id A8308114B0; Wed, 17 Jan 2007 08:08:08 -0500 (EST) Date: Wed, 17 Jan 2007 08:08:07 -0500 Message-ID: MIME-Version: 1.0 In-Reply-To: <20070117103935.GC4018@genius.tao.org.uk> Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-class: urn:content-classes:message Thread-Topic: gmirror disks vs partitions thread-index: Acc6LBJEiqkdBg8xRn+XxrMs5f5oSQAAlUkg References: <20070117103935.GC4018@genius.tao.org.uk> X-MimeOLE: Produced By Microsoft Exchange V6.5 From: "Matthew X. Economou" To: , Cc: Subject: RE: gmirror disks vs partitions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2007 13:27:57 -0000 > Apart from potentially avoiding a whole disk from being copied > during a resync after a crash, are there any other advantages to > using partition level mirroring instead of drive level mirroring? Joe, Partition-level software RAID plus LVM is how the following Slashdot poster manages extendable (and inequally sized disk) arrays on Linux: http://ask.slashdot.org/comments.pl?sid=3D169386&cid=3D14117414 Best wishes, Matthew --=20 "Rogues are very keen in their profession, and know already much more than we can teach them respecting their several kinds of roguery." - A. C. Hobbs in _Locks and Safes_ (1853) From owner-freebsd-fs@FreeBSD.ORG Wed Jan 17 14:03:35 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3EBF916A415 for ; Wed, 17 Jan 2007 14:03:35 +0000 (UTC) (envelope-from infofarmer@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.251]) by mx1.freebsd.org (Postfix) with ESMTP id F18EE13C465 for ; Wed, 17 Jan 2007 14:03:34 +0000 (UTC) (envelope-from infofarmer@gmail.com) Received: by an-out-0708.google.com with SMTP id c24so1216298ana for ; Wed, 17 Jan 2007 06:03:32 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=uJotzmoQDHRpPRsb8PrM50DhJQy2v8ftjWIXY9oQxEhTel8ztL0U7J20Qw6pQeS+/oByNTCOKaQXdac+niJYU0OMXXNeUR4IqOQSGJ4+LPEzPpfw7JKnUKpqT80BsXNsOhr7Q4kHMqGev0hPkCI9OSo81zIjYUPpXeO3hE/bIcE= Received: by 10.78.204.20 with SMTP id b20mr4479629hug.1169042607803; Wed, 17 Jan 2007 06:03:27 -0800 (PST) Received: by 10.78.164.20 with HTTP; Wed, 17 Jan 2007 06:03:26 -0800 (PST) Message-ID: Date: Wed, 17 Jan 2007 17:03:26 +0300 From: "Andrew Pantyukhin" Sender: infofarmer@gmail.com To: "Matthew X. Economou" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20070117103935.GC4018@genius.tao.org.uk> X-Google-Sender-Auth: ac5d49ab91dba309 Cc: stable@freebsd.org, fs@freebsd.org Subject: Re: gmirror disks vs partitions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2007 14:03:35 -0000 On 1/17/07, Matthew X. Economou wrote: > > Apart from potentially avoiding a whole disk from being copied > > during a resync after a crash, are there any other advantages to > > using partition level mirroring instead of drive level mirroring? > > Joe, > > Partition-level software RAID plus LVM is how the following Slashdot > poster manages extendable (and inequally sized disk) arrays on Linux: > > http://ask.slashdot.org/comments.pl?sid=169386&cid=14117414 [...after reading the slashdotter's piece of wisdom...] Yes, but that's the kind of functionality I have always expected to be present in software raid solutions. I hope I'll live to see this implemented in geom. From owner-freebsd-fs@FreeBSD.ORG Wed Jan 17 16:04:29 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D3D8016A407; Wed, 17 Jan 2007 16:04:29 +0000 (UTC) (envelope-from lists@jnielsen.net) Received: from ns1.jnielsen.net (ns1.jnielsen.net [69.55.238.237]) by mx1.freebsd.org (Postfix) with ESMTP id A978513C442; Wed, 17 Jan 2007 16:04:29 +0000 (UTC) (envelope-from lists@jnielsen.net) Received: from localhost (jn@ns1 [69.55.238.237]) (authenticated bits=0) by ns1.jnielsen.net (8.12.9p2/8.12.9) with ESMTP id l0HFdHcG046003; Wed, 17 Jan 2007 07:39:17 -0800 (PST) (envelope-from lists@jnielsen.net) From: John Nielsen To: freebsd-stable@freebsd.org Date: Wed, 17 Jan 2007 10:36:21 -0500 User-Agent: KMail/1.9.5 References: <20070117103935.GC4018@genius.tao.org.uk> In-Reply-To: X-Face: #X5#Y*q>F:]zT!DegL3z5Xo'^MN[$8k\[4^3rN~wm=s=Uw(sW}R?3b^*f1Wu*.<=?utf-8?q?of=5F4NrS=0A=09P*M/9CpxDo!D6?=)IY1w<9B1jB; tBQf[RU-R<,I)e"$q7N7 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200701171036.21534.lists@jnielsen.net> X-Virus-Scanned: ClamAV version 0.88.4, clamav-milter version 0.88.4 on ns1.jnielsen.net X-Virus-Status: Clean Cc: Josef Karthauser , fs@freebsd.org Subject: Re: gmirror disks vs partitions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2007 16:04:29 -0000 On Wednesday 17 January 2007 06:29, Andrew Pantyukhin wrote: > On 1/17/07, Josef Karthauser wrote: > > A poll for opinions if I may? > > > > I've got a few gmirrors running on various machines, all of which > > pair up two drives at the physical level (i.e. mirror /dev/ad0s1 > > with /dev/ad1s1). Of course there are other ways of doing it to, > > like mirroring at the partition level, ie pairing /dev/ad0s1a with > > /dev/ad1s1a, /dev/ad0s1e with /dev/ad0s1e, etc. > > > > Apart from potentially avoiding a whole disk from being copied > > during a resync after a crash, are there any other advantages to > > using partition level mirroring instead of drive level mirroring? > > I can imagine people using partition-level raid to > implement a popular configuration: > > You divide a couple of identical drives proportionally > in two partitions each, place a couple of the first > partitions into gmirror and a couple of the second > ones into gstripe. This way you get both reliable and > fast storage with just two drives. Some strings are > attached. The reduced likelihood of needing to rebuild a given volume is usually enough of an argument for me to mirror at the partition level. Of course, the other side of the coin is that if more than one volume on a given pair of disks needs to be rebuilt, the disks will be twice (or more) as hammered (and less efficient due to the greater number of seeks) during the rebuild(s). If you want to be creative/exotic then it's sometimes useful to use partitions as building blocks for odd (or "advanced") volume configurations. For instance, let's say you're trying to get some disk redundancy for your workstation but you're limited to whatever drives you can scrounge up. (Have _I_ ever been in this position? nah... :) ) You have a 40GB disk, a 60GB disk, and an 80GB disk. If you partition them up right and use gmirror with gstripe, it's possible to use all of the space and still be able to survive the failure of any one disk. Divide everything up into partitions of equal sizes. For an even number of disks you can use the GCD of the sizes as the partition size, but since there's an odd number of disks in this example we'll use GCD/2 or ~10GB. Pair one partition on the 40GB disk with one on the 60GB disk. Then pair all of the partitions on the 80GB disk with the remaining partitions on the 40 and 60 GB disks. Make each pair into a gmirror volume. If you need to boot from the array, pick one pair to be your system volume. The rest of the gmirrors can all be added into a gstripe volume, so you end up with 90GB (or 80+10) of redundant storage with quite good performance (not that I would know, of course). You can use the leftover bits for swap, etc. The two drawbacks to this approach vs a two-disk mirror are increased likelihood of drive failure (due to the greater number of disks) and a more complex recovery procedure if a drive fails (especially if you don't have a spare identical to or slightly larger than the one that failed). Just some thoughts.. JN From owner-freebsd-fs@FreeBSD.ORG Wed Jan 17 22:18:41 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7C4B516A492; Wed, 17 Jan 2007 22:18:41 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 3AC3A13C44B; Wed, 17 Jan 2007 22:18:40 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.4/8.13.4) with ESMTP id l0HMIY8Z076188; Wed, 17 Jan 2007 15:18:40 -0700 (MST) (envelope-from scottl@samsco.org) Message-ID: <45AEA0B5.8060903@samsco.org> Date: Wed, 17 Jan 2007 15:18:29 -0700 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.2pre) Gecko/20070111 SeaMonkey/1.1 MIME-Version: 1.0 To: Andrew Pantyukhin References: <20070117103935.GC4018@genius.tao.org.uk> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (pooker.samsco.org [168.103.85.57]); Wed, 17 Jan 2007 15:18:40 -0700 (MST) X-Spam-Status: No, score=-1.4 required=3.8 tests=ALL_TRUSTED autolearn=failed version=3.1.1 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on pooker.samsco.org Cc: Josef Karthauser , stable@freebsd.org, fs@freebsd.org Subject: Re: gmirror disks vs partitions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2007 22:18:41 -0000 Andrew Pantyukhin wrote: > On 1/17/07, Josef Karthauser wrote: >> A poll for opinions if I may? >> >> I've got a few gmirrors running on various machines, all of which >> pair up two drives at the physical level (i.e. mirror /dev/ad0s1 >> with /dev/ad1s1). Of course there are other ways of doing it to, >> like mirroring at the partition level, ie pairing /dev/ad0s1a with >> /dev/ad1s1a, /dev/ad0s1e with /dev/ad0s1e, etc. >> >> Apart from potentially avoiding a whole disk from being copied >> during a resync after a crash, are there any other advantages to >> using partition level mirroring instead of drive level mirroring? > > I can imagine people using partition-level raid to > implement a popular configuration: > > You divide a couple of identical drives proportionally > in two partitions each, place a couple of the first > partitions into gmirror and a couple of the second > ones into gstripe. This way you get both reliable and > fast storage with just two drives. Some strings are > attached. The head movement that this causes makes it a poor performer. It is an option, but not a terribly popular one. Scott From owner-freebsd-fs@FreeBSD.ORG Wed Jan 17 22:58:41 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0CCDF16A416 for ; Wed, 17 Jan 2007 22:58:41 +0000 (UTC) (envelope-from infofarmer@gmail.com) Received: from nz-out-0506.google.com (nz-out-0506.google.com [64.233.162.227]) by mx1.freebsd.org (Postfix) with ESMTP id AA81313C465 for ; Wed, 17 Jan 2007 22:58:40 +0000 (UTC) (envelope-from infofarmer@gmail.com) Received: by nz-out-0506.google.com with SMTP id i11so11536nzh for ; Wed, 17 Jan 2007 14:58:37 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=TDFhuBcsnt0WrMbw9O7v6+Z7AWdexiJf/zAqMEHSQM9OGM1trc8RqT3eTStuHctC+ZaGXVfjedWWFfm0AUPeBHAaLQt9868xf2ifGrDYMDSlktN6aCoQygI1vzJV4mArhwkEwM0LUhRptjlRELufk7qcKQVm/mRvgQ8DzZwBWH4= Received: by 10.78.160.2 with SMTP id i2mr108597hue.1169074716698; Wed, 17 Jan 2007 14:58:36 -0800 (PST) Received: by 10.78.164.20 with HTTP; Wed, 17 Jan 2007 14:58:36 -0800 (PST) Message-ID: Date: Thu, 18 Jan 2007 01:58:36 +0300 From: "Andrew Pantyukhin" Sender: infofarmer@gmail.com To: "Scott Long" In-Reply-To: <45AEA0B5.8060903@samsco.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20070117103935.GC4018@genius.tao.org.uk> <45AEA0B5.8060903@samsco.org> X-Google-Sender-Auth: 002ddf904bfc6632 Cc: Josef Karthauser , stable@freebsd.org, fs@freebsd.org Subject: Re: gmirror disks vs partitions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2007 22:58:41 -0000 On 1/18/07, Scott Long wrote: > Andrew Pantyukhin wrote: > > On 1/17/07, Josef Karthauser wrote: > >> A poll for opinions if I may? > >> > >> I've got a few gmirrors running on various machines, all of which > >> pair up two drives at the physical level (i.e. mirror /dev/ad0s1 > >> with /dev/ad1s1). Of course there are other ways of doing it to, > >> like mirroring at the partition level, ie pairing /dev/ad0s1a with > >> /dev/ad1s1a, /dev/ad0s1e with /dev/ad0s1e, etc. > >> > >> Apart from potentially avoiding a whole disk from being copied > >> during a resync after a crash, are there any other advantages to > >> using partition level mirroring instead of drive level mirroring? > > > > I can imagine people using partition-level raid to > > implement a popular configuration: > > > > You divide a couple of identical drives proportionally > > in two partitions each, place a couple of the first > > partitions into gmirror and a couple of the second > > ones into gstripe. This way you get both reliable and > > fast storage with just two drives. Some strings are > > attached. > > The head movement that this causes makes it a poor performer. It is > an option, but not a terribly popular one. I hear many desktops and laptops nowadays (used to?) come preconfigured this way. From owner-freebsd-fs@FreeBSD.ORG Thu Jan 18 01:40:24 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B0EFE16A415 for ; Thu, 18 Jan 2007 01:40:24 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from wr-out-0506.google.com (wr-out-0506.google.com [64.233.184.231]) by mx1.freebsd.org (Postfix) with ESMTP id 71A3A13C43E for ; Thu, 18 Jan 2007 01:40:24 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by wr-out-0506.google.com with SMTP id 36so53900wra for ; Wed, 17 Jan 2007 17:40:23 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=GWvC01CWcG7p0Fl5IkXOwUu7Ut1XyDPGtfLeFkh7DB//Bj4ceyYgKmhdL3ztYzRMGO+ebBWAds7KOw6lhjJTtr7rCjqwOhPlXzOwfupA0O1ltO/T9o4R/cbZOahv+NOXZQGGE+kyj5Z1PK41RBqzYzlC1FEXy3EIWtEN1GRFWoQ= Received: by 10.90.35.15 with SMTP id i15mr380374agi.1169082956647; Wed, 17 Jan 2007 17:15:56 -0800 (PST) Received: by 10.90.31.7 with HTTP; Wed, 17 Jan 2007 17:15:56 -0800 (PST) Message-ID: Date: Thu, 18 Jan 2007 10:15:56 +0900 From: "Adrian Chadd" Sender: adrian.chadd@gmail.com To: "Andrew Pantyukhin" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20070117103935.GC4018@genius.tao.org.uk> X-Google-Sender-Auth: 58987f950db2bafd Cc: stable@freebsd.org, fs@freebsd.org Subject: Re: gmirror disks vs partitions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2007 01:40:24 -0000 On 17/01/07, Andrew Pantyukhin wrote: > [...after reading the slashdotter's piece of wisdom...] > > Yes, but that's the kind of functionality I have always > expected to be present in software raid solutions. I > hope I'll live to see this implemented in geom. That made my eyes bleed. Bring on ZFS and its method of managing JBODs. Adrian From owner-freebsd-fs@FreeBSD.ORG Fri Jan 19 05:58:18 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E75B216A415 for ; Fri, 19 Jan 2007 05:58:18 +0000 (UTC) (envelope-from SRS0=nGkO/S=G4=vvelox.net=v.velox@yourhostingaccount.com) Received: from mail04.yourhostingaccount.com (mail04.yourhostingaccount.com [65.254.253.242]) by mx1.freebsd.org (Postfix) with ESMTP id B938B13C448 for ; Fri, 19 Jan 2007 05:58:18 +0000 (UTC) (envelope-from SRS0=nGkO/S=G4=vvelox.net=v.velox@yourhostingaccount.com) Received: from scan06.yourhostingaccount.com ([10.1.1.236] helo=scan06.yourhostingaccount.com) by mail04.yourhostingaccount.com with esmtp (Exim) id 1H7mWd-0001mN-Me for fs@freebsd.org; Fri, 19 Jan 2007 00:42:55 -0500 Received: from authsmtp10.yourhostingaccount.com ([10.1.18.10] ident=exim) by scan06.yourhostingaccount.com with spamscanlookuphost (Exim) id 1H7mWd-0008JZ-P5 for fs@freebsd.org; Fri, 19 Jan 2007 00:42:55 -0500 Received: from authsmtp10.yourhostingaccount.com ([10.1.18.10] helo=authsmtp10.yourhostingaccount.com) by scan06.yourhostingaccount.com with esmtp (Exim) id 1H7mWd-0008JT-8w for fs@freebsd.org; Fri, 19 Jan 2007 00:42:55 -0500 Received: from [69.92.217.33] (helo=vixen42) by authsmtp10.yourhostingaccount.com with esmtpa (Exim) id 1H7mWc-00066B-L2; Fri, 19 Jan 2007 00:42:55 -0500 Date: Thu, 18 Jan 2007 23:42:51 -0600 From: Vulpes Velox To: "Adrian Chadd" Message-ID: <20070118234251.53fd7c8e@vixen42> In-Reply-To: References: <20070117103935.GC4018@genius.tao.org.uk> X-Mailer: Claws Mail 2.7.0 (GTK+ 2.10.8; i386-portbld-freebsd6.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-EN-UserInfo: 0d1ca1697cdb7a831d4877828571b7ab:1570f0de6936c69fef9e164fffc541bc X-EN-AuthUser: vvelox2 Sender: Vulpes Velox Cc: stable@freebsd.org, fs@freebsd.org Subject: Re: gmirror disks vs partitions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jan 2007 05:58:19 -0000 On Thu, 18 Jan 2007 10:15:56 +0900 "Adrian Chadd" wrote: > On 17/01/07, Andrew Pantyukhin wrote: > > > [...after reading the slashdotter's piece of wisdom...] > > > > Yes, but that's the kind of functionality I have always > > expected to be present in software raid solutions. I > > hope I'll live to see this implemented in geom. > > That made my eyes bleed. > > Bring on ZFS and its method of managing JBODs. I second that. I have been way less than impressed with software raid and LVM on linux. I have all ways found not mirroring partitions to be way better. It makes it way easier to repair the damn thing do fewer steps. When ZFS comes available, I plan to actually run it across multiple mirrors. It has built in JBOD, but it does not do mirroring. It just does stripping. From owner-freebsd-fs@FreeBSD.ORG Fri Jan 19 23:52:42 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DE86F16A402; Fri, 19 Jan 2007 23:52:42 +0000 (UTC) (envelope-from tom@samplonius.org) Received: from ly.sdf.com (ly.sdf.com [216.113.193.83]) by mx1.freebsd.org (Postfix) with ESMTP id 8922C13C457; Fri, 19 Jan 2007 23:52:42 +0000 (UTC) (envelope-from tom@samplonius.org) Received: from localhost (localhost [127.0.0.1]) by ly.sdf.com (Postfix) with ESMTP id 6CCF710C6C3; Fri, 19 Jan 2007 15:36:19 -0800 (PST) X-DSPAM-Result: Innocent X-DSPAM-Processed: Fri Jan 19 15:36:18 2007 X-DSPAM-Confidence: 0.9997 X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 45b155f2260202032415231 X-DSPAM-Factors: 27, X-Virus-Scanned: amavisd-new at X-Spam-Score: -4.172 X-Spam-Level: X-Spam-Status: No, score=-4.172 tagged_above=-10 required=6.6 tests=[ALL_TRUSTED=-1.8, AWL=0.327, BAYES_00=-2.599, DSPAM_HAM=-0.1] Received: from ly.sdf.com ([127.0.0.1]) by localhost (ly.sdf.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Pxq0BAD3pZ6R; Fri, 19 Jan 2007 15:36:18 -0800 (PST) Received: from ly.sdf.com (ly.sdf.com [216.113.193.83]) by ly.sdf.com (Postfix) with ESMTP id 5450610C6C2; Fri, 19 Jan 2007 15:36:18 -0800 (PST) Message-ID: <9240774.01169249778121.JavaMail.root@ly.sdf.com> Date: Fri, 19 Jan 2007 15:36:18 -0800 (PST) From: Tom Samplonius To: Vulpes Velox In-Reply-To: <20070118234251.53fd7c8e@vixen42> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Cc: stable@freebsd.org, Adrian Chadd , fs@freebsd.org Subject: Re: gmirror disks vs partitions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jan 2007 23:52:43 -0000 ----- Vulpes Velox wrote: > On Thu, 18 Jan 2007 10:15:56 +0900 > "Adrian Chadd" wrote: > > > On 17/01/07, Andrew Pantyukhin wrote: > > > > > [...after reading the slashdotter's piece of wisdom...] > > > > > > Yes, but that's the kind of functionality I have always > > > expected to be present in software raid solutions. I > > > hope I'll live to see this implemented in geom. > > > > That made my eyes bleed. > > > > Bring on ZFS and its method of managing JBODs. > > I second that. I have been way less than impressed with software raid > and LVM on linux. ... But LVM by itself is a good volume manager. The block level snapshot ability is especially good. LVM can actually notify dependent filesystems so that they flush all data, when the block level snapshot is created. ext3 does not support filesystem based snapshots (like ufs2 does), but LVM snapshots are better than most filesystem snapshots. ZFS is clearly better than LVM+ext3, and is really the only option for really big filesystems right now. ufs2 doesn't support journaling, and background fsck isn't a complete replacement for journalling. ext3 is stable but doesn't really scale well, or have leading performance, and doesn't really work on FreeBSD anyways. XFS is virtually unsupported, as SGI laid off all their filesystem developers when they went into chapter 11, and ReiserFS, besides having some dodgy reliability issues, the head of development is currently in jail for suspicion of murder. So besides, being the best, ZFS is nearly the only choice for really big filesystems. Tom From owner-freebsd-fs@FreeBSD.ORG Sat Jan 20 08:43:51 2007 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8CA6F16A406; Sat, 20 Jan 2007 08:43:51 +0000 (UTC) (envelope-from jon@seaholm.caamora.com.au) Received: from seaholm.caamora.com.au (seaholm.caamora.com.au [203.7.226.5]) by mx1.freebsd.org (Postfix) with ESMTP id 313D013C465; Sat, 20 Jan 2007 08:43:47 +0000 (UTC) (envelope-from jon@seaholm.caamora.com.au) Received: (from jon@localhost) by seaholm.caamora.com.au (8.11.1/8.11.1) id l0K8Lj515434; Sat, 20 Jan 2007 19:21:45 +1100 (EST) Message-ID: <20070120192145.57080@caamora.com.au> Date: Sat, 20 Jan 2007 19:21:45 +1100 From: jonathan michaels To: Andrew Pantyukhin References: <20070117103935.GC4018@genius.tao.org.uk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.84e In-Reply-To: ; from Andrew Pantyukhin on Wed, Jan 17, 2007 at 02:29:33PM +0300 Organisation: Caamora, PO Box 144, Rosebery NSW 1445 Australia Cc: Josef Karthauser , stable@freebsd.org, fs@freebsd.org Subject: Re: gmirror disks vs partitions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jan 2007 08:43:51 -0000 On Wed, Jan 17, 2007 at 02:29:33PM +0300, Andrew Pantyukhin wrote: > On 1/17/07, Josef Karthauser wrote: > > A poll for opinions if I may? i suppose i'm asking the smae here as well ... > > I've got a few gmirrors running on various machines, all of which > > pair up two drives at the physical level (i.e. mirror /dev/ad0s1 > > with /dev/ad1s1). Of course there are other ways of doing it to, > > like mirroring at the partition level, ie pairing /dev/ad0s1a with > > /dev/ad1s1a, /dev/ad0s1e with /dev/ad0s1e, etc. > > > > Apart from potentially avoiding a whole disk from being copied > > during a resync after a crash, are there any other advantages to > > using partition level mirroring instead of drive level mirroring? > > I can imagine people using partition-level raid to > implement a popular configuration: > > You divide a couple of identical drives proportionally > in two partitions each, place a couple of the first > partitions into gmirror and a couple of the second > ones into gstripe. This way you get both reliable and > fast storage with just two drives. Some strings are > attached. my situation is somewhat different, in theat i am providing internet services for a (private) group to access tcp/ip based communications (we are all disabled and couldn't fine "reasonable" priced and competently serviced "ISP" services in our part of teh world, so we decided to do it for our selves) .. sorry thet is teh history and reason behind my participation in/with freebsd (over teh last 10 or so years). we have just recieved several older machines, PIII compaq proliant 5500 with hardware raid works quite nicely wonce it settled down and its batteries regained working voltage so to speak, it is running freebsd 6.10release, ms windows professional 2003 server, and linux debian (sarge v3.1) it is a multi-boot fixit box as well as bing teh basic "fileserver/nfs host" and kernel builder, with its 4 cpu architecture it works well. also came several 233 mhz 2 ide/2 rom drives (cd and dvd) and an 800 mhz PIII similarly equiped. all are intel hardware of some 8-10 years vintage, this is now the basic netowrk backbone, and upgrading from several intel 386dx33 and intel 486dx33/50 machines that have served this netowrk for over 20 years now. now that andrew has 'opened' my eyes so to speak to teh world of software raid and after some extensive reading i discovered RAIDFrame which looked to provide all tehat i am looking for, yes i played with vinum and got burned so badly i was only going to use hardware raid and the basis of my comments to andrew. i too have seen teh raid in freebsd has moved on, so i guess its time for me to move on as well, looks like software raid might just fit the bills that these multiple drive machines are begging .. all have several largis (for me) ide style harddisks, mainly 6-8 gb and i have relic 4 gb scsi harddisks that (as i read in RAIDFrame for freebsd) i'm hoping that i could build some sort of basic media platform for each of teh machines instead of constantly worrying about how to cut up teh operating system software load over teh available spindle count .. its not fun anymore working out where teh system was loading up teh spindles and draging down teh system as a whole .. i'm sure many of teh readers here have expericenced this before from time to time, atleast. i've seen lots of posts about RAIDFrame for freebsd upto about 2002 and perhaos 2003 .. is teh port stabalised and not in need of anymore work, or has it been canned and or droped ??? from what i have read the raidframe package would be an ideal solution, i like very much mr long's introduction on teh freebsd (people) page. this discussion on teh whole had been most enlightening and i hape it will bear much fruit for the geom project in teh long term .. i've been gollowing teh gstripe (here in -stable) i need to keep reminding myself that teh software is not bad, it is being developed and thats why all teh "bad/bug/things going wrong are being reported here in -stable, that what -stable is for/all about. sorry for my post, i'm not very good at comunicationing, its one of teh parts of mybrain that don't work too good, and that is why i'm (struggling) on teh invalid pension. umm i'd also like to take this opportunity to say thank you for al the support freebsd has given me over teh years, it has been a most wonderfull experience, the stability and reliability has been a shining light that i take with me whereever i go, int eh softeware world, and in general as its produced because people band togehter and care about what they do and that is what makes freebsd what it is .. not superieor code and all tehse other things, which i'm sure help, ok just a linny little bit (grin). much appreciations, thanks and gratittude. most kind regards jonathan and caamora dot com dot au -- ================================================================ powered by .. QNX, OS9 and freeBSD -- http://caamora com au/operating system ==== === appropriate solution in an inappropriate world === ==== From owner-freebsd-fs@FreeBSD.ORG Sat Jan 20 18:03:59 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7D2CC16A402 for ; Sat, 20 Jan 2007 18:03:59 +0000 (UTC) (envelope-from ahnjoan@gmail.com) Received: from nz-out-0506.google.com (nz-out-0506.google.com [64.233.162.225]) by mx1.freebsd.org (Postfix) with ESMTP id 4122713C43E for ; Sat, 20 Jan 2007 18:03:59 +0000 (UTC) (envelope-from ahnjoan@gmail.com) Received: by nz-out-0506.google.com with SMTP id i11so227966nzh for ; Sat, 20 Jan 2007 10:03:58 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=F8Il1Os8VCOtGxhiNwu6FlvaYPpvLHEl0o6F65WwZawWvOWaZAoQF0EDYiopKdhE1nD+ug+uE8vY6i3WQH3LhLh3j3HVGDIqGv0Mbk521FnnG7S3QPPuJKcEwB+G4gU0GKgRGveVqBp8Wo/AKtxLN9K/81QefJ2nQA8w42Hh0rc= Received: by 10.65.237.19 with SMTP id o19mr4961495qbr.1169316238459; Sat, 20 Jan 2007 10:03:58 -0800 (PST) Received: by 10.65.181.20 with HTTP; Sat, 20 Jan 2007 10:03:58 -0800 (PST) Message-ID: <5e575c8a0701201003j4de54712h6913f0edd62c4ffd@mail.gmail.com> Date: Sat, 20 Jan 2007 13:03:58 -0500 From: "Ahnjoan Amous" To: "Scott Long" In-Reply-To: <5e575c8a0701130703y6c528cecoed9019472a4956c8@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=WINDOWS-1252; format=flowed Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <5e575c8a0701111155l859d0ecif617dbda43cef842@mail.gmail.com> <45A6A138.8090208@samsco.org> <5e575c8a0701111307s5a839b82ra4ba5c45d554d3b9@mail.gmail.com> <45A7551C.5030006@samsco.org> <5e575c8a0701130703y6c528cecoed9019472a4956c8@mail.gmail.com> Cc: freebsd-fs@freebsd.org Subject: Re: aac scsi raid driver performance X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jan 2007 18:03:59 -0000 Thanks again for all the responses. I have been able to run my tests on the hardware of another customer with significantly different results. (50MB/s writes to hardware RAID, 50MB/s writes to GEOM RAID, 65MB/s to no RAID) The only difference in hardware between the two customers are SCSI disks and CPU speed. (The CPUs in the hardware with significantly better performing write speeds are 3.0G 512L2 1024L3 where the CPUs in the hardware with 20MB/s writes are 2.4G 512L2 (no L3). I'm borrowing a couple of CPUs to test in the slower writing hardware, if that doesn't resolve the problems I'll look in to obtaining alternate SCSI drives for further testing. If anyone has any other suggestions I'd be happy to hear them! I'll post any updates to this thread=85 On 1/13/07, Ahnjoan Amous wrote: > Scott - I disabled read cache on both containers and the end result > for my write tests is still 25M/sec. I have included the output to > aaccli to confirm that I disabled the correct cache. Maybe I'm just > expecting too much from the PE2650 with PERC 3/di. I know there are a > lot of variables in this equation but what speeds would you expect for > this U160 scsi raid controller front ending U320 devices on a box with > no load and no i/o, other than two concurrent dd's to two different > ufs mounted hardware raid 0 volumes? > > Thanks > Ahnjoan > > AAC0> container show cache 2 > Read Cache Setting : DISABLE > Write Cache Setting : ENABLE WHEN PROTECTED > Write Cache Status : Active, protected > > AAC0> container show cache 3 > Read Cache Setting : DISABLE > Write Cache Setting : ENABLE WHEN PROTECTED > Write Cache Status : Active, protected > > > On 1/12/07, Scott Long wrote: > > Turn off read caching > > > > Ahnjoan Amous wrote: > > > I reset the controller to the defaults for everything before I did th= e > > > installation. > > > > > > Each of the containers were created with the following options. > > > Container Type - RAID 0 > > > Container Label - data03 > > > Container Size - 279.396 GB > > > Chunk Size - 64KB > > > Read Caching (Yes/No) - Y > > > Write Caching : Enable when protected > > > Create RAID 5 via - N/A > > > > > > Then the controller itself has a single cache option > > > Drives Write Cache - Disabled > > > > > > Thanks > > > Ahnjoan > > > > > > On 1/11/07, Scott Long wrote: > > >> Did you enable read caching for the arrays? > > >> > > >> Scott > > >> > > >> > > >> Ahnjoan Amous wrote: > > >> > I'm trying to find possible explanations for slow concurrent write= s > > >> > through the > > >> > aac driver. This machine runs under 1% load and has less than 4 > > >> > transfers per > > >> > second to the drives in question when not being used for testing. > > >> > > > >> > When I attempt sequential "dd"s as follow, the results are better = then > > >> > 70MB/sec. > > >> > dd if=3D/dev/zero of=3D/data02/helloworld bs=3D1m count=3D1000 > > >> > 1048576000 bytes transferred in 13.886718 secs (75509274 bytes/= sec) > > >> > dd if=3D/dev/zero of=3D/data03/helloworld bs=3D1m count=3D1000 > > >> > 1048576000 bytes transferred in 14.011323 secs (74837758 bytes/= sec) > > >> > > > >> > When I attempt concurrent "dd"s as follow, with a 1 second sleep > > >> interval > > >> > between starts, the results are better than 40MB/sec > > >> > dd if=3D/dev/zero of=3D/data02/helloworld bs=3D1m count=3D1000 & > > >> > sleep 1 > > >> > dd if=3D/dev/zero of=3D/data03/helloworld bs=3D1m count=3D1000 & > > >> > 1048576000 bytes transferred in 25.269555 secs (41495626 bytes/= sec) > > >> > 1048576000 bytes transferred in 24.935765 secs (42051086 bytes/= sec) > > >> > > > >> > When I attempt concurrent "dd"s as follow, the results are little > > >> better > > >> > than > > >> > 20MB/sec > > >> > dd if=3D/dev/zero of=3D/data02/helloworld bs=3D1m count=3D1000 & > > >> > dd if=3D/dev/zero of=3D/data03/helloworld bs=3D1m count=3D1000 & > > >> > 1048576000 bytes transferred in 44.963408 secs (23320652 bytes/= sec) > > >> > 1048576000 bytes transferred in 45.010065 secs (23296478 bytes/= sec) > > >> > > > >> > I can't account for what causes the huge difference however the > > >> results are > > >> > reproducible. I've run the tests dozens and dozens of times now, = first > > >> > blaming > > >> > the em driver for my slow ggatec/ggated results, then GEOM for my = slow > > >> > local > > >> > mirroring after eliminating the network, and finally blaming the a= ac > > >> driver > > >> > after removing GEOM from the equation. If anyone has ideas on wha= t > > >> I might > > >> > look at or change or test I would love to hear. > > >> > > > >> > > > >> > ****** Misc. Information ****** > > >> > root:somehost:~ > df -k > > >> > Filesystem 1K-blocks Used Avail Capacity Mounted on > > >> > /dev/aacd2s1e 2026030 1024532 839416 55% /data02 > > >> > /dev/aacd3s1e 2026030 1024532 839416 55% /data03 > > >> > Hardware - > > >> > aac - Dell PERC3/Di U160 > > >> > aacd2 - hardware RAID 0, w/1 U320 300G drive > > >> > aacd3 - hardware RAID 0, w/1 U320 300G drive > > >> > _______________________________________________ > > >> > freebsd-fs@freebsd.org mailing list > > >> > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > > >> > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.o= rg" > > >> > > >> > > > > >