Date: Fri, 14 Mar 2003 10:51:56 -0500 From: Bruce Campbell <bruce@engmail.uwaterloo.ca> To: freebsd-questions@freebsd.org, freebsd-hardware@freebsd.org Subject: Re: problem on 1TB filesystem RAID 5 3ware Message-ID: <1047657116.3e71fa9c28266@www.nexusmail.uwaterloo.ca> In-Reply-To: <1047519493.3e6fe105a4f15@www.nexusmail.uwaterloo.ca> References: <1047519493.3e6fe105a4f15@www.nexusmail.uwaterloo.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
Not solved this yet, but I have determined a few things that the problem isn't. Info at: http://www.freebsd.uwaterloo.ca/twiki/bin/view/Freebsd/BackupServerProblem Tested with soft updates off and on, fails in either case, so that isn't it. Seems like the problem is either: - 3ware card or driver - something to do with the large filesystem Quoting Bruce Campbell <bruce@engmail.uwaterloo.ca>: > > File corruption on 2 identical systems, designed to be backup > servers to contain dumps of other systems: > > FreeBSD ecserv18.uwaterloo.ca 4.7-RELEASE FreeBSD 4.7-RELEASE #0: Wed Oct 9 > > 15:08:34 GMT 2002 root@builder.freebsdmall.com:/usr/obj/usr/src/sys/GENERIC > > i386 > > with 1TB /backup partition, on a 3ware 7500-8 ATA RAID card, RAID 5: > > Filesystem 1K-blocks Used Avail Capacity Mounted on > /dev/twed0s1a 20644846 906552 18086708 5% / > procfs 4 4 0 100% /proc > /dev/twed0s1e 938819776 279031856 584682338 32% /backup > > disks are 6 x Western Digital 2000JB (200GB) > > I ran tests on /backup for 10 days on each system (fill disk with > 50GB files of pseudo random data, then reading them all back and > verify contents, then erase, then start over). Tests ran perfectly. > > details on hardware config at: > > http://www.freebsd.uwaterloo.ca/twiki/bin/view/Freebsd/BackupServerHardware > > Then, I was ready to put the systems into production, so I copied > data from my 2 older backup servers (which have 360GB vinum partitions) > and after copying the data (approx 250GB in 325 files) about a dozen > files were corrupt after the copy. I copied via an NFS mount. > > All corruption started on a 64K boundary, except one which was on a 16K > boundary. Recopied the dozen corrupt files, and then only 6 were corrupt. > Same problem on both systems, each which copied from a different source > server. > > File seems corrupt to the end after first corruption starts, I have > not looked for a pattern to see if it is another files contents, > or misplaced contents from the same file. > > fsck shows no problems > > Restarted my test filling with 50GB files again, has run perfectly. > > I plan to try: > > - turn off soft updates > - RAID 10 instead of 5 > - different file system parameters, for example I don't need > 100 million inodes. > - rcp'ing the files > - staring at computer screen > > By the way, 3ware has not officially approved the WD 200GB drive last > time I checked. > > Lots of good experience with the motherboard (ASUS P4S533) and > network card (Intel Pro/100). Lots of good experience with > vinum striped partitions of smaller size (360GB) > > Does anyone have any suggestions ? > > -- > Bruce Campbell > Engineering Computing > CPH-2374B > University of Waterloo > (519)888-4567 ext 5889 > > ---------------------------------------- > This mail sent through www.mywaterloo.ca > -- Bruce Campbell Engineering Computing CPH-2374B University of Waterloo (519)888-4567 ext 5889 ---------------------------------------- This mail sent through www.mywaterloo.ca To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1047657116.3e71fa9c28266>