Date: Mon, 27 Dec 2004 12:36:02 -0800 From: joseph kacmarcik <freebsd-questions@chubbo.net> To: David Gerard <fun@thingy.apana.org.au> Cc: questions@freebsd.org Subject: Re: Cheap NAS using FreeBSD - practical considerations? Message-ID: <1104179762.41d07232a1895@mail.chubbo.net> In-Reply-To: <20041227200011.GC27571@thingy.apana.org.au> References: <20041227200011.GC27571@thingy.apana.org.au>
next in thread | previous in thread | raw e-mail | index | archive | help
> Let's say that, as fine as NetApps are, I can't afford their prices. So I > set up a FreeBSD box with a whole lot of disk attached and use that as > network-attached storage, serving files by NFS, with gigabit ethernet. > > Setting up such a box is trivially easy. But what are the practical > considerations? Have any of you done this, or know anyone who has? Does > serving stay at wire speed? Recommendations for motherboards or > peripherals? freebsd is a fine choice for this application. of course your performance will vary depending on what you're running on the clients; http, database, etc. personally, i'd run off-board raid controllers for your shared storage. leaving the OS on the motherboard is fine. 3ware, adaptec (aac chip, not asr) can be controlled via cmdline on freebsd (3ware has 3dm, gui manager). this will reduce contention on the motherboard. if you're running a variety of applications on the clients and you have the resources on the nfs machine, create seperate raid arrays for each with fitting performance-to-redundancy ratio. ie; raid-50 for web storage to get increased capacity, raid-10 for database to get slightly increased performance. i've found these nfs mount options to give the best performance in my environment: tcp,rsize=32768,wsize=32768. the mount options are suitable for freebsd, linux and solaris. if you have linux clients, use kernel 2.6.x. be sure to start enough nfs servers, and monitor the nfs server with nfsstat. it might be a good idea to graph the various numbers of nfsstat output, so you can recognize trends and understand your storage infrastructure. you'll likely notice that freebsd can maintain saturating it's network. if you have dual-ethernet (or more), you could use an alias IP on the second nic for the nfs clients, or use multiple alias IP's dedicated to the various applications. you'll much more easily spot capacity and performance issues in regards to each application by doing so. currently, i'm serving @30 nfs clients running freebsd-amd64 on a supermicro X6DH8-XG2, two 3.6GHz Xeon, 4GB ram, 2x 3ware 9500s-12MI, 1.8TB RAID-50 array for web/images, and a @300GB RAID-10 array for mysql. if you have the time, try using bonnie++ (or some other benchmarking utility) on a few clients concurrently, and watch cpu, memory and disk usage on the nfs server. bsdsar will record the host data for later research. depending on filesystem size and server speed, you could be well served by using freebsd's mksnap_ffs; taking snapshots similar to the netapp. good luck and have fun! joe
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1104179762.41d07232a1895>