Date: Tue, 10 Jun 2003 01:24:50 -0700 From: Terry Lambert <tlambert2@mindspring.com> To: Eric Anderson <anderson@centtech.com> Cc: freebsd-performance@freebsd.org Subject: Re: Slow disk write speeds over network Message-ID: <3EE595D2.B223CA19@mindspring.com> References: <20030609211526.58641.qmail@web14908.mail.yahoo.com> <3EE4FAED.6090603@centtech.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Eric Anderson wrote: > >>No, considering I would have to update about 600 hosts. Plus, those > >>same hosts are running just fine with my other file servers at the > >>moment. > > > > If your server's network buffers are ok, I would guess that the > > "bottleneck" is on your clients machines. > > > > just my 2 cents... > > Paulo, thanks for the thoughts, however, that doesn't make a whole lot > of sense (although I see why you are saying that), because in this case > I have several fileservers on the same gigabit switches as this > fileserver, and clients can transfer data quickly between all, in both > directions, all the time, except data being sent TO this one host. I've > even tried machines that are idle, on the same switch as this server, > and I still receive slow transfers. Swap cables with another box. BTW: 4 Gigabit cards in one box, with you going to local disk... you've got about 8 cards worth of traffic over your PCI bus. Unless this is a PCI-X based box, you are most likely livelocked; even if it's a PCI-X based box, you could still be livelocked. You haven't said if you were using UDP or TCP for the mounts; you should definitely use TCP with FreeBSD NFS servers; it's also just generally a good idea, since UDP frags act as a fixed non-sliding window: NFS over UDP sucks. Also, you haven't said whether you are using aliases on your network cards; aliases and NFS tend to interact badly. Finally, you probably want to tweak some sysctl's, e.g. net.inet.ip.check_interface=0 net.inet.tcp.inflight_enable=1 net.inet.tcp.inflight_debug=0 net.inet.tcp.msl=3000 net.inet.tcp.inflight_min=6100 net.isr.enable=1 Given your overloading of your bus, that last one is probably the most important one: it enables direct dispatch. You'll also want to enable DEVICE_POLLING in your kernel config file (assuming you have a good ethernet card whose driver supports it): options DEVICE_POLLING options HZ=2000 ...and yet more sysctl's for this: kern.polling.enable=1 kern.polling.user_frac=50 # 0..100; whatever works best If you've got a really terrible Gigabit Ethernet card, then you may be copying all your packets over again (e.g. m_pullup()), and that could be eating your bus, too. -- Terry
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3EE595D2.B223CA19>