Date: Wed, 9 Nov 2011 11:39:34 -0800 From: Garrett Cooper <yanegomi@gmail.com> To: "Daniel O'Connor" <doconnor@gsoft.com.au> Cc: Kurt Touet <ktouet@gmail.com>, "freebsd-current@freebsd.org" <freebsd-current@freebsd.org>, Dan The Man <dan@sunsaturn.com> Subject: Re: samba+zfs Message-ID: <CAGH67wSkoCG1tc6Y_XtErNmmF9JmvQCgUoFU_WLLC_EwAR_ZeQ@mail.gmail.com> In-Reply-To: <82C85C01-62C4-4E75-B3F2-59D703CA5D78@gsoft.com.au> References: <alpine.BSF.2.00.1110272039500.50739@sunsaturn.com> <CAGH67wRZZx0hG9ug2k-5ohCOPJ9sZOU9iFVKg7hv9WM=R761GA@mail.gmail.com> <alpine.BSF.2.00.1111080259270.89703@sunsaturn.com> <alpine.BSF.2.00.1111080328010.89703@sunsaturn.com> <CALM%2B6aJkF=CFq8LA3FrSMYo8La-8txK4h2p4yZtdHshskBU6Vw@mail.gmail.com> <alpine.BSF.2.00.1111082015260.93923@sunsaturn.com> <CALM%2B6aJiM%2BNfikax9vNiYzBJqo3B8WLEXiZgrjUUQa9ngCQaKg@mail.gmail.com> <B4FEAF5B-52AC-40E5-90A3-7BD060BB7A73@gsoft.com.au> <F8AEDB62-4C76-4D95-8CBA-6C58B54C1965@gsoft.com.au> <CAGH67wRJkN%2BsDhxxMVSZx3RXQJm5xyyfW9iwiY1FrkFxLu%2BTWQ@mail.gmail.com> <82C85C01-62C4-4E75-B3F2-59D703CA5D78@gsoft.com.au>
next in thread | previous in thread | raw e-mail | index | archive | help
On Nov 8, 2011, at 11:07 PM, "Daniel O'Connor" <doconnor@gsoft.com.au> wrot=
e:
>
> On 09/11/2011, at 17:32, Garrett Cooper wrote
>>> dd's of large files (spooled backups going to tape) to /dev/null are as=
slow as Samba.
>>
>> - Dedupe?
>
> Nope.
>
>> - Compression?
>
> On the mail spool & ports, but not on the tape spool.
>
>> - How much RAM?
>
> 8GB.
>> - What debug options do you have enabled in the kernel?
>
> It is 8.2-GENERIC so.. no WITNESS (for example)
Ok. 8.2 release or stable?
>> I've been noticing a slowdown in some respects with NFS/SMB, but I
>> suspected it was because I have an re(4) based NIC. ZFS has also wired
>> down a lot of my system memory for the L2ARC=85
>
>
> re isn't great but I wouldn't expect it to slow down over time.. Unless b=
ounce buffers got used more and more or something.
>
> I have an em0 card in this system - but in any case it is slow locally (i=
.e. dd a large file with 64k block size).
Good point. Simple base cases help isolate the root cause. That
being said, my disk speed(s) are a lot better than my network + samba
speeds (zfs:store is mfid0 backed with write cache enabled; zfs:sac is
a single ada(4) backed disk with write cache enabled -- err... it
shouldn't be like that), but I suspect that's misconfiguration on my
part:
$ sysctl hw.model hw.physmem
hw.model: Intel(R) Xeon(R) CPU W3520 @ 2.67GHz
hw.physmem: 12863094784
$ sudo mfiutil show volumes
mfi0 Volumes:
Id Size Level Stripe State Cache Name
mfid0 ( 1860G) RAID-6 64k OPTIMAL Enabled <STORE>
$ zpool status
pool: sac
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
sac ONLINE 0 0 0
ada0p3 ONLINE 0 0 0
errors: No known data errors
pool: store
state: ONLINE
scan: scrub repaired 0 in 10h9m with 0 errors on Mon Nov 7 18:58:01 2011
config:
NAME STATE READ WRITE CKSUM
store ONLINE 0 0 0
mfid0p1 ONLINE 0 0 0
errors: No known data errors
$ zfs list -o name,mountpoint,atime,sync,compression,dedup
NAME MOUNTPOINT ATIME SYNC
COMPRESS DEDUP
sac legacy on standard
off off
sac/root / on standard
off off
sac/scratch /scratch on standard
off off
sac/scratch/freenas /scratch/freenas off standard
on off
sac/scratch/freenas/FreeBSD /scratch/freenas/FreeBSD off standard
on off
sac/usr /usr on standard
off off
sac/var /var on standard
off off
store /store on standard
off off
store/freebsd /store/freebsd on standard
on on
store/home /usr/home on standard
off off
$ dd if=3D/dev/zero of=3Dfoo bs=3D1m count=3D1024
1024+0 records out
1073741824 bytes transferred in 13.426620 secs (79971119 bytes/sec)
$ cd /store
$ dd if=3D/dev/zero of=3Dfoo bs=3D1m count=3D1024
1024+0 records in
1024+0 records out
1073741824 bytes transferred in 7.565117 secs (141933276 bytes/sec)
$ cat /usr/local/etc/smb.conf
[global]
workgroup =3D WORKGROUP
server string =3D BAYONETTA
security =3D user
load printers =3D no
max log size =3D 50
preferred master =3D yes
local master =3D yes
socket options =3D SO_RCVBUF=3D16384 SO_SNDBUF=3D16384
nt acl support =3D yes
inherit acls =3D yes
map acl inherit =3D yes
aio read size =3D 16384
aio write size =3D 16384
[scratch]
path =3D /scratch
writeable =3D yes
[store]
path =3D /store
writeable =3D yes
$
I'll have to:
1. Recheck what Windows 7 says when transferring out to my box
with a large file.
2. Use nc to quickly measure network performance.
3. Try transferring over NFS with both my Macbook and setup
FreeBSD or Linux on the other workstation for testing out NFS
transfers (64kB rsize/wsize of course). Wash, rinse, repeat with
samba.
The last I remember the transfer speeds were pitiful with samba36
(somewhere around 45MBps to my 'store' zpool). I've been conservative
with the socket settings, but it might be time to bump that up along
with the mbuf cluster count (for some odd reason I haven't changed it
from the system default), reboot, and retest.
Thanks,
-Garrett
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAGH67wSkoCG1tc6Y_XtErNmmF9JmvQCgUoFU_WLLC_EwAR_ZeQ>