Date: Sun, 8 Aug 2010 11:07:59 +0800 From: "Alex V. Petrov" <alexvpetrov@gmail.com> To: freebsd-stable@freebsd.org Subject: Re: zpool - low speed write Message-ID: <201008081108.00175.alexvpetrov@gmail.com> In-Reply-To: <4C5DE5AE.5020704@ukr.net> References: <201008042008.25679.alexvpetrov@gmail.com> <4C5DE5AE.5020704@ukr.net>
next in thread | previous in thread | raw e-mail | index | archive | help
=D0=92 =D1=81=D0=BE=D0=BE=D0=B1=D1=89=D0=B5=D0=BD=D0=B8=D0=B8 =D0=BE=D1=82 =
8 =D0=B0=D0=B2=D0=B3=D1=83=D1=81=D1=82=D0=B0 2010 07:01:02 =D0=B0=D0=B2=D1=
=82=D0=BE=D1=80 Vladislav V. Prodan =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=
=BB:
> 04.08.2010 15:08, Alex V. Petrov =D0=BF=D0=B8=D1=88=D0=B5=D1=82:
> > Hi All!
> >=20
> > $ dd if=3D/dev/random of=3D/tank/test bs=3D3M count=3D1000
> > 1000+0 records in
> > 1000+0 records out
> > 3145728000 bytes transferred in 298.153293 secs (10550707 bytes/sec)
> >=20
> >=20
> > Any ideas?
>=20
> Please, show output:
> vmstat -z
> netstat -m
> sysctl -a | grep vm.kmem
> sysctl -a | grep vfs.zfs.arc
> sysctl -a | grep kern.maxvnodes
> sysctl -a | grep vm.kvm
vmstat -z
ITEM SIZE LIMIT USED FREE REQUESTS FAIL=
URES
UMA Kegs: 208, 0, 191, 13, 191, =
0
UMA Zones: 320, 0, 191, 1, 191, =
0
UMA Slabs: 568, 0, 5146, 1301, 24288196, =
0
UMA RCntSlabs: 568, 0, 1631, 1015, 329163, =
0
UMA Hash: 256, 0, 79, 11, 82, =
0
16 Bucket: 152, 0, 16, 109, 167, =
0
32 Bucket: 280, 0, 30, 82, 274, =
2
64 Bucket: 536, 0, 49, 63, 793, =
89
128 Bucket: 1048, 0, 514, 95, 187117, 12=
3359
VM OBJECT: 216, 0, 34974, 213300, 413596412, =
0
MAP: 232, 0, 7, 25, 7, =
0
KMAP ENTRY: 120, 150474, 183, 7040, 66175818, =
0
MAP ENTRY: 120, 0, 24333, 10759, 1025909468, =
0
DP fakepg: 120, 0, 216, 187, 544, =
0
SG fakepg: 120, 0, 667, 1689, 6212, =
0
mt_zone: 2056, 0, 295, 54, 295, =
0
16: 16, 0, 3916, 13724, 325519968, =
0
32: 32, 0, 3598, 1654, 65243323, =
0
64: 64, 0, 16671, 29529, 905000121, =
0
128: 128, 0, 12097, 30011, 473239749, =
0
256: 256, 0, 3300, 4650, 793867992, =
0
512: 512, 0, 5211, 7690, 54770551, =
0
1024: 1024, 0, 239, 697, 5912498, =
0
2048: 2048, 0, 1335, 721, 35819870, =
0
4096: 4096, 0, 715, 617, 7671562, =
0
=46iles: 80, 0, 2258, 1162, 323524959, =
0
TURNSTILE: 136, 0, 1673, 67, 1677, =
0
umtx pi: 96, 0, 0, 0, 0, =
0
MAC labels: 40, 0, 0, 0, 0, =
0
PROC: 1120, 0, 187, 824, 6336408, =
0
THREAD: 984, 0, 1236, 436, 20593, =
0
SLEEPQUEUE: 80, 0, 1673, 67, 1677, =
0
VMSPACE: 392, 0, 159, 831, 6334753, =
0
cpuset: 72, 0, 2, 98, 2, =
0
audit_record: 952, 0, 0, 0, 0, =
0
mbuf_packet: 256, 0, 371, 525, 348284916, =
0
mbuf: 256, 0, 231, 1888, 2247652340, =
0
mbuf_cluster: 2048, 25600, 896, 804, 188084, =
0
mbuf_jumbo_page: 4096, 12800, 27, 754, 593845769, =
0
mbuf_jumbo_9k: 9216, 6400, 0, 0, 0, =
0
mbuf_jumbo_16k: 16384, 3200, 0, 0, 0, =
0
mbuf_ext_refcnt: 4, 0, 0, 672, 774185, =
0
g_bio: 232, 0, 20, 1644, 54750899, =
0
ttyinq: 160, 0, 330, 126, 1275, =
0
ttyoutq: 256, 0, 168, 87, 632, =
0
ata_request: 320, 0, 10, 782, 21908538, =
0
ata_composite: 336, 0, 0, 0, 0, =
0
nv_stack_t: 12288, 0, 10, 10, 22, =
0
VNODE: 472, 0, 32144, 63544, 25299060, =
0
VNODEPOLL: 112, 0, 63, 135, 65, =
0
S VFS Cache: 108, 0, 23801, 111928, 38399522, =
0
L VFS Cache: 328, 0, 4752, 1368, 2652656, =
0
NAMEI: 1024, 0, 0, 764, 1639822943, =
0
NFSMOUNT: 616, 0, 0, 0, 0, =
0
NFSNODE: 656, 0, 0, 0, 0, =
0
DIRHASH: 1024, 0, 183, 433, 391168, =
0
pipe: 728, 0, 294, 811, 10116037, =
0
AIO: 208, 0, 1, 71, 9, =
0
AIOP: 32, 0, 4, 501, 220, =
0
AIOCB: 480, 0, 0, 752, 1725169, =
0
AIOL: 128, 0, 0, 0, 0, =
0
AIOLIO: 272, 0, 0, 0, 0, =
0
ksiginfo: 112, 0, 1002, 846, 6385628, =
0
itimer: 344, 0, 1, 21, 1, =
0
KNOTE: 128, 0, 109, 877, 13079149, =
0
socket: 680, 25602, 638, 1000, 20201690, =
0
unpcb: 240, 25600, 564, 892, 19533253, =
0
ipq: 56, 819, 0, 252, 77, =
0
udp_inpcb: 336, 25608, 23, 648, 490633, =
0
udpcb: 16, 25704, 23, 817, 490633, =
0
tcp_inpcb: 336, 25608, 82, 919, 177794, =
0
tcpcb: 880, 25600, 49, 815, 177794, =
0
tcptw: 72, 5150, 33, 867, 70594, =
0
syncache: 144, 15366, 0, 182, 62709, =
0
hostcache: 136, 15372, 44, 824, 2249, =
0
tcpreass: 40, 1680, 0, 924, 990793, =
0
sackhole: 32, 0, 2, 907, 380653, =
0
sctp_ep: 1272, 25602, 0, 0, 0, =
0
sctp_asoc: 2232, 40000, 0, 0, 0, =
0
sctp_laddr: 48, 80064, 0, 144, 2, =
0
sctp_raddr: 616, 80004, 0, 0, 0, =
0
sctp_chunk: 136, 400008, 0, 0, 0, =
0
sctp_readq: 104, 400032, 0, 0, 0, =
0
sctp_stream_msg_out: 96, 400026, 0, 0, 0, =
0
sctp_asconf: 40, 400008, 0, 0, 0, =
0
sctp_asconf_ack: 48, 400032, 0, 0, 0, =
0
ripcb: 336, 25608, 0, 33, 4, =
0
rtentry: 200, 0, 9, 48, 9, =
0
selfd: 56, 0, 2093, 931, 49944309845, =
0
SWAPMETA: 288, 116519, 134, 399, 67855, =
0
ip4flow: 56, 99351, 159, 1605, 582416, =
0
ip6flow: 80, 99360, 0, 0, 0, =
0
Mountpoints: 752, 0, 11, 14, 13, =
0
=46FS inode: 168, 0, 28873, 96813, 13425752, =
0
=46FS1 dinode: 128, 0, 0, 0, 0, =
0
=46FS2 dinode: 256, 0, 28873, 82712, 13425724, =
0
taskq_zone: 56, 0, 0, 630, 2294, =
0
zio_cache: 776, 0, 40, 1290, 91039501, =
0
zio_buf_512: 512, 0, 0, 0, 0, =
0
zio_data_buf_512: 512, 0, 0, 0, 0, =
0
zio_buf_1024: 1024, 0, 0, 0, 0, =
0
zio_data_buf_1024: 1024, 0, 0, 0, 0, =
0
zio_buf_1536: 1536, 0, 0, 0, 0, =
0
zio_data_buf_1536: 1536, 0, 0, 0, 0, =
0
zio_buf_2048: 2048, 0, 0, 0, 0, =
0
zio_data_buf_2048: 2048, 0, 0, 0, 0, =
0
zio_buf_2560: 2560, 0, 0, 0, 0, =
0
zio_data_buf_2560: 2560, 0, 0, 0, 0, =
0
zio_buf_3072: 3072, 0, 0, 0, 0, =
0
zio_data_buf_3072: 3072, 0, 0, 0, 0, =
0
zio_buf_3584: 3584, 0, 0, 0, 0, =
0
zio_data_buf_3584: 3584, 0, 0, 0, 0, =
0
zio_buf_4096: 4096, 0, 0, 0, 0, =
0
zio_data_buf_4096: 4096, 0, 0, 0, 0, =
0
zio_buf_5120: 5120, 0, 0, 0, 0, =
0
zio_data_buf_5120: 5120, 0, 0, 0, 0, =
0
zio_buf_6144: 6144, 0, 0, 0, 0, =
0
zio_data_buf_6144: 6144, 0, 0, 0, 0, =
0
zio_buf_7168: 7168, 0, 0, 0, 0, =
0
zio_data_buf_7168: 7168, 0, 0, 0, 0, =
0
zio_buf_8192: 8192, 0, 0, 0, 0, =
0
zio_data_buf_8192: 8192, 0, 0, 0, 0, =
0
zio_buf_10240: 10240, 0, 0, 0, 0, =
0
zio_data_buf_10240: 10240, 0, 0, 0, 0, =
0
zio_buf_12288: 12288, 0, 0, 0, 0, =
0
zio_data_buf_12288: 12288, 0, 0, 0, 0, =
0
zio_buf_14336: 14336, 0, 0, 0, 0, =
0
zio_data_buf_14336: 14336, 0, 0, 0, 0, =
0
zio_buf_16384: 16384, 0, 0, 0, 0, =
0
zio_data_buf_16384: 16384, 0, 0, 0, 0, =
0
zio_buf_20480: 20480, 0, 0, 0, 0, =
0
zio_data_buf_20480: 20480, 0, 0, 0, 0, =
0
zio_buf_24576: 24576, 0, 0, 0, 0, =
0
zio_data_buf_24576: 24576, 0, 0, 0, 0, =
0
zio_buf_28672: 28672, 0, 0, 0, 0, =
0
zio_data_buf_28672: 28672, 0, 0, 0, 0, =
0
zio_buf_32768: 32768, 0, 0, 0, 0, =
0
zio_data_buf_32768: 32768, 0, 0, 0, 0, =
0
zio_buf_36864: 36864, 0, 0, 0, 0, =
0
zio_data_buf_36864: 36864, 0, 0, 0, 0, =
0
zio_buf_40960: 40960, 0, 0, 0, 0, =
0
zio_data_buf_40960: 40960, 0, 0, 0, 0, =
0
zio_buf_45056: 45056, 0, 0, 0, 0, =
0
zio_data_buf_45056: 45056, 0, 0, 0, 0, =
0
zio_buf_49152: 49152, 0, 0, 0, 0, =
0
zio_data_buf_49152: 49152, 0, 0, 0, 0, =
0
zio_buf_53248: 53248, 0, 0, 0, 0, =
0
zio_data_buf_53248: 53248, 0, 0, 0, 0, =
0
zio_buf_57344: 57344, 0, 0, 0, 0, =
0
zio_data_buf_57344: 57344, 0, 0, 0, 0, =
0
zio_buf_61440: 61440, 0, 0, 0, 0, =
0
zio_data_buf_61440: 61440, 0, 0, 0, 0, =
0
zio_buf_65536: 65536, 0, 0, 0, 0, =
0
zio_data_buf_65536: 65536, 0, 0, 0, 0, =
0
zio_buf_69632: 69632, 0, 0, 0, 0, =
0
zio_data_buf_69632: 69632, 0, 0, 0, 0, =
0
zio_buf_73728: 73728, 0, 0, 0, 0, =
0
zio_data_buf_73728: 73728, 0, 0, 0, 0, =
0
zio_buf_77824: 77824, 0, 0, 0, 0, =
0
zio_data_buf_77824: 77824, 0, 0, 0, 0, =
0
zio_buf_81920: 81920, 0, 0, 0, 0, =
0
zio_data_buf_81920: 81920, 0, 0, 0, 0, =
0
zio_buf_86016: 86016, 0, 0, 0, 0, =
0
zio_data_buf_86016: 86016, 0, 0, 0, 0, =
0
zio_buf_90112: 90112, 0, 0, 0, 0, =
0
zio_data_buf_90112: 90112, 0, 0, 0, 0, =
0
zio_buf_94208: 94208, 0, 0, 0, 0, =
0
zio_data_buf_94208: 94208, 0, 0, 0, 0, =
0
zio_buf_98304: 98304, 0, 0, 0, 0, =
0
zio_data_buf_98304: 98304, 0, 0, 0, 0, =
0
zio_buf_102400: 102400, 0, 0, 0, 0, =
0
zio_data_buf_102400: 102400, 0, 0, 0, 0, =
0
zio_buf_106496: 106496, 0, 0, 0, 0, =
0
zio_data_buf_106496: 106496, 0, 0, 0, 0, =
0
zio_buf_110592: 110592, 0, 0, 0, 0, =
0
zio_data_buf_110592: 110592, 0, 0, 0, 0, =
0
zio_buf_114688: 114688, 0, 0, 0, 0, =
0
zio_data_buf_114688: 114688, 0, 0, 0, 0, =
0
zio_buf_118784: 118784, 0, 0, 0, 0, =
0
zio_data_buf_118784: 118784, 0, 0, 0, 0, =
0
zio_buf_122880: 122880, 0, 0, 0, 0, =
0
zio_data_buf_122880: 122880, 0, 0, 0, 0, =
0
zio_buf_126976: 126976, 0, 0, 0, 0, =
0
zio_data_buf_126976: 126976, 0, 0, 0, 0, =
0
zio_buf_131072: 131072, 0, 0, 0, 0, =
0
zio_data_buf_131072: 131072, 0, 0, 0, 0, =
0
dmu_buf_impl_t: 224, 0, 4177, 3830, 9957745, =
0
dnode_t: 776, 0, 3230, 665, 3277738, =
0
arc_buf_hdr_t: 208, 0, 2584, 6398, 6339574, =
0
arc_buf_t: 72, 0, 1380, 1970, 6784952, =
0
zil_lwb_cache: 200, 0, 1, 607, 5809, =
0
zfs_znode_cache: 376, 0, 2476, 734, 3979675, =
0
bridge_rtnode: 64, 0, 3, 165, 12, =
0
netstat -m
578/2437/3015 mbufs in use (current/cache/total)
350/1350/1700/25600 mbuf clusters in use (current/cache/total/max)
350/674 mbuf+clusters out of packet secondary zone in use (current/cache)
25/756/781/12800 4k (page size) jumbo clusters in use (current/cache/total/=
max)
0/0/0/6400 9k jumbo clusters in use (current/cache/total/max)
0/0/0/3200 16k jumbo clusters in use (current/cache/total/max)
944K/6333K/7277K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0/0/0 sfbufs in use (current/peak/max)
0 requests for sfbufs denied
0 requests for sfbufs delayed
774157 requests for I/O initiated by sendfile
0 calls to protocol drain routines
sysctl -a | grep vm.kmem
vm.kmem_size_scale: 3
vm.kmem_size_max: 329853485875
vm.kmem_size_min: 0
vm.kmem_size: 1373954048
sysctl -a | grep vfs.zfs.arc
vfs.zfs.arc_meta_limit: 214680320
vfs.zfs.arc_meta_used: 13797096
vfs.zfs.arc_min: 107340160
vfs.zfs.arc_max: 858721280
sysctl -a | grep kern.maxvnodes
kern.maxvnodes: 800000
sysctl -a | grep vm.kvm
vm.kvm_free: 547673337856
vm.kvm_size: 549755809792
=2D----
Alex V. Petrov
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201008081108.00175.alexvpetrov>