From owner-freebsd-stable@FreeBSD.ORG Sun Aug 8 03:08:07 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E4E2A1065672 for ; Sun, 8 Aug 2010 03:08:07 +0000 (UTC) (envelope-from alexvpetrov@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 47DD08FC0C for ; Sun, 8 Aug 2010 03:08:06 +0000 (UTC) Received: by eyh6 with SMTP id 6so3651922eyh.13 for ; Sat, 07 Aug 2010 20:08:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:subject:date :user-agent:references:in-reply-to:mime-version:content-type :content-transfer-encoding:message-id; bh=clTgCu/lYprBRixmlRjLYzhBeB+NNTkxVEovVB17D8I=; b=TIJbcwwKNlSpn7kAxATHEEPLsU0C0fF6Tf9//C6g+ASz0S4VGlcvv7gS0Tkc6Aw6OK NptsXlsdgxu6OiJGF4fkCVkxWsGmiuGrOgXREHiUmQuivcd6z/+dxvdFewMewvtafWxJ XLDp/Dun0aw/79R7+lLEl8+A7WyJY7h6mapM4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:references:in-reply-to:mime-version :content-type:content-transfer-encoding:message-id; b=wwZnBBJ18t94hUnTiODFfgPRBIcwToe4d5YANSwkF87uGImcr/IwmtS8SrHW39o0FI 5kGaKysgxd6cjpMfMtfuU9T1l4/47GQYurtfpItoPIqPE8jBb/z11SNvQ90I5nZtYaHU tG3oMXljcAAoJLkGnR6a1lm8/m6ewuWt+wwYM= Received: by 10.213.100.1 with SMTP id w1mr1245265ebn.67.1281236886227; Sat, 07 Aug 2010 20:08:06 -0700 (PDT) Received: from alex.super ([91.195.101.86]) by mx.google.com with ESMTPS id u9sm5232823eeh.17.2010.08.07.20.08.03 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sat, 07 Aug 2010 20:08:05 -0700 (PDT) From: "Alex V. Petrov" To: freebsd-stable@freebsd.org Date: Sun, 8 Aug 2010 11:07:59 +0800 User-Agent: KMail/1.13.5 (FreeBSD/8.1-STABLE; KDE/4.4.5; amd64; ; ) References: <201008042008.25679.alexvpetrov@gmail.com> <4C5DE5AE.5020704@ukr.net> In-Reply-To: <4C5DE5AE.5020704@ukr.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201008081108.00175.alexvpetrov@gmail.com> Subject: Re: zpool - low speed write X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Aug 2010 03:08:08 -0000 =D0=92 =D1=81=D0=BE=D0=BE=D0=B1=D1=89=D0=B5=D0=BD=D0=B8=D0=B8 =D0=BE=D1=82 = 8 =D0=B0=D0=B2=D0=B3=D1=83=D1=81=D1=82=D0=B0 2010 07:01:02 =D0=B0=D0=B2=D1= =82=D0=BE=D1=80 Vladislav V. Prodan =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0= =BB: > 04.08.2010 15:08, Alex V. Petrov =D0=BF=D0=B8=D1=88=D0=B5=D1=82: > > Hi All! > >=20 > > $ dd if=3D/dev/random of=3D/tank/test bs=3D3M count=3D1000 > > 1000+0 records in > > 1000+0 records out > > 3145728000 bytes transferred in 298.153293 secs (10550707 bytes/sec) > >=20 > >=20 > > Any ideas? >=20 > Please, show output: > vmstat -z > netstat -m > sysctl -a | grep vm.kmem > sysctl -a | grep vfs.zfs.arc > sysctl -a | grep kern.maxvnodes > sysctl -a | grep vm.kvm vmstat -z ITEM SIZE LIMIT USED FREE REQUESTS FAIL= URES UMA Kegs: 208, 0, 191, 13, 191, = 0 UMA Zones: 320, 0, 191, 1, 191, = 0 UMA Slabs: 568, 0, 5146, 1301, 24288196, = 0 UMA RCntSlabs: 568, 0, 1631, 1015, 329163, = 0 UMA Hash: 256, 0, 79, 11, 82, = 0 16 Bucket: 152, 0, 16, 109, 167, = 0 32 Bucket: 280, 0, 30, 82, 274, = 2 64 Bucket: 536, 0, 49, 63, 793, = 89 128 Bucket: 1048, 0, 514, 95, 187117, 12= 3359 VM OBJECT: 216, 0, 34974, 213300, 413596412, = 0 MAP: 232, 0, 7, 25, 7, = 0 KMAP ENTRY: 120, 150474, 183, 7040, 66175818, = 0 MAP ENTRY: 120, 0, 24333, 10759, 1025909468, = 0 DP fakepg: 120, 0, 216, 187, 544, = 0 SG fakepg: 120, 0, 667, 1689, 6212, = 0 mt_zone: 2056, 0, 295, 54, 295, = 0 16: 16, 0, 3916, 13724, 325519968, = 0 32: 32, 0, 3598, 1654, 65243323, = 0 64: 64, 0, 16671, 29529, 905000121, = 0 128: 128, 0, 12097, 30011, 473239749, = 0 256: 256, 0, 3300, 4650, 793867992, = 0 512: 512, 0, 5211, 7690, 54770551, = 0 1024: 1024, 0, 239, 697, 5912498, = 0 2048: 2048, 0, 1335, 721, 35819870, = 0 4096: 4096, 0, 715, 617, 7671562, = 0 =46iles: 80, 0, 2258, 1162, 323524959, = 0 TURNSTILE: 136, 0, 1673, 67, 1677, = 0 umtx pi: 96, 0, 0, 0, 0, = 0 MAC labels: 40, 0, 0, 0, 0, = 0 PROC: 1120, 0, 187, 824, 6336408, = 0 THREAD: 984, 0, 1236, 436, 20593, = 0 SLEEPQUEUE: 80, 0, 1673, 67, 1677, = 0 VMSPACE: 392, 0, 159, 831, 6334753, = 0 cpuset: 72, 0, 2, 98, 2, = 0 audit_record: 952, 0, 0, 0, 0, = 0 mbuf_packet: 256, 0, 371, 525, 348284916, = 0 mbuf: 256, 0, 231, 1888, 2247652340, = 0 mbuf_cluster: 2048, 25600, 896, 804, 188084, = 0 mbuf_jumbo_page: 4096, 12800, 27, 754, 593845769, = 0 mbuf_jumbo_9k: 9216, 6400, 0, 0, 0, = 0 mbuf_jumbo_16k: 16384, 3200, 0, 0, 0, = 0 mbuf_ext_refcnt: 4, 0, 0, 672, 774185, = 0 g_bio: 232, 0, 20, 1644, 54750899, = 0 ttyinq: 160, 0, 330, 126, 1275, = 0 ttyoutq: 256, 0, 168, 87, 632, = 0 ata_request: 320, 0, 10, 782, 21908538, = 0 ata_composite: 336, 0, 0, 0, 0, = 0 nv_stack_t: 12288, 0, 10, 10, 22, = 0 VNODE: 472, 0, 32144, 63544, 25299060, = 0 VNODEPOLL: 112, 0, 63, 135, 65, = 0 S VFS Cache: 108, 0, 23801, 111928, 38399522, = 0 L VFS Cache: 328, 0, 4752, 1368, 2652656, = 0 NAMEI: 1024, 0, 0, 764, 1639822943, = 0 NFSMOUNT: 616, 0, 0, 0, 0, = 0 NFSNODE: 656, 0, 0, 0, 0, = 0 DIRHASH: 1024, 0, 183, 433, 391168, = 0 pipe: 728, 0, 294, 811, 10116037, = 0 AIO: 208, 0, 1, 71, 9, = 0 AIOP: 32, 0, 4, 501, 220, = 0 AIOCB: 480, 0, 0, 752, 1725169, = 0 AIOL: 128, 0, 0, 0, 0, = 0 AIOLIO: 272, 0, 0, 0, 0, = 0 ksiginfo: 112, 0, 1002, 846, 6385628, = 0 itimer: 344, 0, 1, 21, 1, = 0 KNOTE: 128, 0, 109, 877, 13079149, = 0 socket: 680, 25602, 638, 1000, 20201690, = 0 unpcb: 240, 25600, 564, 892, 19533253, = 0 ipq: 56, 819, 0, 252, 77, = 0 udp_inpcb: 336, 25608, 23, 648, 490633, = 0 udpcb: 16, 25704, 23, 817, 490633, = 0 tcp_inpcb: 336, 25608, 82, 919, 177794, = 0 tcpcb: 880, 25600, 49, 815, 177794, = 0 tcptw: 72, 5150, 33, 867, 70594, = 0 syncache: 144, 15366, 0, 182, 62709, = 0 hostcache: 136, 15372, 44, 824, 2249, = 0 tcpreass: 40, 1680, 0, 924, 990793, = 0 sackhole: 32, 0, 2, 907, 380653, = 0 sctp_ep: 1272, 25602, 0, 0, 0, = 0 sctp_asoc: 2232, 40000, 0, 0, 0, = 0 sctp_laddr: 48, 80064, 0, 144, 2, = 0 sctp_raddr: 616, 80004, 0, 0, 0, = 0 sctp_chunk: 136, 400008, 0, 0, 0, = 0 sctp_readq: 104, 400032, 0, 0, 0, = 0 sctp_stream_msg_out: 96, 400026, 0, 0, 0, = 0 sctp_asconf: 40, 400008, 0, 0, 0, = 0 sctp_asconf_ack: 48, 400032, 0, 0, 0, = 0 ripcb: 336, 25608, 0, 33, 4, = 0 rtentry: 200, 0, 9, 48, 9, = 0 selfd: 56, 0, 2093, 931, 49944309845, = 0 SWAPMETA: 288, 116519, 134, 399, 67855, = 0 ip4flow: 56, 99351, 159, 1605, 582416, = 0 ip6flow: 80, 99360, 0, 0, 0, = 0 Mountpoints: 752, 0, 11, 14, 13, = 0 =46FS inode: 168, 0, 28873, 96813, 13425752, = 0 =46FS1 dinode: 128, 0, 0, 0, 0, = 0 =46FS2 dinode: 256, 0, 28873, 82712, 13425724, = 0 taskq_zone: 56, 0, 0, 630, 2294, = 0 zio_cache: 776, 0, 40, 1290, 91039501, = 0 zio_buf_512: 512, 0, 0, 0, 0, = 0 zio_data_buf_512: 512, 0, 0, 0, 0, = 0 zio_buf_1024: 1024, 0, 0, 0, 0, = 0 zio_data_buf_1024: 1024, 0, 0, 0, 0, = 0 zio_buf_1536: 1536, 0, 0, 0, 0, = 0 zio_data_buf_1536: 1536, 0, 0, 0, 0, = 0 zio_buf_2048: 2048, 0, 0, 0, 0, = 0 zio_data_buf_2048: 2048, 0, 0, 0, 0, = 0 zio_buf_2560: 2560, 0, 0, 0, 0, = 0 zio_data_buf_2560: 2560, 0, 0, 0, 0, = 0 zio_buf_3072: 3072, 0, 0, 0, 0, = 0 zio_data_buf_3072: 3072, 0, 0, 0, 0, = 0 zio_buf_3584: 3584, 0, 0, 0, 0, = 0 zio_data_buf_3584: 3584, 0, 0, 0, 0, = 0 zio_buf_4096: 4096, 0, 0, 0, 0, = 0 zio_data_buf_4096: 4096, 0, 0, 0, 0, = 0 zio_buf_5120: 5120, 0, 0, 0, 0, = 0 zio_data_buf_5120: 5120, 0, 0, 0, 0, = 0 zio_buf_6144: 6144, 0, 0, 0, 0, = 0 zio_data_buf_6144: 6144, 0, 0, 0, 0, = 0 zio_buf_7168: 7168, 0, 0, 0, 0, = 0 zio_data_buf_7168: 7168, 0, 0, 0, 0, = 0 zio_buf_8192: 8192, 0, 0, 0, 0, = 0 zio_data_buf_8192: 8192, 0, 0, 0, 0, = 0 zio_buf_10240: 10240, 0, 0, 0, 0, = 0 zio_data_buf_10240: 10240, 0, 0, 0, 0, = 0 zio_buf_12288: 12288, 0, 0, 0, 0, = 0 zio_data_buf_12288: 12288, 0, 0, 0, 0, = 0 zio_buf_14336: 14336, 0, 0, 0, 0, = 0 zio_data_buf_14336: 14336, 0, 0, 0, 0, = 0 zio_buf_16384: 16384, 0, 0, 0, 0, = 0 zio_data_buf_16384: 16384, 0, 0, 0, 0, = 0 zio_buf_20480: 20480, 0, 0, 0, 0, = 0 zio_data_buf_20480: 20480, 0, 0, 0, 0, = 0 zio_buf_24576: 24576, 0, 0, 0, 0, = 0 zio_data_buf_24576: 24576, 0, 0, 0, 0, = 0 zio_buf_28672: 28672, 0, 0, 0, 0, = 0 zio_data_buf_28672: 28672, 0, 0, 0, 0, = 0 zio_buf_32768: 32768, 0, 0, 0, 0, = 0 zio_data_buf_32768: 32768, 0, 0, 0, 0, = 0 zio_buf_36864: 36864, 0, 0, 0, 0, = 0 zio_data_buf_36864: 36864, 0, 0, 0, 0, = 0 zio_buf_40960: 40960, 0, 0, 0, 0, = 0 zio_data_buf_40960: 40960, 0, 0, 0, 0, = 0 zio_buf_45056: 45056, 0, 0, 0, 0, = 0 zio_data_buf_45056: 45056, 0, 0, 0, 0, = 0 zio_buf_49152: 49152, 0, 0, 0, 0, = 0 zio_data_buf_49152: 49152, 0, 0, 0, 0, = 0 zio_buf_53248: 53248, 0, 0, 0, 0, = 0 zio_data_buf_53248: 53248, 0, 0, 0, 0, = 0 zio_buf_57344: 57344, 0, 0, 0, 0, = 0 zio_data_buf_57344: 57344, 0, 0, 0, 0, = 0 zio_buf_61440: 61440, 0, 0, 0, 0, = 0 zio_data_buf_61440: 61440, 0, 0, 0, 0, = 0 zio_buf_65536: 65536, 0, 0, 0, 0, = 0 zio_data_buf_65536: 65536, 0, 0, 0, 0, = 0 zio_buf_69632: 69632, 0, 0, 0, 0, = 0 zio_data_buf_69632: 69632, 0, 0, 0, 0, = 0 zio_buf_73728: 73728, 0, 0, 0, 0, = 0 zio_data_buf_73728: 73728, 0, 0, 0, 0, = 0 zio_buf_77824: 77824, 0, 0, 0, 0, = 0 zio_data_buf_77824: 77824, 0, 0, 0, 0, = 0 zio_buf_81920: 81920, 0, 0, 0, 0, = 0 zio_data_buf_81920: 81920, 0, 0, 0, 0, = 0 zio_buf_86016: 86016, 0, 0, 0, 0, = 0 zio_data_buf_86016: 86016, 0, 0, 0, 0, = 0 zio_buf_90112: 90112, 0, 0, 0, 0, = 0 zio_data_buf_90112: 90112, 0, 0, 0, 0, = 0 zio_buf_94208: 94208, 0, 0, 0, 0, = 0 zio_data_buf_94208: 94208, 0, 0, 0, 0, = 0 zio_buf_98304: 98304, 0, 0, 0, 0, = 0 zio_data_buf_98304: 98304, 0, 0, 0, 0, = 0 zio_buf_102400: 102400, 0, 0, 0, 0, = 0 zio_data_buf_102400: 102400, 0, 0, 0, 0, = 0 zio_buf_106496: 106496, 0, 0, 0, 0, = 0 zio_data_buf_106496: 106496, 0, 0, 0, 0, = 0 zio_buf_110592: 110592, 0, 0, 0, 0, = 0 zio_data_buf_110592: 110592, 0, 0, 0, 0, = 0 zio_buf_114688: 114688, 0, 0, 0, 0, = 0 zio_data_buf_114688: 114688, 0, 0, 0, 0, = 0 zio_buf_118784: 118784, 0, 0, 0, 0, = 0 zio_data_buf_118784: 118784, 0, 0, 0, 0, = 0 zio_buf_122880: 122880, 0, 0, 0, 0, = 0 zio_data_buf_122880: 122880, 0, 0, 0, 0, = 0 zio_buf_126976: 126976, 0, 0, 0, 0, = 0 zio_data_buf_126976: 126976, 0, 0, 0, 0, = 0 zio_buf_131072: 131072, 0, 0, 0, 0, = 0 zio_data_buf_131072: 131072, 0, 0, 0, 0, = 0 dmu_buf_impl_t: 224, 0, 4177, 3830, 9957745, = 0 dnode_t: 776, 0, 3230, 665, 3277738, = 0 arc_buf_hdr_t: 208, 0, 2584, 6398, 6339574, = 0 arc_buf_t: 72, 0, 1380, 1970, 6784952, = 0 zil_lwb_cache: 200, 0, 1, 607, 5809, = 0 zfs_znode_cache: 376, 0, 2476, 734, 3979675, = 0 bridge_rtnode: 64, 0, 3, 165, 12, = 0 netstat -m 578/2437/3015 mbufs in use (current/cache/total) 350/1350/1700/25600 mbuf clusters in use (current/cache/total/max) 350/674 mbuf+clusters out of packet secondary zone in use (current/cache) 25/756/781/12800 4k (page size) jumbo clusters in use (current/cache/total/= max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 944K/6333K/7277K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 774157 requests for I/O initiated by sendfile 0 calls to protocol drain routines sysctl -a | grep vm.kmem vm.kmem_size_scale: 3 vm.kmem_size_max: 329853485875 vm.kmem_size_min: 0 vm.kmem_size: 1373954048 sysctl -a | grep vfs.zfs.arc vfs.zfs.arc_meta_limit: 214680320 vfs.zfs.arc_meta_used: 13797096 vfs.zfs.arc_min: 107340160 vfs.zfs.arc_max: 858721280 sysctl -a | grep kern.maxvnodes kern.maxvnodes: 800000 sysctl -a | grep vm.kvm vm.kvm_free: 547673337856 vm.kvm_size: 549755809792 =2D---- Alex V. Petrov