Date: Thu, 09 Feb 2012 10:27:33 +0600 From: "Eugene M. Zheganin" <emz@norma.perm.ru> To: freebsd-stable <freebsd-stable@FreeBSD.org> Subject: Re: zfs arc and amount of wired memory Message-ID: <4F334B35.5040708@norma.perm.ru> In-Reply-To: <4F32DB30.6020600@FreeBSD.org> References: <4F30E284.8080905@norma.perm.ru> <4F310115.3070507@FreeBSD.org> <4F310C5A.6070400@norma.perm.ru> <4F310E75.7090301@FreeBSD.org> <4F3144A9.2000505@norma.perm.ru> <4F314892.50806@FreeBSD.org> <4F314B5B.100@norma.perm.ru> <4F3186C6.8000904@FreeBSD.org> <4F324F10.2060508@norma.perm.ru> <4F32DB30.6020600@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi.
On 09.02.2012 02:29, Andriy Gapon wrote:
> on 08/02/2012 12:31 Eugene M. Zheganin said the following:
>> Hi.
>>
>> On 08.02.2012 02:17, Andriy Gapon wrote:
>>> [output snipped]
>>>
>>> Thank you. I don't see anything suspicious/unusual there.
>>> Just case, do you have ZFS dedup enabled by a chance?
>>>
>>> I think that examination of vmstat -m and vmstat -z outputs may provide some
>>> clues as to what got all that memory wired.
>>>
>> Nope, I don't have deduplication feature enabled.
> OK. So, did you have a chance to inspect vmstat -m and vmstat -z?
I did. I didn't understand it, but kinda 'felt the atmosphere'. It was
pretty much similar to the output I supplied below. Most of the sizes
were used by 'solaris' and numerous 'zio' caches.
>
> It could be very well possible that swap on zvol doesn't work well when the
> kernel itself is starved on memory.
>
>> So I want to ask - how to report it and what should I include in such pr ?
> I am leaving swap-on-zvol issue aside. Your original problem doesn't seem to be
> ZFS-related. I suspect that you might be running into some kernel memory leak.
> If you manage to reproduce the high wired value again, then vmstat -m and
> vmstat -z may provide some useful information.
>
> In this vein, do you use any out-of-tree kernel modules?
> Also, can you try to monitor your system to see when wired count grows?
>
Nope, I don't have any 3rd party kernel modules.
Yes, I can monitor it, but I have no idea what should I exactly monitor.
This system is running squid with a dozens of authentication helpers,
freeradius + postgresql, sendmail and a perl squid log parser, which
uses postgresql too. net/isc-dhcp, quagga, net/mpd5, a bunch of sendmail
milters, net/samba35, bind. So it's some kind of a corporate production
zoo. As I write this letter, the wired amount of memory increases by 70
megs. Excuse me, 80 megs now.
The output I promised (if it's MORE acceptable in the form of a link to
a paste site, just say it):
[emz@taiga:etc/snmp]# vmstat -m
Type InUse MemUse HighUse Requests Size(s)
hhook 2 1K - 2 128
ithread 85 14K - 85 32,128,256
KTRACE 100 13K - 100 128
linker 280 226K - 384
16,32,64,128,256,512,1024,2048,4096
lockf 94 10K - 20264872 64,128
loginclass 3 1K - 367 64
pci_link 13 2K - 13 16,128
ip6ndp 55 5K - 78 64,128
ip6opt 23 6K - 142134 32,256
temp 146 20K - 114199
16,32,64,128,256,512,1024,2048,4096
devbuf 28285 56235K - 29225
16,32,64,128,256,512,1024,2048,4096
module 291 37K - 291 128
USBdev 39 10K - 39 64,128,512,1024
mtx_pool 2 16K - 2
USB 55 166K - 58 16,32,64,128,256,512,2048,4096
osd 22 1K - 10870 16,64
ddb_capture 1 48K - 1
subproc 831 1312K - 56233 512,4096
proc 2 16K - 2
session 66 9K - 16431 128
pgrp 73 10K - 16581 128
cred 650 102K - 818736 64,256
uidinfo 15 4K - 5420 128,2048
plimit 25 7K - 4948 256
kbdmux 8 18K - 8 16,512,1024,2048
sysctltmp 0 0K - 9741241 16,32,64,128,4096
sysctloid 4837 243K - 4950 16,32,64,128
sysctl 0 0K - 50230 16,32,64
tidhash 1 16K - 1
callout 3 1536K - 3
umtx 2712 339K - 2766 128
p1003.1b 1 1K - 1 16
SWAP 2 1097K - 2 64
bus-sc 84 686K - 2193
16,32,64,128,256,512,1024,2048,4096
bus 861 78K - 4641 16,32,64,128,256,512,1024
devstat 4 9K - 4 32,4096
eventhandler 83 7K - 83 64,128
kobj 194 776K - 231 4096
Per-cpu 1 1K - 1 32
aacbuf 241 72K - 273 64,128,512
rman 219 23K - 449 16,32,128
acpiintr 1 1K - 1 64
sbuf 1 1K - 967
16,32,64,128,256,512,1024,2048,4096
acpica 1641 174K - 50289
16,32,64,128,256,512,1024,2048,4096
DEVFS1 106 53K - 111 512
DEVFS3 261 66K - 269 256
stack 0 0K - 2 256
taskqueue 85 8K - 121 16,32,64,128,1024
Unitno 21 1K - 208557 32,64
DEVFS2 106 2K - 108 16
DEVFS_RULE 54 26K - 54 64,512
DEVFS 39 1K - 40 16,128
Witness 1 128K - 1
iov 0 0K - 12708587 16,32,64,128,256,512
select 1081 136K - 1108 128
ioctlops 0 0K - 5846280
16,32,64,128,256,512,1024,2048,4096
msg 4 30K - 4 2048,4096
sem 4 106K - 4 2048,4096
shm 11 40K - 13055 2048
tty 23 23K - 29 1024,2048
pts 5 2K - 9 256
mbuf_tag 133 13K - 425972406 32,64,128
shmfd 1 8K - 1
DEVFSP 8 1K - 8 64
pcb 618 176K - 1042950 16,32,64,128,1024,2048,4096
soname 221 27K - 70492663 16,32,128
acl 0 0K - 1575 4096
vfscache 1 2048K - 1
vfs_hash 1 1024K - 1
vnodes 6 1K - 17 64,256
vnodemarker 0 0K - 72943 512
mount 256 16K - 1498 16,32,64,128,256,512
BPF 65 74K - 66 128,512,4096
ether_multi 564 32K - 786 16,32,64
ifaddr 330 93K - 339 16,32,64,128,256,512,2048,4096
ifnet 34 67K - 37 128,256,512,2048
clone 8 32K - 8 4096
arpcom 23 1K - 23 16
gif 1 1K - 1 256
lltable 767 209K - 15188 256,512
vlan 103 8K - 465 64,128
routetbl 840 45K - 13455 32,64,128,256,512
igmp 33 9K - 34 256
CARP 32 13K - 32 64,256,1024
ipid 2 24K - 2
ip_moptions 2 1K - 2 64,256
in_multi 23 6K - 24 256
in_mfilter 1 1K - 1 1024
encap_export_host 1 1K - 1 1024
sctp_iter 0 0K - 29 256
sctp_ifn 16 2K - 16 128
sctp_ifa 44 6K - 44 128
sctp_vrf 1 1K - 1 64
sctp_a_it 0 0K - 29 16
hostcache 1 28K - 1
syncache 1 96K - 1
fragment 0 0K - 18 64,128
ip6_moptions 10 2K - 10 32,256
in6_multi 188 28K - 196 32,256
in6_mfilter 5 5K - 5 1024
mld 33 5K - 34 128
NLM 0 0K - 1 32
rpc 64 8K - 398 16,32,64,128,256,512,1024,2048
audit_evclass 179 6K - 218 32
freework 1 1K - 1 64
newblk 1 128K - 1
bmsafemap 1 8K - 1
inodedep 1 1024K - 1
pagedep 1 128K - 1
vm_pgdata 2 129K - 2 128
UMAHash 2 5K - 6 512,1024,2048,4096
NFSD srvcache 0 0K - 91 128
pfs_nodes 21 6K - 21 256
acpitask 1 2K - 1 2048
memdesc 1 4K - 1 4096
GEOM 61 12K - 297 16,32,64,128,256,512,1024,2048
atkbddev 2 1K - 2 64
ata_pci 1 1K - 1 64
CAM dev queue 3 1K - 3 128
CAM queue 11 1K - 56 16,32
acpisem 16 2K - 16 128
CAM SIM 3 1K - 3 256
scsi_cd 0 0K - 4 16
CAM periph 4 1K - 18 16,32,64,128,256
isadev 7 1K - 7 128
entropy 1024 64K - 1024 64
apmdev 1 1K - 1 128
CAM XPT 29 14K - 80 32,64,128,1024,2048
UART 3 2K - 3 16,512,1024
cdev 8 2K - 8 256
io_apic 1 2K - 1 2048
sigio 1 1K - 3 64
filedesc 544 566K - 60333
16,32,64,128,256,512,1024,2048,4096
msi 2 1K - 2 128
nexusdev 3 1K - 3 16
kenv 104 12K - 108 16,32,64,128
kqueue 24 39K - 405980 256,2048,4096
acpidev 27 2K - 27 64
proc-args 219 24K - 449753 16,32,64,128,256
solaris 427482 2357264K - 228638905
16,32,64,128,256,512,1024,2048,4096
kstat_data 4 1K - 4 64
netgraph_msg 0 0K - 17743 64,128,256,512,1024
netgraph_node 11 3K - 26 256
netgraph_hook 14 2K - 46 128
netgraph 3 1K - 9 64,256,512,1024
netgraph_sock 5 1K - 10 128
netgraph_path 0 0K - 8912 16,32
netgraph_mppc 0 0K - 3 1024
netgraph_l2tp 2 3K - 2 128,2048
netgraph_ksock 1 1K - 2 128
netgraph_iface 1 1K - 2 128
netgraph_ppp 1 12K - 2
[emz@taiga:etc/snmp]# vmstat -z
ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP
UMA Kegs: 208, 0, 199, 5, 199, 0, 0
UMA Zones: 896, 0, 199, 1, 199, 0, 0
UMA Slabs: 568, 0, 38730, 827, 2045963, 0, 0
UMA RCntSlabs: 568, 0, 2417, 152, 10161, 0, 0
UMA Hash: 256, 0, 79, 11, 81, 0, 0
16 Bucket: 152, 0, 18, 132, 156, 0, 0
32 Bucket: 280, 0, 35, 119, 299, 0, 0
64 Bucket: 536, 0, 54, 72, 644, 57, 0
128 Bucket: 1048, 0, 467, 262, 112841,14481, 0
VM OBJECT: 216, 0, 39763, 2267, 1794570, 0, 0
MAP: 232, 0, 7, 25, 7, 0, 0
KMAP ENTRY: 120, 150412, 3666, 2131, 4810417, 0, 0
MAP ENTRY: 120, 0, 16746, 8457, 4147673, 0, 0
fakepg: 120, 0, 0, 0, 0, 0, 0
mt_zone: 4112, 0, 283, 77, 283, 0, 0
16: 16, 0, 65795, 565,103575464, 0, 0
32: 32, 0, 14388, 964,32512329, 0, 0
64: 64, 0, 164377, 14879,486891319, 0, 0
128: 128, 0, 17439, 2571,112770466, 0, 0
256: 256, 0, 12848, 15832,32654808, 0, 0
512: 512, 0, 149161, 4727, 6242680, 0, 0
1024: 1024, 0, 1970, 314, 151802, 0, 0
2048: 2048, 0, 1717, 349, 465184, 0, 0
4096: 4096, 0, 7621, 669, 375642, 0, 0
Files: 80, 0, 2210, 805,47573314, 0, 0
TURNSTILE: 136, 0, 1357, 83, 1384, 0, 0
umtx pi: 96, 0, 0, 0, 0, 0, 0
MAC labels: 40, 0, 0, 0, 0, 0, 0
PROC: 1160, 0, 259, 314, 55680, 0, 0
THREAD: 1112, 0, 864, 492, 53330, 0, 0
SLEEPQUEUE: 88, 0, 1357, 122, 1384, 0, 0
VMSPACE: 392, 0, 239, 681, 55662, 0, 0
cpuset: 72, 0, 2, 98, 2, 0, 0
audit_record: 960, 0, 0, 0, 0, 0, 0
mbuf_packet: 256, 0, 2107, 709,137988001, 0, 0
mbuf: 256, 0, 58204, 1065,523117709, 0, 0
mbuf_cluster: 2048, 25600, 2816, 914, 7273137, 0, 0
mbuf_jumbo_page: 4096, 12800, 9, 543, 7683953, 0, 0
mbuf_jumbo_9k: 9216, 6400, 0, 0, 0, 0, 0
mbuf_jumbo_16k: 16384, 3200, 0, 0, 0, 0, 0
mbuf_ext_refcnt: 4, 0, 0, 672, 29126, 0, 0
g_bio: 232, 0, 0, 752, 5733840, 0, 0
ttyinq: 160, 0, 240, 120, 555, 0, 0
ttyoutq: 256, 0, 127, 83, 293, 0, 0
ata_request: 328, 0, 0, 36, 14, 0, 0
ata_composite: 336, 0, 0, 0, 0, 0, 0
taskq_zone: 48, 0, 0, 792, 69760, 0, 0
VNODE: 480, 0, 37315, 2269, 429291, 0, 0
VNODEPOLL: 112, 0, 0, 0, 0, 0, 0
S VFS Cache: 108, 0, 16761, 28350, 349225, 0, 0
L VFS Cache: 328, 0, 18, 1086, 13342, 0, 0
NAMEI: 1024, 0, 0, 384,29340521, 0, 0
NCLNODE: 560, 0, 0, 0, 0, 0, 0
DIRHASH: 1024, 0, 0, 0, 0, 0, 0
zio_cache: 880, 0, 1, 1135,40366913, 0, 0
zio_link_cache: 48, 0, 0, 1440,11333105, 0, 0
zio_buf_512: 512, 0, 0, 0, 0, 0, 0
zio_data_buf_512: 512, 0, 0, 0, 0, 0, 0
zio_buf_1024: 1024, 0, 0, 0, 0, 0, 0
zio_data_buf_1024: 1024, 0, 0, 0, 0, 0, 0
zio_buf_1536: 1536, 0, 0, 0, 0, 0, 0
zio_data_buf_1536: 1536, 0, 0, 0, 0, 0, 0
zio_buf_2048: 2048, 0, 0, 0, 0, 0, 0
zio_data_buf_2048: 2048, 0, 0, 0, 0, 0, 0
zio_buf_2560: 2560, 0, 0, 0, 0, 0, 0
zio_data_buf_2560: 2560, 0, 0, 0, 0, 0, 0
zio_buf_3072: 3072, 0, 0, 0, 0, 0, 0
zio_data_buf_3072: 3072, 0, 0, 0, 0, 0, 0
zio_buf_3584: 3584, 0, 0, 0, 0, 0, 0
zio_data_buf_3584: 3584, 0, 0, 0, 0, 0, 0
zio_buf_4096: 4096, 0, 0, 0, 0, 0, 0
zio_data_buf_4096: 4096, 0, 0, 0, 0, 0, 0
zio_buf_5120: 5120, 0, 0, 0, 0, 0, 0
zio_data_buf_5120: 5120, 0, 0, 0, 0, 0, 0
zio_buf_6144: 6144, 0, 0, 0, 0, 0, 0
zio_data_buf_6144: 6144, 0, 0, 0, 0, 0, 0
zio_buf_7168: 7168, 0, 0, 0, 0, 0, 0
zio_data_buf_7168: 7168, 0, 0, 0, 0, 0, 0
zio_buf_8192: 8192, 0, 0, 0, 0, 0, 0
zio_data_buf_8192: 8192, 0, 0, 0, 0, 0, 0
zio_buf_10240: 10240, 0, 0, 0, 0, 0, 0
zio_data_buf_10240: 10240, 0, 0, 0, 0, 0, 0
zio_buf_12288: 12288, 0, 0, 0, 0, 0, 0
zio_data_buf_12288: 12288, 0, 0, 0, 0, 0, 0
zio_buf_14336: 14336, 0, 0, 0, 0, 0, 0
zio_data_buf_14336: 14336, 0, 0, 0, 0, 0, 0
zio_buf_16384: 16384, 0, 0, 0, 0, 0, 0
zio_data_buf_16384: 16384, 0, 0, 0, 0, 0, 0
zio_buf_20480: 20480, 0, 0, 0, 0, 0, 0
zio_data_buf_20480: 20480, 0, 0, 0, 0, 0, 0
zio_buf_24576: 24576, 0, 0, 0, 0, 0, 0
zio_data_buf_24576: 24576, 0, 0, 0, 0, 0, 0
zio_buf_28672: 28672, 0, 0, 0, 0, 0, 0
zio_data_buf_28672: 28672, 0, 0, 0, 0, 0, 0
zio_buf_32768: 32768, 0, 0, 0, 0, 0, 0
zio_data_buf_32768: 32768, 0, 0, 0, 0, 0, 0
zio_buf_36864: 36864, 0, 0, 0, 0, 0, 0
zio_data_buf_36864: 36864, 0, 0, 0, 0, 0, 0
zio_buf_40960: 40960, 0, 0, 0, 0, 0, 0
zio_data_buf_40960: 40960, 0, 0, 0, 0, 0, 0
zio_buf_45056: 45056, 0, 0, 0, 0, 0, 0
zio_data_buf_45056: 45056, 0, 0, 0, 0, 0, 0
zio_buf_49152: 49152, 0, 0, 0, 0, 0, 0
zio_data_buf_49152: 49152, 0, 0, 0, 0, 0, 0
zio_buf_53248: 53248, 0, 0, 0, 0, 0, 0
zio_data_buf_53248: 53248, 0, 0, 0, 0, 0, 0
zio_buf_57344: 57344, 0, 0, 0, 0, 0, 0
zio_data_buf_57344: 57344, 0, 0, 0, 0, 0, 0
zio_buf_61440: 61440, 0, 0, 0, 0, 0, 0
zio_data_buf_61440: 61440, 0, 0, 0, 0, 0, 0
zio_buf_65536: 65536, 0, 0, 0, 0, 0, 0
zio_data_buf_65536: 65536, 0, 0, 0, 0, 0, 0
zio_buf_69632: 69632, 0, 0, 0, 0, 0, 0
zio_data_buf_69632: 69632, 0, 0, 0, 0, 0, 0
zio_buf_73728: 73728, 0, 0, 0, 0, 0, 0
zio_data_buf_73728: 73728, 0, 0, 0, 0, 0, 0
zio_buf_77824: 77824, 0, 0, 0, 0, 0, 0
zio_data_buf_77824: 77824, 0, 0, 0, 0, 0, 0
zio_buf_81920: 81920, 0, 0, 0, 0, 0, 0
zio_data_buf_81920: 81920, 0, 0, 0, 0, 0, 0
zio_buf_86016: 86016, 0, 0, 0, 0, 0, 0
zio_data_buf_86016: 86016, 0, 0, 0, 0, 0, 0
zio_buf_90112: 90112, 0, 0, 0, 0, 0, 0
zio_data_buf_90112: 90112, 0, 0, 0, 0, 0, 0
zio_buf_94208: 94208, 0, 0, 0, 0, 0, 0
zio_data_buf_94208: 94208, 0, 0, 0, 0, 0, 0
zio_buf_98304: 98304, 0, 0, 0, 0, 0, 0
zio_data_buf_98304: 98304, 0, 0, 0, 0, 0, 0
zio_buf_102400: 102400, 0, 0, 0, 0, 0, 0
zio_data_buf_102400: 102400, 0, 0, 0, 0, 0, 0
zio_buf_106496: 106496, 0, 0, 0, 0, 0, 0
zio_data_buf_106496: 106496, 0, 0, 0, 0, 0, 0
zio_buf_110592: 110592, 0, 0, 0, 0, 0, 0
zio_data_buf_110592: 110592, 0, 0, 0, 0, 0, 0
zio_buf_114688: 114688, 0, 0, 0, 0, 0, 0
zio_data_buf_114688: 114688, 0, 0, 0, 0, 0, 0
zio_buf_118784: 118784, 0, 0, 0, 0, 0, 0
zio_data_buf_118784: 118784, 0, 0, 0, 0, 0, 0
zio_buf_122880: 122880, 0, 0, 0, 0, 0, 0
zio_data_buf_122880: 122880, 0, 0, 0, 0, 0, 0
zio_buf_126976: 126976, 0, 0, 0, 0, 0, 0
zio_data_buf_126976: 126976, 0, 0, 0, 0, 0, 0
zio_buf_131072: 131072, 0, 0, 0, 0, 0, 0
zio_data_buf_131072: 131072, 0, 0, 0, 0, 0, 0
sa_cache: 80, 0, 37248, 2262, 429178, 0, 0
dnode_t: 856, 0, 151801, 1919, 249658, 0, 0
dmu_buf_impl_t: 224, 0, 177337, 2404, 600337, 0, 0
arc_buf_hdr_t: 216, 0, 105487, 1253, 519289, 0, 0
arc_buf_t: 104, 0, 38391, 5205, 559830, 0, 0
zil_lwb_cache: 192, 0, 9, 611, 80899, 0, 0
zfs_znode_cache: 400, 0, 37248, 2325, 429178, 0, 0
pipe: 728, 0, 42, 328, 39330, 0, 0
Mountpoints: 768, 0, 20, 20, 20, 0, 0
ksiginfo: 112, 0, 564, 657, 29202, 0, 0
itimer: 344, 0, 1, 21, 1, 0, 0
KNOTE: 128, 0, 877, 776,90925709, 0, 0
pfsrctrpl: 152, 10000, 0, 0, 0, 0, 0
pfrulepl: 936, 0, 325, 11, 325, 0, 0
pfstatepl: 288, 10010, 3360, 2568, 1170450, 0, 0
pfstatekeypl: 288, 0, 3983, 2621, 1916481, 0, 0
pfstateitempl: 288, 0, 3983, 2439, 1439637, 0, 0
pfaltqpl: 240, 0, 0, 0, 0, 0, 0
pfpooladdrpl: 88, 0, 71, 55, 71, 0, 0
pfrktable: 1296, 1002, 6, 9, 11, 0, 0
pfrkentry: 160, 200016, 21, 27, 21, 0, 0
pffrent: 32, 5050, 0, 0, 0, 0, 0
pffrag: 80, 0, 0, 0, 0, 0, 0
pffrcache: 80, 10035, 0, 0, 0, 0, 0
pffrcent: 24, 50022, 0, 0, 0, 0, 0
pfstatescrub: 40, 0, 0, 0, 0, 0, 0
pfiaddrpl: 120, 0, 0, 0, 0, 0, 0
pfospfen: 112, 0, 700, 26, 700, 0, 0
pfosfp: 40, 0, 410, 94, 410, 0, 0
pfsync: 88, 0, 0, 0, 0, 0, 0
socket: 680, 25602, 1726, 596,25837811, 0, 0
ipq: 56, 819, 0, 126, 396, 0, 0
udp_inpcb: 392, 25600, 152, 728,20355207, 0, 0
udpcb: 16, 25704, 152, 856,20355207, 0, 0
tcp_inpcb: 392, 25600, 1782, 4968, 5284799, 0, 0
tcpcb: 976, 25600, 982, 762, 5284799, 0, 0
tcptw: 72, 5150, 800, 4350, 306578,97901, 0
syncache: 152, 15375, 0, 325, 5052872, 0, 0
hostcache: 136, 15372, 2053, 467, 7364, 0, 0
tcpreass: 40, 1680, 3, 585, 216370, 0, 0
sackhole: 32, 0, 0, 909, 55622, 0, 0
sctp_ep: 1368, 25600, 0, 0, 0, 0, 0
sctp_asoc: 2280, 40000, 0, 0, 0, 0, 0
sctp_laddr: 48, 80064, 0, 288, 43, 0, 0
sctp_raddr: 704, 80000, 0, 0, 0, 0, 0
sctp_chunk: 136, 400008, 0, 0, 0, 0, 0
sctp_readq: 104, 400032, 0, 0, 0, 0, 0
sctp_stream_msg_out: 112, 400026, 0, 0, 0, 0, 0
sctp_asconf: 40, 400008, 0, 0, 0, 0, 0
sctp_asconf_ack: 48, 400032, 0, 0, 0, 0, 0
ripcb: 392, 25600, 6, 34, 8, 0, 0
unpcb: 240, 25600, 570, 550, 197771, 0, 0
rtentry: 200, 0, 756, 137, 1395, 0, 0
selfd: 56, 0, 1504, 701,705385407, 0, 0
SWAPMETA: 288, 116519, 4, 113, 866, 0, 0
NetGraph items: 72, 4118, 0, 812, 115049, 0, 0
NetGraph data items: 72, 522, 0, 493, 189271, 0, 0
P.S. after I pasted this, the wired memory gone down by 70 megs, the
final output (I guess enough is enough can be seen here:
http://dpaste.org/iA5mP/ ).
Thanks.
Eugene.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4F334B35.5040708>