Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 22 Nov 2006 12:41:04 -0800
From:      Jonathan Feally <vulture@netvulture.com>
To:        freebsd-stable@freebsd.org, freebsd-net@freebsd.org
Subject:   6-STABLE (6.2-PRE) and applications (named natd dhcpd) getting stuck in state zoneli (zone limit) - dynamic ipfw rules not working after time - vlans on em
Message-ID:  <4564B5E0.3000404@netvulture.com>

next in thread | raw e-mail | index | archive | help
Sorry to cross post, but the net list didn't help a couple weeks back on 
this.

names, natd, and dhcpd have all been getting stuck in zoneli (zone 
limit) since I upgraded to the box to stable about a month ago. It was 
running a 6.1-STABLE before with out difficulty. Very little has changed 
on the box. All the same applications, same ipfw rules for the most part 
(just more rules for new customers). Most of the time the processes 
cannot be killed. I did get lucky yesterday with dhcpd. It finally died 
about an hour later. I was compiling the latest stable at the time. But 
got up again today to find dhcpd in zoneli. Can someone please point me 
in the correct direction to trouble shoot this problem. I don't really 
know how to get a full dump of what a process is doing, so a quick what 
to do and post back would be great.

More Info On the Setup:
This box is acting as a all-in-one router with traffic shaping.
It has a single em card with 10 tagged vlans on it routing the majority 
of traffic from the customer vlan (vlan125) to the isp vlan (vlan901). 
It also runs dhcpd and named to service the customers on vlan125. All of 
the customer's have a set of ipfw rules like the following.


# ipfw queue 11206 show
q11206: weight 40 pipe 1206  384 KB 1 queues (1 buckets) droptail
    mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes 
Pkt/Byte Drp
  0 tcp    x.y.125.136/1925    a.b.247.137/9580  264461 111550268  0    
0   0

# ipfw queue 21206 show
q21206: weight 60 pipe 1206  384 KB 1 queues (1 buckets) droptail
    mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes 
Pkt/Byte Drp
  0 udp     c.d.55.125/36049   x.y.125.136/17307 239325 38512205  1   76   0

# ipfw queue 31206 show
q31206: weight 21 pipe 10000  384 KB 1 queues (1 buckets) droptail
    mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes 
Pkt/Byte Drp
  0 tcp    x.y.125.136/1925    a.b.247.137/9580  265850 112136314  0    
0   0

# ipfw queue 41206 show
q41206: weight 21 pipe 20000  384 KB 1 queues (1 buckets) droptail
    mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes 
Pkt/Byte Drp
  0 udp     c.d.55.125/36049   x.y.125.136/17307 239447 38467203  0    0   0

# ipfw pipe 1206 show
01206:   2.090 Mbit/s    0 ms  768 KB 0 queues (1 buckets) droptail
q11206: weight 40 pipe 1206  384 KB 1 queues (1 buckets) droptail
    mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes 
Pkt/Byte Drp
  0 tcp    x.y.125.136/1925    a.b.247.137/9580  267269 113411268  0    
0   0
q21206: weight 60 pipe 1206  384 KB 1 queues (1 buckets) droptail
    mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes 
Pkt/Byte Drp
  0 udp     c.d.55.125/36049   x.y.125.136/17307 241121 38680715  0    0   0

# ipfw pipe 10000 show
10000:  20.224 Mbit/s    0 ms   50 sl. 0 queues (1 buckets) droptail
q31206: weight 21 pipe 10000  384 KB 1 queues (1 buckets) droptail
    mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes 
Pkt/Byte Drp
  0 tcp    x.y.125.136/1925    a.b.247.137/9580  268208 113672991  0    
0   0

# ipfw pipe 20000 show
20000:  20.224 Mbit/s    0 ms   50 sl. 0 queues (1 buckets) droptail
q41206: weight 21 pipe 20000  384 KB 1 queues (1 buckets) droptail
    mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes 
Pkt/Byte Drp
  0 udp     c.d.55.125/36049   x.y.125.136/17307 241392 38632811  0    0   0

# The 10000 and 20000 pipes are for the whole link, while the 1206 pipe 
is the customers alloted bandwidth. Customer cannot exceed his 
bandwidth, and if the whole connection is maxed out he would be weighted 
at 21 against other customers competing for isp bandwidth. The more 
bandwidth a customer pays for the bigger his weight in the main pipes.


#Customer rules (#10000-#49999)
19000       0          0 reset tcp from any 25,135,139,445 to x.y.125.136
19000       4        176 reset tcp from any to x.y.125.136 dst-port 
25,135,139,445
19000       0          0 reset tcp from x.y.125.136 to any dst-port 
25,135,139,445
19001  216130   87769342 queue 11206 ip from x.y.125.136 to any out
19002  197604   34923736 queue 21206 ip from any to x.y.125.136 out
19003  216635   87807692 queue 31206 ip from x.y.125.136 to any in
19004  197283   34831273 queue 41206 ip from any to x.y.125.136 in
19005  432765  175577034 allow ip from x.y.125.136 to any
19006  394887   69755009 allow ip from any to x.y.125.136

#Other keep state rules - the keep-state rules will stop working after a 
while.
50000    3325     258473 allow udp from any to any dst-port 53
50000    3020     938086 allow udp from any 53 to any
50001   41462   28661128 divert 8668 ip from any to any via vlan901
50002       0          0 check-state
# Used to forward all un-provisioned customer machines to the local 
httpd so their mac can be recorded.
51004     358      23712 fwd 127.0.0.1,8080 tcp from x.y.125.0/24 to any 
dst-port 80 in keep-state
# Drop anybody not allowed above
51099    1158     141144 deny ip from x.y.125.0/24 to any
# Allow admin vlan1 to go anywhere
60003     257      50939 allow ip from 10.255.1.0/24 to any keep-state
$ Final drop
65000     430      44547 reset tcp from any to any
65001    1887     424765 deny ip from any to any


Below are some dumps of process states when it locks up

#NATD zoneli
  UID   PID  PPID CPU PRI NI   VSZ   RSS MWCHAN STAT  TT       TIME COMMAND
    0     0     0   0  96  0     0     0 -      WLs   ??    0:00.00 
[swapper]
    0     1     0   0   8  0   772   368 wait   ILs   ??    0:00.80 
/sbin/init --
    0     2     0   0   8  0     0     8 crypto DL    ??    0:00.00 [crypto]
    0     3     0   0   8  0     0     8 crypto DL    ??    0:00.00 
[crypto returns]
    0     4     0   0  -8  0     0     8 -      DL    ??    0:12.55 
[g_event]
    0     5     0   0  -8  0     0     8 -      DL    ??    0:17.83 [g_up]
    0     6     0   0  -8  0     0     8 -      DL    ??    0:18.84 [g_down]
    0     7     0   0   8  0     0     8 -      DL    ??    0:00.00 
[kqueue taskq]
    0     8     0   0   8  0     0     8 -      DL    ??    0:00.00 
[acpi_task_0]
    0     9     0   0   8  0     0     8 -      DL    ??    0:00.00 
[acpi_task_1]
    0    10     0 133 171  0     0     8 -      RL    ??  4032:07.99 
[idle: cpu0]
    0    11     0   5 -44  0     0     8 -      WL    ??  215:09.39 
[swi1: net]
    0    12     0   0 -32  0     0     8 -      WL    ??  195:44.05 
[swi4: clock sio]
    0    13     0   0 -36  0     0     8 -      WL    ??    0:00.00 
[swi3: vm]
    0    14     0   0 -16  0     0     8 -      DL    ??   11:00.18 [yarrow]
    0    15     0   0 -40  0     0     8 -      WL    ??    0:00.00 
[swi2: cambio]
    0    16     0   0 -24  0     0     8 -      WL    ??    0:00.00 
[swi6: task queue]
    0    17     0   0   8  0     0     8 -      DL    ??    0:00.00 
[acpi_task_2]
    0    18     0   0 -24  0     0     8 -      WL    ??    0:00.00 
[swi6: Giant taskq]
    0    19     0   0   8  0     0     8 -      DL    ??    0:00.00 
[thread taskq]
    0    20     0   0 -28  0     0     8 -      WL    ??    0:00.00 
[swi5: +]
    0    21     0   0 -52  0     0     8 -      WL    ??    0:00.00 
[irq9: acpi0]
    0    22     0   0 -64  0     0     8 -      WL    ??    0:00.00 
[irq16: uhci0 uhci3]
    0    23     0   0   8  0     0     8 usbevt DL    ??    0:00.02 [usb0]
    0    24     0   0   8  0     0     8 usbtsk DL    ??    0:00.00 
[usbtask]
    0    25     0   0 -64  0     0     8 -      WL    ??    0:00.00 
[irq19: uhci1]
    0    26     0   0   8  0     0     8 usbevt DL    ??    0:00.01 [usb1]
    0    27     0   0 -64  0     0     8 -      WL    ??    0:08.08 
[irq18: uhci2+]
    0    28     0   0   8  0     0     8 usbevt DL    ??    0:00.02 [usb2]
    0    29     0   0   8  0     0     8 usbevt DL    ??    0:00.01 [usb3]
    0    30     0   0 -64  0     0     8 -      WL    ??    0:00.00 
[irq23: ehci0]
    0    31     0   0   8  0     0     8 usbevt DL    ??    0:00.02 [usb4]
    0    32     0   1 -68  0     0     8 -      DL    ??   46:33.63 [em0 
taskq]
    0    33     0   0 -68  0     0     8 -      WL    ??    0:00.00 
[irq20: fxp0]
    0    34     0   0 -64  0     0     8 -      WL    ??    0:00.00 
[irq14: ata0]
    0    35     0   0 -64  0     0     8 -      WL    ??    0:00.00 
[irq15: ata1]
    0    36     0   0 -60  0     0     8 -      WL    ??    0:00.00 
[irq1: atkbd0]
    0    37     0   0 -48  0     0     8 -      WL    ??    0:00.00 
[swi0: sio]
    0    38     0   0 -60  0     0     8 -      WL    ??    0:00.00 
[irq7: ppc0]
    0    39     0   0 -16  0     0     8 psleep DL    ??    0:00.29 
[pagedaemon]
    0    40     0   0  20  0     0     8 psleep DL    ??    0:00.00 
[vmdaemon]
    0    41     0   0 171  0     0     8 pgzero DL    ??    1:07.23 
[pagezero]
    0    42     0   0 -16  0     0     8 psleep DL    ??    0:01.50 
[bufdaemon]
    0    43     0   0  -4  0     0     8 vlruwt DL    ??    0:01.90 [vnlru]
    0    44     0   0  20  0     0     8 syncer DL    ??    2:28.29 [syncer]
    0    45     0   0 -16  0     0     8 sdflus DL    ??    0:03.68 
[softdepflush]
    0    46     0   0 -64  0     0     8 -      DL    ??    0:19.45 
[schedcpu]
    0   163     1 167  20  0  1252   728 pause  Is    ??    0:00.00 
adjkerntz -i
    0  1242     1 167 116  0   528   388 select Is    ??    0:00.00 
/sbin/devd
    0  1486     1   0 -16  0  1572  1112 zoneli Ds    ??    1:47.61 
/sbin/natd -f /etc/natd.conf -n vlan901
    0  1645     1   0  96  0  1404  1112 select Ss    ??    0:18.82 
/usr/sbin/syslogd -l /var/run/log -l /var/named/var/run/log -c
   53  1707     1   0  96  0 13552 12700 select Ss    ??    7:38.07 
/usr/sbin/named -t /var/named -u bind
    0  1717     1   0  96  0  1504  1216 select Is    ??    0:00.26 
/usr/sbin/rpcbind
    0  1772     1 176 118  0  1492  1256 select Is    ??    0:00.00 
/usr/sbin/mountd -r
    0  1774     1 176 118  0  1372  1104 select Is    ??    0:00.03 
nfsd: master (nfsd)
    0  1775  1774 176   4  0  1280   888 -      I     ??    0:00.00 
nfsd: server (nfsd)
    0  1776  1774 176   4  0  1280   888 -      I     ??    0:00.00 
nfsd: server (nfsd)
    0  1777  1774 176   4  0  1280   888 -      I     ??    0:00.00 
nfsd: server (nfsd)
    0  1778  1774 176   4  0  1280   888 -      I     ??    0:00.00 
nfsd: server (nfsd)
    0  1784     1   0  96  0 263584  1112 select Is    ??    0:00.20 
/usr/sbin/rpc.statd
    0  1789     1   0  96  0  1596  1364 select Is    ??    0:00.24 
rpc.lockd: server (rpc.lockd)
    1  1796  1789 176   4  0  1596  1364 nfsloc I     ??    0:00.00 
rpc.lockd: client (rpc.lockd)
    0  1808     1   0  96  0  3204  2508 select Ss    ??    0:01.08 
/usr/local/sbin/racoon
    0  1853     1   0  96  0  2924  1824 select Ss    ??    0:14.87 
/usr/sbin/ntpd -c /etc/ntp.conf -p /var/run/ntpd.pid -f /var/db
    0  1873     1   0  96  0  1288   824 select Is    ??    0:00.36 
/usr/sbin/usbd
    0  1902     1   0  96  0  3520  2648 select Is    ??    0:13.45 
/usr/sbin/sshd
    0  1907     1   0  96  0  3508  2948 select Ss    ??    0:05.87 
sendmail: accepting connections (sendmail)
   25  1911     1   0  20  0  3384  2804 pause  Is    ??    0:00.10 
sendmail: Queue runner@00:30:00 for /var/spool/clientmqueue (se
    0  1917     1   0   8  0  1388  1100 nanslp Is    ??    0:30.23 
/usr/sbin/cron -s
    0  6535     1   0   8  0 11708  9140 nanslp Ss    ??    0:10.04 
/usr/local/apache2/bin/httpd -k start
    0  6565     1   0  96  0  1468  1220 select Is    ??    0:00.00 
/usr/sbin/inetd -wW -C 60
65534 18455  6535   0  20  0 12040  9488 lockf  I     ??    0:00.02 
/usr/local/apache2/bin/httpd -k start
65534 51018  6535   0  20  0 12136  9632 lockf  I     ??    0:00.09 
/usr/local/apache2/bin/httpd -k start
65534 51041  6535   0   4  0 12128  9624 kqread I     ??    0:00.12 
/usr/local/apache2/bin/httpd -k start
65534 51042  6535   0  20  0 12064  9560 lockf  I     ??    0:00.03 
/usr/local/apache2/bin/httpd -k start
65534 51439  6535   0  20  0 12140  9632 lockf  I     ??    0:00.08 
/usr/local/apache2/bin/httpd -k start
65534 51462  6535   0  20  0 12196  9700 lockf  I     ??    0:00.13 
/usr/local/apache2/bin/httpd -k start
65534 51463  6535   0  20  0 12064  9552 lockf  I     ??    0:00.10 
/usr/local/apache2/bin/httpd -k start
65534 51476  6535   0  20  0 12060  9512 lockf  I     ??    0:00.02 
/usr/local/apache2/bin/httpd -k start
65534 51477  6535   0  20  0 12152  9648 lockf  I     ??    0:00.11 
/usr/local/apache2/bin/httpd -k start
65534 51479  6535   0  20  0 12048  9540 lockf  I     ??    0:00.07 
/usr/local/apache2/bin/httpd -k start
 1001 82596     1   0  96  0  2732  2452 select Ss    ??    0:03.93 
/usr/local/sbin/dhcpd -cf /usr/local/etc/dhcpd.conf -lf /var/db
    0 88474  1902   0   4  0  6272  3276 sbwait Is    ??    0:00.04 
sshd: vulture [priv] (sshd)
 1002 88653 88474   0  96  0  6256  3280 select S     ??    0:00.23 
sshd: vulture@ttyp0 (sshd)
    0  6583     1 172   5  0  1344   952 ttyin  Is+   v0    0:00.00 
/usr/libexec/getty Pc ttyv0
    0  6584     1 172   5  0  1344   952 ttyin  Is+   v1    0:00.00 
/usr/libexec/getty Pc ttyv1
    0  6585     1 172   5  0  1344   952 ttyin  Is+   v2    0:00.00 
/usr/libexec/getty Pc ttyv2
    0  6586     1 172   5  0  1344   952 ttyin  Is+   v3    0:00.00 
/usr/libexec/getty Pc ttyv3
    0  6587     1 172   5  0  1344   952 ttyin  Is+   v4    0:00.00 
/usr/libexec/getty Pc ttyv4
    0  6588     1 172   5  0  1344   952 ttyin  Is+   v5    0:00.00 
/usr/libexec/getty Pc ttyv5
    0  6589     1 172   5  0  1344   952 ttyin  Is+   v6    0:00.00 
/usr/libexec/getty Pc ttyv6
    0  6590     1 172   5  0  1344   952 ttyin  Is+   v7    0:00.00 
/usr/libexec/getty Pc ttyv7
    0  1882     1   0 -58  0  3204  2832 bpf    S    con-   0:06.62 
/usr/local/sbin/arpwatch -i vlan125 -f arp.vlan125.dat
 1002 88657 88653   0   8  0  3196  2052 wait   Is    p0    0:00.01 
-bash (bash)
    0 88661 88657   0   8  0  1776  1416 wait   I     p0    0:00.01 su -
    0 88662 88661   1   8  0  3232  2092 wait   S     p0    0:00.06 -su 
(bash)
    0 90806 88662   0  96  0  1500   988 -      R+    p0    0:00.00 ps -axl



#Named zoneli
  UID   PID  PPID CPU PRI NI   VSZ   RSS MWCHAN STAT  TT       TIME 
COMMAND          USER   %CPU %MEM STARTED
    0    10     0 269 171  0     0     8 -      RL    ??  3841:42.30 
[idle: cpu0]     root   79.2  0.0 Sun05PM
    0 57074 56659  87 106  0  4892  4472 select S+    v1    0:10.89 
cvsup system/sta root    9.6  0.9  5:00PM
    0    11     0  42 -44  0     0     8 -      WL    ??  190:27.13 
[swi1: net]      root    4.8  0.0 Sun05PM
    0     0     0   0  96  0     0     0 -      WLs   ??    0:00.00 
[swapper]        root    0.0  0.0 Sun05PM
    0     1     0   0   8  0   772   368 wait   ILs   ??    0:00.08 
/sbin/init --    root    0.0  0.1 Sun05PM
    0     2     0   0   8  0     0     8 crypto DL    ??    0:00.00 
[crypto]         root    0.0  0.0 Sun05PM
    0     3     0   0   8  0     0     8 crypto DL    ??    0:00.00 
[crypto returns] root    0.0  0.0 Sun05PM
    0     4     0   0  -8  0     0     8 -      DL    ??    0:11.47 
[g_event]        root    0.0  0.0 Sun05PM
    0     5     0   0  -8  0     0     8 -      DL    ??    0:14.95 
[g_up]           root    0.0  0.0 Sun05PM
    0     6     0   1  -8  0     0     8 -      DL    ??    0:17.02 
[g_down]         root    0.0  0.0 Sun05PM
    0     7     0   0   8  0     0     8 -      DL    ??    0:00.00 
[kqueue taskq]   root    0.0  0.0 Sun05PM
    0     8     0   0   8  0     0     8 -      DL    ??    0:00.00 
[acpi_task_0]    root    0.0  0.0 Sun05PM
    0     9     0   0   8  0     0     8 -      DL    ??    0:00.00 
[acpi_task_1]    root    0.0  0.0 Sun05PM
    0    12     0   1 -32  0     0     8 -      WL    ??  148:47.86 
[swi4: clock sio root    0.0  0.0 Sun05PM
    0    13     0   0 -36  0     0     8 -      WL    ??    0:00.00 
[swi3: vm]       root    0.0  0.0 Sun05PM
    0    14     0   0 -16  0     0     8 -      DL    ??    8:07.19 
[yarrow]         root    0.0  0.0 Sun05PM
    0    15     0   0 -40  0     0     8 -      WL    ??    0:00.00 
[swi2: cambio]   root    0.0  0.0 Sun05PM
    0    16     0   0 -24  0     0     8 -      WL    ??    0:00.00 
[swi6: task queu root    0.0  0.0 Sun05PM
    0    17     0   0   8  0     0     8 -      DL    ??    0:00.00 
[acpi_task_2]    root    0.0  0.0 Sun05PM
    0    18     0   1 -24  0     0     8 -      WL    ??    0:00.56 
[swi6: Giant tas root    0.0  0.0 Sun05PM
    0    19     0   0   8  0     0     8 -      DL    ??    0:00.00 
[thread taskq]   root    0.0  0.0 Sun05PM
    0    20     0   0 -28  0     0     8 -      WL    ??    0:00.00 
[swi5: +]        root    0.0  0.0 Sun05PM
    0    21     0   0 -52  0     0     8 -      WL    ??    0:00.00 
[irq9: acpi0]    root    0.0  0.0 Sun05PM
    0    22     0   0 -64  0     0     8 -      WL    ??    0:00.00 
[irq16: uhci0 uh root    0.0  0.0 Sun05PM
    0    23     0   0   8  0     0     8 usbevt DL    ??    0:00.01 
[usb0]           root    0.0  0.0 Sun05PM
    0    24     0   0   8  0     0     8 usbtsk DL    ??    0:00.00 
[usbtask]        root    0.0  0.0 Sun05PM
    0    25     0   0 -64  0     0     8 -      WL    ??    0:00.00 
[irq19: uhci1]   root    0.0  0.0 Sun05PM
    0    26     0   0   8  0     0     8 usbevt DL    ??    0:00.01 
[usb1]           root    0.0  0.0 Sun05PM
    0    27     0   1 -64  0     0     8 -      WL    ??    0:06.87 
[irq18: uhci2+]  root    0.0  0.0 Sun05PM
    0    28     0   0   8  0     0     8 usbevt DL    ??    0:00.01 
[usb2]           root    0.0  0.0 Sun05PM
    0    29     0   0   8  0     0     8 usbevt DL    ??    0:00.01 
[usb3]           root    0.0  0.0 Sun05PM
    0    30     0   0 -64  0     0     8 -      WL    ??    0:00.00 
[irq23: ehci0]   root    0.0  0.0 Sun05PM
    0    31     0   0   8  0     0     8 usbevt DL    ??    0:00.02 
[usb4]           root    0.0  0.0 Sun05PM
    0    32     0   4 -68  0     0     8 -      DL    ??   36:30.75 [em0 
taskq]      root    0.0  0.0 Sun05PM
    0    33     0   0 -68  0     0     8 -      WL    ??    0:00.00 
[irq20: fxp0]    root    0.0  0.0 Sun05PM
    0    34     0   0 -64  0     0     8 -      WL    ??    0:00.00 
[irq14: ata0]    root    0.0  0.0 Sun05PM
    0    35     0   0 -64  0     0     8 -      WL    ??    0:00.00 
[irq15: ata1]    root    0.0  0.0 Sun05PM
    0    36     0   0 -60  0     0     8 -      WL    ??    0:00.04 
[irq1: atkbd0]   root    0.0  0.0 Sun05PM
    0    37     0   0 -48  0     0     8 -      WL    ??    0:00.00 
[swi0: sio]      root    0.0  0.0 Sun05PM
    0    38     0   0 -60  0     0     8 -      WL    ??    0:00.00 
[irq7: ppc0]     root    0.0  0.0 Sun05PM
    0    39     0   0 -16  0     0     8 psleep DL    ??    0:00.26 
[pagedaemon]     root    0.0  0.0 Sun05PM
    0    40     0   0  20  0     0     8 psleep DL    ??    0:00.00 
[vmdaemon]       root    0.0  0.0 Sun05PM
    0    41     0   0 171  0     0     8 pgzero DL    ??    0:58.03 
[pagezero]       root    0.0  0.0 Sun05PM
    0    42     0   0 -16  0     0     8 psleep DL    ??    0:01.37 
[bufdaemon]      root    0.0  0.0 Sun05PM
    0    43     0   0  -4  0     0     8 vlruwt DL    ??    0:01.78 
[vnlru]          root    0.0  0.0 Sun05PM
    0    44     0   0  20  0     0     8 syncer DL    ??    2:53.26 
[syncer]         root    0.0  0.0 Sun05PM
    0    45     0   0 -16  0     0     8 sdflus DL    ??    0:03.31 
[softdepflush]   root    0.0  0.0 Sun05PM
    0    46     0   0 -64  0     0     8 -      DL    ??    0:17.67 
[schedcpu]       root    0.0  0.0 Sun05PM
    0   163     1 167  20  0  1252   728 pause  Is    ??    0:00.00 
adjkerntz -i     root    0.0  0.1 Sun05PM
    0  1242     1 165 116  0   528   388 select Is    ??    0:00.00 
/sbin/devd       root    0.0  0.1 Sun05PM
    0  1486     1   0  96  0  1568  1104 select Ss    ??    0:19.48 
/sbin/natd -f /e root    0.0  0.2 Sun05PM
65534  1590  6647   0  20  0 11788  9236 lockf  S     ??    0:00.04 
/usr/local/apach nobody  0.0  1.8  3:14AM
    0  1645     1   0  96  0  1408  1144 select Ss    ??    0:16.44 
/usr/sbin/syslog root    0.0  0.2 Sun05PM
   53  1707     1   0 -16  0  9296  8436 zoneli Ds    ??    5:48.13 
/usr/sbin/named  bind    0.0  1.6 Sun05PM
    0  1717     1   0  96  0  1504  1216 select Is    ??    0:00.23 
/usr/sbin/rpcbin root    0.0  0.2 Sun05PM
    0  1772     1 164 116  0  1492  1256 select Is    ??    0:00.00 
/usr/sbin/mountd root    0.0  0.2 Sun05PM
    0  1774     1 164 116  0  1372  1104 select Is    ??    0:00.03 
nfsd: master (nf root    0.0  0.2 Sun05PM
    0  1775  1774 164   4  0  1280   888 -      I     ??    0:00.00 
nfsd: server (nf root    0.0  0.2 Sun05PM
    0  1776  1774 164   4  0  1280   888 -      I     ??    0:00.00 
nfsd: server (nf root    0.0  0.2 Sun05PM
    0  1777  1774 164   4  0  1280   888 -      I     ??    0:00.00 
nfsd: server (nf root    0.0  0.2 Sun05PM
    0  1778  1774 164   4  0  1280   888 -      I     ??    0:00.00 
nfsd: server (nf root    0.0  0.2 Sun05PM
    0  1784     1   0  96  0 263584  1112 select Is    ??    0:00.19 
/usr/sbin/rpc.st root    0.0  0.2 Sun05PM
    0  1789     1   0  96  0  1596  1364 select Is    ??    0:00.22 
rpc.lockd: serve root    0.0  0.3 Sun05PM
    1  1796  1789 164   4  0  1596  1364 nfsloc I     ??    0:00.00 
rpc.lockd: clien daemon  0.0  0.3 Sun05PM
    0  1808     1   0  96  0  3204  2436 select Ss    ??    0:01.30 
/usr/local/sbin/ root    0.0  0.5 Sun05PM
    0  2101     1   0  96  0  2924  1824 select Ss    ??    0:13.51 
/usr/sbin/ntpd - root    0.0  0.4 Sun05PM
    0  2121     1   0  96  0  1288   824 select Is    ??    0:00.33 
/usr/sbin/usbd   root    0.0  0.2 Sun05PM
    0  2150     1  40 101  0  3520  2648 select Is    ??    0:01.11 
/usr/sbin/sshd   root    0.0  0.5 Sun05PM
    0  2157     1   0  96  0  3508  2944 select Ss    ??    0:05.36 
sendmail: accept root    0.0  0.6 Sun05PM
   25  2161     1   0  20  0  3384  2804 pause  Is    ??    0:00.09 
sendmail: Queue  smmsp   0.0  0.5 Sun05PM
    0  2167     1   9   8  0  1388  1100 nanslp Ss    ??    0:27.45 
/usr/sbin/cron - root    0.0  0.2 Sun05PM
65534  3605  6647   0  20  0 12148  9644 lockf  S     ??    0:00.38 
/usr/local/apach nobody  0.0  1.9  9:06PM
    0  6647     1   0   8  0 11708  9140 nanslp Ss    ??    0:09.25 
/usr/local/apach root    0.0  1.8 Sun05PM
    0  6672     1 158 115  0  1468  1168 select Is    ??    0:00.00 
/usr/sbin/inetd  root    0.0  0.2 Sun05PM
65534 34952  6647   0  20  0 12340  9820 lockf  S     ??    0:01.05 
/usr/local/apach nobody  0.0  1.9  9:48PM
 1001 43220     1   0  96  0  2708  2428 select Ss    ??    0:00.42 
/usr/local/sbin/ dhcpd   0.0  0.5  3:46PM
65534 49660  6647   0  20  0 11788  9236 lockf  S     ??    0:00.03 
/usr/local/apach nobody  0.0  1.8  7:32AM
65534 49710  6647   0  20  0 11788  9236 lockf  S     ??    0:00.02 
/usr/local/apach nobody  0.0  1.8  7:32AM
65534 50579  6647   0   4  0 12132  9624 kqread S     ??    0:00.14 
/usr/local/apach nobody  0.0  1.9 11:07PM
65534 51202  6647   0  20  0 12092  9596 lockf  S     ??    0:00.13 
/usr/local/apach nobody  0.0  1.9 11:10PM
    0 57451  2167   0  -8  0  1388  1164 piperd S     ??    0:00.00 
cron: running jo root    0.0  0.2  5:03PM
    0 57452  2167   0  -8  0  1388  1164 piperd S     ??    0:00.00 
cron: running jo root    0.0  0.2  5:03PM
    0 57454  2167   0  -8  0  1388  1164 piperd S     ??    0:00.00 
cron: running jo root    0.0  0.2  5:03PM
    0 57457 57451   0   8  0  1708  1224 wait   Ss    ??    0:00.00 
/bin/sh -c /etc/ root    0.0  0.2  5:03PM
    0 57458 57454   0   8  0  1708  1224 wait   Ss    ??    0:00.00 
/bin/sh -c /etc/ root    0.0  0.2  5:03PM
    0 57466 57452   1   8  0  1708  1224 wait   Ss    ??    0:00.00 
/bin/sh -c /etc/ root    0.0  0.2  5:03PM
    0 57484 57457   0   8  0  1704  1256 wait   S     ??    0:00.00 
/bin/sh /etc/pin root    0.0  0.2  5:03PM
    0 57495 57466   2   8  0  1704  1256 wait   S     ??    0:00.00 
/bin/sh /etc/pin root    0.0  0.2  5:03PM
    0 57509 57458   0   8  0  1704  1256 wait   S     ??    0:00.00 
/bin/sh /etc/pin root    0.0  0.2  5:03PM
    0 57521 57495   2   8  0  1248   472 nanslp S     ??    0:00.00 
sleep 21         root    0.0  0.1  5:03PM
    0 57523 57484   0   8  0  1248   472 nanslp S     ??    0:00.00 
sleep 22         root    0.0  0.1  5:03PM
    0 57525 57509   0   8  0  1248   472 nanslp S     ??    0:00.00 
sleep 20         root    0.0  0.1  5:03PM
65534 97950  6647   0  20  0 12128  9624 lockf  S     ??    0:00.38 
/usr/local/apach nobody  0.0  1.9  2:57AM
65534 98135  6647   0  20  0 12064  9568 lockf  S     ??    0:00.22 
/usr/local/apach nobody  0.0  1.9  2:58AM
65534 98319  6647   0  96  0 12260  9744 select S     ??    0:00.15 
/usr/local/apach nobody  0.0  1.9  2:59AM
    0  6690     1   0   8  0  1760  1512 wait   Is    v0    0:00.03 
login [pam] (log root    0.0  0.3 Sun05PM
    0 56648  6690   0   8  0  3216  2072 wait   S     v0    0:00.03 
-bash (bash)     root    0.0  0.4  4:58PM
    0 57620 56648   1  96  0  1508  1032 -      R+    v0    0:00.00 ps 
-axeluwww     root    0.0  0.2  5:03PM
    0  6691     1   0   8  0  1740  1488 wait   Is    v1    0:00.02 
login [pam] (log root    0.0  0.3 Sun05PM
    0 56659  6691   0   8  0  3216  2072 wait   I     v1    0:00.04 
-bash (bash)     root    0.0  0.4  4:58PM
    0  6692     1   0   8  0  1736  1460 wait   Is    v2    0:00.02 
login [pam] (log root    0.0  0.3 Sun05PM
    0 57198  6692   0   5  0  3212  2068 ttyin  S+    v2    0:00.01 
-bash (bash)     root    0.0  0.4  5:01PM
    0  6693     1 158   5  0  1344   952 ttyin  Is+   v3    0:00.00 
/usr/libexec/get root    0.0  0.2 Sun05PM
    0  6694     1 158   5  0  1344   952 ttyin  Is+   v4    0:00.00 
/usr/libexec/get root    0.0  0.2 Sun05PM
    0  6695     1 158   5  0  1344   952 ttyin  Is+   v5    0:00.00 
/usr/libexec/get root    0.0  0.2 Sun05PM
    0  6696     1 158   5  0  1344   952 ttyin  Is+   v6    0:00.00 
/usr/libexec/get root    0.0  0.2 Sun05PM
    0  6697     1 158   5  0  1344   952 ttyin  Is+   v7    0:00.00 
/usr/libexec/get root    0.0  0.2 Sun05PM
    0  2130     1   0 -58  0  3216  2844 bpf    S    con-   0:06.37 
/usr/local/sbin/ root    0.0  0.6 Sun05PM





Hopefully somebody can point me in the correct direction soon. dhcpd is 
locking up within 12 hours now with the latest stable from yesterday. If 
its the way I have the vlans all on 1 card, then I can fix that.


Thanks
-Jon






Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4564B5E0.3000404>