Date: Tue, 14 May 2002 01:47:57 -0500 From: "Budec" <budec@qwest.net> To: freebsd-stable@freebsd.org Subject: Buffers fill on network card causing it to hang Message-ID: <NGBBIILBAKIFGHHCHOHPCEMHCPAA.budec@qwest.net>
next in thread | raw e-mail | index | archive | help
I am having an issuse with FreeBSD 4.5 Stable. I just sync my source and
make a new kernel and world, using the generic kernel, but added support for
ipfw. The setup is a single x86 (AMD K6-2 300Mhz, 180 megs ram, IDE, Intel
Pro 100+ Ethernet, Generic Linksys ethernet).
The machine is being used as a firewall/file server amoung other things.
The issuse that I am having is that the fxp0 interface (intel nic) which
hosts the internal network "over flows it buffers". After a large amount of
network traffic the nic will hang and I have to go though the other
interface (or console!) and manually take it down/up (ie: ifconfig down;
ifconfig up) to get it work again, or reboot the entire system. When it is
in this hung state and I try and ping (the internal network), I get the
error message:
PING jenny (192.168.17.20): 56 data bytes
ping: sendto: No buffer space available
ping: sendto: No buffer space available
This was in a high stress network, but had to remove it (and take it home!)
to try and get it working... Sadly FreeBSD has become a "banned" OS in that
envoirment due to this issuse (they think it is to "unsupported and
unstable" to be in a production envoirment, I still like it anyways!).
Currently it is being used as my personal firewall/fileserver at home and
has another clean install of 4.5 release synced to 4.5 stable.
I have gotten a couple suggests from the questions@freebsd.org, but no real
answer on why this keeps happening. I also seen posts of this though google
searches, but have yet to find if it is a configuration issuse, hardware
issuse or Bug. I can rule out hardware as I tried another 4 other nics that
have been working in other boxes. So either config issuse or bug. I am
willing to test differant configurations on this, but have tried everything
I could think of and eveyrthing that has been suggested. Or can anyone
confirm that this is infact a bug?
I tried setting keep alive to both '0' and '1', both had no effect.
NMBCLUSTERS has been increased to varies levels 32768 which should be more
than enough. This had no effect.
Any ideas? I will try anything. Is there any info that would be helpful in
debuging this?
Below is included random bits of informs, everything appears to be fine by
my eyes:
(also is there anyway to look at the buffers that the ping command is
referencing? I thought it was `netstat -m`, but that says it is using 0% of
the buffer when I am having this problem).
bud@hydra:/home/bud/doc/fxp/ --> sysctl -a | grep tcp
tcpcb: 544, 32768, 15, 15, 28
net.inet.tcp.rfc1323: 1
net.inet.tcp.rfc1644: 0
net.inet.tcp.mssdflt: 512
net.inet.tcp.keepidle: 7200000
net.inet.tcp.keepintvl: 75000
net.inet.tcp.sendspace: 32768
net.inet.tcp.recvspace: 65536
net.inet.tcp.keepinit: 75000
net.inet.tcp.delacktime: 100
net.inet.tcp.log_in_vain: 0
net.inet.tcp.blackhole: 0
net.inet.tcp.delayed_ack: 1
net.inet.tcp.drop_synfin: 0
net.inet.tcp.path_mtu_discovery: 1
net.inet.tcp.slowstart_flightsize: 1
net.inet.tcp.local_slowstart_flightsize: 4
net.inet.tcp.newreno: 1
net.inet.tcp.tcbhashsize: 512
net.inet.tcp.do_tcpdrain: 1
net.inet.tcp.pcbcount: 15
net.inet.tcp.icmp_may_rst: 1
net.inet.tcp.strict_rfc1948: 0
net.inet.tcp.isn_reseed_interval: 0
net.inet.tcp.syncookies: 1
net.inet.tcp.syncache.bucketlimit: 30
net.inet.tcp.syncache.cachelimit: 15359
net.inet.tcp.syncache.count: 0
net.inet.tcp.syncache.hashsize: 512
net.inet.tcp.syncache.rexmtlimit: 3
net.inet.tcp.msl: 30000
net.inet.tcp.always_keepalive: 0
bud@hydra:/home/bud/doc/fxp/ --> sysctl -a | grep net
net.local.stream.sendspace: 8192
net.local.stream.recvspace: 8192
net.local.dgram.maxdgram: 2048
net.local.dgram.recvspace: 4096
net.local.inflight: 0
net.inet.ip.portrange.lowfirst: 1023
net.inet.ip.portrange.lowlast: 600
net.inet.ip.portrange.first: 1024
net.inet.ip.portrange.last: 5000
net.inet.ip.portrange.hifirst: 49152
net.inet.ip.portrange.hilast: 65535
net.inet.ip.forwarding: 1
net.inet.ip.redirect: 1
net.inet.ip.ttl: 64
net.inet.ip.rtexpire: 3600
net.inet.ip.rtminexpire: 10
net.inet.ip.rtmaxcache: 128
net.inet.ip.sourceroute: 0
net.inet.ip.intr_queue_maxlen: 50
net.inet.ip.intr_queue_drops: 0
net.inet.ip.accept_sourceroute: 0
net.inet.ip.fastforwarding: 0
net.inet.ip.keepfaith: 0
net.inet.ip.subnets_are_local: 0
net.inet.ip.fw.enable: 1
net.inet.ip.fw.one_pass: 1
net.inet.ip.fw.debug: 1
net.inet.ip.fw.verbose: 1
net.inet.ip.fw.verbose_limit: 0
net.inet.ip.fw.dyn_buckets: 256
net.inet.ip.fw.curr_dyn_buckets: 256
net.inet.ip.fw.dyn_count: 0
net.inet.ip.fw.dyn_max: 1000
net.inet.ip.fw.static_count: 6
net.inet.ip.fw.dyn_ack_lifetime: 300
net.inet.ip.fw.dyn_syn_lifetime: 20
net.inet.ip.fw.dyn_fin_lifetime: 1
net.inet.ip.fw.dyn_rst_lifetime: 1
net.inet.ip.fw.dyn_udp_lifetime: 10
net.inet.ip.fw.dyn_short_lifetime: 5
net.inet.ip.fw.dyn_grace_time: 10
net.inet.ip.maxfragpackets: 8192
net.inet.ip.check_interface: 0
net.inet.icmp.maskrepl: 0
net.inet.icmp.icmplim: 200
net.inet.icmp.drop_redirect: 0
net.inet.icmp.log_redirect: 0
net.inet.icmp.bmcastecho: 0
net.inet.tcp.rfc1323: 1
net.inet.tcp.rfc1644: 0
net.inet.tcp.mssdflt: 512
net.inet.tcp.keepidle: 7200000
net.inet.tcp.keepintvl: 75000
net.inet.tcp.sendspace: 32768
net.inet.tcp.recvspace: 65536
net.inet.tcp.keepinit: 75000
net.inet.tcp.delacktime: 100
net.inet.tcp.log_in_vain: 0
net.inet.tcp.blackhole: 0
net.inet.tcp.delayed_ack: 1
net.inet.tcp.drop_synfin: 0
net.inet.tcp.path_mtu_discovery: 1
net.inet.tcp.slowstart_flightsize: 1
net.inet.tcp.local_slowstart_flightsize: 4
net.inet.tcp.newreno: 1
net.inet.tcp.tcbhashsize: 512
net.inet.tcp.do_tcpdrain: 1
net.inet.tcp.pcbcount: 15
net.inet.tcp.icmp_may_rst: 1
net.inet.tcp.strict_rfc1948: 0
net.inet.tcp.isn_reseed_interval: 0
net.inet.tcp.syncookies: 1
net.inet.tcp.syncache.bucketlimit: 30
net.inet.tcp.syncache.cachelimit: 15359
net.inet.tcp.syncache.count: 0
net.inet.tcp.syncache.hashsize: 512
net.inet.tcp.syncache.rexmtlimit: 3
net.inet.tcp.msl: 30000
net.inet.tcp.always_keepalive: 0
net.inet.udp.checksum: 1
net.inet.udp.maxdgram: 9216
net.inet.udp.recvspace: 41600
net.inet.udp.log_in_vain: 0
net.inet.udp.blackhole: 0
net.inet.accf.unloadable: 0
net.inet.raw.maxdgram: 8192
net.inet.raw.recvspace: 8192
net.link.generic.system.ifcount: 4
net.link.ether.inet.prune_intvl: 300
net.link.ether.inet.max_age: 1200
net.link.ether.inet.host_down_time: 20
net.link.ether.inet.maxtries: 5
net.link.ether.inet.useloopback: 1
net.link.ether.inet.proxyall: 0
net.link.ether.inet.log_arp_wrong_iface: 1
bud@hydra:/home/bud/bin/ --> dmesg
Copyright (c) 1992-2002 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD 4.5-RELEASE-p4 #0: Mon May 13 14:22:48 CDT 2002
bud@hydra:/u2/src/sys/compile/HYDRA2
Timecounter "i8254" frequency 1193182 Hz
Timecounter "TSC" frequency 299941774 Hz
CPU: AMD-K6tm w/ multimedia extensions (299.94-MHz 586-class CPU)
Origin = "AuthenticAMD" Id = 0x570 Stepping = 0
Features=0x8001bf<FPU,VME,DE,PSE,TSC,MSR,MCE,CX8,MMX>
AMD Features=0x400<<b10>>
real memory = 184549376 (180224K bytes)
avail memory = 176369664 (172236K bytes)
Preloaded elf kernel "kernel" at 0xc02cb000.
md0: Malloc disk
npx0: <math processor> on motherboard
npx0: INT 16 interface
pcib0: <Host to PCI bridge> on motherboard
pci0: <PCI bus> on pcib0
isab0: <AcerLabs M1533 portable PCI-ISA bridge> at device 2.0 on pci0
isa0: <ISA bus> on isab0
pci0: <ATI Mach64-GP graphics accelerator> at 3.0 irq 0
fxp0: <Intel Pro 10/100B/100+ Ethernet> port 0xec00-0xec3f mem
0xff900000-0xff9fffff,0xffafd000-0xffafdfff irq 9 at device 5.0 on pci0
fxp0: Ethernet address 00:03:47:40:2c:93
inphy0: <i82555 10/100 media interface> on miibus0
inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
dc0: <ADMtek AN985 10/100BaseTX> port 0xea00-0xeaff mem
0xffafec00-0xffafefff irq 11 at device 7.0 on pci0
dc0: Ethernet address: 00:03:6d:1a:5a:31
miibus1: <MII bus> on dc0
ukphy0: <Generic IEEE 802.3u media interface> on miibus1
ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
atapci0: <AcerLabs Aladdin ATA33 controller> port 0xffa0-0xffaf irq 0 at
device 11.0 on pci0
ata0: at 0x1f0 irq 14 on atapci0
ata1: at 0x170 irq 15 on atapci0
orm0: <Option ROM> at iomem 0xc0000-0xc7fff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> flags 0x1 irq 1 on atkbdc0
kbd0 at atkbd0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 16550A
sio1 at port 0x2f8-0x2ff irq 3 on isa0
sio1: type 16550A
ppc0: <Parallel port> at port 0x378-0x37f irq 7 on isa0
ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/1 bytes threshold
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
IP packet filtering initialized, divert enabled, rule-based forwarding
disabled, default to deny, logging disabled
ad0: 4112MB <SAMSUNG SV0432A> [8912/15/63] at ata0-master UDMA33
ad2: 95396MB <WDC WD1000BB-00CAA0> [193821/16/63] at ata1-master UDMA33
ad3: 78167MB <Maxtor 98196H8> [158816/16/63] at ata1-slave UDMA33
acd0: CDROM <ATAPI 48X CDROM> at ata0-slave using PIO4
Mounting root from ufs:/dev/ad0s1a
ad0s1a: UDMA ICRC error reading fsbn 5167935 of 2583936-2584063 (ad0s1 bn
5167935; cn 321 tn 175 sn 45) retrying
ad0s1a: UDMA ICRC error reading fsbn 5168799 of 2584368-2584447 (ad0s1 bn
5168799; cn 321 tn 189 sn 27) retrying
ad0s1a: UDMA ICRC error reading fsbn 7335423 of 3667680-3667807 (ad0s1 bn
7335423; cn 456 tn 155 sn 18) retrying
ad0s1a: UDMA ICRC error reading fsbn 5196543 of 2598240-2598335 (ad0s1 bn
5196543; cn 323 tn 119 sn 51) retrying
ad0s1a: UDMA ICRC error reading fsbn 5196543 of 2598240-2598335 (ad0s1 bn
5196543; cn 323 tn 119 sn 51) retrying
ad0s1a: UDMA ICRC error reading fsbn 5162799 of 2581368-2581463 (ad0s1 bn
5162799; cn 321 tn 94 sn 12) retrying
ad0s1a: UDMA ICRC error reading fsbn 5162799 of 2581368-2581463 (ad0s1 bn
5162799; cn 321 tn 94 sn 12) retrying
ad0s1a: UDMA ICRC error reading fsbn 5162799 of 2581368-2581463 (ad0s1 bn
5162799; cn 321 tn 94 sn 12) retrying
ad0s1a: UDMA ICRC error reading fsbn 5162799 of 2581368-2581463 (ad0s1 bn
5162799; cn 321 tn 94 sn 12) falling back to PIO mode
bud@hydra:/home/bud/doc/fxp/ --> ifconfig -a
fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
inet 192.168.17.1 netmask 0xffffff00 broadcast 192.168.17.255
ether 00:03:47:40:2c:93
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
dc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
inet 69.231.238.225 netmask 0xfffffff8 broadcast 69.231.238.231
inet 69.231.238.226 netmask 0xffffffff broadcast 69.231.238.226
inet 69.231.238.227 netmask 0xffffffff broadcast 69.231.238.227
inet 69.231.238.228 netmask 0xffffffff broadcast 69.231.238.228
inet 69.231.238.229 netmask 0xffffffff broadcast 69.231.238.229
ether 00:03:6d:1a:5a:31
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
inet 127.0.0.1 netmask 0xff000000
ppp0: flags=8010<POINTOPOINT,MULTICAST> mtu 1500
bud@hydra:/home/bud/doc/fxp/ --> netstat -s
tcp:
47995 packets sent
45922 data packets (55502836 bytes)
24 data packets (29496 bytes) retransmitted
0 resends initiated by MTU discovery
2027 ack-only packets (1959 delayed)
0 URG only packets
0 window probe packets
9 window update packets
13 control packets
25112 packets received
18796 acks (for 55498017 bytes)
14 duplicate acks
0 acks for unsent data
10242 packets (1758833 bytes) received in-sequence
5 completely duplicate packets (195 bytes)
0 old duplicate packets
0 packets with some dup. data (0 bytes duped)
0 out-of-order packets (0 bytes)
0 packets (0 bytes) of data after window
0 window probes
13 window update packets
0 packets received after close
0 discarded for bad checksums
0 discarded for bad header offset fields
0 discarded because packet too short
4 connection requests
11 connection accepts
0 bad connection attempts
0 listen queue overflows
14 connections established (including accepts)
14 connections closed (including 2 drops)
6 connections updated cached RTT on close
6 connections updated cached RTT variance on close
2 connections updated cached ssthresh on close
1 embryonic connection dropped
13409 segments updated rtt (of 13410 attempts)
23 retransmit timeouts
0 connections dropped by rexmit timeout
0 persist timeouts
0 connections dropped by persist timeout
0 keepalive timeouts
0 keepalive probes sent
0 connections dropped by keepalive
13713 correct ACK header predictions
6238 correct data packet header predictions
11 syncache entries added
0 retransmitted
0 dupsyn
0 dropped
11 completed
0 bucket overflow
0 cache overflow
0 reset
0 stale
0 aborted
0 badack
0 unreach
0 zone failures
0 cookies sent
0 cookies received
udp:
360 datagrams received
0 with incomplete header
0 with bad data length field
0 with bad checksum
0 with no checksum
4 dropped due to no socket
0 broadcast/multicast datagrams dropped due to no socket
0 dropped due to full socket buffers
0 not for hashed pcb
356 delivered
181 datagrams output
ip:
35691 total packets received
0 bad header checksums
0 with size smaller than minimum
0 with data size < data length
0 with ip length > max ip packet size
0 with header length < data size
0 with data length < header length
0 with bad options
0 with incorrect version number
0 fragments received
0 fragments dropped (dup or out of space)
0 fragments dropped after timeout
0 packets reassembled ok
29354 packets for this host
0 packets for unknown/unsupported protocol
6337 packets forwarded (0 packets fast forwarded)
0 packets not forwardable
0 packets received for unknown multicast group
0 redirects sent
48181 packets sent from this host
3269 packets sent with fabricated ip header
0 output packets dropped due to no bufs, etc.
0 output packets discarded due to no route
0 output datagrams fragmented
0 fragments created
0 datagrams that can't be fragmented
0 tunneling packets that can't find gif
0 datagrams with bad address in header
icmp:
4 calls to icmp_error
0 errors not generated 'cuz old message was icmp
Output histogram:
destination unreachable: 4
0 messages with bad code fields
0 messages < minimum length
0 bad checksums
0 messages with bad length
0 multicast echo requests ignored
0 multicast timestamp requests ignored
Input histogram:
destination unreachable: 4
0 message responses generated
0 invalid return addresses
0 no return routes
ICMP address mask responses are disabled
igmp:
0 messages received
0 messages received with too few bytes
0 messages received with bad checksum
0 membership queries received
0 membership queries received with invalid field(s)
0 membership reports received
0 membership reports received with invalid field(s)
0 membership reports received for groups to which we belong
0 membership reports sent
bud@hydra:/home/bud/doc/fxp/ --> netstat -m
142/592/131072 mbufs in use (current/peak/max):
130 mbufs allocated to data
12 mbufs allocated to packet headers
128/250/32768 mbuf clusters in use (current/peak/max)
648 Kbytes allocated to network (0% of mb_map in use)
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines
bud@hydra:/home/bud/doc/fxp/ --> netstat -i
Name Mtu Network Address Ipkts Ierrs Opkts Oerrs
Coll
fxp0 1500 <Link#1> 00:03:47:40:2c:93 24000 0 47338 0
0
fxp0 1500 192.168.17 hydra 19667 -
3975 - -
dc0 1500 <Link#2> 00:03:6d:1a:5a:31 3883 0 3279 0
0
dc0 1500 69.231.238.22 63.231.238.225 406 -
3270 - -
dc0 1500 69.231.238.22 63.231.238.226 0 -
0 - -
dc0 1500 69.231.238.22 63.231.238.227 0 -
0 - -
dc0 1500 69.231.238.22 63.231.238.228 0 -
0 - -
dc0 1500 69.231.238.22 63.231.238.229 1489 -
0 - -
lo0 16384 <Link#3> 5663 0 5663 0
0
lo0 16384 your-net localhost 5607 -
5607 - -
ppp0* 1500 <Link#4> 0 0 0 0
0
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?NGBBIILBAKIFGHHCHOHPCEMHCPAA.budec>