From owner-freebsd-net@FreeBSD.ORG Wed Mar 26 02:51:38 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C42FAC2C for ; Wed, 26 Mar 2014 02:51:38 +0000 (UTC) Received: from mail-yk0-x231.google.com (mail-yk0-x231.google.com [IPv6:2607:f8b0:4002:c07::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 83E8AAF0 for ; Wed, 26 Mar 2014 02:51:38 +0000 (UTC) Received: by mail-yk0-f177.google.com with SMTP id q200so98241ykb.22 for ; Tue, 25 Mar 2014 19:51:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=R8Sj21yHT2kV4iges3HeRaKt9OsTaMm/eWwcbY79tRQ=; b=wgTAhoU9T+COkCovIURukbyRw8XJunB5RfLrBekfzmUIHLmlIz50dyWX2XEWR1Pidm pPRmlQPyoA7nWR9l7ES8UY/ik49v0t2ifQOAcIXWwk3p88EGNn1elsFvG/VSRLQm2D0Z kI0/n3oZsGDgYDAxmV1wIdXMXnJIlcKIJILhic5dCOw64G3asC1w8fMwXC43yT1+Asoh /sU41ymgrUlS7qhWUspTFMYDdEM88auEbhmIptKcCyJIT+K3VYa8HSf8DNGy6r7tie/w z6Kyw18y7jXXX901JcaySbFdkkgGN9EYa0uLue9Fxz8u74nufXTT5+b9lW3vCg8AbogW aS+Q== X-Received: by 10.236.8.68 with SMTP id 44mr79360468yhq.39.1395802297763; Tue, 25 Mar 2014 19:51:37 -0700 (PDT) Received: from kmatoMacBook-Pro.local ([27.24.140.240]) by mx.google.com with ESMTPSA id u5sm15566919yhg.25.2014.03.25.19.51.34 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 25 Mar 2014 19:51:37 -0700 (PDT) Message-ID: <533240B4.50809@gmail.com> Date: Wed, 26 Mar 2014 10:51:32 +0800 From: k simon User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Christopher Forgeron Subject: Re: syslogd:sendto: no buffer available on 10-stable References: <53310C57.5070102@gmail.com> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 02:51:38 -0000 Thanks, Christopher. But I think my problem may does not related to TSO issue. I have tried disable tso with "ifconfig igb(x) -tso" and ovserved with "netstat -ihw 1", and found "oErrs" does not disappeared. Regards Simon 于 14-3-25 22:08, Christopher Forgeron 写道: > Hi Simon, > > Try checking out the "9.2 ixgbe tx queue hang' thread here, and see if > it applies to you. > > > On Tue, Mar 25, 2014 at 1:55 AM, k simon > wrote: > > Hi,Lists: > I have got lots of "no buffer available" on 10-stable with igb nic. > But em and bce works well. And I tried force igb to 4 or 8 queues and > set hw.igb.rx_process_limit="-1", but have nothing helped. > > > Regards > Simon > > > > # uname -a > FreeBSD sq-l1-n2 10.0-STABLE FreeBSD 10.0-STABLE #0 r262469: Tue Feb 25 > 13:27:11 CST 2014 > root@sq-l1-n2:/usr/obj/usr/src/sys/stable-10-262458 amd64 > > > # netstat -mb > 19126/73289/92415 mbufs in use (current/cache/total) > 13289/46841/60130/524288 mbuf clusters in use (current/cache/total/max) > 13289/46812 mbuf+clusters out of packet secondary zone in use > (current/cache) > 5638/22605/28243/262144 4k (page size) jumbo clusters in use > (current/cache/total/max) > 0/0/0/77672 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/43690 16k jumbo clusters in use (current/cache/total/max) > 53914K/202424K/256338K bytes allocated to network (current/cache/total) > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > > > # netstat -di > Name Mtu Network Address Ipkts Ierrs Idrop > Opkts Oerrs Coll Drop > igb0 1500 00:1b:21:70:5f:80 17212101113 1355809 0 > 19612978862 0 0 > igb1 1500 00:1b:21:70:5f:81 76601294282 81162751 0 > 74236432686 0 0 > lo0 16384 20532742636 0 0 > 20522475797 0 0 > lo0 - your-net localhost 2736994243 - - > 20520227166 - - > > > # sysctl hw.igb > hw.igb.rxd: 2048 > hw.igb.txd: 2048 > hw.igb.enable_aim: 1 > hw.igb.enable_msix: 1 > hw.igb.max_interrupt_rate: 12000 > hw.igb.buf_ring_size: 4096 > hw.igb.header_split: 0 > hw.igb.num_queues: 1 > hw.igb.rx_process_limit: 1000 > > > # sysctl dev.igb.1 > dev.igb.1.%desc: Intel(R) PRO/1000 Network Connection version - 2.4.0 > dev.igb.1.%driver: igb > dev.igb.1.%location: slot=0 function=1 > dev.igb.1.%pnpinfo: vendor=0x8086 device=0x10c9 subvendor=0x8086 > subdevice=0xa04c class=0x020000 > dev.igb.1.%parent: pci8 > dev.igb.1.nvm: -1 > dev.igb.1.enable_aim: 1 > dev.igb.1.fc: 0 > dev.igb.1.rx_processing_limit: 4096 > dev.igb.1.link_irq: 3 > dev.igb.1.dropped: 0 > dev.igb.1.tx_dma_fail: 0 > dev.igb.1.rx_overruns: 0 > dev.igb.1.watchdog_timeouts: 0 > dev.igb.1.device_control: 1086325313 > dev.igb.1.rx_control: 67141634 > dev.igb.1.interrupt_mask: 4 > dev.igb.1.extended_int_mask: 2147483651 > dev.igb.1.tx_buf_alloc: 0 > dev.igb.1.rx_buf_alloc: 0 > dev.igb.1.fc_high_water: 58976 > dev.igb.1.fc_low_water: 58960 > dev.igb.1.queue0.no_desc_avail: 10874 > dev.igb.1.queue0.tx_packets: 74509997338 > dev.igb.1.queue0.rx_packets: 76837720630 > dev.igb.1.queue0.rx_bytes: 35589607860237 > dev.igb.1.queue0.lro_queued: 0 > dev.igb.1.queue0.lro_flushed: 0 > dev.igb.1.mac_stats.excess_coll: 0 > dev.igb.1.mac_stats.single_coll: 0 > dev.igb.1.mac_stats.multiple_coll: 0 > dev.igb.1.mac_stats.late_coll: 0 > dev.igb.1.mac_stats.collision_count: 0 > dev.igb.1.mac_stats.symbol_errors: 0 > dev.igb.1.mac_stats.sequence_errors: 0 > dev.igb.1.mac_stats.defer_count: 0 > dev.igb.1.mac_stats.missed_packets: 81162751 > dev.igb.1.mac_stats.recv_no_buff: 176691324 > dev.igb.1.mac_stats.recv_undersize: 0 > dev.igb.1.mac_stats.recv_fragmented: 0 > dev.igb.1.mac_stats.recv_oversize: 0 > dev.igb.1.mac_stats.recv_jabber: 0 > dev.igb.1.mac_stats.recv_errs: 0 > dev.igb.1.mac_stats.crc_errs: 0 > dev.igb.1.mac_stats.alignment_errs: 0 > dev.igb.1.mac_stats.coll_ext_errs: 0 > dev.igb.1.mac_stats.xon_recvd: 0 > dev.igb.1.mac_stats.xon_txd: 0 > dev.igb.1.mac_stats.xoff_recvd: 0 > dev.igb.1.mac_stats.xoff_txd: 0 > dev.igb.1.mac_stats.total_pkts_recvd: 76925709917 > dev.igb.1.mac_stats.good_pkts_recvd: 76837704301 > dev.igb.1.mac_stats.bcast_pkts_recvd: 49174716 > dev.igb.1.mac_stats.mcast_pkts_recvd: 282670 > dev.igb.1.mac_stats.rx_frames_64: 31057121854 > dev.igb.1.mac_stats.rx_frames_65_127: 19996324498 > dev.igb.1.mac_stats.rx_frames_128_255: 1171960837 > dev.igb.1.mac_stats.rx_frames_256_511: 2295894674 > dev.igb.1.mac_stats.rx_frames_512_1023: 2026241811 > dev.igb.1.mac_stats.rx_frames_1024_1522: 20290160627 > dev.igb.1.mac_stats.good_octets_recvd: 36204302378783 > dev.igb.1.mac_stats.good_octets_txd: 59038220741656 > dev.igb.1.mac_stats.total_pkts_txd: 90973292365 > dev.igb.1.mac_stats.good_pkts_txd: 90973292359 > dev.igb.1.mac_stats.bcast_pkts_txd: 2408182 > dev.igb.1.mac_stats.mcast_pkts_txd: 246782 > dev.igb.1.mac_stats.tx_frames_64: 24604769631 > dev.igb.1.mac_stats.tx_frames_65_127: 21373976133 > dev.igb.1.mac_stats.tx_frames_128_255: 3047554762 > dev.igb.1.mac_stats.tx_frames_256_511: 3751820879 > dev.igb.1.mac_stats.tx_frames_512_1023: 3481550512 > dev.igb.1.mac_stats.tx_frames_1024_1522: 34713620448 > dev.igb.1.mac_stats.tso_txd: 9549335102 > dev.igb.1.mac_stats.tso_ctx_fail: 0 > dev.igb.1.interrupts.asserts: 16929843739 > dev.igb.1.interrupts.rx_pkt_timer: 76834839132 > dev.igb.1.interrupts.rx_abs_timer: 0 > dev.igb.1.interrupts.tx_pkt_timer: 0 > dev.igb.1.interrupts.tx_abs_timer: 76835042438 > dev.igb.1.interrupts.tx_queue_empty: 90970561829 > dev.igb.1.interrupts.tx_queue_min_thresh: 0 > dev.igb.1.interrupts.rx_desc_min_thresh: 0 > dev.igb.1.interrupts.rx_overrun: 0 > dev.igb.1.host.breaker_tx_pkt: 0 > dev.igb.1.host.host_tx_pkt_discard: 0 > dev.igb.1.host.rx_pkt: 2865171 > dev.igb.1.host.breaker_rx_pkts: 0 > dev.igb.1.host.breaker_rx_pkt_drop: 0 > dev.igb.1.host.tx_good_pkt: 2730536 > dev.igb.1.host.breaker_tx_pkt_drop: 0 > dev.igb.1.host.rx_good_bytes: 36204307257585 > dev.igb.1.host.tx_good_bytes: 59038220743158 > dev.igb.1.host.length_errors: 0 > dev.igb.1.host.serdes_violation_pkt: 0 > dev.igb.1.host.header_redir_missed: 0 > > > > # cat /var/run/dmesg.boot |grep igb > igb0: mem > 0xd5ee0000-0xd5efffff,0xd5800000-0xd5bfffff,0xd5edc000-0xd5edffff irq 18 > at device 0.0 on pci8 > igb0: Using MSIX interrupts with 2 vectors > igb0: Ethernet address: 00:1b:21:70:5f:80 > igb1: mem > 0xd5ea0000-0xd5ebffff,0xd5400000-0xd57fffff,0xd5ed8000-0xd5edbfff irq 19 > at device 0.1 on pci8 > igb1: Using MSIX interrupts with 2 vectors > igb1: Ethernet address: 00:1b:21:70:5f:81 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org > " > >