Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 27 Jan 2009 18:56:02 +0200
From:      "Yony Yossef" <yonyossef.lists@gmail.com>
To:        "'Kip Macy'" <kmacy@freebsd.org>, "Yony Yossef" <yonyossef.lists@gmail.com>
Cc:        johan@nocrew.org, freebsd-net@freebsd.org, Eitan Shefi <eitans@mellanox.co.il>, Amit Krig <amitk@mellanox.co.il>, Liran Liss <liranl@mellanox.co.il>
Subject:   RE: freebsd 7.0-RELEASE BUG ping: sendto: No buffer space available
Message-ID:  <002201c980a0$25903420$220f000a@mtl.com>
In-Reply-To: <3c1674c90901261212l5432b99fpd77bcff96883bd21@mail.gmail.com>
References:  <000001c97e23$0d81df20$39ed1aac@mtl.com> <3c1674c90901261212l5432b99fpd77bcff96883bd21@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help

> "No buffer space available" is ENOBUFS. This error is 
> returned when the ifq is full - see IFQ_ENQUEUE in if_var.h. 
> You'll almost certainly want to set ifq_len to larger than 
> the default for a 10 gigabit driver. This most likely to be a 
> combination of ifq_len being too small and inadequate txq 
> overflow handling. However, without the code to look at I can 
> only guess.
> 
> Cheers,
> Kip


---- my Tx loop is pretty normal (I've removed debug calls and irrelevant
parts):

while (!IFQ_DRV_IS_EMPTY(&dev->if_snd)) {

	IFQ_DRV_DEQUEUE(&dev->if_snd, m_head);
	if (m_head == NULL)
		break;

	if ( (err = mtnic_xmit_encap(priv, &m_head, &used)) != 0) {
            	priv->ifp->if_drv_flags |= IFF_DRV_OACTIVE;
            	IFQ_LOCK(&dev->if_snd);                
           		IFQ_DRV_PREPEND(&dev->if_snd, m_head);
            	IFQ_UNLOCK(&dev->if_snd);            
            	break;    
    	}

    	ETHER_BPF_MTAP(dev, m_head);
}


---- my ifq_max_len is set to 4096, I believe it's bigger than the other
10GigE drivers (cxgb is using 1K, for instance):

dev->if_snd.ifq_drv_maxlen = MTNIC_TX_IFQ_MAXLEN;    // = 4096
IFQ_SET_MAXLEN(&dev->if_snd, dev->if_snd.ifq_drv_maxlen);
IFQ_SET_READY(&dev->if_snd);


---- there's another strange symptom to this hang: after the 20 minutes I've
mentioned (caused by the sender full IFQ) I have to ping the sender from the
receiver side in order to retain traffic. During the 20 minutes it will not
work, but after it passes a single ping will make the sender IFQ go back to
life.

Yony




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?002201c980a0$25903420$220f000a>