Date: Sun, 23 Nov 2014 05:58:14 +0200 From: Alexander Motin <mav@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r274853 - in head: sys/dev/iscsi usr.sbin/ctld usr.sbin/iscsid Message-ID: <54715B56.8010905@FreeBSD.org> In-Reply-To: <20141122180318.GA3921@ox> References: <201411221509.sAMF9JIh016800@svn.freebsd.org> <20141122180318.GA3921@ox>
next in thread | previous in thread | raw e-mail | index | archive | help
On 22.11.2014 20:03, Navdeep Parhar wrote: > On Sat, Nov 22, 2014 at 03:09:19PM +0000, Alexander Motin wrote: >> Author: mav >> Date: Sat Nov 22 15:09:18 2014 >> New Revision: 274853 >> URL: https://svnweb.freebsd.org/changeset/base/274853 >> >> Log: >> For both iSCSI initiator and target increase socket buffer sizes before >> establishing connection. >> >> This is a workaround for Chelsio TOE driver, that does not update socket >> buffer size in hardware after connection established, and unless that is >> done beforehand, kernel code will stuck, attempting to send/receive full >> PDU at once. > > cxgbe's TOE driver does update the rcv window in the hardware in > response to changes in the socket buffer size (see t4_rcvd in > cxgbe/tom/t4_cpl_io.c). > > The bug is that the driver tries to avoid sending too many small updates > to the chip, and instead tries to build up a balance of credits before > an update. This doesn't interact well with some applications that set > the sockbuf size to a high fixed value and then wait for the entire > buffer to fill up before draining it. The driver ends up waiting for > the application to drain the socket's receive buffer so that pru_rcvd > (t4_rcvd) can then send a big batch of credits to the chip, the > application is waiting for the receive to reach its high water mark > before draining the buffer, but the TOE on the chip has shut down the > receive window because it's out of credits. If you see the peer sending > zero window probes and the TOE refusing to budge, then the problem > you've observed is definitely the problem I just described. Yes, it looks like this. You may find the packet dump with the problem here: https://people.freebsd.org/~mav/rcvbuf.dump > I'll fix this in the TOE driver. I can send you an early patch in case > you'd like to try out the fix before it gets committed. Thank you. I'll be happy to try it. -- Alexander Motin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?54715B56.8010905>