From owner-freebsd-current@FreeBSD.ORG Mon Nov 5 20:05:59 2007 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5C44016A417 for ; Mon, 5 Nov 2007 20:05:59 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id A0DBB13C4A5 for ; Mon, 5 Nov 2007 20:05:58 +0000 (UTC) (envelope-from andre@freebsd.org) Received: (qmail 24819 invoked from network); 5 Nov 2007 19:41:38 -0000 Received: from dotat.atdotat.at (HELO [62.48.0.47]) ([62.48.0.47]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 5 Nov 2007 19:41:38 -0000 Message-ID: <472F778A.6080206@freebsd.org> Date: Mon, 05 Nov 2007 21:05:30 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8b) Gecko/20050217 MIME-Version: 1.0 To: Ian FREISLICH References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: current@freebsd.org Subject: Re: sendfile() not detecting closed connections. X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Nov 2007 20:05:59 -0000 Ian FREISLICH wrote: > Andre Oppermann wrote: > >>Ian FREISLICH wrote: >> >>>Hi >>> >>>System is 8.0-CURRENT. I have the following piece of code: >>> >>> rename(path, data); >>> stat(data, &sb); >>> len = snprintf(buffer, MAXBUFLEN, "BYTES %lld\r\n", sb.st_size); >>> write(connection, buffer, len); >>> sleep(10); >>> if ((sendfile(fd, connection, 0, sb.st_size, NULL, >>> &sbytes, 0)) == -1 || sbytes != sb.st_size) { >>> syslog(facility, "Problem writing data: %s, wrote %lld", >>> strerror(errno), sbytes); >>> respool(fd, path); >>> unlink(data) >>> close(fd); >>> return(-1); >>> } >>> else >>> syslog(facility, "Download successful %ld", sbytes); >>> close(fd); >>> unlink(data); >>> >>>If, during the sleep, I terminate the connection so that netstat >>>reports: >>> >>>tcp4 0 0 127.0.0.1.666 127.0.0.1.58239 CLOSE_WA > > IT > >>>tcp4 0 0 127.0.0.1.58239 127.0.0.1.666 FIN_WAIT > > _2 > >>>sendfile() reports success for files less than about 64k in size, >>>but I haven't been able to figure out where the threshold is. It >>>erroneously reports that 41000 of the 64k were sent, but will say >>>the whole file was transferred up to about 64k. The connection >>>filedescriptor is blocking. >>> >>>Any ideas? >> >>sendfile() reports the bytes written into the send socket buffer. If >>there is a connection error it doesn't (and never did) look at how >>much data was still in the socket buffer. The sendfile(2) man page >>says: "[sbytes] If non-NULL, the system will write the total number >>of bytes sent on the socket to the variable pointed to by sbytes." >>This could be changed to subtract the remaining data in the socket >>buffer before reporting back. One has to be careful though about >>other writes so that the number never goes negative. There may be >>more data remaining in the socket buffer than from this write attempt >>alone. > > > The connection was closed about 6 seconds before I called sendfile(). > Would sendfile() write to the socket buffer of a socket closed that > long ago? The sendfile(2) syscall checks if the socket is still connected: if ((so->so_state & SS_ISCONNECTED) == 0) { error = ENOTCONN; goto out; } Due to timing circumstances the connection may still be technically connected while at the same time disconnecting when FIN exchange has not completed yet. To prevent a sendfile(2) call on a disconnecting socket another test has to be added: if (so->so_state & SS_ISDISCONNECTING) { error = EPIPE; /* or ESHUTDOWN? */ goto out; } Please add this test after line 1841 in sys/kern/uipc_syscalls.c and test again. The error code ESHUTDOWN seems more appropriate but is so far not documented in the sendfile(2) man page. As it seems this bug has been present since the introduction of the sendfile syscall. -- Andre