Date: Tue, 22 Mar 2005 16:54:06 GMT From: James Juran <James.Juran@baesystems.com> To: freebsd-gnats-submit@FreeBSD.org Subject: kern/79138: close while sending on connected UNIX-domain socket can return ENOTCONN, should return EPIPE or 0 Message-ID: <200503221654.j2MGs67V014609@www.freebsd.org> Resent-Message-ID: <200503221700.j2MH06K8058261@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 79138
>Category: kern
>Synopsis: close while sending on connected UNIX-domain socket can return ENOTCONN, should return EPIPE or 0
>Confidential: no
>Severity: non-critical
>Priority: low
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Tue Mar 22 17:00:06 GMT 2005
>Closed-Date:
>Last-Modified:
>Originator: James Juran
>Release: 5.3-RELEASE
>Organization:
BAE Systems Information Technology, LLC
>Environment:
FreeBSD fbsd 5.3-RELEASE FreeBSD 5.3-RELEASE #1: Mon Mar 22 09:15:42 EST 2005 root@fbsd:/usr/obj/usr/src/sys/PREEMPT i386 (built with options PREEMPTION FULL_PREEMPTION)
>Description:
The test program shown below, when run on a uniproc system compiled with PREEMPTION and FULL_PREEMPTION, shows that calling send() for more than 2048 bytes on a UNIX-domain connection-oriented socket can return ENOTCONN, even when the socket is connected. This happens when the other side of the connection does a partial recv() and then closes the connection with data still in the send buffer. The send() should return either EPIPE, if the close happens before the send completes, or 0 if the send completes before the close takes effect.
What happens is that sosend() breaks the send into 2048-byte chunks and calls uipc_send() for each one. uipc_send() wakes up a process waiting in recv(). In the GENERIC kernel on uniproc, this wakeup doesn't actually take effect until sosend() tries to get the SOCKBUF_LOCK at the bottom of the main loop. It then checks SBS_CANTSENDMORE on the next loop iteration, and returns EPIPE as it should. However, with option FULL_PREEMPTION, the process doing the recv() gets run right away, and performs its close before the second call to uipc_send(). uipc_send() then fails the check for SS_ISCONNECTED and returns ENOTCONN.
Even though this failure requires FULL_PREEMPTION to reproduce on uniproc, I would think that on SMP there would be a chance of failure even on a standard kernel.
>How-To-Repeat:
Script started on Tue Mar 22 10:47:37 2005
bash-3.00$ cat sock-exec.c
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
int start_server(const char *name)
{
pid_t pid;
pid = fork();
if (pid == -1)
perror("fork failed");
else if (pid == 0)
{
/* Child process */
execl("sock-lpc", NULL);
perror("execl returned?");
}
return 1;
}
int run_client(const char *pathname)
{
int s;
struct sockaddr_un un;
int ret;
char buf[8193];
s = socket(PF_LOCAL, SOCK_STREAM, 0);
if (s == -1)
{
printf("socket failed: %s\n", strerror(errno));
exit(1);
}
memset(&un, 0, sizeof(un));
un.sun_family = AF_UNIX;
strcpy(un.sun_path, pathname);
ret = connect(s, (struct sockaddr *)&un,
sizeof(un.sun_family) + strlen(un.sun_path) + 1);
if (ret == -1)
{
printf("connect failed: %s\n", strerror(errno));
exit(1);
}
memset(buf, 'A', sizeof(buf));
ret = send(s, buf, sizeof(buf), 0);
if (ret == -1)
{
printf("send failed: %s\n", strerror(errno));
exit(1);
}
printf("send returned %d\n", ret);
return 1;
}
int main(void)
{
if (!start_server("/tmp/AAAA"))
return 1;
/* Let the child process get the socket established */
sleep(2);
if (!run_client("/tmp/AAAA"))
return 1;
puts("Parent process finished.");
return 0;
}
bash-3.00$ cat sock-lpc.c
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
int main(void)
{
int s;
const char name[] = "/tmp/AAAA";
struct sockaddr_un un;
int conn;
char buf[8193];
FILE *fp;
int ret;
fp = fopen("/tmp/child-log.txt", "w");
if (!fp)
exit(1);
s = socket(PF_UNIX, SOCK_STREAM, 0);
if (s == -1)
{
fprintf(fp, "socket failed: %s", strerror(errno));
exit(1);
}
fprintf(fp, "socket for %s is %d\n", name, s);
unlink(name);
memset(&un, 0, sizeof(un));
un.sun_family = AF_UNIX;
strcpy(un.sun_path, name);
ret = bind(s, (struct sockaddr *)&un, sizeof(un.sun_family) +
strlen(un.sun_path) + 1);
if (ret == -1)
{
fprintf(fp, "bind failed: %s\n", strerror(errno));
exit(1);
}
fprintf(fp, "Returned %d from bind to %s\n", ret, name);
fprintf(fp, "Child process %d about to call listen\n", getpid());
ret = listen(s, 2);
if (ret == -1)
{
fprintf(fp, "listen failed: %s\n", strerror(errno));
exit(1);
}
fprintf(fp, "Returned %d from listen\n", ret);
conn = accept(s, NULL, NULL);
fprintf(fp, "accept returned %d\n", conn);
if (conn == -1)
{
fprintf(fp, "No connection: %s\n", strerror(errno));
exit(1);
}
ret = recv(conn, buf, sizeof(buf), 0);
if (ret == -1)
{
fprintf(fp, "recv failed: %s\n", strerror(errno));
exit(1);
}
fprintf(fp, "recv returned %d\n", ret);
close(conn);
close(s);
fclose(fp);
return 0;
}
bash-3.00$ gcc -Wall -Werror -o sock-exec sock-exec.c
bash-3.00$ gcc -Wall -Werror -o sock-lpc sock-lpc.c
bash-3.00$ ./sock-exec
send failed: Socket is not connected
bash-3.00$ cat /tmp/child-log.txt
socket for /tmp/AAAA is 4
Returned 0 from bind to /tmp/AAAA
Child process 550 about to call listen
Returned 0 from listen
accept returned 5
recv returned 2048
bash-3.00$ ktrace ./sock-exec
send failed: Socket is not connected
bash-3.00$ kdump
[...]
552 sock-exec CALL socket(0x1,0x1,0)
552 sock-exec RET socket 3
552 sock-exec CALL connect(0x3,0xbfbfea40,0xb)
552 sock-exec NAMI "/tmp/AAAA"
552 sock-exec RET connect 0
552 sock-exec CALL sendto(0x3,0xbfbfca20,0x2001,0,0,0)
552 sock-exec RET sendto -1 errno 57 Socket is not connected
>Fix:
Reversing the checks for SBS_CANTSENDMORE and SS_ISCONNECTED in uipc_send(), as in this patch, should avoid this problem. However, I don't have an SMP system to test on and I'm not sure this is the correct fix.
Index: uipc_usrreq.c
===================================================================
RCS file: /home/ncvs/src/sys/kern/uipc_usrreq.c,v
retrieving revision 1.151
diff -u -p -f -u -p -r1.151 uipc_usrreq.c
--- uipc_usrreq.c 21 Feb 2005 21:58:16 -0000 1.151
+++ uipc_usrreq.c 22 Mar 2005 16:45:33 -0000
@@ -436,6 +436,13 @@ uipc_send(struct socket *so, int flags,
}
case SOCK_STREAM:
+ SOCKBUF_LOCK(&so->so_snd);
+ if (so->so_snd.sb_state & SBS_CANTSENDMORE) {
+ SOCKBUF_UNLOCK(&so->so_snd);
+ error = EPIPE;
+ break;
+ }
+
/* Connect if not connected yet. */
/*
* Note: A better implementation would complain
@@ -443,21 +450,18 @@ uipc_send(struct socket *so, int flags,
*/
if ((so->so_state & SS_ISCONNECTED) == 0) {
if (nam != NULL) {
+ SOCKBUF_UNLOCK(&so->so_snd);
error = unp_connect(so, nam, td);
if (error)
break; /* XXX */
+ SOCKBUF_LOCK(&so->so_snd);
} else {
+ SOCKBUF_UNLOCK(&so->so_snd);
error = ENOTCONN;
break;
}
}
- SOCKBUF_LOCK(&so->so_snd);
- if (so->so_snd.sb_state & SBS_CANTSENDMORE) {
- SOCKBUF_UNLOCK(&so->so_snd);
- error = EPIPE;
- break;
- }
if (unp->unp_conn == NULL)
panic("uipc_send connected but no connection?");
so2 = unp->unp_conn->unp_socket;
>Release-Note:
>Audit-Trail:
>Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200503221654.j2MGs67V014609>