Date: Fri, 29 Mar 2013 00:47:53 +0100 From: Jilles Tjoelker <jilles@stack.nl> To: Andriy Gapon <avg@FreeBSD.org> Cc: freebsd-net@FreeBSD.org, FreeBSD Hackers <freebsd-hackers@FreeBSD.org> Subject: Re: close(2) while accept(2) is blocked Message-ID: <20130328234753.GA88844@stack.nl> In-Reply-To: <515475C7.6010404@FreeBSD.org> References: <515475C7.6010404@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Mar 28, 2013 at 06:54:31PM +0200, Andriy Gapon wrote: > So, this started as a simple question, but the answer was quite > unexpected to me. > Let's say we have an opened and listen-ed socket and let's assume that > we know that one thread is blocked in accept(2) and another thread is > calling close(2). What is going to happen? > Turns out that practically nothing. For kernel the close call would > be almost a nop. > My understanding is this: > - when socket is created, its reference count is 1 > - when accept(2) is called, fget in kernel increments the reference > count (kept in an associated struct file) > - when close(2) is called, the reference count is decremented > The reference count is still greater than zero, so fdrop does not call > fo_close. That means that in the case of a socket soclose is not > called. > I am sure that the reference counting in this case is absolutely > correct with respect to managing kernel side structures. I agree this is expected and correct from the kernel point of view. > But I am not that it is correct with respect to hiding the explicit > close(2) call from other threads that may be waiting on the socket. In > other words, I am not sure if fo_close is supposed to signify that > there are no uses of a file, or that userland close-d the file. Or > perhaps these should be two different methods. It would be possible to keep track of the file descriptor number but I think it is not worth the large amount of extra code. Keeping track of file descriptor number would be necessary to interrupt waits after a close or dup2 on the file descriptor that was passed to the blocking call, even if the object remains open on a different file descriptor number or in a different process. Also, most people would use the new functionality incorrectly anyway. A close() on a file descriptor another thread is using is risky since it is in most cases impossible to prove that the other thread is in fact blocked on the file descriptor and not preempted right before making the system call. In the latter case, the other thread might accept a connection from a different socket created later. A dup2() of /dev/null onto the file descriptor would be safer. > Additional note is that shutdown(2) doesn't wake up the thread in > accept(2) either. At least that's true for unix domain sockets. > Not sure if this is a bug. I think it is a bug. It works properly for IPv4 TCP sockets. The resulting error is [ECONNABORTED] for a blocking socket which likely leads to infinite looping if the thread does not know about the shutdown(2) (because that error normally means the accept should be retried later). For a non-blocking socket the error is [EWOULDBLOCK] which also leads to infinite looping and is certainly wrong because select/poll do report the socket as readable. Both of these are in kern_accept() in sys/kern/uipc_syscalls.c. POSIX does not say which error code we should return here but these two are almost certainly wrong (it is usable for waking up threads stuck in accept() if those threads check a variable after every accept() failure and do not rely on the exact value of errno). Linux returns [EINVAL] for both blocking and non-blocking sockets, probably from the POSIX error condition "The socket is not accepting connections." In our man page that error condition is formulated "listen(2) has not been called on the socket descriptor." which is clearly not the case. Also, I think a non-blocking accept() should immediately fail with the head->so_error if it is set, rather than returning [EWOULDBLOCK] until another connection arrives. Likewise, filt_solisten() in sys/kern/uipc_socket.c only returns true if there is a connection, not if there was an error or shutdown() has been called. On the other hand, sopoll_generic() looks correct. Error reporting on non-blocking accept() might usefully be postponed until there is a connection or the socket has been shut down, to reduce context switches. > But the summary seems to be is that currently it is not possible to > break a thread out of accept(2) (at least without resorting to > signals). Pthread cancellation works better than raw signals for this use case. In a proper implementation such as in FreeBSD 9.0 or newer and used properly, it allows avoiding the resource leak that may happen when calling longjmp() or pthread_exit() in a signal handler just after accept() has created a new socket. -- Jilles Tjoelker
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20130328234753.GA88844>