Date: Thu, 4 Jun 2009 04:42:39 +1000 (EST) From: Bruce Evans <brde@optusnet.com.au> To: Kostik Belousov <kostikbel@gmail.com> Cc: Vlad Galu <dudu@dudu.ro>, freebsd-stable@freebsd.org, Oliver Fromme <olli@freebsd.org>, bde@freebsd.org Subject: Re: poll()-ing a pipe descriptor, watching for POLLHUP Message-ID: <20090604030724.V1181@besplex.bde.org> In-Reply-To: <20090603150710.GN1927@deviant.kiev.zoral.com.ua> References: <ad79ad6b0906030515k2e41f4b9t25f752af8ef3866c@mail.gmail.com> <20090603123208.GK1927@deviant.kiev.zoral.com.ua> <ad79ad6b0906030535o4b1a959ev6bc2b34af4e7304e@mail.gmail.com> <ad79ad6b0906030610y7e3beb05w5a3a39eaf7ebe2be@mail.gmail.com> <20090603143051.GM1927@deviant.kiev.zoral.com.ua> <20090603150710.GN1927@deviant.kiev.zoral.com.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 3 Jun 2009, Kostik Belousov wrote: > On Wed, Jun 03, 2009 at 05:30:51PM +0300, Kostik Belousov wrote: >> On Wed, Jun 03, 2009 at 04:10:34PM +0300, Vlad Galu wrote: >>> Hm, I was having an issue with an internal piece of software, but >>> never checked what kind of pipe caused the problem. Turns out it was a >>> FIFO, and I got bitten by the same bug described here: >>> http://lists.freebsd.org/pipermail/freebsd-bugs/2006-March/017591.html >>> >>> The problem is that the reader process isn't notified when the writer G>>> process exits or closes the FIFO fd... >> >> So you did found the relevant PR with long audit trail and patches >> attached. You obviously should contact the author of the patches, >> Oliver Fromme, who is FreeBSD committer for some time (CCed). >> >> I agree that the thing shall be fixed finally. Skimming over the >> patches in kern/94772, I have some doubts about removal of >> POLLINIGNEOF flag. The reason is that we are generally do not >> remove exposed user interfaces. Maybe, but this flag was not a documented interface, and too much ugliness might be required to preserve its behaviour bug-for-bug compatibly (the old buggy behaviour would probably be more wanted for compatibility than the strange behaviour given by this flag! > I forward-ported Bruce' patch to the CURRENT. It passes the tests > from tools/regression/fifo and a test from kern/94772. Thanks. I won't be committing it any time soon, so you should. I rewrote the test programs extensively (enclosed at the end) in Oct 2007 and updated the kernel patches to match. Please run the new tests to see if you are missing anything important in the kernel part. If so, I will search for the kernel patches later (actually, now -- enclosed in the middle). I just ran them under RELENG_7 and unpatched -current and found no differences with the Oct 2007 version for RELENG_7 in the old test output. The old test output is in the following subdirectories: 4: FreeBSD-4 7: FreeBSD-7 l: Linux-2.6.10 m: my version of FreeBSD-5.2 including patches for this problem. AFAIR, the FreeBSD output in "m" is the same as the Linux output in all except a couple of cases where Linux select is inconsistent with itself and/or with Linux poll. However, the differences in the saved output are that the Linux output is mysteriously missing results for tests 5-8. The tests attempt to test certain race possibilities in a non-racy way. This is not easy and the differences might be due to some races/states not occurring under Linux. POSIX finally specified the behaviour strictly enough for it to be possible to test it a couple of years ago. I didn't follow all the developments and forget the details, but it was close to the Linux behaviour. > For my liking, I did not removed POLLINIGNEOF. > > diff --git a/sys/fs/fifofs/fifo_vnops.c b/sys/fs/fifofs/fifo_vnops.c > index 66963bc..7e279ca 100644 > --- a/sys/fs/fifofs/fifo_vnops.c > +++ b/sys/fs/fifofs/fifo_vnops.c > @@ -226,11 +226,47 @@ fail1: > if (ap->a_mode & FREAD) { > fip->fi_readers++; > if (fip->fi_readers == 1) { > + SOCKBUF_LOCK(&fip->fi_readsock->so_rcv); > + if (fip->fi_writers > 0) > + fip->fi_readsock->so_rcv.sb_state |= > + SBS_COULDRCV; My current version is in fact completely different. It doesn't have SBS_COULDRCV, but uses a generation count. IIRC, this is the same method as is used in Linux, and is needed for the same reasons (something to do with keeping new "connections" separate from old ones). So I will try to enclose the components of the patch in the order of your diff (might miss some). First one: % Index: fifo_vnops.c % =================================================================== % RCS file: /home/ncvs/src/sys/fs/fifofs/fifo_vnops.c,v % retrieving revision 1.100 % diff -u -2 -r1.100 fifo_vnops.c % --- fifo_vnops.c 23 Jun 2004 00:35:50 -0000 1.100 % +++ fifo_vnops.c 17 Oct 2007 11:36:23 -0000 % @@ -36,4 +36,5 @@ % #include <sys/fcntl.h> % #include <sys/file.h> % +#include <sys/filedesc.h> % #include <sys/kernel.h> % #include <sys/lock.h> % @@ -61,4 +62,5 @@ % long fi_readers; % long fi_writers; % + int fi_wgen; % }; % % @@ -182,8 +184,11 @@ % struct ucred *a_cred; % struct thread *a_td; % + int a_fdidx; % } */ *ap; % { % struct vnode *vp = ap->a_vp; % struct fifoinfo *fip; % + struct file *fp; % + struct filedesc *fdp; % struct thread *td = ap->a_td; % struct ucred *cred = ap->a_cred; % @@ -240,4 +245,10 @@ % } % } % + fdp = td->td_proc->p_fd; % + FILEDESC_LOCK(fdp); % + fp = fget_locked(fdp, ap->a_fdidx); % + /* Abuse f_msgcount as a generation count. */ % + fp->f_msgcount = fip->fi_wgen - fip->fi_writers; % + FILEDESC_UNLOCK(fdp); % } % if (ap->a_mode & FWRITE) { % @@ -248,10 +259,10 @@ % fip->fi_writers++; % if (fip->fi_writers == 1) { % - SOCKBUF_LOCK(&fip->fi_writesock->so_rcv); % + SOCKBUF_LOCK(&fip->fi_readsock->so_rcv); % fip->fi_readsock->so_rcv.sb_state &= ~SBS_CANTRCVMORE; % - SOCKBUF_UNLOCK(&fip->fi_writesock->so_rcv); % + SOCKBUF_UNLOCK(&fip->fi_readsock->so_rcv); % if (fip->fi_readers > 0) { % wakeup(&fip->fi_readers); % - sorwakeup(fip->fi_writesock); % + sorwakeup(fip->fi_readsock); % } % } % @@ -588,6 +599,8 @@ % if (ap->a_fflag & FWRITE) { % fip->fi_writers--; % - if (fip->fi_writers == 0) % + if (fip->fi_writers == 0) { % socantrcvmore(fip->fi_readsock); % + fip->fi_wgen++; % + } % } % fifo_cleanup(vp); % @@ -669,2 +682,28 @@ % return (ap->a_flags & F_FLOCK ? EOPNOTSUPP : EINVAL); % } % + % +fo_poll_t fifo_poll_f; % +int % +fifo_poll_f(struct file *fp, int events, struct ucred *cred, struct thread *td) % +{ % + struct file filetmp; % + struct fifoinfo *fip; % + int levents, revents = 0; % + % + fip = fp->f_vnode->v_fifoinfo; % + levents = events & (POLLIN | POLLPRI | POLLRDNORM | POLLRDBAND); % + if (levents) { % + filetmp.f_data = fip->fi_readsock; % + filetmp.f_cred = fp->f_cred; % + if (fp->f_msgcount == fip->fi_wgen) % + levents |= POLLINIGNEOF; % + revents |= soo_poll(&filetmp, levents, cred, td); % + } % + levents = events & (POLLOUT | POLLWRNORM | POLLWRBAND); % + if (levents) { % + filetmp.f_data = fip->fi_writesock; % + filetmp.f_cred = fp->f_cred; % + revents |= soo_poll(&filetmp, levents, cred, td); % + } + return (revents); +} Large parts of this (the existence of fifo_poll_f()...) are already in -current -- I copied them from -current to fix a critical bug. The above version of fifo_poll_f() is essentially much simpler. It still uses POLLINIGNEOF but sets it better using the generation counts. I don't remember why f_msgcount is abused as a generation count instead of adding a second generation count; probably it needs to go in `struct file' and I didn't want to edit file.h and/or bloat `struct file'. > diff --git a/sys/kern/sys_generic.c b/sys/kern/sys_generic.c > index 616c5b7..98ccc9e 100644 > --- a/sys/kern/sys_generic.c > +++ b/sys/kern/sys_generic.c > @@ -1226,8 +1226,8 @@ pollscan(td, fds, nfd) > * POLLERR if appropriate. > */ > selfdalloc(td, fds); > - fds->revents = fo_poll(fp, fds->events, > - td->td_ucred, td); > + fds->revents = fo_poll(fp, > + fds->events | POLLPOLL, td->td_ucred, td); > if (fds->revents != 0) > n++; > } I now seem to have only style fixes in pollscan() and selscan(). I don't need POLLPOLL. IIRC, the more intelligent/state-dependent setting of POLLINIGNEOF handles this for fifos, while different treatments work better for nameless pipes and sockets. The following is for nameless pipes: % Index: sys_pipe.c % =================================================================== % RCS file: /home/ncvs/src/sys/kern/sys_pipe.c,v % retrieving revision 1.171 % diff -u -2 -r1.171 sys_pipe.c % --- sys_pipe.c 27 Mar 2004 19:50:22 -0000 1.171 % +++ sys_pipe.c 16 Oct 2007 07:18:55 -0000 % @@ -1296,6 +1295,5 @@ % if (events & (POLLIN | POLLRDNORM)) % if ((rpipe->pipe_state & PIPE_DIRECTW) || % - (rpipe->pipe_buffer.cnt > 0) || % - (rpipe->pipe_state & PIPE_EOF)) % + (rpipe->pipe_buffer.cnt > 0)) % revents |= events & (POLLIN | POLLRDNORM); % POLLHUP will be set a little later if PIPE_EOF, and it is at best confusing for PIPE_EOF to imply POLLIN and POLLRDNORM (especially, "normal input"). I now vaguely remember that the differences with Linux are in this area. IIRC, the above makes nameless pipes behave like fifos in Linux (and now in FreeBSD), while nameless pipes behave differently from fifos in Linux. The tests assert that POLLIN and POLLHUP are set when there is EOF and (previously buffered) data to read, and that POLLIN becomes clear on reading the data. This is also be related to the longstanding bug in gdb -- "echo 'print 0' | gdb" exits before reading any input because it sees POLLIN | POLLHUP and thinks that POLLHUP implies no more input. poll() is hard to use unless it clears POLLIN once the input is read, and gdb-6.1 still doesn't understand how to use poll() whether or not poll() does the wrong thing here. gdb used to work OK using select(): With select(), there is nothing like POLLHUP and select() returns the equivalent of POLLIN for EOF with no data; then the reader must try a read() and hope that a read() of 0 means EOF. With broken poll(), readers must try a read similarly since they can't trust POLLIN if POLLHUP is also set. With non-broken poll(), they can trust POLLIN and not quit until poll() returns only POLLHUP. The tests describe this with different details. > diff --git a/sys/kern/uipc_socket.c b/sys/kern/uipc_socket.c > index 7341d3f..a13d648 100644 > --- a/sys/kern/uipc_socket.c > +++ b/sys/kern/uipc_socket.c > @@ -2706,6 +2706,42 @@ sopoll_generic(struct socket *so, int events, struct ucred *active_cred, > if (sowriteable(so)) > revents |= events & (POLLOUT | POLLWRNORM); > > + /* > + * SBS_CANTRCVMORE (which is checked by soreadable()) normally > + * implies EOF (and thus readable) and hung up, but for > + * compatibility with other systems and to obtain behavior that > + * is otherwise unavailable we make the case of poll() on a fifo > + * that has never had any writers during the lifetime of any > + * current reader special: then we pretend that the fifo is > + * unreadable unless it contains non-null data, and that it is > + * not hung up. The POLLPOLL flag is set by poll() to identify > + * poll() here, and the SBS_COULDRCV flag is set by the fifo > + * layer to indicate a fifo that is not in the special state. > + */ > + if (so->so_rcv.sb_state & SBS_CANTRCVMORE) { > + if (!(events & POLLPOLL) || so->so_rcv.sb_state & SBS_COULDRCV) > + revents |= POLLHUP; /* finish settings */ > + else if (!(so->so_rcv.sb_cc >= so->so_rcv.sb_lowat || > + !TAILQ_EMPTY(&so->so_comp) || so->so_error)) > + revents &= ~(POLLIN | POLLRDNORM); /* undo settings */ > + } > + > + /* > + * Testing of hangup for writers could be optimized by combining > + * it with testing for writeability, but we keep the test separate > + * and with the same organization as the test for readers for > + * clarity. Note that writeable implies not hung up, so if POLLHUP > + * is set here then (POLLOUT | POLLWRNORM) is not set above, as > + * standards require. Less obviously, if POLLHUP was set above for > + * a reader, then the output flags cannot have been set above for > + * a writer. Even less obviously, we cannot end up with both > + * POLLHUP output flags set in revents after ORing the revents for > + * the read and write socket in fifo_poll(). > + */ > + if (so->so_snd.sb_state & SBS_CANTSENDMORE) > + revents |= POLLHUP; > + > + > if (events & (POLLPRI | POLLRDBAND)) > if (so->so_oobmark || (so->so_rcv.sb_state & SBS_RCVATMARK)) > revents |= events & (POLLPRI | POLLRDBAND); My version is only subtly different from the above (except it doesn't add any comments -- not sure why). It uses a soreadable1() macro instead of getting tangled up trying to use soreadable(): % Index: uipc_socket.c % =================================================================== % RCS file: /home/ncvs/src/sys/kern/uipc_socket.c,v % retrieving revision 1.189 % diff -u -2 -r1.189 uipc_socket.c % --- uipc_socket.c 24 Jun 2004 04:28:30 -0000 1.189 % +++ uipc_socket.c 17 Oct 2007 10:44:50 -0000 % @@ -1862,4 +1860,8 @@ % } % % +#define soreadable1(so) \ % + ((so)->so_rcv.sb_cc >= (so)->so_rcv.sb_lowat || \ % + !TAILQ_EMPTY(&(so)->so_comp) || (so)->so_error) % + % int % sopoll(struct socket *so, int events, struct ucred *active_cred, % @@ -1869,12 +1871,7 @@ % % if (events & (POLLIN | POLLRDNORM)) % - if (soreadable(so)) % + if (soreadable1(so)) % revents |= events & (POLLIN | POLLRDNORM); % % - if (events & POLLINIGNEOF) % - if (so->so_rcv.sb_cc >= so->so_rcv.sb_lowat || % - !TAILQ_EMPTY(&so->so_comp) || so->so_error) % - revents |= POLLINIGNEOF; % - % if (events & (POLLOUT | POLLWRNORM)) % if (sowriteable(so)) % @@ -1885,8 +1882,12 @@ % revents |= events & (POLLPRI | POLLRDBAND); % % + if ((events & POLLINIGNEOF) == 0) % + if (so->so_rcv.sb_state & SBS_CANTRCVMORE) % + revents |= POLLHUP; % + if (so->so_rcv.sb_state & SBS_CANTSENDMORE) % + revents |= POLLHUP; % + % if (revents == 0) { % - if (events & % - (POLLIN | POLLINIGNEOF | POLLPRI | POLLRDNORM | % - POLLRDBAND)) { % + if (events & (POLLIN | POLLPRI | POLLRDNORM | POLLRDBAND)) { % SOCKBUF_LOCK(&so->so_rcv); % selrecord(td, &so->so_rcv.sb_sel); > diff --git a/sys/sys/poll.h b/sys/sys/poll.h > index c955f32..cfd5f01 100644 > --- a/sys/sys/poll.h > +++ b/sys/sys/poll.h > @@ -71,6 +71,10 @@ struct pollfd { > #define POLLINIGNEOF 0x2000 /* like POLLIN, except ignore EOF */ > #endif > > +#ifdef _KERNEL > +#define POLLPOLL 0x8000 /* system call is actually poll() */ > +#endif > + > /* > * These events are set if they occur regardless of whether they were > * requested. I don't have any changes here (still have POLLINIGNEOF). > diff --git a/sys/sys/sockbuf.h b/sys/sys/sockbuf.h > index b8e6699..0da4fa1 100644 > --- a/sys/sys/sockbuf.h > +++ b/sys/sys/sockbuf.h > @@ -56,6 +56,7 @@ > #define SBS_CANTSENDMORE 0x0010 /* can't send more data to peer */ > #define SBS_CANTRCVMORE 0x0020 /* can't receive more data from peer */ > #define SBS_RCVATMARK 0x0040 /* at mark on input */ > +#define SBS_COULDRCV 0x0080 /* could receive previously (or now) */ > > struct mbuf; > struct sockaddr; > I still had this patch, but no references to SBS_COULDRCV. Now I don't have it :-). Now for the test programs: #! /bin/sh # This is a shell archive. Remove anything before this line, then unpack # it by saving it into a file and typing "sh file". To overwrite existing # files, type "sh file -c". You can also feed this as standard input via # unshar, or by typing "sh <file", e.g.. If this archive is complete, you # will see the following message at the end: # "End of shell archive." # Contents: ./4/pipeselect.out ./4/pipepoll.out ./pipepoll.c # ./pipeselect.c ./m/pipeselect.out ./m/pipepoll.out # ./7/pipeselect.out ./7/pipepoll.out ./Makefile ./l/pipepoll.out # ./l/pipeselect.out # Wrapped by bde@besplex.bde.org on Thu Jun 4 03:05:36 2009 PATH=/bin:/usr/bin:/usr/ucb ; export PATH if test -f './4/pipeselect.out' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'./4/pipeselect.out'\" else echo shar: Extracting \"'./4/pipeselect.out'\" \(957 characters\) sed "s/^X//" >'./4/pipeselect.out' <<'END_OF_FILE' X1..20 Xok 1 Pipe state 4: expected clear; got clear Xok 2 Pipe state 5: expected set; got set Xok 3 Pipe state 6: expected set; got set Xok 4 Pipe state 6a: expected set; got set Xok 5 Sock state 4: expected clear; got clear Xok 6 Sock state 5: expected set; got set Xok 7 Sock state 6: expected set; got set Xok 8 Sock state 6a: expected set; got set Xnot ok 9 FIFO state 0: expected clear; got set Xok 10 FIFO state 1: expected clear; got clear Xok 11 FIFO state 2: expected set; got set Xok 12 FIFO state 2a: expected clear; got clear Xok 13 FIFO state 3: expected set; got set Xok 14 FIFO state 4: expected clear; got clear Xok 15 FIFO state 5: expected set; got set Xok 16 FIFO state 6: expected set; got set Xok 17 FIFO state 6a: expected set; got set Xnot ok 18 FIFO state 6b: expected clear; got set Xok 19 FIFO state 6c: expected set; got set Xok 20 FIFO state 6d: expected set; got set END_OF_FILE if test 957 -ne `wc -c <'./4/pipeselect.out'`; then echo shar: \"'./4/pipeselect.out'\" unpacked with wrong size! fi # end of './4/pipeselect.out' fi if test -f './4/pipepoll.out' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'./4/pipepoll.out'\" else echo shar: Extracting \"'./4/pipepoll.out'\" \(1049 characters\) sed "s/^X//" >'./4/pipepoll.out' <<'END_OF_FILE' X1..20 Xok 1 Pipe state 4: expected 0; got 0 Xok 2 Pipe state 5: expected POLLIN; got POLLIN Xok 3 Pipe state 6: expected POLLIN | POLLHUP; got POLLIN | POLLHUP Xnot ok 4 Pipe state 6a: expected POLLHUP; got POLLIN | POLLHUP Xok 5 Sock state 4: expected 0; got 0 Xok 6 Sock state 5: expected POLLIN; got POLLIN Xnot ok 7 Sock state 6: expected POLLIN | POLLHUP; got POLLIN Xnot ok 8 Sock state 6a: expected POLLHUP; got POLLIN Xnot ok 9 FIFO state 0: expected 0; got POLLIN Xok 10 FIFO state 1: expected 0; got 0 Xok 11 FIFO state 2: expected POLLIN; got POLLIN Xok 12 FIFO state 2a: expected 0; got 0 Xnot ok 13 FIFO state 3: expected POLLHUP; got POLLIN Xok 14 FIFO state 4: expected 0; got 0 Xok 15 FIFO state 5: expected POLLIN; got POLLIN Xnot ok 16 FIFO state 6: expected POLLIN | POLLHUP; got POLLIN Xnot ok 17 FIFO state 6a: expected POLLHUP; got POLLIN Xnot ok 18 FIFO state 6b: expected 0; got POLLIN Xnot ok 19 FIFO state 6c: expected POLLHUP; got POLLIN Xnot ok 20 FIFO state 6d: expected POLLHUP; got POLLIN END_OF_FILE if test 1049 -ne `wc -c <'./4/pipepoll.out'`; then echo shar: \"'./4/pipepoll.out'\" unpacked with wrong size! fi # end of './4/pipepoll.out' fi if test -f './pipepoll.c' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'./pipepoll.c'\" else echo shar: Extracting \"'./pipepoll.c'\" \(5929 characters\) sed "s/^X//" >'./pipepoll.c' <<'END_OF_FILE' X#include <sys/poll.h> X#include <sys/socket.h> X#include <sys/stat.h> X X#include <err.h> X#include <fcntl.h> X#include <signal.h> X#include <stdio.h> X#include <stdlib.h> X#include <unistd.h> X X#define FIFONAME "fifo.tmp" X#define FT_END 3 X#define FT_FIFO 2 X#define FT_PIPE 0 X#define FT_SOCKETPAIR 1 X Xstatic int filetype; X Xstatic const char * Xdecode_events(int events) X{ X char *ncresult; X const char *result; X X switch (events) { X case POLLIN: X result = "POLLIN"; X break; X case POLLHUP: X result = "POLLHUP"; X break; X case POLLIN | POLLHUP: X result = "POLLIN | POLLHUP"; X break; X default: X asprintf(&ncresult, "%#x", events); X result = ncresult; X break; X } X return (result); X} X Xstatic void Xreport(int num, const char *state, int expected, int got) X{ X if (expected == got) X printf("ok %-2d ", num); X else X printf("not ok %-2d", num); X printf(" %s state %s: expected %s; got %s\n", X filetype == FT_PIPE ? "Pipe" : X filetype == FT_SOCKETPAIR ? "Sock" : "FIFO", X state, decode_events(expected), decode_events(got)); X fflush(stdout); X} X Xstatic pid_t cpid; Xstatic pid_t ppid; Xstatic volatile sig_atomic_t state; X Xstatic void Xcatch(int sig) X{ X state++; X} X Xstatic void Xchild(int fd, int num) X{ X struct pollfd pfd; X int fd2; X char buf[256]; X X if (filetype == FT_FIFO) { X fd = open(FIFONAME, O_RDONLY | O_NONBLOCK); X if (fd < 0) X err(1, "open for read"); X } X pfd.fd = fd; X pfd.events = POLLIN; X X if (filetype == FT_FIFO) { X if (poll(&pfd, 1, 0) < 0) X err(1, "poll"); X report(num++, "0", 0, pfd.revents); X } X kill(ppid, SIGUSR1); X X usleep(1); X while (state != 1) X ; X if (filetype != FT_FIFO) { X /* X * The connection cannot be restablished. Use the code that X * delays the read until after the writer disconnects since X * that case is more interesting. X */ X state = 4; X goto state4; X } X if (poll(&pfd, 1, 0) < 0) X err(1, "poll"); X report(num++, "1", 0, pfd.revents); X kill(ppid, SIGUSR1); X X usleep(1); X while (state != 2) X ; X if (poll(&pfd, 1, 0) < 0) X err(1, "poll"); X report(num++, "2", POLLIN, pfd.revents); X if (read(fd, buf, sizeof buf) != 1) X err(1, "read"); X if (poll(&pfd, 1, 0) < 0) X err(1, "poll"); X report(num++, "2a", 0, pfd.revents); X kill(ppid, SIGUSR1); X X usleep(1); X while (state != 3) X ; X if (poll(&pfd, 1, 0) < 0) X err(1, "poll"); X report(num++, "3", POLLHUP, pfd.revents); X kill(ppid, SIGUSR1); X X /* X * Now we expect a new writer, and a new connection too since X * we read all the data. The only new point is that we didn't X * start quite from scratch since the read fd is not new. Check X * startup state as above, but don't do the read as above. X */ X usleep(1); X while (state != 4) X ; Xstate4: X if (poll(&pfd, 1, 0) < 0) X err(1, "poll"); X report(num++, "4", 0, pfd.revents); X kill(ppid, SIGUSR1); X X usleep(1); X while (state != 5) X ; X if (poll(&pfd, 1, 0) < 0) X err(1, "poll"); X report(num++, "5", POLLIN, pfd.revents); X kill(ppid, SIGUSR1); X X usleep(1); X while (state != 6) X ; X /* X * Now we have no writer, but should still have data from the old X * writer. Check that we have both a data condition and a hangup X * condition, and that the data can read the data in the usual way. X * Since Linux does this, programs must not quit reading when they X * see POLLHUP; they must see POLLHUP without POLLIN (or another X * input condition) before they decide that there is EOF. gdb-6.1.1 X * is an example of a broken program that quits on POLLHUP only -- X * see its event-loop.c. X */ X if (poll(&pfd, 1, 0) < 0) X err(1, "poll"); X report(num++, "6", POLLIN | POLLHUP, pfd.revents); X if (read(fd, buf, sizeof buf) != 1) X err(1, "read"); X if (poll(&pfd, 1, 0) < 0) X err(1, "poll"); X report(num++, "6a", POLLHUP, pfd.revents); X if (filetype == FT_FIFO) { X /* X * Check that POLLHUP is not seen by new readers but is X * still seen by old readers. X */ X fd2 = open(FIFONAME, O_RDONLY | O_NONBLOCK); X if (fd2 < 0) X err(1, "open for read"); X pfd.fd = fd2; X if (poll(&pfd, 1, 0) < 0) X err(1, "poll"); X report(num++, "6b", 0, pfd.revents); X pfd.fd = fd; X if (poll(&pfd, 1, 0) < 0) X err(1, "poll"); X report(num++, "6c", POLLHUP, pfd.revents); X close(fd2); X if (poll(&pfd, 1, 0) < 0) X err(1, "poll"); X report(num++, "6d", POLLHUP, pfd.revents); X } X close(fd); X kill(ppid, SIGUSR1); X X exit(0); X} X Xstatic void Xparent(int fd) X{ X usleep(1); X while (state != 1) X ; X if (filetype == FT_FIFO) { X fd = open(FIFONAME, O_WRONLY | O_NONBLOCK); X if (fd < 0) X err(1, "open for write"); X } X kill(cpid, SIGUSR1); X X usleep(1); X while (state != 2) X ; X if (write(fd, "", 1) != 1) X err(1, "write"); X kill(cpid, SIGUSR1); X X usleep(1); X while (state != 3) X ; X if (close(fd) != 0) X err(1, "close for write"); X kill(cpid, SIGUSR1); X X usleep(1); X while (state != 4) X ; X if (filetype != FT_FIFO) X return; X fd = open(FIFONAME, O_WRONLY | O_NONBLOCK); X if (fd < 0) X err(1, "open for write"); X kill(cpid, SIGUSR1); X X usleep(1); X while (state != 5) X ; X if (write(fd, "", 1) != 1) X err(1, "write"); X kill(cpid, SIGUSR1); X X usleep(1); X while (state != 6) X ; X if (close(fd) != 0) X err(1, "close for write"); X kill(cpid, SIGUSR1); X X usleep(1); X while (state != 7) X ; X} X Xint Xmain(void) X{ X int fd[2], num; X X num = 1; X printf("1..20\n"); X fflush(stdout); X signal(SIGUSR1, catch); X ppid = getpid(); X for (filetype = 0; filetype < FT_END; filetype++) { X switch (filetype) { X case FT_FIFO: X if (mkfifo(FIFONAME, 0666) != 0) X err(1, "mkfifo"); X fd[0] = -1; X fd[1] = -1; X break; X case FT_SOCKETPAIR: X if (socketpair(AF_UNIX, SOCK_STREAM, AF_UNSPEC, X fd) != 0) X err(1, "socketpair"); X break; X case FT_PIPE: X if (pipe(fd) != 0) X err(1, "pipe"); X break; X } X state = 0; X switch (cpid = fork()) { X case -1: X err(1, "fork"); X case 0: X (void)close(fd[1]); X child(fd[0], num); X break; X default: X (void)close(fd[0]); X parent(fd[1]); X break; X } X num += filetype == FT_FIFO ? 12 : 4; X } X (void)unlink(FIFONAME); X return (0); X} END_OF_FILE if test 5929 -ne `wc -c <'./pipepoll.c'`; then echo shar: \"'./pipepoll.c'\" unpacked with wrong size! fi # end of './pipepoll.c' fi if test -f './pipeselect.c' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'./pipeselect.c'\" else echo shar: Extracting \"'./pipeselect.c'\" \(6476 characters\) sed "s/^X//" >'./pipeselect.c' <<'END_OF_FILE' X#include <sys/socket.h> X#include <sys/select.h> X#include <sys/stat.h> X X#include <err.h> X#include <fcntl.h> X#include <signal.h> X#include <stdio.h> X#include <stdlib.h> X#include <unistd.h> X X#define FIFONAME "fifo.tmp" X#define FT_END 3 X#define FT_FIFO 2 X#define FT_PIPE 0 X#define FT_SOCKETPAIR 1 X X#define SETUP(fd, rfds, tv) do { \ X FD_ZERO(&(rfds)); \ X FD_SET((fd), &(rfds)); \ X (tv).tv_sec = 0; \ X (tv).tv_usec = 0; \ X} while (0) X Xstatic int filetype; X Xstatic const char * Xdecode_events(int events) X{ X return (events ? "set" : "clear"); X} X Xstatic void Xreport(int num, const char *state, int expected, int got) X{ X if (!expected == !got) X printf("ok %-2d ", num); X else X printf("not ok %-2d", num); X printf(" %s state %s: expected %s; got %s\n", X filetype == FT_PIPE ? "Pipe" : X filetype == FT_SOCKETPAIR ? "Sock" : "FIFO", X state, decode_events(expected), decode_events(got)); X fflush(stdout); X} X Xstatic pid_t cpid; Xstatic pid_t ppid; Xstatic volatile sig_atomic_t state; X Xstatic void Xcatch(int sig) X{ X state++; X} X Xstatic void Xchild(int fd, int num) X{ X fd_set rfds; X struct timeval tv; X int fd1, fd2; X char buf[256]; X X if (filetype == FT_FIFO) { X fd = open(FIFONAME, O_RDONLY | O_NONBLOCK); X if (fd < 0) X err(1, "open for read"); X } X if (fd >= FD_SETSIZE) X errx(1, "fd = %d too large for select()", fd); X X if (filetype == FT_FIFO) { X SETUP(fd, rfds, tv); X if (select(fd + 1, &rfds, NULL, NULL, &tv) < 0) X err(1, "select"); X report(num++, "0", 0, FD_ISSET(fd, &rfds)); X } X kill(ppid, SIGUSR1); X X usleep(1); X while (state != 1) X ; X if (filetype != FT_FIFO) { X /* X * The connection cannot be restablished. Use the code that X * delays the read until after the writer disconnects since X * that case is more interesting. X */ X state = 4; X goto state4; X } X SETUP(fd, rfds, tv); X if (select(fd + 1, &rfds, NULL, NULL, &tv) < 0) X err(1, "select"); X report(num++, "1", 0, FD_ISSET(fd, &rfds)); X kill(ppid, SIGUSR1); X X usleep(1); X while (state != 2) X ; X SETUP(fd, rfds, tv); X if (select(fd + 1, &rfds, NULL, NULL, &tv) < 0) X err(1, "select"); X report(num++, "2", 1, FD_ISSET(fd, &rfds)); X if (read(fd, buf, sizeof buf) != 1) X err(1, "read"); X SETUP(fd, rfds, tv); X if (select(fd + 1, &rfds, NULL, NULL, &tv) < 0) X err(1, "select"); X report(num++, "2a", 0, FD_ISSET(fd, &rfds)); X kill(ppid, SIGUSR1); X X usleep(1); X while (state != 3) X ; X SETUP(fd, rfds, tv); X if (select(fd + 1, &rfds, NULL, NULL, &tv) < 0) X err(1, "select"); X report(num++, "3", 1, FD_ISSET(fd, &rfds)); X kill(ppid, SIGUSR1); X X /* X * Now we expect a new writer, and a new connection too since X * we read all the data. The only new point is that we didn't X * start quite from scratch since the read fd is not new. Check X * startup state as above, but don't do the read as above. X */ X usleep(1); X while (state != 4) X ; Xstate4: X SETUP(fd, rfds, tv); X if (select(fd + 1, &rfds, NULL, NULL, &tv) < 0) X err(1, "select"); X report(num++, "4", 0, FD_ISSET(fd, &rfds)); X kill(ppid, SIGUSR1); X X usleep(1); X while (state != 5) X ; X SETUP(fd, rfds, tv); X if (select(fd + 1, &rfds, NULL, NULL, &tv) < 0) X err(1, "select"); X report(num++, "5", 1, FD_ISSET(fd, &rfds)); X kill(ppid, SIGUSR1); X X usleep(1); X while (state != 6) X ; X /* X * Now we have no writer, but should still have data from the old X * writer. Check that we have both a data condition and a hangup X * condition, and that the data can read the data in the usual way. X * Since Linux does this, programs must not quit reading when they X * see POLLHUP; they must see POLLHUP without POLLIN (or another X * input condition) before they decide that there is EOF. gdb-6.1.1 X * is an example of a broken program that quits on POLLHUP only -- X * see its event-loop.c. X */ X SETUP(fd, rfds, tv); X if (select(fd + 1, &rfds, NULL, NULL, &tv) < 0) X err(1, "select"); X report(num++, "6", 1, FD_ISSET(fd, &rfds)); X if (read(fd, buf, sizeof buf) != 1) X err(1, "read"); X SETUP(fd, rfds, tv); X if (select(fd + 1, &rfds, NULL, NULL, &tv) < 0) X err(1, "select"); X report(num++, "6a", 1, FD_ISSET(fd, &rfds)); X if (filetype == FT_FIFO) { X /* X * Check that POLLHUP is not seen by new readers but is X * still seen by old readers. X */ X fd2 = open(FIFONAME, O_RDONLY | O_NONBLOCK); X if (fd2 < 0) X err(1, "open for read"); X fd1 = fd; X fd = fd2; X SETUP(fd, rfds, tv); X if (select(fd + 1, &rfds, NULL, NULL, &tv) < 0) X err(1, "select"); X report(num++, "6b", 0, FD_ISSET(fd, &rfds)); X fd = fd1; X SETUP(fd, rfds, tv); X if (select(fd + 1, &rfds, NULL, NULL, &tv) < 0) X err(1, "select"); X report(num++, "6c", 1, FD_ISSET(fd, &rfds)); X close(fd2); X SETUP(fd, rfds, tv); X if (select(fd + 1, &rfds, NULL, NULL, &tv) < 0) X err(1, "select"); X report(num++, "6d", 1, FD_ISSET(fd, &rfds)); X } X close(fd); X kill(ppid, SIGUSR1); X X exit(0); X} X Xstatic void Xparent(int fd) X{ X usleep(1); X while (state != 1) X ; X if (filetype == FT_FIFO) { X fd = open(FIFONAME, O_WRONLY | O_NONBLOCK); X if (fd < 0) X err(1, "open for write"); X } X kill(cpid, SIGUSR1); X X usleep(1); X while (state != 2) X ; X if (write(fd, "", 1) != 1) X err(1, "write"); X kill(cpid, SIGUSR1); X X usleep(1); X while (state != 3) X ; X if (close(fd) != 0) X err(1, "close for write"); X kill(cpid, SIGUSR1); X X usleep(1); X while (state != 4) X ; X if (filetype != FT_FIFO) X return; X fd = open(FIFONAME, O_WRONLY | O_NONBLOCK); X if (fd < 0) X err(1, "open for write"); X kill(cpid, SIGUSR1); X X usleep(1); X while (state != 5) X ; X if (write(fd, "", 1) != 1) X err(1, "write"); X kill(cpid, SIGUSR1); X X usleep(1); X while (state != 6) X ; X if (close(fd) != 0) X err(1, "close for write"); X kill(cpid, SIGUSR1); X X usleep(1); X while (state != 7) X ; X} X Xint Xmain(void) X{ X int fd[2], num; X X num = 1; X printf("1..20\n"); X fflush(stdout); X signal(SIGUSR1, catch); X ppid = getpid(); X for (filetype = 0; filetype < FT_END; filetype++) { X switch (filetype) { X case FT_FIFO: X if (mkfifo(FIFONAME, 0666) != 0) X err(1, "mkfifo"); X fd[0] = -1; X fd[1] = -1; X break; X case FT_SOCKETPAIR: X if (socketpair(AF_UNIX, SOCK_STREAM, AF_UNSPEC, X fd) != 0) X err(1, "socketpair"); X break; X case FT_PIPE: X if (pipe(fd) != 0) X err(1, "pipe"); X break; X } X state = 0; X switch (cpid = fork()) { X case -1: X err(1, "fork"); X case 0: X (void)close(fd[1]); X child(fd[0], num); X break; X default: X (void)close(fd[0]); X parent(fd[1]); X break; X } X num += filetype == FT_FIFO ? 12 : 4; X } X (void)unlink(FIFONAME); X return (0); X} END_OF_FILE if test 6476 -ne `wc -c <'./pipeselect.c'`; then echo shar: \"'./pipeselect.c'\" unpacked with wrong size! fi # end of './pipeselect.c' fi if test -f './m/pipeselect.out' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'./m/pipeselect.out'\" else echo shar: Extracting \"'./m/pipeselect.out'\" \(961 characters\) sed "s/^X//" >'./m/pipeselect.out' <<'END_OF_FILE' X1..20 Xok 1 Pipe state 4: expected clear; got clear Xok 2 Pipe state 5: expected set; got set Xok 3 Pipe state 6: expected set; got set Xok 4 Pipe state 6a: expected set; got set Xok 5 Sock state 4: expected clear; got clear Xok 6 Sock state 5: expected set; got set Xok 7 Sock state 6: expected set; got set Xok 8 Sock state 6a: expected set; got set Xok 9 FIFO state 0: expected clear; got clear Xok 10 FIFO state 1: expected clear; got clear Xok 11 FIFO state 2: expected set; got set Xok 12 FIFO state 2a: expected clear; got clear Xok 13 FIFO state 3: expected set; got set Xok 14 FIFO state 4: expected clear; got clear Xok 15 FIFO state 5: expected set; got set Xok 16 FIFO state 6: expected set; got set Xok 17 FIFO state 6a: expected set; got set Xok 18 FIFO state 6b: expected clear; got clear Xok 19 FIFO state 6c: expected set; got set Xok 20 FIFO state 6d: expected set; got set END_OF_FILE if test 961 -ne `wc -c <'./m/pipeselect.out'`; then echo shar: \"'./m/pipeselect.out'\" unpacked with wrong size! fi # end of './m/pipeselect.out' fi if test -f './m/pipepoll.out' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'./m/pipepoll.out'\" else echo shar: Extracting \"'./m/pipepoll.out'\" \(1055 characters\) sed "s/^X//" >'./m/pipepoll.out' <<'END_OF_FILE' X1..20 Xok 1 Pipe state 4: expected 0; got 0 Xok 2 Pipe state 5: expected POLLIN; got POLLIN Xok 3 Pipe state 6: expected POLLIN | POLLHUP; got POLLIN | POLLHUP Xok 4 Pipe state 6a: expected POLLHUP; got POLLHUP Xok 5 Sock state 4: expected 0; got 0 Xok 6 Sock state 5: expected POLLIN; got POLLIN Xok 7 Sock state 6: expected POLLIN | POLLHUP; got POLLIN | POLLHUP Xok 8 Sock state 6a: expected POLLHUP; got POLLHUP Xok 9 FIFO state 0: expected 0; got 0 Xok 10 FIFO state 1: expected 0; got 0 Xok 11 FIFO state 2: expected POLLIN; got POLLIN Xok 12 FIFO state 2a: expected 0; got 0 Xok 13 FIFO state 3: expected POLLHUP; got POLLHUP Xok 14 FIFO state 4: expected 0; got 0 Xok 15 FIFO state 5: expected POLLIN; got POLLIN Xok 16 FIFO state 6: expected POLLIN | POLLHUP; got POLLIN | POLLHUP Xok 17 FIFO state 6a: expected POLLHUP; got POLLHUP Xok 18 FIFO state 6b: expected 0; got 0 Xok 19 FIFO state 6c: expected POLLHUP; got POLLHUP Xok 20 FIFO state 6d: expected POLLHUP; got POLLHUP END_OF_FILE if test 1055 -ne `wc -c <'./m/pipepoll.out'`; then echo shar: \"'./m/pipepoll.out'\" unpacked with wrong size! fi # end of './m/pipepoll.out' fi if test -f './7/pipeselect.out' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'./7/pipeselect.out'\" else echo shar: Extracting \"'./7/pipeselect.out'\" \(969 characters\) sed "s/^X//" >'./7/pipeselect.out' <<'END_OF_FILE' X1..20 Xok 1 Pipe state 4: expected clear; got clear Xok 2 Pipe state 5: expected set; got set Xok 3 Pipe state 6: expected set; got set Xok 4 Pipe state 6a: expected set; got set Xok 5 Sock state 4: expected clear; got clear Xok 6 Sock state 5: expected set; got set Xok 7 Sock state 6: expected set; got set Xok 8 Sock state 6a: expected set; got set Xok 9 FIFO state 0: expected clear; got clear Xok 10 FIFO state 1: expected clear; got clear Xok 11 FIFO state 2: expected set; got set Xok 12 FIFO state 2a: expected clear; got clear Xnot ok 13 FIFO state 3: expected set; got clear Xok 14 FIFO state 4: expected clear; got clear Xok 15 FIFO state 5: expected set; got set Xok 16 FIFO state 6: expected set; got set Xnot ok 17 FIFO state 6a: expected set; got clear Xok 18 FIFO state 6b: expected clear; got clear Xnot ok 19 FIFO state 6c: expected set; got clear Xnot ok 20 FIFO state 6d: expected set; got clear END_OF_FILE if test 969 -ne `wc -c <'./7/pipeselect.out'`; then echo shar: \"'./7/pipeselect.out'\" unpacked with wrong size! fi # end of './7/pipeselect.out' fi if test -f './7/pipepoll.out' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'./7/pipepoll.out'\" else echo shar: Extracting \"'./7/pipepoll.out'\" \(1019 characters\) sed "s/^X//" >'./7/pipepoll.out' <<'END_OF_FILE' X1..20 Xok 1 Pipe state 4: expected 0; got 0 Xok 2 Pipe state 5: expected POLLIN; got POLLIN Xok 3 Pipe state 6: expected POLLIN | POLLHUP; got POLLIN | POLLHUP Xnot ok 4 Pipe state 6a: expected POLLHUP; got POLLIN | POLLHUP Xok 5 Sock state 4: expected 0; got 0 Xok 6 Sock state 5: expected POLLIN; got POLLIN Xnot ok 7 Sock state 6: expected POLLIN | POLLHUP; got POLLIN Xnot ok 8 Sock state 6a: expected POLLHUP; got POLLIN Xok 9 FIFO state 0: expected 0; got 0 Xok 10 FIFO state 1: expected 0; got 0 Xok 11 FIFO state 2: expected POLLIN; got POLLIN Xok 12 FIFO state 2a: expected 0; got 0 Xnot ok 13 FIFO state 3: expected POLLHUP; got 0 Xok 14 FIFO state 4: expected 0; got 0 Xok 15 FIFO state 5: expected POLLIN; got POLLIN Xnot ok 16 FIFO state 6: expected POLLIN | POLLHUP; got POLLIN Xnot ok 17 FIFO state 6a: expected POLLHUP; got 0 Xok 18 FIFO state 6b: expected 0; got 0 Xnot ok 19 FIFO state 6c: expected POLLHUP; got 0 Xnot ok 20 FIFO state 6d: expected POLLHUP; got 0 END_OF_FILE if test 1019 -ne `wc -c <'./7/pipepoll.out'`; then echo shar: \"'./7/pipepoll.out'\" unpacked with wrong size! fi # end of './7/pipepoll.out' fi if test -f './Makefile' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'./Makefile'\" else echo shar: Extracting \"'./Makefile'\" \(494 characters\) sed "s/^X//" >'./Makefile' <<'END_OF_FILE' X# This makefile has been uglified for portability. X# Nothing yet works with gmake for the path to the sources. X.PATH: .. X XPROG= pipepoll pipeselect XCFLAGS+= -Werror -Wall X Xall: ${PROG} Xpipepoll: pipepoll.c Xpipeselect: pipeselect.c X Xpipepoll pipeselect: X ${CC} ${CFLAGS} ${LDFLAGS} -o $@ $@.c X Xtest: all X -for prog in ${PROG}; do \ X ./$${prog} > $${prog}.out.new; \ X diff -u1 $${prog}.out $${prog}.out.new; \ X done X Xclean: X for prog in ${PROG}; do \ X rm -f $${prog} $${prog}.out.new; \ X done END_OF_FILE if test 494 -ne `wc -c <'./Makefile'`; then echo shar: \"'./Makefile'\" unpacked with wrong size! fi # end of './Makefile' fi if test -f './l/pipepoll.out' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'./l/pipepoll.out'\" else echo shar: Extracting \"'./l/pipepoll.out'\" \(834 characters\) sed "s/^X//" >'./l/pipepoll.out' <<'END_OF_FILE' X1..20 Xok 1 Pipe state 4: expected 0; got 0 Xok 2 Pipe state 5: expected POLLIN; got POLLIN Xok 3 Pipe state 6: expected POLLIN | POLLHUP; got POLLIN | POLLHUP Xok 4 Pipe state 6a: expected POLLHUP; got POLLHUP Xok 9 FIFO state 0: expected 0; got 0 Xok 10 FIFO state 1: expected 0; got 0 Xok 11 FIFO state 2: expected POLLIN; got POLLIN Xok 12 FIFO state 2a: expected 0; got 0 Xok 13 FIFO state 3: expected POLLHUP; got POLLHUP Xok 14 FIFO state 4: expected 0; got 0 Xok 15 FIFO state 5: expected POLLIN; got POLLIN Xok 16 FIFO state 6: expected POLLIN | POLLHUP; got POLLIN | POLLHUP Xok 17 FIFO state 6a: expected POLLHUP; got POLLHUP Xok 18 FIFO state 6b: expected 0; got 0 Xok 19 FIFO state 6c: expected POLLHUP; got POLLHUP Xok 20 FIFO state 6d: expected POLLHUP; got POLLHUP END_OF_FILE if test 834 -ne `wc -c <'./l/pipepoll.out'`; then echo shar: \"'./l/pipepoll.out'\" unpacked with wrong size! fi # end of './l/pipepoll.out' fi if test -f './l/pipeselect.out' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'./l/pipeselect.out'\" else echo shar: Extracting \"'./l/pipeselect.out'\" \(772 characters\) sed "s/^X//" >'./l/pipeselect.out' <<'END_OF_FILE' X1..20 Xok 1 Pipe state 4: expected clear; got clear Xok 2 Pipe state 5: expected set; got set Xok 3 Pipe state 6: expected set; got set Xok 4 Pipe state 6a: expected set; got set Xok 9 FIFO state 0: expected clear; got clear Xok 10 FIFO state 1: expected clear; got clear Xok 11 FIFO state 2: expected set; got set Xok 12 FIFO state 2a: expected clear; got clear Xok 13 FIFO state 3: expected set; got set Xok 14 FIFO state 4: expected clear; got clear Xok 15 FIFO state 5: expected set; got set Xok 16 FIFO state 6: expected set; got set Xok 17 FIFO state 6a: expected set; got set Xok 18 FIFO state 6b: expected clear; got clear Xok 19 FIFO state 6c: expected set; got set Xok 20 FIFO state 6d: expected set; got set END_OF_FILE if test 772 -ne `wc -c <'./l/pipeselect.out'`; then echo shar: \"'./l/pipeselect.out'\" unpacked with wrong size! fi # end of './l/pipeselect.out' fi echo shar: End of shell archive. exit 0 Bruce
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20090604030724.V1181>