Date: Thu, 10 Aug 2000 18:08:38 -0700 From: Alfred Perlstein <bright@wintelcom.net> To: Michael Owens <owensmk@earthlink.net> Cc: hackers@FreeBSD.ORG Subject: Re: R. Stevens select() Collisions Scenario Message-ID: <20000810180838.V4854@fw.wintelcom.net> In-Reply-To: <39934E64.A5BEE8EE@earthlink.net>; from owensmk@earthlink.net on Thu, Aug 10, 2000 at 07:52:52PM -0500 References: <39934E64.A5BEE8EE@earthlink.net>
next in thread | previous in thread | raw e-mail | index | archive | help
* Michael Owens <owensmk@earthlink.net> [000810 17:51] wrote:
> Purpose:
>
> I am trying to write a non-blocking, preforked server, specifically to
> run on FreeBSD, and have a general question as to whether or not my
> strategy is sound.
>
>
> Problem:
>
> In Unix Network Programming Vol. 1 (section 27.6, p.741), Stevens
> mentions a scenario under a preforked server design where multiple
> children are calling select() on the same descriptor, causing collisions
> for the kernel to resolve. The children would look something like...
> .
> .
> .
> for( ; ; ){
> FD_SET(listfd,&rset);
> Select(listfd+1,&rset,NULL,NULL,NULL);
> if(FD_ISSET(listfd,&rset)==0)
> err_quit("listenfd readable");
>
> clilen = addrlen;
> connfd = Accept(listenfd, cliaddr, &clilen);
>
> process(connfd);
> Close(connfd);
> }
> .
> .
> .
>
> Hypothesis:
>
> If you were implement the server such that the children used
> non-blocking I/O, so that select() returned immediately, would this
> alleviate the problem of collisions in the kernel and its associated
> overhead?
Select ignores non-blocking IO, if you have a non-blocking socket and
perform a select on it, you _will_ block (unless you use a zero valued
timeout).
> If so, would you then have to worry about mutual exclusion when select
> did return a ready descriptor for accept(). For example, a child gets
> switched just after calling select() but just before it makes it to
> accept(). Thus, the next child would also receive the same descriptor
> ready for accept()? Now only one of the two children will get to
> accept() first, leaving the other blocking on accept(). Stevens mentions
> a similar case at the end of the Nonblocking I/O chapter (section 15.6,
> p. 422) and to avoid this he recommends 1) setting the listening
> descriptor to non-blocking and 2) ignoring EWOULDBLOCK, ECONNABORTED,
> etc. on accept().
Yes, that is correct and the correct thing to do.
> So, if you make the listening descriptor non-blocking, and treat
> select() and accept() appropriately, you should be alright in both
> avoiding select collisions and there overhead, as well as avoid children
> blocking due to losing possible race conditions for the listen
> descriptor in the event (albiet small) of a context switch between
> select() and accept().
Another way of doing this would be to have a 'master' accept process
that uses fd passing (it's in your book) to hand off connections to
children, this would avoid select collisions at the cost of an additional
context switch.
>
> Apology:
>
> The reason I bother you with all this is that while I (think I)
> understand the logic, I (know I) am at the limits of my understanding,
> and might be missing some important considerations as to other goings on
> in the kernel, and that there might be a better way to go about all
> this.
Well for one thing I wouldn't be using select, I would use FreeBSD's
kqueue mechanism, it's vastly superior to select and poll, although
given the choice of select vs poll I would use poll().
>
> Summary:
>
> Ultimately, all I am seeking to do is have an efficient and scalable
> server design, and to my knowledge, this would consist of a number of
> preforked children who are non-blocking/multiplexing themselves.
> Furthermore, each child will have a pool of worker threads which will
> handle jobs sent by different clients. As long as the threads' work
> doesn't entail an operation that blocks for appreciable amount of time
> (so that a single thread puts the whole child to sleep), it would seem
> like this might be a decent proposal: multiple clients and multiple jobs
> being handled simultaneously by each child, and the number of children
> can be controlled by the parent according to the load and limits of the
> hardware.
>
> Does this seem like a good way to go about it?
Yes. :) When FreeBSD gets scheduler activations you'll be able to change
to a single threaded process that will have excellent performance, the
scheduler activations are just around the corner.
>
> Thanks:
you're welcome.
>
> Thanks.
too much coffee today? :)
-Alfred
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20000810180838.V4854>