Date: Wed, 13 Feb 2002 17:30:10 +0600 From: Alexey Dokuchaev <danfe@cytherea.weblab.nsu.ru> To: "Eugene L. Vorokov" <vel@bugz.infotecs.ru> Cc: freebsd-hackers@freebsd.org, thttpd@acme.com Subject: Re: THTTPD web server: problems with KQUEUE on FreeBSD 4.5-STABLE Message-ID: <20020213173010.E14414@cytherea.weblab.nsu.ru> In-Reply-To: <200202131028.g1DASTY88632@bugz.infotecs.ru>; from vel@bugz.infotecs.ru on Wed, Feb 13, 2002 at 01:28:29PM %2B0300 References: <20020213160014.A97359@cytherea.weblab.nsu.ru> <200202131028.g1DASTY88632@bugz.infotecs.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Feb 13, 2002 at 01:28:29PM +0300, Eugene L. Vorokov wrote:
> > I still wonder, whether this problem occurs because of how thttpd does
> > things, or how FreeBSD implements kqueue stuff, however, I am not sure
> > that I will have enough time to dig deep into this. Right now I'm pretty
> > happy with the fact that I got thttpd working again, and those "200 0"
> > messages are no longer in my logs. However, it is still an issue to
> > worry about, I believe, and I will be a lot more happy if my experience
> > helps either folks to find and fix some probable bugs (if any) in their
> > excellent software.
>
> Hm, there is no such thing as kqread() I think. There are kqueue() and
Oh, I merely meant that thttpd process was in kqread state in top(1) output.
Sorry for confusing you.
> kevent(). You didn't show the piece of code that uses it, so it's hard
> to say what the problem is. I recommend you to read jlemon's paper
OK, this is how kqueue mechanism is used in thttpd. This is from fdwatch.c
source file, starting at line 300 (I apologize for it's length):
#ifdef HAVE_KQUEUE
static struct kevent* kqevents;
static int nkqevents;
static int* kqrfdidx;
static int kq;
static int
kqueue_init( int nfiles )
{
kq = kqueue();
if ( kq == -1 )
return -1;
kqevents = (struct kevent*) malloc( sizeof(struct kevent) * nfiles );
kqrfdidx = (int*) malloc( sizeof(int) * nfiles );
if ( kqevents == (struct kevent*) 0 || kqrfdidx == (int*) 0 )
return -1;
(void) memset( kqevents, 0, sizeof(struct kevent) * nfiles );
(void) memset( kqrfdidx, 0, sizeof(int) * nfiles );
return 0;
}
static void
kqueue_add_fd( int fd, int rw )
{
kqevents[nkqevents].ident = fd;
kqevents[nkqevents].flags = EV_ADD;
switch ( rw )
{
case FDW_READ: kqevents[nkqevents].filter = EVFILT_READ; break;
case FDW_WRITE: kqevents[nkqevents].filter = EVFILT_WRITE; break;
default: break;
}
++nkqevents;
}
static void
kqueue_del_fd( int fd )
{
kqevents[nkqevents].ident = fd;
kqevents[nkqevents].flags = EV_DELETE;
switch ( fd_rw[fd] )
{
case FDW_READ: kqevents[nkqevents].filter = EVFILT_READ; break;
case FDW_WRITE: kqevents[nkqevents].filter = EVFILT_WRITE; break;
}
++nkqevents;
}
static int
kqueue_watch( long timeout_msecs )
{
int i, r;
if ( timeout_msecs == INFTIM )
r = kevent(
kq, kqevents, nkqevents, kqevents, nfiles, (struct timespec*) 0 );
else
{
struct timespec ts;
ts.tv_sec = timeout_msecs / 1000L;
ts.tv_nsec = ( timeout_msecs % 1000L ) * 1000000L;
r = kevent( kq, kqevents, nkqevents, kqevents, nfiles, &ts );
}
nkqevents = 0;
if ( r == -1 )
return -1;
for ( i = 0; i < r; ++i )
kqrfdidx[kqevents[i].ident] = i;
return r;
}
static int
kqueue_check_fd( int fd )
{
int ridx = kqrfdidx[fd];
if ( kqevents[ridx].ident != fd )
return 0;
if ( kqevents[ridx].flags & EV_ERROR )
return 0;
switch ( fd_rw[fd] )
{
case FDW_READ: return kqevents[ridx].filter == EVFILT_READ;
case FDW_WRITE: return kqevents[ridx].filter == EVFILT_WRITE;
default: return 0;
}
}
static int
kqueue_get_fd( int ridx )
{
return kqevents[ridx].ident;
}
#else /* HAVE_KQUEUE */
In my previous mail I cited the code where error is often returned, and
that confuses the server. It was this one:
if ( ! fdwatch_check_fd( hc->conn_fd ) ) {
/* Something went wrong. */ printf("!!!!!!!!!!!!!!!!!!\n");
clear_connection( c, &tv ); }
The fdwatch_check_fd() function is merely this one:
/* Check if a descriptor was ready. */
int
fdwatch_check_fd( int fd )
{
#ifdef HAVE_KQUEUE
return kqueue_check_fd( fd );
#else
# ifdef HAVE_POLL
return poll_check_fd( fd );
# else /* HAVE_POLL */
# ifdef HAVE_SELECT
return select_check_fd( fd );
# else /* HAVE_SELECT */
return 0;
# endif /* HAVE_SELECT */
# endif /* HAVE_POLL */
#endif /* HAVE_KQUEUE */
}
I really do hope that it will help someone to spot any possible problems
with this kqueue code.
> at http://www.freebsd.org/~jlemon/kqueue.pdf to see how to you should
> work with kqueue() and kevent().
> As for performance, I can confirm that kqueue() functions is better
> than select()/poll() at least in situations where we have to do non-blocking
> i/o on some large number of fd's (say, several thousands), and actually
> each time we see some activity only on small number of them (say,
> several hundred). That's how ircd usually works, and system load goes
> down significantly with kqueue() comparing to select().
WBR,
Alexey
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020213173010.E14414>