Date: Thu, 16 Nov 2000 14:35:46 -0800 From: Bakul Shah <bakul@bitblocks.com> To: "Nicolai Petri" <nicolai@petri.cc> Cc: freebsd-hackers@freebsd.org Subject: Re: Multithreaded tcp-server or non-blocking ? Message-ID: <200011162235.RAA27714@thunderer.cnchost.com> In-Reply-To: Your message of "Thu, 16 Nov 2000 11:38:14 %2B0100." <021501c04fb9$574f9030$6732a8c0@atomic.dk>
next in thread | previous in thread | raw e-mail | index | archive | help
> What's the best approach for a simple web-server(never more the 10 clients)
> ? Is it using pthread and a thread per connection . Or to make a
> non-blocking single thread server. Can people show me some simple examples
> of the 2 techniques ?
>
> And what's the pro's and con's for the 2 methods ???
The simplest solution is to just fork a new process on
accepting a new connection. A sample implementation is
included at the end. This solution doesn't scale too well as
a process is quite heavy weight and uses much more resources
than the other two solutions. One variation avoids the cost
of fork/exit per connection. The connection accepting
process passes the new file descriptor in a message to a free
server process. The acceptor creates server threads as
needed and the server threads can be made to exit when they
are idle for a while.
The thread per connection solution is also quite simple and
significantly more efficient that process per connection
since a thread context switch is a lot cheaper than a process
context switch. But you need to deal with concurrency issues
if you have to share a bunch of state and your threads can be
preempted. With this and the next solution you can also run
into file descriptor limits. I don't have an example I can
show you.
A non-blocking single thread server is likely to be more
portable than the pthread solution.
See
http://www.bitblocks.com/src/select-test.c
for an example.
The basic problem here is what to do if deep down in some
function you need to wait for some response from a client.
As an example, consider the case where function A is called
when a new connection comes in. A calls B and B calls C and
C wants to block. You have no choice but to implement this
as a state machine and break functions B, C in B1, B2, C1 and
C2 and carry around appropriate state. So you have to take
something like
A(int fd) { // called on a new connection
...
B(fd);
...
}
B(int fd) {
...
C(fd);
...
}
C(int fd) {
...
read(fd, buf, len);
...
}
and change it to
A(int fd, Context* c) { // called on a new connection
switch (c->state) {
case init:
...
B1(fd, c);
break;
case more:
B2(fd, c);
...
}
}
B1(int fd, Context* c) {
...
C1(fd, c);
}
B2(int fd, Context* c) {
C2(fd, c);
...
}
C1(int fd, Context* c) {
...
c->state = more;
}
C2(int fd, Context* c) {
len = read(fd, bufptr, len);
if (read enough)
c->state = init;
else
update bufptr
...
}
If you have a complex interaction with the client, the number
of states can explode. Worse, such interactions may get
added as the application evolves so you are forced to do a
conversion like the one shown above. In effect here you are
implementing threading in a application specific way and the
c->state variable has a role analogous to a program counter
of a thread.
At the same time concurrency issues are easier to deal with
than the pthreads case since this is cooperative
multithreading.
In my (somewhat dated) experience this solution was a bit
more efficient than the pthreads version but significantly
more complex -- this situation may have changed in pthreads'
favor. Also, typically here you require much less memory to
store a connection's state than what you spend with a pthread
-- an issue if you have thousands of connections. If only a
few are active at a time this single thread select loop
solution is a perfectly usable solution and uses the least
resources.
You didn't ask for this but note that for handling a very
large number of clients (thousands to millions) you have to
use a combination of techniques to deal with various resource
limits. For example, you can push timeconsuming diskio
requests to separate threads or ever separate machines.
You can use multiple processes to deal with file descriptor
limits and so on.
-- bakul
/* A simple fork-a-server-on-accepting-a-connection solution */
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <sys/wait.h>
#include <signal.h>
void
reaper(int ignore)
{
int status;
wait(&status);
}
void
process_request(int fd)
{
char buffer[11];
int len = sprintf(buffer, "%d\n", getpid());
write(fd, buffer, len);
}
int
main(int argc, char** argv)
{
u_short port = argc > 1? strtoul(argv[1], 0, 0) : 1234;
int s = socket(PF_INET, SOCK_STREAM, 0);
int one = 1;
struct sockaddr_in addr;
if (s < 0) {
perror("socket");
exit(1);
}
bzero(&addr, sizeof addr);
addr.sin_len = sizeof addr;
addr.sin_family = AF_INET;
addr.sin_port = htons(port);
if (bind(s, (struct sockaddr*)&addr, sizeof addr) < 0) {
perror("bind");
exit(1);
}
if (setsockopt(s, SOL_SOCKET, SO_REUSEPORT, &one, sizeof one) < 0) {
perror("setsockopt");
exit(1);
}
if (listen(s, 5) < 0) {
perror("listen");
exit(1);
}
if (signal(SIGCHLD, reaper) < 0) {
perror("signal");
exit(1);
}
for (;;) {
int len = sizeof addr;
int fd = accept(s, (struct sockaddr*)&addr, &len);
int pid;
if (fd < 0)
continue;
pid = fork();
if (pid) { /*child */
process_request(fd);
exit(0);
}
/* parent */
close(fd);
if (pid < 0) {
perror("fork");
continue;
}
}
}
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200011162235.RAA27714>