Date: Fri, 16 Jul 1999 23:45:47 +0000 (GMT) From: Terry Lambert <tlambert@primenet.com> To: unknown@riverstyx.net (Tani Hosokawa) Cc: tlambert@primenet.com, davids@webmaster.com, chat@FreeBSD.ORG Subject: Re: Known MMAP() race conditions ... ? Message-ID: <199907162345.QAA18013@usr05.primenet.com> In-Reply-To: <Pine.LNX.4.10.9907151824410.12970-100000@avarice.riverstyx.net> from "Tani Hosokawa" at Jul 15, 99 06:42:54 pm
next in thread | previous in thread | raw e-mail | index | archive | help
> > > Thank you for spewing some geek-speak into an unrelated discussion. I
> > > welcome you to bring forth a new webserver that can duplicate everything
> > > that Apache can do with the same developer support that Apache has using a
> > > different model.
>
> > You, yourself, indicated that the process architecture was in flux;
> > if so, then it would be a good idea if it got away from threads
> > dependencies before it solidifies to something that won't run at
> > all on some platforms.
>
> This isn't a discussion about Apache; Apache is just an example of piece
> of software that requires threads. Your argument was that making
> FreeBSD's threads efficient was a waste of time because nobody needs
> threads. My argument is that "needing" threads is irrelevant, because
> people use threads.
No, my argument was against getting al het up about FreeBSD not
having kernel threads, or all excited that kernel threads are
somehow important for performance.
> I just thought of another high profile system that uses threads. MySQL.
> It's all threaded. Threads for MySQL are damn cool, 'coz it leaves a tiny
> memory footprint, especially when compared to other similar systems like
> PostGres that are process oriented.
Yes. And you'll notice that it runs just fine on FreeBSD. So I guess
FreeBSD does have working threads.
> Or, the aio_* functions that have been bandied around a bit. AFAIK, those
> are all implemented in their respective OS with kernel threads, are they
> not?
No. They are not. They are implemented with context structures,
which the kernel updates when the operation is completed as a
result of a wakeup() call -- in other words, the operation still
blocks the call, but it does not block the process that made the
call. See the man pages for aio_wait and aio_cancel for details of
the operation of the context structure.
> > I believe the Zeus server, which the Linux folks were complaining
> > was not being tested by Mindcraft (instead of Apache) supports
> > this model, since it requires the ability to do clustering, and
> > migrating a thread across machines is computationally hard.
>
> Zeus uses select, and has the problem that it blocks on I/O. It's not
> using the model you suggest.
Wait. Does it use "select", or does it "block on I/O". Select
does not return a 1 bit unless I/O will not block.
> I've heard this can be a serious issue on, say, serving files off NFS.
> Doubly so on FreeBSD, I suppose...
Wrong.
fcntl( fd, F_SETFL, O_NONBLOCK);
It will return an "EWOULDBLOCK" on a failed attempt to read or write,
which could be reattempted later.
> > > So, what we get back to is, what's one thing that's wrong with FreeBSD?
> > > Threads.
> >
> > That's actually "what's wrong with applications?". The closest you
> > can get to blaiming FreeBSD for this is "why can't FreeBSD run
> > these applications that rely on threads in an SMP scalable way?". I
> > could easily ask a similar question: "why can't FreeBSD run Microsoft
> > Office 2000?".
>
> No, it's "what's wrong with FreeBSD?".
Funny; mySQL, ACAP, and LDAP are all threaded, and they all run fine
on FreeBSD.
So does nay other pthreads application that conforms to POSIX 1003.1c.
Any application which doesn't run (e.g. the claims of one thread
blocking all other threads) is, by definition of POSIX 1003.1c, broken.
> > It matters if the Apache process architecture is in flux, and can
> > be affected in such a way as to not unneccessarily preclude having
> > good performance on dozens of platforms for the benefit of having
> > good performance on a single platform that happens to be the
> > current benchmark target.
>
> So, go talk to the Apache Group and tell them your opinions. A lot of
> them have pretty strong opinions about this.
>
> This isn't an Apache issue. This is a simple question relating to one
> aspect of FreeBSD, and whether or not you agree with that aspect has no
> relevance.
Apache will run on FreeBSD. THe Apache group uses FreeBSD to serve
their web pages. They're not going to break that.
> So? My point is, was, and will be, that FreeBSD doesn't *currently* have
> good threading. I never said that FreeBSD can't *get* good threading.
FreeBSD currently has good threading.
FreeBSD currently does not have SMP scalable threading. Neither does
Linux, given their lack of scheduler activations and CPU affinity code.
Note that all these scheduler problems "just go away", if you are not
so stupid as to implement your threads as kernel objects (Like Linux
and Solaris and SVR4).
It's not that "FreeBSD can't do blah blah blah", it's that "there has
been no compelling reason, so far, for FreeBSD to do blah blah blah"
(for the purposes of this discussion, "SMP scalable threads" is
"blah blah blah").
SMP scalability for user space threading in FreeBSD is a matter of
1) a small user space threads scheduler change to allow for
one stack per each anticipated CPU in user space.
2) the addition of one u_init32_t to the proc structure.
3) a straight forward addition of per CPU ready-to-run queues
and simple reference to the u_init32_t proc structure
element in schedcpu() that defaults to "0" (all bits not
set). Processes without the 0'th bit set would pick a
CPU based on which CPU has the lowest number of processess
in its ready to run queue.
4) a system call to allow more than one CPU to be asked to
visit user space (probably via a minor mod to rfork(2) to
add RFADDONECPU and RFADDALLCPU flags) that sets bits in
the u_init32_t, examines the global containing the number
of CPU's in the system, and adds an entry to the per CPU
ready-to-run queue of one (or more, if desired) CPU.
5) a minor change to execve(), exit1(), and fork1() to manage
the u_init32_t contents on a fork1() or execve() entry,
and to deschedule from other CPU's any scheduled entries
into user space on an execve() or exit1().
In other words, childs play.
Now that I've listed it out, it would take a day, tops, for most
programmers I have ever worked with to implement. Even the
Microsoft Windows hacks, if I point them at the right files and
tell them where to type "make".
Note 1: Complicated crap, like high and low watermarking, average
per CPU load calculation, and CPU affinity (other than maybe
locking a single process the first time you call "rfork(RFADDONECPU)")
could come later as people gave a damn about it, and could prove that
the oft asked question "how do I make a process stay on one CPU?" was
actually meaningful with regard to performance.
Note 2: In my experience, complicated crap generally doesn't result
in the performance imporvements you naievely think it will when you
first go at the problem (e.g. kernel threads).
Note 3: A better implementation would use a pointer to an allocated
bit array that gets allocated based on the number of CPU's, instead
of a u_int32_t. On a 32 bit box, this would be the same increase
in size of the proc structure, at the cost of an additional pointer
dereference.
> > > That's naive. Especially when you're talking about development
> > > on what's supposedly a "server OS".
> >
> > Server != threads.
>
> Server = runs server software, which means it is probably going to be
> doing one of a few tasks, mostly related to returning files or data. If
> it can't do that, then it's a bad server OS. And if a large number of
> software products require threads (and they do! they do! that's the
> point!) and FreeBSD can't do threads well, then it's a bad server OS.
>
> Ok, not necessarily bad -- just not suited to those particular tasks.
Server class software is generally software which is very aware of
performance issues.
Software which is very aware of issues doesn't use threads (or at
least doesn't use threads in version 2.0). Non-blocking I/O, or
async I/O, if they are data pumps and need to interleave I/O instead
of increasing CPU cycles thrown at the problem -- not likely, given
that a single Xeon, the WC CDROM FTP server, can saturate an OC3,
and tops out on I/O capability, not instructions -- is a better
use of the CPU, than spending time in a user space (or kernel, if
the threads are badly implemented) scheduler.
> > Even given the existance of the evil "configure" program I really
> > doubt the Apache people aren't (very) wise to this.
>
> Yes, they are. That's the point of the APR and MPM projects. However,
> threading's in there!
Uh, you are misreading this... read it again:
> > Even given the existance of the evil "configure" program I really
> > doubt the Apache people aren't (very) wise to this.
***** ******
It's a double negative indicating that I believe they are aware,
despite using "configure", of how to program server class software.
Terry Lambert
terry@lambert.org
---
Any opinions in this posting are my own and not those of my present
or previous employers.
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-chat" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199907162345.QAA18013>