Date: Wed, 28 May 1997 10:32:32 -0400 (EDT) From: Christopher Sedore <cmsedore@mailbox.syr.edu> To: Ben Black <black@zen.cypher.net> Cc: Ruslan Shevchenko <rssh@cki.ipri.kiev.ua>, FreeBSD-Hackers@FreeBSD.ORG Subject: Re: async socket stuff Message-ID: <Pine.SOL.3.95.970528095615.11635A-100000@gamera.syr.edu> In-Reply-To: <Pine.LNX.3.91.970527234336.10761C-100000@zen.cypher.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 27 May 1997, Ben Black wrote: > > > > > > it is a kludge because it is in there for EXACTLY one purpose. you > > > couldn't use the facilities for any but transferring a file from one host > > > to another. it is only there to get around a general problem for a > > > specific application. that's a kludge. > > > > Gee, one of the most common applications for web servers, news servers, > > ftp servers. Probably a general problem that represents 60-80% of today's > > internet traffic. > > > > and since 99% of the world's computers are connected to the internet as > web, news, or ftp servers... No argument with that point, though it was my impression that this was a primary use for FreeBSD. > > > > It seems to me that a good test set for what facilities should be in the > > > > kernel might include: > > > > > > > > 1. can the kernel do it significantly more efficiently that the user? > > > > > > and if not, is that because the kernel needs to be reworked rather than > > > hacked up to solve a single problem. > > > > If I understood the site you referenced in your other post, your > > alternative is putting an IO interpreter into the kernel? Heck, why not > > just put java in kernel space and write all the I/O in java. Sounds like > > a smaller, faster kernel to me. Next you'll want interpreters to handle > > complex permissions manipulation, and page remapping, etc. > > > > 1) stick to what i actually said, and stop putting ridiculous words in my > mouth. I didn't put words in your mouth, though my sarcasm might have been a bit excessive. > 2) it is not a complete lagnuage interpreter, it is simply a compact > system for dynamically adding system call to the kernel by aggregating > existing system calls. contrast this with your approach which requires > adding special purpose code to the kernel for EVERY "common" sequence of > syscalls. I understood this, but its case is not made by the docs I read. It looks like a nice thing, but I'd like to see more discussion and an implementation before I support it generally. > > > > I prefer simple, reliable system calls to interpreters in the kernel any > > day. I also prefer solutions that are likely to exist soon rather than > > ones which may not exist for months or years. > > > > so you prefer solutions that involve no thoughtful design and testing. > this explains why you are pushing an obviously flawed solution > implemented by M$. The fact that something is simple does not mean that there was no thought involved. This is an example of something that is a simple addition that provides a nice performance boost for what I thought were important applications. If you're that concerned about bloat, perhaps it should be made an LKM. > > > > Not if that code can provide a non-trivial performance boost, *and* reduce > > application code sizes by roughly the same amount multiplied by the number > > of processes running using such facilities. > > > > 1K reduction in the size of the 100 or so web server running on a pretty > busy system at once...you saved a whole 100k! and all it cost was adding > yet more uneeded bloat to the kernel. the solution i offered has the > potential to boost performance for applications which aren't even around > yet. your solution becomes bloat as soon as application needs change. I'm generally willing to trade off 100k for 1k in the kernel if I get a nice performance boost. > > > > #2 is really a test question for a library rather than the kernel, but, > > > > where applicable, it should also be applied to kernel calls. > > > > > > > > Maybe not, but I don't think I could even get my code to run under FBSD > > > > without some non-trivial rearrangement because of facilities that don't > > > > exist or are costly to use (I'm not happy about the idea of checking 800 > > > > connections with an FD_ISSET() loop). > > > > > > > > > > so use 800 threads with a single fd each. > > > > And I do this with FreeBSD using what, the thread package that implements > > it as a select() in the background? So, in addition to reimplementing the > > I/O subsystem, you'd like me to add multithreading, too? > > > > it was an either or. and neither matters since i doubt you will actaully > be implementing any of this. My point was that the only thread packages available now (corrections welcome) just use non-blocking I/O and select() to simulate threads in userland. I'm not sure what you are refering to with regard to implementation. I already have an application that runs with 800 or so threads on NT, each managing a connection. NT is bugging me with a few problems I can't get Microsoft to address. I'm trying to figure out what I have to do to get a FreeBSD port with the same (or better) performance, or if I'm better off pushing on Microsoft to fix the problems I have with NT. I'm getting to the point where MS is a bit too unresponsive at the same time they're arrogant. > > > > > > and you have presented no evidence that it needs to be in the kernel. i > > > have suggested fbufs and various eros techniques while you have just > > > listed why *something* is needed. > > > > > > it *can* be done cleanly and quickly assuming proper kernel support. > > > adding an application specific system call to the kernel is just a bad idea. > > > > So far the kernel possibilities you have presented are not implemented for > > FreeBSD and probably won't be any time in the near future. Some of them > > have problems that don't have clear win/win outcomes (like I/O > > interpreters in the kernel). > > > > no, they are not yet implemented, but the source is there, the core team > is eager to make the system better...what's stopping you? Time. I probably could implement transmitfile for FreeBSD in a few weeks with everything else I have on my plate. I could not reengineer FreeBSD's I/O system in 10x that timeframe. > a small syscall aggregator in the kernel would take up perhaps 5-10k and > provide limitless applications. your solution solves a single current > problem and totally ignores the rest of the world. No disagreement here. I'd be happy to see your solution implemented, and I'd use it if it provided benefit to me. > > I can certainly imagine that some or all of these may provide more optimal > > I/O services. I don't think that you've demonstrated that the net gain of > > providing all this generalized service will result in greater overall > > efficiency than implementing my single, clean optimization, since I'd > > argue that the majority use will be the simple case, and you'll have made > > that case less efficient than it might be otherwise. > > > > of course you'd argue the majority would use the simple case: you are > only considering a SINGLE application. I'm willing to argue that your solution would, in a majority of cases, be used to implement mine. Mine would be faster and smaller for this case. I'll grant that this is a single application (or better said, single problem) solution. I happen to think it is a problem worthy of a solution. In the future, when a better I/O system (like the one you describe/reference) comes along, my call could be implemented in a library. > > I think transmitfile has a win/win outcome--it is small and simple to > > implement in the kernel, and provides code reduction and a non-trivial > > performance boost on the userland side. > > > > the code reduction is a non-issue and just because something *can* be > implemented in the kernel doesn't mean it *should* be implemented in the > kernel. if something is only supposed to help out a small fraction of > applications it belongs in userland. show me how TransmitFile() helps > emacs or gcc or xwin. i'd bet more people use those applications than > run their own web/ftp/news servers on their machine. Maybe so, but I don't see how it negatively affects them in any serious way. I do see how it can offer significant benefits to people running web/ftp/news servers where FreeBSD is trying to gain usage. My main point is that it cannot be implemented outside the kernel with what I would consider to be reasonable efficiency. If the async I/O stuff eliminates the buffer copies then some portion of my argument becomes moot. Even then I might still argue for its existence. -Chris
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.SOL.3.95.970528095615.11635A-100000>