Date: Tue, 4 Mar 2003 13:51:18 -0800 From: Sean Chittenden <sean@chittenden.org> To: "Alan L. Cox" <alc@imimic.com> Cc: arch@freebsd.org Subject: Re: Should sendfile() to return ENOBUFS? Message-ID: <20030304215118.GJ79234@perrin.int.nxad.com> In-Reply-To: <3E64FEA0.CCA21C7@imimic.com> References: <3E64FEA0.CCA21C7@imimic.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--D+UG5SQJKkIYNVx0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable > The current sf_buf implementation has a simple problem that could > account for your frequent blocking. Let me describe an extreme > example that will make it clear. Suppose you have a web server that > delivers nothing but a single file of 8 pages, or 32K bytes of data, > to its clients. Here's the punchline: If you had 1,000 concurrent > requests, you could wind up allocating 8,000 sf_bufs. Given that > the main purpose of the sf_buf is simply to provide an in-kernel > virtual address for the page, one sf_buf per page should suffice. > Sf_bufs are already reference counted. So, the principle change > would be to add a directory data structure that could answer the > question "Does this page already have an allocated sf_buf?" This is an excellent suggestion and one that I hadn't even thought of, thank you. You're right in assuming that I'm sending out only a few hundred files per server to many thousands of clients, so this would be ideal in terms of performance. The problem that I can see with this is, "what happens when a file changes on disk?" Some how the page of data needs to be flushed and re-read. For files that are constantly in transit, their ref count would never hit zero so the data sent would never change (in theory, or it would mix/match pages from the new and old file: a problem not encountered with the current sendfile() implementation). Shutting down the server and waiting for the buffers to clear isn't a valid option in my book or with the possibility of out of sequence pages makes sendfile() something of a data integrity liability. Since the various pages types are all aligned, caching of sf_buf's along with the above directory structure would be quite a bit more efficient for my case, but possibly too efficient. Is there a mechanism for reseting highly used, but changed file pages? There could very likely is a way of doing this, but I'm not familiar with it. -sc --=20 Sean Chittenden --D+UG5SQJKkIYNVx0 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Comment: Sean Chittenden <sean@chittenden.org> iD8DBQE+ZR/W3ZnjH7yEs0ERAqlxAKDLFlX/YJ6MzAkV6yA7v7ixO8NgggCg59fT NwRxgLSCeAy12P/8LbmnXYo= =h3y7 -----END PGP SIGNATURE----- --D+UG5SQJKkIYNVx0-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030304215118.GJ79234>