Date: Tue, 4 Mar 2003 18:27:42 -0500 (EST) From: Andrew Gallatin <gallatin@cs.duke.edu> To: arch@freebsd.org Cc: Sean Chittenden <sean@chittenden.org> Subject: Re: Should sendfile() to return ENOBUFS? Message-ID: <15973.13934.694598.353417@grasshopper.cs.duke.edu> In-Reply-To: <3E64FEA0.CCA21C7@imimic.com> References: <3E64FEA0.CCA21C7@imimic.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Alan L. Cox writes: > Sean, > > The current sf_buf implementation has a simple problem that could > account for your frequent blocking. Let me describe an extreme example > that will make it clear. Suppose you have a web server that delivers > nothing but a single file of 8 pages, or 32K bytes of data, to its > clients. Here's the punchline: If you had 1,000 concurrent requests, > you could wind up allocating 8,000 sf_bufs. Given that the main purpose > of the sf_buf is simply to provide an in-kernel virtual address for the > page, one sf_buf per page should suffice. Sf_bufs are already reference > counted. So, the principle change would be to add a directory data > structure that could answer the question "Does this page already have an > allocated sf_buf?" In a reply I previously sent privately to Alan, I suggested: One off-the-cuff idea would be to trade the u_int cow field of a vm_page for a struct sf_buf *sf_buf ptr, and to move the cow field into the sf_buf. That way, the sendfile and zero-copy code could find the relevant sfbuf without doing any additional hashing beyond what they needed to do to find the page. If page->sf_buf == NULL, then an sf buf is alloc'ed off the list, and page->sf_buf = new_sfbuf. Otherwise, a refcnt is incremented. The vm_fault() code would change so that it first checked for a non-null sf buf, then checked the cow count in the sf buf. This increases the size of a vm_page of 4 bytes on a 64-bit platform (or maybe 8, depending on the size of a vm_page), but should not affect the 32-bit platforms. There'd be a 4-byte size increase per sf_buf, but the decrease in the number of sf_buf's in flight should more than make up for the bloat. Alan suggested that once this was done: alc> the next step would be to manage sf_buf's as a sort alc> of "mapping cache". This could reduce the number of TLB shootdowns on alc> SMPs; and on 64-bit architectures we should be using the "mapping of alc> all RAM". Unfortunately, I don't have any time to implement this, nor does Alan. Is there any interest in this idea? Anbody like it enough to implement it? Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?15973.13934.694598.353417>