From owner-freebsd-stable@FreeBSD.ORG Fri Oct 29 16:06:11 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 78325106564A; Fri, 29 Oct 2010 16:06:11 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 973988FC1B; Fri, 29 Oct 2010 16:06:08 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA07885; Fri, 29 Oct 2010 19:06:03 +0300 (EEST) (envelope-from avg@icyb.net.ua) Message-ID: <4CCAF0EB.4080001@icyb.net.ua> Date: Fri, 29 Oct 2010 19:06:03 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.11) Gecko/20101021 Lightning/1.0b2 Thunderbird/3.1.5 MIME-Version: 1.0 To: Artemiev Igor References: <3D1C350B94A44E5D95BAA1596D1EBF13@vosz.local> <20101029090417.GA17537@two.kliksys.ru> <4CCABFC2.3040701@icyb.net.ua> <4CCADD37.7000306@icyb.net.ua> <20101029152602.GA18613@two.kliksys.ru> In-Reply-To: <20101029152602.GA18613@two.kliksys.ru> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: 8.1-STABLE: zfs and sendfile: problem still exists X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Oct 2010 16:06:11 -0000 on 29/10/2010 18:26 Artemiev Igor said the following: > On Fri, Oct 29, 2010 at 05:41:59PM +0300, Andriy Gapon wrote: > >> What svn revision of FreeBSD source tree did you test? > > r213936. Revision seems a little old. > >> Ah, I think I see what's going on. >> Either sendfile should (have an option to) use VOP_GETPAGES to request data or ZFS >> mappedread should use vm_grab_page instead of vm_lookup_page for UIO_NOCOPY case. >> Currently ZFS would read a whole FS block into ARC, but populate only one page >> with data and for the rest it would just wastefully do uiomove(UIO_NOCOPY) from >> ARC data. >> So, e.g. zpool iostat would show that there are only few actual reads from a pool. >> The rest of the time must be spent churning over the data already in ARC and >> doing page-per-VOP_READ copies from it. > I can test it, but what allocflags? VM_ALLOC_RETRY|VM_ALLOC_NORMAL? Probably yes, but have to be careful there. First, do vm_page_grab only for UIO_NOCOPY case. Second, the first page is already "shared busy" after vm_page_io_start() call in kern_sendfile; so you might need VM_ALLOC_IGN_SBUSY for that page to avoid a deadlock. I think that it may be good to separate UIO_NOCOPY/sendfile case from mappedread into a function of its own. P.S. doing VOP_GETPAGES instead of vn_rdwr() in kern_sendfile() might be a better idea still. But there are some additional details to that, e.g. a mount/fs flag to tell which mechanism is preferred. Because, as I've been told, vn_rdwr() has better performance than VOP_GETPAGES. Although, I don't understand why it could/should be that way. -- Andriy Gapon