From owner-freebsd-hackers@FreeBSD.ORG Tue Aug 12 19:37:10 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C8D50DA6 for ; Tue, 12 Aug 2014 19:37:10 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A3F7A25CD for ; Tue, 12 Aug 2014 19:37:10 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 982AAB95B; Tue, 12 Aug 2014 15:37:09 -0400 (EDT) From: John Baldwin To: freebsd-hackers@freebsd.org Subject: Re: IO chunking Date: Tue, 12 Aug 2014 12:57:18 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201408121257.18814.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 12 Aug 2014 15:37:09 -0400 (EDT) Cc: "Pokala, Ravi" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Aug 2014 19:37:10 -0000 On Tuesday, August 12, 2014 2:56:26 am Pokala, Ravi wrote: > Hi folks, > > I'm doing moderately-large block IO (16KB - 1MB) directly against drive > devices (i.e. /dev/adaX), and I see that `iostat -d adaX' reports a > transaction size of at most 128KB. I believe this is because transactions > are limited to at most MAXPHYS bytes (128KB), and requests larger than > that are broken into smaller chunks; is that correct? If so, where does > that chunking happen? In low-level GEOM code (geom_io.c, geom_dev.c)? In > CAM? In the drive device driver? In VFS? Note that you can increase MAXPHYS (though you will want to ensure your storage controller drivers correctly report their maximum supported size and don't just hardcode MAXPHYS). The limit appears to be throughout the stack, though largely enforced at the top (e.g. in physio() before entering GEOM or the b_pages[] array in struct buf). Certainly I've seen folks run with MAXPHYS of 512k, but check your drivers. -- John Baldwin