Date: Tue, 12 Aug 2014 20:59:57 +0000 From: "Pokala, Ravi" <rpokala@panasas.com> To: John Baldwin <jhb@freebsd.org>, "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org> Subject: Re: IO chunking Message-ID: <D00FC785.11B8D3%rpokala@panasas.com> In-Reply-To: <201408121257.18814.jhb@freebsd.org> References: <D00F08A8.11B800%rpokala@panasas.com> <201408121257.18814.jhb@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
-----Original Message----- From: John Baldwin <jhb@freebsd.org> Date: Tuesday, August 12, 2014 at 9:57 AM To: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org> Cc: Ravi Pokala <rpokala@panasas.com> Subject: Re: IO chunking >On Tuesday, August 12, 2014 2:56:26 am Pokala, Ravi wrote: >> Hi folks, >>=20 >> I'm doing moderately-large block IO (16KB - 1MB) directly against drive >> devices (i.e. /dev/adaX), and I see that `iostat -d adaX' reports a >> transaction size of at most 128KB. I believe this is because >>transactions >> are limited to at most MAXPHYS bytes (128KB), and requests larger than >> that are broken into smaller chunks; is that correct? If so, where does >> that chunking happen? In low-level GEOM code (geom_io.c, geom_dev.c)? In >> CAM? In the drive device driver? In VFS? > >Note that you can increase MAXPHYS (though you will want to ensure your >storage controller drivers correctly report their maximum supported size >and don't just hardcode MAXPHYS). Yeah. For a semi-related issue, we're planning on upping MAXPHYS to 256KB. That still artificially limits the transaction size, but it's a bit better. >The limit appears to be throughout the stack, though largely enforced >at the top (e.g. in physio() before entering GEOM or the b_pages[] array >in struct buf). Looking... sys/kern/kern_physio.c: 101 /* Don't exceed drivers iosize limit */ 102 if (bp->b_bcount > dev->si_iosize_max) 103 bp->b_bcount =3D dev->si_iosize_max; Yeah, that's probably what it is. It looks like geom_dev.c::g_dev_taste() sets si_iosize_max to MAXPHYS, and nothing in the ATA layer changes it. >Certainly I've seen folks run with MAXPHYS of 512k, but check your >drivers. Yeah. We've been running w/ 256KB in another branch for a while, so it looks like everything we use is okay. Thanks, Ravi >--=20 >John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?D00FC785.11B8D3%rpokala>