Date: Sun, 29 Jan 2012 16:08:10 +0100 From: Tijl Coosemans <tijl@coosemans.org> To: freebsd-current@freebsd.org Subject: Re: posix_fadvise noreuse disables file caching Message-ID: <201201291608.16741.tijl@coosemans.org> In-Reply-To: <201201251129.22368.jhb@freebsd.org> References: <201201191739.48327.tijl@coosemans.org> <201201201412.13269.jhb@freebsd.org> <201201251129.22368.jhb@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--nextPart4901043.djbVi398T5 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: quoted-printable On Wednesday 25 January 2012 17:29:22 John Baldwin wrote: > On Friday, January 20, 2012 2:12:13 pm John Baldwin wrote: >> On Thursday, January 19, 2012 11:39:42 am Tijl Coosemans wrote: >>> I recently noticed that multimedia/vlc generates a lot of disk IO when >>> playing media files. For instance, when playing a 320kbps mp3 gstat >>> reports about 1250kBps (=3D10000kbps). That's quite a lot of overhead. >>>=20 >>> It turns out that vlc sets POSIX_FADV_NOREUSE on the entire file and >>> reads in chunks of 1028 bytes. FreeBSD implements NOREUSE as if >>> O_DIRECT was specified during open(2), i.e. it disables all caching. >>> That means every 1028 byte read turns into a 32KiB read (new default >>> block size in 9.0) which explains the above numbers. >>>=20 >>> I've copied the relevant vlc code below (modules/access/file.c:Open()). >>> It's interesting to see that on OSX it sets F_NOCACHE which disables >>> caching too, but combined with F_RDAHEAD there's still read-ahead >>> caching. >>>=20 >>> I don't think POSIX intended for NOREUSE to mean O_DIRECT. It should >>> still cache data (and even do read-ahead if F_RDAHEAD is specified), >>> and once data is fetched from the cache, it can be marked WONTNEED. >>=20 >> POSIX doesn't specify O_DIRECT, so it's not clear what it asks for. >>=20 >>> Is it possible to implement it this way, or if not to just ignore >>> the NOREUSE hint for now? >>=20 >> I think it would be good to improve NOREUSE, though I had sort of >> assumed that applications using NOREUSE would do their own buffering >> and read full blocks. We could perhaps reimplement NOREUSE by doing >> the equivalent of POSIX_FADV_DONTNEED after each read to free buffers >> and pages after the data is copied out to userland. I also have an >> XXX about whether or not NOREUSE should still allow read-ahead as it >> isn't very clear what the right thing to do there is. HP-UX (IIRC) >> has an fadvise() that lets you specify multiple policies, so you >> could specify both NOREUSE and SEQUENTIAL for a single region to >> get read-ahead but still release memory once the data is read once. > > So I've came up with this untested patch. It uses > VOP_ADVISE(FADV_DONTNEED) after read(2) calls to a NOREUSE region, and > leaves read-ahead caching enabled for NOREUSE. FADV_DONTNEED doesn't > do any good really for writes (it only flushes clean buffers), so I've > left write(2) operations as using IO_DIRECT still. Does this sound > reasonable? I've not yet tested this at all: The patch drastically improves vlc, but there's still a tiny overhead. Without NOREUSE the disk is read in chunks of 128KiB (F_RDAHEAD buffer size). With NOREUSE there's an extra transfer of 32KiB (block size). --nextPart4901043.djbVi398T5 Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iF4EABEIAAYFAk8lYOAACgkQfoCS2CCgtivk2AD/RqvjPc4BSLqq/BzerszDDolM cwQBJoh2eMsxfpiJroMA/2pRubGDhtqL3nG7zIYfgm/2fcH2LYFm7W9ZrUCLxTgD =VmXF -----END PGP SIGNATURE----- --nextPart4901043.djbVi398T5--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201201291608.16741.tijl>