Date: Sun, 04 Nov 2007 13:14:06 +0100 From: Kris Kennaway <kris@FreeBSD.org> To: freebsd-current@freebsd.org, Peter Schuller <peter.schuller@infidyne.com> Subject: Re: ZFS slowness (not using cache?) Message-ID: <472DB78E.5030103@FreeBSD.org> In-Reply-To: <20071104110521.GA12145@outcold.yadt.co.uk> References: <200711021208.25913.Thomas.Sparrevohn@btinternet.com> <20071103164231.GB23714@outcold.yadt.co.uk> <200711040948.25732.peter.schuller@infidyne.com> <20071104110521.GA12145@outcold.yadt.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
David Taylor wrote: > On Sun, 04 Nov 2007, Peter Schuller wrote: > >>> For example, pkg_delete seems to be _extremely_ slow and ^T reports that >>> it is stuck waiting on zfs:(&zio->io_cv) for an unreasonable (IMO) amount >>> of time. >> FWIW, I have seen pkg_install (and possibly other pkg_* tools) being extremely >> slow seemingly as a result of the active set of files it touches exceededing >> the amount cached. In particular I had this problem after converting to ZFS, >> but prior to switching to amd64 and more RAM. >> >> It would sit and churn on disk I/O forever, entirely seek bound. Tracing the >> processes showed it traversing the package database over and over (presumably >> recursively following dependencies or some such). So the same files were >> touched any number of times. As a result, with too little cached, runtime >> exploded (it took hours and hours upgrading my desktop using *binary* >> pre-built packages because the larger packages with a lot of dependencies >> would take forever to install and delete). > > Hmm. That prompted me to have a look at the arcstats, and I'm now rather > confused. It seems to have plenty of cache free, but not actually using > it properly. > > I'm running i386 for now (on an amd64 motherboard), with 4GB of RAM > (~3.5Gb usable) and the following settings in loader.conf: > > geom_stripe_load="YES" > geom_label_load="YES" > zfs_load="YES" > snd_driver_load="YES" > nvidia_load="YES" > > hw.ata.atapi_dma="1" > kern.maxfiles="25000" > kern.ipc.shmmax=67108864 > kern.ipc.shmall=32768 > > kern.maxdsiz="900M" > vm.kmem_size=1450M > vfs.zfs.arc_max=500M > vfs.root.mountfrom="zfs:tank/i386_root" > > This results in (sysctl): > > vm.kmem_size_scale: 3 > vm.kmem_size_max: 335544320 # I have just noticed this is about 300MB, > # far lower than vm.kmem_size. Is that a problem? No but you can't set kmem to 1500MB without first increasing the kernel address space (1GB on i386) to some suitably larger value via KVA_PAGES. > vm.kmem_size_min: 0 > vm.kmem_size: 1520435200 > > vfs.zfs.arc_min: 47513600 > vfs.zfs.arc_max: 524288000 > kstat.zfs.misc.arcstats.c_min: 47513600 > kstat.zfs.misc.arcstats.c_max: 524288000 > kstat.zfs.misc.arcstats.size: 86967808 > > Regardless of what I do, I can't seem to get arcstats.size to exceed 100MB. > It initially goes up with disk usage, but then starts to drop again, causing > it to hold steady around 70-90MB. Something seems to be agressively pushing > data out of the cache, despite it being <20% full, which seems rather fishy. > Does anyone know what's going on? Try increasing kern.maxvnodes. Kris >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?472DB78E.5030103>