Date: Thu, 27 Mar 2014 09:26:20 -0500 (CDT) From: Bob Friesenhahn <bfriesen@simple.dallas.tx.us> To: Joar Jegleim <joar.jegleim@gmail.com> Cc: "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org> Subject: Re: zfs l2arc warmup Message-ID: <alpine.GSO.2.01.1403270904280.1735@freddy.simplesystems.org> In-Reply-To: <CAFfb-hpi20062%2BHCrSVhey1hVk9TAcOZAWgHSAP93RSov3sx4A@mail.gmail.com> References: <CAFfb-hpi20062%2BHCrSVhey1hVk9TAcOZAWgHSAP93RSov3sx4A@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 27 Mar 2014, Joar Jegleim wrote: > Is this how 'you' do it to warmup the l2arc, or am I missing something ? > > The thing is with this particular pool is that it serves somewhere > between 20 -> 30 million jpegs for a website. The front page of the > site will for every reload present a mosaic of about 36 jpegs, and the > jpegs are completely randomly fetched from the pool. > I don't know what jpegs will be fetched at any given time, so I'm > installing about 2TB of l2arc ( the pool is about 1.6TB today) and I > want the whole pool to be available from the l2arc . Your usage pattern is the opposite of what the ARC is supposed to do. The ARC is supposed to keep most-often accessed data in memory (or retired to L2ARC) based on access patterns. It does not seem necessary for your mosaic to be truely random across 20 -> 30 million jpegs. Random across 1000 jpegs which are circulated in time would produce a similar effect. The application building your web page mosiac can manage which files will be included in the mosaic and achieve the same effect as a huge cache by always building the mosiac from a known subset of files. The 1000 jpegs used for the mosaics can be cycled over time from a random selection, with old ones being removed. This approach assures that in-memory caching is effective since the same files will be requested many times by many clients. Changing the problem from an OS-oriented one to an application-oriented one (better algorithm) gives you more control and better efficiency. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.GSO.2.01.1403270904280.1735>