Date: Thu, 27 Mar 2014 09:53:41 -0500 From: Karl Denninger <karl@denninger.net> To: freebsd-fs@freebsd.org Subject: Re: zfs l2arc warmup Message-ID: <53343B75.6090807@denninger.net> In-Reply-To: <alpine.GSO.2.01.1403270904280.1735@freddy.simplesystems.org> References: <CAFfb-hpi20062%2BHCrSVhey1hVk9TAcOZAWgHSAP93RSov3sx4A@mail.gmail.com> <alpine.GSO.2.01.1403270904280.1735@freddy.simplesystems.org>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] On 3/27/2014 9:26 AM, Bob Friesenhahn wrote: > On Thu, 27 Mar 2014, Joar Jegleim wrote: >> Is this how 'you' do it to warmup the l2arc, or am I missing something ? >> >> The thing is with this particular pool is that it serves somewhere >> between 20 -> 30 million jpegs for a website. The front page of the >> site will for every reload present a mosaic of about 36 jpegs, and the >> jpegs are completely randomly fetched from the pool. >> I don't know what jpegs will be fetched at any given time, so I'm >> installing about 2TB of l2arc ( the pool is about 1.6TB today) and I >> want the whole pool to be available from the l2arc . > > Your usage pattern is the opposite of what the ARC is supposed to do. > The ARC is supposed to keep most-often accessed data in memory (or > retired to L2ARC) based on access patterns. > > It does not seem necessary for your mosaic to be truely random across > 20 -> 30 million jpegs. Random across 1000 jpegs which are circulated > in time would produce a similar effect. > > The application building your web page mosiac can manage which files > will be included in the mosaic and achieve the same effect as a huge > cache by always building the mosiac from a known subset of files. The > 1000 jpegs used for the mosaics can be cycled over time from a random > selection, with old ones being removed. This approach assures that > in-memory caching is effective since the same files will be requested > many times by many clients. > > Changing the problem from an OS-oriented one to an > application-oriented one (better algorithm) gives you more control and > better efficiency. > > Bob That's true, but the other option if he really does want it to be random across the entire thing, given the size (which is not outrageous) and that the resource is going to be read-nearly-only, is to put them on SSDs and ignore the L2ARC entirely. These days that's not a terribly expensive answer as with a read-mostly-always environment you're not going to run into a rewrite life-cycle problem on rationally-priced SSDs (e.g. Intel 3500s). Now an ARC cache miss is not all *that* material since there is no seek or rotational latency penalty. HOWEVER, with that said it's still expensive compared against rotating rust for bulk storage, and as Bob noted a pre-select middleware process would result in no need for a L2ARC and allow the use of a pool with much-smaller SSDs for the actual online retrieval function. Whether the coding time and expense is a good trade against the lower hardware cost to do it the "raw" way is a fair question. -- -- Karl karl@denninger.net [-- Attachment #2 --] 0 *H 01 0 + 0 *H O0K030 *H 01 0 UUS10UFlorida10U Niceville10U Cuda Systems LLC1 0UCuda Systems LLC CA1/0- *H customer-service@cudasystems.net0 130824190344Z 180823190344Z0[1 0 UUS10UFlorida10UKarl Denninger1!0 *H karl@denninger.net0"0 *H 0 bi՞] MNԿawx?`)'ҴcWgR@BlWh+ u}ApdCF JVй~FOL }EW^bچYp3K& ׂ(R lxڝ.xz?6 &nsJ +1v9 v/( kqĪp[vjcK%fϻe?iq]z lyzFO'ppdX//Lw(3 JI A*S#՟H[f|CGqJKoo y.oEuOw$ /섀$삻J9b|A P~8]D1YI<"""Y^T2iQ2 b yH)] Ƶ0y$ _N 6XqMC 9 XgώjGTP"#nˋ"Bk1 00 U 0 0 `HB0 U 0, `HB OpenSSL Generated Certificate0 U |8 ˴d[20 U #0]Af4 U3x&^"408 `HB+)https://cudasystems.net:11443/revoked.crl0 *H gBwH] j\x`( &gW32"Uf^. ^Iϱ k!DQA g{(w /)\N'[ oRW @C HO>)XrTNɘ!u`xt5(=f\-l3<@C6mnhv##1ŃbH͍_Nq aʷ?rk$^9TIa!kh,D -ct1 0001 0 UUS10UFlorida10U Niceville10U Cuda Systems LLC1 0UCuda Systems LLC CA1/0- *H customer-service@cudasystems.net0 + ;0 *H 1 *H 0 *H 1 140327145341Z0# *H 1;0Sȫ =T0l *H 1_0]0 `He*0 `He0 *H 0*H 0 *H @0+0 *H (0 +71001 0 UUS10UFlorida10U Niceville10U Cuda Systems LLC1 0UCuda Systems LLC CA1/0- *H customer-service@cudasystems.net0 *H 101 0 UUS10UFlorida10U Niceville10U Cuda Systems LLC1 0UCuda Systems LLC CA1/0- *H customer-service@cudasystems.net0 *H (Zc&%tΊh*a1i5LFH.kiZ*-.LڰuȀN/jpW,NTr,Nv-CN p.5r=~iX`9b.LK> e1{~D#o&%0 [p) dql5e #_ #,Ψ crjiOC l~ D E/s׆?N!NPE@9[A-IE8i)u\vv%`#leEǝǠxp:+{<E fBWv9W!' փH <P]P&@ * U=b"};] ps]L$쏶O`'Vk3Mp. Őx_J_#o!Q/-)?͜ #O ުxңE )p. ˸]M\J&n L ɐshelp
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?53343B75.6090807>