From owner-freebsd-fs@FreeBSD.ORG Thu Mar 27 10:21:07 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 19B4A53B for ; Thu, 27 Mar 2014 10:21:07 +0000 (UTC) Received: from mail-ee0-x231.google.com (mail-ee0-x231.google.com [IPv6:2a00:1450:4013:c00::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A01A8ACD for ; Thu, 27 Mar 2014 10:21:06 +0000 (UTC) Received: by mail-ee0-f49.google.com with SMTP id c41so2622725eek.22 for ; Thu, 27 Mar 2014 03:21:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=bCwJ7SxefoRzE7/D/BPZTNjPiPwxQpHx/sWDGedktks=; b=ADRXNy6FlNaMcoiXH6dW9IjMmG31g9GZr+c7VA6scHH0JcypL2mjQeOyb8PP4PS+Iz ARm/NeHJ8HpcpRVrf4gepcfojvyUzfwKenlQOLXKBPJ0eafU4vd+J30Duiqk5epDDhee kuuQOCvTFpeZ19IYyy2p4BWPzR6hq2O5PGVOQv6p1QCa8XU6LRjSkix7pVtDbiqnD47/ IzsChKEqpcITX10ZGqN9LhINmiByYG3UV6x5GoUJvWtyWRxkwe0RTKR+EL0bXbP11aib Ovm8oXGiutaAvB2QWCItZBpgIH1Ct1xt7JQGO5rfWWFgjJDR+AFM9mcnKZN4L279r3XK aOGg== X-Received: by 10.15.73.134 with SMTP id h6mr1008890eey.3.1395915663397; Thu, 27 Mar 2014 03:21:03 -0700 (PDT) Received: from [192.168.1.129] ([193.173.55.180]) by mx.google.com with ESMTPSA id l42sm3383924eew.19.2014.03.27.03.21.02 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 27 Mar 2014 03:21:02 -0700 (PDT) Message-ID: <5333FB8F.7010500@gmail.com> Date: Thu, 27 Mar 2014 11:21:03 +0100 From: Johan Hendriks User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Joar Jegleim Subject: Re: zfs l2arc warmup References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 10:21:07 -0000 Joar Jegleim schreef: > Hi list ! > > I struggling to get a clear understanding of how the l2arc get warm ( zfs). > It's a FreeBSD 9.2-RELEASE server. > > From various forum I've come up with this which I have in my /boot/loader.conf > # L2ARC tuning > # Maximum number of bytes written to l2arc per feed > # 8MB (actuall=vfs.zfs.l2arc_write_max*(1000 / vfs.zfs.l2arc_feed_min_ms)) > # so 8MB every 200ms = 40MB/s > vfs.zfs.l2arc_write_max=8388608 > # Mostly only relevant at the first few hours after boot > # write_boost, speed to fill l2arc until it is filled (after boot) > # 70MB, same rule applys, multiply by 5 = 350MB/s > vfs.zfs.l2arc_write_boost=73400320 > # Not sure > vfs.zfs.l2arc_headroom=2 > # l2arc feeding period > vfs.zfs.l2arc_feed_secs=1 > # minimum l2arc feeding period > vfs.zfs.l2arc_feed_min_ms=200 > # control whether streaming data is cached or not > vfs.zfs.l2arc_noprefetch=1 > # control whether feed_min_ms is used or not > vfs.zfs.l2arc_feed_again=1 > # no read and write at the same time > vfs.zfs.l2arc_norw=1 > > But what I really wonder is how does the l2arc get warmed up ? > I'm thinking of 2 scenarios: > > a.: when arc is full, stuff that evict from arc is put over in l2arc, > that means that files in the fs that are never accessed will never end > up in l2arc, right ? > > b.: zfs run through fs in the background and fill up the l2arc for any > file, regardless if it has been accessed or not ( this is the > 'feature' I'd like ) > > I suspect scenario a is what really happens, and if so, how does > people warmup the l2arc manually (?) > I figured that if I rsync everything from the pool I want to be > cache'ed, it will fill up the l2arc for me, which I'm doing right now. > But it takes 3-4 days to rsync the whole pool . > > Is this how 'you' do it to warmup the l2arc, or am I missing something ? > > The thing is with this particular pool is that it serves somewhere > between 20 -> 30 million jpegs for a website. The front page of the > site will for every reload present a mosaic of about 36 jpegs, and the > jpegs are completely randomly fetched from the pool. > I don't know what jpegs will be fetched at any given time, so I'm > installing about 2TB of l2arc ( the pool is about 1.6TB today) and I > want the whole pool to be available from the l2arc . > > > Any input on my 'rsync solution' to warmup the l2arc is much appreciated :) > > A nice blog about the L2ARC https://blogs.oracle.com/brendan/entry/test https://blogs.oracle.com/brendan/entry/l2arc_screenshots regards Johan