From owner-freebsd-fs@FreeBSD.ORG Thu Mar 27 07:50:08 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5517ACB6 for ; Thu, 27 Mar 2014 07:50:08 +0000 (UTC) Received: from mail-wg0-x22f.google.com (mail-wg0-x22f.google.com [IPv6:2a00:1450:400c:c00::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E7792BBC for ; Thu, 27 Mar 2014 07:50:07 +0000 (UTC) Received: by mail-wg0-f47.google.com with SMTP id x12so2144708wgg.18 for ; Thu, 27 Mar 2014 00:50:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=mZKUshbA6/GVjNJe4IISX8T0rOLFGoSbdEnSBdPhkB8=; b=JR0NrSBCCjGjnoQPl31g0DOnyz2ifaMlztNmo0rKAdl4dSksYiK8OAyc976lAWzR37 FhZLlZA7f8Si6oMaqnbl4ScyyFeKN5wxfIDdsjVGuULTDNKpbIrecslIUpx8lEwveb6i 6KWzv8X0S2rW3miSJDGvablFuGwd7nv+zK3LUZvitoxLZMoDf/s4oEkZTqisxjLCkbCo lJgLCD/TUvAnKlX1U/0/nIb8Izr6ctjzTn7neP0mmyO4O1YsxQD3qhInWAqdufKCPEaW X9wETglnyK3yNug8n5Fwcc6ObjGnYpYqGEFPgbwl4l5KwizNnqwZcIjBINroTZn0ehbM qOgw== MIME-Version: 1.0 X-Received: by 10.180.97.72 with SMTP id dy8mr2623801wib.5.1395906606072; Thu, 27 Mar 2014 00:50:06 -0700 (PDT) Received: by 10.216.146.195 with HTTP; Thu, 27 Mar 2014 00:50:06 -0700 (PDT) Date: Thu, 27 Mar 2014 08:50:06 +0100 Message-ID: Subject: zfs l2arc warmup From: Joar Jegleim To: "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 07:50:08 -0000 Hi list ! I struggling to get a clear understanding of how the l2arc get warm ( zfs). It's a FreeBSD 9.2-RELEASE server. >From various forum I've come up with this which I have in my /boot/loader.conf # L2ARC tuning # Maximum number of bytes written to l2arc per feed # 8MB (actuall=vfs.zfs.l2arc_write_max*(1000 / vfs.zfs.l2arc_feed_min_ms)) # so 8MB every 200ms = 40MB/s vfs.zfs.l2arc_write_max=8388608 # Mostly only relevant at the first few hours after boot # write_boost, speed to fill l2arc until it is filled (after boot) # 70MB, same rule applys, multiply by 5 = 350MB/s vfs.zfs.l2arc_write_boost=73400320 # Not sure vfs.zfs.l2arc_headroom=2 # l2arc feeding period vfs.zfs.l2arc_feed_secs=1 # minimum l2arc feeding period vfs.zfs.l2arc_feed_min_ms=200 # control whether streaming data is cached or not vfs.zfs.l2arc_noprefetch=1 # control whether feed_min_ms is used or not vfs.zfs.l2arc_feed_again=1 # no read and write at the same time vfs.zfs.l2arc_norw=1 But what I really wonder is how does the l2arc get warmed up ? I'm thinking of 2 scenarios: a.: when arc is full, stuff that evict from arc is put over in l2arc, that means that files in the fs that are never accessed will never end up in l2arc, right ? b.: zfs run through fs in the background and fill up the l2arc for any file, regardless if it has been accessed or not ( this is the 'feature' I'd like ) I suspect scenario a is what really happens, and if so, how does people warmup the l2arc manually (?) I figured that if I rsync everything from the pool I want to be cache'ed, it will fill up the l2arc for me, which I'm doing right now. But it takes 3-4 days to rsync the whole pool . Is this how 'you' do it to warmup the l2arc, or am I missing something ? The thing is with this particular pool is that it serves somewhere between 20 -> 30 million jpegs for a website. The front page of the site will for every reload present a mosaic of about 36 jpegs, and the jpegs are completely randomly fetched from the pool. I don't know what jpegs will be fetched at any given time, so I'm installing about 2TB of l2arc ( the pool is about 1.6TB today) and I want the whole pool to be available from the l2arc . Any input on my 'rsync solution' to warmup the l2arc is much appreciated :) -- ---------------------- Joar Jegleim Homepage: http://cosmicb.no Linkedin: http://no.linkedin.com/in/joarjegleim fb: http://www.facebook.com/joar.jegleim AKA: CosmicB @Freenode ----------------------