From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 17:31:01 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DD5811065670 for ; Fri, 7 Jan 2011 17:31:01 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 62ACF8FC0C for ; Fri, 7 Jan 2011 17:31:01 +0000 (UTC) Received: by bwz12 with SMTP id 12so10185057bwz.13 for ; Fri, 07 Jan 2011 09:31:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=1Yi5N3hvDSCXYSQkVwz9eYR7fmiak1i4ZTI850WACJs=; b=QQwoRPKe9AUyXHX7OYlsbTDXDdORCTzEMcenySZHd4eTqUZA1KPKQDhseHGTTs3OHP vQ2aUY0PeEQ0niPj3E4AVTHKTyqgZpp/EOHheHpkgUMmEgoUBwpSe75rcZfRqrUpS6Si kvNvfoPBmpGPdzumD3zRMYCK7ZkKPO//E0uO4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=mxSSLAdlqxWTFLpIAJ8f5KqjimsyDDP+sUXjFoW1UZ+ubg8JqFxlLxHfVsHH/kwaM8 7tp23Rt7EHlsYbBB4VHM3MuPVUoig/zYzK/C36FwCpztZKh6ZUnrkApblzibkuBAdGck Cxp5Cle2mDV/qmMWdgch3Fl0Pxh6v0bfTrmyw= MIME-Version: 1.0 Received: by 10.204.122.65 with SMTP id k1mr1865293bkr.80.1294421460046; Fri, 07 Jan 2011 09:31:00 -0800 (PST) Sender: artemb@gmail.com Received: by 10.204.22.18 with HTTP; Fri, 7 Jan 2011 09:30:59 -0800 (PST) In-Reply-To: <20110107111635.GL28453@over-yonder.net> References: <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> <4D25BF91.7070304@my.gd> <4D25C760.4070703@digsys.bg> <20110107111635.GL28453@over-yonder.net> Date: Fri, 7 Jan 2011 09:30:59 -0800 X-Google-Sender-Auth: XQVsKOSToS0snEgteMPtVsWmQdE Message-ID: From: Artem Belevich To: "Matthew D. Fuller" Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-stable@freebsd.org, Daniel Kalchev Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 17:31:01 -0000 On Fri, Jan 7, 2011 at 3:16 AM, Matthew D. Fuller wrote: > On Thu, Jan 06, 2011 at 03:45:04PM +0200 I heard the voice of > Daniel Kalchev, and lo! it spake thus: >> >> You should also know that having large L2ARC requires that you also >> have larger ARC, because there are data pointers in the ARC that >> point to the L2ARC data. Someone will do good to the community to >> publish some reasonable estimates of the memory needs, so that >> people do not end up with large but unusable L2ARC setups. > > Estimates I've read in the past are that L2ARC consumes ARC space at > around 1-2%. Each record in L2ARC takes about 250 bytes in ARC. If I understand it correctly, not all records are 128K which is default record size on ZFS. If you end up with a lot of small records (for instance, if you have a lot of small files or due to a lot of synchronous writes or if record size is set to a lower value) then you could potentially end up with much higher ARC requirements. So, 1-2% seems to be a reasonable estimate assuming that ZFS deals with ~10K-20K records most of the time. If you mostly store large files your ratio would probably be much better. One way to get specific ratio for *your* pool would be to collect record size statistics from your pool using "zdb -L -b " and then calculate L2ARC:ARC ratio based on average record size. I'm not sure, though whether L2ARC stores records in compressed or uncompressed form. --Artem