From owner-freebsd-fs@FreeBSD.ORG Mon Oct 5 18:28:34 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 003A9106566B; Mon, 5 Oct 2009 18:28:33 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-yx0-f184.google.com (mail-yx0-f184.google.com [209.85.210.184]) by mx1.freebsd.org (Postfix) with ESMTP id 6626F8FC0C; Mon, 5 Oct 2009 18:28:33 +0000 (UTC) Received: by yxe14 with SMTP id 14so4183840yxe.7 for ; Mon, 05 Oct 2009 11:28:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=+jMFWEDWcEi2OyS0hHNqSQvDUrlE15u6cfaTFptYegk=; b=eOFkx4djNY8jkFfdijVm5+8i4t91zMyc2FSfX5zbOw9CB8Cefk7rEmCDc0NJuyEflT Fp09A9ovCLAZjBRGbgOc3j8N3WbneoV2FyDbPB7tQWnr6OXSU5+vug6TVmR5Cp0i7wt8 sHynWA8RwfK15WWa9ySbUANwtm0BXyhZ/aplk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=C26FMtaIFiYAOs8AY+EKc1uVNmT1rDUpaNdYAfb6OFs+pgp9mZYGVD+9jnRDE55EIN nY5zo8apy91wDTZD8HRVmleqgNBs577gJJPq14ukZxWFSgK0STMiTQWCgi+F8WGWfqXz 3g/GR5m/ph97StH9pUFCh1hkwVmh9aEiFU5NU= MIME-Version: 1.0 Sender: artemb@gmail.com Received: by 10.90.16.34 with SMTP id 34mr157806agp.47.1254767312445; Mon, 05 Oct 2009 11:28:32 -0700 (PDT) In-Reply-To: References: <4AC1E540.9070001@fsn.hu> <4AC5B2C7.2000200@fsn.hu> <20091002184526.GA1660@garage.freebsd.pl> <20091003000909.GD1660@garage.freebsd.pl> Date: Mon, 5 Oct 2009 11:28:32 -0700 X-Google-Sender-Auth: e3489b7891f75ece Message-ID: From: Artem Belevich To: Pawel Jakub Dawidek Content-Type: multipart/mixed; boundary=001636283b62ef22e804753447f5 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: ARC size constantly shrinks, then ZFS slows down extremely X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Oct 2009 18:28:34 -0000 --001636283b62ef22e804753447f5 Content-Type: text/plain; charset=ISO-8859-1 I left my box running over the weekend (vm.kmem_size=8G) in a loop doing a build and then deleting build results. Each loop cycle takes about 5 hours. All in all it touches about 2GB of sources and produces about 30GB of object files and stuff. This morning ARC size is around 2.5G. Now and then it dips down to 1G. I've attached graph with memory stats and ARC size. --Artem > =============================================================== > Now, the same experiment, with vm.kmem_size=8G > vm.kmem_size: 8589934592 > vfs.zfs.arc_min: 939524096 > vfs.zfs.arc_max: 7516192768 > > ARC grows to 6.2G: > Mem: 47M Active, 13M Inact, 7376M Wired, 31M Buf, 473M Free > > Then it quickly shrinks to 4.6G and grows to 6.2G again, shrinks again, etc.. > > What's different from the previous case is that after a while ZFS > adjusts target size (kstat.zfs.misc.arcstats.c) down to ~5.8G and > after that ZFS size oscillates between 4.2G and 5.6G. Another > observation -- ARC shrinking happens when system is left with ~512M of > free memory. Yet another observation is that even with ARC peak of > ~5.8G, system has about 7.5G wired. Where did almost 2G of difference > go? Fragmentation? > > I've tried both experiments with and without L2ARC -- behavior seems > to be the same. > > --Artem > --001636283b62ef22e804753447f5--