From owner-freebsd-fs@FreeBSD.ORG Tue May 31 19:21:34 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C44D4106566C; Tue, 31 May 2011 19:21:34 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 6651C8FC13; Tue, 31 May 2011 19:21:34 +0000 (UTC) Received: by yxl31 with SMTP id 31so2837763yxl.13 for ; Tue, 31 May 2011 12:21:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=KgF7aG9NGFf0CLgWBH1Z4bdlgtNcRZThbICQE2AgFzw=; b=Pz/H9B27WONxs7j+mRaM9DCF5FGtsDpSy9MQatttbO9H8pEQELz0DBi8oyamaEA3ba u1YS29FaSohb+mlBq1jHKZAVKoyQtF4tKftLWa+mPIJq/5Et22AKbad1e4ufapvoXWqW 6RVek35Q9qF927wsQPmAbCXvUXhsNYJQ0PfYk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=YSdAa/RdVgqwJv1jcdwNRYXaBoeok4RaIG5YAGkdon5+nfJyQvAQsgAsix293G5B1S 90lrSSwe7rK1HtBOUJzp9Wxo5Ova0U+b8QukgZ+YijeXmtREYuu9rxyzz1z/aoSr4pC4 nH6MmcoJ8O2QepC4V7UQNCZ/FQhjJvlOOQ98c= MIME-Version: 1.0 Received: by 10.236.187.100 with SMTP id x64mr8069682yhm.317.1306869693497; Tue, 31 May 2011 12:21:33 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.236.208.70 with HTTP; Tue, 31 May 2011 12:21:33 -0700 (PDT) In-Reply-To: References: <0EFD28CD-F2E9-4AE2-B927-1D327EC99DB9@bitgravity.com> <4DE50811.5060606@FreeBSD.org> <4DE51DD3.6040602@FreeBSD.org> Date: Tue, 31 May 2011 12:21:33 -0700 X-Google-Sender-Auth: dAqwjHPBbM_crSUM1o93-bGoJkI Message-ID: From: Artem Belevich To: David P Discher Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, Andriy Gapon Subject: Re: ZFS: arc_reclaim_thread running 100%, 8.1-RELEASE, LBOLT related X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 May 2011 19:21:34 -0000 On Tue, May 31, 2011 at 12:07 PM, David P Discher wrot= e: > > On May 31, 2011, at 9:56 AM, Andriy Gapon wrote: > >>> >>> FYI, we've already changed clock_t for opensolaris code to int64_t in >>> r218169 regardless of $MACHINE. >> >> I think that's a good solution. >> I hope that we don't have any place where osol clock_t would somehow mix= with fbsd >> clock_t with detrimental effects. > > Well, clock_t to int64_t only fixes l2arc_reclaim_thread. > > If you look at lines 1712-1725 in sys/cddl/contrib/opensolaris/uts/common= /fs/zfs/arc.c, the arc_evict(), (8.1-release), you'll see that clock_t is n= ot use. =A0So, yes, clock_t will fix the l2arc issue, but not the arc_recla= im/arc evict issue of the signed 64bit int overflowing for LBOLT. No disagreement there. The way LBOLT is implemented now would indeed cause the overflow that leads to CPU hogging you've reported. > And from the conversation on this thread, there isn't any agreement on ho= w to really fix it. Andriy has proposed potential fix that would eliminate multiplication of gethrtime result by hz and would delay overflow by few hundred years. Should be good enough. > Someone please explain to me the units of LBOLT? HZ ticks per second. > > hz is cycles per second right ? > > =A0 =A0#define =A0 =A0 LBOLT =A0 ((gethrtime() * hz) / NANOSEC) > > > gethrtime() is returning nanoseconds. =A0 How is LBOLT in ticks-per-secon= d ? gethrtime is indeed nanoseconds. After division by NANOSES it becomes old good seconds. Multiplication by hz (which is ticks/sec) the whole thing represents just 'ticks' > > In sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c : > > =A0 =A0152 /* number of seconds before growing cache again */ > =A0 =A0153 static int =A0 =A0 =A0 =A0 =A0 =A0 =A0arc_grow_retry =3D 60; That would be in seconds. > =A0 =A0... > =A0 =A02255 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 /* reset the = growth delay for every reclaim */ > =A0 =A02256 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 growtime =3D = LBOLT + (arc_grow_retry * hz); (arc_grow_retry in seconds * hz in ticks/sec) --> ticks > > > LBOLT is then clearly using it math with ticks-per-second. =A0 I'm I just= crazy ? =A0It seems to me that in this case, line 2256 ... can't be added = together. LBOLT is in 'ticks', not ticks/sec. --Artem