From owner-freebsd-fs@FreeBSD.ORG Tue Feb 8 21:03:03 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 912A1106564A for ; Tue, 8 Feb 2011 21:03:03 +0000 (UTC) (envelope-from gldisater@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 462668FC14 for ; Tue, 8 Feb 2011 21:03:02 +0000 (UTC) Received: by yxh35 with SMTP id 35so2565683yxh.13 for ; Tue, 08 Feb 2011 13:03:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=ivdqO4A8PtoqGzP734T0qWvdwFwUGf62+eGo+YXl6Fk=; b=dhAIRYFUedXQ/Kz64xDWKRG7ISJY10ScHTJMRig7Zm6y3gAKYpmqHruH9gkirCgMYy sc10jEVOQIaPyfkgXBcqSvU1dcn+BTOW4lgHWPeKHI1AYmzYtZMGJepUFF5YCTW9G+La eqJl+oi9SE/niyZf/7z78xHL7sOyI3nmyv3Tw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; b=gxjeCwJb7FhsWsFKxT6bLsa9mwuXegPhLR76+ClRuyIkONZzBerBG+CjsTrioe/HUl 8jgLEfTEfF4By446hKUDeyeE25z/MA6B7x/r4tdfUzfqpnSvsTm7LTsJBZllbn/Y9j0r QBfVQRCfREI3MKk6CDhfU5m42nqWBOJtNHAV4= Received: by 10.90.52.20 with SMTP id z20mr417711agz.107.1297197558507; Tue, 08 Feb 2011 12:39:18 -0800 (PST) Received: from [192.168.1.2] (CPE00236909a9fe-CM00195eca698c.cpe.net.cable.rogers.com [99.236.13.52]) by mx.google.com with ESMTPS id b11sm7501926ana.38.2011.02.08.12.39.14 (version=SSLv3 cipher=RC4-MD5); Tue, 08 Feb 2011 12:39:14 -0800 (PST) Message-ID: <4D51A969.5090203@gmail.com> Date: Tue, 08 Feb 2011 15:36:57 -0500 From: Jeremy Faulkner User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20110130 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <4D510BBB.1060708@kkip.pl> <20110208102727.GA8555@icarus.home.lan> <4D511F65.2050503@kkip.pl> <4D519F97.2000805@kkip.pl> In-Reply-To: <4D519F97.2000805@kkip.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: Memory leak in ZFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Feb 2011 21:03:03 -0000 On 02/08/11 14:55, Bartosz Stec wrote: > W dniu 2011-02-08 11:48, Bartosz Stec pisze: > We didn't need to wait 2 days :) > Now I can confirm that sendfile under SAMBA + ZFS are responsible for > issue. Here's sample output from my monitoring script[1] (update every 2 > seconds): > > PHYSMEM: 2027.41 MB > ACTIVE: 61.14 MB > INACTIVE: 40.01 MB > WIRED: 1303.86 MB > CACHED: .50 MB > FREE: 552.30 MB > SUM: 1957.82 MB > ------------------------ > MISSING: 69.58 MB > > PHYSMEM: 2027.41 MB > ACTIVE: 61.14 MB > INACTIVE: 40.07 MB > WIRED: 1303.86 MB > CACHED: .50 MB > FREE: 551.80 MB > SUM: 1957.38 MB > ------------------------ > MISSING: 70.02 MB > > PHYSMEM: 2027.41 MB > ACTIVE: 61.14 MB > INACTIVE: 40.13 MB > WIRED: 1303.86 MB > CACHED: .50 MB > FREE: 551.30 MB > SUM: 1956.94 MB > ------------------------ > MISSING: 70.46 MB > > PHYSMEM: 2027.41 MB > ACTIVE: 61.14 MB > INACTIVE: 40.19 MB > WIRED: 1303.86 MB > CACHED: .50 MB > FREE: 550.80 MB > SUM: 1956.51 MB > ------------------------ > MISSING: 70.89 MB > > PHYSMEM: 2027.41 MB > ACTIVE: 61.14 MB > INACTIVE: 40.24 MB > WIRED: 1303.86 MB > CACHED: .50 MB > FREE: 550.42 MB > SUM: 1956.18 MB > ------------------------ > MISSING: 71.22 MB > > PHYSMEM: 2027.41 MB > ACTIVE: 61.14 MB > INACTIVE: 40.30 MB > WIRED: 1303.86 MB > CACHED: .50 MB > FREE: 549.92 MB > SUM: 1955.74 MB > ------------------------ > MISSING: 71.66 MB > > PHYSMEM: 2027.41 MB > ACTIVE: 61.14 MB > INACTIVE: 40.38 MB > WIRED: 1303.86 MB > CACHED: .50 MB > FREE: 549.30 MB > SUM: 1955.19 MB > ------------------------ > MISSING: 72.21 MB > > PHYSMEM: 2027.41 MB > ACTIVE: 61.14 MB > INACTIVE: 40.44 MB > WIRED: 1303.86 MB > CACHED: .50 MB > FREE: 548.80 MB > SUM: 1954.76 MB > ------------------------ > MISSING: 72.64 MB > > This behaviour has been seen while copying 600MB file from SAMBA share > with sendfile enabled. > It doesn't happen when writing to samba share, and it doesn't happen > with sendfile disabled, both ways. > For me it looks like memory which leaks should be added to wired pool > and belongs to ARC, but appareantly this doesn't work well and WIRED: > 1303.86 MB all the time. > > [1] http://pastebin.com/sQUyQbmm > Also occurring with the zfsv28 patch for 8-STABLE gldisater@constans:~% zpool upgrade This system is currently running ZFS pool version 28. All pools are formatted using this version. gldisater@constans:~% sh memleak-detect.sh PHYSMEM: 12268.94 MB ACTIVE: 427.57 MB INACTIVE: 724.98 MB WIRED: 10155.28 MB CACHED: 28.32 MB FREE: 556.16 MB SUM: 11892.32 MB ------------------------ MISSING: 376.61 MB gldisater@constans:~% uname -a FreeBSD constans 8.2-PRERELEASE FreeBSD 8.2-PRERELEASE #14: Tue Jan 18 15:26:02 EST 2011 root@constans:/usr/obj/usr/src/sys/CONSTANS amd64 gldisater@constans:~% uptime 3:22PM up 8 days, 5:07, 2 users, load averages: 0.21, 0.22, 0.24 -- Jeremy Faulkner