From owner-freebsd-fs@FreeBSD.ORG Wed Nov 19 04:12:49 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 129D8868 for ; Wed, 19 Nov 2014 04:12:49 +0000 (UTC) Received: from mail-wi0-f169.google.com (mail-wi0-f169.google.com [209.85.212.169]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 99B555E8 for ; Wed, 19 Nov 2014 04:12:48 +0000 (UTC) Received: by mail-wi0-f169.google.com with SMTP id r20so7677515wiv.2 for ; Tue, 18 Nov 2014 20:12:41 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=e+GLIpQPR9bSFzCqRMXNUhWJ8bo/57uxd1wZgyhW9rU=; b=eCBdt0Z7N+nWy47CuR4rcWz8qGZalAjnOI+ee+Bppf+YbWGGXrhTOnSiRI8TjRURE0 EmLDZxo5U6ya8yqLIGxNFemLBBff/jHCfoRcNY9ELCtmcDvdYVAosT9kaLZMO6GfeyLj fTU0Et9zSO1UopF9V43jJJqkTYHzxdm837UibTokmio1SFaJl9w002wTErBjrVdVikf3 UtaqPrzBSwqSxGU9+nqMchRe5wsWrCC+gPor6UWqnb2sZnorL4K7MNQU3LL7xTI3fi9Q EMuDL00hTcst4zKYwYP5tmutbGRrHIx41UD7PgunS5unVK8Kgtd5aNEVNTMsq/x8j823 W9mQ== X-Gm-Message-State: ALoCoQn050G/7870no6gNMOd8vjQbpOZiNp3fQyOkyj9SFWGHcOCzPXotfdeGMsvjW8dMOo53QIi X-Received: by 10.180.11.168 with SMTP id r8mr1594197wib.74.1416370360926; Tue, 18 Nov 2014 20:12:40 -0800 (PST) Received: from [10.10.1.68] (82-69-141-170.dsl.in-addr.zen.co.uk. [82.69.141.170]) by mx.google.com with ESMTPSA id mw7sm472629wib.14.2014.11.18.20.12.38 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 18 Nov 2014 20:12:39 -0800 (PST) Message-ID: <546C18F4.1090209@multiplay.co.uk> Date: Wed, 19 Nov 2014 04:13:40 +0000 From: Steven Hartland User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Re: No more free space after upgrading to 10.1 and zpool upgrade References: <20141116080128.GA20042@exhan.dylanleigh.net> <20141118054443.GA40514@core.summit> <546B8203.5040607@platinum.linux.pl> <546B9754.4060906@delphij.net> <20141119013611.GA52102@core.summit> <546C01C5.7080605@delphij.net> In-Reply-To: <546C01C5.7080605@delphij.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Nov 2014 04:12:49 -0000 On 19/11/2014 02:34, Xin Li wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > On 11/18/14 17:36, Emil Mikulic wrote: >> On Tue, Nov 18, 2014 at 11:00:36AM -0800, Xin Li wrote: >>> On 11/18/14 09:29, Adam Nowacki wrote: >>>> This commit is to blame: >>>> http://svnweb.freebsd.org/base?view=revision&revision=268455 >>>> >>>> 3.125% of disk space is reserved. >> This is the sort of thing I suspected, but I didn't spot this >> commit. >> >>> Note that the reserved space is so that one can always delete >>> files, etc. to get the pool back to a usable state. >> What about the "truncate -s0" trick? That doesn't work reliably? >> >>> I've added a new tunable/sysctl in r274674, but note that tuning >>> is not recommended >> Thanks!! >> >> Can you give us an example of how (and when) to tune the sysctl? > sysctl vfs.zfs.spa_slop_shift=6 would tune down the reserved space to > 1/(2^6) (=1.5625%). > > Personally I would never tune it. At this level of space your pool is > already running at degraded performance, by the way. Don't do that. > >> Regarding r268455, this is kind of a gotcha for people who are >> running their pools close to full - should this be mentioned in >> UPDATING or in the release notes? >> >> I understand that ZFS needs free space to be able to free more >> space, but 3% of a large pool is a lot of bytes. > Well, if you look at UFS, the reservation ratio is about 7.5% (8/108). > > File systems need free space to do allocation efficiently; even with > mostly static contents, performance would suffer because at high level > of space usage the file system would spend more time on looking up for > free space and the resulted allocation is likely to be more > fragmented. For ZFS, this means many essential operations like > resilvering would be much slower, which is a real threat to data > recoverability. > The new space map code should help with that and a fixed 3.125% is a large portion of a decent sized pool. On our event cache box for example thats 256GB which feels like silly amount to reserve. Does anyone have any stats which backup the need for this amount of free space on large pool arrays, specifically with spacemaps enabled?