From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 18 09:59:41 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C5DF97ED; Mon, 18 Nov 2013 09:59:41 +0000 (UTC) Received: from mail-ee0-x235.google.com (mail-ee0-x235.google.com [IPv6:2a00:1450:4013:c00::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 109852C41; Mon, 18 Nov 2013 09:59:40 +0000 (UTC) Received: by mail-ee0-f53.google.com with SMTP id b57so2361474eek.12 for ; Mon, 18 Nov 2013 01:59:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=z8SjCiJNVsJYQIQevLms4rBH10/eVvjRNnAsaPnx910=; b=gbBZO8svj8R5kM0wCmtYbbSwCgyU+Fad2BcKXXRq6WR9n8VeydpMOwcl8SlpK9RZYZ SPLelGk9dWsI3tJaLIg8/ImVC1YnNXcJ1vF8swI2RPAj0ZaIaLedPx7P/RTTQ8MpAf26 ePk7JtG8RGGCWpTtPg7L1FsIAN0+py9sHSa+dudQWyF/xnMwdBjoRHTtus94ZWZQV0iK Zqqow3pXsFaDe2LSTPZWkEFzgeAN1o7g1XbhXD3KprV9Y7x/Bk7ON2ilJPjxqUCWdqwQ T5QawgbeKYcLyU2pwuAlbuUMHLkipl0omokO9JxDb2frdazmVos9JuED7CX5QYJjtVzj bcUQ== X-Received: by 10.14.109.1 with SMTP id r1mr11909280eeg.32.1384768779114; Mon, 18 Nov 2013 01:59:39 -0800 (PST) Received: from mavbook.mavhome.dp.ua ([178.137.150.35]) by mx.google.com with ESMTPSA id o47sm36065475eem.21.2013.11.18.01.59.36 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 18 Nov 2013 01:59:38 -0800 (PST) Sender: Alexander Motin Message-ID: <5289E506.2070207@FreeBSD.org> Date: Mon, 18 Nov 2013 11:59:34 +0200 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: Luigi Rizzo Subject: Re: UMA cache back pressure References: <52894C92.60905@FreeBSD.org> <5289DBF9.80004@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" , Adrian Chadd , "freebsd-current@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Nov 2013 09:59:41 -0000 On 18.11.2013 11:45, Luigi Rizzo wrote: > > > > On Mon, Nov 18, 2013 at 10:20 AM, Alexander Motin > wrote: > > On 18.11.2013 10:41, Adrian Chadd wrote: > > Your patch does three things: > > * adds a couple new buckets; > > > These new buckets make bucket size self-tuning more soft and > precise. Without them there are buckets for 1, 5, 13, 29, ... items. > While at bigger sizes difference about 2x is fine, at smallest ones > it is 5x and 2.6x respectively. New buckets make that line look like > 1, 3, 5, 9, 13, 29, reducing jumps between steps, making algorithm > work softer, allocating and freeing memory in better fitting chunks. > Otherwise there is quite a big gap between allocating 128K and > 5x128K of RAM at once. > > > just curious (and i do not understand whether the "1, 5 ..." are object > sizes in bytes or what), Buckets include header (~3 pointers), plus number of item pointers. So on amd64 1, 5, 13 mean 32, 64, 128 bytes per bucket. It is not really about saving memory on buckets themselves since they are very small, comparing to stored items. We could use bigger (like 16 items) bucket zone for allocating all smaller ones, overwriting just their items limit. But more zones potentially means also lower zone lock congestion there, so why not? > would it make sense to add some instrumentation > code (a small array of counters i presume) to track the actual number > of requests for exact object sizes, and perhaps at runtime create buckets > trying to reduce waste ? Since 10.0 buckets are also allocated from UMA cache zones, so all stats, garbage collection, etc. work by the same rules, which you can see in `vmstat -z`. > Following your reasoning there seems to be still a big gap between > some of the numbers you quote in the sequence. Big (2x) gaps between big numbers is less important since once we got there it means we have not so much memory pressure and should not be hurt by many extra frees. At lower numbers it may be more important. -- Alexander Motin