From owner-freebsd-current@FreeBSD.ORG Mon Nov 18 08:41:50 2013 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E87C359F; Mon, 18 Nov 2013 08:41:49 +0000 (UTC) Received: from mail-qe0-x229.google.com (mail-qe0-x229.google.com [IPv6:2607:f8b0:400d:c02::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8C13E2719; Mon, 18 Nov 2013 08:41:49 +0000 (UTC) Received: by mail-qe0-f41.google.com with SMTP id x7so3878272qeu.14 for ; Mon, 18 Nov 2013 00:41:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=G+T0OAuaFI/yyb4AkjROiIwYoZzdv9PevOuywYhjg+I=; b=f/wS6uq+Gr/7F3GSlqBFqzGBiHPw6JrwNTDOX2Hm2n2lVwEsTEFZPVON3B9JERZHJY vtI8sdfLATQreDpQTAQmq2G6g44oNezeACdLN89WV4QNJLSGByp121OXnUeVcN4THkuA tfdYnDsuk61qKwctA4Bn0Zy64OZZnhV7gZfLrM56nqRqF4XUClnsqIhIPV0L1H+AC3Wi /A9EzZuiEXQOB2l4j9yf476DXwTLEvtR8nFmo1hp3zoyibiPT1w3K6c/XnngZcEKd1Is qU83sCcJJcqvmaUZwh6elbJ1U4M/e4bxeSPHvnwpgmS6ZQbKFAD09k4qwETE4XKJ8yaG pTog== MIME-Version: 1.0 X-Received: by 10.224.64.200 with SMTP id f8mr32262534qai.55.1384764108825; Mon, 18 Nov 2013 00:41:48 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.207.66 with HTTP; Mon, 18 Nov 2013 00:41:48 -0800 (PST) In-Reply-To: <52894C92.60905@FreeBSD.org> References: <52894C92.60905@FreeBSD.org> Date: Mon, 18 Nov 2013 00:41:48 -0800 X-Google-Sender-Auth: NbolgVcs7EvAmjwQ51Qzypcoosk Message-ID: Subject: Re: UMA cache back pressure From: Adrian Chadd To: Alexander Motin Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-hackers@freebsd.org" , "freebsd-current@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Nov 2013 08:41:50 -0000 Hi! Your patch does three things: * adds a couple new buckets; * reduces some lock contention * does the aggressive backpressure. So, do you get any benefits from just the first one, or first two? -adrian On 17 November 2013 15:09, Alexander Motin wrote: > Hi. > > I've created patch, based on earlier work of avg@, to add back pressure to > UMA allocation caches. The problem of physical memory or KVA exhaustion > existed there for many years and it is quite critical now for improving > systems performance while keeping stability. Changes done in memory > allocation last years improved situation. but haven't fixed completely. My > patch solves remaining problems from two sides: a) reducing bucket sizes > every time system detects low memory condition; and b) as last-resort > mechanism for very low memory condition, it cycling over all CPUs to purge > their per-CPU UMA caches. Benefit of this approach is in absence of any > additional hard-coded limits on cache sizes -- they are self-tuned, based on > load and memory pressure. > > With this change I believe it should be safe enough to enable UMA allocation > caches in ZFS via vfs.zfs.zio.use_uma tunable (at least for amd64). I did > many tests on machine with 24 logical cores (and as result strong allocation > cache effects), and can say that with 40GB RAM using UMA caches, allowed by > this change, by two times increases results of SPEC NFS benchmark on ZFS > pool of several SSDs. To test system stability I've run the same test with > physical memory limited to just 2GB and system successfully survived that, > and even showed results 1.5 times better then with just last resort measures > of b). In both cases tools/umastat no longer shows unbound UMA cache growth, > that makes me believe in viability of this approach for longer runs. > > I would like to hear some comments about that: > http://people.freebsd.org/~mav/uma_pressure.patch > > Thank you. > > -- > Alexander Motin > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"