From owner-freebsd-net@FreeBSD.ORG Wed Feb 25 23:36:28 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9F3118F1; Wed, 25 Feb 2015 23:36:28 +0000 (UTC) Received: from khavrinen.csail.mit.edu (khavrinen.csail.mit.edu [IPv6:2001:470:8b2d:1e1c:21b:21ff:feb8:d7b0]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "khavrinen.csail.mit.edu", Issuer "Client CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 25F54CEC; Wed, 25 Feb 2015 23:36:28 +0000 (UTC) Received: from khavrinen.csail.mit.edu (localhost [127.0.0.1]) by khavrinen.csail.mit.edu (8.14.9/8.14.9) with ESMTP id t1PNaQ0b062094 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL CN=khavrinen.csail.mit.edu issuer=Client+20CA); Wed, 25 Feb 2015 18:36:26 -0500 (EST) (envelope-from wollman@khavrinen.csail.mit.edu) Received: (from wollman@localhost) by khavrinen.csail.mit.edu (8.14.9/8.14.9/Submit) id t1PNaQbl062091; Wed, 25 Feb 2015 18:36:26 -0500 (EST) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <21742.23674.220013.63261@khavrinen.csail.mit.edu> Date: Wed, 25 Feb 2015 18:36:26 -0500 From: Garrett Wollman To: Alfred Perlstein Subject: Re: Implementing backpressure in the NFS server In-Reply-To: <54EE5AE9.1000908@freebsd.org> References: <21742.18390.976511.707403@khavrinen.csail.mit.edu> <54EE5AE9.1000908@freebsd.org> X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (khavrinen.csail.mit.edu [127.0.0.1]); Wed, 25 Feb 2015 18:36:26 -0500 (EST) Cc: freebsd-fs@freebsd.org, freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 23:36:28 -0000 < said: > I think your other suggestions are fine, however the problem is that: > 1) they seem complex for an edge case > 2) turning them on may tank performance for no good reason if the > heuristic is met but we're not in the bad situation I'm OK with trading off performance for one user against availability for the other 800. I'm pleased to hear that FreeNAS was fine with 256 threads as the default; I'm going to try running with that to see if we encounter any other scaling issues as a result. (Our servers are all 12-core, 24-thread systems with buckets of memory, and I remember increasing the thread count as high as 128 previously, but I also remember having to back it down to 64 for some reason.) -GAWollman