From owner-freebsd-net@FreeBSD.ORG Sun Oct 24 18:50:26 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B56C2106564A; Sun, 24 Oct 2010 18:50:26 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 8DE7C8FC0C; Sun, 24 Oct 2010 18:50:26 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 2F02F46B99; Sun, 24 Oct 2010 14:50:26 -0400 (EDT) Date: Sun, 24 Oct 2010 19:50:26 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Lawrence Stewart In-Reply-To: <4CC2254C.7070104@freebsd.org> Message-ID: References: <4CA5D1F0.3000307@freebsd.org> <4CA9B6AC.20403@freebsd.org> <4CBB6CE9.1030009@freebsd.org> <4CC2254C.7070104@freebsd.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-net@freebsd.org, Andre Oppermann , Sriram Gorti Subject: Re: Question on TCP reassembly counter X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Oct 2010 18:50:26 -0000 On Sat, 23 Oct 2010, Lawrence Stewart wrote: >> One observation though: net.inet.tcp.reass.cursegments was non-zero (it was >> just 1) after 30 rounds, where each round is (as earlier) 15-concurrent >> instances of netperf for 20s. This was on the netserver side. And, it was >> zero before the netperf runs. On the other hand, Andre told me (in a >> separate mail) that this counter is not relevant anymore - so, should I >> just ignore it ? > > It's relevant, just not guaranteed to be 100% accurate at any given point in > time. The value is calculated based on synchronised access to UMA zone stats > and unsynchronised access to UMA per-cpu zone stats. The latter is safe, but > causes the overall result to potentially be inaccurate due to use of stale > data. The accuracy vs overhead tradeoff was deemed worthwhile for > informational counters like this one. > > That being said, I would not expect the value to remain persistently at 1 > after all TCP activity has finished on the machine. It won't affect > performance, but I'm curious to know if the calculation method has a flaw. > I'll try to reproduce locally, but can you please confirm if the value stays > at 1 even after many minutes of no TCP activity? It's possible we should revisit the current synchronisation model for per-CPU caches in this regard. We switched to soft critical sessions when the P4 Xeon was a popular CPU line -- it had extortionately expensive atomic operations, even when a cache line was in the local cache. If we were to move back to mutexes for per-CPU caches, then we could acquire all the locks in sequence and get an atomic snapshot across them all (if desired). This isn't a hard technical change, but would require very careful performance evaluation. Robert