From owner-freebsd-arch@FreeBSD.ORG  Mon May 24 20:51:25 2004
Return-Path: <owner-freebsd-arch@FreeBSD.ORG>
Delivered-To: freebsd-arch@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP
	id 8DEFC16A4CE; Mon, 24 May 2004 20:51:25 -0700 (PDT)
Received: from fledge.watson.org (fledge.watson.org [204.156.12.50])
	by mx1.FreeBSD.org (Postfix) with ESMTP
	id 19DFE43D31; Mon, 24 May 2004 20:51:25 -0700 (PDT)
	(envelope-from robert@fledge.watson.org)
Received: from fledge.watson.org (localhost [127.0.0.1])
	by fledge.watson.org (8.12.11/8.12.11) with ESMTP id i4P3nCN6075871;
	Mon, 24 May 2004 23:49:12 -0400 (EDT)
	(envelope-from robert@fledge.watson.org)
Received: from localhost (robert@localhost)i4P3nCfT075868;
	Mon, 24 May 2004 23:49:12 -0400 (EDT)
	(envelope-from robert@fledge.watson.org)
Date: Mon, 24 May 2004 23:49:11 -0400 (EDT)
From: Robert Watson <rwatson@FreeBSD.org>
X-Sender: robert@fledge.watson.org
To: Matthew Dillon <dillon@apollo.backplane.com>
In-Reply-To: <200405250339.i4P3dLBX090505@apollo.backplane.com>
Message-ID: <Pine.NEB.3.96L.1040524234219.98946L-100000@fledge.watson.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
cc: arch@FreeBSD.org
cc: Eivind Eklund <eivind@FreeBSD.org>
Subject: Re: Network Stack Locking
X-BeenThere: freebsd-arch@freebsd.org
X-Mailman-Version: 2.1.1
Precedence: list
List-Id: Discussion related to FreeBSD architecture
	<freebsd-arch.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-arch>,
	<mailto:freebsd-arch-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-arch>
List-Post: <mailto:freebsd-arch@freebsd.org>
List-Help: <mailto:freebsd-arch-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-arch>,
	<mailto:freebsd-arch-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 25 May 2004 03:51:25 -0000


On Mon, 24 May 2004, Matthew Dillon wrote:

>     Deep message queues aren't necessarily a problem and, in fact, having
>     one or two dozen messages backed up in a protocol thread's message
>     port is actually good because the thread can then process all the
>     messages in a tight loop (cpu and cache locality of reference).  If
>     designed properly, this directly mitigates the cost of a thread switch
>     as system load increases.  So message queueing has the opposite effect...
>     per-unit handling overhead *decreases* as system load increases.

Actually, this was the specific point I was making also :-).  The question
I was asking was about the depth of the message queues between protocol
stack layers in actual measurements -- are you observing substantial
coallescing between layers as a result of the queues at this point?  I'm
looking for emperical evidence that the coallescing does make up for the
extra context switches of the model in practice...

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert@fledge.watson.org      Senior Research Scientist, McAfee Research