From owner-freebsd-net@FreeBSD.ORG Thu Apr 17 15:11:58 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F28DA106566C; Thu, 17 Apr 2008 15:11:58 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id A8AB58FC0C; Thu, 17 Apr 2008 15:11:58 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 36C1846C14; Thu, 17 Apr 2008 11:11:57 -0400 (EDT) Date: Thu, 17 Apr 2008 16:11:57 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Alexander Sack In-Reply-To: <3c0b01820804170752x68714a6cj10919e6cb7ffda16@mail.gmail.com> Message-ID: <20080417160922.X77343@fledge.watson.org> References: <3c0b01820804160929i76cc04fdy975929e2a04c0368@mail.gmail.com> <200804161456.20823.jkim@FreeBSD.org> <3c0b01820804161328m77704ca0g43077a9718d446d4@mail.gmail.com> <200804161654.22452.jkim@FreeBSD.org> <3c0b01820804161402u3aac4425n41172294ad33a667@mail.gmail.com> <20080417112329.G47027@delplex.bde.org> <3c0b01820804170643w6b771ce9jdfc2dc5b240922b@mail.gmail.com> <20080417145652.J71628@fledge.watson.org> <3c0b01820804170705l9c196d9rff45fa4919ace8c@mail.gmail.com> <20080417152837.E71628@fledge.watson.org> <3c0b01820804170752x68714a6cj10919e6cb7ffda16@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-net@freebsd.org, Dieter , Jung-uk Kim Subject: Re: bge dropping packets issue X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2008 15:11:59 -0000 On Thu, 17 Apr 2008, Alexander Sack wrote: > Gotcha, this is really good to know. FreeBSD is a new OS for me to work on > but I'm learning so much everyday! I believe Solaris uses ithreads natively > and I do know there are some drivers that forcefully create kernel threads > to handle interrupts as way to scale (I have no idea the ramification of > this on FreeBSD or if the overhead of creating a new kthread outweighs the > benefits). This is also true on FreeBSD -- on the whole, drivers up to 1gbps create one ithread per device (typically interface), but our 10gbps drivers [increasingly] instantiate multiple ithreads and direct traffic to multiple CPUs using different MSIX interrupts. So for 1gbps forwarding or bridging with if_em, we'll run the code in two ithreads. We direct dispatch our network stack by default in 7.x, meaning that we run everything up to local socket delivery in the ithread. If you're doing multi-interface forwarding, you might experiment with setting net.isr.direct=0 (via sysctl) to see how that affects performance -- this pushes network layer (and higher) processing to a separate netisr thread. Robert N M Watson Computer Laboratory University of Cambridge