From owner-freebsd-net@FreeBSD.ORG Wed Apr 16 20:28:57 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 724261065670 for ; Wed, 16 Apr 2008 20:28:57 +0000 (UTC) (envelope-from pisymbol@gmail.com) Received: from yw-out-2324.google.com (yw-out-2324.google.com [74.125.46.30]) by mx1.freebsd.org (Postfix) with ESMTP id 2433B8FC20 for ; Wed, 16 Apr 2008 20:28:56 +0000 (UTC) (envelope-from pisymbol@gmail.com) Received: by yw-out-2324.google.com with SMTP id 2so1469933ywt.13 for ; Wed, 16 Apr 2008 13:28:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=Z+kIlDQp9tvFwmNbjAFvjddunSM3Uo71Lp238hKe7tc=; b=xeqT2uzmlSNdG1E9ZK5gYvbDPQrhsnnUGRRG9TsT3OlrSyH+dzW9lAn6EnKRsYMY/YJ5SpilDJ0xUPJCNSy8zJFocUFSX8jeMGzKs8eyQEb8ergnTMYERfFoTsrTmzZHCy0H+DfF9iVuc/x1N3hCqUWzYtxV2xovb47MMIshku0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=LYKcY0pM1HFHRwxoEBTRW3ufpBuqiS1jV3cHNd/O3QCr2f25GtDFqFp23Pv9S1Xo4ESZWAxS0IQvO8BIwDdUfMAxd8hPIZ01U5IM9+Exf452/KMupa+iS48NSd5+XlWLzVp1sCe2y9+qfZh0OUj7WOW7E8codpsTujb9gK4VYaY= Received: by 10.151.43.19 with SMTP id v19mr678488ybj.195.1208377719528; Wed, 16 Apr 2008 13:28:39 -0700 (PDT) Received: by 10.151.15.21 with HTTP; Wed, 16 Apr 2008 13:28:39 -0700 (PDT) Message-ID: <3c0b01820804161328m77704ca0g43077a9718d446d4@mail.gmail.com> Date: Wed, 16 Apr 2008 16:28:39 -0400 From: "Alexander Sack" To: "Jung-uk Kim" In-Reply-To: <200804161456.20823.jkim@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3c0b01820804160929i76cc04fdy975929e2a04c0368@mail.gmail.com> <200804161732.RAA23071@sopwith.solgatos.com> <3c0b01820804161120udb54ab3tea4bf7baade0061f@mail.gmail.com> <200804161456.20823.jkim@FreeBSD.org> Cc: freebsd-net@freebsd.org, Dieter Subject: Re: bge dropping packets issue X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2008 20:28:57 -0000 On Wed, Apr 16, 2008 at 2:56 PM, Jung-uk Kim wrote: > [CC trimmed] > > > On Wednesday 16 April 2008 02:20 pm, Alexander Sack wrote: > > Dieter: Thanks, at 20Mbps! That's pretty aweful. > > > > JK: Thanks again. Wow, I searched the list and didn't see much > > discussion with respect to bge and packet loss! I will try the > > rest of that patch including pushing the TCP receive buffer up > > (though I don't think that's going to help in this case). The > > above is based on just looking at code.... > > > > I guess some follow-up questions would be: > > > > 1) Why isn't BGE_SSLOTS tunable (to a point)? Why can't that be > > added the driver? I noticed that CURRENT has added a lot more > > SYSCTL information. Moreover it seems the Linux driver can set it > > up to 1024. > > IIRC, Linux tg3 uses one ring for both standard and jumbo. I'm talking about the number of slots within the ring not the number of RX queues. I believe the bnx4 driver (thought the tg stuff was deprecated??) uses 4 rings (one for each port perhaps) and reads hardware register at ISR time to flip between them. The BGE_SLLOTS define I think could be a tunable which would allow someone to tune the driver to use more or less RX slots. I still feel that is a good idea instead of forcing someone to recompile! :(! > > 2) bge_tick() uses the same global mutex for its callout as the > > rest of the driver. Moreover, it really doesn't have to hold it > > while updating statistics, they are reads of volatile registers > > anyway (blocking the ISR doesn't prevent the firmware from updating > > its stat struct). Would there be any interest in using a separate > > mutex for the callout itself and then just hold the lock for the > > other small calls (bge_asf_driver_up(), bge_watchdog())? I'm > > experimenting with right now just dropping the BGE mutex around the > > bge_stats_update() calls to give more time to bge_rxeof() to drain > > rx_bd's. I admit that bge_tick doesn't do a whole lot but it seems > > this driver is very sensitive to resource starvation and I'm trying > > to get the BGE driver to drain as much rx_bd's as possible to avoid > > drops due to the firmware having no place to put them! I'd like to experiment some more with this to see if I can get 100MBps 100% non-dropping (may not be possible with small changes but it'll be a character building experience none the less). > > 3) How does interrupt non-DEVICE_POLLING perform? > > If you just use default values, it won't perform very well. There are > some patches around the net to automatically adjust these numbers, > e.g., > > http://docs.freebsd.org/cgi/mid.cgi?20071117194615.L67319 > http://mail-index.netbsd.org/tech-kern/2004/03/16/0000.html Thanks will take a look! This really sucks... :D! -aps