From owner-freebsd-net@FreeBSD.ORG Sun Apr 3 06:34:45 2005 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DE42516A4CE for ; Sun, 3 Apr 2005 06:34:45 +0000 (GMT) Received: from lunarlander.maxgigapop.net (128-8-243-144.umd.edu [128.8.243.144]) by mx1.FreeBSD.org (Postfix) with ESMTP id E4B2543D31 for ; Sun, 3 Apr 2005 06:34:44 +0000 (GMT) (envelope-from chris@maxgigapop.net) Received: (qmail 84227 invoked by uid 1000); 3 Apr 2005 06:34:44 -0000 Received: from localhost (sendmail-bs@127.0.0.1) by localhost with SMTP; 3 Apr 2005 06:34:44 -0000 Date: Sun, 3 Apr 2005 01:34:44 -0500 (EST) From: Chris Tracy To: freebsd-net@freebsd.org Message-ID: <20050403011248.S72062@lunarlander.maxgigapop.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Subject: can't get linerate TCP iperf between two Intel PRO/1000 82544GC cards X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2005 06:34:46 -0000 Hi, I have been attempting to get iperf to generate a line-rate TCP flow (~989Mbps) across a GigE link but can only get a maximum of around 912Mbps. The bottleneck appears to be on the sending side, as top shows the CPU for the iperf process at 50% (machine is a dual Xeon 2.4GHz with SMP and HTT enabled) and the WCPU column goes to 99%. Presumably the CPU is busy computing checksums, so I took a look at if_em.c to find out more about what kind of hardware checksum offloading features might be available, and I found this: if (adapter->hw.mac_type >= em_82543) { if (ifp->if_capenable & IFCAP_TXCSUM) ifp->if_hwassist = EM_CHECKSUM_FEATURES; I wasn't sure which chipset I had, so I checked the output of pciconf: # pciconf -v -l |grep 8254 device = '82544GC Gigabit Ethernet Controller (LOM)' I then tried an ifconfig em0, and it indeed shows the rxcsum and txcsum options listed in the output. However, executing something like 'ifconfig em0 txcsum' doesn't seem to help with the CPU issue. I took a look at /usr/src/sys/dev/em/README and didn't see the GC flavor of this card mentioned anywhere, nor any details or known limitations with regard to checksum offloading. I'm assuming GC means Gigabit Copper since these are the 1000bT versions. I also have some 82544EI/GC cards which I haven't gotten around to testing yet because of a jumbo frame issue on one of the layer3 devices in between the hosts with those cards. At this point, all that I can think of doing is to try and add some debug statements to the driver so that I might be able to get a better idea of what's going on with the hardware checksum...maybe it is already offloading correctly and the CPU bottleneck is being caused by something else? I can provide additional details if needed, but please include my email in the reply as I'm not subscribed to the list. Any help would be most appreciated. I've had no problems doing 989Mbps in Linux, but I'd really prefer to run FreeBSD. Thanks, -Chris