From owner-freebsd-current@FreeBSD.ORG Sat Jun 11 00:25:40 2011 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 001551065673; Sat, 11 Jun 2011 00:25:39 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id BD1388FC0A; Sat, 11 Jun 2011 00:25:39 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id E5F357300A; Sat, 11 Jun 2011 02:41:50 +0200 (CEST) Date: Sat, 11 Jun 2011 02:41:50 +0200 From: Luigi Rizzo To: John Baldwin Message-ID: <20110611004150.GB59898@onelab2.iet.unipi.it> References: <20110609035952.GA30464@onelab2.iet.unipi.it> <201106090911.51817.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201106090911.51817.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i Cc: freebsd-current@freebsd.org Subject: Re: any place to look at for PCI-express performance issues ? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Jun 2011 00:25:40 -0000 On Thu, Jun 09, 2011 at 09:11:51AM -0400, John Baldwin wrote: > On Wednesday, June 08, 2011 11:59:52 pm Luigi Rizzo wrote: > > hi, > > during my tests with netmap with 10Gbit cards (82599, dual port), > > i notice that a motherboard with an AMD 880G chipset > > is performing significantly worse than an intel based one. > > In both cases the NIC is mounted on a 16x PCIe slot, > > and in both cases the driver reports the use 5Gb/4x per port. > > > > On the intel i reach easily 14.88Mpps, on the AMD the card tops > > at 1.8Mpps, and is not CPU limited (changing dev.cpu.0.freq does not change > > the throughput). > > Disabling flow control does not help (and in any case > > the other end of the link is the same), and since > > I am using the same picobsd image (based on FreeBSD/i386 > > head w/ my netmap code) i suspect that the difference in > > performance has to do with the PCIe controller. just for the records: the AMD motherboard works fine and can reach 14.88Mpps, i was just doing a couple of mistakes in my AMD tests, including the use of a slot with 16x form factor but only 4 lanes connected. This said, the i7-870 is about twice as fast as the Athlon II X4-635 in generating packets for the same clock speed. I think the different cache size might have some impact on the result given the Athlon has no L3 cache and the test program surely overflows the 512k L2 cache (i am using a total of 8k packet buffers, touching 64 bytes each for the payload, plus 24 bytes each for descriptors). Unfortunately at these speeds even small things matter a lot! cheers luigi