From owner-freebsd-net Wed Oct 16 7:39:19 2002 Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 69B5C37B401 for ; Wed, 16 Oct 2002 07:39:17 -0700 (PDT) Received: from carp.icir.org (carp.icir.org [192.150.187.71]) by mx1.FreeBSD.org (Postfix) with ESMTP id E699F43E6A for ; Wed, 16 Oct 2002 07:39:16 -0700 (PDT) (envelope-from rizzo@carp.icir.org) Received: from carp.icir.org (localhost [127.0.0.1]) by carp.icir.org (8.12.3/8.12.3) with ESMTP id g9GEdGpJ034796; Wed, 16 Oct 2002 07:39:16 -0700 (PDT) (envelope-from rizzo@carp.icir.org) Received: (from rizzo@localhost) by carp.icir.org (8.12.3/8.12.3/Submit) id g9GEdGCo034795; Wed, 16 Oct 2002 07:39:16 -0700 (PDT) (envelope-from rizzo) Date: Wed, 16 Oct 2002 07:39:16 -0700 From: Luigi Rizzo To: Petri Helenius Cc: Lars Eggert , freebsd-net@FreeBSD.ORG Subject: Re: ENOBUFS Message-ID: <20021016073916.A34626@carp.icir.org> References: <065901c27495$56a94c40$8c2a40c1@PHE> <3DAC8FAD.30601@isi.edu> <068b01c2749f$32e7cf70$8c2a40c1@PHE> <20021015161055.A27443@carp.icir.org> <06c901c274d8$e5280b80$8c2a40c1@PHE> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit User-Agent: Mutt/1.2.5.1i In-Reply-To: <06c901c274d8$e5280b80$8c2a40c1@PHE>; from pete@he.iki.fi on Wed, Oct 16, 2002 at 08:57:19AM +0300 Sender: owner-freebsd-net@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Wed, Oct 16, 2002 at 08:57:19AM +0300, Petri Helenius wrote: > > > > how large are the packets and how fast is the box ? > > Packets go out at an average size of 1024 bytes. The box is dual > P4 Xeon 2400/400 so I think it should qualify as "fast" ? I disabled yes, it qualifies as fast. With this kind of box, a trivial program can send short (18 byte payload, 64 byte total) UDP frames at 5-600kpps, with quite a bit of time i suspect is being spent in the userland-kernel transition (with some tricks to skip that i went up to ~680kpps). > The information I´m looking for is how to instrument where the hard to tell -- see if short packets you get the same performance i mention above, then maybe try some tricks such as sending short bursts (5-10 pkts at a time) on each of the interfaces. Maybe using a UP kernel as opposed to an SMP one might give you slightly better performance, i am not sure though. There might be some minor optimizations here and there which could possibly help (e.g. make th em driver use m_getcl(), remove IPSEC from the kernel if you have it) but you are essentially close to the speed you can get with that box (within a factor of 2, probably). cheers luigi > > on a fast box you should be able to generate packets faster than wire > > speed for sizes around 500bytes, meaning that you are going to saturate > > the queue no matter how large it is. > > > > cheers > > luigi > > > > > em-interface is running 66/64 and is there a way to see interface queue > depth? > > > em0: port > 0x3040-0x307f > > > mem 0xfc220000-0xfc23ffff irq 17 at device 3.0 on pci2 > > > em0: Speed:1000 Mbps Duplex:Full > > > pcib2: at device 29.0 on pci1 > > > IOAPIC #2 intpin 0 -> irq 16 > > > IOAPIC #2 intpin 6 -> irq 17 > > > IOAPIC #2 intpin 7 -> irq 18 > > > pci2: on pcib2 > > > > > > The OS is 4.7-RELEASE. > > > > > > Pete > > > > > > > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > > with "unsubscribe freebsd-net" in the body of the message > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message