From owner-freebsd-net@FreeBSD.ORG Wed Jul 2 10:05:52 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9CB361065672 for ; Wed, 2 Jul 2008 10:05:52 +0000 (UTC) (envelope-from if@xip.at) Received: from chile.gbit.at (ns1.xip.at [193.239.188.99]) by mx1.freebsd.org (Postfix) with ESMTP id ECBA68FC12 for ; Wed, 2 Jul 2008 10:05:51 +0000 (UTC) (envelope-from if@xip.at) Received: (qmail 7835 invoked from network); 2 Jul 2008 12:05:49 +0200 Received: from unknown (HELO filebunker.xip.at) (86.59.10.180) by chile.gbit.at with (DHE-RSA-AES256-SHA encrypted) SMTP; 2 Jul 2008 12:05:49 +0200 Date: Wed, 2 Jul 2008 12:05:49 +0200 (CEST) From: Ingo Flaschberger To: Paul In-Reply-To: <486B4F11.6040906@gtcomm.net> Message-ID: References: <4867420D.7090406@gtcomm.net> <200806301944.m5UJifJD081781@lava.sentex.ca> <20080701004346.GA3898@stlux503.dsto.defence.gov.au> <20080701010716.GF3898@stlux503.dsto.defence.gov.au> <486986D9.3000607@monkeybrains.net> <48699960.9070100@gtcomm.net> <20080701033117.GH83626@cdnetworks.co.kr> <4869ACFC.5020205@gtcomm.net> <4869B025.9080006@gtcomm.net> <486A7E45.3030902@gtcomm.net> <486A8F24.5010000@gtcomm.net> <486A9A0E.6060308@elischer.org> <486B41D5.3060609@gtcomm.net> <486B4F11.6040906@gtcomm.net> User-Agent: Alpine 1.10 (LFD 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Cc: FreeBSD Net Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jul 2008 10:05:52 -0000 Dear Paul, > I still don't like the huge hit ipfw and lagg take :/ I think, you can't use fastforward with with lagg. > ** I tried polling in UP mode and I got some VERY interesting results.. > CPU is 44% idle (idle polling isn't on) but I'm getting errors! It's doing > 530kpps with ipfw loaded, which without polling uses 100% cpu but now it says > my cpu is 44% idle? that makes no sense.. If it was idle why am I getting > errors? I only get errors when em taskq was eating 100% cpu.. > Idle polling on/off makes no difference. > user_frac is set to 5 .. what are your values: kern.polling.reg_frac= kern.polling.user_frac= kern.polling.burst_max= I use: kern.polling.reg_frac=20 kern.polling.user_frac=20 kern.polling.burst_max=512 if you need more than 1000, you need to change the code: src/sys/kern/kern_poll.c #define MAX_POLL_BURST_MAX 1000 > So my maximum without polling is close to 800kpps but if I push that it > starts locking me from doing things, or how many kpps do you want to achieve? > HZ=2000 for this test (512/512 descriptors) you mean: hw.em.rxd=512 hw.em.txd=512 ? can you try with polling: hw.em.rxd=4096 hw.em.txd=4096 Kind regards, Ingo Flaschberger