From owner-freebsd-net@FreeBSD.ORG Mon Apr 18 17:59:59 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E7A4E1065677; Mon, 18 Apr 2011 17:59:58 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: from mail-px0-f172.google.com (mail-px0-f172.google.com [209.85.212.172]) by mx1.freebsd.org (Postfix) with ESMTP id AE2798FC1F; Mon, 18 Apr 2011 17:59:58 +0000 (UTC) Received: by pxi6 with SMTP id 6so7324874pxi.17 for ; Mon, 18 Apr 2011 10:59:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=ihHJU7Hu4jCYH+KgYyJNt8Z4aaMBikKuu1wTXvAwtWI=; b=ct2jsNC+wJnoGi5EGVut0OOGscfDJpFfu4i10fZmP1b3J92sEQjMvJhm3D28U7CPoh k5wPFyx8+NnR6QIUFVpwgbuXeHYOPHDxBVJ5xL+ZhgcysQUER6jqEWD7f/La9WTaLnac g2Cg7MOVDZlyCibQ6H/j3U3Rq5rPeu+Z/ILfU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=RN1vX543KvL4BOgHjNahLw0bXVTEX5UUxzHDG7rrg/il/bVufsZ+IkVOR28bjUBuE1 HfgLCh7KeZhfoQOUz9uIBs9gsFFnFaDLrvplBf42/OOwCh9dRC+45FSiMG5PwIkjqAZP BoQ0O0HokA+htFAgdnJzABASBDTOJTdr2BrPI= MIME-Version: 1.0 Received: by 10.68.52.227 with SMTP id w3mr7157450pbo.312.1303149598276; Mon, 18 Apr 2011 10:59:58 -0700 (PDT) Sender: kmacybsd@gmail.com Received: by 10.68.41.101 with HTTP; Mon, 18 Apr 2011 10:59:58 -0700 (PDT) In-Reply-To: References: Date: Mon, 18 Apr 2011 19:59:58 +0200 X-Google-Sender-Auth: 4W4o67wytIJimfwcFuDUxU2aTns Message-ID: From: "K. Macy" To: Ingo Flaschberger Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org, Ingo Flaschberger Subject: Re: Routing enhancement - reduce routing table locking X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Apr 2011 17:59:59 -0000 On Mon, Apr 18, 2011 at 7:28 PM, K. Macy wrote: > 400kpps is not a large enough measure to reach any conclusions. A > system like that should be able to push at least 2.3Mpps with > flowtable. I'm not saying that what you've done is not an improvement, > but rather that you're hitting some other bottleneck. The output of > pmc and LOCK_PROFILING might be insightful. It occurred to me that I should add a couple of qualifications to the previous statements. 1.6Mpps is line rate for GigE and I only know of it to be achievable by igb hardware. The most I've seen em hardware achieve is 1.1Mpps. Furthermore, in order to achieve that you would have to enable IFNET_MULTIQUEUE in the driver, because by default the driver uses the traditional (slow) IFQ as opposed overloading if_transmit and doing its own queueing when needed. Support for efficient multi-queue software queueing is provided by buf_ring, a lock-free multi-producer ring buffer written just for this purpose. Thus, the fairly low transmit rate may be attributable to driver locking. Cheers > > Thanks, > Kip > > On Mon, Apr 18, 2011 at 7:12 PM, Ingo Flaschberger wrote: >> >>> It would be great to see flowtable going back to its intended use. >>> However, I would be surprised if this actually scales to Mpps. I don't >>> have any high end hardware at the moment to test, what is the highest >>> packet rate you've seen? i.e. simply generating small packets. >> >> Currently I have no tests available, but I have seen at a appliance with= : >> Intel Q35 >> Quad Core cpu >> Intel em desktop pcie cards >> >> ~ 200mbit 64byte packets - ~ 400kpps without packetloss. >> >> Without patch flowtable and fastforward had the same speed as flowtable, >> fastfoward and standard forward. >> >> That means, with the patch the standard forward patch had the same speed= as >> the fastforward path. >> >> It seems, I'm hitting some other speedlimits at my system, so there was = no >> real difference between flowtable, fastforward with and without the patc= h. >> >> I would be great if someone could load a system with a full tables (400k >> routes) and do some tests at 10gbe speed. >> >> Kind regards, >> =A0 =A0 =A0 =A0Ingo Flaschberger >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >