From owner-freebsd-net@FreeBSD.ORG Fri Apr 20 08:31:08 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id 240BF106564A; Fri, 20 Apr 2012 08:31:08 +0000 (UTC) (envelope-from melifaro@FreeBSD.org) Received: from dhcp170-36-red.yandex.net (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx2.freebsd.org (Postfix) with ESMTP id 43E1B14DEE2; Fri, 20 Apr 2012 08:31:06 +0000 (UTC) Message-ID: <4F911DCD.30001@FreeBSD.org> Date: Fri, 20 Apr 2012 12:26:53 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111117 Thunderbird/8.0 MIME-Version: 1.0 To: Andre Oppermann References: <20120419133018.GA91364@onelab2.iet.unipi.it> <4F907011.9080602@freebsd.org> <20120419204622.GA94904@onelab2.iet.unipi.it> <4F907FB4.3080400@freebsd.org> In-Reply-To: <4F907FB4.3080400@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: net@freebsd.org, Luigi Rizzo , "K. Macy" , current@freebsd.org Subject: Re: Some performance measurements on the FreeBSD network stack X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Apr 2012 08:31:08 -0000 On 20.04.2012 01:12, Andre Oppermann wrote: > On 19.04.2012 22:34, K. Macy wrote: >>>> This is indeed a big problem. I'm working (rough edges remain) on >>>> changing the routing table locking to an rmlock (read-mostly) which >>> >> >> This only helps if your flows aren't hitting the same rtentry. >> Otherwise you still convoy on the lock for the rtentry itself to >> increment and decrement the rtentry's reference count. > > The rtentry lock isn't obtained anymore. While the rmlock read > lock is held on the rtable the relevant information like ifp and > such is copied out. No later referencing possible. In the end > any referencing of an rtentry would be forbidden and the rtentry > lock can be removed. The second step can be optional though. > >>> i was wondering, is there a way (and/or any advantage) to use the >>> fastforward code to look up the route for locally sourced packets ? >>> >> >> If the number of peers is bounded then you can use the flowtable. Max >> PPS is much higher bypassing routing lookup. However, it doesn't scale From my experience, turning fastfwd on gives ~20-30% performance increase (10G forwarding with firewalling, 1.4MPPS). ip_forward() uses 2 lookups (ip_rtaddr + ip_output) vs 1 ip_fastfwd(). The worst current problem IMHO is number of locks packet have to traverse, not number of lookups. >> to arbitrary flow numbers. > > In theory a rmlock-only lookup into a default-route only routing > table would be faster than creating a flow table entry for every > destination. It a matter of churn though. The flowtable isn't > lockless in itself, is it? > -- WBR, Alexander