From owner-cvs-all@FreeBSD.ORG Wed May 11 11:33:13 2005 Return-Path: Delivered-To: cvs-all@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2C1A116A4CE; Wed, 11 May 2005 11:33:13 +0000 (GMT) Received: from relay.bestcom.ru (relay.bestcom.ru [217.72.144.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2E50E43D3F; Wed, 11 May 2005 11:33:12 +0000 (GMT) (envelope-from glebius@FreeBSD.org) Received: from cell.sick.ru (root@cell.sick.ru [217.72.144.68]) by relay.bestcom.ru (8.13.1/8.12.9) with ESMTP id j4BBX9Z6045261 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Wed, 11 May 2005 15:33:10 +0400 (MSD) (envelope-from glebius@FreeBSD.org) Received: from cell.sick.ru (glebius@localhost [127.0.0.1]) by cell.sick.ru (8.13.1/8.12.8) with ESMTP id j4BBX9Bt028390 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 11 May 2005 15:33:09 +0400 (MSD) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.sick.ru (8.13.1/8.13.1/Submit) id j4BBX8qB028389; Wed, 11 May 2005 15:33:09 +0400 (MSD) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.sick.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Wed, 11 May 2005 15:33:08 +0400 From: Gleb Smirnoff To: src-committers@FreeBSD.org, cvs-src@FreeBSD.org, cvs-all@FreeBSD.org Message-ID: <20050511113308.GA28318@cell.sick.ru> References: <200505111126.j4BBQOdL062708@repoman.freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <200505111126.j4BBQOdL062708@repoman.freebsd.org> User-Agent: Mutt/1.5.6i X-Virus-Scanned: ClamAV version devel-20050125, clamav-milter version 0.80ff on relay.bestcom.ru X-Virus-Status: Clean Subject: Re: cvs commit: src/sys/netgraph/netflow netflow.c netflow.h ng_netflow.c ng_netflow.h X-BeenThere: cvs-all@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: CVS commit messages for the entire tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2005 11:33:13 -0000 On Wed, May 11, 2005 at 11:26:24AM +0000, Gleb Smirnoff wrote: T> A new version of NetFlow node. T> T> The most significant changes are: T> - Use UMA zone instead of own chunk of memory. T> - Lock each hash entry separately. T> - Expire items "actively" - interrupt method can expire flows T> from hash slot, when it searches through it. T> - Remove global tailqueue. Make callout thread search through T> every hash slot. T> - Export datagram is detached from private data and filled. If T> it is incomplete, it is attached back. Another thread will T> continue working with it. T> T> Lesser, but also important speedups: T> - Flows in hash slot are stored in tailqueue. Whenever a flow is T> hit, it is moved to the begging, so it can be located quicker. T> - When callout thread works with hash slot it bails out if T> slot mutex is contested. With this new version I'm now processing live traffic via switch mirror port. The traffic pattern is quite unpleasant for NetFlow, there are lots of small flows in it - this is traffic of a big web services data center. Now more than 140 kpps is processed, half a gigabit. More than 21k exports are created per second. Average NetFlow cache size is 170k entries. I've been told about results of 190 kpps on RELENG_5. -- Totus tuus, Glebius. GLEBIUS-RIPN GLEB-RIPE