From owner-freebsd-current@FreeBSD.ORG Sun Jul 12 07:09:18 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9096D106566B for ; Sun, 12 Jul 2009 07:09:18 +0000 (UTC) (envelope-from mat.macy@gmail.com) Received: from mail-gx0-f217.google.com (mail-gx0-f217.google.com [209.85.217.217]) by mx1.freebsd.org (Postfix) with ESMTP id 498608FC13 for ; Sun, 12 Jul 2009 07:09:18 +0000 (UTC) (envelope-from mat.macy@gmail.com) Received: by gxk17 with SMTP id 17so2285244gxk.19 for ; Sun, 12 Jul 2009 00:09:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=sGl/ytSF0vCzRW2Cq3egsS0L7BmvdwyRNGLxPhK92y0=; b=N7k+04/si22j3LOJCrGxYr1/9JGvLviR9c/I25Ws7LFpKUIZxtaXduEmFpBAjHNv4i Rvz1EL5xrX6dzfPEVFWrEpJo3qZmVXrxXsXvPPkbSFsMzIe4iqMzmBPZzfzct+gXmD+p ql2+TF9FHhp403XaiG+j2m3TjTseo1LFhX128= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=AO6w8ahA8f3SyhixnfB8S4qqDx693vKgd8o+wYDSxbbGCGUq+z41YRuNPfYCpIBDrB TTUdo1EZ+HTaeu2HScP8SfGQxOPGoCB/AQ92YmarLeOOPNgLJL6RkBGVYP+tadqipvpE +pwgZ8SsZx7kV2MpW54tHoiXZyM8+Whu2drzo= MIME-Version: 1.0 Sender: mat.macy@gmail.com Received: by 10.100.177.15 with SMTP id z15mr5383075ane.27.1247382557604; Sun, 12 Jul 2009 00:09:17 -0700 (PDT) In-Reply-To: References: Date: Sun, 12 Jul 2009 00:09:17 -0700 X-Google-Sender-Auth: 35f4f1a20c2ca52e Message-ID: <3c1674c90907120009o330da19ds68c45d0dab6ef81f@mail.gmail.com> From: Kip Macy To: Scott Ullrich Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Current Subject: Re: Flowtables -- any tuning hints? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Jul 2009 07:09:18 -0000 On Sat, Jul 11, 2009 at 10:24 AM, Scott Ullrich wrote: > Hello Freebsd-current@ folks, > > I see with the commit "svn commit: r191259 - head/sys/netinet" > flowtables have been added.. =A0Cool! > > Does anyone have any tuning hints for this addition -- specifically > how much memory does the hash table consume? =A0 Or better yet does any > documentation exist for this newly added feature? > > Looking for an easy way to calculate max flows for the amount of > memory installed in a FreeBSD machine. You want to avoid hash collisions. So, generally speaking you want the hash table to be sized 2x larger than the number of unique connection destinations. You want the maximum number of flows to be as large as the maximum number of unique destinations x number of cores. When you get to the case of hundreds of thousands of unique destinations as in the case of a small ISP doing IP forwarding, you're probably better off disabling the flowtable. For most other workloads its likely to be a clear win. Running a process on an 8-core system with 8 threads each calling sendto(...) with 10 bytes I can push 3.5 - 4Mpps (with cxgb - you won't get this with most cards) with the flowtable enabled. With the flowtable disabled lock contention causes performance to degrade to 330kpps with the aforementioned workload. Let me know if you have any issues. Cheers, Kip