From owner-freebsd-questions@FreeBSD.ORG Thu May 27 16:12:55 2010 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 14DBA1065676 for ; Thu, 27 May 2010 16:12:55 +0000 (UTC) (envelope-from m.seaman@infracaninophile.co.uk) Received: from smtp.infracaninophile.co.uk (unknown [IPv6:2001:8b0:151:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 935188FC08 for ; Thu, 27 May 2010 16:12:54 +0000 (UTC) Received: from seedling.black-earth.co.uk (seedling.black-earth.co.uk [81.187.76.163]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.4/8.14.4) with ESMTP id o4RGCR9x018403 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 27 May 2010 17:12:27 +0100 (BST) (envelope-from m.seaman@infracaninophile.co.uk) Message-ID: <4BFE99EB.50208@infracaninophile.co.uk> Date: Thu, 27 May 2010 17:12:27 +0100 From: Matthew Seaman Organization: Infracaninophile User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: Kevin Wilcox References: In-Reply-To: X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.96.1 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=2.0 required=5.0 tests=DKIM_ADSP_ALL,SPF_FAIL autolearn=no version=3.3.1 X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on lucid-nonsense.infracaninophile.co.uk Cc: Free BSD Questions list Subject: Re: FreeBSD router - large scale X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 May 2010 16:12:55 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 27/05/2010 16:00:12, Kevin Wilcox wrote: > Hello everyone. > > We're in the very early stages of considering [Free|Open]BSD on > commodity hardware to handle NAT *and* firewall duties for (what I > consider to be) a sizable deployment. Overall bandwidth is low, only a > gigabit connection, but we handle approximately fifteen thousand > devices. DHCP and DNS would be passed through to other servers, this > hardware would only be responsible for address translation and pf. > > I've done this on a very, very small scale (small/home office, small > business) but I'm curious how many other folks are doing it on this > scale, the hardware they are running on and any "gotchas" they may > have faced. Does pf on FreeBSD take advantage of multiple cores/SMP? > Is it preferable, as with OpenBSD, to go for a very stout processor > without much consideration to cores? Would freebsd-net@ be a better > place to ask this? > > I'm getting ready to start digging in to memory and other resources > needed based on available documentation but real-world usage is much > preferred to my academic assessment. I've used OpenBSD/pf + carp for several sites; also + relayd for a reasonably high traffic website, plus various setups using IPSec tunnels. All very successfully. On a reasonably fast modern processor, PF can run pretty much at GB wirespeed for straight packet forwarding or NAT. Doing serious crypto slows things up somewhat. The hardest job I've had an OpenBSD firewall do is actually as a mid-level firewall between a DMZ full of web servers and a back-end database layer. The thing to watch out for is running out of states in PF. It's trivial to change that in the config, and given a machine with 1GB or so RAM dedicated to running PF, you can up the number of states by a factor of a hundred or more without problem. Also if you know all your connections are from directly attached networks and very low latency, you can be a lot more aggressive about dropping old states. PF is basically single-threaded -- even on FreeBSD, multiple cores won't help you a great deal. (Unless you've got anything else running on the firewall, when several cores is really useful, of course.) On the other hand, PF is not hugely CPU intensive. Better to spend your money on the best NICs you can afford. There are some useful enhancements in OpenBSD-4.7/pf which haven't made it into FreeBSD yet -- FreeBSD pf is basically equivalent to about OpenBSD-4.1 I think. FreeBSD is compatible with more varieties of amd64/i386 based hardware, and it does threading and multi-cpu very much better. Cheers, Matthew - -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate JID: matthew@infracaninophile.co.uk Kent, CT11 9PW -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.14 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkv+mesACgkQ8Mjk52CukIyB4gCff56iOhw7jRwmH4jzhaRmZPiK COwAoINJQZ8YRk3s4plAuoru4CIdQr/h =xyZm -----END PGP SIGNATURE-----