From owner-freebsd-questions@FreeBSD.ORG Wed Jun 23 15:21:28 2010 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 44069106566B for ; Wed, 23 Jun 2010 15:21:28 +0000 (UTC) (envelope-from kevin.wilcox@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id F0E598FC08 for ; Wed, 23 Jun 2010 15:21:27 +0000 (UTC) Received: by mail-gx0-f182.google.com with SMTP id 3so754960gxk.13 for ; Wed, 23 Jun 2010 08:21:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=QQ2G2aPhK36mue1FGDVCFa6uGxOAGe5cZg39Lb6I0N4=; b=Jwl0oMomN5eH2IdhL+Z32kw0D4klH/zuVaAuEj2x0EMU7gpfEb9Gzi2nKHFTcbV3fg u9rnVi76mF4bxw/C93sHgYZ+j6FdwU8BUDLqDeRdX17GiJWPfH8CkOPuTsGgcLHXWoda ArxX04FMXu3tIxeYy5JgoVR/JNwUliG7gTD34= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=HFu6tzLoXZS7Yl+K6gN+lltwoBZQ55lUCgAWa6ML7I1mWC5hGee9HPrgjCzRIlPSJ3 R1uDToq1DSG1BG5CRmsGJ6d92NmCoB9JHcnFNmtIxHUvGS9MkVdx5bglR9tRuKUeagQA buzV9wGE0039WVVB1DbivS/2uzcb8Y+uFDicY= MIME-Version: 1.0 Received: by 10.90.219.1 with SMTP id r1mr5201601agg.118.1277306487688; Wed, 23 Jun 2010 08:21:27 -0700 (PDT) Received: by 10.90.98.5 with HTTP; Wed, 23 Jun 2010 08:21:27 -0700 (PDT) In-Reply-To: <4BFE99EB.50208@infracaninophile.co.uk> References: <4BFE99EB.50208@infracaninophile.co.uk> Date: Wed, 23 Jun 2010 11:21:27 -0400 Message-ID: From: Kevin Wilcox To: Matthew Seaman Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: Free BSD Questions list Subject: Re: FreeBSD router - large scale X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jun 2010 15:21:28 -0000 On 27 May 2010 12:12, Matthew Seaman wrot= e: > The hardest job I've had an OpenBSD firewall do is actually as a > mid-level firewall between a DMZ full of web servers and a back-end > database layer. =C2=A0The thing to watch out for is running out of states= in > PF. =C2=A0It's trivial to change that in the config, and given a machine = with > 1GB or so RAM dedicated to running PF, you can up the number of states > by a factor of a hundred or more without problem. =C2=A0Also if you know = all > your connections are from directly attached networks and very low > latency, you can be a lot more aggressive about dropping old states. Matthew - thanks for the information! For other reasons I'm limited to about 500k states...since our typical hardware build has at least 4GB of RAM, I'm not overly concerned about RAM exhaustion when routing. As I stated in another post the potential for something like a squid cache does exist, in which case I'll take all the RAM I can get my hands on (a 16GB+ build is not out of the question at that point). Preliminary testing has been favorable. My big concerns have mostly been related to state and packets per second. The first test environment was as follows: | one NIC, 4 routable addresses | | ------------------------------ | FreeBSD 8 Router | ------------------------------ | | one NIC with aliases for | 10.10.10.254 | 10.10.20.254 | 10.10.30.254 | 10.10.40.254 | ---------------- | switch | ---------------- Attached to the switch are four workstations/laptops: 10.10.10.1/255.255.255.0 10.10.20.1/255.255.255.0 10.10.30.1/255.255.255.0 10.10.40.1/255.255.255.0 All connections are gigabit. The idea is that in a production environment, we'll have multiple /22 networks coming in so I wanted to test having multiple network aliases. There will be a pool of public addresses for the outside interface(s), possibly as large as a class C but probably 20 - 30 addresses. By using sticky-address on a NAT rule, we can watch each RFC-1918 address get mapped to a different outside address via round-robin while enforcing that all connections from one inside host are consistently mapped to the same external address. Generating 10k active pings on each of the workstations/laptops, we were able to get an idea of how the machine would respond with 80k active states (two per connection, one in each direction). Adding in a couple of BitTorrent and HTTP .iso downloads only supported the conclusions we were beginning to form. Currently I'm testing it with multiple BitTorrent downloads and a very lively World of Warcraft installer. While nowhere near an indication of what we could expect in production it is showing us RAM usage, processor usage and state maintenance behaviour that gives us pretty good indications that we can go ahead and test in a larger environment. Like I said, we are otherwise limited to approximately 500k states (actually 250k connections) and only about half of that will be allotted for the population this project is targeting so testing with 100k states is actually pretty realistic at this point. We will wait, of course, to attempt a production deployment until after we have tested with a larger sample of the target population. Thanks to everyone for their comments and suggestions, both on and off list= ! kmw --=20 A: Maybe because some people are too annoyed by top-posting. Q: Why do I not get an answer to my question(s)? A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?