From owner-freebsd-pf@FreeBSD.ORG Thu Jul 5 19:55:26 2007 Return-Path: X-Original-To: freebsd-pf@freebsd.org Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BF60F16A46D for ; Thu, 5 Jul 2007 19:55:26 +0000 (UTC) (envelope-from pergesu@gmail.com) Received: from ik-out-1112.google.com (ik-out-1112.google.com [66.249.90.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3FD3D13C4AD for ; Thu, 5 Jul 2007 19:55:25 +0000 (UTC) (envelope-from pergesu@gmail.com) Received: by ik-out-1112.google.com with SMTP id c21so2160171ika for ; Thu, 05 Jul 2007 12:55:24 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=D3ufnkWdvUxAOE62HdWKvHCYa5HimDv8LKORfQFw2MfcoiDBC1fIO9vuPlr+FYZ+wH2y9zaTnTwV6dpuqP+rbu4ghFmfRdvOawj50l3Zw0OFsuNOaaJVEn48m2kHeaiK1YlXE99NHdwgy/4/r9IaIo8mCtrgImqdBqj21HR0O30= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=O3qAGJNzVE7VOuf3nvpinnXlugqV1w1teu1nV66FJtsdExduDLthXEpPJfOPEtyDAQJB6QZ3YP92SIhjlkkB+jfBiauQAjSBS5vDNda9JNHmf+l3QPT2HimL8771iKJqWP+13xCiH3gut9l8LA/ITfIZpGgsJPOtSsgiVnd3cLk= Received: by 10.78.185.15 with SMTP id i15mr4894514huf.1183665324268; Thu, 05 Jul 2007 12:55:24 -0700 (PDT) Received: by 10.78.200.15 with HTTP; Thu, 5 Jul 2007 12:55:24 -0700 (PDT) Message-ID: <810a540e0707051255w269b7362g576bce5695ba76ab@mail.gmail.com> Date: Thu, 5 Jul 2007 13:55:24 -0600 From: "Pat Maddox" To: "Greg Hennessy" In-Reply-To: <-7932512891363606358@unknownmsgid> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <810a540e0707050222s55a62641je0138e931832e86@mail.gmail.com> <-7932512891363606358@unknownmsgid> Cc: freebsd-pf@freebsd.org Subject: Re: Losing connections/performance with PF turned on X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2007 19:55:26 -0000 On 7/5/07, Greg Hennessy wrote: > > > > We're doing some stress testing on our server, > > CPU ? Memory ? Xeon 3060 (dual core @ 2.4 Ghz) 2 gigs of ram > > and noticed that when > > we turn PF on, we lose connections and have a drastic reduction in > > performance. > > > > We used SIEGE for 120 seconds, 50 connections, on req/conn > > > > [snip] > > > # --- DEFAULT POLICY > > block log all > > > > What drops are you seeing in the firewall logs for the missing connections ? I'm not very familiar with pf at this point. Here's a snippet of the log: pat@~: sudo tcpdump -n -e -ttt -r /var/log/pflog | grep CLIENT reading from file /var/log/pflog, link-type PFLOG (OpenBSD pflog file) 281. 491774 rule 2/0(match): block in on em0: CLIENT.56441 > SERVER.80: . ack 3842266997 win 5080 000117 rule 2/0(match): block in on em0: CLIENT.56456 > SERVER.80: P 3759758688:3759758883(195) ack 769179073 win 1460 000007 rule 2/0(match): block in on em0: CLIENT.56442 > SERVER.80: . ack 2278771587 win 5804 000005 rule 2/0(match): block in on em0: CLIENT.56442 > SERVER.80: F 0:0(0) ack 628 win 5804 000111 rule 2/0(match): block in on em0: CLIENT.56437 > SERVER.80: . ack 21684384 win 2184 > Are you monitoring the number of entries in the state table with pfctl -si ? > The default is iirc 10k, a benchmarking tool can easily chew through this. > > > > Greg I reran the benchmarks and monitored the # of entries, we hit 10k pretty quickly. Kept upping it until we got to 35k which is where we stopped seeing any returns. We still dropped some connections (99.6% of requests came back successfully), and the throughput was 3.4 Mbp as opposed to the 9.8 Mbps we get with the firewall off. I'll be doing a lot more testing over the next few days, so I'll have better info in a couple days...but if you can shed any light on this I'd really appreciate it. Pat