From owner-freebsd-current@FreeBSD.ORG Fri Jan 25 17:59:43 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1C4D616A419 for ; Fri, 25 Jan 2008 17:59:43 +0000 (UTC) (envelope-from stefan.lambrev@moneybookers.com) Received: from blah.sun-fish.com (blah.sun-fish.com [217.18.249.150]) by mx1.freebsd.org (Postfix) with ESMTP id 9CFE613C447 for ; Fri, 25 Jan 2008 17:59:42 +0000 (UTC) (envelope-from stefan.lambrev@moneybookers.com) Received: by blah.sun-fish.com (Postfix, from userid 1002) id 3F8D71B10EBB; Fri, 25 Jan 2008 18:59:41 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on blah.cmotd.com X-Spam-Level: X-Spam-Status: No, score=-10.6 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.2.3 Received: from hater.haters.org (unknown [192.168.25.10]) by blah.sun-fish.com (Postfix) with ESMTP id 205CA1B10C26 for ; Fri, 25 Jan 2008 18:59:38 +0100 (CET) Message-ID: <479A2389.2000802@moneybookers.com> Date: Fri, 25 Jan 2008 19:59:37 +0200 From: Stefan Lambrev User-Agent: Thunderbird 2.0.0.9 (X11/20071120) MIME-Version: 1.0 To: freebsd-current@freebsd.org Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.91.2/5550/Fri Jan 25 08:02:45 2008 on blah.cmotd.com X-Virus-Status: Clean Subject: FreeBSD 7, bridge, PF and syn flood = very bad performance X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jan 2008 17:59:43 -0000 Greetings, Does anyone try to see PF with "keep state" in action when under syn flood attack? I tried to get some help in freebsd-pf@, because the test firewall, that I build hardly can handle 2-5MB/s syn flood. Unfortunately I do not saw useful advice. The problem is that a quad core bridge firewall running freebsd 7 amd64 with PF is near useless and can't handle "small" SYN ddos. Here is the schema that I'm testing: web server (freebsd) - freebsd (bridged interfaces) - gigabit switch - clients + flooders In this configuration ~25MB/s syn flood (and I think this limit is because of my switch) is not a problem and the web server responds without a problem. With this configuration netperf -l 610 -p 10303 -H 10.3.3.1 shows 116MB/s stable speed , so I guess there are no problems with cables, hardware and etc :) But when I start pf (see below the config file) the traffic drops to 2-3MB/s and the web server is hardly accessible. It seems that device polling helps a lot in this situation, and at least the bridge firewall is accessible. Without "polling" the firewall is so heavily loaded that even commands like "date" take few seconds to finish, with 2 cores at ~100% idle at same time. I have "flat profiles" from hwpmc, and I think it indicates a problem: (bridge, pf enabled, polling enabled, sched_ule - I have profiles and for other combinations too if needed) % cumulative self self total time seconds seconds calls ms/call ms/call name 24.0 268416.00 268416.00 0 100.00% _mtx_lock_sleep [1] 6.7 343572.50 75156.50 0 100.00% pf_state_compare_ext_gwy [2] 6.7 418405.50 74833.00 0 100.00% pf_src_compare [3] 3.9 462298.50 43893.00 0 100.00% pf_state_compare_lan_ext [4] 3.6 503019.50 40721.00 0 100.00% pf_test [5] 3.2 538671.50 35652.00 0 100.00% pf_test_tcp [6] 3.1 573743.50 35072.00 0 100.00% bcopy [7] 2.3 599406.50 25663.00 0 100.00% bridge_pfil [8] 2.0 622096.50 22690.00 0 100.00% pf_state_compare_id [9] 2.0 644331.50 22235.00 0 100.00% pfsync_pack_state [10] 1.5 661563.50 17232.00 0 100.00% uma_zalloc_arg [11] 1.3 676296.50 14733.00 0 100.00% bzero [12] 1.3 690842.00 14545.50 0 100.00% bus_dmamap_load_mbuf_sg [13] 1.2 704521.50 13679.50 0 100.00% uma_zfree_arg [14] 1.2 717554.50 13033.00 0 100.00% bridge_rtnode_lookup [15] 1.2 730575.50 13021.00 0 100.00% _rw_rlock [16] 1.1 743076.50 12501.00 0 100.00% rn_match [17] 1.0 754016.50 10940.00 0 100.00% bridge_forward [18] 1.0 764766.00 10749.50 0 100.00% em_encap [19] 0.9 775400.00 10634.00 0 100.00% _rw_runlock [20] ........ Also when running without polling top -S shows that external em0 interface's taskq takes 100% CPU The situation is even worse when running with SCHED_BSD. Here is my test pf.conf quick #macros ext_if="em0" int_if="em1" br_if="bridge0" www="10.3.3.1" #sets set skip on lo0 set skip on $int_if set skip on $br_if set limit states 20000000 set limit src-nodes 19000000 set optimization aggressive table persist file "/etc/abusive_hosts" block log quick from to any block log quick from any to pass in on $ext_if proto tcp from any to $www port { 80, 443 } flags S/SA keep state \ (source-track rule, max-src-conn-rate 150/10, max-src-states 500, max-src-nodes 4000000, overload flush global) -- Best Wishes, Stefan Lambrev ICQ# 24134177