From owner-freebsd-pf@freebsd.org Wed Aug 26 14:44:52 2015 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2A9D19C39BB for ; Wed, 26 Aug 2015 14:44:52 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-lb0-f176.google.com (mail-lb0-f176.google.com [209.85.217.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9D922978 for ; Wed, 26 Aug 2015 14:44:50 +0000 (UTC) (envelope-from ml@my.gd) Received: by lbbpu9 with SMTP id pu9so121412648lbb.3 for ; Wed, 26 Aug 2015 07:44:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=qrRMHpXQ4YvMXKJuhpckkRIHllXSSbUCih9tFoFhTlg=; b=R52YGAxSyvi/G/Zb66f7yrT+6f9/m+jBFi4Cz93PRrtevKuoK5PTxvWGFML3wCBO/k sOPoQnM0wyrlIDilh/dE5XnNHNUDP6tXBdF+5RaL/VCTwpdH3me17nfNQHhFTbzm9RpN 0Ao4W7VyMwAYbqoKV6v0QrKPcFjp0NgmIXXZ9NFgApy/HtAki91ZbpQF2ynsU5SFrPff 6JFl42tKBfy+WgxvG5bxBEy1jkHtNzCe0XYSzMqzDPQ/HBXu03uwL9yZDxYQJ4M8M9jG ofKn/t/7tJ6t/b1EtghSFUo7Rm8Cu93cOQONtv8FccSCQySdPgDAGdHaKBpbFn7gXtVe CtAQ== X-Gm-Message-State: ALoCoQntQkoV6m01xXFQJkAXY2Ec/4s40VGzAJ9QJHM0BRJJ9St2XRhFlGZ3+wVFDw7FIyHQ60YF MIME-Version: 1.0 X-Received: by 10.152.28.105 with SMTP id a9mr31796541lah.9.1440599808943; Wed, 26 Aug 2015 07:36:48 -0700 (PDT) Received: by 10.112.60.34 with HTTP; Wed, 26 Aug 2015 07:36:48 -0700 (PDT) In-Reply-To: <894145A3DDBDEF4880E00D334DCD87263EC814A8@MXS2.zuv.uni-muenchen.de> References: <894145A3DDBDEF4880E00D334DCD87263EC814A8@MXS2.zuv.uni-muenchen.de> Date: Wed, 26 Aug 2015 16:36:48 +0200 Message-ID: Subject: Re: Machine freezes when loading pf ruleset From: Damien Fleuriot To: Kolontai Andrej Cc: "freebsd-pf@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Aug 2015 14:44:52 -0000 On 26 August 2015 at 16:09, Kolontai Andrej < Andrej.Kolontai@verwaltung.uni-muenchen.de> wrote: > >1.5k rules seems like a lot for PF to handle. > > > >Is that 1.5k rules you've written in the conf, or 1.5k rules from `pfctl > -sr | wc -l' ? > > Yes, that's what is in the conf files. The latter command gives around > 3400... > > >I would suggest you find a way to drastically lower that. > > Given the number of networks, devices and applications that seems to be > difficult. Also, we have some policies on our firewall which contribute to > the large number of rules: > * strict minimum principle. Only traffic that is explicitly needed passes > the fw. That gives a lot of combinations. > * most rules are either "pass in" or "pass out" . So every connection > needs to have a pass in and a pass out rule. This does not mean that we > have every rule twice. It's just that inbound and outbound policies are > different things and we try not to mix them (accidently). > * using "quick" is actually not allowed except in very special cases as it > tends to cause unexpected side effects. > > What we do is, of course, using tables and anchors to make the evaluation > of the rule set as efficient as possible. > > I need to say that in normal operation, the machines perform really well > with absolutely no sign of shortage in any resource. > > A typical top screen looks like this: > > last pid: 59875; load averages: 0.29, 0.33, 0.36 > > up 0+02:55:51 15:02:22 > 38 processes: 1 running, 37 sleeping > CPU: 0.0% user, 0.0% nice, 0.0% system, 0.7% interrupt, 99.2% idle > Mem: 3588K Active, 1883M Inact, 2333M Wired, 2037M Buf, 89G Free > Swap: 117G Total, 117G Free > ... > > There's plenty of memory and cpu power. Networking is (physically) capable > of 40 gbit/s (4 aggregated 10 gbit links). I have never seen a cpu load > higher than 1.something. > > The problem occurs only in the moment I load the ruleset and only if the > machine is in master state. If I only knew what is happening at that > point... I can't make any sense of the log outputs. > > >You may also wish to ensure : > >- you're using, if at all possible, a *dedicated* pfsync link (like a > cable on a dedicated interface between the boxes) > > Yes, to some degree that is true. Pfsync runs over separate physical > interfaces. But the machines are at different places. That means that the > pfsync net is in fact just another vlan which runs over the same fiber link > as everything else. Yes, we used to use IPsec to secure it. Right now, > Ipsec is turned off for pfsync to exclude the possibility that this causes > the problem. But that's apparently not the case. It still happens. > > > Well the machine seems to be very apt for the job indeed with plenty of spare resources. While I believe your filtering on egress traffic to be somewhat redundant since you already filter inbound connections, it is not my place to comment on that, not to mention that would certainly entail huge changes on your side. I do believe however that when you reload the ruleset PF sets some kind of a lock. I'm afraid I'm not familiar enough with the internals of PF to dig deeper into this, but that may be worth a look into.