From owner-freebsd-net@FreeBSD.ORG Sat Mar 9 13:37:57 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 8451DBDB for ; Sat, 9 Mar 2013 13:37:57 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: from mail-ea0-x22a.google.com (mail-ea0-x22a.google.com [IPv6:2a00:1450:4013:c01::22a]) by mx1.freebsd.org (Postfix) with ESMTP id F2FA01D5 for ; Sat, 9 Mar 2013 13:37:56 +0000 (UTC) Received: by mail-ea0-f170.google.com with SMTP id a15so551164eae.15 for ; Sat, 09 Mar 2013 05:37:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:from:to:subject:date:user-agent:cc:references :in-reply-to:mime-version:content-type:content-transfer-encoding :message-id:x-gm-message-state; bh=ZfG3s3NFULdgi+srDZR0jl6TBJ+5Egt1AF4+r9mpUUI=; b=VNYp6mQA58OvDbOrGDKwua95jkxnt5bDmiRLkMFAoaIX0HmrjQv3OXLCkKSKc1ZK0z lm/4EXWW/SMwK6+2GAjHGU6584I61gwNEFQnsbFTP2smzhtK4oHjHJ0/i3YDrfjUjJdS D8Qole7pOddTcoQtqUa4w2FB0dquxQInAIsOFe6sH7ULwoz7LFm3VdCOQbpzJE2HDcWM mmVkekHVnJZQd2+hPiAO7DoT37TV7JRzoEj+Aicrt0zt8dnkPZOLX9aH0iotYPXqTO2G hvOr3kE0NGxB6dPC8zkU/YRfA5l4pKJjkco0X8S3bmmoarsHxPjPM0OHP5qs9bRviS16 FP6Q== X-Received: by 10.14.183.198 with SMTP id q46mr16472183eem.1.1362836275730; Sat, 09 Mar 2013 05:37:55 -0800 (PST) Received: from zvezda.localnet ([37.81.64.97]) by mx.google.com with ESMTPS id 44sm13262429eek.5.2013.03.09.05.37.53 (version=TLSv1 cipher=RC4-SHA bits=128/128); Sat, 09 Mar 2013 05:37:54 -0800 (PST) From: Kajetan Staszkiewicz To: Ermal =?utf-8?q?Lu=C3=A7i?= Subject: Re: [patch] Source entries removing is awfully slow. Date: Sat, 9 Mar 2013 14:37:51 +0100 User-Agent: KMail/1.13.5 (Linux/3.6.6-vegeta.1; KDE/4.4.5; x86_64; ; ) References: <201303081419.17743.vegeta@tuxpowered.net> <201303082151.00895.vegeta@tuxpowered.net> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201303091437.51945.vegeta@tuxpowered.net> X-Gm-Message-State: ALoCoQn09kjRt4d+P7fNlvvJYQ+w9TlP8yVULZMp79p7cm7tRAdsSin3D8UK8LSQTNNIBYIE08hQ Cc: "freebsd-net@freebsd.org" , "freebsd-pf@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Mar 2013 13:37:57 -0000 Dnia sobota, 9 marca 2013 o 13:14:16 Ermal Lu=C3=A7i napisa=C5=82(a): > On Fri, Mar 8, 2013 at 9:51 PM, Kajetan Staszkiewicz >=20 > wrote: > > Dnia pi=C4=85tek, 8 marca 2013 o 21:11:43 Ermal Lu=C3=A7i napisa=C5=82(= a): > > > Is this FreeBSD 9.x or HEAD? > >=20 > > I found the problem and developed the patch on 9.1. > >=20 > Can you please test this more 'beautiful' patch. Oh, somehow I did not notice an existing implementation for doubly linked l= ist.=20 I'm quite new to kernel programming. > Its similar to yours but also delays src state removal to the proper purge > thread. I'll try it right after the weekend. > Though the src node removal option through pfctl -K does a lot of job to > cleanup things > Still need to undertand why it takes so much time for you to loop through > 500K states. That is because the loop will not be called just once. `pfctl -K 0.0.0.0/0 -K ip.of.internal.server.behind.this.loadbalancer` will= =20 match multiple Source entries, up to a thousand of them in normal condition= s=20 ("normal" for my loadbalancers) and many many more when under a DDoS attack. > The purge thread does that every tick by partitioning it to a few per time > slot but still minutes is way loong. >=20 > Can you please try to give a top -SH view of the time when this happens a= nd > a pfctl -vvsa output? I'll try on Monday, although as far as I remember the machine was quite fro= zen=20 during this operation. =2D-=20 | pozdrawiam / greetings | powered by Debian, CentOS and FreeBSD | | Kajetan Staszkiewicz | jabber,email: vegeta()tuxpowered net | | Vegeta | www: http://vegeta.tuxpowered.net | `------------------------^---------------------------------------'