From owner-freebsd-hackers@freebsd.org Thu Apr 28 14:51:00 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9768CB1F471 for ; Thu, 28 Apr 2016 14:51:00 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi0-x236.google.com (mail-oi0-x236.google.com [IPv6:2607:f8b0:4003:c06::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5BC4C13C5 for ; Thu, 28 Apr 2016 14:51:00 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi0-x236.google.com with SMTP id x19so85954820oix.2 for ; Thu, 28 Apr 2016 07:51:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc; bh=6nUnKlLvuBWxxOn3B7vaQr4f8CNSDQGqnKZmE6We8tQ=; b=ZRYtaIQwtVKh886vwfXMYgWYQOE5U1K4VtrHmbhQmBdtDHDMhBBsjYSfyrTp75OHZA YGptZ5vL6edsAJbPJkbx6zOba9+/eipM+Sc/WPSiJTd1F/TYBMDu67rcuik+mpCMRfqk 5p9xIv8yXEJRaS6M9kyBuxCUpEI9+NUcBmw0hH67hNf9c5GHOxZnMxzknkVYst676ffo WLPDhOfU7//aFn2GhIiE5EXEGrHRjwxEOzgOmes05UzbDTRKa5yrrHZtaueJq6zk5Bxz H/ktXTi+J120ALDb81mxZ2Z1Pe5a8G9ShxMgTdvT9Y3YGl/HcgGAA4O2G+oXV7RZ8eeK mClQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc; bh=6nUnKlLvuBWxxOn3B7vaQr4f8CNSDQGqnKZmE6We8tQ=; b=BX+twFXJnE3emPR5YDL0qtXf8AXYScQ/bwtcOpnfd1a1SHOoA5CVasyCtnn/fjhgPP 8/zKOyP2UkkeaHStpp09TLzUMWtXubgQjNqjqcFPy2eUDgp06mdxO4t63ikaW5ztDyXQ vg5Z5YUKivpOReSG3yjp4/ySAYz6yv5i3vBkAcyMw7n8+OZYNVVxeXnLme5CzT6xutIE mJ89G1xCD//nTBsmiTIZSXcm3zM5DLETgcU0HU3eXO5epGH4mtI6daXI1qWh2kWOma6T OC1AlR1xkEkxmL/MjAs8hH/XLX437dAsb+pQGe4X5uEB/U0l+IoRfVIYIUzbYTRrUqfR wE/Q== X-Gm-Message-State: AOPr4FWO0dqQq2NU4e2p2YQtkFbsj6yBoDO3xZ3Cpny0qD5SU4LZyFQb+tjCIoVl8Cj1Z/lPKDiKLO2XkZ1lOw== MIME-Version: 1.0 X-Received: by 10.157.38.40 with SMTP id a37mr7343618otb.152.1461855059696; Thu, 28 Apr 2016 07:50:59 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.202.64.138 with HTTP; Thu, 28 Apr 2016 07:50:59 -0700 (PDT) In-Reply-To: References: Date: Thu, 28 Apr 2016 08:50:59 -0600 X-Google-Sender-Auth: py33b7vvEgF-J1e-EIZy6vyt1Jo Message-ID: Subject: Re: Best option to process packet ACL From: Alan Somers To: =?UTF-8?Q?Z=C3=A9_Claudio_Pastore?= Cc: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Apr 2016 14:51:00 -0000 On Wed, Apr 27, 2016 at 1:21 PM, Z=C3=A9 Claudio Pastore wrote: > Hello everyone, > > I would like to hear your suggestion regarding the best approach to proce= ss > IP packets for filtering, in such a way I can avoid lowering my pps rate. > > Today a have a simple application proxies http application. It's dual > threaded on a 4 core system with low CPU power. The current application > uses two threads, one for control and one for data flow processing. > > I need to implement a simple set of stateless filtering, I will process > only: > > - src-ip > - dst-ip > - src-port > - dst-port > - iplen > - proto (tcp/udp/other) > > My current rate of requests per second is high, around 200K. I have no id= ea > how I can leverage the IDLE CPUs the best way to implement this ACL > filtering trying not to impact on the pps rate I have today. > > I have implemented it serial today (not threaded) and I get 40% performan= ce > loss. I will handle max 128 filter rules, this is a decision which is mad= e. > This is going to be first match wins. > > My current plans are to test: > > 1) Create 6 threads, one to test each aspect of the ACL (src-ip, dst-ip, > etc) the first thread that returns false to parent thread I stop processi= ng > that rule and go to the next, and tell all other threads to die/exit sinc= e > they don't matter anymore. > > 2) Create one thread to process a batch of rules, say, 8 rules per thread > per request. Don't know if I would limit total number of threads and lock > requests while threads ar e busy. > > 3) Someone suggested "do as pf/ipfw do" but I have no idea how it's done, > how multithreaded it is and what is done on each thread. > > 4) Other suggestion? > > This is going to run FreeBSD 11, I use libevent2 on the current applicati= on > so far. > > Thanks. > _______________________________________________ > > Is there some reason why you can't simply use pf or ipfw? ipfw can do everything you described. pf can do most of it, but I'm not sure if pf can filter on iplen. If I were you, I wouldn't attempt to write my own userland firewall until I was absolutely sure that neither pf nor ipfw would work. If that's the case, then I would try using diverter sockets. With a diverter socket, pf or ipfw does most of the work, but when it encounters a packet it can't process it pushes it up to a userland helper. The userland helper processes the packet and then tells pf or ipfw what to do with it. In realistic applications, pf or ipfw also creates a temporary rule based on the userland helper's decision. Applying the temporary rule in the future is far faster than invoking the userland helper. After a certain amount of time, the temporary rule will expire again. Here's an example in action: http://daemonforums.org/showthread.php?t=3D8846 -Alan