From owner-freebsd-net@FreeBSD.ORG Sat Dec 8 10:35:17 2007 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D9F816A418 for ; Sat, 8 Dec 2007 10:35:17 +0000 (UTC) (envelope-from peter@alastria.net) Received: from nebula.thdo.uk.alastria.net (unknown [IPv6:2001:ba8:0:1f0::5]) by mx1.freebsd.org (Postfix) with ESMTP id 0D83A13C455 for ; Sat, 8 Dec 2007 10:35:16 +0000 (UTC) (envelope-from peter@alastria.net) Received: from [10.10.4.10] (dragon.lancs.uk.alastria.net [88.96.139.34]) (authenticated bits=0) by nebula.thdo.uk.alastria.net (8.13.3/8.13.3) with ESMTP id lB8AZSsC083296 for ; Sat, 8 Dec 2007 10:35:29 GMT (envelope-from peter@alastria.net) Message-ID: <475A735F.8000907@alastria.net> Date: Sat, 08 Dec 2007 10:35:11 +0000 From: Peter Wood Organization: Alastria Networks Limited User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) MIME-Version: 1.0 To: freebsd-net@freebsd.org References: <4755EFDD.8070609@isc.org> <20071205021851.V87930@fledge.watson.org> <20071205093244.U87930@fledge.watson.org> <20071205094657.P87930@fledge.watson.org> In-Reply-To: <20071205094657.P87930@fledge.watson.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Flag: NO X-Virus-Status: No X-Spam-Score: 0.137 () RCVD_IN_SORBS_DUL X-Spam-Ultra-Flag: NO X-Spam-Low-Flag: NO X-Spam-Flag: NO X-Spam-High-Flag: NO X-Scanned-By: MIMEDefang 2.51 on 212.13.198.8 Subject: Re: Aggregating many ports into one for tcpdump server. (also sampling before libpcap) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Dec 2007 10:35:17 -0000 Morning, >>> Looking thru the archives, it seems ng_one2many (in this case >>> 'many2one') is what I am looking for. Am I barking the right tree here? Strangely enough this is the exact situation I was looking into on Friday for two mirror ports from our border routers via aggregation switches. I had seen the netgraph solution however I had initially ignored if_bridge as I don't want the packets to be sent to the opposing devices. >> I've had several reports of significantly improved packet capture >> rates at high speeds with it, but it's not yet in the tree because we >> feel it needs more evaluation and review. I hope to ship some form of >> zero-copy BPF buffer support in FreeBSD 8, and possibly even MFC it. >> Any feedback you might have would be most helpful. As I am about to reinstall the server in question, I too shall give the zero copy code a go and report back. For reference on our two links the mirrored data is fed into snort (as well as tcpdump for "interactive" investigation) at about 700mbs average. Roberts suggestion of a 10Gbe interface hits home for me as we're in the middle of planning (or should I say plotting) an upgrade to our connection to the UK academic network to 10Gbe (although at maximum of 2.5Gbs due to our RENs connection, we're working on that too ;). At which point we might have to consider using sampling, unfortunately the aggregation switch we use doesn't support sampling on a mirror port. I know it's a tad off topic, but having a quick look that's not something I see libpcap shouting about. After very quick thinking would that have to be implemented in the kernel before the packets where passed to BPF? I'd prefer to use sampling rather then just accepting kernel droped packets to ensure fair selection over a time period, rather then only collecting the start of that period and then nothing else. I'd be willing to look into implementing that perhaps in the same way that Juniper Networks do for their sampling, ie. a maximum number of packets to be sampled in a second, how often to sample in terms of packets and then when sampling how many packets it should sample. Cheers, Peter Wood Network Security Specialist Information Systems Services Lancaster University -- Peter Wood