From owner-freebsd-net@FreeBSD.ORG Fri Sep 10 05:50:09 2004 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8ADF516A4CE for ; Fri, 10 Sep 2004 05:50:09 +0000 (GMT) Received: from diaspar.rdsnet.ro (diaspar.rdsnet.ro [213.157.165.9]) by mx1.FreeBSD.org (Postfix) with ESMTP id AD1B943D46 for ; Fri, 10 Sep 2004 05:50:08 +0000 (GMT) (envelope-from dudu@diaspar.rdsnet.ro) Received: (qmail 620 invoked by uid 89); 10 Sep 2004 05:50:37 -0000 Received: from unknown (HELO diaspar.rdsnet.ro) (dudu@diaspar.rdsnet.ro@213.157.165.9) by 0 with AES256-SHA encrypted SMTP; 10 Sep 2004 05:50:37 -0000 Date: Fri, 10 Sep 2004 08:50:36 +0300 From: Vlad GALU To: freebsd-net@freebsd.org Message-Id: <20040910085036.11a5d3b3.dudu@diaspar.rdsnet.ro> In-Reply-To: <20040910053619.GB14470@cell.sick.ru> References: <20040905121111.GA78276@cell.sick.ru> <4140834C.3000306@freebsd.org> <20040909171018.GA11540@cell.sick.ru> <414093DE.A6DC6E67@freebsd.org> <41409CB5.836DE816@freebsd.org> <20040909193507.GA12168@cell.sick.ru> <4140B603.8E979D72@freebsd.org> <20040909200052.GD12168@cell.sick.ru> <20040909234126.3f1c7cf3.dudu@diaspar.rdsnet.ro> <20040910053619.GB14470@cell.sick.ru> Organization: Romania Data Systems X-Mailer: Sylpheed version 0.9.12 (GTK+ 1.2.10; i386-portbld-freebsd4.10) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [TEST/REVIEW] Netflow implementation X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Sep 2004 05:50:09 -0000 On Fri, 10 Sep 2004 09:36:19 +0400 Gleb Smirnoff wrote: > On Thu, Sep 09, 2004 at 11:41:26PM +0300, Vlad GALU wrote: > V> This made me raise my eyebrow. I wrote a small tool that we use > V> in > V> production at RDS: http://freshmeat.net/projects/glflow. The way I > V> designed it, it is supposed to clean up the flow tree once in a > V> while and remove 'old' flows (that haven't had any packet matching > V> them in the last X seconds). The problem is that I currently have > V> about 400-500k active flows on a 700Mbps link. Every 10 seconds the > V> software removes about 100-200k of them in no more than 0.2-0.3 > V> seconds. Of course, I couldn't possibly send them over a socket > V> somewhere else at that speed,, and chose to open a tempfile, mmap() > V> it, write the expired flows to the buffer. When the buffer exceeds > V> a programatically chosen number of packets, it is msync()-ed, > V> munmap()-ed and a new file is open. > > If you remove 100-200k of flows every 10 seconds, this means you have > 10 - 20 kpps of worm traffic or more! Impressive. That, and the fact that we get a DoS attack every minute :( > 200k flows expired in 10 seconds is 666 export dgrams per second, 7 > times more than I usually have in my testbed. Not sure, that > ng_netflow + ng_ksocket can do this. > > Do you collect all this traffic on a single router? > For now, yes. I've planned into developing this to a distributed solution, using the spread toolkit (www.spread.org). My goal is to have an instance of glflow running in every branch of our national backbone and filter the destination from the very source of the flood, so we can save traffic. Right now there's only one instance running at the entrypoint of our national backbone ring. However, since I'm going to leave RDS in a week or so, I'll probably find another testbed. I hope it will be one with at least half the traffic we have here. > V> Do you accidentally have a better storage model ? I've been > V> trying to > V> dump these binary files to SQL but for a 42 meg binary log the > V> necessary SQL storage went to about 150 megs, which is a bit over > V> reasonable, considering the fact that the software dumps a binary > V> file every 5 to 10 seconds. > > Dumping to SQL is a bad idea. I have tried it, too :) > > V> P.S. I haven't yet tried to aggregate the flows between reading > V> them from the binary file and inserting the data into SQL. I > V> thought it would take too much time to be able to keep up with the > V> newly created dumps. > > This is how many people do. However I don't know details. > > -- > Totus tuus, Glebius. > GLEBIUS-RIPN GLEB-RIPE > ---- If it's there, and you can see it, it's real. If it's not there, and you can see it, it's virtual. If it's there, and you can't see it, it's transparent. If it's not there, and you can't see it, you erased it.