From owner-freebsd-net@FreeBSD.ORG Thu Sep 1 04:49:22 2005 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 98CEB16A41F; Thu, 1 Sep 2005 04:49:22 +0000 (GMT) (envelope-from ganbold@micom.mng.net) Received: from publicd.ub.mng.net (publicd.ub.mng.net [202.179.0.88]) by mx1.FreeBSD.org (Postfix) with ESMTP id B3A5B43D45; Thu, 1 Sep 2005 04:49:21 +0000 (GMT) (envelope-from ganbold@micom.mng.net) Received: from [202.179.0.164] (helo=ganbold.micom.mng.net) by publicd.ub.mng.net with esmtpa (Exim 4.43 (FreeBSD)) id 1EAhN2-000K4c-2Z; Thu, 01 Sep 2005 14:12:16 +0900 Message-Id: <6.2.1.2.2.20050901133026.03582b30@202.179.0.80> X-Mailer: QUALCOMM Windows Eudora Version 6.2.1.2 Date: Thu, 01 Sep 2005 13:49:16 +0900 To: freebsd-net@freebsd.org From: Ganbold Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Cc: glebius@FreeBSD.org Subject: ng_netflow/ipfw/bridge problems and Netflow best practices X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Sep 2005 04:49:22 -0000 Hi, I'm newbie to Netflow and I'm trying to use ng_netflow because it is fast and uses less CPU. I'm trying to collect Netflow traffic from FreeBSD 5.4 machine. Collector (flow-tools) runs on same machine. This FreeBSD has 3 interfaces and it acts as bridging firewall using IPFW2. It also uses dummynet. host# uname -an FreeBSD machine.mng.net 5.4-STABLE FreeBSD 5.4-STABLE #4: Fri Aug 12 09:58:18 ULAST 2005 tsgan@machine.mng.net:/usr/obj/usr/src/sys/PRXY i386 host# ifconfig xl0: flags=8943 mtu 1500 media: Ethernet 100baseTX status: active xl1: flags=8943 mtu 1500 media: Ethernet 100baseTX status: active vr0: flags=8843 mtu 1500 inet x.x.x.x netmask 0xffffffe0 broadcast x.x.x.x media: Ethernet autoselect (100baseTX ) status: active I'm running ng_netflow module and ngctl with following parameters to catch both incoming and outgoing traffic: ngctl mkpeer xl1: tee lower right ngctl connect xl1: xl1:lower upper left ngctl name xl1:lower xl1_tee ngctl mkpeer xl1_tee: netflow left2right iface0 ngctl name xl1:lower.left2right netflow ngctl connect xl1_tee: netflow: right2left iface1 ngctl msg netflow: setifindex { iface=0 index=2 } ngctl msg netflow: setifindex { iface=1 index=1 } ngctl mkpeer netflow: ksocket export inet/dgram/udp ngctl msg netflow:export connect inet/127.0.0.1:8818 ngctl mkpeer xl0: tee lower right ngctl connect xl0: xl0:lower upper left ngctl name xl0:lower xl0_tee ngctl mkpeer xl0_tee: netflow left2right iface2 ngctl name xl0:lower.left2right netflow0 ngctl msg netflow0: setifindex { iface=2 index=4 } ngctl connect xl0_tee: netflow0: right2left iface3 ngctl msg netflow0: setifindex { iface=3 index=3 } ngctl mkpeer netflow0: ksocket export inet/dgram/udp ngctl msg netflow0:export connect inet/127.0.0.1:8818 However I have 2 issues. 1. Firewall dynamic rules count almost doubles when starts ng_netflow traffic. 2. Firewall behaves abnormally, customers complained that they couldn't connect to Internet. Is this known issue? How can I fix those? I rebooted firewall and I tried following: ngctl mkpeer xl1: tee lower left ngctl connect xl1: xl1:lower upper right ngctl mkpeer xl1:lower one2many left2right many0 ngctl connect xl1:lower.left2right xl1:lower many1 right2left ngctl name xl1:lower.right2left o2m ngctl mkpeer o2m: netflow one iface0 ngctl name o2m:one netflow ngctl mkpeer netflow: ksocket export inet/dgram/udp ngctl msg netflow:export connect inet/127.0.0.1:8818 Same problems as before I had after that. I don't know yet how to solve these problems. Can somebody in this list help me to solve above problems? Maybe somebody already had these issues and solved already. Afterwards I tried softflowd and it is working fine except it adds 5% overhead to CPU. That is why I prefer ng_netfow instead of softflowd. I'm using flow-tools and flowscan to collect traffic and make report using CUflow. Is there any better way to make nice graphs and reports? What other tools should I try? What is the best practice? I appreciate if somebody can give me some hints and advices. It would be great if someone can share configuration samples and best practices. thanks in advance, Ganbold