From owner-freebsd-net@FreeBSD.ORG Sat Aug 2 12:55:20 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2E18A106570D for ; Sat, 2 Aug 2008 12:55:20 +0000 (UTC) (envelope-from mtm@wubethiopia.com) Received: from dire.wubethiopia.com (j071.v.rootbsd.net [208.79.82.223]) by mx1.freebsd.org (Postfix) with ESMTP id DA1118FC18 for ; Sat, 2 Aug 2008 12:55:19 +0000 (UTC) (envelope-from mtm@wubethiopia.com) Received: from rogue.mike.lan (unknown [213.55.65.29]) by dire.wubethiopia.com (Postfix) with ESMTPSA id 301E94FDA22A; Sat, 2 Aug 2008 12:54:54 +0000 (UTC) Message-ID: <48945A79.50300@wubethiopia.com> Date: Sat, 02 Aug 2008 16:00:41 +0300 From: Mike Makonnen User-Agent: Thunderbird 2.0.0.12 (X11/20080323) MIME-Version: 1.0 To: Patrick Tracanelli References: <48918DB5.7020201@wubethiopia.com> <4891CD13.20600@freebsdbrasil.com.br> <48922E9D.1020507@elischer.org> <4893328C.2040105@freebsdbrasil.com.br> <4894447C.3000800@wubethiopia.com> In-Reply-To: <4894447C.3000800@wubethiopia.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Julian Elischer Subject: Re: Application layer classifier for ipfw X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2008 12:55:20 -0000 Mike Makonnen wrote: > Patrick Tracanelli wrote: >> >> To let you know of my current (real world) tests: >> >> - Wireless Internet Provider 1: >> - 4Mbit/s of Internet Traffic >> - Classifying default protocols + soulseek + ssh >> - Classifying 100Mbit/s of dump over ssh >> >> Results in: >> No latency added, very low CPU usage, no packets dropping. >> >> - Wireless ISP 2: >> - 21 Mbit/s of Internet Traffic >> - Classifying default protocols + soulseek + ssh >> >> Results in: >> No tcp or udp traffic at all; everything that gets diverted never >> comes out of the divert socket, and ipfw-classifyd logs >> >> Aug 1 12:07:35 ourofino last message repeated 58 times >> Aug 1 12:17:54 ourofino ipfw-classifyd: Loaded Protocol: bittorrent >> (rule 50000) >> Aug 1 12:17:54 ourofino ipfw-classifyd: Loaded Protocol: edonkey >> (rule 50000) >> Aug 1 12:17:54 ourofino ipfw-classifyd: Loaded Protocol: fasttrack >> (rule 50000) >> Aug 1 12:17:54 ourofino ipfw-classifyd: Loaded Protocol: gnutella >> (rule 1000) >> Aug 1 12:17:54 ourofino ipfw-classifyd: Loaded Protocol: soulseek >> (rule 50000) >> Aug 1 12:17:54 ourofino ipfw-classifyd: Loaded Protocol: ssh (rule >> 50000) >> Aug 1 12:18:28 ourofino ipfw-classifyd: unable to write to divert >> socket: Operation not permitted >> Aug 1 12:18:50 ourofino last message repeated 90 times > > Hmmm... this part means that the call to sendto(2) to write the packet > back into network stack failed. This explains why you are not seein g > any traffic comming back out of the divert socket, but I don't see why > it would suddenly fail with a permission error. Could this be a kernel > bug? >> Aug 1 12:18:51 ourofino ipfw-classifyd: packet dropped: input queue >> full >> Aug 1 12:19:11 ourofino last message repeated 94 times >> >> Raised queue len a lot (up to 40960), when the application starts it >> uses up to 25% CPU and a second after that, CPU usage gets lower the >> 0.1%. > > This looks like a deadlock. If it weren't able to process packets fast > enough the cpu usage should be high even as it's spewing "packet > dropped" messages. Can you send me some more information like memory > usage and the firewall script you are using? How much of the 21Mbits/s > of traffic is P2P? If you reduce the number of protocols you are > trying to match against does the behavior change? Using netstat -w1 > -I can you tell me how many packets per second we're > talking about for 4Mbits/s and 21Mbit/s? Also, the timestamps from the > log file seem to show that the daemon is running for approx. 34 sec. > before the first "unable to write to write to divert socket" message. > Is it passing traffic during this time? Thanks. > > I've uploaded a newer version. Can you try that also please. It includes: > o SIGHUP forces it to re-read its configuration file > o rc.d script > o minor optimization (calls pthread_cond_signal with the mutex > unlocked) > o code cleanup > > Also, for your convenience I have attached a patch against the earlier > version that removes a debugging printf that should remove spammage to > your log files (the current version has it removed already). > Ooops, a few minutes after I sent this email I found a couple of bugs (one major, and one minor). They were in the original tarball as well as the newer one I uploaded earlier today. I've uploaded a fixed version of the code. Can you try that instead please. Also, to help track down performance issues I've modified the Makefile to build a profiled version of the application so you can use gprof(1) to figure out where any problems lie. Cheers. -- Mike Makonnen | GPG-KEY: http://people.freebsd.org/~mtm/mtm.asc mtm @ FreeBSD.Org | AC7B 5672 2D11 F4D0 EBF8 5279 5359 2B82 7CD4 1F55 FreeBSD | http://www.freebsd.org