From owner-freebsd-ports@FreeBSD.ORG Mon Sep 28 11:30:20 2009 Return-Path: Delivered-To: ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15612106566B; Mon, 28 Sep 2009 11:30:20 +0000 (UTC) (envelope-from zvladov@gmail.com) Received: from mail-fx0-f222.google.com (mail-fx0-f222.google.com [209.85.220.222]) by mx1.freebsd.org (Postfix) with ESMTP id 6AB278FC1E; Mon, 28 Sep 2009 11:30:19 +0000 (UTC) Received: by fxm22 with SMTP id 22so3918373fxm.36 for ; Mon, 28 Sep 2009 04:30:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=uuXOrH38yKWrfQjtoxpZ1dqKgm/FcGMh3/Tq07fZ49U=; b=N48jE8MWvOlyFx4so7oJql/zc3YbWkgGSVlWcA1OUpCBHGKxJXKr4P4//s1epe1OIP fEh/U1iul25dJEGGcj7IR9Ov5xu70Tamf95AgHvyO7DbzPkW2JQg3ED3bthIwBEwIf04 isVqwnhaFvwcMbGIaGPpIwJFz7MaJJ80EH1mA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=X/KBwqu6bhvjfHM3rXYjDGobZ31WVkwDoLR6EtXgFSwQkd14wjwdMY+ECNnkbjadma HIRepC24aQpKU/bUYRXSrodMCTJJFB/rddXwjCY1e03Z20iHDsO19gq58GucBw/qyQoT TfyoiBTVQt8cU2fmsrCQZmsHARwDhFiUhn5Yk= MIME-Version: 1.0 Received: by 10.204.25.152 with SMTP id z24mr3020963bkb.44.1254137417844; Mon, 28 Sep 2009 04:30:17 -0700 (PDT) In-Reply-To: <20090927161137.4becc379.stas@FreeBSD.org> References: <20090927161137.4becc379.stas@FreeBSD.org> Date: Mon, 28 Sep 2009 14:30:17 +0300 Message-ID: From: Zvezdelin Vladov To: Stanislav Sedov Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: ports@freebsd.org, theraphim@gmail.com, anikina@gmail.com, zhnichkov@gmail.com Subject: Re: Flow-tools and Flow-tools-ng - Flow-Capture - supposed to be memory leak - patch X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Sep 2009 11:30:20 -0000 Hello Stanislav, According to the author of the patch: "Paul, Upon further investigation, it appears as if there is an alignment issue in 'struct msgip' inside of 'struct ftnet'. I've looked at some other code, a= nd there appear to be a handful of useful CMSG_* macros to help deal with this= . RFC 2292 details them. The attached patch works cleanly on my system and fixes the issue at hand. Hopefully it doesn't break other platforms. It's there for anybody who is interested. Thanks, Mark" I'll try to describe what I saw from the debugging of the flow-tools code (not the NG branch, but the regular one) and from the patch provided. That is how I understand it. So, in the first - original source of the flow-tools code, there were some structures that (re)declared the IP header, and dealt with it directly. While this worked for the IPv4, 32bit code, the same actions applied to 64code base, doesn't work. And the whole code, that finds different exporters, doesn't work. The whole scene is changed then - while you should have at most say - 10-60 different exporters, now because the IP header isn't decoded correctly you have almost millions of different exporters, and for each of these exporters the code takes memory. And it starts to look like a regular memory leak -- but in reality it is totally different problem - IP header decoding problem. And In the patch, there is a new method deployed to describe and decode the IP header, and the whole method is implemented by the regular work with specially purposed macro-es -- taken from RFC 2299 -- Advanced Sockets API for IPv6 (1998). So, in my opinion, it is perfectly suitable patch for the FreeBSD to include it. After all it does what it really has to do - to patch a problem with the IP header decoding. And patch it the portable way. Hope this helps. BR, Z.Vladov Of course, you may think differently. On Sun, Sep 27, 2009 at 3:11 PM, Stanislav Sedov wrote: > On Sat, 15 Aug 2009 10:04:46 +0300 > Zvezdelin Vladov mentioned: > >> Dear Sir/Madam, >> >> Please, publish in the official ports build patch system, the patch >> bellow for the problem >> that manifest itself only on the amd64 platform, and it is one and the >> same for both the >> flow-tools and flow-tools-ng - i.e. flow-capture eats all of the RAM >> and SWAP of the >> machine, until killed by the kernel. >> >> The problem itself is in the built-in mechanism in the flow-tools,( >> instead of using the >> macros in the FreeBSD,) to read msg structures and control-information >> from recvmsg >> calls. >> >> I've managed to patch myself the source, but my patch is ugly one, and >> consist of >> just commenting out the usage of the hack-in tools to read msg structure= s of the >> original source (but worked). >> >> The patch I am pasting here is from the author, and concerns the same pr= oblem. >> It looks much better than mine, and If I've new about it, I would be >> spending 4 days >> debugging....;-( >> > > Hello, Zvezdelin! > > Can you elaborate a bit more on the patch contents? =A0I'm not entirely s= ure > what it does and furthermore flow-tools source code does not contain any > references to the CMSG_XXX macros, so it is effectively a no-op for > net/flow-tools. =A0It might work for flow-tools-ng though. > > It'd be helpful to have a small description of what these changes does, > so it will be possible to elaborate the same fix for net/flow-tools. > > -- > Stanislav Sedov > ST4096-RIPE >