Date: Sat, 15 Aug 2009 10:04:46 +0300 From: Zvezdelin Vladov <zvladov@gmail.com> To: ports@FreeBSD.org Cc: stas@freebsd.org, zhnichkov@gmail.com, anikina@gmail.com, theraphim@gmail.com Subject: Flow-tools and Flow-tools-ng - Flow-Capture - supposed to be memory leak - patch Message-ID: <c3a09b6b0908150004t5556522hbaa3933153ed09b3@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Dear Sir/Madam, Please, publish in the official ports build patch system, the patch bellow for the problem that manifest itself only on the amd64 platform, and it is one and the same for both the flow-tools and flow-tools-ng - i.e. flow-capture eats all of the RAM and SWAP of the machine, until killed by the kernel. The problem itself is in the built-in mechanism in the flow-tools,( instead of using the macros in the FreeBSD,) to read msg structures and control-information from recvmsg calls. I've managed to patch myself the source, but my patch is ugly one, and consist of just commenting out the usage of the hack-in tools to read msg structures of the original source (but worked). The patch I am pasting here is from the author, and concerns the same problem. It looks much better than mine, and If I've new about it, I would be spending 4 days debugging....;-( Best Regards, Z.Vladov Mark R. mark at inetu.net Tue Oct 7 14:53:09 EDT 2008 Previous message: [Flow-tools] ftpdu_seq_check lost flows Next message: [Flow-tools] limit for number of filter elements Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] -------------------------------------------------------------------------------- Paul, Upon further investigation, it appears as if there is an alignment issue in 'struct msgip' inside of 'struct ftnet'. I've looked at some other code, and there appear to be a handful of useful CMSG_* macros to help deal with this. RFC 2292 details them. The attached patch works cleanly on my system and fixes the issue at hand. Hopefully it doesn't break other platforms. It's there for anybody who is interested. Thanks, Mark On Tue, 7 Oct 2008, Mark R. wrote: > > Paul, > > This is where I'm picking up the garbage data: > > ftnet.loc_addr.sin_addr.s_addr = ftnet.msgip.ip.s_addr > > The kernel is GENERIC plus IPFW. Nothing terribly oddball. I'm wondering if > this is a variable type issue somewhere. Running a 32bit build on amd64 and > the same box doesn't exhibit this issue. > > If you care to look into this, I can provide access to the box in question. > If not, I'll might eventually figure it out. > > I'm not at the point of caring about a memory leak. When I get there, I can > run it through valgrind. > > Thanks, > Mark > > On Tue, 7 Oct 2008, Paul P Komkoff Jr wrote: > >> Replying to Mark R.: >>> Are there any known issues with flow-tools on 64-bit platforms? I'm >>> trying to run 0.68 on FreeBSD 7.0/amd64 and running into some odd >>> behavior with flow-capture and flow-fanout. >> >> There is at least one known memory leak that only happens on 64-bit >> FreeBSD, that I still haven't fixed (mainly because it requires to set up >> FreeBSD somewhere). >> >> This one that you telling here is new. And it's serious, I wonder why >> nobody else seeing it. Maybe your kernel is broken? >> Or maybe it's ipv4-over-ipv6 issue? >> I don't have access to freebsd system to verify how recvmsg works >> there, sorry. >> >> -- >> Paul P 'Stingray' Komkoff Jr // http://stingr.net/key <- my pgp key >> This message represents the official view of the voices in my head >> > _______________________________________________ > Flow-tools mailing list > flow-tools at splintered.net > http://mailman.splintered.net/mailman/listinfo/flow-tools -------------- next part -------------- --- lib/ftlib.h-orig 2008-03-09 13:09:01.000000000 +0000 +++ lib/ftlib.h 2008-10-07 17:41:11.000000000 +0000 @@ -459,11 +459,16 @@ int fd; /* fd receiving flows on */ struct mymsghdr msg; /* recvmsg data */ struct { - struct cmsghdr hdr; #ifdef IP_RECVDSTADDR +#ifdef CMSG_DATA + char cbuf[CMSG_SPACE(sizeof(struct sockaddr_storage))]; +#else + struct cmsghdr hdr; struct in_addr ip; +#endif /* CMSG_DATA */ #else #ifdef IP_PKTINFO + struct cmsghdr hdr; struct in_pktinfo pktinfo; #endif /* else */ #endif /* IP RECVDSTADDR */ --- src/flow-fanout.c-orig 2008-01-27 20:48:55.000000000 +0000 +++ src/flow-fanout.c 2008-10-07 17:42:33.000000000 +0000 @@ -124,6 +124,11 @@ int i, n, detach, one, ret, offset, hdr_len; int npeers, tx_delay; int stat_interval, stat_next, src_ip_spoof; +#ifdef IP_RECVDSTADDR +#ifdef CMSG_DATA + struct cmsghdr *cmsg; +#endif +#endif time_startup = time((time_t)0L); @@ -625,12 +630,24 @@ #ifdef IP_RECVDSTADDR /* got destination IP back? */ +#ifdef CMSG_DATA + for (cmsg = CMSG_FIRSTHDR(&ftnet.msg); cmsg != NULL; + cmsg = CMSG_NXTHDR(&ftnet.msg, cmsg)) { + if (cmsg->cmsg_level == IPPROTO_IP && + cmsg->cmsg_type == IP_RECVDSTADDR) { + memcpy(&ftnet.loc_addr.sin_addr.s_addr, + CMSG_DATA(cmsg), sizeof(struct in_addr)); + break; + } + } +#else if ((ftnet.msgip.hdr.cmsg_level == IPPROTO_IP) && (ftnet.msgip.hdr.cmsg_type == IP_RECVDSTADDR)) { ftnet.loc_addr.sin_addr.s_addr = ftnet.msgip.ip.s_addr; } else { ftnet.loc_addr.sin_addr.s_addr = 0; } +#endif /* CMSG_DATA */ #else #ifdef IP_PKTINFO if ((ftnet.msgip.hdr.cmsg_level == IPPROTO_IP) && --- src/flow-receive.c-orig 2008-01-27 20:48:55.000000000 +0000 +++ src/flow-receive.c 2008-10-07 17:42:35.000000000 +0000 @@ -93,6 +93,11 @@ char fmt_src_ip[32], fmt_dst_ip[32], fmt_dst_port[32]; char xl_rec[FT_IO_MAXREC], *out_rec; int stat_interval, stat_next; +#ifdef IP_RECVDSTADDR +#ifdef CMSG_DATA + struct cmsghdr *cmsg; +#endif +#endif time_startup = time((time_t)0L); @@ -488,12 +493,24 @@ #ifdef IP_RECVDSTADDR /* got destination IP back? */ +#ifdef CMSG_DATA + for (cmsg = CMSG_FIRSTHDR(&ftnet.msg); cmsg != NULL; + cmsg = CMSG_NXTHDR(&ftnet.msg, cmsg)) { + if (cmsg->cmsg_level == IPPROTO_IP && + cmsg->cmsg_type == IP_RECVDSTADDR) { + memcpy(&ftnet.loc_addr.sin_addr.s_addr, + CMSG_DATA(cmsg), sizeof(struct in_addr)); + break; + } + } +#else if ((ftnet.msgip.hdr.cmsg_level == IPPROTO_IP) && (ftnet.msgip.hdr.cmsg_type == IP_RECVDSTADDR)) { ftnet.loc_addr.sin_addr.s_addr = ftnet.msgip.ip.s_addr; } else { ftnet.loc_addr.sin_addr.s_addr = 0; } +#endif /* CMSG_DATA */ #else #ifdef IP_PKTINFO if ((ftnet.msgip.hdr.cmsg_level == IPPROTO_IP) && --- src/flow-capture.c-orig 2008-01-27 20:48:55.000000000 +0000 +++ src/flow-capture.c 2008-10-07 17:42:30.000000000 +0000 @@ -170,6 +170,11 @@ int stat_interval, stat_next, child_status; int v_flag; int preserve_umask; +#ifdef IP_RECVDSTADDR +#ifdef CMSG_DATA + struct cmsghdr *cmsg; +#endif +#endif time_startup = time((time_t)0L); @@ -621,7 +626,12 @@ ftnet.msg.msg_name = &ftnet.rem_addr; ftnet.msg.msg_namelen = sizeof ftnet.rem_addr; ftnet.msg.msg_control = &ftnet.msgip; + +#ifdef CMSG_DATA + ftnet.msg.msg_controllen = CMSG_LEN(sizeof(struct sockaddr_storage)); +#else ftnet.msg.msg_controllen = sizeof ftnet.msgip; +#endif while (1) { @@ -853,12 +863,24 @@ #ifdef IP_RECVDSTADDR /* got destination IP back? */ +#ifdef CMSG_DATA + for (cmsg = CMSG_FIRSTHDR(&ftnet.msg); cmsg != NULL; + cmsg = CMSG_NXTHDR(&ftnet.msg, cmsg)) { + if (cmsg->cmsg_level == IPPROTO_IP && + cmsg->cmsg_type == IP_RECVDSTADDR) { + memcpy(&ftnet.loc_addr.sin_addr.s_addr, + CMSG_DATA(cmsg), sizeof(struct in_addr)); + break; + } + } +#else if ((ftnet.msgip.hdr.cmsg_level == IPPROTO_IP) && (ftnet.msgip.hdr.cmsg_type == IP_RECVDSTADDR)) { ftnet.loc_addr.sin_addr.s_addr = ftnet.msgip.ip.s_addr; } else { ftnet.loc_addr.sin_addr.s_addr = 0; } +#endif /* CMSG_DATA */ #else #ifdef IP_PKTINFO if ((ftnet.msgip.hdr.cmsg_level == IPPROTO_IP) &&
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?c3a09b6b0908150004t5556522hbaa3933153ed09b3>