Date: Mon, 26 May 2008 22:27:37 +0200 From: "Mathieu Prevot" <mathieu.prevot@gmail.com> To: "Marc Spitzer" <mspitzer@gmail.com> Cc: hackers@freebsd.org, Dan Nelson <dnelson@allantgroup.com> Subject: Re: network bitrate of a poll of processes Message-ID: <3e473cc60805261327g38349857ga87dcfa0b855c1f6@mail.gmail.com> In-Reply-To: <8c50a3c30805261237y6573a2c9ra4af56477d93af78@mail.gmail.com> References: <3e473cc60805260752o2f573cf2h12910a45cf6849e6@mail.gmail.com> <20080526151614.GA5582@dan.emsphone.com> <3e473cc60805260914r537f113eh8d840a94c0a5d958@mail.gmail.com> <8c50a3c30805261237y6573a2c9ra4af56477d93af78@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
2008/5/26 Marc Spitzer <mspitzer@gmail.com>: > On Mon, May 26, 2008 at 12:14 PM, Mathieu Prevot > <mathieu.prevot@gmail.com> wrote: >> 2008/5/26 Dan Nelson <dnelson@allantgroup.com>: >>> In the last episode (May 26), Mathieu Prevot said: >>>> Hi, >>>> >>>> I would like to know the bitrate of a pool of child processes that use >>>> a network connection, how can I have something like netstat -w1 >>>> provide but at the process level ? >>> >>> If you can segregate them to their own UID, you can use an ipfw "count >>> ip from any to any uid 6666" rule to count the packets. Another option >>> would be to jail them to a dedicated IP address and count traffic on >>> that IP. >> >> Both are nice... thank you. And don't you have something more portable >> ie. that could run on "any" posix os - like a sh script that use >> standard tools - and that doesn't need root user intervention ? >> >> Mathieu > > I do not think so, firewall for example is controlled by root. And > users do not generally have detailed access to other users > information. You can look into the BASM framework to track what your > apps are doing and if you know the behavior to expect you can use > netflow/argus/tcpdump to track that and post process the logs in each > case into something useful. > > What are you trying to do, details do help? > > marc I have a script that generates many child processes that download videos (~10MB) in this case, or do heavy data processing/analysing. I can tune the number of workers as I want, but It's a shared cluster so I would like to 1) measure 2) optimize/control (by number of workers ? at the script level ! so I can update thing very dynamically) a) CPU load b) IO on drive(s) c) network load and I prefer solutions that don't need root access/intervention. I can measure indirectly network load using `date +%s` and `du -b |sed 's/\t.*//' ` regularly... but I may have to deal with several 10GB later - and actually the number of files is more critical than their size. Maybe writing a (very) quick and dirty python app that listen/attach to a group of processes (for the network case) ? Mathieu
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3e473cc60805261327g38349857ga87dcfa0b855c1f6>