Date: Wed, 13 Aug 2008 14:59:45 -0400 From: Bill Moran <wmoran@potentialtech.com> To: Christopher Cowart <ccowart@rescomp.berkeley.edu> Cc: freebsd-questions@freebsd.org Subject: Re: Lots of accounting data Message-ID: <20080813145945.80d08986.wmoran@potentialtech.com> In-Reply-To: <20080813184603.GA25990@hal.rescomp.berkeley.edu> References: <20080813184603.GA25990@hal.rescomp.berkeley.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
In response to Christopher Cowart <ccowart@rescomp.berkeley.edu>:
> Hello,
>
> I'm playing a game of cat and mouse with process accounting and disk
> space. I built some boxes with 9GB /var partitions, rolled them into
> production, and after about 4 days of full load, /var filled up.
>
> Looking at the size of /var/account/acct{,.0}, and figuring I'd be
> seeing a 200% load increase in about a month, I created a new label from
> the large chunk of free space I saved for situations like this. 40GB
> mounted to /var/account: usage was down to 20%, and I thought the crisis
> was averted.
>
> About a week and a half later, I get a disk full e-mail from nagios and
> > +pid 94696 (gzip), uid 0 inumber 6 on /var/account: filesystem full
> in my dailies again. My /var/account/acct file was 17GB in size. Add one
> rotation before compression and I completely lose that feeling of
> cleverness I had when I gave accounting a dedicated 40GB partition.
>
> If you're wondering how I can possibly have this much accounting data,
> two `vmstat -f' invocations 100 seconds apart show 32282 forks (an
> average of 323 per second). These boxes are running squid with a
> redirect script to implement a captive portal. There are generally
> several hundred unauthenticated users; all of their http traffic, from
> firefox to the little weather widgets and spyware phoning home, gets
> proxied through squid and subsquently a redirect script that, among
> other things, does some text munging on the URL, and queries various
> ipfw tables to determine what "context" the user is in. Some of this
> could be optimized to launch fewer processes, but the code would be less
> maintainable.
>
> I only really see two options, neither of which I particularly like:
> * Throw more disk at the problem (but given what I've seen, I don't
> like the odds that within a month or two, I'll realize I didn't give
> it enough).
> * Turn off accounting on these boxes.
* Rotate and compress more frequently; and store less history?
--
Bill Moran
http://www.potentialtech.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080813145945.80d08986.wmoran>