Date: Sun, 15 Apr 2007 11:53:58 +0300 From: Diomidis Spinellis <dds@aueb.gr> To: Robert Watson <rwatson@FreeBSD.org> Cc: arch@freebsd.org, re@freebsd.org Subject: Re: Accounting changes Message-ID: <4621E826.6050306@aueb.gr> In-Reply-To: <20070414170218.M76326@fledge.watson.org> References: <461958CC.4040804@aueb.gr> <20070414170218.M76326@fledge.watson.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Robert Watson wrote:
>
> On Mon, 9 Apr 2007, Diomidis Spinellis wrote:
>
>> I'd like your permission to perform a change during the slush period.
>>
>> Following some gentle prodding by bde, I'm working on an overhaul of
>> the accounting system. The reason for the change is that time values
>> with AHZ == 64 (1/64 second) are very often zero on modern machines.
>> The change involves storing acct(5) time values as proper IEEE-487
>> floats with AHZ being 1000000 (i.e storing \mu s). For the change
>> I'll touch the following files/directories:
>>
>> sys/sys/acct.h
>> sys/kern/kern_acct.c
>> usr.sbin/sa
>> usr.bin/lastcomm
>> share/man/man5/acct.5
>> UPDATING
>>
>> The change will render old daily accounting files incompatible with
>> the new commands and formats. Because the summary databases don't
>> store any version information, the old versions of these will also
>> become unusable. Therefore users will need to clear
>> /var/account{acct*,savacct,usracct}. Apparently, this problem has
>> occurred multiple times in the past, when we changed the size of
>> elements like uid_t and dev_t.
>
> What do you think of the idea of changing the file format a little to
> include a short file header at the front, and that the first field of
> that head is zero-filled u_int32_t, and the second a version number?
> Right now, the first field of the acct structure is the name of the
> command, which will always be a non-nul string, so always have a first
> character non-nul. If we see non-nul data in the file header's first
> field, we use the old structure layout, and otherwise we check the
> version number and use the new layout? This would provide backwards
> compatibility for reading old accounting data, which I would think would
> generally be desirable, and allow us to explicitly version the file in
> the future.
There are three types of accounting files: acct(5), which contains
records written by the kernel, and two others containing sa(8)
aggregated results of acct per user and per command. I'd prefer to keep
acct(5) free of version data. This 1) simplifies the kernel, which
would otherwise have to write version data each time the file is turned
over, 2) is consistent with current practice for storing such files, and
3) simplifies the processing programs, which can process data at
arbritrary locations in the file. (lastcomm reads records backwards, and
recently I updated it to also work on the output of tail -f).
Typically, these files are aggregated and turned over daily; IMO doing
this also over an OS upgrade isn't a big deal.
The aggregated files are different. They are dbm-based, and their
structure is opaque. I could therefore easily add a special "file
version" record to them. My concern is how to handle multiple file
versions over a long time period. Hard-coding legacy file formats in
the code is inelegant. I'd prefer to add a text-based import/export
capability, so that administrators can migrate data by exporting it
before an OS upgrade and importing it afterwords.
Even that could be an overkill. Based on the problems Bruce and I have
seen in the accounting system, my impression is that not many people use
it nowadays. Once/if we get dtrace in FreeBSD, I'd prever to scrap t
existing accounting infrastructure and re-implement it on top of that.
Diomidis - http://www.spinellis.gr
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4621E826.6050306>