Date: Mon, 17 Nov 2003 16:09:35 -0500 From: Damian Gerow <damian@sentex.net> To: isp@freebsd.org Subject: Re: Daily/weekly/monthly output aggregation Message-ID: <20031117210935.GK98840@sentex.net> In-Reply-To: <6.0.0.22.0.20031117154856.01b4eb58@pop.face2interface.com> References: <20031117203641.GG98840@sentex.net> <20031117204102.GI61630@complx.LF.net> <6.0.0.22.0.20031117154856.01b4eb58@pop.face2interface.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Thus spake Marty Landman (MLandman@face2interface.com) [17/11/03 15:57]:
> As a developer I'd like to throw my 2 cents in; although this stmt may come
> as no news to anyone else imho the issue is what to parse out as
> significant. With the underlined caveat that once you make (what's in
> essence then) a policy decision about what system output is significant
> enough to pass along to the admin as worthy of review the danger is in
> everything that /isn't/ passed along.
Developer input is what I need at this point -- I have done development work
in the past, but I very quickly moved into sysadmin work.
> At least now you've got the gnawing feeling that you're behind in reading
> the stuff; once you implement a system to decide what's worth reading
I put 'read' in quotes, because I usually give each one a ten-second
once-over. 75% of the time, that's good enough, but I have missed more than
a couple of problems that I shouldn't have.
> you've gotten rid of that guilt pang. Should that evolve into a sense of
> false security - well I can only speculate how many server crashes could've
> been avoided if not for feelings of false security.
Being security-concious, this is a big concern. Hence, my paper-napkin
draft of what needs to be done:
Everything gets stored in a SQL database, since it is the cure to any and
every computing problem that has ever been introduced.
Store a table of hostnames, and whether or not they are active. When we run
the report generator, we can check to see if a hostname did *not* check in.
If not, we send an alert.
Each report is mailed to an address, that pipes the message to a program.
This program would break each report down into its already-labelled
sections, and store it *verbatim* in the database. This makes looking up
past reports much, much easier.
The report generator would be run via a cron job. The idea at this point is
to:
- make sure all currently active servers have checked in, with the
appropriate reports
- detect any new servers that checked in
- do, essentially, a diff against today and yesterday for each host
(also do a diff against today and last week, when necessary)
- if no changes, pring a 'Host OK' status
- otherwise, print a line for every change.
The output of this would be one e-mail, that would be sent out however you
want it to be sent out.
I already have bigger ideas for this (i.e. paging if more that 'root/toor'
found with userid zero, paging if known hosts did not check in/unknown hosts
did check in, collision/error rate jumps too high, etc.), but I'd like to
avoid feature creap for now.
Any thoughts/suggestions/comments?
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20031117210935.GK98840>