From owner-freebsd-security@FreeBSD.ORG Fri Jan 14 07:54:47 2005 Return-Path: Delivered-To: freebsd-security@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9FFF416A4CE for ; Fri, 14 Jan 2005 07:54:47 +0000 (GMT) Received: from figg.securenet.com.au (ns2.isecure.com.au [202.125.4.72]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8678143D58 for ; Fri, 14 Jan 2005 07:54:42 +0000 (GMT) (envelope-from Stanley.Hopcroft@IPAustralia.Gov.AU) Received: from iron.securenet.com.au (iron.isecure.com.au [202.125.4.94]) j0E7sfMw014383 for ; Fri, 14 Jan 2005 18:54:41 +1100 Received: (from uucp@localhost) by iron.securenet.com.au (8.12.6/8.12.6) id j0E7sfDB012243 for ; Fri, 14 Jan 2005 18:54:41 +1100 (EST) Received: from nodnsquery(10.11.3.10) by iron.securenet.com.au via csmap (V6.0) id srcAAAWkaq6x; Fri, 14 Jan 05 18:54:41 +1100 Received: from vmail.aipo.gov.au (localhost [127.0.0.1]) j0E7seff001994 for ; Fri, 14 Jan 2005 18:54:41 +1100 Received: from stan.aipo.gov.au (wf-142.aipo.gov.au [192.168.1.142]) by vmail.aipo.gov.au (8.12.9p2/8.12.9) with ESMTP id j0E7scFb020212 for ; Fri, 14 Jan 2005 18:54:39 +1100 (EST) (envelope-from Stanley.Hopcroft@IPAustralia.Gov.AU) Received: from stan.aipo.gov.au (localhost [127.0.0.1]) by stan.aipo.gov.au (8.12.11/8.12.11) with ESMTP id j0E7scwt000276 for ; Fri, 14 Jan 2005 18:54:38 +1100 (EST) (envelope-from anwsmh@IPAustralia.Gov.AU) Received: (from anwsmh@localhost) by stan.aipo.gov.au (8.12.11/8.12.11/Submit) id j0E7sbSI000275 for freebsd-security@freebsd.org; Fri, 14 Jan 2005 18:54:37 +1100 (EST) (envelope-from anwsmh@IPAustralia.Gov.AU) X-Authentication-Warning: stan.aipo.gov.au: anwsmh set sender to anwsmh@IPAustralia.Gov.AU using -f Date: Fri, 14 Jan 2005 18:54:37 +1100 From: Stanley Hopcroft To: freebsd-security@freebsd.org Message-ID: <20050114075435.GA239@IPAustralia.Gov.AU> References: <200501131232.44441.mjohnston@skyweb.ca> <87wtug26a8.fsf@gray.impulse.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="----------=_1105689280-17516-261" Content-Disposition: inline In-Reply-To: <87wtug26a8.fsf@gray.impulse.net> User-Agent: Mutt/1.4.2.1i X-Scanned-By: MIMEDefang 2.48 on 10.0.100.191 Subject: Re: Aggregating logs from numerous FreeBSD machines X-BeenThere: freebsd-security@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Security issues [members-only posting] List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Jan 2005 07:54:47 -0000 This is a multi-part message in MIME format... ------------=_1105689280-17516-261 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Dear Folks, On Thu, Jan 13, 2005 at 04:39:11PM -0800, Ted Cabeen wrote: > Mark Johnston writes: > > > Hi folks, > > > > My stack of trusty FreeBSD servers always seems to be growing, and it's > > getting to the point where the daily and security output mail is too much to > > make good use of. I'm looking for suggestions for log monitoring and > > aggregation tools, especially from a monitoring-for-security perspective. > > .. snip .. > > syslog-ng is useful for separating incoming log entries by server, > facility and priority. I'd start with that. You could then use > something like logwatch or logcheck to mail you or trigger a nagios > warning on strange log lines. > a helpful way of looking at the problem may be 1 data collection/aggregation log forwarding is the way to go (there is free code to forward events from MS event logs to syslog [these are Win binaries] for collecting all events. Mr Cabeens suggestion of using the better classification of syslog-ng sounds very helpful on the host that is collecting the syslog'd events. 2 event correlation and or filtering. Programs like logsurfer and swatch can be used to react to simuli in the event stream (ie the logs being tailed) and react by forking shell scripts, mailing, highlighting the message on a viewer etc. The SourceForge project SEC can analyse multiple log files (the number is probably limited by the resources of your analysis/logging host) and do the above + process events (ie mesages that occur with a particular time sequence eg within an interval of one another, or after a message ...) SEC also does useful things such as compression (ie many stimuli one response). Actively developed. Junk free mail list. Mr John Rouillard gave a paper on SEC at the last LISA conference (Boston ?). SEC like Swatch is a Perl application and the rules can use arbitrary in-line Perl code. People use it for lots of things including real time Snort log analysis. Lastlu, I am not sure if the name is a conscious pun, but SEC is absolutely completely unrelated to the Tivoli TEC product. If you appreciate, TECs capabilities you'll do more with SEC and have more fun (unless you happen to love Prolog and rules based processing). Yours sincerely. -- Stanley Hopcroft IP Australia Ph: (02) 6283 3189 Fax: (02) 6281 1353 PO Box 200 Woden ACT 2606 http://www.ipaustralia.gov.au ------------=_1105689280-17516-261 Content-Type: text/plain; name="disclaimer.txt" Content-Disposition: inline; filename="disclaimer.txt" Content-Transfer-Encoding: 7bit MIME-Version: 1.0 X-Mailer: MIME-tools 5.415 (Entity 5.415) -- This message contains privileged and confidential information only for use by the intended recipient. If you are not the intended recipient of this message, you must not disseminate, copy or use it in any manner. If you have received this message in error, please advise the sender by reply e-mail. Please ensure all e-mail attachments are scanned for viruses prior to opening or using. ------------=_1105689280-17516-261--