From owner-freebsd-questions@FreeBSD.ORG Sat Feb 23 12:24:04 2008 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A1E4C16A401 for ; Sat, 23 Feb 2008 12:24:04 +0000 (UTC) (envelope-from fbsd.questions@rachie.is-a-geek.net) Received: from snoogles.rachie.is-a-geek.net (rachie.is-a-geek.net [66.230.99.27]) by mx1.freebsd.org (Postfix) with ESMTP id 5DFDB13C4E1 for ; Sat, 23 Feb 2008 12:24:04 +0000 (UTC) (envelope-from fbsd.questions@rachie.is-a-geek.net) Received: from localhost (localhost [127.0.0.1]) by snoogles.rachie.is-a-geek.net (Postfix) with ESMTP id 7DF4A1CC8B; Sat, 23 Feb 2008 03:24:03 -0900 (AKST) From: Mel To: freebsd-questions@freebsd.org Date: Sat, 23 Feb 2008 13:24:01 +0100 User-Agent: KMail/1.9.7 References: <47BFF649.9060104@jessikat.plus.net> In-Reply-To: <47BFF649.9060104@jessikat.plus.net> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200802231324.02203.fbsd.questions@rachie.is-a-geek.net> Cc: Robin Becker Subject: Re: duplicate message removal X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Feb 2008 12:24:04 -0000 On Saturday 23 February 2008 11:32:41 Robin Becker wrote: > Trouble is many of the emails get huge because of repeated messages > typically stuff like > > xxx.yyy.com login failures: > Feb 22 20:07:54 app3 sshd[56886]: reverse mapping checking getaddrinfo > for 216-194-26-66.ny.ny.metconnect.net failed - POSSIBLE BREAKIN ATTEMPT! > > etc etc > > All these servers are running denyhosts, but we still see lots of these > messages. > > I was wondering if there's any simple compression script which notices > the repetitions (apart from timestamp) and can remove the many > duplicates etc etc. Well, there's two things you can do: Attack the problem: Install sysutils/grok on the machines and start it with following config: file "/var/log/auth.log" { type "ssh-illegal-user" { match = "Invalid user %USERNAME% from %IP%"; threshold = 5; # 5 hits ... key = "%IP%"; # from a single ip ... interval = 60; # in 1 minutes reaction = "/sbin/pfctl -t sshscan -Tadd %IP%"; }; type "ssh-scan-possible" { match = "Did not receive identification string from %IP%"; threshold = 3; interval = 60; reaction = "/sbin/pfctl -t sshscan -Tadd %IP%"; }; }; If you replace: my $TAIL = "tail -0f"; with my $TAIL = "/usr/bin/tail -0F"; # capital F in grok, you will never have to look at it again. Else you will have to restart it whenever newsyslog(8) rotates the auth.log. Note that this uses pf, and assumes the table is blocking table. I'm sure it can be done with other firewalls as well. Attack the report: daily_status_security_loginfail_enable="NO" >>/etc/periodic.conf Then write your own and put it in /usr/local/etc/periodic/security/. I've written something similar with php for mail rejects, that groups sender or hostname and error message together, but any scripting language that you're comfy with should work. -- Mel