From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 27 18:46:46 2006 Return-Path: X-Original-To: hackers@freebsd.org Delivered-To: freebsd-hackers@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6BDC116A401 for ; Mon, 27 Mar 2006 18:46:46 +0000 (UTC) (envelope-from mwlucas@bewilderbeast.blackhelicopters.org) Received: from bewilderbeast.blackhelicopters.org (bewilderbeast.blackhelicopters.org [198.22.63.43]) by mx1.FreeBSD.org (Postfix) with ESMTP id E27B743D46 for ; Mon, 27 Mar 2006 18:46:45 +0000 (GMT) (envelope-from mwlucas@bewilderbeast.blackhelicopters.org) Received: from bewilderbeast.blackhelicopters.org (mwlucas@localhost [127.0.0.1]) by bewilderbeast.blackhelicopters.org (8.12.10/8.12.10) with ESMTP id k2RIkiWG058720; Mon, 27 Mar 2006 13:46:44 -0500 (EST) (envelope-from mwlucas@bewilderbeast.blackhelicopters.org) Received: (from mwlucas@localhost) by bewilderbeast.blackhelicopters.org (8.12.10/8.12.10/Submit) id k2RIkiRb058719; Mon, 27 Mar 2006 13:46:44 -0500 (EST) (envelope-from mwlucas) Date: Mon, 27 Mar 2006 13:46:44 -0500 From: "Michael W. Lucas" To: Maxim Konovalov Message-ID: <20060327184643.GA58674@bewilderbeast.blackhelicopters.org> References: <20060327160130.GA57689@bewilderbeast.blackhelicopters.org> <20F3E06D-5727-4531-A81B-DF64765D1564@SARENET.ES> <20060327173841.GA58274@bewilderbeast.blackhelicopters.org> <20060327214209.U87890@mp2.macomnet.net> <20060327181501.GA58448@bewilderbeast.blackhelicopters.org> <20060327222836.J89207@mp2.macomnet.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060327222836.J89207@mp2.macomnet.net> User-Agent: Mutt/1.4.1i X-Spam-Score: (0) X-Scanned-By: MIMEDefang 2.39 Cc: hackers@freebsd.org Subject: Re: syslogd not draining X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Mar 2006 18:46:46 -0000 On Mon, Mar 27, 2006 at 10:35:11PM +0400, Maxim Konovalov wrote: > [....] > > > > > >ns1/etc;netstat -s | grep full > > > > > >Warning: sysctl(net.inet6.ip6.rip6stats): No such file or directory > > > > > > 122066 dropped due to full socket buffers > > > > > >ns1/etc; > > > > > > > > > > > >I've doubled kern.ipc.maxsockbuf a couple of times now, and yet it > > > > > >still happens. > > > > > > That's not enough. You need to teach syslogd to use this new value. > > > > I don't see this in syslogd(8); I presume it require source hacking? > > Yes. OK, I'm going to avoid that for the moment. I haven't touched C in five years now, I'd probably confuse it even worse. Besides, I've had centralized logging hosts with this much activity -- and far more -- previously. I can't believe that this environment is so special that it requires that sort of customization. > [...] > > > netstat -sp udp | grep 'datagrams received'; sleep 10; \ > > > netstat -sp udp | grep 'datagrams received' > > > > 158169 dropped due to full socket buffers > > 2467251 datagrams received > > sleeping... > > 158903 dropped due to full socket buffers > > 2468299 datagrams received > > ~100 datagrams per second, not a lot. Perhaps they are huge. Not that I've noticed. It's syslogd, DHCP, DNS, and flow-capture from a variety of devices, all generally small packets. > > > How much cpu time does syslogd use? > > > > Not much. ps -ax | grep syslog gives: > > > > 4317 ?? Ss 0:01.60 /usr/sbin/syslogd -l /var/run/log -l > > /var/named/var/run/log > > Try to remove a log socket for named and restart syslogd. Removed the named socket and restarted. We'll see what happens. > > Process has been running for about five minutes at that point. > > > > Another point that might be of interest: > > > > ns1/etc;rc.d/syslogd restart Stopping syslogd. Waiting for PIDS: > > 4317, 4317, 4317, 4317, 4317, 4317, 4317, 4317, 4317, 4317, 4317, > > 4317, 4317, 4317, 4317, 4317, 4317, 4317 Starting syslogd. > > What's the /var filesystem type? Something like gmirror? Nope. It's a big SATA drive with a swap partition at the top and the rest vanilla UFS2: ad4: 38146MB at ata2-master SATA150 ad5: 476940MB at ata2-slave SATA150 ns1~;mount /dev/ad4s1a on / (ufs, local) devfs on /dev (devfs, local) /dev/ad4s1d on /tmp (ufs, local, soft-updates) /dev/ad4s1e on /usr (ufs, local, soft-updates) /dev/ad4s1f on /home (ufs, local, soft-updates) /dev/ad5s1d on /var (ufs, local, soft-updates) devfs on /var/named/dev (devfs, local) > diff -u /etc/syslog.conf /usr/src/etc/syslog.conf? # $FreeBSD: src/etc/syslog.conf,v 1.28 2005/03/12 12:31:16 glebius Exp $ -#$Id: syslog.conf,v 1.11 2006/03/17 18:56:18 system_mwl Exp system_mwl $ # # Spaces ARE valid field separators in this file. However, # other *nix-like systems still insist on using tabs as field # separators. If you are sharing this file between systems, you # may want to use only tabs as field separators here. # Consult the syslog.conf(5) manpage. -*.err;kern.warning;auth.notice;mail.crit;local4.none /var/log/console.log -#*.err;kern.warning;auth.notice;mail.crit;local4.none /dev/console -*.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err;local0.none;local 1.none;local2.none;local3.none;local4.none;local5.none;local6.none;local7.none / var/log/messages +*.err;kern.warning;auth.notice;mail.crit /dev/console +*.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err /var/log/message s security.* /var/log/security auth.info;authpriv.info /var/log/auth.log mail.info /var/log/maillog lpr.info /var/log/lpd-errs ftp.info /var/log/xferlog cron.* /var/log/cron -daemon.debug /var/log/daemon.debug *.=debug /var/log/debug.log *.emerg * -local0.* /var/log/router -local1.* /var/log/switch -#local2.* /var/log/kvm -#local 2-3 can be used -local4.* /var/log/pix -local5.* /var/log/vpn -local7.* /var/log/dhcpd # uncomment this to log all writes to /dev/console to /var/log/console.log #console.info /var/log/console.log # uncomment this to enable logging of all log messages to /var/log/all.log # touch /var/log/all.log and chmod it to mode 600 before it will work -*.* /var/log/all.log +#*.* /var/log/all.log # uncomment this to enable logging to a remote loghost named loghost #*.* @loghost # uncomment these if you're running inn @@ -40,5 +30,3 @@ *.* /var/log/slip.log !ppp *.* /var/log/ppp.log -!flow-capture -*.* /var/log/flow-capture -- Michael W. Lucas mwlucas@FreeBSD.org, mwlucas@BlackHelicopters.org http://www.BlackHelicopters.org/~mwlucas/ "The cloak of anonymity protects me from the nuisance of caring." -Non Sequitur