From owner-freebsd-current Sun Mar 21 20:38:59 1999 Delivered-To: freebsd-current@freebsd.org Received: from detlev.UUCP (52-sweet.camalott.com [208.239.153.52]) by hub.freebsd.org (Postfix) with ESMTP id 5FD9C15171 for ; Sun, 21 Mar 1999 20:38:39 -0800 (PST) (envelope-from joelh@gnu.org) Received: (from joelh@localhost) by detlev.UUCP (8.9.3/8.9.1) id WAA00694; Sun, 21 Mar 1999 22:38:17 -0600 (CST) (envelope-from joelh@gnu.org) Date: Sun, 21 Mar 1999 22:38:17 -0600 (CST) Message-Id: <199903220438.WAA00694@detlev.UUCP> X-Authentication-Warning: detlev.UUCP: joelh set sender to joelh@gnu.org using -f To: freebsd-current@freebsd.org Subject: syslogd meets The Sorcerer's Apprentice From: Joel Ray Holveck Reply-To: joelh@gnu.org Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I'd like to add safeguards to keep syslogd from cascading its own error messages. To describe more fully: I just came back from a weekend getaway and discovered my crash box screaming bloody murder. I haven't had any odd experiments running for quite some time now. A quick look at the top of /var/log/messages.0 showed: Mar 18 00:00:00 detlev newsyslog[543]: logfile turned over Mar 21 15:44:25 detlev syslogd: /dev/ttyv1: Too many open files in system Mar 21 15:44:25 detlev syslogd: /dev/ttyv2: Too many open files in system Mar 21 15:44:25 detlev syslogd: /dev/ttyp0: Too many open files in system Mar 21 15:44:25 detlev syslogd: /dev/ttyp2: Too many open files in system Mar 21 15:44:25 detlev syslogd: /dev/ttyp3: Too many open files in system Mar 21 15:44:25 detlev syslogd: /dev/console: Too many open files in system: Too many open files in system Mar 21 15:44:25 detlev syslogd: /dev/ttyv1: Too many open files in system Mar 21 15:44:25 detlev syslogd: /dev/ttyv2: Too many open files in system Mar 21 15:44:25 detlev syslogd: /dev/ttyp0: Too many open files in system Mar 21 15:44:25 detlev syslogd: /dev/ttyp2: Too many open files in system Mar 21 15:44:25 detlev syslogd: /dev/ttyp3: Too many open files in system Mar 21 15:44:25 detlev /kernel: file: table is full Mar 21 15:44:25 detlev last message repeated 41 times Mar 21 15:44:25 detlev sendmail[17482]: PAA17481: SYSERR(UID0): Can't create transcript file xfPAA17482: Too many open files in system Mar 21 15:44:25 detlev sendmail[17482]: PAA17481: SYSERR(UID0): Can't open /dev/null: Too many open files in system Mar 21 15:44:26 detlev syslogd: /var/run/utmp: Too many open files in system Mar 21 15:44:26 detlev syslogd: /var/run/utmp: Too many open files in system Mar 21 15:44:26 detlev /kernel: file: table is full Mar 21 15:44:26 detlev syslogd: /var/run/utmp: Too many open files in system Mar 21 15:44:26 detlev last message repeated 3 times Mar 21 15:44:26 detlev /kernel: file: table is full Mar 21 15:44:26 detlev syslogd: /var/run/utmp: Too many open files in system Mar 21 15:44:26 detlev last message repeated 3 times After that it settled down to just repeating: Mar 21 15:45:30 detlev syslogd: /dev/ttyv1: Too many open files in system Mar 21 15:45:30 detlev syslogd: /dev/ttyv2: Too many open files in system Mar 21 15:45:30 detlev syslogd: /dev/ttyp0: Too many open files in system Mar 21 15:45:30 detlev syslogd: /dev/ttyp2: Too many open files in system Mar 21 15:45:30 detlev syslogd: /dev/ttyp3: Too many open files in system Mar 21 15:45:30 detlev /kernel: file: table is full Evidently it was recording about 125 log messages per second. I came home to discover a full /var, a 25 MB uncompressed messages.0, no /var/log/messages at all, and overall a fairly unhappy computer. (Evidently, when newsyslog ran, things were already hosed. It could move messages.0, but not gzip it or create /var/log/messages.) This is all from a March 13 build, I believe, if it matters. MAXUSERS is set to 20, and there were probably six logins including a small X session at the time. All had been idle for hours. (I figure something was leaking fd's bad.) Now, I don't know what set this off. I moved the messages.0 to another filesystem, rebooted, recreated /var/log/messages, and restarted syslogd. Like I say, I don't know what set this off. That's once concern, but will take more investigation. My concern is whether we should keep syslogd from going into Sorcerer's Apprentice mode like this. What do you think about adding safeguards against syslogd logging more than, say, thirty messages per hour saying why can't log messages? Cheers, joelh -- Joel Ray Holveck - joelh@gnu.org Fourth law of programming: Anything that can go wrong wi sendmail: segmentation violation - core dumped To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message