From owner-freebsd-questions@FreeBSD.ORG Mon Dec 12 15:23:21 2005 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B368416A424 for ; Mon, 12 Dec 2005 15:23:21 +0000 (GMT) (envelope-from robert@webtent.com) Received: from columbus.webtent.org (static-70-110-70-42.tampfl.fios.verizon.net [70.110.70.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id E50B443D69 for ; Mon, 12 Dec 2005 15:23:08 +0000 (GMT) (envelope-from robert@webtent.com) Received: from localhost (localhost [127.0.0.1]) by linux.local (Postfix) with ESMTP id 3C34CCFAB5 for ; Mon, 12 Dec 2005 09:19:20 -0500 (EST) From: Robert Fitzpatrick To: FreeBSD Content-Type: text/plain Organization: WebTent Networking, Inc. Date: Mon, 12 Dec 2005 09:19:19 -0500 Message-Id: <1134397159.6334.21.camel@columbus> Mime-Version: 1.0 X-Mailer: Evolution 2.2.1 Content-Transfer-Encoding: 7bit Subject: Reaching kern.maxfiles X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2005 15:23:21 -0000 Been running this FreeBSD 5.2.1 server since 5.2 was released, never had an issue couldn't resolve, especially with the help of this and other lists. Now I'm stumped. I posted this issue last week, someone suggested I look at: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/configtuning-kernel-limits.html The problem with reaching kern.maxfiles started immediately after doing several upgrades including Perl 5.8.4 to 5.8.7 and other minor level upgrades - Postfix 2.2.3 to 2.2.6, amavisd-new 2.2 to 2.3, SpamAssassin 3.0 to 3.1 and others. Since stopping and starting Postfix every 10 minutes alliviates the issue, I portdowngraded Postfix, amavisd-new and SpamAssassin to previous packages and still reach the kern.maxfiles of over 12000 within minutes. Anyone have an inclination of what I can do to track down the cause of this issue? Do I need to increase kern.maxfiles (but why) or am I going to have to reinstall FreeBSD (I thought I'd never say that to fix a problem, like Winblows). I have already checked all partitions where a lot of issues were found on /var, but went to single user mode and repaired it to a clean state, disks (RAID 5) all seemed fine. The lsof command shows mostly smtp and smtpd processes with open files during this issue, but a lot of postfix clean up processes with open files to various libraries. This is just a snippet of the 11000 files open at one time, but most entries are like these: smtpd 10522 root 111u KQUEUE 0xc756f900 count=0, state=0 smtpd 10522 root 112u KQUEUE 0xc728b000 count=0, state=0 smtpd 10522 root 113u KQUEUE 0xc7c72100 count=0, state=0 smtpd 10522 root 114u KQUEUE 0xc7cb3a00 count=0, state=0 smtpd 10522 root 115u KQUEUE 0xca892800 count=0, state=0 smtpd 10522 root 116u KQUEUE 0xc7d89d00 count=0, state=0 smtpd 10522 root 117u KQUEUE 0xc72bdc00 count=0, state=0 smtpd 10522 root 118u KQUEUE 0xc7d5d500 count=0, state=0 cleanup 10530 postfix cwd VDIR 4,17 512 965640 /var/spool/postfix cleanup 10530 postfix rtd VDIR 4,13 512 2 / cleanup 10530 postfix txt VREG 4,16 250855 2499176 /usr/local/libexec/postfix/cleanup cleanup 10530 postfix txt VREG 4,13 135188 94210 /libexec/ld-elf.so.1 cleanup 10530 postfix txt VREG 4,16 97995 2246523 /usr/local/lib/libsasl2.so.2 cleanup 10530 postfix txt VREG 4,16 28036 1413244 /usr/lib/libpam.so.2 cleanup 10530 postfix txt VREG 4,13 32740 70658 /lib/libcrypt.so.2 cleanup 10530 postfix txt VREG 4,16 259651 2248530 /usr/local/lib/libssl.so.4 cleanup 10530 postfix txt VREG 4,16 1308218 2248522 /usr/local/lib/libcrypto.so.4 cleanup 10530 postfix txt VREG 4,16 969676 2496614 /usr/local/lib/db42/libdb-4.2.so.2 cleanup 10530 postfix txt VREG 4,16 241030 2241407 /usr/local/lib/libldap-2.2.so.7 cleanup 10530 postfix txt VREG 4,16 61071 2241404 /usr/local/lib/liblber-2.2.so.7 cleanup 10530 postfix txt VREG 4,16 53192 2242521 /usr/local/lib/libpcre.so.0 cleanup 10530 postfix txt VREG 4,13 892348 70667 /lib/libc.so.5 cleanup 10530 postfix txt VREG 4,16 111336 1415475 /usr/lib/libc_r.so.5 cleanup 10530 postfix txt VREG 4,16 204560 1413283 /usr/lib/libssl.so.3 cleanup 10530 postfix txt VREG 4,13 1106720 70679 /lib/libcrypto.so.3 cleanup 10530 postfix txt VREG 4,16 47428 2237797 /usr/local/lib/nss_ldap.so.1 cleanup 10530 postfix 0u VCHR 2,2 0t0 8 /dev/null cleanup 10530 postfix 1u VCHR 2,2 0t0 8 /dev/null cleanup 10530 postfix 2u VCHR 2,2 0t0 8 /dev/null cleanup 10530 postfix 3u PIPE 0xc8864e1c 16384 ->0xc7ac9810 cleanup 10530 postfix 4u PIPE 0xc7ac9810 16384 ->0xc8864e1c cleanup 10530 postfix 5u PIPE 0xc72160ac 16384 ->0xc6e1d2b0 cleanup 10530 postfix 6u unix 0xc71180f0 0t0 public/cleanup cleanup 10530 postfix 7u unix 0xc71614b0 0t0 ->0xc6d674b0 cleanup 10530 postfix 8u VREG 4,17 0 965683 /var/spool/postfix/pid/unix.cleanup cleanup 10530 postfix 9u unix 0xcad4d2d0 0t0 public/cleanup cleanup 10530 postfix 10r VREG 4,16 49152 2386167 /usr/local/etc/postfix/transport.map.db cleanup 10530 postfix 11r VREG 4,16 49152 2386167 /usr/local/etc/postfix/transport.map.db cleanup 10530 postfix 12u IPv4 0xc71648a0 0t0 TCP directory.webtent.net:60741->directory.webtent.net:ldap (ESTABLISHED) cleanup 10530 postfix 13u VREG 4,17 0 966093 /var/spool/postfix/incoming/632A6EBDCD cleanup 10530 postfix 14u unix 0xc7235690 0t0 ->0xc7217a50 cleanup 10531 postfix cwd VDIR 4,17 512 965640 /var/spool/postfix cleanup 10531 postfix rtd VDIR 4,13 512 2 / cleanup 10531 postfix txt VREG 4,16 250855 2499176 /usr/local/libexec/postfix/cleanup cleanup 10531 postfix txt VREG 4,13 135188 94210 /libexec/ld-elf.so.1 cleanup 10531 postfix txt VREG 4,16 97995 2246523 /usr/local/lib/libsasl2.so.2 cleanup 10531 postfix txt VREG 4,16 28036 1413244 /usr/lib/libpam.so.2 cleanup 10531 postfix txt VREG 4,13 32740 70658 /lib/libcrypt.so.2 cleanup 10531 postfix txt VREG 4,16 259651 2248530 /usr/local/lib/libssl.so.4 cleanup 10531 postfix txt VREG 4,16 1308218 2248522 /usr/local/lib/libcrypto.so.4 cleanup 10531 postfix txt VREG 4,16 969676 2496614 /usr/local/lib/db42/libdb-4.2.so.2 Clues? -- Robert