Date: Thu, 11 Mar 2004 22:12:25 -0500 From: "Justin Baugh, KSC" <baughj@discordians.net> To: freebsd-questions@freebsd.org Subject: Pernicious problem with vfork / qmail / qmail-scanner Message-ID: <40512A99.3050508@discordians.net>
next in thread | raw e-mail | index | archive | help
Hello list,
I am having a hell of a time with qmail/qmail-scanner, and I am hoping
someone on the list can help me. Normally this setup has worked
completely fine on any FreeBSD machine I've set it up on.
This is on 5.1-p10 (which I'm still running as I've yet to upgrade to
5.2.1R, and I am wondering if that just might be the problem).
I am running a large mailserver which feeds the message to
qmail-scanner, and then to spamd & a virus scanner (Kaspersky). This is
a very beefy box (dual 2.4ghz Xeon, 2gb RAM) which should be able to
handle many, many connections.
My connection limits on tcpserver are set to 60 concurrent connections.
concurrencyincoming/remote/local are all appropriate for this value.
When I suddenly get a number of connections (5-10), or when there are
more than 15-20 concurrent sessions in progress, I begin to get an error
message from tcpserver and also from qmail-scanner: "Cannot fork:
Resource temporarily unavailable".
Using truss, I see that the processes are just sitting there doing
vfork/nanosleep over and over and over, in a "Resource temporarily
unavailable" loop. Once the processes get into this state, they don't
get out of it, and it seems to build up over time (??) - i.e. more and
more forking errors until everything is hosed. The only way to fix it is
to kill off all the qmail-scanner processes.
I have tried messing with every sysctl value I know to change. I've
checked kern.maxproc, kern.maxfiles, kern.maxfilesperproc,
kern.maxprocperuid, etc. I've also looked at kern.somaxconn, etc, mostly
just shooting in the dark. All of these are at sane limits and I am
nowhere near the limits on open files or the number of processes. I've
also checked the max datasize (kern.maxdsize), etc etc. I am nowhere
near the maximum of open files or processes (both of which are
insanely high: maxusers = 384).
At the moment I'm not even using softlimit in the qmail startup script
in order to eliminate it as a cause. tcpserver is set to allow 60
concurrent qmail processes; the problems start to occur at about 20
processes, or when there is a very sudden increase in the number of
connections At the moment there are no set limits on resource
consumption for tcpserver or its children - at least not ones that I am
setting!
Return of limits
Resource limits (current):
cputime infinity secs
filesize infinity kb
datasize 1572864 kb
stacksize 262144 kb
coredumpsize infinity kb
memoryuse infinity kb
memorylocked infinity kb
maxprocesses 5547
openfiles 11095
sbsize infinity bytes
vmemoryuse infinity kb
Return of limits -U {qscand|qmaild} (no surprise here as I checked
login.conf)
Resource limits for class default:
cputime infinity secs
filesize infinity kb
datasize infinity kb
stacksize infinity kb
coredumpsize infinity kb
memoryuse infinity kb
memorylocked infinity kb
maxprocesses infinity
openfiles infinity
sbsize infinity bytes
vmemoryuse infinity kb
Return of ulimit as qmaild/qscand (qmail / qmail-scanner users)
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) 1572864
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 11095
pipe size (512 bytes, -p) 1
stack size (kbytes, -s) 262144
cpu time (seconds, -t) unlimited
max user processes (-u) 5547
virtual memory (kbytes, -v) unlimited
Related sysctls:
kern.maxproc: 6164
kern.maxprocperuid: 5547
kern.maxfiles: 12328
kern.maxfilesperproc: 11095
kern.maxusers: 384
Note, I'm nowhere near these limits:
kern.openfiles: 421
ps -aux | wc -l returns 145
What else could be causing these problems? I would be very grateful for
any assistance anyone can provide!
-Justin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?40512A99.3050508>