Date: Mon, 24 Jun 2002 15:25:37 -0700 (PDT) From: Patrick Thomas <root@utility.clubscholarship.com> To: <freebsd-hackers@freebsd.org> Subject: tunings for many httpds... Message-ID: <20020624151650.I68572-100000@utility.clubscholarship.com>
next in thread | raw e-mail | index | archive | help
As a splinter to the ongoing KVA/crash/memory discussion, I am wondering:
- given a machine that will run 250+ httpds and another ~800 misc.
processes, what system tunings would any of you suggest other than the
ones I have done:
In my kernel: maxusers=256 (was 512, change to 256 didn't help)
options SHMMAXPGS=16384
options SHMMAX="(SHMMAXPGS*PAGE_SIZE+1)"
options SHMSEG=256
options SEMMNI=384
options SEMMNS=768
options SEMMNU=384
options SEMMAP=384
(all this SHM and SEM stuff is to run multiple postgres')
and at boot time:
sysctl -w jail.sysvipc_allowed=1
sysctl -w kern.ipc.shmall=65535
sysctl -w kern.ipc.shmmax=134217728
sysctl -w net.inet.tcp.syncookies=0
Anything obvious I am missing ? Terry seems to think:
<quote>
It's obvious that you are running a large number of httpd's; the
sbwait in this case could be reasonably assumed to be waits based
on "sendfile" for a change in so->so_snd->sb_cc; if that's the
case, then it may be that you are simply running out of mbufs,
and are deadlocking. This can happen if you have enough data in
the pipe that you can not receive more data (e.g. the m_pullup()
in tcp_input() could fail before other things would fail).
</quote>
Two things about this interested me:
a) watching `top` output anytime of the day, i see several httpd processes
in "sbwait" - granted I can only see 40 lines of processes or so in `top`,
but usually at least two show "sbwait". Worrisome ?
b) As I showed him, the netstat -m output 30-60 seconds before the crash
looks very benign:
524/2576/34816 mbufs in use (current/peak/max):
500 mbufs allocated to data
24 mbufs allocated to packet headers
273/2254/8704 mbuf clusters in use (current/peak/max)
5152 Kbytes allocated to network (19% of mb_map in use)
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines
Is it possible that within 30 seconds or so current mbufs would skyrocket
and my percentage of mb_map in use would skyrocket and I would start to
see requests for memory denied ?
All comments appreciated.
--PT
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020624151650.I68572-100000>