Date: Sat, 8 May 1999 12:11:14 -0700 (PDT) From: Matthew Dillon <dillon@apollo.backplane.com> To: Kevin Day <toasty@nfs.dragondata.com> Cc: toasty@home.dragondata.com (Kevin Day), current@FreeBSD.ORG Subject: Re: -current NFS crash (out of mbuf clusters) Message-ID: <199905081911.MAA55730@apollo.backplane.com> References: <199905081842.NAA14324@nfs.dragondata.com>
next in thread | previous in thread | raw e-mail | index | archive | help
:> :I'm sure by now Matt is gonna kill me. :)
:> :
:> :-current from 2 days ago.
:...
:
:> netstat -m -M vmcore.XX -N kernel.XX
:>
:
:1014/2144 mbufs in use:
: 714 mbufs allocated to data
: 300 mbufs allocated to packet headers
:638/1324/1536 mbuf clusters in use (current/peak/max)
:2916 Kbytes allocated to network (48% in use)
:0 requests for memory denied
:0 requests for memory delayed
:0 calls to protocol drain routines
:
:What does this tell you?
:
:Kevin
It tells me your userbase is out of control :-) From the looks
of it, hundreds of cron jobs are starting up simultaniously
and overloading some system resource.
I would also recommend:
vmstat -m -M vmcore.XX -N kernel.XX
It is possible that the machine was attacked from the outside since you
are allowing eggdrops to be run. An IP spoofing attack can eat a
considerable amount of KVM due to temporary routes and, in fact, run
it out, leaving no memory left for mbufs. If so, this will show up
in the vmstat.
A quick side note on eggdrops: We allowed them at BEST.COM, but
after four years our machines and networks were getting attacked
virtually every day by IRC bozos. Also, the users who tend to run
eggdrops also tend to be stupid - often logging in from compromised
machines, so we also had a huge problem with these user's accounts being
compromised. We eventually gave up and banned bots entirely. Things have
been a whole lot quieter since.
Another thing you can do in regards to the cron jobs is go through
all your user's crons, many of which are probably running bot check
scripts every 10 minutes, and adjust them to run only once an hour,
plus scramble the 'minute' so they do not all run simultaniously.
I've seen IRC bozos setup cron jobs that run botcheck once a minute.
We gave them one warning, and if they did not heed it we kicked them off.
-Matt
Matthew Dillon
<dillon@backplane.com>
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199905081911.MAA55730>