Date: Sat, 23 Feb 2008 12:19:39 +1100 From: Lawrence Stewart <lstewart@freebsd.org> To: Dirk Meyer <dirk.meyer@dinoex.sub.org>, Gil Bidon <gil.bidon@gmail.com> Cc: freebsd-stable@freebsd.org Subject: Re: FreeBSD 6.3 update and process freeze Message-ID: <47BF74AB.6040001@freebsd.org> In-Reply-To: <vCAdgSVokS@dmeyer.dinoex.sub.org> References: <2de9b4690802201459y7d916bc9i3a0d5548d3025327@mail.gmail.com> <vCAdgSVokS@dmeyer.dinoex.sub.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Dirk Meyer wrote: > Gil Bidon schrieb:, > >> Since i've updated my machine to FreeBSD 6.3 some process randomly start >> using 100% CPU or are locked to STOP state (in top). >> >> Is there a known solution / patch ? Except the rollback :) > > I see this for mail/antivirus-milter too. > But I have not been able to trace it down, > the machine is remote and locks up fully. > On normal load it runs fine, > Maybe an resource-limit is not enforced? > > It looks similar to the nagios thread problem. > To add another data point, I've been seeing similar problems as reported in this thread on a server I upgraded 2 weeks ago from 6.2-RELEASE-p10 to 6.3-RELEASE. I run the spamass-milter-0.3.1_3 port with sendmail and since the upgrade, spamass-milter processes would randomly wedge and spin using 100% CPU. I used GDB to attach to a spinning process and got the following backtrace: (gdb) backtrace 0 0x281a8198 in pthread_sigmask () from /lib/libpthread.so.2 1 0x281a8148 in sigprocmask () from /lib/libpthread.so.2 2 0x281b260c in pthread_mutexattr_init () from /lib/libpthread.so.2 3 0x281a52db in fork () from /lib/libpthread.so.2 4 0x0804d944 in ?? () 5 0x0804e0cf in ?? () 6 0x28089d31 in mi_engine () from /usr/lib/libmilter.so.3 7 0x28089427 in mi_engine () from /usr/lib/libmilter.so.3 8 0x28088c15 in mi_handle_session () from /usr/lib/libmilter.so.3 9 0x2808833b in mi_opensocket () from /usr/lib/libmilter.so.3 10 0x281ac3c9 in pthread_create () from /lib/libpthread.so.2 11 0x2826a837 in _ctx_start () from /lib/libc.so.6 Some time on google found these seemingly related links: http://unix.derkeiler.com/Mailing-Lists/FreeBSD/stable/2008-01/msg00570.html http://www.freebsd.org/cgi/query-pr.cgi?pr=119920 My understanding of the way spamass-milter works is that it forks a child for each email it processes, which does make it seem plausible I'm seeing the pthread fork problem. Assuming this is true, the solution I'm currently testing might solve the problem i.e using /etc/libmap.conf to globally map libpthread to libthr using the following line: libpthread.so.2 libthr.so.2 You of course have to restart all the processes that link against libpthread.so.2 in order for the libmap.conf change to take effect. I only made the change and restarted the processes a couple of days ago so I won't know for a week or two whether it has fixed the problem. So far everything is working as it did before so nothing appears to have regressed since the change. Cheers, Lawrence
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?47BF74AB.6040001>