Date: Thu, 27 Aug 1998 13:51:40 -0700 (PDT) From: Archie Cobbs <archie@whistle.com> To: archer@lucky.net (Alexander Litvin) Cc: wollman@khavrinen.lcs.mit.edu, current@FreeBSD.ORG Subject: Re: encountered possible VM bug ? Message-ID: <199808272051.NAA27400@bubba.whistle.com> In-Reply-To: <199808270714.KAA02669@grape.carrier.kiev.ua> from Alexander Litvin at "Aug 27, 98 10:14:19 am"
next in thread | previous in thread | raw e-mail | index | archive | help
Alexander Litvin writes: > >> Under out of memory conditions, inetd tends to fall in a weird > >> state. Apparently this is an interaction between the malloc library > >> and inetd, but nobody has managed yet to find out exactly what > >> happens. Maybe error checking is lacking somewhere. > > GW> No, this is the ``daemons dying'' bug which nobody has fixed yet. > GW> When the system runs out of swap, some random selection of processes > GW> which are in swap get corrupted. Usually this results in a daemon > GW> which dies whenever it fork()s, but sometimes it is manifested as > GW> other sorts of corruption. The message you see from realloc is > GW> indicative of a corrupted pointer. > > Really, I was under impression, that it is the problem just with fork(). > But now I may confirm that processes get corrupted in different manners. > E.g., I have now a specially written dummy daemon running, which I > was able to corrupt (intentionally exhausting swap) in such a way that > it successfully forks. Than child process sleeps (just to give me > chance to attach to it with debugger), allocates memory, accesses it > -- and during all that it doesn't get SIGSEGV. But then it dies when > trying to syslog(3). It seems that the corruption is in mmaped ld.so > or libc.3.1.so. > > If anybody cares, I may try to give any other details. At Whistle, we've seen this bug every so often for a long time. The common elements seem to be: 1. memory mapping is in use 2. a fork() is happening or just happened But #1 and #2 are not necessarily both related to the same process. This bug has been around for a *long* time, in both 2.x and 3.x. Running out of swap may or may not be related, not sure... I think we've seen this when swap was not an issue. Perhaps running out of swap amplifies the problem. It's really hard to pin down, because the panic seems to come a while after the initial damage is done. We've seen random processes crashing every time they try to fork(), kernel panic's because of some process being on two different queues at the same time (eg, sleep and runnable), and other manifestations. A common manifestation is that a file being written out contains some random page of memory from some other file -- we think the other file is a currently mmap'd file. Julian and Terry can supply more details. -Archie ___________________________________________________________________________ Archie Cobbs * Whistle Communications, Inc. * http://www.whistle.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199808272051.NAA27400>