Date: Mon, 24 Jun 2002 11:40:36 -0700 (PDT) From: Patrick Thomas <root@utility.clubscholarship.com> To: Terry Lambert <tlambert2@mindspring.com> Cc: Nielsen <nielsen@memberwebs.com>, <hackers@freebsd.org> Subject: Re: (jail) problem and a (possible) solution ? Message-ID: <20020624112143.X68572-100000@utility.clubscholarship.com> In-Reply-To: <3D158E12.CDB87171@mindspring.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Terry, I made an initial change to the kernel of reducing maxusers from 512 to 256 - you said that 3gig is right on the border of needing extra KVA or not, so I thought maybe this unnecessarily high maxusers might be puching me "over the top". However, as long as I was changing the kernel, I also added DDB. The bad news is, it crashed again. The good news is, I dropped to the debugger and got the wait channel info you wanted with `ps`. Here are the last four columns of ps output for the first two pages of processes (roughly 900 procs were running at the time of the halt, so of course I can't give you them all, especially since I am copying by hand) 3 select c0335140 local 3 select c0335140 trivial-rewrite 3 select c0335140 cleanup 3 select c0335140 smtpd 3 select c0335140 imapd 2 httpd 2 httpd 3 sbwait e5ff6a8c httpd 3 lockf c89b7d40 httpd 3 sbwait e5fc8d0c httpd 2 httpd 3 select c0335140 top 3 accept e5fc9ef6 httpd 3 select c0335140 imapd 3 select c0335140 couriertls 3 select c0335140 imapd 2 couriertls 3 ttyin c74aa630 bash 3 select c0335140 sshd 3 select c0335140 tt++ So there it all is. Does this confirm your feeling that I need to increase KVA? Or does it show you that one of the one or two other low probablity problems is occurring? thanks, PT On Sun, 23 Jun 2002, Terry Lambert wrote: > Patrick Thomas wrote: > > I think I'll just decrease my swap size from 2 gigs to 1 gig - is that a > > reasonable alternative that provides the same benefit and possible > > solution to this problem ? > > > > ...since bsically 0 swap has ever been used on the machine anyway... > > Not really. > > The code in machdep.c allocated pmaps for swapped memory based > on the size of real memory, rather than based on available swap. > > The reason it does this is that you can (effectively) add an > arbitrary amount of swap later with "swapon", without the swap > devices at the time being known to the kernel at boot. THis > makes it impossible to prereserve the number of pmap pages that > will be needed for the actual amount of swap. > > Matt Dillon made some autosizing changes after I complained > about this before. My actual complaint was to implicate the > size of real memory available relative to the size of the full > address space. The change he made attempts to autosize, and > doesn't quite mirror this policy directly. THis code is not > available in 4.5. I believe that it was back-ported to 4.6, > but you would have to look at the CVS log on machdep.c to be > sure about this -- it may only be in -current. > > The upshot of this is that having a lot of memory reserves > pmap entries at 4K per 4M of real OR virtual memory. The > result of this is that at 4G of physical RAM, you actually > end up allocating more pmap's than 1G of memory can contain, > since the total of physical RAM plus swap over 1024 is > larger than 1G minus the amount taken by an idle kernel, not > including the page mappings. > > If you have 3G of real RAM (which you do), then you are on > the borderline of running out. When you factor in the amount > of *potential* swap that machdep.c reserves, plus tuning for > maxfiles/sockets/inpcb/tcpcb/mbufs/etc. (if any), PLUS the > RAM taken up for things associated with running over 1000 > processes (as your system does), then you end up exhausting > the amount of VM space available. > > As I said before, though, the only way to know for sure if > this is your real problem is to break to the debugger after > the lockup (it's *not* a crash), and check out the wait channels > for the processes thar are unable to run. > > If you want a tweak for 4.5 that has about a 95% proability of > masking the problem, then you need to up the KVA space. > > Unfortunately, it's not really possible to tell you where > every byte of memory is going. Also, unfortunately, the > pmap's for swappable memory are not themselves swappable > (or this would not be a problem). Probably, pmaps for > swap and for file backing store for exectuables should be > allocated when they are needed, not preallocated (they can > be, if you are not out of RAM, or have RAM, but are out of > KVA space in which to create mappings) [see "growkernel"]. > > Taking out 1G of physical memory from the box might also > fix the problem without a kernel tweak, FWIW. > > However, right now, you need to cause the problem, enter > the debugger, and use "ps" in the debugger to examine the > wait channels. > > -- Terry > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020624112143.X68572-100000>