From owner-freebsd-hackers Mon May 6 1:34:48 2002 Delivered-To: freebsd-hackers@freebsd.org Received: from harrier.prod.itd.earthlink.net (harrier.mail.pas.earthlink.net [207.217.120.12]) by hub.freebsd.org (Postfix) with ESMTP id 0A4E437B403 for ; Mon, 6 May 2002 01:34:45 -0700 (PDT) Received: from pool0126.cvx22-bradley.dialup.earthlink.net ([209.179.198.126] helo=mindspring.com) by harrier.prod.itd.earthlink.net with esmtp (Exim 3.33 #2) id 174dx8-0007I2-00; Mon, 06 May 2002 01:34:38 -0700 Message-ID: <3CD63FFD.BAECB0BF@mindspring.com> Date: Mon, 06 May 2002 01:34:05 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Patrick Thomas Cc: Anthony Schneider , freebsd-hackers@FreeBSD.ORG Subject: Re: what causes a userland to stop, but allows kernel to continue? References: <20020505223641.M86733-100000@utility.clubscholarship.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Patrick Thomas wrote: > Are NMBCLUSTERS and mbuf determined by 'maxusers' ? > > I have maxusers=512 ... comments ? > > When you suggest 'clamp the total number of sockets that are permittedto > be open' ... how is this done - is there a sysctl that corresponds to > "total number of sockets that are permitted to be open" ? It's the number of tcpcb's, which is based on the value of "maxfiles" at boot time, when the zalloci mapping reservation occurs in machdep.c; you are still limited by available physical memory... so if you run out of real physical memory before you run out of pre-reserved page mapping space, things will break. There are different types of limits: soft limits Limits that are enforced by software, but are generally administrative in nature hard limits Limits that are enforced by software, which are administrative, but generally apply to allocations and reservations of resources at boot time, and can not be changed successfully at run time (e.g. you can up "maxfiles", but it will not increase the number of sockets you can have open, because it will not reserve more tcpcb's, etc., the KVA for which is allocated at boot time). hard hard limits Limits which can;t be increased because you lack the physical resources necessary to deal with them (e.g. things which are allocated out of the KVA space can only come from the KVA space, so when you are out of KVA, you are screwed... or backing pages for such allocations, once you run out of physical memory). > I am also a little confused how this performance issue is solved by > _lowering_ a tunable value - all of my problems up to this point (ran out > of file descriptors, ran out of ptys, etc.) were solved by increasing > them. The short answer is that this is not a performance issue, it's a resource starvation issue of some kind (or it's a bug, biut this is much less likely to be the case, or other people would have seen and fixed the problem already). Basically, the symptoms look like a deadly embrace or starvation deadlock of processes for physical memory -- which is why the keyboard freezes on you. It's basically something that happens when you hit a physical memory or KVA overcommit -- so the only thing you can do to avoid it is to not overcommit those things. In some cases, it will happen because you have dirty pages that can not be forced out to swap... because you are out of swap. Adding more swap will work around these cases. Basically, "maxusers" is the worst possible way to tune things, unless you are very careful with knowing how much memory you have committed per user for the load you have. Without knowing exactly what it is you are running out of, it's going to be impossible to tell you what you need more of in order to fix the problem. I suggested "mbufs" because it's a common culprit in this type of lockup. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message