Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 06 May 2002 01:34:05 -0700
From:      Terry Lambert <tlambert2@mindspring.com>
To:        Patrick Thomas <root@utility.clubscholarship.com>
Cc:        Anthony Schneider <aschneid@mail.slc.edu>, freebsd-hackers@FreeBSD.ORG
Subject:   Re: what causes a userland to stop, but allows kernel to continue?
Message-ID:  <3CD63FFD.BAECB0BF@mindspring.com>
References:  <20020505223641.M86733-100000@utility.clubscholarship.com>

next in thread | previous in thread | raw e-mail | index | archive | help
Patrick Thomas wrote:
> Are NMBCLUSTERS and mbuf determined by 'maxusers' ?
> 
> I have maxusers=512 ... comments ?
> 
> When you suggest 'clamp the total number of sockets that are permittedto
> be open' ... how is this done - is there a sysctl that corresponds to
> "total number of sockets that are permitted to be open" ?

It's the number of tcpcb's, which is based on the value of "maxfiles"
at boot time, when the zalloci mapping reservation occurs in machdep.c;
you are still limited by available physical memory... so if you run out
of real physical memory before you run out of pre-reserved page mapping
space, things will break.

There are different types of limits:

soft limits		Limits that are enforced by software, but are
			generally administrative in nature

hard limits		Limits that are enforced by software, which
			are administrative, but generally apply to
			allocations and reservations of resources at
			boot time, and can not be changed successfully
			at run time (e.g. you can up "maxfiles", but
			it will not increase the number of sockets you
			can have open, because it will not reserve more
			tcpcb's, etc., the KVA for which is allocated
			at boot time).

hard hard limits	Limits which can;t be increased because you
			lack the physical resources necessary to deal
			with them (e.g. things which are allocated out
			of the KVA space can only come from the KVA
			space, so when you are out of KVA, you are
			screwed... or backing pages for such allocations,
			once you run out of physical memory).

> I am also a little confused how this performance issue is solved by
> _lowering_ a tunable value - all of my problems up to this point (ran out
> of file descriptors, ran out of ptys, etc.) were solved by increasing
> them.

The short answer is that this is not a performance issue, it's a resource
starvation issue of some kind (or it's a bug, biut this is much less
likely to be the case, or other people would have seen and fixed the
problem already).

Basically, the symptoms look like a deadly embrace or starvation
deadlock of processes for physical memory -- which is why the
keyboard freezes on you.

It's basically something that happens when you hit a physical memory
or KVA overcommit -- so the only thing you can do to avoid it is to
not overcommit those things.

In some cases, it will happen because you have dirty pages that can
not be forced out to swap... because you are out of swap.  Adding more
swap will work around these cases.

Basically, "maxusers" is the worst possible way to tune things, unless
you are very careful with knowing how much memory you have committed per
user for the load you have.

Without knowing exactly what it is you are running out of, it's going
to be impossible to tell you what you need more of in order to fix the
problem.  I suggested "mbufs" because it's a common culprit in this
type of lockup.

-- Terry

To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3CD63FFD.BAECB0BF>