Date: Mon, 21 Aug 2006 12:15:31 -0600 From: Chris Jones <cdjones-freebsd-hackers@novusordo.net> To: Remko Lodder <remko@FreeBSD.org> Cc: freebsd-hackers@freebsd.org Subject: Re: [SoC] Jail Resource Limits Message-ID: <5E256748-E46C-46FE-80B9-6E1DBD8F1208@novusordo.net> In-Reply-To: <44E9D322.8090900@FreeBSD.org> References: <98C569DD-B559-4243-8C15-0DA1D832C387@novusordo.net> <44E9D322.8090900@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 21-Aug-06, at 9:37 AM, Remko Lodder wrote: > Before my little question, i would like to take the opportunity to > thank you for the hardwork, this is a much requested feature and > will greatly appriciated by the crowd :-) Cheers for that!. Thanks! I do some web hosting with jails on the side, and've often wished this sort of thing were around. I suppose necessity truly is the mother of invention. ;) > My little question: How does this impact on performance? I noticed > all your p4 commits and saw some pieces of code that constantly > monitor the usage of the jail (wrt CPU and Memory); doesn't that > reduce our performance? Is that performance impact (if any) also > growing when using more jails? The changes can be broken out into three major groups: the memory- limiter (kern_jail:jpager_td), the CPU usage monitor (sched_4bsd:schedcpu), and the process priority resetter (sched_4bsd:resetpriority). The memory-limiter essentially idles if the sysctl for memory limits is turned off or the jail doesn't have a memory limit; otherwise, we need to go over all processes and check whether they're in the jail and sum their RSS (in prison_memory). This could really use having a quick way of getting a list of processes in a jail (perhaps a new entry in the proc struct for next-process-in-this-jail?). If we're over the limit, we then have to go over all processes again and check whether they're in the jail and amenable to being pushed out. If so, we then go and push them out in the same way that the pagedaemon does. This bit's going to be relatively slow, and also could use that quick get-processes-in-jail method. The CPU usage monitor keeps track of the number of CPU usage shares allocated in schedcpu_thread, which runs every second and then calls schedcpu (see below). This one's linear with the number of jails and again could probably benefit from a quick check to see whether we're enforcing CPU shares at all. An even better method would be to just recalculate the total outstanding shares when the jail or jail_set_resource_limits syscalls are called or the sysctl for unjailed shares changes, rather than all the time. The amount of CPU time each KSE uses is already collected by the system, so we just re- use that. schedcpu needs to lock the allprison mutex and reset each prison's estimated CPU time at line 466, so that'll be about linear with the number of jails. Hmm, looking at it, there's a really obvious optimization that I forgot to put in (skip the lock & reset if we have no jails), which I'll want to add. Next, it will increment the CPU time if the sysctl is set and the process is jailed. We're looking at about eight loads if the compiler's smart, three compares and two ors, an if, and maybe an add. Unfortunately, this time's linear with the number of processes. The big hit comes in the process priority resetter, which runs when we consider what priority to run a process at (this runs intermittently for each active process). If the sysctl is set and the process is jailed, we do some reasonably hefty (some multiplies) calculations to change it. This'll be roughly linear with the number of jailed processes. Overall, I don't think there'll be a noticeable performance impact for non-pathological numbers of processes or jails, but I don't have any stats to back that up. > I think these things are simple, but important to know; I do not > have a full featured setup (sadly) to measure the performance > impact of this, If you are able to find someone, that would be > really great imo. Consider this a general appeal for anyone who can help me do some analysis on the performance testing, or give me some pointers on how to collect the relevant data. :) Cheers, Chris
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5E256748-E46C-46FE-80B9-6E1DBD8F1208>