Date: Wed, 19 Feb 2003 23:50:43 +0100 From: Marko Zec <zec@tel.fer.hr> To: Mooneer Salem <mooneer@translator.cx> Cc: FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: Per-jail CPU limits? Message-ID: <3E540A43.215CDECD@tel.fer.hr> References: <FHEMJMOKKMJDGKFOHHEPKEDKFAAA.mooneer@translator.cx>
next in thread | previous in thread | raw e-mail | index | archive | help
Mooneer Salem wrote: > Hello, > > I've been looking at the kernel source, in particular the scheduler > in the past few weeks. I found a place in kern_switch.c where per-jail > CPU controls could be added (in particular, in the kse_reassign() function). > >From looking at that function, I could loop through td_runq until I either: > > 1. Found a thread that isn't jailed, > 2. Found a jailed thread, but determine it's safe to let it run because > it does not go over sysctl-defined limits, or > 3. Find no usable thread, in which case the KSE would theoretically switch > over to the idle process until it's time to repeat the process again. > > This should allow the use of the standard FreeBSD scheduler, except for > the jail limits. The question is, how do we determine the total CPU used > by the jail? I found the kg_estcpu entry in struct ksegrp, which the thread > has a pointer to, but would that be enough? Is there a different approach we > could take that would solve this problem? A rudimentary CPU usage limiting on per virtual image basis (virtual image can be considered a jailed environment with its own independent network stack instance) was implemented using an algorithm very similar to what you proposed, so you might check the original patch against 4.7-RELEASE kernel at http://www.tel.fer.hr/zec/BSD/vimage/index.html As I didn't have enough time yet to make a usable port to 5.0, my assumptions regarding programming in -CURRENT might be slightly wrong, but I guess you'll have to: 1) extend the jail structure to hold CPU usage accounting information on per-jail basis; 2) update this field when doing normal per-process CPU accounting in kern/kern_clock.c / statclock(); 3) do some decay filtering to ensure stability and "smoothness" of the acquired per-jail CPU usage data; 4) in kern/kern_switch.c / chooseproc() implement the steps you originally defined as 1. to 3. 5) on each HZ tick in kern/kern_synch.c / schedclock() check the current process/jail hasn't consumed more CPU time than it was allowed, and if it has, reschedule a new process. This is necessary to ensure acceptable interactive response for processes/jails running with administratively restricted CPU resources, otherwise the process could consume the entire time quantum (10 ms by default), and would than have to wait annoyingly long in order for the average per-jail CPU usage to drop under the defined threshold. 6) optionally, extend procrunnable() in kern/kern_switch.c to return 0 in case there are over-the-CPU-limit processes in active run queue, in order for idle loop to be able to execute the halt instruction, instead of unnecessarily looping endlessly through chooseproc() until the next clock tick. This can be especially useful on laptops where you don't want a process with CPU usage limit to actually burn the battery power in idle loop, and also burn your lap at the same time :) Note: everything I wrote is based on my experience with 4.7-R kernel, in 5.0 many things have changed replacing process with threads as the atomic entities for scheduling, so probably the function naming and some logic has changed also... Marko To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3E540A43.215CDECD>