Date: Sat, 28 Aug 2004 19:13:55 -0700 From: Julian Elischer <julian@elischer.org> To: jreff@freebsd.org, FreeBSD current mailing list <current@freebsd.org> Subject: New scheduler interface patch Message-ID: <41313BE3.2070906@elischer.org>
next in thread | raw e-mail | index | archive | help
(CC'd to current) Discussing the patch at: http://www.freebsd.org/~julian/f.diff or available to committers as p4 branch "nsched2" AS discussed before there were several "goals" in this change. 1/ remove the KSE structure from general view. (if not delete it entirely) (I admit it was a mistake to make it visible in the first place.. we learn) 2/ clean up thread and ksegrp shutdown. 3/ Make the libthr interface behave according to the rules that threads were supposed to behave by (I am working on documenting this) i.e all threads under a single KSEGRP are limitted by the ksegrp's concurrency and may be handled in some special ways by schedulers. (e.g. affinity etc). Threads under different KSEGRPs are treated as completely separate from each other by the schedulers and not restricted in any way.. Threads in different processes are treated separatly by the scheduler because they are in different ksegrps. the schedulers should only dimmly be aware of the concept of a process, but keenly aware of kergrps and threads. More on this later. 4/ Make the schedulers a bit more independent from each other and, in a related goal: 5/ move all fields in the thread, and ksegrp structures that were scheduler internal implementation dependent into the scheduler specific extensions for those structures.. thus runqueue hooks etc are no longer generally visible. This allows development of schedulers that use completely different internal structures, such as puting a thread on multiple queues at once. There are a 6th and 7th goal for this particular patch: 6/ No changes to functionality of either sheduler (as far as possible) 7/ Comments, field-names, structure names etc. should be left as much alone as possible, even if generally making them inaccurate, in order to make the diff itself as small as possible. This should make it possible for you or others to prove to yourself that the schedulers are basically unalterred in a functional manner, without drowning the diff in non-functional nomenclature and comment changes. Optimisations which can now be made are NOT included in these diffs for the reason given above. I also used macros to achieve some of this. My intention is that after the FUNCTIONAL change has been committed and we all agree that functionality has not changed, that a completely separate pass should be made to fix all that stuff and to clean it up.. The aim is to have a working schedule(s) at all times.. Method: phase 1. I first made a set of diffs that absteacted all the current KSE handling to a set of sched_calls, and manipulated the number of KSEs via a sched_set_concurrency() call. (and some variantes of it.. init_concurrency.. reset_concurrency() etc.) The KSE structires were moved in their entirety to the sched_{4bsd,ule}.c files to ENSURE that nothing else was accessing them. Since these new sched_ methods were common to both schedulers I added them to kern_switch.c (at the bottom). This file was then INCLUDED into both schedulers so it could pick up their local definitions of the KSE structure etc. (these diffs are available on request) Phase 2 Once that was done and nothing outside the schedulers could see the KSE sructures I then merged the KSE sructure and the td_sched structure. The resulting structure is the td_sched structure however the fields from the kse still have their old names and cod estill refers to KSEs and compiles due to the macro "#define kse td_sched" . This meant that every thread has a KSE at all times. I could then remove all the kse_alloc() etc. stuff as they are allocated as part of the thread. Some fields moved from the ksegrp to the kg_sched extension. I added counters to the ksegrp structure to implement the concurrency calls. Nothing outside the schedulers was changed in this phase as all relevent code had already been isolated to the schedulers in phase 1. Phase 3 (Not yet done) After committing and testing, a pass over the code to rename items,correct comments, make new optimisations etc. will be made. Phase 4 Experiment with radically different and revolutionary schedulers. problems: * There may be an interration between the concurrency counters and the preemption code that was just re-enabled. A preemptionmay cause the ksegrp to lose track of the real status. I need to check this in the next day. This could lead to a threaded process 'freezing'. * Load Average under ULE is completely whacked.. I hope you (jeffr) can see what I've done wrong here. * I think I need to look at the new libthr interface code and make an alternate thr_create() that allows libthr to create all its threads under a single ksegrp, with a concurrency of NCPU (the suggested maximum for a ksegrp concurrency). The old thr_create was a strange hybrid.. All threads scheduled independently with no cuncurrency limit (i.e able to flood the run queues) but all onder one KSEGRP. Performance testing will be needed to check which way to go.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?41313BE3.2070906>