Date: Tue, 28 Jan 2003 12:11:27 -0800 (PST) From: Julian Elischer <julian@elischer.org> To: arch@freebsd.org Subject: threads and the scheduler(s). Message-ID: <Pine.BSF.4.21.0301281125460.6521-100000@InterJet.elischer.org>
next in thread | raw e-mail | index | archive | help
At the beginning of this we spec'd out a design and several things
were said.
1/ the names of these entities might change as we discover the
implementation details,
and
2/ Sme of thes structures (thread, KSE, KSEGRP, proc) might become
more or less 'virtual' when we try implement them.
Well that time has happenned.
In a stroke of brilliance David Xu has added a small 5th element,
an 'upcall' structure, that pretty much changes the relationship
between all the others.
** What follows is only true for Threaded processes **
The 'upcall' structure is a 'token with data'. Only a thread that has
an upcall structure can cross into userland. Any thread that wishes to
cross to userland but does not have one, must simply save it's state to
its userland mailbox, and exit, relying on some other thread to
eventually deliver the completion notice to the UTS.
There is a lot of shuffling here and there but the end result is that
the KSE structure is in effect only known by the KSE/BSD4.4 scheduler.
If we we to put a couple of callbacks into the scheduler
for fork/exit/(etc) points, we could completely
isolate the existance of the KSE to the KSE scheduler.
This would mean that the entities that the rest of the kernel
would know about were:
proc -- owns resources and memeory map.
ksegrp (possibly rename this now.. (subproc?)
thread -- the thing that gets to sleep, run, etc.
The KSE becomes an internal scheduler detail that is used to force some
kind of fairness between threaded and unthreaded processes.
it should be theoretically possible to write a scheduler that uses a
different internal abstraction. We've been working towards this for a
while and over the last few months we've isolated all the 'hard work'
code regarding KSEs to kern_switch.c and kern_threads.c
There are some other places where KSEs are referenced, but usually they
are just creating on, removing one, or doing other housekeeping with
them, and most of these could be moved to scheduler specific callbacks.
here is an almost complete list of what the KSE is used for OUSIDE of
the SCHEDULER files.
Some of these could be moved to within those files by extending the
scheduler interface, (e.g. sched_fork() could add a KSE to a process)
while others of these uses could move to other structures..
For example, KEF_ASTPENDING might be better stored in the upcall
structure or the thread structure (of the thread that is going up)
WHat we need to decide is "What exactly does ASTPENDING mean in the
context of a threaded program. Same for NEEDRESCHED.
##############################
./alpha/alpha/genassym.c:ASSYM(KE_FLAGS, offsetof(struct kse,
ke_flags));
./ddb/db_ps.c: db_printf("[CPU %d]", td->td_kse->ke_oncpu);
./i386/i386/genassym.c:ASSYM(KE_FLAGS, offsetof(struct kse, ke_flags));
./i386/i386/sys_machdep.c: td->td_kse->ke_flags |= KEF_NEEDRESCHED;
./i386/i386/trap.c: KASSERT((td->td_kse->ke_thread == td),
("syscall:kse/thr
ead mismatch"));
./i386/isa/npx.c: td->td_kse->ke_flags |= KEF_ASTPENDING;
./ia64/ia64/genassym.c:ASSYM(KE_FLAGS, offsetof(struct kse, ke_flags));
./kern/init_main.c: ke->ke_sched = kse0_sched;
./kern/init_main.c: ke->ke_oncpu = 0;
./kern/init_main.c: ke->ke_state = KES_THREAD;
./kern/init_main.c: ke->ke_thread = td;
./kern/kern_clock.c: td->td_kse->ke_flags |=
KEF_ASTPENDING;
./kern/kern_clock.c: td->td_kse->ke_flags |=
KEF_ASTPENDING;
./kern/kern_clock.c: * ke->ke_uticks, p->p_sticks, p->p_iticks, and
p->p_estcpu.
This function
./kern/kern_clock.c: * ke->ke_uticks, p->p_sticks, p->p_iticks, and
p->p_estcpu.
./kern/kern_fork.c: bzero(&ke2->ke_startzero,
./kern/kern_fork.c: (unsigned) RANGEOF(struct kse, ke_startzero,
ke_endz
ero));
./kern/kern_fork.c: ke2->ke_state = KES_THREAD;
./kern/kern_fork.c: ke2->ke_thread = td2;
./kern/kern_fork.c: td->td_kse->ke_oncpu = PCPU_GET(cpuid);
./kern/kern_idle.c: td->td_kse->ke_flags |= KEF_IDLEKSE;
./kern/kern_intr.c: if (ctd->td_kse->ke_flags &
KEF_IDLEKSE)
./kern/kern_intr.c: curthread->td_kse->ke_flags |=
KEF_NEEDR
ESCHED;
./kern/kern_ktr.c: (td->td_kse->ke_flags & KEF_IDLEKSE) == 0 &&
./kern/kern_mutex.c: ((td)->td_kse != NULL && (td)->td_kse->ke_oncpu
!= NOCPU
)
./kern/kern_proc.c: kp->ki_rqindex = ke->ke_rqindex;
./kern/kern_proc.c: kp->ki_oncpu = ke->ke_oncpu;
./kern/kern_sig.c: ke->ke_flags |=
KEF_ASTPENDING;
./kern/subr_smp.c: id = td->td_kse->ke_oncpu;
./kern/subr_smp.c: td->td_kse->ke_flags |=
KEF_NEEDRESCHED;
./kern/subr_trap.c: (td->td_kse->ke_flags & KEF_ASTPENDING) ==
0))
./kern/subr_trap.c: flags = ke->ke_flags;
./kern/subr_trap.c: ke->ke_flags &= ~(KEF_ASTPENDING |
KEF_NEEDRESCHED);
./kern/subr_witness.c: * td->td_kse->ke_oncpu to get the list of
spinlocks for
this thread
./posix4/ksched.c: td->td_kse->ke_flags
|=
KEF_NEEDRESCHED;
./posix4/ksched.c: td->td_kse->ke_flags
|=
KEF_NEEDRESCHED;
./posix4/ksched.c: curthread->td_kse->ke_flags |= KEF_NEEDRESCHED;
./powerpc/powerpc/genassym.c:ASSYM(KE_FLAGS, offsetof(struct kse,
ke_flags));
./sys/sched.h:extern struct ke_sched *kse0_sched;
./security/mac_lomac/mac_lomac.c: curthread->td_kse->ke_flags |=
KEF_ASTPE
NDING;
./sparc64/sparc64/genassym.c:ASSYM(KE_FLAGS, offsetof(struct kse,
ke_flags));
################################################
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.4.21.0301281125460.6521-100000>