Date: Sat, 6 Jun 2015 02:03:30 +0300 From: Stefan Andritoiu <stefan.andritoiu@gmail.com> To: freebsd-virtualization@freebsd.org Subject: Implementation of gang-scheduling for bhyve VCPUs Message-ID: <CAO3d8=aCf-02jx27TsDizAjSSKN=BiP85bKu6ZK-AmLaPXavQQ@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hello, I have broken the implementation of gang-scheduling in three steps: 1. Marking threads that should be gang scheduled - add a custom field "int gang" to the FreeBSD thread structure, defined in proc.h - create a custom ioctl request code VM_SET_GANG, that will set the 'gang' field of the VCPU thread, to the gang it is part of - in bhyverun.c, in the fbsdrun_start_thread() function that is invoked when a new pthread is created (a new VCPU is added), call the ioctl function with VM_SET_GANG request code 2.When a thread that must be gang scheduled is selected from a CPU's thread queue, inform other CPUs so they can schedule a thread belonging to that gang - add two new fields to struct tdq: bool gang_leader /* set if CPU is gang leader */ int scheduled_gang /* set if current CPU needs to search it's queue for a thread belonging to scheduled_gang */ - in sched_ule.c, in the sched_choose() function, after a thread is selected, check if it is part of a gang (has the gang field set) - if so, mark the CPU it is on as gang-leader (set gang_leader in tdq), and use function smp_rendezvous() to send IPI to all other CPUs to run schedule_gang() /* custom function defined in sched_ule.c*/ - in function schedule_gang(), set field scheduled_gang, and inform the processor it needs to reschedule (not yet sure how) 3. Scheduling rest of threads - in function tdq_choose(), check first if the tdq's scheduled_gang field is set. If so, reset scheduled_gang field to 0 and return the first thread in the tdq belonging to the gang. If sched_gang==0 or no thread belonging to the gang can be found continue with normal thread selection. This is the implementation of gang-scheduling on FreeBSD the way I have sketched it. If anyone has any thoughts/suggestions, what will work, what will not work, what could have been done better/simpler/easier, anything at all, I would really appreciate and welcome it. Thank you, Stefan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAO3d8=aCf-02jx27TsDizAjSSKN=BiP85bKu6ZK-AmLaPXavQQ>