Date: Tue, 24 Feb 2009 14:46:11 -0800 From: Justin Teller <justin.teller@gmail.com> To: freebsd-current@freebsd.org Subject: Possible case of sched_ule never honoring cpu_set affinity? Message-ID: <c47019cc0902241446s434cfb59wbf56e59c75f9c44e@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
--000e0cd5d000bafbb20463b1e2f8 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit I believe I've found a case where sched_ule can never honor cpu_set affinity, and I've attached a suggestion to fix it -- if I'm way off on how this should be implemented, please let me know! Imagine the case where sched_affinity is called and the thread is on a RUNQ on the wrong CPU -- sched_affinity simply exits, relying on something else in the scheduler to migrate the thread if need be. The next time the thread is chosen to run, it runs on the wrong CPU. Worse, if the thread never goes to sleep or is chosen to be moved by the load balancer, the thread will continue running on the wrong CPU indefinitely. Attached is a suggestion of how to change the scheduler to honor affinity in this case. With the attached diff, sched_ule will allow a thread to run on the wrong CPU for one slice. Then when the thread moves through sched_switch, if it was running on the wrong CPU, it will be migrated to the right CPU. I've written a test where one thread will bind another thread to a particular CPU (using cpusets) then waits until it's running on the right CPU before binding it to a different CPU (and continuing ad nauseam). Without the change, the test will sometimes hang waiting for the second thread to get on the correct CPU -- with the change, it works every time. Thanks! -Justin --000e0cd5d000bafbb20463b1e2f8 Content-Type: application/octet-stream; name="sched_ule.affinity_change.diff" Content-Disposition: attachment; filename="sched_ule.affinity_change.diff" Content-Transfer-Encoding: base64 X-Attachment-Id: f_frl5gcn80 LS0tIHNjaGVkX3VsZS5jCTIwMDktMDItMDMgMTU6NDc6MTguMDAwMDAwMDAwIC0wODAwCisrKyBz Y2hlZF91bGVfdXBkYXRlZC5jCTIwMDktMDItMjQgMTQ6MjI6NTMuMDAwMDAwMDAwIC0wODAwCkBA IC0xODI2LDYgKzE4MjYsOCBAQAogCQlzcnFmbGFnID0gKGZsYWdzICYgU1dfUFJFRU1QVCkgPwog CQkgICAgU1JRX09VUlNFTEZ8U1JRX1lJRUxESU5HfFNSUV9QUkVFTVBURUQgOgogCQkgICAgU1JR X09VUlNFTEZ8U1JRX1lJRUxESU5HOworCQlpZiAoVEhSRUFEX0NBTl9TQ0hFRCh0ZCwgdHMtPnRz X2NwdSkpCisJCQl0cy0+dHNfY3B1ID0gc2NoZWRfcGlja2NwdSh0ZCwwKTsKIAkJaWYgKHRzLT50 c19jcHUgPT0gY3B1aWQpCiAJCQl0ZHFfcnVucV9hZGQodGRxLCB0ZCwgc3JxZmxhZyk7CiAJCWVs c2UK --000e0cd5d000bafbb20463b1e2f8--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?c47019cc0902241446s434cfb59wbf56e59c75f9c44e>