Date: Wed, 25 Sep 2019 09:29:25 +0200 From: Michael Tuexen <tuexen@fh-muenster.de> To: Peter Holm <pho@FreeBSD.org> Cc: Alexander Motin <mav@freebsd.org>, src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r352658 - head/sys/kern Message-ID: <36A3B2B8-3230-4B97-AABC-BEA9F040AC24@fh-muenster.de> In-Reply-To: <20190925051317.GA77827@x8.osted.lan> References: <201909242001.x8OK1KAV099968@repo.freebsd.org> <20190925051317.GA77827@x8.osted.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] > On 25. Sep 2019, at 07:13, Peter Holm <pho@FreeBSD.org> wrote: > > On Tue, Sep 24, 2019 at 08:01:20PM +0000, Alexander Motin wrote: >> Author: mav >> Date: Tue Sep 24 20:01:20 2019 >> New Revision: 352658 >> URL: https://svnweb.freebsd.org/changeset/base/352658 >> >> Log: >> Fix/improve interrupt threads scheduling. >> >> Doing some tests with very high interrupt rates I've noticed that one of >> conditions I added in r232207 to make interrupt threads in most cases >> run on local CPU never worked as expected (worked only if previous time >> it was executed on some other CPU, that is quite opposite). It caused >> additional CPU usage to run full CPU search and could schedule interrupt >> threads to some other CPU. >> >> This patch removes that code and instead reuses existing non-interrupt >> code path with some tweaks for interrupt case: >> - On SMT systems, if current thread is idle, don't look on other threads. >> Even if they are busy, it may take more time to do fill search and bounce >> the interrupt thread to other core then execute it locally, even sharing >> CPU resources. It is other threads should migrate, not bound interrupts. >> - Try hard to keep interrupt threads within LLC of their original CPU. >> This improves scheduling cost and supposedly cache and memory locality. >> >> On a test system with 72 threads doing 2.2M IOPS to NVMe this saves few >> percents of CPU time while adding few percents to IOPS. >> >> MFC after: 1 month >> Sponsored by: iXsystems, Inc. >> >> Modified: >> head/sys/kern/sched_ule.c >> >> Modified: head/sys/kern/sched_ule.c >> ============================================================================== >> --- head/sys/kern/sched_ule.c Tue Sep 24 18:18:11 2019 (r352657) >> +++ head/sys/kern/sched_ule.c Tue Sep 24 20:01:20 2019 (r352658) >> @@ -1251,7 +1251,7 @@ sched_pickcpu(struct thread *td, int flags) > > Could this be yours? > > FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs > FreeBSD/SMP: 2 package(s) x 6 core(s) x 2 hardware threads > random: unblocking device. > Firmware Warning (ACPI): Invalid length for FADT/Pm1aControlBlock: 32, using default 16 (20190703/tbfadt-850) > ioapic0 <Version 2.0> irqs 0-23 > ioapic1 <Version 2.0> irqs 24-47 > ioapic2 <Version 2.0> irqs 48-71 > Launching APs: 13 6 18 17 5 9 8 19 7 10 1 11 2 12 14 15 20 4 21 16 22 3 23 > panic: sched_pickcpu: Failed to find a cpu. > cpuid = 0 > time = 1 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffffff8254a830 > vpanic() at vpanic+0x19d/frame 0xffffffff8254a880 > panic() at panic+0x43/frame 0xffffffff8254a8e0 > sched_pickcpu() at sched_pickcpu+0x4c1/frame 0xffffffff8254a990 > sched_add() at sched_add+0x6e/frame 0xffffffff8254a9d0 > gtaskqueue_start_threads() at gtaskqueue_start_threads+0x124/frame 0xffffffff8254aa70 > taskqgroup_cpu_create() at taskqgroup_cpu_create+0x135/frame 0xffffffff8254aab0 > taskqgroup_adjust() at taskqgroup_adjust+0x1ad/frame 0xffffffff8254ab20 > mi_startup() at mi_startup+0x210/frame 0xffffffff8254ab70 > btext() at btext+0x2c > KDB: enter: panic > [ thread pid 0 tid 100000 ] > Stopped at kdb_enter+0x3b: movq $0,kdb_why > db> Also triggered by syzkaller: https://syzkaller.appspot.com/bug?id=27e65a528a1431f0aeea7b7814e685bddf769747 Best regards Michael > > - Peter [-- Attachment #2 --] 0 *H 010 `He 0 *H 00 %ف0 *H 01 0 UDE1+0)U "T-Systems Enterprise Services GmbH1 0 U T-Systems Trust Center1%0#U T-TeleSec GlobalRoot Class 20 160222133822Z 310222235959Z01 0 UDE1E0CU <Verein zur Foerderung eines Deutschen Forschungsnetzes e. V.10U DFN-PKI1-0+U$DFN-Verein Certification Authority 20"0 *H 0 `fAsMg9Z`GNWKVlUvFEc>pԫT#7( ]Ę7 fxJ2)y T_Jx4twMYZ/l< 5e 2mZjKb84YE6Ixjm[k<q;gz6N)=Q\ 4 u9 i%=+ ;U*^3 1Fk Y t0p0U 0 U 2&JJK 0 U #0Y 6 y"kaҸ,˂J0U 003U ,0*0 +!,0 +!, 0g 0LU E0C0A?=;http://pki0336.telesec.de/rl/TeleSec_GlobalRoot_Class_2.crl0+z0x0,+0 http://ocsp0336.telesec.de/ocspr0H+0<http://pki0336.telesec.de/crt/TeleSec_GlobalRoot_Class_2.cer0 *H >eV-;qOں)!F.8y8tv]bL3j2,LG`Utk,ykMP f0"rM>[ϙR9. D~i;u ;"lGG4}=wMOY>GJXoB .M<I&6,"͋xU;D RiТSW{$Jer900c ,=0 *H 01 0 UDE1E0CU <Verein zur Foerderung eines Deutschen Forschungsnetzes e. V.10U DFN-PKI1-0+U$DFN-Verein Certification Authority 20 160524113840Z 310222235959Z01 0 UDE1E0CU <Verein zur Foerderung eines Deutschen Forschungsnetzes e. V.10U DFN-PKI1%0#U DFN-Verein Global Issuing CA0"0 *H 0 ;y G i9٢0r )_KhLk {-n:W Q6pmc.@уLf= di囤rh9QyӮ/]cqK9zBi@p翞 MGH_e*I- ? T/:W~xJA}S_ȳ*0$? MR6$eǖs$hYv~ x<K7S:8Gmd=<ZEJʀ|LL 00U 00U 0)U "0 0 +!, 0 +!,0 U k:S୲2 ;t0 U #02&JJK 0U 00@><:http://cdp1.pca.dfn.de/global-root-g2-ca/pub/crl/cacrl.crl0@><:http://cdp2.pca.dfn.de/global-root-g2-ca/pub/crl/cacrl.crl0+003+0'http://ocsp.pca.dfn.de/OCSP-Server/OCSP0J+0>http://cdp1.pca.dfn.de/global-root-g2-ca/pub/cacert/cacert.crt0J+0>http://cdp2.pca.dfn.de/global-root-g2-ca/pub/cacert/cacert.crt0 *H xENU j>xh5 ?k8w,>̨͠3$b6LxnPn`OR1CFubnfQB1gPI9 ˟ʆHK+f8 W-L_>\)9l{F 8ܰ?q 3Hc%%<z{Wn8︠/[Kl=w 3Sv&%Ao/|~ `]AaI)ب-uJU4"00Ƞ !16BR0 *H 01 0 UDE1E0CU <Verein zur Foerderung eines Deutschen Forschungsnetzes e. V.10U DFN-PKI1%0#U DFN-Verein Global Issuing CA0 190604142910Z 220603142910Z0|1 0 UDE1 0 U Fachhochschule Muenster1200U )Fachbereich Elektrotechnik und Informatik10U Michael Tuexen0"0 *H 0 ̯ʐpK,s>G#=)# 5" @W : qI6D Ni&A@ڑOqm8{ ET'C+ti?8%FU!eե #HK? U<o'0!Ssmd'Ur؉wHZ;09Le.6P7`xyL(5:C+aK gW N0J0>U 7050 +!,0+!,0+!,0 U 0 0U 0 U %0++0 U ATy"Q 0 U #0k:S୲2 ;t0 U 0tuexen@fh-muenster.de0U 00?=;9http://cdp1.pca.dfn.de/dfn-ca-global-g2/pub/crl/cacrl.crl0?=;9http://cdp2.pca.dfn.de/dfn-ca-global-g2/pub/crl/cacrl.crl0+003+0'http://ocsp.pca.dfn.de/OCSP-Server/OCSP0I+0=http://cdp1.pca.dfn.de/dfn-ca-global-g2/pub/cacert/cacert.crt0I+0=http://cdp2.pca.dfn.de/dfn-ca-global-g2/pub/cacert/cacert.crt0 *H 7VYgUEyAȝ&jdZm`'3n / {b͠1M݇ݨJv^|b:ɐR<DW_x$iqi7yoֹMqc s&| )rm씎hK4{Ăl}w<)hhUA#][ZiX\ЗŁ@\NVd v /*@}\C" VeՀ t)$ۉ%)ڧ10001 0 UDE1E0CU <Verein zur Foerderung eines Deutschen Forschungsnetzes e. V.10U DFN-PKI1%0#U DFN-Verein Global Issuing CA !16BR0 `He 0 *H 1 *H 0 *H 1 190925072926Z0/ *H 1" *k!jnd>,mE(Y' 8M0 +71001 0 UDE1E0CU <Verein zur Foerderung eines Deutschen Forschungsnetzes e. V.10U DFN-PKI1%0#U DFN-Verein Global Issuing CA !16BR0 *H 101 0 UDE1E0CU <Verein zur Foerderung eines Deutschen Forschungsnetzes e. V.10U DFN-PKI1%0#U DFN-Verein Global Issuing CA !16BR0 *H Ib]_i !_LW2 ޟȁƦKMՈoۜdx+گ`3q||y| ŤMQϒV<hBnte+ 5SZc.]cwt{ɏ"?A*Zu߃Wuw&ӹܶ 7/& }da-RJS i\dĊM9z cL6KۀW[DƬ|P&*@Y KEڥFjb 2 Md
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?36A3B2B8-3230-4B97-AABC-BEA9F040AC24>