Date: Fri, 10 Mar 2017 15:30:21 +0100 From: Alexandre Martins <alexandre.martins@stormshield.eu> To: Konstantin Belousov <kostikbel@gmail.com> Cc: freebsd-current <freebsd-current@freebsd.org> Subject: Re: smp_rendezvous_action: Are atomics correctly used ? Message-ID: <1881786.W3Fpph0Tg6@pc-alex> In-Reply-To: <20170310135716.GI16105@kib.kiev.ua> References: <2092905.6A8RAGlt18@pc-alex> <1584060.AQ0HNTQbMq@pc-alex> <20170310135716.GI16105@kib.kiev.ua>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] Le vendredi 10 mars 2017, 15:57:16 Konstantin Belousov a crit : > On Fri, Mar 10, 2017 at 02:24:52PM +0100, Alexandre Martins wrote: > > Le jeudi 9 mars 2017, 16:25:17 Konstantin Belousov a ?crit : > > > On Thu, Mar 09, 2017 at 02:52:09PM +0100, Alexandre Martins wrote: > > > > Le jeudi 9 mars 2017, 15:07:54 Konstantin Belousov a ?crit : > > > > > On Thu, Mar 09, 2017 at 10:59:27AM +0100, Alexandre Martins wrote: > > > > > > I have the save question for the cpu_ipi_pending here: > > > > > > > > > > > > https://svnweb.freebsd.org/base/head/sys/x86/x86/mp_x86.c?view=ann > > > > > > otat > > > > > > e#l1 > > > > > > 080> > > > > > > > > > > > > Le jeudi 9 mars 2017, 10:43:14 Alexandre Martins a ?crit : > > > > > > > Hello, > > > > > > > > > > > > > > I'm curently reading the code of the function > > > > > > > smp_rendezvous_action, > > > > > > > in > > > > > > > kern/subr_smp.c file. In that function, i see that the variable > > > > > > > smp_rv_waiters is read in some while() loop in a non-atomic way. > > > > > > > > > > > > > > https://svnweb.freebsd.org/base/head/sys/kern/subr_smp.c?view=an > > > > > > > nota > > > > > > > te#l > > > > > > > 412 > > > > > > > https://svnweb.freebsd.org/base/head/sys/kern/subr_smp.c?view=an > > > > > > > nota > > > > > > > te#l > > > > > > > 458 > > > > > > > https://svnweb.freebsd.org/base/head/sys/kern/subr_smp.c?view=an > > > > > > > nota > > > > > > > te#l > > > > > > > 472 > > > > > > > > > > > > > > I suspect one of my freeze to be due by that. > > > > > > > > > > You should provide either evidence or, at least, some reasoning > > > > > supporting > > > > > your claims. > > > > > > > > I curently have a software watchdog that triger and does a coredump. > > > > In > > > > the > > > > coredumps, I always see a CPU trying to write-lock a "rm lock". Every > > > > time, > > > > that CPU is spinning into the smp_rendezvous_action, in the first > > > > while > > > > loop) while the others are into the idle threads. > > > > > > > > The fact is that freeze is not clear and I start to search "exotic" > > > > causes > > > > to explain it. > > > > > > This sounds as the 'usual' deadlock, where some other thread owns rmlock > > > in > > > read mode. I recommend you to follow the > > > https://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/ke > > > rnel debug-deadlocks.html > > > > Just a last question, for my personnal knowledge. > > > > In ARM >= 6, for atomic acces, the code should (?) use LDREX and STREX > > for, I quote : "Use LDREX and STREX to implement interprocess > > communication in multiple-processor and shared-memory systems." (see here > > : > > http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0489e/Cihbg > > hef.html > In my previous response to you, I explicitely defined what 'atomic' > means when adjected to the term 'load'. The *EX instructions are used on > ll/sc architectures to implement read/modify/write atomic operations, > which are different from load (read) operations. Ok ! Because we just want to read the value, there is no need to use the *EX version. *EX is intended to be use when a modification will be done thereafter. > > > But, in that while loop, it's a standard "LDR" that is used. Is it correct > > too, and why ? > > Which 'that while loop' ? > while (atomic_load_acq_int(&smp_rv_waiters[3]) < ncpus) > cpu_spinwait(); > This one ? No, I point the one at line 412, 458 and 472: 412: while (smp_rv_waiters[0] < smp_rv_ncpus) cpu_spinwait(); 458: while (smp_rv_waiters[1] < smp_rv_ncpus) cpu_spinwait(); 472: while (smp_rv_waiters[2] < smp_rv_ncpus) cpu_spinwait(); > > Because the semantic of the normal load + DMB barrier provides the expected > semantic of atomic_load_acq(), as explained in atomic(9) and utilized by > the author of the code. So, the writer must use LDREX/STREX to modify the value and use dmb to make visible to other CPU the write. The readers can read simply the value without the barrier because cache coherancy protcol will update the value automaticaly. I think I finally got it ! Thank you so much ! Best regards, -- Alexandre Martins STORMSHIELD [-- Attachment #2 --] 0 *H 010 `He 0 *H 00n 0 *H 0H1 0 UFR10U STORMSHIELD1#0!U Stormshield Root Authority0 140904150710Z 240901150710Z0I1 0 UFR10U STORMSHIELD1$0"U Stormshield Users Authority0"0 *H 0 X6[t. D Uge0-2;h@eȻClΫp B# M,FY=.{ya{2πߢ 7 <d~O;ޅԋ&C 8ُ6@CXX>|abqenmI, O&'۰@%OhW& {52D%_8#f]G0ct y \ v0t0 U ml||uu4[ׁ0 U #0BgDaP0U 00U 0 `HB0 *H N9-؞>m-K!M-7zD6IZʾr>q? u ?xr6_'NջZ7]V\5Y&) )m@~^Qdp/H3ͦ` o⬋U,z,0Bdp!C2K8.r>0-1!C%3U2ϠMg~3 ͟wͲAZ&vF/BA$_ڀV! s7)=-nP> qH~g/Rs,PAJVm#QXJo */V|s Ŀj~y߽j 3]v3ƌqe`Y|HB3lC!^J2$4A`Z+( 7b{e˄wʈ<L=6[3`>IQF aiܝ ?:_yԊ / `0W0? (˻0 *H 0I1 0 UFR10U STORMSHIELD1$0"U Stormshield Users Authority0 160901151108Z 170901151108Z0p1 0 UFR10U STORMSHIELD10U Alexandre MARTINS1/0- *H alexandre.martins@stormshield.eu0"0 *H 0 ~ {x~;#3BgXC[rSVuv#>aL0w}""v`B)Db8qHkH\d qB6rd|:%Ze[wi3)$! hXε< صVO}# EjJe gk0%L퉬Q'b}3"*(_ T- w?gR H,\aWO4Ǘ Y3TR(Wn7Sv1n8xIan 00 U D0IM_W]A$v#<0 U #0ml||uu4[ׁ0 U 0 0U 0 `HB0 U %0++0JU C0A0?=;9https://pki.netasq.com/auth/certificaterevocationlist.crl0U 00U 0+U $0" alexandre.martins@stormshield.eu0 *H 5hhN ̛U92+ E jr@|f{`J2X!ȳ.ڤ|^*$"vlHtSeeAܟ-bC$*)Ж WJL >5@|N%·%{i-4akG앏]%.ݤp]7B/ *1WE@ zZoq@fk0NpHObD $9jXḓSk~LKX100R0I1 0 UFR10U STORMSHIELD1$0"U Stormshield Users Authority (˻0 `He 0 *H 1 *H 0 *H 1 170310143021Z0( *H 100 `He0 *H 0/ *H 1" 8 r)Νֶ.T0 *H HL$Vq Ϧ!9&]0 i2ֆcLDu}GxU ,CXP) -SylW5c61yig&VE ַK[jXV:hTmNo]qtwk?IG*BC$1 n?JF 䃻2ԣ [ka (2tϝ6)~t.`JTct(4%ШS /(Ohelp
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1881786.W3Fpph0Tg6>