From owner-svn-src-projects@FreeBSD.ORG Tue Jul 31 09:59:30 2012 Return-Path: Delivered-To: svn-src-projects@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0B619106564A; Tue, 31 Jul 2012 09:59:30 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 97A348FC16; Tue, 31 Jul 2012 09:59:29 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q6V9xYSX022605; Tue, 31 Jul 2012 12:59:34 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q6V9xMpi087414; Tue, 31 Jul 2012 12:59:22 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q6V9xMYH087413; Tue, 31 Jul 2012 12:59:22 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 31 Jul 2012 12:59:22 +0300 From: Konstantin Belousov To: Alexander Motin Message-ID: <20120731095922.GC2676@deviant.kiev.zoral.com.ua> References: <201207301350.q6UDobCI099069@svn.freebsd.org> <20120730143943.GY2676@deviant.kiev.zoral.com.ua> <20120730145912.GZ2676@deviant.kiev.zoral.com.ua> <20120731093735.GB2676@deviant.kiev.zoral.com.ua> <5017A82B.3040704@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="P6FkfbahCsDDKxle" Content-Disposition: inline In-Reply-To: <5017A82B.3040704@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Attilio Rao , Davide Italiano , src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: Re: svn commit: r238907 - projects/calloutng/sys/kern X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2012 09:59:30 -0000 --P6FkfbahCsDDKxle Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jul 31, 2012 at 12:40:59PM +0300, Alexander Motin wrote: > On 31.07.2012 12:37, Konstantin Belousov wrote: > >On Mon, Jul 30, 2012 at 09:48:08PM +0100, Attilio Rao wrote: > >>On Mon, Jul 30, 2012 at 3:59 PM, Konstantin Belousov > >> wrote: > >>>On Mon, Jul 30, 2012 at 03:51:22PM +0100, Attilio Rao wrote: > >>>>On 7/30/12, Konstantin Belousov wrote: > >>>>>On Mon, Jul 30, 2012 at 03:24:26PM +0100, Attilio Rao wrote: > >>>>>>On 7/30/12, Davide Italiano wrote: > >>>>>>>On Mon, Jul 30, 2012 at 4:02 PM, Attilio Rao > >>>>>>>wrote: > >>>>>>>Thanks for the comment, Attilio. > >>>>>>>Yes, it's exactly what you thought. If direct flag is equal to one > >>>>>>>you're sure you're processing a callout which runs directly from > >>>>>>>hardware interrupt context. In this case, the running thread cannot > >>>>>>>sleep and it's likely you have TDP_NOSLEEPING flags set, failing t= he > >>>>>>>KASSERT() in THREAD_NO_SLEEPING() and leading to panic if kernel is > >>>>>>>compiled with INVARIANTS. > >>>>>>>In case you're running from SWI context (direct equals to zero) co= de > >>>>>>>remains the same as before. > >>>>>>>I think what I'm doing works due the assumption thread running nev= er > >>>>>>>sleeps. Do you suggest some other way to handle this? > >>>>>> > >>>>>>Possibly the quicker way to do this is to have a way to deal with t= he > >>>>>>TDP_NOSLEEPING flag in recursed way, thus implement the same logic = as > >>>>>>VFS_LOCK_GIANT() does, for example. > >>>>>>You will need to change the few callers of THREAD_NO_SLEEPING(), but > >>>>>>the patch should be no longer than 10/15 lines. > >>>>> > >>>>>There are already curthread_pflags_set/restore KPI designed exactly = to > >>>>>handle > >>>>>nested private thread flags. > >>>> > >>>>Yes, however I would use curthread_pflags* KPI within > >>>>THREAD_NO_SLEEPING() as this name is much more explicit. > >>>> > >>>Sure, hiding it in THREAD_NO_SLEEPING (THREAD_NO_SLEEP_ENTER/LEAVE ?) > >>>is the way to use curthread_pflags_set there. > >>> > >>>As a second though, on the other hand, is it safe to modify td_flags > >>>from the interrupt context at all ? Probably yes if interrupt handler > >>>always leave td_pflags in the same state on leave as it was on entry, > >>>but couldn't too smart compiler cause inconsistent view of td_pflags > >>>inside the handler ? > >> > >>Can you think of any? Because I cannot think of a case where a nested > >>interrupt can messup with already compiled code, unless it leaks a > >>cleanup. > >In principle, compiler might compile the > > x |=3D a; > >into whatever it finds suitable, e.g. it could write 0 temporary into > >x if the corresponding instruction sequence is considered faster. > > > >No sane compiler for x86 does this. > >> > >>I was more worried about the compiler reordering operations before > >>locking could really see it, but I think in this case the functions > >>call to sleepqueue (at least) works as a sequence point so we are > >>safe. > >> > >>> > >>>>>Also, I wonder, should you assert somehow that direct dispatch canno= t=20 > >>>>>block > >>>>>as well ? > >>>> > >>>>Yes, it would be optimal, but I don't think we have a flag for that > >>>>right now, do we? > >>> > >>>I am not aware of such flag, this might be a good reason to introduce = it, > >>>if issue about td_pflags is just a product of my imagination. > >> > >>I think you should be good to go. Do you plan to work on such a patch? > > > >Ok, I looked closely at the direct dispatch and TD_NOBLOCKING. I now > >think that such flag is not needed. > > > >Am I right that direct dispatch executes callback while owning cc_lock > >spinlock ? >=20 > No, does not now. It was so originally, but was fixed recently, as it=20 > caused LOR deadlocks. Hm, ok. Probably I misread the diff. Anyway, I believe that both direct interrupt dispatch and IPIs take critical sections around handlers. This should have the same effect for assertion in the mi_switch(). >=20 > >If true, then TD_NOBLOCKING is definitely not needed for > >direct dispatch. For thread to be blocked, it shall be scheduled off the > >CPU, going through mi_switch(). And mi_switch() asserts that critical > >section level is exactly 1, which assertion fails due to direct dispatch > >context owning spinlock. >=20 > --=20 > Alexander Motin --P6FkfbahCsDDKxle Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlAXrHoACgkQC3+MBN1Mb4g8fACeLrio17rJnv6l1ydVPnKbUuHB NL4AmgOrwGM6JMEWFn/JGLbFP0hmlsdp =HxrS -----END PGP SIGNATURE----- --P6FkfbahCsDDKxle--