Date: Thu, 13 Nov 2014 02:14:27 -0800 From: Alfred Perlstein <alfred@freebsd.org> To: Adrian Chadd <adrian@freebsd.org> Cc: Hans Petter Selasky <hps@selasky.org>, "freebsd-arch@freebsd.org" <freebsd-arch@freebsd.org> Subject: Re: Questions about locking; turnstiles and sleeping threads Message-ID: <54648483.5060107@freebsd.org> In-Reply-To: <CAJ-VmonbEfxz9Bgw9O9f-5%2Bb=UM1b1nzPK9zfAAnmYKVumOKkQ@mail.gmail.com> References: <CAJ-VmomrauhCMoF_dZfMWWhZp0EgwfE9RmxL5Pc37PhLSzZ6Qg@mail.gmail.com> <20141112212613.21037929@kan> <CAJ-Vmok-8znyycyOBS_ZQU275zFy%2BzuZ2C-jt4N3DnuEVS=PWg@mail.gmail.com> <CAJ-Vmo=R4ayAn-d986TC3CzZ4y23jnDW=uTyo4O=x4Ae1wB60A@mail.gmail.com> <CAJ-Vmok40-W%2B%2BdUJPcfO74%2BuudCgbs3tR5nwrR-aNuCy=5o0HQ@mail.gmail.com> <546472DA.3080006@freebsd.org> <CAJ-VmomQdNDWqdC7o-nuGfypDDgDsJX9ouhx%2Br7ckZptgzQ10Q@mail.gmail.com> <5464764E.9080308@freebsd.org> <CAJ-VmonbrB3s5SsABo1Kotfz1Dn3cpvbnRMu4J9ikPmyS_aTgQ@mail.gmail.com> <54647D1E.9010904@freebsd.org> <CAJ-VmonbEfxz9Bgw9O9f-5%2Bb=UM1b1nzPK9zfAAnmYKVumOKkQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Would need more context to help on this. I can't tell based on your description which thread is holding which lock. If A is waiting for callout C to stop AND there exists a thread B that is contending against C for a lock, you should be fine so long as there is no lock cycle against A. Would be best if you pointed at some code and gave descriptions. -Alfred On 11/13/14, 1:52 AM, Adrian Chadd wrote: > Hm, the more I dig into this, the more I realise it's not a 1:45am > question to ask. > > Specifically, callout_stop_safe() takes 'safe', which says "are we > waiting around for this callout to finish if it started". Ie, > callout_drain() is callout_stop_safe(c, 1) ; callout_stop() is > callout_stop_safe(c, 0). > > If safe is 1, then it'll potentially put the current thread to sleep > in order to wait for it to synchronise with the callout that's > running. It's sleeping with cc_lock which is the per-callwheel lock > and it's doing that with whatever other locks are held. That's the > situation which is tripping things up. > > The manpage says that no locks should be held that the callout may > block on, which isn't the case here at all - I'm trying to grab a lock > in another thread that the caller _into_ the callout subsystem holds. > The manpage doesn't mention anything about this. Sniffle. > > > > -adrian > > On 13 November 2014 01:42, Alfred Perlstein <alfred@freebsd.org> wrote: >> OK that makes more sense. >> >> I've cc'd Hans for the usb issue. >> >> >> On 11/13/14, 1:38 AM, Adrian Chadd wrote: >>> It looks like the initial firings are because the check I put in >>> didn't check to see if it's MPSAFE. >>> >>> eg: >>> >>> ip6_input -> tcp6_input -> tcp_input -> tcp_do_segment -> >>> tcp_timer_active -> callout_stop_safe; called with tcpinp held. >>> closefp() -> closef() -> fdrop -> soclose() -> sofree() -> >>> tcp_usr_detach() -> tcp_discardcb() -> callout_stop_safe() with the >>> tcpinp and tcp locks held. >>> ioctl -> sys_ioctl-> devfs_ioctl_f -> acpi_ackSleepState -> >>> callout_stop_safe; with ACPI global lock held; >>> suspend path -> hdaa_suspend -> callout_stop_safe() with HDA driver mutex >>> held >>> >>> So we can't just put the simple witness check from _sleep() in >>> _callout_stop_safe(), it looks like it's mis-firing on MPSAFE callouts >>> (which the tcp timers all are) and that won't go via the sleepq. >>> It looks like the acpi callout is also mpsafe, as well as the HDA jack >>> callout. >>> >>> However, I did pick up this: >>> >>> detach path -> usbd_transfer_drain() -> usbd_transfer_stop() -> >>> ehci_device_intr_close() -> usbd_transfer_done() -> >>> callout_stop_safe() with USB HUB mutex held >>> >>> The usbd_transfer_done() callout is initialised with a mutex, but the >>> witness code should've detected it wasn't callout->c_lock and thus >>> warned. >>> >>> >>> >>> -adrian >>> >>> On 13 November 2014 01:13, Alfred Perlstein <alfred@freebsd.org> wrote: >>>> On 11/13/14, 1:09 AM, Adrian Chadd wrote: >>>>> On 13 November 2014 00:59, Alfred Perlstein <alfred@freebsd.org> wrote: >>>>>> On 11/12/14, 11:25 PM, Adrian Chadd wrote: >>>>>>> On 12 November 2014 19:48, Adrian Chadd <adrian@freebsd.org> wrote: >>>>>>>> kan pointed out that we should likely do a WITNESS_WARN() in the >>>>>>>> relevant spots in the callout code so we catch these before it >>>>>>>> happens. >>>>>>>> >>>>>>>> I'll see what we can add to -HEAD to be pro-active about it. >>>>>>> Amusingly, I tried adding it and it made my laptop turn to soup very >>>>>>> quickly - among other things, the TCP timer callouts are all setup >>>>>>> with non sleep locks held. >>>>>>> >>>>>> Howso? You only have to worry about callout_drain(), most other >>>>>> callout >>>>>> functions should be safe-ish.... >>>>> yeah, except for all the places where they drain the timer once >>>>> something happens so it doesn't fire. >>>>> >>>>> :) >>>> >>>> What is the backtrace...? >>>> >>>> >>> _______________________________________________ >>> freebsd-arch@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-arch >>> To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >>> > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?54648483.5060107>