Date: Tue, 10 Sep 2019 17:40:59 +0300 From: Konstantin Belousov <kostikbel@gmail.com> To: Ian Lepore <ian@freebsd.org> Cc: "O'Connor, Daniel" <darius@dons.net.au>, "freebsd-arm@FreeBSD.org" <freebsd-arm@FreeBSD.org> Subject: Re: Is it a good idea to use a usb-serial adapter for PPS input? Yes, it is. Message-ID: <20190910144059.GA2559@kib.kiev.ua> In-Reply-To: <b5c8493ef95dcc49ed21fd7c7cf52808e5f0bfe8.camel@freebsd.org> References: <B9EFA4D4-C1AD-4181-B421-F6BD53434FA5@dons.net.au> <bfd784f6ac9c939dbe6a243336bc3b6eab02d4f5.camel@freebsd.org> <61B1AAF3-40F6-47BC-8F05-7491C13BF288@dons.net.au> <9E142F1A-5E8C-4410-91F5-7C80B3D0A15B@dons.net.au> <9D2ACA87-C2DB-40D9-9638-B0E215A4EEC0@dons.net.au> <0A7796DA-508B-4FE6-B5C0-391EC5F46C86@dons.net.au> <20190908134205.GO2559@kib.kiev.ua> <25BF53F1-23CF-4619-AB13-110566D6DC82@dons.net.au> <20190909164501.GT2559@kib.kiev.ua> <b5c8493ef95dcc49ed21fd7c7cf52808e5f0bfe8.camel@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Sep 09, 2019 at 04:19:48PM -0600, Ian Lepore wrote: > On Mon, 2019-09-09 at 19:45 +0300, Konstantin Belousov wrote: > > On Mon, Sep 09, 2019 at 05:20:25PM +0930, O'Connor, Daniel wrote: > > > > > > > > > > On 8 Sep 2019, at 23:12, Konstantin Belousov <kostikbel@gmail.com > > > > > wrote: > > > > > I suppose it would be good to change it to the same structure > > > > > as the feed forward clock stuff, that way it is much easier to > > > > > change the number of hands at compile time.. > > > > > > > > > > > > > The reason to not increase it by default is the same as the > > > > reason why it > > > > was reduced. But since I did still not provided the analysis why > > > > increasing > > > > timehands count helps for Ian' and your case, I think that making > > > > it easy > > > > to increase the timehands number is due. > > > > > > I am a bit worried (based on your commit log for r303383) that the > > > code now relies on it being 2 for correct function, although given > > > I increased it to 10 and this system works fine perhaps not :) > > > > > > > It means that the sliding window where consumer should reach the > > writer > > to read the current timehands is larger. I think that in reality the > > cases where the latency of get*time*(9) KPI increased are very rare. > > > > Still the question is why your system have the negative impact from > > reducing the number of timehands. Interrupt should never interrupt > > tc_windup() because it is protected by spinlock. Silly question, are > > spinlocks functional on this machine ? > > > > <dropping usb@ from the cc, since this is purely arm stuff> > > Yep, spinlocks work fine. > > The problem sequence that happens, as I remember it, is that the system > is sleeping in cpu_idle() which has stopped the single core with the > WFI (wait-for-interrupt) instruction. The wakeup from WFI is an > eventtimer event that was scheduled to handle state->nexthard to keep > the timecounter updated. As part of handling that event, the system > calls the timecounter's tc_poll_pps function (which is dmtpps_poll() in > arm/ti/am335x/am335x_dmtpps.c). That captures the pps event time using > the current timehands, then schedules a taskqueue task to finish > processing the captured data. By the time the taskqueue task runs, > tc_windup() has been called more times than there are timehands, so > pps_event() rejects the data because th->th_generation captured in > dmtpps_poll() does not match th_generation in that set of timehands > anymore. Is it really needed to schedule task for pps_event() ? We don't for tc_windup(), and pps_event() is quite similar. And for pps_event(), we might use same opportunistic locking as for tc_windup(). > > When I was trying to track this down a couple years ago, part of why I > gave up was that I couldn't point my finger at anything that was > happening and say "here is the problem". It's odd to me that > tc_windup() gets called 3 or 4 times so quickly like that, but odd is > not the same as wrong. > > Ironically, a very busy system doesn't suffer this problem. The bad > sequence happens only when the system is very quiet, spending most of > its time in cpu_idle(). It's something about that path where you come > out of cpu_idle() to process a hardclock event followed by a taskqueue > task that causes the multiple tc_windup calls to happen. If hardclock > events are interrupting other processes, you somehow don't get as many > calls to tc_windup between the pps_capture() at interrupt time and the > pps_event() call from the taskqueue. > > The problem also doesn't happen on multi-core systems, perhaps because > the taskqueue event begins running sooner on a different core (that is > a wild guess). It requires the combo of single core, mostly-idle, and > pps-capture driver. > > -- Ian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20190910144059.GA2559>