From owner-freebsd-arm@freebsd.org Mon Sep 9 22:19:53 2019 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D6B6EE35C8 for ; Mon, 9 Sep 2019 22:19:53 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound3d.ore.mailhop.org (outbound3d.ore.mailhop.org [54.186.57.195]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46S2f14lqtz4kHD for ; Mon, 9 Sep 2019 22:19:53 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1568067592; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=Q493efEjfHomdjuRg+8RGp7X+0+wCcecfAY7J+tzMChi+zrdHFQ5D5vViHAS5gDEtkfVRQw/G061T YgAINKDP1wgg1Aw0xkDFH/WkyT5vk4GYtqDp04MpYNUhABOIxubcdjBsPCKdg55LqwWiO8WL59Y8ke tyyeYq+zFBrNLFk8Vk4A2FL2KXF4O//lzSl/jy+ZDfzeXXAQMRcuLICqa55GF0zsYNCWyqp9dcIMFW SywaiT/1rsUFeBx16cYparhWtdu0PqCRgo/0geIG7NJrd2/SX8rNHbZhlTNB1P7SIqAM2maCTK8mNF tFC+gSIbzgG3zvKspQCcRBcF1aELFcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=r7A2+xYKwNn5blhPIVASuMQ8Y3lcBVtA63MeB6Wp48s=; b=Tz9KtfUod3fT9GmXpASgu9B4WakjvBECy0RQXWC+NnrbcGdw3BnvlbF4Klfo9Z9gxfxjZuIiNNzD+ 8GuVhA0KpN5Rvck8YnIUX4yioWpJ4TDb/BsHZyP+cXgEo8xhaas7H8jaN8LTDwDBDD+RethuAg738q YMKKkdVQuKgEpM5nl7QTAS6djb+SCBCxQk+mf3ylsRy1zCd56DK18AsKGDjEqWz1DKkpPZOHv5f/2Y Ijy45ChkBl1t2Du9fW2s0roZ3EsnB4uzt+i2O5ouBhLOCpjrEfZQyrFqZ1wynRGS1ts4va56/BhIB0 esIzQFZDGD4xaiGWRenUm0zR8UrwY6g== ARC-Authentication-Results: i=1; outbound3.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=r7A2+xYKwNn5blhPIVASuMQ8Y3lcBVtA63MeB6Wp48s=; b=LmEF4QuxiPgrmnUugsqSHuITYSqGl8cBbarC9NBPxvBGU6+yBoj2FSf5DHFrAv5AHpBRWZunHBnhj 9AyQx1mc+1sDAOlUN/8CllnRcpZ136QM/K9+v5eQtWBfVCeHwAA+nsaNsIaHIOcj54Pv2eVkXC+oZH rE3+VM2vB13qZdUyGkLah44EhxTvhxaz8nv0ED1mEm0xDVvic6SSnq1PEnlkFQ7dY3ti46UHW9nrfM dC8XOkR7WDVmxUzhI2aS9LYHPUEnuYiK2Y4vycx6ErQv4D9eC/vFGSXqk312LJOpyK2Nv1Y8k066fX NVuzLYOiaOwOJANJNEu9G/Lvr6q75WA== X-MHO-RoutePath: aGlwcGll X-MHO-User: efd6408f-d34f-11e9-b67d-cdd75d6ce7a8 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound3.ore.mailhop.org (Halon) with ESMTPSA id efd6408f-d34f-11e9-b67d-cdd75d6ce7a8; Mon, 09 Sep 2019 22:19:50 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x89MJm2f063925; Mon, 9 Sep 2019 16:19:48 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: Subject: Re: Is it a good idea to use a usb-serial adapter for PPS input? Yes, it is. From: Ian Lepore To: Konstantin Belousov , "O'Connor, Daniel" Cc: "freebsd-arm@FreeBSD.org" Date: Mon, 09 Sep 2019 16:19:48 -0600 In-Reply-To: <20190909164501.GT2559@kib.kiev.ua> References: <61B1AAF3-40F6-47BC-8F05-7491C13BF288@dons.net.au> <9E142F1A-5E8C-4410-91F5-7C80B3D0A15B@dons.net.au> <9D2ACA87-C2DB-40D9-9638-B0E215A4EEC0@dons.net.au> <0A7796DA-508B-4FE6-B5C0-391EC5F46C86@dons.net.au> <20190908134205.GO2559@kib.kiev.ua> <25BF53F1-23CF-4619-AB13-110566D6DC82@dons.net.au> <20190909164501.GT2559@kib.kiev.ua> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46S2f14lqtz4kHD X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-2.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:16509, ipnet:54.186.0.0/15, country:US] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Sep 2019 22:19:53 -0000 On Mon, 2019-09-09 at 19:45 +0300, Konstantin Belousov wrote: > On Mon, Sep 09, 2019 at 05:20:25PM +0930, O'Connor, Daniel wrote: > > > > > > > On 8 Sep 2019, at 23:12, Konstantin Belousov > > > wrote: > > > > I suppose it would be good to change it to the same structure > > > > as the feed forward clock stuff, that way it is much easier to > > > > change the number of hands at compile time.. > > > > > > > > > > The reason to not increase it by default is the same as the > > > reason why it > > > was reduced. But since I did still not provided the analysis why > > > increasing > > > timehands count helps for Ian' and your case, I think that making > > > it easy > > > to increase the timehands number is due. > > > > I am a bit worried (based on your commit log for r303383) that the > > code now relies on it being 2 for correct function, although given > > I increased it to 10 and this system works fine perhaps not :) > > > > It means that the sliding window where consumer should reach the > writer > to read the current timehands is larger. I think that in reality the > cases where the latency of get*time*(9) KPI increased are very rare. > > Still the question is why your system have the negative impact from > reducing the number of timehands. Interrupt should never interrupt > tc_windup() because it is protected by spinlock. Silly question, are > spinlocks functional on this machine ? > Yep, spinlocks work fine. The problem sequence that happens, as I remember it, is that the system is sleeping in cpu_idle() which has stopped the single core with the WFI (wait-for-interrupt) instruction. The wakeup from WFI is an eventtimer event that was scheduled to handle state->nexthard to keep the timecounter updated. As part of handling that event, the system calls the timecounter's tc_poll_pps function (which is dmtpps_poll() in arm/ti/am335x/am335x_dmtpps.c). That captures the pps event time using the current timehands, then schedules a taskqueue task to finish processing the captured data. By the time the taskqueue task runs, tc_windup() has been called more times than there are timehands, so pps_event() rejects the data because th->th_generation captured in dmtpps_poll() does not match th_generation in that set of timehands anymore. When I was trying to track this down a couple years ago, part of why I gave up was that I couldn't point my finger at anything that was happening and say "here is the problem". It's odd to me that tc_windup() gets called 3 or 4 times so quickly like that, but odd is not the same as wrong. Ironically, a very busy system doesn't suffer this problem. The bad sequence happens only when the system is very quiet, spending most of its time in cpu_idle(). It's something about that path where you come out of cpu_idle() to process a hardclock event followed by a taskqueue task that causes the multiple tc_windup calls to happen. If hardclock events are interrupting other processes, you somehow don't get as many calls to tc_windup between the pps_capture() at interrupt time and the pps_event() call from the taskqueue. The problem also doesn't happen on multi-core systems, perhaps because the taskqueue event begins running sooner on a different core (that is a wild guess). It requires the combo of single core, mostly-idle, and pps-capture driver. -- Ian