From owner-freebsd-arm@freebsd.org Mon Aug 19 15:52:08 2019 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C321ACA061 for ; Mon, 19 Aug 2019 15:52:08 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound2m.ore.mailhop.org (outbound2m.ore.mailhop.org [54.149.155.156]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46Bz2J4BbJz3ydj for ; Mon, 19 Aug 2019 15:52:08 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1566229926; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=nz0Z/2sqqez1ElCOMgiNUoPiukkCV4VF5WlX8ZErJzJd5aVLJR8DBaZy5icT5bCO6somHEbyn+8na J856QGL5q6Fe1TpQmggfc0GuQmoF2zoU0oYJhkSl6feXgQpwSpEVavXAEerIugMNF9mU/xx1Glg5Pq dBLLQSW4zlufcxl5XRH6LKlphnLRZQ0OG6zCdrwr2K5uA/x1goJys3gqirpRQpw1FHaTHJq9P7/fuU fQXmDPIiY0nG9rqf2Hwh/DAcCBIvIXoLgqjkL5ANmUqOHHO3qYwbnRtb6xJ4+JJM/n8JwlidA0/xBw L//kjQyyTxOZO19byV3TzAMPWdGbUIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=1s36/kbLRuPHZahLoSbUDpqMfuFjKo42SUI2XyOrdYQ=; b=itovK+hzExaUt5H4BTQ32qaEzZCVJ08hoTWSroNeS4NiSu9Q/ADrrAZ9J2kEwbuH+liT1HEoL2APO Sl/OFxJTVGdzJrsb+wqwX/SUMuwFo915Eax0fPGTEoEzUj4zi0ey8tOZhqiyTUkPG9gOTrytRlgeOO eT/mxz6I1xAqGw8SJ5oa2AXsqsP7w6VF4y0d2u6NbkkRsmkg6I9ON7RwyKvfhA36cVa8qJCdwPpKWn sOzP+TTAwhkmEMPVuNoYC1oKpt/n2S3tcDtPJspeGBgHNZQMv1A1bgJ0AEO1fg95blCsHGve7jx9vU c9ZBDouAqoDHZROXT2n+l3fGBDkGMKg== ARC-Authentication-Results: i=1; outbound4.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=1s36/kbLRuPHZahLoSbUDpqMfuFjKo42SUI2XyOrdYQ=; b=eOHn1Lo2pbi1JZEUcWOKXDU2CwjGVLrUcF/Sno3h972uqcKhtIlbjWnAgAD0Q8dr622eZqUV5JPnK qKohnwUKUzBZvgCNbtsXdyaFkwl2V+QKGGhkwdkI8W/YQrszSIpSFE/oJBSWQrU6mGUuJkdRuQ1nWq /7Dhey0Afj3BTsfBXSLzUUzeBVcGCFQa6q0bVKiWop2/rMM+Nqzp3hyOWFY/OWxD/kgtUvAkrDUW8q iOZHM3fSwBFur6x8OV84Hnrwf1FTBD5bG2w0pDM4WmLC1dfDNj8YpYiPxVehvDrnMmLcWi2WGCbdMZ EloKUcwjKJghOUgG/sOLw9FfI4IcPFA== X-MHO-RoutePath: aGlwcGll X-MHO-User: 49f9d5ec-c299-11e9-85ec-13b9aae3a1d2 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound4.ore.mailhop.org (Halon) with ESMTPSA id 49f9d5ec-c299-11e9-85ec-13b9aae3a1d2; Mon, 19 Aug 2019 15:52:05 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x7JFq39k076197; Mon, 19 Aug 2019 09:52:03 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <8220eafa44d1f3a4d38e3ecfd18f2ff9c7f3ce97.camel@freebsd.org> Subject: Re: dmtpps driver on beaglebone [was: Is it a good idea to use a usb-serial adapter for PPS input? Yes, it is.] From: Ian Lepore To: Konstantin Belousov Cc: Warner Losh , "freebsd-arm@FreeBSD.org" Date: Mon, 19 Aug 2019 09:52:03 -0600 In-Reply-To: <20190819152244.GN71821@kib.kiev.ua> References: <61B1AAF3-40F6-47BC-8F05-7491C13BF288@dons.net.au> <9E142F1A-5E8C-4410-91F5-7C80B3D0A15B@dons.net.au> <20190819152244.GN71821@kib.kiev.ua> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46Bz2J4BbJz3ydj X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.983,0]; ASN(0.00)[asn:16509, ipnet:54.148.0.0/15, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2019 15:52:08 -0000 On Mon, 2019-08-19 at 18:22 +0300, Konstantin Belousov wrote: > On Mon, Aug 19, 2019 at 08:47:46AM -0600, Ian Lepore wrote: > > I first ran into the problem on a beaglebone a few years ago, not that > > long after the number was reduced to two. The number 4 is pretty much > > just a vague-ish memory of the debugging I did back then. When you > > don't have enough timehands, the symptom is that you lose most of the > > pps events (but enough get captured that ntpd will more or less run > > properly on the reduced data). > > > > The busier the system is, the more events get captured succesfully, > > because the loss is caused when the pps pulse happens while sleeping in > > cpu_idle(); the path that unwinds out of there leads to calling > > tc_windup 3 times, iirc, which means the value latched in the > > timecounter hardware belongs to a set of timehands whose generation > > count is not valid anymore because of all the tc_windup calls, and the > > event gets discarded because of the generation mismatch. > > Is the problem specific to the platform then ? I think it is specific to timecounters that implement pps capture using the polling mechanism (tc_poll_pps pointer is non-NULL), and I remember that there was something about the sequence of events that made me think it would only be a problem on a single-core system. Or maybe just way more likely on a single-core system (I've got similarly- structured drivers on multicore systems that don't have this problem). But I don't think it was so much platform-specific as specific to that pair of conditions. > > The issue you see might come from tc_ticktock(), but it is strange. > All calls to tc_windup() are serialized by spinlock. tc_ticktock(), > potentially being called from interrupt context, uses try-lock to avoid > spinning while other CPU does the update. But this is impossible on UP > machine, because spinlock in top half disables interrupts, which means > that event that triggers tc_ticktock() call and which trylock fails > should be impossible, until the top half finishes. > > Having too many timehands is bad because reader get a higher chance to > find obsoleted th pointer. With the rework of the generation counts for > timehands reader would not use stalled data, but it might have to spin > somewhat prolonged time. > > In fact, it might be worth trying reducing the timehands count from two > to one, since then there is no non-current th at all. The driver involved is arm/ti/am335x/am335x_dmptpps.c. The basic sequence that occurs is: The dmtpps_poll() function is called (via the timecounter.tc_poll_pps pointer), and it calls pps_capture(), then schedules a taskqueue_fast task to do the rest of the processing. That task function calls pps_event(), and most of the time pps_event() does nothing because it hits this early out: /* If the timecounter was wound up underneath us, bail out. */ if (pps->capgen == 0 || pps->capgen != atomic_load_acq_int(&pps->capth->th_generation)) return; I forget the exact sequence, but there was something about getting from the point where the taskqueue task begins running to the point where it calls pps_event() that almost always burned through 3 timehands generations, but if the machine was busy enough, sometimes there would only be one windup and the event would get processed properly. I don't remember why I used a taskqueue task to separate the capture and event processing. Maybe so I could use a mutex to protect the pps event state, and it was written before I extended the internal pps api stuff to let drivers use a spin mutex with pps_ioctl(). -- Ian