From owner-freebsd-usb@freebsd.org Fri Aug 2 21:16:42 2019 Return-Path: Delivered-To: freebsd-usb@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8F7F5BC012 for ; Fri, 2 Aug 2019 21:16:42 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 460g2f3KLsz4WNq for ; Fri, 2 Aug 2019 21:16:42 +0000 (UTC) (envelope-from ian@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 703EBBC011; Fri, 2 Aug 2019 21:16:42 +0000 (UTC) Delivered-To: usb@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7005BBC010 for ; Fri, 2 Aug 2019 21:16:42 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound3d.ore.mailhop.org (outbound3d.ore.mailhop.org [54.186.57.195]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 460g2f1rkKz4WNp for ; Fri, 2 Aug 2019 21:16:38 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1564780597; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=GHUxBdYL7IDGdDqqxpfsSwBcWX5R5fbciVvFqqEDcl0ZCUus0Tq9FQR2eocBL0cbiYyk396cZr6u3 q3mGiz9ZCN8G3eaxfxPx7c1vJ/rpknT2N4fYGVDNX1NmphjsKPIr6HikgEi+5Boq/4EcgdDGB22Q/Z VV8szdq4+6w7T8+eUjB9m6PGUGfcBZVx7xo3rY7bRFBT1LNaPHK7J3yGYEjwgQCtPFGZTcI9XkY5P+ ccvzwOMSqqDzrLipmVQTLurYuNNUROtEZ8yDRnbbSx3j4tbk/eRrMblQyQXIYfMkrF5zRQzuJhmUb+ kdHRnXjyqbUFAsl8wHzBSTrmboQJdwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:date:to:from:subject: message-id:dkim-signature:from; bh=fk5Wcs3Iz1Lm5Hkyzn5pZaMfjSL3G9a8YbzYHGhqb34=; b=lBoVF+1vjeyUARYfAbciFizW7dHvX+clMxT/SCQPOVImIssW3cJEry7DTmMV9FyNLBhVz7fEYnZhk 3YP6koJEJfT25vCy6VUE8WGNhxKOp9QiaML5y91FC4i29yX+U+ihkM3+193eOgTDECdlaIyLnaaMes gWG41+HSTXvuyy2CeVDDfG51j7VYidCvdlnNUZAAdl8H53HbtGs7dTpv6qAj476BISdn0eHlEW3qUZ S7nfbmy3Eg5SR0B9orTEx59VAiQVxTRyB8mcru6QIwP8EUxpgQZBc//an6pNcBAsGrK3yBpd3IGh4t VA3j09C1A/XWvIbUSkQrytuFQ/+B7zg== ARC-Authentication-Results: i=1; outbound3.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:date:to:from:subject: message-id:from; bh=fk5Wcs3Iz1Lm5Hkyzn5pZaMfjSL3G9a8YbzYHGhqb34=; b=FGYDuZheCCepVmtImPrmA/wCW99+csxDDanf3t2Qf3j91bbyjNg9SW8yK4Am4JE/csFSfinkn0qlI 7XMfA6yndfkoRidw7Xz1yEgwiWlQunqO75qewmFkq63COFdYOiLCglR45oqUOoAUZ/wyBW58qx8qsJ i/wivb5z5AQAJGMJ9xxcW+30Al4ljJvXL+wld2g0TJeK2fkkEK5vpaER1y34tKR5QznCvBGuc9nQWp X9bypbsHiKMzz1T1jwlqu4eWqNP5ofyzdVgdeW1jepRNxtbOWhPNG9i9eLltuQlmQ25FjHMxifrmQ7 Qsh6am7wPvq8rUsUw/+ZGSP2k3D+4kQ== X-MHO-RoutePath: aGlwcGll X-MHO-User: cecc6a2d-b56a-11e9-b679-cdd75d6ce7a8 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound3.ore.mailhop.org (Halon) with ESMTPSA id cecc6a2d-b56a-11e9-b679-cdd75d6ce7a8; Fri, 02 Aug 2019 21:16:36 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x72LGZUk006832; Fri, 2 Aug 2019 15:16:35 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: Subject: Is it a good idea to use a usb-serial adapter for PPS input? Yes, it is. From: Ian Lepore To: "freebsd-arm@FreeBSD.org" , usb@FreeBSD.org Date: Fri, 02 Aug 2019 15:16:34 -0600 Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 460g2f1rkKz4WNp X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.31 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_SPAM_SHORT(0.31)[0.306,0]; ASN(0.00)[asn:16509, ipnet:54.186.0.0/15, country:US] X-BeenThere: freebsd-usb@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FreeBSD support for USB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Aug 2019 21:16:42 -0000 Since we added support for accepting a PPS signal on a USB-serial adapter a couple years ago, I've seen people express reluctance to use it several times. Usually they cite concerns about latency and jitter. I decided it was time to do some rigorous testing, and post the results. Complete details on the test setup appear below. The tl;dr summary is: A single PPS source is fanned out to 4 different types of inputs on a Wandboard system (armv7, imx6 SoC). Ntpd is configured to steer only to PPS(0), and the offset and jitter numbers for the other sources allow comparing the various PPS processing paths. After running about 12 hours, the results are: remote refid st t when poll reach delay offset jitter ===================================================================== oPPS(0) .gpt. 0 l 8 16 377 0.000 0.000 0.002 PPS(1) .gpio. 0 l 7 16 377 0.000 -0.002 0.002 PPS(2) .usb1. 0 l 6 16 377 0.000 -0.201 0.034 PPS(3) .usb2. 0 l 5 16 377 0.000 -0.215 0.026 *tflex.my.lan .GPS. 1 u 32 64 377 0.568 0.038 0.061 PPS(0) is fed to a hardware timer block within the imx6 SoC. The PPS pulse triggers hardware capture of the kernel clock, eliminating latency and jitter due to interrupt processing. PPS(1) is fed to a generic gpio input pin on the SoC, handled by the standard gpiopps driver processing a pin-change interrupt. PPS(2) is an FTDI 232R, a USB 1.1 serial adapter, connected to a port on a USB 2.0 hub that's connected to a USB 2.0 host port on the Wandboard. PPS(3) is an FTDI 4232H, a USB 2.0 serial adapter, connected to a port on the same USB hub as PPS(2). Unfortunately, the uart driver for the imx6 SoC doesn't support the CTS or CD signals, so I couldn't measure native uart performance directly. I would expect the performance to be comparable to the gpio pin input. As shown above, there is a 2 microsecond latency and virtually no jitter on the GPIO input. The USB 1.1 and USB 2.0 adapters performed essentially identically to each other, with about 200 microseconds of latency and negligible jitter. There was no difference in performance between using the CTS versus the CD pins on the adapters for input. To see if lots of USB bus activity increased latency or noise, I connected a USB SATA dock containing an SSD drive to the same USB hub as the serial adapters, and ran a continuous dd(1) from the drive to /dev/null. Surprisingly, there was absolutely no difference in the results during that run. Most people are not worried about their kernel clock being 200 microseconds off from UTC, even if they're using the PPS signal from a GPS receiver. So I think most people should feel completely at ease using a USB serial adapter as the input device for a PPS signal. Test setup details... PPS measurements are made using the kernel clock. Typically the kernel clock is sourced from a hardware clock which isn't particularly accurate in terms of frequency. All clocks drift; cheap crystals on computer boards drift a lot. Ntpd will align both the frequency and phase of the kernel clock using a PPS signal, but to compare the various processing paths a PPS signal can go through, you must be able to determine how much error came from the reference path and how much from the path being tested. In an ideal world, there would be no measurement jitter, or frequency drift in the kernel clock, and thus all PPS measurements made would be directly comparable to each other without having to ascribe any part of the differences between sources to the kernel clock. I am able to configure a Wandboard imx6 system so that the frequency and phase alignment of kernel time is "perfect" with respect to one of the PPS inputs. Since all the PPS inputs are sourced by fanning out the same source PPS signal, any difference in the offset or jitter reported by ntpd directly represents differences in the processing paths taken by those signals. The test setup consists of a commercial precision timing system which generates a 10 MHz clock signal from a GPS-disciplined rubidium oscillator, and it generates a PPS pulse that is derived from that 10 MHz clock using a simple "divide by 10 million" counter. So the leading edge of the PPS pulse is phase-coherent with the leading edge of one of the clock pulses. The 10 MHz clock signal is fed to an external clock input pin on the imx6 SoC. Within the imx6 timer block, that clock signal drives a 32- bit counter register, and that counter is used to implement a kernel timecounter. The PPS signal is also fed into that imx6 timer block, and the leading edge of the PPS pulse causes the hardware to latch the current value of the 32-bit timecounter into a capture register. This captured value is then used to generate a PPS measurement that doesn't incorporate any interrupt processing latency or jitter. Because the PPS pulse and the kernel timecounter clock are derived from the same source, the kernel clock will never appear to drift in frequency. That is, when ntpd compares two successive PPS measurements, it will always find that exactly 1 billion nanoseconds elapased between PPS pulses. At startup, there will be some offset between the kernel clock and the PPS pulse, and ntpd will slowly steer out that error until the beginning of the kernel's second exactly matches the PPS pulse edge. I force ntpd to step the clock at startup; that typically leaves about a hundred microseconds of offset, and it takes a couple hours to slowly reduce that to zero. Once that happens, the system is in a steady state, where ntpd will never again have to apply any correction to the kernel clock. You can see this state in the output of ntptime(8) which reports on the kernel clock's phase and frequency. The important numbers are flagged with => root # ntptime ntp_gettime() returns code 0 (OK) time e0eed6a9.b6875d58 Fri, Aug 2 2019 15:35:05.713, (.713003500), maximum error 8000 us, estimated error 1 us, TAI offset 0 ntp_adjtime() returns code 0 (OK) modes 0x0 (), => offset 0.000 us, frequency 0.000 ppm, interval 4 s, maximum error 8000 us, estimated error 1 us, status 0x2001 (PLL,NANO), time constant 4, precision 1.000 us, tolerance 496 ppm, => pps frequency 0.000 ppm, stability 0.000 ppm, jitter 0.000 us, intervals 0, jitter exceeded 0, stability exceeded 0, errors 0. Finally, for making these measurements, I wasn't worried about jitter and latency that occurs down in the "handful of nanoseconds" range. For example, a single PPS source is fanned out into 4 different inputs on the Wandboard, and I didn't worry at all about things like using equal cable lengths, because I was looking for differences on the order of microseconds, not nanos (and with a 10 MHz timecounter, the measurement resolution is 100ns, so I can't see those little differences anyway). -- Ian