Date: Tue, 7 Nov 2017 14:17:10 -0700 From: Warner Losh <imp@bsdimp.com> To: Konstantin Belousov <kostikbel@gmail.com> Cc: John Baldwin <jhb@freebsd.org>, src-committers <src-committers@freebsd.org>, "svn-src-all@freebsd.org" <svn-src-all@freebsd.org>, "svn-src-head@freebsd.org" <svn-src-head@freebsd.org> Subject: Re: svn commit: r325506 - in head: sbin/ifconfig sys/net sys/sys Message-ID: <CANCZdfpiCZph8K_WXH2id0oedJu4TYmd0rL3hQM9NzN4Tf4opg@mail.gmail.com> In-Reply-To: <20171107174431.GQ2566@kib.kiev.ua> References: <201711070929.vA79TFTc096109@repo.freebsd.org> <2707886.I9a5Fez8At@ralph.baldwin.cx> <20171107173926.GP2566@kib.kiev.ua> <20171107174431.GQ2566@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Nov 7, 2017 at 10:44 AM, Konstantin Belousov <kostikbel@gmail.com> wrote: > On Tue, Nov 07, 2017 at 07:39:26PM +0200, Konstantin Belousov wrote: > > On Tue, Nov 07, 2017 at 09:06:52AM -0800, John Baldwin wrote: > > > On Tuesday, November 07, 2017 09:29:15 AM Konstantin Belousov wrote: > > > > Author: kib > > > > Date: Tue Nov 7 09:29:14 2017 > > > > New Revision: 325506 > > > > URL: https://svnweb.freebsd.org/changeset/base/325506 > > > > > > > > Log: > > > > Add a place for a driver to report rx timestamps in nanoseconds > from > > > > boot for the received packets. > > > > > > > > The rcv_tstmp field overlaps the place of Ln header length > indicators, > > > > not used by received packets. The basic pkthdr rearrangement > change > > > > in sys/mbuf.h was provided by gallatin. > > > > > > > > There are two accompanying M_ flags: M_TSTMP means that there is > the > > > > timestamp (and it was generated by hardware). > > > > > > > > Another flag M_TSTMP_HPREC indicates that the timestamp is > > > > high-precision. Practically M_TSTMP_HPREC means that hardware > > > > provided additional precision comparing with the stamps when the > flag > > > > is not set. E.g., for ConnectX all packets are stamped by hardware > > > > when PCIe transaction to write out the completion descriptor is > > > > performed, but PTP packet are stamped on port. For Intel cards, > when > > > > PTP assist is enabled, only PTP packets are stamped in the limited > > > > number of registers, so if Intel cards ever start support this > > > > mechanism, they would always set M_TSTMP | M_TSTMP_HPREC if > hardware > > > > timestamp is present for the given packet. > > > > > > > > Add IFCAP_HWRXTSTMP interface capability to indicate the support > for > > > > hardware rx timestamping, and ifconfig(8) command to toggle it. > > > > > > Hmm, other NICs (Chelsio T4 and later for example) support timestamps > that > > > aren't in nanoseconds but some other frequency (which are themselves > useful). > > > It would be nice to have a more flexible interface that supports not > only ns > > > timestamps. Perhaps a way to expose a direct hardware timestamp as a > > > "number" without a specific frequency? > > > > ConnectX does not provide ns-clocked counter either. It is some internal > > clock driven by a cristal with > 100MHz frequency. > > > > There is no much space in the pkthdr, and the request to provide the > > timestamp was in the context where the wall clock or some closely related > > timer is needed. Of course, I can put raw hardware timestamp into the > > packet header, but only instead of the reduced value. Then the consumer > > of the timestamp would need to find the interface which received the > > packet and call its method to convert ? We have only one consumer in > > tree (SO_TIMESTAMP) and perhaps one possible another consumer (TCP) for > > this data, both of which require wall clock, so would need to call into > > the method. > > > > Also please see the discussion in the referenced review about accuracy of > > the convertion. > > > > Important example are Intel cards where is only limited number of > > latched registers, and only PtP packets are stamped. This (and some > > quirk in ConnectX) explains the high-precision flag. > > > > And another consideration which was one of the strong argument for me > when I thought about this stuff: the convertion of the hardware timestamp > to the useful clock stamp depends on the clock calibraton data which might > not be available long time after the packet receive. In other words, when > the consumer would call into the interface method to convert raw timestamp, > it might be already not convertable (in kern_tc.c terms, timehands were > switched by tc_windup()). > On the other hand, if you are trying to steer or calibrate the clock in userland, doing the conversions may be losing data if the clocks are fast enough... But that was one of the big lessons I learned when doing timing gear: raw data is better than cooked data in userland, even if you instantly cook it in the same way you would have cooked it in the kernel before publishing. Given the quality of these clocks, though, I doubt it matters much at all... Warner
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfpiCZph8K_WXH2id0oedJu4TYmd0rL3hQM9NzN4Tf4opg>