Date: Tue, 7 Nov 2017 19:44:31 +0200 From: Konstantin Belousov <kostikbel@gmail.com> To: John Baldwin <jhb@freebsd.org> Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r325506 - in head: sbin/ifconfig sys/net sys/sys Message-ID: <20171107174431.GQ2566@kib.kiev.ua> In-Reply-To: <20171107173926.GP2566@kib.kiev.ua> References: <201711070929.vA79TFTc096109@repo.freebsd.org> <2707886.I9a5Fez8At@ralph.baldwin.cx> <20171107173926.GP2566@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Nov 07, 2017 at 07:39:26PM +0200, Konstantin Belousov wrote: > On Tue, Nov 07, 2017 at 09:06:52AM -0800, John Baldwin wrote: > > On Tuesday, November 07, 2017 09:29:15 AM Konstantin Belousov wrote: > > > Author: kib > > > Date: Tue Nov 7 09:29:14 2017 > > > New Revision: 325506 > > > URL: https://svnweb.freebsd.org/changeset/base/325506 > > > > > > Log: > > > Add a place for a driver to report rx timestamps in nanoseconds from > > > boot for the received packets. > > > > > > The rcv_tstmp field overlaps the place of Ln header length indicators, > > > not used by received packets. The basic pkthdr rearrangement change > > > in sys/mbuf.h was provided by gallatin. > > > > > > There are two accompanying M_ flags: M_TSTMP means that there is the > > > timestamp (and it was generated by hardware). > > > > > > Another flag M_TSTMP_HPREC indicates that the timestamp is > > > high-precision. Practically M_TSTMP_HPREC means that hardware > > > provided additional precision comparing with the stamps when the flag > > > is not set. E.g., for ConnectX all packets are stamped by hardware > > > when PCIe transaction to write out the completion descriptor is > > > performed, but PTP packet are stamped on port. For Intel cards, when > > > PTP assist is enabled, only PTP packets are stamped in the limited > > > number of registers, so if Intel cards ever start support this > > > mechanism, they would always set M_TSTMP | M_TSTMP_HPREC if hardware > > > timestamp is present for the given packet. > > > > > > Add IFCAP_HWRXTSTMP interface capability to indicate the support for > > > hardware rx timestamping, and ifconfig(8) command to toggle it. > > > > Hmm, other NICs (Chelsio T4 and later for example) support timestamps that > > aren't in nanoseconds but some other frequency (which are themselves useful). > > It would be nice to have a more flexible interface that supports not only ns > > timestamps. Perhaps a way to expose a direct hardware timestamp as a > > "number" without a specific frequency? > > ConnectX does not provide ns-clocked counter either. It is some internal > clock driven by a cristal with > 100MHz frequency. > > There is no much space in the pkthdr, and the request to provide the > timestamp was in the context where the wall clock or some closely related > timer is needed. Of course, I can put raw hardware timestamp into the > packet header, but only instead of the reduced value. Then the consumer > of the timestamp would need to find the interface which received the > packet and call its method to convert ? We have only one consumer in > tree (SO_TIMESTAMP) and perhaps one possible another consumer (TCP) for > this data, both of which require wall clock, so would need to call into > the method. > > Also please see the discussion in the referenced review about accuracy of > the convertion. > > Important example are Intel cards where is only limited number of > latched registers, and only PtP packets are stamped. This (and some > quirk in ConnectX) explains the high-precision flag. > And another consideration which was one of the strong argument for me when I thought about this stuff: the convertion of the hardware timestamp to the useful clock stamp depends on the clock calibraton data which might not be available long time after the packet receive. In other words, when the consumer would call into the interface method to convert raw timestamp, it might be already not convertable (in kern_tc.c terms, timehands were switched by tc_windup()).
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20171107174431.GQ2566>