From owner-freebsd-hackers@freebsd.org Sun Apr 1 14:13:52 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C3A66F9E99E for ; Sun, 1 Apr 2018 14:13:52 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1a.eu.mailhop.org (outbound1a.eu.mailhop.org [52.58.109.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 458706E230 for ; Sun, 1 Apr 2018 14:13:51 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: e34b07f2-35b6-11e8-91c6-33ffc249f3e8 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound1.eu.mailhop.org (Halon) with ESMTPSA id e34b07f2-35b6-11e8-91c6-33ffc249f3e8; Sun, 01 Apr 2018 14:13:46 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id w31EDdwk075763; Sun, 1 Apr 2018 08:13:39 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <1522592019.49673.166.camel@freebsd.org> Subject: Re: Add option -Z to syslogd(8) to use ISO 8601 timestamps From: Ian Lepore To: Ben Woods , Julian Elischer Cc: "freebsd-hackers@freebsd.org" Date: Sun, 01 Apr 2018 08:13:39 -0600 In-Reply-To: References: <4fd424ca-ff93-315c-fa87-fb9c738b9b50@freebsd.org> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Apr 2018 14:13:52 -0000 On Sun, 2018-04-01 at 18:24 +0800, Ben Woods wrote: > My testing shows that if syslogd(8) is not running at the time a > message > comes in from syslog(3), the message is dropped (not buffered until > syslogd > is once again running). > This means there would be no significant time delay issues causing > incorrect timestamps if syslogd(8) adds the timestamp instead of > syslog(3). > https://svnweb.freebsd.org/base/head/lib/libc/gen/syslog.c?revision=3 > 26025&view=markup#l171 The important issue with source-side timestamping doesn't have to do with whether syslogd is running or not, it has to do with mutiple threads and processes feeding data to syslogd at high speed and accurately timestamping the events being logged. Mutexes are involved in libc syslog(3) for multithreaded apps, and buffers are involved which can lead to arbitrary ordering of messages generated by various processes, and only source-side timestamps can make it clear in what order things really happened. In addition, milliseconds are not nearly fine-grained enough for fractional seconds.  Even microseconds aren't really good enough on modern hardware. -- Ian