From owner-freebsd-hackers@freebsd.org Sun Apr 1 10:24:59 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C674BF80400 for ; Sun, 1 Apr 2018 10:24:59 +0000 (UTC) (envelope-from woodsb02@gmail.com) Received: from mail-ot0-x22b.google.com (mail-ot0-x22b.google.com [IPv6:2607:f8b0:4003:c0f::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3D96D826DD; Sun, 1 Apr 2018 10:24:59 +0000 (UTC) (envelope-from woodsb02@gmail.com) Received: by mail-ot0-x22b.google.com with SMTP id f47-v6so4318797oth.2; Sun, 01 Apr 2018 03:24:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Xpd4lSXq+pjYwP4TVATcHAtMF587pqHGpz2Ru3KzZPs=; b=juF6g/LlJ552j78u2JxCXWBTXsGJGY9tp2lB4kQzJvyHryv6YWmuenNXJGbPw8RkOb 7qA2rBi+ZtSWFO29GeM4+FohI0Z5FK/3rWpRovif3QbVAM2iWcmSfhVkP0HvB4TRfexT tpWEZMlzD5BoFzsvHaQXRsBCtW8tc9HJLM2fUucfgxFDSTBXsyO/4Q3Nlrh0TNVMeJVQ AEblt/rq4EVD9JFW8SHfgsGavpMVNSUVg5d8zrQQy1t4A2hme7KgG6QcH9Mr5sWc7pR8 iOyxUbHfT2iSP7TYedReVhErBBCMfabiAHM0p8O1XsiSZH+F1hvwMIguTaZzQBzYqU0w k6rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Xpd4lSXq+pjYwP4TVATcHAtMF587pqHGpz2Ru3KzZPs=; b=cKacDVgmiE/WRNDZ5SUYCRZm6qMlK9pl6v4hcR8JpRKZkDw34ElJ8pOw9biiq4zgCC DEk/Wbz0DjK8EyZUeXjU32VTAUkm++Sht8vVkvX1hhMTru8PsKPUINm1arhL6S9taiAe cPqays8HGVm8cFzJKLQvl8/S0SzA8BlRugJHpjJPCS0+NODJcAi5ML2fxRtQGh1LfFOh B31cdM2jdEl9aYP6JAYPpPcYQOaPy385nsnJoVAMS+Pgn3NFCO/MNmKFfeuT+DbPWP1C ZM5zS2foW9hLt0yQ3EBGidzCcm/ugxW828IKPd5zmjmBX0VzL93DEiRMFx+1WA1wOvSf hhMQ== X-Gm-Message-State: ALQs6tDWAuXJCL/Crx70Zqz5X3myueEmRBasygbHpoENCnxb3dSjKCE7 JJqE4lcuDJPRWtVKRN3HygGaPGh6ift5J6C8Nws3PFoj X-Google-Smtp-Source: AIpwx4/SqXQhwg6IqiLzFV7pFU6+Txs0xbe8MBDf48/d6r+z3IwWHSgLY7VpSVe8/d4gu6SQkj/iDGdJE0ExaJyJrmU= X-Received: by 2002:a9d:2785:: with SMTP id c5-v6mr3456907otb.260.1522578298467; Sun, 01 Apr 2018 03:24:58 -0700 (PDT) MIME-Version: 1.0 Received: by 10.201.15.247 with HTTP; Sun, 1 Apr 2018 03:24:57 -0700 (PDT) In-Reply-To: <4fd424ca-ff93-315c-fa87-fb9c738b9b50@freebsd.org> References: <4fd424ca-ff93-315c-fa87-fb9c738b9b50@freebsd.org> From: Ben Woods Date: Sun, 1 Apr 2018 18:24:57 +0800 Message-ID: Subject: Re: Add option -Z to syslogd(8) to use ISO 8601 timestamps To: Julian Elischer Cc: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Apr 2018 10:25:00 -0000 On 1 April 2018 at 11:25, Julian Elischer wrote: > is there a matching change for syslog(3) to actually generate new > timestamps? > > The syslog protocol is unique that either end can be responsible for the > timestamp. > The approach the OpenBSD team has taken has been to remove all timestamping from syslog(3): "Do not include a timestamp in the syslog message. There is no need -- syslogd will fill it in immediately upon reception on the other side of sendsyslog(2). Our libc only talks to our syslogd, which will fix the timestamp before forwarding. syslog_r has done this for a long time already. ok tedu bluhm" https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/lib/libc/gen/syslog.c.diff?r1=1.32&r2=1.33&f=h https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/lib/libc/gen/syslog_r.c.diff?r1=1.9&r2=1.10&f=h The FreeBSD syslog(3) code still adds a timestamp, so there is an option to either remove this code from syslog(3) similar to OpenBSD, or update the syslog(3) code to support either timestamp. My testing shows that if syslogd(8) is not running at the time a message comes in from syslog(3), the message is dropped (not buffered until syslogd is once again running). This means there would be no significant time delay issues causing incorrect timestamps if syslogd(8) adds the timestamp instead of syslog(3). https://svnweb.freebsd.org/base/head/lib/libc/gen/syslog.c?revision=326025&view=markup#l171 Note that when testing my patch with the logger(1) tool, which uses syslog(3), I have found that the RFC3164 timestamp format applied by syslog(3) is replaced with the RFC5424 timestamp format in syslogd(8), as per the description below. In the case of the new timestamp form you would have to strip off the > incoming old form timestamp and replace it. > Indeed. This is handled by the changes in the parsemsg() function, which effectively sets the -T option (RemoteAddDate = 1) if it detects the old timestamp format when the -Z option has been set. Note that my proposed implementation does not convert the old format to the new format (and assume the current year), but instead simply strips the old timestamp and applies a new one. This is as per OpenBSD's implementation. Regards, Ben