From owner-dev-commits-src-all@freebsd.org Wed Jun 2 16:34:29 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id ADB8C646B98; Wed, 2 Jun 2021 16:34:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FwF3n46SMz3t44; Wed, 2 Jun 2021 16:34:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7544A1A41C; Wed, 2 Jun 2021 16:34:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 152GYTXF053453; Wed, 2 Jun 2021 16:34:29 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 152GYTbp053452; Wed, 2 Jun 2021 16:34:29 GMT (envelope-from git) Date: Wed, 2 Jun 2021 16:34:29 GMT Message-Id: <202106021634.152GYTbp053452@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Ceri Davies Subject: git: 774bb1c256fb - main - periodic: add support for .xz and .zcat compressed logs MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: ceri X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 774bb1c256fbc58a7e8d0d1f7d6427007105b334 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jun 2021 16:34:29 -0000 The branch main has been updated by ceri (doc committer): URL: https://cgit.FreeBSD.org/src/commit/?id=774bb1c256fbc58a7e8d0d1f7d6427007105b334 commit 774bb1c256fbc58a7e8d0d1f7d6427007105b334 Author: Ceri Davies AuthorDate: 2021-06-02 16:28:28 +0000 Commit: Ceri Davies CommitDate: 2021-06-02 16:28:28 +0000 periodic: add support for .xz and .zcat compressed logs Also improve temporary file usage in 200.accounting, add an xref to zstd(1) to newsyslog.conf.5, and clarify in periodic.conf that "daily accounting" means process accounting and "monthly accounting" is login accounting. PR: 253868 Reviewed by: allanjude Approved by: blackend (mentor) Differential Revision: https://reviews.freebsd.org/D29267 --- share/man/man5/periodic.conf.5 | 2 +- usr.sbin/newsyslog/newsyslog.conf.5 | 1 + .../periodic/etc/daily/460.status-mail-rejects | 6 +++++ usr.sbin/periodic/etc/monthly/200.accounting | 30 ++++++++++++++++------ usr.sbin/periodic/etc/security/800.loginfail | 2 ++ usr.sbin/periodic/etc/security/900.tcpwrap | 2 ++ 6 files changed, 34 insertions(+), 9 deletions(-) diff --git a/share/man/man5/periodic.conf.5 b/share/man/man5/periodic.conf.5 index 46174da2772a..ef80578e5372 100644 --- a/share/man/man5/periodic.conf.5 +++ b/share/man/man5/periodic.conf.5 @@ -365,7 +365,7 @@ daily. .Pq Vt bool Set to .Dq Li YES -if you want to rotate your daily accounting files. +if you want to rotate your daily process accounting files. No rotations are necessary unless .Va accounting_enable is enabled in diff --git a/usr.sbin/newsyslog/newsyslog.conf.5 b/usr.sbin/newsyslog/newsyslog.conf.5 index b897389b99dd..cf1a36305b00 100644 --- a/usr.sbin/newsyslog/newsyslog.conf.5 +++ b/usr.sbin/newsyslog/newsyslog.conf.5 @@ -415,6 +415,7 @@ entry: .Xr bzip2 1 , .Xr gzip 1 , .Xr xz 1 , +.Xr zstd 1 , .Xr syslog 3 , .Xr chown 8 , .Xr newsyslog 8 , diff --git a/usr.sbin/periodic/etc/daily/460.status-mail-rejects b/usr.sbin/periodic/etc/daily/460.status-mail-rejects index ce633640b6a4..6161a5525cc1 100755 --- a/usr.sbin/periodic/etc/daily/460.status-mail-rejects +++ b/usr.sbin/periodic/etc/daily/460.status-mail-rejects @@ -52,6 +52,12 @@ case "$daily_status_mail_rejects_enable" in elif [ -f /var/log/maillog.$n.bz2 ] then bzcat -fc /var/log/maillog.$n.bz2 + elif [ -f /var/log/maillog.$n.xz ] + then + xzcat -f /var/log/maillog.$n.xz + elif [ -f /var/log/maillog.$n.zst ] + then + zstdcat -fc /var/log/maillog.$n.zst fi n=$(($n - 1)) done diff --git a/usr.sbin/periodic/etc/monthly/200.accounting b/usr.sbin/periodic/etc/monthly/200.accounting index 46f153de535a..9d50ba182777 100755 --- a/usr.sbin/periodic/etc/monthly/200.accounting +++ b/usr.sbin/periodic/etc/monthly/200.accounting @@ -18,16 +18,30 @@ case "$monthly_accounting_enable" in W=/var/log/utx.log rc=0 remove=NO + filetoread=$W.0 if [ ! -f $W.0 ] then - if [ -f $W.0.gz ] + if [ -f $W.0.gz ] || [ -f $W.0.bz2 ] || [ -f $W.0.xz ] || [ -f $W.0.zst ] then + TMP=`mktemp -t accounting` remove=YES - zcat $W.0.gz > $W.0 || rc=1 - elif [ -f $W.0.bz2 ] - then - remove=YES - bzcat $W.0.bz2 > $W.0 || rc=1 + filetoread=$TMP + if [ -f $W.0.gz ] + then + zcat $W.0.gz > $TMP || rc=1 + elif [ -f $W.0.bz2 ] + then + bzcat $W.0.bz2 > $TMP || rc=1 + elif [ -f $W.0.xz ] + then + xzcat $W.0.xz > $TMP || rc=1 + elif [ -f $W.0.zst ] + then + zstdcat $W.0.zst > $TMP || rc=1 + else + # shouldn't get here, unless something disappeared under us. + rc=2 + fi else echo '$monthly_accounting_enable is set but' \ "$W.0 doesn't exist" @@ -39,10 +53,10 @@ case "$monthly_accounting_enable" in echo "" echo "Doing login accounting:" - rc=$(ac -p -w $W.0 | sort -nr -k 2 | tee /dev/stderr | wc -l) + rc=$(ac -p -w $filetoread | sort -nr -k 2 | tee /dev/stderr | wc -l) [ $rc -gt 0 ] && rc=1 fi - [ $remove = YES ] && rm -f $W.0;; + [ $remove = YES ] && rm -f $TMP;; *) rc=0;; esac diff --git a/usr.sbin/periodic/etc/security/800.loginfail b/usr.sbin/periodic/etc/security/800.loginfail index 4c78f441639a..ef169482001c 100755 --- a/usr.sbin/periodic/etc/security/800.loginfail +++ b/usr.sbin/periodic/etc/security/800.loginfail @@ -53,6 +53,8 @@ catmsgs() { case $f in *.gz) zcat -f $f;; *.bz2) bzcat -f $f;; + *.xz) xzcat -f $f;; + *.zst) zstdcat -f $f;; esac done [ -f ${LOG}/auth.log ] && cat $LOG/auth.log diff --git a/usr.sbin/periodic/etc/security/900.tcpwrap b/usr.sbin/periodic/etc/security/900.tcpwrap index 10b02e9e2bb6..b6c44126bc1c 100755 --- a/usr.sbin/periodic/etc/security/900.tcpwrap +++ b/usr.sbin/periodic/etc/security/900.tcpwrap @@ -53,6 +53,8 @@ catmsgs() { case $f in *.gz) zcat -f $f;; *.bz2) bzcat -f $f;; + *.xz) xzcat -f $f;; + *.zst) zstdcat -f $f;; esac done [ -f ${LOG}/messages ] && cat $LOG/messages