From owner-svn-src-head@freebsd.org Mon May 15 18:51:49 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CFD4ED6E609 for ; Mon, 15 May 2017 18:51:49 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1b.ore.mailhop.org (outbound1b.ore.mailhop.org [54.200.247.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B174935B for ; Mon, 15 May 2017 18:51:49 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: 9db9ad73-399f-11e7-bfb5-0d159cd3c324 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.ore.mailhop.org (Halon) with ESMTPSA id 9db9ad73-399f-11e7-bfb5-0d159cd3c324; Mon, 15 May 2017 18:52:17 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id v4FIpfXJ001778; Mon, 15 May 2017 12:51:41 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <1494874301.59865.116.camel@freebsd.org> Subject: Re: svn commit: r318250 - in head: etc etc/newsyslog.conf.d etc/syslog.d tools/build/mk From: Ian Lepore To: "Ngie Cooper (yaneurabeya)" Cc: src-committers , "svn-src-all@freebsd.org" , "svn-src-head@freebsd.org" Date: Mon, 15 May 2017 12:51:41 -0600 In-Reply-To: <60A3FDE1-F68E-49D5-972F-A0C8993E0FEE@gmail.com> References: <201705131537.v4DFbgWV045290@pdx.rh.CN85.dnsmgr.net> <2229085.lB46rKsq7o@ralph.baldwin.cx> <1494870201.59865.103.camel@freebsd.org> <0B1B76CF-9CC4-422B-8356-AE425413850B@gmail.com> <60A3FDE1-F68E-49D5-972F-A0C8993E0FEE@gmail.com> Content-Type: text/plain; charset="windows-1251" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 May 2017 18:51:49 -0000 On Mon, 2017-05-15 at 11:24 -0700, Ngie Cooper (yaneurabeya) wrote: > > > > On May 15, 2017, at 10:47, Ngie Cooper (yaneurabeya) > mail.com> wrote: > > > > > > > > > > On May 15, 2017, at 10:43, Ian Lepore wrote: > > … > > > > > > > > That's only a good argument for keeping the lines in the > > > monolithic > > > file if those lines will be ignored when a file in the .conf.d > > > directory provides conflicting config.  Otherwise my embedded > > > product > > > that drops different rules for rotating /var/log/messages into > > > .conf.d > > > STILL has to programmatically edit the monolithic file to remove > > > the > > > standard rule(s). > > Bingo. This is part of the reason why I did this, apart from > > being selfish in not wanting to handle a handful of unnecessary > > entries in newsyslog/syslogd on all of my systems where I set these > > knobs to no. > > Literally all I did was `dd+p` in vim in the new files. I didn’t > > add or subtract any overall entries. > And for context, we (Isilon) run newsyslog more frequently than > upstream (sometimes every couple minutes, sometimes multiple times an > hour/day), attributing to unnecessary creation and modification of > log files (like this) that we (Isilon) don’t care about at all, which > causes other potential issues with root media wear, potential for > filesystem corruption, reduced number of available inodes/space > because it allocates at least the frag size for a file, etc. > We also run log rotation frequently (every 5 minutes) on our embedded systems.  That was super-important back when we logged to a ram filesystem, then newsyslog would rotate and compress from there to sdcard.  Now we just log to sdcard because when something goes wrong it's almost always the end of the log that has the clues (unless something triggers runaway log spewage) and we often lost it. A local modification we still apply to our syslogd allows you to specify in syslog.conf that for any given file, syslogd itself should limit the size of that file if the rotation doesn't take care of it soon enough.  That was important to prevent filling up a ram filesystem with runaway logging.  Now it's a feature we don't use much, but before I throw it overboard I thought I'd mention it here in case others have any use for it... The way the feature worked was you could put at the end of a line in syslog.conf "R nnnnK" -- the 'R' had a historical meaning that's now lost, so we bacronymed it to mean "Recycle" -- when the log grows to nnnnK in size, the last 32K of the file becomes a circular buffer that continually gets rewritten until a normal file rotation happens.  If something triggers runaway logging, the hope is that that event got captured somewhere before the final 32k of the file, so you still have some clues for postmortem analysis. -- Ian