Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 26 Jun 2019 08:49:56 -0600
From:      Ian Lepore <ian@freebsd.org>
To:        Michael Sierchio <kudzu@tenebras.com>
Cc:        freebsd-hackers@freebsd.org, freebsd-embedded@freebsd.org
Subject:   Re: Limiting the size of syslogd output files using options in syslog.conf
Message-ID:  <a61521b41a79d96bbc8fe0a87bf4e8609395fcb0.camel@freebsd.org>
In-Reply-To: <CAHu1Y72_hKcTcAvohsJOLvwwh2y3mseJGYJQbB0DqxkioBc_iA@mail.gmail.com>
References:  <bf597c3830ebabbef6ac422a5d648e1eed13fac5.camel@freebsd.org> <CAHu1Y72_hKcTcAvohsJOLvwwh2y3mseJGYJQbB0DqxkioBc_iA@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 2019-06-25 at 16:06 -0700, Michael Sierchio wrote:
> Apologies, but I don't understand the point of this at all.  I object
> especially to idea of patching syslogd when EVERYTHING you seek can
> be
> accomplished with appropriate settings in /etc/newsyslog.conf and
> changing
> its entry in /etc/crontab to run more frequently. You can control the
> size
> and number of files there.
> 
> You can also do remote syslog.
> 
> Do you need to rotate logs more than once per minute?
> 
> 

Newsyslog handles expected log growth.  I guess you've never had a
hardware device or other problem unexpectedly trigger the writing of
hundreds or thousands of lines of output per minute to syslog?  It
happens.  You write apps that try to strike a sensible balance between
logging important events and not logging so much it leads to problems,
then something unexpected happens and your carefully crafted logging
starts happening at 200hz and actually contributes to the problem.

We've been using this code at $work since 2003; it appears in dozens of
different products ranging from lab instruments to national timescale
systems.  People using the products experiencing such problems just
want the device to keep working as best it can.  They don't want
repeated mysterious reboots because of things like /var filling up. 
(In the case of products like lab instruments, they don't really
perceive the thing to be a computer running an OS at all.)

And most of all, when something does go wrong and they get our support
engineers to look into it, they don't want to hear "We can't figure out
what happened originally to cause this problem because all the logging
got rotated out of existence."  Newsyslog does exactly the opposite of
what you want in that situation... it purposely destroys the valuable
information about the original problem while carefully preserving the
useless fallout that follows.

Over the years we've upstreamed every freebsd customization we've ever
done at $work except this one.  The only reason I've been carrying this
one local mod is because it seemed too specialized to be useful to
anyone else.  Then yesterday someone working on an embedded-system
product asked me on irc how to solve the problem of unexpected log
spewage, so I offered the solution we've been using.

-- Ian

> 
> On Tue, Jun 25, 2019 at 3:25 PM Ian Lepore <ian@freebsd.org> wrote:
> 
> > I've posted a review of a small syslogd change which lets you set a
> > limit on the size of syslogd output files in /etc/syslog.conf.  The
> > idea is to prevent filling up a filesystem on emmc or sdcard or
> > other
> > small storage device on an embedded system with unexpected logging
> > triggered by some error or failing hardware.
> > 
> > If you're interested, please see the review at
> > 
> > https://reviews.freebsd.org/D20764
> > 
> > -- Ian
> > 
> > _______________________________________________
> > freebsd-embedded@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-embedded
> > To unsubscribe, send any mail to "
> > freebsd-embedded-unsubscribe@freebsd.org
> > "
> > 
> 
> 




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?a61521b41a79d96bbc8fe0a87bf4e8609395fcb0.camel>