From owner-freebsd-hackers@freebsd.org Wed Jun 26 14:50:10 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 719EB15CA161 for ; Wed, 26 Jun 2019 14:50:10 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1a.eu.mailhop.org (outbound1a.eu.mailhop.org [52.58.109.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BDC2970AEF for ; Wed, 26 Jun 2019 14:50:09 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1561560601; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=Pb7LEQukgTdQGycqv9V8HIbiKaqY4Bx4Ine7VBAFDWs1uACJO44bkN78zDCVF4KnT742k0gOMbIqE LGACLVnq0RI2Wrf6CJwt32joSN0jkfI1ZizJBzd0ytyoomVu/9IFozi0cUgsH/dxN5DkZKK1IVRXN+ ZoVlsV2v7d++thki8ETVsmj/354A11ZUNEFrIs1VIRoSBi7thLu+ti3dsWBonObk3M3T7S3tj5BTyf OO0W2jwDz7SKMAo/mMILckGOrK9FDLNVi6u3+tdWDqINHtLcWYwlrU29ZzpOoaONd+x+IS5taX1Ywj ZCoN9HjRWw7LLJnLA/b7rHSSZeOVgRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=mShiKk6bGlJpZdxfTAZKtDSSxUYY1w81EfM2mEQkUAY=; b=GCpaLpSjFi/S+B7powMn4wQRm/0BE9/Z3slJwnhlTKmYtrkgAzM45EMlOKRmrZLT4ZcYgHrRWFWbV 37+N5rIfw3DchNIQQsDvKDb6bjdLPcf8a1fpGCLIN5ZI20pUyzS91PwrvVFKbuZ4/1T0Z2ztnRGB8U iOp6wT5dqWYjRFKMHjLJLiyb3XbEXpvTpqMGxaOgfCde6hgtsp4sw54C8H6dBQI0w24GRvsbjPhOQf nIFP6JuCVUCVWVWxN92icCC+vAxVJsmlhUk5I03KHwfiyxGJHFZxe1M4rJPCEQKsXIAg+wohgr4jzv 5Kd0FGZeyFXGpKVJqcXpNhohPiAOFZg== ARC-Authentication-Results: i=1; outbound2.eu.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=mShiKk6bGlJpZdxfTAZKtDSSxUYY1w81EfM2mEQkUAY=; b=tiISJ2kQw+SP4ikJCjZ7Hyu/EtBlfgNiE2bB/Y/meuIm/94qKjoL/JE8Q3CuBd2PM4TSB46rjarZo HsnnFoSXMPPHw2htQbIyQv+1FdDcZ9J54LLwY8uHz3qNx2onL0hkA9nHmnALvWWzDNyK6dKQ50GMJu KzXX8GQ4IFv4qQh3UgqfyGUTrDQ6hiy2vGn2h4Yeubqt0NNRGA28nGpcZga4lN4FBogajQVCT1+ogi erGmYmWY5rC5ntNWcDYEDyHSjcFhmiJzmIPStlMMVA5TjCibItxpoleTnaKOCf8oFsqN8Qr7WBy171 8aMI++i14OZW+2nhAExldAk6zSDtpJA== X-MHO-RoutePath: aGlwcGll X-MHO-User: aa1e52f3-9821-11e9-bbea-3f55ba53af12 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound2.eu.mailhop.org (Halon) with ESMTPSA id aa1e52f3-9821-11e9-bbea-3f55ba53af12; Wed, 26 Jun 2019 14:49:58 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x5QEnuVK051004; Wed, 26 Jun 2019 08:49:56 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: Subject: Re: Limiting the size of syslogd output files using options in syslog.conf From: Ian Lepore To: Michael Sierchio Cc: freebsd-hackers@freebsd.org, freebsd-embedded@freebsd.org Date: Wed, 26 Jun 2019 08:49:56 -0600 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: BDC2970AEF X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.978,0]; ASN(0.00)[asn:16509, ipnet:52.58.0.0/15, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jun 2019 14:50:10 -0000 On Tue, 2019-06-25 at 16:06 -0700, Michael Sierchio wrote: > Apologies, but I don't understand the point of this at all. I object > especially to idea of patching syslogd when EVERYTHING you seek can > be > accomplished with appropriate settings in /etc/newsyslog.conf and > changing > its entry in /etc/crontab to run more frequently. You can control the > size > and number of files there. > > You can also do remote syslog. > > Do you need to rotate logs more than once per minute? > > Newsyslog handles expected log growth. I guess you've never had a hardware device or other problem unexpectedly trigger the writing of hundreds or thousands of lines of output per minute to syslog? It happens. You write apps that try to strike a sensible balance between logging important events and not logging so much it leads to problems, then something unexpected happens and your carefully crafted logging starts happening at 200hz and actually contributes to the problem. We've been using this code at $work since 2003; it appears in dozens of different products ranging from lab instruments to national timescale systems. People using the products experiencing such problems just want the device to keep working as best it can. They don't want repeated mysterious reboots because of things like /var filling up. (In the case of products like lab instruments, they don't really perceive the thing to be a computer running an OS at all.) And most of all, when something does go wrong and they get our support engineers to look into it, they don't want to hear "We can't figure out what happened originally to cause this problem because all the logging got rotated out of existence." Newsyslog does exactly the opposite of what you want in that situation... it purposely destroys the valuable information about the original problem while carefully preserving the useless fallout that follows. Over the years we've upstreamed every freebsd customization we've ever done at $work except this one. The only reason I've been carrying this one local mod is because it seemed too specialized to be useful to anyone else. Then yesterday someone working on an embedded-system product asked me on irc how to solve the problem of unexpected log spewage, so I offered the solution we've been using. -- Ian > > On Tue, Jun 25, 2019 at 3:25 PM Ian Lepore wrote: > > > I've posted a review of a small syslogd change which lets you set a > > limit on the size of syslogd output files in /etc/syslog.conf. The > > idea is to prevent filling up a filesystem on emmc or sdcard or > > other > > small storage device on an embedded system with unexpected logging > > triggered by some error or failing hardware. > > > > If you're interested, please see the review at > > > > https://reviews.freebsd.org/D20764 > > > > -- Ian > > > > _______________________________________________ > > freebsd-embedded@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-embedded > > To unsubscribe, send any mail to " > > freebsd-embedded-unsubscribe@freebsd.org > > " > > > >