Date: Tue, 3 May 2016 12:30:06 +0200 (CEST) From: =?ISO-8859-1?Q?Trond_Endrest=F8l?= <Trond.Endrestol@fagskolen.gjovik.no> To: Wolfgang Zenker <wolfgang@lyxys.ka.sub.org> Cc: freebsd-stable@FreeBSD.org Subject: Re: Recent stable: bsnmpd eats up memory and cpu Message-ID: <alpine.BSF.2.20.1605031228060.1206@mail.fig.ol.no> In-Reply-To: <alpine.BSF.2.20.1605020846470.1206@mail.fig.ol.no> References: <20160501220107.GA58930@lyxys.ka.sub.org> <alpine.BSF.2.20.1605020846470.1206@mail.fig.ol.no>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, 2 May 2016 09:10+0200, Trond Endrestøl wrote: > On Mon, 2 May 2016 00:01+0200, Wolfgang Zenker wrote: > > > Hi, > > > > after updating some 10-STABLE systems a few days ago, I noticed that on > > two of those systems bsnmpd started to use up a lot of cpu time, and the > > available memory shrinked until rendering the system unusable. Killing > > bsnmpd stops the cpu usage but does not free up memory. > > Both affected systems are amd64, one having moved from r297555 to > > r298723, the other from r297555 to r298722. Another amd64 system > > that went from r297555 to r298722 appears to be not affected. > > The two affected systems are on an internal LAN segment and there > > is currently no application connecting to snmp on those machines. > > > > What would be useful debugging data to collect in this case? > > I believe I've seen the very same on my systems. All of them got > updated last Friday due to the recent NTP fix. Prior to last Friday, > they all ran stable/10 from early March, r296648-ish. Neither of them > run bsnmpd, but they offer a lot of network services. > > Three of my i386 systems each with 1 GiB of memory ran out of swap > space, Sunday afternoon. > > This night a mail server running i386 with 4 GiB of memory died while > handling mail. From the messages I could glean on /dev/ttyvb (due to > custom logging) before rebooting, is that it's all networking related. > > SpamAssassin and syslogd on the mail server managed to transmit these > lines to the central log host before dying: > > May 2 00:05:17 <mail.err> [HOSTNAME] spamc[63613]: connect to spamd on ::1 failed, retrying (#1 of 3): Connection refused > May 2 00:05:17 <mail.err> [HOSTNAME] spamc[63613]: connect to spamd on 127.0.0.1 failed, retrying (#1 of 3): Connection refused > May 2 00:05:18 <mail.err> [HOSTNAME] spamc[63613]: connect to spamd on ::1 failed, retrying (#2 of 3): Connection refused > May 2 00:05:18 <mail.err> [HOSTNAME] spamc[63613]: connect to spamd on 127.0.0.1 failed, retrying (#2 of 3): Connection refused > May 2 00:05:19 <mail.err> [HOSTNAME] spamc[63613]: connect to spamd on ::1 failed, retrying (#3 of 3): Connection refused > May 2 00:05:19 <mail.err> [HOSTNAME] spamc[63613]: connect to spamd on 127.0.0.1 failed, retrying (#3 of 3): Connection refused > May 2 00:05:19 <mail.err> [HOSTNAME] spamc[63613]: connection attempt to spamd aborted after 3 retries > > May 2 00:52:17 <mail.err> [HOSTNAME] sm-mta[63740]: u41Mp86h063740: Milter (spamassassin): error creating socket: No buffer space available > May 2 00:52:17 <mail.err> [HOSTNAME] sm-mta[63739]: u41Mp8r9063739: Milter (spamassassin): error creating socket: No buffer space available > May 2 00:52:17 <mail.info> [HOSTNAME] sm-mta[63740]: u41Mp86h063740: Milter (spamassassin): to error state > May 2 00:52:17 <mail.info> [HOSTNAME] sm-mta[63739]: u41Mp8r9063739: Milter (spamassassin): to error state > > All of the amd64 systems with 4 GiB or 8 GiB of memory are apparently > unaffected. > > Maybe it's time to convert the remaining i386 systems to amd64 > systems, and add some memory while I'm at it. > > The bug is either in the kernel or in libc, or both. You might want to try out the patch created by Mark Johnston: http://lists.freebsd.org/pipermail/freebsd-current/2016-May/061015.html I'm in the process of testing this patch on one i386 stable/10 system and on one amd64 stable/10 system. -- +-------------------------------+------------------------------------+ | Vennlig hilsen, | Best regards, | | Trond Endrestøl, | Trond Endrestøl, | | IT-ansvarlig, | System administrator, | | Fagskolen Innlandet, | Gjøvik Technical College, Norway, | | tlf. mob. 952 62 567, | Cellular...: +47 952 62 567, | | sentralbord 61 14 54 00. | Switchboard: +47 61 14 54 00. | +-------------------------------+------------------------------------+ From owner-freebsd-stable@freebsd.org Tue May 3 11:41:14 2016 Return-Path: <owner-freebsd-stable@freebsd.org> Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA8DEB2B718 for <freebsd-stable@mailman.ysv.freebsd.org>; Tue, 3 May 2016 11:41:14 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 9C5A519C5; Tue, 3 May 2016 11:41:14 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id 9A6211143; Tue, 3 May 2016 11:41:14 +0000 (UTC) Date: Tue, 3 May 2016 11:41:14 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org Message-ID: <125543220.54.1462275674418.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Jenkins build became unstable: FreeBSD_stable_10 #241 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkKKb2VAfYQKfu1t7qk4nR5qzUBEI+UqT4BPec4qHVhqUy0FFdq50sMH+3y9bCDNOufctov6VqTNffZ3YXArnZK95YF0OX97fh+E9txYOUX1adc+TikcKjuYpHmL5dE62eaZTI+4A5jnRonskQ1PaoIFz0Kbu4mWzkFsmdiXTraGzomXq4cHUCATA2+K4eDYgjXEQI30z3GOMmmZ4t/+6QGk1cMb/BqMWHbn80AsRCb4tU7Hpd72XLDpsuO7YRP1Q0CjmNAuBOTj+sFiiOe6U9HpqOlQN+iFUvBdZo/ybuy5Kh71cAaYQNL68cYdZJ6binH/DkG3KY/fS7DFYAeuwjwIDAQAB X-Jenkins-Job: FreeBSD_stable_10 X-Jenkins-Result: UNSTABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Production branch of FreeBSD source code <freebsd-stable.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-stable>, <mailto:freebsd-stable-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-stable/>; List-Post: <mailto:freebsd-stable@freebsd.org> List-Help: <mailto:freebsd-stable-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-stable>, <mailto:freebsd-stable-request@freebsd.org?subject=subscribe> X-List-Received-Date: Tue, 03 May 2016 11:41:14 -0000 See <https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/241/>;
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.BSF.2.20.1605031228060.1206>