From owner-freebsd-net@FreeBSD.ORG Thu Mar 22 14:09:26 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0BC541065670 for ; Thu, 22 Mar 2012 14:09:26 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id D1E4C8FC0A for ; Thu, 22 Mar 2012 14:09:25 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 87BE446B2C; Thu, 22 Mar 2012 10:09:25 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 14F9FB940; Thu, 22 Mar 2012 10:09:25 -0400 (EDT) From: John Baldwin To: freebsd-net@freebsd.org Date: Thu, 22 Mar 2012 08:16:57 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p10; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201203220816.58217.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 22 Mar 2012 10:09:25 -0400 (EDT) Cc: Traiano Welcome Subject: Re: FreeBSD: syslog-ng: I/O error occurred while writing; fd='xx', error='No buffer space available (yy)' X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 14:09:26 -0000 On Thursday, March 22, 2012 6:03:21 am Traiano Welcome wrote: > Hi List > > I've been seeing the following in the messages log of my freebsd syslog > server for quite some time now: > > --- > Mar 20 12:19:12 syslog2 syslog-ng[35313]: I/O error occurred while > writing; fd='12', error='No buffer space available (55)' > Mar 20 12:19:12 syslog2 syslog-ng[35313]: Connection broken; > time_reopen='60' > Mar 20 12:19:12 syslog2 syslog-ng[35313]: I/O error occurred while > writing; fd='13', error='No buffer space available (55)' > Mar 20 12:19:12 syslog2 syslog-ng[35313]: Connection broken; > time_reopen='60' > --- > > These happen at a frequency of about 7 per minute on average. See attached > trend graphs for an idea of the volume of traffic we're doing, as well as > the memory and cpu utilisation trends on this server during this period. > As can be seen from the graphs, load does not seem to be the issue. > Occasionally during the week, the system freezes and requires a reboot, I > think it's related to the above message, though I'm not sure. > > My question is: What does this error mean, and how can I resolve it? > > I have tried to frame this as an operating system kernel resource issue, > and experimented with increasing the freebsd kernel sysctls for UDP > performance: It means that the network driver has "filled" up with packets. Are you using igb(4)? We have to crank the number of descriptors assigned to igb to the max to workaround this at work (we get DNS timeouts during a simple boot otherwise). hw.igb.maxtxd is the tunable you would set. The max value you can set it to is 4096. -- John Baldwin