From owner-freebsd-net@FreeBSD.ORG Thu Apr 10 11:22:45 2014 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E0D3DC1E; Thu, 10 Apr 2014 11:22:45 +0000 (UTC) Received: from mailhost.dlr.de (mailhost.dlr.de [129.247.252.33]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mailhost.dlr.de", Issuer "DLR CA - G02" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 728EB1660; Thu, 10 Apr 2014 11:22:45 +0000 (UTC) Received: from DLREXHUB01.intra.dlr.de (172.21.152.130) by dlrexedge02.dlr.de (172.21.163.101) with Microsoft SMTP Server (TLS) id 14.3.174.1; Thu, 10 Apr 2014 13:22:15 +0200 Received: from KNOP-BEAGLE.kn.op.dlr.de (129.247.178.136) by smtp.dlr.de (172.21.152.151) with Microsoft SMTP Server (TLS) id 14.3.174.1; Thu, 10 Apr 2014 13:22:14 +0200 Date: Thu, 10 Apr 2014 13:22:52 +0200 From: Hartmut Brandt X-X-Sender: brandt_h@KNOP-BEAGLE.kn.op.dlr.de To: Vladislav Prodan Subject: Re[2]: Some gruesome moments with performance of FreeBSD at over 20K interfaces In-Reply-To: <1397127901.499782177.24smhe7a@frv35.fwdcdn.com> Message-ID: References: <1397077963.756961709.gspkmzvd@frv35.fwdcdn.com> <1397127901.499782177.24smhe7a@frv35.fwdcdn.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Cc: stable@freebsd.org, hackers@freebsd.org, net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 11:22:46 -0000 On Thu, 10 Apr 2014, Vladislav Prodan wrote: VP>> On Wed, 9 Apr 2014, Vladislav Prodan wrote: VP>> VP>> VP>b) Service bsnmpd started at 12K interfaces, but immediately loaded CPU VP>> VP>at 80-100% VP>> VP>> I could imagine that this is because of the statistics polling. bsnmp VP>> implements 64-bit interface statistics but we have only 32-bit statistics VP>> in the kernel. So it polls the kernel statistics for each interface on a VP>> rate that ensures that 32-bit don't overflow. If the interfaces are GBit VP>> or, worse, 10GBit interfaces the polling rate is rather high (in the order VP>> of seconds). VP>> VP>> You should either make sure that the interfaces report sensible bitrates VP>> (I doubt that 20k interfaces could all be GBit interfaces) or force a slower VP>> polling interval by setting begemotIfForcePoll.0 to some large value. VP>> VP>> harti VP>> VP> VP>Thanks for the tip. VP> VP>At least 10 interfaces to be 1Gb, and the rest no more than 50M. VP>BegemotIfForcePoll parameter in this case a little help, you will be forced to stand another value for Gigabit Interface begemotIfForcePoll ... Yeah. There is only one parameter. You are running -STABLE, right? On current the statistics are 64-bit (I wonder whether the operation on these values is automatically atomic on all our platforms, though). harti