From owner-freebsd-net@FreeBSD.ORG  Fri Dec 17 19:31:42 2010
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id D2178106566B
	for <net@freebsd.org>; Fri, 17 Dec 2010 19:31:42 +0000 (UTC)
	(envelope-from eugen@grosbein.pp.ru)
Received: from grosbein.pp.ru (grosbein.pp.ru [89.189.172.146])
	by mx1.freebsd.org (Postfix) with ESMTP id 44DE28FC08
	for <net@freebsd.org>; Fri, 17 Dec 2010 19:31:41 +0000 (UTC)
Received: from grosbein.pp.ru (localhost [127.0.0.1])
	by grosbein.pp.ru (8.14.4/8.14.4) with ESMTP id oBHJEbW3003069
	for <net@freebsd.org>; Sat, 18 Dec 2010 01:14:37 +0600 (NOVT)
	(envelope-from eugen@grosbein.pp.ru)
Message-ID: <4D0BB69D.4000106@grosbein.pp.ru>
Date: Sat, 18 Dec 2010 01:14:37 +0600
From: Eugene Grosbein <eugen@grosbein.pp.ru>
User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US;
	rv:1.9.1.9) Gecko/20100518 Thunderbird/3.0.4
MIME-Version: 1.0
To: net@freebsd.org
Content-Type: text/plain; charset=KOI8-R
Content-Transfer-Encoding: 7bit
Cc: 
Subject: netisr queue overflows
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Dec 2010 19:31:42 -0000

Hi!

I run pretty busy 4-core router that has 4 gigabit ethernet interfaces:
em0 and em1 grouped to lagg0 that has IP address and
igb0 and igb1 grouped to lagg1 that has no IP address but many vlans
that carry PPPoE traffic.

sysctl net.inet.ip.intr_queue_drops shows me 15408340 packet drops
due to netisr queue overflow. I have in /boot/loader.conf:

net.isr.maxthreads=4
# default value
#net.isr.maxqlimit=10240
# for rtsock
net.route.netisr_maxqlen=4096
# for other protocols (IP & PPPoE?)
net.isr.defaultqlimit=4096

How do I obtain current queue lengths for monitoring/tuning?
I see over 125Kpss for lagg0 and 132Kpps for lagg1.

/etc/sysctl.conf:

net.isr.direct=0
net.isr.direct_force=0
net.inet.ip.intr_queue_maxlen=4096

dev.em.0.rx_int_delay=200
dev.em.0.tx_int_delay=200
dev.em.0.rx_abs_int_delay=4000
dev.em.0.tx_abs_int_delay=4000
dev.em.0.rx_processing_limit=4096

dev.em.1.rx_int_delay=200
dev.em.1.tx_int_delay=200
dev.em.1.rx_abs_int_delay=4000
dev.em.1.tx_abs_int_delay=4000
dev.em.1.rx_processing_limit=4096

dev.igb.0.rx_processing_limit=4096
dev.igb.1.rx_processing_limit=4096

kern.ipc.nmbclusters=100000
kern.ipc.nmbjumbop=100000
kern.ipc.maxsockbuf=83886080

net.graph.maxdgram=8388608
net.graph.recvspace=8388608

Eugene Grosbein