From owner-freebsd-net@FreeBSD.ORG  Wed Jul 18 22:42:28 2007
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
X-Original-To: freebsd-net@freebsd.org
Delivered-To: freebsd-net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id E664116A406
	for <freebsd-net@freebsd.org>; Wed, 18 Jul 2007 22:42:28 +0000 (UTC)
	(envelope-from jfvogel@gmail.com)
Received: from nz-out-0506.google.com (nz-out-0506.google.com [64.233.162.239])
	by mx1.freebsd.org (Postfix) with ESMTP id A50F313C4C5
	for <freebsd-net@freebsd.org>; Wed, 18 Jul 2007 22:42:28 +0000 (UTC)
	(envelope-from jfvogel@gmail.com)
Received: by nz-out-0506.google.com with SMTP id l8so283429nzf
	for <freebsd-net@freebsd.org>; Wed, 18 Jul 2007 15:42:28 -0700 (PDT)
DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta;
	h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
	b=fnOEQPee70bvm7bPEf+gcryUnhp3WmsFt6BhpJ8saQHgy4fdVe1Czum0OusOUkEnrtqVZmg34X1a8b1ZQ/5CEqjLipDs6dygTcsdPImGNwtc7g68hpmSiMZl/xKc0xtIxj87jNUQMR2eYePRsXRFQ8jnir7tVfc4VXxxhX3YU3E=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta;
	h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
	b=OMM9+yDMcDbJmj/oC+TDc5/YsB56kOgGHGjPw7NaCh7n1j9+9+igtg3BbIhbt45k3fMrrtMAIgVkK9+OK/oH+oZJr5cKKiTjtm+Kr5QODB4FNFDurQvrEEPcDmfGHbUY7MEFmB45XgGGVBQGgN9BatZS3eZQHy6rz1HU4m8ltK8=
Received: by 10.115.108.1 with SMTP id k1mr1955280wam.1184798547438;
	Wed, 18 Jul 2007 15:42:27 -0700 (PDT)
Received: by 10.114.103.14 with HTTP; Wed, 18 Jul 2007 15:42:27 -0700 (PDT)
Message-ID: <2a41acea0707181542q23d36733o339448ad86f480cb@mail.gmail.com>
Date: Wed, 18 Jul 2007 15:42:27 -0700
From: "Jack Vogel" <jfvogel@gmail.com>
To: freebsd-net <freebsd-net@freebsd.org>, 
	"FreeBSD Current" <freebsd-current@freebsd.org>
In-Reply-To: <2a41acea0707181152r4c662b00tf1be7b6b4dce74f1@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <2a41acea0707181152r4c662b00tf1be7b6b4dce74f1@mail.gmail.com>
Cc: 
Subject: 10G and socket alloc failure
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 18 Jul 2007 22:42:29 -0000

While testing out the ixgbe driver we've observed this failure in the stack
code, here is the info:

The test engineer is using iperf, typically with 16 threads. If the
driver is using
either legacy or MSI interrupts we will see broken pipes, in dmesg its
due to sonewconn() failing in syncache_socket().

Whats interesting is that when I have multiple RX queues configured and
using MSI/X this doesnt happen, at least not very often.

It does not seem to hurt performance horribly, iperf just spawns another
thread, but I was wondering if there is some underlying tuneable I dont
know about that would stop this from happening??

And any theory about why it doesnt happen with multiple queues?

Jack