From owner-freebsd-net@FreeBSD.ORG  Wed Oct 26 11:33:46 2011
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 07728106566B
	for <freebsd-net@freebsd.org>; Wed, 26 Oct 2011 11:33:46 +0000 (UTC)
	(envelope-from nitroboost@gmail.com)
Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com
	[209.85.220.182])
	by mx1.freebsd.org (Postfix) with ESMTP id B08A08FC0A
	for <freebsd-net@freebsd.org>; Wed, 26 Oct 2011 11:33:45 +0000 (UTC)
Received: by vcbfo13 with SMTP id fo13so2015637vcb.13
	for <freebsd-net@freebsd.org>; Wed, 26 Oct 2011 04:33:44 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type;
	bh=/KqvgwTldAMQ1Q1IwSpkfL4Tms3+uGzuQJoO1LmduFk=;
	b=xd3hJiAu2e9CWQLYvnXdDYlAAHqwzuKNIcs4i6bnrWETvCtvggjgbc2mMA5jFG66W7
	7mmqXtANkjO8zqXQbz1pnlG9q8wHtNRRS18LGpwkVLtJz4vAUJAADMMb5N3Zr/STkWDT
	ssAvwRUFIkAEkPZQh3Ef4bTBGGB6EuIR1i9xY=
MIME-Version: 1.0
Received: by 10.182.59.5 with SMTP id v5mr2396508obq.78.1319628824788; Wed, 26
	Oct 2011 04:33:44 -0700 (PDT)
Received: by 10.182.35.193 with HTTP; Wed, 26 Oct 2011 04:33:44 -0700 (PDT)
In-Reply-To: <4EA7E203.3020306@sepehrs.com>
References: <CAAAm0r0RXEJo4UiKS=Ui0e5OQTg6sg-xcYf3mYB5+vk8i8557w@mail.gmail.com>
	<4E8F157A.40702@sentex.net>
	<CAAAm0r2JH43Rct7UxQK2duH1p43Nepnj5mpb6bXo==DPayhJLg@mail.gmail.com>
	<4E8F51D4.1060509@sentex.net>
	<CACqU3MVwLaepFymZJkaVk6p=SpykGhqs=VYFjLh9fP9S=AxDhg@mail.gmail.com>
	<CAAAm0r1DKvoL9=Ket9up=4+5xiCzTTZJK99FhF9jcCA28B0M+A@mail.gmail.com>
	<CAAAm0r3XdsMHZh+P_NF-txZasdExzwZ8ymmGQgGhJQds0fOiBQ@mail.gmail.com>
	<CAAAm0r1iS3z-7CBJ=xYDf+JOA1Q2nU0O54Twbyb7FjvgWHjKVw@mail.gmail.com>
	<4EA7E203.3020306@sepehrs.com>
Date: Wed, 26 Oct 2011 04:33:44 -0700
Message-ID: <CAAAm0r3Nr2t8cCetPkFnLQ-3KwqHw_0SpqbtvYPRUkSP=9n8CA@mail.gmail.com>
From: Jason Wolfe <nitroboost@gmail.com>
To: Hooman Fazaeli <fazaeli@sepehrs.com>
Content-Type: text/plain; charset=ISO-8859-1
X-Content-Filtered-By: Mailman/MimeDel 2.1.5
Cc: freebsd-net@freebsd.org
Subject: Re: Intel 82574L interface wedging on em 7.1.9/7.2.3 when MSIX
	enabled
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 26 Oct 2011 11:33:46 -0000

Hooman,

I have run with dev.em.X.flow_control=0, which should have the same result
as hw.em.fc_setting=0, and net.inet.tcp.tso is also 0.  I'm not sure the
remaining options would be able to produce the scenario I'm seeing, but I'm
open to giving it a try with no options on the interfaces.  I've also added
ifconfig output to the collection.

options=219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
ifconfig emX -rxcsum -txcsum -vlanhwtag -tso -wol
options=88<VLAN_MTU,VLAN_HWCSUM>

It's always TX, but these servers push ~12x what they receive, so I'm
guessing it could happen to either buffer given the right traffic patterns.
 While looking through commits I also found a patch to add a couple sysctls
for em, which I'm adding - http://freshbsd.org/commit/freebsd/r223676

Thanks,
Jason

On Wed, Oct 26, 2011 at 3:33 AM, Hooman Fazaeli <fazaeli@sepehrs.com> wrote:

> Hi Jason
>
> Have you tried:
>
> hw.em.fc_setting="0" (in loader.conf)
> ifconfig emX -tso -lro -rxcsum -txcsum -vlanhwtag -wol
>
> with MSIX and no multiqueue.
>
> Advanced features has always been a source of problem.
> It is worth a try and help to narrow down possibilities.
>
> It would also be helpful if you provide 'ifconfig' output
> when the problem happens.
>
>  And a question: Does interface RX also hangs or it is just TX?