From owner-freebsd-net@FreeBSD.ORG  Wed Oct 31 12:48:44 2012
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@FreeBSD.org
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
 by hub.freebsd.org (Postfix) with ESMTP id B2081559;
 Wed, 31 Oct 2012 12:48:44 +0000 (UTC)
 (envelope-from glebius@FreeBSD.org)
Received: from cell.glebius.int.ru (glebius.int.ru [81.19.64.117])
 by mx1.freebsd.org (Postfix) with ESMTP id 29A228FC08;
 Wed, 31 Oct 2012 12:48:43 +0000 (UTC)
Received: from cell.glebius.int.ru (localhost [127.0.0.1])
 by cell.glebius.int.ru (8.14.5/8.14.5) with ESMTP id q9VCmgQM045363;
 Wed, 31 Oct 2012 16:48:42 +0400 (MSK)
 (envelope-from glebius@FreeBSD.org)
Received: (from glebius@localhost)
 by cell.glebius.int.ru (8.14.5/8.14.5/Submit) id q9VCmg1j045362;
 Wed, 31 Oct 2012 16:48:42 +0400 (MSK)
 (envelope-from glebius@FreeBSD.org)
X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to
 glebius@FreeBSD.org using -f
Date: Wed, 31 Oct 2012 16:48:42 +0400
From: Gleb Smirnoff <glebius@FreeBSD.org>
To: R?mi Pauchet <remi.pauchet@netasq.com>, jfv@FreeBSD.org,
 melifaro@FreeBSD.org
Subject: Re: panic ixgbevf / SMP under high network load
Message-ID: <20121031124842.GS70741@FreeBSD.org>
References: <B722D1A8-AA5F-4B80-8F84-B3935D72A498@netasq.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="WlEyl6ow+jlIgNUh"
Content-Disposition: inline
In-Reply-To: <B722D1A8-AA5F-4B80-8F84-B3935D72A498@netasq.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
Cc: freebsd-net@FreeBSD.org
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 31 Oct 2012 12:48:44 -0000


--WlEyl6ow+jlIgNUh
Content-Type: text/plain; charset=koi8-r
Content-Disposition: inline

  [back to list and adding Jack to Cc]

On Thu, Oct 25, 2012 at 10:06:40AM +0200, R?mi Pauchet wrote:
R> I'm testing network performance of FreeBSD using vmware esxi 5.1 with SR-IOV
R> 
R> I'm using  FreeBSD 8.3 kernel GENERIC, 4 cpus, ixgbevf driver with an Intel 82599EB dual 10 Gbps network interface
R> 
R> After a few seconds of udp ipv4 load (5Gbps x2, frame size 700), I have the following panic :

Remi reported that attached patch fixes the panic. Looks like ixv_rxeof() isn't
thread safe since doesn't expect its state being changed while lock is temporarily
dropped.

This is deja vu of an old problem in em(4):

http://freshbsd.org/commit/freebsd/r151314

Similar fix can be made for ixgbe(4). However, recently we had discussion on
removing this unlock entirely from drivers. Unlock/lock removal would not only
fix such kind of problems, but also would speed up processing.

Discussion starts here:

http://lists.freebsd.org/pipermail/freebsd-net/2012-October/033520.html

-- 
Totus tuus, Glebius.

--WlEyl6ow+jlIgNUh
Content-Type: text/x-diff; charset=koi8-r
Content-Disposition: attachment; filename="ixv.c.diff"

Index: ixv.c
===================================================================
--- ixv.c	(revision 242127)
+++ ixv.c	(working copy)
@@ -3250,9 +3250,7 @@
                         if (tcp_lro_rx(&rxr->lro, m, 0) == 0)
                                 return;
         }
-	IXV_RX_UNLOCK(rxr);
         (*ifp->if_input)(ifp, m);
-	IXV_RX_LOCK(rxr);
 }
 
 static __inline void

--WlEyl6ow+jlIgNUh--