From owner-freebsd-net@FreeBSD.ORG  Mon Jan 28 19:55:52 2013
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by hub.freebsd.org (Postfix) with ESMTP id 97DC7DEC
 for <net@freebsd.org>; Mon, 28 Jan 2013 19:55:52 +0000 (UTC)
 (envelope-from vijju.singh@gmail.com)
Received: from mail-ee0-f44.google.com (mail-ee0-f44.google.com [74.125.83.44])
 by mx1.freebsd.org (Postfix) with ESMTP id 0C4E67E5
 for <net@freebsd.org>; Mon, 28 Jan 2013 19:55:51 +0000 (UTC)
Received: by mail-ee0-f44.google.com with SMTP id l10so1541926eei.3
 for <net@freebsd.org>; Mon, 28 Jan 2013 11:55:45 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:x-received:in-reply-to:references:date:message-id
 :subject:from:to:cc:content-type;
 bh=8ag6GqB/7lN5FVSbTOjPk3RkUfQidxD6ucye6GqWrb0=;
 b=zP0EuDoL4Bx1fwCz4AWK52+VCG0gEpTay/AtZOjbgRB7A2eCx5S0CzNJjCO8oDWmgm
 +QkRNlJQKdtWwssNm66jO775uS2NcUpvgYGMngPmZXeRL1YnC1mTmkoTKuySsr9EDCeC
 Txvr1Z8BxBTvXlXFNT/JAJHo18jP5cM4vihubn4ReAlpJlreh+pZcetTcKHA2DgJUBRY
 8HdiLOKvEN3rPrGfVq0FtKPckheKHkMyIbe0q1nAxZztdQyFvDrQlTS+rpn9HO/nMlgB
 g60gU+1T7xcVQpaiC9XUBVsvYnXCOvBIwz5w4s6yvrgc1B7ARykc6dERUEHtawyF8zmw
 46HQ==
MIME-Version: 1.0
X-Received: by 10.14.173.196 with SMTP id v44mr50622147eel.29.1359402945049;
 Mon, 28 Jan 2013 11:55:45 -0800 (PST)
Received: by 10.223.161.80 with HTTP; Mon, 28 Jan 2013 11:55:44 -0800 (PST)
In-Reply-To: <CA+hQ2+hwd8CHyKCN9v2iQZ0XT+FTW8inReR-Z-HCagATL76Vmg@mail.gmail.com>
References: <CALCNsJRdwKKB5DwRAFmenXkY8u8bmBh7QTjyrjqf_Q1HDw+3QQ@mail.gmail.com>
 <20130128070439.GB85353@onelab2.iet.unipi.it>
 <CALCNsJSK3wO45YE4TaYF=LWcab7iECfdo0FArvLbaAbPT6U9Yg@mail.gmail.com>
 <CA+hQ2+hwd8CHyKCN9v2iQZ0XT+FTW8inReR-Z-HCagATL76Vmg@mail.gmail.com>
Date: Mon, 28 Jan 2013 11:55:44 -0800
Message-ID: <CALCNsJRUp=ocjRdtRcpGCqtGF0Zv2J4--GqAAoJP6hxc_o3QfA@mail.gmail.com>
Subject: Re: ixgbe & msi/x
From: Vijay Singh <vijju.singh@gmail.com>
To: Luigi Rizzo <rizzo@iet.unipi.it>
Content-Type: text/plain; charset=ISO-8859-1
Cc: "freebsd-net@freebsd.org" <net@freebsd.org>
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 28 Jan 2013 19:55:52 -0000

> that was my case too. I have not gone too far into my investigation but
> should
> note that not _all_ interrupts were lost; my symptoms were queue overflows
> under netmap even at a low 2 Mpps, which with 2k entries in the rx ring
> means
> that the interrupt was delayed for more than 1ms, well above the moderation
> delay.

This would be consistent with what I am seeing. I saw that vmstat -i
reported some interrupt rate for the rx rings but even a simple ping
at that point would lead to input errors - queue overflows.


> So just to clarify, which one of these symptoms did you see
> 1) no rx interrupts at all at any rx rate
> 2) occasional missing interrupts/drops as the rx pps increase
> 3) complete loss of rx interrupts above some pps threshold ?

I think it would be closest to 3. The same HW runs fine when I disable
msi/x and use legacy interrupts.

-vijay