From owner-freebsd-net@FreeBSD.ORG Mon Jan 28 19:32:55 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 3679E962 for ; Mon, 28 Jan 2013 19:32:55 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-lb0-f181.google.com (mail-lb0-f181.google.com [209.85.217.181]) by mx1.freebsd.org (Postfix) with ESMTP id A07C86FA for ; Mon, 28 Jan 2013 19:32:54 +0000 (UTC) Received: by mail-lb0-f181.google.com with SMTP id gm6so4305947lbb.26 for ; Mon, 28 Jan 2013 11:32:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=30hziINk/UrpewsOIYHDDkETlDaBRJDmB6+SkGDpdiA=; b=0PnssKTsa/jgrGjKnFZTyW1LTaKl1DsPdfSXw2ZByQJ1PYo9yVhONRWAEpEX81HNn4 4w8dkS86rshaepE8A2TRsGgbMZb5YdY+SXPlyDKioprKLMNv8cUdlMf1ocW0cgCUGJYx NoIvHjQHl6ORC5OBDKCgt/TJRewyTfhmmw75pV8lEVdv3X3aoBXzr6nyvDtp86/RY23z 3AwItHPBx7U60u4h3JakUTFG9OeiVEB+gCIR8Uwf4Weuh7c42QDijnbxd3nAhtwlLEp4 iUthlvKZd1nfSkG2i1TwZSH8PAi2F+Sld9GrBTsHRTLenFbKae2h4PivWOVsiAW4jJAf wweA== MIME-Version: 1.0 X-Received: by 10.152.123.34 with SMTP id lx2mr14217626lab.52.1359401573116; Mon, 28 Jan 2013 11:32:53 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.114.93.200 with HTTP; Mon, 28 Jan 2013 11:32:52 -0800 (PST) In-Reply-To: References: <20130128070439.GB85353@onelab2.iet.unipi.it> Date: Mon, 28 Jan 2013 11:32:52 -0800 X-Google-Sender-Auth: k6o7qNOO21y6EbfW5QrIQncwOAw Message-ID: Subject: Re: ixgbe & msi/x From: Luigi Rizzo To: Vijay Singh Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jan 2013 19:32:55 -0000 On Mon, Jan 28, 2013 at 9:15 AM, Vijay Singh wrote: > > just curious, is this happening under behyve or also native, > > and is it always occurring or it is occasional ? > > Native, and it happens when the pps rate is high, even if the > aggregate bandwidth is low. > > that was my case too. I have not gone too far into my investigation but should note that not _all_ interrupts were lost; my symptoms were queue overflows under netmap even at a low 2 Mpps, which with 2k entries in the rx ring means that the interrupt was delayed for more than 1ms, well above the moderation delay. With these symptoms I would normally blame the os scheduler, but in this case it seems a bit hard given that the machine has 4 cores at 2.8 GHz and no other processes running. So just to clarify, which one of these symptoms did you see 1) no rx interrupts at all at any rx rate 2) occasional missing interrupts/drops as the rx pps increase 3) complete loss of rx interrupts above some pps threshold ? cheers luigi