From owner-freebsd-net@FreeBSD.ORG Mon Aug 13 21:16:03 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B4A85106564A for ; Mon, 13 Aug 2012 21:16:03 +0000 (UTC) (envelope-from vijju.singh@gmail.com) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 492578FC14 for ; Mon, 13 Aug 2012 21:16:02 +0000 (UTC) Received: by eeke52 with SMTP id e52so1224543eek.13 for ; Mon, 13 Aug 2012 14:15:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=Ftpg0zqM+DzaCAQROnxXTZA4CZbE0b6aXc5Jx/Jxwr8=; b=f2xQjRiOqxQHsO0zgFwgsufPfIGxhDyvIOuum3WjpHeo0n5NpfESF6kB2tI2t8/BP2 6N6jHvJiX7+7QWzbpplDEnpHMNVPSz0+g5rTAM5wJXbQQCWUy4jChyRvog9ftaFxo23s WYjYbW3zpB8ak+b2LCOFKuqHcVNLko1Xafkfi2pj6nw1P3nPdLzVjI3eg04qgQ4vp8dm RZiK/fRNLoZBA/8m2eitimc7ENzvTyN6m2QXsXo6Cr4sW9Y/tWcmNovDb48HkfijZ5g0 3zXJQp/S7SLybvTV4qeVnW+tYr2yvehTs1JWCeqANRcO6voL5f9LLg1P1El/XnFrBkF/ 7AzQ== MIME-Version: 1.0 Received: by 10.14.211.3 with SMTP id v3mr11710943eeo.43.1344892555860; Mon, 13 Aug 2012 14:15:55 -0700 (PDT) Received: by 10.14.120.199 with HTTP; Mon, 13 Aug 2012 14:15:55 -0700 (PDT) Date: Mon, 13 Aug 2012 14:15:55 -0700 Message-ID: From: Vijay Singh To: net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Subject: ixgbe rx & tx locks X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2012 21:16:03 -0000 Folks, I've been looking into lock contention on the ixgbe rx & tx locks. I have collected this data: debug.lock.prof.stats: max wait_max total wait_total count avg wait_avg cnt_hold cnt_lock name 263 304 1329357 858873 322010 4 2 0 45210 sys/dev/ixgbe/ixgbe.c:1349 (sleep mutex:e2a:tx(0)) 415 189 14314889 62933 739895 19 0 0 13694 sys/dev/ixgbe/ixgbe.c:1445 (sleep mutex:e2a:tx(0)) So the lock contention is coming from lock acquisitions in ixgbe_handle_que() and ixgbe_msix_que(). I am unable to understand why, since I would not expect the deferred task to execute in parallel with the msix interrupt handler. rx is similar: debug.lock.prof.stats: max wait_max total wait_total count avg wait_avg cnt_hold cnt_lock name 88 893 672058 269855 821847 0 0 0 753 sys/dev/ixgbe/ixgbe.c:4211 (sleep mutex:e6b:rx(0)) 893 25 3994289 1937 1093238 3 0 0 561 sys/dev/ixgbe/ixgbe.c:4276 (sleep mutex:e6b:rx(0)) Any thoughts? -vijay