From owner-freebsd-net@FreeBSD.ORG Wed Jun 17 12:54:52 2015 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EC759A1B for ; Wed, 17 Jun 2015 12:54:51 +0000 (UTC) (envelope-from lakshmi.n@msystechnologies.com) Received: from mail-ig0-f172.google.com (mail-ig0-f172.google.com [209.85.213.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C379BDD0 for ; Wed, 17 Jun 2015 12:54:51 +0000 (UTC) (envelope-from lakshmi.n@msystechnologies.com) Received: by igbiq7 with SMTP id iq7so66175545igb.1 for ; Wed, 17 Jun 2015 05:54:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to:cc :content-type; bh=nRR4dln20fQfoww0YDuCUIkMuWE7ZxtnnFWqyrPRRpA=; b=KNZCPyUKqblWTJkNtC4Z1Z+djcVxXaceJyRaCauiyI6v+mFN1RbxCwGNWPtmkXbr77 icyavlobykEYBiZ63s9nI1LdIV4ff5Mn8SqsNd1qa4tQChORBbG5gOgMJPI3xs+0YcRJ soSlfGoMGgpwPN6CjrSKJPmKMFmc9ODiNw5EIvm8p2qNzZ8eyOtMskHx5+D6BqfCP8SK f0Bhpezst/YhkZ/HEHnhGwcCXKM0b1knK+KbWZdwXMP/lJygmdFj79tAyhc/kZCB9mqc jUmoJDl7wB9UBCNJtZxZvkssHUJ8NFgqQghd0FG4gYyNahqQv+HYs4meizxJT2X8JGTR Xc2A== X-Gm-Message-State: ALoCoQkWHD4li8pGsJCmIRoBBbz//ljNthoMGLX67AkknI0jWAlq9eA6fUFcuOnAIkcBAzu/IDxH MIME-Version: 1.0 X-Received: by 10.107.12.72 with SMTP id w69mr7219655ioi.10.1434538811488; Wed, 17 Jun 2015 04:00:11 -0700 (PDT) Received: by 10.64.116.66 with HTTP; Wed, 17 Jun 2015 04:00:11 -0700 (PDT) Date: Wed, 17 Jun 2015 16:30:11 +0530 Message-ID: Subject: Reg Intel Fortville IXL driver on 11-CURRENT From: Lakshmi Narasimhan Sundararajan To: freebsd-net@freebsd.org Cc: "Pokala, Ravi" , "Lewis, Fred" , "Kloss, Joe" , "Tallam, Sreen" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jun 2015 12:54:52 -0000 Hi FreeBSD folks, I am part of Panasas and working on evaluating IXL over FreeBSD [11-CURRENT] on Intel Taylor Pass platform for our next product. In that regard, while evaluating performance, we found the Tx performance to be very poor. We found it to be so because the tx interrupts are spread over all the CPUs even if the traffic handling process is pinned at a particular cpu. And we narrowed it down to the below lines of code in the tx path within the IXL driver. It seems to me that the logic for finding whether tx queue is stalled might be incorrect or too aggressive. Either ways, removing the below lines makes the tx interrupt being handled on the same cpu on which the request was raised and the performance is very good as expected. Filename: sys/dev/ixl/ixl_txrx.c [lakshmis@mau-bsd-10a ~/fortville/hol/sys/dev/ixl]$ diff -c5pt ixl_txrx.c ixl_txrx.c.mod *** ixl_txrx.c Fri Jun 12 06:56:51 2015 --- ixl_txrx.c.mod Fri Jun 12 06:56:33 2015 *************** ixl_mq_start(struct ifnet *ifp, struct m *** 96,112 **** --- 96,115 ---- } else #endif i = m->m_pkthdr.flowid % vsi->num_queues; } else i = curcpu % vsi->num_queues; + + #if 0 /* ** This may not be perfect, but until something ** better comes along it will keep from scheduling ** on stalled queues. */ if (((1 << i) & vsi->active_queues) == 0) i = ffsl(vsi->active_queues); + #endif que = &vsi->queues[i]; txr = &que->txr; err = drbr_enqueue(ifp, txr->br, m); [lakshmis@mau-bsd-10a ~/fortville/hol/sys/dev/ixl]$ Would appreciate your feedback on the same. Thanks, LN