From owner-freebsd-net@FreeBSD.ORG Thu Mar 15 08:37:34 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D3C371065670; Thu, 15 Mar 2012 08:37:34 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pz0-f45.google.com (mail-pz0-f45.google.com [209.85.210.45]) by mx1.freebsd.org (Postfix) with ESMTP id 8093B8FC14; Thu, 15 Mar 2012 08:37:34 +0000 (UTC) Received: by dadp14 with SMTP id p14so12293111dad.18 for ; Thu, 15 Mar 2012 01:37:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=Yrc/B0YQdpCldJQ2Yd0BBD1cQv0nYwifyMyXityR81c=; b=jDRcjQys7li9SblENqW3Knt7YQ2F937bH4ZyFrdqkqsfqAzjVV6/bhhClWyhOcEjBY YE5UnwOoVTQLdZt6PP55LZscBtd8SlqR2f19ldPK9iZ0c78gDgyIPcgBfMA4I2sHks4c FqaDFcqyvLXc9r0ffHZPUpluqCfuMdEGgZWzui7QbWYa0/SJx8JFWB5RN6ZhV67hVhU+ s/HDaSoDH6Me3aBpbEoE6PnDTQrQKWN9d7/xbvm0hTFnOLd9nEEUbhW/5TZExFszYRsd t6qG1v5pSIFjIoeWOz4/n9kh+10YffhJXUOOd57V5QCFkTW22LdxpH/PmMrCgBOMYSus UsUA== Received: by 10.68.236.3 with SMTP id uq3mr3371946pbc.63.1331800654334; Thu, 15 Mar 2012 01:37:34 -0700 (PDT) Received: from pyunyh@gmail.com ([114.111.62.249]) by mx.google.com with ESMTPS id b10sm1407561pbr.46.2012.03.15.01.37.31 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 15 Mar 2012 01:37:33 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Thu, 15 Mar 2012 17:37:29 -0700 From: YongHyeon PYUN Date: Thu, 15 Mar 2012 17:37:29 -0700 To: Juli Mallett Message-ID: <20120316003729.GA3872@michelle.cdnetworks.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org Subject: Re: MSI-X + em(4) = Refresh mbufs: hdr dmamap load failure - 22 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Mar 2012 08:37:34 -0000 On Thu, Mar 15, 2012 at 12:39:30AM -0700, Juli Mallett wrote: > All, > > On both stable/9 and trunk I see that with one of either the 82571EB > or 82574L I am flooded with messages in the form of: > > Refresh mbufs: hdr dmamap load failure - 22 > > If I disable msix, then the messages go away. I am not sure why msix > vs. non-msix would matter in this case unless in the msix case there's > some kind of case of spurious interrupts causing em_rxeof to be called > without any packets available. If that happens then perhaps > e1000_rx_unrefreshed() is called when no buffers have been processed > and then em_refresh_mbufs wrongly refreshes the whole ring? > > This seems like it would be a problem because the > bus_dmamap_load_mbuf_sg code is called unconditionally, even when a > new mbuf isn't being allocated. In that case, the mapping already > exists. Wouldn't it be necessary to unload and then reload the mbuf? > So either it's a bug that em_refresh_mbufs is being called at all, or > it's naively reusing mbufs in a way that actually guarantees an error, > right? Also, in the case where it frees, only m_free is called ??? is > there never a case where that should be an m_freem? I can imagine > some, but they are likely impossible with the receive path of the > driver. (I don't know for sure because the receive path and the mbuf > refresh code keep changing and I've been unable to keep up.) > > I don't know which part it is, of course, because I don't know what > port it's coming from. Like three other printfs in the driver where > which device is being used matters tremendously, it uses a bare printf > and not a device_printf. I could modify the driver, but for now > disabling msix is easier than continuing to load new kernels to try to > debug the problem. > > Is anyone else seeing this? Has anyone further investigated the Some time ago I also tried to debug low performance issue of 82574. At that time I didn't encounter the issue you mentioned but I saw spurious interrupts with MSI-X. This issue is also mentioned in 82574 data sheet(7.4.5 Clearing Interrupt Causes). However I couldn't explain odd interrupt behavior(interrupt moderation, MSI vs. MSI-X) of controller since the controller did not work as described in data sheet. > problem? Is there a patch floating around and I just haven't found > the right search terms? > > Thanks in advance, > Juli. > > PS: Yes, I know this is kind of a crappy bug report, sorry. I've had > a limited amount of time to investigate so far, and don't want to > delay reporting it until I am able to get more time with the > problematic hardware.