From owner-freebsd-net@FreeBSD.ORG Mon Oct 31 00:25:28 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5CF55106564A for ; Mon, 31 Oct 2011 00:25:28 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id DAB098FC0A for ; Mon, 31 Oct 2011 00:25:27 +0000 (UTC) Received: by wwi18 with SMTP id 18so7763883wwi.31 for ; Sun, 30 Oct 2011 17:25:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=7uA/KwtvLWugU+2xQYFHX9hss+4m5FNi1rp/7dDnKpw=; b=kCJuBoBVBU6+vBOUNqmV0qARj/k5oNksC9Csd6y/X4ksQPiyh4yBOQZ3/lGmKeW43h WQVCDh8NprvzlgvGDO6bDNtSRbcCsNX/ejKGu8tpK2gyn5LD2fZoeU6QBLrLrob99XR9 yaswh6POBVfOXoegjCw/PuSU/zXIZcrpw0Jjc= Received: by 10.227.198.140 with SMTP id eo12mr14917598wbb.20.1320020726755; Sun, 30 Oct 2011 17:25:26 -0700 (PDT) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id b5sm29424131wbh.4.2011.10.30.17.25.22 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 30 Oct 2011 17:25:25 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Sun, 30 Oct 2011 17:23:49 -0700 From: YongHyeon PYUN Date: Sun, 30 Oct 2011 17:23:49 -0700 To: Hooman Fazaeli Message-ID: <20111031002349.GA1679@michelle.cdnetworks.com> References: <4EA7E203.3020306@sepehrs.com> <4EA80818.3030504@sentex.net> <4EA80F88.4000400@hotplug.ru> <4EA82715.2000404@gmail.com> <4EA8FA40.7010504@hotplug.ru> <4EA91836.2040508@gmail.com> <4EA959EE.2070806@hotplug.ru> <4EAD116A.8090006@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4EAD116A.8090006@gmail.com> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org, Jason Wolfe , Emil Muratov , Jack Vogel Subject: Re: Intel 82574L interface wedging on em 7.1.9/7.2.3 when MSIX enabled X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Oct 2011 00:25:28 -0000 On Sun, Oct 30, 2011 at 12:27:14PM +0330, Hooman Fazaeli wrote: > > I finally managed to re-produce an affect similar to Jason's case. It > may not be the exact same issue, but it is a serious problem and must > be addressed. > > 1. Push out packet on em/igb with high rate. > 2. Disconnect cable and wait for a few seconds. "netstat -ind" shows that > Drops are increasing. > 3. Re-connect the cable. Both sides of like re-negotiate and the links > comes up. > 4. But ..., no packets is ever transmitted again and Drops still increasing! > > This is because em/lem/igb and some other interfaces (i.e., bce) have > a check at the very beginning of their _start function > which checks link status and immediately returns if it is inactive. > This behavior causes if_snd to fills up in step 2 and when this happens, > IFQ_HANDOFF never calls if_start again, even when the link becomes > active again. > > A cable unplug is not necessary to trigger the issue. Any temporary > link loss (e.i., during re-negotiation) can potentially lead to > aforementioned problem. > > IMHO, this is not a driver issue and the real fix would be to change > IFQ_HANDOFF to call if_start when the queue is full. > It would be normal not to call IFQ_HANDOFF when driver queue is full since it would be waste of time to invoke driver TX routine again. If driver fails to send queued frames again when it re-established a link it sounds like a driver bug. Of course, there is a problem here as driver can send queued frames without knowing how much time the frame queued in TX queue. Depending on application this behavior may or may not useful. > Jason, If you are interested, I can prepare a patch for you > to address this issue in if_em and see if it helps. >