From owner-freebsd-net@FreeBSD.ORG Wed Oct 26 14:04:50 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7E0D5106564A for ; Wed, 26 Oct 2011 14:04:50 +0000 (UTC) (envelope-from gpm@hotplug.ru) Received: from gate.pikinvest.ru (gate.pikinvest.ru [87.245.155.170]) by mx1.freebsd.org (Postfix) with ESMTP id 2BEFD8FC13 for ; Wed, 26 Oct 2011 14:04:49 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailgate.pik.ru (Postfix) with ESMTP id 285441C0867; Wed, 26 Oct 2011 17:48:17 +0400 (MSD) Received: from EX03PIK.PICompany.ru (unknown [192.168.156.52]) by mailgate.pik.ru (Postfix) with ESMTP id E61A81C0864; Wed, 26 Oct 2011 17:48:14 +0400 (MSD) Received: from [192.168.148.9] ([192.168.148.9]) by EX03PIK.PICompany.ru with Microsoft SMTPSVC(6.0.3790.4675); Wed, 26 Oct 2011 17:48:11 +0400 Message-ID: <4EA80F88.4000400@hotplug.ru> Date: Wed, 26 Oct 2011 17:47:52 +0400 From: Emil Muratov User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.23) Gecko/20110922 Thunderbird/3.1.15 MIME-Version: 1.0 To: Mike Tancsa References: <4E8F157A.40702@sentex.net> <4E8F51D4.1060509@sentex.net> <4EA7E203.3020306@sepehrs.com> <4EA80818.3030504@sentex.net> In-Reply-To: <4EA80818.3030504@sentex.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 26 Oct 2011 13:48:11.0505 (UTC) FILETIME=[E6C02E10:01CC93E5] Cc: freebsd-net@freebsd.org, Hooman Fazaeli , Jack Vogel , Jason Wolfe Subject: Re: Intel 82574L interface wedging on em 7.1.9/7.2.3 when MSIX enabled X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Oct 2011 14:04:50 -0000 >> Hooman, >> >> I have run with dev.em.X.flow_control=0, which should have the same result >> as hw.em.fc_setting=0, and net.inet.tcp.tso is also 0. I'm not sure the >> remaining options would be able to produce the scenario I'm seeing, but I'm >> open to giving it a try with no options on the interfaces. I've also added >> ifconfig output to the collection. >> >> options=219b >> ifconfig emX -rxcsum -txcsum -vlanhwtag -tso -wol >> options=88 >> >> It's always TX, but these servers push ~12x what they receive, so I'm >> guessing it could happen to either buffer given the right traffic patterns. >> While looking through commits I also found a patch to add a couple sysctls >> for em, which I'm adding - http://freshbsd.org/commit/freebsd/r223676 > Can you provide some more details as to how traffic flows on these servers ? Are they going through them, or are they generating the traffic ? I wonder given a smaller CPU, it would be easier to trigger the condition somehow. We recently got a Soekris 6501 which has onboard 4 such em nics, but has just a 1Ghz Atom. > Hi All, I've got almost the same problem with intel 82574L based nic. My platform is nvidia ion running Atom 1.6 and nic is an external PCI-express adapter. Unlike Jason's case mine is always stuck in receiving traffic, it's Ierrs increasing while Ipkts not. Thanks to Jason's script I can see those locks and interface flapping every several hours. My system is not a heavy loaded server but just a home nas/router, usually routing at 100 mbps or less. Nither disabling MSIX nor tuning txd rxd doesn't help me.