From owner-freebsd-net@FreeBSD.ORG Mon Oct 31 09:44:19 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 00986106564A for ; Mon, 31 Oct 2011 09:44:19 +0000 (UTC) (envelope-from hoomanfazaeli@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 73D248FC0C for ; Mon, 31 Oct 2011 09:44:18 +0000 (UTC) Received: by bkbzs2 with SMTP id zs2so3161688bkb.13 for ; Mon, 31 Oct 2011 02:44:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type; bh=Z8FUXpJfIjVMqsp7sBPzYfx/ynV2sXSjr4f30qftT5M=; b=i3iCcqS7HxlYEJa81ASLYjsULa8FczPAw69lHTBrMbcKhGQKhxxyDH5QktlnUNIvDx YocNzf0mCyTPkoecU9ONGwstMHAG81Io0ihRUGNplT/OdiYO2UJHX7zRSAV7umHSutT1 e4khbzmdI3dmjNuur3fGi0Hr3PF4vclIOxOzo= Received: by 10.204.132.211 with SMTP id c19mr4190366bkt.94.1320054257215; Mon, 31 Oct 2011 02:44:17 -0700 (PDT) Received: from [127.0.0.1] ([84.241.57.181]) by mx.google.com with ESMTPS id e14sm16480417bka.0.2011.10.31.02.44.14 (version=SSLv3 cipher=OTHER); Mon, 31 Oct 2011 02:44:16 -0700 (PDT) Message-ID: <4EAE6DEA.7040701@gmail.com> Date: Mon, 31 Oct 2011 13:14:10 +0330 From: Hooman Fazaeli User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.23) Gecko/20110920 Thunderbird/3.1.15 MIME-Version: 1.0 To: Emil Muratov References: <4E8F157A.40702@sentex.net> <4E8F51D4.1060509@sentex.net> <4EA7E203.3020306@sepehrs.com> <4EA80818.3030504@sentex.net> <4EA80F88.4000400@hotplug.ru> <4EA82715.2000404@gmail.com> <4EA8FA40.7010504@hotplug.ru> <4EA91836.2040508@gmail.com> <4EA959EE.2070806@hotplug.ru> <4EAD116A.8090006@gmail.com> <4EAE58A2.9040803@gmail.com> <4EAE68B7.5030000@hotplug.ru> In-Reply-To: <4EAE68B7.5030000@hotplug.ru> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: pyunyh@gmail.com, freebsd-net@freebsd.org, Jack Vogel , Jason Wolfe Subject: Re: Intel 82574L interface wedging on em 7.1.9/7.2.3 when MSIX enabled X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Oct 2011 09:44:19 -0000 On 10/31/2011 12:51 PM, Emil Muratov wrote: > On 31.10.2011 12:13, Hooman Fazaeli wrote: >> >>> >>> >>> Thanks for looking into this. I'd be happy to test any patch thrown my way, but keep in mind my issue is only tickled when MSI-X is enabled. My interfaces aren't bouncing, though it might be >>> possible some unique path in the MSI-X code is causing a throughput hang akin to connectivity loss? >>> >>> Jack is the delta your speaking to the 7.2.4 code? I did manage to get the code from Intel compiled with a couple minutes of work, but haven't loaded it up yet as I didn't see anything that >>> caught my untrained eye in the diffs. I'll wait until its ported over and would be happy to test if needed. >>> >>> Conveniently enough I just received another report from my test boxes with a pretty stock loader.conf. I had forgotten to remove the advanced options from the interfaces after I cycled them to >>> pick up the fc_setting=0. Fixed that up just meow. >>> >>> hw.em.fc_setting="0" >>> cc_cubic_load="YES" >>> >>> >> >> Jason >> >> Attached is a patch for if_em.c. It flushes interface queue when it is full >> and link is not active. Please note that when this happens, drops are increasing >> on interface and this will trigger your scripts as before. You need to change >> a little the scripts as follows: >> >> check interface TX status >> if (interface TX seems hung) { >> sleep 5 >> check interface TX status >> if (interface TX seems hung) { >> reset the interface. >> } >> } >> >> For MULTIQUEUE, it just disables the check for link status (which is not good). >> so pls. test in non-MULTIQUEUE mode. >> >> The patch also contains some minor fixups to compile on 7 plus >> a fix from r1.69 which addressed RX hang problem (the fix was >> later removed in r1.70). I included it for Emil to give it a >> try. >> >> Pls. let me know if you have any problems with patch. >> >> >> > > Hi! Thanks for the update. But I can't make it, there is an error in build process. Can you kindly take a look at it? > > > -emil@ion-/usr/src/sys/dev/e1000 > --(0)> sudo patch < /home/emil/patches/if_em/if_em.c.patch > Password: > Hmm... Looks like a unified diff to me... > The text leading up to this was: > -------------------------- > |--- if_em.c.orig 2011-10-31 11:43:35.000000000 +0330 > |+++ if_em.c 2011-10-31 11:43:35.000000000 +0330 > -------------------------- > Patching file if_em.c using Plan A... > Hunk #1 succeeded at 85. > Hunk #2 succeeded at 101. > Hunk #3 succeeded at 382 (offset -29 lines). > Hunk #4 succeeded at 400 (offset -29 lines). > Hunk #5 succeeded at 857 (offset -29 lines). > Hunk #6 succeeded at 960 (offset -29 lines). > Hunk #7 succeeded at 1420 (offset -29 lines). > Hunk #8 succeeded at 1436 (offset -29 lines). > Hunk #9 succeeded at 1466 (offset -29 lines). > Hunk #10 succeeded at 2230 (offset -29 lines). > Hunk #11 succeeded at 2338 (offset -29 lines). > Hunk #12 succeeded at 2350 (offset -29 lines). > Hunk #13 succeeded at 3799 (offset -29 lines). > Hunk #14 succeeded at 5164 with fuzz 2 (offset -29 lines). > Hunk #15 succeeded at 5616 (offset -4 lines). > done > > -emil@ion-/usr/src/sys/dev/e1000 > --(0)> sudo patch < /home/emil/patches/if_em/if_em.h.patch > Hmm... Looks like a unified diff to me... > The text leading up to this was: > -------------------------- > |--- if_em.h.orig 2011-10-31 11:43:34.000000000 +0330 > |+++ if_em.h 2011-10-31 11:43:35.000000000 +0330 > -------------------------- > Patching file if_em.h using Plan A... > Hunk #1 succeeded at 438. > done > > > #root@ion-/usr/src/sys/modules/em > #-(0)> make > Warning: Object directory not changed from original /usr/src/sys/modules/em > awk -f @/tools/makeobjops.awk @/kern/device_if.m -h > awk -f @/tools/makeobjops.awk @/kern/bus_if.m -h > awk -f @/tools/makeobjops.awk @/dev/pci/pci_if.m -h > :> opt_inet.h > cc -O2 -pipe -march=nocona -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdinc -I/usr/src/sys/modules/em/../../dev/e1000 -I. -I@ -I@/contrib/altq > -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -fno-common -fno-omit-frame-pointer -mcmodel=kernel -mno-red-zone -mfpmath=387 > -mno-sse -mno-sse2 -mno-sse3 -mno-mmx -mno-3dnow -msoft-float -fno-asynchronous-unwind-tables - ffreestanding -fstack-protector -std=iso9899:1999 -fstack-protector -Wall > -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing- prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -c > /usr/src/sys/modules/em/../../dev/e1000/if _em.c > /usr/src/sys/modules/em/../../dev/e1000/if_em.c:387: error: 'sysctl__hw_em_children' undeclared here (not in a function) > *** Error code 1 > > Stop in /usr/src/sys/modules/em. > > > > Please sync your sys/dev/e1000 with HEAD and try again: setenv CVSROOT :pserver:anoncvs@anoncvs.FreeBSD.org:/home/ncvs cvs login password: cd /usr/src cvs co -rHEAD sys/dev/e1000