Date: Tue, 28 Apr 2015 14:19:34 +0200 From: Mark Schouten <mark@tuxis.nl> To: Chris Forgeron <cforgeron@acsi.ca>, "freebsd-net@FreeBSD.org" <freebsd-net@FreeBSD.org> Subject: Re: Frequent hickups on the networking layer Message-ID: <4294293883-7795@kerio.tuxis.nl> In-Reply-To: <46D80686C389884BB0C047851038EC4501C98541EB@AA-EX0.acsi.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] I've disabled TSO a while ago, after my networking stopped working completely, with `ifconfig em0 -tso`. em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000 options=4209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWTSO> ether 00:25:90:dc:0f:a2 nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> media: Ethernet autoselect (1000baseT <full-duplex>) status: active nfsv212: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=3<RXCSUM,TXCSUM> ether 00:25:90:dc:0f:a2 inet6 fe80::225:90ff:fedc:fa2%nfsv212 prefixlen 64 scopeid 0x6 inet6 fd::212:31:3:111:fffe prefixlen 64 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> media: Ethernet autoselect (1000baseT <full-duplex>) status: active vlan: 212 parent interface: em0 nfsv308: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000 options=3<RXCSUM,TXCSUM> ether 00:25:90:dc:0f:a2 inet 10.38.0.253 netmask 0xffffff00 broadcast 10.38.0.255 inet6 fe80::225:90ff:fedc:fa2%nfsv308 prefixlen 64 scopeid 0x8 inet6 fd:308::30 prefixlen 64 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> media: Ethernet autoselect (1000baseT <full-duplex>) status: active vlan: 308 parent interface: em0 Or should I disable VLAN_HWTSO as well. If so, how? Met vriendelijke groeten, -- Kerio Operator in de Cloud? https://www.kerioindecloud.nl/ Mark Schouten | Tuxis Internet Engineering KvK: 61527076 | http://www.tuxis.nl/ T: 0318 200208 | info@tuxis.nl Van: Chris Forgeron <cforgeron@acsi.ca> Aan: Mark Schouten <mark@tuxis.nl>, "freebsd-net@FreeBSD.org" <freebsd-net@FreeBSD.org> Verzonden: 28-4-2015 13:58 Onderwerp: RE: Frequent hickups on the networking layer What network care are you using? There have been a few reports of issues with TSO, if you check the list. I had some myself a while ago, but are now resolved thanks to a few helpful folk here. You could increase your mbufs, but if the problem is a leak/error in the stack, then you're just delaying the behaviour. -----Original Message----- From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-net@freebsd.org] On Behalf Of Mark Schouten Sent: Tuesday, April 28, 2015 5:48 AM To: freebsd-net@FreeBSD.org Subject: Frequent hickups on the networking layer Hi, I've got a FreeBSD 10.1-RELEASE box running with iscsi on top of ZFS. I've had some major issues with it where it would stop processing traffic for a minute or two, but that's 'fixed' by disabling TSO. I do have frequent iscsi errors, which are luckily fixed on the iscsi layer, but they do cause an occasional errormessage on both the iscsi client and server. Also, I see input errors on the FreeBSD server, but I'm unable to find out what those are. I do see a relation between iscsi-errormessages and the number of ethernet inputerrors on the server. I saw this message [1] which made me have a look at `vmstat -z`, and that shows me the following: vmstat -z | head -n 1; vmstat -z | sort -k 6 -t , | tail -10 ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP zio_data_buf_94208: 94208, 0, 162, 5, 135632, 0, 0 zio_data_buf_98304: 98304, 0, 118, 9, 101606, 0, 0 zio_link_cache: 48, 0, 6, 30870,24853549414, 0, 0 8 Bucket: 64, 0, 145, 2831,148672720, 11, 0 32 Bucket: 256, 0, 859, 731,231513474, 52, 0 mbuf_jumbo_9k: 9216, 604528, 7230, 2002,11764806459,108298123, 0 64 Bucket: 512, 0, 808, 352,147120342,16375582, 0 256 Bucket: 2048, 0, 500, 50,307051808,189685088, 0 vmem btag: 56, 0, 1671605, 1291509,198933250,36431, 0 128 Bucket: 1024, 0, 410, 106,65267164,772374, 0 I am using jumboframes. Could it be that the inputerrors AND my frequent hickups come from all those failures to allocate 9k jumbo mbufs? And can I increase the in [1] mentioned sysctls at will? Thanks [1]: https://lists.freebsd.org/pipermail/freebsd-questions/2013-August/252827.html Met vriendelijke groeten, -- Kerio Operator in de Cloud? https://www.kerioindecloud.nl/ Mark Schouten | Tuxis Internet Engineering KvK: 61527076 | http://www.tuxis.nl/ T: 0318 200208 | info@tuxis.nl [-- Attachment #2 --] 0 *H 01 0 + 0 *H 00,]~!dn=0 *H 01 0 UGB10UGreater Manchester10USalford10U COMODO CA Limited1907U0COMODO Client Authentication and Secure Email CA0 140701000000Z 150701235959Z0 1 0 *H mark@tuxis.nl0"0 *H 0 ,e1`4jEtuBeE")>ԁY 7ՀI&]KͨqkRnZÚC$jw4[ߞ? f)Sm6,M%#R 2U\?blOZǣۍq }(/nKNoovtڪmrw4Ԣ -H2R 0!0PnB_n?uW+RoqCڇvx|1B/ Lan / )` 00 U #0zN t[xcd'/[y{0 U ~tDeHO3 1M]0U 0 U 0 0 U %0+ +10 `HB 0FU ?0=0; +10+0)+ https://secure.comodo.net/CPS0WU P0N0LJHFhttp://crl.comodoca.com/COMODOClientAuthenticationandSecureEmailCA.crl0+|0z0R+0Fhttp://crt.comodoca.com/COMODOClientAuthenticationandSecureEmailCA.crt0$+0http://ocsp.comodoca.com0U 0 mark@tuxis.nl0 *H |0D7 -Ѐ1Ν {(pSiW;@Fλ2I)l`ϥ#<v.|Y;oa95z.dJ5 _[/>1I/ QSby@o@=T<cW4$?^KXkO=Uw NCdFC ӡLWk?@\˯K OdAWL{yC0zb%|{&1ӷ >g ʋlg !&600mOj3""2zq0 *H 01 0 UUS1 0 UUT10USalt Lake City1 0 U The USERTRUST Network1!0 U http://www.usertrust.com1604U-UTN-USERFirst-Client Authentication and Email0 110428000000Z 200530104838Z01 0 UGB10UGreater Manchester10USalford10U COMODO CA Limited1907U0COMODO Client Authentication and Secure Email CA0"0 *H 0 [KW^/@ȣSX_fe2N2}UxLUB'qi2@'Vbqi c^`ʢAj HmeC*.+c8w߱ڂ2jgo \5Tq 7 PSl Y1 LR@[HhJ$ :q_㬿;%qh=XF<hmz!W42~JRrd&N`ohQcB}"cөΞD\[5 K0G0 U #0g}ĝ&p KPH|=n}0 U zN t[xcd'/[y{0U 0U 0 0U 00U 0XU Q0O0MKIGhttp://crl.usertrust.com/UTN-USERFirst-ClientAuthenticationandEmail.crl0t+h0f0=+01http://crt.usertrust.com/UTNAddTrustClient_CA.crt0%+0http://ocsp.usertrust.com0 *H ־xWUm3DRB JAIZҭsn>&|L0(B<%> u=9fѡMo(l tZڱuz/y VtCr`9 G:eH<=%`I?C 3_н`j;:< I3B)93i.EMiڀ =]|Gm]W0KID~y83:]&XaU!ՙC@B0Ұun004=+'44pT0 *H 0o1 0 USE10U AddTrust AB1&0$U AddTrust External TTP Network1"0 UAddTrust External CA Root0 050607080910Z 200530104838Z01 0 UUS1 0 UUT10USalt Lake City1 0 U The USERTRUST Network1!0 U http://www.usertrust.com1604U-UTN-USERFirst-Client Authentication and Email0"0 *H 0 9}A;bF7`u9eJGHjM5BI/ |1Nd.)բdąQ5yNh{ zɤ2 O0 nFxoY^/m/묡j.g5y iF͠ v:z'[=s "HaLi.1 ,CZ qYں gT: we tbh ~GeMW(t40b0, 00 U #0z4&&T$T0 U g}ĝ&p KPH|=n}0U 0U 00U 00U 0DU =0;09753http://crl.usertrust.com/AddTrustExternalCARoot.crl05+)0'0%+0http://ocsp.usertrust.com0 *H c(1{b#1sSQ L֟/g~x3t&dp bP#4Vp4nx7 _j̉_|>Q5|`k:+߳}[| P-sxt1^˚ƹ7urDg% R%G<N 6wH\-?`q`q 6 lKuI;ٟMx&-n_ 10001 0 UGB10UGreater Manchester10USalford10U COMODO CA Limited1907U0COMODO Client Authentication and Secure Email CA,]~!dn=0 + 0 *H 1 *H 0 *H 1 150428121934Z0# *H 1盪m`680y *H 1l0j0 `He*0 `He0 `He0 *H 0*H 0 *H @0+0 *H (0 *H NYU C,gv4i2A , _ˍ%Y Xmkp:U.keƗ$,WԔ1 B1tk%jv.xsUb`91\A]!C`tᶶ}Dy@lN3_r'2xU >,.kWU r9&D9袕E ^ E]di9ˉeX߳X WU
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4294293883-7795>