From owner-freebsd-stable@FreeBSD.ORG Fri Jan 15 19:35:05 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3FE3B106568B for ; Fri, 15 Jan 2010 19:35:05 +0000 (UTC) (envelope-from alan.bryan@yahoo.com) Received: from web50507.mail.re2.yahoo.com (web50507.mail.re2.yahoo.com [206.190.38.83]) by mx1.freebsd.org (Postfix) with SMTP id CBA4E8FC21 for ; Fri, 15 Jan 2010 19:35:04 +0000 (UTC) Received: (qmail 14504 invoked by uid 60001); 15 Jan 2010 19:35:01 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1263584101; bh=cFnaB6/HJbjcGq6ZKIZEd0oEtp1qGo2VmDnbi1UwsRs=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=Vip3q0Lp83Ik2slW9DQSTkqqLhl4CS6XUetZb+YpZ9bqtzllkPxHXrYqq+b0CNICAlOHrd5rPyz1wio9g+XM9e+FTvEpJfr7h+dzxlg/3k+GCg0I4b81kmgqDqN4RWPuDQYdMvydonjlpcgL5kyacVJ4ER0Du5YtHrlXbSI86cU= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=EaK3v7uAXMVAIAVbC6UFJ7z4qf9Rg9ezL/nMSWo7di75wyP7ATstNqpq7kYRT0IjkHrM7yWaFqjcf2w8/zpQ6LKJ+KIpD1Ose2bL1xXeDZHYJ1McfX8DJ49WJINeGZl4KBkFBuJAmoeFY59gvkvqyBalAvLLKLBOeKPUu0zZ6d8=; Message-ID: <659970.14031.qm@web50507.mail.re2.yahoo.com> X-YMail-OSG: _CjFxQYVM1n2mx4mUGz8OAV32aQg3UU0sGqbECBr.u.XfVHVNDN.uIeXTfJIGHLt7BApJ4cZeqlHp29pf5CsdxJzflJuPZkaBf7SSrPMObTWUXdwHm07fzz2G5915f7sGsMBYg18bYI6hsR8YeGbjjT1g4CE2v8ZxckMYJvaC9rYHAi1hWHbKxR5OyAQyxJfcMcBk7gpADhX2JKRwnDESrK.rq6dNLBTORDvTPC_1_FowgdTMcwkDtty9pm1eYpw4vjkH7UNzDrs_qplIjR.lScBZCzIwpOapkSwpCfwW1uakXxMCZ1IpbqTVl5ro95N33xr3CMRA67YYdQ_gvfIiF1gBFe2oO_MjmrubESktuVVL7YfPR7vFEIU6A2GMEmD4N6Lit0xG5fG Received: from [99.24.6.121] by web50507.mail.re2.yahoo.com via HTTP; Fri, 15 Jan 2010 11:35:01 PST X-Mailer: YahooMailClassic/9.0.20 YahooMailWebService/0.8.100.260964 Date: Fri, 15 Jan 2010 11:35:01 -0800 (PST) From: alan bryan To: Chuck Swiger In-Reply-To: <81046D84-0BF4-4F3C-A262-B28A39F9D4A0@mac.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: General problems with checksums (txcsum/rxcsum) on FreeBSD 8.0? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2010 19:35:05 -0000 =0A=0A--- On Fri, 1/15/10, Chuck Swiger wrote:=0A=0A> Fro= m: Chuck Swiger =0A> Subject: Re: General problems with ch= ecksums (txcsum/rxcsum) on FreeBSD 8.0?=0A> To: "alan bryan" =0A> Cc: freebsd-stable@freebsd.org=0A> Date: Friday, January 15, 2= 010, 11:21 AM=0A> Hi--=0A> =0A> On Jan 15, 2010, at 11:12 AM, alan bryan wr= ote:=0A> [ ... ]=0A> > I'm currently sifting through a tcpdump in wireshark= =0A> and there are all sorts of messages in there about checksums=0A> being= incorrect - both TCP and UDP.=0A> =0A> If you run tcpdump on a machine, it= normally will receive=0A> the traffic being sent from that machine before = the=0A> checksums are computed, especially if HW checksumming is=0A> being = used.=A0 For reliable detection of these problems,=0A> you need to look at = the traffic either on a hub or via the=0A> monitoring or span port of a sma= rt switch, although simply=0A> glancing at the checksum stats from "netstat= -s" on both=0A> sides should indicate whether significant error rates are= =0A> happening.=0A> =0A> Regards,=0A> -- =0A> -Chuck=0A=0A=0AAh - thanks fo= r the explanation as I'm new to tcpdump/wireshark. Need to learn more about= those tools still.=0A=0A=0A=0AWatching "systat -ip 1" gives results like:= =0A=0A /0 /1 /2 /3 /4 /5 /6 /7 /8 /9 = /10=0A Load Average |||||=0A=0A IP Input = IP Output=0A 4534 total packets received 4395 total pack= ets sent=0A 0 - with bad checksums 4395 - generated locally= =0A 0 - too short for header 0 - output drops=0A 0 = - too short for data 42 output fragments generated=0A 0 -= with invalid hlen 0 - fragmentation failed=0A 0 - with= invalid length 0 destinations unreachable=0A 0 - with in= valid version 0 packets output via raw IP=0A 0 - jumbogram= s=0A 84 total fragments received UDP Statistics=0A 0= - fragments dropped 3989 total input packets=0A 0 - fragm= ents timed out 0 - too short for header=0A 14 - packets re= assembled ok 0 - invalid checksum=0A 0 packets forwarded = 0 - no checksum=0A 0 - unreachable dests 0 = - invalid length=0A 0 - redirects generated 0 - no socket= for dest port=0A 0 option errors 0 - no socket f= or broadcast=0A 0 unwanted multicasts 0 - socket buffer= full=0A 4464 delivered to upper layer 3989 total output packets= =0A=0A=0AThe total IP packets for this box should normally be in the hundre= ds per second - it stays constant at this 4K level. =0A=0AAnd the matching= netstat -s:=0A=0Atcp:=0A=0947312791 packets sent=0A=09=0928828783 data pac= kets (26317159806 bytes)=0A=09=09206 data packets (633246 bytes) retransmit= ted=0A=09=0911 data packets unnecessarily retransmitted=0A=09=090 resends i= nitiated by MTU discovery=0A=09=0912364262 ack-only packets (944193 delayed= )=0A=09=090 URG only packets=0A=09=092518 window probe packets=0A=09=093212= 717 window update packets=0A=09=092903587 control packets=0A=0959176152 pac= kets received=0A=09=0934785356 acks (for 26320662084 bytes)=0A=09=091849829= duplicate acks=0A=09=090 acks for unsent data=0A=09=0939787782 packets (24= 426616075 bytes) received in-sequence=0A=09=09601 completely duplicate pack= ets (14975 bytes)=0A=09=090 old duplicate packets=0A=09=090 packets with so= me dup. data (0 bytes duped)=0A=09=09101 out-of-order packets (114828 bytes= )=0A=09=092 packets (0 bytes) of data after window=0A=09=090 window probes= =0A=09=094411949 window update packets=0A=09=0976 packets received after cl= ose=0A=09=090 discarded for bad checksums=0A=09=090 discarded for bad heade= r offset fields=0A=09=090 discarded because packet too short=0A=09=090 disc= arded due to memory problems=0A=09905931 connection requests=0A=091098039 c= onnection accepts=0A=090 bad connection attempts=0A=090 listen queue overfl= ows=0A=09523 ignored RSTs in the windows=0A=092003969 connections establish= ed (including accepts)=0A=092002786 connections closed (including 4056 drop= s)=0A=09=091187984 connections updated cached RTT on close=0A=09=091191288 = connections updated cached RTT variance on close=0A=09=09294899 connections= updated cached ssthresh on close=0A=090 embryonic connections dropped=0A= =0934785321 segments updated rtt (of 29019225 attempts)=0A=0942636 retransm= it timeouts=0A=09=093512 connections dropped by rexmit timeout=0A=092554 pe= rsist timeouts=0A=09=090 connections dropped by persist timeout=0A=090 Conn= ections (fin_wait_2) dropped because of timeout=0A=0923 keepalive timeouts= =0A=09=0923 keepalive probes sent=0A=09=090 connections dropped by keepaliv= e=0A=0979387 correct ACK header predictions=0A=0916380715 correct data pack= et header predictions=0A=091098039 syncache entries added=0A=09=090 retrans= mitted=0A=09=090 dupsyn=0A=09=090 dropped=0A=09=091098039 completed=0A=09= =090 bucket overflow=0A=09=090 cache overflow=0A=09=090 reset=0A=09=090 sta= le=0A=09=090 aborted=0A=09=090 badack=0A=09=090 unreach=0A=09=090 zone fail= ures=0A=091098039 cookies sent=0A=090 cookies received=0A=092 SACK recovery= episodes=0A=090 segment rexmits in SACK recovery episodes=0A=090 byte rexm= its in SACK recovery episodes=0A=090 SACK options (SACK blocks) received=0A= =0955 SACK options (SACK blocks) sent=0A=090 SACK scoreboard overflow=0Audp= :=0A=0979558178 datagrams received=0A=090 with incomplete header=0A=090 wit= h bad data length field=0A=092 with bad checksum=0A=09929 with no checksum= =0A=091622 dropped due to no socket=0A=0927825 broadcast/multicast datagram= s undelivered=0A=090 dropped due to full socket buffers=0A=090 not for hash= ed pcb=0A=0979528729 delivered=0A=0979523714 datagrams output=0A=090 times = multicast source filter matched=0Asctp:=0A=090 input packets=0A=09=090 data= grams=0A=09=090 packets that had data=0A=09=090 input SACK chunks=0A=09=090= input DATA chunks=0A=09=090 duplicate DATA chunks=0A=09=090 input HB chunk= s=0A=09=090 HB-ACK chunks=0A=09=090 input ECNE chunks=0A=09=090 input AUTH = chunks=0A=09=090 chunks missing AUTH=0A=09=090 invalid HMAC ids received=0A= =09=090 invalid secret ids received=0A=09=090 auth failed=0A=09=090 fast pa= th receives all one chunk=0A=09=090 fast path multi-part data=0A=090 output= packets=0A=09=090 output SACKs=0A=09=090 output DATA chunks=0A=09=090 retr= ansmitted DATA chunks=0A=09=090 fast retransmitted DATA chunks=0A=09=090 FR= 's that happened more than once to same chunk.=0A=09=090 intput HB chunks= =0A=09=090 output ECNE chunks=0A=09=090 output AUTH chunks=0A=09=090 ip_out= put error counter=0A=09Packet drop statistics:=0A=09=090 from middle box=0A= =09=090 from end host=0A=09=090 with data=0A=09=090 non-data, non-endhost= =0A=09=090 non-endhost, bandwidth rep only=0A=09=090 not enough for chunk h= eader=0A=09=090 not enough data to confirm=0A=09=090 where process_chunk_dr= op said break=0A=09=090 failed to find TSN=0A=09=090 attempt reverse TSN lo= okup=0A=09=090 e-host confirms zero-rwnd=0A=09=090 midbox confirms no space= =0A=09=090 data did not match TSN=0A=09=090 TSN's marked for Fast Retran=0A= =09Timeouts:=0A=09=090 iterator timers fired=0A=09=090 T3 data time outs=0A= =09=090 window probe (T3) timers fired=0A=09=090 INIT timers fired=0A=09=09= 0 sack timers fired=0A=09=090 shutdown timers fired=0A=09=090 heartbeat tim= ers fired=0A=09=090 a cookie timeout fired=0A=09=090 an endpoint changed it= s cookiesecret=0A=09=090 PMTU timers fired=0A=09=090 shutdown ack timers fi= red=0A=09=090 shutdown guard timers fired=0A=09=090 stream reset timers fir= ed=0A=09=090 early FR timers fired=0A=09=090 an asconf timer fired=0A=09=09= 0 auto close timer fired=0A=09=090 asoc free timers expired=0A=09=090 inp f= ree timers expired=0A=090 packet shorter than header=0A=090 checksum error= =0A=090 no endpoint for port=0A=090 bad v-tag=0A=090 bad SID=0A=090 no memo= ry=0A=090 number of multiple FR in a RTT window=0A=090 RFC813 allowed sendi= ng=0A=090 RFC813 does not allow sending=0A=090 max burst dosn't allow sendi= ng=0A=090 look ahead tells us no memory in interface=0A=090 numbers of wind= ow probes sent=0A=090 times an output error to clamp down on next user send= .=0A=090 times sctp_senderrors were caused from a user=0A=090 number of in = data drops due to chunk limit reached=0A=090 number of in data drops due to= rwnd limit reached=0A=090 times a ECN reduced the cwnd=0A=090 used express= lookup via vtag=0A=090 collision in express lookup.=0A=090 times the sende= r ran dry of user data on primary=0A=090 same for above=0A=090 sacks the sl= ow way=0A=090 window update only sacks sent=0A=090 sends with sinfo_flags != =3D0=0A=090 unordered sends=0A=090 sends with EOF flag set=0A=090 sends wit= h ABORT flag set=0A=090 times protocol drain called=0A=090 times we did a p= rotocol drain=0A=090 times recv was called with peek=0A=090 cached chunks u= sed=0A=090 cached stream oq's used=0A=090 unread messages abandonded by clo= se=0A=090 send burst avoidance, already max burst inflight to net=0A=090 se= nd cwnd full avoidance, already max burst inflight to net=0A=090 number of = map array over-runs via fwd-tsn's=0Aip:=0A=09153027955 total packets receiv= ed=0A=090 bad header checksums=0A=090 with size smaller than minimum=0A=090= with data size < data length=0A=090 with ip length > max ip packet size=0A= =090 with header length < data size=0A=090 with data length < header length= =0A=090 with bad options=0A=090 with incorrect version number=0A=0917594719= fragments received=0A=0919 fragments dropped (dup or out of space)=0A=0911= fragments dropped after timeout=0A=092948502 packets reassembled ok=0A=091= 38381640 packets for this host=0A=0914 packets for unknown/unsupported prot= ocol=0A=090 packets forwarded (0 packets fast forwarded)=0A=090 packets not= forwardable=0A=0955 packets received for unknown multicast group=0A=090 re= directs sent=0A=09126477794 packets sent from this host=0A=090 packets sent= with fabricated ip header=0A=090 output packets dropped due to no bufs, et= c.=0A=090 output packets discarded due to no route=0A=091732555 output data= grams fragmented=0A=0910345284 fragments created=0A=090 datagrams that can'= t be fragmented=0A=090 tunneling packets that can't find gif=0A=090 datagra= ms with bad address in header=0Aicmp:=0A=091622 calls to icmp_error=0A=090 = errors not generated in response to an icmp message=0A=09Output histogram:= =0A=09=09echo reply: 2=0A=09=09destination unreachable: 1622=0A=090 message= s with bad code fields=0A=090 messages < minimum length=0A=090 bad checksum= s=0A=090 messages with bad length=0A=090 multicast echo requests ignored=0A= =090 multicast timestamp requests ignored=0A=09Input histogram:=0A=09=09des= tination unreachable: 14=0A=09=09echo: 2=0A=092 message responses generated= =0A=090 invalid return addresses=0A=090 no return routes=0A=09ICMP address = mask responses are disabled=0Aigmp:=0A=090 messages received=0A=090 message= s received with too few bytes=0A=090 messages received with bad checksum=0A= =090 membership queries received=0A=090 membership queries received with in= valid field(s)=0A=090 membership reports received=0A=090 membership reports= received with invalid field(s)=0A=090 membership reports received for grou= ps to which we belong=0A=090 membership reports sent=0Aip6:=0A=09352688 tot= al packets received=0A=090 with size smaller than minimum=0A=090 with data = size < data length=0A=090 with bad options=0A=090 with incorrect version nu= mber=0A=090 fragments received=0A=090 fragments dropped (dup or out of spac= e)=0A=090 fragments dropped after timeout=0A=090 fragments that exceeded li= mit=0A=090 packets reassembled ok=0A=09352688 packets for this host=0A=090 = packets forwarded=0A=090 packets not forwardable=0A=090 redirects sent=0A= =09352692 packets sent from this host=0A=090 packets sent with fabricated i= p header=0A=090 output packets dropped due to no bufs, etc.=0A=091 output p= acket discarded due to no route=0A=090 output datagrams fragmented=0A=090 f= ragments created=0A=090 datagrams that can't be fragmented=0A=090 packets t= hat violated scope rules=0A=090 multicast packets which we don't join=0A=09= Input histogram:=0A=09=09TCP: 352688=0A=09Mbuf statistics:=0A=09=09352688 o= ne mbuf=0A=09=090 one ext mbuf=0A=09=090 two or more ext mbuf=0A=090 packet= s whose headers are not continuous=0A=090 tunneling packets that can't find= gif=0A=090 packets discarded because of too many headers=0A=090 failures o= f source address selection=0A=09352687 forward cache hit=0A=090 forward cac= he miss=0A=09Source addresses selection rule applied:=0A=09=0950384 first c= andidate=0A=09=0950384 same address=0Aicmp6:=0A=090 calls to icmp6_error=0A= =090 errors not generated in response to an icmp6 message=0A=090 errors not= generated because of rate limitation=0A=09Output histogram:=0A=09=09multic= ast listener report: 4=0A=090 messages with bad code fields=0A=090 messages= < minimum length=0A=090 bad checksums=0A=090 messages with bad length=0A= =09Histogram of error messages to be generated:=0A=09=090 no route=0A=09=09= 0 administratively prohibited=0A=09=090 beyond scope=0A=09=090 address unre= achable=0A=09=090 port unreachable=0A=09=090 packet too big=0A=09=090 time = exceed transit=0A=09=090 time exceed reassembly=0A=09=090 erroneous header = field=0A=09=090 unrecognized next header=0A=09=090 unrecognized option=0A= =09=090 redirect=0A=09=090 unknown=0A=090 message responses generated=0A=09= 0 messages with too many ND options=0A=090 messages with bad ND options=0A= =090 bad neighbor solicitation messages=0A=090 bad neighbor advertisement m= essages=0A=090 bad router solicitation messages=0A=090 bad router advertise= ment messages=0A=090 bad redirect messages=0A=090 path MTU changes=0Arip6:= =0A=090 messages received=0A=090 checksum calcurations on inbound=0A=090 me= ssages with bad checksum=0A=090 messages dropped due to no socket=0A=090 mu= lticast messages dropped due to no socket=0A=090 messages dropped due to fu= ll socket buffers=0A=090 delivered=0A=090 datagrams output=0A=0A=0A=0A=0A= =0ANow for the NFS (FreeBSD 8.0) server side:=0A=0A /0 = /1 /2 /3 /4 /5 /6 /7 /8 /9 /10=0A Load Average |= =0A=0A IP Input IP Output=0A 5388 to= tal packets received 4717 total packets sent=0A 0 - with bad = checksums 4717 - generated locally=0A 0 - too short for hea= der 0 - output drops=0A 0 - too short for data = 65 output fragments generated=0A 0 - with invalid hlen = 0 - fragmentation failed=0A 0 - with invalid length 0 des= tinations unreachable=0A 0 - with invalid version 0 packet= s output via raw IP=0A 0 - jumbograms=0A 808 total fragments re= ceived UDP Statistics=0A 0 - fragments dropped 4= 714 total input packets=0A 0 - fragments timed out 0 - to= o short for header=0A 137 - packets reassembled ok 0 - invalid= checksum=0A 0 packets forwarded 0 - no checksum=0A = 0 - unreachable dests 0 - invalid length=0A 0 - r= edirects generated 0 - no socket for dest port=0A 0 optio= n errors 0 - no socket for broadcast=0A 0 unwante= d multicasts 0 - socket buffer full=0A 4717 delivered to u= pper layer 4714 total output packets=0A=0A=0AAgain - packet volume sho= uld be in the hundreds right now with bursts during traffic - but this is a= constant. There is no growth in disk usage over time - nothing is actuall= y getting written out anywhere. If I kill everything on the FreeBSD 7.0 cl= ient the packets still transfer until the client is rebooted. Then everyth= ing is usually OK again and then often serveral hrs later the FreeBSD 8.0 s= erver loses networking or simply reboots.=0A=0AAnd to complete the stats, h= ere's netstat -s for the FreeBSD 8.0 NFS server:=0A=0Atcp:=0A=0911241344 pa= ckets sent=0A=09=098436896 data packets (24959317163 bytes)=0A=09=09100 dat= a packets (112180 bytes) retransmitted=0A=09=091 data packet unnecessarily = retransmitted=0A=09=090 resends initiated by MTU discovery=0A=09=092282609 = ack-only packets (6239 delayed)=0A=09=090 URG only packets=0A=09=090 window= probe packets=0A=09=09518530 window update packets=0A=09=093305 control pa= ckets=0A=0917239617 packets received=0A=09=0910845777 acks (for 24959571358= bytes)=0A=09=09560 duplicate acks=0A=09=090 acks for unsent data=0A=09=091= 0437746 packets (4690003575 bytes) received in-sequence=0A=09=0917 complete= ly duplicate packets (11070 bytes)=0A=09=090 old duplicate packets=0A=09=09= 0 packets with some dup. data (0 bytes duped)=0A=09=0939 out-of-order packe= ts (43478 bytes)=0A=09=090 packets (0 bytes) of data after window=0A=09=090= window probes=0A=09=091130377 window update packets=0A=09=09132 packets re= ceived after close=0A=09=090 discarded for bad checksums=0A=09=090 discarde= d for bad header offset fields=0A=09=090 discarded because packet too short= =0A=09=090 discarded due to memory problems=0A=093 connection requests=0A= =093310 connection accepts=0A=091 bad connection attempt=0A=090 listen queu= e overflows=0A=090 ignored RSTs in the windows=0A=093313 connections establ= ished (including accepts)=0A=093302 connections closed (including 1 drop)= =0A=09=092233 connections updated cached RTT on close=0A=09=092233 connecti= ons updated cached RTT variance on close=0A=09=0921 connections updated cac= hed ssthresh on close=0A=090 embryonic connections dropped=0A=098077952 seg= ments updated rtt (of 6908327 attempts)=0A=0910 retransmit timeouts=0A=09= =090 connections dropped by rexmit timeout=0A=090 persist timeouts=0A=09=09= 0 connections dropped by persist timeout=0A=090 Connections (fin_wait_2) dr= opped because of timeout=0A=0910 keepalive timeouts=0A=09=099 keepalive pro= bes sent=0A=09=091 connection dropped by keepalive=0A=091341332 correct ACK= header predictions=0A=095196626 correct data packet header predictions=0A= =093316 syncache entries added=0A=09=0918 retransmitted=0A=09=092 dupsyn=0A= =09=090 dropped=0A=09=093310 completed=0A=09=090 bucket overflow=0A=09=090 = cache overflow=0A=09=090 reset=0A=09=096 stale=0A=09=090 aborted=0A=09=090 = badack=0A=09=090 unreach=0A=09=090 zone failures=0A=093316 cookies sent=0A= =090 cookies received=0A=0918 SACK recovery episodes=0A=0989 segment rexmit= s in SACK recovery episodes=0A=09101595 byte rexmits in SACK recovery episo= des=0A=09640 SACK options (SACK blocks) received=0A=0977 SACK options (SACK= blocks) sent=0A=090 SACK scoreboard overflow=0A=090 packets with ECN CE bi= t set=0A=090 packets with ECN ECT(0) bit set=0A=090 packets with ECN ECT(1)= bit set=0A=090 successful ECN handshakes=0A=090 times ECN reduced the cong= estion window=0Audp:=0A=0991334533 datagrams received=0A=090 with incomplet= e header=0A=090 with bad data length field=0A=098 with bad checksum=0A=0916= with no checksum=0A=09254 dropped due to no socket=0A=094339 broadcast/mul= ticast datagrams undelivered=0A=0956 dropped due to full socket buffers=0A= =090 not for hashed pcb=0A=0991329876 delivered=0A=0991316517 datagrams out= put=0A=090 times multicast source filter matched=0Asctp:=0A=090 input packe= ts=0A=09=090 datagrams=0A=09=090 packets that had data=0A=09=090 input SACK= chunks=0A=09=090 input DATA chunks=0A=09=090 duplicate DATA chunks=0A=09= =090 input HB chunks=0A=09=090 HB-ACK chunks=0A=09=090 input ECNE chunks=0A= =09=090 input AUTH chunks=0A=09=090 chunks missing AUTH=0A=09=090 invalid H= MAC ids received=0A=09=090 invalid secret ids received=0A=09=090 auth faile= d=0A=09=090 fast path receives all one chunk=0A=09=090 fast path multi-part= data=0A=090 output packets=0A=09=090 output SACKs=0A=09=090 output DATA ch= unks=0A=09=090 retransmitted DATA chunks=0A=09=090 fast retransmitted DATA = chunks=0A=09=090 FR's that happened more than once to same chunk=0A=09=090 = intput HB chunks=0A=09=090 output ECNE chunks=0A=09=090 output AUTH chunks= =0A=09=090 ip_output error counter=0A=09Packet drop statistics:=0A=09=090 f= rom middle box=0A=09=090 from end host=0A=09=090 with data=0A=09=090 non-da= ta, non-endhost=0A=09=090 non-endhost, bandwidth rep only=0A=09=090 not eno= ugh for chunk header=0A=09=090 not enough data to confirm=0A=09=090 where p= rocess_chunk_drop said break=0A=09=090 failed to find TSN=0A=09=090 attempt= reverse TSN lookup=0A=09=090 e-host confirms zero-rwnd=0A=09=090 midbox co= nfirms no space=0A=09=090 data did not match TSN=0A=09=090 TSN's marked for= Fast Retran=0A=09Timeouts:=0A=09=093 iterator timers fired=0A=09=090 T3 da= ta time outs=0A=09=090 window probe (T3) timers fired=0A=09=090 INIT timers= fired=0A=09=090 sack timers fired=0A=09=090 shutdown timers fired=0A=09=09= 0 heartbeat timers fired=0A=09=090 a cookie timeout fired=0A=09=090 an endp= oint changed its cookiesecret=0A=09=090 PMTU timers fired=0A=09=090 shutdow= n ack timers fired=0A=09=090 shutdown guard timers fired=0A=09=090 stream r= eset timers fired=0A=09=090 early FR timers fired=0A=09=090 an asconf timer= fired=0A=09=090 auto close timer fired=0A=09=090 asoc free timers expired= =0A=09=090 inp free timers expired=0A=090 packet shorter than header=0A=090= checksum error=0A=090 no endpoint for port=0A=090 bad v-tag=0A=090 bad SID= =0A=090 no memory=0A=090 number of multiple FR in a RTT window=0A=090 RFC81= 3 allowed sending=0A=090 RFC813 does not allow sending=0A=090 times max bur= st prohibited sending=0A=090 look ahead tells us no memory in interface=0A= =090 numbers of window probes sent=0A=090 times an output error to clamp do= wn on next user send=0A=090 times sctp_senderrors were caused from a user= =0A=090 number of in data drops due to chunk limit reached=0A=090 number of= in data drops due to rwnd limit reached=0A=090 times a ECN reduced the cwn= d=0A=090 used express lookup via vtag=0A=090 collision in express lookup=0A= =090 times the sender ran dry of user data on primary=0A=090 same for above= =0A=090 sacks the slow way=0A=090 window update only sacks sent=0A=090 send= s with sinfo_flags !=3D0=0A=090 unordered sends=0A=090 sends with EOF flag = set=0A=090 sends with ABORT flag set=0A=090 times protocol drain called=0A= =090 times we did a protocol drain=0A=090 times recv was called with peek= =0A=090 cached chunks used=0A=090 cached stream oq's used=0A=090 unread mes= sages abandonded by close=0A=090 send burst avoidance, already max burst in= flight to net=0A=090 send cwnd full avoidance, already max burst inflight t= o net=0A=090 number of map array over-runs via fwd-tsn's=0Aip:=0A=091223213= 95 total packets received=0A=090 bad header checksums=0A=090 with size smal= ler than minimum=0A=090 with data size < data length=0A=090 with ip length = > max ip packet size=0A=090 with header length < data size=0A=090 with data= length < header length=0A=090 with bad options=0A=090 with incorrect versi= on number=0A=0916507934 fragments received=0A=0919 fragments dropped (dup o= r out of space)=0A=09970 fragments dropped after timeout=0A=092761830 packe= ts reassembled ok=0A=09108575291 packets for this host=0A=090 packets for u= nknown/unsupported protocol=0A=090 packets forwarded (0 packets fast forwar= ded)=0A=090 packets not forwardable=0A=090 packets received for unknown mul= ticast group=0A=090 redirects sent=0A=09102560932 packets sent from this ho= st=0A=090 packets sent with fabricated ip header=0A=090 output packets drop= ped due to no bufs, etc.=0A=090 output packets discarded due to no route=0A= =093269107 output datagrams fragmented=0A=0919499266 fragments created=0A= =090 datagrams that can't be fragmented=0A=090 tunneling packets that can't= find gif=0A=090 datagrams with bad address in header=0Aicmp:=0A=09254 call= s to icmp_error=0A=090 errors not generated in response to an icmp message= =0A=09Output histogram:=0A=09=09echo reply: 1132=0A=09=09destination unreac= hable: 254=0A=090 messages with bad code fields=0A=090 messages less than t= he minimum length=0A=090 messages with bad checksum=0A=090 messages with ba= d length=0A=090 multicast echo requests ignored=0A=090 multicast timestamp = requests ignored=0A=09Input histogram:=0A=09=09echo reply: 6=0A=09=09echo: = 1132=0A=091132 message responses generated=0A=090 invalid return addresses= =0A=090 no return routes=0A=09ICMP address mask responses are disabled=0Aig= mp:=0A=090 messages received=0A=090 messages received with too few bytes=0A= =090 messages received with wrong TTL=0A=090 messages received with bad che= cksum=0A=090 V1/V2 membership queries received=0A=090 V3 membership queries= received=0A=090 membership queries received with invalid field(s)=0A=090 g= eneral queries received=0A=090 group queries received=0A=090 group-source q= ueries received=0A=090 group-source queries dropped=0A=090 membership repor= ts received=0A=090 membership reports received with invalid field(s)=0A=090= membership reports received for groups to which we belong=0A=090 V3 report= s received without Router Alert=0A=090 membership reports sent=0Aip6:=0A=09= 166 total packets received=0A=090 with size smaller than minimum=0A=090 wit= h data size < data length=0A=090 with bad options=0A=090 with incorrect ver= sion number=0A=090 fragments received=0A=090 fragments dropped (dup or out = of space)=0A=090 fragments dropped after timeout=0A=090 fragments that exce= eded limit=0A=090 packets reassembled ok=0A=09166 packets for this host=0A= =090 packets forwarded=0A=090 packets not forwardable=0A=090 redirects sent= =0A=090 packets sent from this host=0A=090 packets sent with fabricated ip = header=0A=090 output packets dropped due to no bufs, etc.=0A=090 output pac= kets discarded due to no route=0A=090 output datagrams fragmented=0A=090 fr= agments created=0A=090 datagrams that can't be fragmented=0A=090 packets th= at violated scope rules=0A=09166 multicast packets which we don't join=0A= =09Input histogram:=0A=09=09ICMP6: 166=0A=09Mbuf statistics:=0A=09=090 one = mbuf=0A=09=09166 one ext mbuf=0A=09=090 two or more ext mbuf=0A=090 packets= whose headers are not continuous=0A=090 tunneling packets that can't find = gif=0A=090 packets discarded because of too many headers=0A=090 failures of= source address selection=0A=09Source addresses selection rule applied:=0Ai= cmp6:=0A=090 calls to icmp6_error=0A=090 errors not generated in response t= o an icmp6 message=0A=090 errors not generated because of rate limitation= =0A=090 messages with bad code fields=0A=090 messages < minimum length=0A= =090 bad checksums=0A=090 messages with bad length=0A=09Histogram of error = messages to be generated:=0A=09=090 no route=0A=09=090 administratively pro= hibited=0A=09=090 beyond scope=0A=09=090 address unreachable=0A=09=090 port= unreachable=0A=09=090 packet too big=0A=09=090 time exceed transit=0A=09= =090 time exceed reassembly=0A=09=090 erroneous header field=0A=09=090 unre= cognized next header=0A=09=090 unrecognized option=0A=09=090 redirect=0A=09= =090 unknown=0A=090 message responses generated=0A=090 messages with too ma= ny ND options=0A=090 messages with bad ND options=0A=090 bad neighbor solic= itation messages=0A=090 bad neighbor advertisement messages=0A=090 bad rout= er solicitation messages=0A=090 bad router advertisement messages=0A=090 ba= d redirect messages=0A=090 path MTU changes=0Arip6:=0A=090 messages receive= d=0A=090 checksum calculations on inbound=0A=090 messages with bad checksum= =0A=090 messages dropped due to no socket=0A=090 multicast messages dropped= due to no socket=0A=090 messages dropped due to full socket buffers=0A=090= delivered=0A=090 datagrams output=0A=0A=0A=0A=0A=0A=0A