Date: Sun, 27 Mar 2011 21:38:57 -0400 From: "J. Hellenthal" <jhell@DataIX.net> To: Stefan `Sec` Zehl <sec@42.org> Cc: freebsd-net@freebsd.org Subject: Re: The tale of a TCP bug Message-ID: <alpine.BSF.2.00.1103272054380.9813@qvfongpu.qngnvk.ybpny> In-Reply-To: <20110326224340.GB23803@ice.42.org> References: <4D8B99B4.4070404@FreeBSD.org> <201103250825.10674.jhb@freebsd.org> <20110325194109.GB25392@ice.42.org> <201103251640.16147.jhb@freebsd.org> <20110326140212.GB45402@ice.42.org> <20110326224340.GB23803@ice.42.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --80310268-1666522583-1301276363=:9813 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sat, 26 Mar 2011 18:43, sec@ wrote: > Hi, > >> On Fri, Mar 25, 2011 at 16:40 -0400, John Baldwin wrote: >>> And the problem is that the code that uses 'adv' to determine if it >>> sound send a window update to the remote end is falsely succeeding due >>> to the overflow causing tcp_output() to 'goto send' but that it then >>> fails to send any data because it thinks the remote window is full? > > On a whim I wanted to find out, how often that overflow is triggered in > normal operation, and whipped up a quick counter-sysctl. > > --- sys/netinet/tcp_output.c.org 2011-01-04 19:27:00.000000000 +0100 > +++ sys/netinet/tcp_output.c 2011-03-26 18:49:30.000000000 +0100 > @@ -87,6 +87,11 @@ > extern struct mbuf *m_copypack(); > #endif > > +VNET_DEFINE(int, adv_neg) = 0; > +SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, adv_neg, CTLFLAG_RD, > + &VNET_NAME(adv_neg), 1, > + "How many times adv got negative"); > + > VNET_DEFINE(int, path_mtu_discovery) = 1; > SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, path_mtu_discovery, CTLFLAG_RW, > &VNET_NAME(path_mtu_discovery), 1, > @@ -573,6 +578,10 @@ > long adv = min(recwin, (long)TCP_MAXWIN << tp->rcv_scale) - > (tp->rcv_adv - tp->rcv_nxt); > > + if(min(recwin, (long)TCP_MAXWIN << tp->rcv_scale) < > + (tp->rcv_adv - tp->rcv_nxt)) > + adv_neg++; > + > if (adv >= (long) (2 * tp->t_maxseg)) > goto send; > if (2 * adv >= (long) so->so_rcv.sb_hiwat) > > I booted my main (web/shell) box with (only) this patch: > > 11:36PM up 3:50, 1 user, load averages: 2.29, 1.51, 0.73 > net.inet.tcp.adv_neg: 2466 > > That's approximately once every 5 seconds. That's way more often than I > suspected. > > CU, > Sec > With this patch applied with John's on a 32-bit box I can repeatedly bump this sysctl with an SSL connection to another destination. Doesn't seem to matter what the destination is. curl -q https://www.changeip.com/ip.asp It also bumps in SSL connections to other protocols too. This behavior does not seem to be happening with non-SSL connections. Attached is a script that I am using to monitor the sysctl here just for reference. L = Last value C = Current value D = Difference I = Log interval S = Seconds since last change * = marked changed line /bin/sh ./adv_neg_mon.sh 7 |tee -a adv_neg.log [...] L:41 C:41 D:0 I:7 S:7.000000e+01 L:41 C:41 D:0 I:7 S:7.700000e+01 L:41 C:43 D:2 I:7 S:8.400000e+01 * L:43 C:88 D:45 I:7 S:7.000000e+00 * - -- Regards, J. Hellenthal (0x89D8547E) JJH48-ARIN -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (FreeBSD) Comment: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x89D8547E iQEcBAEBAgAGBQJNj+a7AAoJEJBXh4mJ2FR+VssIAI7QSUUb6jvZdMWxxVGPpr6o vPGDqPfWxNcih4D5SZxJJtsslnunpAcOjSWK8YGvOCINt8XhexVOSklyHuyvjIWd 4ijywngx5H2RT22c6wTdNPOfsZzoBkvLZZ2mj2cUF1ISxrvgy5syMp/TnANE3kul Mqf29HA8t3qYQCfb6zuFoWGdYI5Ahfsks4rljZJy/5bRQfNceJwBjUGnSlL0651m Bl4GpcNWA0fbuJeUgEzIK6mOpNdoI+PrZv6GEG7LErLaVtr+43gET/YITuGv1jY3 dlQ1WkHZSnaG/S7vpWbb2W/cuJ8ak6esbM74x8KakiOnLeJgy0MYK8oqYJyN3aI= =l+iW -----END PGP SIGNATURE----- --80310268-1666522583-1301276363=:9813 Content-Type: TEXT/PLAIN; charset=US-ASCII; name=adv_neg_mon.sh Content-Transfer-Encoding: BASE64 Content-ID: <alpine.BSF.2.00.1103272138570.9813@qvfongpu.qngnvk.ybpny> Content-Description: adv_neg monitor Content-Disposition: attachment; filename=adv_neg_mon.sh IyEvYmluL3NoDQoNCnRyYXAgJ2V4aXQgMScgMg0KDQpVUERBVEU9JDEgOzog JHtVUERBVEU6PTV9DQoNCndoaWxlIHRydWU7IGRvDQoJTlZBTD0kKHN5c2N0 bCAtbiBuZXQuaW5ldC50Y3AuYWR2X25lZykNCglpZiBbIC16ICIkTFZBTCIg XTsgdGhlbg0KCQlMVkFMPSR7TlZBTH0NCglmaQ0KCWlmIFsgIiROVkFMIiAt Z3QgIiRMVkFMIiBdOyB0aGVuDQoJCWVjaG8gIkw6JExWQUwgQzokTlZBTCBE OiQoKCR7TlZBTH0tJHtMVkFMfSkpIEk6JHtVUERBVEV9IFM6JChwcmludGYg JWUgJHtVU0VDU30pICoiDQoJCVVTRUNTPSR7VVBEQVRFfQ0KCWVsc2UNCgkJ ZWNobyAiTDokTFZBTCBDOiROVkFMIEQ6JCgoJHtOVkFMfS0ke0xWQUx9KSkg SToke1VQREFURX0gUzokKHByaW50ZiAlZSAke1VTRUNTfSkiDQoJCVVTRUNT PSQoKCR7VVNFQ1N9KyR7VVBEQVRFfSkpDQoJZmkNCglMVkFMPSR7TlZBTH0N CglzbGVlcCAkVVBEQVRFDQpkb25lDQo= --80310268-1666522583-1301276363=:9813--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.BSF.2.00.1103272054380.9813>