Date: Tue, 10 Nov 2009 08:42:24 +0000 From: Doug Rabson <dfr@rabson.org> To: Rick Macklem <rmacklem@uoguelph.ca> Cc: O.Seibert@cs.ru.nl, dfr@FreeBSD.org, freebsd-current@FreeBSD.org, Rui Paulo <rpaulo@FreeBSD.org>, kometen@gmail.com Subject: Re: Help needed: TCP Wizards (was 8.0-RC1 NFS client timeout issue) Message-ID: <3D80A486-FB8F-451D-BF44-F62C647D8556@rabson.org> In-Reply-To: <Pine.GSO.4.63.0911091744570.18527@muncher.cs.uoguelph.ca> References: <Pine.GSO.4.63.0911011644410.19276@muncher.cs.uoguelph.ca> <4AF0B7DF.9030405@freebsd.org> <Pine.GSO.4.63.0911051121340.5409@muncher.cs.uoguelph.ca> <030A8229-9707-4F70-B4BE-584F1BF9ECEC@FreeBSD.org> <Pine.GSO.4.63.0911091744570.18527@muncher.cs.uoguelph.ca>
index | next in thread | previous in thread | raw e-mail
On 9 Nov 2009, at 23:04, Rick Macklem wrote:
>
>
> On Thu, 5 Nov 2009, Rui Paulo wrote:
>
>>> Now, here's where someone may be able to help?
>>> Grep'ng around, I found 4 places where the TCP stack called
>>> ip_output()
>>> (one in each of tcp_output.c, tcp_subr.c, tcp_timewait.c and
>>> tcp_syncache.c) and I put a printf like this just before them:
>>> if (flags & TH_RST)
>>> printf("sent a reset\n");
>>>
>>> (The exact format varies for each, because of where the TCP
>>> header flags are and have different printf messages.)
>>> Now, the weird part is, that when the extraneous RST is sent to the
>>> server, I don't get any printf. (I do get a few of these, but at
>>> other
>>> times for what appear to be legitimate RSTs.)
>>> I can't see anywhere else that the TCP stack would send an RST
>>> and, so,
>>> I'm stuck w.r.t. figuring out what is sending them?
> Ok, if you found the previous posts entertaining, you might enjoy
> this:-)
>
> Along with the printfs before all the ip_output() calls, I added calls
> inside ip_output() and, eventually, even calls in front of every
> if_output(). Never got anything that indicated an RST was being sent.
> (I only saw what I expected, which was an ACK reply being sent.)
>
> BUT, at almost exactly the same time, there were the FreeBSD8-CURRENT
> client->server RST packets on the server's snoop trace.
>
> Hmm, did a tcpdump in the client and, yes, the same packets were
> there.
>
> To keep it simple, I had done the dinosaur thing and plugged both the
> client and server into an old, dumb 10baseT hub, so that I could
> easily
> watch everything. (I also had an uplink cable to the net port in the
> wall, so I could move kernels around from the machine I usually build
> them on.)
>
> I was at the point where I couldn't conceivably figure out where the
> FreeBSD-CURRENT client was generating these RSTs. So guess what...
> --> it wasn't
>
> I unplugged the uplink cable and, no more RSTs. I've been testing for
> long enough now, that I am 99% certain they were being injected. Since
> the from address and even the MAC address is correct, I can only
> assume
> that it was the Cisco switch that was doing the injecting. (How else
> could a packet come in from the Cisco switch with the MAC address of
> the FreeBSD-CURRENT client machine?)
>
> It was usually triggered by a server reboot. After the server reboot,
> the server does send an RST to the client. This seems legit, but might
> be what makes Cisco think that "bad things" are happening? (I have no
> access to info about the switches or their configuration, although the
> campus standard is for these ports to be used by a single desktop
> machine
> only and not a switch or hub.)
>
> So, it seems that FreeBSD8-CURRENT reconnects fine now, so long as
> nothing is injecting RSTs into the newly created connection.
>
> Well, I'm not sure I found this fun, but hopefully others are
> entertained:-), rick
>
We had some issues at work where the Cisco intrusion detection thing
was resetting connections bogusly. Do you have IDS enabled on your
Cisco gear?
home |
help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3D80A486-FB8F-451D-BF44-F62C647D8556>