Date: Thu, 18 May 1995 19:26:48 -0400 (EDT) From: "Michael C. Newell" <mnewell@lupine.nsi.nasa.gov> To: Peter Wemm <peter@haywire.dialix.com> Cc: hackers@FreeBSD.org Subject: Re: Hmm. _really wierd_. Packets going out wrong interface... Message-ID: <Pine.SUN.3.91.950518191911.19992E-100000@lupine.nsi.nasa.gov> In-Reply-To: <Pine.SV4.3.91.950519061418.23362A-100000@haywire.DIALix.COM>
next in thread | previous in thread | raw e-mail | index | archive | help
Wait till the REALLY nasty one hits - when for some reason it decides
that your default route should be out some other interface...
We had ppp0: 198.116.2.4->198.116.75.1, where 2.4 was the local Ethernet and
75.1 was the remote tail site Ethernet. At one point routing decided that
2.4 wasn't ed0 anymore, it was ppp0. Don't know why or how, but the end
result was everything got routed through ppp0. Not a good thing, and the
only fix that worked was to reboot the local server.
Major bummer. :{0
To fix that we picked a class C subnet specifically for the local server
ppp interfaces; thus our two ppp links that used to be
Local Remotes
198.116.2.4 -> 198.116.75.1
198.116.2.4 -> 198.116.75.33
became
198.116.75.17 -> 198.116.75.65
198.116.75.19 -> 198.116.75.33
Things worked MUCH better then; the ONLY problem we had after that was
that routed and gated would simply not announce the 198.116.75 network
out to the 198.116.2.0 network. That's when we assigned 198.116.75.1 as
an alias to 198.116.2.4 and ran gated with a specific announce to
specified; that fixed the problem. But I still don't understand why it
has to be that way...
Thanks,
Mike
On Fri, 19 May 1995, Peter Wemm wrote:
> Date: Fri, 19 May 1995 06:54:49 +0800 (WST)
> From: Peter Wemm <peter@haywire.dialix.com>
> To: hackers@FreeBSD.org
> Subject: Hmm. _really wierd_. Packets going out wrong interface...
>
> Something really wierd is going on..
>
> When the system is choosing which interface a packet has to go out on,
> something is getting crossed over somewhere.
>
> Scenario:
> super small C-class subnet, 192.203.228.64 -> 192.203.228.79.
> .79 is broadcast address. .65 is haywire, .69 is jhome.
>
> Original config:
> haywire (SVR4, lachman tcp) ethernet at .65, netmask 0xfffffff0, and has
> a ppp link .65 -> .3, netmask also 0xfffffff0. This represents no
> problem, as all lookups that deal with IFF_POINTTOPOINT interfaces
> compare the destination address with the value being looked up. This
> prevents the need for large numbers of IP addresses for the server with
> lots of ppp/slip connections. it works reliably and deterministically.
>
> I tried that with a FreeBSD machine.. whooo.. that turned out to be
> interesting.
>
> I fired up the PPP link on jhome, .69 -> .3 with the same netmask,
> 0xfffffff0. I've been trying to figure out where the hell the broadcasts
> have gone.
>
> Talk about surprise.. The broadcasts destined for the local ethernet
> went out the PPP link! (so says tcpdump anyway..)
>
> It appears that even though the networking code goes to *great* lengths
> to make sure that destination address is the only one that counts on P2P
> links, there's still one or two left that are using (localaddr & netmask).
>
> So.. after killing and restarting gated, I then got:
> Data modified on freelist: word 0 of object 0xf0733c80 size 36 previous
> type in_multi (0xf066d440 != 0xdeadc0de)
>
> And.. I was just about to write that rwhod is broadcasting to the local
> ethernet to confuse the issue, I discover that it's currently going over
> the ppp link too... I was sure that it was going to ethernet before, but
> that might have been before I started messing with ifconfig.
>
> pinging the ethernet broadcast address yields a response from the
> ethernet hosts as expected (even though it's not got SO_BROADCAST set)..
> but things that DO have SO_BROADCAST seem to go out on the non-broadcast
> ppp link..
>
> Real strange...
>
> The thought occurred that it might be an idea to try using the remote
> subnet for both addresses.. (ie: .3 -> .8 and vice versa), but that wont
> work when (if) the other end becomes a freebsd machine, because the
> remote end wont be able to broadcast to it's ethernet..
>
> Puzzled..
> -Peter
>
Thanks,
Mike
+--------------------------------------+------------------------------------+
|Mike Newell | The opinions expressed herein are |
|NASA Science Internet Network Systems | my own, and do not necessarily |
|Sterling Software, Inc. | reflect those of the NSI program, |
|MNewell@nsipo.nasa.gov | Sterling Software, NASA, or anyone |
|+1-202-434-8954 | else. |
+--------------------------------------+------------------------------------+
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.SUN.3.91.950518191911.19992E-100000>