Date: Thu, 01 Nov 2012 18:59:04 -0500 From: "Douglas K. Rand" <rand@meridian-enviro.com> To: freebsd-net@freebsd.org Cc: Ryan Langseth <langseth@meridian-enviro.com> Subject: Trouble with TCP/UDP picking source addresses Message-ID: <50930CC8.4050406@meridian-enviro.com>
next in thread | raw e-mail | index | archive | help
We have an 8.3 system that picks the wrong, or at least inconvenient,
source IP address for UDP and TCP packets. This *only* happens when
sending packets to itself, never when sending packets to other hosts.
And not when sending packets to 127.0.0.1.
I *think* the problem might be related to this system being a CARP
backup system.
The local system has as its "internal" IP address 10.100.2.11. This is
on a VLAN interface called internal0. It also has a carp0 interface in
backup state with 10.100.2.1 as an address.
When I send TCP or UDP packets from the system back to itself using
10.100.2.11, tcpdump (watching lo0) shows the source address as 10.100.2.1:
18:47:44.742063 IP 10.100.2.1.45061 > 10.100.2.11.53: 31845+ A?
puppet.r2.ivr.meridian-enviro.com. (51)
And of course I see named trying to reply to this request on the
internal0 interface, but the packet gets routed out to the other CARP
host that is currently master:
18:47:44.742245 IP 10.100.2.11.53 > 10.100.2.1.45061: 31845* 2/3/3 CNAME
front0-vpn.r2.ivr.meridian-enviro.com., A 10.100.2.10 (201)
I can "fix" this by destroying and re-creating the carp0 interface.
What I'm thinking is that somehow the kernel is latching onto the IP
address of carp0 instead of internal0. Perhaps because carp0 is created
before internal0 during boot?
If I destroy and re-create carp0, then internal0 is earlier in the list
of interfaces. At least how I seem them with ifconfig.
I've verified that both UDP and TCP do the same thing. But ICMP doesn't
for some reason, it picks the "right" address of 10.100.2.11 as the
source address when I use ping.
We have another 8.3 based firewall with a similar, but not quite
identical, configuration. But it doesn't exhibit the problem, it picks a
source from the VLAN interface, not the CARP interface.
I was wondering if anybody has any ideas.
Here is the output from ifconfig:
bce0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> \
metric 0 mtu 1500
options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,\
JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
ether 84:8f:69:e3:a1:51
inet 65.101.96.19 netmask 0xfffffff8 broadcast 65.101.96.23
media: Ethernet autoselect (1000baseT <full-duplex,\
flowcontrol,rxpause,txpause>)
status: active
bce1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> \
metric 0 mtu 1500
options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,\
JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
ether 84:8f:69:e3:a1:53
media: Ethernet autoselect (1000baseT <full-duplex,\
flowcontrol,rxpause,txpause>)
status: active
bce2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,\
JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
ether 84:8f:69:e3:a1:55
media: Ethernet autoselect
bce3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> \
metric 0 mtu 1500
options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,\
JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
ether 84:8f:69:e3:a1:57
inet 10.254.3.11 netmask 0xffffff00 broadcast 10.254.3.255
media: Ethernet autoselect (1000baseT <full-duplex,\
flowcontrol,master,rxpause,txpause>)
status: active
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
lo0: flags=8149<UP,LOOPBACK,RUNNING,PROMISC,MULTICAST> \
metric 0 mtu 16384
options=3<RXCSUM,TXCSUM>
inet6 fe80::1%lo0 prefixlen 64 scopeid 0xc
inet6 ::1 prefixlen 128
inet 127.0.0.1 netmask 0xff000000
nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
pfsync0: flags=41<UP,RUNNING> metric 0 mtu 1460
pfsync: syncdev: bce3 syncpeer: 10.254.3.10 maxupd: 128
pflog0: flags=141<UP,RUNNING,PROMISC> metric 0 mtu 33152
carp0: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
inet 10.100.2.1 netmask 0xffffff00
carp: BACKUP vhid 12 advbase 1 advskew 150
internal0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> \
metric 0 mtu 1500
options=103<RXCSUM,TXCSUM,TSO4>
ether 84:8f:69:e3:a1:53
inet 10.100.2.11 netmask 0xffffff00 broadcast 10.100.2.255
media: Ethernet autoselect (1000baseT <full-duplex,\
flowcontrol,rxpause,txpause>)
status: active
vlan: 1 parent interface: bce1
management0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> \
metric 0 mtu 1500
options=103<RXCSUM,TXCSUM,TSO4>
ether 84:8f:69:e3:a1:53
inet 10.253.0.11 netmask 0xffffff00 broadcast 10.253.0.255
media: Ethernet autoselect (1000baseT <full-duplex,\
flowcontrol,rxpause,txpause>)
status: active
vlan: 410 parent interface: bce1
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?50930CC8.4050406>