Date: Wed, 25 Jun 2008 23:03:43 +0200 From: Giulio Ferro <auryn@zirakzigil.org> To: freebsd-net@freebsd.org Subject: SOLVED (was Re: Problem clarification (was: Problems with vlan + carp + alias)) Message-ID: <4862B2AF.70202@zirakzigil.org> In-Reply-To: <486000B5.9090703@zirakzigil.org> References: <486000B5.9090703@zirakzigil.org>
next in thread | previous in thread | raw e-mail | index | archive | help
I finally got the problem, and it had nothing to do either with vlans or with carp. The firewall I was setting up was meant to replace an existing freebsd firewall which didn't use vlans (it had a lot of nics). The problem was that the network port where our ISP brings the internet connection still had the old aliased mac addresses in its arp cache. For some reason when I plugged in the new firewall, only the base non-aliased address was updated in the ISP switch arp cache (if someone can throw a guess at why, I'm eager to listen). The ISP router was still looking for the aliased addresses with the old macs, so it didn't find them. Moreover, I inadvertently put the vlan internet interface in promiscuous mode, so with tcpdump I also picked up those packets with wrong mac address which weren't meant for me. To make the story short, I called the technical customer care of the ISP and I requested them to reset the arp cache of the port. Done that, everything worked without a glitch. The new firewall is now up and running in production with vlan + carp. Everything seems fine. Thanks to everybody who answered my plea... :-) Giulio Ferro wrote: > After some more tests I've finally realized that the problem is with > vlan and alias. I've taken carp out of the picture. > > > (Please read my previous message on the topic to understand the scenario, > I've reported it below) > > Here is what matters in /etc/rc.conf: > > ----------------------------------------------------------- > ... > ifconfig_bce0="inet 192.168.26.1 netmask 255.255.255.0" > ... > ifconfig_vlan128="inet x.y.z.132 netmask 255.255.255.224 vlan 128 > vlandev bce0" > ifconfig_vlan128_alias0="x.y.z.133 netmask 255.255.255.255" > ifconfig_vlan128_alias1="x.y.z.134 netmask 255.255.255.255" > ifconfig_vlan128_alias2="x.y.z.135 netmask 255.255.255.255" > ifconfig_vlan128_alias3="x.y.z.136 netmask 255.255.255.255" > ifconfig_vlan128_alias4="x.y.z.137 netmask 255.255.255.255" > ifconfig_vlan128_alias5="x.y.z.138 netmask 255.255.255.255" > ifconfig_vlan128_alias6="x.y.z.139 netmask 255.255.255.255" > ifconfig_vlan128_alias7="x.y.z.140 netmask 255.255.255.255" > ifconfig_vlan128_alias8="x.y.z.141 netmask 255.255.255.255" > ... > defaultrouter="x.y.z.129" > ----------------------------------------------------------- > > netstat -rn > ----------------------------------------------------------- > default x.y.z.129 UGS 0 9869 vlan12 > x.y.z.128/27 link#11 UC 0 0 vlan12 > x.y.z.129 00:00:0c:07:ac:0a UHLW 2 52 vlan12 1107 > x.y.z.130 00:d0:03:8a:9b:fc UHLW 1 0 vlan12 1147 > x.y.z.131 00:d0:03:8a:9b:fd UHLW 1 0 vlan12 1144 > x.y.z.133/32 link#11 UC 0 0 vlan12 > x.y.z.134/32 link#11 UC 0 0 vlan12 > x.y.z.135/32 link#11 UC 0 0 vlan12 > x.y.z.136/32 link#11 UC 0 0 vlan12 > x.y.z.137/32 link#11 UC 0 0 vlan12 > x.y.z.138/32 link#11 UC 0 0 vlan12 > x.y.z.139/32 link#11 UC 0 0 vlan12 > x.y.z.140/32 link#11 UC 0 0 vlan12 > x.y.z.141/32 link#11 UC 0 0 vlan12 > ----------------------------------------------------------- > > ifconfig vlan128 > ----------------------------------------------------------- > vlan128: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 > mtu 1500 > options=3<RXCSUM,TXCSUM> > ether 00:1e:c9:ad:fa:c9 > inet x.y.z.132 netmask 0xffffffe0 broadcast x.y.z.159 > inet x.y.z.133 netmask 0xffffffff broadcast x.y.z.133 > inet x.y.z.134 netmask 0xffffffff broadcast x.y.z.134 > inet x.y.z.135 netmask 0xffffffff broadcast x.y.z.135 > inet x.y.z.136 netmask 0xffffffff broadcast x.y.z.136 > inet x.y.z.137 netmask 0xffffffff broadcast x.y.z.137 > inet x.y.z.138 netmask 0xffffffff broadcast x.y.z.138 > inet x.y.z.139 netmask 0xffffffff broadcast x.y.z.139 > inet x.y.z.140 netmask 0xffffffff broadcast x.y.z.140 > inet x.y.z.141 netmask 0xffffffff broadcast x.y.z.141 > media: Ethernet autoselect (1000baseTX <full-duplex>) > status: active > vlan: 128 parent interface: bce0 > ----------------------------------------------------------- > > Tests: > No problem when I try to ping the default gateway from my fw > No problem when I ping my fw from an external internet address > > Problems: > - I cannot ping the router from one of the aliased address: > ping -S x.y.z.133 x.y.z.129 > - I cannot ping the aliased addresses from an external internet address > > Note : I can see the packets with tcpdump travelling from and to the > aliased > address. It seems the interface won't process them for some reason. > > This seems suspiciously like a bug to me... > > > -------------------------------------------------------------------------------------- > > (previous message on vlan + carp +alias) > -------------------------------------------------------------------------------------- > > > > Primeroz lists wrote: >> What is tcpdump showing for ping on 192.168.10.11 >> <http://192.168.10.11>; ? can you see echo reply exiting vlan10 >> interface ? >> >> what if you try from your server to "ping -S 192.168.10.11 >> <http://192.168.10.11>; 192.168.10.254 <http://192.168.10.254>" ? >> >> >> > First of all I'm sorry for the late reply. Yesterday I could do some more > in-depth test to analyze this strange behavior of my firewall. > > The 192.168.10.0/24 range I used in the previous example isn't the real > one, I just used it for simplicity´s sake. > The true range, the one which has been assigned by the ISP to my customer > is: x.y.z.128/27. (x.y.z corresponds to a true public IP address) > > I've deactivated the firewall, so we have one less thing to worry about: > /etc/rc.d/pf stop > This is a pure network configuration issue. > > Here is the relevant part in /etc/rc.conf: > --------------------------------------------------- > ... > ifconfig_bce0="inet 192.168.26.1 netmask 255.255.255.0" > ... > cloned_interfaces="vlan5 vlan25 vlan30 vlan40 vlan128 carp5 carp25 > carp30 carp40 carp128" > ... > ifconfig_vlan128="inet x.y.z.157 netmask 255.255.255.224 vlan 128 > vlandev bce0" > ... > ifconfig_carp128="vhid 128 pass qweq x.y.z.132 netmask 255.255.255.255" > ifconfig_carp128_alias0="x.y.z.133 netmask 255.255.255.255" > ifconfig_carp128_alias1="x.y.z.134 netmask 255.255.255.255" > ifconfig_carp128_alias2="x.y.z.135 netmask 255.255.255.255" > ifconfig_carp128_alias3="x.y.z.136 netmask 255.255.255.255" > ifconfig_carp128_alias4="x.y.z.137 netmask 255.255.255.255" > ifconfig_carp128_alias5="x.y.z.138 netmask 255.255.255.255" > ifconfig_carp128_alias6="x.y.z.139 netmask 255.255.255.255" > ifconfig_carp128_alias7="x.y.z.140 netmask 255.255.255.255" > ifconfig_carp128_alias8="x.y.z.141 netmask 255.255.255.255" > ... > defaultrouter="x.y.z.129" > --------------------------------------------------- > > On my managed switch I've set 2 ports: > 1) the one where the bce0 interface is plugged in : mode trunk with > all the vlans above > 2) the one where the ISP internet is plugged in : mode access with > vlan 128 > > I've also set the ip interface of my switch to x.y.z.155 vlan 128 > > > Here is the relevant part of netstat -rn on my machine > --------------------------------------------------- > default x.y.z.129 UGS 0 13966 vlan12 > x.y.z/27 link#11 UC 0 0 vlan12 > x.y.z.132 x.y.z.132 UH 0 0 carp12 > x.y.z.133 x.y.z.133 UH 0 0 carp12 > x.y.z.134 x.y.z.134 UH 0 0 carp12 > x.y.z.135 x.y.z135 UH 0 0 carp12 > x.y.z.136 x.y.z.136 UH 0 0 carp12 > x.y.z.137 x.y.z.137 UH 0 0 carp12 > x.y.z.138 x.y.z.138 UH 0 0 carp12 > x.y.z.139 x.y.z.139 UH 0 0 carp12 > x.y.z.140 x.y.z.140 UH 0 0 carp12 > x.y.z.141 x.y.z.141 UH 0 0 carp12 > x.y.z.155 00:1e:c9:90:4a:c0 UHLW 1 8 vlan12 1183 > > --------------------------------------------------- > > > > Here come the tests. > 1) From the firewall : basic > I can ping both the default gateway (x.y.z.129) and the switch > interface (x.y.z.155) > I can ping a generic internet address (a.b.c.d) > With tcpdump I can see the packets leaving as x.y.z.157 and coming > with the same > address > > 2) from the switch : basic > I can ping the firewall's vlan address (x.y.z.157) > I can ping _ALL_ the carp interfaces, base and alias: > ping x.y.z.157 -> OK > ping x.y.z.132 -> OK > ping x.y.z.133 -> OK > ... > ping x.y.z.141 -> OK > > 3) from the internet : basic > From an external internet address I can ping the vlan address: > ping x.y.z.157 -> OK > > 4) from the firewall : advanced > From the firewall I can ping the switch address from one of the carp > base and aliased address: > ping -S x.y.z.132 x.y.z.155 -> OK > ping -S x.y.z.133 x.y.z.155 -> OK > > I _cannot_ ping the default router from one of the carp addresses: > ping -S x.y.z.132 x.y.z.129 -> NOT OK > ping -S x.y.z.133 x.y.z.129 -> NOT OK > By using tcpdump on the vlan128 interface I can see the packets > _BOTH_ leaving and coming from the carp addresses. It just seems > that the carp interfaces can't process the packets properly. > > 5) from the internet : advanced > From an external internet address I _cannot_ ping the carp addresses > (x.y.z.132 and up) > As above, I can see the incoming packets with > tcpdump -i vlan128 -n icmp > > > Ok, that was long. I hope someone can help to shed light into this, to > see > whether this is a bug or not. > I stress again that the _same_ configuration works as it should on a > physical > (non-vlan) interface. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4862B2AF.70202>