From owner-freebsd-stable@FreeBSD.ORG Tue Jun 14 21:14:50 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DDDAF1065674 for ; Tue, 14 Jun 2011 21:14:49 +0000 (UTC) (envelope-from korvus@comcast.net) Received: from qmta03.westchester.pa.mail.comcast.net (qmta03.westchester.pa.mail.comcast.net [76.96.62.32]) by mx1.freebsd.org (Postfix) with ESMTP id 9F70E8FC1B for ; Tue, 14 Jun 2011 21:14:49 +0000 (UTC) Received: from omta24.westchester.pa.mail.comcast.net ([76.96.62.76]) by qmta03.westchester.pa.mail.comcast.net with comcast id vwcv1g0051ei1Bg53x1aS3; Tue, 14 Jun 2011 21:01:34 +0000 Received: from [192.168.2.164] ([206.210.89.202]) by omta24.westchester.pa.mail.comcast.net with comcast id vx1P1g00W4Mx3R23kx1R7j; Tue, 14 Jun 2011 21:01:32 +0000 Message-ID: <4DF7CC21.6040500@comcast.net> Date: Tue, 14 Jun 2011 17:01:21 -0400 From: Steve Polyack User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110516 Lightning/1.0b2 Thunderbird/3.1.10 MIME-Version: 1.0 To: Damien Fleuriot References: <4DF72488.6050806@my.gd> <4DF793B5.903@my.gd> <4DF79B72.2090805@comcast.net> <99A75196-BE3C-466C-9B0B-CF874C1287B5@my.gd> In-Reply-To: <99A75196-BE3C-466C-9B0B-CF874C1287B5@my.gd> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-stable@freebsd.org" Subject: Re: Networking - CARP interfaces X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Jun 2011 21:14:50 -0000 On 06/14/2011 04:51 PM, Damien Fleuriot wrote: > > On 14 Jun 2011, at 19:33, Steve Polyack wrote: > >> On 06/14/2011 01:00 PM, Damien Fleuriot wrote: >>> I can confirm that this scenario causes problems, see below: >>> >>> ### ON FIREWALL 1 , carp master for carp0, carp1, carp2 >>> carp2: flags=49 metric 0 mtu 1500 >>> inet 192.168.224.254 netmask 0xffffff00 >>> carp: MASTER vhid 224 advbase 1 advskew 50 >>> >>> >>> ### ON FIREWALL 2 , carp backup for carp0, carp1, carp2 >>> carp2: flags=49 metric 0 mtu 1500 >>> inet 192.168.234.254 netmask 0xffffff00 >>> carp: BACKUP vhid 234 advbase 1 advskew 100 >>> >>> >>> Now, I add a dummy IP to carp2 on FIREWALL 2, which is supposedly backup: >>> >>> ifconfig carp2 inet 192.168.234.207 alias >>> >>> Result: >>> >>> ### ON FIREWALL 1, carp master for carp0, carp1, carp2 >>> carp2: flags=49 metric 0 mtu 1500 >>> inet 192.168.224.254 netmask 0xffffff00 >>> carp: MASTER vhid 224 advbase 1 advskew 50 >>> >>> ### ON FIREWALL 2, carp backup for carp0, carp1, but no longer carp2 >>> carp2: flags=49 metric 0 mtu 1500 >>> inet 192.168.234.254 netmask 0xffffff00 >>> inet 192.168.234.207 netmask 0xffffff00 >>> carp: MASTER vhid 234 advbase 1 advskew 100 >>> >>> >>> After I remove the extraneous IP, the interface becomes backup again: >>> >>> >>> # This was a long time ago >>> carp0: MASTER -> BACKUP (more frequent advertisement received) >>> carp0: link state changed to DOWN >>> carp2: MASTER -> BACKUP (more frequent advertisement received) >>> carp2: link state changed to DOWN >>> carp1: MASTER -> BACKUP (more frequent advertisement received) >>> carp1: link state changed to DOWN >>> carp2: link state changed to DOWN >>> # This was when I ran my tests >>> carp2: INIT -> MASTER (preempting) >>> carp2: link state changed to UP >>> carp2: MASTER -> BACKUP (more frequent advertisement received) >>> carp2: link state changed to DOWN >> Did you give this enough time to reasonably settle? Sometimes when the interfaces initially come up, they will become MASTER for a bit before backing down. >> > I think I did but I can do try again tomorrow evening just to make sure. > > Oh god, if only dmesg entries were timestamped... > > >>> This entails that hosts in a given carp vhid must have the exact same IP >>> addresses configured on that interface. >>> >>> While this is perfectly understandable in a master-backup scenario, this >>> is a bit more annoying for us in a master-backup + backup-backup >>> scenario with 2 datacenters. >>> >>> I'll just have to adapt and ensure they have the same IP addresses then. >> I have a suspicion that the important part may be the number of IP addresses on the CARP interface. If CARP sends an advertisement from each IP alias on a CARP interface, then I think that would explain what you are seeing - and also possibly give you a workaround by adding two more bogus IPs on your primary datacenter firewalls (where IPs W and Z are normally missing). >> >> - Steve >> > I'll give it a try, although I think in a scenario where the carp interfaces have the same number of IPs and these IPs differ, both interfaces will claim mastership. > > Will post results. Now that I look at the spec, it looks like both the count and the addresses themselves are provided in VRRP packets. CARP likely does the same. I can't speak for whether these things are considered along with the VHID and password, but it's worth a shot. I think you are correct, though. http://www.networksorcery.com/enp/protocol/vrrp.htm - Steve