From owner-freebsd-net@FreeBSD.ORG Tue Mar 27 08:37:41 2007 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6FDC216A400 for ; Tue, 27 Mar 2007 08:37:41 +0000 (UTC) (envelope-from stefan.lambrev@sun-fish.com) Received: from blah.sun-fish.com (blah.sun-fish.com [217.18.249.150]) by mx1.freebsd.org (Postfix) with ESMTP id AD8CC13C48A for ; Tue, 27 Mar 2007 08:37:40 +0000 (UTC) (envelope-from stefan.lambrev@sun-fish.com) Received: from blah.sun-fish.com (localhost [127.0.0.1]) by blah.sun-fish.com (Postfix) with ESMTP id 6082B1B10F09; Tue, 27 Mar 2007 10:37:39 +0200 (CEST) Received: from [192.168.3.125] (hater.cmotd.com [192.168.3.125]) by blah.sun-fish.com (Postfix) with ESMTP id 5ADA81B10EA4; Tue, 27 Mar 2007 10:37:39 +0200 (CEST) Message-ID: <4608D7D1.4070304@sun-fish.com> Date: Tue, 27 Mar 2007 11:37:37 +0300 From: Stefan Lambrev User-Agent: Thunderbird 1.5.0.10 (X11/20070326) MIME-Version: 1.0 To: Ross Draper References: <3DDDCC38D00FA545A6C012475EF2DC0302AF85DF@LQEVS1.gcapmedia.com> In-Reply-To: <3DDDCC38D00FA545A6C012475EF2DC0302AF85DF@LQEVS1.gcapmedia.com> Content-Type: multipart/mixed; boundary="------------010107000605050406040108" X-Virus-Scanned: ClamAV using ClamSMTP on BLAH Cc: freebsd-net@freebsd.org Subject: Re: Vrrp/CARP/ucarp Problems X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: stefan.lambrev@sun-fish.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Mar 2007 08:37:41 -0000 This is a multi-part message in MIME format. --------------010107000605050406040108 Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: quoted-printable HI all, Ross Draper wrote: > Hi All > =20 > I was wondering if I could get some advice from those of you who have > successfully implemented ip address failover systems such as carp and > freevrrpd. > =20 > I am trying to set up a high availability web loadbalancer using a pair= > of freebsd 6.2 boxes. I have tried a number of ways to perform failover= > but always seem to be hitting a problem. > =20 > UCARP > Pro's:This would be my ideal solution as the startup/shutdown scripts > enable me to stop and start my applications and add aliases to adaptors= > easily. > Cons: When the backup box is rebooted it always comes up advertising > itself as the master then after a few seconds reverts to backup, > although I was under the impression it was supposed to wait and listen > for advertisements(it doesnt seem to). The backup boxes initial > gratuitous arp as a master is sufficient to poison any traffic from the= > local router to the shared ip address. Only solution was to use arp-sk > to send gratuitous arps every few secs, however, arp-sk was a bit flake= y > and it was a bodge. > =20 > CARP > Pro's: stable and built into the kernel. Could enable acive/active arp > load sharing at a later point. > Cons: There is a Freebsd bug (I've seen it discussed on the lists) wher= e > the creation and destroyal of a carp interface causes a kernel panic. > Also, there is no support for start/stop scripts. > =20 I do not have experience with ucarp and freevrrpd, so I can talk only=20 about CARP :) The bug you are talking is fixed in -CURRENT, and you can trigger it=20 only if you have more then 1 carp interface per host. I fetch changes from -current and made patch for -stable, that seems to=20 work without problems. There are other bugs, and I'm not sure what is their status, but you=20 always can search for PR. I do not think start/stop scripts are problem as average sysadmin can=20 solve this for itself :) > =20 > Freevrrpd > Pros: Mac address changing removes some of the arp timeout > issues/gratuitous arp problems and it supports start/stop scripts > Cons: I'm finding that upon rebooting the backup unit it correctly > starts as a backup, then three seconds later syslogs that it is the > master and changes its mac address accordingly. although a sniff of the= > network traffic indicates it is sending the right advertisements(lower > priority), it never goes into backup mode again. > =20 > So, what am I doing wrong? Are these common problems, or something that= > appears specific to my hosts/switches? are there more suitable options?= > The loadbalancers are all single homed and I have tried a mixture of xl= , > bge and fxp cards. =20 > =20 > Any help/suggestions much appreciated, also, any links to a perl based > gratuitous arp util would be great! > =20 > Many thanks > > Ross=20 > > PS - Apologies if you see multiple copies of this message, I seem to be= > having trouble getting mails onto the list. > > > > All correspondence, attachments and agreements remain strictly subject = to fully executed contract. (c) GCap Media plc 2006. All rights remain re= served. This e-mail (and any attachments) contains information which may = be confidential, subject to intellectual property protection and may be l= egally privileged and protected from disclosure and unauthorised use. It = is intended solely for the use of the individual(s) or entity to whom it = is addressed and others specifically authorised to receive it. If you are= not the intended recipient of this e-mail or any parts of it please tele= phone 020 7054 8000 immediately upon receipt. No other person is authoris= ed to copy, adapt, forward, disclose, distribute or retain this e-mail in= any form without prior specific permission in writing from an authorised= representative of GCap Media plc. We will not accept liability for any c= laims arising as a result of the use of the internet to transmit informat= ion by or to GCap Media plc. > > GCap Media plc. Registered address: 30 Leicester Square, London WC2H 7L= A. Registered in England & Wales with No. 923454 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > =20 P.S. the attached patch is little old so I'm not sure it still apply=20 cleanly to the latest -stable :) I tested base functionality with patched carp, but still do not have=20 server in production with it, so be careful! --=20 Best Wishes, Stefan Lambrev ICQ# 24134177 --------------010107000605050406040108 Content-Type: text/plain; name="carp.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="carp.patch" --- src/sys/netinet/ip_carp.c.orig Thu Feb 1 18:53:55 2007 +++ src/sys/netinet/ip_carp.c Tue Feb 6 18:41:24 2007 @@ -191,7 +191,7 @@ static void carp_input_c(struct mbuf *, struct carp_header *, sa_family_t); static int carp_clone_create(struct if_clone *, int); static void carp_clone_destroy(struct ifnet *); -static void carpdetach(struct carp_softc *); +static void carpdetach(struct carp_softc *, int); static int carp_prepare_ad(struct mbuf *, struct carp_softc *, struct carp_header *); static void carp_send_ad_all(void); @@ -406,9 +406,7 @@ if (sc->sc_carpdev) CARP_SCLOCK(sc); - carpdetach(sc); - if (sc->sc_carpdev) - CARP_SCUNLOCK(sc); + carpdetach(sc, 1); /* Returns unlocked. */ mtx_lock(&carp_mtx); LIST_REMOVE(sc, sc_next); @@ -420,7 +418,7 @@ } static void -carpdetach(struct carp_softc *sc) +carpdetach(struct carp_softc *sc, int unlock) { struct carp_if *cif; @@ -450,9 +448,10 @@ sc->sc_carpdev->if_carp = NULL; CARP_LOCK_DESTROY(cif); FREE(cif, M_IFADDR); - } + } else if (unlock) + CARP_UNLOCK(cif); + sc->sc_carpdev = NULL; } - sc->sc_carpdev = NULL; } /* Detach an interface from the carp. */ @@ -471,7 +470,7 @@ CARP_LOCK(cif); for (sc = TAILQ_FIRST(&cif->vhif_vrs); sc; sc = nextsc) { nextsc = TAILQ_NEXT(sc, sc_list); - carpdetach(sc); + carpdetach(sc, 0); } } --------------010107000605050406040108--