From owner-freebsd-stable@FreeBSD.ORG Thu Sep 11 17:05:15 2014 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CDDD0760 for ; Thu, 11 Sep 2014 17:05:15 +0000 (UTC) Received: from mail-oi0-x233.google.com (mail-oi0-x233.google.com [IPv6:2607:f8b0:4003:c06::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9378F8EA for ; Thu, 11 Sep 2014 17:05:15 +0000 (UTC) Received: by mail-oi0-f51.google.com with SMTP id e131so5756142oig.10 for ; Thu, 11 Sep 2014 10:05:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=bWwvRzKHgTrtROA9l+D2AlKNsmMO9SRsaDHqlzaOhIM=; b=z7hgUyYMr3LT/Zn8GnULvIQRz3ZURzL0Vctad/HMCNkz56t3WCPDI+0UX9ygqtkEYO bJ8t32cw3Hhp3gE5Q7kAZ7IqhMfsb3vrCRD6Qe+W3QGZA0Eh9hUG/lJUtkpfpc44g9dJ pNdD7xqrE0E5XltvxOxZ9tA/yoG1ag0ZxLpyDRgDHY+5QzJnvpudKqyAnO9FJRoMrx9O SA4SHIaU1X3TVMItsVvSVkm48PrJ448aUIJBqemLG9VO0FZdJ/q0cTu+eaQXd6SmS4fr bDpYsiKDczOJ+WUYLnIe+lYRo0M4JeBXJr5AqkNPZmDKsBsar6UqUuSw0SVU4mVw2Tzq oTjg== MIME-Version: 1.0 X-Received: by 10.60.130.170 with SMTP id of10mr2802762oeb.10.1410455114393; Thu, 11 Sep 2014 10:05:14 -0700 (PDT) Received: by 10.202.199.11 with HTTP; Thu, 11 Sep 2014 10:05:14 -0700 (PDT) In-Reply-To: <7925563B043E419996CD7FEE8C7DFDB6@multiplay.co.uk> References: <7925563B043E419996CD7FEE8C7DFDB6@multiplay.co.uk> Date: Thu, 11 Sep 2014 10:05:14 -0700 Message-ID: Subject: Re: Using CARP with multiple IP aliases (FBSD 10.0) From: Freddie Cash To: Steven Hartland Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: FreeBSD Stable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Sep 2014 17:05:16 -0000 On Thu, Sep 11, 2014 at 9:34 AM, Steven Hartland wrote: > I can't say I've used it in that way and I'm not sure how carp decides ho= w > to fail over when it has multiple IP's available. > =E2=80=8BI'm hoping, and my testing appears to corroborate, that it fails b= ased on the interface state, and all IPs transfer over at once (CARP systctl set to fail everything at once if any one interface state changes).=E2=80=8B > I can confirm you don't need all the params when adding an IP to vhid. > so you can for example configure the vhid and then add the IP's or do > as you have done and configure it on the first IP. > =E2=80=8BThat's good to hear. Will simplify things a bit.=E2=80=8B > Best thing to do is try it and see. > =E2=80=8BThat's scheduled for tomorrow morning. :) I'll try it first with= only setting pass/advskew on the vhid once, and just adding the alias IPs to the vhid. If that doesn't fix things, then I'll try with a separate vhid per IP. The reason I was asking about this is that I have a pair of systems in place now (sys1 and sys2, with sys1 configured with advskew 1 to make it always master) where everything works wonderfully for between 5 and 15 minutes. If I down an interface on sys1, or physically remove a cable from sys1, everything fails over to sys2 and traffic continues normally.=E2=80=8B =E2=80=8B If I bring the interface back up on sys1, then everything fails = back over to sys1 and traffic continues. After 5-15 minutes, though, igb0 on both boxes switches to master state. :( igb1, igb2, and igb3 on sys2 all stay in backup state. And then traffic slows to a crawl as the upstream switch gets confused and sends packets=E2=80=8B randomly between the two hosts. Manually changing state to backup on igb0 on sys2 fixes things for about 3 seconds, and then it switches back to master. Once this happens, tcpdump on both systems only shows VRRPv2 packets from sys1, nothing from sys2.=E2=80=8B I have to reboot sys2 in order to get th= ings working again. As I said, this is the first time I've used CARP with multiple shared IPs on an interface (NAT firewall), so I may be doing things "wrong" or non-optimally. :) --=20 Freddie Cash fjwcash@gmail.com