From owner-freebsd-net@freebsd.org Tue Sep 8 13:23:25 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D2A733C9019 for ; Tue, 8 Sep 2020 13:23:25 +0000 (UTC) (envelope-from SRS0=b0HP=CR=perdition.city=julien@bebif.be) Received: from orval.bbpf.belspo.be (orval.bbpf.belspo.be [193.191.208.90]) by mx1.freebsd.org (Postfix) with ESMTP id 4Bm5SY19Hhz3VqB for ; Tue, 8 Sep 2020 13:23:24 +0000 (UTC) (envelope-from SRS0=b0HP=CR=perdition.city=julien@bebif.be) Received: from x1 (unknown [77.109.102.38]) by orval.bbpf.belspo.be (Postfix) with ESMTPSA id AE89D1D4FC26; Tue, 8 Sep 2020 15:23:17 +0200 (CEST) Date: Tue, 8 Sep 2020 15:23:14 +0200 From: Julien Cigar To: Michael Gmelin , freebsd-net@freebsd.org Subject: Re: CARP over VLAN over LAGG Message-ID: <20200908132314.2txabgcuz4wmsq7n@x1> Mail-Followup-To: Michael Gmelin , freebsd-net@freebsd.org References: <20200831083705.pvrjk4srdohlxklf@x1> <8A98D287-4202-493B-8515-4377740B126A@grem.de> <20200901081323.amsc55h5xnikbycu@x1> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20200901081323.amsc55h5xnikbycu@x1> X-Rspamd-Queue-Id: 4Bm5SY19Hhz3VqB X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of SRS0=b0HP=CR=perdition.city=julien@bebif.be designates 193.191.208.90 as permitted sender) smtp.mailfrom=SRS0=b0HP=CR=perdition.city=julien@bebif.be X-Spamd-Result: default: False [-1.86 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.96)[-0.961]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-0.96)[-0.958]; MIME_GOOD(-0.10)[text/plain]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_NA(0.00)[perdition.city]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.54)[-0.538]; RCPT_COUNT_TWO(0.00)[2]; FORGED_SENDER(0.30)[julien@perdition.city,SRS0=b0HP=CR=perdition.city=julien@bebif.be]; RCVD_NO_TLS_LAST(0.10)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:2611, ipnet:193.191.192.0/19, country:BE]; FROM_NEQ_ENVFROM(0.00)[julien@perdition.city,SRS0=b0HP=CR=perdition.city=julien@bebif.be]; MAILMAN_DEST(0.00)[freebsd-net] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Sep 2020 13:23:25 -0000 On Tue, Sep 01, 2020 at 10:13:23AM +0200, Julien Cigar wrote: > On Mon, Aug 31, 2020 at 01:55:52PM +0200, Michael Gmelin wrote: > > > > > > > On 31. Aug 2020, at 10:37, Julien Cigar wrote: > > > > > > On Fri, Aug 28, 2020 at 04:52:01PM +0200, Julien Cigar wrote: > > >> Hello, > > >> > > >> I have a "highly available" router/firewall with the following > > >> configuration (1). Those are plugged in two 2930F (with VSF) using LACP. > > >> It works well, except that I have some weird issues with the CARP > > >> demotion counter when I'm unplugging some interfaces involved in the > > >> lagg/carp setup, for example if I unplug/replug igb0 and igb1 in this > > >> case: > > >> > > >> (dmesg): > > >> igb0: link state changed to DOWN > > >> igb1: link state changed to DOWN > > >> carp: demoted by 240 to 240 (send error 50 on vlan11) > > >> carp: 11@vlan11: MASTER -> BACKUP (more frequent advertisement received) > > >> vlan11: deletion failed: 3 > > >> igb1: link state changed to UP > > >> igb0: link state changed to UP > > >> > > >> then the CARP status stays to BACKUP unless I demote the CARP demotion > > >> counter manually with: sudo sysctl net.inet.carp.demotion=-240: > > >> > > >> (dmesg): > > >> carp: demoted by -240 to 0 (sysctl) > > >> carp: 11@vlan11: BACKUP -> MASTER (preempting a slower master) > > >> > > >> I guess this is because it takes some time for lagg/lacp to converge and > > >> thus carp thinks that there is a problematic condition as it experiences > > >> problems with sending announcements.. > > >> > > >> What it the best way to handle this? > > > > > > I'm wondering if setting net.inet.carp.senderr_demotion_factor to "0" > > > could be a solution? Are there any downsides of setting this to "0" > > > instead of "240"? > > > > > > > Sharing your pf.conf from both hosts could be helpful analyzing the issue. > > Here is my pf.conf (it's the same on both host): > https://gist.github.com/silenius/b758851f03c28ef8caaa53cfe381c455 > > However, I don't think pf is the issue here, the problem is that there > is a slight delay when LAGG/LACP converge and thus CARP increase the > demotion counter by net.inet.carp.senderr_demotion_factor (240). I can confirm that after setting net.inet.carp.senderr_demotion_factor=0 (instead of 240) it works as expected. > > > > > -m > > > > > > -- > Julien Cigar > Belgian Biodiversity Platform (http://www.biodiversity.be) > PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 > No trees were killed in the creation of this message. > However, many electrons were terribly inconvenienced. > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" -- Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced.