From owner-freebsd-questions@freebsd.org Wed Feb 19 19:35:03 2020 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 273F42432EA for ; Wed, 19 Feb 2020 19:35:03 +0000 (UTC) (envelope-from SRS0=Oc3R=4H=mail.sermon-archive.info=doug@sermon-archive.info) Received: from mail.sermon-archive.info (sermon-archive.info [71.177.216.148]) by mx1.freebsd.org (Postfix) with ESMTP id 48N7GX45mSz3DkC for ; Wed, 19 Feb 2020 19:34:59 +0000 (UTC) (envelope-from SRS0=Oc3R=4H=mail.sermon-archive.info=doug@sermon-archive.info) Received: from [10.0.1.251] (mini [10.0.1.251]) by mail.sermon-archive.info (Postfix) with ESMTPSA id 48N7GT1wHZz2fjVJ; Wed, 19 Feb 2020 11:34:57 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: Switching to backup Network From: Doug Hardie In-Reply-To: Date: Wed, 19 Feb 2020 11:34:56 -0800 Cc: FreeBSD Content-Transfer-Encoding: quoted-printable Message-Id: References: <64F39D12-E061-4726-B58E-943D61963944@mail.sermon-archive.info> <50d6c0e2-8e70-0743-1e9c-f4c36847a015@kicp.uchicago.edu> To: Valeri Galtsev X-Mailer: Apple Mail (2.3445.104.11) X-Virus-Scanned: clamav-milter 0.101.4 at mail X-Virus-Status: Clean X-Rspamd-Queue-Id: 48N7GX45mSz3DkC X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of SRS0=Oc3R=4H=mail.sermon-archive.info=doug@sermon-archive.info designates 71.177.216.148 as permitted sender) smtp.mailfrom=SRS0=Oc3R=4H=mail.sermon-archive.info=doug@sermon-archive.info X-Spamd-Result: default: False [-1.33 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.89)[-0.888,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:71.177.216.148]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(-0.05)[asn: 5650(-0.20), country: US(-0.05)]; NEURAL_HAM_LONG(-0.99)[-0.993,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[148.216.177.71.list.dnswl.org : 127.0.10.0]; FORGED_SENDER(0.30)[bc979@lafn.org,SRS0=Oc3R=4H=mail.sermon-archive.info=doug@sermon-archive.info]; RCVD_NO_TLS_LAST(0.10)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:5650, ipnet:71.177.216.0/23, country:US]; FROM_NEQ_ENVFROM(0.00)[bc979@lafn.org,SRS0=Oc3R=4H=mail.sermon-archive.info=doug@sermon-archive.info]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Feb 2020 19:35:03 -0000 > On 19 February 2020, at 06:32, Valeri Galtsev = wrote: >=20 >=20 >=20 >> On Feb 18, 2020, at 7:29 PM, Doug Hardie wrote: >>=20 >>> On 18 February 2020, at 12:25, Valeri Galtsev = wrote: >>>=20 >>>=20 >>>=20 >>> On 2020-02-18 14:19, Doug Hardie wrote: >>>> One of my clients has a machine running 12.1 that is connected via = two different NICs to two different WANs. He has drops from 2 different = ISPs to provide redundancy. I have configured each of the DNS names with = both IP addresses so that web access will switch over to the backup when = the primary is down. Setfib and pf are used to make that work. That = works fine (although there is a DNS timeout involved). The problem is = that all the servers on the machine talk out via the primary IP address. = While web access continues, the server initiated functions fail because = the next hop is down. Is there a way to switch everything over to the = backup network in this case? I don't find anything that enables = automatic changes to the default network. >>>> Also, when the backup network goes down, the default network entry = for setfib 1 route is deleted. I have to manually enter that when it = comes backup. I am initially setting that in /etc/rc.local. Is there a = way to make it either remain, or be restored? >>>>=20 >>>=20 >>> I would look into link aggregation (lagg): >>>=20 >>> = https://www.freebsd.org/doc/en/books/handbook/network-aggregation.html >>>=20 >>> I used that to make my FreeBSD laptop switch over from WiFi to = ethernet interface when the last link is available. Worked neat for me. >>>=20 >>> Valeri >>>=20 >>=20 >> Lagg looks neat, but my first setup didn't work. I suspect the issue = is the IP addresses. Each of the two networks have quite different IPs. = Both are fixed IP addresses but from different allocations. It appears = that lagg requires the use of one IP for both networks. All the = examples use just one IP address for both networks. >>=20 >=20 > I did not look into setting lagg for two ethernet interfaces, mine was = for ethernet + WiFi. But they were definitely on different networks. = Additional simplicity of my case was: both interfaces were DHCP clients = (that took care of routing auto-magically). There was one =E2=80=9Chack=E2= =80=9D I did in my case: I have a vague recollection I made both = adapters under lagg have the same MAC address (by changing one of them), = but that likely was due to my insufficient knowledge or laziness when I = was setting it up. Expert probably will do it without that hack. I still = have a feeling, one can do it for static IPs (my laptop configuration = does work for interfaces on different networks), that=E2=80=99s why I = suggested to look into that. >=20 > Hopefully, some lagg expert will offer you help. It may make sense to = ask on freebsd-net list. >=20 > Valeri >=20 I discovered there are three things you lose when you get older. The = first is your memory. I've forgotten the other two. Apparently I had started a development earlier and actually got a = solution working, but never included it in my client's system. I = encountered it, by accident, last night and included it. This only = applies to the code I developed for the client, but I replaced the = connect and tcp_connect calls with new modules that test the returns = from those and then use setfib to try the backup if the primary is not = available. This solves the big issue for my client as his faxes now are = being delivered using the backup network when the primary is down. It = doesn't solve the problem for things like email, but the client usually = uses his PCs for that. I just lose things like the periodic emails = (they aren't lost, but significantly delayed as postfix is being used). The primary benefit of this is my client's customers are not affected by = the down time. My client now reports that he thinks the problem is = caused by the cable from the NIC to the router being too short and = having too much stress. This is a remote location so I have no idea = what it looks like. However, it is interesting to note that my client = did the physical setup and is a Microsoft Certified Network Engineer ;-) -- Doug