From owner-freebsd-net@freebsd.org Wed Jan 17 22:06:36 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0EB59EBB06E for ; Wed, 17 Jan 2018 22:06:36 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lf0-x22d.google.com (mail-lf0-x22d.google.com [IPv6:2a00:1450:4010:c07::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7788DF54 for ; Wed, 17 Jan 2018 22:06:35 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lf0-x22d.google.com with SMTP id a204so15654973lfa.2 for ; Wed, 17 Jan 2018 14:06:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=8g3fN0+moZeGW5zoVzxJ2d2Aeozxf7JfDNI55Zoql8k=; b=G6Q4PWp+ktUktOyseahb5TaXVLvRJOcadxZwEnvIf7WzV9AJiRMKOmLqQvohvGD4G7 ujsIDWEL8qjB/17SKneCuq+S9F/g34jyRLt9zhm0+z6Un4VZZfnaLiapGlP3XXqw3aEV e0IcAfqbVfMMtciF3vmPh2DitVqd70P2a6kjknXbbrjjT4waOug0T4+2TUxaqdCZGkgW r2nco0GL0W8yUI9WnLoRrepRntOcxr1Ak8liYk1R+blnSR5WjQLwGOoagGfjV9lCwbCA khm3tP4TNpVKoM4JpuvnP+yT882DE3ZrEtZwY2PsAfGjUZUly6TvlKP6csJdNW1VEIeg o5Yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=8g3fN0+moZeGW5zoVzxJ2d2Aeozxf7JfDNI55Zoql8k=; b=klh6Emqls8T3+SNUzvgU8RAJxa6cow1Yizw8Ngxv/DvXIC/mm/boRYGNKPEOp+MxDr nVPQAUuT5kEMx8IG/LBo71vTnlgLFQGcyk8vKTe8hINsfKbtzXdup0LQw/N4gh4QRyIC 9tP/okQTuM9xe/kWAq6ZRjdWaCm9pIo0lSuukOiB5ozIKQSDEhHT/FnCisrbL2XYmTu+ Dal7oHVqSqcWH7QUANsTtu3k6M3mAg/NU3SO83es5UhFhV23QMEwohwA+XjtFBWUqcRk wcIh3QqPxnzjN+SeryQe6Q8ZKwe3HnNfrhlR+kN0q2Ya6C3fbJxxZUauyj6zeLdvOJV3 B04w== X-Gm-Message-State: AKwxytfpaq5A4P34aVeWW3yMUx2fIIbLCgnPwxgpTNqE6BZIC/zg8Omi RLv8fhe/XoEsl6bHkRsZLq2itAZHoISpha5A+bA= X-Google-Smtp-Source: ACJfBov085vwi//hj7QfFHD8d5kldvl5kOnivwrdazKD1F8XS9vuFiLo9CjFZphLeYVZuDnxkwAiR861J2GuT9fEnzk= X-Received: by 10.25.151.209 with SMTP id z200mr11270305lfd.41.1516226793283; Wed, 17 Jan 2018 14:06:33 -0800 (PST) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.179.64.22 with HTTP; Wed, 17 Jan 2018 14:06:32 -0800 (PST) In-Reply-To: References: From: Alan Somers Date: Wed, 17 Jan 2018 15:06:32 -0700 X-Google-Sender-Auth: UDJ1ynVDgRBwrhTPeUanxpXD3z0 Message-ID: Subject: Re: Allowing a local subnet route to change to a different ifnet To: Ryan Stone Cc: freebsd-net Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2018 22:06:36 -0000 On Wed, Jan 17, 2018 at 2:56 PM, Ryan Stone wrote: > I'm going to prefix this question by noting that I realize that the > configuration that I am about to describe is quite nonsensical. > Unfortunately, it seems that under older versions of FreeBSD (possibly > FreeBSD 7-vintage), the configuration mostly "worked", and now at > $WORK I am responsible for making the configuration continue to work > in the future. The customer in this case is completely unwilling to > modify their configuration. > > I have a customer that has configured two different IPs on the same > subnet on two different interfaces. The behaviour that they want is > that if the link on one of the two interfaces goes down, the route to > that subnet will migrate to the other IP on the other interface as a > quasi-failover behaviour. Under FreeBSD 7, we had a daemon that > accomplished this by detecting the link loss and then using "route > change" to move the route to the up interface. If the subnet in > question was 192.168.1.0/24, for example, we could run "route change > 192.1.68.1.0/24 -ifp em1" to migrate the route. > > Running on -head I run into two issues. The first comes out of > r264986, which changes the behaviour of RTM_CHANGE. The code path > changed significantly, but the part that impacts me is that now any > RTM_CHANGE command with the gateway set NULL gets EINVAL immediately > where previously it was allowed. I've hacked around this problem > locally for testing purposes but I really don't understand the code > well enough at this point to see what a real fix would look like. > > The second issue is quite complex. The root cause is that the routing > code sets IFA_ROUTE on any ifaddr that has a local subnet route > associated with it. If I don't migrate this flag to the new ifaddr, > very bizarre behaviour follows. I've done a prototype that detects > when this migration is needed in rtrequest1_fib_change() and it works, > but IFA_ROUTE seems to otherwise be handled in individual L3 protocols > like in and in6, so I'm worried that this is a layering violation. My > patch looks like this: > https://people.freebsd.org/~rstone/patches/route-change-subnet.diff > > > My first, and most important question, is whether a patch that would > allow a subnet route to be migrated to a different interface be > something that would be acceptable in FreeBSD? If so, I need guidance > on what a proper fix for both issues would look like so that I can > implement fixes that I can upstream. > > Thanks, > Ryan > I'm sorry for you Ryan; this sounds like a doozy. I know you said that the customer is unwilling to change, but would they consider using a lagg(4) interface? Using lagg with laggproto=failover is designed to solve exactly this problem. They wouldn't have to recable anything, and they could keep their single IP address. If not, you should see PR 189088; I think it's related. -Alan