From owner-freebsd-net@FreeBSD.ORG Mon Apr 1 20:58:47 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id CACC0925 for ; Mon, 1 Apr 2013 20:58:47 +0000 (UTC) (envelope-from ambrisko@ambrisko.com) Received: from mail.ambrisko.com (mail.ambrisko.com [70.91.206.90]) by mx1.freebsd.org (Postfix) with ESMTP id AC2A5DA for ; Mon, 1 Apr 2013 20:58:47 +0000 (UTC) X-Ambrisko-Me: Yes Received: from server2.ambrisko.com (HELO internal.ambrisko.com) ([192.168.1.2]) by ironport.ambrisko.com with ESMTP; 01 Apr 2013 14:01:17 -0700 Received: from ambrisko.com (localhost [127.0.0.1]) by internal.ambrisko.com (8.14.4/8.14.4) with ESMTP id r31KweuD049887; Mon, 1 Apr 2013 13:58:40 -0700 (PDT) (envelope-from ambrisko@ambrisko.com) Received: (from ambrisko@localhost) by ambrisko.com (8.14.4/8.14.4/Submit) id r31KwdkI049885; Mon, 1 Apr 2013 13:58:39 -0700 (PDT) (envelope-from ambrisko) Date: Mon, 1 Apr 2013 13:58:39 -0700 From: Doug Ambrisko To: Richard Tector Subject: Re: Route next-hop interface behaviour Message-ID: <20130401205839.GA24595@ambrisko.com> References: <51588F43.7060609@tector.org.uk> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="3uo+9/B/ebqu+fSQ" Content-Disposition: inline In-Reply-To: <51588F43.7060609@tector.org.uk> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2013 20:58:47 -0000 --3uo+9/B/ebqu+fSQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Sun, Mar 31, 2013 at 08:32:19PM +0100, Richard Tector wrote: | Hi, | | I'm not sure if it is expected behaviour but when configuring a static | route (default or otherwise) the outbound interface recorded in the | table does not update when the system's IP changes interface, even when | removing and re-adding it. | | Fairly simple topology, system running 9.1-STABLE (30th March) with just | one of four 'em' interfaces in use - em0 with 10.0.2.199. Default route | to 10.0.2.1. | | Whilst troubleshooting some odd TCP behaviour I thought I'd try a | different interface and so downed the active interface and brought up | one on a PCI card, and swapped the cable over: | | # ifconfig em0 down | # ifconfig em2 inet 10.0.2.199/24 | # ifconfig em2 up | | ##### | If I then ping an external host it shows as the destination network | being inaccessible: | | root@daffy:~ # ping 212.23.6.76 | PING 212.23.6.76 (212.23.6.76): 56 data bytes | ping: sendto: Network is down | | | ##### | Can contact the next hop just fine: | | root@daffy:~ # ping 10.0.2.1 | PING 10.0.2.1 (10.0.2.1): 56 data bytes | 64 bytes from 10.0.2.1: icmp_seq=0 ttl=64 time=0.211 ms | | | ##### | Routing table shows that the default route is still bound to em0 even | though the next hop is on em2: | | root@daffy:~ # netstat -rfinet -n | Routing tables | | Internet: | Destination Gateway Flags Refs Use Netif Expire | default 10.0.2.1 UGS 0 3141 em0 | ^^^ | 10.0.2.0/24 link#4 U 1 255 em2 | 10.0.2.199 link#1 UHS 1 0 lo0 | 127.0.0.1 link#9 UH 0 0 lo0 | | | ##### | Removing the default route and re-adding also leaves it on the old | interface: | | root@daffy:~ # route delete default | delete net default | root@daffy:~ # route add default 10.0.2.1 | add net default: gateway 10.0.2.1 | root@daffy:~ # netstat -rfinet -n | Routing tables | | Internet: | Destination Gateway Flags Refs Use Netif Expire | default 10.0.2.1 UGS 0 0 em0 | 10.0.2.0/24 link#4 U 0 688 em2 | 10.0.2.199 link#1 UHS 1 0 lo0 | 127.0.0.1 link#9 UH 0 0 lo0 | | I can understand the initial problem, but surely when I re-add the route | it should do a lookup against the table for the next hop, ie. | 10.0.2.0/24, and use the associated Netif? | | What's interesting is that if I remove the IP configuration from em0 it | removes the default route above (even though the interface is downed). | Re-adding the route works. You can try this patch: Index: net/if.c =================================================================== --- net/if.c (revision 248707) +++ net/if.c (working copy) @@ -1532,6 +1532,8 @@ IFNET_RLOCK_NOSLEEP(); TAILQ_FOREACH(ifp, &V_ifnet, if_link) { + if ((ifp->if_flags & IFF_UP) == 0) + continue; IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family != addr->sa_family) @@ -1620,6 +1622,8 @@ IFNET_RLOCK_NOSLEEP(); TAILQ_FOREACH(ifp, &V_ifnet, if_link) { + if ((ifp->if_flags & IFF_UP) == 0) + continue; if ((ifp->if_flags & IFF_POINTOPOINT) == 0) continue; IF_ADDR_RLOCK(ifp); @@ -1672,6 +1676,8 @@ */ IFNET_RLOCK_NOSLEEP(); TAILQ_FOREACH(ifp, &V_ifnet, if_link) { + if ((ifp->if_flags & IFF_UP) == 0) + continue; IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { char *cp, *cp2, *cp3; Index: net/if_ethersubr.c =================================================================== --- net/if_ethersubr.c (revision 248707) +++ net/if_ethersubr.c (working copy) @@ -812,6 +871,11 @@ #if defined(NETATALK) struct llc *l; #endif + /* Discard packet if interface is not up */ + if ((ifp->if_flags & IFF_UP) == 0) { + m_freem(m); + return; + } KASSERT(ifp != NULL, ("%s: NULL interface pointer", __func__)); The issue is that the routing doesn't look to see if the NIC is up or not. It just looks at the IP address. So it tries to send it out the first matching NIC that could be down. Doug A. --3uo+9/B/ebqu+fSQ Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="up.patch" Index: net/if.c =================================================================== --- net/if.c (revision 248707) +++ net/if.c (working copy) @@ -1532,6 +1532,8 @@ IFNET_RLOCK_NOSLEEP(); TAILQ_FOREACH(ifp, &V_ifnet, if_link) { + if ((ifp->if_flags & IFF_UP) == 0) + continue; IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family != addr->sa_family) @@ -1620,6 +1622,8 @@ IFNET_RLOCK_NOSLEEP(); TAILQ_FOREACH(ifp, &V_ifnet, if_link) { + if ((ifp->if_flags & IFF_UP) == 0) + continue; if ((ifp->if_flags & IFF_POINTOPOINT) == 0) continue; IF_ADDR_RLOCK(ifp); @@ -1672,6 +1676,8 @@ */ IFNET_RLOCK_NOSLEEP(); TAILQ_FOREACH(ifp, &V_ifnet, if_link) { + if ((ifp->if_flags & IFF_UP) == 0) + continue; IF_ADDR_RLOCK(ifp); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { char *cp, *cp2, *cp3; Index: net/if_ethersubr.c =================================================================== --- net/if_ethersubr.c (revision 248707) +++ net/if_ethersubr.c (working copy) @@ -812,6 +871,11 @@ #if defined(NETATALK) struct llc *l; #endif + /* Discard packet if interface is not up */ + if ((ifp->if_flags & IFF_UP) == 0) { + m_freem(m); + return; + } KASSERT(ifp != NULL, ("%s: NULL interface pointer", __func__)); --3uo+9/B/ebqu+fSQ--