From owner-freebsd-hackers@freebsd.org Mon Jan 14 21:53:47 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 18FEC148919B for ; Mon, 14 Jan 2019 21:53:47 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-ed1-x542.google.com (mail-ed1-x542.google.com [IPv6:2a00:1450:4864:20::542]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F35EB77E07 for ; Mon, 14 Jan 2019 21:53:45 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: by mail-ed1-x542.google.com with SMTP id f9so892572eds.10 for ; Mon, 14 Jan 2019 13:53:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=longcount-org.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=8/eEi3ZtVPjhVJnS4mECDRCsTPJqkkGXySGWRGa69Qw=; b=Smt+YZ+i/vE7h9xgdedWpsI6kMirrSoOmPJbwd5ZK/wLIGGV34JkJf13rCVBZ+eg69 DKP5ig+nNRBH927wDR9r90lTEJRqUw9sSHowyp4jguQC0ieUiAIwV4B6UJBZbvp38Qhd qdMOGFaVd7dajDLqIfxlt6lG4PcaDToueztOqvdZtJqxw81e+5jlllBziS05aeCvfSXc 2F+crG5KxyRFByAbiQpxxKzsc5KZN0WVG0TSGXZ2anjw0Dxo/NbUFiR/6e0dFCaGq/ym Bd2Re4pe1Ca3dQMMx6T14IH/ft08bQ6huSPFC6VLtt5RxfdnxQ4/CEKIssj3ZR10iilR VxzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=8/eEi3ZtVPjhVJnS4mECDRCsTPJqkkGXySGWRGa69Qw=; b=qz8+ohKvXQianqEAUNposvyJSAqA2uxIhXQSNwushxeXPPLIKScrFFflRfZYS661NU Dv4HDyYM+UCzd6PS5Pp8VWVYSPCy0WZ6nPlK9LHskPJxj+0d4mZwJAK2jXD4Db6weoqQ +C1HAlly2bA4Z1e70oT9fTI14yQECewiDMlaYOl/4jRTMd8NYCxDVsMKZ6u+nuuIS653 bjglahbmdjtuJwNkoyuu6pcCfUNYuq6+wlD39hcK0/q3tUi9d8rRk/i6SrwHwJS4YzDB wQ0Maj8pd+MzrwJvTIio/FymCxdEm4K1bzAwZPt7TYdgtJLCLC/fTjHtS6NWfTd30LZQ DRgg== X-Gm-Message-State: AJcUukeAu4mT0aoQh4cRrJsyMh6kHghfVN7o53L1ceB59WyrAtEOGDlE +6fh2gjrLmZNzZtpFMt0+LkL40S44YnSQoBo6+Y0iEgMcGM= X-Google-Smtp-Source: ALg8bN4CF4U56lJw81hBCBu0psMeAQzdhfs35wxC4CyF8hnB3IvHFmM9OSDgBzoDH2aUZq/Z3xDpZIgFDkabbWtyEwo= X-Received: by 2002:a17:906:b799:: with SMTP id dt25-v6mr796680ejb.217.1547502824618; Mon, 14 Jan 2019 13:53:44 -0800 (PST) MIME-Version: 1.0 References: <201901141821.x0EILVem036356@pdx.rh.CN85.dnsmgr.net> In-Reply-To: From: Mark Saad Date: Mon, 14 Jan 2019 16:53:31 -0500 Message-ID: Subject: Re: Removing an alias can remove routes ? To: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: F35EB77E07 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=longcount-org.20150623.gappssmtp.com header.s=20150623 header.b=Smt+YZ+i X-Spamd-Result: default: False [-2.40 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.984,0]; R_DKIM_ALLOW(-0.20)[longcount-org.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.995,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[longcount.org]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[longcount-org.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[cached: alt1.aspmx.l.google.com]; RCVD_IN_DNSWL_NONE(0.00)[2.4.5.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.37)[-0.372,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; IP_SCORE(-0.73)[ip: (0.20), ipnet: 2a00:1450::/32(-2.02), asn: 15169(-1.77), country: US(-0.08)] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Jan 2019 21:53:47 -0000 On Mon, Jan 14, 2019 at 4:48 PM Mark Saad wrote: > > On Mon, Jan 14, 2019 at 1:21 PM Rodney W. Grimes > wrote: > > > > > On Mon, Jan 14, 2019 at 11:59 AM Rodney W. Grimes > > > wrote: > > > > > > > > > All > > > > > I ran into a peculiar over the weekend on two FreeBSD 10-STABLE > > > > > servers that has me at a loss. Both servers have the same setup lacp > > > > > lagg wtih vlans using the lagg as a parent dev. > > > > > On the vlans I have some alias along with the primary ip. When a > > > > > alias was added or deleted routes that sourced out of the primary ip > > > > > on that vlan were dropped from the routing table. > > > > > > > > > > > > > > > So for example. I have lagg0.vlan1044 . (both servers are similarly > > > > > configured as routers / firewalls ) > > > > > > > > > > [root@pineapple01 ~]# ifconfig lagg0.vlan1044 > > > > > lagg0.vlan1044: flags=8843 > > > > > metric 0 mtu 1500 > > > > > options=300 > > > > > ether 00:0f:53:20:9b:d0 > > > > > inet6 fe80::20f:53ff:fe20:9bd0%lagg0.vlan1044 prefixlen 64 > > > > > scopeid 0x80 > > > > > inet 10.24.213.84 netmask 0xffffffe0 broadcast 10.24.213.95 > > > > > inet 10.24.212.129 netmask 0xffffffff broadcast 10.24.212.129 > > > > > inet 10.24.213.163 netmask 0xfffffff0 broadcast 10.24.213.175 > > > > > inet 10.24.213.76 netmask 0xffffffe0 broadcast 10.24.213.95 > > > > > inet 10.24.213.94 netmask 0xffffffe0 broadcast 10.24.213.95 > > > > > inet 10.24.213.89 netmask 0xffffffe0 broadcast 10.24.213.95 > > > > > inet 10.24.213.75 netmask 0xffffffe0 broadcast 10.24.213.95 > > > > > nd6 options=21 > > > > > media: Ethernet autoselect > > > > > status: active > > > > > vlan: 1044 parent interface: lagg0 > > > > > > > > > > [root@pineapple01 ~]# netstat -nr4Wl > > > > > ... > > > > > 192.168.144.32/27 10.24.213.65 UGS 0 1500 lagg0.vlan1044 > > > > > 192.168.144.96/27 10.24.213.65 UGS 0 1500 lagg0.vlan1044 > > > > > 192.168.23.0/24 10.24.213.65 UGS 0 1500 lagg0.vlan1044 > > > > > 192.168.120.0/21 10.24.213.65 UGS 0 1500 lagg0.vlan1044 > > > > > > > > > > > > > > > So I wanted to remove the alias ended in 163 and fix its netmask back to /32 > > > > > > > > > > I ran this > > > > > > > > > > ifconfig lagg0.vlan1044 inet 10.24.213.163/24 -alias && ifconfig > > > > ^ delete? > > > > > > I use -alias which is an alias of -delete > > > > > > > > > > > > lagg0.vlan1044 inet 10.24.213.163/32 -alias > > > > This is also a delete? > > I am concerend that what you think the command you > > typed is not actually the command you typed and what > > you did actually type has the bad side effects. > > > > > > > > > and shortly there after all of the routes that went out lagg0.vlan1044 > > > > > were gone . I quickly undid my change and put the routes back but I am > > > > > not sure what caused this ? Anyone have any ideas I have done this in > > > > > the past with out issue and I am unsure whats changed other then the > > > > > box have a long up time of 463 days . > > > > > > > > I believe what happened here is that 10.24.213.163/24 when reduced > > > > to a network address is 10.24.213.0/24, which probably got sent to > > > > the route removal code, which since the route to that covers the > > > > gateway at 10.24.213.65 that gateway was no longer accessable > > > > so all routes via it got removed. > > > > > > > > > > ok I see what you are saying and it makes sense to me; do you know why > > > the routing code does not see the primary ip ( the non-alias one) is > > > still > > > live on the interface before deciding to drop the routes associated with it ? > > > > It should, but I can not see enough of your routing table to > > guess as to exactly what the kernel did when you removed > > this interface. The routes I see above all would go through > > the interface you showed you removed, thus they would all > > go away. > > > > -- > > Rod Grimes rgrimes@freebsd.org > > > Ok some further testing shows what appears to be unintended results. > > On my test box I do the following > > ifconfig vlan98 create > ifconfig vlan98 vlan 98 vlandev lagg0 > ifconfig vlan98 inet 10.1.68.12/26 > ifconfig vlan98 inet 10.1.68.13/26 alias > ifconfig vlan98 inet 10.1.68.14/28 alias > ifconfig vlan98 inet 10.1.68.15/32 alias > > route add 10.24.213.0/24 10.1.68.11 > route add 10.24.214.0/24 10.1.68.11 > route add 10.24.215.0/24 10.1.68.11 > > root@potato2:~ # netstat -nr4Wl > Routing tables > > Internet: > Destination Gateway Flags Use Mtu Netif Expire > default 10.21.160.1 UGS 1216 1500 igb0 > 10.1.68.0/26 link#12 U 0 1500 vlan98 > 10.1.68.12 link#12 UHS 0 16384 lo0 > 10.1.68.13 link#12 UHS 0 16384 lo0 > 10.1.68.13/32 link#12 U 0 1500 vlan98 > 10.1.68.14 link#12 UHS 0 16384 lo0 > 10.1.68.14/32 link#12 U 0 1500 vlan98 > 10.1.68.15 link#12 UHS 0 16384 lo0 > 10.1.68.15/32 link#12 U 0 1500 vlan98 > 10.21.160.0/21 link#3 U 26985 1500 igb0 > 10.21.160.85 link#3 UHS 0 16384 lo0 > 10.24.213.0/24 10.1.68.11 UGS 0 1500 vlan98 > 10.24.214.0/24 10.1.68.11 UGS 0 1500 vlan98 > 10.24.215.0/24 10.1.68.11 UGS 0 1500 vlan98 > 127.0.0.1 link#5 UH 840 16384 lo0 > > Then I realize that my masks are wrong and I change the aliases back to "/32's" > > root@potato2:~ # ifconfig vlan98 > vlan98: flags=8843 metric 0 mtu 1500 > ether 00:0f:53:20:9d:00 > inet 10.1.68.12 netmask 0xffffffc0 broadcast 10.1.68.63 > inet6 fe80::20f:53ff:fe20:9d00%vlan98 prefixlen 64 scopeid 0xc > inet 10.1.68.13 netmask 0xffffffff broadcast 10.1.68.13 > inet 10.1.68.14 netmask 0xffffffff broadcast 10.1.68.14 > inet 10.1.68.15 netmask 0xffffffff broadcast 10.1.68.15 > nd6 options=21 > media: Ethernet autoselect > status: active > vlan: 98 parent interface: lagg0 > root@potato2:~ # netstat -nr4Wl > Routing tables > > Internet: > Destination Gateway Flags Use Mtu Netif Expire > default 10.21.160.1 UGS 1217 1500 igb0 > 10.1.68.0/26 link#12 U 0 1500 vlan98 > 10.1.68.12 link#12 UHS 0 16384 lo0 > 10.1.68.13 link#12 UHS 0 16384 lo0 > 10.1.68.13/32 link#12 U 0 1500 vlan98 > 10.1.68.14 link#12 UHS 0 16384 lo0 > 10.1.68.14/32 link#12 U 0 1500 vlan98 > 10.1.68.15 link#12 UHS 0 16384 lo0 > 10.1.68.15/32 link#12 U 0 1500 vlan98 > 10.21.160.0/21 link#3 U 27166 1500 igb0 > 10.21.160.85 link#3 UHS 0 16384 lo0 > 10.24.213.0/24 10.1.68.11 UGS 0 1500 vlan98 > 10.24.214.0/24 10.1.68.11 UGS 0 1500 vlan98 > 10.24.215.0/24 10.1.68.11 UGS 0 1500 vlan98 > 127.0.0.1 link#5 UH 840 16384 lo0 > root@potato2:~ # > > So far no problems. > > Then I see ohh my primary ip needs to be a "/24" > > > root@potato2~ # ifconfig vlan98 inet 10.1.68.12/24 > root@potato2:~ # netstat -nr4Wl > Routing tables > > Internet: > Destination Gateway Flags Use Mtu Netif Expire > default 10.21.160.1 UGS 1218 1500 igb0 > 10.1.68.0/24 link#12 U 0 1500 vlan98 > 10.1.68.12 link#12 UHS 0 16384 lo0 > 10.1.68.13 link#12 UHS 0 16384 lo0 > 10.1.68.13/32 link#12 U 0 1500 vlan98 > 10.1.68.14 link#12 UHS 0 16384 lo0 > 10.1.68.14/32 link#12 U 0 1500 vlan98 > 10.1.68.15 link#12 UHS 0 16384 lo0 > 10.1.68.15/32 link#12 U 0 1500 vlan98 > 10.21.160.0/21 link#3 U 27230 1500 igb0 > 10.21.160.85 link#3 UHS 0 16384 lo0 > 10.24.213.0/24 10.1.68.11 UGS 0 1500 vlan98 > 10.24.214.0/24 10.1.68.11 UGS 0 1500 vlan98 > 10.24.215.0/24 10.1.68.11 UGS 0 1500 vlan98 > 127.0.0.1 link#5 UH 840 16384 lo0 > root@potato2:~ # > > > So far so good; then I accidentally hit up arrow and enter / or re-run > the promotion to /24 again two times > > IE: > root@potato2:~ # ifconfig vlan98 inet 10.1.68.12/24 > root@potato2:~ # ifconfig vlan98 inet 10.1.68.12/24 > > root@potato2:~ # netstat -nr4Wl > Routing tables > > Internet: > Destination Gateway Flags Use Mtu Netif Expire > default 10.21.160.1 UGS 1223 1500 igb0 > 10.1.68.0/24 link#12 U 0 1500 vlan98 > 10.1.68.12 link#12 UHS 0 16384 lo0 > 10.1.68.15 link#12 UHS 0 16384 lo0 > 10.1.68.15/32 link#12 U 0 1500 vlan98 > 10.21.160.0/21 link#3 U 27847 1500 igb0 > 10.21.160.85 link#3 UHS 0 16384 lo0 > 127.0.0.1 link#5 UH 868 16384 lo0 > > > > What just happened to my routes and check out what just happened to my aliases > > root@potato2:~ # ifconfig vlan98 > vlan98: flags=8843 metric 0 mtu 1500 > ether 00:0f:53:20:9d:00 > inet6 fe80::20f:53ff:fe20:9d00%vlan98 prefixlen 64 scopeid 0xc > inet 10.1.68.15 netmask 0xffffffff broadcast 10.1.68.15 > inet 10.1.68.12 netmask 0xffffff00 broadcast 10.1.68.255 > nd6 options=21 > media: Ethernet autoselect > status: active > vlan: 98 parent interface: lagg0 > > > > Anyone have an idea what happened here ? > > -- > mark saad | nonesuch@longcount.org To be clear this is on 10-STABLE from 2017 however on 12-STABLE from Dec 2018 this is still acting odd but in a slightly diferent way. When I change the aliases from /28 and /26 back to /32 when I run root@ostrich:~ # ifconfig vlan98 inet 10.1.68.13/32 alias no issues root@ostrich:~ # ifconfig vlan98 inet 10.1.68.14/32 alias poof my routes are removed. Again anyone have any idea whats going on here ? -- mark saad | nonesuch@longcount.org