From owner-freebsd-net@freebsd.org Fri Feb 12 17:41:07 2021 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5576C52B044 for ; Fri, 12 Feb 2021 17:41:07 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DcglP3WN6z3Pjh; Fri, 12 Feb 2021 17:41:05 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-pg1-x532.google.com with SMTP id n10so96373pgl.10; Fri, 12 Feb 2021 09:41:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=3wS1dECgWTbYD/Dq2deZQdttpErnSbNwS4DRjZCQvnc=; b=A/4oe6XljJMlxjsbL6FCgy4GZnVnc+u+VBxUMOh545+7R8SOZsub1mOGr34HurcLbl iG0yin9Hjxvww1OjAhDKq6F7noeEJNJ+yBSVF3AEhswZ9KpJiz3EgcPfvhHe3SjgxAZU 9o8G8AvEcO5lpH3GpkDFiNhAIOdvSBd5dn5AIChchMH3dbuBDvXl7UFIbUyt3xsc5UG1 sIk4sS9I5GF9hrBKd0fJlk3GJbFBPdseaqJ34peAdgttjxLgiSW4Gdlnmq0vBXaSeeNQ DzbM/EpwudiqBZe75Py9oK6M4OHU+71mkpwKNCO5hIpHfABzUOGEJioXUEGdIjwBWdKb Olzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=3wS1dECgWTbYD/Dq2deZQdttpErnSbNwS4DRjZCQvnc=; b=msJ1/mXRie26UBXqpH1Hr6/x/uYBYPFS48bwsqQnbM9ZZYsofNiuXwM9vLZ2Ie+AlG i19cafjJpDH0bHkzRiiCbA5wWpSuau/XjiF1WrHvUXSiGxGloy+Tr+sTOQNcacmUQ8Vb QOJ9TDBUrTr/943RpZn512y17+xO5jcvDnRe+KuWtlEqPcveUNayvej4WUMk4NaZn2si RJ/l+r065xJSogRSIU3qUvqnVbuSatM6gnUA+M9WS8yl0Gwl7AmHdGutozuKLBTQ7opS fZlnM9LRlP9L7GBkHicQ+xVe3CBVebOhCS5HxvKR8ZNAqwp3iWBkA5Lo4uO4reNAijWC yZ0w== X-Gm-Message-State: AOAM533/ylPX3tUUZf6KWk5ANMrSQDAxTh/JaPE2wwrHTewgb5oyouD5 eotgzQ65mpbmc9vbvN3f/O6bSJ2SRbfql82+/83/Mp0YJ6Zf1A== X-Google-Smtp-Source: ABdhPJyilZrnonoHu58veJTkKwjMlvCqU1YZh7kQvfucpDcVAdKMDbm9HIHFVjr1X9fj6IPZGZ8WEXRPVVvDLdXnjkc= X-Received: by 2002:aa7:8f0a:0:b029:1de:4d20:8346 with SMTP id x10-20020aa78f0a0000b02901de4d208346mr4119613pfr.15.1613151663664; Fri, 12 Feb 2021 09:41:03 -0800 (PST) MIME-Version: 1.0 From: Ryan Stone Date: Fri, 12 Feb 2021 12:40:52 -0500 Message-ID: Subject: ifaddr reference count leaks that seem to be related to routing code To: freebsd-net Cc: "Alexander V. Chernikov" Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4DcglP3WN6z3Pjh X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=A/4oe6Xl; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rysto32@gmail.com designates 2607:f8b0:4864:20::532 as permitted sender) smtp.mailfrom=rysto32@gmail.com X-Spamd-Result: default: False [-3.98 / 15.00]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::532:from]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::532:from:127.0.2.255]; NEURAL_HAM_MEDIUM(-0.98)[-0.976]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::532:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-net]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-Mailman-Approved-At: Fri, 12 Feb 2021 19:23:00 +0000 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Feb 2021 17:41:07 -0000 On main I'm seeing leaks of ifaddr reference counts. I can reproduce this in a bhyve VM by repeating this: vlan=`ifconfig vlan create vlandev vtnet0 vlan 777` ifconfig $vlan up ifconfig $vlan 1.0.0.1/24 ifconfig $vlan destroy If I watch the ifaddr column of vmstat -m while I repeat this, I see it increasing by one every time. Poking around with dtrace confirms that it's an AF_INET address that's getting leaked. Looking at the calls to ifa_ref and ifa_free, the routing code looks suspicious. I see that the route add path takes one reference here: https://cgit.freebsd.org/src/tree/sys/net/route/route_ctl.c#n588 kernel`rib_add_route+0x11e kernel`rib_handle_ifaddr_info+0xd7 kernel`in_handle_ifaddr_route+0x1cd kernel`in_addprefix+0x18b kernel`in_control+0xada kernel`ifioctl+0x3d8 kernel`kern_ioctl+0x289 kernel`sys_ioctl+0x12a kernel`amd64_syscall+0x12e kernel`0xffffffff8108b9be (for me directly in the else case, or rt_getifa_fib() will take one on the ifa it grabs) I see a second reference taken in the nhop code: kernel`get_nhop+0xd8 kernel`nhop_create_from_info+0x1df kernel`rib_add_route+0x12e kernel`rib_handle_ifaddr_info+0xd7 kernel`in_handle_ifaddr_route+0x1cd kernel`in_addprefix+0x18b kernel`in_control+0xada kernel`ifioctl+0x3d8 kernel`kern_ioctl+0x289 kernel`sys_ioctl+0x12a kernel`amd64_syscall+0x12e kernel`0xffffffff8108b9be But I only see one reference released when the nhop is cleaned up: kernel`destroy_nhop_epoch+0x2d kernel`epoch_call_task+0x13a kernel`gtaskqueue_run_locked+0xa7 kernel`gtaskqueue_thread_loop+0x94 kernel`fork_exit+0x80 kernel`0xffffffff8108c11e It looks to me like we can either lose the ifa_ref() from create_rtentry entirely (if the route add code runs within the net_epoch), or we need to release that reference before returning from rib_add_route(). Can somebody take a look at this and confirm my findings? Thanks