From owner-freebsd-net@freebsd.org  Tue Mar 14 08:40:47 2017
Return-Path: <owner-freebsd-net@freebsd.org>
Delivered-To: freebsd-net@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A639D0A250
 for <freebsd-net@mailman.ysv.freebsd.org>;
 Tue, 14 Mar 2017 08:40:47 +0000 (UTC) (envelope-from mike@karels.net)
Received: from mail.karels.net (mail.karels.net [63.231.190.5])
 by mx1.freebsd.org (Postfix) with ESMTP id 6CB111A17;
 Tue, 14 Mar 2017 08:40:45 +0000 (UTC) (envelope-from mike@karels.net)
Received: from mail.karels.net (localhost [127.0.0.1])
 by mail.karels.net (8.15.2/8.15.2) with ESMTP id v2E8ecH2040827;
 Tue, 14 Mar 2017 03:40:38 -0500 (CDT) (envelope-from mike@karels.net)
Message-Id: <201703140840.v2E8ecH2040827@mail.karels.net>
To: "Andrey V. Elsukov" <bu7cher@yandex.ru>
cc: freebsd-net@FreeBSD.org, Eugene Grosbein <eugen@freebsd.org>,
 "Alexander V. Chernikov" <melifaro@freebsd.org>, karels@FreeBSD.org
From: Mike Karels <mike@karels.net>
Reply-to: mike@karels.net
Subject: Re: LLE reference leak in the L2 cache
In-reply-to: Your message of Tue, 14 Mar 2017 09:47:26 +0300.
 <18d77ab0-f818-d711-196b-69f10877ae80@yandex.ru>
Date: Tue, 14 Mar 2017 03:40:38 -0500
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 14 Mar 2017 08:40:47 -0000

> Hi All,

> Eugene has reported about the following assertion in the ARP code:
> 	http://www.grosbein.net/freebsd/crash/arp-kassert.txt

> After some investigation I found that L2 cache has reference leak, that
> can lead to integer overflow and this assertion.
> The one of the ways to reproduce this overflow can be demonstrated with
> simple IP forwarding, when ip_forward() is used (not ip_tryforward).

> I asked olivier@ to reproduce this leak and he got this result:
> 	http://slexy.org/view/s21ql7nA0q

> After further investigation I found similar leak in the IPv6 TCP path.
> Simple iperf test shows these results:

> # dtrace -n 'fbt::in6_lltable_dump_entry:entry {printf("%d",
> args[1]->lle_refcnt);}'
> dtrace: description 'fbt::in6_lltable_dump_entry:entry ' matched 1 probe
> CPU     ID                    FUNCTION:NAME
>  51  18589     in6_lltable_dump_entry:entry 55721
>  51  18589     in6_lltable_dump_entry:entry 1
>  51  18589     in6_lltable_dump_entry:entry 1
>  51  18589     in6_lltable_dump_entry:entry 2
>  38  18589     in6_lltable_dump_entry:entry 111417
>  38  18589     in6_lltable_dump_entry:entry 1
>  38  18589     in6_lltable_dump_entry:entry 1

> --
> WBR, Andrey V. Elsukov

Thanks!  Could you try the following patch (compiles, but untested):

Index: netinet/ip_input.c
===================================================================
--- netinet/ip_input.c	(revision 315160)
+++ netinet/ip_input.c	(working copy)
@@ -60,6 +60,7 @@
 #include <net/if_types.h>
 #include <net/if_var.h>
 #include <net/if_dl.h>
+#include <net/if_llatbl.h>
 #include <net/route.h>
 #include <net/netisr.h>
 #include <net/rss_config.h>
@@ -1066,6 +1067,8 @@
 	if (error == EMSGSIZE && ro.ro_rt)
 		mtu = ro.ro_rt->rt_mtu;
 	RO_RTFREE(&ro);
+	if (ro.ro_lle)
+		LLE_FREE(ro.ro_lle);
 
 	if (error)
 		IPSTAT_INC(ips_cantforward);
Index: netinet6/ip6_forward.c
===================================================================
--- netinet6/ip6_forward.c	(revision 315160)
+++ netinet6/ip6_forward.c	(working copy)
@@ -52,6 +52,7 @@
 #include <net/if.h>
 #include <net/if_var.h>
 #include <net/netisr.h>
+#include <net/if_llatbl.h>
 #include <net/route.h>
 #include <net/pfil.h>
 
@@ -431,4 +432,6 @@
 out:
 	if (rt != NULL)
 		RTFREE(rt);
+	if (rin6.ro_lle)
+		LLE_FREE(rin6.ro_lle);
 }

Thanks,
		Mike