From owner-freebsd-net@freebsd.org Tue Mar 14 08:52:06 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06C9ED0AAAF for ; Tue, 14 Mar 2017 08:52:06 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward4h.cmail.yandex.net (forward4h.cmail.yandex.net [IPv6:2a02:6b8:0:f35::111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Yandex CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 971431301; Tue, 14 Mar 2017 08:52:05 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from smtp1p.mail.yandex.net (smtp1p.mail.yandex.net [77.88.29.84]) by forward4h.cmail.yandex.net (Yandex) with ESMTP id 6568020A22; Tue, 14 Mar 2017 11:52:01 +0300 (MSK) Received: from smtp1p.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp1p.mail.yandex.net (Yandex) with ESMTP id F3E741780A04; Tue, 14 Mar 2017 11:51:58 +0300 (MSK) Received: by smtp1p.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id uC5ZMZEacu-pwiWBEOt; Tue, 14 Mar 2017 11:51:58 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1489481518; bh=iWO0sD+GmBRRHdx/t0L+YRkdaip2v1ZsyBYAp2LH3Jk=; h=Subject:To:References:Cc:From:Message-ID:Date:In-Reply-To; b=XHk1f0IN8ZlesoMvzXBXTihG3S9cty0WD3lCDUQOWQ4y3r4K6MstYZ08xtURF6efV XZfxowp9IrV5i2CT+txSKBz1RO95REnJMSk4kcNlB4+5qcmaWYf6ZePa9q/a+VWF2u Z5+dO54Kf9i+5we44ffqETJfnEfJz3xzV8FFl0H8= Authentication-Results: smtp1p.mail.yandex.net; dkim=pass header.i=@yandex.ru X-Yandex-Suid-Status: 1 0,1 0,1 0,1 0,1 0 Subject: Re: LLE reference leak in the L2 cache To: mike@karels.net References: <201703140840.v2E8ecH2040827@mail.karels.net> Cc: freebsd-net@FreeBSD.org, Eugene Grosbein , "Alexander V. Chernikov" , karels@FreeBSD.org From: "Andrey V. Elsukov" Openpgp: id=E6591E1B41DA1516F0C9BC0001C5EA0410C8A17A Message-ID: <3a4c5d87-d42e-5615-5d2b-2a8801376600@yandex.ru> Date: Tue, 14 Mar 2017 11:50:18 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 In-Reply-To: <201703140840.v2E8ecH2040827@mail.karels.net> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="TjpHPX55uB1j9qoa7heNJEpbS6baGEhOb" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Mar 2017 08:52:06 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --TjpHPX55uB1j9qoa7heNJEpbS6baGEhOb Content-Type: multipart/mixed; boundary="k4Ij0dbjUQOfN0rPMsXHB3gtsdh4mIeFw"; protected-headers="v1" From: "Andrey V. Elsukov" To: mike@karels.net Cc: freebsd-net@FreeBSD.org, Eugene Grosbein , "Alexander V. Chernikov" , karels@FreeBSD.org Message-ID: <3a4c5d87-d42e-5615-5d2b-2a8801376600@yandex.ru> Subject: Re: LLE reference leak in the L2 cache References: <201703140840.v2E8ecH2040827@mail.karels.net> In-Reply-To: <201703140840.v2E8ecH2040827@mail.karels.net> --k4Ij0dbjUQOfN0rPMsXHB3gtsdh4mIeFw Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 14.03.2017 11:40, Mike Karels wrote: >> Hi All, >=20 >> Eugene has reported about the following assertion in the ARP code: >> http://www.grosbein.net/freebsd/crash/arp-kassert.txt >=20 >> After some investigation I found that L2 cache has reference leak, tha= t >> can lead to integer overflow and this assertion. >> The one of the ways to reproduce this overflow can be demonstrated wit= h >> simple IP forwarding, when ip_forward() is used (not ip_tryforward). >=20 >> I asked olivier@ to reproduce this leak and he got this result: >> http://slexy.org/view/s21ql7nA0q >=20 >> After further investigation I found similar leak in the IPv6 TCP path.= >> Simple iperf test shows these results: >=20 >> # dtrace -n 'fbt::in6_lltable_dump_entry:entry {printf("%d", >> args[1]->lle_refcnt);}' >> dtrace: description 'fbt::in6_lltable_dump_entry:entry ' matched 1 pro= be >> CPU ID FUNCTION:NAME >> 51 18589 in6_lltable_dump_entry:entry 55721 >> 51 18589 in6_lltable_dump_entry:entry 1 >> 51 18589 in6_lltable_dump_entry:entry 1 >> 51 18589 in6_lltable_dump_entry:entry 2 >> 38 18589 in6_lltable_dump_entry:entry 111417 >> 38 18589 in6_lltable_dump_entry:entry 1 >> 38 18589 in6_lltable_dump_entry:entry 1 >=20 >> -- >> WBR, Andrey V. Elsukov >=20 > Thanks! Could you try the following patch (compiles, but untested): >=20 > Index: netinet/ip_input.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- netinet/ip_input.c (revision 315160) > +++ netinet/ip_input.c (working copy) > @@ -60,6 +60,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -1066,6 +1067,8 @@ > if (error =3D=3D EMSGSIZE && ro.ro_rt) > mtu =3D ro.ro_rt->rt_mtu; > RO_RTFREE(&ro); > + if (ro.ro_lle) > + LLE_FREE(ro.ro_lle); > =20 > if (error) > IPSTAT_INC(ips_cantforward); I think it would be better to set RT_LLE_CACHE flag only for protocols that expect presence of L2 cache. I.e. only for the TCP and UDP and do it in the corresponding protocol output routine, not in the ip[6]_output.= > Index: netinet6/ip6_forward.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- netinet6/ip6_forward.c (revision 315160) > +++ netinet6/ip6_forward.c (working copy) > @@ -52,6 +52,7 @@ > #include > #include > #include > +#include > #include > #include > =20 > @@ -431,4 +432,6 @@ > out: > if (rt !=3D NULL) > RTFREE(rt); > + if (rin6.ro_lle) > + LLE_FREE(rin6.ro_lle); > } I don't think this chunk will help. ip6_forward() doesn't use ip6_output(). And IPv6 forwarding is not affected by this problem. Look at the tcp_output(), it uses local route variable for IPv6 output. I'm not sure, but probably SCTP also can be affected by this problem. --=20 WBR, Andrey V. Elsukov --k4Ij0dbjUQOfN0rPMsXHB3gtsdh4mIeFw-- --TjpHPX55uB1j9qoa7heNJEpbS6baGEhOb Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEzBAEBCAAdFiEE5lkeG0HaFRbwybwAAcXqBBDIoXoFAljHrsoACgkQAcXqBBDI oXrQLQf/XbF6ju2Ztg2cWJw0IRhz0HknR5wIdxxMPcCfFmKo3in2BLMSGC0GTQFm BQj9upaVe5qXAqBj61TCbZYCqOvjlU8rHayB9q/W8RMeq7S7U0Gtkzl9snpVDGRE gw7KC42Pt3KeJzIJnNABiW4sZGx7q43+B2jpI1IAmZnXCuq9jsgd06zT7GXeijCl NUiwx7xevP9ytOjtHslN2oE4iFXZIdNyr4b4nPJ46bViKIvdF3hcdUccyiCHlPMW GJr01gEE023PuOQlkuRbOEF2O5uFIk59oFV/G8C2mf0PmDyoaBPLetL5s5X7wSeF xSFmwwd+QtaLn3sy7UF23LNTFyMeBg== =QTqL -----END PGP SIGNATURE----- --TjpHPX55uB1j9qoa7heNJEpbS6baGEhOb--