Date: Tue, 14 Mar 2017 11:50:18 +0300 From: "Andrey V. Elsukov" <bu7cher@yandex.ru> To: mike@karels.net Cc: freebsd-net@FreeBSD.org, Eugene Grosbein <eugen@freebsd.org>, "Alexander V. Chernikov" <melifaro@freebsd.org>, karels@FreeBSD.org Subject: Re: LLE reference leak in the L2 cache Message-ID: <3a4c5d87-d42e-5615-5d2b-2a8801376600@yandex.ru> In-Reply-To: <201703140840.v2E8ecH2040827@mail.karels.net> References: <201703140840.v2E8ecH2040827@mail.karels.net>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --TjpHPX55uB1j9qoa7heNJEpbS6baGEhOb Content-Type: multipart/mixed; boundary="k4Ij0dbjUQOfN0rPMsXHB3gtsdh4mIeFw"; protected-headers="v1" From: "Andrey V. Elsukov" <bu7cher@yandex.ru> To: mike@karels.net Cc: freebsd-net@FreeBSD.org, Eugene Grosbein <eugen@freebsd.org>, "Alexander V. Chernikov" <melifaro@freebsd.org>, karels@FreeBSD.org Message-ID: <3a4c5d87-d42e-5615-5d2b-2a8801376600@yandex.ru> Subject: Re: LLE reference leak in the L2 cache References: <201703140840.v2E8ecH2040827@mail.karels.net> In-Reply-To: <201703140840.v2E8ecH2040827@mail.karels.net> --k4Ij0dbjUQOfN0rPMsXHB3gtsdh4mIeFw Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 14.03.2017 11:40, Mike Karels wrote: >> Hi All, >=20 >> Eugene has reported about the following assertion in the ARP code: >> http://www.grosbein.net/freebsd/crash/arp-kassert.txt >=20 >> After some investigation I found that L2 cache has reference leak, tha= t >> can lead to integer overflow and this assertion. >> The one of the ways to reproduce this overflow can be demonstrated wit= h >> simple IP forwarding, when ip_forward() is used (not ip_tryforward). >=20 >> I asked olivier@ to reproduce this leak and he got this result: >> http://slexy.org/view/s21ql7nA0q >=20 >> After further investigation I found similar leak in the IPv6 TCP path.= >> Simple iperf test shows these results: >=20 >> # dtrace -n 'fbt::in6_lltable_dump_entry:entry {printf("%d", >> args[1]->lle_refcnt);}' >> dtrace: description 'fbt::in6_lltable_dump_entry:entry ' matched 1 pro= be >> CPU ID FUNCTION:NAME >> 51 18589 in6_lltable_dump_entry:entry 55721 >> 51 18589 in6_lltable_dump_entry:entry 1 >> 51 18589 in6_lltable_dump_entry:entry 1 >> 51 18589 in6_lltable_dump_entry:entry 2 >> 38 18589 in6_lltable_dump_entry:entry 111417 >> 38 18589 in6_lltable_dump_entry:entry 1 >> 38 18589 in6_lltable_dump_entry:entry 1 >=20 >> -- >> WBR, Andrey V. Elsukov >=20 > Thanks! Could you try the following patch (compiles, but untested): >=20 > Index: netinet/ip_input.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- netinet/ip_input.c (revision 315160) > +++ netinet/ip_input.c (working copy) > @@ -60,6 +60,7 @@ > #include <net/if_types.h> > #include <net/if_var.h> > #include <net/if_dl.h> > +#include <net/if_llatbl.h> > #include <net/route.h> > #include <net/netisr.h> > #include <net/rss_config.h> > @@ -1066,6 +1067,8 @@ > if (error =3D=3D EMSGSIZE && ro.ro_rt) > mtu =3D ro.ro_rt->rt_mtu; > RO_RTFREE(&ro); > + if (ro.ro_lle) > + LLE_FREE(ro.ro_lle); > =20 > if (error) > IPSTAT_INC(ips_cantforward); I think it would be better to set RT_LLE_CACHE flag only for protocols that expect presence of L2 cache. I.e. only for the TCP and UDP and do it in the corresponding protocol output routine, not in the ip[6]_output.= > Index: netinet6/ip6_forward.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- netinet6/ip6_forward.c (revision 315160) > +++ netinet6/ip6_forward.c (working copy) > @@ -52,6 +52,7 @@ > #include <net/if.h> > #include <net/if_var.h> > #include <net/netisr.h> > +#include <net/if_llatbl.h> > #include <net/route.h> > #include <net/pfil.h> > =20 > @@ -431,4 +432,6 @@ > out: > if (rt !=3D NULL) > RTFREE(rt); > + if (rin6.ro_lle) > + LLE_FREE(rin6.ro_lle); > } I don't think this chunk will help. ip6_forward() doesn't use ip6_output(). And IPv6 forwarding is not affected by this problem. Look at the tcp_output(), it uses local route variable for IPv6 output. I'm not sure, but probably SCTP also can be affected by this problem. --=20 WBR, Andrey V. Elsukov --k4Ij0dbjUQOfN0rPMsXHB3gtsdh4mIeFw-- --TjpHPX55uB1j9qoa7heNJEpbS6baGEhOb Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEzBAEBCAAdFiEE5lkeG0HaFRbwybwAAcXqBBDIoXoFAljHrsoACgkQAcXqBBDI oXrQLQf/XbF6ju2Ztg2cWJw0IRhz0HknR5wIdxxMPcCfFmKo3in2BLMSGC0GTQFm BQj9upaVe5qXAqBj61TCbZYCqOvjlU8rHayB9q/W8RMeq7S7U0Gtkzl9snpVDGRE gw7KC42Pt3KeJzIJnNABiW4sZGx7q43+B2jpI1IAmZnXCuq9jsgd06zT7GXeijCl NUiwx7xevP9ytOjtHslN2oE4iFXZIdNyr4b4nPJ46bViKIvdF3hcdUccyiCHlPMW GJr01gEE023PuOQlkuRbOEF2O5uFIk59oFV/G8C2mf0PmDyoaBPLetL5s5X7wSeF xSFmwwd+QtaLn3sy7UF23LNTFyMeBg== =QTqL -----END PGP SIGNATURE----- --TjpHPX55uB1j9qoa7heNJEpbS6baGEhOb--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3a4c5d87-d42e-5615-5d2b-2a8801376600>