From owner-freebsd-current@freebsd.org Tue Sep 3 14:07:21 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 18132DD90D; Tue, 3 Sep 2019 14:07:21 +0000 (UTC) (envelope-from yuripv@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46N80S5qPGz4QW9; Tue, 3 Sep 2019 14:07:20 +0000 (UTC) (envelope-from yuripv@freebsd.org) Received: by freefall.freebsd.org (Postfix, from userid 1452) id 7CA061B4B7; Tue, 3 Sep 2019 14:06:36 +0000 (UTC) X-Original-To: yuripv@localmail.freebsd.org Delivered-To: yuripv@localmail.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mx1.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by freefall.freebsd.org (Postfix) with ESMTPS id 6CE10C5CD; Mon, 22 Apr 2019 11:25:33 +0000 (UTC) (envelope-from owner-src-committers@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 338ED865A5; Mon, 22 Apr 2019 11:25:33 +0000 (UTC) (envelope-from owner-src-committers@freebsd.org) Received: by freefall.freebsd.org (Postfix, from userid 538) id 14B47C5CC; Mon, 22 Apr 2019 11:25:33 +0000 (UTC) Delivered-To: src-committers@localmail.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mx1.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by freefall.freebsd.org (Postfix) with ESMTPS id CB3E0C5CA for ; Mon, 22 Apr 2019 11:25:30 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pl1-x641.google.com (mail-pl1-x641.google.com [IPv6:2607:f8b0:4864:20::641]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3235C865A1; Mon, 22 Apr 2019 11:25:30 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mail-pl1-x641.google.com with SMTP id w24so5702510plp.2; Mon, 22 Apr 2019 04:25:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=fHIktrEk9er+LLxwC69feHZT3nvs1eOw25g/mbv/KIk=; b=t2oAAcVA/cnpts16FQ0puPFRLWGgWkzw6ohirY/CEmhch5ZIDL5k/HzOnrjMJnM2qE zgsqwIOJ5fERRltuYm8ngaymO5RQx84SVVwOBTTsJyFNt/rEFPN2Out7ozXINKBJggSb RVU22R7PlFTTnQljqBB1JO2ZKgA6rFoHKv+8rVq53WnKel7Cd5hEv7llF67ZoFoP6b7i /qdZFMYcWcp0I5Co249QUptL1p+0MDyVaTAVpWky0WzPBnB21omdzC5R/XYHIcGuMAZ8 oZWhnUpnnaUXXO4kevp9tosOgU8uFu1lvnUcBvXrNH7s8ihnQvSy3iTEcjgDmBTx6Ai8 H2qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=fHIktrEk9er+LLxwC69feHZT3nvs1eOw25g/mbv/KIk=; b=SU1qZhYWVq8ObbA1nx32cp/shmWpfUw4Lx4XBIFGG5gYRZOk2hQNeNsKFk+4aJo57X SWOSGK5YF31uP1h1nOiBxUb/+lWm3mBAHQ/a2/Uc2Xo8x0snBxHYchGNqbxuTjR5kBk1 gZBtpIQ8ILKfHRZwLRliKWW7KhJFnm4hPnGDEKinPwTIMnUR7kGXt84qMPJIrVfTOs5L JKTw6tgs6bZi4JoTw23n5jiJFUoJE77GcKUCGXLfmH62gkmdx03FrogvH05Q2qYZYuCv 5vkWtXk6im1/iaCTlmEtE6J41z+R0663M1W+475nDTqa7QT8G5ukjr/RMubTeXFQ3j8s VB6A== X-Gm-Message-State: APjAAAXVJqkBzI2ywePAm70/Z0Nkwp8p01cqPeAwZgFURFcvGJv3rpSs DvI51+mPmLD6q21xcGxkrngX0zhNQLw= X-Google-Smtp-Source: APXvYqze44MOEAzDMojKvzphAeIW+xmiPYCvmv2OFpTDzhjb3HjmEwRlhXA6hZMMrm+rFyXMeHIH5A== X-Received: by 2002:a17:902:42:: with SMTP id 60mr19754000pla.79.1555932328982; Mon, 22 Apr 2019 04:25:28 -0700 (PDT) Received: from [192.168.20.7] (c-73-19-52-228.hsd1.wa.comcast.net. [73.19.52.228]) by smtp.gmail.com with ESMTPSA id v12sm16904988pfe.148.2019.04.22.04.25.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Apr 2019 04:25:28 -0700 (PDT) From: Enji Cooper Message-Id: <8EAC0CFE-E22F-478F-813F-A07E68C0518D@gmail.com> Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Panic with r346530 [Re: svn commit: r346530 - in head/sys: netinet netinet6] In-Reply-To: Cc: src-committers , svn-src-all , svn-src-head , FreeBSD CURRENT To: Hans Petter Selasky References: <201904220727.x3M7ROpR009729@repo.freebsd.org> <2F3D6B17-AF4F-4B0F-B20E-5EF41DE851F9@gmail.com> <87917500-0381-79d8-a34b-819848abed32@selasky.org> X-Mailer: Apple Mail (2.3445.104.8) Precedence: bulk X-Loop: FreeBSD.org Sender: owner-src-committers@freebsd.org X-Rspamd-Queue-Id: 338ED865A5 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.94 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.94)[-0.941,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] Status: O Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Date: Tue, 03 Sep 2019 14:07:21 -0000 X-Original-Date: Mon, 22 Apr 2019 04:25:27 -0700 X-List-Received-Date: Tue, 03 Sep 2019 14:07:21 -0000 Hi Hans, > On Apr 22, 2019, at 1:32 AM, Hans Petter Selasky = wrote: >=20 > On 4/22/19 10:10 AM, Hans Petter Selasky wrote: >> On 4/22/19 9:52 AM, Enji Cooper wrote: >>>=20 >>>> On Apr 22, 2019, at 12:27 AM, Hans Petter Selasky = wrote: >>>>=20 >>>> Author: hselasky >>>> Date: Mon Apr 22 07:27:24 2019 >>>> New Revision: 346530 >>>> URL: https://svnweb.freebsd.org/changeset/base/346530 >>>>=20 >>>> Log: >>>> Fix panic in network stack due to memory use after free in = relation to >>>> fragmented packets. >>>>=20 >>>> When sending IPv4 and IPv6 fragmented packets and a fragment is = lost, >>>> the mbuf making up the fragment will remain in the temporary = hashed >>>> fragment list for a while. If the network interface departs = before the >>>> so-called slow timeout clears the packet, the fragment causes a = panic >>>> when the timeout kicks in due to accessing a freed network = interface >>>> structure. >>>>=20 >>>> Make sure that when a network device is departing, all hashed = IPv4 and >>>> IPv6 fragments belonging to it, get freed. >>>>=20 >>>> Backtrace: >>>> panic() >>>> icmp6_reflect() >>>>=20 >>>> hlim =3D ND_IFINFO(m->m_pkthdr.rcvif)->chlim; >>>> ^^^^ rcvif->if_afdata[AF_INET6] is NULL. >>>>=20 >>>> icmp6_error() >>>> frag6_freef() >>>> frag6_slowtimo() >>>> pfslowtimo() >>>> softclock_call_cc() >>>> softclock() >>>> ithread_loop() >>>>=20 >>>> Differential Revision: https://reviews.freebsd.org/D19622 >>>> Reviewed by: bz (network), adrian >>>> MFC after: 1 week >>>> Sponsored by: Mellanox Technologies >=20 > Should be fixed by >=20 > r346535 >=20 > Else I'll revert. ... The code compiles, but unfortunately panics when running the test suite. = =46rom https://ci.freebsd.org/job/FreeBSD-head-amd64-test/10926/console: 03:05:01 1st 0xffffffff820967f0 allprison (allprison) @ = /usr/src/sys/kern/kern_jail.c:966 03:05:01 2nd 0xffffffff820c47f0 vnet_sysinit_sxlock = (vnet_sysinit_sxlock) @ /usr/src/sys/net/vnet.c:575 03:05:01 stack backtrace: 03:05:01 #0 0xffffffff80c477f3 at witness_debugger+0x73 03:05:01 #1 0xffffffff80c4753d at witness_checkorder+0xa7d 03:05:01 #2 0xffffffff80be9088 at _sx_slock_int+0x68 03:05:01 #3 0xffffffff80d0ef97 at vnet_alloc+0x117 03:05:01 #4 0xffffffff80ba4111 at kern_jail_set+0x1bb1 03:05:01 #5 0xffffffff80ba5b70 at sys_jail_set+0x40 03:05:01 #6 0xffffffff810b2e16 at amd64_syscall+0x276 03:05:01 #7 0xffffffff8108b44d at fast_syscall_common+0x101 03:05:01 panic: mtx_lock() of destroyed mutex @ = /usr/src/sys/netinet/ip_reass.c:628 03:05:01 cpuid =3D 1 03:05:01 time =3D 1555927501 03:05:01 KDB: stack backtrace: 03:05:01 db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame = 0xfffffe0030eec630 03:05:01 vpanic() at vpanic+0x19d/frame 0xfffffe0030eec680 03:05:01 panic() at panic+0x43/frame 0xfffffe0030eec6e0 03:05:02 __mtx_lock_flags() at __mtx_lock_flags+0x12e/frame = 0xfffffe0030eec730 03:05:02 ipreass_cleanup() at ipreass_cleanup+0x86/frame = 0xfffffe0030eec770 03:05:02 if_detach_internal() at if_detach_internal+0x786/frame = 0xfffffe0030eec7f0 03:05:02 if_detach() at if_detach+0x3d/frame 0xfffffe0030eec810 03:05:02 lo_clone_destroy() at lo_clone_destroy+0x16/frame = 0xfffffe0030eec830 03:05:02 if_clone_destroyif() at if_clone_destroyif+0x21f/frame = 0xfffffe0030eec880 03:05:02 if_clone_detach() at if_clone_detach+0xb8/frame = 0xfffffe0030eec8b0 03:05:02 vnet_loif_uninit() at vnet_loif_uninit+0x26/frame = 0xfffffe0030eec8d0 03:05:02 vnet_destroy() at vnet_destroy+0x124/frame 0xfffffe0030eec900 03:05:02 prison_deref() at prison_deref+0x29d/frame 0xfffffe0030eec940 03:05:02 sys_jail_remove() at sys_jail_remove+0x28f/frame = 0xfffffe0030eec990 03:05:02 amd64_syscall() at amd64_syscall+0x276/frame 0xfffffe0030eecab0 03:05:02 fast_syscall_common() at fast_syscall_common+0x101/frame = 0xfffffe0030eecab0 03:05:02 --- syscall (508, FreeBSD ELF64, sys_jail_remove), rip =3D = 0x80031e12a, rsp =3D 0x7fffffffe998, rbp =3D 0x7fffffffea20 --- 03:05:02 KDB: enter: panic 03:05:02 [ thread pid 13109 tid 100150 ] 03:05:02 Stopped at kdb_enter+0x3b: movq $0,kdb_why 03:05:02 db:0:kdb.enter.panic> show pcpu 03:05:02 cpuid =3D 1 03:05:02 dynamic pcpu =3D 0xfffffe0080191800 03:05:02 curthread =3D 0xfffff80005c1f000: pid 13109 tid 100150 = "jail" 03:05:02 curpcb =3D 0xfffffe0030eecb80 03:05:02 fpcurthread =3D 0xfffff80005c1f000: pid 13109 "jail" 03:05:02 idlethread =3D 0xfffff800032765a0: tid 100004 "idle: cpu1" 03:05:02 curpmap =3D 0xfffff8013d837130 03:05:02 tssp =3D 0xffffffff821cd388 03:05:02 commontssp =3D 0xffffffff821cd388 03:05:02 rsp0 =3D 0xfffffe0030eecb80 03:05:02 gs32p =3D 0xffffffff821d3fc0 03:05:02 ldt =3D 0xffffffff821d4000 03:05:02 tss =3D 0xffffffff821d3ff0 03:05:02 tlb gen =3D 314416 03:05:02 curvnet =3D 0xfffff80139320200 03:05:02 spin locks held: 03:05:02 db:0:kdb.enter.panic> alltrace Either the sys/netinet/ or sys/netipsec/ tests triggered the = panic. Not sure which right now. Cheers, -Enji=