From owner-freebsd-current@freebsd.org Thu Aug 20 17:05:56 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8B4083C2E52 for ; Thu, 20 Aug 2020 17:05:56 +0000 (UTC) (envelope-from eric@vangyzen.net) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4BXWJ42cdbz4frT for ; Thu, 20 Aug 2020 17:05:56 +0000 (UTC) (envelope-from eric@vangyzen.net) Received: by mailman.nyi.freebsd.org (Postfix) id 5815D3C2EC5; Thu, 20 Aug 2020 17:05:56 +0000 (UTC) Delivered-To: current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 57DEC3C319F for ; Thu, 20 Aug 2020 17:05:56 +0000 (UTC) (envelope-from eric@vangyzen.net) Received: from smtp.vangyzen.net (hotblack.vangyzen.net [IPv6:2607:fc50:1000:7400:216:3eff:fe72:314f]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BXWJ33gG8z4g6r; Thu, 20 Aug 2020 17:05:55 +0000 (UTC) (envelope-from eric@vangyzen.net) Received: from disco.vangyzen.net (unknown [70.97.188.230]) by smtp.vangyzen.net (Postfix) with ESMTPSA id 7A1775648F; Thu, 20 Aug 2020 12:05:48 -0500 (CDT) To: current@freebsd.org, kevans@freebsd.org From: Eric van Gyzen Subject: LOR: tun_ioctl after tun_mtx Message-ID: <41921dc3-b1e9-b823-12f4-d108319c08d4@vangyzen.net> Date: Thu, 20 Aug 2020 12:05:45 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.8.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4BXWJ33gG8z4g6r X-Spamd-Bar: / X-Spamd-Result: default: False [0.20 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[vangyzen.net]; NEURAL_SPAM_MEDIUM(0.05)[0.050]; NEURAL_SPAM_SHORT(0.45)[0.455]; FREEFALL_USER(0.00)[eric]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:36236, ipnet:2607:fc50:1000::/36, country:US]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[current]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2020 17:05:56 -0000 I see this LOR on head r364364 while running the tcptestsuite (ports/net/tcptestsuite). In fact, I interrupted a test with Ctrl-C, and got a panic. I assume it's the same, since the test was twiddling the MTU, but I haven't looked closely. Eric lock order reversal: (sleepable after non-sleepable) 1st 0xfffff802238ea690 tun_mtx (tun_mtx, sleep mutex) @ /usr/src/sys/net/if_tuntap.c:1628 2nd 0xffffffff81d99d28 tun_ioctl (tun_ioctl, sx) @ /usr/src/sys/net/if_tuntap.c:1326 lock order tun_ioctl -> tun_mtx established at: #0 0xffffffff80c432dd at witness_checkorder+0x46d #1 0xffffffff80bb38e4 at __mtx_lock_flags+0x94 #2 0xffffffff80cfad2b at tuninit+0x4b #3 0xffffffff80cfa44f at tunifioctl+0x6f #4 0xffffffff80dc398f at in6_update_ifa+0xa8f #5 0xffffffff80dc96f0 at in6_ifattach+0x5b0 #6 0xffffffff80dc577e at in6_if_up+0x7e #7 0xffffffff80ceb289 at if_up+0x69 #8 0xffffffff80cec2f7 at ifhwioctl+0xd07 #9 0xffffffff80ced475 at ifioctl+0x395 #10 0xffffffff80c490ae at kern_ioctl+0x28e #11 0xffffffff80c48d77 at sys_ioctl+0x127 #12 0xffffffff81015820 at amd64_syscall+0x140 #13 0xffffffff80febb3e at fast_syscall_common+0xf8 lock order tun_mtx -> tun_ioctl attempted at: #0 0xffffffff80c43c3c at witness_checkorder+0xdcc #1 0xffffffff80be0247 at _sx_xlock+0x67 #2 0xffffffff80cfa411 at tunifioctl+0x31 #3 0xffffffff80ceba5b at ifhwioctl+0x46b #4 0xffffffff80cf9101 at tunioctl+0x5b1 #5 0xffffffff80a7b0fc at devfs_ioctl+0xcc #6 0xffffffff80cc9bf2 at vn_ioctl+0x132 #7 0xffffffff80a7b76e at devfs_ioctl_f+0x1e #8 0xffffffff80c490ae at kern_ioctl+0x28e #9 0xffffffff80c48d77 at sys_ioctl+0x127 #10 0xffffffff81015820 at amd64_syscall+0x140 #11 0xffffffff80febb3e at fast_syscall_common+0xf8 local/tcptestsuite/tcptestsuite_atf_test:snd_syn_mss_inherited_from_mtu_72_ipv4 -> ^C[-- Signal caught; please wait for cleanup --] Sleeping thread (tid 100505, pid 61414) owns a non-sleepable lock KDB: stack backtrace of thread 100505: sched_switch() at sched_switch+0x5b2/frame 0xfffffe00627165a0 mi_switch() at mi_switch+0x155/frame 0xfffffe00627165c0 sleepq_switch() at sleepq_switch+0x109/frame 0xfffffe0062716600 _sx_xlock_hard() at _sx_xlock_hard+0x42e/frame 0xfffffe00627166a0 _sx_xlock() at _sx_xlock+0xba/frame 0xfffffe00627166e0 tunifioctl() at tunifioctl+0x31/frame 0xfffffe0062716720 ifhwioctl() at ifhwioctl+0x46b/frame 0xfffffe00627167a0 tunioctl() at tunioctl+0x5b1/frame 0xfffffe0062716810 devfs_ioctl() at devfs_ioctl+0xcc/frame 0xfffffe0062716860 vn_ioctl() at vn_ioctl+0x132/frame 0xfffffe0062716970 devfs_ioctl_f() at devfs_ioctl_f+0x1e/frame 0xfffffe0062716990 kern_ioctl() at kern_ioctl+0x28e/frame 0xfffffe0062716a00 sys_ioctl() at sys_ioctl+0x127/frame 0xfffffe0062716ad0 amd64_syscall() at amd64_syscall+0x140/frame 0xfffffe0062716bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0062716bf0 --- syscall (54, FreeBSD ELF64, sys_ioctl), rip = 0x8005818fa, rsp = 0x7fffffffd408, rbp = 0x7fffffffd540 --- panic: sleeping thread cpuid = 4 time = 1597942792 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00652545e0 vpanic() at vpanic+0x182/frame 0xfffffe0065254630 panic() at panic+0x43/frame 0xfffffe0065254690 propagate_priority() at propagate_priority+0x219/frame 0xfffffe00652546d0 turnstile_wait() at turnstile_wait+0x380/frame 0xfffffe0065254720 __mtx_lock_sleep() at __mtx_lock_sleep+0x1cc/frame 0xfffffe00652547b0 __mtx_lock_flags() at __mtx_lock_flags+0xe5/frame 0xfffffe0065254800 tunifioctl() at tunifioctl+0xdc/frame 0xfffffe0065254840 ifhwioctl() at ifhwioctl+0x2b1/frame 0xfffffe00652548c0 ifioctl() at ifioctl+0x395/frame 0xfffffe0065254990 kern_ioctl() at kern_ioctl+0x28e/frame 0xfffffe0065254a00 sys_ioctl() at sys_ioctl+0x127/frame 0xfffffe0065254ad0 amd64_syscall() at amd64_syscall+0x140/frame 0xfffffe0065254bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0065254bf0 --- syscall (54, FreeBSD ELF64, sys_ioctl), rip = 0x8004b48fa, rsp = 0x7fffffffd428, rbp = 0x7fffffffdc50 --- KDB: enter: panic [ thread pid 61418 tid 100573 ] Stopped at kdb_enter+0x37: movq $0,0x10b70b6(%rip)