From owner-freebsd-net@freebsd.org Fri Sep 11 19:25:06 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E08423E35F4 for ; Fri, 11 Sep 2020 19:25:06 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4Bp5LV5gNjz3V4T for ; Fri, 11 Sep 2020 19:25:06 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id C27593E3787; Fri, 11 Sep 2020 19:25:06 +0000 (UTC) Delivered-To: net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C22853E3734; Fri, 11 Sep 2020 19:25:06 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from cell.glebi.us (glebi.us [162.251.186.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "cell.glebi.us", Issuer "cell.glebi.us" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Bp5LV4Gw1z3VGG; Fri, 11 Sep 2020 19:25:06 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from cell.glebi.us (localhost [127.0.0.1]) by cell.glebi.us (8.16.1/8.16.1) with ESMTPS id 08BGBnOQ066394 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 11 Sep 2020 09:11:49 -0700 (PDT) (envelope-from glebius@freebsd.org) Received: (from glebius@localhost) by cell.glebi.us (8.16.1/8.16.1/Submit) id 08BGBmfU066393; Fri, 11 Sep 2020 09:11:48 -0700 (PDT) (envelope-from glebius@freebsd.org) X-Authentication-Warning: cell.glebi.us: glebius set sender to glebius@freebsd.org using -f Date: Fri, 11 Sep 2020 09:11:48 -0700 From: Gleb Smirnoff To: xtouqh@mm.st Cc: current@freebsd.org, net@freebsd.org Subject: Re: bridge/igb panic: sleepq_add: td 0xfffffe01bbce5300 to sleep on wchan 0xffffffff8157d9a0 with sleeping prohibited Message-ID: <20200911161148.GP1269@FreeBSD.org> References: <8172451b-81ea-aacb-02bd-b5005d18646d@mm.st> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="EgVrEAR5UttbsTXg" Content-Disposition: inline In-Reply-To: <8172451b-81ea-aacb-02bd-b5005d18646d@mm.st> X-Rspamd-Queue-Id: 4Bp5LV4Gw1z3VGG X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:27348, ipnet:162.251.186.0/24, country:US] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Sep 2020 19:25:06 -0000 --EgVrEAR5UttbsTXg Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, can you please try out this patch? This is configuration event and we should use sleepable lock to synchronize, not epoch. On Fri, Sep 11, 2020 at 10:47:41AM +0300, xtouqh@mm.st wrote: x> Updating from latest CURRENT snapshot x> (FreeBSD-13.0-CURRENT-amd64-20200910-1544934ffb2) to r365620 broke the x> bridges with igb (I350-T2) for me. Booting to kernel.old and/or x> commenting the entries in rc.conf helps. x> x> rc.conf: x> ---- x> cloned_interfaces="bridge0 bridge1 tap0 tap1 tap2 tap3" x> ifconfig_em0="inet ..." x> ifconfig_igb0="up" x> ifconfig_igb1="up" x> ifconfig_bridge0="addm igb0 addm tap0 addm tap1" x> ifconfig_bridge1="addm igb1 addm tap2 addm tap3" x> ---- x> x> NICs (em0 is on-board, igb0/igb1 is addon I350-T2 card): x> ---- x> em0: mem 0x92d00000-0x92d1ffff at x> device 31.6 numa-domain 0 on pci0 x> em0: Using 1024 TX descriptors and 1024 RX descriptors x> em0: Using an MSI interrupt x> em0: Ethernet address: e0:d5:5e:6c:aa:36 x> em0: netmap queues/slots: TX 1/1024, RX 1/1024 x> igb0: mem x> 0xfbb00000-0xfbbfffff,0xfbc84000-0xfbc87fff at device 0.0 numa-domain 0 x> on pci16 x> igb0: Using 1024 TX descriptors and 1024 RX descriptors x> igb0: Using 8 RX queues 8 TX queues x> igb0: Using MSI-X interrupts with 9 vectors x> igb0: Ethernet address: a0:36:9f:0a:cf:42 x> igb0: netmap queues/slots: TX 8/1024, RX 8/1024 x> igb1: mem x> 0xfba00000-0xfbafffff,0xfbc80000-0xfbc83fff at device 0.1 numa-domain 0 x> on pci16 x> igb1: Using 1024 TX descriptors and 1024 RX descriptors x> igb1: Using 8 RX queues 8 TX queues x> igb1: Using MSI-X interrupts with 9 vectors x> igb1: Ethernet address: a0:36:9f:0a:cf:43 x> igb1: netmap queues/slots: TX 8/1024, RX 8/1024 x> ---- x> x> panic: x> ---- x> panic: sleepq_add: td 0xfffffe01bbce5300 to sleep on wchan x> 0xffffffff8157d9a0 with sleeping prohibited x> cpuid = 16 x> time = 1599808542 x> KDB: stack backtrace: x> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame x> 0xfffffe01ba658c40 x> vpanic() at vpanic+0x182/frame 0xfffffe01ba658c90 x> panic() at panic+0x43/frame 0xfffffe01ba658cf0 x> sleepq_add() at sleepq_add+0x359/frame 0xfffffe01ba658d40 x> _sleep() at _sleep+0x20c/frame 0xfffffe01ba658df0 x> pause_sbt() at pause_sbt+0xfe/frame 0xfffffe01ba658e20 x> e1000_reset_hw_82580() at e1000_reset_hw_82580+0x1c8/frame x> 0xfffffe01ba658e60 x> em_if_stop() at em_if_stop+0x1b/frame 0xfffffe01ba658e80 x> iflib_stop() at iflib_stop+0xbd/frame 0xfffffe01ba658ed0 x> iflib_if_ioctl() at iflib_if_ioctl+0x397/frame 0xfffffe01ba658f40 x> bridge_mutecaps() at bridge_mutecaps+0x145/frame 0xfffffe01ba658fb0 x> bridge_ioctl_add() at bridge_ioctl_add+0x468/frame 0xfffffe01ba659000 x> bridge_ioctl() at bridge_ioctl+0x32b/frame 0xfffffe01ba6590d0 x> in_control() at in_control+0x322/frame 0xfffffe01ba659180 x> ifioctl() at ifioctl+0x3e8/frame 0xfffffe01ba659250 x> kern_ioctl() at kern_ioctl+0x28e/frame 0xfffffe01ba6592c0 x> sys_ioctl() at sys_ioctl+0x127/frame 0xfffffe01ba659390 x> amd64_syscall() at amd64_syscall+0x140/frame 0xfffffe01ba6594b0 x> fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01ba6594b0 x> --- syscall (54, FreeBSD ELF64, sys_ioctl), rip = 0x8004b4aba, rsp = x> 0x7fffffffe2b8, rbp = 0x7fffffffe360 --- x> Uptime: 14s x> Dumping 3794 out of 97961 x> MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% x> ---- x> _______________________________________________ x> freebsd-current@freebsd.org mailing list x> https://lists.freebsd.org/mailman/listinfo/freebsd-current x> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" -- Gleb Smirnoff --EgVrEAR5UttbsTXg Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="lagg_register_vlan-sleeps.diff" Index: if_lagg.c =================================================================== --- if_lagg.c (revision 364434) +++ if_lagg.c (working copy) @@ -476,10 +476,10 @@ lagg_register_vlan(void *arg, struct ifnet *ifp, u if (ifp->if_softc != arg) /* Not our event */ return; - LAGG_RLOCK(); + sx_slock(&sc->sc_sx); CK_SLIST_FOREACH(lp, &sc->sc_ports, lp_entries) EVENTHANDLER_INVOKE(vlan_config, lp->lp_ifp, vtag); - LAGG_RUNLOCK(); + sx_sunlock(&sc->sc_sx); } /* @@ -495,10 +495,10 @@ lagg_unregister_vlan(void *arg, struct ifnet *ifp, if (ifp->if_softc != arg) /* Not our event */ return; - LAGG_RLOCK(); + sx_slock(&sc->sc_sx); CK_SLIST_FOREACH(lp, &sc->sc_ports, lp_entries) EVENTHANDLER_INVOKE(vlan_unconfig, lp->lp_ifp, vtag); - LAGG_RUNLOCK(); + sx_sunlock(&sc->sc_sx); } static int --EgVrEAR5UttbsTXg--