Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 3 Oct 2015 01:10:46 +0300
From:      Nikos Vassiliadis <nvass@gmx.com>
To:        freebsd-net@freebsd.org
Subject:   carp on if_bridge deadlock
Message-ID:  <560F00E6.5010503@gmx.com>

next in thread | raw e-mail | index | archive | help
Hi,

I am trying to use carp over an if_bridge and am getting
this LOR:
> login: lock order reversal:
>  1st 0xfffff8000848a018 if_bridge (if_bridge) @ /usr/src/sys/modules/if_bridge/../../net/if_bridge.c:2315
>  2nd 0xfffff80003a58778 carp_if (carp_if) @ /usr/src/sys/modules/carp/../../netinet/ip_carp.c:1118
> KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0092c3b6a0
> witness_checkorder() at witness_checkorder+0xe7a/frame 0xfffffe0092c3b720
> __mtx_lock_flags() at __mtx_lock_flags+0xa8/frame 0xfffffe0092c3b770
> carp_forus() at carp_forus+0x7a/frame 0xfffffe0092c3b7b0
> bridge_input() at bridge_input+0x338/frame 0xfffffe0092c3b820
> ether_nh_input() at ether_nh_input+0x2ab/frame 0xfffffe0092c3b860
> netisr_dispatch_src() at netisr_dispatch_src+0x86/frame 0xfffffe0092c3b8d0
> ether_input() at ether_input+0x4f/frame 0xfffffe0092c3b900
> vtnet_rxq_eof() at vtnet_rxq_eof+0x845/frame 0xfffffe0092c3b9b0
> vtnet_rx_vq_intr() at vtnet_rx_vq_intr+0x4e/frame 0xfffffe0092c3b9e0
> intr_event_execute_handlers() at intr_event_execute_handlers+0xe4/frame 0xfffffe0092c3ba20
> ithread_loop() at ithread_loop+0xa6/frame 0xfffffe0092c3ba70
> fork_exit() at fork_exit+0x84/frame 0xfffffe0092c3bab0
> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0092c3bab0
> --- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Eventually all network activity will stop and this will happen:
> load: 0.33  cmd: ifconfig 665 [*carp_if] 228.29r 0.00u 0.01s 0% 2716k
Or:
> load: 2.86  cmd: ifconfig 720 [running] 104.92r 0.00u 36.53s 100% 2716k

A single ping to the carp address is enough to trigger the problem.
The debugger says:
> db> show all locks
> Process 669 (sysctl) thread 0xfffff800084269a0 (100082)
> exclusive sleep mutex Giant (Giant) r = 0 (0xffffffff81cc4e90) locked @ /usr/src/sys/kern/kern_sysctl.c:164
> Process 668 (ifconfig) thread 0xfffff8000842e9a0 (100079)
> exclusive sx carp_sx (carp_sx) r = 0 (0xffffffff820b0ac0) locked @ /usr/src/sys/modules/carp/../../netinet/ip_carp.c:1644
> Process 12 (intr) thread 0xfffff800038a89a0 (100010)
> exclusive sleep mutex carp_softc (carp_softc) r = 0 (0xfffff80008266508) locked @ /usr/src/sys/kern/kern_mutex.c:158
> Process 12 (intr) thread 0xfffff80003a679a0 (100031)
> exclusive sleep mutex carp_if (carp_if) r = 0 (0xfffff8000825b078) locked @ /usr/src/sys/modules/carp/../../netinet/ip_carp.c:1118
> exclusive sleep mutex if_bridge (if_bridge) r = 0 (0xfffff80008467018) locked @ /usr/src/sys/modules/if_bridge/../../net/if_bridge.c:2315
> db>

There is a related PR
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200319
But I couldn't apply cleanly the patch to HEAD or 10-STABLE.

Thanks for any insights,
Nikos




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?560F00E6.5010503>