Date: Sun, 21 Oct 2012 22:21:45 +0400 From: "Alexander V. Chernikov" <melifaro@FreeBSD.org> To: Ryan Stone <rysto32@gmail.com> Cc: Jack Vogel <jfvogel@gmail.com>, net@freebsd.org Subject: Re: ixgbe & if_igb RX ring locking Message-ID: <50843D39.5000003@FreeBSD.org> In-Reply-To: <CAFMmRNzA23r_a0nrnsUN8ZoHGL5c7OF1-nCqWpN42FLc50d7eg@mail.gmail.com> References: <5079A9A1.4070403@FreeBSD.org> <20121015162926.GV89655@FreeBSD.org> <CAFMmRNxT=GWxc6r7B81ENjzwJmfea3016Sh-DxJEGBwybM0QwQ@mail.gmail.com> <507D4F11.2030704@FreeBSD.org> <CAFMmRNy1h86ZXAH4oK%2BLOK6_VeTsbo=d6oYk669bisKfHbJ56A@mail.gmail.com> <20121016124733.GC89655@glebius.int.ru> <CAFMmRNzA23r_a0nrnsUN8ZoHGL5c7OF1-nCqWpN42FLc50d7eg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 16.10.2012 17:20, Ryan Stone wrote:
> On Tue, Oct 16, 2012 at 8:47 AM, Gleb Smirnoff<glebius@freebsd.org> wrote:
>> Can you please provide hints how can SIOCADDMULTI lead to obtaining RX
>> lock in the stock driver?
>
> It doesn't. But it does acquire the core lock, and the core lock is
> acquired before the RX lock (in ixgbe_init, for instance).
I still can't get how any of these can happen.
We basically has 2 related places where ixgbe driver interact with OS:
1) forwarding path (ixgbe_rxeof via ithread / taskq)
2) ioctl path
I'm currently talking about multicast only.
So, first can lead us to:
ixgbe_rxeof() (rxr->rx_mtx "ix:rx")
ether_input()
...
udp_input() (V_udbinfo "udp" READ lock)
udp_input() (inp->inp_lock "udpinp" READ lock)
>
2) ioctl part can lead to the following locks to be held:
IXGBE_CORE_LOCK() in every ioctl ("ix?:core" mtx)
ixgbe_init_locked()
ixgbe_setup_receive_structures()
ixgbe_setup_receive_ring() (rxr->rx_mtx "ix:rx")
.. with any possibly locks prepended (like in_multi_mtx or any other)
> Last time I checked(FreeBSD 8.2), this is not true. The problematic
> (and convoluted) ordering is:
>
> ix:rx -> udp -> udpinp -> in_multi_mtx -> ix:core -> ix:rx
>
> udp -> udpinp -> in_multi_mtx is defined in subr_witness.c.
Actually subr_witness defines the following:
1) udp -> udpinp -> so_snd
2) udpinp -> in_multi_mtx -> .. -> ..
>
> ix:core -> ix:rx is fairly obvious, and happens in places like
ixgbe_init.
>
Yes. But how can I trigger reverted order (e.g. ixgbe_init[_locked]()
(actually, ioctl) on packet input ?
(And if such place really exists we should consider fixing it, but not
the driver side)
> ix:rx -> udp is also fairly obvious (here's one backtrace):
The same question, where "udp" -> "ix:rx" can happen ?
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?50843D39.5000003>