Date: Tue, 25 Sep 2012 13:40:58 -0700 From: Jack Vogel <jfvogel@gmail.com> To: Vijay Singh <vijju.singh@gmail.com> Cc: freebsd-net@freebsd.org, John Baldwin <jhb@freebsd.org> Subject: Re: ixgbe rx & tx locks Message-ID: <CAFOYbcmuxBmOfC6P0UBG5RR7xJy_i5mTr2NGMRcTJEJrQmjQ_Q@mail.gmail.com> In-Reply-To: <CALCNsJQ740ceDzpd5n7QAALn-uJ-GdWxPTkQJuMJUMTUGJjOUg@mail.gmail.com> References: <CALCNsJSSQSWV7vNVR-Sn8CPDKbUBBLpSH0b-HYMJo3SXvkOY=w@mail.gmail.com> <201208161736.47250.jhb@freebsd.org> <CAFOYbc=bXv_V4egO94ftkTFoLQ2uM50Sw3X5kgYoyKQcd0s31A@mail.gmail.com> <201208170941.54482.jhb@freebsd.org> <CALCNsJQ740ceDzpd5n7QAALn-uJ-GdWxPTkQJuMJUMTUGJjOUg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Ah yes, at one time I was keeping the RX side lock when calling the stack,
but then as I recall that had problems, so the code now releases and
reaquires
as you can see. It results in some contention but I'm not sure that's
avoidable.
I've seen some LRO related panics on the 1G driver that may be related to
this lock release, or that's one theory I have..
Thanks for the testing Vijay!
Jack
On Tue, Sep 25, 2012 at 1:19 PM, Vijay Singh <vijju.singh@gmail.com> wrote:
> > Vijay, can you test this to see if it helps with your test case?
> >
> >> Jack
>
> John, apologies for the delay. I have some data to share now.
>
> With your patch, the transmit side lock contention is all gone.
> However I still see receive side contention. I have MSI/X enabled,
> with one hw queue per-port.
>
> debug.lock.prof.stats:
> lock held_at
> contested_from instances
> e1b:rx(0) ( sys/dev/ixgbe/ixgbe.c:4314) (
> sys/dev/ixgbe/ixgbe.c:4249) 6814
> e2b:rx(0) ( sys/dev/ixgbe/ixgbe.c:4314) (
> sys/dev/ixgbe/ixgbe.c:4249) 6962
>
> These are from:
>
> 4297 static bool
> 4298 ixgbe_rxeof(struct ix_queue *que, int count)
> 4299 {
> .....
> 4314 IXGBE_RX_LOCK(rxr);
>
> and
>
> 4220 static __inline void
> 4221 ixgbe_rx_input(struct rx_ring *rxr, struct ifnet *ifp, struct
> mbuf *m, u32 ptype)
> 4222 {
> ....
> 4247 IXGBE_RX_UNLOCK(rxr);
> 4248 (*ifp->if_input)(ifp, m);
> 4249 IXGBE_RX_LOCK(rxr);
>
> So it seems like the interrupt handler is still contending with a
> taskqueue handler on the receive side.
>
> -vijay
>
> PS: The interface names are custom.
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFOYbcmuxBmOfC6P0UBG5RR7xJy_i5mTr2NGMRcTJEJrQmjQ_Q>