Date: Mon, 15 Oct 2012 15:36:57 -0700 From: Adrian Chadd <adrian@freebsd.org> To: John Baldwin <jhb@freebsd.org> Cc: "Alexander V. Chernikov" <melifaro@freebsd.org>, freebsd-net@freebsd.org, Jack Vogel <jfvogel@gmail.com>, net@freebsd.org, Luigi Rizzo <rizzo@iet.unipi.it> Subject: Re: ixgbe & if_igb RX ring locking Message-ID: <CAJ-Vmo=qMJXwYDUEPHRn49SrAOP%2BNt3v9FaxFXq8wB6Q_uCmPg@mail.gmail.com> In-Reply-To: <201210151414.27318.jhb@freebsd.org> References: <5079A9A1.4070403@FreeBSD.org> <201210150904.27567.jhb@freebsd.org> <20121015163210.GW89655@FreeBSD.org> <201210151414.27318.jhb@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
The reason why I've started moving net80211 and ath _away_ from using direct dispatch (for now) and to using a taskqueue for TX (and RX) is because it's too freaking annoying right now to deal with all the crazy long-held locks to guarantee consistency between multiple transmitting threads. Considering that the driver and net80211 stack: * sometimes is PCI, sometimes is USB (with all the differing thread models that exist there); * sometimes bridge traffic, sometimes route traffic, sometimes source or terminate TCP/UDP connections; * sometimes has one sender, sometimes has multiple senders, with some other modules in between (bridge, pf, ipfw, etc) with locks being held here and there; * since the stack(s) like doing direct dispatch, RX very often causes TX to occur, which for some drivers will block on a long-held driver lock (with all the LORs that occur) - and drivers that do this (eg iwn) will simply drop the lock before passing the packet up. Dropping the lock before passing net80211_input*() .. is just plain silly. Now, I'd _like_ to eventually make net80211/ath support direct dispatch, but that also requires making sure only -one- transmitter is working at once. I'd like to not have the extra context switch overhead, but I haven't seen a better way of doing it yet. It's fun to see the gige/10ge driver have lots of long held locks with lots of concurrent sender processes possibly blocking until TX completes.. so I wonder if that has scaling issues for lots of connections/sending processes. Adrian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-Vmo=qMJXwYDUEPHRn49SrAOP%2BNt3v9FaxFXq8wB6Q_uCmPg>