Date: Wed, 29 Jan 2020 13:44:44 -0800 From: Eric Joyner <erj@freebsd.org> To: Hans Petter Selasky <hps@selasky.org> Cc: freebsd-net@freebsd.org Subject: Re: Issue with epoch_drain_callbacks and unloading iavf(4) [using iflib] Message-ID: <CAKdFRZi3UoRuz=OXnBG=NVcJe605x9OwrLmdCyD98mDeTpbf0Q@mail.gmail.com> In-Reply-To: <0e2e97f2-df75-3c6f-9bdd-e8c2ab7bf79e@selasky.org> References: <CAKdFRZjxp=mTkUzFU8qsacP86OQOC9vCDCQ%2BO2iF7svRRGDK8w@mail.gmail.com> <0e2e97f2-df75-3c6f-9bdd-e8c2ab7bf79e@selasky.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Jan 29, 2020 at 1:41 PM Hans Petter Selasky <hps@selasky.org> wrote: > On 2020-01-29 22:30, Eric Joyner wrote: > > Hi freebsd-net, > > > > We've encountered an issue with unloading the iavf(4) driver on FreeBSD > > 12.1 (and stable). On a VM with two iavf(4) interfaces, if we send heavy > > traffic to iavf1 and try to kldunload the driver, the kldunload process > > hangs on iavf0 until iavf1 stops receiving traffic. > > > > After some debugging, it looks like epoch_drain_callbacks() [via > > if_detach_internal()] tries to switch CPUs to run on one that iavf1 is > > using for RX processing, but since iavf1 is busy, it can't make the > switch, > > so cpu_switch() just hangs and nothing happens until iavf1's RX thread > > stops being busy. > > > > I can work around this by inserting a kern_yield(PRI_USER) somewhere in > one > > of the iavf txrx functions that iflib calls into (e.g. > > iavf_isc_rxd_available), but that's not a proper fix. Does anyone know > what > > to do to prevent this from happening? > > > > Wildly guessing, does maybe epoch_drain_callbacks() need a higher > priority > > than the PI_SOFT used in the group taskqueues used in iflib's RX > processing? > > > > Hi, > > Which scheduler is this? ULE or BSD? > > EPOCH(9) expects some level of round-robin scheduling on the same > priority level. Setting a higher priority on EPOCH(9) might cause epoch > to start spinning w/o letting the lower priority thread which holds the > EPOCH() section to finish. > > --HPS > > Hi Hans, kern.sched.name gives me "ULE" - Eric
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAKdFRZi3UoRuz=OXnBG=NVcJe605x9OwrLmdCyD98mDeTpbf0Q>