Date: Sun, 12 Oct 2014 02:43:50 +0000 From: bugzilla-noreply@freebsd.org To: freebsd-bugs@FreeBSD.org Subject: [Bug 194311] New: [ixgbe] DROP_EN needs to be set early to avoid RX queue hanging Message-ID: <bug-194311-8@https.bugs.freebsd.org/bugzilla/>
next in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194311 Bug ID: 194311 Summary: [ixgbe] DROP_EN needs to be set early to avoid RX queue hanging Product: Base System Version: 11.0-CURRENT Hardware: Any OS: Any Status: Needs Triage Severity: Affects Only Me Priority: --- Component: kern Assignee: freebsd-bugs@FreeBSD.org Reporter: adrian@freebsd.org >From Luigi Rizzo: reviving this thread: i am just running experiments on 10.1 beta3 and even setting dev.ix.*.fc=0 and flipping the interface up and down does not seem to help: if i read only from a subset of the queues, the entire rx unit stalls eventually. I need to drain all queues to keep moving. Just tested this with 8 instances of netmap-ipfw running on an 8-core machine (8 queues enabled). netmap-ipfw netmap:ix0-0 netmap:ix1-0 netmap-ipfw netmap:ix0-1 netmap:ix1-1 ... and the source on another box is blasting on multiple queues with pkt-gen -f tx -i ix0 -d 10.0.10.0-10.0.10.255 I going to look at the driver's code now to see if/how this issue can be addressed. ... don't have a way to test this on HEAD. Do you know if there is any change that could be related ? On 10.1 beta 3 I noticed is that when I open a single queue on an ixgbe (and with incoming traffic), the rx unit stalls no matter what the previous state of dev.ix.*.fc (i.e. the DROP_EN bits) or QDE are. In this state, toggling DROP_EN has no effect, whereas something that seems effective is setting the QDE bit(s) in the PFQDE register for all queues, _after_ i open the device. >From the above my take is the following: - on NIC reset, the SRRCTL register starts at 0 including DROP_EN; same goes for PFQDE.QDE - setting SRRCTL.DROP_EN must happen before the receive unit is started; - conversely, toggling PFQDE.QDE has effect even when the receive unit has started - sysctl dev.ix.*.fc sets/clear DROP_EN but at the wrong time (the right time seems to be the window between reset and start) I am going to run more tests to figure out. cheers ... Index: sys/dev/ixgbe/ixgbe.c =================================================================== --- sys/dev/ixgbe/ixgbe.c (revision 272974) +++ sys/dev/ixgbe/ixgbe.c (working copy) @@ -4377,6 +4377,20 @@ srrctl &= ~IXGBE_SRRCTL_BSIZEPKT_MASK; srrctl |= bufsz; srrctl |= IXGBE_SRRCTL_DESCTYPE_ADV_ONEBUF; + + /* + * Set DROP_EN iff we have no flow control and >1 queue. + * Note that srrctl was cleared shortly before during reset, + * so we do not need to clear the bit, but do it just in case + * this code is moved elsewhere. + */ + if (adapter->num_queues > 1 && + adapter->hw.fc.requested_mode == ixgbe_fc_none) { + srrctl |= IXGBE_SRRCTL_DROP_EN; + } else { + srrctl &= ~IXGBE_SRRCTL_DROP_EN; + } + IXGBE_WRITE_REG(hw, IXGBE_SRRCTL(i), srrctl); /* Setup the HW Rx Head and Tail Descriptor Pointers */ -- You are receiving this mail because: You are the assignee for the bug.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-194311-8>