Date: Wed, 1 Oct 2014 11:48:48 +0200 From: Luigi Rizzo <rizzo@iet.unipi.it> To: Adrian Chadd <adrian@freebsd.org>, "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Cc: Elof Ofel <elofu17@hotmail.com>, "Alexander V. Chernikov" <melifaro@freebsd.org> Subject: individual queue blocking entire rx unit on ixgbe (Re: How do I balance bandwidth over several virtual NICs?) Message-ID: <CA%2BhQ2%2BhYY7AWE7jJWAg8uYER2LRSPz8TcVPuamsRBLj_WQyakw@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
reviving this thread: i am just running experiments on 10.1 beta3 and even setting dev.ix.*.fc=0 and flipping the interface up and down does not seem to help: if i read only from a subset of the queues, the entire rx unit stalls eventually. I need to drain all queues to keep moving. Just tested this with 8 instances of netmap-ipfw running on an 8-core machine (8 queues enabled). netmap-ipfw netmap:ix0-0 netmap:ix1-0 netmap-ipfw netmap:ix0-1 netmap:ix1-1 ... and the source on another box is blasting on multiple queues with pkt-gen -f tx -i ix0 -d 10.0.10.0-10.0.10.255 I going to look at the driver's code now to see if/how this issue can be addressed. cheers luigi On Tue, Sep 23, 2014 at 6:00 PM, Adrian Chadd <adrian@freebsd.org> wrote: > Ah, this behaviour. > > It's called DROP_EN on the intel igb / ixgbe hardware. Grep the > drivers for that particular register bit/setting. > > Set that bit for an RX queue and it'll instruct the MAC to drop frames > destined if that RX ring is full to it and keep receiving on the other > rings. Otherwise yes, receiving on that ring with the ring full cuases > the MAC to stop receiving on all rings until that ring has free space. > > You flip this on with ixgbe and igb by disabling tx/rx flowcontrol > (sysctl dev.ix|igb.X.fc=0) before configuring the interface. > > > > -a > -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+-------------------------------
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2BhQ2%2BhYY7AWE7jJWAg8uYER2LRSPz8TcVPuamsRBLj_WQyakw>