Date: Sat, 11 May 2013 15:16:50 -0700 From: Adrian Chadd <adrian@freebsd.org> To: Hooman Fazaeli <hoomanfazaeli@gmail.com> Cc: Barney Cordoba <barney_cordoba@yahoo.com>, =?ISO-8859-1?B?IkNs6W1lbnQgSGVybWFubiAobm9kZW5zKSI=?= <nodens2099@gmail.com>, Eugene Grosbein <egrosbein@rdtc.ru>, freebsd-net@freebsd.org Subject: Re: High CPU interrupt load on intel I350T4 with igb on 8.3 Message-ID: <CAJ-Vmom47bzkFZKhb%2BT8Xb88jo1YxD=jh4EK5OgPDd9xn_x2gg@mail.gmail.com> In-Reply-To: <518EA643.5010505@gmail.com> References: <1368287797.70288.YahooMailClassic@web121603.mail.ne1.yahoo.com> <518EA643.5010505@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, The motivation behind the locking scheme in igb in friends is for a very specific, userland-traffic-origin workload. Sure, it may or may not work well for forwarding/filtering workloads. If you want to fix it, let's have a discussion about how to do it, followed by some patches to do so. Adrian On 11 May 2013 13:12, Hooman Fazaeli <hoomanfazaeli@gmail.com> wrote: > On 5/11/2013 8:26 PM, Barney Cordoba wrote: >> Clearly you don't understand the problem. Your logic is that because other drivers are defective also; therefore its not a driver problem? The problem is caused by a multi-threaded driver that >> haphazardly launches tasks and that doesn't manage the case that the rest of the system can't handle the load. It's no different than a driver that barfs when mbuf clusters are exhausted. The answer >> isn't to increase memory or mbufs, even though that may alleviate the problem. The answer is to fix the driver, so that it doesn't crash the system for an event that is wholly predictable. igb has >> 1) too many locks and 2) exasperates the problem by binding to cpus, which causes it to not only have to wait for the lock to free, but also for a specific cpu to become free. So it chugs along >> happily until it encounters a bottleneck, at which point it quickly blows up the entire system in a domino effect. It needs to manage locks more efficiently, and also to detect when the backup is >> unmanageable. Ever since FreeBSD 5 the answer has been "it's fixed in 7, or its fixed in 9, or it's fixed in 10". There will always be bottlenecks, and no driver should blow up the system no matter >> what intermediate code may present a problem. Its the driver's responsibility to behave and to drop packets if necessary. BC > > And how the driver should behave? You suggest dropping the packets. Even if we accept > that dropping packets is a good strategy in all configurations (which I doubt), the driver is > definitely not the best place to implement it, since that involves duplication of similar > code between drivers. Somewhere like the Ethernet layer is a much better choice to watch > load of packets and drop them to prevent them to eat all the cores. Furthermore, ignoring > the fact that pf is not optimized for multi-processors and blaming drivers for not adjusting > themselves with the this pf's fault, is a bit unfair, I believe. > > > -- > > Best regards. > Hooman Fazaeli > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-Vmom47bzkFZKhb%2BT8Xb88jo1YxD=jh4EK5OgPDd9xn_x2gg>