Date: Fri, 18 Nov 2011 18:06:15 +0100 From: Luigi Rizzo <rizzo@iet.unipi.it> To: John Baldwin <jhb@freebsd.org> Cc: Matteo Landi <matteo@matteolandi.net>, freebsd-current@freebsd.org Subject: Re: ixgbe and fast interrupts Message-ID: <20111118170615.GA9762@onelab2.iet.unipi.it> In-Reply-To: <201111180800.06593.jhb@freebsd.org> References: <CALJ8J_HPZewO12uanb=kctQYwepMssr63E0DQh9CqV6PGaC=JA@mail.gmail.com> <201111170953.58151.jhb@freebsd.org> <CALJ8J_HZiJfh_id%2BggaYZpU5UhUjZy==CK9jmdR4pePjfi=2gw@mail.gmail.com> <201111180800.06593.jhb@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Nov 18, 2011 at 08:00:06AM -0500, John Baldwin wrote: > On Friday, November 18, 2011 3:46:02 am Matteo Landi wrote: > > > you probably want to be using MSI-X for a 10G NIC instead of INTx anyway. > > > > Why do you say that? Is MSI-X faster than INTx in terms of interrupt > > latency? When should I use MSI-X, instead of fast filters interrupts > > (fast interrupt?), instead of ithread interrupts? Thanks in advace. > > With MSI-X you can have more than one interrupt and those interrupts can be > distributed across CPUs. This means you can (somewhat) tie each queue on your > NIC to a different CPU. > > MSI-X vs INTx is orthogonal to fast vs filter, but in general MSI and MSI-X > interrupts are not shared, and require no interrupt masking in hardware (they > are effectively edge-triggered), so using a filter for MSI is rather pointless > and only adds needless complexity. For MSI I would just use a theraded > interrupt handler. For INTx, I would only use a fast interrupt handler if > there is a really good reason to do so (e.g. em(4) does so to work around > broken Intel Host-PCI bridges). A bit more context: Matteo is looking at the latency of RPCs across the network involving userspace processes, and possibly using the netmap API. As we understand it: if you are not using a filter, the interrupt calls a "predefined" filter (kern_intr.c::intr_event_schedule_thread() ? ) which wakes up the handler thread which in turn wakes up the user process. This means two scheduler invocations on each side. In the case of netmap, all the handler needs to do is a selwakeup() of the user thread blocked on the file descriptor, so if this can be done in the filter we can save an extra step through the scheduler. cheers luigi
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20111118170615.GA9762>