Date: Thu, 1 Oct 2009 14:26:55 -0400 From: John Baldwin <jhb@freebsd.org> To: Bruce Evans <brde@optusnet.com.au> Cc: arch@freebsd.org Subject: Re: Interrupt Descriptions Message-ID: <200910011426.56047.jhb@freebsd.org> In-Reply-To: <20091002024550.X21519@delplex.bde.org> References: <200909301732.20589.jhb@freebsd.org> <200910010949.20849.jhb@freebsd.org> <20091002024550.X21519@delplex.bde.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thursday 01 October 2009 1:14:01 pm Bruce Evans wrote: > On Thu, 1 Oct 2009, John Baldwin wrote: > > > On Wednesday 30 September 2009 8:02:57 pm Bruce Evans wrote: > >> On Wed, 30 Sep 2009, John Baldwin wrote: > >> > >>> A few folks have asked recently for the ability to add descriptive strings to > >>> registered interrupt handlers. This is especially true since the advent of > >>> MSI with multiple interrupts per device. I hacked up a prototype today that > >> > >> Interrupt names should be no longer than 4 (5 works sometimes) characters so > >> that they can be displayed by systat -v. > > > > For the machines this is targeted at (i.e. ones with devices using MSI/MSI-X) > > the number of interrupts is already well beyond what systat -v can currently > > display anyway. > > This should be fixed before expanding the names into descriptions. To me this change doesn't make systat any more or less broken than it was before, so I consider systat an orthogonal problem. > I wonder what uses all these interrupts. Most FreeBSD cluster machines use > a whole 3 interrupts apart from acpi_timer, with 1 barely active (1 disk, > 1 net, and 1 serial i/o probably only used for a serial console). My > workstations use about 10 plus timers, but most are usually inactive > except for net, keyboard and sometimes disk. Does anyone have systems > more than about 16 disk controllers and 16 network cards so as to need > more than 32 interrupts? As Jack mentioned, many newer devices are starting to take advantage of MSI to use multiple interrupts for a single device. A typical cxgb(4) adapter will use up to 9 interrupts (1 for link/misc, and 1 per-CPU for a RX/TX queueset). The Intel 10gb driver takes this even farther by giving each queueset separate RX/TX interrupts and it supports up to 16 queues giving up to 33 total interrupts per device. Even the igb(4) driver currently uses 1 interrupt for link, 1 for each RX queue, and 1 for each TX queue and it supports having at least 4 RX queues (possibly as many as 8?). Devices of this nature using lots of interrupts is also what drove the change to use per-CPU IDTs in 8.0 to increase the number of available IDT vectors for these types of devices that want MSI interrupts tied to every CPU. > > I believe most folks use vmstat -i or ithread stats in top > > to measure real-time interrupts at this point. Possibly systat could grow a > > new interrupt-only mode that has sufficient room for full interrupt names. > > I don't find static info and thus vmstat -i very useful. vmstat with a > repeat interval is useful, but for some reason vmstat -i never supported > the repeat interval (its multi-line display is certainly unsuitable for > repetitions being readable), and with systat -v repetition in vmstat -i > is rarely missed. A repeating vmstat -i would have even more space > problems than systat -v, since it would want to repeat single line output. For certain tests simply doing a before and after of vmstat -i and figuring out the diff manually (sometimes just having a diff at all is important rather than the number (e.g. did some action I just did generate an interrupt?)) is sufficient. > systat and most things just want a unique tag, not a description or even > a full name. The full name could be looked up from a tag by looking at > vmstat -i output which could show both. This could be interesting perhaps. I would probably want to change my proposed intrcnt structure to contain three things then: a unique tag, a full name, and a count. One possible idea for the unique "tag" would be to use the interrupt handler name if there is only a single handler or to use the 'irqXXX' name if the interrupt is shared. -- John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200910011426.56047.jhb>