Date: Thu, 21 Jun 2012 18:20:06 +0530 From: Venkat Duvvuru <venkatduvvuru.ml@gmail.com> To: John Baldwin <jhb@freebsd.org> Cc: freebsd-net@freebsd.org Subject: Re: MSI-X limitation in freebsd 8.2 Message-ID: <CAGdae7Yum_MP99kSK1Xi8z02APnZPoXFKR22fTuE5tiuorYMhw@mail.gmail.com> In-Reply-To: <201206210816.22774.jhb@freebsd.org> References: <CAGdae7YW0jt4gTmHufHFTrcdtFGhngu_Wi%2BMDg1nY_T0J1C_SQ@mail.gmail.com> <201206201054.40824.jhb@freebsd.org> <CAGdae7anhqdzacVt-BMv3e7GvKFO4QbYEVxd9k-agGjLfPi_iQ@mail.gmail.com> <201206210816.22774.jhb@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
While I'm able to ping to the machine..every command on the current active session is sluggish infact doesn't complete, unable to open another ssh session..an already opened ssh session doesn't react which is running "systat -vmstat"... The only thing I could do is to successfully complete "top -P" command once..even top hung for the second time. /Venkat On Thu, Jun 21, 2012 at 5:46 PM, John Baldwin <jhb@freebsd.org> wrote: > On Thursday, June 21, 2012 5:47:48 am Venkat Duvvuru wrote: > > John - Thanks for the reply. > > > > All the CPUs are ~100% idle. I don't see any interrupt storm on any of > the > > irqs (vmstat -i). > > > > One observation I made is that I see messages like these in dmesg > > > > ===> mem > 0xfaf60000-0xfaf7ffff,0xfaf40000-0xfaf5ffff,0xfaf1c000-0xfaf1ffff > > irq 40 at device 0.1 on pci6 > > > > Looking at the irq value I think it is the INTx irq range which shouldn't > > have probably got allocated as the device is msix capable and there are > > vectors allocated for these devices in the range (256-380). > > > > Could this be a problem? > > No, that line is output before the driver's attach routine is run, so it > will > always show INTx IRQ value even if it isn't used. > > > The scenario where I am hitting this problem is a setup with 4 NICs, each > > NIC with two ports and each port using up 4 msix vectors. The system is > > fine till some ports are up but once I ifup the 5th port, the system > > becomes sluggish. > > > > I'm not sure whether all the 30 vectors are from a single cpu..I don't > know > > how to get that information. > > Unfortunately there isn't an easy way. I have this gdb script which can > display it from kgdb on x86: > > define irqs > set $e = event_list->tqh_first > while ($e != 0) > if ($e->ie_source != 0 && $e->ie_handlers.tqh_first != 0) > set $src = (struct intsrc *)$e->ie_source > if ($src->is_pic->pic_enable_source == &ioapic_enable_source) > set $_cpu = ((struct ioapic_intsrc *)$src)->io_cpu > else > if ($src->is_pic->pic_enable_source == &msi_enable_source) > set $_cpu = ((struct msi_intsrc *)$src)->msi_cpu > else > set $_cpu = 0 > end > end > printf "CPU %d: %s\n", $_cpu, $e->ie_fullname > end > set $e = $e->ie_list.tqe_next > end > end > > document irqs > Dump list of IRQs with associated CPU. > end > > However, unless the driver is using BUS_BIND_IRQ() or you are using cpuset > -x, > the interrupts should be round-robin assigned among CPUs. > > What exactly do you mean by sluggish? Trying to interact with the box over > SSH is sluggish? Is there a change in RTT if you are pinging the box, is > there a change in performance of TCP or UDP streams to/from the box? > > -- > John Baldwin >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAGdae7Yum_MP99kSK1Xi8z02APnZPoXFKR22fTuE5tiuorYMhw>