From owner-freebsd-stable@FreeBSD.ORG Fri Jun 10 10:02:14 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1716C106564A for ; Fri, 10 Jun 2011 10:02:14 +0000 (UTC) (envelope-from bartosz.stec@it4pro.pl) Received: from mainframe.kkip.pl (kkip.pl [87.105.164.78]) by mx1.freebsd.org (Postfix) with ESMTP id 3408F8FC15 for ; Fri, 10 Jun 2011 10:02:12 +0000 (UTC) Received: from mb01.admin.lan.kkip.pl ([10.66.3.0]) by mainframe.kkip.pl with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1QUyXR-0009jc-Fb; Fri, 10 Jun 2011 12:02:09 +0200 Message-ID: <4DF1EB98.9080109@it4pro.pl> Date: Fri, 10 Jun 2011 12:02:00 +0200 From: Bartosz Stec Organization: IT4Pro User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9.2.17) Gecko/20110414 Lightning/1.0b2 Thunderbird/3.1.10 MIME-Version: 1.0 To: Jeremy Chadwick References: <4DC6A277.4030801@it4pro.pl> <4DC6E23B.2040207@it4pro.pl> <4DC81E22.5030806@it4pro.pl> <4DCC3844.6070008@it4pro.pl> <4DCD1672.4020405@FreeBSD.org> <4DF1DC26.5010600@it4pro.pl> <20110610093728.GA39453@icarus.home.lan> In-Reply-To: <20110610093728.GA39453@icarus.home.lan> X-Authenticated-User: bartosz.stec@it4pro.pl X-Authenticator: plain X-Sender-Verify: SUCCEEDED (sender exists & accepts mail) X-Spam-Score: -8.1 X-Spam-Score-Int: -80 X-Exim-Version: 4.76 (build at 12-May-2011 10:41:54) X-Date: 2011-06-10 12:02:09 X-Connected-IP: 10.66.3.0:1511 X-Message-Linecount: 704 X-Body-Linecount: 687 X-Message-Size: 36654 X-Body-Size: 35717 X-Received-Count: 1 X-Recipient-Count: 3 X-Local-Recipient-Count: 3 X-Local-Recipient-Defer-Count: 0 X-Local-Recipient-Fail-Count: 0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: FreeBSD Stable , "Vogel, Jack" Subject: Re: Panic during kernel booting on HP Proliant DL180G6 and latest STABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Jun 2011 10:02:14 -0000 W dniu 2011-06-10 11:37, Jeremy Chadwick pisze: > On Fri, Jun 10, 2011 at 10:56:06AM +0200, Bartosz Stec wrote: >> W dniu 2011-05-13 13:30, Andriy Gapon pisze: >>> on 12/05/2011 22:43 Bartosz Stec said the following: >>>> Allright, if anyone want to follow: http://www.freebsd.org/cgi/query-pr.cgi?pr=156974 >>>> >>> I suspect that your best course here is to add some diagnostic printfs in >>> igb_refresh_mbufs and in m_getzone to see what value is actually passed to >>> m_getjcl and why. >>> >> You're probably right, although I would need some guidance how to do >> it. Sorry :( >> >> Yesterday I rebuilt this machine with fresh sources to upgrade >> filesystem to ZFSv28 which was MFC'ed lately. After that server >> started to panic regardless of igb loaded as module or integrated >> into kernel. >> >> Finally I found that problem was caused by MSIX somehow. >> No more panic after >> >> hw.pci.enable_msix=0 >> >> was included in loader.conf. >> >> There's also no panic with GENERIC kernel. >> >> Should I really care abou MSIX? > Yes, you absolutely should. The performance hit, depending on your > interrupt load, can be quite dramatic with this turned off. Remember: > what you just turned off was MSI-X for the *entire system*, not just for > NICs. Please read about it here: > > http://en.wikipedia.org/wiki/Message_Signaled_Interrupts > > CC'ing Jack Vogel here, because your issue pertains to igb(4). Please > provide output from "pciconv -lvcb" that pertain to your igb NIC(s), as > well as relevant dmesg info that pertains to them (driver, etc.). You > can XXX out MAC addresses if you're worried. > > Jack, original thread starts here: > > http://lists.freebsd.org/pipermail/freebsd-stable/2011-May/thread.html#62596 > Thanks Jeremy, you are helpful as always :) Full dmesg output (was already included in this thread): http://pastebin.com/Es0CKD64 #pciconf -lcvb hostb0@pci0:0:0:0: class=0x060000 card=0x330b103c chip=0x34068086 rev=0x13 hdr=0x00 vendor = 'Intel Corporation' device = 'QuickPath Architecture I/O Hub to ESI Port' class = bridge subclass = HOST-PCI cap 05[60] = MSI supports 2 messages, vector masks cap 10[90] = PCI-Express 2 root port max data 128(128) link x4(x4) cap 01[e0] = powerspec 3 supports D0 D3 current D0 ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected ecap 000d[150] = unknown 1 ecap 000b[160] = unknown 0 pcib1@pci0:0:1:0: class=0x060400 card=0x330b103c chip=0x34088086 rev=0x13 hdr=0x01 vendor = 'Intel Corporation' device = 'QuickPath Architecture I/O Hub PCI Express Root Port 1' class = bridge subclass = PCI-PCI cap 0d[40] = PCI Bridge card=0x330b103c cap 05[60] = MSI supports 2 messages, vector masks cap 10[90] = PCI-Express 2 root port max data 256(256) link x4(x4) cap 01[e0] = powerspec 3 supports D0 D3 current D0 ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected ecap 000d[150] = unknown 1 ecap 000b[160] = unknown 0 pcib2@pci0:0:3:0: class=0x060400 card=0x330b103c chip=0x340a8086 rev=0x13 hdr=0x01 vendor = 'Intel Corporation' device = 'QuickPath Architecture I/O Hub PCI Express Root Port 3' class = bridge subclass = PCI-PCI cap 0d[40] = PCI Bridge card=0x330b103c cap 05[60] = MSI supports 2 messages, vector masks cap 10[90] = PCI-Express 2 root port max data 256(256) link x0(x16) cap 01[e0] = powerspec 3 supports D0 D3 current D0 ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected ecap 000d[150] = unknown 1 ecap 000b[160] = unknown 0 pcib3@pci0:0:7:0: class=0x060400 card=0x330b103c chip=0x340e8086 rev=0x13 hdr=0x01 vendor = 'Intel Corporation' device = 'QuickPath Architecture I/O Hub PCI Express Root Port 7' class = bridge subclass = PCI-PCI cap 0d[40] = PCI Bridge card=0x330b103c cap 05[60] = MSI supports 2 messages, vector masks cap 10[90] = PCI-Express 2 root port max data 256(256) link x0(x16) cap 01[e0] = powerspec 3 supports D0 D3 current D0 ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected ecap 000d[150] = unknown 1 ecap 000b[160] = unknown 0 pcib4@pci0:0:9:0: class=0x060400 card=0x330b103c chip=0x34108086 rev=0x13 hdr=0x01 vendor = 'Intel Corporation' device = 'QuickPath Architecture I/O Hub PCI Express Root Port 9' class = bridge subclass = PCI-PCI cap 0d[40] = PCI Bridge card=0x330b103c cap 05[60] = MSI supports 2 messages, vector masks cap 10[90] = PCI-Express 2 root port max data 256(256) link x0(x4) cap 01[e0] = powerspec 3 supports D0 D3 current D0 ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected ecap 000d[150] = unknown 1 none0@pci0:0:20:0: class=0x080000 card=0x000b003c chip=0x342e8086 rev=0x13 hdr=0x00 vendor = 'Intel Corporation' device = 'QuickPath Architecture I/O Hub System Management Registers' class = base peripheral subclass = interrupt controller cap 10[40] = PCI-Express 2 root endpoint max data 128(128) link x0(x0) none1@pci0:0:20:1: class=0x080000 card=0x000b003c chip=0x34228086 rev=0x13 hdr=0x00 vendor = 'Intel Corporation' device = 'QuickPath Architecture I/O Hub GPIO and Scratch Pad Registers' class = base peripheral subclass = interrupt controller cap 10[40] = PCI-Express 2 root endpoint max data 128(128) link x0(x0) none2@pci0:0:20:2: class=0x080000 card=0x000b003c chip=0x34238086 rev=0x13 hdr=0x00 vendor = 'Intel Corporation' device = 'QuickPath Architecture I/O Hub Control Status and RAS Registers' class = base peripheral subclass = interrupt controller cap 10[40] = PCI-Express 2 root endpoint max data 128(128) link x0(x0) uhci0@pci0:0:26:0: class=0x0c0300 card=0x330b103c chip=0x3a378086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'USB UHCI Controller *4' class = serial bus subclass = USB bar [20] = type I/O Port, range 32, base 0xc800, size 32, enabled cap 13[50] = PCI Advanced Features: FLR TP ehci0@pci0:0:26:7: class=0x0c0320 card=0x330b103c chip=0x3a3c8086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'USB EHCI Controller *2' class = serial bus subclass = USB bar [10] = type Memory, range 32, base 0xfaff8000, size 1024, enabled cap 01[50] = powerspec 2 supports D0 D3 current D0 cap 0a[58] = EHCI Debug Port at offset 0xa0 in map 0x14 cap 13[98] = PCI Advanced Features: FLR TP pcib5@pci0:0:28:0: class=0x060400 card=0x330b103c chip=0x3a408086 rev=0x00 hdr=0x01 vendor = 'Intel Corporation' device = 'PCI Express Port 1' class = bridge subclass = PCI-PCI cap 10[40] = PCI-Express 1 root port max data 128(128) link x0(x4) cap 05[80] = MSI supports 1 message cap 0d[90] = PCI Bridge card=0x330b103c cap 01[a0] = powerspec 2 supports D0 D3 current D0 ecap 0002[100] = VC 1 max VC0 ecap 0005[180] = unknown 1 pcib6@pci0:0:28:4: class=0x060400 card=0x330b103c chip=0x3a488086 rev=0x00 hdr=0x01 vendor = 'Intel Corporation' device = 'PCI Express Port 5' class = bridge subclass = PCI-PCI cap 10[40] = PCI-Express 1 root port max data 128(128) link x1(x2) cap 05[80] = MSI supports 1 message cap 0d[90] = PCI Bridge card=0x330b103c cap 01[a0] = powerspec 2 supports D0 D3 current D0 ecap 0002[100] = VC 1 max VC0 ecap 0005[180] = unknown 1 uhci1@pci0:0:29:0: class=0x0c0300 card=0x330b103c chip=0x3a348086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'USB UHCI Controller *1' class = serial bus subclass = USB bar [20] = type I/O Port, range 32, base 0xc880, size 32, enabled cap 13[50] = PCI Advanced Features: FLR TP uhci2@pci0:0:29:1: class=0x0c0300 card=0x330b103c chip=0x3a358086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'USB UHCI Controller *2' class = serial bus subclass = USB bar [20] = type I/O Port, range 32, base 0xcc00, size 32, enabled cap 13[50] = PCI Advanced Features: FLR TP uhci3@pci0:0:29:2: class=0x0c0300 card=0x330b103c chip=0x3a368086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'USB UHCI Controller *3' class = serial bus subclass = USB bar [20] = type I/O Port, range 32, base 0xd000, size 32, enabled cap 13[50] = PCI Advanced Features: FLR TP ehci1@pci0:0:29:7: class=0x0c0320 card=0x330b103c chip=0x3a3a8086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'USB EHCI Controller *1' class = serial bus subclass = USB bar [10] = type Memory, range 32, base 0xfaffa000, size 1024, enabled cap 01[50] = powerspec 2 supports D0 D3 current D0 cap 0a[58] = EHCI Debug Port at offset 0xa0 in map 0x14 cap 13[98] = PCI Advanced Features: FLR TP pcib7@pci0:0:30:0: class=0x060401 card=0x330b103c chip=0x244e8086 rev=0x90 hdr=0x01 vendor = 'Intel Corporation' device = '82801 Family (ICH2/3/4/5/6/7/8/9,63xxESB) Hub Interface to PCI Bridge' class = bridge subclass = PCI-PCI cap 0d[50] = PCI Bridge card=0x330b103c isab0@pci0:0:31:0: class=0x060100 card=0x330b103c chip=0x3a168086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'LPC Interface Controller' class = bridge subclass = PCI-ISA cap 09[e0] = vendor (length 12) Intel cap 1 version 0 features: SATA RAID-5, 4 PCI-e x1 slots atapci0@pci0:0:31:2: class=0x010601 card=0x330b103c chip=0x3a228086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = '6 port SATA AHCI Controller' class = mass storage subclass = SATA bar [10] = type I/O Port, range 32, base 0xd880, size 8, enabled bar [14] = type I/O Port, range 32, base 0xd800, size 4, enabled bar [18] = type I/O Port, range 32, base 0xd480, size 8, enabled bar [1c] = type I/O Port, range 32, base 0xd400, size 4, enabled bar [20] = type I/O Port, range 32, base 0xd080, size 32, enabled bar [24] = type Memory, range 32, base 0xfaffc000, size 2048, enabled cap 05[80] = MSI supports 16 messages cap 01[70] = powerspec 3 supports D0 D3 current D0 cap 12[a8] = SATA Index-Data Pair cap 13[b0] = PCI Advanced Features: FLR TP igb0@pci0:7:0:0: class=0x020000 card=0x323f103c chip=0x10c98086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' class = network subclass = ethernet bar [10] = type Memory, range 32, base 0xfbe60000, size 131072, enabled bar [14] = type Memory, range 32, base 0xfbe40000, size 131072, enabled bar [18] = type I/O Port, range 32, base 0xe880, size 32, enabled cap 01[40] = powerspec 3 supports D0 D3 current D0 cap 05[50] = MSI supports 1 message, 64 bit, vector masks enabled with 1 message cap 11[70] = MSI-X supports 10 messages in map 0x1c cap 10[a0] = PCI-Express 2 endpoint max data 256(512) link x4(x4) ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected ecap 0003[140] = Serial 1 d8d385ffffe3e4f6 ecap 000e[150] = unknown 1 ecap 0010[160] = unknown 1 igb1@pci0:7:0:1: class=0x020000 card=0x323f103c chip=0x10c98086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' class = network subclass = ethernet bar [10] = type Memory, range 32, base 0xfbee0000, size 131072, enabled bar [14] = type Memory, range 32, base 0xfbec0000, size 131072, enabled bar [18] = type I/O Port, range 32, base 0xec00, size 32, enabled cap 01[40] = powerspec 3 supports D0 D3 current D0 cap 05[50] = MSI supports 1 message, 64 bit, vector masks enabled with 1 message cap 11[70] = MSI-X supports 10 messages in map 0x1c cap 10[a0] = PCI-Express 2 endpoint max data 256(512) link x4(x4) ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected ecap 0003[140] = Serial 1 d8d385ffffe3e4f6 ecap 000e[150] = unknown 1 ecap 0010[160] = unknown 1 vgapci0@pci0:2:0:0: class=0x030000 card=0x010019a2 chip=0x0522102b rev=0x02 hdr=0x00 vendor = 'Matrox Electronic Systems Ltd.' device = 'Matrox G200e (ServerEngines) - English (G200e)' class = display subclass = VGA bar [10] = type Prefetchable Memory, range 32, base 0xf8000000, size 16777216, enabled bar [14] = type Memory, range 32, base 0xfbdfc000, size 16384, enabled bar [18] = type Memory, range 32, base 0xfb000000, size 8388608, enabled cap 01[dc] = powerspec 2 supports D0 D3 current D0 cap 10[e4] = PCI-Express 1 legacy endpoint max data 128(128) link x1(x1) cap 05[54] = MSI supports 1 message -- Bartosz Stec