Date: Fri, 27 Feb 2009 19:35:59 -0600 From: Robert Noland <rnoland@FreeBSD.org> To: John Baldwin <jhb@freebsd.org> Cc: freebsd-current@freebsd.org, Sergey G Nasonov <snasonov@bcc.ru> Subject: Re: Interrupt stom on cardbus device Message-ID: <1235784959.1289.67.camel@widget.2hip.net> In-Reply-To: <200902271502.37051.jhb@freebsd.org> References: <200902271730.07660.snasonov@bcc.ru> <200902271403.46828.jhb@freebsd.org> <1235761864.1289.54.camel@widget.2hip.net> <200902271502.37051.jhb@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--=-5+2y9fg0Psi3xLSxPUxb Content-Type: multipart/mixed; boundary="=-cGHxln2TL/XYKr1w7YVL" --=-cGHxln2TL/XYKr1w7YVL Content-Type: text/plain Content-Transfer-Encoding: quoted-printable On Fri, 2009-02-27 at 15:02 -0500, John Baldwin wrote: > On Friday 27 February 2009 2:11:04 pm Robert Noland wrote: > > On Fri, 2009-02-27 at 14:03 -0500, John Baldwin wrote: > > > On Friday 27 February 2009 1:50:28 pm Robert Noland wrote: > > > > On Fri, 2009-02-27 at 12:08 -0500, John Baldwin wrote: > > > > > On Friday 27 February 2009 9:30:06 am Sergey G Nasonov wrote: > > > > > > Hello all, > > > > > > I have get an issue after recent kernel recompile. > > > > > > The problem appears after switch from X to text console and bac= k to X11. > > > > > > After that vmstat -i show an interrupt storm on cardbus device= : > > > > > >=20 > > > > > > > vmstat -i > > > > > > interrupt total rate > > > > > > irq1: atkbd0 6483 3 > > > > > > irq9: acpi0 3236 1 > > > > > > irq12: psm0 347988 167 > > > > > > irq14: ata0 16431 7 > > > > > > irq16: cbb0 uhci2+ 13624982 6556 > > > > > > irq20: uhci0 14 0 > > > > > > irq22: ehci0 2 0 > > > > > > cpu0: timer 4154687 1999 > > > > > > irq256: em0 53736 25 > > > > > > irq257: hdac0 5797 2 > > > > > > cpu1: timer 4153683 1998 > > > > > > irq258: vgapci0 235585 113 > > > > > > Total 22602624 10877 > > > > > >=20 > > > > > > I suppose that the issue related with the latest MSI interrupt=20 > > > > > > handler changes for intel graphics chipset. My laptop has i965G= M. > > > > > > pciconf -lv: > > > > > >=20 > > > > > > vgapci0@pci0:0:2:0: class=3D0x030000 card=3D0x20b517aa chip= =3D0x2a028086=20 > > > > > > rev=3D0x0c hdr=3D0x00 > > > > > > vendor =3D 'Intel Corporation' > > > > > > device =3D 'Mobile 965 Express Integrated Graphics Cont= roller' > > > > > > class =3D display > > > > > > subclass =3D VGA > > > > > >=20 > > > > > > When I added my device to drm_msi_blacklist and recompile drm m= odules=20 > > > the=20 > > > > > > problem disappear. > > > > > > Is it possible to resolve this problem without moving the devic= e to the=20 > > > > > > drm_msi_blacklist? > > > > > > I can test any patches or provide additional detail if it is re= quired. =20 > > > > > > Thanks. > > > > >=20 > > > > > It seems the device is still interrupting on its INTx line perhap= s in=20 > > > addition=20 > > > > > to the MSI interrupts. > > > >=20 > > > > Hrm, I did most all of that development on a 965gm. When you VT sw= itch, > > > > the irq handler gets uninstalled and reinstalled when you return to= X. > > > > There was an eratta on the 965gm suggesting that msi didn't work ri= ght, > > > > but I was never able to produce the issue. Intel was having major > > > > issues with this on linux and I finally convinced them to turn msi = back > > > > on. My irq handler and Eric's are very similar, so I'm not sure wh= at > > > > could be going on here. > > > >=20 > > > > There is however an issue with vblanks that might be related. Coul= d you > > > > try http://people.freebsd.org/~rnoland/drm-move_vblank_init.patch a= nd > > > > see if that helps? > > >=20 > > > In this case the issue isn't that MSI isn't working I think, but that= the=20 > > > hardware is sending interrupts via both routes (MSI and INTx). If th= at=20 > > > happens, then you will see an interrupt storm on the INTx line, but F= reeBSD=20 > > > will only notice if another device is sharing the same IRQ line. So = if your=20 > > > test machine has vgapci0 on irq 22 and you have no other devices on I= RQ 22,=20 > > > then the storm would go unnoticed. This is most likely a chip bug (u= nless=20 > > > the driver has to explicitly disable INTx interrupts when using MSI).= It=20 > > > would probably be a good idea to add a hw.drm.msi_enable tunable (or=20 > > > hw.drm.msi) that people can use to disable MSI perhaps. > >=20 > > Ok, I do have docs on the 965, so I'll look at this. The linux version > > does not do this, unless the OS does it in the background somewhere. Ok, so I looked over the 965 docs again and noticed PCIR_COMMAND bit 10. Then I pulled up the AMD docs on their PCIE cards and they also have this bit. I made an test patch for just the i915 driver to ensure that this fixes the issue, but it seems like a more general fix is in order. I'm proposing to disable INTx when we setup MSI/MSIX interrupts. I talked with scottl@ about this a bit last night and this seems like the right thing to do, or at least it shouldn't hurt much... John, what do you think of the attached patch? robert. > Perhaps Eric can help to answer a question about the hardware in this cas= e. >=20 > > I thought about adding a tuneable, but I have to figure out how that > > works first... ;) >=20 > You can basically do something like this: >=20 > int drm_msi =3D 1; /* Enable by default. */ > TUNABLE_INT("hw.drm.msi", &drm_msi); >=20 > And later don't enable msi if it is zero. >=20 --=20 Robert Noland <rnoland@FreeBSD.org> FreeBSD --=-cGHxln2TL/XYKr1w7YVL Content-Disposition: attachment; filename="pci_disable_intx.patch" Content-Transfer-Encoding: base64 Content-Type: text/x-patch; name="pci_disable_intx.patch"; charset="us-ascii" SW5kZXg6IGRldi9wY2kvcGNpLmMNCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0NCi0tLSBkZXYvcGNpL3BjaS5jCShyZXZp c2lvbiAxODkwNDQpDQorKysgZGV2L3BjaS9wY2kuYwkod29ya2luZyBjb3B5KQ0KQEAgLTI4NjQs NiArMjg2NCw4IEBADQogCQkJfQ0KIAkJCW10ZS0+bXRlX2hhbmRsZXJzKys7DQogCQl9DQorCQkv KiBEaXNhYmxlIElOVHggaWYgd2UgYXJlIHVzaW5nIE1TSS9NU0lYICovDQorCQlwY2lfc2V0X2Nv bW1hbmRfYml0KGRldiwgY2hpbGQsIFBDSU1fQ01EX0lOVHhESVMpOw0KIAliYWQ6DQogCQlpZiAo ZXJyb3IpIHsNCiAJCQkodm9pZClidXNfZ2VuZXJpY190ZWFyZG93bl9pbnRyKGRldiwgY2hpbGQs IGlycSwNCkBAIC0yOTE4LDYgKzI5MjAsOCBAQA0KIAkJCWlmIChtdGUtPm10ZV9oYW5kbGVycyA9 PSAwKQ0KIAkJCQlwY2lfbWFza19tc2l4KGNoaWxkLCByaWQgLSAxKTsNCiAJCX0NCisJCS8qIFJl c3RvcmUgSU5UeCBjYXBhYmlsaXR5IGZvciBNU0kvTVNJWCAqLw0KKwkJcGNpX2NsZWFyX2NvbW1h bmRfYml0KGRldiwgY2hpbGQsIFBDSU1fQ01EX0lOVHhESVMpOw0KIAl9DQogCWVycm9yID0gYnVz X2dlbmVyaWNfdGVhcmRvd25faW50cihkZXYsIGNoaWxkLCBpcnEsIGNvb2tpZSk7DQogCWlmIChk ZXZpY2VfZ2V0X3BhcmVudChjaGlsZCkgPT0gZGV2ICYmIHJpZCA+IDApDQpJbmRleDogZGV2L3Bj aS9wY2lyZWcuaA0KPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PQ0KLS0tIGRldi9wY2kvcGNpcmVnLmgJKHJldmlzaW9uIDE4 OTA0NCkNCisrKyBkZXYvcGNpL3BjaXJlZy5oCSh3b3JraW5nIGNvcHkpDQpAQCAtNjAsNiArNjAs NyBAQA0KICNkZWZpbmUJUENJTV9DTURfUEVSUkVTUEVOCTB4MDA0MA0KICNkZWZpbmUJUENJTV9D TURfU0VSUkVTUEVOCTB4MDEwMA0KICNkZWZpbmUJUENJTV9DTURfQkFDS1RPQkFDSwkweDAyMDAN CisjZGVmaW5lCVBDSU1fQ01EX0lOVHhESVMJMHgwNDAwDQogI2RlZmluZQlQQ0lSX1NUQVRVUwkw eDA2DQogI2RlZmluZQlQQ0lNX1NUQVRVU19DQVBQUkVTRU5UCTB4MDAxMA0KICNkZWZpbmUJUENJ TV9TVEFUVVNfNjZDQVBBQkxFCTB4MDAyMA0K --=-cGHxln2TL/XYKr1w7YVL-- --=-5+2y9fg0Psi3xLSxPUxb Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.10 (FreeBSD) iEYEABECAAYFAkmolP8ACgkQM4TrQ4qfROMT/QCfcdtyE+x8KI8yAVc+ZrjgSYW3 5kUAnimDJstxFoFdpof2osSw74YWVyN4 =SOT3 -----END PGP SIGNATURE----- --=-5+2y9fg0Psi3xLSxPUxb--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1235784959.1289.67.camel>