From owner-freebsd-drivers@FreeBSD.ORG Sun Feb 8 18:01:16 2009 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E9B71065670 for ; Sun, 8 Feb 2009 18:01:16 +0000 (UTC) (envelope-from jason.harmening@gmail.com) Received: from yw-out-2324.google.com (yw-out-2324.google.com [74.125.46.28]) by mx1.freebsd.org (Postfix) with ESMTP id E43C58FC19 for ; Sun, 8 Feb 2009 18:01:15 +0000 (UTC) (envelope-from jason.harmening@gmail.com) Received: by yw-out-2324.google.com with SMTP id 2so384148ywt.13 for ; Sun, 08 Feb 2009 10:01:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:date:from:to:cc:subject :message-id:x-mailer:mime-version:content-type :content-transfer-encoding; bh=LJm0XtJHXSf+p4If/HX/ORfDlZBUHC8aiAKcF3ZxLf0=; b=C1Eg4H+gjGpjbqgUf3pyAhUj4yxapZWYU2ueqWeRNkVfbVBrfYAAPZMwRiql4eGtBq UYDYvMDSBu3GE4SVWEN6mX0BGQm7xBy6/8ejLkjneywZ8DODRciRZfqBvFJ5TizcRIUj EfM+by47hRbcoaSVWLETNbUXD5SBneXsYPc9A= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:x-mailer:mime-version :content-type:content-transfer-encoding; b=ZiLDXO1G+0whYodQiCC9movkJho7FHGUanmD0FGzVSko93NqNNa+2sMIGKe9T12jNX lW6uDBtXGCUZvaRTsYapW9srwifI/yC+Dq4467987DlUq3wbbxdV4ym4lxYBtex2GPaN HpcHBoxziltDfoYenycBF3yzjHrIziSMON4SY= Received: by 10.100.214.15 with SMTP id m15mr1025940ang.44.1234114691336; Sun, 08 Feb 2009 09:38:11 -0800 (PST) Received: from CORONA ([24.174.5.175]) by mx.google.com with ESMTPS id b29sm8803554ana.51.2009.02.08.09.38.10 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sun, 08 Feb 2009 09:38:10 -0800 (PST) Date: Sun, 8 Feb 2009 11:32:02 -0600 From: Jason Harmening To: freebsd-drivers@freebsd.org Message-ID: <20090208113202.6d591589@CORONA> X-Mailer: Claws Mail 3.5.0 (GTK+ 2.12.11; amd64-portbld-freebsd7.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Subject: PCI-Express interrupt issues X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Feb 2009 18:01:16 -0000 Hi All, I'm the maintainer for the FreeBSD cx88 driver (multimedia/cx88). I've recently encountered some issues w/ interrupt handling, specifically on PCI-Express systems, and I was hoping someone would be able to help me. Issue #1: The cx88 driver has split interrupt handling between filters and ithreads since filters became available w/ 7.0. For the past year, the filters have been set up to return FILTER_SCHEDULE_THREAD if the status register indicates an interrupt, FILTER_STRAY otherwise. Everything has worked fine. However, I recently stumbled across some documentation indicating that FILTER_SCHEDULE_THREAD shouldn't be returned alone--instead, the filter should return FILTER_HANDLED | FILTER_SCHEDULE_THREAD. So I modified the cx88 driver to do that instead, and that's where things turned strange: On machine #1, which uses a VIA K8T800 chipset, everything still worked fine. On machine #2, which uses a VIA K8T890 chipset, the threaded interrupt handlers were no longer invoked. It's as though the bus driver saw FILTER_HANDLED in the bitmask and assumed the interrupt was already processed without checking to see if an ithread should be scheduled. What's interesting is that the only significant difference between the K8T800 in machine #1 and the K8T890 in machine #2 is that the K8T890 supports PCI-Express, while the K8T800 is PCI-only. Issue #2: We're working on adding support for a newer generation of PCI-Express cx88 devices, which support message signaled interrupts. When MSIs are enabled for these boards, the interrupts seem to simply stop firing after the first several. During transfers across the cx88's onboard I2C bus, the first several (e.g. 8-10) bytes will be transferred successfully, which means the I2C transfer interrupts are working for those bytes. But after that the passive thread will timeout waiting to be signaled by the interrupt handler. The I2C interrupts are handled entirely in their own filter (no ithread here), and they can occur with high frequency (e.g. several hundred per second for large firmware loads). We don't see anything in the system log indicating that an interrupt storm is detected or that throttling is coming into play here. If we revert to legacy INTx interrupts, everything works fine. All other transfer logic is identical between the legacy and MSI cases. The MSI behavior is identical between machine #2 w/ the K8T890 and a third machine w/ an nForce4. We haven't been able to try it anywhere else yet. I'm hoping someone will be able to shed some light on these issues--neither one is a real emergency, but they're both kind of troubling. Thanks, Jason