From owner-freebsd-current@FreeBSD.ORG Fri Jun 4 18:30:36 2010 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F0EA106564A for ; Fri, 4 Jun 2010 18:30:36 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.152]) by mx1.freebsd.org (Postfix) with ESMTP id E125B8FC16 for ; Fri, 4 Jun 2010 18:30:35 +0000 (UTC) Received: by fg-out-1718.google.com with SMTP id l26so182244fgb.13 for ; Fri, 04 Jun 2010 11:30:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:subject:x-enigmail-version:content-type :content-transfer-encoding; bh=lXwWEr9rnQcaORekT38Tzk5R1Fq+zekeVTcJ01XQ3uQ=; b=GH8QsX6cSg28Blrp1FP+ocajMxckv8QWYLnj4LzLatpvo2WDKyX/3tzmOXmTNVlqvm elVOcaJGFisY7BeUjQjQKF4tYOWJ+z2aZyPpgjHX8MTPoqdWztYhWcBxnYNEfJeCKxyh jdJ109tvHOkNNqjonNPTn4DFNBhoIpCnQ1pSI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:subject :x-enigmail-version:content-type:content-transfer-encoding; b=VFDQsQtZeVzoBBxaZwOI47PWqEM0xCrVFKDWPfJ0cEn3U9SUxG31uJDWr8TzksYcWD 292YMOE1+jbBcwXwkVYxWW/vTMO6wzVzTZlc5HDZaM/tGEU1KYGxsa/1J4k9IG2o0HHP AyvxMXfGuLpJWEGke9143sCXQ9INmsltGg8lE= Received: by 10.223.147.209 with SMTP id m17mr12215847fav.11.1275676234719; Fri, 04 Jun 2010 11:30:34 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (pc.mavhome.dp.ua [212.86.226.226]) by mx.google.com with ESMTPS id g10sm6434643fai.0.2010.06.04.11.30.33 (version=SSLv3 cipher=RC4-MD5); Fri, 04 Jun 2010 11:30:34 -0700 (PDT) Sender: Alexander Motin Message-ID: <4C094635.40002@FreeBSD.org> Date: Fri, 04 Jun 2010 21:30:13 +0300 From: Alexander Motin User-Agent: Thunderbird 2.0.0.24 (X11/20100402) MIME-Version: 1.0 To: FreeBSD-Current X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit Subject: ioapic_assign_cpu() on active level-triggered interrupt X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jun 2010 18:30:36 -0000 Hi. I am working on driver for HPET event timers. It works mostly fine, except after some cases when ioapic_assign_cpu() called while timer is active. Under interrupt rate of 10KHz it is enough a dozen cpuset runs to break it (with 1KHz - few dozens). When it happens, I can see that timer is still running, interrupt status register is changing, but no interrupts received. Timer uses level-triggered interrupts, so it is tolerant to interrupt losses. I have tried to not acknowledge some, and they have immediately got back to me again, as expected for level-triggering. Timer runs in periodic mode, so it doesn't need handling to continue counting. I have reproduced it on two different i386 SMP systems: Core2Duo+ICH10 and Core i5+PCH. With more experiments I have found that I can't trigger this issue if following patch applied: --- io_apic.c.prev 2010-06-02 10:55:56.000000000 +0300 +++ io_apic.c 2010-06-04 17:45:51.000000000 +0300 @@ -363,7 +366,10 @@ ioapic_assign_cpu(struct intsrc *isrc, u printf(") to lapic %u vector %u\n", intpin->io_cpu, intpin->io_vector); } + ioapic_disable_source(isrc, PIC_NO_EOI); + DELAY(10); ioapic_program_intpin(intpin); + ioapic_enable_source(isrc); /* * Free the old vector after the new one is established. This is done * to prevent races where we could miss an interrupt. It is is almost a hack and 10us is completely experimental. But it looks like changing interrupt's APIC and vector in some moments of interrupt processing may be not a good idea. Can somebody explain this behavior and propose some solution? Have somebody seen it for regular PCI devices? -- Alexander Motin