From owner-freebsd-stable@FreeBSD.ORG Wed Sep 27 19:30:46 2006 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B824E16A40F; Wed, 27 Sep 2006 19:30:46 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id 750FB43D5C; Wed, 27 Sep 2006 19:30:45 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from [10.10.3.185] ([165.236.175.187]) (authenticated bits=0) by pooker.samsco.org (8.13.4/8.13.4) with ESMTP id k8RJUM4r020393; Wed, 27 Sep 2006 13:30:27 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <451AD147.9060607@samsco.org> Date: Wed, 27 Sep 2006 13:30:15 -0600 From: Scott Long User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20060206 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Peter Jeremy References: <451A1375.5080202@gneto.com> <20060927071538.GF22229@e-Gitt.NET> <451A4189.5020906@samsco.org> <20060927152824.GJ22229@e-Gitt.NET> <20060927155553.GB14563@icarus.home.lan> <20060927155904.GM22229@e-Gitt.NET> <451AA7B1.5080202@samsco.org> <20060927191402.GB932@turion.vk2pj.dyndns.org> In-Reply-To: <20060927191402.GB932@turion.vk2pj.dyndns.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=3.8 tests=none autolearn=failed version=3.1.1 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on pooker.samsco.org Cc: freebsd-stable@freebsd.org, Oliver Brandmueller , John Baldwin Subject: Re: 6.2 SHOWSTOPPER - em completely unusable on 6.2 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2006 19:30:46 -0000 Peter Jeremy wrote: > On Wed, 2006-Sep-27 10:32:49 -0600, Scott Long wrote: > >>My theory here is that something in the kernel, likely VM/VFS, is >>holding the Giant lock for an inordinate amount of time. > > > In the past (RELENG_5) I've had major problems with syncer delaying > interrupt threads for long periods (I've seen 8msec). See > http://lists.freebsd.org/pipermail/freebsd-stable/2005-February/012346.html > I'm not sure if this is still a problem (but I am still having some > problems which may be caused by excessive interrupt and will be doing > some debugging as I get time). > > >>I have a few ideas on tracking down the root cause, but they are pretty >>pretty painful and slow. > > > In my case, I was fairly certain that the problem I was seeing was > excessive interrupt latency for my driver. The approach I took was to > capture TSC, IRQ number and curproc address in lapic_handle_intr(), > atpic_handle_intr() and at the beginning of my interrupt handler into > a ring buffer. I'd dump the ring buffer into a file using a userland > tool and then post-process the file looking for oddities. In my case, > there was a _very_ high correlation between long latencies and syncer. > If anyone's interested in this approach, I can provide the relevant > code diffs. > Yes, I was thinking about the syncer too, but the timeouts for ethernet interfaces are measured in seconds, not milliseconds. > >>2. Add INTR_FAST shims to the usb and ichsmb drivers so that neither >>uses an ithread. > > > The problem I ran into with this approach was that my interrupt > handler needs to use psignal(9) - which requires PROC_LOCK() which > (AFAIK) isn't allowed in an INTR_FAST handler. You can define a very simple INTR_FAST handler that just disables the interrupt at the device and then schedules a taskqueue to do the real work. This is what I did for if_em, actually. > > It would be useful if our interrupt subsystem allowed both INTR_FAST > and normal interrupt handlers to be defined. If an INTR_FAST handler > is defined then it gets executed and defines whether its associated > interrupt thread handler needs to be triggered. If there's no > INTR_FAST handler then the interrupt thread is always triggered. > This was an SoC2006 project, and I believe it will be committed fairly soon. Scott