From owner-freebsd-arm@freebsd.org Fri Aug 14 11:45:06 2015 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4575099F02C for ; Fri, 14 Aug 2015 11:45:06 +0000 (UTC) (envelope-from onwahe@gmail.com) Received: from mail-ig0-x22e.google.com (mail-ig0-x22e.google.com [IPv6:2607:f8b0:4001:c05::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 03E101A9F for ; Fri, 14 Aug 2015 11:45:06 +0000 (UTC) (envelope-from onwahe@gmail.com) Received: by igfj19 with SMTP id j19so10506742igf.0 for ; Fri, 14 Aug 2015 04:45:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=EqsnqIXpeUi21tQXAyGm8X+Grl2V6FfOmTB8n/qI1Fc=; b=RxuUPzSON9qkDsf/0ZtJITPGiZOJgBi7ieaOdFWT3ezxNuLbctyGEE9sRyFnimJx9W cS4xsk4OeRM3PjNlQiyYcSyVssEO93pUVCr6q5Dr5f1XdrqlTICOM3fg/x8iK6wAjy0G CbmsMqkNSMnnPPWYNJ3MsKUkSm6FV2cXUZ+vmYBeoRezZ7EwMcKiA6qeZI1ltV/Lap6t 9wkbKkxzJ8GuniOG+t8325FPH8NmzrrCbKcqrObNYOlUcmv1rhPKxzi+BRC9o50iIS9J 7OCKohJV8UjUeU3wlzeoXcavKMO6TIA208rs89QyY1B6yrZIeGZHYow1zCT2OeRWJZ/u X9Ng== MIME-Version: 1.0 X-Received: by 10.50.164.131 with SMTP id yq3mr1970892igb.26.1439552705349; Fri, 14 Aug 2015 04:45:05 -0700 (PDT) Received: by 10.64.106.106 with HTTP; Fri, 14 Aug 2015 04:45:05 -0700 (PDT) In-Reply-To: References: <55A7D8CE.4020809@selasky.org> <55B23276.8090703@selasky.org> <55B73113.2020308@selasky.org> <55B8AB76.7030603@selasky.org> <55B8B297.1010008@selasky.org> <20150729154516.GH78154@funkthat.com> <55B8F5EC.2050908@selasky.org> <46ad096c958.1a82a175@mail.schwarzes.net> <55B9C3E2.5040501@selasky.org> <46ae815c7c3.447237c8@mail.schwarzes.net> <46aece00b53.3c1cdc1f@mail.schwarzes.net> <55BB2A5F.9000502@selasky.org> <55BB3CC6.4030002@selasky.org> <55BB6A7F.3060402@selasky.org> <55CB4BAF.3050702@selasky.org> Date: Fri, 14 Aug 2015 13:45:05 +0200 Message-ID: Subject: Re: DWC OTG TX path optimisation for 11-current From: Svatopluk Kraus To: Hans Petter Selasky Cc: "freebsd-arm@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Aug 2015 11:45:06 -0000 On Wed, Aug 12, 2015 at 8:16 PM, Svatopluk Kraus wrote: > On Wed, Aug 12, 2015 at 3:35 PM, Hans Petter Selasky wrote: >> On 07/31/15 17:11, Svatopluk Kraus wrote: >>> >>> Note that I got about 24000 interrupts per a second during buildworld >>> too. If I remember it correctly, it was about 4000 before r285935. >>> When the trigger is pulled, the count is changing (in range of 4000 to >>> 21000). It looks like something is taking more time (interrupt >>> servicing probably). According to "systat -v", disk is going to be >>> 100% (and more) busy very often for example. >> >> >> Hi, >> >> Finally my RPI-2 arrived :-) > > > It's great! ;) I can do some debugging but it's hard to go deeper > without usb controller specification. > > >> >> After r285935 we are polling for data more frequently, so the IRQ rate will >> go up. 3x 8000 IRQs/second would be normal to query the device every 125us, >> which the EHCI does. >> >> Can you give me some more information about your setup, so that I can >> reproduce it? > > I use in-tree RPI2 configuration but with INVARIANTS and > INVARIANT_SUPPORT options disabled. This is crucial as it's hard to > trigger it with these options enabled. > > I use usb disk with three partitions: > First one is mounted as a root. > Second one is used for swapping. > The third one is not mounted. > > The disk is connected thru usb hub. > > My fstab is empty. > My kernel is loaded from SD card. > I use UART as console. > > root@raspberry-pi:~ # usbconfig > ugen0.1: at usbus0, cfg=0 md=HOST spd=HIGH > (480Mbps) pwr=SAVE (0mA) > ugen0.2: at usbus0, cfg=0 md=HOST > spd=HIGH (480Mbps) pwr=SAVE (2mA) > ugen0.3: at usbus0, cfg=0 md=HOST > spd=HIGH (480Mbps) pwr=ON (2mA) > ugen0.4: at usbus0, cfg=0 md=HOST spd=HIGH > (480Mbps) pwr=SAVE (0mA) > ugen0.5: at usbus0, cfg=0 md=HOST spd=HIGH > (480Mbps) pwr=SAVE (0mA) > ugen0.6: at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (0mA) > > > FreeBSD source tree is on /usr/src and I run native buildworld : "make > -j6 buildworld" > I run the buildworld on ssh to keep my console "clean and free" for > debugging. The trigger is pulled in about 25 minutes, however it's > more expessive with more time elapsed. > > So far it looks that it happens during "stage 2.1: cleaning up the > object tree" when a lot of small and different disk request are done. > > I have tried to get big picture of the problem with KTR option > enabled, but it's not possible as it's not triggered when all KTR > classes are compiled in. What works is the following configuration: > > options KTR > options KTR_ENTRIES=65536 > options KTR_MASK=(KTR_SPARE3) > options KTR_COMPILE=(KTR_SPARE3) > > However, even using of KTR_SPARE3 class for debugging is limited > considering the trigger. > > I will send you more info about my investigation tomorow. > Well, I have made a decision finally to not influence you by my own investigation for now, especially when I have nothing solid yet. Important thing is if you are able to trigger it. On the other hand ... (1) Triggered or not, ACCORDING to KTR I have noticed that time elapsed between two SOF interrupts (no other interrupt between them) is sometimes much smaller (in long intervals) then it should be. It was even 10 times smaller than 125 microsecond. In these moments, system is 60% and more of time in interrupt. Maybe the intervals, when system is in this state, are much longer if the trigger is pulled. Note that this is quite confusing as a number of interrupts per a second is about 24000 and stable when system is okay. When system is bad, the count varies from let's say 4000 to 21000. The interrupt processing time looks same in both cases. It goes so far that I blame CP15 monitor counter which measures time in ktr. ;) Anyway, the relative intervals should be correct, so true is that system stays in interrupt routine for long time. (2) I made a test and set my root filesystem on SD card while "make buildworld" was done on usb disk. It took much longer to trigger it, however, even full reset of usb controller (which I was able to do in this configuration and which worked) did not help as system remained in its bad state. Svata