From owner-freebsd-arm@freebsd.org Fri Aug 14 11:42:45 2015 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C7C729B8FE6 for ; Fri, 14 Aug 2015 11:42:45 +0000 (UTC) (envelope-from onwahe@gmail.com) Received: from mail-io0-x22b.google.com (mail-io0-x22b.google.com [IPv6:2607:f8b0:4001:c06::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8B0C01A45 for ; Fri, 14 Aug 2015 11:42:45 +0000 (UTC) (envelope-from onwahe@gmail.com) Received: by iods203 with SMTP id s203so81698906iod.0 for ; Fri, 14 Aug 2015 04:42:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=hmkkaO7Lb1/o5uSU2o5Pwg/N2PxKa97DG+aEKi3tFDI=; b=uJ7tpwCwlz3YxuzAw4yx3C7h6qZmEpVaAO3an9O2wj8QUQxm78NbpM74/XTXNpJHHf StUGrV/OvchU32w/EVWDPWskinRdEvUJkONGKCz+46+CkPlHsmJbaf8wFVcoXV/h6Yf8 xgJ/OqeMnRYD04fnnpdn6WVOtmMc15D0QWQjTpUjTy6LaH5lfm8Z23W20DCpnvVp06Ue 2vX0L80m0lQBqxUpYFUd5YXpw+QFylVXxSa3NK/rKJ56cJdjzDzCrR7CIpPF31otciSA /PCWFVtqTocblUhOsFmlmtrQubbaVQbXXoNmNv8znaCK8W0GJnwEYCFObq7nXCp6iot3 hrUA== MIME-Version: 1.0 X-Received: by 10.107.128.83 with SMTP id b80mr14629702iod.84.1439552564937; Fri, 14 Aug 2015 04:42:44 -0700 (PDT) Received: by 10.64.106.106 with HTTP; Fri, 14 Aug 2015 04:42:44 -0700 (PDT) In-Reply-To: <55CDB9BC.5000005@selasky.org> References: <55A7D8CE.4020809@selasky.org> <55B73113.2020308@selasky.org> <55B8AB76.7030603@selasky.org> <55B8B297.1010008@selasky.org> <20150729154516.GH78154@funkthat.com> <55B8F5EC.2050908@selasky.org> <46ad096c958.1a82a175@mail.schwarzes.net> <55B9C3E2.5040501@selasky.org> <46ae815c7c3.447237c8@mail.schwarzes.net> <46aece00b53.3c1cdc1f@mail.schwarzes.net> <55BB2A5F.9000502@selasky.org> <55BB3CC6.4030002@selasky.org> <55BB6A7F.3060402@selasky.org> <55CB4BAF.3050702@selasky.org> <55CDB9BC.5000005@selasky.org> Date: Fri, 14 Aug 2015 13:42:44 +0200 Message-ID: Subject: Re: DWC OTG TX path optimisation for 11-current From: Svatopluk Kraus To: Hans Petter Selasky Cc: "freebsd-arm@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Aug 2015 11:42:45 -0000 On Fri, Aug 14, 2015 at 11:49 AM, Hans Petter Selasky wrote: > On 08/12/15 20:16, Svatopluk Kraus wrote: >> >> On Wed, Aug 12, 2015 at 3:35 PM, Hans Petter Selasky >> wrote: >>> >>> On 07/31/15 17:11, Svatopluk Kraus wrote: >>>> >>>> >>>> Note that I got about 24000 interrupts per a second during buildworld >>>> too. If I remember it correctly, it was about 4000 before r285935. >>>> When the trigger is pulled, the count is changing (in range of 4000 to >>>> 21000). It looks like something is taking more time (interrupt >>>> servicing probably). According to "systat -v", disk is going to be >>>> 100% (and more) busy very often for example. >>> >>> >>> >>> Hi, >>> >>> Finally my RPI-2 arrived :-) >> >> >> >> It's great! ;) I can do some debugging but it's hard to go deeper >> without usb controller specification. >> >> >>> >>> After r285935 we are polling for data more frequently, so the IRQ rate >>> will >>> go up. 3x 8000 IRQs/second would be normal to query the device every >>> 125us, >>> which the EHCI does. >>> >>> Can you give me some more information about your setup, so that I can >>> reproduce it? >> >> >> I use in-tree RPI2 configuration but with INVARIANTS and >> INVARIANT_SUPPORT options disabled. This is crucial as it's hard to >> trigger it with these options enabled. >> >> I use usb disk with three partitions: >> First one is mounted as a root. >> Second one is used for swapping. >> The third one is not mounted. >> >> The disk is connected thru usb hub. >> >> My fstab is empty. >> My kernel is loaded from SD card. >> I use UART as console. >> >> root@raspberry-pi:~ # usbconfig >> ugen0.1: at usbus0, cfg=0 md=HOST spd=HIGH >> (480Mbps) pwr=SAVE (0mA) >> ugen0.2: at usbus0, cfg=0 md=HOST >> spd=HIGH (480Mbps) pwr=SAVE (2mA) >> ugen0.3: at usbus0, cfg=0 md=HOST >> spd=HIGH (480Mbps) pwr=ON (2mA) >> ugen0.4: at usbus0, cfg=0 md=HOST spd=HIGH >> (480Mbps) pwr=SAVE (0mA) >> ugen0.5: at usbus0, cfg=0 md=HOST spd=HIGH >> (480Mbps) pwr=SAVE (0mA) >> ugen0.6: at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON >> (0mA) >> >> >> FreeBSD source tree is on /usr/src and I run native buildworld : "make >> -j6 buildworld" >> I run the buildworld on ssh to keep my console "clean and free" for >> debugging. The trigger is pulled in about 25 minutes, however it's >> more expessive with more time elapsed. >> >> So far it looks that it happens during "stage 2.1: cleaning up the >> object tree" when a lot of small and different disk request are done. >> >> I have tried to get big picture of the problem with KTR option >> enabled, but it's not possible as it's not triggered when all KTR >> classes are compiled in. What works is the following configuration: >> >> options KTR >> options KTR_ENTRIES=65536 >> options KTR_MASK=(KTR_SPARE3) >> options KTR_COMPILE=(KTR_SPARE3) >> >> However, even using of KTR_SPARE3 class for debugging is limited >> considering the trigger. >> >> I will send you more info about my investigation tomorow. >> >> Svata >> > > Hi, > > Here are some more clues. I've found a situation where if one USB class > driver gets too busy, the others gets starved. I have a patch in the works > to implement a round robin mechanism for this. > > In the meanwhile, maybe you can try to: > > cpuset -l 1-3 -p XXX > > Where XXX is pid for the usbus0 processes, typically 13 with my RPI-2. > > The DWC OTG interrupts on CPU #0 and if the usbus0 processes are running > there too, it is suboptimal. Well, I have tried it and it ran fine but it almost blocked buildworld in time when the trigger is pulled usually. According to systat, the disk is almost all time 100% and more busy. Svata