From owner-freebsd-arm@FreeBSD.ORG Thu Jul 16 19:56:29 2009 Return-Path: Delivered-To: freebsd-arm@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 498AE1065672; Thu, 16 Jul 2009 19:56:29 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe15.swipnet.se [212.247.155.193]) by mx1.freebsd.org (Postfix) with ESMTP id 0B6488FC12; Thu, 16 Jul 2009 19:56:27 +0000 (UTC) (envelope-from hselasky@c2i.net) X-Cloudmark-Score: 0.000000 [] X-Cloudmark-Analysis: v=1.0 c=1 a=Hrwt8fWgTlIA:10 a=MXw7gxVQKqGXY79tIT8aFQ==:17 a=B_zZn5-Rd8HGBMezT6QA:9 a=JNpn4SdkRBrMK3k_QEYA:7 a=TTBpQAZNaQe9nYvObQmjU1kUzmgA:4 a=n26B4fOH_80_4pcn:21 a=2bjXZ3cnzzwmxv3j:21 Received: from [62.113.132.61] (account mc467741@c2i.net HELO laptop.adsl.tele2.no) by mailfe15.swip.net (CommuniGate Pro SMTP 5.2.13) with ESMTPA id 534963832; Thu, 16 Jul 2009 21:56:26 +0200 From: Hans Petter Selasky To: Rafal Jaworowski Date: Thu, 16 Jul 2009 21:56:04 +0200 User-Agent: KMail/1.11.4 (FreeBSD/8.0-BETA1; KDE/4.2.4; i386; ; ) References: <200906231035.43096.kosmo@semihalf.com> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-2" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200907162156.06598.hselasky@c2i.net> Cc: Marcel Moolenaar , Piotr =?iso-8859-2?q?Zi=EAcik?= , freebsd-usb@freebsd.org, freebsd-arm@freebsd.org, thompsa@freebsd.org Subject: Re: CPU Cache and busdma usage in USB X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to the StrongARM Processor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Jul 2009 19:56:29 -0000 On Tuesday 14 July 2009 19:20:22 Rafal Jaworowski wrote: > >> Please note these problems should be considered as a showstopper > >> for the release since USB is currently broken on at least three ARM > >> platforms in the tree (Marvell). Hi, First of all I'm not able to boot into userland on my ARM board. Maybe I missed something. I'm using a custom built userland image. Trying to mount root from ufs:/dev/md0 warning: no time-of-day clock registered, system time will not be set accurately Jul 16 18:53:22 init: login_getclass: unknown class 'daemon' Fatal kernel mode data abort: 'Alignment Fault 3' trapframe: 0xc622cc60 FSR=00000003, FAR=6e75203a, spsr=600000d3 r0 =60000093, r1 =c10594a0, r2 =c10594a0, r3 =00000002 r4 =c10acbd0, r5 =00000000, r6 =6e75203a, r7 =c10594a0 r8 =c622cd74, r9 =00000000, r10=c0ab081c, r11=c622cccc r12=c622ccac, ssp=c622ccac, slr=c020a144, pc =c00c02c8 [thread pid 4 tid 100008 ] Stopped at _thread_lock_flags+0x24: ldr r4, [r6] db> With the following patch in arm/cpufunc.c my UMASS device gets detected at boot time (KB9202B ARM board). Else I get a USB request timeout followed by a panic. But data transfers of more than 16KByte are likely to be broken. struct cpu_functions arm9_cpufuncs = { ... - /*XXX*/ arm9_dcache_wbinv_range, /* dcache_inv_range */ + /*XXX*/ arm9_dcache_inv_range, /* dcache_inv_range */ }; Looking at the assembly code behind "arm9_dcache_inv_range" in "cpufunc_asm_arm9.S" I see that "arm9_dcache_wbinv_all" gets called if the area to invalidate is greater equal to 16KByte. This is wrong. When asked to invalidate, then no memory must be flushed in that region. Because USB works like this: invalidate (TD) read field from (TD) Next IRQ invalidate (TD) read field from (TD) If the invalidate call flushes any data, you clearly see from the pseudo code above that there is a chance that the last read TD field in the cache gets written back before the next invalidate at the next IRQ, when the invalidate function is implemented like a writeback+invalidate!?? Piotr and Rafal: How does ARM work? Are there two caches? One read cache and one write cache. Or are they the same. What happens if you read a value from RAM into cache, then writeback+invalidate it. Does it get written back to RAM or does it get discarded? Sorry, I have some holes in my ARM knowledge which you need to fill in. Reference: ENTRY(arm9_dcache_inv_range) ldr ip, .Larm9_line_size cmp r1, #0x4000 bcs .Larm9_dcache_wbinv_all Anything unclear? Working version of kernel with my patch having stick plugged in at boot: ugen1.2: at usbus1 umass0: on usbus1 umass0: SCSI over Bulk-Only; quirks = 0x0000 umass0:0:0:-1: Attached to scbus0 (probe0:umass-sim0:0:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0 (probe0:umass-sim0:0:0:0): CAM Status: SCSI Status Error (probe0:umass-sim0:0:0:0): SCSI Status: Check Condition (probe0:umass-sim0:0:0:0): UNIT ATTENTION asc:28,0 (probe0:umass-sim0:0:0:0): Not ready to ready change, medium may have changed (probe0:umass-sim0:0:0:0): Retrying Command (per Sense Data) da0 at umass-sim0 bus 0 target 0 lun 0 da0: Removable Direct Access SCSI-0 device da0: 1.000MB/s transfers da0: 3842MB (7868416 512 byte sectors: 255H 63S/T 489C) Stock 8-current code: ugen1.2: at usbus1 umass0: on usbus1 umass0: SCSI over Bulk-Only; quirks = 0x0000 umass0:0:0:-1: Attached to scbus0 (probe0:umass-sim0:0:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0 (probe0:umass-sim0:0:0:0): CAM Status: SCSI Status Error (probe0:umass-sim0:0:0:0): SCSI Status: Check Condition (probe0:umass-sim0:0:0:0): UNIT ATTENTION asc:28,0 (probe0:umass-sim0:0:0:0): Not ready to ready change, medium may have changed (probe0:umass-sim0:0:0:0): Retrying Command (per Sense Data) da0 at umass-sim0 bus 0 target 0 lun 0 da0: Removable Direct Access SCSI-0 device da0: 1.000MB/s transfers da0: 3842MB (7868416 512 byte sectors: 255H 63S/T 489C) ugen1.2: at usbus1 (disconnected) umass0: at uhub1, port 1, addr 2 (disconnected) (da0:umass-sim0:0:0:0): lost device (da0:umass-sim0:0:0:0): removing device entry ugen1.2: at usbus1 umass0: on usbus1 umass0: SCSI over Bulk-Only; quirks = 0x0000 umass0:0:0:-1: Attached to scbus0 Sleeping thread (tid 100031, pid 19) owns a non-sleepable lock sched_switch() at sched_switch+0x10 scp=0xc00e7830 rlv=0xc00d2228 (mi_switch+0x108) rsp=0xc6271ddc rfp=0xc6271e04 r7=0xc11306f0 r6=0x00000000 r5=0x000000c0 r4=0x00000000 ... umass0: on usbus1 umass0: SCSI over Bulk-Only; quirks = 0x0000 umass0:0:0:-1: Attached to scbus0 (probe0:umass-sim0:0:0:0): AutoSense Failed usb_alloc_device:1586: set address 3 failed (USB_ERR_IOERROR, ignored) usb_alloc_device:1624: getting device descriptor at addr 3 failed, USB_ERR_IOERROR! usbd_req_re_enumerate:1539: addr=3, set address failed! (USB_ERR_IOERROR, ignored) usbd_req_re_enumerate:1553: getting device descriptor at addr 3 failed, USB_ERR_IOERROR! usbd_req_re_enumerate:1539: addr=3, set address failed! (USB_ERR_IOERROR, ignored) usbd_req_re_enumerate:1553: getting device descriptor at addr 3 failed, USB_ERR_IOERROR! ugen1.3: <(null)> at usbus1 (disconnected) uhub_reattach_port:435: could not allocate new device! (da0:umass-sim0:0:0:0): got CAM status 0x4 (da0:umass-sim0:0:0:0): fatal error, failed to attach to device (da0:umass-sim0:0:0:0): lost device (da0:umass-sim0:0:0:0): removing device entry ugen1.2: at usbus1 (disconnected) umass0: at uhub1, port 1, addr 2 (disconnected) ugen1.2: at usbus1 umass0: on usbus1 umass0: SCSI over Bulk-Only; quirks = 0x0000 vm_fault(0xc0ac65f4, 0, 1, 0) -> 1 Fatal kernel mode data abort: 'Translation Fault (S)' trapframe: 0xc6271a38 FSR=00000005, FAR=00000014, spsr=a00000d3 r0 =c0accdcc, r1 =00000000, r2 =c11306f0, r3 =00000004 r4 =00000000, r5 =00000000, r6 =cc05f270, r7 =c1168d78 r8 =c6271b80, r9 =0000000d, r10=00000030, r11=c6271a98 r12=c6271a6c, ssp=c6271a84, slr=c0209ee8, pc =c00fd518 [thread pid 19 tid 100031 ] Stopped at turnstile_broadcast+0x30: ldr r2, [r1, #0x014] Are there any pending patches which I should have put into the code before building? --HPS