From owner-freebsd-arm@freebsd.org Sun Sep 11 16:20:36 2016 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 66A5EBD6BDE for ; Sun, 11 Sep 2016 16:20:36 +0000 (UTC) (envelope-from e.moe@rcn.com) Received: from smtp.rcn.com (smtp.rcn.com [69.168.97.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 33BAC17A for ; Sun, 11 Sep 2016 16:20:35 +0000 (UTC) (envelope-from e.moe@rcn.com) X_CMAE_Category: , , X-CNFS-Analysis: v=2.2 cv=Ctr9STwD c=1 sm=1 tr=0 a=dx250bZxW1HngxdpQMIA3g==:117 a=dx250bZxW1HngxdpQMIA3g==:17 a=IkcTkHD0fZMA:10 a=7Qk2ozbKAAAA:8 a=OA2lqS22AAAA:8 a=MblHmfhBUmSBXmIPL6UA:9 a=eCJyFBme0dIDkcO9:21 a=9dHPNxKPubeS_gJt:21 a=QEXdDO2ut3YA:10 a=1lyxoWkJIXJV6VJUPhuM:22 a=047cQLXTEfCjWPUe_kQW:22 X-CM-Score: 0 X-Scanned-by: Cloudmark Authority Engine X-Authed-Username: ZS5tb2VAcmNuLmNvbQ== Authentication-Results: smtp01.rcn.cmh.synacor.com header.from=e.moe@rcn.com; sender-id=neutral Authentication-Results: smtp01.rcn.cmh.synacor.com smtp.mail=e.moe@rcn.com; spf=neutral; sender-id=neutral Authentication-Results: smtp01.rcn.cmh.synacor.com smtp.user=e.moe; auth=pass (PLAIN) Received-SPF: neutral (smtp01.rcn.cmh.synacor.com: 24.148.20.233 is neither permitted nor denied by domain of rcn.com) Received: from [24.148.20.233] ([24.148.20.233:21111] helo=[192.168.3.100]) by smtp.rcn.com (envelope-from ) (ecelerity 3.6.23.54417 r(Core:3.6.23.0)) with ESMTPSA (cipher=DHE-RSA-AES256-SHA) id AD/87-04139-25485D75; Sun, 11 Sep 2016 12:20:34 -0400 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: Unaligned access in ubldr.bin From: Erik Moe In-Reply-To: Date: Sun, 11 Sep 2016 11:20:34 -0500 Cc: =?utf-8?Q?Micha=C5=82_Stanek?= , freebsd-arm Content-Transfer-Encoding: quoted-printable Message-Id: <26CCA10F-E369-402E-B34D-CE66FEDF9746@rcn.com> References: To: Warner Losh X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Sep 2016 16:20:36 -0000 > On Sep 9, 2016, at 10:31 AM, Warner Losh wrote: >=20 > On Thu, Sep 8, 2016 at 5:23 PM, Erik Moe wrote: >>=20 >> On Sep 7, 2016, at 8:22 AM, Warner Losh wrote: >>=20 >> On Wed, Sep 7, 2016 at 12:46 AM, Erik Moe wrote: >>=20 >> Hello, >>=20 >> I=E2=80=99ve been trying to bring up FreeBSD on the USB Armory based = on the i.MX53. >> I=E2=80=99ve made myself a u-boot patch that will load and start = ubldr.bin but I=E2=80=99m >> running into a issue with what I think is an unaligned access issue: >>=20 >>=20 >> which version of u-boot are you using? >>=20 >>=20 >> I was using 2016.07, but also tried u-boot from head (b615267). >=20 > When forward porting the u-boot in the ports tree, I noticed that = 2016.05 worked > w/o changing anything in ubldr and 2016.07 and head from a week ago = both > gave me unaligned accesses. Maybe you could try it as an experiment? = Failed > in exactly the same way on the two boards I tried it on (pandaboard = and BBB). I was going to try 2016.05 as you suggested, but ended up trying a = different experiment instead. What I saw was ubldr.bin getting loaded, = doing it=E2=80=99s self relocation, calling main and then failing in the = getopt call in api_parse_cmdline_sig. Since I wasn=E2=80=99t passing the = api signature on the command line to ubldr and ubldr will fallback to = searching memory anyway for api signature I changed the following in = glue.c: =46rom this: if (!api_parse_cmdline_sig(argc, argv, &sig) && = !api_search_sig(&sig)) To this: if (!api_search_sig(&sig)) That got around the unaligned access issue and ubldr was able to boot = the kernel, though I think I have issues with the dtb that I=E2=80=99m = building: U-Boot 2016.07 (Aug 31 2016 - 04:10:12 -0500) CPU: Freescale i.MX53 rev2.1 at 800 MHz Reset cause: POR Board: Inverse Path USB armory MkI I2C: ready DRAM: 512 MiB WARNING: Caches not enabled MMC: FSL_SDHC: 0 reading u-boot.env ** Unable to read "u-boot.env" from mmc0:1 ** Using default environment In: serial Out: serial Err: serial Net: CPU Net Initialization Failed No ethernet found. starting USB... USB0: USB EHCI 1.00 scanning bus 0 for devices... 1 USB Device(s) found scanning usb for storage devices... 0 Storage Device(s) found Hit any key to stop autoboot: 0 Booting from: mmc 0 ubldr.bin reading ubldr.bin 225796 bytes read in 28 ms (7.7 MiB/s) ## No elf image at address 0x72000000 ## Starting application at 0x70800000 ... Consoles: U-Boot console Compatible U-Boot API signature found @0x8f55a3f0 FreeBSD/armv6 U-Boot loader, Revision 1.2 (root@dora, Thu Sep 8 18:57:27 CDT 2016) DRAM: 512MB MMC Device 1 not found Number of U-Boot devices: 1 U-Boot env: loaderdev=3D'mmc 0' Found U-Boot device: disk Checking unit=3D0 slice=3D partition=3D... good. Booting from disk0s2a: /boot/kernel/kernel data=3D0x5f3b64+0x9449c = syms=3D[0x4+0x84470+0x4+0x98ccc] Hit [Enter] to boot immediately, or any other key for command prompt. Booting [/boot/kernel/kernel]... /boot/dtb/imx53-usbarmory.dtb size=3D0x4590 Loaded DTB from file 'imx53-usbarmory.dtb'. Kernel entry at 0x70a00100... Kernel args: (null) I think the heart of the issue is that when clang builds ubldr it=E2=80=99= s generating code that causing an unaligned access. It=E2=80=99s the = call to getopt in api_parse_cmdline_sig: while ((c =3D getopt (argc, argv, "a:")) !=3D -1) The third parameter =E2=80=9Ca:=E2=80=9D in the .rodata.str1.1 section = is not aligned on a word boundary: [ 3264] U-Boot console [ 3273] a: [ 3276] UBootAPI So when this section of code runs, it gets the unaligned access = exception at address 24ca4: if (optreset || !*place) { /* update scanning = pointer */ optreset =3D 0; if (optind >=3D nargc || *(place =3D nargv[optind]) !=3D = '-') { 24c94: e1510007 cmp r1, r7 24c98: aa000011 bge 24ce4 24c9c: e7965101 ldr r5, [r6, r1, lsl #2] 24ca0: e59f21d0 ldr r2, [pc, #464] ; 24e78 = 24ca4: e5825000 str r5, [r2] 24ca8: e5d53000 ldrb r3, [r5] 24cac: e353002d cmp r3, #45 ; 0x2d That=E2=80=99s my theory anyway since r2 contains the address of the = constant =E2=80=9Ca:=E2=80=9D, which ends of being 70833cd9. If I=E2=80=99= m right I=E2=80=99m not sure how to fix this. >=20 >> ## Starting application at 0x70800000 ... >> data abort >> pc : [<70824bac>] lr : [<7081846c>] >> sp : 8f550c98 ip : 70835a00 fp : 8f550cb0 >> r10: 00000002 r9 : 70838d58 r8 : 70833cd9 >> r7 : 707fff08 r6 : 000054f0 r5 : 70833cd9 r4 : 00000000 >> r3 : 70828fc4 r2 : 70833cd9 r1 : 00000001 r0 : 7083705c >> Flags: Nzcv IRQs off FIQs off Mode SVC_32 >> Resetting CPU ... >>=20 >> I=E2=80=99ve hand dissambled the code at pc=3D0x70824bac: >>=20 >> 0x70824ba8: 0xe59f21d0 ldr r2, [pc, #464] >> 0x70824bac: 0xe5825000 str r5, [r2] >> 0x70824bb0: 0xe5d53000 ldrb r3, [r5] >> 0x70824bb4: 0xe353002d cmp r3, #45 >> 0x70824bb8: 0x1a00000b bne #+44 >>=20 >> The offending instructions is "str r5, [r2]=E2=80=9D where r2 =3D = 0x70833cd9, which >> sort of makes sense since it isn=E2=80=99t aligned on a 4 byte = boundary. I=E2=80=99m new to >> arm ARM, so I=E2=80=99m not really sure. My questions are these: >>=20 >> 1.) Doesn=E2=80=99t ARMv6 and higher architecture allow unaligned = access? Is there >> something that u-boot needs be doing in initialization to allow = unaligned >> access? >>=20 >>=20 >> It does, but usually that's configured later in boot. >>=20 >> 2.) Does ubldr make the assumption that unaligned access is allowed = and >> maybe shouldn=E2=80=99t? I would think not since ubldr has been = around for a while >> and works on numerous ARM processors. >>=20 >>=20 >> It may be that the compiler is generating bad code in this case? You >> might check to see how we're building it and to see if that's causing >> problems. >>=20 >>=20 >> Per Michal Stanek=E2=80=99s suggestion I tried adding = "-mno-unaligned-access=E2=80=9D to the >> build of ubldr. It=E2=80=99s still failing in getopt, though not it = the same spot: >=20 > How about adding it to the ubldr build? IIRC, it should be default in > FreeBSD's clang though. I did try adding -mno-unaligned-access to the build of ubldr, but still = got the unaligned access issue. The patch I used is attached. >=20 >> ## Starting application at 0x70800000 ... >> data abort >> pc : [<70824bd0>] lr : [<70818454>] >> sp : 8f559a78 ip : 70835a90 fp : 8f559a90 >> r10: 00000002 r9 : 70838df0 r8 : 70833d73 >> r7 : 707fff08 r6 : 00005530 r5 : 70833d73 r4 : 00000000 >> r3 : 70828fe8 r2 : 70833d73 r1 : 00000001 r0 : 708370ec >> Flags: Nzcv IRQs off FIQs off Mode SVC_32 >> Resetting CPU ... >>=20 >> 0x70824bcc 0xe59f21d0 ldr r2, [pc, #464] >> 0x70824bd0 0xe5825000 str r5, [r2] >> 0x70824bd4 0xe5d53000 ldrb r3, [r5] >> 0x70824bd8 0xe353002d cmp r3, #45 >>=20 >> 70833d72: 00 61 3a 00 55 42 6f 6f 74 41 50 49 00 49 44 = .a:.UBootAPI.ID >>=20 >> r2 contains 70833d73 which points to =E2=80=9C0x61 0x3a 0x00=E2=80=9D, = which is the literal >> string =E2=80=9Ca:=E2=80=9D. Looking at the lr register the calling = function is >> =E2=80=9Capi_parse_cmdline_sig=E2=80=9D. Looking at the code: >>=20 >> api_parse_cmdline_sig(int argc, char **argv, struct api_signature = **sig) >> { >> unsigned long api_address; >> int c; >>=20 >> api_address =3D 0; >> opterr =3D 0; >> optreset =3D 1; >> optind =3D 1; >>=20 >> while ((c =3D getopt (argc, argv, "a:")) !=3D -1) >> switch (c) { >> case 'a': >> api_address =3D strtoul(optarg, NULL, 16); >> break; >> default: >> break; >> } >>=20 >> It=E2=80=99s the third argument to getopt that seems to be on an odd = address. As >> for the SCTLR.A bit being enabled, that is definitely something = u-boot does, >> because the default state is for it to be cleared after reset. I see = this >> in arch/arm/cpu/armv7/start.S: >>=20 >> /* >> * disable MMU stuff and caches >> */ >> mrc p15, 0, r0, c1, c0, 0 >> bic r0, r0, #0x00002000 @ clear bits 13 (--V-) >> bic r0, r0, #0x00000007 @ clear bits 2:0 (-CAM) >> orr r0, r0, #0x00000002 @ set bit 1 (--A-) Align >> orr r0, r0, #0x00000800 @ set bit 11 (Z---) BTB >> #ifdef CONFIG_SYS_ICACHE_OFF >> bic r0, r0, #0x00001000 @ clear bit 12 (I) I-cache >> #else >> orr r0, r0, #0x00001000 @ set bit 12 (I) I-cache >> #endif >> mcr p15, 0, r0, c1, c0, 0 >>=20 >> I tried to clear that flag, but that didn=E2=80=99t work either, but = I=E2=80=99m not sure if >> this is correct: >>=20 >> mac p15, 0, r0, c1, c0, 0 >> bic r0, r0, #0x00000002 >> mar p15, 0, r0, c1, c0, 0 >=20 > That looks like it should do the right thing. You could just remove > the orr line that ors in bit 1 too. Have you tried that? >=20 > Warner Erik