Date: Tue, 22 Dec 2015 16:23:51 +0100 From: Bartosz Szczepanek <bsz@semihalf.com> To: freebsd-arm@freebsd.org, skra@freebsd.org Cc: Marcin Wojtas <mw@semihalf.com> Subject: Translation Fault (L1) while using pmap-v6-new Message-ID: <CABLO=%2BkGzcnOLJ=_X8EfNw-YcBL%2BOFhf9G8YQb5RRasNLH9C_Q@mail.gmail.com>
index | next in thread | raw e-mail
[-- Attachment #1 --] Hello, currently I'm working on support for Armada38x on FreeBSD-CURRENT (patchset was submitted to Phabricator - https://reviews.freebsd.org/D4210). After switching to ARM_NEW_PMAP problems related with PCIe subsystem emerged, even though that worked fine on FreeBSD-10.2. My setup consists of Marvell Armada38x GP development board equipped with Cortex-A9, PCIe controller serviced by arm/mv/mv_pci.c driver and RealTek GE PCI card (re driver). Enabling ARM_NEW_PMAP leads to 'Translation Fault (L1)' on write: > Starting Network: lo0 re0. > lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 > options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> > re0: flags=8802<FataBROADCAST,SIMPLEl keX,MULTICAST> metrnelric 0 mtu 1500 > mod options=8209b<Re daXCSUM,TXCSUM,VLAta > aN_MTU,VLAN_HWTAGbortGING,VLAN_HWCSUM: 'T,WOL_MAGIC,LINKSransTATE> > on F ether 64lati:70:02:10:f7:20 > ault (L1)' on write > trapframe: 0xefe78b40 > FSR=00000805, FAR=80000060, spsr=60000013 > r0 =c0e796c0, r1 =80000000, r2 =00000004, r3 =00010000 > r4 =c57f1000, r5 =c57f1000, r6 =00000000, r7 =00000001 > r8 =c0e47e8c, r9 =c5be3780, r10=c57efb00, r11=efe78be0 > r12=efe78d43, ssp=efe78bd0, slr=c0971ef4, pc =c0971f64 > [ thread pid 241 tid 100068 ] > Stopped at re_gmii_readreg+0x50: str r3, [r1, #0x060] > db> (re_gmii_readreg is function in re driver, I made it non-static so it is visible in debugger) Address it crashes on lies in the PCI devices' memory range, and it was accessed successfully several times during boot proccess before crash (I put printfs in the exact function where fault occurs). So it seems just like the memory mapping has disappeared at some point. I put kdb_enter in re attach function (long before translation fault), from that point I see that 0x80000000 mapping exists: > pcib0: <Marvell Integrated PCI/PCI-E Controller> mem 0xf1080000-0xf1081fff irq 1 on ofwbus0 > [ thread pid 0 tid 100000 ] > Stopped at kdb_enter+0x58: ldrb r15, [r15, r15, ror r15]! > db> show pmap > pmap: 0xC0EAC544 > PT2MAP: 0xBFC00000 > pt2tab: 0xC0F04000 > 0x80000000: Section 0x8001041A, s:1 g:1 > 0x80100000: Section 0x8011041A, s:1 g:1 > 0x80200000: Section 0x8021041A, s:1 g:1 > 0x80300000: Section 0x8031041A, s:1 g:1 > ... Doing 'show pmap' after crash gives me long, long log without 0x80000000 occuring. On the other hand, adding vtophys(0x80000060) line before affected write operation translates address correctly. It is visible in log attached. I've also tried to track various functions removing mapping in pmap-v6-new, but with no luck. However, problem seems to lie there, as system boots fine without ARM_NEW_PMAP option. I would be grateful for you advice - what else can I do to investigate the issue? Best regards, Bartosz Szczepanek [-- Attachment #2 --] ## Starting application at 0x00900000 ... KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2015 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-CURRENT #191 38b3003(devel-fbsd-a38x-bsz-upstream)-dirty: Tue Dec 22 15:59:50 CET 2015 bsz@fbsd:/usr/home/bsz/build/arm.armv6/usr/home/bsz/freebsd-netasq/sys/ARMADA38X arm FreeBSD clang version 3.7.0 (tags/RELEASE_370/final 246257) 20150906 CPU: Cortex A9-r4 rev 1 (Cortex-A core) Supported features: ARM_ISA THUMB2 JAZELLE THUMBEE ARMv4 Security_Ext WB disabled EABT branch prediction enabled LoUU:2 LoC:2 LoUIS:2 Cache level 1: 32KB/32B 4-way data cache WB Read-Alloc Write-Alloc 32KB/32B 4-way instruction cache Read-Alloc real memory = 2147479552 (2047 MB) avail memory = 2095722496 (1998 MB) SOC: Marvell 88F6828, TClock 250MHz Instruction cache prefetch enabled, data cache prefetch disabled FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs random: entropy device external interface ofwbus0: <Open Firmware Device Tree> simplebus0: <Flattened device tree simple bus> on ofwbus0 simplebus1: <Flattened device tree simple bus> on simplebus0 gic0: <ARM Generic Interrupt Controller> mem 0xd000-0xdfff,0xc100-0xc1ff on simplebus1 gic0: pn 0x390, arch 0x1, rev 0x2, implementer 0x43b irqs 192 mpic0: <Marvell Integrated Interrupt Controller> mem 0x20a00-0x20ccf,0x21000-0x210ff,0x20400-0x204ff irq 18 on simplebus1 mp_tmr0: <ARM MPCore Timers> mem 0xc200-0xc21f irq 2 on simplebus1 Timecounter "MPCore" frequency 800000000 Hz quality 800 mp_tmr1: <ARM MPCore Timers> mem 0xc600-0xc61f irq 3 on simplebus1 Event timer "MPCore" frequency 800000000 Hz quality 1000 twsi0: <Marvell Integrated I2C Bus Controller> mem 0x11000-0x1101f irq 6 on simplebus1 iicbus0: <Philips I2C bus> on twsi0 iic0: <I2C generic I/O> on iicbus0 uart0: <16550 or compatible> mem 0x12000-0x120ff irq 8 on simplebus1 uart0: console (853,n,8,1) timer0: <Marvell CPU Timer> mem 0x20300-0x20333,0x20704-0x20707,0x18260-0x18263 on simplebus1 timer0: only watchdog attached pmsu0: <Power Management Service Unit> mem 0x22000-0x22fff on simplebus1 ehci0: <Marvell Integrated USB 2.0 controller> mem 0x58000-0x584ff irq 27 on simplebus1 usbus0: EHCI version 1.0 usbus0: set host controller mode usbus0 on ehci0 rtc0: <Marvell Integrated RTC> mem 0xa3800-0xa381f,0x184a0-0x184ab irq 29 on simplebus1 xhci0: <Marvell Integrated USB 3.0 controller> mem 0xf0000-0xf3fff,0xf4000-0xf7fff irq 34 on simplebus1 xhci0: 32 bytes context size, 32-bit DMA usbus1 on xhci0 xhci1: <Marvell Integrated USB 3.0 controller> mem 0xf8000-0xfbfff,0xfc000-0xfffff irq 35 on simplebus1 xhci1: 32 bytes context size, 32-bit DMA usbus2 on xhci1 pcib0: <Marvell Integrated PCI/PCI-E Controller> mem 0xf1080000-0xf1081fff irq 1 on ofwbus0 [ thread pid 0 tid 100000 ] Stopped at kdb_enter+0x58: ldrb r15, [r15, r15, ror r15]! db> c pci0: <PCI bus> on pcib0 re0: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> port 0x81000000-0x810000ff mem 0x80000000-0x80000fff,0x80004000-0x80007fff at device 0.0 on pci0 re0: Chip rev. 0x2c000000 re0: MAC rev. 0x00200000 vtophys: 0x80000060 vtophys: 0x80000060 vtophys: 0x80000060 miibus0: <MII bus> on re0 ukphy0: <Generic IEEE 802.3u media interface> PHY 1 on miibus0 vtophys: 0x80000060 vtophys: 0x80000060 vtophys: 0x80000060 vtophys: 0x80000060 vtophys: 0x80000060 vtophys: 0x80000060 ukphy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow vtophys: 0x80000060 re0: Using defaults for TSO: 65518/35/2048 re0: Ethernet address: 64:70:02:10:f7:20 cryptosoft0: <software crypto> Timecounters tick every 10.000 msec IPsec: Initialized Security Association Processing. usbus0: 480Mbps High Speed USB v2.0 usbus1: 5.0Gbps Super Speed USB v3.0 usbus2: 5.0Gbps Super Speed USB v3.0 Swap zone entries reduced from 256169 to 170779. Release APs mountroot: invalid file system specification. Loader variables: Manual root filesystem specification: <fstype>:<device> [options] Mount <device> using filesystem <fstype> and with the specified (optional) option list. eg. ufs:/dev/da0s1a zfs:tank cd9660:/dev/cd0 ro (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /) ? List valid disk boot devices . Yield 1 second (for background tasks) <empty line> Abort manual input mountroot> . ugen1.1: <Marvell> at usbus1 ugen2.1: <Marvell> at usbus2 uhub0: <Marvell XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus1 uhub1: <Marvell XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus2 ugen0.1: <Marvell> at usbus0 uhub2: <Marvell EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0 uhub1: 2 ports with 2 removable, self powered uhub0: 2 ports with 2 removable, self powered uhub2: 1 port with 1 removable, self powered mountroot> gic0: Spurious interrupt detected: last irq: 29 on CPU0 . mountroot> . ugen0.2: <vendor 0x0930> at usbus0 umass0: <vendor 0x0930 USB Flash Memory, class 0/0, rev 2.00/1.00, addr 2> on usbus0 da0 at umass-sim0 bus 0 scbus0 target 0 lun 0 da0: < USB Flash Memory 1.00> Removable Direct Access SPC-2 SCSI device da0: Serial Number CC52AF4C8244CEC0D29BA31B da0: 40.000MB/s transfers da0: 7396MB (15148608 512 byte sectors) da0: quirks=0x2<NO_6_BYTE> mountroot> ufs:da0s1a Trying to mount root from ufs:da0s1a []... Setting hostuuid: 76c63c5d-5af8-11e5-ba13-293fd6d20050. Setting hostid: 0x20257b83. Entropy harvesting:sysctl: unknown oid 'kern.random.sys.harvest.interrupt': No such file or directory interruptssysctl: unknown oid 'kern.random.sys.harvest.ethernet': No such file or directory ethernetsysctl: unknown oid 'kern.random.sys.harvest.point_to_point': No such file or directory point_to_pointsysctl: unknown oid 'kern.random.sys.harvest.swi': No such file or directory swi. Starting file system checks: /dev/da0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/da0s1a: clean, 412901 free (269 frags, 51579 blocks, 0.0% fragmentation) Mounting local file systems:. random: unblocking device. Writing entropy file:. Setting hostname: a38x. Starting Network: lo0 re0. lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> re0: flags=8802<vtopBROADCAST,SIMPLEhys:X,MULTICAST> met 0x8ric 0 mtu 1500 XCSUM,TXCSUM,VLA8209b<R060 N_MTU,VLAN_HWTAGFataGING,VLAN_HWCSUMl ke,WOL_MAGIC,LINKSrnelTATE> e da ether 64 mod:70:02:10:f7:20 ta abort: 'Translation Fault (L1)' on write trapframe: 0xefe80b28 FSR=00000805, FAR=80000060, spsr=60000013 r0 =c0e79740, r1 =80000000, r2 =00000001, r3 =00010000 r4 =c57f1000, r5 =00000001, r6 =00000000, r7 =00000001 r8 =c0e47f0c, r9 =c5be5780, r10=c57efb00, r11=efe80bc8 r12=00000000, ssp=efe80bb8, slr=c096f520, pc =c096f538 [ thread pid 241 tid 100068 ] Stopped at re_gmii_readreg+0x74: str r3, [r1, #0x060] db> bt Tracing pid 241 tid 100068 td 0xc5be1000 db_trace_self() at db_trace_self pc = 0xc0d16614 lr = 0xc09436e4 (db_hex2dec+0x1f4) sp = 0xefe80838 fp = 0xefe80850 db_hex2dec() at db_hex2dec+0x1f4 pc = 0xc09436e4 lr = 0xc0943338 (db_command_loop+0x2f4) sp = 0xefe80858 fp = 0xefe808f8 r4 = 0x00000000 r5 = 0x00000000 r6 = 0xc0d8c274 r10 = 0xc0eaac5c db_command_loop() at db_command_loop+0x2f4 pc = 0xc0943338 lr = 0xc09430b8 (db_command_loop+0x74) sp = 0xefe80900 fp = 0xefe80910 r4 = 0xc0d6a377 r5 = 0xc0d849a0 r6 = 0xc0eaac48 r7 = 0xefe80b28 r8 = 0xc0e9fa00 r9 = 0xc0e43360 r10 = 0xc0e9fa04 db_command_loop() at db_command_loop+0x74 pc = 0xc09430b8 lr = 0xc0945ddc (db_fetch_ksymtab+0x2d0) sp = 0xefe80918 fp = 0xefe80a30 r4 = 0x00000000 r5 = 0xc0eaac54 r6 = 0xc0e9fa20 r10 = 0xc0e9fa04 db_fetch_ksymtab() at db_fetch_ksymtab+0x2d0 pc = 0xc0945ddc lr = 0xc0abe454 (kdb_trap+0x180) sp = 0xefe80a38 fp = 0xefe80a60 r4 = 0x00000000 r5 = 0x00000805 r6 = 0xc0e9fa20 r7 = 0xefe80b28 kdb_trap() at kdb_trap+0x180 pc = 0xc0abe454 lr = 0xc0d31988 (abort_handler+0x714) sp = 0xefe80a68 fp = 0xefe80a88 r4 = 0xefe80b28 r5 = 0x00000013 r6 = 0x80000060 r7 = 0x00000005 r8 = 0x00000805 r9 = 0xc5be1000 r10 = 0xefe80b28 abort_handler() at abort_handler+0x714 pc = 0xc0d31988 lr = 0xc0d312f4 (abort_handler+0x80) sp = 0xefe80a90 fp = 0xefe80b20 r4 = 0x00000023 r5 = 0x00000005 r6 = 0x00000000 r7 = 0x00000805 r8 = 0x00000013 r10 = 0xefe80b28 abort_handler() at abort_handler+0x80 pc = 0xc0d312f4 lr = 0xc0d17cbc (exception_exit) sp = 0xefe80b28 fp = 0xefe80bc8 r4 = 0xc57f1000 r5 = 0x00000001 r6 = 0x00000000 r7 = 0x00000001 r8 = 0xc0e47f0c r9 = 0xc5be5780 r10 = 0xc57efb00 exception_exit() at exception_exit pc = 0xc0d17cbc lr = 0xc096f520 (re_gmii_readreg+0x5c) sp = 0xefe80bb8 fp = 0xefe80bc8 r0 = 0xc0e79740 r1 = 0x80000000 r2 = 0x00000001 r3 = 0x00010000 r4 = 0xc57f1000 r5 = 0x00000001 r6 = 0x00000000 r7 = 0x00000001 r8 = 0xc0e47f0c r9 = 0xc5be5780 r10 = 0xc57efb00 r12 = 0x00000000 re_gmii_readreg() at re_gmii_readreg+0x74 pc = 0xc096f538 lr = 0xc09720b8 (re_gmii_writereg+0x2af0) sp = 0xefe80bd0 fp = 0xefe80be0 r4 = 0xc56c6e80 r5 = 0xc57f1000 r6 = 0x00000001 r10 = 0xc57efb00 re_gmii_writereg() at re_gmii_writereg+0x2af0 pc = 0xc09720b8 lr = 0xc09558cc (ukphy_status+0x7c) sp = 0xefe80be8 fp = 0xefe80c08 r4 = 0xc584a200 r5 = 0xc584a300 r6 = 0x00000001 r7 = 0xc0d6e54d ukphy_status() at ukphy_status+0x7c pc = 0xc09558cc lr = 0xc095583c (mii_phy_flowstatus+0x1e8) sp = 0xefe80c10 fp = 0xefe80c18 r4 = 0x00000003 r5 = 0xc584a200 r6 = 0xc57fe420 r7 = 0xc0d6e54d r8 = 0xc57efb00 r9 = 0xc5be5780 r10 = 0x00000000 mii_phy_flowstatus() at mii_phy_flowstatus+0x1e8 pc = 0xc095583c lr = 0xc09539a0 (mii_pollstat+0x5c) sp = 0xefe80c20 fp = 0xefe80c30 r4 = 0xc57efb00 r5 = 0xc584a200 mii_pollstat() at mii_pollstat+0x5c pc = 0xc09539a0 lr = 0xc0973044 (re_gmii_writereg+0x3a7c) sp = 0xefe80c38 fp = 0xefe80c48 r4 = 0xefe80d40 r5 = 0xc57efb00 r6 = 0xc57f3144 r10 = 0x00000000 re_gmii_writereg() at re_gmii_writereg+0x3a7c pc = 0xc0973044 lr = 0xc0b5a708 (ifmedia_ioctl+0x188) sp = 0xefe80c50 fp = 0xefe80c68 r4 = 0x0000002d r5 = 0xefe80d40 r6 = 0xc0286938 r7 = 0xc5be1000 ifmedia_ioctl() at ifmedia_ioctl+0x188 pc = 0xc0b5a708 lr = 0xc0b52734 (ifioctl+0xa88) sp = 0xefe80c70 fp = 0xefe80ce8 r4 = 0x0000002d r5 = 0xc0286938 r6 = 0xc0286938 r7 = 0xc5be1000 r8 = 0xc5839000 r10 = 0x00000000 ifioctl() at ifioctl+0xa88 pc = 0xc0b52734 lr = 0xc0ad90b0 (kern_ioctl+0x200) sp = 0xefe80cf0 fp = 0xefe80d30 r4 = 0xc5be1000 r5 = 0xc0286938 r6 = 0x00000003 r7 = 0xc0ae0638 r8 = 0xefe80d40 r9 = 0xc5b2b000 r10 = 0x00000000 kern_ioctl() at kern_ioctl+0x200 pc = 0xc0ad90b0 lr = 0xc0ad8e5c (sys_ioctl+0xfc) sp = 0xefe80d38 fp = 0xefe80de0 r4 = 0x00000028 r5 = 0xefe80e00 r6 = 0xc0286938 r7 = 0x00000000 r8 = 0xefe80d40 r9 = 0xc5be1000 r10 = 0x40000000 sys_ioctl() at sys_ioctl+0xfc pc = 0xc0ad8e5c lr = 0xc0d30eb8 (swi_handler+0x31c) sp = 0xefe80de8 fp = 0xefe80e48 r4 = 0xc5be1000 r5 = 0xc5c32a98 r6 = 0x00000000 r7 = 0xc0eac620 r8 = 0x00000000 r9 = 0xefe80df8 r10 = 0xbfbfee18 swi_handler() at swi_handler+0x31c pc = 0xc0d30eb8 lr = 0xc0d17c4c (swi_exit) sp = 0xefe80e50 fp = 0xbfbfe670 r4 = 0x00035f20 r5 = 0xbfbfdeb8 r6 = 0xbfbfdeba r7 = 0x00000036 r8 = 0x00000000 r9 = 0x00037cdc r10 = 0xbfbfee18 swi_exit() at swi_exit pc = 0xc0d17c4c lr = 0xc0d17c4c (swi_exit) sp = 0xefe80e50 fp = 0xbfbfe670 db>help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CABLO=%2BkGzcnOLJ=_X8EfNw-YcBL%2BOFhf9G8YQb5RRasNLH9C_Q>