Date: Tue, 30 Jul 2019 14:43:35 +0200 From: "O. Hartmann" <o.hartmann@walstatt.org> To: Toomas Soome <tsoome@me.com> Cc: "O. Hartmann" <ohartmann@walstatt.org>, "freebsd-embedded@freebsd.org" <embedded@freebsd.org>, FreeBSD CURRENT <freebsd-current@freebsd.org> Subject: Re: PCengines APU2C4, 12-STABLE: bootloader failure: Panic: free: guard2 fail @ 0x1000 + 2311663946 from Message-ID: <20190730144331.24f83e0c@freyja> In-Reply-To: <93EFC481-50A1-491C-87BB-12A35F3302EE@me.com> References: <20190724101132.5560377a@thor.intern.walstatt.dynvpn.de> <B7BF6016-E4B1-4B34-9C22-CC7CD789FA79@me.com> <20190724120720.4382397c@thor.intern.walstatt.dynvpn.de> <20190724154901.0c114ebe@thor.intern.walstatt.dynvpn.de> <93EFC481-50A1-491C-87BB-12A35F3302EE@me.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 24 Jul 2019 18:07:22 +0300 Toomas Soome <tsoome@me.com> wrote: > > On 24 Jul 2019, at 16:48, O. Hartmann <ohartmann@walstatt.org> wrote: > >=20 > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA256 > >=20 > > Am Wed, 24 Jul 2019 12:06:53 +0200 > > "O. Hartmann" <o.hartmann@walstatt.org <mailto:o.hartmann@walstatt.org>> > > schrieb:=20 > >> -----BEGIN PGP SIGNED MESSAGE----- > >> Hash: SHA256 > >>=20 > >> Am Wed, 24 Jul 2019 12:09:16 +0300 > >> Toomas Soome <tsoome@me.com> schrieb: > >> =20 > >>>> On 24 Jul 2019, at 11:11, O. Hartmann <ohartmann@walstatt.org> wrote: > >>>>=20 > >>>> -----BEGIN PGP SIGNED MESSAGE----- > >>>> Hash: SHA256 > >>>>=20 > >>>> Hallo, > >>>>=20 > >>>> on APU2C4 from PCengines with latest firmware apu2_v4.9.0.7.rom, Sea= BIOS > >>>> rel-1.12.1.3-0-g300e8b7, booting via legacy MBR FreeBSD 12-STABLE > >>>> r350274 (the same with r350115) fails to boot with an immediate load= er > >>>> error: > >>>>=20 > >>>> [...] > >>>> SeaBIOS (version rel-1.12.1.3-0-g300e8b7) > >>>>=20 > >>>> Press F10 key now for boot menu > >>>>=20 > >>>> Booting from Hard Disk... > >>>> / > >>>>=20 > >>>> onsoles: internal video/keyboard =20 > >>>> IOS drive C: is disk0=20 > >>>> IOS drive D: is disk1=20 > >>>> IOS 639kB/3404444kB available memory=20 > >>>>=20 > >>>> reeBSD/x86 bootstrap loader, Revision 1.1 =20 > >>>> Mon Apr 15 21:28:11 CEST 2019 root@thor)=20 > >>>> anic: free: guard2 fail @ 0x1000 + 2311663946 from > >>>> X=C3=A7u0=C3=A7}4=C3=A7l$=E2=99=A6=E2=94=9C=C3=AD@=E2=94=A4=E2=99=A0= :2106163957 -> Press a key on the console to reboot > >>>> <-- [=E2=80=A6] =20 > >>>=20 > >>>=20 > >>> This is definitely something =E2=80=9Cfunny=E2=80=9D, we are apparent= ly attempting to > >>> free pointer 0x1000 which is definitely wrong because our heap should= be > >>> just below 4GB line. Since we do get list of disks printed, also memo= ry > >>> and version, it means we get error from interpretator - it is possible > >>> the stack did clash with bss and hence the corruption. =20 > >>=20 > >> I realized that I have defined=20 > >>=20 > >> WITH_KERNEL_RETPOLINE=3DYES > >>=20 > >> and since I use to build NanoBSD with -DNO_CLEAN, I'm just now compili= ng a > >> clean NanoBSD with RETPOLINE mitigations disabled so far - trying to c= heck > >> whether either of the ways to build causes the issue. > >> =20 > >>>=20 > >>> You can try to press space on first spinner and enter alternate loade= r on > >>> boot: prompt. (enter ?/boot on boot: prompt to see the file list). = =20 > >>=20 > >> I try a soon as the build process has finished and if the problem is t= hen > >> still present. =20 > >=20 > >=20 > > With a fresh build and no RETPOLINE mitigation (neither kernel nor worl= d) > > the phenomenon as described above is still the same. I tried an alterna= tive > > loader as requested, but without success. When choosing loader_4th, I g= et > > this error: > >=20 > > [...] > > FreeBSD/x86 boot > > Default: 0:ad(0p3)/boot/loader > > boot: /boot/loader_4th/ > >=20 > > onsoles: internal video/keyboard > > IOS drive C: is disk0 > > IOS drive D: is disk1 > > IOS 639kB/3404444kB available memory > >=20 > > reeBSD/x86 bootstrap loader, Revision 1.1 > > Wed Jul 24 12:51:12 CEST 2019 root@thor) > > anic: No heap setup =20 > > -> Press a key on the console to reboot <=E2=80=94 =20 > > =20 >=20 > Now this is bad. if my math is correct, this system is supposed to have 3= GB > of RAM, so are there specific build exceptions in place? see > stand/i386/loader/main.c, function main, after call to bios_getmem(). >=20 > rgds, > toomas Hello Toomas, the PCengine APU2C4 is supposed to have 4GB of RAM - wouldn't have a 64bit system seen the whole range? On 32bit systems, there was a memory hole I as= sume for memory mapped I/O of several PCI devices. This is the first time I hav= e a look at the memory reported by the kernel and its confusing me a bit. I do not have any kind of specific build exceptions in place targetting the memory. Of course, for memory usage and image size optimizations I defined several WITHOUT_ and WITH_ tags for build and install - but they never caus= ed any trouble and have not been changed so far. >=20 > >=20 > > Loader loader_simp ends up in stuck console with no output: > >=20 > > [...] > > FreeBSD/x86 boot > > Default: 0:ad(0p3)/boot/loader > > boot: /boot/loader_4th/ > >=20 > > onsoles: internal video/keyboard > > IOS drive C: is disk0 > > IOS drive D: is disk1 > > IOS 639kB/3404444kB available memory > >=20 > > reeBSD/x86 bootstrap loader, Revision 1.1 > > Wed Jul 24 12:59:23 CEST 2019 root@thor) > > [...] > >=20 > > regards > > oh =20 > >> =20 > >>>=20 > >>> =20 > >>>>=20 > >>>>=20 > >>>>=20 > >>>>=20 > >>>> Booting 12.0-STABLE #78 r349288: Sat Jun 22 09:10:25 CEST 2019 amd64 > >>>> works fine with nothing changed except the OS version. > >>>>=20 > >>>>=20 > >>>> Booting 2.0-STABLE #78 r349288 works fine: > >>>>=20 > >>>> [...] > >>>> SeaBIOS (version rel-1.12.1.3-0-g300e8b7) > >>>>=20 > >>>> Press F10 key now for boot menu > >>>>=20 > >>>> Booting from Hard Disk... > >>>> | > >>>>=20 > >>>> onsoles: internal video/keyboard =20 > >>>> IOS drive C: is disk0=20 > >>>> IOS drive D: is disk1=20 > >>>> IOS 639kB/3404444kB available memory=20 > >>>>=20 > >>>> reeBSD/x86 bootstrap loader, Revision 1.1=20 > >>>> Mon Apr 15 21:28:11 CEST 2019 root@thor)=20 > >>>> oading /boot/defaults/loader.conf=20 > >>>> oading /boot/device.hints=20 > >>>> oading /boot/loader.conf=20 > >>>> oading /boot/loader.conf.local=20 > >>>> Loading kernel... > >>>> /boot/kernel/kernel text=3D0xb005e8 \ > >>>> [...] > >>>>=20 > >>>> In the message taken from the serial console the first column of > >>>> characters is lost due to an error in the output which seems FreeBSD > >>>> related.=20 > >>>=20 > >>> It certainly does look weird - sio_putc() is used in boot2 and it=E2= =80=99s > >>> implementation is using same principe as comc_putchat() in comconsole= .c > >>> (even if it is asm versus c code). Since the serial data is interpret= ed > >>> by terminal, it feels more about terminal emulator issue (line > >>> discipline, cabling, usb to serial dongle?) =20 > >>=20 > >> We use here a null modem cabling with an integrated USB-to-UART/TTL > >> converter, which is attached to a FreeBSD CURRENT (most recent) box: > >>=20 > >> [...] > >> ugen2.3: <FTDI FT232R USB UART> at usbus2 > >> uftdi0 on uhub4 > >> uftdi0: <FT232R USB UART> on usbus2 > >> [...] > >>=20 > >> it is a=20 > >> StarTech.com 1 Port USB Nullmodem RS232 Adapter Kabel (USB 2.0 FTDI > >> chipset). > >>=20 > >> Regards, > >> oh > >>=20 > >> =20 > >>>=20 > >>> rgds, > >>> toomas > >>>=20 > >>> =20 > >>>>=20 > >>>> The file /boot/loader.conf.local contains these lines in both, worki= ng > >>>> and non-working, scenario: > >>>>=20 > >>>> [...] > >>>> boot_serial=3D"YES" > >>>> # serial speed in bits/s > >>>> comconsole_speed=3D"115200" > >>>> console=3D"comconsole" > >>>>=20 > >>>> autoboot_delay=3D"0" > >>>>=20 > >>>> verbose_loading=3D"YES" > >>>> loader_logo=3D"orb" > >>>> beastie_disable=3D"YES" > >>>>=20 > >>>> ### Microcode > >>>> #cpu_microcode_load=3D"YES" # Set this to YES to load= and > >>>> apply a #cpu_microcode_name=3D"/boot/firmware/intel-ucode.bin" # Set= this > >>>> to the microcode #cpu_microcode_type=3D"cpu_microcode" # Requir= ed for > >>>> the kernel to find # the microcode update file. > >>>>=20 > >>>>=20 > >>>> # disable Process Table Isolation > >>>> #vm.pmap.pti=3D0 > >>>>=20 > >>>> kern.geom.label.gptid.enable=3D0 > >>>>=20 > >>>> # Limit the phys. memory > >>>> #hw.physmem=3D1073741824 # 1 G > >>>> #hw.physmem=3D536870912 # 512 MB > >>>> #hw.physmem=3D268435456 # 256 MB > >>>>=20 > >>>> # Da mehr als 1 igb NIC an Bord! Siehe man igb(4) > >>>> kern.ipc.nmbclusters=3D757350 > >>>> #kern.ipc.nmbjumbo9k=3D8192 > >>>>=20 > >>>> # NIC > >>>> #hw.em.max_interrupt_rate=3D32000 > >>>> hw.em.max_interrupt_rate=3D16000 > >>>>=20 > >>>> #If non-zero, enable EXPERIMENTAL feature to improve concurrent Fort= una > >>>> performance kern.random.fortuna.concurrent_read=3D"1" > >>>>=20 > >>>> # Forward Information Bases (FIBs) > >>>> net.fibs=3D10 > >>>> net.add_addr_allfibs=3D0 > >>>>=20 > >>>> [...] > >>>>=20 > >>>>=20 > >>>> Again, with the exact same setting 12-STABLE r349288 boots fine, > >>>> rr350274 doesn't. FreeBSD 12-STABLE r > >>>>=20 > >>>> Can someone please help? > >>>>=20 > >>>> Thanks in advance, oh =20 [...]
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20190730144331.24f83e0c>