Date: Wed, 24 Jul 2019 21:49:46 +0300 From: Toomas Soome <tsoome@me.com> To: "Hartmann, O." <ohartmann@walstatt.org> Cc: "freebsd-embedded@freebsd.org" <embedded@freebsd.org>, FreeBSD CURRENT <freebsd-current@freebsd.org> Subject: Re: PCengines APU2C4, 12-STABLE: bootloader failure: Panic: free: guard2 fail @ 0x1000 + 2311663946 from Message-ID: <D00BEA50-DEE1-428B-9F4A-D0EA255ED582@me.com> In-Reply-To: <20190724203003.27310532@hermann.fritz.box> References: <20190724101132.5560377a@thor.intern.walstatt.dynvpn.de> <B7BF6016-E4B1-4B34-9C22-CC7CD789FA79@me.com> <20190724120720.4382397c@thor.intern.walstatt.dynvpn.de> <20190724154901.0c114ebe@thor.intern.walstatt.dynvpn.de> <93EFC481-50A1-491C-87BB-12A35F3302EE@me.com> <20190724203003.27310532@hermann.fritz.box>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 24 Jul 2019, at 21:30, Hartmann, O. <ohartmann@walstatt.org> wrote: >=20 > On Wed, 24 Jul 2019 18:07:22 +0300 > Toomas Soome <tsoome@me.com <mailto:tsoome@me.com>> wrote: >=20 >>> On 24 Jul 2019, at 16:48, O. Hartmann <ohartmann@walstatt.org = <mailto:ohartmann@walstatt.org>> >>> wrote: >>>=20 >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA256 >>>=20 >>> Am Wed, 24 Jul 2019 12:06:53 +0200 >>> "O. Hartmann" <o.hartmann@walstatt.org = <mailto:o.hartmann@walstatt.org> >>> <mailto:o.hartmann@walstatt.org <mailto:o.hartmann@walstatt.org>>> = schrieb:=20 >>>> -----BEGIN PGP SIGNED MESSAGE----- >>>> Hash: SHA256 >>>>=20 >>>> Am Wed, 24 Jul 2019 12:09:16 +0300 >>>> Toomas Soome <tsoome@me.com> schrieb: >>>>=20 >>>>>> On 24 Jul 2019, at 11:11, O. Hartmann <ohartmann@walstatt.org> >>>>>> wrote: >>>>>>=20 >>>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>>> Hash: SHA256 >>>>>>=20 >>>>>> Hallo, >>>>>>=20 >>>>>> on APU2C4 from PCengines with latest firmware apu2_v4.9.0.7.rom, >>>>>> SeaBIOS rel-1.12.1.3-0-g300e8b7, booting via legacy MBR FreeBSD >>>>>> 12-STABLE r350274 (the same with r350115) fails to boot with an >>>>>> immediate loader error: >>>>>>=20 >>>>>> [...] >>>>>> SeaBIOS (version rel-1.12.1.3-0-g300e8b7) >>>>>>=20 >>>>>> Press F10 key now for boot menu >>>>>>=20 >>>>>> Booting from Hard Disk... >>>>>> / >>>>>>=20 >>>>>> onsoles: internal video/keyboard =20 >>>>>> IOS drive C: is disk0=20 >>>>>> IOS drive D: is disk1=20 >>>>>> IOS 639kB/3404444kB available memory=20 >>>>>>=20 >>>>>> reeBSD/x86 bootstrap loader, Revision 1.1 =20 >>>>>> Mon Apr 15 21:28:11 CEST 2019 root@thor)=20 >>>>>> anic: free: guard2 fail @ 0x1000 + 2311663946 from >>>>>> X=C3=A7u0=C3=A7}4=C3=A7l$=E2=99=A6=E2=94=9C=C3=AD@=E2=94=A4=E2=99=A0= :2106163957 -> Press a key on the console to >>>>>> reboot <-- [=E2=80=A6] =20 >>>>>=20 >>>>>=20 >>>>> This is definitely something =E2=80=9Cfunny=E2=80=9D, we are = apparently >>>>> attempting to free pointer 0x1000 which is definitely wrong >>>>> because our heap should be just below 4GB line. Since we do get >>>>> list of disks printed, also memory and version, it means we get >>>>> error from interpretator - it is possible the stack did clash >>>>> with bss and hence the corruption. =20 >>>>=20 >>>> I realized that I have defined=20 >>>>=20 >>>> WITH_KERNEL_RETPOLINE=3DYES >>>>=20 >>>> and since I use to build NanoBSD with -DNO_CLEAN, I'm just now >>>> compiling a clean NanoBSD with RETPOLINE mitigations disabled so >>>> far - trying to check whether either of the ways to build causes >>>> the issue.=20 >>>>>=20 >>>>> You can try to press space on first spinner and enter alternate >>>>> loader on boot: prompt. (enter ?/boot on boot: prompt to see the >>>>> file list). =20 >>>>=20 >>>> I try a soon as the build process has finished and if the problem >>>> is then still present. =20 >>>=20 >>>=20 >>> With a fresh build and no RETPOLINE mitigation (neither kernel nor >>> world) the phenomenon as described above is still the same. I tried >>> an alternative loader as requested, but without success. When >>> choosing loader_4th, I get this error: >>>=20 >>> [...] >>> FreeBSD/x86 boot >>> Default: 0:ad(0p3)/boot/loader >>> boot: /boot/loader_4th/ >>>=20 >>> onsoles: internal video/keyboard >>> IOS drive C: is disk0 >>> IOS drive D: is disk1 >>> IOS 639kB/3404444kB available memory >>>=20 >>> reeBSD/x86 bootstrap loader, Revision 1.1 >>> Wed Jul 24 12:51:12 CEST 2019 root@thor) >>> anic: No heap setup =20 >>> -> Press a key on the console to reboot <=E2=80=94 =20 >>>=20 >>=20 >> Now this is bad. if my math is correct, this system is supposed to >> have 3GB of RAM, so are there specific build exceptions in place? see >> stand/i386/loader/main.c, function main, after call to bios_getmem(). >>=20 >> rgds, >> toomas >=20 > I'm afraid I do not understand exactly the point. The base system is a > PCengine APU2C4 - which has 4 GB of memory total. I do not build the > NanoBSD image with any kind of buikd exceptions - neither did I with > the FreeBSD versions that worked so far (see my initial post for the > revision numbers). >=20 > Should there be build execeptions in place for recent developments? If we end up with error message like =E2=80=99No heap setup=E2=80=99, it = means there is something terribly wrong about heap setup, however, it = [the loader] certainly does seem to work with =E2=80=9Ccommon=E2=80=9D = build, therefore it seems like we should look into the values we get in = your case. The heap setup is done by setheap(heap_bottom, heap_top); So we would need to dig out more details to see what is really going = on:) rgds, toomas >=20 >>=20 >>>=20 >>> Loader loader_simp ends up in stuck console with no output: >>>=20 >>> [...] >>> FreeBSD/x86 boot >>> Default: 0:ad(0p3)/boot/loader >>> boot: /boot/loader_4th/ >>>=20 >>> onsoles: internal video/keyboard >>> IOS drive C: is disk0 >>> IOS drive D: is disk1 >>> IOS 639kB/3404444kB available memory >>>=20 >>> reeBSD/x86 bootstrap loader, Revision 1.1 >>> Wed Jul 24 12:59:23 CEST 2019 root@thor) >>> [...] >>>=20 >>> regards >>> oh =20 >>>>=20 >>>>>=20 >>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> Booting 12.0-STABLE #78 r349288: Sat Jun 22 09:10:25 CEST 2019 >>>>>> amd64 works fine with nothing changed except the OS version. >>>>>>=20 >>>>>>=20 >>>>>> Booting 2.0-STABLE #78 r349288 works fine: >>>>>>=20 >>>>>> [...] >>>>>> SeaBIOS (version rel-1.12.1.3-0-g300e8b7) >>>>>>=20 >>>>>> Press F10 key now for boot menu >>>>>>=20 >>>>>> Booting from Hard Disk... >>>>>> | >>>>>>=20 >>>>>> onsoles: internal video/keyboard =20 >>>>>> IOS drive C: is disk0=20 >>>>>> IOS drive D: is disk1=20 >>>>>> IOS 639kB/3404444kB available memory=20 >>>>>>=20 >>>>>> reeBSD/x86 bootstrap loader, Revision 1.1=20 >>>>>> Mon Apr 15 21:28:11 CEST 2019 root@thor)=20 >>>>>> oading /boot/defaults/loader.conf=20 >>>>>> oading /boot/device.hints=20 >>>>>> oading /boot/loader.conf=20 >>>>>> oading /boot/loader.conf.local=20 >>>>>> Loading kernel... >>>>>> /boot/kernel/kernel text=3D0xb005e8 \ >>>>>> [...] >>>>>>=20 >>>>>> In the message taken from the serial console the first column of >>>>>> characters is lost due to an error in the output which seems >>>>>> FreeBSD related.=20 >>>>>=20 >>>>> It certainly does look weird - sio_putc() is used in boot2 and >>>>> it=E2=80=99s implementation is using same principe as = comc_putchat() in >>>>> comconsole.c (even if it is asm versus c code). Since the serial >>>>> data is interpreted by terminal, it feels more about terminal >>>>> emulator issue (line discipline, cabling, usb to serial >>>>> dongle?) =20 >>>>=20 >>>> We use here a null modem cabling with an integrated >>>> USB-to-UART/TTL converter, which is attached to a FreeBSD CURRENT >>>> (most recent) box: >>>>=20 >>>> [...] >>>> ugen2.3: <FTDI FT232R USB UART> at usbus2 >>>> uftdi0 on uhub4 >>>> uftdi0: <FT232R USB UART> on usbus2 >>>> [...] >>>>=20 >>>> it is a=20 >>>> StarTech.com 1 Port USB Nullmodem RS232 Adapter Kabel (USB 2.0 >>>> FTDI chipset). >>>>=20 >>>> Regards, >>>> oh >>>>=20 >>>>=20 >>>>>=20 >>>>> rgds, >>>>> toomas >>>>>=20 >>>>>=20 >>>>>>=20 >>>>>> The file /boot/loader.conf.local contains these lines in both, >>>>>> working and non-working, scenario: >>>>>>=20 >>>>>> [...] >>>>>> boot_serial=3D"YES" >>>>>> # serial speed in bits/s >>>>>> comconsole_speed=3D"115200" >>>>>> console=3D"comconsole" >>>>>>=20 >>>>>> autoboot_delay=3D"0" >>>>>>=20 >>>>>> verbose_loading=3D"YES" >>>>>> loader_logo=3D"orb" >>>>>> beastie_disable=3D"YES" >>>>>>=20 >>>>>> ### Microcode >>>>>> #cpu_microcode_load=3D"YES" # Set this to YES to >>>>>> load and apply a >>>>>> #cpu_microcode_name=3D"/boot/firmware/intel-ucode.bin" # Set this >>>>>> to the microcode #cpu_microcode_type=3D"cpu_microcode" # >>>>>> Required for the kernel to find # the microcode update file. >>>>>>=20 >>>>>>=20 >>>>>> # disable Process Table Isolation >>>>>> #vm.pmap.pti=3D0 >>>>>>=20 >>>>>> kern.geom.label.gptid.enable=3D0 >>>>>>=20 >>>>>> # Limit the phys. memory >>>>>> #hw.physmem=3D1073741824 # 1 G >>>>>> #hw.physmem=3D536870912 # 512 MB >>>>>> #hw.physmem=3D268435456 # 256 MB >>>>>>=20 >>>>>> # Da mehr als 1 igb NIC an Bord! Siehe man igb(4) >>>>>> kern.ipc.nmbclusters=3D757350 >>>>>> #kern.ipc.nmbjumbo9k=3D8192 >>>>>>=20 >>>>>> # NIC >>>>>> #hw.em.max_interrupt_rate=3D32000 >>>>>> hw.em.max_interrupt_rate=3D16000 >>>>>>=20 >>>>>> #If non-zero, enable EXPERIMENTAL feature to improve concurrent >>>>>> Fortuna performance kern.random.fortuna.concurrent_read=3D"1" >>>>>>=20 >>>>>> # Forward Information Bases (FIBs) >>>>>> net.fibs=3D10 >>>>>> net.add_addr_allfibs=3D0 >>>>>>=20 >>>>>> [...] >>>>>>=20 >>>>>>=20 >>>>>> Again, with the exact same setting 12-STABLE r349288 boots fine, >>>>>> rr350274 doesn't. FreeBSD 12-STABLE r >>>>>>=20 >>>>>> Can someone please help? >>>>>>=20 >>>>>> Thanks in advance, oh =20 >>>=20 >>> [...] >>>=20 >>>=20 >>> - --=20 >>> O. Hartmann >>>=20 >>> Ich widerspreche der Nutzung oder =C3=9Cbermittlung meiner Daten = f=C3=BCr >>> Werbezwecke oder f=C3=BCr die Markt- oder Meinungsforschung (=C2=A7 = 28 Abs. 4 >>> BDSG). -----BEGIN PGP SIGNATURE----- >>>=20 >>> iHUEARYIAB0WIQSy8IBxAPDkqVBaTJ44N1ZZPba5RwUCXThhzQAKCRA4N1ZZPba5 >>> R5SrAQDMEZQPGXsD9dqGsq9wNOPYPY7o0y/sBm9ovRkyepI5VwD/WUulyFprjlPI >>> hKYI/AJ/VVqI3EfPd/dUiSyBf1toeQw=3D >>> =3DaLz0 >>> -----END PGP SIGNATURE----- >>> _______________________________________________ >>> freebsd-current@freebsd.org <mailto:freebsd-current@freebsd.org> = <mailto:freebsd-current@freebsd.org = <mailto:freebsd-current@freebsd.org>> >>> mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-current = <https://lists.freebsd.org/mailman/listinfo/freebsd-current>; >>> <https://lists.freebsd.org/mailman/listinfo/freebsd-current = <https://lists.freebsd.org/mailman/listinfo/freebsd-current>>; To >>> unsubscribe, send any mail to >>> "freebsd-current-unsubscribe@freebsd.org = <mailto:freebsd-current-unsubscribe@freebsd.org> >>> <mailto:freebsd-current-unsubscribe@freebsd.org = <mailto:freebsd-current-unsubscribe@freebsd.org>>" =20 >>=20 >> _______________________________________________ >> freebsd-current@freebsd.org <mailto:freebsd-current@freebsd.org> = mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-current = <https://lists.freebsd.org/mailman/listinfo/freebsd-current>; >> To unsubscribe, send any mail to >> "freebsd-current-unsubscribe@freebsd.org = <mailto:freebsd-current-unsubscribe@freebsd.org>"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?D00BEA50-DEE1-428B-9F4A-D0EA255ED582>