From owner-freebsd-embedded@freebsd.org Wed Jul 24 18:50:02 2019 Return-Path: Delivered-To: freebsd-embedded@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 62C45B4B96 for ; Wed, 24 Jul 2019 18:50:02 +0000 (UTC) (envelope-from tsoome@me.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id C3E2A6D2BA for ; Wed, 24 Jul 2019 18:50:01 +0000 (UTC) (envelope-from tsoome@me.com) Received: by mailman.nyi.freebsd.org (Postfix) id C105BB4B91; Wed, 24 Jul 2019 18:50:01 +0000 (UTC) Delivered-To: embedded@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C0922B4B90 for ; Wed, 24 Jul 2019 18:50:01 +0000 (UTC) (envelope-from tsoome@me.com) Received: from pv50p00im-ztdg10011901.me.com (pv50p00im-ztdg10011901.me.com [17.58.6.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 330566D2B5 for ; Wed, 24 Jul 2019 18:49:59 +0000 (UTC) (envelope-from tsoome@me.com) Received: from nazgul.lan (148-52-235-80.sta.estpak.ee [80.235.52.148]) by pv50p00im-ztdg10011901.me.com (Postfix) with ESMTPSA id 0E7BF8009DA; Wed, 24 Jul 2019 18:49:48 +0000 (UTC) From: Toomas Soome Message-Id: Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: PCengines APU2C4, 12-STABLE: bootloader failure: Panic: free: guard2 fail @ 0x1000 + 2311663946 from Date: Wed, 24 Jul 2019 21:49:46 +0300 In-Reply-To: <20190724203003.27310532@hermann.fritz.box> Cc: "freebsd-embedded@freebsd.org" , FreeBSD CURRENT To: "Hartmann, O." References: <20190724101132.5560377a@thor.intern.walstatt.dynvpn.de> <20190724120720.4382397c@thor.intern.walstatt.dynvpn.de> <20190724154901.0c114ebe@thor.intern.walstatt.dynvpn.de> <93EFC481-50A1-491C-87BB-12A35F3302EE@me.com> <20190724203003.27310532@hermann.fritz.box> X-Mailer: Apple Mail (2.3445.104.11) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-07-24_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1812120000 definitions=main-1907240200 X-Rspamd-Queue-Id: C3E2A6D2BA X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.23 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; FORGED_RECIPIENTS_FORWARDING(0.00)[]; FORWARDED(0.00)[embedded@mailman.nyi.freebsd.org]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[me.com]; SPF_FAIL_FORWARDING(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all]; URI_COUNT_ODD(1.00)[11]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[me.com:+]; DMARC_POLICY_ALLOW(0.00)[me.com,quarantine]; MX_GOOD(-0.01)[cached: mx1.mail.icloud.com]; DMARC_POLICY_ALLOW_WITH_FAILURES(-0.50)[]; NEURAL_HAM_SHORT(-0.87)[-0.870,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[me.com]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; FORGED_RECIPIENTS(0.00)[embedded@freebsd.org ..,freebsd-embedded@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[148.52.235.80.zen.spamhaus.org : 127.0.0.10]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[me.com:s=1a1hai]; RCVD_COUNT_FIVE(0.00)[5]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; IP_SCORE(-1.15)[ip: (0.86), ipnet: 2610:1c1:1::/48(-3.72), asn: 11403(-2.84), country: US(-0.05)]; DWL_DNSWL_LOW(-1.00)[me.com.dwl.dnswl.org : 127.0.5.1] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2019 18:50:02 -0000 > On 24 Jul 2019, at 21:30, Hartmann, O. wrote: >=20 > On Wed, 24 Jul 2019 18:07:22 +0300 > Toomas Soome > wrote: >=20 >>> On 24 Jul 2019, at 16:48, O. Hartmann > >>> wrote: >>>=20 >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA256 >>>=20 >>> Am Wed, 24 Jul 2019 12:06:53 +0200 >>> "O. Hartmann" >>> >> = schrieb:=20 >>>> -----BEGIN PGP SIGNED MESSAGE----- >>>> Hash: SHA256 >>>>=20 >>>> Am Wed, 24 Jul 2019 12:09:16 +0300 >>>> Toomas Soome schrieb: >>>>=20 >>>>>> On 24 Jul 2019, at 11:11, O. Hartmann >>>>>> wrote: >>>>>>=20 >>>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>>> Hash: SHA256 >>>>>>=20 >>>>>> Hallo, >>>>>>=20 >>>>>> on APU2C4 from PCengines with latest firmware apu2_v4.9.0.7.rom, >>>>>> SeaBIOS rel-1.12.1.3-0-g300e8b7, booting via legacy MBR FreeBSD >>>>>> 12-STABLE r350274 (the same with r350115) fails to boot with an >>>>>> immediate loader error: >>>>>>=20 >>>>>> [...] >>>>>> SeaBIOS (version rel-1.12.1.3-0-g300e8b7) >>>>>>=20 >>>>>> Press F10 key now for boot menu >>>>>>=20 >>>>>> Booting from Hard Disk... >>>>>> / >>>>>>=20 >>>>>> onsoles: internal video/keyboard =20 >>>>>> IOS drive C: is disk0=20 >>>>>> IOS drive D: is disk1=20 >>>>>> IOS 639kB/3404444kB available memory=20 >>>>>>=20 >>>>>> reeBSD/x86 bootstrap loader, Revision 1.1 =20 >>>>>> Mon Apr 15 21:28:11 CEST 2019 root@thor)=20 >>>>>> anic: free: guard2 fail @ 0x1000 + 2311663946 from >>>>>> X=C3=A7u0=C3=A7}4=C3=A7l$=E2=99=A6=E2=94=9C=C3=AD@=E2=94=A4=E2=99=A0= :2106163957 -> Press a key on the console to >>>>>> reboot <-- [=E2=80=A6] =20 >>>>>=20 >>>>>=20 >>>>> This is definitely something =E2=80=9Cfunny=E2=80=9D, we are = apparently >>>>> attempting to free pointer 0x1000 which is definitely wrong >>>>> because our heap should be just below 4GB line. Since we do get >>>>> list of disks printed, also memory and version, it means we get >>>>> error from interpretator - it is possible the stack did clash >>>>> with bss and hence the corruption. =20 >>>>=20 >>>> I realized that I have defined=20 >>>>=20 >>>> WITH_KERNEL_RETPOLINE=3DYES >>>>=20 >>>> and since I use to build NanoBSD with -DNO_CLEAN, I'm just now >>>> compiling a clean NanoBSD with RETPOLINE mitigations disabled so >>>> far - trying to check whether either of the ways to build causes >>>> the issue.=20 >>>>>=20 >>>>> You can try to press space on first spinner and enter alternate >>>>> loader on boot: prompt. (enter ?/boot on boot: prompt to see the >>>>> file list). =20 >>>>=20 >>>> I try a soon as the build process has finished and if the problem >>>> is then still present. =20 >>>=20 >>>=20 >>> With a fresh build and no RETPOLINE mitigation (neither kernel nor >>> world) the phenomenon as described above is still the same. I tried >>> an alternative loader as requested, but without success. When >>> choosing loader_4th, I get this error: >>>=20 >>> [...] >>> FreeBSD/x86 boot >>> Default: 0:ad(0p3)/boot/loader >>> boot: /boot/loader_4th/ >>>=20 >>> onsoles: internal video/keyboard >>> IOS drive C: is disk0 >>> IOS drive D: is disk1 >>> IOS 639kB/3404444kB available memory >>>=20 >>> reeBSD/x86 bootstrap loader, Revision 1.1 >>> Wed Jul 24 12:51:12 CEST 2019 root@thor) >>> anic: No heap setup =20 >>> -> Press a key on the console to reboot <=E2=80=94 =20 >>>=20 >>=20 >> Now this is bad. if my math is correct, this system is supposed to >> have 3GB of RAM, so are there specific build exceptions in place? see >> stand/i386/loader/main.c, function main, after call to bios_getmem(). >>=20 >> rgds, >> toomas >=20 > I'm afraid I do not understand exactly the point. The base system is a > PCengine APU2C4 - which has 4 GB of memory total. I do not build the > NanoBSD image with any kind of buikd exceptions - neither did I with > the FreeBSD versions that worked so far (see my initial post for the > revision numbers). >=20 > Should there be build execeptions in place for recent developments? If we end up with error message like =E2=80=99No heap setup=E2=80=99, it = means there is something terribly wrong about heap setup, however, it = [the loader] certainly does seem to work with =E2=80=9Ccommon=E2=80=9D = build, therefore it seems like we should look into the values we get in = your case. The heap setup is done by setheap(heap_bottom, heap_top); So we would need to dig out more details to see what is really going = on:) rgds, toomas >=20 >>=20 >>>=20 >>> Loader loader_simp ends up in stuck console with no output: >>>=20 >>> [...] >>> FreeBSD/x86 boot >>> Default: 0:ad(0p3)/boot/loader >>> boot: /boot/loader_4th/ >>>=20 >>> onsoles: internal video/keyboard >>> IOS drive C: is disk0 >>> IOS drive D: is disk1 >>> IOS 639kB/3404444kB available memory >>>=20 >>> reeBSD/x86 bootstrap loader, Revision 1.1 >>> Wed Jul 24 12:59:23 CEST 2019 root@thor) >>> [...] >>>=20 >>> regards >>> oh =20 >>>>=20 >>>>>=20 >>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> Booting 12.0-STABLE #78 r349288: Sat Jun 22 09:10:25 CEST 2019 >>>>>> amd64 works fine with nothing changed except the OS version. >>>>>>=20 >>>>>>=20 >>>>>> Booting 2.0-STABLE #78 r349288 works fine: >>>>>>=20 >>>>>> [...] >>>>>> SeaBIOS (version rel-1.12.1.3-0-g300e8b7) >>>>>>=20 >>>>>> Press F10 key now for boot menu >>>>>>=20 >>>>>> Booting from Hard Disk... >>>>>> | >>>>>>=20 >>>>>> onsoles: internal video/keyboard =20 >>>>>> IOS drive C: is disk0=20 >>>>>> IOS drive D: is disk1=20 >>>>>> IOS 639kB/3404444kB available memory=20 >>>>>>=20 >>>>>> reeBSD/x86 bootstrap loader, Revision 1.1=20 >>>>>> Mon Apr 15 21:28:11 CEST 2019 root@thor)=20 >>>>>> oading /boot/defaults/loader.conf=20 >>>>>> oading /boot/device.hints=20 >>>>>> oading /boot/loader.conf=20 >>>>>> oading /boot/loader.conf.local=20 >>>>>> Loading kernel... >>>>>> /boot/kernel/kernel text=3D0xb005e8 \ >>>>>> [...] >>>>>>=20 >>>>>> In the message taken from the serial console the first column of >>>>>> characters is lost due to an error in the output which seems >>>>>> FreeBSD related.=20 >>>>>=20 >>>>> It certainly does look weird - sio_putc() is used in boot2 and >>>>> it=E2=80=99s implementation is using same principe as = comc_putchat() in >>>>> comconsole.c (even if it is asm versus c code). Since the serial >>>>> data is interpreted by terminal, it feels more about terminal >>>>> emulator issue (line discipline, cabling, usb to serial >>>>> dongle?) =20 >>>>=20 >>>> We use here a null modem cabling with an integrated >>>> USB-to-UART/TTL converter, which is attached to a FreeBSD CURRENT >>>> (most recent) box: >>>>=20 >>>> [...] >>>> ugen2.3: at usbus2 >>>> uftdi0 on uhub4 >>>> uftdi0: on usbus2 >>>> [...] >>>>=20 >>>> it is a=20 >>>> StarTech.com 1 Port USB Nullmodem RS232 Adapter Kabel (USB 2.0 >>>> FTDI chipset). >>>>=20 >>>> Regards, >>>> oh >>>>=20 >>>>=20 >>>>>=20 >>>>> rgds, >>>>> toomas >>>>>=20 >>>>>=20 >>>>>>=20 >>>>>> The file /boot/loader.conf.local contains these lines in both, >>>>>> working and non-working, scenario: >>>>>>=20 >>>>>> [...] >>>>>> boot_serial=3D"YES" >>>>>> # serial speed in bits/s >>>>>> comconsole_speed=3D"115200" >>>>>> console=3D"comconsole" >>>>>>=20 >>>>>> autoboot_delay=3D"0" >>>>>>=20 >>>>>> verbose_loading=3D"YES" >>>>>> loader_logo=3D"orb" >>>>>> beastie_disable=3D"YES" >>>>>>=20 >>>>>> ### Microcode >>>>>> #cpu_microcode_load=3D"YES" # Set this to YES to >>>>>> load and apply a >>>>>> #cpu_microcode_name=3D"/boot/firmware/intel-ucode.bin" # Set this >>>>>> to the microcode #cpu_microcode_type=3D"cpu_microcode" # >>>>>> Required for the kernel to find # the microcode update file. >>>>>>=20 >>>>>>=20 >>>>>> # disable Process Table Isolation >>>>>> #vm.pmap.pti=3D0 >>>>>>=20 >>>>>> kern.geom.label.gptid.enable=3D0 >>>>>>=20 >>>>>> # Limit the phys. memory >>>>>> #hw.physmem=3D1073741824 # 1 G >>>>>> #hw.physmem=3D536870912 # 512 MB >>>>>> #hw.physmem=3D268435456 # 256 MB >>>>>>=20 >>>>>> # Da mehr als 1 igb NIC an Bord! Siehe man igb(4) >>>>>> kern.ipc.nmbclusters=3D757350 >>>>>> #kern.ipc.nmbjumbo9k=3D8192 >>>>>>=20 >>>>>> # NIC >>>>>> #hw.em.max_interrupt_rate=3D32000 >>>>>> hw.em.max_interrupt_rate=3D16000 >>>>>>=20 >>>>>> #If non-zero, enable EXPERIMENTAL feature to improve concurrent >>>>>> Fortuna performance kern.random.fortuna.concurrent_read=3D"1" >>>>>>=20 >>>>>> # Forward Information Bases (FIBs) >>>>>> net.fibs=3D10 >>>>>> net.add_addr_allfibs=3D0 >>>>>>=20 >>>>>> [...] >>>>>>=20 >>>>>>=20 >>>>>> Again, with the exact same setting 12-STABLE r349288 boots fine, >>>>>> rr350274 doesn't. FreeBSD 12-STABLE r >>>>>>=20 >>>>>> Can someone please help? >>>>>>=20 >>>>>> Thanks in advance, oh =20 >>>=20 >>> [...] >>>=20 >>>=20 >>> - --=20 >>> O. Hartmann >>>=20 >>> Ich widerspreche der Nutzung oder =C3=9Cbermittlung meiner Daten = f=C3=BCr >>> Werbezwecke oder f=C3=BCr die Markt- oder Meinungsforschung (=C2=A7 = 28 Abs. 4 >>> BDSG). -----BEGIN PGP SIGNATURE----- >>>=20 >>> iHUEARYIAB0WIQSy8IBxAPDkqVBaTJ44N1ZZPba5RwUCXThhzQAKCRA4N1ZZPba5 >>> R5SrAQDMEZQPGXsD9dqGsq9wNOPYPY7o0y/sBm9ovRkyepI5VwD/WUulyFprjlPI >>> hKYI/AJ/VVqI3EfPd/dUiSyBf1toeQw=3D >>> =3DaLz0 >>> -----END PGP SIGNATURE----- >>> _______________________________________________ >>> freebsd-current@freebsd.org = > >>> mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-current = >>> > To >>> unsubscribe, send any mail to >>> "freebsd-current-unsubscribe@freebsd.org = >>> >" =20 >>=20 >> _______________________________________________ >> freebsd-current@freebsd.org = mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-current = >> To unsubscribe, send any mail to >> "freebsd-current-unsubscribe@freebsd.org = "