Date: Sat, 5 Nov 2016 05:42:22 +0000 (UTC) From: <girivs82@yahoo.com> To: "freebsd-arm@freebsd.org" <freebsd-arm@freebsd.org> Subject: Low level boot loader for freebsd Message-ID: <234108602.628399.1478324542467@mail.yahoo.com> References: <234108602.628399.1478324542467.ref@mail.yahoo.com>
index | next in thread | previous in thread | raw e-mail
Serious problems with blank emails, so I'm using my alternate email address. Hopefully this will go through.-----------------------------------Just an idea I'm throwing out. I decided to do a bit of baremetal coding on my Jetson-TK1 that got me thinking. Right now FreeBSD relies on uboot as the low level bootloader and ubldr uses the u-boot API to bootstrap on top of it. I'm thinking of extending the functionality of ubldr to remove the u-boot dependency and use our own baremetal lowlevel hookups (well, name might have to be changed since it won't be using u-boot anymore). Advantages:1. Low level bootstrapping code part of FreeBSD tree. Using the libstand library and the low level hookups, we'd get the low level bootloader compiled while building world+kernel.2. Sharing the same fdt. Since the low level bootloader uses the same dts/dtsi files as the kernel, there is no reason to have two of them. The same fdt can be shared between the bootloader and the kernel. Theoretically we could still do that now between u-boot and freebsd, but being maintained by different projects, the shared dts will have an unavoidable dichotomy.3. Seamless bootstrapping. Right now, the bootloader and kernel are completely separate entities, so once the kernel takes control, it ends up having to initialize a bunch of HW that the bootloader has already take care of. This makes sense now because there are no guarantees on what has and has not been initialized by u-boot and what could change in future. With the bootloader part of the freebsd base, we might actually be able to just do post-init work on the kernel and this could actually improve boot times (and in the embedded world, this counts for a lot)4. Use our default toolchain. Clang (binutils is unavoidable at this point till the llvm linker is mature). I already compiled my bare-metal code on Jetson-TK1 using Clang+binutils and it ran quite well, llvm has has gotten pretty mature at this point. I stopped using gcc for my arm compilation a long time ago, even for the freebsd kernel. Disadvantages:1. Embedded world is fragmented, too many machine/arch specific variations. Code maintainability could be a nightmare.2. Testing low level is difficult. You don't even have a uart console in bare-metal, so you'll have to rely on JTAG (sometimes really expensive lauterbachs) or poring through assembly. Plus we need to find people with embedded hardware willing to test them.3. Adoptability. I don't think people will warm up to a new bootloader soon, particularly those who have u-boot experience. Without contributors, this will die a slow agonizing death. What does the community think? Is this something that could actually help? If yes, I'm willing to start working on a proof-of-concept. My test vehicle would be the Jetson TK1. From owner-freebsd-arm@freebsd.org Sat Nov 5 23:13:43 2016 Return-Path: <owner-freebsd-arm@freebsd.org> Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2398BED618 for <freebsd-arm@mailman.ysv.freebsd.org>; Sat, 5 Nov 2016 23:13:43 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-56.reflexion.net [208.70.210.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 737D2873 for <freebsd-arm@freebsd.org>; Sat, 5 Nov 2016 23:13:42 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 16590 invoked from network); 5 Nov 2016 23:13:25 -0000 Received: from unknown (HELO rtc-sm-01.app.dca.reflexion.local) (10.81.150.1) by 0 (rfx-qmail) with SMTP; 5 Nov 2016 23:13:25 -0000 Received: by rtc-sm-01.app.dca.reflexion.local (Reflexion email security v8.10.1) with SMTP; Sat, 05 Nov 2016 19:13:40 -0400 (EDT) Received: (qmail 17766 invoked from network); 5 Nov 2016 23:13:40 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 5 Nov 2016 23:13:40 -0000 Received: from [192.168.1.106] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 7122FEC8BDB for <freebsd-arm@freebsd.org>; Sat, 5 Nov 2016 16:13:35 -0700 (PDT) From: Mark Millard <markmi@dsl-only.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.1 \(3251\)) Subject: FYI: Some BPi-M3 notes, including self-hosted buildworld/buildkernel in about 9.5 hours (lldb build included) Message-Id: <F9B05171-38EF-47F3-8B53-F25138F027B2@dsl-only.net> Date: Sat, 5 Nov 2016 16:13:34 -0700 To: freebsd-arm <freebsd-arm@freebsd.org> X-Mailer: Apple Mail (2.3251) X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Porting FreeBSD to ARM processors." <freebsd-arm.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-arm>, <mailto:freebsd-arm-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-arm/>; List-Post: <mailto:freebsd-arm@freebsd.org> List-Help: <mailto:freebsd-arm-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-arm>, <mailto:freebsd-arm-request@freebsd.org?subject=subscribe> X-List-Received-Date: Sat, 05 Nov 2016 23:13:43 -0000 I did a from-scratch -j 5 buildworld buildkernel of stable/11 -r308135 on a BPi-M3 and it took about 9.5 hours. The "Other notes" part below lists the src-env.conf (src.conf) sort of content, clang's FULL, EXTRAS, and LLDB included. But WITH_SYSTEM_COMPILER= was in use as well. The BPi-M3 was running -r308135 that had been cross-built with -mcpu=cortex-a7 listed in XCFLAGS and XCXXFLAGS. The ALLWINNER kernel configuration file was included in my BPIM3-NODBG and then BPIM3-NODBG made sure that the performance-tied debug options were disabled in the cross build. (This style of configuration handling adapts to head's ALLWINNER configuration use as well --without editing BPIM3-NODBG generally.) The BPi-M3 is a V1.2 board (with a 1.7mm-inner/4mm-outer power connector) and has/had: A) A heat sink on the A83T chip. B) A 3.3 V fan above the A83T chip. C) A 240 GByte USB SSD used for / (the root file system) and swap. [The SSD is a USB-stick style one, no external power.] D) There is a 3.1 GByte swap parition on the USB SSD. [Given bugzilla 206048 I always avoid using files as swap.] E) The mmc media used was a Samsung 32GB EVO+. [But (C) makes this largely irrelevant to the performance.] F) A 15W power supply (output: 5V, 3A capable). G) Appropriate serial port hardware attached. H) "-mcpu=cortex-a7" listed in XCFLAGS and XCXXFLAGS for buildworld/buildkernel. I) The ALLWINNER kernel configuration file was included in my BPIM3-NODBG and then BPIM3-NODBG made sure that the performance-tied debug options were disabled. (stable/11 -r308135) J) The "Other notes" part below lists the src-env.conf (src.conf) sort of content, clang's FULL, EXTRAS, and LLDB included. But WITH_SYSTEM_COMPILER= was in use as well. FreeBSD only supports using 4 of the 8 cores. If 6-8 of them were usable then the time for a build with some -j from 6 to 9 might be 7 hours or less. For comparison: As I remember last I did -j 5 buildworld buildkernel of an earlier stable/11 on a rpi2 it was between 14 hours and 15 hours, no heat sink, no fan, but an external USB SSD for / (root filesystem) and swap. (That USB SSD on a powered hub.) I found the "sysctl -a" aw_ items for the BPi-M3 and the thermal results reported were 35C or a little less while 4 cores were busy and had been for some time. Both cluster's reports always agreed when observed --despite only 4 cores being in use. (Apparently thermal monitoring and throttling are done by stable/11 for the BPi-M3 --but not via powerd. powerd reports that the cpufreq facilities it uses are are not available and quits.) I also used the Ubuntu 16.04 with Mate image and looked around some: 0) All 8 cores (4 from each cluster) are available. 1) The two clusters of 4 cores each are not treated as NUMA from what I can tell. [For example the Ubuntu top's NUMA selections report lack of a NUMA context.] 2) (1) is despite the early boot referencing 2 "sockets" [amd64 terminology?] for what Allwinner calls "clusters". 3) All the cores are reported as running at the same speed when the speed changes, even when only one core was doing much of anything and top shows most as 0.0% used. This seemed to be true for all the policy settings that I played with. So it appears the Ubuntu image keeps the two clusters of cores set up the same: when one changes so does the other to match, cores uniform as well. That and avoiding any assumption of a cache level that spans all the cores (both clusters) would seem to be the general trend for how things are handled to enable 8 cores. But that claim is made from the external presentation, not from analysis of the source code. Other notes: I've had the BPi-M3 board change its Ethernet MAC address 3 times so far. MAC stability may require some explicit control/override somewhere. I used kern.cam.boot_delay="10000" in /boot/loader.conf so that mounting / on the USB SSD would reliably go well on the BPi-M3. I have had the serial connection drop text, unlike the rpi2 using the same serial hardware and such. I've also had the BPi-M3 context switch to odd characters being shown. Crochet tends to set up a file as the swap area. Given bugzilla 206048 I avoid this configuration on all architectures and create and use a swap partition instead. (I usually have swap set up and enabled.) ~/src.configs/src.conf.bpim3-clang-bootstrap.bpim3-host was: > TO_TYPE=armv6 > # > KERNCONF=BPIM3-NODBG > TARGET=arm > .if ${.MAKE.LEVEL} == 0 > TARGET_ARCH=${TO_TYPE} > .export TARGET_ARCH > .endif > # > #WITH_CROSS_COMPILER= > WITH_SYSTEM_COMPILER= > # > #CPUTYPE=soft > WITH_LIBCPLUSPLUS= > WITH_BINUTILS_BOOTSTRAP= > #WITHOUT_CLANG_BOOTSTRAP= > WITH_CLANG= > WITH_CLANG_IS_CC= > WITH_CLANG_FULL= > WITH_CLANG_EXTRAS= > WITH_LLDB= > # > WITH_BOOT= > WITHOUT_LIB32= > WITHOUT_LIBSOFT= > # > WITHOUT_ELFTOOLCHAIN_BOOTSTRAP= > WITHOUT_GCC_BOOTSTRAP= > WITHOUT_GCC= > WITHOUT_GCC_IS_CC= > WITHOUT_GNUCXX= > # > NO_WERROR= > #WERROR= > MALLOC_PRODUCTION= > # > WITH_DEBUG_FILES= > # > XCFLAGS+= -mcpu=cortex-a7 > XCXXFLAGS+= -mcpu=cortex-a7 > # There is no XCPPFLAGS but XCPP gets XCFLAGS content. === Mark Millard markmi at dsl-only.nethelp
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?234108602.628399.1478324542467>