Date: Tue, 3 Jun 2025 18:41:41 +0100 From: Nuno Teixeira <eduardo@freebsd.org> To: "Simon J. Gerraty" <sjg@juniper.net> Cc: Mark Millard <marklmi@yahoo.com>, FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: incremental bulds from scratch with beinstall.sh Message-ID: <CAFDf7UJsvGZiyUYMKm-1dNt7wXS1uw2ofbLLSMydJumexGzHyA@mail.gmail.com> In-Reply-To: <85314.1747165929@kaos.jnpr.net> References: <28F2BDE7-5903-4C04-A570-6A407F19D5F2.ref@yahoo.com> <28F2BDE7-5903-4C04-A570-6A407F19D5F2@yahoo.com> <49396.1746554966@kaos.jnpr.net> <C9CCEA0D-C397-4FAA-8459-E8A2CD468358@yahoo.com> <87401.1746562441@kaos.jnpr.net> <4ACBBC16-3BB6-436A-B0B1-A18F088B000E@yahoo.com> <4421.1746572832@kaos.jnpr.net> <2CA19E21-0F2F-465A-BE8E-81ACDEE42D23@yahoo.com> <10858.1746584427@kaos.jnpr.net> <CAFDf7UKiKQnCNi-q4-m1QzP1qbobmoBr-UokQjBuJULuWPJC4w@mail.gmail.com> <27515.1747072855@kaos.jnpr.net> <CAFDf7UJs_yLJXAGKWaODhXh46%2BNxZug4y1HHk_c6ZvyRRvw1oA@mail.gmail.com> <3D5D0CE0-64A9-4A99-8282-61918987BA5C@yahoo.com> <CAFDf7UJxm_PNvenk8wnNAk=SusunOydXpJG4j-Ud_xc3vgVKCg@mail.gmail.com> <8FA02BF7-2653-4B4A-8EFD-84C148473AF5@yahoo.com> <91572.1747150994@kaos.jnpr.net> <DE730DA5-5F4F-4EF0-90CD-25EE413488EE@yahoo.com> <85314.1747165929@kaos.jnpr.net>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] Hello! https://reviews.freebsd.org/D50313 has been acepted and soon it will be committed. I still have a lot of difficulties about how this works and how to be configured. Thinking if it will be some config examples be provided? Thanks Simon J. Gerraty <sjg@juniper.net> escreveu (terça, 13/05/2025 à(s) 20:52): > Mark Millard <marklmi@yahoo.com> wrote: > > > If you are going to head down that path, I would highly recommend > > > using the 'mk' wrapper from > > > https://www.crufty.net/ftp/pub/sjg/sb-tools.tar.gz > > > We've used that model at work for over 20 years. > > > Described in https://www.crufty.net/sjg/docs/sb-tools.htm > > > > > > In a nutshell; each tree has a .sandbox-env file which can tune its > > > environment (as well as mark the top of the "sandbox"). > > > There are a plethora of other hooks to tune. > > > I find it especially useful with Emac's M-x compile > > > > I will take a look. > > > > Using my aarch64 context as an example (it > > has more variations than my amd64 > > environment, since I do nothing for i386 > > but aarch64 is also set up for armv7): > > you can do combinations. > > Eg Junos currently builds for last I counted, over 35 combinations of > architecture and OS we provide symlinks to 'mk' to simply usage. > Eg. you can always do 'mk --machine arm64,aarch64' but 'mk-arm64,aarch64' > is > less typing and allows for auto-completion. > > FreeBSD by uses MACHINE and MACHINE_ARCH as TARGET_SPEC_VARS, > Junos however only supports a single MACHINE_ARCH per MACHINE but > multiple TARGET_OS's (bsd15,bsd12,wrl9,...) so we use > MACHINE and TARGET_OS as TARGET_SPEC_VARS. > > You can of course extend that for your own use, so that you could have > mk --machine arm64,aarch64,something to set > TARGET_SPEC=arm64,aarch64,something > > but that assumes that the objects for arm64,aarch64,something should be > kept separate from those for arm64,aarch64,other > > If "something" and "other" really represent things to be built for > arm64,aarch64, then setting up targets for them is another option. > > There is nothing to stop you of course from using any of the scripts > below in conjuction with the setup supported by 'mk'. > > Things that everyone is likely to need/use can/should be setup as > targets so 'mk-arm64,aarch64 dbg-kernel' "just worked" > but things that do not fit that criteria are better served as scripts > such as you have. > > This is in some way a reflection of the difference b/w an emedded vendor > who is typically only interested in a very small sub-set of the > architectures supported by the project, but wants them handled the same > way by 100's if not 1000's of devs. Eg we put all our Juniper specific > targets under a juniper/ tree - that works with the same DIRDEPS_BUILD > bits as FreeBSD can. > > > I have 8 aarch64 scripts that have the > > likes of __MAKE_CONF (and more) specified > > that do individual system builds of main's > > kernel or world: > > > > # ls -C1 ~/build-sys-*dbg-*.sh > > /root/build-sys-main-CA7-dbg-kernel.sh > > /root/build-sys-main-CA7-dbg-world.sh > > /root/build-sys-main-CA7-nodbg-kernel.sh > > /root/build-sys-main-CA7-nodbg-world.sh > > /root/build-sys-main-CA76-dbg-kernel.sh > > /root/build-sys-main-CA76-dbg-world.sh > > /root/build-sys-main-CA76-nodbg-kernel.sh > > /root/build-sys-main-CA76-nodbg-world.sh > > > > The above in turn involve use of appropriate > > files from: > > > > # ls -C1 ~/src.configs/* > > /root/src.configs/make.conf > > /root/src.configs/src.conf.CA7-dbg-clang.aarch64-host > > /root/src.configs/src.conf.CA7-nodbg-clang.aarch64-host > > /root/src.configs/src.conf.CA76-dbg-clang.aarch64-host > > /root/src.configs/src.conf.CA76-nodbg-clang.aarch64-host > > > > ~/src.configs/make.conf is common to all 8. > > Which is consistent with how one might setup a .sandbox-env for that > tree - it might do nothing more than set > > export __MAKE_CONF=$HOME/src.configs/make.conf > > > > > They also use my git worktree: /usr/main-src/ > > > > ( /usr/src/ is from PkgBase and, so, has no > > .git/ repository. A different /usr/*-src/ > > has the .git repository. ) > > > > > > I also have 7 scripts that run more than > > one of those ~/build-sys-*dbg-*.sh in a > > sequence: > > > > # ls -C1 ~/build-sys-*[67]-[kw]*.sh > > /root/build-sys-main-CA7-kernel.sh > > /root/build-sys-main-CA7-world-kernel.sh > > /root/build-sys-main-CA7-world.sh > > /root/build-sys-main-CA76-kernel.sh > > /root/build-sys-main-CA76-world-kernel.sh > > /root/build-sys-main-CA76-world.sh > > /root/build-sys-main-CA76_CA7-world-kernel.sh > > > > (Each of those 7 build both -dbg- and -nodbg- > > variations.) > > Sounds like something more suited to a set of targets - which could > reduce redundant work? > > > > > > > For reference: > > > > CA76: cortex-a76 (aarch64) > > CA7: cortex-a7 (armv7) > > > > # ls -dC1 /usr/obj/BUILDs/*/ > > /usr/obj/BUILDs/main-CA7-dbg-clang/ > > /usr/obj/BUILDs/main-CA7-nodbg-clang/ > > /usr/obj/BUILDs/main-CA76-dbg-clang/ > > /usr/obj/BUILDs/main-CA76-nodbg-clang/ > > > > I do not use ~/src.configs/make.conf with > > poudriere-devel for package builds. I avoid > > doing package builds outside of poudriere > > in normal circumstances. > > > > I normally do not build stable/* or releng/*.* > > systems, just using official FreeBSD builds > > for such. (Long ago I used to build more > > variations.) > > > > I only build amd64 systems on amd64; I only > > build aarch64 and armv7 on aarch64. (Long > > ago I used to cross build little endian > > systems on amd64.) > > We cross-build everything ;-) > -- Nuno Teixeira FreeBSD UNIX: <eduardo@FreeBSD.org> Web: https://FreeBSD.org [-- Attachment #2 --] <div dir="ltr"><div>Hello!</div><div><br></div><div><a href="https://reviews.freebsd.org/D50313">https://reviews.freebsd.org/D50313</a>; has been acepted and soon it will be committed.<br><br></div><div>I still have a lot of difficulties about how this works and how to be configured.<br></div><div>Thinking if it will be some config examples be provided?</div><div><br></div><div>Thanks</div></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">Simon J. Gerraty <<a href="mailto:sjg@juniper.net">sjg@juniper.net</a>> escreveu (terça, 13/05/2025 à(s) 20:52):<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Mark Millard <<a href="mailto:marklmi@yahoo.com" target="_blank">marklmi@yahoo.com</a>> wrote:<br> > > If you are going to head down that path, I would highly recommend<br> > > using the 'mk' wrapper from<br> > > <a href="https://www.crufty.net/ftp/pub/sjg/sb-tools.tar.gz" rel="noreferrer" target="_blank">https://www.crufty.net/ftp/pub/sjg/sb-tools.tar.gz</a><br>; > > We've used that model at work for over 20 years.<br> > > Described in <a href="https://www.crufty.net/sjg/docs/sb-tools.htm" rel="noreferrer" target="_blank">https://www.crufty.net/sjg/docs/sb-tools.htm</a><br>; > ><br> > > In a nutshell; each tree has a .sandbox-env file which can tune its<br> > > environment (as well as mark the top of the "sandbox").<br> > > There are a plethora of other hooks to tune.<br> > > I find it especially useful with Emac's M-x compile<br> > <br> > I will take a look.<br> > <br> > Using my aarch64 context as an example (it<br> > has more variations than my amd64<br> > environment, since I do nothing for i386<br> > but aarch64 is also set up for armv7):<br> <br> you can do combinations.<br> <br> Eg Junos currently builds for last I counted, over 35 combinations of<br> architecture and OS we provide symlinks to 'mk' to simply usage.<br> Eg. you can always do 'mk --machine arm64,aarch64' but 'mk-arm64,aarch64' is<br> less typing and allows for auto-completion.<br> <br> FreeBSD by uses MACHINE and MACHINE_ARCH as TARGET_SPEC_VARS,<br> Junos however only supports a single MACHINE_ARCH per MACHINE but<br> multiple TARGET_OS's (bsd15,bsd12,wrl9,...) so we use<br> MACHINE and TARGET_OS as TARGET_SPEC_VARS.<br> <br> You can of course extend that for your own use, so that you could have<br> mk --machine arm64,aarch64,something to set<br> TARGET_SPEC=arm64,aarch64,something<br> <br> but that assumes that the objects for arm64,aarch64,something should be<br> kept separate from those for arm64,aarch64,other<br> <br> If "something" and "other" really represent things to be built for<br> arm64,aarch64, then setting up targets for them is another option.<br> <br> There is nothing to stop you of course from using any of the scripts<br> below in conjuction with the setup supported by 'mk'.<br> <br> Things that everyone is likely to need/use can/should be setup as<br> targets so 'mk-arm64,aarch64 dbg-kernel' "just worked"<br> but things that do not fit that criteria are better served as scripts<br> such as you have.<br> <br> This is in some way a reflection of the difference b/w an emedded vendor<br> who is typically only interested in a very small sub-set of the<br> architectures supported by the project, but wants them handled the same<br> way by 100's if not 1000's of devs. Eg we put all our Juniper specific<br> targets under a juniper/ tree - that works with the same DIRDEPS_BUILD<br> bits as FreeBSD can.<br> <br> > I have 8 aarch64 scripts that have the<br> > likes of __MAKE_CONF (and more) specified<br> > that do individual system builds of main's<br> > kernel or world:<br> > <br> > # ls -C1 ~/build-sys-*dbg-*.sh<br> > /root/build-sys-main-CA7-dbg-kernel.sh<br> > /root/build-sys-main-CA7-dbg-world.sh<br> > /root/build-sys-main-CA7-nodbg-kernel.sh<br> > /root/build-sys-main-CA7-nodbg-world.sh<br> > /root/build-sys-main-CA76-dbg-kernel.sh<br> > /root/build-sys-main-CA76-dbg-world.sh<br> > /root/build-sys-main-CA76-nodbg-kernel.sh<br> > /root/build-sys-main-CA76-nodbg-world.sh<br> > <br> > The above in turn involve use of appropriate<br> > files from:<br> > <br> > # ls -C1 ~/src.configs/*<br> > /root/src.configs/make.conf<br> > /root/src.configs/src.conf.CA7-dbg-clang.aarch64-host<br> > /root/src.configs/src.conf.CA7-nodbg-clang.aarch64-host<br> > /root/src.configs/src.conf.CA76-dbg-clang.aarch64-host<br> > /root/src.configs/src.conf.CA76-nodbg-clang.aarch64-host<br> > <br> > ~/src.configs/make.conf is common to all 8.<br> <br> Which is consistent with how one might setup a .sandbox-env for that<br> tree - it might do nothing more than set<br> <br> export __MAKE_CONF=$HOME/src.configs/make.conf<br> <br> > <br> > They also use my git worktree: /usr/main-src/<br> > <br> > ( /usr/src/ is from PkgBase and, so, has no<br> > .git/ repository. A different /usr/*-src/<br> > has the .git repository. )<br> > <br> > <br> > I also have 7 scripts that run more than<br> > one of those ~/build-sys-*dbg-*.sh in a<br> > sequence:<br> > <br> > # ls -C1 ~/build-sys-*[67]-[kw]*.sh<br> > /root/build-sys-main-CA7-kernel.sh<br> > /root/build-sys-main-CA7-world-kernel.sh<br> > /root/build-sys-main-CA7-world.sh<br> > /root/build-sys-main-CA76-kernel.sh<br> > /root/build-sys-main-CA76-world-kernel.sh<br> > /root/build-sys-main-CA76-world.sh<br> > /root/build-sys-main-CA76_CA7-world-kernel.sh<br> > <br> > (Each of those 7 build both -dbg- and -nodbg-<br> > variations.)<br> <br> Sounds like something more suited to a set of targets - which could<br> reduce redundant work?<br> <br> > <br> > <br> > For reference:<br> > <br> > CA76: cortex-a76 (aarch64)<br> > CA7: cortex-a7 (armv7)<br> > <br> > # ls -dC1 /usr/obj/BUILDs/*/<br> > /usr/obj/BUILDs/main-CA7-dbg-clang/<br> > /usr/obj/BUILDs/main-CA7-nodbg-clang/<br> > /usr/obj/BUILDs/main-CA76-dbg-clang/<br> > /usr/obj/BUILDs/main-CA76-nodbg-clang/<br> > <br> > I do not use ~/src.configs/make.conf with<br> > poudriere-devel for package builds. I avoid<br> > doing package builds outside of poudriere<br> > in normal circumstances.<br> > <br> > I normally do not build stable/* or releng/*.*<br> > systems, just using official FreeBSD builds<br> > for such. (Long ago I used to build more<br> > variations.)<br> > <br> > I only build amd64 systems on amd64; I only<br> > build aarch64 and armv7 on aarch64. (Long<br> > ago I used to cross build little endian<br> > systems on amd64.)<br> <br> We cross-build everything ;-)<br> </blockquote></div><div><br clear="all"></div><br><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><font color="#888888">Nuno Teixeira</font></div><div><div><font color="#888888"> FreeBSD UNIX: <eduardo@FreeBSD.org> Web: <a href="https://FreeBSD.org" rel="noreferrer" target="_blank">https://FreeBSD.org</a><br></font></div></div></div></div>;help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFDf7UJsvGZiyUYMKm-1dNt7wXS1uw2ofbLLSMydJumexGzHyA>