Date: Sat, 27 Apr 2002 17:04:43 +0300 From: Ruslan Ermilov <ru@FreeBSD.org> To: John Baldwin <jhb@FreeBSD.org> Cc: Marcel Moolenaar <marcel@xcllnt.net>, cvs-committers@FreeBSD.org, cvs-all@FreeBSD.org, Makoto Matsushita <matusita@jp.FreeBSD.org> Subject: Re: cvs commit: src Makefile Makefile.inc1 src/etc Makefile src/ Message-ID: <20020427140443.GA35685@sunbay.com> In-Reply-To: <XFMail.20020427002332.jhb@FreeBSD.org> References: <20020427033118.GA583@athlon.pn.xcllnt.net> <XFMail.20020427002332.jhb@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --]
On Sat, Apr 27, 2002 at 12:23:32AM -0400, John Baldwin wrote:
>
> On 27-Apr-2002 Marcel Moolenaar wrote:
> > On Fri, Apr 26, 2002 at 11:10:22PM -0400, John Baldwin wrote:
> >>
> >> distribute stuff already uses what is in /usr/obj, so it would just simply
> >> involve adding an extra buildworld after the world. I would actually prefer
> >> that as it still preserves the "clean room" atmosphere that a release is
> >> supposed to have.
> >
> > How would this work for a cross-release? The bits in /usr/obj may not
> > be for the architecture you're building the release on.
>
> The cross-release kind of would work like this:
>
> 1. You start off with matching src/ and obj/ like we do now.
> 2. You install that obj/ into the chroot.
> 3. You do a non-cross make world in the chroot like we do now.
> 4. What we do next is do all the make distribute stuff using
> what is in /usr/obj in the chroot.
>
> We do step 3 to try to get "clean room" binaries to use in stage 4.
>
> Now for a cross-release we need to make sure the binaries in /usr/obj
> in the chroot are cross-built binaries. Ruslan's current approach is
> to do this by make step 3 be a cross-buildworld instead of a full world.
> This means, then, that any tools for the new world you need for 4
> need to be built as build-tools or cross-tools or the like. What I'm
> suggesting is instead to insert a step at 3.5 to do a cross-build world
> and then we still have the right tools installed and don't have to worry
> about using the right build/cross tools for the release scripts.
>
After so much noise, let me tell you the truth now. :-)
The old version worked like this (the numbering does not resemble
the release.X steps):
1. Install the same world that you're currently running into a chroot
environment.
2. Check out fresh sources in the chroot's /usr/src and chroot into
there. (The next steps are done in the chroot'ed environment.)
3. Build world from /usr/src and install it under root. This has
the following disadvantages:
o Installworld from step 1 should install the world _compatible_
with the world built in this step, otherwise the new world may
not be able to run with the current kernel. That means that
you should run a relatively fresh (ideally, the same /usr/src
stuff as in chroot) world before rolling a release. That's
why I wrote that my patch obviates the need to have an up-to-
date world.
o You can't install and run the world for a different arch.
4. Distribute the same world under /R/stage/trees, and make
additional distributions like krb4, krb5 and fix non-crypto
stuff (FIXCRYPTO). (The old version fixed non-crypto stuff
later, for no apparent reason.)
5. Rebuild GENERIC and additional kernels in the new world
environment.
6. Build crunched binaries, boot and fixit.
7. Create distribution tarballs using some sample tools like
sed(1) and tar(1).
8. Build mfsroot.flp: copy some misc. stuff from the either
the / (e.g. /sbin/dhclient-script) or /usr/src (etc/services),
set up the "boot" crunched binary from step 6, copy the
necessary device nodes, docs, boot blocks, etc., unnecessarily
re-make _all_ the GENERIC kernel's modules we already built in
step 5, and install a subset of them under mfsfd/stand/modules,
and finally create a floppy image with this stuff with the
doFS.sh script.
9. Create the BOOTMFS kernel config, compile the BOOTMFS kernel,
and install it in two incarnations, small (kern.flp) and big
(boot.flp) that also has a copy of mfsroot.flp.
10. Create a fixit.flp using the "fixit" crunched binary we built
in step 6 and copy some stuff from the "base" dist.
11. Set up FTP and CD-ROM distributions.
That's how it works now:
1. Install the same world that you're currently running into a chroot
environment. Note: this world doesn't need to be very fresh, it
should only be compatible with the currently running kernel, for
obvious reasons.
2. Check out fresh sources in the chroot's /usr/src and chroot into
there. (The next steps are done in the chroot'ed environment.)
3. Build world from /usr/src but _do not_ install it under root.
This has the following advantages:
o Installworld from step 1 _does not_ need to be compatible with
the world built in this step.
o We may build world for a different arch (no knobs are provided
in this milestone commit).
4. Distribute the world we built in step 3 under /R/stage/trees, and
make additional distributions like krb4, krb5 and fix non-crypto
stuff (FIXCRYPTO). Note the new way how additional distributions
are built: because we do not install the new world (it may be for
a different arch or be incompatible with the currently running
kernel) they are built the same way as the main world, as the
sub-world (thanks to the SUBDIR_OVERRIDE knob) using the right
cross-tools (compiler, binutils, etc.), libraries and headers
from ${WORLDTMP}.
5. Rebuild GENERIC and additional kernels using the "buildkernel"
Makefile.inc1 target which uses the right cross-tools.
6. Build crunched binaries, boot and fixit, using the right
cross-tools, libraries and headers. This is done by importing
the WMAKEENV idea from ../Makefile.inc1. Note: we may need
to bootstrap crunchgen(1) to do it. (Just to avoid any possible
ambiguities, bootstrapping means "rebuild crunchgen(1) from
/usr/src in the running (host) environment, using the /usr/bin/cc
compiler, /usr/lib libraries, and /usr/include headers, and
install them under root. But so far (even for 4.x doing the
5.0 release) this is not necessary. Anyway, I plan to fix it
so that we eliminate any possible problems if someone (joe@?)
adds some nice new stuff to crunchgen(1). :-)
7. Create distribution tarballs using some sample tools like
sed(1) and tar(1). This step was not even touched by my
changes because it does not depend on the new world's
binaries (unless we introduce some incompatibilities in
sed(1) and tar(1), in which case we'd need to bootstrap
them too the same way as we (will) bootstrap crunchgen(1).
8. Build mfsroot.flp: copy some misc. stuff from the "base"
dist (compare with the old version) or /usr/src, set up
the "boot" crunched binary from step 6, copy the
necessary device nodes, docs, boot blocks (from "base"
dist), etc., install the subset of the GENERIC kernel's
modules we already built in step 5, and finally create a
floppy image with this stuff with the doFS.sh script,
making sure to use the boot blocks from the "base" dist
(remember: we are aimed at different arch support).
9. Create the BOOTMFS kernel config, compile the BOOTMFS kernel
with Makefile.inc1:buildkernel, using the right cross-tools,
and install it in two incarnations, small (kern.flp) and big
(boot.flp) that also has a copy of mfsroot.flp. (See how
doMFSKERN was fixed to achieve this.) I have some minor
optimizations planned for this step too like avoiding the
unnecessary "buildkernel" step on the second invokation.
10. Create a fixit.flp using the "fixit" crunched binary we built
in step 6 and copy some stuff from the "base" dist. This
stage wasn't modified.
11. Set up FTP and CD-ROM distributions. These stages were not
modified either.
I have intentionally excluded the doc.1 and ports.1 steps from
this discussion, because they seem to be not affected by these
changes.
You'll probably not be surprised now that the milestone #2 commit
that will bring us to the level of supporting cross-arch "make
release"s will be a really minor patch for release/Makefile that
will:
1. Borrow the TARGET_ARCH and TARGET definitions from Makefile.inc1.
2. Rename MACHINE_ARCH -> TARGET_ARCH and MACHINE -> TARGET all
around release/Makefile.
3. Ensure world-related targets are passed the TARGET_ARCH and TARGET.
[4. There will probably some other fixups that I missed in the
milestone #1 sweep.]
Cheers,
--
Ruslan Ermilov Sysadmin and DBA,
ru@sunbay.com Sunbay Software AG,
ru@FreeBSD.org FreeBSD committer,
+380.652.512.251 Simferopol, Ukraine
http://www.FreeBSD.org The Power To Serve
http://www.oracle.com Enabling The Information Age
[-- Attachment #2 --]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (FreeBSD)
Comment: For info see http://www.gnupg.org
iD8DBQE8yq/7Ukv4P6juNwoRAoJ9AJ44TjlROn8H3EAiVZBXPreVPMT1IACePOp9
qRARZvMUtUiGP81QeanvPes=
=88MI
-----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020427140443.GA35685>