Date: Thu, 27 Dec 2012 16:42:48 +1100 From: "Michael Vale" <masked@internode.on.net> To: <freebsd-hackers@freebsd.org>, <freebsd-arch@freebsd.org>, <freebsd-ports@freebsd.org> Subject: Cross Compiling of ports Makefiles. Message-ID: <E49F944634B24A7387BC937CBE19D2D9@forexamplePC>
next in thread | raw e-mail | index | archive | help
Hi,=20
For those of you who are aware I=E2=80=99ve been implementing a complete =
cross-compiling series of functions to ports makefiles.
I had a good 3+ week break since my last email with a patch to show, and =
I=E2=80=99ve totally re-written it and have started from scratch. Not =
including any of Ray=E2=80=99s Zrouter code either.
While it=E2=80=99s still a work in progress, i have outlined the entire =
system to produce target installs into the same staging directory as a =
bsd system ready to be flashed onto NAND for embedded, complete with pkg =
registry and ldconfig, everything has been thought of. - The reason I =
have chosen this method for the ports to be installed into a tree is so =
they can be compliled after build/install kernel/world and be combined =
into one firmware image seemlessly. Some ports won=E2=80=99t just be =
optional applications for future embedded firmware images, =
they=E2=80=99ll be an integral part of it. The goal here is to be able =
to build complete firmware images in one fowl swoop. Perhaps beyond the =
scope most of you out there but I may wish to pick and choose exclude =
required parts of the BSD system and replace them with the busybox port =
and replace libc with google=E2=80=99s Bionic, uClibc or even musl. =
This cannot be achieved currently with the likes of tinderbox and =
pourdiere
It will still be possible to build packages though.
Due to the nature of cross building first i=E2=80=99ll lay out the =
options and then tell you which one I am implementing first as there are =
reasons for having different build-enviornments/toolchains.
Ok, firstly I was going to give you all detail of all possible =
cross-compiling scenarios as I outline them. but I=E2=80=99ll have you =
know it=E2=80=99s much of a muchness, there is the pros and cons to each =
and every different step, the one i=E2=80=99m about to put to you now is =
the most feature complete and quickest to implement. That =
doesn=E2=80=99t mean building without a DESTDIR JAIL in the future and =
just using the build system and it=E2=80=99s tools without a new =
toolchain doesn=E2=80=99t make sense (sometimes it does!) and that =
i=E2=80=99m not going to do it or that I=E2=80=99m not going to do a =
full '=E2=80=99Canadian Cross=E2=80=99.
Ultimately as a goal the minimal command do invoke cross compliation is =
TARGET(_ARCH)=3D${ARCH} make.
This could go on for hours, so after just deleted to extra paragraphs, =
i=E2=80=99m going to summerise.
first we check for CLANG (as the x-compiler) or if we need to install =
xdev (bsd make of gcc compiled for target arch).
(ok so some of this wont be in Makefile order (upside down and back to =
front), but im just spitting it out as it comes)
if GNU configure is used, it usually pretty good at detecting the =
compilers executable path from the TARGET triple alone, for worse case =
scenario also set ${CC}=E2=80=99s path at the beginning of global env =
${PATH} to override any subsequent.
pre-chroot: is mostly used to declare global env variables to keep the =
build from failing and making sure the install will complete.
do-chroot: and we have to firstly install and BUILD_DEPENDS, remember =
these can be libraries too and they have to be built with the build =
machines usual stuff and installed in their usual place (lucky we are =
using a CHROOTED JAIL here! we could easy make a mess otherwise) =
remembering sometimes some depends can be both a BUILD dep AND a RUN dep =
to the TARGET. That=E2=80=99s okay, they should always be declared as =
correctly and never have to cross-compile a BUILD depend. However a =
BUILD depend can be build twice, (once for the build system) and again =
(as a TARGET) for the TARGET as a RUN depend for the TARGET.
The beauty of doing this work is we can now treat the lib and run =
depends more suitably. During this process we can strip the libs, =
exclude the headers and change the directory structure to one, save on =
inodes, and second pkg register, libtool and ld require the files are =
installed into the root tree correctly in order for them to build valid =
databases and register them. Now, BUILD/HOST system has already had =
it=E2=80=99s tail cut off by DESTDIR. Now there is plenty of ways we =
can install everything into a valid sub-directory and have DESTDIR still =
considered ROOT and PREFIX or LOCALDIR doesn=E2=80=99t have some obscure =
prepending directory that doesn=E2=80=99t exist in the =
CROSS_STAGING_ROOT. Some ways include adding a variable in bsd.lib.mk =
and in every single one of make=E2=80=99s install targets between =
${DESTDIR} and ${LOCALBASE} or ${PREFIX}. And we could include if =
statements for cross, this would leave it at that and we could go ahead =
and simply install into a sub-directory before pkg, ldconfig and =
firmware image packing occurs, but I=E2=80=99d rather keep all =
cross-building to bsd.cross.mk and include it in bsd.port.mk and instead =
within DESTDIR do-chroot: re-define ${DESTDIR} as ${_bldroot}${DESTDIR} =
and all TARGET_LIBS, RUN_DEPENDS and TARGET install in a CHROOTED=3Dno =
chroot.
Doing the same thing could also prevent the need for a DESTDIR JAIL =
install at all and just use the real build machine=E2=80=99s build env, =
rather than a jail. Regardless. We still have to install these targets =
and their DESTDIR is skewed. There is a few options,
One is to have a MAKEOBJDIRPREFIX like option, and redefine every =
target=E2=80=99s DESTDIR ${makeobjDESTDIR} before running do-install. =
Now i=E2=80=99ve yet to complete this stage, but I believe this is the =
way to do it.
There are other options but they aren=E2=80=99t as elegant/will make =
baby jesus cry.
Now the install of these targets won=E2=80=99t require a chroot. A =
chroot could be done, and that would be okay for one port. but if there =
is already a cross compiled system in there ready for flashing to disk, =
theres no way to chroot without moving files temporarially form the =
existing target system and copying or building programs like /bin/sh =
that will execute on the build machine and allow chroot to run.
We can patch/sed PLIST files, for pkg register to work, patch/sed/edit =
ldconfig=E2=80=99s db, and some other steps. But I don=E2=80=99t like =
that idea.
that=E2=80=99s why I=E2=80=99m opting with the other option and that is =
to create some INSTALL_DEPENDS or =
CROSS_COMPILING_CHROOT_INSTALL_DEPENDS, if you will. just /bin/sh and =
another few=20
build TARGET port in jailed DESTDIR/CHROOTED=3Dyes.
this is achieved by installing all build dependencies first...
Sorry, I=E2=80=99m too tired to continue on any further!
I wanted to wait until the initial plan works, shoot an email off then =
get into the good stuff. But it=E2=80=99s taking me longer than I =
thought even just to describe all the processes.
I didn=E2=80=99t want to submit half-baked Makefiles that don=E2=80=99t =
work, but I can only write about half of one anyway haha!
Anyway, I=E2=80=99m going to spend some time working on them in the next =
few days, so please expect an update.
From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 27 09:40:47 2012
Return-Path: <owner-freebsd-hackers@FreeBSD.ORG>
Delivered-To: freebsd-hackers@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
by hub.freebsd.org (Postfix) with ESMTP id C9D0FD37
for <freebsd-hackers@freebsd.org>; Thu, 27 Dec 2012 09:40:47 +0000 (UTC)
(envelope-from wojtek@wojtek.tensor.gdynia.pl)
Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl
[188.252.31.196])
by mx1.freebsd.org (Postfix) with ESMTP id 12C5E8FC13
for <freebsd-hackers@freebsd.org>; Thu, 27 Dec 2012 09:40:46 +0000 (UTC)
Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1])
by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qBR9Iljq035122
for <freebsd-hackers@freebsd.org>; Thu, 27 Dec 2012 10:18:47 +0100 (CET)
(envelope-from wojtek@wojtek.tensor.gdynia.pl)
Received: from localhost (wojtek@localhost)
by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qBR9Ilc2035119
for <freebsd-hackers@freebsd.org>; Thu, 27 Dec 2012 10:18:47 +0100 (CET)
(envelope-from wojtek@wojtek.tensor.gdynia.pl)
Date: Thu, 27 Dec 2012 10:18:47 +0100 (CET)
From: Wojciech Puchar <wojtek@wojtek.tensor.gdynia.pl>
To: freebsd-hackers@freebsd.org
Subject: cvs deprecated
Message-ID: <alpine.BSF.2.00.1212271016340.35118@wojtek.tensor.gdynia.pl>
User-Agent: Alpine 2.00 (BSF 1167 2008-08-23)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7
(wojtek.tensor.gdynia.pl [127.0.0.1]); Thu, 27 Dec 2012 10:18:47 +0100 (CET)
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers>;
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 27 Dec 2012 09:40:47 -0000
but i can't find "moron guide" for using svn to update tree.
I never used cvs or svn myself just want to
1) get latest FreeBSD 9-* sources
2) get latest HEAD sources.
could someone just tell me what command (and/or config file) i have to use
today, as cvs will not work soon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E49F944634B24A7387BC937CBE19D2D9>