Date: Sat, 15 Feb 2020 19:14:03 -0500 From: Dan McGrath <danmcgrath.ca@gmail.com> To: "@lbutlr" <kremels@kreme.com> Cc: FreeBSD <freebsd-ports@freebsd.org> Subject: Re: Starting with poudriere Message-ID: <CAK82gMFVQuTzFBXYk0QVu-5gGwxaUHvdyGP_03LmOzkSN5uoNA@mail.gmail.com> In-Reply-To: <3743CEAE-BCC9-479E-8367-F3DA0E30496E@kreme.com> References: <3743CEAE-BCC9-479E-8367-F3DA0E30496E@kreme.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi,
Just a bit of a heads up that poudriere will require you to be on the new
version of FreeBSD before you can build for it on the current system. For
example, if you are running 12.1, and you upgrade poudriere's jail to 13.0,
it will complain that you have to be running that version on the host you
are using it on. Ideally, poudriere should be running on it's own dedicated
system, not the one you intend to deploy to.
As for how I use it here, I use salt to deploy to the poudriere machine,
with one repo per host/jail. It's been good, but things get very hairy with
this approach since all of the ports need to be recompiled whenever you
update the jail (say from 12.1-RELEASE-p1 to 12.1-RELEASE-p2). When you
start to have multiple port trees, multiple jails and multiple sets, the
permutations of this start to really kick you in the face :) I built on a
tiny old quad core VM, which makes matters worse.
In terms of how I actually use things, originally I started off doing
"poudriere options ...", and would go through pressing enter for every menu
config option, and for each repo (host/jail) that I have. This was a pain,
especially during quarterly changes where the on disk files in
/usr/local/etc/poudriere.d/ for each <jail>-<tree>-<set> directory would
have to be copied to a new filename in order to start from the previous
settings, and then change what is new etc. Eventually I learned that I
could just erase all that, skip the "poudriere options", and specify the
defaults that I want/need to override in the
/usr/local/etc/poudriere.d/make.conf file. That I could specify options per
port, as needed with something like this:
OPTIONS_SET+=3D GSSAPI_NONE
OPTIONS_UNSET+=3D CUPS DBUS X11 DOCBOOK EXAMPLES KERBEROS PERFSCHEMA PERFSC=
HM
GSSAPI_BASE GSSAPI_HEIMDAL GSSAPI_MIT WAYLAND OPENGL XCB
DEFAULT_VERSIONS+=3D ssl=3Dopenssl
# Python
DEFAULT_VERSIONS+=3D python=3D3.7 python3=3D3.7
# Ensure apache uses event MPM or ZTS will disable and php extensions will
fail
# to build inside poduriere!
WITH_MPM=3Devent
# Apache 2.4
www_apache24_UNSET+=3D MPM_PREFORK MPM_WORKER
www_apache24_SET+=3D MPM_EVENT
# Ensure ZTS
www_mod_php72_SET+=3D ZTS
lang_php72_SET+=3D ZTS
That should get you an idea how to setup the make.conf, at least. With this
approach, I don't have to deal with the headaches of going through millions
of menu options every time stuff changes. While I still need to migrate to
more of a "single repo" approach, it is nice having dedicated repos for
each jail, but it does mean that you are compiling the same stuff over and
over, so be warned! Interesting note was that I originally had ccache
installed in the system, and even with SSD's over network storage, was
finding that the I/O was actually adding to my compile times, so I just
disable it now.
As for your question of the file to use to build a bunch of ports, the
approach I use is a layout like this:
/poudriere/
/poudriere/113amd64
/poudriere/113amd64/2019Q4
/poudriere/113amd64/2019Q4/host_a
/poudriere/113amd64/2019Q4/host_b
/poudriere/113amd64/2020Q1
/poudriere/113amd64/2020Q1/host_a
/poudriere/113amd64/2020Q1/host_b
/poudriere/113amd64/bulk-2019Q4.sh
/poudriere/113amd64/bulk-2020Q1.sh
/poudriere/113amd64/clean-2019Q4.sh
/poudriere/113amd64/clean-2020Q1.sh
/poudriere/121amd64
/poudriere/121amd64/2019Q4
/poudriere/121amd64/2019Q4/host_a
/poudriere/121amd64/2019Q4/host_b
/poudriere/121amd64/2020Q1
/poudriere/121amd64/2020Q1/host_a
/poudriere/121amd64/2020Q1/host_b
/poudriere/121amd64/bulk-2019Q4.sh
/poudriere/121amd64/bulk-2020Q1.sh
/poudriere/121amd64/clean-2019Q4.sh
/poudriere/121amd64/clean-2020Q1.sh
/poudriere/bulk-all.sh
/poudriere/update-2019Q4.sh
/poudriere/update-2020Q1.sh
Where the bulk-all.sh script just runs the bulk-<port quarter>.sh and clean
scripts. As sample of one of them would be:
#!/bin/sh
#
# Because variables
#
JAIL=3D"113amd64"
TREE=3D"2020Q1"
LIST_PREFIX=3D"/poudriere/${JAIL}/${TREE}"
# Bulk compile
for lcv in ${LIST_PREFIX}/*; do
echo "Bulk for $lcv..."
# read -p "Press ENTER to continue. " INPUT
nice poudriere bulk -j $JAIL -p $TREE -z $(basename ${lcv}) -f
"${lcv}"
done
Of course, the clean script would replace s/bulk/clean/, but you get the
idea. More of a prototype script than anything, but it "works", although
could most certainly be done better!
Hopefully this gives you a bit of insight and understanding into the
system, and possible approaches to setting things up. Best of luck! o/
Cheers,
Danny McGrath
On Sat, Feb 15, 2020 at 6:33 PM @lbutlr <kremels@kreme.com> wrote:
> I=E2=80=99ve setup a new FreeBSD 12.1 system and am going to give poudrie=
re a
> whirl on it.
>
> The machine is currently setup as a two drive zfs mirror and I have used
> pkg to install a few basic things (sudo, zsh, poudriere itself, etc).
>
> I am reading through <
> https://wiki.freebsd.org/VladimirKrstulja/Guides/Poudriere#A3._Create_the=
_list_of_ports_wanted_in_the_repo>
> and I have some questions.
>
> First, the example of says that if we don=E2=80=99t change any options, t=
he
> defaults will be used. OK, but it makes no mention on how we might change
> or set options? Is this simply talking about the build options panels tha=
t
> sometimes come up (Build examples, support ssl, use sqlite or MySQL, etc)=
,
> and when I first go to build a =E2=80=9Cpersistent list=E2=80=9D those op=
tion screens will
> come up and I can make changes and they will be saved? Or is there
> something else that I need to do?
>
> Second, let's say that I want to setup the machine to build postfix,
> dovecot, apache, pigeonhole, marisadb, postfixadmin, rpoundcube, and hord=
e
> all in on =E2=80=9Cpersistent list=E2=80=9D since all of these pacajkges =
are need to do
> mail and webmail, do I simply list the packages in a single file the way
> the page lists www/firefox?
>
> Once I have poudriere setup and running and I=E2=80=99m comfortable with =
it, do I
> need to do anything to have it =E2=80=9Ctake over=E2=80=9D sudo and anyth=
ing else that I
> installed via pkg?
>
> Sorry for the rather basic questions.
>
>
> --
> "Are you pondering what I'm pondering?"
> "I think so, Brain, but why would anyone want to see Snow White and
> the Seven Samurai?=E2=80=9D
>
>
> _______________________________________________
> freebsd-ports@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-ports
> To unsubscribe, send any mail to "freebsd-ports-unsubscribe@freebsd.org"
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAK82gMFVQuTzFBXYk0QVu-5gGwxaUHvdyGP_03LmOzkSN5uoNA>