From owner-freebsd-current@freebsd.org Sun Aug 19 14:39:37 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BAA73106A8E1 for ; Sun, 19 Aug 2018 14:39:36 +0000 (UTC) (envelope-from gurenchan@gmail.com) Received: from mail-it0-x22b.google.com (mail-it0-x22b.google.com [IPv6:2607:f8b0:4001:c0b::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5B1537969F for ; Sun, 19 Aug 2018 14:39:36 +0000 (UTC) (envelope-from gurenchan@gmail.com) Received: by mail-it0-x22b.google.com with SMTP id j81-v6so17284640ite.0 for ; Sun, 19 Aug 2018 07:39:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=7V6p4MCd9t7Y1d54rQdFGuusOUfwNyk/psx0VumfXYA=; b=hkDm86YCferFLRHJccO18afDYJffe9AMEvawnktVSQ4/+pgboK/n9G3XIjv+q93+8Q bNm8JGEivX8nICdJLLldz5P9kVDy+kNmxSi44xDBlSLzjccLXfm60k+DHQ24cMEDCui5 xI/4Dlq7MpnQbFFMyFceDNnfMpbOykJcL38Gi92a/WFblvW9K5yKl3in2XG5JqsrwyWa 6wTxu+42RjwhySGe/LyNVHSWwIXGqEQmNOFlGyVxkNz8ozQi5YVPjUEW5eBG31DwtWIK 72icYZsvYraRi8tOJK5xUXSqfEYGTiZb5GTHbvCd9wCuMr19eh9sldFITRZdFrzlkGpx RrwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=7V6p4MCd9t7Y1d54rQdFGuusOUfwNyk/psx0VumfXYA=; b=ACx0vRgIT0CXLBtjGNmEMDl+3szserWkK2rnftxD8RV8iMq+apy1CEA6QGyERHbCOF NumRk/mp2RffhZ2ZwNRJx3l/zLB0NhOiCT9OIoHdzYj5Bry6DSlEbNi1erI/9DqNQBtM N8GjDXRjjqwdKD3q+kThl3sS+ACBTJyZD7zbhkGPRLYiqsLH5PC1je7nVpRxfDW9ktfI rR00bSoVX9rRkniw+xVXiLcNNvseFgwP/ownaT9cPcHllv2h1FNgiAg6l88+DhDB/6Dh LEydcHynQHYJKMPiMJCyQv1ln1wQGCYz+iZYZ0mymsfvubjCUn/fC+h/Vj/Rh/hT4HDL msNg== X-Gm-Message-State: AOUpUlESAvEd52snwPN/Bl1A5onjHakBQTbFOriyUdZ2bELNx9wdHmVF eBmjrl5gcRrH3lc8pO6z/63zPpTGrcGunsHpH2cw1h3U X-Google-Smtp-Source: AA+uWPwxg6NmB1buhgjeG8qHiLJKhjpWPzbpMTiB5QsbQT4f+IPI+XdHJFKkT+ltkSHkEDvb8EkxbbvtSycU8PcmxkU= X-Received: by 2002:a24:7f94:: with SMTP id r142-v6mr32046203itc.137.1534689575737; Sun, 19 Aug 2018 07:39:35 -0700 (PDT) MIME-Version: 1.0 References: <20180818223420.rjisst4vuxzxbcrl@kazhap> <20180819122727.4d22f99d@thor.intern.walstatt.dynvpn.de> In-Reply-To: <20180819122727.4d22f99d@thor.intern.walstatt.dynvpn.de> From: blubee blubeeme Date: Sun, 19 Aug 2018 22:39:24 +0800 Message-ID: Subject: Re: Sharing compiled builds between multiple 12-CURRENT boxes. To: "O. Hartmann" Cc: mail@dbalan.in, FreeBSD current Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Aug 2018 14:39:37 -0000 On Sun, Aug 19, 2018 at 6:30 PM O. Hartmann wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > Am Sun, 19 Aug 2018 00:34:20 +0200 > Dhananjay Balan schrieb: > > > Hi, > > > > I run 12-CURRENT on few machines, some more powerful that other (all > > of them x86_64, march varies). > > > > Is there is a way to avoid building CURRENT on all machines? Rather > > than building everywhere, can I just build it on the big server that I > > have and copy and update my laptop? > > > > - > > Dhananjay Balan > [...] > > Yes, you can. > We do this via a custom build and creating packages as this is introduced > at > > https://wiki.freebsd.org/PkgBase > > But beware! As many others have already stated, take care about to use th= e > least common > denominator of achitectural specifications amongst your pools! This means > to not use any > kind of optimizations for a specific CPU type for pkg-base distributed > builds! > > Because we build the local OS for fast/server machines always(!) with > optimisations, the > pkg-base builds are done in a separate way - which is very easy if you've > once understood > the really neat and great build framework of FreeBSD's! > > Instead of building the traditional (probably optimised) system from > /usr/src > and /usr/obj, now you've to consider a separate path like > /pool/sources/CURRENT/src (our > way, since we also try to build packages and object trees for 11.X), then > setup a small > build script that essentially sets > > MAKEOBJDIRPREFIX > KERNCONFDIR > KERNCONF > > and especially > > __MAKE_CONF > SRCCONF > SRC_ENV_CONF > > if you use usually optimisations for the native target build on the > building host and > more generic binaries for the Intel x86 crap for redistribution. > > Doing so, we also perform builds for some ARM64 based experimental boxes. > > The next step is then up to you how to distribute. We copy all the pkg > stuff coming out > of the build cycle to a folder which is accessible by pkg via HTTP(S) - s= o > www/apache24 > is our platform to redistribute the binaries over the network and even to > remote sites > (beware of the security implications!). You also can distribute the > obj-folder (/usr/obj, > or, if using another approach, like /pool/sources/CURRENT/obj) via NFS. > > Once you've understood the (easy to learn) concept of building the source > tree, creating > the packages for pkg-base and having dealt with the more labor for the > setting of a > distribution server, you can use the most potent server/box on you networ= k > for building > dedicated distributions > > Even a "Release" build is possible as long as there are not pitfalls like > they occured > during the transition from 11.X to more 12-CURRENT spcific development > (i.e. LINT). > > If you use pkg-base as mentioned above, be aware to setup a proper > pkg.conf file as > introduced on the Wiki and please be also aware of the fact, that there i= s > at the moment > no sharp separation between pkg-base and oridnary pkg for packages - so > take the > warinig serious, that pkg delete -a will not only kill all packages, it > also will kill > all packages installed for the base system! > > Once installed you'll see how fast compared to source build the pkg-base > installation > is (although I still prefer source build, optimised builds ...). > > And by the way: depending on the sophistication of your build script for > dedicated > tree builds as mentioned, via manipulations of __MAKE_CONF, SRCCONF you'r= e > able to also > build optimised binary trees for different CPU types, but you have to dea= l > yourself > with the fact that you've to put the binaries into a proper place and the= n > delegate the > URI in pkg.conf to the correct branch of your tree. The ABI environmental > variable > doesn't take care of the "set" you may have used to optimise. But this is > something > you're able to deal with easily yourself after having setup your basic > build > environment. > > Regards, > > oh > > - -- > O. Hartmann > > Ich widerspreche der Nutzung oder =C3=9Cbermittlung meiner Daten f=C3=BCr > Werbezwecke oder f=C3=BCr die Markt- oder Meinungsforschung (=C2=A7 28 Ab= s. 4 BDSG). > -----BEGIN PGP SIGNATURE----- > > iLUEARMKAB0WIQQZVZMzAtwC2T/86TrS528fyFhYlAUCW3lGDwAKCRDS528fyFhY > lMBeAf4nVIFEAgegbZyKDZvTQQD/9Q8mN+IY8aJ7Fzza23KgWQsM3ZP59Orh0mi2 > PA94ywn5hOjfTgzdYcp1lq3MCE84AgCBGAWAMTag87ru89JHm75NLsgDvEjPpYgG > 9hW1Vrdoj9EeiR2HTv2ncTc/AkFPNhdyhe+EJqUg01rtAd9sdmdM > =3D/rzO > -----END PGP SIGNATURE----- > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org= " > I would say setup a poudriere build bot on your most powerful machine to build and test all the applications and tools that your weaker machines need, then setup a custom pkg repo so once poudriere builds all the ports and create packages from them. Then you use the custom pkg url to update your other machines. Quick primer on poudriere: https://www.freebsd.org/doc/handbook/ports-poudriere.html Digital Ocean did a write up on exactly what you're requesting here: https://www.digitalocean.com/community/tutorials/how-to-set-up-a-poudriere-= build-system-to-create-packages-for-your-freebsd-servers Best, Owen