From owner-freebsd-pkg@freebsd.org Fri Sep 27 14:45:00 2019 Return-Path: Delivered-To: freebsd-pkg@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8176B124E0A for ; Fri, 27 Sep 2019 14:45:00 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46fvhr2mYNz3PnQ for ; Fri, 27 Sep 2019 14:45:00 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp.infracaninophile.co.uk [IPv6:2001:8b0:151:1:c4ea:bd49:619b:6cb3]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "Let's Encrypt Authority X3" (verified OK)) (Authenticated sender: matthew/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 2513F1F655 for ; Fri, 27 Sep 2019 14:45:00 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from leaf.local (unknown [88.212.184.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: m.seaman@infracaninophile.co.uk) by smtp.infracaninophile.co.uk (Postfix) with ESMTPSA id CA53C22BE for ; Fri, 27 Sep 2019 14:44:58 +0000 (UTC) Authentication-Results: smtp.infracaninophile.co.uk; dmarc=none (p=none dis=none) header.from=FreeBSD.org Authentication-Results: smtp.infracaninophile.co.uk/CA53C22BE; dkim=none; dkim-atps=neutral Subject: Re: poudriere: build on on server, rsync to another To: freebsd-pkg@freebsd.org References: <6c130e1c-57df-40da-b9ae-169f8b20238d@www.fastmail.com> From: Matthew Seaman Message-ID: <5404b363-e887-728b-112d-a125aa8f8d5b@FreeBSD.org> Date: Fri, 27 Sep 2019 15:44:55 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: <6c130e1c-57df-40da-b9ae-169f8b20238d@www.fastmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-pkg@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Binary package management and package tools discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Sep 2019 14:45:00 -0000 On 27/09/2019 15:26, Dan Langille wrote: > Hello, > > I am working on putting a new poudriere build server into production at work. > > At present, we build packages on one server & deploy them from another server. > > The package build server mounts an NFS share from /usr/local/poudriere/data/packages > > The package distribution server mounts that same NFS share in RO-mode. All hosts install from this server. > > I want to change this. The goal: > > * poudriere builds to local disk > * /usr/local/poudriere/data/packages will be rsync'd later to NFS > * the build & rsync process will not be serial/linked (i.e. a build will not trigger an rsync) > > My concern: will this work? Will the package distribution server get incomplete sets of packages? > > Thank you. > If you enable the ATOMIC_PACKAGE_REPOSITORY setting in poudriere.conf, then you won't see inconsistencies because you're rysnc'ing while poudriere is in the middle of re-building the repository. You might see inconsistencies because someone is trying to install packages while you're in the middle of rsyncing an updated repository. However, the atomic package repository mechanism works by juggling symbolic links at the top of the /usr/local/poudriere/data/packages directory tree -- if you arrange things to rsync any subdirectories first, and then finally synch over the symbolic link bits, you should be golden. Although, just sticking a webserver onto your poudriere build box would likely be easier overall and less effort to maintain. Cheers, Matthew