From owner-freebsd-ports@FreeBSD.ORG Fri Sep 9 07:25:32 2011 Return-Path: Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B225C1065674 for ; Fri, 9 Sep 2011 07:25:32 +0000 (UTC) (envelope-from erwin@mail.droso.net) Received: from mail.droso.net (grizzly.droso.net [IPv6:2a01:4f8:100:9424::3]) by mx1.freebsd.org (Postfix) with ESMTP id 534C88FC0A for ; Fri, 9 Sep 2011 07:25:32 +0000 (UTC) Received: by mail.droso.net (Postfix, from userid 1001) id 131F9651DF; Fri, 9 Sep 2011 09:07:16 +0200 (CEST) Date: Fri, 9 Sep 2011 09:07:16 +0200 From: Erwin Lansing To: freebsd-ports@freebsd.org Message-ID: <20110909070715.GJ71220@droso.net> References: <20110908045328.C6E2E1EE8F1@keeper.homelinux.org> <201109081326.11474.erichfreebsdlist@ovitrap.com> <4E69539A.7080703@delphij.net> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <4E69539A.7080703@delphij.net> X-Operating-System: FreeBSD/amd64 8.2-RELEASE User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: The cost of a source based package system X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Sep 2011 07:25:32 -0000 On Thu, Sep 08, 2011 at 04:45:30PM -0700, Xin LI wrote: > > Both portmaster and portupgrade have 'package' mode, which uses > packages when available. If one can live with default optimization > (which is usually good anyways) and if most times the default options > would satisfy his/her need, or if the port doesn't provide any > options, binary packages would save a lot of time. > > The real problem for FreeBSD's packaging system is, in my opinion, we > do not maintain branches and ports tree is a fast moving target, > making it impractical to build packages and push to mirrors. Absolotely right. There is also so ongoing work to move away from the current model. We already are using a consistent model for from which src branch we build packages on. Previously, this was RELENG_X on a given day, where that day was defined by "whenever the portmgr running that architecture has time to do so". This is now oldest supported minor release within each major branch. More on: http://www.freebsd.org/portmgr/policies_eol.html We also need to do this for the ports tree. The best way to do so is to provide consistent package sets which are not overwritten on the mirrors when a new package set becomes available, but let the user move between sets manually. This requires some large changes, both to the code and infrastructure which we are working on. The code needs to be able to identify which package set is installed and an ability to move to a new set. The PKGNG project will provide us that. Also, the current mirror infrastructure cannot support the extra amount of data needed. Currently, a full set of packages for one architecture is about 30G. Just supporting the Tier-1 architectures on 2-3 live branches add up, especially if we also need to keep multipe full sets around. There's a short presentation from BSDCan with some bullet points: http://people.freebsd.org/~erwin/presentations/20110512-BSDCan-packages-summary.pdf This is also a topic we'll talk more about at EuroBSDCon next month. > > My $0.02: It might be worthy to experiment a branched development > model and only pull up changes at a much lower pace to branch (e.g. > create a branch near a release and drop the branch after a few weeks > once a new one is created, and only pullup changes when there is need, > like because security vulnerability or serious reliability/performance > issue), it would be easier to produce binary package and sync them > across mirrors. > I don't think brances is the right solution here. Talking to the pkgsrc people, they use quite a lot of time on pullups and we have almost 3 times as many packages. This would not scale to ports. Unfortunately, as I do like the model in principle. I think we can come near though wil clearly defined EOL/EOS lifetimes for the package sets (see slides). Erwin -- Erwin Lansing http://droso.org Prediction is very difficult especially about the future erwin@FreeBSD.org