Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 20 Apr 2017 21:19:11 +0000
From:      Grzegorz Junka <list1@gjunka.com>
To:        freebsd-ports@freebsd.org
Subject:   Re: Is pkg quarterly really needed?
Message-ID:  <399feac5-71d7-25ab-80da-84bd6c2eeeda@gjunka.com>
In-Reply-To: <20170420195712.GK74780@home.opsec.eu>
References:  <58F61A8D.1030309@a1poweruser.com> <CALfReyctL3vTt756oyh1ZTf%2BkgpAOHwp_SUZQCFQiZDccFNMow@mail.gmail.com> <ljhffcphq3bqr8dk2lrlld11ola28b7gqp@4ax.com> <29e44642-e301-f07c-afe3-bad735d8ee5e@freebsd.org> <20170420053722.GD31559@lonesome.com> <b9d24938-5502-cc69-30ed-1941c2517849@gjunka.com> <20170420084452.GH74780@home.opsec.eu> <99a57878-ae39-d2a4-fe35-023dae8b320b@gjunka.com> <20170420171119.GJ74780@home.opsec.eu> <127a5f89-93ba-aee4-14d3-41e2f2d71892@gjunka.com> <20170420195712.GK74780@home.opsec.eu>

next in thread | previous in thread | raw e-mail | index | archive | help
Hi :)

On 20/04/2017 19:57, Kurt Jaeger wrote:
> Hi!
>
>> I understand that the main problem with quarterly branches is that they
>> start from an unstable edge and mature with time, then after three
>> months at the most mature point they are being deleted and replaced with
>> a new unstable edge. So, there is no good point of reference to use as a
>> source of packages.
> First, let me say that for my use cases, the pkg tree is in consistent
> state most of the time. We have a set of approx. 2500 pkg's build etc,
> so roughly 10% of the full repo. We update the ports tree and build the
> repo every week, just to see if all is fine, and we update
> and build, if we need a package we do not yet have in our repo.
> For the thrills, look at our repo at https://repo.nepustil.net

I am maintaining four sets of pkg's for my desktop, laptop and server 
(in two variants) - anything between 100 to 1300 pkg's resulted from 
selecting around 10-200 ports depending on the set. I usually first 
build for my desktop, then if everything is fine I rebuild the set for 
my laptop, and finally the server. I hardly remember a single upgrade to 
the longest set that didn't result in some packages being broken. Sure, 
most of those breakages resulted from me selecting non-standard options 
(but if I wanted standard options then I would simply install from the 
official distribution, wouldn't I?). Only last weekend I filled two bugs 
against ports that didn't compile when either non-default options were 
selected in those packages or in some of their dependent packages.

So, my experience is quite different to yours...

> Hmm, let's try this thought experiment.
>
> For each package, keep the whole repo when it was last build sucessfully
> (Note: we're not yet discussing whether it runs!)
>
> Worst case: We have n ports and n repos. The question is:
> what would be the average case ? Would that be a sustainable model ?
>
> Now, the next question: Even if we have all the repos,
> would we find *one* repo where *two* packagew we'd like to have
> are in a consistent (build-!)state ? What about the 250+
> that are normally needed for a small server ? Or the 1200+
> for a multi-role server (some will say: "nah, don't do it, use
> container/jails/ whatever virtualisiation").

If the whole repository builds doesn't it mean by default that any 
subset also builds? All packages are build incrementally, the packages 
that don't depend on any other packages are build first, then any 
packages that depend on them, and so on. Sure, it doesn't mean that they 
also run properly, but that's a different story. If packages are build 
then people can install them and fill bugs. That would be the reason for 
having CURRENT, STABLE and RELEASE where fixes would be gradually 
progressed between them.

>> What I described is taking the good points (maturing the code through
>> progressing version upgrades from CURRENT, through STABLE to RELEASE)
> Now, if we have ports-HEAD, and changes to that (especially fixes and
> features to the ports infrastructure), it's getting more and more difficult
> to backport fixes from ports-HEAD to the ports-STABLE versions, because
> those do not contain dependencies and infrastructure changes.
> If we backport those, we have to roll forward some other
> changes. This gets out of hand very quickly.

I see where you are getting at. My assumption was that only version 
upgrades are progressed from CURRENT to STABLE to RELEASE. If a port 
can't be progressed because it requires infrastructure change then, 
well, it won't. STABLE and then RELEASE are meant to be more stable so 
we would prefer older versions that work rather than new versions that 
break. Infrastructure changes would have to be progressed eventually but 
that could be done in batches where most fixes in more edge branches 
have been fixed.

> So:
>
> If we take the sum of the brain time our maintainers and committers
> deliver to keep HEAD in a moving (different from a stagnating) state,
> and try to estimate how much *more* time would be needed to
> keep additional trees in working conditions, only updating
> what is needed, under the assumption that infrastructure changes
> *need* to happen ? What would that additional brain time be ?
> My inituition tells me that it would probably break the model.

On the other hand developers would be more inclined to do changes in 
CURRENT if they know that they are not going to break ports for the 
majority of users who should use STABLE or RELEASE. According to the 
principle "Failing by design 
<https://hbr.org/2011/04/failing-by-design>". They would be also more 
confident when porting changes to more stable branches knowing that they 
have been tested by many users and if something could have gone wrong 
most likely already did.


> Now, if someone wants to *experiment* with that, we
> already have quite a few people doing that: By setting up
> our own repo boxes, where we build the trees to our liking.

I am not trying to solve the problem for myself. I already have my own 
repo box. I am just investigating options as many people seem to be 
unhappy by the current state of affair. That's not to say that I am not 
annoyed by the frequent breakages in the ports that I am trying to build 
for my own purpose of course.

> The FreeBSD svn commits are public. So, if someone wants
> to use it, and select and pick to provide a stable repo,
> we would all welcome the effort. If it's a sustained effort over
> some time, clusteradm etc would probably add that repo
> to the infrastructure. We can even call it the 'caveat'-repo.
>
> It's just that asking the team that's barely keeping up
> to do that *on top*, that will probably not work.

That was supposed to be more like *instead* rather than *on top*.

>>  From that short description it should be more or less
>> obvious if that approach is better/doable when opposed to quarterly
>> branches?
> There's a way to find out: Try it.
>

Not the best way TBH. I would rather hear some opinions first as I am 
sure there are plenty of conditions and requirements I haven't even 
imagined myself yet.

Grzegorz




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?399feac5-71d7-25ab-80da-84bd6c2eeeda>