Date: Wed, 2 Dec 2020 17:28:58 +0100 From: =?UTF-8?Q?Ulrich_Sp=C3=B6rlein?= <uqs@freebsd.org> To: Marc Branchaud <marcnarc@gmail.com> Cc: Warner Losh <imp@bsdimp.com>, git@freebsd.org, "portmgr@FreeBSD.org" <portmgr@freebsd.org> Subject: Re: Using git branches for ports (was: Re: converting rmport to git) Message-ID: <CAJ9axoTqFJ-gQee0Um5-LDBHc0cXpK=YQOJ0y2kr%2BPcODydgAg@mail.gmail.com> In-Reply-To: <af365a33-1d53-ba0d-29fd-571dc72c7594@gmail.com> References: <20201129164707.GA31739@freefall.freebsd.org> <CAP7rwchHsz3x4v%2BWmxEqTyXOvJx6fUbRuKnirBPEPVzz7gMw0A@mail.gmail.com> <14871125-A032-4980-8DB1-0210E34D5A11@FreeBSD.org> <20201130105337.GA42359@freefall.freebsd.org> <7246FB00-655B-4BD4-BC99-B87E4595969C@FreeBSD.org> <20201201095906.GA50345@freefall.freebsd.org> <CANCZdfrMP--yZBedSoCBhAFuDzhxLt3BbNi=DCoErrXTtY=qtg@mail.gmail.com> <af365a33-1d53-ba0d-29fd-571dc72c7594@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Sorry didn't read all of this :). I think the proposal has merit for a
ports maintainer maintaining individual ports. But I don't see how it can
work:
- Where's Mk/ then? What about MOVED and UPDATING?
- How does one check out the full ports tree so that build dependent ports
works?
- How can you do a shlib bump with all the required PORTREVISION bumps? It
would need elaborate scripting support to produce the 2 dozen commits
required for that.
- Say you worked on such a bump across dozens of ports, you'd need more
scripts to merge or rebase changes that have happened in the mean time.
I'm sure there are more issues. It would require us to properly slice out
individual ports vs. things like Mk/ and my guess the people working on the
ports infra as such prefer to have a full tree visible (and grep'able)
Interesting idea though ...
Uli
On Wed, Dec 2, 2020 at 5:08 PM Marc Branchaud <marcnarc@gmail.com> wrote:
> On 2020-12-01 11:36 a.m., Warner Losh wrote:
> >
> > To be honest, though, I think this is an area where some experimentation
> to
> > understand the alternatives is needed because this use case is relatively
> > rare in the larger open source community.
>
> OK, so I just have to ask (and I apologize if I'm opening a can of worms
> that has already been discussed, or that nobody wants to look at; I'll
> drop this if it's just noise):
>
> Have you considered using a branch for each port? Yes, I'm talking
> about 41,000+ branches. Git should not have any trouble dealing with that.
>
> There are a few advantages to this approach:
>
> * Each port's change history is fully isolated and easy to track.
> (Don't worry about having lots of near-duplicate files in different
> branches or directories, as git is very efficient at dealing with this.)
>
> * MFCs are proper git merges, which means that it is very easy to
> understand which changes have landed where.
>
> * Cases like removing/re-adding a port would take place on that port's
> branch, making it obvious just how that work was done.
>
> If this sounds appealing, then the real question is whether or not this
> approach trips up any important cases that arise when working on ports.
>
> I can't answer that, but in the grand tradition of git branch ASCII-art,
> here is a pretty picture to help understand what this approach might
> look like. In the following:
> - "a" thru "f" represent commits of some work on a port (net/gsk, for
> example).
> - "M#" represent git merge commits of some net/gsk changes.
> - "m" represents a git merge commit of some other port's changes.
> My proposed branch names are on the left. Commit history proceeds from
> left to right.
>
> main ....--m---m--M1--m--M2--m--M3--m
> / / /
> net/gsk ....-a--b---c--d--e---f
> \
> 2020Q4 ....---m---M4---m---m
>
> So the net/gsk port evolves on its own "net/gsk" branch, with commits
> a..f. We see that the a and b changes were merged into the "main"
> branch by merge-commit M1. Merge commit M2 brought in changes c and d,
> and then merge M3 brought changes e and f into "main". Meanwhile, only
> changes a and b have been merged into the "2020Q4" branch (commit M4).
>
> Both the "main" and "2020Q4" branches also contain merges from other
> ports' branches (the "m" commits). The mainline branches ("main",
> "2020Q4", etc) would consist almost entirely of merge commits.
>
> The net/gsk changes in the mainline branches can be easily obtained from
> simple git commands. To see the net/gsk work that has happened in a
> mainline branch like 2020Q4, just do
> git log 2020Q4 -- net/gsk
> That will list commits a, b and M4. No need to do any patch-level
> analysis.
>
> That command will also work with the existing git repo migrated form
> svn. But the branch-based model has some additional power. For
> example, a command like
> git log --oneline --graph 2020Q4 -- net/gsk
> will output an ASCII-art picture of the 2020Q4 branch's view of the
> net/gsk port, similar to what I drew above.
>
> More importantly, it's easy to see where any particular piece of the
> net/gsk work has landed:
> git branch -a --contains <b>
> would report the "main" and "2020Q4" branches (here the <b> is the
> SHA-ID of commit b). No need to deal with "combined" MFCs or
> did-this-change-match-that-patch problems.
>
> What about the rmport script? The branches I'm describing contain the
> full ports tree -- they're not "partial" or "sparse" in any way. So to
> remove the net/gsk port, rmport would just checkout the "net/gsk" branch
> and do the removal there. Then that can be merged (manually or
> automatically) into whatever mainline branch is desired. There's no
> need to remove the "net/gsk" branch though, and it's better to keep it
> around in case someone wants to revive the net/gsk port in the future.
>
> This branch-based model can be adopted atop the transitioned ports repo
> as it stands today. There's no need (nor is it possible) to
> retroactively translate the svn history into this structure. Sure, the
> migrated svn history isn't amenable to tricks like "git branch
> --contains", but that will become less important as time marches on.
> And the migrated history can still be teased out using patch-level
> commands like "git cherry".
>
> Those are my main points, so you can stop reading here if you're already
> annoyed! I'm now going to delve into some of the flexibility that this
> approach offers.
>
>
> In this model the net/gsk port is free to evolve as it needs to in the
> "net/gsk" branch. From the above we see that changes a and b were
> deemed good enough to put into 2020Q4, but changes c-f are still a bit
> experimental and they're still being validated on the "main" branch.
> (I'm making some assumptions here about how people develop the ports.
> Apologies if I got it wrong; I'm sure this model can accommodate a
> different workflow.)
>
> In fact, that "net/gsk" branch can itself contain sub-branches for
> special circumstances. Let's say that commit b has a bug. We'd like to
> fix that bug in both "main" and "2020Q4", but if we just plop the fix
> onto the tip of the "net/gsk" branch (as commit g, say) that change will
> have commits c-f has part of its history:
>
> main ....--m---m--M1--m--M2--m--M3--m
> / / /
> net/gsk ....-a--b---c--d--e---f---g
> \
> 2020Q4 ....---m---M4---m---m
>
> If we just merged g into 2020Q4, we'd also bring in the c-f changes
> which we do not want to have on the 2020Q4 branch.
>
> So instead, we can fix the bug in a mini branch based on the b commit,
> then merge that work where it's needed:
>
> main ....--m---m--M1--m--M2--m--M3--m--M6
> / / / /
> net/gsk ....-a--b---c--d--e---f------g'
> |\ /
> | \---------b'-----/
> \ \
> 2020Q4 ....---m---M4---m---m--M5
>
> Here we've fixed the bug with commit b', which is based directly on
> commit b and so we can merge b' into the "2020Q4" branch (commit M5),
> with the confidence that we're only bringing in the exact bug fix we
> need. Meanwhile, we also merge b' onto the tip of the "net/gsk" branch
> (commit g'), fixing the bug on the port's own branch, and then merge g'
> into the "main" branch as commit M6. It is completely clear what
> happened to the net/gsk port, and how those changes were brought into
> the mainline branches.
>
> One last wrinkle about this picture: Note how I did not put a name on
> the branch with the b' commit. Git is perfectly happy to deal with this
> kind of anonymous branching, and so there's no need to pollute the
> central FreeBSD-ports repository with names for these kinds of branches.
> But that does not prevent the net/gsk developer from having a *local*
> name for that branch in their own, local clone of the repository. The
> developer can name their local branch whatever makes sense to them.
> When they push one of the merge commits (M5, g' or M6) to the central
> repo, the b' commit rides along but without the developer's local branch
> name. The history recorded in the central repository is as depicted,
> with b' living on a nameless branch.
>
> I can't believe you've read all of this! Thanks!
>
> M.
>
> _______________________________________________
> freebsd-git@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-git
> To unsubscribe, send any mail to "freebsd-git-unsubscribe@freebsd.org"
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ9axoTqFJ-gQee0Um5-LDBHc0cXpK=YQOJ0y2kr%2BPcODydgAg>