Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 8 Nov 2015 21:05:30 +0100
From:      Oliver Pinter <oliver.pinter@hardenedbsd.org>
To:        =?UTF-8?Q?Ulrich_Sp=C3=B6rlein?= <uqs@freebsd.org>
Cc:        Alfred Perlstein <alfred@freebsd.org>, freebsd-git@freebsd.org,  FreeBSD-Current <freebsd-current@freebsd.org>, git-admin@freebsd.org
Subject:   Re: FYI: SVN to GIT converter currently broken, github is falling behind
Message-ID:  <CAPQ4ffu3__uttr14dMQd0Hs34YtYh8MDX0ot8vFS8n6FQgqchw@mail.gmail.com>
In-Reply-To: <CAJ9axoRcCUoLzyGN-JkJEn%2ByinWdVoSKUcBr7eas5638t6jBUg@mail.gmail.com>
References:  <CAJ9axoTuuBt4%2Bg4o1%2BLy9VmNfAa3pcMhcPr2ws8T1kCm=Om=tg@mail.gmail.com> <CAJ9axoRBcFD=-d=pzJJvYempEO-EyR_kAiK3EZQ_hp%2B7_J1iyQ@mail.gmail.com> <563EAAB8.5020702@freebsd.org> <CAJ9axoQmgT0B23UtmzGeMcvS%2BCHxC16FL53fPGObBZoxEC03aQ@mail.gmail.com> <CAJ9axoRcCUoLzyGN-JkJEn%2ByinWdVoSKUcBr7eas5638t6jBUg@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Nov 8, 2015 at 12:06 PM, Ulrich Sp=C3=B6rlein <uqs@freebsd.org> wro=
te:
> 2015-11-08 11:32 GMT+01:00 Ulrich Sp=C3=B6rlein <uqs@freebsd.org>:
>> 2015-11-08 2:51 GMT+01:00 Alfred Perlstein <alfred@freebsd.org>:
>>>>
>>> Uli,
>>>
>>> One of the biggest concerns I've heard from folks using FreeBSD's git m=
irror
>>> is that the hashes can change.
>>>
>>> I have a question about this.   Is it possible to keep track of what th=
e
>>> "official" git mirror (on github) is doing and keep that as a log.  The=
n
>>> that log can be used to replay commits when there is a divergence probl=
em.
>>>
>>> What I'm basically saying is that let's take this small example:
>>>
>>> importer is working fine @rev 10000
>>> imports 10000
>>> imports 10001
>>> imports 10002
>>> something happens to importer to give indeterminate shas.
>>> imports 10003 - sha is "unstable" sha3
>>> imports 10004 - sha is "unstable" sha4
>>> imports 10005 - sha is "unstable" sha5
>>> imports 10006 - sha is "unstable" sha6
>>> importer is fixed
>>>
>>>
>>> At this point normally we'd rewind the importer to 10002 and then force
>>> update the affected branches.
>>>
>>> My question is... can the imports of 10003, 10004, 10005 and 10006 be p=
ut
>>> into the importer such that any "mirror site" that re-does the import u=
sing
>>> the most up to date importer will get the same shas.
>>>
>>> That would allow to proceed with 10007, etc without force pushing.
>>>
>>> This should be possible based on querying "git" for the meta data assoc=
iated
>>> with sha3..sha6 and then forcing those commits to have the same meta da=
ta.
>>>
>>> This would eliminate the concern about shas in the mirror changing that=
 I've
>>> heard.
>>
>> The goal of the conversion is that everyone can re-do the conversion
>> in their basement and come up with the same history and checksums.
>> This was not the case when I first started, as there was some
>> non-deterministic hash structure being used in svn2git. This was fixed
>> in the code and then all converter runs produced the very same
>> results.
>>
>> The scenario that we have right now, is that one of the merge commits
>> done about two weeks ago is being handled different by svn2git w/ svn
>> v1.8 vs. svn v1.9 and I haven't investigated yet how the API's
>> behavior changed to cause this. I'm afraid I also swapped out all my
>> knowledge about svn2git internals and will have to redo this all from
>> scratch :/
>>
>> Your suggestion could only work, if we hard-code this svn revision
>> special handling into svn2git, either in the code or by providing more
>> mappings and rules to the process. svn2git should run hermetic and not
>> poke at github's commits to see how things were handled in the past.
>> It has to be self-sufficient and must not depend on github.
>>
>> This would also only work, if the "breakage" window was very small,
>> but it is already about two weeks long and will surely increase till I
>> find the proper fix.
>>
>> So, to take a stand here: this sort of kludge is unlikely to ever
>> happen. Git commit hashes *might* change in the future. I really don't
>> see how this is a big deal anyway.  It happened once and I'm trying to
>> have it never happen again. But why are people afraid of this
>> happening? Every "official" git commit is tagged with a SVN revision
>> and the contents of those revisions are obviously correct (just not
>> the ancestry and the commit objects, possibly). So it would be easy to
>> write a script that replays VendorA's git history and swaps out the
>> new official commits for the old official commits. There would be no
>> merge conflicts.
>>
>> I can see how this would be annoying if you have 100 developers and
>> dozens of branches that are far from mainline FreeBSD. But I'm sure
>> these companies that depend on git will come forward and donate some
>> of their developer manpower to help me with keeping the converter
>> stable/deterministic. Right? Right? :) :)
>>
>> Cheers,
>> Uli
>


Hi Uli!

I can not find your original svn2git repo in gitorius
(https://gitorious.org/ is down) , could you please the source code
somewhere to git-repo? For example github.com/freebsd/svn2git?

> Quick update: doc is so far unaffected by svn 1.9, but for ports, the
> drift happened as of Jul 18, so you'd need to special case a lot of
> commits.
>
> Here's the same commit, and the difference between 1.8 and 1.9:
>
> % git cat-file commit 803795d
> tree 7fc83aba022834da5c218114b09ad4640735bcc0
> parent c96fb0418e545a569b5975b4d878a30a948c29d5
> author olgeni <olgeni@FreeBSD.org> 1437203525 +0000
> committer olgeni <olgeni@FreeBSD.org> 1437203525 +0000
>
> Upgrade to version 0.4.1.
> % git cat-file commit 61ca43b
> tree 7fc83aba022834da5c218114b09ad4640735bcc0
> parent c96fb0418e545a569b5975b4d878a30a948c29d5
> author olgeni <olgeni@FreeBSD.org> 1437203529 +0000
> committer olgeni <olgeni@FreeBSD.org> 1437203529 +0000
>
> Upgrade to version 0.4.1.
>
>
> In case you don't see it, there's a 4s difference in the timestamps
> for authoring and committing. Here's the original:
>
> % svn log -vc392405 svn://svn.freebsd.org/ports
> ------------------------------------------------------------------------
> r392405 | olgeni | 2015-07-18 09:12:05 +0200 (Sat, 18 Jul 2015) | 2 lines
> Changed paths:
>    M /head/www/elixir-maru/Makefile
>    M /head/www/elixir-maru/distinfo
>
> Upgrade to version 0.4.1.
>
> ------------------------------------------------------------------------
>
> So yeah, svn 1.9 returned a timestamp that was off by 4s. WTF?
>
> For base it's actually even more complicated than I had thought so
> far. But let's take this one step at time ...
> _______________________________________________
> freebsd-git@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-git
> To unsubscribe, send any mail to "freebsd-git-unsubscribe@freebsd.org"



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAPQ4ffu3__uttr14dMQd0Hs34YtYh8MDX0ot8vFS8n6FQgqchw>