From owner-freebsd-git@freebsd.org Thu Sep 26 14:26:59 2019 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3418E1284E3 for ; Thu, 26 Sep 2019 14:26:59 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-io1-f66.google.com (mail-io1-f66.google.com [209.85.166.66]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46fHLV1lgpz4JXK for ; Thu, 26 Sep 2019 14:26:57 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-io1-f66.google.com with SMTP id q1so7125951ion.1 for ; Thu, 26 Sep 2019 07:26:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=kfQVTR8HyHKAjB/xtjRXJH9iv9cT7fGk01688so4NOg=; b=lmYu5Bf9jaJPijaO1NWggIO2XY7BlW5F4hFjne8FI5H1mhMe8Ej/y69N/vffC+5a35 np8i0YJ4OS+5fPDO9tLs+HTBjOODjtyR93AfZNBELxEtwzbYcF7DfpRkIlD3KnUabmAa 4lX9HpE0O+2Bg5AS1VrNqLYNNbQTBb+6CzqbzAnCa0xmo/sES6WWeCFnm1oPRbl5wHW4 hBZGo/6CMJ0IGTby3Rnff9yNf5e5xyJVXzZfZnNkFX73Bp59FziDKWHflZNzozDxVvEb elv2LCj0hhDj3lam/KDWELHV8c7o6/4xwyuAQ79OptPjIKcFEe4f8uMI0LPqnnH+WFmN 7Img== X-Gm-Message-State: APjAAAWC0yGLDPgb3bGmG/MlNHtVTnOWL0nlijkARYze9iITEIVylbG8 zSDw2h8JmPEDxZnYaXbvqSArXlQXNy71HsmRxPA= X-Google-Smtp-Source: APXvYqwjfZreMMukPR2s7yUHymlc3HYfKb6BMQtBdb3PpBCTYIOD9d+a/NBH/K2Mjr6vKPZL25MG1EGsmGTtkzb9vMM= X-Received: by 2002:a02:6616:: with SMTP id k22mr3613546jac.129.1569508016219; Thu, 26 Sep 2019 07:26:56 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Ed Maste Date: Thu, 26 Sep 2019 10:26:43 -0400 Message-ID: Subject: Re: Service disruption: git converter currently down To: Warner Losh Cc: Sean Chittenden , freebsd-git@freebsd.org, =?UTF-8?Q?Ulrich_Sp=C3=B6rlein?= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 46fHLV1lgpz4JXK X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.66 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com X-Spamd-Result: default: False [-3.20 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[66.166.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-1.20)[ip: (-0.46), ipnet: 209.85.128.0/17(-3.29), asn: 15169(-2.18), country: US(-0.05)]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[66.166.85.209.rep.mailspike.net : 127.0.0.17]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Sep 2019 14:26:59 -0000 On Wed, 25 Sep 2019 at 15:50, Warner Losh wrote: > > git log always requires added care. There's not actually 9000 commits the= re. The tree looks fine topologically. Its purely an artifact of git log. This seems to be getting into a philosophical discussion of what it means for a commit to exist. But, given the constraints in the way git represents commits the history crafted by the svn-git exporter indeed shows thousands of "phantom" commits. The converter should (and with uqs tweaks, would) represent the offending commit here as if it were a cherry pick, not a merge. In order to really represent this correctly we need to add to git metadata tracking file operations. Recording that path d1/f1 was copied from d2/f2 at some hash would allow us to properly represent this case as well as renames/moves. >> git log --first-parent isn't really a solution here either, because >> there are cases where one legitimately does want history from both >> parents, especially working in downstream projects. > > I'm pretty sure it would be fine, even in that case. It's not fine, because it omits the commits I want to see. >> > I'd offer the opinion that needing to know about things like git log -= -first-parent vs having to rebase every single downstream fork, >> >> We won't need to rebase every fork - in no case should the path >> forward be worse than uqs's suggestion of a merge from both old/new >> conversions. > > IMHO, uqs suggestion is a complete non-starter, at least the "git diff | = git patch" one. It destroys all local history, commit messages, etc. Except= for the most trivial cases, it's not really going to fly with our users. H= is other, followup ones might be workable into scripts. diff | patch is not the suggestion; the suggestion is to perform a merge from the "new" conversion. Other options (e.g. some sort of scripted commit replay) are at least no worse than that base case. > I'm not sure you can merge, as there's no common ancestor that's recent e= nough to give it a chance at succeeding (since the different exports would = have different hashes starting fairly early in our history). My experience = with qemu is that long-lived merge-updated branches become quite difficult = to cope with after a while. It took me three weeks to sort out that relativ= ely simple repo. In fact, the merge works fine, even with completely unrelated histories. You can try this by merging 'svn_head' (from git svn) to 'master' (from svn2git), using `git merge --allow-unrelated-histories origin/svn_head`. The resulting history has two copies of every commit, but the file contents are unchanged over the merge. If you try this in a tree with changes (i.e., try applying it to a long-running merge-based branch) every modified file will result in a conflict, but they can be trivially resolved in favour of the first version. From that point on merging from the "new" conversion will work as expected. > A rebase has a chance of working for people following a 'rebase' work flo= w. Indeed, for rebase workflow it's fairly straightforward. > However, for people like CHERIBSD who follow a 'merge from upstream' mode= l which never rebases (since that would be anti-social to their down stream= s), I'm having trouble understanding how that could work. At work, we basic= ally do the merge from upstream with collapse model, which I'm having troub= le seeing how to move from old hashes to new. I'd like to know what the pla= n for that would be and would happily test any solution there with a copy o= f our repo. I'd even be happy to run experiments in advance of there being = something more public available to see what options do or don't work. Could you expand on the "merge from upstream with collapse" - specifically, can you provide an example command used when merging from FreeBSD?