From owner-freebsd-git@freebsd.org Sun Dec 4 11:04:37 2016 Return-Path: Delivered-To: freebsd-git@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A61A0C66E4A; Sun, 4 Dec 2016 11:04:37 +0000 (UTC) (envelope-from uspoerlein@gmail.com) Received: from mail-wm0-x242.google.com (mail-wm0-x242.google.com [IPv6:2a00:1450:400c:c09::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 505A43CB; Sun, 4 Dec 2016 11:04:37 +0000 (UTC) (envelope-from uspoerlein@gmail.com) Received: by mail-wm0-x242.google.com with SMTP id u144so9479508wmu.0; Sun, 04 Dec 2016 03:04:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=feKiq3OQ91QWqEvcjU6j3gqHoTHzZIl/KBw08b/7IbA=; b=TPw13BbbT2mivw223fTuAO5jiedfNnlQiVhAfwBAYOMGHueS+C8Nm9oCPqngxt/7q7 933AyXypUhyCdQwRqjhnshlWyDgtxYCRM9wLn7yqaIbbWsRbNqMQ5WCpseJ77MWo3G8g ymEeHjIqqup07hqAM6l1wbBxxMIydJCbmLLAeQVeDshkm+UP1vCbWKrDuYiYoPJZYBZ7 5mUvN6HRAYMzXR2S32c+MUf0HsCzQeMr5INmUIgfCWzcH5+T5g8FNzMdJdtfOFbHigwO VLd/wKl+zeBFXOGvn4X9ZgUMq1omwjHCn1ZMJaNxR1qgKEKeGzWZ8OOzP6aqc/lLQZYe LEZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=feKiq3OQ91QWqEvcjU6j3gqHoTHzZIl/KBw08b/7IbA=; b=jpOFtMymDxhpF961+KeaydNEn5wy2cHZzowwDw3S1Mc6yHwApGohmZklY0rJ6sSTLi gVjh94ziD4Jrbc3JD09OoCXDhylcy9Q6n1O7//nV3fOBp1QCIjeJWaO4IUcQBXE6A9yR HQxibJEjZ0UhG/0wOoA1m1qgOgWL5Tz/SaHRIuPkXMgWUpIt9E1DONAWUAdKiBwEwXvZ CXD6a+zOh9rYBhRqn4hXrH+jGS2nly6SMlTA1GL2gLT6OmKT4lVUTvrUIkzyv5VDsEPc JUQp/PEuDBVYlwQrku8UiYWNsGPk55Igb6Ec+zKmudFlRsyehURa5APup1FJD4gwjIvu qzxg== X-Gm-Message-State: AKaTC01V82i6+S+bC55nyDYeRPspTjx9kWDwe1EEeehdy6OvNzZLYlgXvIYLx73/zjKDkYgsgveBspzzoq+Y4g== X-Received: by 10.25.21.27 with SMTP id l27mr20499674lfi.163.1480849475240; Sun, 04 Dec 2016 03:04:35 -0800 (PST) MIME-Version: 1.0 Sender: uspoerlein@gmail.com Received: by 10.25.196.140 with HTTP; Sun, 4 Dec 2016 03:04:34 -0800 (PST) In-Reply-To: <86zikeekj6.fsf@FreeBSD.org> References: <86zikeekj6.fsf@FreeBSD.org> From: =?UTF-8?Q?Ulrich_Sp=C3=B6rlein?= Date: Sun, 4 Dec 2016 12:04:34 +0100 X-Google-Sender-Auth: GfrYqIygrJhejCMsZECPCiQX6bM Message-ID: Subject: Re: Hash changes in the freebsd-ports GitHub mirror To: Raphael Kubo da Costa Cc: freebsd-ports@freebsd.org, freebsd-git@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Dec 2016 11:04:37 -0000 Sorry for seeing this message so late, here's what happened: 1. There was metadata corruption in the SVN repo that is used as the source for the svn2git conversion, this repo is kept up-to-date using svnsync, and it turns out svnsync does not operate atomically. 2. Someone "fixed" the corrupt SVN repos, but that means we can no longer reproduce what is published on github <- this happened last year and we've had this Damocles sword dangling above the repos for that long 3. bitbucket changed their permissions model and somehow this caused git to chew up 100% cpu when doing anything inside the "src" repo (wtf?) 4. I stopped everything, fixed bitbucket ssh keys, started deleting bitbucket branches as we're approaching the 2G limit 5. seeing that a proper repack was in order, I did a git repack on base/ports/doc 6. ??? 7. svn2git started to re-convert freebsd-ports from rev 1 (wtf wtf?) Because (7) used the fixed repo, this should now actually be the proper 1:1 conversion from SVN ... unless there's more metadata corruption that we did not fix. I am currently checking this with another run on a different machine, but that machine is slower and not even half done yet. The interesting thing to note is that: a) obviously no one is doing the conversion in-house and found out that they get different hashes, although this is documented on https://wiki.freebsd.org/GitWorkflow b) we still have the same problem for src and doc. We can *not* reproduce the version that is published on github (different timestamps/authors on the commit metadata) Expect more turbulences Uli 2016-12-02 10:40 GMT+01:00 Raphael Kubo da Costa : > Hi all, > > I tried running `git pull` a few minutes ago and had a ton of conflicts. > > It turns out all hashes after c96fb0418e545a569b5975b4d878a30a948c29d5 > ("fix issues related with USES=fonts" from 2015-07-18, aka r392404) are > now different in all GitHub branches. > > Is anyone else experiencing this? Was this change intentional? >