From owner-freebsd-git@freebsd.org Wed Jan 4 22:25:15 2017 Return-Path: Delivered-To: freebsd-git@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D7FF2C9F2F3 for ; Wed, 4 Jan 2017 22:25:15 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-it0-x22c.google.com (mail-it0-x22c.google.com [IPv6:2607:f8b0:4001:c0b::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AE6091923 for ; Wed, 4 Jan 2017 22:25:12 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-it0-x22c.google.com with SMTP id c20so315401294itb.0 for ; Wed, 04 Jan 2017 14:25:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:from:date:message-id:subject:to; bh=MKMY3eKendcyCS7PtcOf3NFx+sFUSgw6T3x7E72LFYk=; b=ZkiO4WL8o7BYEmvpIh/hEv+e05yjj/aAS+uKMTmEKv+EFkP/0tSQQ3WPQDsGydNnuC vXURV1KRzxUW7v9E8yt0SJBYsfBnEJyF6zo6GdPkeik2+v+WvhQKBBPGNiIFj0bsCipO iAsIeEKXL76qo27cDnobITcIih9jtY/zg7Ia7vopD2L+3XLoPax2RsiVe3dMtQsMolvs 731tNeAjexS2hcmvfFzvdektUmxJ6j0G2kgVr8izv1qvktK/m/3TtyCz94jOm3CL+V3W y7hmT8/tvV9o2bufQ01UnB/ji5oZyeUQRE1Bj1XovTjdXgpk8+RZuTUdHgM34vqQBYuc bAGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:from:date:message-id:subject :to; bh=MKMY3eKendcyCS7PtcOf3NFx+sFUSgw6T3x7E72LFYk=; b=eAbvW8ohyWE3gvOkHl7zf2RGQkgIMVKW9fNzIO6H4dbTbbOkDXtu9qnk9do7LUCql3 p5LWQjxRuffdzoWKt4+48jbwUrgztxcgtZKlDpQDZ4ZqG8VT3v1m7tPkx0MH3BLA9jLN bqMxosJvbHpIezv3CL+lY/zmlQJt350SOeoTius4FmfMJSGQ2eHBLsr8zWvgN/8995M8 W5Rr3W++B4/l7GkhjXa6P+WPAC7E77VMZuNigekAdnr7+v42quRvdMpYX3KQ9fLYh2Es DmuPLvtqmJ6haM0h1p0YbdHOoJgq/iEBdvfvwdxB598bzuZuQB+QR+VuhTrXE046bk2+ iizQ== X-Gm-Message-State: AIkVDXLaN3KYku0JbKClC0+JezPwmMQ/KtAShsO7ISkvz0CMKGd9/Typgk/Z0g1mHUQ3LtYhwh9kr4G46V/s6w== X-Received: by 10.36.212.66 with SMTP id x63mr49990790itg.14.1483568711868; Wed, 04 Jan 2017 14:25:11 -0800 (PST) MIME-Version: 1.0 Sender: carpeddiem@gmail.com Received: by 10.107.175.133 with HTTP; Wed, 4 Jan 2017 14:24:51 -0800 (PST) From: Ed Maste Date: Wed, 4 Jan 2017 17:24:51 -0500 X-Google-Sender-Auth: jwxbltyNkJC_aeKFFblPFZU_H6o Message-ID: Subject: FreeBSD svn2git aborting: potential workaround To: freebsd-git@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jan 2017 22:25:15 -0000 I'm trying to reproduce the svn2git conversion locally in order to investigate nonreproducibility issues. I'm in the process of cloning a local copy of the svn repository with svnsync, and am running uqs' version of svn2git (obtained from http://svn.freebsd.org/base/user/uqs/git_conv) against that. Initially svn2git was aborting on me: Exporting revision 70017 ....... 7 modifications from SVN /head/ to freebsd-base.git/master done Exporting revision 70018 . 1 modifications from SVN /head/ to freebsd-base.git/master done Exporting revision 70019 ...... 6 modifications from SVN /stable/4/ to freebsd-base.git/stable/4 done Exporting revision 70020 . 1 modifications from SVN /head/ to freebsd-base.git/master done Exporting revision 70021 . 1 modifications from SVN /stable/4/ to freebsd-base.git/stable/4 done Exporting revision 70022 .............Failed to write to process: Error writing to process Abort trap (core dumped) When restarting from an empty git repo it aborted after the same revision. When this happens more detailed information is emitted in the log-freebsd-base.git logfile: progress SVN r70019 branch stable/4 = :70720 progress SVN r70020 branch master = :70721 error: failed to apply delta fatal: Can't load tree 7d74bacb97098f62f3df6c00dea2ca477526c5b6 fast-import: dumping crash report to fast_import_crash_27754 And the referenced fast_import_crash_27754 has a lot more information in it, although it was not helpful to me in trying to take the problem to root cause. I did find that adding "--commit-interval 10" to the svn2git command-line avoided the abort, and also cleared up a diverging git hash between the official mirror and my local clone. In the official mirror r17572 is f7b4db0557177ace5810a3905ec4eaea478862b2; in my clone it was 9aec003234685887537aeda5d6408cb54d2a066f, but trying to show that hash resulted in: % git show 9aec003234685887537aeda5d6408cb54d2a066f error: failed to apply delta error: failed to read object b7e21351967a525e1e4146ab43a981d0ddf34d71 at offset 117543675 from ./objects/pack/pack-c4fb2c8279432e4e8701c0b35df7cce79a96fa1d.pack fatal: packed object b7e21351967a525e1e4146ab43a981d0ddf34d71 (stored in ./objects/pack/pack-c4fb2c8279432e4e8701c0b35df7cce79a96fa1d.pack) is corrupt It seems to me that git is at least partially at fault here. With the default settings (i.e., no --commit-interval) either svn2git is generating valid fast-import data and git is responsible for the failure, or svn2git is generating invalid data and git is doing insufficient validation. "--commit-interval 10" may be overly conservative (I think the default is 1000), but it seems to be an effective workaround until we can take this to root cause.