From owner-freebsd-git@freebsd.org Sat Dec 3 17:29:13 2016 Return-Path: Delivered-To: freebsd-git@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E3C8AC65FC6 for ; Sat, 3 Dec 2016 17:29:13 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-qt0-x233.google.com (mail-qt0-x233.google.com [IPv6:2607:f8b0:400d:c0d::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 98C3C91B for ; Sat, 3 Dec 2016 17:29:13 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: by mail-qt0-x233.google.com with SMTP id p16so279302243qta.0 for ; Sat, 03 Dec 2016 09:29:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=jzPnCP8wAVR29P9WbgaAxfeUsM0W6H4WckZlCVU/t9A=; b=vGLmmE7TQmpCMqU7gvBXfzZ/mdLLgUzedRkA9sT/fr4QN7Yvyf7ytgYENiDv3Upk/r vRWGdYSAo+oucd+5iZ4tJF+KONGteuIRoIEmncnb8odyrdQ0SfVrwJFkIL5sJQ0KpIxY oUD2c+PLFUXAkC517du+HX+6AEZGC14VT99eZyHqGrw7HSBHu29Dh6CNKKuimf9W7GEd h9thli3/NpfnsKsS+dSOsdqceMCee77SqnJ+Lam3S7NdCwOXm5Oz1ulEDwyLwzQsTNhC JEtUJEhBwbI22zcneqN4twjhvK9tkm/GAGTP3V+s0lD4/285XTXITsdvUFH3cVnFtJp6 pqQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=jzPnCP8wAVR29P9WbgaAxfeUsM0W6H4WckZlCVU/t9A=; b=FD1/KsmjwpY2fli/d9PIf41OhZUW3mfoWUhXiZ/h4GiTIagMHr7BmNl6JtIAOwIatd XylpjgeJfEuoirYmTCCx9wlORmT9oE7RqUORB7Cp+KFj/tYp7wK4cAa2NBIHrFQPLDCO VEzJbX4a+qFX3N7qsUbMbJPIYALSZT64xs1qNf+RxxET4PstKWfnouWPn5WqLr2kHIG/ Dzju7MJuoqPjRQX4BNSxNzVHgSUFWyG+IOTUkniiMF7nPKdA081Ab2cgA8PFuiui9QfI ah5SA4ygkheXb7LWyjfSzcIXVyMr1osRxYS0b2EKvpkC+945RJgFOoGGiDNS/tRpJrXn QXoQ== X-Gm-Message-State: AKaTC012iPLfeUquBgTGCdV7U7zuIf8SphXN4glKS1rmcyibnJM80sgBfV/6I4r0g7D5L+RT X-Received: by 10.200.49.247 with SMTP id i52mr42800885qte.108.1480786152729; Sat, 03 Dec 2016 09:29:12 -0800 (PST) Received: from mutt-hardenedbsd (pool-100-16-218-231.bltmmd.fios.verizon.net. [100.16.218.231]) by smtp.gmail.com with ESMTPSA id 124sm5348055qki.14.2016.12.03.09.29.11 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 03 Dec 2016 09:29:11 -0800 (PST) Date: Sat, 3 Dec 2016 12:29:10 -0500 From: Shawn Webb To: Ulrich Sp??rlein Cc: freebsd-git@freebsd.org Subject: Re: Git mirroring halted for freebsd-base Message-ID: <20161203172910.GB82677@mutt-hardenedbsd> References: <20161203172151.GA82677@mutt-hardenedbsd> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="RASg3xLB4tUQ4RcS" Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD mutt-hardenedbsd 12.0-CURRENT-HBSD FreeBSD 12.0-CURRENT-HBSD X-PGP-Key: http://pgp.mit.edu/pks/lookup?op=vindex&search=0x6A84658F52456EEE User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Dec 2016 17:29:14 -0000 --RASg3xLB4tUQ4RcS Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Dec 03, 2016 at 06:25:29PM +0100, Ulrich Sp??rlein wrote: > 2016-12-03 18:21 GMT+01:00 Shawn Webb : > > On Sat, Dec 03, 2016 at 12:42:56PM +0100, Ulrich Sp??rlein wrote: > >> 2016-12-02 11:07 GMT+01:00 Ulrich Sp??rlein : > >> > The conversion process started chewing up 100% cpu without making mu= ch progress, first attempts to rectify this have failed. > >> > > >> > The svn2git conversion and pushes to github have been halted. Pushes= to bitbucket also have been halted (we're reaching the 2GB limit imposed b= y bitbucket). > >> > > >> > I'll update this thread in about 24h. > >> > Uli > >> > >> Service is fully restored now for the github mirror, sorry for the dow= ntime. > >> Bitbucket will stop working soon because of the size limitations. > >> Speak up if you require this mirror to be kept up-to-date. > >> > >> Cheers, > >> Uli > > > > Looks like it might be easier for some downstream projects to fully > > recreate their ports repositories from scratch than to try to merge from > > upstream. >=20 > What are you referring to here? The ports repo at https://github.com/freebsd/freebsd-ports was force pushed. Now attempts at merging in upstream's ports tree into hardenedbsd's causes merge conflicts for hundreds of files, including files we didn't change. So I'm forced to either inspect hundreds of files, manually merging in the changes or recreate our ports tree from scratch, re-importing our changes in a single atomic commit. The second option sounds more appealing, though we'd lose the entire history of our changes. Additionally, anyone downstream from HardenedBSD might have to do the same. Domino affect. >=20 > > What caused the issue? What is going to be done to prevent it from > > happening again? >=20 > I have no root cause, other than bitbucket changing permissions and > somehow git ending up using 100% CPU for most of the operations. So no guarantees this massive screw-up won't happen again? --=20 Shawn Webb Cofounder and Security Engineer HardenedBSD GPG Key ID: 0x6A84658F52456EEE GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE --RASg3xLB4tUQ4RcS Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJYQwDlAAoJEGqEZY9SRW7u2IQQAKYkGsd+OtpOpaYDejfIMHmD HdW8l2OoWqwc/AQdCCwhjDNpLAGqFM1zWp32JVyg47xyL7SVXBosAjMn5zz5Iz13 L7Ysr6HSHFtN6R90GKOmUqNm/kMD+k8CIvD/I8/ziGNSKsxCZL7We6HHs3ecqEtB fa+ryfM9/I4T2/d80rYtaaF0LQQx/r4OIEIzVduUKbbdzRmBGihvdOgwhXTgrp+R s1yyLcQI3GKb44SpTFH+RYQQxhHt8XD3FasV1a0bxvZgmAzHwL/I81koID/D0D6D pQ2fKLyyMFIfoTXcC1MBIm9vmjW4Azg2NoiijapX0JXPovrEdE1nZZHPgCeu0wAG xvMJhaUK//tgJLBHsWbOSFwPxeoWuOLndG+VkN15DjAH0bGng2r1GgbOsVm6VoYS 1pXWktZjvaWMrhzTvADX8S9fG8PFPhkA6bcwBN3oW5GsAS4g/EEMW56NW5V5yB8o tvr3YNYTU3VOJVumkpgPqL2ofxzQh6MVMV06ZfJMjZk+36rojrkaGSwCoGpXEjcy qEJIk2N7Q1XsYjKUI9I+KU4i4d1yK4uhpwRkC9lVw3rNMAKRPNCp98Ycn1dgm2AV kb3u3bHFTaRG+I5ROmQPpC+nNJ0XH/VjsjqwuX6NvX9NlQgb6mE/mym9kjVYvEqM oL3GEviBcJfdzCisA1qf =mTN/ -----END PGP SIGNATURE----- --RASg3xLB4tUQ4RcS--