Date: Mon, 23 Sep 2019 17:16:17 -0400 From: Shawn Webb <shawn.webb@hardenedbsd.org> To: Ulrich Sp??rlein <uspoerlein@gmail.com> Cc: Sean Chittenden <sean@chittenden.org>, freebsd-git@freebsd.org Subject: Re: Service disruption: git converter currently down Message-ID: <20190923211617.ifajkj2pwss346ub@mutt-hbsd> In-Reply-To: <CAJ9axoQjJpOO7G4e5HK0KPO3xJNh8RbQSLZ3C8NFPKtygub_bw@mail.gmail.com> References: <CAJ9axoR41gM5BGzT-nPJqqjym1cPYv31dDUwXwi4wsApfDJW%2Bw@mail.gmail.com> <CAJ9axoToynYpF=ZdWdtn_CkkA2nVkgtckQSu%2BcMis1NOXgUdnA@mail.gmail.com> <CAJ9axoR2VXFo9_hx9Z1Qwgs7U-dkan56hrUKO9f7uN6Wpd15xQ@mail.gmail.com> <CAHevUJHwDet8pBdrE4SN3nuoAUgP-ixpCz9uOTdwbE31UDDsbA@mail.gmail.com> <CAJ9axoSVhmSkNS6S1zTiwK5F3uUM%2B-8D2478=irZMeTjNxpnfg@mail.gmail.com> <20190923183424.ebnghzf67mx56aom@mutt-hbsd> <CAJ9axoQ-g2Qa2Qnr%2BPOD63s8sTH2Gsi7Rh2VMcZzQF5dd_kBvA@mail.gmail.com> <20190923185113.dyvxxn36gvj4dtu5@mutt-hbsd> <CAJ9axoQjJpOO7G4e5HK0KPO3xJNh8RbQSLZ3C8NFPKtygub_bw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--czzanpxa67by6knx Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Sep 23, 2019 at 10:38:36PM +0200, Ulrich Sp??rlein wrote: > Am Mo., 23. Sept. 2019 um 20:51 Uhr schrieb Shawn Webb > <shawn.webb@hardenedbsd.org>: > > > > On Mon, Sep 23, 2019 at 08:42:10PM +0200, Ulrich Sp??rlein wrote: > > > Am Mo., 23. Sept. 2019 um 20:34 Uhr schrieb Shawn Webb > > > <shawn.webb@hardenedbsd.org>: > > > > > > > > Hey Ulrich, > > > > > > > > I appreciate your hard work in maintaining the git mirror. Work like > > > > this can sometimes go unthanked. I want to take a moment to show > > > > appreciation for you and the FreeBSD project in maintaining the git > > > > mirror. > > > > > > > > I do have a few concerns with what was stated in your email. I've > > > > written my concerns inline. I hope this discussion is a positive on= e, > > > > wherein upstream and downstream can effectively come to a conclusio= n. > > > > > > > > On Mon, Sep 23, 2019 at 08:16:25PM +0200, Ulrich Sp??rlein wrote: > > > > > Am Mo., 23. Sept. 2019 um 19:51 Uhr schrieb Sean Chittenden > > > > > <sean@chittenden.org>: > > > > > >> > > > > > >> Please note however, that more "garbage" metadata escaped from= SVN into > > > > > >> github, meaning 3rd parties have a hard time re-running the co= nversion and > > > > > >> making sure that it matches SVN down to the metadata (i.e. tim= estamps). > > > > > >> > > > > > >> Eventually, this will have to be re-rolled and a new "master" = branch will > > > > > >> be force-pushed into github. There's no timeline for this yet. > > > > > > > > > > > > > > > > > > Wait, what? Can you elaborate? > > > > > > > > > > > > Discussion of a force-push to github has occurred a few times a= nd been explicitly ruled out because most of our corporate citizens use git= hub to integrate changes from FreeBSD. Rerolling master was universally re= jected when we socialized wanting to do this due to the level of disruption= this would cause. The feedback was that this would be a high-cost, low-va= lue operation. In the tradeoffs of purity vs pragmatism, pragmatism wins e= very time (that is the FreeBSD way). > > > > > > > > > > > > -sc > > > > > > > > > > > > > > > This is not just about pragmatism and the disruption it would cau= se is > > > > > vastly overblown by people who don't seem to know much about the = git > > > > > storage model. > > > > > > > > > > There *is* garbage metadata in the published version on github, t= here > > > > > *is* a disclaimer on https://wiki.freebsd.org/GitWorkflow since > > > > > forever, and the cost of switching from 1 published branch to ano= ther > > > > > is literally: > > > > > > > > > > - git diff origin/broken_master mybranch > mybranch.patch > > > > > - git checkout -b fixed_branch origin/fixed_master > > > > > - patch < mybranch.patch > > > > > > > > Such a workflow breaks historical accuracy. Instead of `git annotat= e` > > > > showing the history properly, it's now based on an "epoch commit". > > > > Sure such a commit brings the branch to a working condition, but at > > > > the cost of history. > > > > > > Is there really that much value in having "git blame" work in that en= vironment? > > > My mental model is of short-lived branches that get upstreamed, so I'= m biased > > > towards this not being all that big of a problem (for some at least). > > > > > > > > > > > > > It should also be possible to merge both broken and fixed master = into > > > > > your branch (at the exact same SVN revision in time) and then you= can > > > > > follow fixed_master going forward. You'll schlepp around double t= he > > > > > commit history, but not tree objects. > > > > > If you want to retain history, you can upstream the changes prior= to > > > > > the switch > > > > > > > > I so wish that were possible for certain downstream projects. We're > > > > unable to upstream the majority of our work. To argue "upstream your > > > > work and you won't be affected" is to choose an argument that does = not > > > > reflect the reality of a growing portion of FreeBSD's downstream > > > > consumers: the inability to work effectively with upstream. > > > > > > :/ > > > > > > I'm 80% sure that you can just merge both branches and things will be= fine > > > (though the exact incantation will surely be black magic). I'd love to > > > try this on > > > an actual repo though, I don't have the time to craft some test repo = to verify > > > this assumption, and then find out that other repos are different). > > > > HardenedBSD's github repo has existed since 2013, with branches > > stemming from that work existing still today. Perhaps HardenedBSD is > > somewhat in a special case: we aim to provide the BSD community with a > > clean-room reimplementation of publicly-documented parts of the > > grsecurity patchset. > > > > With FreeBSD not taking the same approach, we will have very > > long-lived branches. For example, our hardened/current/master branch > > follows FreeBSD's HEAD and syncs every six hours. Meaning, we maintain > > our patches, resolving whatever few merge conflicts arrive. The > > hardened/current/master branch was created so many years ago, I've > > forgotten when it was actually created (perhaps in 2013?) > > > > Though HardenedBSD's cause for existence may be a special case, this > > problem can be viewed in a general fashion. I'm confident HardenedBSD > > is not alone in facing issues of these types. > > > > Thanks, >=20 > What I don't understand is how a security focused project can trust a > random source for the svn2git conversion. I could have planted a bunch > of backdoors and then come up with some SVN metadata corruption > conspiracy as to why the commit hashes are different. Why would you > trust me? >=20 > HardenedBSD of all people should be running the converter themselves > and check that the content really matches what is in SVN (which it > currently doesn't for metadata). There's far easier ways for upstream to introduce code into downstream's infrastructure. Just adding `nc -l -p 1234 -e /bin/sh` to a Makefile would do it. Or introducing a vulnerability into telnet in 2019. The possibilities are limited to the imagination and the number of XKCD-obeying butterflies one has on-hand on any given Monday. We at HardenedBSD have to trust FreeBSD in many (uncomfortable) ways. These trust issues are shared amongst all derivatives of FreeBSD, regardless of underlying version control system. Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD Tor-ified Signal: +1 443-546-8752 Tor+XMPP+OTR: lattera@is.a.hacker.sx GPG Key ID: 0xFF2E67A277F8E1FA GPG Key Fingerprint: D206 BB45 15E0 9C49 0CF9 3633 C85B 0AF8 AB23 0FB2 --czzanpxa67by6knx Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAl2JNhoACgkQ/y5nonf4 4fqlDBAAglY5L6xmE3j3o0XKvaQbBesNsDgySAm8uwGWd/5FYKzLuCjlJL989auI +0qxn+waRykOzXxdmyOK7sZh2pbMi3TAxZKekzT9kw+dBx+o77gYvw5ajC+kn9DE 4b857ohbMXxxRyXPWsiEJKC2sMgd4uald2Wc2TG44xtfcDB0+6fL6YJCeDLGLezz fiXU1EqpBEFw05J9qYH9YF3txvQPLoxf/guBaKN8cJBYp4dR4eGNrhagiDaE39o7 UwUSpvQmSuFIOUBTF4cC661iSy0PUbLhq/Y5HzYVA2INdfsvzmjszE1Nl2QRk96D qDHSMe1KwZLqvyosWV3MhnnKi62kIJsEIjY+uLJjne/3/uMoOcbQ8UthOzxpTuJF 79l+or2nQNBzCNZw5Dc84X5r6Lxz3DGH2s6bG7r7g1JXfAXSjBE+i9xdTfyN0smc 0Ab8rpj5aq6yRSUWOQY2UCeDHj1BbsSfNcNpzM146BnSLZkW6TSfqw+kn3AIdN1G NKvkPkHNrOhMyYdRqJaYekt/QsHRsa0aXVn5l+QC7DbBX0M3SdaAOJbObABsZnul 5chn24BBzh85Ui6OeFbTOLv3CQOkmWxUhQaqHPKo44vdF4AVHt6om1qfJyoBbSlw p3GbBKLJfmfnqaTFnAVEuMivgCFWkB1q7C/rXYkrbxBCq8WMMIM= =yc8P -----END PGP SIGNATURE----- --czzanpxa67by6knx--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20190923211617.ifajkj2pwss346ub>