Date: Sat, 26 Sep 2020 06:42:49 -0700 From: David Wolfskill <david@catwhisker.org> To: Ulrich =?iso-8859-1?Q?Sp=F6rlein?= <uqs@freebsd.org> Cc: git@freebsd.org Subject: Re: Question on (my) workflow migration svn -> git Message-ID: <20200926134249.GX1390@albert.catwhisker.org> In-Reply-To: <20200926124212.GL92039@acme.spoerlein.net> References: <20200924201715.GR1390@albert.catwhisker.org> <20200924205326.GB64154@spindle.one-eyed-alien.net> <CAJ9axoR=0zPxUSF-wH-ZT%2BeeHN3-ZB%2B=M6NcbWR1zKncU2m%2BKA@mail.gmail.com> <20200926020946.GK1390@albert.catwhisker.org> <20200926124212.GL92039@acme.spoerlein.net>
next in thread | previous in thread | raw e-mail | index | archive | help
--IqfqxVCWwOOCfbAU Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Sep 26, 2020 at 02:42:12PM +0200, Ulrich Sp=F6rlein wrote: > ... > > The original was ~2.5 GB; the bare clone, ~1.2 GB. >=20 > Ok, this is where you went wrong, I think. Your copy of > freebsd-cgit-beta is a regular clone, yes? Then all the `refs` in there > are suitable for a checked out copy, but not to serve as a "master copy" > of sorts. > (fun fact, you can convert your existing clone into a bare repo by > taking the .git content and 2-3 quick edits to the config therein, but > that's more advanced and would require some more trial and error round > trips) :-} > Please start from this and only this: >=20 > git clone --bare https://cgit-beta.freebsd.org/src.git OK; I've started that process. > The worktree stuff is then as simple as this: >=20 > % cd src.git > % git show-ref|grep stable.12 > e9c4330183dcfe8a5d26843613adc0a348fd0544 refs/heads/stable/12 > % git worktree add ../stable_12 stable/12 > Preparing worktree (checking out 'stable/12') > Updating files: 100% (81262/81262), done. > HEAD is now at e9c4330183d MFC r360483,360484: Make nvmecontrol work > with nda like it does with nvd, and associated bits. > % git worktree add ../current main > Preparing worktree (checking out 'main') > Updating files: 100% (82279/82279), done. > HEAD is now at 9b8dbe73434 Revert most of r360179. > % git worktree list > /tmp/src.git (bare) > /tmp/current 9b8dbe73434 [main] > /tmp/stable_12 e9c4330183d [stable/12] Thanks. I'll poke at that later today. > For debugging, please have a look at `git show-ref` outputs of the > various repos, this will tell you what refs are known to them, and if > stable/12 is not there, all your later attempts to check it out were > doomed to fail. >=20 > Why might stable/12 not be there? For that, please check `config` in the > bare repo (actually it omits any `fetch` directives, as it defaults to > fetching em all), or `.git/config` in a non-bare repo, it'll tell you > what it fetches from where. I suspect that this implies a far deeper understanding of git's internals than I have (or, really, want to have). :-} > > For my use case, it is important to me to keep my local private > > mirrors as close as possible to "the same" (as one another) except > > for the defined periods during which they are being updated. >=20 > Hmm, instead of rsync one into the other and potentially clobbering > custom branches or pulling massive pack files over and over again, I > think this can be better achieved by only checking out the youngest > common commit. Err...? This is a private local mirror of the FreeBSD repo. I do not commit to it. I do not create branches in it. Other than the "mirroring," it is read-only for my purposes. Actually, for the (CVS and) svn case(s), the repos are "owned" by a different user, and I'd need to engage in "privilege escalation" to write to them. I would prefer to be able to maintain that separation using git, but it's not a hard requirement. And the rsync takes perhaps as long as ... a minute? Even svnsync doesn't take long: svnsync started at Sat Sep 26 08:30:00 UTC 2020 svnsync for src ended at Sat Sep 26 08:30:05 UTC 2020 exit status 0 svnsync for ports ended at Sat Sep 26 08:30:09 UTC 2020 exit status 0 svnsync for doc ended at Sat Sep 26 08:30:10 UTC 2020 exit status 0 svnsync started at Sat Sep 26 10:30:00 UTC 2020 svnsync for src ended at Sat Sep 26 10:30:01 UTC 2020 exit status 0 svnsync for ports ended at Sat Sep 26 10:30:02 UTC 2020 exit status 0 svnsync for doc ended at Sat Sep 26 10:30:03 UTC 2020 exit status 0 (I got in the habit of doing the nightly updates in two passes back in the CVS days, when tagging a new branch required updating every file in the branch. [I emphatically do not miss that.] While svn doesn't appear to need that, I kept the approach -- for one thing, it wasn't broken; for another, it provided a bit of a backstop should something happen for one of the attempts.) For reference, the above svnsync runs were: * src, from r366149 -> r366181 -> r366183 * ports, from r550013 -> r550136 -> r550155 * doc, from r54527 -> r54540 -> r54541 And I have finished my "daily updates" for the day, running: FreeBSD g1-48.catwhisker.org 12.2-STABLE FreeBSD 12.2-STABLE #823 r366182M/= 366183: Sat Sep 26 03:38:27 PDT 2020 root@g1-55.catwhisker.org:/common/= S1/obj/usr/src/amd64.amd64/sys/CANARY amd64 on my laptop, and having built & smoke-tested: FreeBSD g1-48.catwhisker.org 13.0-CURRENT FreeBSD 13.0-CURRENT #12 r366181M= /366183: Sat Sep 26 05:33:13 PDT 2020 root@g1-48.catwhisker.org:/common= /S4/obj/usr/src/amd64.amd64/sys/CANARY amd64 1300117 1300117 on it earlier. The build machine is presently running: FreeBSD freebeast.catwhisker.org 12.2-STABLE FreeBSD 12.2-STABLE #1029 r366= 182M/366183: Sat Sep 26 03:37:33 PDT 2020 root@freebeast.catwhisker.org= :/common/S1/obj/usr/src/amd64.amd64/sys/GENERIC amd64 1202502 1202502 and built and smoke-tested: FreeBSD freebeast.catwhisker.org 13.0-CURRENT FreeBSD 13.0-CURRENT #1045 r3= 66181M/366183: Sat Sep 26 05:09:57 PDT 2020 root@freebeast.catwhisker.o= rg:/common/S4/obj/usr/src/amd64.amd64/sys/GENERIC amd64 1300117 1300117 earlier. (The build machine would normally be powered off at this point in the day, but this is Saturday, so it's running pudriere, building packages for tomorrow's update of the "production" machines. As with repo-updating, I do weekly package-building in two stages: the "heavy lift" on Saturday, and a "catch-up" on Sunday -- which mostly works out well enough.) > So instead of rsync-over-ssh, you would do the following when synching > them up: >=20 > % hash=3D`ssh otherhost "cd /path/to/checkout && git show -s --format=3D%= H"` > % git fetch --prune [in the bare repo, this runs after the above, so > it's guaranteed to have that ref] > [actually, I think you can also git pull in the worktree and it will > adjust the bare repo accordingly] > % cd /worktree && git reset --hard ${hash} Kinky. :-} > Hmm, I think you must pull actually, yeah, try with pulling in your > checkout instead of the fetch. > % cd /worktree && git pull && git reset --hard ${hash} >=20 > This will bring your worktree up the latest, but the bump it down to > whatever the otherhost had checked out last. That seems a bit saner than poking around determining repo hashes and =2E.. stuff; thanks. :-) Please note that the laptop treats the build machine as its source-of-truth as often as possible, and "setlles" (temporarily) for svn.freebsd.org when the build machine is unavailable. Once the build machine is available again, the next update resets the laptop to mirroring the build machine again. (And since the build machine uses svn.freebsd.org as its source-of-truth, this should not be a large difference -- in most cases, it's no difference at all.) > Cheers > Uli Oh -- that `git clone --bare` finished. So: g1-48(12.2-S)[5] cd src.git/ g1-48(12.2-S)[6] git show-ref|grep stable.12 e9c4330183dcfe8a5d26843613adc0a348fd0544 refs/heads/stable/12 g1-48(12.2-S)[7] git show-ref | grep stable/12 e9c4330183dcfe8a5d26843613adc0a348fd0544 refs/heads/stable/12 g1-48(12.2-S)[8] git worktree add /bkp/tmp/git/stable_12 stable/12 Preparing worktree (checking out 'stable/12') Updating files: 100% (81262/81262), done. HEAD is now at e9c4330183d MFC r360483,360484: Make nvmecontrol work with n= da like it does with nvd, and associated bits. g1-48(12.2-S)[9] git worktree add /bkp/tmp/git/head main Preparing worktree (checking out 'main') Updating files: 100% (82279/82279), done. HEAD is now at 9b8dbe73434 Revert most of r360179. g1-48(12.2-S)[10]=20 (My repo mirrors live on a dedicated file system -- in part so I can leave it unmounted when I'm experimenting with head. So I put the "worktrees" on a different file system from the repo -- and if those worktrees are to be my src directories, they will be on separate file systems (on separate slices, even) anyway in real life.) In any case, it looks as if we have head & stable/12 sources based on the sane (bare) repo, so that's progress. I'll plan on re-visiting the "re-silvering" of the mirror somewhat later this weekend. Thanks again! Peace, david --=20 David H. Wolfskill david@catwhisker.org "We aren't hiding. We are named witnesses to Donald Trump's unfitness for office." -- Miles Taylor, former chief of staff to former DHS Secretary Kirstjen Nielsen See http://www.catwhisker.org/~david/publickey.gpg for my public key. --IqfqxVCWwOOCfbAU Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGTBAEBCgB9FiEE4owz2QxMJyaxAefyQLJg+bY2PckFAl9vRVlfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEUy OEMzM0Q5MEM0QzI3MjZCMTAxRTdGMjQwQjI2MEY5QjYzNjNEQzkACgkQQLJg+bY2 PcnI+Af9GibcawMD28KI+7389aUqlcdYjLynw7VzxzOEh+AV1gxHSRpV4mv3biNi J/aaglrWUOPjNDhrhSczOF0NqhT1OcB8d0YJb6bM6AVG8ZzUp2x/LpGqhTEHnRx1 cGsiJ/E7YYHLgu7ojuz3iCIm5iba7KdlbgMYFy5M8lyVaPZOhdbx14MmpghD7zbs uvv4ConDYt2Ul+3fPBoRa5o6K2X42XlGycUIk8SU2K7Y8B2ywCZAwCA/m6ieS+Ve Y92XzBf5MSyBYik5dYMLFdt8Y4XshjmxGvcYufRmTF4UBPUub2z6cJ8IKZtdjp6c WXXtwZ+Cihuzf4XhiTdjS+xFiKV68g== =Mbi0 -----END PGP SIGNATURE----- --IqfqxVCWwOOCfbAU--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20200926134249.GX1390>