Date: Sat, 26 Dec 2020 11:44:06 +0100 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= <uqs@freebsd.org> To: FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: src: continued use of Subversion for getting updates Message-ID: <X%2BcT9s0pIWILiOT0@acme.spoerlein.net> In-Reply-To: <20201226015530.z2iL0%steffen@sdaoden.eu> References: <E1ks2I6-0005nh-BQ@rmmprod05.runbox> <20201223143545.Wf_Ww%steffen@sdaoden.eu> <X%2BZUa7NyatH2ktYI@acme.spoerlein.net> <20201225214041.jVKMU%steffen@sdaoden.eu> <CANCZdfpgFUFzW3b_c2sa1zYcuNFjrBw_mtkCzT4_D5tajXCkXA@mail.gmail.com> <CANCZdfpNM19OyQtXH0OLHxzrQf4dqpBcRZzfBCuKfRyo5Wcziw@mail.gmail.com> <20201225222510.JyYwH%steffen@sdaoden.eu> <CANCZdfpidSxN-3fqWC8z=9ANytzF9y9WcgmEejGTsihfa-Pr-Q@mail.gmail.com> <20201226015530.z2iL0%steffen@sdaoden.eu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, 2020-12-26 at 02:55:30 +0100, Steffen Nurpmeso wrote:
>Warner Losh wrote in
> <CANCZdfpidSxN-3fqWC8z=9ANytzF9y9WcgmEejGTsihfa-Pr-Q@mail.gmail.com>:
> |On Fri, Dec 25, 2020 at 3:25 PM Steffen Nurpmeso <steffen@sdaoden.eu> \
> |wrote:
> |> Warner Losh wrote in
> |> <CANCZdfpNM19OyQtXH0OLHxzrQf4dqpBcRZzfBCuKfRyo5Wcziw@mail.gmail.com>:
> |>|> On Fri, Dec 25, 2020 at 2:41 PM Steffen Nurpmeso <steffen@sdaoden.eu>
> |>|> wrote:
> |>|>> Ulrich Spörlein wrote in
> |>|>> <X+ZUa7NyatH2ktYI@acme.spoerlein.net>:
> |>|>>|On Wed, 2020-12-23 at 15:35:45 +0100, Steffen Nurpmeso wrote:
> ...
> |>|>>|>I really dislike that vendor imports have been tagged. Because
> ...
> |>|>>|That's basically what was done? I don't understand what you're saying
> |>|>>|here ...
> |>|>> Well, cgit-beta did not have had all these tags if i recall
> ...
> |>|> It had them, but not under the refs/head/vendor space but under the
> |>|> refs/vendor space.
> |>
> |> These are not tags but branches. I have nothing against the
> ...
> |>|> They did exist. They were under refs/vendor rather than
> |> refs/head/vendor
> |>|> though.
> |>
> |> Under refs/tags/vendor? refs/tags/ is the "special" namespace
> |> managed by "git tag", this is different than the rest.
> ...
> |>|>> there is
> |>|>>
> |>|>> #?0|kent:free-src.git$ git sr|wc -l
> |>|>> 2137
> |>|>>
> |>|>> but if i go for "the real" FreeBSD itself it is just
> |>|>>
> |>|>> #?0|kent:free-src.git$ git sr | grep -v vendor | wc -l
> |>|>> 19
> |>|>
> |>|You might be happier tracking on github, once we start pushing there as
> |> the
> |>|vendor branches won't be published there.
> |>
> |> No problem with any number of branches, Warner. Just tags under
> |> refs/tags this is above.
> |
> |They were tags in the cgit-beta as well (and a few branches). I don't
> |believe that detail changed, but my old copies of the repo are gone.
>
>Yes mine is gone, for a month. Then i am sorry that i did not
>speak out by then.
They were annotated tags, git doesn't care where you actually store
them. Yes, refs/tags/ is special but more in terms of automatically
pulling them down, not in whether that has commit objects or tag
objects.
> ...
> |>|>> Which is a pity since all these references will be checked during
> |>|>> "git fetch" unless i am mistaken.
> |>|>
> |>|Yes. So far it's been doing it quite quickly for me, but I'm decently
> |>|connected...
> |>
> |> Yes, terrible here, shared with many.
> |
> |You may be happier running custom refs, or grab from github. The source of
> |truth has a lot of stuff. We may work to prune some of the vendor branches
>
>I am not interested in the entire repo, yes. For almost a decade
>i had only the sources as such, without history, for me this is an
>improvement. As i am not a BSD developer it is for snooping
>around only. And when i report bugs they are not fixed, but i am
>not alone with this. Nonetheless: interest in FreeBSD here, ok.
You say you're not interested in the entire repo, but as soon as you
fetch `main` or any of the stable branches you get the entire repo
anyway. So I don't understand what you're talking about ...
>
> |into their own repos in the future, but today there's a lot of stuff that's
> |there, some of which is for the convenience of the developer and you may
> |need to trim (at least in the short term).
>
> | ...
> |>|>>|That's a valid point, we debated whether to keep vendor tags and
> |>|>> decided
> |>|>>|for now to replicate what we have in SVN. We can still delete all the
> |>|>>|vendor tags on the main repo anytime we want ...
> |>|>>
> |>|>> I personally would track that in the commit message of the import
> |>|>> on the vendor branch that anyway exists(!), and then when merging
> |>|>> this into the mainline, but not create a real tag in the tag
> |>|>> namespace. Also the backups/ and such, because why?
> |>|>
> |>|We need tags to keep track of what's been done, and to revert and do
> |> other
> |>|management things with vendor imports.
> |>
> |> But why? You have the commit on a topic/vendor branch, and yppou
> |> revert nothing but the commit. In fact doing so messes the tag,p
> |> it has to be retagged when you do re-commit an import proper,
> |> which requires a forced push even!
> |
> |I'm not sure I follow. Unless I misunderstand, you are describing a
> |different problem with different issues.
>
>No, maybe i was mistaken. I never did a vendor import myself.
>..But looking at the history i see lots of import disasters ;-))
>Look for example at llvm 10.x this year, it consumed three _tags_!
>
>I am luckily free to say that this is merde (without the intention
>to annoy the poor one who did it) and am going to talk. I presume
>most of you can do git(1) better than i, i use it for a decade
>(almost exactly in fact!), but have never stepped forward, and
>have a very primitive way of daily use.
>
>There should be a real per-vendor branch, say [vendor/sqlite].
>The way FreeBSD seems to do vendor imports this should even be
>easy, since vendor stuff is usually in separate directories.
>Create it once it is needed first, merge into main via --no-ff so
>that you get real "merge commits". You can see the difference
>below.
Umm, no offense but wtf are you talking about? I asked this earlier but
you didn't reply to that part of the question. Of course the vendor
branches are stand-alone vendor branches.
% git log --reverse --compact-summary -n2 origin/vendor/sqlite3
commit c80e66e8e79185b1e7c999decef3d4adfdb902de (tag: vendor/sqlite3/sqlite-3320300)
Author: Cy Schubert <cy@FreeBSD.org>
Date: 2020-07-07 13:48:26 +0000
Import sqlite 3.32.3 (3320300).
Notes:
svn path=/vendor/sqlite3/dist/; revision=362990
svn path=/vendor/sqlite3/sqlite-3320300/; revision=362991; tag=vendor/sqlite3/sqlite-3320300
configure | 20 +++++++-------
configure.ac | 2 +-
sqlite3.c | 325 +++++++++++++++++...
sqlite3.h | 6 ++---
tea/configure | 18 ++++++-------
tea/configure.ac | 2 +-
6 files changed, 272 insertions(+), 101 deletions(-)
commit 2793f2eef2be94a38e38babede1b01c3c50196fe (tag: vendor/sqlite3/sqlite-3330000, origin/vendor/sqlite3, freebsd/vendor/sqlite3)
Author: Cy Schubert <cy@FreeBSD.org>
Date: 2020-08-21 22:54:38 +0000
Import sqlite 3.32.3 (3330000).
Notes:
svn path=/vendor/sqlite3/dist/; revision=364467
svn path=/vendor/sqlite3/sqlite-3330000/; revision=364469; tag=vendor/sqlite3/sqlite-3330000
Makefile.am | 2 +-
Makefile.in | 2 +-
configure | 20 +-
configure.ac | 2 +-
shell.c | 1618 +++++++++++++++++--
sqlite3.c | 19228 +++++++++++++++....
sqlite3.h | 1378 ++++++++--------
sqlite3rc.h (new) | 3 +
tea/configure | 18 +-
tea/configure.ac | 2 +-
10 files changed, 12150 insertions(+), 10123 deletions(-)
% git ls-tree -r origin/vendor/sqlite3
100644 blob a1e89e18ad20c227845f2099cb9894c799265d19 INSTALL
100644 blob 694419b27dfd74741846d068c3e5a626d14b1797 Makefile.am
100644 blob 9355b147a8fd3cd10edcf89ac955bf3f9a9d56ba Makefile.fallback
100644 blob 842fa864581de8788d5e74eab961c01bd157a778 Makefile.in
100644 blob 746162a00c04298d13bb01ac26e77ad2cca566a4 Makefile.msc
100644 blob 6e62a4e13854dc976b1f7e16caa6c0dd042ca90c README.txt
100644 blob 3475a47e6e813a7e8c2ff0893d4ee28442bd15fb Replace.cs
100644 blob 53c1fc39d80b85fb98daa8607ce6b6b6af8fc7f2 aclocal.m4
100755 blob a85b723c7e67d46316e85e7422bd5088e9136042 compile
100755 blob f50dcdb6de2af0a2e33f44704da3ec1286e5f291 config.guess
100755 blob 1d8e98bcee23a0421e4fafe9a6c9ac75180cff25 config.sub
100755 blob 9aed16a74091aea4ee9911922ed79ab1b72d4cd5 configure
100644 blob a83dac3ac1425d4f13d0186da67861683fb2bae1 configure.ac
100755 blob b39f98f9ae9f950391abb09f4fa03ee113a07ac6 depcomp
100755 blob 59990a10492675f2e87d5e5df17b566d145d9aee install-sh
100755 blob a736cf994256132aefd49c1f11118ad7ba31d924 ltmain.sh
100755 blob f62bbae306c7e1bc28896aab8fe7bfb700a9a33e missing
100644 blob a1a77e49fa5f91625ee02efc36818ffab9fe1219 shell.c
100644 blob 80353b0eecd9848204c711d6264ca116b2d1064b sqlite3.1
100644 blob a82744931c0b7f1ed3baf079bcc75090448d00b4 sqlite3.c
100644 blob 910b687aa7df2d45ada1e7a57bd1794ae3fd4227 sqlite3.h
100644 blob 3799671e613b34a4551d1010c13e334b89123ea1 sqlite3.pc.in
100644 blob 5a856490d64a45062c5e8d648e51cc8b6d09dbdc sqlite3.rc
100644 blob 78c19a0d10ce60b29720889c2c1d0ba746ef9169 sqlite3ext.h
100644 blob 80feb9e1cd61e4d89ff9c8a0595024430a6229f1 sqlite3rc.h
100644 blob 3e481dadfe84fb65f64c81bcb3cd08a1ddc0855c tea/Makefile.in
100644 blob 99dc8b8f03cda56e80e8dbf594bfe1feb3f880ae tea/README
100644 blob 0b057391d2919535a4427b09987c06287eb6e05e tea/aclocal.m4
100755 blob 4ff25cb1ce06affe22a8b2f9efb157376e270cb2 tea/configure
100644 blob bfc4eca248c1da282960b16f45e08150e4478360 tea/configure.ac
100644 blob 13913e5583d8258ff244f945710f39dacf0adb9d tea/doc/sqlite3.n
100644 blob 4d722eb6c3c7f32b4ad9cdb9da3d1c15e4cf528a tea/generic/tclsqlite3.c
100644 blob 723c4cd3c6e91a9f86ee9fe667923a316ba17d03 tea/license.terms
100644 blob bc585f73b3007cf63086532cddc38bfe4a246236 tea/pkgIndex.tcl.in
100644 blob 7c34c3f926031734a8e1a4234a7ab131bdefda3e tea/tclconfig/install-sh
100644 blob 4b4bd1e888964cc55c78a97591f40538cd5b21ec tea/tclconfig/tcl.m4
100644 blob 88b66f173cb3d41a97023dc547d77fb0fcdafcff tea/win/makefile.vc
100644 blob e00f1b49965d07d0ac4862354131776bdaf714eb tea/win/nmakehlp.c
100644 blob 99471053c8c0a38e9571d8809402b032b250401c tea/win/rules.vc
They are tagged because we tagged them in SVN and in SVN it's very
annoying to get the history of a branch only. And yes, we talked in the
WG about dropping the tags eventually, as they serve little purpose.
>Then there are basically two options. One is to simply switch to
>the vendor branch, work and commit there as often as necessary,
>and then merge to main when done. The other is to instead rebase the
>vendor branch to main first _before_ you start the work, any
>_maybe_, _possibly_ again if the final push fails.
>
>The former approach is perfect for vendor stuff which really
>stands for itself, you can see below why this is so (the history
>of the vendor branch is really only about the vendor branch, at
>least after it has been created).
>
>The latter is the only way to go if there are conflicts because
>the vendor package reaches out into the base system, like i would
>expect for llvm. Then you want to rebase to main so that any
>merge conflict resolving happen on the actually current state of
>the art, of course, not on some old tree.
>
>Like this the history of the vendor branch is not "clean" in that
>it includes commits of the main branch that happened in the
>meantime, but .. i would not care about this.
>(One could also get the same vendor-branch-history-only also for
>this kind, of course, easily even. For this i would checkout the
>branch, "git rm -rf '*'", do "git archive --format=tar --prefix=
>main | tar -xf -", then "git add .", then "git commit -m
>SYNC-WITH-MAIN", .. and then do the normal vendor branch stuff.)
Please explain how this is different from the above? I'm really not sure
what you're saying here ...
>
> |But the nice thing about refs is that you can always change them
> later if
> |the current scheme isn't working out... So far, it is for most people, so
>
>This requires forced pushes and thus special configuration.
>"Normally", it is said, this should be avoided. The fossil scm
>does not even support it i think. I for myself do that, but only
>on development branch, release and stable and master branches are
>immutable (but in disasters).
Adding or removing refs or tags doesn't need a force push. Please stop
spreading this misinformation.
As for your network trouble, have you measured the bandwidth difference
between: git fetch and git fetch --no-tags and if there's a big
difference I'd suggest to simply use the later?
I just tried to measure the difference with trafshow and it was pretty
much 9k up 560k down for both flags. Then I deleted all my local copies
of the tags and it went to 8k up, 560k down. Probably a measurement
error. Am I holding this wrong?
Cheers
Uli
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?X%2BcT9s0pIWILiOT0>