Date: Wed, 2 Oct 2024 09:29:00 -0600 From: Warner Losh <imp@bsdimp.com> To: Brooks Davis <brooks@freebsd.org> Cc: Shawn Webb <shawn.webb@hardenedbsd.org>, Matthias Fechner <mfechner@freebsd.org>, git@freebsd.org Subject: Re: Impossible to push my ports directory to my CI/CD pipeline Message-ID: <CANCZdfo-CjFA5v9oB5571GDUALwh%2BNpnd_uu7rS26-LWnxPrag@mail.gmail.com> In-Reply-To: <Zv1mFbI0CIToXD-D@spindle.one-eyed-alien.net> References: <483fb132-0d1c-443a-9b44-b7f2f087fb3d@freebsd.org> <736exensjgsyhbytunglk6qnk3dh4fyyvdc5jktzrv3zzujbgr@db6amzcu53h2> <Zv1mFbI0CIToXD-D@spindle.one-eyed-alien.net>
next in thread | previous in thread | raw e-mail | index | archive | help
--0000000000008f88d70623801996 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, Oct 2, 2024, 9:26=E2=80=AFAM Brooks Davis <brooks@freebsd.org> wrot= e: > On Tue, Oct 01, 2024 at 07:37:00PM +0000, Shawn Webb wrote: > > On Mon, Sep 30, 2024 at 01:09:32PM UTC, Matthias Fechner wrote: > > > Dear all, > > > > > > I already tried to address that problem on the all-developer list, bu= t > it is > > > maybe better to put it here. > > > I think by accident someone pushed LFS object with a commit of > java/eclipse. > > > > > > git remote -v > > > freebsd git@gitrepo.freebsd.org:ports.git (fetch) > > > freebsd git@gitrepo.freebsd.org:ports.git (push) > > > freebsd-https?? https://git.freebsd.org/ports.git (fetch) > > > freebsd-https?? https://git.freebsd.org/ports.git (push) > > > githubfreebsd?? https://github.com/freebsd/freebsd-ports.git (fetch) > > > githubfreebsd?? https://github.com/freebsd/freebsd-ports.git (push) > > > origin? git@gitlab.fechner.net:mfechner/Gitlab.git (fetch) > > > origin? git@gitlab.fechner.net:mfechner/Gitlab.git (push) > > > > > > git lfs fetch --all freebsd > > > fetch: 22 object(s) found, done. > > > fetch: Fetching all references... > > > [99a670c083e000008ee29dd972b60cbad6ade1d6ec024b30ff45f41da3d19e1d] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [c89e491472c8af1d153e2e2801836fad0e38f428d0250ed982e16baa5effbe4f] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [f7db6794c6c3e7a03a3987537b8a0cdb1bde2996ed7c7eec50f2e5df1e7a30c2] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [e8e2d306fd11358a07b4ee10bfef19de3b518793568fc498cdd989170d31712b] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [5dbe70b5c83520df09c100815bb164c8903da5eb8bffbd094aaaed54b212d73a] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [8813907b0f7b001080a346ed2d0307a1c749b85688fdfa1d270fe59a07aa3909] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [1d41b04bdd4115c52aac8177e1b899403a3d7d9f6d147cd864463029648a4714] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [c0155fc8e7c4b9736d81db638fed95bc9ff297453bfd0bbd7dd5d882ebca6e89] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [6d0aef4ba9e96d4c63baae8db9cdce7c634140a2bda7da670e698e38e0df57ce] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [a928176ec92ad347404b2454e5f4f321aaef8ba862a60b0c28adc3bdcc619a4a] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [42fa73302624dc511e6d8958c66c82f31b8d9155f2980343b6bcdc25345e1dcd] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [ed324ddc16ccb8b9fab3041da394b844166b6778c39cd12708bcade05b41c0da] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [9ef847d4e699b5dbe49ac74ee35596fdddc22c6335bc96affee19a99ecd077f5] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [f5417c277c6ca509334010bb6999c56c3ab5d3751204c74dc00c2fa25d93dfae] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [a1ffa591430df806983c8798113264f12ea679bce4b785d70d2868dc91cec883] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [b2e525c38598dac6885a75e7eaffa346c8aca3dbdda73cb0275f50103ec3d245] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [5db9424884396cf5bfc46a6b22dacbe245f43a7e1eda6aeac831905dc921802e] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [b5e05ef77effb4048e72e38605700151fdf4eae214cfa358e9e81a48e124cb52] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [ca2c21bf21e1fa2f53c72653ea57e76506c305bdd44c6416542ebbf2ab8b1b83] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [4fc786126dd64f936554a12c4752787c4989453654fc5d6fd119b6e5eeafec6f] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > [f56db5e81304e441260b84d6c2805ae7050ea56760d763d2b32e6be928fa1cae] > Object > > > does not exist on the server: [404] Object does not exist on the serv= er > > > error: failed to fetch some objects from > > > 'https://github.com/freebsd/freebsd-ports.git/info/lfs' > > > > > > I have my own testing pipeline in place to automatically test my > commits > > > against changes I did to ports. > > > > > > But as LFS is now in the FreeBSD repository, but the files cannot be > > > retrieved from the FreeBSD repository I'm now locked out for my CI/CD > > > pipeline and my automated tests. > > > > > > If I now try to push to my local repo I get: > > > > > > git push origin > > > Warning: untrusted X11 forwarding setup failed: xauth key data not > generated > > > Locking support detected on remote "origin". Consider enabling it wit= h: > > > ? $ git config > > > lfs. > https://gitlab.fechner.net/mfechner/Gitlab.git/info/lfs.locksverify true > > > Unable to find source for object > > > 99a670c083e000008ee29dd972b60cbad6ade1d6ec024b30ff45f41da3d19e1d (try > > > running git lfs fetch --all) > > > Uploading LFS objects:?? 0% (0/21), 0 B | 0 B/s, done. > > > error: failed to push some refs to 'gitlab.fechner.net: > mfechner/Gitlab.git' > > > > > > Does anyone have an idea how that can be fixed? > > > > Hey Matthias, > > > > With commit c94e4ced329298efb06189d1c6db26e3d6ff3eef[1], FreeBSD > > introduced LFS objects into the ports git history. This commit was > > reverted with commit 3ec21e417cd5752c96b88587a5a605567ba5cd19[2]. > > > > Even though the offending commit was reverted, the git commit history > > still contains LFS references. This caused HardenedBSD's self-hosted > > GitLab to become unhappy in exactly the same way yours did. > > > > I ended up disabling LFS for the ports repo in the GitLab side. This > > caused GitLab to become happy again. Problem is, now everyone > > downstream from us (and FreeBSD, too) will need to disable LFS on > > their repos/forks, too. > > I'm curious if you can turn LFS back on once the broken change is past? > That that ports should ever use LFS, but it seems somewhat better if > GitLab configs only need to be non-default during bootstrapping. > > At this point there's probably nothing to be done other than workarounds > as rewriting history to remove this stuff would be quite disruptive > after several weeks of commits. > > We could really use some hooks to block both things that look like lfs > metadata and submodule bits. (The latter is somewhat challenging in that > we need to allow them on the vendor side to support OpenZFS.) > I didn't think we imported the submodules of OpenZFS to the vendor branch, except to the extent that data was plain files in that import not otherwise used... Warner > > --0000000000008f88d70623801996 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto"><div><br><br><div class=3D"gmail_quote"><div dir=3D"ltr" = class=3D"gmail_attr">On Wed, Oct 2, 2024, 9:26=E2=80=AFAM Brooks Davis <= <a href=3D"mailto:brooks@freebsd.org">brooks@freebsd.org</a>> wrote:<br>= </div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex">On Tue, Oct 01, 2024 at 07:37:00PM +00= 00, Shawn Webb wrote:<br> > On Mon, Sep 30, 2024 at 01:09:32PM UTC, Matthias Fechner wrote:<br> > > Dear all,<br> > > <br> > > I already tried to address that problem on the all-developer list= , but it is<br> > > maybe better to put it here.<br> > > I think by accident someone pushed LFS object with a commit of ja= va/eclipse.<br> > > <br> > > git remote -v<br> > > freebsd git@gitrepo.freebsd.org:ports.git (fetch)<br> > > freebsd git@gitrepo.freebsd.org:ports.git (push)<br> > > freebsd-https?? <a href=3D"https://git.freebsd.org/ports.git" rel= =3D"noreferrer noreferrer" target=3D"_blank">https://git.freebsd.org/ports.= git</a> (fetch)<br> > > freebsd-https?? <a href=3D"https://git.freebsd.org/ports.git" rel= =3D"noreferrer noreferrer" target=3D"_blank">https://git.freebsd.org/ports.= git</a> (push)<br> > > githubfreebsd?? <a href=3D"https://github.com/freebsd/freebsd-por= ts.git" rel=3D"noreferrer noreferrer" target=3D"_blank">https://github.com/= freebsd/freebsd-ports.git</a> (fetch)<br> > > githubfreebsd?? <a href=3D"https://github.com/freebsd/freebsd-por= ts.git" rel=3D"noreferrer noreferrer" target=3D"_blank">https://github.com/= freebsd/freebsd-ports.git</a> (push)<br> > > origin? git@gitlab.fechner.net:mfechner/Gitlab.git (fetch)<br> > > origin? git@gitlab.fechner.net:mfechner/Gitlab.git (push)<br> > > <br> > > git lfs fetch --all freebsd<br> > > fetch: 22 object(s) found, done.<br> > > fetch: Fetching all references...<br> > > [99a670c083e000008ee29dd972b60cbad6ade1d6ec024b30ff45f41da3d19e1d= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [c89e491472c8af1d153e2e2801836fad0e38f428d0250ed982e16baa5effbe4f= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [f7db6794c6c3e7a03a3987537b8a0cdb1bde2996ed7c7eec50f2e5df1e7a30c2= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [e8e2d306fd11358a07b4ee10bfef19de3b518793568fc498cdd989170d31712b= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [5dbe70b5c83520df09c100815bb164c8903da5eb8bffbd094aaaed54b212d73a= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [8813907b0f7b001080a346ed2d0307a1c749b85688fdfa1d270fe59a07aa3909= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [1d41b04bdd4115c52aac8177e1b899403a3d7d9f6d147cd864463029648a4714= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [c0155fc8e7c4b9736d81db638fed95bc9ff297453bfd0bbd7dd5d882ebca6e89= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [6d0aef4ba9e96d4c63baae8db9cdce7c634140a2bda7da670e698e38e0df57ce= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [a928176ec92ad347404b2454e5f4f321aaef8ba862a60b0c28adc3bdcc619a4a= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [42fa73302624dc511e6d8958c66c82f31b8d9155f2980343b6bcdc25345e1dcd= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [ed324ddc16ccb8b9fab3041da394b844166b6778c39cd12708bcade05b41c0da= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [9ef847d4e699b5dbe49ac74ee35596fdddc22c6335bc96affee19a99ecd077f5= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [f5417c277c6ca509334010bb6999c56c3ab5d3751204c74dc00c2fa25d93dfae= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [a1ffa591430df806983c8798113264f12ea679bce4b785d70d2868dc91cec883= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [b2e525c38598dac6885a75e7eaffa346c8aca3dbdda73cb0275f50103ec3d245= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [5db9424884396cf5bfc46a6b22dacbe245f43a7e1eda6aeac831905dc921802e= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [b5e05ef77effb4048e72e38605700151fdf4eae214cfa358e9e81a48e124cb52= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [ca2c21bf21e1fa2f53c72653ea57e76506c305bdd44c6416542ebbf2ab8b1b83= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [4fc786126dd64f936554a12c4752787c4989453654fc5d6fd119b6e5eeafec6f= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > [f56db5e81304e441260b84d6c2805ae7050ea56760d763d2b32e6be928fa1cae= ] Object<br> > > does not exist on the server: [404] Object does not exist on the = server<br> > > error: failed to fetch some objects from<br> > > '<a href=3D"https://github.com/freebsd/freebsd-ports.git/info= /lfs" rel=3D"noreferrer noreferrer" target=3D"_blank">https://github.com/fr= eebsd/freebsd-ports.git/info/lfs</a>'<br> > > <br> > > I have my own testing pipeline in place to automatically test my = commits<br> > > against changes I did to ports.<br> > > <br> > > But as LFS is now in the FreeBSD repository, but the files cannot= be<br> > > retrieved from the FreeBSD repository I'm now locked out for = my CI/CD<br> > > pipeline and my automated tests.<br> > > <br> > > If I now try to push to my local repo I get:<br> > > <br> > > git push origin<br> > > Warning: untrusted X11 forwarding setup failed: xauth key data no= t generated<br> > > Locking support detected on remote "origin". Consider e= nabling it with:<br> > > ? $ git config<br> > > lfs.<a href=3D"https://gitlab.fechner.net/mfechner/Gitlab.git/inf= o/lfs.locksverify" rel=3D"noreferrer noreferrer" target=3D"_blank">https://= gitlab.fechner.net/mfechner/Gitlab.git/info/lfs.locksverify</a> true<br> > > Unable to find source for object<br> > > 99a670c083e000008ee29dd972b60cbad6ade1d6ec024b30ff45f41da3d19e1d = (try<br> > > running git lfs fetch --all)<br> > > Uploading LFS objects:?? 0% (0/21), 0 B | 0 B/s, done.<br> > > error: failed to push some refs to 'gitlab.fechner.net:mfechn= er/Gitlab.git'<br> > > <br> > > Does anyone have an idea how that can be fixed?<br> > <br> > Hey Matthias,<br> > <br> > With commit c94e4ced329298efb06189d1c6db26e3d6ff3eef[1], FreeBSD<br> > introduced LFS objects into the ports git history. This commit was<br> > reverted with commit 3ec21e417cd5752c96b88587a5a605567ba5cd19[2].<br> > <br> > Even though the offending commit was reverted, the git commit history<= br> > still contains LFS references. This caused HardenedBSD's self-host= ed<br> > GitLab to become unhappy in exactly the same way yours did.<br> > <br> > I ended up disabling LFS for the ports repo in the GitLab side. This<b= r> > caused GitLab to become happy again. Problem is, now everyone<br> > downstream from us (and FreeBSD, too) will need to disable LFS on<br> > their repos/forks, too.<br> <br> I'm curious if you can turn LFS back on once the broken change is past?= <br> That that ports should ever use LFS, but it seems somewhat better if<br> GitLab configs only need to be non-default during bootstrapping.<br> <br> At this point there's probably nothing to be done other than workaround= s<br> as rewriting history to remove this stuff would be quite disruptive<br> after several weeks of commits.<br> <br> We could really use some hooks to block both things that look like lfs<br> metadata and submodule bits.=C2=A0 (The latter is somewhat challenging in t= hat<br> we need to allow them on the vendor side to support OpenZFS.)<br></blockquo= te></div></div><div dir=3D"auto"><br></div><div dir=3D"auto">I didn't t= hink we imported the submodules of OpenZFS to the vendor branch, except to = the extent that data was plain files in that import not otherwise used...</= div><div dir=3D"auto"><br></div><div dir=3D"auto">Warner</div><div dir=3D"a= uto"><div class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"= margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <br> </blockquote></div></div></div> --0000000000008f88d70623801996--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfo-CjFA5v9oB5571GDUALwh%2BNpnd_uu7rS26-LWnxPrag>