From owner-freebsd-git@freebsd.org Sun Nov 29 16:47:07 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 74C2F4A79A0 for ; Sun, 29 Nov 2020 16:47:07 +0000 (UTC) (envelope-from rene@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4CkZ5l2glBz4swZ for ; Sun, 29 Nov 2020 16:47:07 +0000 (UTC) (envelope-from rene@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 5A2084A7A27; Sun, 29 Nov 2020 16:47:07 +0000 (UTC) Delivered-To: git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 59E934A77EF for ; Sun, 29 Nov 2020 16:47:07 +0000 (UTC) (envelope-from rene@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CkZ5l29gvz4ssy; Sun, 29 Nov 2020 16:47:07 +0000 (UTC) (envelope-from rene@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1606668427; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=xEg5GnH90gKpGGphc0ID0ov29Ss6ZTlkCvXnmyLSkOQ=; b=c5Dcr3WSGsXwJx4sNfn4Qn6YZZnobR0i9bhBEj1RAKy0ERvrUEXvIwX3lIBZetbwlaWDjr jPCKusEE4h54i9rIDD96vsQGF37p77IHZdhvnyVm9uNY80uD4TCXMcksQHxnBOJRIUtv9y L8XJG+8l0FapP1Huo1uq49Zr3MwYS/Fk1TRbOYiRrNhKdzAHItG8LkfifjgWgez0VbPPUz RbKgMnXgS2rtDy9QBhH3WeLZunc5enWuGDgRsVNjaatdIEyiVFaPu1CdoLJPM1Ry/v5rad Z0+YBe3b55GRGQmXeMGjFimcuQjP+HB2Nla7szSF4Le4odmMGe8dcgL9ScJhsA== Received: by freefall.freebsd.org (Postfix, from userid 1185) id 40D74A110; Sun, 29 Nov 2020 16:47:07 +0000 (UTC) Date: Sun, 29 Nov 2020 16:47:07 +0000 From: Rene Ladan To: crees@freebsd.org, git@freebsd.org, portmgr@freebsd.org Subject: converting rmport to git Message-ID: <20201129164707.GA31739@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1606668427; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=xEg5GnH90gKpGGphc0ID0ov29Ss6ZTlkCvXnmyLSkOQ=; b=BVW8vSSZOc42OO8kWGQiIsq623Gg3ZPX7SPbyomYAXk3YXt/NBgL2Mvbgt+9sMPVtvC4si mQyOpMSNPDW+tZbmbHQJhSzLdN5xJc0aBqVW5nHKJuDLKRFqh1vAjViT0qFWbvctzy1s1P /rw5s6jsCqK+28MR9mHyHlXTNNupFi+CLylShhqIi851/zwYPxEwctW3MrrJD7Ne7elP1I oAiQJw49xPXaE0N9qe6uTCIyhchwHL/oYZDSsRW41QIC5Yxl+KqReF0jdwwiZdNuSeA8lZ 8gl0F0uUm57wwZwOlTmE3+M+lvEXa4OyvHS6RdZYEuMNi7wMdroltsS+1pYEFw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1606668427; a=rsa-sha256; cv=none; b=nGFxjFfFL44DlL1e22PgREAzj9SDF77i9nYM32ZHMYMH7RpKNexHs73S+IbtHSTEdixiVZ wKKXzRViXcvGdvPo0iIauxlGzgZmUlWvV8vh5LUWgUdgLSaTJay3xvFjEvDMWnKpS6p/d+ Q8yI0bUgI0hTFeJWONsMJ2Yg6C0XMpXMiQRA+LjrBiyWVT0PzNFo0Yn0vq4i6n7hHZO4+Z z+eyZ8F9LsZqzx58M0HkkWqFqMSrIDvLYncQKwfCF4NzwPHgo6ttny3RiHmdX4eGJd1WXf mshltZtsmNrZQHWIBroRGQOgoxUKu0ejFFBXdyTkr5f+Vf5X2frvJWY8j6SRmA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Nov 2020 16:47:07 -0000 Hello, I started converting some scripts in the port tree to git, and hit a design decision with Tools/scripts/rmport. In the current SVN world it checks out a sparse copy of the tree to work on (just LEGAL, MOVED, category/Makefile and the port directory itself) and commits from that copy. This has the advantage of not clobbering an existing work tree and indeed does not need a pre-existing checkout at all. I could try to replicate this with git, but it seems like partial checkouts (not to be confused with shallow checkouts) are a bit of an afterthought [1] and need reasonably recent versions of the git binary and protocol. An alternative would be to just create a temporary branch on an existing checkout, say rmport-$USER-$EPOCH, do the removal work there, merge that branch to main, remove the temporary branch and push the new main. This feels less cryptic to me. It might also allow for better handling of merge conflicts with MOVED in case that file gets updated by someone else and you are in the middle of a big removal (say Python 2.7). Currently such conflicts crash the script (no work is lost because the temporary checkout is left alone in that case). What do you think? It looks like we can use a similar solution to the addport and perhaps mfh scripts. [1] https://unix.stackexchange.com/questions/233327/is-it-possible-to-clone-only-part-of-a-git-project René From owner-freebsd-git@freebsd.org Sun Nov 29 18:26:29 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 013A54A9CDC for ; Sun, 29 Nov 2020 18:26:29 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4CkcJN40ZWz3GYJ for ; Sun, 29 Nov 2020 18:26:28 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mailman.nyi.freebsd.org (Postfix) id 876F54A9CDB; Sun, 29 Nov 2020 18:26:28 +0000 (UTC) Delivered-To: git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 872CA4A9EA6 for ; Sun, 29 Nov 2020 18:26:28 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com [IPv6:2607:f8b0:4864:20::732]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CkcJN3BjQz3Gf9 for ; Sun, 29 Nov 2020 18:26:28 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x732.google.com with SMTP id h20so8962374qkk.4 for ; Sun, 29 Nov 2020 10:26:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=yDOYbG0YzYI5N44FSVQ4mALSzJp5FQ416wM3B0J1TKE=; b=cMQSho68dDKyZu6dehV4myKo+5XNh8Xx35hNuSvyE23RgfBjJj8WCLxA9s0F7Tm55U nO3lz7eIoYLjycvoF9XU/5v3FeWlekvNWr7Khf+/4GUSD36UuHwdYfVGb88tgcGZttMa IYbF4Xj6EQ7mlviVDJtLKTBqOKiumSi06K48IvE4NKcZkNYxDjMlFMfflU3BBrUCvLJx b86RisvrhKt1HO1YFLghtEWFVCfdfhFYh8syjQxVptbZa00oYr70GwDtjpdRkr87Vmzm SaV9iUT0A3lH54YlxJY4lbUfbyPiBMvDYlaJ+mu7C0Z579SHjPpXVmpqseqCLRE/9euW sbxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=yDOYbG0YzYI5N44FSVQ4mALSzJp5FQ416wM3B0J1TKE=; b=Be+sL16aHE7UuKOSn7YQKw7+nlBn9H/ddpwwlQKpS8T4aBDwjPcLz24M9v4VOBHF8p rSm+gXT8K231Uucu2Mv/+ImB+l8/JZ2Eeei0qaV+N+WuQnfDu698o0QKJaST/WAgq3WW xKNVQ7RlkdOX3PCz9YOCwz24ayeNfIl20W3nXML5gEgtFS4DWJDLZ9b+mF3sgJh2ZRde 7nvezvVWBNzw8OLcyfZ0q4DYlGCxNOJ1agO2N4nhqAarLLNJwHpfJhx4TbgEgy7RSOOw kMOmahI4BNz0WcWvCeSXVGRG7UKCWfTD0SeWQSGe47acRVxpbva2jgNoqyPHRKK9Yy+4 BzRA== X-Gm-Message-State: AOAM531wbdgWtOELvXB1KnoqbRcOAzYdLVm8/uZkl7F6sFi3KVeyNZ18 FAiF2pcJipf+7lHRXna5iDhC8nvfbGNBYjAnRskpBA== X-Google-Smtp-Source: ABdhPJwP/Fq3iRWlV2TMNKaJvj21LNeyRqyC1u9FJT8V0E0s+gg5FBPzBd5IjRqglJANWPdBk8cRbRDg+Gc74UFaYtk= X-Received: by 2002:ae9:f311:: with SMTP id p17mr19305689qkg.206.1606674387435; Sun, 29 Nov 2020 10:26:27 -0800 (PST) MIME-Version: 1.0 References: <20201129164707.GA31739@freefall.freebsd.org> In-Reply-To: <20201129164707.GA31739@freefall.freebsd.org> From: Warner Losh Date: Sun, 29 Nov 2020 11:26:16 -0700 Message-ID: Subject: Re: converting rmport to git To: Rene Ladan Cc: Chris Rees , git@freebsd.org, Ports Management Team X-Rspamd-Queue-Id: 4CkcJN3BjQz3Gf9 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Nov 2020 18:26:29 -0000 On Sun, Nov 29, 2020 at 9:47 AM Rene Ladan wrote: > Hello, > > I started converting some scripts in the port tree to git, and hit a design > decision with Tools/scripts/rmport. > Thanks Rene! > In the current SVN world it checks out a sparse copy of the tree to work on > (just LEGAL, MOVED, category/Makefile and the port directory itself) and > commits from that copy. This has the advantage of not clobbering an > existing > work tree and indeed does not need a pre-existing checkout at all. > There's two approaches for this in the git world. First would be worktrees. These are checkouts of the repo on different branches. It's not sparse, but it's not terrible all things considered. It would most closely replicate what you're trying to do, though at the cost of disk space. There's ways to do partial checkouts, but I've not had good luck with them. The second is to assume everybody is using branches in their main tree and can move to a new temporary branch at any time (eg have the tool do nothing in a dirty tree). > I could try to replicate this with git, but it seems like partial checkouts > (not to be confused with shallow checkouts) are a bit of an afterthought > [1] > and need reasonably recent versions of the git binary and protocol. An > alternative would be to just create a temporary branch on an existing > checkout, say rmport-$USER-$EPOCH, do the removal work there, merge that > branch to main, remove the temporary branch and push the new main. This > feels > less cryptic to me. It might also allow for better handling of merge > conflicts > with MOVED in case that file gets updated by someone else and you are in > the > middle of a big removal (say Python 2.7). Currently such conflicts crash > the > script (no work is lost because the temporary checkout is left alone in > that > case). > That's not a bad idea. branches are cheap in git. they let you fully test that you've not broken things, if you want, and can be rebased if there's a mid-air collision while all this is in progress... > What do you think? It looks like we can use a similar solution to the > addport > and perhaps mfh scripts. > > [1] > https://unix.stackexchange.com/questions/233327/is-it-possible-to-clone-only-part-of-a-git-project I think we'll want to have MFC/MFH scripts. One of the open items is the exact form these will take so we can take an accurate census of commits that could be merged. There's issues all over the place here that become less troublesome if the MFC/MFH commit message is decorated in a standardized way (or if standardized notes are used). It's unclear what to do here and I suspect we'll have at least a short period of chaos on the MFC front when viewed in the rear view mirror of whatever grows up to take its place. git can do some of this today w/o help, but only if we cherry-pick w/o modification. And that's unreasonable: we have to tweak MFCs all the time, and there's a proud history in at least src of collapsing several MFCs. I'm afraid I've not looked at the ports world to know if these things are the same or not, but I suspect at least an occasional tweak is needed for MFHs. tl;dr: branch based solutions are likely the ticket here. Warner From owner-freebsd-git@freebsd.org Mon Nov 30 10:53:37 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 763B34773D2 for ; Mon, 30 Nov 2020 10:53:37 +0000 (UTC) (envelope-from rene@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4Cl2CP2cJ9z4vLS for ; Mon, 30 Nov 2020 10:53:37 +0000 (UTC) (envelope-from rene@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 57A914774C4; Mon, 30 Nov 2020 10:53:37 +0000 (UTC) Delivered-To: git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 577124774C2 for ; Mon, 30 Nov 2020 10:53:37 +0000 (UTC) (envelope-from rene@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Cl2CP253yz4vQl; Mon, 30 Nov 2020 10:53:37 +0000 (UTC) (envelope-from rene@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1606733617; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sQS8MqfC18KT9fiBrhYBa+6hapwy4D48TYL2V/j7Y2Q=; b=M1um4QT9a0tddsiqIvBpP/Z1vS39/gC9Lmz5Vhy3duCVIbRO4fytvwYZnOc1jTa82AVKUO Bbw8sp2Zs7UV0YouwA/8FjGdNnwc3lcWrmVDWgoZhsmxn9ho2BQSvn3UA4W7EBCNHjhwJq /pdzoNsDVxj7qBWpwdst7TXH/HIcA9ehs20BQZZi6NlJeIZrkqcsCfy+SseZRMCjgE5lIL pctCHpUREYSNCwQQSY6vcHNLX3POgEGMtPuxFXn1dw1ILF6fftMXfh1pZaru9Otdw75AYV WZHg6e3FknKXmmiCbg0n3gjWVs7XO5WjfpgD5Fp1ETVWB249/9xbGbKhIqYEiA== Received: by freefall.freebsd.org (Postfix, from userid 1185) id 39C8E15AB9; Mon, 30 Nov 2020 10:53:37 +0000 (UTC) Date: Mon, 30 Nov 2020 10:53:37 +0000 From: Rene Ladan To: Chris Rees Cc: Adam Weinberger , git@freebsd.org, "portmgr@FreeBSD.org" Subject: Re: converting rmport to git Message-ID: <20201130105337.GA42359@freefall.freebsd.org> References: <20201129164707.GA31739@freefall.freebsd.org> <14871125-A032-4980-8DB1-0210E34D5A11@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <14871125-A032-4980-8DB1-0210E34D5A11@FreeBSD.org> ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1606733617; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sQS8MqfC18KT9fiBrhYBa+6hapwy4D48TYL2V/j7Y2Q=; b=YhJLkdFu0fnv9u+VyFvqcJwSpUENI9ng8Sdpz+aKDvJ+kIhtfJnPXHL8XjTpXaJle5WsrR V0C/WetOucsPRH5+jcr8ChcT047wU+p2f93wThoDgQE6K6hvCg7mCMwch1gR4zNT/YqqId LlLXafQPvRH71mee21ri1bOmdWeun6JblmqXrdflscFuh/K6bRMB6M/wmVjdZJOBujnG2x uq7OHh9B1A9NxilHpcLH/7tdFR6A7NPI4Gi8/YbK3fpK1lsc1hAnDtsXpmCnqAyrjv+19c KTlGugTMb4X7fF7FHxJo2omJFc7CZH354FjRda6uFzJ0lBeU8izbU9On90sFkg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1606733617; a=rsa-sha256; cv=none; b=SVvvT43nKckYVr6OJ1UPxFsIDAdaDOkSmQ7qjkAAtM6GSnBkaV1QjskFz7HOPAl3Yje5lo 0qr7XdyI8pozy/wyZWyLP/fqT49e/wUaXh/Osx9CjKZ+2pdTKiaZ4CJiSEofso+Cc7vM8x ZXBYbIhuoJaYSqYMA8hNSDQ8CrWVdb5N2G5fx2TwN6/hZW6yBabeUDGVbYVzEpdcVuLHjS MGON8VoCAeGczCn2/GoXgAiok8MaSpN89XRHbDJdaiK87CP8ODOIkCAUjlFpqmxAZMRlNn bKmZFOSpOVeEwoX/8OPw0P/ebVRpDPtN7WrB7siNdPUXsmcuMVGqzMhr5ZfdrQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Nov 2020 10:53:37 -0000 On Sun, Nov 29, 2020 at 11:26:41PM +0000, Chris Rees wrote: > Hey, > > On 29 November 2020 17:12:21 GMT, Adam Weinberger wrote: > >On Sun, Nov 29, 2020 at 9:47 AM Rene Ladan wrote: > >> > >> Hello, > >> > >> I started converting some scripts in the port tree to git, and hit a > >design > >> decision with Tools/scripts/rmport. > >> > >> In the current SVN world it checks out a sparse copy of the tree to > >work on > >> (just LEGAL, MOVED, category/Makefile and the port directory itself) > >and > >> commits from that copy. This has the advantage of not clobbering an > >existing > >> work tree and indeed does not need a pre-existing checkout at all. > > > >Git uses topic branches for not-clobbering-the-tree purposes, so your > >proposal below seems like the git analogue of an svn sparse tree. > > > >> I could try to replicate this with git, but it seems like partial > >checkouts > >> (not to be confused with shallow checkouts) are a bit of an > >afterthought [1] > >> and need reasonably recent versions of the git binary and protocol. > >An > > > >If our users will have to install git from ports or pkg, is there much > >concern that people won't be on a reasonably recent version of git? > > Nah, we can just check the version and complain easily. > > >> alternative would be to just create a temporary branch on an existing > >> checkout, say rmport-$USER-$EPOCH, do the removal work there, merge > >that > >> branch to main, remove the temporary branch and push the new main. > >This feels > > > >That definitely seems more consistent with the git workflow. Create a > >topic branch, do the thing, merge the branch, push the change. > > > Looks like a great idea. I'm starting to get used to git, but I'm happy if you (René) want to push ahead with this, as I'm sure my approach would be dreadful.... > OK, I'll rewrite the script to use the branch workflow. This does assume having an existing clone of the git repository present, but I guess that is a safe assumption if you are a ports committer :) The addport and mfh scripts could use a similar workflow. Speaking of addport, it uses your website to check if a port pre-existed, that website should learn about git too? Regards, René From owner-freebsd-git@freebsd.org Mon Nov 30 11:57:47 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A20334790BA for ; Mon, 30 Nov 2020 11:57:47 +0000 (UTC) (envelope-from mat@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4Cl3dR43w4z3GjQ for ; Mon, 30 Nov 2020 11:57:47 +0000 (UTC) (envelope-from mat@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 8B63747907C; Mon, 30 Nov 2020 11:57:47 +0000 (UTC) Delivered-To: git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8B288478E3B for ; Mon, 30 Nov 2020 11:57:47 +0000 (UTC) (envelope-from mat@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Cl3dR3R8pz3Gcx; Mon, 30 Nov 2020 11:57:47 +0000 (UTC) (envelope-from mat@freebsd.org) Received: from mail.j.mat.cc (owncloud.cube.mat.cc [79.143.240.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.mat.cc", Issuer "Let's Encrypt Authority X3" (verified OK)) (Authenticated sender: mat/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4CB192DA3C; Mon, 30 Nov 2020 11:57:47 +0000 (UTC) (envelope-from mat@freebsd.org) Received: from aching.in.mat.cc (unknown [IPv6:2a01:678:42:0:e187:c79:7202:4467]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: mat@mat.cc) by mail.j.mat.cc (Postfix) with ESMTPSA id 3CF1E942D84; Mon, 30 Nov 2020 11:57:45 +0000 (UTC) Date: Mon, 30 Nov 2020 12:57:43 +0100 From: Mathieu Arnold To: Rene Ladan Cc: Chris Rees , Adam Weinberger , git@freebsd.org, "portmgr@FreeBSD.org" Subject: Re: converting rmport to git Message-ID: <20201130115743.elfr6y2anpuoh5gy@aching.in.mat.cc> References: <20201129164707.GA31739@freefall.freebsd.org> <14871125-A032-4980-8DB1-0210E34D5A11@FreeBSD.org> <20201130105337.GA42359@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="btwk5hfwtnk4m6op" Content-Disposition: inline In-Reply-To: <20201130105337.GA42359@freefall.freebsd.org> X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Nov 2020 11:57:47 -0000 --btwk5hfwtnk4m6op Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Nov 30, 2020 at 10:53:37AM +0000, Rene Ladan wrote: > OK, I'll rewrite the script to use the branch workflow. This does assume > having an existing clone of the git repository present, but I guess that > is a safe assumption if you are a ports committer :) You really should not to worry about branches, the script could emit a warning if it runs on the default branch, which will be something like "main" but the script should not worry about all that, it should just assume it has the checkout of the thing you need to remove ports from, let the user worry about working on some branch or not. --=20 Mathieu Arnold --btwk5hfwtnk4m6op Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAABCgB9FiEEVhwchfRfuV0unqO5KesJApEdfgIFAl/E3jNfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDU2 MUMxQzg1RjQ1RkI5NUQyRTlFQTNCOTI5RUIwOTAyOTExRDdFMDIACgkQKesJApEd fgKjVQ//SItt+oD3wajPAaAU3pd2Pxsud0CA62RYoXKzDhGk8Ig2YM3Fa33c+FKx /cuCQEHwsaPnHFbODx7mZ739OHhF+6dF/O01i1LVT0Xo0eisCUKQKRc79VDJs6DW ukYDV0KEPUR9SlhGY+gtnrAYCozppq5rA7GOGYasprdhENZIjSBsX177CNI+nvoM W+KKop2rS5flJY5iRAru8G+IXxUa34h4K94LFuY8Bo6QJHFP1p0JYUDmrJzWN5JR twhOE4J6smF7jzUHy8HrYA9EBBQpF2WGXTdRHXZH2UHWfP9RakEEDkctP2C2buLE 1Inok2Qp0I7FkDXGDeYwei3gTzrZIyRTcnT5h9WisVJr43WGyvpc1sxG1bAdic/E Q0rjaJg11q+Ml6VfhuXQD/QhDz5eP1bY4cXE986cWYFb3a1pRUJoa+kxMeQz+K+r Yb+mPoifFG1mkuf+PHRMfwG+Kdfn8cMHhqcclfe2qkjsj9J550Id460mHsOZWmbc UOcyouxD6VS+XB9iCCkidup8hGiYLAHvLgaeDIs83wwSRHl5eMkH6NfwssY+cNLx NkaV1HQQSMcttd8FXzVZj1TnVu8qE/dmuvdBD2wG+V38k1y0klfBD7NTChG3hjUe V4mxht8K1eI/6we9sFWMJ5AEPwGIZuCyFn1L/sr6A15qYdthld8= =5k9C -----END PGP SIGNATURE----- --btwk5hfwtnk4m6op-- From owner-freebsd-git@freebsd.org Mon Nov 30 15:06:53 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6CCFB47F061 for ; Mon, 30 Nov 2020 15:06:53 +0000 (UTC) (envelope-from david@isnic.is) Received: from mx01.isnic.is (mx01.isnic.is [IPv6:2001:67c:6c:58::133]) by mx1.freebsd.org (Postfix) with ESMTP id 4Cl7qc3Tn9z3mTb; Mon, 30 Nov 2020 15:06:52 +0000 (UTC) (envelope-from david@isnic.is) Received: from localhost (wg-client01.isnic.is [185.93.159.98]) by mx01.isnic.is (Postfix) with ESMTPS id 48C1E1F56B; Mon, 30 Nov 2020 15:06:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=isnic.is; s=20200921; t=1606748803; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=LoY6ZMLKMA6gR8nmr7PaSI2meQYvLr9ZmphP8gtktvE=; b=yVNd7NtZtCWFTvpnQ+ie81yNT+RVKglNZWbJy8C4+xzb8xOlSyEEzeXfzMMsTY6rS/L0qE o3fXmXbkDua0794IIEgn3Lw3BMoH8g5RzKpfVIAE748GS/BRgHJmqo43/EOv45M9LizDbJ MPqi1bb5mFDV5RDo6uME+uaxWVc4QChU4170i+34Apu/N/k78XE3aVjXgW+PBrMX7jmmef kCDfBbo+jciwaRA2X8k1a44MHRvtFbiUtxRQTmog5REK+u1e28CghxyVgA2x4eeiuRx8Ri mlvF1zAI/s2wZDyAnvKsvbQDJPDUu+Ylu3j4G0e/Y7ti8T2Ka/HsXEmdqrgMWA== Date: Mon, 30 Nov 2020 15:06:42 +0000 From: =?iso-8859-1?B?RGF27fA=?= Steinn Geirsson To: Li-Wen Hsu Cc: freebsd-git@freebsd.org Subject: Re: 504 errors from cgit-beta Message-ID: <20201130150642.GB6221@mail> References: <20201112155659.GQ913@mail> <20201113.032709.2108746957258946268.yasu@utahime.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="t0UkRYy7tHLRMCai" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Rspamd-Queue-Id: 4Cl7qc3Tn9z3mTb X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=isnic.is header.s=20200921 header.b=yVNd7NtZ; dmarc=none; spf=pass (mx1.freebsd.org: domain of david@isnic.is designates 2001:67c:6c:58::133 as permitted sender) smtp.mailfrom=david@isnic.is X-Spamd-Result: default: False [-3.89 / 15.00]; MID_RHS_NOT_FQDN(0.50)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:67c:6c:58::133:from]; R_DKIM_ALLOW(-0.20)[isnic.is:s=20200921]; FREEFALL_USER(0.00)[david]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2001:67c:6c::/48]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; DMARC_NA(0.00)[isnic.is]; ARC_NA(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; SPAMHAUS_ZRD(0.00)[2001:67c:6c:58::133:from:127.0.2.255]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DKIM_TRACE(0.00)[isnic.is:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-1.00)[-1.000]; SIGNED_PGP(-2.00)[]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; R_MIXED_CHARSET(1.11)[subject]; ASN(0.00)[asn:1850, ipnet:2001:67c:6c::/48, country:EU]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Nov 2020 15:06:53 -0000 --t0UkRYy7tHLRMCai Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Nov 13, 2020 at 05:33:12PM +0800, Li-Wen Hsu wrote: > On Fri, Nov 13, 2020 at 2:28 AM Yasuhiro KIMURA wrote: > > > > From: Dav=ED=F0 Steinn Geirsson > > Subject: 504 errors from cgit-beta > > Date: Thu, 12 Nov 2020 15:56:59 +0000 > > > > > We are getting frequent 504 errors when running `git fetch` against an > > > existing checkout of `ports.git` from https://cgit-beta.freebsd.org/p= orts.git: > > > > > > ``` > > > $ git fetch cgit-beta > > > error: RPC failed; HTTP 504 curl 22 The requested URL returned error:= 504 > > > fatal: the remote end hung up unexpectedly > > > ``` > > > > I experienced same error when accessing Emacs git remository with > > HTTPS. Following is bug report that I submitted to report the issue. > > > > https://savannah.nongnu.org/support/?110322 > > > > As you can see, site administrator fixed the issue by icreasing > > `fastcgi_read_timeout` and `proxy_read_timeout` parameters of > > nginx. Since cgit-beta also uses nginx this may also fix your > > error. In my case, however, access always failed and never > > succeeded. So cause may be different from the one of my case. >=20 > Thanks, I have checked this, indeed some requests' handlers don't have > a long enough timeout setting and I've relaxed them. Hope this solves > some people's issues. Please check it again, and if it still fails for > you, we might need to have more information to debug. This problem disappeared after your changes, but as of this weekend seems to be happening again: user@ssh:~/foo/ports$ git fetch -v cgit-beta POST git-upload-pack (gzip 3272 to 1703 bytes) POST git-upload-pack (gzip 2577 to 1354 bytes) error: RPC failed; HTTP 504 curl 22 The requested URL returned error: 504 G= ateway Time-out fatal: the remote end hung up unexpectedly Is it possible some web server config got overwritten during the last batch of changes? Thanks! -Dav=ED=F0 >=20 > Li-Wen > _______________________________________________ > freebsd-git@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-git > To unsubscribe, send any mail to "freebsd-git-unsubscribe@freebsd.org" --t0UkRYy7tHLRMCai Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAABCgAdFiEEvylfYbt7o3c60Grm/+HlKLuPmJoFAl/FCoIACgkQ/+HlKLuP mJpqsgf/ShByUmIQzC+S1DQKDQBXHhhwebLyXeJfTOUBhA9uVKlb/qaIw9G4egQL vaxyw+hmb6tv/BtwAXXWWL+LMSYRq+iwtWuAd8H1vL5Ur0sB2tMTKDmYKFGQZ0qN 21jF3eN8M4beYuwD00PH9UsOWiZ1Yl2vwXXCH/Z8OBdXZ2kZ+cZSX9BpylsxueLg Ef5C8HuaKfwBy4Pth8nB1GYXKhY2/AJe8KVu9QDMvT6ZReVRQ72ExEmLHE3V3Wbm eKx2IvmSeTrof0YFm5FrJYM62YCVoJspkBIjcaTpcT4yrR944pBhen8c8MdbKT+R vVxtejTSqJnSeouxjhx5QLIeF7Bphg== =Kprv -----END PGP SIGNATURE----- --t0UkRYy7tHLRMCai-- From owner-freebsd-git@freebsd.org Mon Nov 30 18:52:45 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 81D904A54DE for ; Mon, 30 Nov 2020 18:52:45 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4ClDrF1qDrz4XRh for ; Mon, 30 Nov 2020 18:52:45 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mailman.nyi.freebsd.org (Postfix) id 3E30D4A5339; Mon, 30 Nov 2020 18:52:45 +0000 (UTC) Delivered-To: git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3DCC94A53B0 for ; Mon, 30 Nov 2020 18:52:45 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x830.google.com (mail-qt1-x830.google.com [IPv6:2607:f8b0:4864:20::830]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ClDrF0nYLz4XLq for ; Mon, 30 Nov 2020 18:52:44 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x830.google.com with SMTP id k4so4284039qtj.10 for ; Mon, 30 Nov 2020 10:52:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Jz5p7i8hpbM2ybebcF+vVXplIje5rM3OWkLbRe7GECI=; b=kisoqMRT9Ob+pCOhNxG8Q0nOuNoq+wqdjGgL3w7PvckqOGQRhlx2HWe+kC1FK1eDLD FciD8yirrfRUmhTmYCOty37PRRbYsgIfhF+LcbhoGh++sM0mbgp6Mw9MnJPkbzaRSwpz 1mstVI2Hy4aDxs5LS6sG0wMG5j4P2tKe3yKVS0/Ke9ZCb79H6u3uu3cAruaZHzgIBIqI BeY+RqgI1D80YFC/40cB03BAARYM+gw1aDgHYIGgZFYyfNR0vPhsVuiBnrQSbTaDi89U LuXVRbWDzPvbdZFq0ujUNylZ0pM2SPrO+0eeKx+1ZSKnE9dXpJ+yWcPC1DT31Ym9BtQr ZNxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Jz5p7i8hpbM2ybebcF+vVXplIje5rM3OWkLbRe7GECI=; b=BV9vsBbb7mv1fX/tcNNC+4KdBUXTyzLtrBy/K/Ad1O4E6JhAHJE8npNIDl61dwBIkO lg0s/zay/4eoM8MGyJsCpzTj/u6xzQDx4iP4hBQsWJ2zQC1Q5b+LVe+IE5XXLbe++R7d zp4z3wKD/3A9wM29oV/oLYLXcjgVQLxhA8cqIK+rNG7XXxmCXt0HT9sg/SXmiGKxPo5Y vMRH7xiviO/m9QR80wF1r6qbX0p9uh29lEZpbksQL5e7WEwsnObRKCf4nw3fc6GydO+m e0xL7mDjB9PXbtKvqi0aU191stA29nzR4MhN+47YlDm9hZ1U6RzJcIGm1CmxaANVCe1+ GWwQ== X-Gm-Message-State: AOAM530H9smVz1K7QUW393836u9bY+kBZVlf/c5mi26XJkTLPvFlOO1F tFz/V8bPSmkLU1CFgTAznWhJ37eA0eCdmeLPCmyHEw== X-Google-Smtp-Source: ABdhPJzBCxyITl+KI0PHQokRavfgYJSFx6P+pQIecetwjn2BRM/ny+QluyTrUfQMiDYyklFwoIXTHxK6lbFPxH3KvLM= X-Received: by 2002:ac8:4802:: with SMTP id g2mr23191363qtq.235.1606762363826; Mon, 30 Nov 2020 10:52:43 -0800 (PST) MIME-Version: 1.0 References: <20201129164707.GA31739@freefall.freebsd.org> <14871125-A032-4980-8DB1-0210E34D5A11@FreeBSD.org> <20201130105337.GA42359@freefall.freebsd.org> <20201130115743.elfr6y2anpuoh5gy@aching.in.mat.cc> In-Reply-To: <20201130115743.elfr6y2anpuoh5gy@aching.in.mat.cc> From: Warner Losh Date: Mon, 30 Nov 2020 11:52:32 -0700 Message-ID: Subject: Re: converting rmport to git To: Mathieu Arnold Cc: Rene Ladan , Chris Rees , Adam Weinberger , git@freebsd.org, "portmgr@FreeBSD.org" X-Rspamd-Queue-Id: 4ClDrF0nYLz4XLq X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Nov 2020 18:52:45 -0000 On Mon, Nov 30, 2020 at 4:57 AM Mathieu Arnold wrote: > On Mon, Nov 30, 2020 at 10:53:37AM +0000, Rene Ladan wrote: > > OK, I'll rewrite the script to use the branch workflow. This does assume > > having an existing clone of the git repository present, but I guess that > > is a safe assumption if you are a ports committer :) > > You really should not to worry about branches, the script could emit a > warning if it runs on the default branch, which will be something like > "main" but the script should not worry about all that, it should just > assume it has the checkout of the thing you need to remove ports from, > let the user worry about working on some branch or not. > A clean tree is the only real requirement, and all you need to check for / assume. However, it would be nice to allow for the steps to create a branch and/or worktree, if desired. There are times you'd want to run this in a mode that just does it and then you push. And there's times you need to run more extensive testing / regression. Automating the branching dance would allow for that. Making it optional would address the likely more typical case of 'I just want to remove this port'. Warner From owner-freebsd-git@freebsd.org Mon Nov 30 21:24:52 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 776654A8C90 for ; Mon, 30 Nov 2020 21:24:52 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2a05:fc87:1:5::15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "www.spoerlein.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ClJCm23phz4k4M; Mon, 30 Nov 2020 21:24:51 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: from localhost (acme.spoerlein.net [IPv6:2a05:fc87:1:5:0:0:0:15]) by acme.spoerlein.net (8.16.1/8.15.2) with ESMTPS id 0AULOoeX000777 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 30 Nov 2020 22:24:50 +0100 (CET) (envelope-from uqs@freebsd.org) Date: Mon, 30 Nov 2020 22:24:50 +0100 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: =?utf-8?B?RGF2w63DsA==?= Steinn Geirsson Cc: Li-Wen Hsu , freebsd-git@freebsd.org Subject: Re: 504 errors from cgit-beta Message-ID: References: <20201112155659.GQ913@mail> <20201113.032709.2108746957258946268.yasu@utahime.org> <20201130150642.GB6221@mail> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20201130150642.GB6221@mail> User-Agent: Mutt/2.0.1 (2020-11-14) X-Rspamd-Queue-Id: 4ClJCm23phz4k4M X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Nov 2020 21:24:52 -0000 On Mon, 2020-11-30 at 15:06:42 +0000, Davíð Steinn Geirsson wrote: >On Fri, Nov 13, 2020 at 05:33:12PM +0800, Li-Wen Hsu wrote: >> On Fri, Nov 13, 2020 at 2:28 AM Yasuhiro KIMURA wrote: >> > >> > From: Davíð Steinn Geirsson >> > Subject: 504 errors from cgit-beta >> > Date: Thu, 12 Nov 2020 15:56:59 +0000 >> > >> > > We are getting frequent 504 errors when running `git fetch` against an >> > > existing checkout of `ports.git` from https://cgit-beta.freebsd.org/ports.git: >> > > >> > > ``` >> > > $ git fetch cgit-beta >> > > error: RPC failed; HTTP 504 curl 22 The requested URL returned error: 504 >> > > fatal: the remote end hung up unexpectedly >> > > ``` >> > >> > I experienced same error when accessing Emacs git remository with >> > HTTPS. Following is bug report that I submitted to report the issue. >> > >> > https://savannah.nongnu.org/support/?110322 >> > >> > As you can see, site administrator fixed the issue by icreasing >> > `fastcgi_read_timeout` and `proxy_read_timeout` parameters of >> > nginx. Since cgit-beta also uses nginx this may also fix your >> > error. In my case, however, access always failed and never >> > succeeded. So cause may be different from the one of my case. >> >> Thanks, I have checked this, indeed some requests' handlers don't have >> a long enough timeout setting and I've relaxed them. Hope this solves >> some people's issues. Please check it again, and if it still fails for >> you, we might need to have more information to debug. > >This problem disappeared after your changes, but as of this weekend seems >to be happening again: > >user@ssh:~/foo/ports$ git fetch -v cgit-beta >POST git-upload-pack (gzip 3272 to 1703 bytes) >POST git-upload-pack (gzip 2577 to 1354 bytes) >error: RPC failed; HTTP 504 curl 22 The requested URL returned error: 504 Gateway Time-out >fatal: the remote end hung up unexpectedly > >Is it possible some web server config got overwritten during the last >batch of changes? This is most definitely fallout from the commit hashes changing. That means your client will upload basically all the hashes or packs for the server to compare what it does and does not have. What is your up/downstream bandwidth situation like? Could you try some more tracing as outlined here: https://stackoverflow.com/questions/27442134/git-fetch-hangs-on-git-upload-pack What sort of custom work do you have in there (branches, etc)? I'm curious to find out a way to reset this non-destructively ... and I have an idea. Li-Wen, we need to further up the timeouts here, freebsd repos are big, after all. Cheers Uli From owner-freebsd-git@freebsd.org Tue Dec 1 09:52:24 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D6B5B4A3DD9 for ; Tue, 1 Dec 2020 09:52:24 +0000 (UTC) (envelope-from david@isnic.is) Received: from mx01.isnic.is (mx01.isnic.is [IPv6:2001:67c:6c:58::133]) by mx1.freebsd.org (Postfix) with ESMTP id 4ClcpJ0H8fz3Gyw; Tue, 1 Dec 2020 09:52:23 +0000 (UTC) (envelope-from david@isnic.is) Received: from localhost (wg-client01.isnic.is [185.93.159.98]) by mx01.isnic.is (Postfix) with ESMTPS id A36FB20186; Tue, 1 Dec 2020 09:52:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=isnic.is; s=20200921; t=1606816340; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=zl84DIC00ohmo+ZOe1X9+9ehv2kE2VwjiWr2lQcw5bw=; b=vq2x1GDtHMKRjjU9VrqOC24L0ZHTmHhtzZbcNElzxHe7iK6k8MvlfUKIhm+u0kV/8FGXu1 MMkzsAFtm/0oFohkpOgPtGpNbPdz+RYdtkvn71C0ErsKjmRn/FgN6oDAK4N2lXODKVQNro 2nHgaKPSE8TlOzJCWxwepWHQq5LtyE1Kv1ODb/r0ZplEUCC4L1S77YsfefzbgBsBLUMU1Z q8orsrpzh+/bTs1qJw/eZF44VS7NyLKUj3Lp3PZzpB8XNQ8EdyQaUhhNLYGVwiviA+ktFU XIPfqjatYJKVgo33CTXZ9Rpr5Zp9Tafo0lMCHYE2rwKsHxjXPnsm6xBHe8YRSg== Date: Tue, 1 Dec 2020 09:52:18 +0000 From: =?iso-8859-1?B?RGF27fA=?= Steinn Geirsson To: Ulrich =?iso-8859-1?Q?Sp=F6rlein?= Cc: Li-Wen Hsu , freebsd-git@freebsd.org Subject: Re: 504 errors from cgit-beta Message-ID: <20201201095218.GC6221@mail> References: <20201112155659.GQ913@mail> <20201113.032709.2108746957258946268.yasu@utahime.org> <20201130150642.GB6221@mail> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="sXc4Kmr5FA7axrvy" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Rspamd-Queue-Id: 4ClcpJ0H8fz3Gyw X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=isnic.is header.s=20200921 header.b=vq2x1GDt; dmarc=none; spf=pass (mx1.freebsd.org: domain of david@isnic.is designates 2001:67c:6c:58::133 as permitted sender) smtp.mailfrom=david@isnic.is X-Spamd-Result: default: False [-3.87 / 15.00]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:67c:6c:58::133:from]; R_DKIM_ALLOW(-0.20)[isnic.is:s=20200921]; FREEFALL_USER(0.00)[david]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2001:67c:6c::/48:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_NA(0.00)[isnic.is]; TO_DN_SOME(0.00)[]; SPAMHAUS_ZRD(0.00)[2001:67c:6c:58::133:from:127.0.2.255]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DKIM_TRACE(0.00)[isnic.is:+]; NEURAL_HAM_SHORT(-0.98)[-0.977]; NEURAL_HAM_LONG(-1.00)[-1.000]; SIGNED_PGP(-2.00)[]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; R_MIXED_CHARSET(1.11)[subject]; ASN(0.00)[asn:1850, ipnet:2001:67c:6c::/48, country:EU]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Dec 2020 09:52:24 -0000 --sXc4Kmr5FA7axrvy Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Nov 30, 2020 at 10:24:50PM +0100, Ulrich Sp=F6rlein wrote: > On Mon, 2020-11-30 at 15:06:42 +0000, Dav=ED=F0 Steinn Geirsson wrote: > > On Fri, Nov 13, 2020 at 05:33:12PM +0800, Li-Wen Hsu wrote: > > > On Fri, Nov 13, 2020 at 2:28 AM Yasuhiro KIMURA wr= ote: > > > > > > > > From: Dav=ED=F0 Steinn Geirsson > > > > Subject: 504 errors from cgit-beta > > > > Date: Thu, 12 Nov 2020 15:56:59 +0000 > > > > > > > > > We are getting frequent 504 errors when running `git fetch` again= st an > > > > > existing checkout of `ports.git` from https://cgit-beta.freebsd.o= rg/ports.git: > > > > > > > > > > ``` > > > > > $ git fetch cgit-beta > > > > > error: RPC failed; HTTP 504 curl 22 The requested URL returned er= ror: 504 > > > > > fatal: the remote end hung up unexpectedly > > > > > ``` > > > > > > > > I experienced same error when accessing Emacs git remository with > > > > HTTPS. Following is bug report that I submitted to report the issue. > > > > > > > > https://savannah.nongnu.org/support/?110322 > > > > > > > > As you can see, site administrator fixed the issue by icreasing > > > > `fastcgi_read_timeout` and `proxy_read_timeout` parameters of > > > > nginx. Since cgit-beta also uses nginx this may also fix your > > > > error. In my case, however, access always failed and never > > > > succeeded. So cause may be different from the one of my case. > > >=20 > > > Thanks, I have checked this, indeed some requests' handlers don't have > > > a long enough timeout setting and I've relaxed them. Hope this solves > > > some people's issues. Please check it again, and if it still fails for > > > you, we might need to have more information to debug. > >=20 > > This problem disappeared after your changes, but as of this weekend see= ms > > to be happening again: > >=20 > > user@ssh:~/foo/ports$ git fetch -v cgit-beta > > POST git-upload-pack (gzip 3272 to 1703 bytes) > > POST git-upload-pack (gzip 2577 to 1354 bytes) > > error: RPC failed; HTTP 504 curl 22 The requested URL returned error: 5= 04 Gateway Time-out > > fatal: the remote end hung up unexpectedly > >=20 > > Is it possible some web server config got overwritten during the last > > batch of changes? >=20 > This is most definitely fallout from the commit hashes changing. That mea= ns > your client will upload basically all the hashes or packs for the server = to > compare what it does and does not have. >=20 > What is your up/downstream bandwidth situation like? Could you try some m= ore > tracing as outlined here: https://stackoverflow.com/questions/27442134/gi= t-fetch-hangs-on-git-upload-pack > What sort of custom work do you have in there (branches, etc)? I'm curious > to find out a way to reset this non-destructively ... and I have an idea. Up/downstream should be good. Speedtests show ~100-160Mbit/s in both directions. Cloning a repo from cgit-beta.freebsd.org I see 7.75 MiB/s. The checkout I was working from had two branches: `upstream` which is a 1:1 clone of the state of the `main` branch on cgit-beta, and `main` which is the same but also has a couple of local ports in commits that get rebased on top of `upstream` when it's updated. When this error occurred I was on the `upstream` branch. This was a manual test, but normally the same update-then-rebase process happens as part of a CI job which was also failing. It seems this is fixed now, as the last 4 runs of the CI job were successful (first successful run was at 18:42 UTC). If I see a similar error again I'll follow the linked steps and send a more detailed trace. Thanks for your assistance, -Dav=ED=F0 >=20 > Li-Wen, we need to further up the timeouts here, freebsd repos are big, > after all. >=20 > Cheers > Uli --sXc4Kmr5FA7axrvy Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAABCgAdFiEEvylfYbt7o3c60Grm/+HlKLuPmJoFAl/GElIACgkQ/+HlKLuP mJoKvQf/XmzIXD2KQeqEmOuwpyWYvmzKzZGuGgcarqVWBbbvWgtbQ01nTDZ6d9PF 6Fzf737iyiXW01awpcC5BUdZtsJT7ItLYqHSnJ1O2lQFZZkUKwxhlA231yZIR3wn 2ZzRozkAAlBKyGqpV+QZ8U7296eUU9qZU1ugGcgiVqCZvFhiVbaFMxJZE3J5fqfN fSl22R/pb0lJmjb4Wzl5U9gYrDz/zeuuRifNq5xB4abpnS4TpWyHgQU1u/jMl0/p OjWkc9kYCTjwYaacsVXX8Z5adtYcag3ZxdIRmwXw77Vu5LO6jFBWkqUdWMJ6FnqQ M4m9yBClvxATx8GJNzbwV9knBm0tjw== =26C5 -----END PGP SIGNATURE----- --sXc4Kmr5FA7axrvy-- From owner-freebsd-git@freebsd.org Tue Dec 1 09:59:06 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BCA3C4A41FC for ; Tue, 1 Dec 2020 09:59:06 +0000 (UTC) (envelope-from rene@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4Clcy24cHsz3HYf for ; Tue, 1 Dec 2020 09:59:06 +0000 (UTC) (envelope-from rene@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 9C44B4A41FB; Tue, 1 Dec 2020 09:59:06 +0000 (UTC) Delivered-To: git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9C0094A3F4E for ; Tue, 1 Dec 2020 09:59:06 +0000 (UTC) (envelope-from rene@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Clcy22zldz3HbJ; Tue, 1 Dec 2020 09:59:06 +0000 (UTC) (envelope-from rene@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1606816746; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XRuvhQ0Gw+vyugy35AeMjCKBWW/RkyH+mncZ8HUG2oM=; b=mZWeeFZcE/fNPXExAwbmaI47wqeB7ryHFn9chtnYJ37I1nigjsobZu4ghsi36mI8+T9pwe TnUIfYbpGp7xxTMn1zFaKcIxIUD9+akCb68rFJQk6GCfU/+16ZQu+TDE1cAKmHtrSScoU5 UyX+aKUlcWdn00rbhtKqsLz5vHgGJXo1rAO0feEtDYHlHM6jwdtiLdMnxjRS6VwcQCpTjk wqf4ajpAaj1cD+JB+fSSxSyZGMpchhEq+HFDQO+HAztfet1x6cRggaIA4bcn5HZa1l19jW a9fiXzgJVcGEXAE5LXeajZVUrn1+2SynsSnBNHICc5gRMGYqGoXOuTpWsXg3tw== Received: by freefall.freebsd.org (Postfix, from userid 1185) id 4DFA9D602; Tue, 1 Dec 2020 09:59:06 +0000 (UTC) Date: Tue, 1 Dec 2020 09:59:06 +0000 From: Rene Ladan To: Chris Rees Cc: Adam Weinberger , git@freebsd.org, "portmgr@FreeBSD.org" Subject: Re: converting rmport to git Message-ID: <20201201095906.GA50345@freefall.freebsd.org> References: <20201129164707.GA31739@freefall.freebsd.org> <14871125-A032-4980-8DB1-0210E34D5A11@FreeBSD.org> <20201130105337.GA42359@freefall.freebsd.org> <7246FB00-655B-4BD4-BC99-B87E4595969C@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <7246FB00-655B-4BD4-BC99-B87E4595969C@FreeBSD.org> ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1606816746; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XRuvhQ0Gw+vyugy35AeMjCKBWW/RkyH+mncZ8HUG2oM=; b=csQt5LGCHamg0+p5AX9bFj1FEURvl8G9AdPA6BFXaokdFzhMhd32gQVC+iwUm/GTKJ9Fsa BqjyaAUZ5Tre9hyAUmPbkvKEntjjQQ+ijjyqbGR21HrO7iXdfyb2oracgvMzr0ELXsnkYg eAfnpsLAqLyGPIZwqS+5VqW2uzvXGJI2AJL3JtskYHWRZ++tsiRgEwi28H6rZRWBiNHv5F xhPtXNIjG2YxnXuqlWzD+aPGdRP5nUFsmfmuReH4HeO4vNMkmg+5keomZSGiYlyyBLKoX4 TJ/OFoTgdweWi2qWNTRy9RqIZVURf08sut5xmgdrlrAYzTIr0wE9XuYRql7Vmw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1606816746; a=rsa-sha256; cv=none; b=KMlJa111BWj8X2B34IVoQuq5sdZI1wp9CPOHfAau0/BQduZ7jJb9fpSWOZ31itFNvNUsEf 2Swg70bjLnWvbRzfgUlXMGLXFH1O0B30du8f8weicqCRhvnoLWBu/H4+qJ0yHcGwMb+gQA hsdBzgpwfPOisCXSAdEoVKYaGXrvN2yz/TGNkV4vDuGDTHfUC7Mpu3u8Iwc5d1D6DsBs1n DhaRlIvmyyySHQYNRDTKOGTo2ypAO3nxCcuKpEnYRaVnYd9NPCHu4SdhS1/QDvloVnbGzJ i5/W/crYiGZ73J3cIOUdSt0+4AoKgPPgG20MWhZhoRDbzHnp8b/slpZ6YnNR0g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Dec 2020 09:59:06 -0000 On Mon, Nov 30, 2020 at 08:19:40PM +0000, Chris Rees wrote: > > > On 30 November 2020 10:53:37 GMT, Rene Ladan wrote: > >On Sun, Nov 29, 2020 at 11:26:41PM +0000, Chris Rees wrote: > >> Hey, > >> > >> On 29 November 2020 17:12:21 GMT, Adam Weinberger > >wrote: > >> >On Sun, Nov 29, 2020 at 9:47 AM Rene Ladan wrote: > >> >> > >> >> Hello, > >> >> > >> >> I started converting some scripts in the port tree to git, and hit > >a > >> >design > >> >> decision with Tools/scripts/rmport. > >> >> > >> >> In the current SVN world it checks out a sparse copy of the tree > >to > >> >work on > >> >> (just LEGAL, MOVED, category/Makefile and the port directory > >itself) > >> >and > >> >> commits from that copy. This has the advantage of not clobbering > >an > >> >existing > >> >> work tree and indeed does not need a pre-existing checkout at all. > >> > > >> >Git uses topic branches for not-clobbering-the-tree purposes, so > >your > >> >proposal below seems like the git analogue of an svn sparse tree. > >> > > >> >> I could try to replicate this with git, but it seems like partial > >> >checkouts > >> >> (not to be confused with shallow checkouts) are a bit of an > >> >afterthought [1] > >> >> and need reasonably recent versions of the git binary and > >protocol. > >> >An > >> > > >> >If our users will have to install git from ports or pkg, is there > >much > >> >concern that people won't be on a reasonably recent version of git? > >> > >> Nah, we can just check the version and complain easily. > >> > >> >> alternative would be to just create a temporary branch on an > >existing > >> >> checkout, say rmport-$USER-$EPOCH, do the removal work there, > >merge > >> >that > >> >> branch to main, remove the temporary branch and push the new main. > >> >This feels > >> > > >> >That definitely seems more consistent with the git workflow. Create > >a > >> >topic branch, do the thing, merge the branch, push the change. > >> > >> > >> Looks like a great idea. I'm starting to get used to git, but I'm > >happy if you (René) want to push ahead with this, as I'm sure my > >approach would be dreadful.... > >> > >OK, I'll rewrite the script to use the branch workflow. This does > >assume > >having an existing clone of the git repository present, but I guess > >that > >is a safe assumption if you are a ports committer :) > > > >The addport and mfh scripts could use a similar workflow. Speaking of > >addport, it uses your website to check if a port pre-existed, that > >website > >should learn about git too? > > Well... I don't think git actually does git cp properly does it? As in, doesn't it cleverly work it out for itself? Git only knows about mv, not cp :( > > The only reason I wrote the removed ports thing is because we were losing the link between old and new version when a port was removed and readded. Is git likely to DTRT without it? I don't know, perhaps we can search the git history itself. René From owner-freebsd-git@freebsd.org Tue Dec 1 15:17:25 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 126784AC106 for ; Tue, 1 Dec 2020 15:17:25 +0000 (UTC) (envelope-from uspoerlein@gmail.com) Received: from mail-oi1-f170.google.com (mail-oi1-f170.google.com [209.85.167.170]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Clm1J731nz3w7b; Tue, 1 Dec 2020 15:17:24 +0000 (UTC) (envelope-from uspoerlein@gmail.com) Received: by mail-oi1-f170.google.com with SMTP id v202so2069528oia.9; Tue, 01 Dec 2020 07:17:24 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=32EeU+GH8eiGN/d3fmV5Fe4QgzOXxrtdutM8Fr2mGkQ=; b=trV5f1UZyAfNHt9fqiYCikUqtpFPQ+vJG2l6egaFyKvJGT/ZlQVoCjYic5UUqvixyx LrBB/Jdev1DpaeIuksHJ3DeTE6BUnrjaoCqehHN8mbBiAI4m85FGgi1Zj9Hh0NppqT2/ zTfp3gF/MJDlN/dWDlZkUgmEkZhMVbST3fiQN1eNYVxAM+MQbLtuEX1sk6tD7jN9RZfq yrP1xokqNQfuI3GnLyKDmeLrCcUA3ov4ArPK8lLMF50+7xxUkbntxVd7Fo4tE0/jR34x 1IiOUN2WbhGZlMKyEJafyXbJJXBvIsiX2I1jxgn+oy6tC9pVT3IqlFRAtlWLFvAKpI7X cxUg== X-Gm-Message-State: AOAM533GdQYU76UACcrhxvKs52d8j/x/jTFz98M0aiGNH8gsToxGEfJx GrjqhbS//LlMc835XQAoKA7G5lf4oeqTQD6kJm6/8Hrw X-Google-Smtp-Source: ABdhPJyzZAmq5s3AVS/y/mhjaJQst1/ie8eAYWVoXidDa8DGINgHpQS0wEr4mJdJXqkI02Rj/JFJwhw32Oh8XBZIXlU= X-Received: by 2002:a54:4d8f:: with SMTP id y15mr2066219oix.150.1606835843435; Tue, 01 Dec 2020 07:17:23 -0800 (PST) MIME-Version: 1.0 References: <20201112155659.GQ913@mail> <20201113.032709.2108746957258946268.yasu@utahime.org> <20201130150642.GB6221@mail> <20201201095218.GC6221@mail> In-Reply-To: <20201201095218.GC6221@mail> From: =?UTF-8?Q?Ulrich_Sp=C3=B6rlein?= Date: Tue, 1 Dec 2020 16:17:12 +0100 Message-ID: Subject: Re: 504 errors from cgit-beta To: =?UTF-8?Q?Dav=C3=AD=C3=B0_Steinn_Geirsson?= Cc: Li-Wen Hsu , freebsd-git@freebsd.org X-Rspamd-Queue-Id: 4Clm1J731nz3w7b X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Dec 2020 15:17:25 -0000 On Tue, Dec 1, 2020 at 10:52 AM Dav=C3=AD=C3=B0 Steinn Geirsson wrote: > On Mon, Nov 30, 2020 at 10:24:50PM +0100, Ulrich Sp=C3=B6rlein wrote: > > On Mon, 2020-11-30 at 15:06:42 +0000, Dav=C3=AD=C3=B0 Steinn Geirsson w= rote: > > > On Fri, Nov 13, 2020 at 05:33:12PM +0800, Li-Wen Hsu wrote: > > > > On Fri, Nov 13, 2020 at 2:28 AM Yasuhiro KIMURA > wrote: > > > > > > > > > > From: Dav=C3=AD=C3=B0 Steinn Geirsson > > > > > Subject: 504 errors from cgit-beta > > > > > Date: Thu, 12 Nov 2020 15:56:59 +0000 > > > > > > > > > > > We are getting frequent 504 errors when running `git fetch` > against an > > > > > > existing checkout of `ports.git` from > https://cgit-beta.freebsd.org/ports.git: > > > > > > > > > > > > ``` > > > > > > $ git fetch cgit-beta > > > > > > error: RPC failed; HTTP 504 curl 22 The requested URL returned > error: 504 > > > > > > fatal: the remote end hung up unexpectedly > > > > > > ``` > > > > > > > > > > I experienced same error when accessing Emacs git remository with > > > > > HTTPS. Following is bug report that I submitted to report the > issue. > > > > > > > > > > https://savannah.nongnu.org/support/?110322 > > > > > > > > > > As you can see, site administrator fixed the issue by icreasing > > > > > `fastcgi_read_timeout` and `proxy_read_timeout` parameters of > > > > > nginx. Since cgit-beta also uses nginx this may also fix your > > > > > error. In my case, however, access always failed and never > > > > > succeeded. So cause may be different from the one of my case. > > > > > > > > Thanks, I have checked this, indeed some requests' handlers don't > have > > > > a long enough timeout setting and I've relaxed them. Hope this > solves > > > > some people's issues. Please check it again, and if it still fails > for > > > > you, we might need to have more information to debug. > > > > > > This problem disappeared after your changes, but as of this weekend > seems > > > to be happening again: > > > > > > user@ssh:~/foo/ports$ git fetch -v cgit-beta > > > POST git-upload-pack (gzip 3272 to 1703 bytes) > > > POST git-upload-pack (gzip 2577 to 1354 bytes) > > > error: RPC failed; HTTP 504 curl 22 The requested URL returned error: > 504 Gateway Time-out > > > fatal: the remote end hung up unexpectedly > > > > > > Is it possible some web server config got overwritten during the last > > > batch of changes? > > > > This is most definitely fallout from the commit hashes changing. That > means > > your client will upload basically all the hashes or packs for the serve= r > to > > compare what it does and does not have. > > > > What is your up/downstream bandwidth situation like? Could you try some > more > > tracing as outlined here: > https://stackoverflow.com/questions/27442134/git-fetch-hangs-on-git-uploa= d-pack > > What sort of custom work do you have in there (branches, etc)? I'm > curious > > to find out a way to reset this non-destructively ... and I have an ide= a. > > Up/downstream should be good. Speedtests show ~100-160Mbit/s in both > directions. Cloning a repo from cgit-beta.freebsd.org I see 7.75 MiB/s. > > The checkout I was working from had two branches: `upstream` which is > a 1:1 clone of the state of the `main` branch on cgit-beta, and `main` > which is the same but also has a couple of local ports in commits that > get rebased on top of `upstream` when it's updated. When this error > occurred I was on the `upstream` branch. > > This was a manual test, but normally the same update-then-rebase process > happens as part of a CI job which was also failing. > > It seems this is fixed now, as the last 4 runs of the CI job were > successful > (first successful run was at 18:42 UTC). If I see a similar error again > I'll > follow the linked steps and send a more detailed trace. > > I saw that we had tons of loose objects in src and doc (but not in ports) and I gc'ed them today around UTC 9:00 or so. Maybe ports did auto-gc yesterday? I have some fixes to how we push things into the repo that might (or might not) reduce the number of loose objects we end up with. I'm puzzled that doc of all places would result in loose objects. For src this is expected due to the elaborate re-writes I'm doing post-conversion. Hmmm From owner-freebsd-git@freebsd.org Tue Dec 1 16:37:00 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E8A804ADDBB for ; Tue, 1 Dec 2020 16:37:00 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4Clnn84zGXz4Vsq for ; Tue, 1 Dec 2020 16:37:00 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mailman.nyi.freebsd.org (Postfix) id AAA9B4ADF09; Tue, 1 Dec 2020 16:37:00 +0000 (UTC) Delivered-To: git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AA6B64ADAE0 for ; Tue, 1 Dec 2020 16:37:00 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x830.google.com (mail-qt1-x830.google.com [IPv6:2607:f8b0:4864:20::830]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Clnn84F9nz4WF1 for ; Tue, 1 Dec 2020 16:37:00 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x830.google.com with SMTP id r6so1133803qtm.3 for ; Tue, 01 Dec 2020 08:37:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=sR3Wn9B3Q4KzrnCHaaHFtkv/zTGW6nMKNNy5AK4nJug=; b=aZUtZbyAqE3DLD6NkPzfkZQqoHLLU4x8ZJUbGsi52mGtcNqnUEosINojUYw5oKuQAG ac0IfYFhLEytF3ZfgB3WviYbHpuCingyyYfDLqSkTG6OxcGVtykW7Fxe9T24Py76TThn EfdgUe2MWbGE1FQBS8X4Fpfri00woesQnsJP/Zro2ku35+gKgKP5Q5cblsqYaExx1jXA iadVvwP4xKLNdnFHl8C4+QYQm2NTcF1PL2+xCjQoiJTZo8SrX2jiOFfVDKKBxaX8F3Qm wl7d9cHTocCrYPUUu5rxdPty7fS5VM/hBly06Kqb2xtnocZu3d1QnHm552bx08XZESUQ kxIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=sR3Wn9B3Q4KzrnCHaaHFtkv/zTGW6nMKNNy5AK4nJug=; b=o7WJWXc9wR7hO3MhcNn8mFFKdE+bubGnTTfug2fmlD0FOGkI5aKLAn31kGRC+tp2Mm VWhrKPxJhuYLZDnzzEYz1gIvf0LtjoG2QPxXtn7yZWLhw08fkbLFVbbQW6Gu55DLlkum 9YkBfiNcXyRSGV9LsA104cT2Y+vdFOGXsjrbldVoFndSw2vSolB2BgbKIU8TA+aMvtZa i/w+YvbWcm2M7KgP5dFA1reyRMgpp2Y/szNmaricrzrGqEf6RraEVHswEoai4X6oWBIi MsBIefb5g6UBj1fd0vWLOwxau6w51+fOPvQajR1SOuqx3p4SL9BFhuufWGx3XWDweb32 ro7g== X-Gm-Message-State: AOAM530Y10oYffvOTKr+NHhG3sVwyNi/WgizLoRbkxQkFaV7tPWUeh+/ 3MQ5yecFCz6fsjzzNFRE3CaRLBIZ8q3uuNCfJhhx1A== X-Google-Smtp-Source: ABdhPJycUlVtpfmHoG/0nY6bm7Q+uRW8tQvAYJETDOQvwzGaKvO6iINYT/Hr3Adlk/6YJ2qWLWbR9AMnLGEk3DW+lZw= X-Received: by 2002:ac8:67da:: with SMTP id r26mr3614016qtp.101.1606840619192; Tue, 01 Dec 2020 08:36:59 -0800 (PST) MIME-Version: 1.0 References: <20201129164707.GA31739@freefall.freebsd.org> <14871125-A032-4980-8DB1-0210E34D5A11@FreeBSD.org> <20201130105337.GA42359@freefall.freebsd.org> <7246FB00-655B-4BD4-BC99-B87E4595969C@FreeBSD.org> <20201201095906.GA50345@freefall.freebsd.org> In-Reply-To: <20201201095906.GA50345@freefall.freebsd.org> From: Warner Losh Date: Tue, 1 Dec 2020 09:36:48 -0700 Message-ID: Subject: Re: converting rmport to git To: Rene Ladan Cc: Chris Rees , Adam Weinberger , git@freebsd.org, "portmgr@FreeBSD.org" X-Rspamd-Queue-Id: 4Clnn84F9nz4WF1 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Dec 2020 16:37:01 -0000 On Tue, Dec 1, 2020 at 2:59 AM Rene Ladan wrote: > On Mon, Nov 30, 2020 at 08:19:40PM +0000, Chris Rees wrote: > > > > > > On 30 November 2020 10:53:37 GMT, Rene Ladan wrote: > > >On Sun, Nov 29, 2020 at 11:26:41PM +0000, Chris Rees wrote: > > >> Hey, > > >> > > >> On 29 November 2020 17:12:21 GMT, Adam Weinberger > > >wrote: > > >> >On Sun, Nov 29, 2020 at 9:47 AM Rene Ladan wrote= : > > >> >> > > >> >> Hello, > > >> >> > > >> >> I started converting some scripts in the port tree to git, and hi= t > > >a > > >> >design > > >> >> decision with Tools/scripts/rmport. > > >> >> > > >> >> In the current SVN world it checks out a sparse copy of the tree > > >to > > >> >work on > > >> >> (just LEGAL, MOVED, category/Makefile and the port directory > > >itself) > > >> >and > > >> >> commits from that copy. This has the advantage of not clobbering > > >an > > >> >existing > > >> >> work tree and indeed does not need a pre-existing checkout at all= . > > >> > > > >> >Git uses topic branches for not-clobbering-the-tree purposes, so > > >your > > >> >proposal below seems like the git analogue of an svn sparse tree. > > >> > > > >> >> I could try to replicate this with git, but it seems like partial > > >> >checkouts > > >> >> (not to be confused with shallow checkouts) are a bit of an > > >> >afterthought [1] > > >> >> and need reasonably recent versions of the git binary and > > >protocol. > > >> >An > > >> > > > >> >If our users will have to install git from ports or pkg, is there > > >much > > >> >concern that people won't be on a reasonably recent version of git? > > >> > > >> Nah, we can just check the version and complain easily. > > >> > > >> >> alternative would be to just create a temporary branch on an > > >existing > > >> >> checkout, say rmport-$USER-$EPOCH, do the removal work there, > > >merge > > >> >that > > >> >> branch to main, remove the temporary branch and push the new main= . > > >> >This feels > > >> > > > >> >That definitely seems more consistent with the git workflow. Create > > >a > > >> >topic branch, do the thing, merge the branch, push the change. > > >> > > >> > > >> Looks like a great idea. I'm starting to get used to git, but I'm > > >happy if you (Ren=C3=A9) want to push ahead with this, as I'm sure my > > >approach would be dreadful.... > > >> > > >OK, I'll rewrite the script to use the branch workflow. This does > > >assume > > >having an existing clone of the git repository present, but I guess > > >that > > >is a safe assumption if you are a ports committer :) > > > > > >The addport and mfh scripts could use a similar workflow. Speaking of > > >addport, it uses your website to check if a port pre-existed, that > > >website > > >should learn about git too? > > > > Well... I don't think git actually does git cp properly does it? As in= , > doesn't it cleverly work it out for itself? > Git only knows about mv, not cp :( > I also think that git mv is just a convenience for git rm and git add.... > > > > The only reason I wrote the removed ports thing is because we were > losing the link between old and new version when a port was removed and > readded. Is git likely to DTRT without it? > I don't know, perhaps we can search the git history itself. > The typical way I've seen this done is to add the metadata to either the 'bring it back' commit (so like our MFC After: lines), or using the 'notes' feature in some way. The upside to either of these techniques is that you preserve the data. the downside is that neither is known by git log. I've also seen weird merge commits attempted for this since git log knows about them... The only problem is they are sufficiently weird that git log does odd things for everybody across them... I don't anticipate that git will do the right thing w/o help because the ports tree has so many nearly identical files. For simple ports, the Makefile could have been copied from anywhere, likewise some of the pkg* files. Though for simple ones, lost history doesn't lose much that's useful. For complex ones, it likely would do close to the right thing since complex ports, kinda by definition, are different. Though I'd test this notion (because as I'm typing this I find myself making too many mental reservations)... To be honest, though, I think this is an area where some experimentation to understand the alternatives is needed because this use case is relatively rare in the larger open source community. Warner From owner-freebsd-git@freebsd.org Wed Dec 2 14:34:04 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6758C47C375 for ; Wed, 2 Dec 2020 14:34:04 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4CmM0r1hKKz4rVd for ; Wed, 2 Dec 2020 14:34:04 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 3745D47C73C; Wed, 2 Dec 2020 14:34:04 +0000 (UTC) Delivered-To: git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 357A847C5E4 for ; Wed, 2 Dec 2020 14:34:04 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2a05:fc87:1:5::15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "www.spoerlein.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CmM0q47F2z4rFl; Wed, 2 Dec 2020 14:34:03 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: from localhost (acme.spoerlein.net [IPv6:2a05:fc87:1:5:0:0:0:15]) by acme.spoerlein.net (8.16.1/8.15.2) with ESMTPS id 0B2EXlt2003684 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Wed, 2 Dec 2020 15:33:47 +0100 (CET) (envelope-from uqs@freebsd.org) Date: Wed, 2 Dec 2020 15:33:47 +0100 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Warner Losh Cc: Rene Ladan , Chris Rees , Adam Weinberger , git@freebsd.org, "portmgr@FreeBSD.org" Subject: Re: converting rmport to git Message-ID: References: <20201129164707.GA31739@freefall.freebsd.org> <14871125-A032-4980-8DB1-0210E34D5A11@FreeBSD.org> <20201130105337.GA42359@freefall.freebsd.org> <7246FB00-655B-4BD4-BC99-B87E4595969C@FreeBSD.org> <20201201095906.GA50345@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: User-Agent: Mutt/2.0.1 (2020-11-14) X-Rspamd-Queue-Id: 4CmM0q47F2z4rFl X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Dec 2020 14:34:04 -0000 On Tue, 2020-12-01 at 09:36:48 -0700, Warner Losh wrote: >On Tue, Dec 1, 2020 at 2:59 AM Rene Ladan wrote: >> On Mon, Nov 30, 2020 at 08:19:40PM +0000, Chris Rees wrote: >> > The only reason I wrote the removed ports thing is because we were >> losing the link between old and new version when a port was removed and >> readded. Is git likely to DTRT without it? >> I don't know, perhaps we can search the git history itself. >> > >The typical way I've seen this done is to add the metadata to either the >'bring it back' commit (so like our MFC After: lines), or using the 'notes' >feature in some way. The upside to either of these techniques is that you >preserve the data. the downside is that neither is known by git log. > >I've also seen weird merge commits attempted for this since git log knows >about them... The only problem is they are sufficiently weird that git log >does odd things for everybody across them... > >I don't anticipate that git will do the right thing w/o help because the >ports tree has so many nearly identical files. For simple ports, the >Makefile could have been copied from anywhere, likewise some of the pkg* >files. Though for simple ones, lost history doesn't lose much that's >useful. For complex ones, it likely would do close to the right thing since >complex ports, kinda by definition, are different. Though I'd test this >notion (because as I'm typing this I find myself making too many mental >reservations)... > >To be honest, though, I think this is an area where some experimentation to >understand the alternatives is needed because this use case is relatively >rare in the larger open source community. Isn't the ports copy or move fixable by doing it in 2 steps? That is, one commit is copying or moving the file without any modification inside the file whatsoever (sic!). This is obviously a bogus state for the ports tree, so a 2nd commit then changes the content of that new file to adjust its name or category or whatever. The good thing about a DVCS is that these 2 commits will obviously be pushed together, atomically, so no one will ever see the broken intermediate state. Unless of course you want to `git bisect` stuff, so it's not a panacea and also brittle as it depends on a human doing the right thing. Cheers Uli From owner-freebsd-git@freebsd.org Wed Dec 2 16:08:12 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C785C47ECA7 for ; Wed, 2 Dec 2020 16:08:12 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4CmP5S4GZYz3DxQ for ; Wed, 2 Dec 2020 16:08:12 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 9276547EC0F; Wed, 2 Dec 2020 16:08:12 +0000 (UTC) Delivered-To: git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 923B447EAC0 for ; Wed, 2 Dec 2020 16:08:12 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com [IPv6:2607:f8b0:4864:20::732]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CmP5S3TYlz3Dcw; Wed, 2 Dec 2020 16:08:12 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: by mail-qk1-x732.google.com with SMTP id v143so1693392qkb.2; Wed, 02 Dec 2020 08:08:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=sOR9ZOJmTLn00I85aznevMdo/0B/Ib6KB/ypYkPGgm8=; b=cPRyMoWsQPW+O+IFC8wc5Me/xPhZ3KbonFuuc9ZMaL0GM0uMWRxXH9464U3HyXZ63b EOoEe0tiFQ17S9VshoxM04AX6R0gTofmnorYmpthZELl2qCWFWzViFJEESQPIKT201qJ +PgD7qVwoZmcX14qfaiEH1k14ubgWNaSi/+nKe/4EJAptFA2KK0h+outaiqYHyEeiek9 m2zSc5KDPsat8sCBYB6P7O9O9dNhBztNElxfkrZkHPjuaVngWgtYpFKlf6rtNrL3OG5v uMsR5bYmPMKQmjDWuyyiqjsoBqZ9AFHLxzaaJt39MWiYdF4zu39M6qrNfjkGyOlaudjn jLxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=sOR9ZOJmTLn00I85aznevMdo/0B/Ib6KB/ypYkPGgm8=; b=t115pDaWt1YLZQBTRYnGOR8MzIYqaa2rXzIRQm7AFwT0TvC5YzPH4T/9tk7ML0/I+f sIdu0r9uWO/JU5OylbAqTo+PFjgMzZ6BRU40Sbo9MM6x21ydevwDZENINjjMyMS/eHmy 4p7kOf4B4EgrdoVTsqcH4ClIgWPslSVxqfv9MNCV8mICdewSc/34LeVVEsCQgquElM1P K2K2GKSIdSibTITnHammJvQF/80//Zb+TQXdfTuiR5eEq4L9eecgGxIavKHLUDDlSBhd jWw99FO6rhjARtLnNVFwSsGKQ7q1Tpg4nGM5FsPPn3aHB3ctQUlKXrN4nz/K31SlCbx8 CmYg== X-Gm-Message-State: AOAM532Gj9KcVcLQE6gFHFljBhB72goX20xwSnL7RXgMQiH6sM7NZ4CZ SptIvQVrJKvJhefQuJwe2hI= X-Google-Smtp-Source: ABdhPJxal9VSG2MFr/0yECPx1pxMr91BsCraGf23WYGZDhjdCa3/+jOnyO0pIeIaPK4utXTiytF1pg== X-Received: by 2002:a05:620a:2189:: with SMTP id g9mr3305656qka.488.1606925291512; Wed, 02 Dec 2020 08:08:11 -0800 (PST) Received: from [192.168.222.18] (192-222-183-158.qc.cable.ebox.net. [192.222.183.158]) by smtp.gmail.com with ESMTPSA id c6sm2104272qkg.54.2020.12.02.08.08.10 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 08:08:10 -0800 (PST) Subject: Using git branches for ports (was: Re: converting rmport to git) To: Warner Losh Cc: git@freebsd.org, "portmgr@FreeBSD.org" References: <20201129164707.GA31739@freefall.freebsd.org> <14871125-A032-4980-8DB1-0210E34D5A11@FreeBSD.org> <20201130105337.GA42359@freefall.freebsd.org> <7246FB00-655B-4BD4-BC99-B87E4595969C@FreeBSD.org> <20201201095906.GA50345@freefall.freebsd.org> From: Marc Branchaud Message-ID: Date: Wed, 2 Dec 2020 11:08:10 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4CmP5S3TYlz3Dcw X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Dec 2020 16:08:12 -0000 On 2020-12-01 11:36 a.m., Warner Losh wrote: > > To be honest, though, I think this is an area where some experimentation to > understand the alternatives is needed because this use case is relatively > rare in the larger open source community. OK, so I just have to ask (and I apologize if I'm opening a can of worms that has already been discussed, or that nobody wants to look at; I'll drop this if it's just noise): Have you considered using a branch for each port? Yes, I'm talking about 41,000+ branches. Git should not have any trouble dealing with that. There are a few advantages to this approach: * Each port's change history is fully isolated and easy to track. (Don't worry about having lots of near-duplicate files in different branches or directories, as git is very efficient at dealing with this.) * MFCs are proper git merges, which means that it is very easy to understand which changes have landed where. * Cases like removing/re-adding a port would take place on that port's branch, making it obvious just how that work was done. If this sounds appealing, then the real question is whether or not this approach trips up any important cases that arise when working on ports. I can't answer that, but in the grand tradition of git branch ASCII-art, here is a pretty picture to help understand what this approach might look like. In the following: - "a" thru "f" represent commits of some work on a port (net/gsk, for example). - "M#" represent git merge commits of some net/gsk changes. - "m" represents a git merge commit of some other port's changes. My proposed branch names are on the left. Commit history proceeds from left to right. main ....--m---m--M1--m--M2--m--M3--m / / / net/gsk ....-a--b---c--d--e---f \ 2020Q4 ....---m---M4---m---m So the net/gsk port evolves on its own "net/gsk" branch, with commits a..f. We see that the a and b changes were merged into the "main" branch by merge-commit M1. Merge commit M2 brought in changes c and d, and then merge M3 brought changes e and f into "main". Meanwhile, only changes a and b have been merged into the "2020Q4" branch (commit M4). Both the "main" and "2020Q4" branches also contain merges from other ports' branches (the "m" commits). The mainline branches ("main", "2020Q4", etc) would consist almost entirely of merge commits. The net/gsk changes in the mainline branches can be easily obtained from simple git commands. To see the net/gsk work that has happened in a mainline branch like 2020Q4, just do git log 2020Q4 -- net/gsk That will list commits a, b and M4. No need to do any patch-level analysis. That command will also work with the existing git repo migrated form svn. But the branch-based model has some additional power. For example, a command like git log --oneline --graph 2020Q4 -- net/gsk will output an ASCII-art picture of the 2020Q4 branch's view of the net/gsk port, similar to what I drew above. More importantly, it's easy to see where any particular piece of the net/gsk work has landed: git branch -a --contains would report the "main" and "2020Q4" branches (here the is the SHA-ID of commit b). No need to deal with "combined" MFCs or did-this-change-match-that-patch problems. What about the rmport script? The branches I'm describing contain the full ports tree -- they're not "partial" or "sparse" in any way. So to remove the net/gsk port, rmport would just checkout the "net/gsk" branch and do the removal there. Then that can be merged (manually or automatically) into whatever mainline branch is desired. There's no need to remove the "net/gsk" branch though, and it's better to keep it around in case someone wants to revive the net/gsk port in the future. This branch-based model can be adopted atop the transitioned ports repo as it stands today. There's no need (nor is it possible) to retroactively translate the svn history into this structure. Sure, the migrated svn history isn't amenable to tricks like "git branch --contains", but that will become less important as time marches on. And the migrated history can still be teased out using patch-level commands like "git cherry". Those are my main points, so you can stop reading here if you're already annoyed! I'm now going to delve into some of the flexibility that this approach offers. In this model the net/gsk port is free to evolve as it needs to in the "net/gsk" branch. From the above we see that changes a and b were deemed good enough to put into 2020Q4, but changes c-f are still a bit experimental and they're still being validated on the "main" branch. (I'm making some assumptions here about how people develop the ports. Apologies if I got it wrong; I'm sure this model can accommodate a different workflow.) In fact, that "net/gsk" branch can itself contain sub-branches for special circumstances. Let's say that commit b has a bug. We'd like to fix that bug in both "main" and "2020Q4", but if we just plop the fix onto the tip of the "net/gsk" branch (as commit g, say) that change will have commits c-f has part of its history: main ....--m---m--M1--m--M2--m--M3--m / / / net/gsk ....-a--b---c--d--e---f---g \ 2020Q4 ....---m---M4---m---m If we just merged g into 2020Q4, we'd also bring in the c-f changes which we do not want to have on the 2020Q4 branch. So instead, we can fix the bug in a mini branch based on the b commit, then merge that work where it's needed: main ....--m---m--M1--m--M2--m--M3--m--M6 / / / / net/gsk ....-a--b---c--d--e---f------g' |\ / | \---------b'-----/ \ \ 2020Q4 ....---m---M4---m---m--M5 Here we've fixed the bug with commit b', which is based directly on commit b and so we can merge b' into the "2020Q4" branch (commit M5), with the confidence that we're only bringing in the exact bug fix we need. Meanwhile, we also merge b' onto the tip of the "net/gsk" branch (commit g'), fixing the bug on the port's own branch, and then merge g' into the "main" branch as commit M6. It is completely clear what happened to the net/gsk port, and how those changes were brought into the mainline branches. One last wrinkle about this picture: Note how I did not put a name on the branch with the b' commit. Git is perfectly happy to deal with this kind of anonymous branching, and so there's no need to pollute the central FreeBSD-ports repository with names for these kinds of branches. But that does not prevent the net/gsk developer from having a *local* name for that branch in their own, local clone of the repository. The developer can name their local branch whatever makes sense to them. When they push one of the merge commits (M5, g' or M6) to the central repo, the b' commit rides along but without the developer's local branch name. The history recorded in the central repository is as depicted, with b' living on a nameless branch. I can't believe you've read all of this! Thanks! M. From owner-freebsd-git@freebsd.org Wed Dec 2 16:29:11 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AC86F47F313 for ; Wed, 2 Dec 2020 16:29:11 +0000 (UTC) (envelope-from uspoerlein@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4CmPYg2swZz3Ff0 for ; Wed, 2 Dec 2020 16:29:11 +0000 (UTC) (envelope-from uspoerlein@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 60B5B47F0CC; Wed, 2 Dec 2020 16:29:11 +0000 (UTC) Delivered-To: git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6076747F0CB for ; Wed, 2 Dec 2020 16:29:11 +0000 (UTC) (envelope-from uspoerlein@gmail.com) Received: from mail-oi1-f172.google.com (mail-oi1-f172.google.com [209.85.167.172]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CmPYg27j5z3FsP; Wed, 2 Dec 2020 16:29:11 +0000 (UTC) (envelope-from uspoerlein@gmail.com) Received: by mail-oi1-f172.google.com with SMTP id f11so2159448oij.6; Wed, 02 Dec 2020 08:29:11 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=/ecUH3qQXilpNUrb0sQvaNNpikk0tesj5QxWUWo83Pw=; b=ZLroN92/hcXhkvOsCzL3u3xtPRMJj+KWltrXzJQZMWGjKF33Qj5i6a/1fVClB/SIyU Iox0eafLtNphM/B5/9FRjqtqTQ+1D3bNCo7cfBL6R312YyDygsCClR50IpX4ho/8RD62 jKZTmjlFZGSTAGGCzXDL2zS0Knx7PSC698ABhTl27wVNfVl2Z8W5JRg5OiL4oTq1gGxo 834ronjt0PK1U5arIb9FPIzevACPThyoQ62mkPPNCl4KhjFAnZP+CgplD38RCFlAj2qA I9NZEr9foBgS3b9BXyuBCnVFxDEncDgxrYw7TRd5ndEMAag5ykCPgDM21sUVAAT5kbiY CLpw== X-Gm-Message-State: AOAM530Ohpt4qzpwlE/fx9FxYpwULoJ4m5HkyM/6Ala73MhCDYwDu8cy 3gXEmcVfESsqP9TnwQ098Qe7aU+UggrR3YDuT4He6EhEWeTChw== X-Google-Smtp-Source: ABdhPJzfOmYr2v3ww08x0+crE/l3iXHHxkOrYxzWgHPInDWT8rrkSMqP+z/z5psmJlQJkmWYd4vf+ozEzYmqQEllGog= X-Received: by 2002:a54:4d8f:: with SMTP id y15mr2200204oix.150.1606926549952; Wed, 02 Dec 2020 08:29:09 -0800 (PST) MIME-Version: 1.0 References: <20201129164707.GA31739@freefall.freebsd.org> <14871125-A032-4980-8DB1-0210E34D5A11@FreeBSD.org> <20201130105337.GA42359@freefall.freebsd.org> <7246FB00-655B-4BD4-BC99-B87E4595969C@FreeBSD.org> <20201201095906.GA50345@freefall.freebsd.org> In-Reply-To: From: =?UTF-8?Q?Ulrich_Sp=C3=B6rlein?= Date: Wed, 2 Dec 2020 17:28:58 +0100 Message-ID: Subject: Re: Using git branches for ports (was: Re: converting rmport to git) To: Marc Branchaud Cc: Warner Losh , git@freebsd.org, "portmgr@FreeBSD.org" X-Rspamd-Queue-Id: 4CmPYg27j5z3FsP X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Dec 2020 16:29:11 -0000 Sorry didn't read all of this :). I think the proposal has merit for a ports maintainer maintaining individual ports. But I don't see how it can work: - Where's Mk/ then? What about MOVED and UPDATING? - How does one check out the full ports tree so that build dependent ports works? - How can you do a shlib bump with all the required PORTREVISION bumps? It would need elaborate scripting support to produce the 2 dozen commits required for that. - Say you worked on such a bump across dozens of ports, you'd need more scripts to merge or rebase changes that have happened in the mean time. I'm sure there are more issues. It would require us to properly slice out individual ports vs. things like Mk/ and my guess the people working on the ports infra as such prefer to have a full tree visible (and grep'able) Interesting idea though ... Uli On Wed, Dec 2, 2020 at 5:08 PM Marc Branchaud wrote: > On 2020-12-01 11:36 a.m., Warner Losh wrote: > > > > To be honest, though, I think this is an area where some experimentation > to > > understand the alternatives is needed because this use case is relatively > > rare in the larger open source community. > > OK, so I just have to ask (and I apologize if I'm opening a can of worms > that has already been discussed, or that nobody wants to look at; I'll > drop this if it's just noise): > > Have you considered using a branch for each port? Yes, I'm talking > about 41,000+ branches. Git should not have any trouble dealing with that. > > There are a few advantages to this approach: > > * Each port's change history is fully isolated and easy to track. > (Don't worry about having lots of near-duplicate files in different > branches or directories, as git is very efficient at dealing with this.) > > * MFCs are proper git merges, which means that it is very easy to > understand which changes have landed where. > > * Cases like removing/re-adding a port would take place on that port's > branch, making it obvious just how that work was done. > > If this sounds appealing, then the real question is whether or not this > approach trips up any important cases that arise when working on ports. > > I can't answer that, but in the grand tradition of git branch ASCII-art, > here is a pretty picture to help understand what this approach might > look like. In the following: > - "a" thru "f" represent commits of some work on a port (net/gsk, for > example). > - "M#" represent git merge commits of some net/gsk changes. > - "m" represents a git merge commit of some other port's changes. > My proposed branch names are on the left. Commit history proceeds from > left to right. > > main ....--m---m--M1--m--M2--m--M3--m > / / / > net/gsk ....-a--b---c--d--e---f > \ > 2020Q4 ....---m---M4---m---m > > So the net/gsk port evolves on its own "net/gsk" branch, with commits > a..f. We see that the a and b changes were merged into the "main" > branch by merge-commit M1. Merge commit M2 brought in changes c and d, > and then merge M3 brought changes e and f into "main". Meanwhile, only > changes a and b have been merged into the "2020Q4" branch (commit M4). > > Both the "main" and "2020Q4" branches also contain merges from other > ports' branches (the "m" commits). The mainline branches ("main", > "2020Q4", etc) would consist almost entirely of merge commits. > > The net/gsk changes in the mainline branches can be easily obtained from > simple git commands. To see the net/gsk work that has happened in a > mainline branch like 2020Q4, just do > git log 2020Q4 -- net/gsk > That will list commits a, b and M4. No need to do any patch-level > analysis. > > That command will also work with the existing git repo migrated form > svn. But the branch-based model has some additional power. For > example, a command like > git log --oneline --graph 2020Q4 -- net/gsk > will output an ASCII-art picture of the 2020Q4 branch's view of the > net/gsk port, similar to what I drew above. > > More importantly, it's easy to see where any particular piece of the > net/gsk work has landed: > git branch -a --contains > would report the "main" and "2020Q4" branches (here the is the > SHA-ID of commit b). No need to deal with "combined" MFCs or > did-this-change-match-that-patch problems. > > What about the rmport script? The branches I'm describing contain the > full ports tree -- they're not "partial" or "sparse" in any way. So to > remove the net/gsk port, rmport would just checkout the "net/gsk" branch > and do the removal there. Then that can be merged (manually or > automatically) into whatever mainline branch is desired. There's no > need to remove the "net/gsk" branch though, and it's better to keep it > around in case someone wants to revive the net/gsk port in the future. > > This branch-based model can be adopted atop the transitioned ports repo > as it stands today. There's no need (nor is it possible) to > retroactively translate the svn history into this structure. Sure, the > migrated svn history isn't amenable to tricks like "git branch > --contains", but that will become less important as time marches on. > And the migrated history can still be teased out using patch-level > commands like "git cherry". > > Those are my main points, so you can stop reading here if you're already > annoyed! I'm now going to delve into some of the flexibility that this > approach offers. > > > In this model the net/gsk port is free to evolve as it needs to in the > "net/gsk" branch. From the above we see that changes a and b were > deemed good enough to put into 2020Q4, but changes c-f are still a bit > experimental and they're still being validated on the "main" branch. > (I'm making some assumptions here about how people develop the ports. > Apologies if I got it wrong; I'm sure this model can accommodate a > different workflow.) > > In fact, that "net/gsk" branch can itself contain sub-branches for > special circumstances. Let's say that commit b has a bug. We'd like to > fix that bug in both "main" and "2020Q4", but if we just plop the fix > onto the tip of the "net/gsk" branch (as commit g, say) that change will > have commits c-f has part of its history: > > main ....--m---m--M1--m--M2--m--M3--m > / / / > net/gsk ....-a--b---c--d--e---f---g > \ > 2020Q4 ....---m---M4---m---m > > If we just merged g into 2020Q4, we'd also bring in the c-f changes > which we do not want to have on the 2020Q4 branch. > > So instead, we can fix the bug in a mini branch based on the b commit, > then merge that work where it's needed: > > main ....--m---m--M1--m--M2--m--M3--m--M6 > / / / / > net/gsk ....-a--b---c--d--e---f------g' > |\ / > | \---------b'-----/ > \ \ > 2020Q4 ....---m---M4---m---m--M5 > > Here we've fixed the bug with commit b', which is based directly on > commit b and so we can merge b' into the "2020Q4" branch (commit M5), > with the confidence that we're only bringing in the exact bug fix we > need. Meanwhile, we also merge b' onto the tip of the "net/gsk" branch > (commit g'), fixing the bug on the port's own branch, and then merge g' > into the "main" branch as commit M6. It is completely clear what > happened to the net/gsk port, and how those changes were brought into > the mainline branches. > > One last wrinkle about this picture: Note how I did not put a name on > the branch with the b' commit. Git is perfectly happy to deal with this > kind of anonymous branching, and so there's no need to pollute the > central FreeBSD-ports repository with names for these kinds of branches. > But that does not prevent the net/gsk developer from having a *local* > name for that branch in their own, local clone of the repository. The > developer can name their local branch whatever makes sense to them. > When they push one of the merge commits (M5, g' or M6) to the central > repo, the b' commit rides along but without the developer's local branch > name. The history recorded in the central repository is as depicted, > with b' living on a nameless branch. > > I can't believe you've read all of this! Thanks! > > M. > > _______________________________________________ > freebsd-git@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-git > To unsubscribe, send any mail to "freebsd-git-unsubscribe@freebsd.org" > From owner-freebsd-git@freebsd.org Wed Dec 2 16:32:34 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F29EB47F489 for ; Wed, 2 Dec 2020 16:32:34 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4CmPdZ5ZSpz3FtR for ; Wed, 2 Dec 2020 16:32:34 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mailman.nyi.freebsd.org (Postfix) id BF3DE47F488; Wed, 2 Dec 2020 16:32:34 +0000 (UTC) Delivered-To: git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BDE4447F487 for ; Wed, 2 Dec 2020 16:32:34 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x729.google.com (mail-qk1-x729.google.com [IPv6:2607:f8b0:4864:20::729]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CmPdZ4Y5rz3G4x for ; Wed, 2 Dec 2020 16:32:34 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x729.google.com with SMTP id u4so1763260qkk.10 for ; Wed, 02 Dec 2020 08:32:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=AklrxOc/KZDIbRLeXqYqrx4wrTHsdG6yypYKFpK+hNw=; b=HDjp9g8pXxOe86ulOmkdJUhcRlpvrs7M/VA2LWR93rCuDmqAKk43cflyyQpwQJyVR0 /ff1HK6Uz4JfTb+fmf3p+qE7oFSPQea1xRWVUBgX7rDRtJwME3h97GF8LQp5znabFhUy +0iLZ0QOWUgjpbtospXCAOPoX/kWnZY1Ui3aaTObRt+QYYeM7zuIdied7LOsfhA3UQeE SIH0D6FcCZPMqee+/hjq5eHr9d1AOlkW7jA2wUuqZ0TP4sAF9dr+SZwBeaGGR34W88SN alDu0WBhSeuYLFGG7k+4ap3VeGGHEOIWlCyldMrBkM98J9iRX8I8QuSCSmJgmM7IP/kr 3hVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=AklrxOc/KZDIbRLeXqYqrx4wrTHsdG6yypYKFpK+hNw=; b=T+SfywMYxrpppWYuzSiYRqgX4NfApFu+DGJ9UQl/hy28PXBH9ybzJpwartPf+wC640 jxzOJ8sprSre6Tv4Mo5nVoK992rnM6hB9P99tJ0x1svYqXhy3BvEtZPi67cqC7sXWbF6 Nbsuhn0ch0d7qfFbYp9sPltsM/Qi2LJ1E5fhMbgYdGX/2oW4BbXkl6yCSmndggdg/c5n 2PYSET191XMY1549kzc9EBguCLx2GzGT4SS3XhC6GKxJrw3GJ+4JXGhDeIuN2quIb0Ej XoxFcbWamsezx4BFo2ljZLyOtSKoicHzNfM5ZYW48xhZA6F576K1wV3hiyiCiRhUgBXr nJTg== X-Gm-Message-State: AOAM530iwxOeSwBRjJWKc5LHwAICLsSIwp8aiFqW0AHjwZa6sWzrz+7Z 7gxm8NslFtwL3lVXDyickBrI5ejRReTEU9PWbJBEB3nRznppAA== X-Google-Smtp-Source: ABdhPJyjAMY4eqGcXezAnXqswkAfzumAPD/RdHgd0fg+jzJUWONMPkfdTfzGRrHJeQzhfIrvA/ewojOCdt2APMYfXLU= X-Received: by 2002:a05:620a:88e:: with SMTP id b14mr3352225qka.195.1606926752644; Wed, 02 Dec 2020 08:32:32 -0800 (PST) MIME-Version: 1.0 References: <20201129164707.GA31739@freefall.freebsd.org> <14871125-A032-4980-8DB1-0210E34D5A11@FreeBSD.org> <20201130105337.GA42359@freefall.freebsd.org> <7246FB00-655B-4BD4-BC99-B87E4595969C@FreeBSD.org> <20201201095906.GA50345@freefall.freebsd.org> In-Reply-To: From: Warner Losh Date: Wed, 2 Dec 2020 09:32:21 -0700 Message-ID: Subject: Re: Using git branches for ports (was: Re: converting rmport to git) To: Marc Branchaud Cc: git@freebsd.org, "portmgr@FreeBSD.org" X-Rspamd-Queue-Id: 4CmPdZ4Y5rz3G4x X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Dec 2020 16:32:35 -0000 Top posting... I'm not sure how productive it would be to do the blow by blow since it seems to be fraught With respect, I think this is too complicated to be workable. git merges don't work quite the way you think they do (merging an individual commit isn't a thing in git, it's specifically called out as a cherry-pick for a reason). Such a complex merge history will create way more issues than it solves, imho. "proper git merges" is not something that we should strive for, since merges in git are very very different than they are in subversion (where proper merges are preferable). Many of the things that you call 'merges' would more properly be done as cherry picks. There's no benefit from forcing them to be merges, and IMHO, and it creates nothing but trouble. There's also issues with complex ports that span dozens of different directories that would be problematic. While there are ways that we may be able to use branches in a productive way, but I'm having trouble seeing how this proposal accomplishes that. Warner On Wed, Dec 2, 2020 at 9:08 AM Marc Branchaud wrote: > On 2020-12-01 11:36 a.m., Warner Losh wrote: > > > > To be honest, though, I think this is an area where some experimentation > to > > understand the alternatives is needed because this use case is relatively > > rare in the larger open source community. > > OK, so I just have to ask (and I apologize if I'm opening a can of worms > that has already been discussed, or that nobody wants to look at; I'll > drop this if it's just noise): > > Have you considered using a branch for each port? Yes, I'm talking > about 41,000+ branches. Git should not have any trouble dealing with that. > > There are a few advantages to this approach: > > * Each port's change history is fully isolated and easy to track. > (Don't worry about having lots of near-duplicate files in different > branches or directories, as git is very efficient at dealing with this.) > > * MFCs are proper git merges, which means that it is very easy to > understand which changes have landed where. > > * Cases like removing/re-adding a port would take place on that port's > branch, making it obvious just how that work was done. > > If this sounds appealing, then the real question is whether or not this > approach trips up any important cases that arise when working on ports. > > I can't answer that, but in the grand tradition of git branch ASCII-art, > here is a pretty picture to help understand what this approach might > look like. In the following: > - "a" thru "f" represent commits of some work on a port (net/gsk, for > example). > - "M#" represent git merge commits of some net/gsk changes. > - "m" represents a git merge commit of some other port's changes. > My proposed branch names are on the left. Commit history proceeds from > left to right. > > main ....--m---m--M1--m--M2--m--M3--m > / / / > net/gsk ....-a--b---c--d--e---f > \ > 2020Q4 ....---m---M4---m---m > > So the net/gsk port evolves on its own "net/gsk" branch, with commits > a..f. We see that the a and b changes were merged into the "main" > branch by merge-commit M1. Merge commit M2 brought in changes c and d, > and then merge M3 brought changes e and f into "main". Meanwhile, only > changes a and b have been merged into the "2020Q4" branch (commit M4). > > Both the "main" and "2020Q4" branches also contain merges from other > ports' branches (the "m" commits). The mainline branches ("main", > "2020Q4", etc) would consist almost entirely of merge commits. > > The net/gsk changes in the mainline branches can be easily obtained from > simple git commands. To see the net/gsk work that has happened in a > mainline branch like 2020Q4, just do > git log 2020Q4 -- net/gsk > That will list commits a, b and M4. No need to do any patch-level > analysis. > > That command will also work with the existing git repo migrated form > svn. But the branch-based model has some additional power. For > example, a command like > git log --oneline --graph 2020Q4 -- net/gsk > will output an ASCII-art picture of the 2020Q4 branch's view of the > net/gsk port, similar to what I drew above. > > More importantly, it's easy to see where any particular piece of the > net/gsk work has landed: > git branch -a --contains > would report the "main" and "2020Q4" branches (here the is the > SHA-ID of commit b). No need to deal with "combined" MFCs or > did-this-change-match-that-patch problems. > > What about the rmport script? The branches I'm describing contain the > full ports tree -- they're not "partial" or "sparse" in any way. So to > remove the net/gsk port, rmport would just checkout the "net/gsk" branch > and do the removal there. Then that can be merged (manually or > automatically) into whatever mainline branch is desired. There's no > need to remove the "net/gsk" branch though, and it's better to keep it > around in case someone wants to revive the net/gsk port in the future. > > This branch-based model can be adopted atop the transitioned ports repo > as it stands today. There's no need (nor is it possible) to > retroactively translate the svn history into this structure. Sure, the > migrated svn history isn't amenable to tricks like "git branch > --contains", but that will become less important as time marches on. > And the migrated history can still be teased out using patch-level > commands like "git cherry". > > Those are my main points, so you can stop reading here if you're already > annoyed! I'm now going to delve into some of the flexibility that this > approach offers. > > > In this model the net/gsk port is free to evolve as it needs to in the > "net/gsk" branch. From the above we see that changes a and b were > deemed good enough to put into 2020Q4, but changes c-f are still a bit > experimental and they're still being validated on the "main" branch. > (I'm making some assumptions here about how people develop the ports. > Apologies if I got it wrong; I'm sure this model can accommodate a > different workflow.) > > In fact, that "net/gsk" branch can itself contain sub-branches for > special circumstances. Let's say that commit b has a bug. We'd like to > fix that bug in both "main" and "2020Q4", but if we just plop the fix > onto the tip of the "net/gsk" branch (as commit g, say) that change will > have commits c-f has part of its history: > > main ....--m---m--M1--m--M2--m--M3--m > / / / > net/gsk ....-a--b---c--d--e---f---g > \ > 2020Q4 ....---m---M4---m---m > > If we just merged g into 2020Q4, we'd also bring in the c-f changes > which we do not want to have on the 2020Q4 branch. > > So instead, we can fix the bug in a mini branch based on the b commit, > then merge that work where it's needed: > > main ....--m---m--M1--m--M2--m--M3--m--M6 > / / / / > net/gsk ....-a--b---c--d--e---f------g' > |\ / > | \---------b'-----/ > \ \ > 2020Q4 ....---m---M4---m---m--M5 > > Here we've fixed the bug with commit b', which is based directly on > commit b and so we can merge b' into the "2020Q4" branch (commit M5), > with the confidence that we're only bringing in the exact bug fix we > need. Meanwhile, we also merge b' onto the tip of the "net/gsk" branch > (commit g'), fixing the bug on the port's own branch, and then merge g' > into the "main" branch as commit M6. It is completely clear what > happened to the net/gsk port, and how those changes were brought into > the mainline branches. > > One last wrinkle about this picture: Note how I did not put a name on > the branch with the b' commit. Git is perfectly happy to deal with this > kind of anonymous branching, and so there's no need to pollute the > central FreeBSD-ports repository with names for these kinds of branches. > But that does not prevent the net/gsk developer from having a *local* > name for that branch in their own, local clone of the repository. The > developer can name their local branch whatever makes sense to them. > When they push one of the merge commits (M5, g' or M6) to the central > repo, the b' commit rides along but without the developer's local branch > name. The history recorded in the central repository is as depicted, > with b' living on a nameless branch. > > I can't believe you've read all of this! Thanks! > > M. > > From owner-freebsd-git@freebsd.org Wed Dec 2 18:31:17 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 330E94A2E59 for ; Wed, 2 Dec 2020 18:31:17 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4CmSGY0FDZz3Prr for ; Wed, 2 Dec 2020 18:31:17 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: by mailman.nyi.freebsd.org (Postfix) id 069444A2EED; Wed, 2 Dec 2020 18:31:17 +0000 (UTC) Delivered-To: git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 065B44A3302 for ; Wed, 2 Dec 2020 18:31:17 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CmSGX6gy6z3PXC; Wed, 2 Dec 2020 18:31:16 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id F06E23C0199; Wed, 2 Dec 2020 18:31:15 +0000 (UTC) Date: Wed, 2 Dec 2020 18:31:15 +0000 From: Brooks Davis To: Ulrich Sp??rlein Cc: Warner Losh , Chris Rees , "portmgr@FreeBSD.org" , git@freebsd.org, Adam Weinberger Subject: Re: converting rmport to git Message-ID: <20201202183115.GE18452@spindle.one-eyed-alien.net> References: <20201129164707.GA31739@freefall.freebsd.org> <14871125-A032-4980-8DB1-0210E34D5A11@FreeBSD.org> <20201130105337.GA42359@freefall.freebsd.org> <7246FB00-655B-4BD4-BC99-B87E4595969C@FreeBSD.org> <20201201095906.GA50345@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="cHMo6Wbp1wrKhbfi" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-Rspamd-Queue-Id: 4CmSGX6gy6z3PXC X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Dec 2020 18:31:17 -0000 --cHMo6Wbp1wrKhbfi Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Dec 02, 2020 at 03:33:47PM +0100, Ulrich Sp??rlein wrote: > On Tue, 2020-12-01 at 09:36:48 -0700, Warner Losh wrote: > >On Tue, Dec 1, 2020 at 2:59 AM Rene Ladan wrote: > >> On Mon, Nov 30, 2020 at 08:19:40PM +0000, Chris Rees wrote: > >> > The only reason I wrote the removed ports thing is because we were > >> losing the link between old and new version when a port was removed and > >> readded. Is git likely to DTRT without it? > >> I don't know, perhaps we can search the git history itself. > >> > > > >The typical way I've seen this done is to add the metadata to either the > >'bring it back' commit (so like our MFC After: lines), or using the 'not= es' > >feature in some way. The upside to either of these techniques is that you > >preserve the data. the downside is that neither is known by git log. > > > >I've also seen weird merge commits attempted for this since git log knows > >about them... The only problem is they are sufficiently weird that git l= og > >does odd things for everybody across them... > > > >I don't anticipate that git will do the right thing w/o help because the > >ports tree has so many nearly identical files. For simple ports, the > >Makefile could have been copied from anywhere, likewise some of the pkg* > >files. Though for simple ones, lost history doesn't lose much that's > >useful. For complex ones, it likely would do close to the right thing si= nce > >complex ports, kinda by definition, are different. Though I'd test this > >notion (because as I'm typing this I find myself making too many mental > >reservations)... > > > >To be honest, though, I think this is an area where some experimentation= to > >understand the alternatives is needed because this use case is relatively > >rare in the larger open source community. >=20 > Isn't the ports copy or move fixable by doing it in 2 steps? That is,=20 > one commit is copying or moving the file without any modification inside= =20 > the file whatsoever (sic!). This is obviously a bogus state for the=20 > ports tree, so a 2nd commit then changes the content of that new file to= =20 > adjust its name or category or whatever. >=20 > The good thing about a DVCS is that these 2 commits will obviously be=20 > pushed together, atomically, so no one will ever see the broken=20 > intermediate state. Unless of course you want to `git bisect` stuff, so= =20 > it's not a panacea and also brittle as it depends on a human doing the=20 > right thing. The state need not even be broken, just partially orphaned (e.g. perform the copy, but don't hook the port up until the second commit.) Likewise for resurrection, revert the removal, but don't include the addition to /Makefile in the revert. This would work best if deletes are done as separate commits, but that shouldn't be a big deal. -- Brooks --cHMo6Wbp1wrKhbfi Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJfx91zAAoJEKzQXbSebgfA/SQIAIKLaFbsvcimG9zYBwaXNkmr pkH+m8ogGuqu1yZR32Lz9tk+qivJmHY8pJkBTyeUg6pFGkr9ycn9DGHG5l271xr4 LnpA6yoaIV8ID5sEGNYJ/LsosoxvDcGHK9D4O86T8+GJewq4/+PcfakHFFGrj3nW opoc4oRqBSOw94ZYH4rlkGgA6vFdFleaGekqJlmhQSuBe5S2C/V71imwB10rMWm8 yUv/XdEvSX/M7eqaYo/uioZaxOJHMKJ6OfNyFD16xCVAgO3lcXhUv4Z40MoWpH5o 8DxHU1H9REnyulNVeEPcZpoWsBEND0wrCANVmTkrMs7DxMJDq0vxZszDgN0VDBY= =WB/8 -----END PGP SIGNATURE----- --cHMo6Wbp1wrKhbfi-- From owner-freebsd-git@freebsd.org Wed Dec 2 21:15:28 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D967B4A681D for ; Wed, 2 Dec 2020 21:15:28 +0000 (UTC) (envelope-from bakul@iitbombay.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4CmWw04gtTz3pLl for ; Wed, 2 Dec 2020 21:15:28 +0000 (UTC) (envelope-from bakul@iitbombay.org) Received: by mailman.nyi.freebsd.org (Postfix) id A09084A66BC; Wed, 2 Dec 2020 21:15:28 +0000 (UTC) Delivered-To: git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A05544A6627 for ; Wed, 2 Dec 2020 21:15:28 +0000 (UTC) (envelope-from bakul@iitbombay.org) Received: from mail-pl1-x62c.google.com (mail-pl1-x62c.google.com [IPv6:2607:f8b0:4864:20::62c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CmWvz61KSz3pT2 for ; Wed, 2 Dec 2020 21:15:27 +0000 (UTC) (envelope-from bakul@iitbombay.org) Received: by mail-pl1-x62c.google.com with SMTP id b23so1819104pls.11 for ; Wed, 02 Dec 2020 13:15:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iitbombay-org.20150623.gappssmtp.com; s=20150623; h=from:content-transfer-encoding:mime-version:subject:message-id:date :to; bh=Sl+wbmi4m4s7c41l8WjF8lePLtwYyqIT8PDN7dqBz4s=; b=rpSTzRsLW8P5LyyQBQuDaM9wlmrBCu9Tzci5snwc2DQFnH7LVMGi7305sr8ejZj1DX UFOsq6EAQpCJaMuq0T+jGuS9DTm9hIhbtdwytpGam1Jr1cwtTJFqRp3ZR1wRLQEF1nKh T54odK2yGYFZO5sk1vByK9r3/OZH++RafgI6etVjPI/SIe6jQ9hmNaMye/9VflmT7j2E BvSYy3NxvmQzbm6T5ySKaJS5o2Zf8DP0GWnYVpwFrhhdhE9aFnAHgWDq8QA6F0QeCePR 8wBQSRX2S797t2qLpV9hhcvxgfCn/9DeGo86gYlcONdY6TQiL7oHIBypTeS0sU2suxxQ J6FQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:content-transfer-encoding:mime-version :subject:message-id:date:to; bh=Sl+wbmi4m4s7c41l8WjF8lePLtwYyqIT8PDN7dqBz4s=; b=j9GuByaLiCzZ6g+47KeROJB9iBpPu0709jjIXUV/Qj9n7NkQqX7uRoZcE33juAG8+n A3bBVKH3xJg/y7E1gwCX1qvv9gWQjbPMR4975iZxvTk7R5YZYeB8KvyjTtO1V9BfX0Dj Yav23pkwOUFpk0JEhpdlrN68VzWhBtuao5dqMLCmU+r5vBysVh1YWqwCke7BjZ7UFTno tancCL6bjQ/+CbHL7TtnZfKe91zHv7oC3Y/6CXQfNdb/g5Gjrfynyj1iGyj8rHBrhquX ja0OwWWsoIXX6g/xPgdK1nWPOHsSVmEp32YXoP6zH+QPvYS55muIhu4zyXGv/nFm3tc0 DRmw== X-Gm-Message-State: AOAM53000lWjdBEHC4Sw1G2+44LGp1gngzPmX/CdU78BhzWfy+i9F6hV fT36q+u5KNpYS1PiImmfquCVmBg0ZwtMPg== X-Google-Smtp-Source: ABdhPJzSUzx+hDSyM9Ksxc8E8Gdgu7Ksy04XXWJt8tuQy4O95zHKpVDZxhTWRUbkQPUvexbAOYJqlA== X-Received: by 2002:a17:902:8698:b029:da:1d7a:f5ef with SMTP id g24-20020a1709028698b02900da1d7af5efmr4354219plo.67.1606943725775; Wed, 02 Dec 2020 13:15:25 -0800 (PST) Received: from [192.168.1.113] (172-125-77-130.lightspeed.sntcca.sbcglobal.net. [172.125.77.130]) by smtp.gmail.com with ESMTPSA id m4sm17610pfd.203.2020.12.02.13.15.24 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 02 Dec 2020 13:15:25 -0800 (PST) From: Bakul Shah Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.20.0.2.21\)) Subject: git fetch --all from cgit-beta.freebsd.org stopped working Message-Id: <8F11027D-16C7-450F-836A-9D34B08B071A@iitbombay.org> Date: Wed, 2 Dec 2020 13:15:22 -0800 To: git@freebsd.org X-Mailer: Apple Mail (2.3654.20.0.2.21) X-Rspamd-Queue-Id: 4CmWvz61KSz3pT2 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=iitbombay-org.20150623.gappssmtp.com header.s=20150623 header.b=rpSTzRsL; dmarc=none; spf=pass (mx1.freebsd.org: domain of bakul@iitbombay.org designates 2607:f8b0:4864:20::62c as permitted sender) smtp.mailfrom=bakul@iitbombay.org X-Spamd-Result: default: False [-3.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[iitbombay-org.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::62c:from]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[iitbombay-org.20150623.gappssmtp.com:s=20150623]; FREEFALL_USER(0.00)[bakul]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[git@freebsd.org]; DMARC_NA(0.00)[iitbombay.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::62c:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::62c:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Dec 2020 21:15:28 -0000 The last commit I see is 38dddf6 (Tue Dec 1 15:53:12 2020 +0000). Since then it always fails. git fetch --all -v shows lots and lots of lines like = [up to date] ... -> ... https://github.com/freebsd/freebsd seems to be up to date. I am mirroring https://cgit-beta.freebsd.org/src.git. What is the magic incantation for getting this working again? Thanks! Bakul From owner-freebsd-git@freebsd.org Thu Dec 3 10:47:16 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 647F54A2F25 for ; Thu, 3 Dec 2020 10:47:16 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4Cmswh2PSmz3Nwp for ; Thu, 3 Dec 2020 10:47:16 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 5054B4A2F21; Thu, 3 Dec 2020 10:47:16 +0000 (UTC) Delivered-To: git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 501394A2E25 for ; Thu, 3 Dec 2020 10:47:16 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2a05:fc87:1:5::15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "www.spoerlein.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Cmswh0kcJz3P1y for ; Thu, 3 Dec 2020 10:47:15 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: from localhost (acme.spoerlein.net [IPv6:2a05:fc87:1:5:0:0:0:15]) by acme.spoerlein.net (8.16.1/8.15.2) with ESMTPS id 0B3AlBmC040223 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 3 Dec 2020 11:47:13 +0100 (CET) (envelope-from uqs@freebsd.org) Date: Thu, 3 Dec 2020 11:47:11 +0100 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Bakul Shah Cc: git@freebsd.org Subject: Re: git fetch --all from cgit-beta.freebsd.org stopped working Message-ID: References: <8F11027D-16C7-450F-836A-9D34B08B071A@iitbombay.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <8F11027D-16C7-450F-836A-9D34B08B071A@iitbombay.org> User-Agent: Mutt/2.0.1 (2020-11-14) X-Rspamd-Queue-Id: 4Cmswh0kcJz3P1y X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Dec 2020 10:47:16 -0000 On Wed, 2020-12-02 at 13:15:22 -0800, Bakul Shah wrote: >The last commit I see is 38dddf6 (Tue Dec 1 15:53:12 2020 +0000). >Since then it always fails. > > git fetch --all -v > >shows lots and lots of lines like > > = [up to date] ... -> ... > >https://github.com/freebsd/freebsd seems to be up to date. I am >mirroring https://cgit-beta.freebsd.org/src.git. What is the >magic incantation for getting this working again? > >Thanks! > >Bakul Hi there, those 2 repositories are actually totally unrelated (but soon will merge). Do you see an error when fetching, or just no further updates that you would expect? Please note that cgit-beta updates only about 4-5 times a day and can lag for 5-6h or more. Is that what you're seeing here? hth Uli From owner-freebsd-git@freebsd.org Thu Dec 3 14:49:58 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6A0274A80D7 for ; Thu, 3 Dec 2020 14:49:58 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4CmzJk1MFwz3rKB for ; Thu, 3 Dec 2020 14:49:58 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 2BFCD4A80D6; Thu, 3 Dec 2020 14:49:58 +0000 (UTC) Delivered-To: git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2BC344A805A for ; Thu, 3 Dec 2020 14:49:58 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: from mail-qv1-xf2a.google.com (mail-qv1-xf2a.google.com [IPv6:2607:f8b0:4864:20::f2a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CmzJj3LfYz3rGL; Thu, 3 Dec 2020 14:49:57 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: by mail-qv1-xf2a.google.com with SMTP id cv2so1023698qvb.9; Thu, 03 Dec 2020 06:49:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=QSNBOWWdGHmB9Y8BKDYw1S/JY4agn1jaMQ3tq8foo7M=; b=QZ+RibCuMUYTEwzGoFWzJygevNW30ZBplP7AZCRuT8fFNMaVTREuabYIkwRQzkwjeo RBr722PkJpzdwIaQLli/h84jvqHV+FZ2qg1DaILLUKIh+KM32W6gyv1J0YIaoe7oeA9q w3a7bF4HbGeVHeU5govDCdmdouMhbDckXdqSYUxn2OO3nQzw00urMOzS+rh6/+2qTP6B F8oXTW8L5uuN8qMirUsz5ax2PO5h06ZFj2MunipHEbwpdvr8SFZejQplkz7G/tonetmh Urtl15M8g9fBklVsQoLktTD1QSwwaOwqlrEnCPzwd9C4BHO03wlnTJ/8oX2aot2NQ3mD uHQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:references:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=QSNBOWWdGHmB9Y8BKDYw1S/JY4agn1jaMQ3tq8foo7M=; b=eOPq4PdQtJ7SvDa2iN5E11N50/MVkc1pV/1EbHSCPB5iQqLgtAIUCzBhZBNZQAIZlh 4zT6DvWud6MTjJOOyveH+uSOaZ8ioaUJJDrdBv/d5etIoub3v6TTGVJThxStnffC6OkH FHdVYz9EK2DCN3vOBvaFxAI34OktlJCpBnSaRZS9kPS/orhk3j/acn2BbRWPu+UPHEXn FDjlyDYSwtZ+bN/9Bc8Ld77vdvBlKGIawyV2ttKpDJMK5Il8b5ID8DuCPqlQN5KX8cY/ zNWrPEvA8bnFD6KNVyT4PEPPlXPN6xgjATe1/db0yOWLD4ESFqkRwi+Qui12CkW3GCA5 xpfA== X-Gm-Message-State: AOAM532sodrL/JSbtHx0l6eqo1VP5plGR4Tw1DagaE7E69Zil+UDblY7 lBpkpzXQKBqEdmpSpGosRyY= X-Google-Smtp-Source: ABdhPJydrsObgM1Jku4Th5OtvKlIXehGhzve1A81IAer5ak7Z73x8fs9+tSgA3Oht0qLgFha5/xM9g== X-Received: by 2002:a0c:ba20:: with SMTP id w32mr3479872qvf.50.1607006996580; Thu, 03 Dec 2020 06:49:56 -0800 (PST) Received: from [192.168.222.18] (192-222-183-158.qc.cable.ebox.net. [192.222.183.158]) by smtp.gmail.com with ESMTPSA id l28sm1537898qkl.7.2020.12.03.06.49.55 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 03 Dec 2020 06:49:55 -0800 (PST) Subject: Re: Using git branches for ports (was: Re: converting rmport to git) From: Marc Branchaud To: Warner Losh , =?UTF-8?Q?Ulrich_Sp=c3=b6rlein?= Cc: git@freebsd.org, "portmgr@FreeBSD.org" References: <20201129164707.GA31739@freefall.freebsd.org> <14871125-A032-4980-8DB1-0210E34D5A11@FreeBSD.org> <20201130105337.GA42359@freefall.freebsd.org> <7246FB00-655B-4BD4-BC99-B87E4595969C@FreeBSD.org> <20201201095906.GA50345@freefall.freebsd.org> Message-ID: <4dccb673-7860-b2b4-501d-5c28b60d53c3@gmail.com> Date: Thu, 3 Dec 2020 09:49:54 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4CmzJj3LfYz3rGL X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=QZ+RibCu; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of marcnarc@gmail.com designates 2607:f8b0:4864:20::f2a as permitted sender) smtp.mailfrom=marcnarc@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::f2a:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::f2a:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::f2a:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Dec 2020 14:49:58 -0000 Thanks to Warner and Ulrich for setting me straight. The points you both raised clearly show how my proposal is impractical, so I'm dropping the idea. I believe I understand why the quarterly Ports branches have to be constructed by cherry-picking commits from the main branch. What I find unfortunate is that this practice makes it impossible to use Git's standard tools to answer questions like "Which branches contain this piece of work?" (which seems to be an important question, given what I've read on freebsd-git over the last little while). While it's certainly possible to use Git's notes and/or special commit message lines to build a system on top of Git that provides the desired level of traceability, I'd like to think that would be done as a last resort. It's pretty obvious that y'all have been thinking about how FreeBSD should use Git for quite a while. Is there any record of discussions or evaluations of various Git approaches for FreeBSD that I could read? I'm keen to understand how the project has arrived at its current practices. Thanks for taking the time to indulge my curiosity! M. From owner-freebsd-git@freebsd.org Thu Dec 3 15:45:07 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E33EC4A8DDE for ; Thu, 3 Dec 2020 15:45:07 +0000 (UTC) (envelope-from bakul@iitbombay.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4Cn0XM52FSz3v15 for ; Thu, 3 Dec 2020 15:45:07 +0000 (UTC) (envelope-from bakul@iitbombay.org) Received: by mailman.nyi.freebsd.org (Postfix) id AAEAA4A908D; Thu, 3 Dec 2020 15:45:07 +0000 (UTC) Delivered-To: git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AA9B44A8DDD for ; Thu, 3 Dec 2020 15:45:07 +0000 (UTC) (envelope-from bakul@iitbombay.org) Received: from mail-oo1-xc29.google.com (mail-oo1-xc29.google.com [IPv6:2607:f8b0:4864:20::c29]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Cn0XM4Cvdz3v6D for ; Thu, 3 Dec 2020 15:45:07 +0000 (UTC) (envelope-from bakul@iitbombay.org) Received: by mail-oo1-xc29.google.com with SMTP id t23so614524oov.4 for ; Thu, 03 Dec 2020 07:45:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iitbombay-org.20150623.gappssmtp.com; s=20150623; h=content-transfer-encoding:from:mime-version:subject:date:message-id :references:cc:in-reply-to:to; bh=30rMSGgwWB4LB8JVUgOY82hr+22sQJ6YrWPGJuomU58=; b=VFM5z/2FLAQIchwjg5lb61An/NilvStytjiU5cO5VG3tJZJyjFZXows3U5YIyizkfc T/wl0l3TgPrAWtglf53Nnq2fdupyLb+NbHefk/bJy2+Ea4FbJ/D+Rq7r1QoRmvQ7HlKM 3/iZTbz4Dut17tjJNZ8ZN2FoxEpBHzIqMZ3DTfSXjL9JuDqKU8QBZhfuBxHAoRVGwwHJ u0E5Chqvd2JdBrD+elMfXMb0BzsI6P3tLR0PrpXSdnnDnz1D/YVDslKMXHb9kemOSxrl tGXOO7uqem+Xst5m6x0jKfN5bpLujhyqe2VV5IaX9KRqE21+kzvj9WbiCVBp6PlDdPXf BGKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:from:mime-version :subject:date:message-id:references:cc:in-reply-to:to; bh=30rMSGgwWB4LB8JVUgOY82hr+22sQJ6YrWPGJuomU58=; b=fQihTMQPHD5YRitzEOkDRjjh2OxywPF9FaJscAnw5Sukl76SVlAC6k3pziUDIrj1iC JG7gjfGwBQdoHlENcsHogyY/Wk963afXTdka+B93fZ93lAzVwdMw0rbf/XnCHBSRJCtJ sIDXisGjycVPHe8unZoLiWBw6RZwdLeq2LZdiWkyeKXxRfUUBkdkthwrz38iBCITfeTg 5MJkQORwfps+HpNoqLY0wVLunF3S9nOs0o+Rp3Ua9ugPYvVmPy2sRE/5d07mrDnZ5Je8 U2jrfnDj2K39BXJDNDSHZWBj041lfCN1GuqiXlka6M/31uTkhPIuUyTp6a5MWktZRFYT yN+Q== X-Gm-Message-State: AOAM533u5mvBxm7go5IbRqs3JTZ56ph44bXG7DxZpSC2kStBgRcFBjpD ZjlEqPj+bkBtbQvlfeAnLm7wuan3Mg+PEw== X-Google-Smtp-Source: ABdhPJwkilBseG73kSL9PnV9dGovWepnGw5ibgfHwKOl2YGiJegDILrrBsJ85jwLzZNxLXYCx7gJ4Q== X-Received: by 2002:a4a:e972:: with SMTP id i18mr2502013ooe.17.1607010306164; Thu, 03 Dec 2020 07:45:06 -0800 (PST) Received: from ?IPv6:2600:1700:42f2:310:348d:b362:20ac:d615? ([2600:1700:42f2:310:348d:b362:20ac:d615]) by smtp.gmail.com with ESMTPSA id l1sm298569otj.17.2020.12.03.07.45.05 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 03 Dec 2020 07:45:05 -0800 (PST) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable From: Bakul Shah Mime-Version: 1.0 (1.0) Subject: Re: git fetch --all from cgit-beta.freebsd.org stopped working Date: Thu, 3 Dec 2020 07:45:03 -0800 Message-Id: <2644324C-6AF6-463D-B604-73009CD01739@iitbombay.org> References: Cc: git@freebsd.org In-Reply-To: To: =?utf-8?Q?Ulrich_Sp=C3=B6rlein?= X-Mailer: iPad Mail (18B92) X-Rspamd-Queue-Id: 4Cn0XM4Cvdz3v6D X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Dec 2020 15:45:07 -0000 On Dec 3, 2020, at 2:47 AM, Ulrich Sp=C3=B6rlein wrote: >=20 > Please note that cgit-beta updates only about 4-5 times a day and can lag f= or 5-6h or more. Is that what you're seeing here? It had lagged for more than a day but it is working again. Sorry for the noi= se & Thanks for your response.= From owner-freebsd-git@freebsd.org Thu Dec 3 21:01:11 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2EAC24AF3E7 for ; Thu, 3 Dec 2020 21:01:11 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4Cn7Y30pq6z4lL0 for ; Thu, 3 Dec 2020 21:01:11 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 1B8A84AF916; Thu, 3 Dec 2020 21:01:11 +0000 (UTC) Delivered-To: git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1A2E94AF472 for ; Thu, 3 Dec 2020 21:01:11 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2a05:fc87:1:5::15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "www.spoerlein.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Cn7Y26Fdhz4l6d for ; Thu, 3 Dec 2020 21:01:10 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: from localhost (acme.spoerlein.net [IPv6:2a05:fc87:1:5:0:0:0:15]) by acme.spoerlein.net (8.16.1/8.15.2) with ESMTPS id 0B3L125R077161 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 3 Dec 2020 22:01:03 +0100 (CET) (envelope-from uqs@freebsd.org) Date: Thu, 3 Dec 2020 22:01:02 +0100 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Bakul Shah Cc: git@freebsd.org Subject: Re: git fetch --all from cgit-beta.freebsd.org stopped working Message-ID: References: <2644324C-6AF6-463D-B604-73009CD01739@iitbombay.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <2644324C-6AF6-463D-B604-73009CD01739@iitbombay.org> User-Agent: Mutt/2.0.1 (2020-11-14) X-Rspamd-Queue-Id: 4Cn7Y26Fdhz4l6d X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Dec 2020 21:01:11 -0000 On Thu, 2020-12-03 at 07:45:03 -0800, Bakul Shah wrote: >On Dec 3, 2020, at 2:47 AM, Ulrich Spörlein wrote: >> >> Please note that cgit-beta updates only about 4-5 times a day and can lag for 5-6h or more. Is that what you're seeing here? > >It had lagged for more than a day but it is working again. Sorry for the noise & >Thanks for your response. Hmm, I had to fix some excessive loose objects that tripped up the auto gc runs, but I thought the warning message during git push was just that, a warning, not an error. IOW, it was probably my fault somehow ... Cheers Uli