From owner-freebsd-git@freebsd.org Wed Nov 18 22:31:00 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 330C64739DD for ; Wed, 18 Nov 2020 22:31:00 +0000 (UTC) (envelope-from dan@langille.org) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CbyFZ6hS9z4lfk for ; Wed, 18 Nov 2020 22:30:58 +0000 (UTC) (envelope-from dan@langille.org) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id EB82C5C003F for ; Wed, 18 Nov 2020 17:30:57 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Wed, 18 Nov 2020 17:30:57 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=langille.org; h= from:content-type:content-transfer-encoding:mime-version:subject :message-id:date:to; s=fm2; bh=PHJJn9Jf2kYW1gO0GJ/nuQa7uqH1Hm8JG pNeMfrddn0=; b=XzyDJ9Cv9iV3gvRJwa8+AwrNhZqVpOgKwBxlcDL46wbJWLjNi ZokfuK38ax1i8Ci+pPXyEuC7t5LErI71lkKSF4A5/anPGCNUvPuRC5CtK3dOTXHQ U33fBYi/XkGLd6v0iThpPEDsB17KursiIQkKW9CtwOs8olvhrBZqp1g5mglSrH0G 0225JpXIjla3pLqlcGLVXQCNlx/6HbGjOqfm2zHzumB8IZCX4gTt+A3Cl03CHt69 +3EvyYXxj41k9zerLyoBBWT6gbNAxUpkjhaPKeZuY/2RDb4nn8abzgnzraSVBDNu N4fPdLRIdJ5aTdDW+5/mYn0aa5UkjbgpGnzHg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=PHJJn9 Jf2kYW1gO0GJ/nuQa7uqH1Hm8JGpNeMfrddn0=; b=YqCeEKiUj+zXZLlv20tvfk Iduaen9qnYBoYSJIcZoNmFpbfeioKm2V+vNALU9B8w0PpCAdYMWjdhTyESnQB2OT 5+tmGaAKfrlsZ92sjEUoFvgGy46VHmRJHeoqAISptpFyDOaFdNtnRSixbxBXq8u5 xGIieooMyrJZb8MnU2yipfqgXJNzE4zOCQBrju1Ns0bHS+pcC1DBk73G0QLgN9yd 460YN1Jc0DY1uxFpcMJVdg8tMuyIuI/6Z/FN0jqIuz1nrEbwqzCLj0PDsFPKpNsg Pg65fp4J+XrZ/RcKrZjEG3L2ge+xcaT9ee682YzxoaDEu5AqVrh+DqwKeIW2DLmA == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrudefhedgudeifecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhephfgtgfgguffkfffvofesthhqmh dthhdtvdenucfhrhhomhepffgrnhcunfgrnhhgihhllhgvuceouggrnheslhgrnhhgihhl lhgvrdhorhhgqeenucggtffrrghtthgvrhhnpefgtdefgeduueduvedtteetkedvvdevvd etheeuvdehgfeljeffheetvddtheegveenucffohhmrghinhepghhithhhuhgsrdgtohhm necukfhppeejvddrjeekrdduledtrdduledtnecuvehluhhsthgvrhfuihiivgeptdenuc frrghrrghmpehmrghilhhfrhhomhepuggrnheslhgrnhhgihhllhgvrdhorhhg X-ME-Proxy: Received: from [10.222.78.31] (pool-72-78-190-190.phlapa.fios.verizon.net [72.78.190.190]) by mail.messagingengine.com (Postfix) with ESMTPA id 7FBE7328005D for ; Wed, 18 Nov 2020 17:30:57 -0500 (EST) From: Dan Langille Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.4\)) Subject: checking out a commits on port quarterly branch Message-Id: <5E8AEDD5-327A-46A0-A186-EDBC616A9F89@langille.org> Date: Wed, 18 Nov 2020 17:30:56 -0500 To: freebsd-git@freebsd.org X-Mailer: Apple Mail (2.3608.120.23.2.4) X-Rspamd-Queue-Id: 4CbyFZ6hS9z4lfk X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=langille.org header.s=fm2 header.b=XzyDJ9Cv; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=YqCeEKiU; dmarc=pass (policy=none) header.from=langille.org; spf=pass (mx1.freebsd.org: domain of dan@langille.org designates 66.111.4.29 as permitted sender) smtp.mailfrom=dan@langille.org X-Spamd-Result: default: False [-3.60 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[66.111.4.29:from]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.29]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[langille.org:+,messagingengine.com:+]; DMARC_POLICY_ALLOW(-0.50)[langille.org,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RECEIVED_SPAMHAUS_PBL(0.00)[72.78.190.190:received]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[66.111.4.29:from]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[langille.org:s=fm2,messagingengine.com:s=fm1]; FREEFALL_USER(0.00)[dan]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[66.111.4.29:from:127.0.2.255]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.29:from]; MAILMAN_DEST(0.00)[freebsd-git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2020 22:31:00 -0000 When FreshPorts processes a commit, it needs a working copy of the repo = as it was at the time of that commit. I have this working on head. I am trying to figure out how to do that = when the commit is on a branch. To get a copy of the branch, I do: $ git checkout branches/2020Q4 $ git branch branches/2020Q2 branches/2020Q3 * branches/2020Q4 master Next, I want the tree as it existed at commit = 46433baae934d92698422495b72f811839caa1a9 i.e. = https://github.com/freebsd/freebsd-ports/commit/46433baae934d92698422495b7= 2f811839caa1a9 My first attempt is $ git checkout 46433baae934d92698422495b72f811839caa1a9 Note: switching to '46433baae934d92698422495b72f811839caa1a9'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in = this state without impacting any branches by switching back to a branch. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -c with the switch command. Example: git switch -c Or undo this operation with: git switch - Turn off this advice by setting config variable advice.detachedHead to = false HEAD is now at 46433baae934 MFH: r555565 That "MFH: r555565' message indicates that I am at the right commit. Is this a concern? $ git branch * (HEAD detached at 46433baae934) branches/2020Q2 branches/2020Q3 branches/2020Q4 master Am I doing this right? Thank you --=20 Dan Langille - BSDCan / PGCon dan@langille.org From owner-freebsd-git@freebsd.org Wed Nov 18 23:42:13 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DF144474CEC for ; Wed, 18 Nov 2020 23:42:13 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: from mail-qt1-x836.google.com (mail-qt1-x836.google.com [IPv6:2607:f8b0:4864:20::836]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Cbzqm4Lr0z4pWh for ; Wed, 18 Nov 2020 23:42:12 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: by mail-qt1-x836.google.com with SMTP id g15so3036605qtq.13 for ; Wed, 18 Nov 2020 15:42:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-language:content-transfer-encoding; bh=iInUK31bUK4kWimApSW4Rl1Y9PJHY9kYvnWugNfW39w=; b=LaSuyb05rOXff4o5GRIMoWQM48QF1LeG9LxnAqpqP7nqImKwb4oZTgmNK03SnwIYL5 SoF7wU2G3BXsCdkt9+jfT20kGdx+S1zLN5M+KbBtlylB3BpjVff1/PKrw2LbOGBaoLZw qj5mj9Ir/ntJy9tUHnYbV4m8yRjQjkc/TNhVe6NT+8Z83tiMOnn2CWrBS5n+UIQusQ3B zPCFB4CDH6Tz53uwLKrxeM52kbKscGjRop9VO7JsA6htSDw5DRfKseLrr9ltH6AIUQQw NOPYZzlq89oj9gXWy3rL0FN/bOa+XDd184YzRG+jIwilygR/XjjSzuiWYgUU4o3YAK9B OJ9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=iInUK31bUK4kWimApSW4Rl1Y9PJHY9kYvnWugNfW39w=; b=U5uV65W0zYCU4+A429++kTIBQEO8LhrHaIHz72u7IZD8xllBG9372xF9HpBc5h0nDB JMN8VYBdIf6knQ7PTEl42T2zpgEQXtqjA6M/FL2bqvVgcvX9bikv+xqDilnp7tUuNBT0 +VwjIShT1Gyez+7EQbGz/ZZcKQu8uyfTeRFtMILYSMnnpR5g4KIwuXd027icsSPr72wg mvzBs6wA/faeHRHfKX2Zcr0UNfQcZaY98ul2btiuimiFraOCJeSq5BO9CmPQEDyiI9td lRTBYthqqVjutrQWwx5OeREtXLRbKPYKQxa1DZgYCwnWXYVhmLIezthB/XJTKgZqwPhY lKbw== X-Gm-Message-State: AOAM530WtZ2TTDtXRVXalnsjmL+HQTmMXIpxBkZgYR93/BIAe2eIPEll DWCiX9vuerJ8NKq+QOLxMteeyj3tzRbft2Fc X-Google-Smtp-Source: ABdhPJxukiEcVKiDtGGQlBwwNveHZHmreU7XOF7lJSJxa6hSesgBE3EgbPduEf2XNoIKWgMF+PAHPQ== X-Received: by 2002:ac8:5059:: with SMTP id h25mr7509171qtm.283.1605742931665; Wed, 18 Nov 2020 15:42:11 -0800 (PST) Received: from [192.168.222.18] (192-222-183-158.qc.cable.ebox.net. [192.222.183.158]) by smtp.gmail.com with ESMTPSA id 137sm17584214qkj.109.2020.11.18.15.42.10 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 18 Nov 2020 15:42:11 -0800 (PST) Subject: Re: checking out a commits on port quarterly branch To: Dan Langille , freebsd-git@freebsd.org References: <5E8AEDD5-327A-46A0-A186-EDBC616A9F89@langille.org> From: Marc Branchaud Message-ID: <47863540-fbe8-bd1e-25bd-da17f4ca5d06@gmail.com> Date: Wed, 18 Nov 2020 18:42:09 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: <5E8AEDD5-327A-46A0-A186-EDBC616A9F89@langille.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4Cbzqm4Lr0z4pWh X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=LaSuyb05; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of marcnarc@gmail.com designates 2607:f8b0:4864:20::836 as permitted sender) smtp.mailfrom=marcnarc@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::836:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::836:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::836:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2020 23:42:13 -0000 Hi Dan, You're doing it right, assuming that you're not going to be adding any new work atop that checked-out commit. Your working copy does indeed reflect the code as of commit 46433baae934d92698422495b72f811839caa1a9. I've found that the best way to think of git branches is simply as a label for a particular commit SHA ID. Git has other ways of labeling a commit (notably tags, and you can actually create your own, separate sets of labels if you like). Git internally calls these labels "refs". The special thing about a branch "ref" is that, when you have the branch checked-out and create a new commit, git automatically updates the branch's label to refer to the new commit. But you can also checkout a commit using its SHA ID, as you did. In this case you are not using a branch, so git is warning you that it is not tracking a name for any new commits you might create. Git still lets you do anything you want in this "detached" state, with the caveat that if you make new commits then it's up to you to figure out how to keep track of them (SHA IDs being a bit hard to memorize). BTW, checking out a tag also puts git into the "detached" state, because tags are considered static and so they do not get automatically updated when you add new commits. For regular development it's important to understand when you're in the "detached" state, because it's easy to lose your work. Hence the big warning message (which you can disable). But if you're just looking to build the code at a particular commit, you're fine being "detached" from any actual branch. M. On 2020-11-18 5:30 p.m., Dan Langille wrote: > When FreshPorts processes a commit, it needs a working copy of the repo as it was at the time of that commit. > > I have this working on head. I am trying to figure out how to do that when the commit is on a branch. > > To get a copy of the branch, I do: > > $ git checkout branches/2020Q4 > $ git branch > branches/2020Q2 > branches/2020Q3 > * branches/2020Q4 > master > > Next, I want the tree as it existed at commit 46433baae934d92698422495b72f811839caa1a9 > > i.e. https://github.com/freebsd/freebsd-ports/commit/46433baae934d92698422495b72f811839caa1a9 > > My first attempt is > > $ git checkout 46433baae934d92698422495b72f811839caa1a9 > Note: switching to '46433baae934d92698422495b72f811839caa1a9'. > > You are in 'detached HEAD' state. You can look around, make experimental > changes and commit them, and you can discard any commits you make in this > state without impacting any branches by switching back to a branch. > > If you want to create a new branch to retain commits you create, you may > do so (now or later) by using -c with the switch command. Example: > > git switch -c > > Or undo this operation with: > > git switch - > > Turn off this advice by setting config variable advice.detachedHead to false > > HEAD is now at 46433baae934 MFH: r555565 > > That "MFH: r555565' message indicates that I am at the right commit. > > Is this a concern? > > $ git branch > * (HEAD detached at 46433baae934) > branches/2020Q2 > branches/2020Q3 > branches/2020Q4 > master > > > Am I doing this right? > > Thank you > > From owner-freebsd-git@freebsd.org Thu Nov 19 01:49:51 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 22F3347A39F for ; Thu, 19 Nov 2020 01:49:51 +0000 (UTC) (envelope-from dan@langille.org) Received: from wout1-smtp.messagingengine.com (wout1-smtp.messagingengine.com [64.147.123.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Cc2g203Wmz3GNJ for ; Thu, 19 Nov 2020 01:49:49 +0000 (UTC) (envelope-from dan@langille.org) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 04FF8A03 for ; Wed, 18 Nov 2020 20:49:47 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Wed, 18 Nov 2020 20:49:48 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=langille.org; h= from:content-type:content-transfer-encoding:mime-version:subject :message-id:date:to; s=fm2; bh=KCe+Cs3XF7C1POUpNCTnLEMnXUZ/AtI8o oMysJdYfRs=; b=QjqSddUqN4njMD1SYCh47uy0B1H1wpBc9RRPZ9GoDuOkT8Dbh FKeL7MK+jJn0e/qpYHWOxSJpPUdim+SzUnbQ3Id58nhEX9it+cvrDPlVZCGIb88R uQaJbgYPswvveEwC/LaP/Xp/UsX3PEfk5ne5ZrbIrsvaxmYgvbeRaH4cZrNKNnCa TwNgk2X0KiG4ADm2M0QzDMwqu/dFY5CIYWa/2y/FOs6xkGO5B+i8JyGFRRiGbrMT acmXLONX1LnNzRmN71NZNNWezmzso+JzomHZKDB+90W+3qLUr73WlcIT7Rq2Uvnx E4GlInfMdBgVPs2Pt8Mlwn/bNo2s0PtQ5fs+w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=KCe+Cs 3XF7C1POUpNCTnLEMnXUZ/AtI8ooMysJdYfRs=; b=IpK8etA6Wmtr9mGO8oZq07 6AvS9Cr9vKS9vQjm2Ii8DwuLoQX2lZVj28oYABkUSHZS0UZVwmeCxSJ5ATuyvwaF b5O6EA1G8Vf5fBCmlM9OXMHmWUTra24GKbruPviVg5lhOW21MnaNb16vqiE9Xoqk G9WSFdQ1JVvvYXZ6oCjead7nf4KxKQ/vm7KPMqK2eYSG6zKCpFqTzlIBXo9o6WeA foitd2GwTQXwzHQI3YwCBoR1zIcM7T5otyS4UDMOcY7e7ZF5W20w/2uOkl/rcca1 XjvCHuwsS5GVCBMaG0fKmsHQ2IdpsaVFvE3rB3N8KH3BbWpv6uFGiJzF/9ZKcU/Q == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrudefiedgfeekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhtgfgggfukfffvffosehtqhhmtd hhtddvnecuhfhrohhmpeffrghnucfnrghnghhilhhlvgcuoegurghnsehlrghnghhilhhl vgdrohhrgheqnecuggftrfgrthhtvghrnhepkeevteelkeevkeetfeehgeeufeffjeelvd fgtdfhgefffefgfeefuefhhffgkefhnecukfhppeejvddrjeekrdduledtrdduledtnecu vehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepuggrnheslh grnhhgihhllhgvrdhorhhg X-ME-Proxy: Received: from [10.222.78.31] (pool-72-78-190-190.phlapa.fios.verizon.net [72.78.190.190]) by mail.messagingengine.com (Postfix) with ESMTPA id 3E63E328005A for ; Wed, 18 Nov 2020 20:49:47 -0500 (EST) From: Dan Langille Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.4\)) Subject: Monitoring commits on all branches Message-Id: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> Date: Wed, 18 Nov 2020 20:49:46 -0500 To: freebsd-git@freebsd.org X-Mailer: Apple Mail (2.3608.120.23.2.4) X-Rspamd-Queue-Id: 4Cc2g203Wmz3GNJ X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=langille.org header.s=fm2 header.b=QjqSddUq; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=IpK8etA6; dmarc=pass (policy=none) header.from=langille.org; spf=pass (mx1.freebsd.org: domain of dan@langille.org designates 64.147.123.24 as permitted sender) smtp.mailfrom=dan@langille.org X-Spamd-Result: default: False [-3.60 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.24]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[langille.org:+,messagingengine.com:+]; DMARC_POLICY_ALLOW(-0.50)[langille.org,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RECEIVED_SPAMHAUS_PBL(0.00)[72.78.190.190:received]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11403, ipnet:64.147.123.0/24, country:US]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.24:from]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[langille.org:s=fm2,messagingengine.com:s=fm1]; FREEFALL_USER(0.00)[dan]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; RWL_MAILSPIKE_VERYGOOD(0.00)[64.147.123.24:from]; MAILMAN_DEST(0.00)[freebsd-git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 01:49:51 -0000 How can a repo be monitored for commits on all branches? I know how to ask a given branch: do you have any commits after = foo_hash? How do I: * get a list of all commits since foo_hash * know which branch each of those commits was on (e.g. master, = branches/2020Q4) Thanks. --=20 Dan Langille - BSDCan / PGCon dan@langille.org From owner-freebsd-git@freebsd.org Thu Nov 19 02:18:22 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EFF9847B2E2 for ; Thu, 19 Nov 2020 02:18:22 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x82d.google.com (mail-qt1-x82d.google.com [IPv6:2607:f8b0:4864:20::82d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Cc3Hw45Xhz3J7Q for ; Thu, 19 Nov 2020 02:18:20 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x82d.google.com with SMTP id g20so3351037qtu.4 for ; Wed, 18 Nov 2020 18:18:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=NyVshYQnEse9z4E04cMiGRusgHugg4LhVFtxQRtPIi8=; b=Peuc8NWHE+ihng9/lazRhENkxvzAPljRJhKqy5IoevNsdXrZAofwM1L7kTNp3Xu45x 90+Rp4wS+dcnTUkH1ZpweY7/csOSJBLep2SkwwOqJHTdgb+6ByEYur8PHGUzAFWMkhNo YX9xat0fxRIc4S9aXHCpU/2K1kce5c0Daqm8hDixGN2jKo4D9fgybJ/jqcoYSypRPkVZ w566DDTRQkJuNnHxCI8ghGXCn+pbYIrnvelwECoy/jUFVb2pfaANsgR7s6308I0SYKkZ xYUAEZUeOFD1ojNAseV8U6M3saHVGSIx6UukEbJX04edxYhUs4wyKUKLthG0xB99kEfe 0afQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=NyVshYQnEse9z4E04cMiGRusgHugg4LhVFtxQRtPIi8=; b=cmH9gzr/RYc5F5yjfwwi9/ZIE8tsHq1GPxY0iNpWo8y87hEC4On34JkhRnmjw39Fcb rK8LIJam+HsKN8ZGYdGIEWO0e64X+u8EM7wAs1SFORmP+hNtuyM1g+p9R8XukU2mHta6 g4ErUSjBPqZIifaxAtXqVcRMwPxRlbzuRlTpqqMaplS8T4ag5RU2XqQZQ/uosEsIspl5 AIZye0nvDXY06oQTLbjxYkSbrf8ydLpNbvNxeN4AqIAGsI6UnBEVRov7Tp794zbZIK/G N+9cWOGvM362ZWlgDU6KJ6YP/Ogd9BmeZKvY3dcEYHb8pAv35DgEDqVO7IuoEAJzULus uSmA== X-Gm-Message-State: AOAM5336ZwASsZvFPidzaZOo0UUzOI9O1XeEQ2K75DERchu3FEgIa3RC 8udCECiCdvSD4UwF3fjwbzk6jArLsIweGhC+7QEFKOpvLvk7tQ== X-Google-Smtp-Source: ABdhPJxkEpKPqf2PlW1g+EDeiIQ7rBkGu+pn9aaQziWIDVZA1wHyBZ7vVaY/YE8Wn7axKZBAKk//WSl0X1l8bWVblD0= X-Received: by 2002:ac8:5351:: with SMTP id d17mr8490791qto.235.1605752299325; Wed, 18 Nov 2020 18:18:19 -0800 (PST) MIME-Version: 1.0 References: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> In-Reply-To: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> From: Warner Losh Date: Wed, 18 Nov 2020 19:18:07 -0700 Message-ID: Subject: Re: Monitoring commits on all branches To: Dan Langille Cc: freebsd-git@freebsd.org X-Rspamd-Queue-Id: 4Cc3Hw45Xhz3J7Q X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=Peuc8NWH; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::82d) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-1.02 / 15.00]; ARC_NA(0.00)[]; MAILMAN_DEST(0.00)[freebsd-git]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.98)[0.983]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::82d:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::82d:from]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::82d:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 02:18:23 -0000 On Wed, Nov 18, 2020, 6:49 PM Dan Langille wrote: > How can a repo be monitored for commits on all branches? > > I know how to ask a given branch: do you have any commits after foo_hash? > > How do I: > > * get a list of all commits since foo_hash > git log $hash..HEAD will list all the commits on the branch. Between branches 'since' has little to no meaning. * know which branch each of those commits was on (e.g. master, > branches/2020Q4) > You need to know which branches are interesting. There are some git log options... --date-order might help since dates are weird in git... All of this is a consequence of git being distributed. Warner Thanks. > > -- > Dan Langille - BSDCan / PGCon > dan@langille.org > > > _______________________________________________ > freebsd-git@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-git > To unsubscribe, send any mail to "freebsd-git-unsubscribe@freebsd.org" > From owner-freebsd-git@freebsd.org Thu Nov 19 08:54:00 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7C1FD2EBBEB for ; Thu, 19 Nov 2020 08:54:00 +0000 (UTC) (envelope-from mat@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CcD4S33hfz3v9j; Thu, 19 Nov 2020 08:54:00 +0000 (UTC) (envelope-from mat@freebsd.org) Received: from mail.j.mat.cc (owncloud.cube.mat.cc [IPv6:2a01:678:4:1::228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.mat.cc", Issuer "Let's Encrypt Authority X3" (verified OK)) (Authenticated sender: mat/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4099F164D; Thu, 19 Nov 2020 08:54:00 +0000 (UTC) (envelope-from mat@freebsd.org) Received: from aching.in.mat.cc (unknown [IPv6:2a01:678:42:0:2e66:df0e:5de6:b4de]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: mat@mat.cc) by mail.j.mat.cc (Postfix) with ESMTPSA id 57E08942D84; Thu, 19 Nov 2020 08:53:57 +0000 (UTC) Date: Thu, 19 Nov 2020 09:52:23 +0100 From: Mathieu Arnold To: Dan Langille Cc: freebsd-git@freebsd.org Subject: Re: Monitoring commits on all branches Message-ID: <20201119085223.nsrp2e45oyepsvds@aching.in.mat.cc> References: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="qoacccmzawas6bhq" Content-Disposition: inline In-Reply-To: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 08:54:00 -0000 --qoacccmzawas6bhq Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Nov 18, 2020 at 08:49:46PM -0500, Dan Langille wrote: > How can a repo be monitored for commits on all branches? >=20 > I know how to ask a given branch: do you have any commits after foo_hash? >=20 > How do I: >=20 > * get a list of all commits since foo_hash All commits on the branch foo_hash is: git log $foo_hash...branch_name > * know which branch each of those commits was on (e.g. master, branches/2= 020Q4) You will need to keep track of the latest commit on each branch separately, because there is absolutely no relation or ordering possible between branches. You could be tempted to use `--since `, but you cannot, because as git is distributed, commit dates are by no mean increasing as you get to the tip of the branch. For example, I work on something on a private branch, commit it, and open a review, a few days pass, the review gets accepted, and I then rebase my one commit branch on the main branch and git push. The latest commit on that branch will be mine, which has a date a few days in the past, the previous commit probably has a date a few hours ago. --=20 Mathieu Arnold --qoacccmzawas6bhq Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAABCgB9FiEEVhwchfRfuV0unqO5KesJApEdfgIFAl+2MkJfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDU2 MUMxQzg1RjQ1RkI5NUQyRTlFQTNCOTI5RUIwOTAyOTExRDdFMDIACgkQKesJApEd fgIOpA/9F+S267282jm5lt7yS82Ob1NlR/8zR8WoQHxXSrd1eriTU4ie56e23PAC N34Rb6+55A8mldAQvKNLNgzkXb3TqY62JiunXNbk72cz/dqmyMfJBbGAGJKxfbxG +ia1PQB66S9pmlsGH2gZCATAo3sRsHJ548M73HzdYc7/LBj/O4H2L9mQzAMk7LmI SvKGd3ytT+ABEmJEnhWEt5wCVZ8esmFtCeJK3gMXQCBeQsFbN17Yc9P+WqoY6ShC ineiQsjZ5pviIH52hCPqoXyByLXUMHHKqBTWpQHfauVjmsCFwOhM29AHmNcrv87m ocnyTXyVsqIm88iSsutp3+mQphtOOdjGff7KFML9r2eesVTQUANliivIdj/DrNqq S7+ePNMV4WUd4c1zt5E+dSBcaVQuwTlQk5ZBsTEQHW9iGe+l/wMYWSwUHidq0bxC A8zGXouvanpPP31q+bwUmaEGZYJ7w9EjOBCQx8RTlUV2t35Rwjf/5GMAcIO6rjhC bc3yJBOp+rQAw5aGbYFJB8ZORx658AQ0/rCny+cLk7/ibUzYd3zDl/EnoETgSQMp sBfBuNbh1P2W0rgNLCLzEjlCer2VluCx0XcNjeKM/5MedZB0D+TXPySaqSEMBW2v 2A81UAilVeL0V7SfnLZ+L7PU1zR3g7XpIB99ntMy2UYpofki+aw= =3JPV -----END PGP SIGNATURE----- --qoacccmzawas6bhq-- From owner-freebsd-git@freebsd.org Thu Nov 19 09:34:41 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CB0982ECED3 for ; Thu, 19 Nov 2020 09:34:41 +0000 (UTC) (envelope-from david@isnic.is) Received: from mx01.isnic.is (mx01.isnic.is [IPv6:2001:67c:6c:58::133]) by mx1.freebsd.org (Postfix) with ESMTP id 4CcDzN2b2Lz4RQZ; Thu, 19 Nov 2020 09:34:40 +0000 (UTC) (envelope-from david@isnic.is) Received: from localhost (wg-client01.isnic.is [185.93.159.98]) by mx01.isnic.is (Postfix) with ESMTPS id 176AA2994B; Thu, 19 Nov 2020 09:34:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=isnic.is; s=20200921; t=1605778471; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=5Kddx/Ykegs3VFuK1+L/BHKwWyRjA0aXa801CzZ7uFk=; b=seREbf0oYFs7bbP+hh6CbOJ4SDIebSOGBN7plhOxCdNZUvCanHaIKReZbxV8ohGEzkZN9V t9OQDWl5qMW6Ihb1VkfWusVshJsKXQ8vdytdcpUamgDO+5GTHE/6cVUh4JH/VX1ZTrn5CG 5ZNdOMN9FENPCVIRkMf98APdJsM/9sh/Aljek6Ntd1rUQUiGBrZxzpHuZERWgKbrGaEkU0 Ii8atAhgomlCk8gdTLrpk+vJgKOf6L8AXPDvzbGjVArS8l2sab1tYmUjFvXEN1YkQK7pqL MiwLFmeYuylJK9/fuJm1ofJ8JnJF6LqHwS72qZkH+SoLx8225CIKpYPUYDNBOA== Date: Thu, 19 Nov 2020 09:34:30 +0000 From: =?iso-8859-1?B?RGF27fA=?= Steinn Geirsson To: Li-Wen Hsu Cc: Yasuhiro KIMURA , freebsd-git@freebsd.org Subject: Re: 504 errors from cgit-beta Message-ID: <20201119093429.GG895@mail> References: <20201112155659.GQ913@mail> <20201113.032709.2108746957258946268.yasu@utahime.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="Qf1oXS95uex85X0R" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Rspamd-Queue-Id: 4CcDzN2b2Lz4RQZ X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=isnic.is header.s=20200921 header.b=seREbf0o; dmarc=none; spf=pass (mx1.freebsd.org: domain of david@isnic.is designates 2001:67c:6c:58::133 as permitted sender) smtp.mailfrom=david@isnic.is X-Spamd-Result: default: False [-3.89 / 15.00]; MID_RHS_NOT_FQDN(0.50)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:67c:6c:58::133:from]; R_DKIM_ALLOW(-0.20)[isnic.is:s=20200921]; FREEFALL_USER(0.00)[david]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2001:67c:6c::/48]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; DMARC_NA(0.00)[isnic.is]; ARC_NA(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; SPAMHAUS_ZRD(0.00)[2001:67c:6c:58::133:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[isnic.is:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; SIGNED_PGP(-2.00)[]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; R_MIXED_CHARSET(1.11)[subject]; ASN(0.00)[asn:1850, ipnet:2001:67c:6c::/48, country:EU]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 09:34:41 -0000 --Qf1oXS95uex85X0R Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Nov 13, 2020 at 05:33:12PM +0800, Li-Wen Hsu wrote: > On Fri, Nov 13, 2020 at 2:28 AM Yasuhiro KIMURA wrote: > > > > From: Dav=ED=F0 Steinn Geirsson > > Subject: 504 errors from cgit-beta > > Date: Thu, 12 Nov 2020 15:56:59 +0000 > > > > > We are getting frequent 504 errors when running `git fetch` against an > > > existing checkout of `ports.git` from https://cgit-beta.freebsd.org/p= orts.git: > > > > > > ``` > > > $ git fetch cgit-beta > > > error: RPC failed; HTTP 504 curl 22 The requested URL returned error:= 504 > > > fatal: the remote end hung up unexpectedly > > > ``` > > > > I experienced same error when accessing Emacs git remository with > > HTTPS. Following is bug report that I submitted to report the issue. > > > > https://savannah.nongnu.org/support/?110322 > > > > As you can see, site administrator fixed the issue by icreasing > > `fastcgi_read_timeout` and `proxy_read_timeout` parameters of > > nginx. Since cgit-beta also uses nginx this may also fix your > > error. In my case, however, access always failed and never > > succeeded. So cause may be different from the one of my case. >=20 > Thanks, I have checked this, indeed some requests' handlers don't have > a long enough timeout setting and I've relaxed them. Hope this solves > some people's issues. Please check it again, and if it still fails for > you, we might need to have more information to debug. Sorry for the late reply, I can confirm we have not seen this issue since this change was made. Thanks! >=20 > Li-Wen > _______________________________________________ > freebsd-git@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-git > To unsubscribe, send any mail to "freebsd-git-unsubscribe@freebsd.org" -Dav=ED=F0 --Qf1oXS95uex85X0R Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAABCgAdFiEEvylfYbt7o3c60Grm/+HlKLuPmJoFAl+2PCUACgkQ/+HlKLuP mJpslgf/dFY4XoE/QFT1u4qcHSSfcFOvbPmQFQLxyf7pXUZbPhzm6Q+eFuzm1b/B mUeb/9CU7DNLNNpzYdaFzxJ/+4MyZVOC88RJ1Z3TIvkuZr8f7PqQxw/5CGjZl7VX 6jHxrlHYv9/pvf3nwsIqVKmAdpUcsBmMXz3l2cuDiDvrvIZhP4wQjwaUiIRwJKHX /hbJt8ANXRjW+leJZGemAyV4QI0hsvj460NvOmEDSHfsDcR7+MWT19gMkO2gVUPR NHNjxXRE9qgWV0gfxEDLF8MBx+q+eCqAw4VzDAAxEzx3t8z2BFUH5HC9O96P40PL 2+gdR+mNcQQ7wVySXUynxhVGzqUulA== =s6lu -----END PGP SIGNATURE----- --Qf1oXS95uex85X0R-- From owner-freebsd-git@freebsd.org Thu Nov 19 13:03:52 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C8E0546B41B for ; Thu, 19 Nov 2020 13:03:52 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2a05:fc87:1:5::15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "www.spoerlein.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CcKcm1Dn8z4dxf for ; Thu, 19 Nov 2020 13:03:51 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: from localhost (acme.spoerlein.net [IPv6:2a05:fc87:1:5:0:0:0:15]) by acme.spoerlein.net (8.16.1/8.15.2) with ESMTPS id 0AJD3fMt085225 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 19 Nov 2020 14:03:42 +0100 (CET) (envelope-from uqs@freebsd.org) Date: Thu, 19 Nov 2020 14:03:41 +0100 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Marc Branchaud Cc: Dan Langille , freebsd-git@freebsd.org Subject: Re: checking out a commits on port quarterly branch Message-ID: References: <5E8AEDD5-327A-46A0-A186-EDBC616A9F89@langille.org> <47863540-fbe8-bd1e-25bd-da17f4ca5d06@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <47863540-fbe8-bd1e-25bd-da17f4ca5d06@gmail.com> User-Agent: Mutt/2.0.1 (2020-11-14) X-Rspamd-Queue-Id: 4CcKcm1Dn8z4dxf X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:39540, ipnet:2a05:fc87::/32, country:CH] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 13:03:52 -0000 Just to add, losing work in git is actually very hard. While it's true that committing to a detached head doesn't track the new commits in a permanent way, you can actually use `git reflog` to see everything that you checked out for the last 90d or so. To me, resetting the index is a more scary proposition, because I've still not wrapped my head around how that works and what --hard does, etc. Cheers Uli On Wed, 2020-11-18 at 18:42:09 -0500, Marc Branchaud wrote: >Hi Dan, > >You're doing it right, assuming that you're not going to be adding any >new work atop that checked-out commit. Your working copy does indeed >reflect the code as of commit 46433baae934d92698422495b72f811839caa1a9. > >I've found that the best way to think of git branches is simply as a >label for a particular commit SHA ID. Git has other ways of labeling a >commit (notably tags, and you can actually create your own, separate >sets of labels if you like). Git internally calls these labels "refs". > The special thing about a branch "ref" is that, when you have the >branch checked-out and create a new commit, git automatically updates >the branch's label to refer to the new commit. > >But you can also checkout a commit using its SHA ID, as you did. In >this case you are not using a branch, so git is warning you that it is >not tracking a name for any new commits you might create. Git still >lets you do anything you want in this "detached" state, with the caveat >that if you make new commits then it's up to you to figure out how to >keep track of them (SHA IDs being a bit hard to memorize). BTW, >checking out a tag also puts git into the "detached" state, because tags >are considered static and so they do not get automatically updated when >you add new commits. > >For regular development it's important to understand when you're in the >"detached" state, because it's easy to lose your work. Hence the big >warning message (which you can disable). But if you're just looking to >build the code at a particular commit, you're fine being "detached" from >any actual branch. > > M. > > >On 2020-11-18 5:30 p.m., Dan Langille wrote: >> When FreshPorts processes a commit, it needs a working copy of the repo as it was at the time of that commit. >> >> I have this working on head. I am trying to figure out how to do that when the commit is on a branch. >> >> To get a copy of the branch, I do: >> >> $ git checkout branches/2020Q4 >> $ git branch >> branches/2020Q2 >> branches/2020Q3 >> * branches/2020Q4 >> master >> >> Next, I want the tree as it existed at commit 46433baae934d92698422495b72f811839caa1a9 >> >> i.e. https://github.com/freebsd/freebsd-ports/commit/46433baae934d92698422495b72f811839caa1a9 >> >> My first attempt is >> >> $ git checkout 46433baae934d92698422495b72f811839caa1a9 >> Note: switching to '46433baae934d92698422495b72f811839caa1a9'. >> >> You are in 'detached HEAD' state. You can look around, make experimental >> changes and commit them, and you can discard any commits you make in this >> state without impacting any branches by switching back to a branch. >> >> If you want to create a new branch to retain commits you create, you may >> do so (now or later) by using -c with the switch command. Example: >> >> git switch -c >> >> Or undo this operation with: >> >> git switch - >> >> Turn off this advice by setting config variable advice.detachedHead to false >> >> HEAD is now at 46433baae934 MFH: r555565 >> >> That "MFH: r555565' message indicates that I am at the right commit. >> >> Is this a concern? >> >> $ git branch >> * (HEAD detached at 46433baae934) >> branches/2020Q2 >> branches/2020Q3 >> branches/2020Q4 >> master >> >> >> Am I doing this right? >> >> Thank you >> >> >_______________________________________________ >freebsd-git@freebsd.org mailing list >https://lists.freebsd.org/mailman/listinfo/freebsd-git >To unsubscribe, send any mail to "freebsd-git-unsubscribe@freebsd.org" From owner-freebsd-git@freebsd.org Thu Nov 19 16:16:37 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CF8CE46F17D for ; Thu, 19 Nov 2020 16:16:37 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: from mail-qt1-x82b.google.com (mail-qt1-x82b.google.com [IPv6:2607:f8b0:4864:20::82b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CcPv90Ywlz4qkX for ; Thu, 19 Nov 2020 16:16:36 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: by mail-qt1-x82b.google.com with SMTP id g17so4754263qts.5 for ; Thu, 19 Nov 2020 08:16:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-language:content-transfer-encoding; bh=wTiU34XArvVx3zKxyfcCcBjxrOgZBX3pp9QXgBOUJKo=; b=OBggC8lIuCfFzVy4EWbHzweGd9r3F46jMZAj2B09k4WPTW0JNjyR8hHODdlO/jm0bX zLnputAwmU9TjEB+zU62wJjGn/qYEh4y9SfE/oFh/+Fgue3eL0r1fCwMuJzS5w7fwaNC M5HQ8Q3jmvxaaKBmdkF7SP/SFYW1irBZITTHNy7qEoKmOSL3cIJOhCNe8hd/rcfVyT1l HzjhJ0j83rax9Nq2HyPXjlKilLSQMgBKZu8PmWf9MxJHeXIi9lZOrdDIB4IpJxAnjIkQ aby9ufFA6XHLVo1KcgfIs2wP7/jCPumXPzymhGpnyOWtUNvp6JDDFnkLgGwCAgg8h9tN tJKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=wTiU34XArvVx3zKxyfcCcBjxrOgZBX3pp9QXgBOUJKo=; b=VCPQQsMdoVmAtoM0YR5RxYYcrNBz8bDRZ/Q9MJSfysWFaRDv74JCUU/MDaN6v3+5g/ 47pOwp9pvk4Zvn/ddcNDHbFADInn299uQfVIO13MKkG3q2ej9uV2DQFSamUf1haJq2H1 fh0nQKPipSjBm4F9W39T5CQ2hTG7XZpSMbUreiwgHU9SVwWPPfClhOfSjPFO85q2JP03 s4/FkIgDeQT8WieZUHgfqQKGtcHlnh6zmAbNy8aX5qdEuHP/Q2vlYhebzXYbSTfgk5Ju geNI0ZMheKg5nBmS/vQd2sU8LmSquvauAsOP6zUY8dTH3YjxZlSQDVS69iasf9mZHc3y Ap3Q== X-Gm-Message-State: AOAM531CyyfPXzPj4XF2JGarv8H+m+6ZRp5ze7+NIBMQtrotnAWLAeqA 6dNQjwka0tYa6uM9MKqlbcY= X-Google-Smtp-Source: ABdhPJwRePPzvzW6ncK1d+GLHbk33hEO8P599ZPnu3Q+X4FNRz4LOd/ibOIluhIQrhJ2CnMFheglMw== X-Received: by 2002:ac8:5351:: with SMTP id d17mr11461585qto.235.1605802595864; Thu, 19 Nov 2020 08:16:35 -0800 (PST) Received: from [192.168.222.18] (192-222-183-158.qc.cable.ebox.net. [192.222.183.158]) by smtp.gmail.com with ESMTPSA id g9sm192102qtq.21.2020.11.19.08.16.35 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 19 Nov 2020 08:16:35 -0800 (PST) Subject: Re: Monitoring commits on all branches To: Dan Langille , freebsd-git@freebsd.org References: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> From: Marc Branchaud Message-ID: <3c9f6285-ae7c-1062-2dd3-42f8c953a230@gmail.com> Date: Thu, 19 Nov 2020 11:16:34 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4CcPv90Ywlz4qkX X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=OBggC8lI; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of marcnarc@gmail.com designates 2607:f8b0:4864:20::82b as permitted sender) smtp.mailfrom=marcnarc@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::82b:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::82b:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::82b:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 16:16:37 -0000 On 2020-11-18 8:49 p.m., Dan Langille wrote: > How can a repo be monitored for commits on all branches? > > I know how to ask a given branch: do you have any commits after foo_hash? > > How do I: > > * get a list of all commits since foo_hash A quick a note about Warner's reply: > git log $hash..HEAD "HEAD" is just a git nickname for "whatever you have currently checked-out" (which can be a branch, a tag, or "detached" commit SHA ID). > * know which branch each of those commits was on (e.g. master, branches/2020Q4) Unfortunately you'll find most normal git advice to be a bit frustrating with the FreeBSD repos, because FreeBSD doesn't work the way most people use git. Specifically, the FreeBSD project does not ever merge branches (in the git sense of the word "merge"). Things would be very, very much easier if the FreeBSD project were to use git-style merging. I believe there are discussions underway about adjusting the whole MFC process for the git world. I admit that part of my motivation in writing this message is to provide grist for that mill. Fortunately even without git-merged branches, there are still git tools that help, though they're not as precise as one would like. Let's look at a concrete example with the beta ports git repo (which I just cloned), and compare the 2020Q4 and main branches. I'll start with some overall exploration, then address your specific question. There are 298 commits in the 2020Q4 branch. I know this because git merge-base origin/main origin/branches/2020Q4 tells me where 2020Q4 branched off of main: commit 5dbe4e5f775ea2. And git rev-list 5dbe4e5f775ea2..origin/branches/2020Q4 | wc -l says "299". (The "rev-list" command is a bare-bones version of "log" that only lists commit SHA IDs.) Meanwhile there have been 4538 commits to the main branch since commit 5dbe4e5f775ea2. As far as git is concerned, those 299 commits in 2020Q4 are *different* from anything in main. Even though most of them made the exact same code changes, they were created at different times, often by different authors, and they have different commit messages. But you can still ask git to look at the code-change level to see which 2020Q4 commits exactly replicated the code change from main: git cherry -v origin/main origin/branches/2020Q4 This little piece of magic looks at the 299 commits in 2020Q4 that are not in main and compares their code changes to the 4538 commits in main that are not in 2020Q4. It prints out the 299 2020Q4 commit SHA IDs, prefixed with either a "- " or a "+ ". The -v appends the commit message's first line: - 394d9746e5eea73f56334b2e7ddbdc8f686d6541 MFH: r550869 + 1ac9571956759c91d852ee92859a12e52dcbde48 MFH: r550885 r550886 - fd411bdfda55488b84de75e6b043c513a281abf0 MFH: r551209 - 533cdaa97457b3318aebcc53f7a1a46ea66721da MFH: r551236 ...... A "-" means that the commit matches the code change made by a commit in main, while a "+" means that the commit's code change does not *exactly* match any main commit since commit 5dbe4e5f775ea2. So git cherry -v origin/main origin/branches/2020Q4 | grep ^- shows us the 234 2020Q4 commits that made the exact same change as a commit in main. And git cherry -v origin/main origin/branches/2020Q4 | grep ^+ shows us that there are 41 not-exactly-the-same-change commits in 2020Q4. Mostly these are ones that combined two or more MFH's into one commit (e.g. 2020Q4 commit 1ac95719567), or that changed a file in a slightly different way (see the first patch hunk of 2020Q4 commit cbd002878f2, compared to its counterpart in main: commit a5d21ea16b6). Now to your specific question: Given a commit, how can we tell which branches contain that code change? Let's look at main commit 6a9a8389d609 which I've determined, through manual spelunking, matches 2020Q4's commit 02eba4048564. At a basic level, "git cherry" can tell us that *something* in 2020Q4 made the same change as commit 6a9a8389d609. Here I reversed the order of the branch names in the command: git cherry origin/branches/2020Q4 origin/main | grep 6a9a8389d609 This outputs: - 6a9a8389d609ca0370c8c6eb8f993c1aa4071681 and the "-" tells me that 6a9a8389d609's code change is *somewhere* in 2020Q4 unique 299 commits. Unfortunately there's no convenient git command that'll tell you *which* 2020Q4 commit replicated commit 6a9a8389d609. For that, we need to do a bit of scripting: -----8<-----8<-----8<-----8<----- #!/bin/sh TARGET="6a9a8389d609" BASE=`git merge-base origin/branches/2020Q4 origin/main` TARGET_PATCH_ID=`git show -p $TARGET | git patch-id --stable | cut -f 1 -d ' '` for REV in `git rev-list $BASE..origin/branches/2020Q4`; do PATCH_ID=`git show -p $REV | git patch-id --stable | cut -f 1 -d ' '` if [ "$PATCH_ID" = "$TARGET_PATCH_ID" ]; then echo "Found a commit that replicated target commit $TARGET:" echo git show -s $REV exit 0 fi done echo "Did not find any commit that exactly replicated $TARGET." exit 1 ----->8----->8----->8----->8----- This only looks at the 2020Q4 branch, but it's easily adapted to look at a user-specified branch, or multiple branches. (In the above I used "git patch-id", which is what "git cherry" uses internally to identify a commit's code changes.) I hope all this helps a bit! M. From owner-freebsd-git@freebsd.org Thu Nov 19 17:16:39 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A07584703BC for ; Thu, 19 Nov 2020 17:16:39 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x72a.google.com (mail-qk1-x72a.google.com [IPv6:2607:f8b0:4864:20::72a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CcRDQ3Bbdz4v3d for ; Thu, 19 Nov 2020 17:16:38 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x72a.google.com with SMTP id l2so6145639qkf.0 for ; Thu, 19 Nov 2020 09:16:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=uzWEIlxFYrhYaCwJ81Z4ASNBiShYXTeOYWcDsrSpw5I=; b=arrFuQu/kSzZntOaJRsOGK/2nTSz3fufBGH1i24GvhmzMuppS9lNio5fNXj+xz6958 7vFj/3fysMUq7hN6jRzPFakmPONVeQQlNLXKwQ0reiKx1kX95oZHmWnEVgQ/I3Wkc0zo zhxsnAdrZ4UIiNKfxv+VzJfIosXk607WepSBQOepam419j8wH8vGpCVYIJk6gqmG/c21 wZkCi42qrroMDRFdZ34O5g8BVSwiwXQ6Tne/kblUN8sLmkEv/TMqp/8hithTj6/RoCv/ NPKYcl34Anx9Eg3+6PN5APKHFxIr+sdMaNiCoGFKLd3ay5bx8hNWh/oSDPXM5VRsXov8 E4qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=uzWEIlxFYrhYaCwJ81Z4ASNBiShYXTeOYWcDsrSpw5I=; b=hYgjH80QwrKdrblYF5t7KX77Iy2vSHiNeJr12RFCNEDRqY8ty2Af5OxydgMHRh9MFI hkpfYl/Eat9VRALQ5OXccwp72mlgv33RPwYC8wOE12t5qV0iefHtYa6uCBSiEddE46ac dZI9yIRuHRMJpamIESK8/ihfqepN97eeMa9vWIa3Gqb4F+XqXl7L6l+GQdtlbalKevNy EGqr10nSar1EH7T8Kb3WeXBjGDc7X7Dau9ZMK57GV5JQJdVF8kMZtxqvzXXTenxshowf f/leaqrpTlasb7tTheDBo0tVbfwp8M7bbCCvN86aFG0bunvKIJv+bsjQmor86wf7oBIw 2w8w== X-Gm-Message-State: AOAM530Jn3Ek1JKPovnlpuswl0zNEkqbFt6oKxluYkMTvBlRBnWxQNgt FyUBFQjp2m2IjngqXdSHQutxEigYoBa3ZMRQhOIQhDlD/GvC7Q== X-Google-Smtp-Source: ABdhPJxgyfLmbVOF2C7qYQPn1KUhcDBWdlQBS7yBbdKNp/9j+W4ZLVYyyPEGffJn8VkYSLWTZKY87SEXmMHHTJlwIiM= X-Received: by 2002:a05:620a:228f:: with SMTP id o15mr12314129qkh.206.1605806196150; Thu, 19 Nov 2020 09:16:36 -0800 (PST) MIME-Version: 1.0 References: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> <3c9f6285-ae7c-1062-2dd3-42f8c953a230@gmail.com> In-Reply-To: <3c9f6285-ae7c-1062-2dd3-42f8c953a230@gmail.com> From: Warner Losh Date: Thu, 19 Nov 2020 10:16:26 -0700 Message-ID: Subject: Re: Monitoring commits on all branches To: Marc Branchaud Cc: Dan Langille , freebsd-git@freebsd.org X-Rspamd-Queue-Id: 4CcRDQ3Bbdz4v3d X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=arrFuQu/; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::72a) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-3.00 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::72a:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::72a:from]; R_SPF_NA(0.00)[no SPF record]; FREEMAIL_TO(0.00)[gmail.com]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::72a:from]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; MAILMAN_DEST(0.00)[freebsd-git] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 17:16:39 -0000 Thanks Marc! This is great advice... more comments below... On Thu, Nov 19, 2020 at 9:16 AM Marc Branchaud wrote: > On 2020-11-18 8:49 p.m., Dan Langille wrote: > > How can a repo be monitored for commits on all branches? > > > > I know how to ask a given branch: do you have any commits after foo_hash? > > > > How do I: > > > > * get a list of all commits since foo_hash > > A quick a note about Warner's reply: > > > git log $hash..HEAD > > "HEAD" is just a git nickname for "whatever you have currently > checked-out" (which can be a branch, a tag, or "detached" commit SHA ID). > > > * know which branch each of those commits was on (e.g. master, > branches/2020Q4) > > Unfortunately you'll find most normal git advice to be a bit frustrating > with the FreeBSD repos, because FreeBSD doesn't work the way most people > use git. Specifically, the FreeBSD project does not ever merge branches > (in the git sense of the word "merge"). Things would be very, very much > easier if the FreeBSD project were to use git-style merging. I believe > there are discussions underway about adjusting the whole MFC process for > the git world. I admit that part of my motivation in writing this > message is to provide grist for that mill. > FreeBSD src will be doing cherry-picks. There's only pain and suffering from merge commits in this environment. Git's tools are adequate to cope with individual and squashed cherry picks. > Fortunately even without git-merged branches, there are still git tools > that help, though they're not as precise as one would like. > They are for src. I suspect for ports they might not be. > Let's look at a concrete example with the beta ports git repo (which I > just cloned), and compare the 2020Q4 and main branches. I'll start with > some overall exploration, then address your specific question. > > There are 298 commits in the 2020Q4 branch. I know this because > git merge-base origin/main origin/branches/2020Q4 > tells me where 2020Q4 branched off of main: commit 5dbe4e5f775ea2. And > git rev-list 5dbe4e5f775ea2..origin/branches/2020Q4 | wc -l > says "299". (The "rev-list" command is a bare-bones version of "log" > that only lists commit SHA IDs.) > > Meanwhile there have been 4538 commits to the main branch since commit > 5dbe4e5f775ea2. > > As far as git is concerned, those 299 commits in 2020Q4 are *different* > from anything in main. Even though most of them made the exact same > code changes, they were created at different times, often by different > authors, and they have different commit messages. > True. > But you can still ask git to look at the code-change level to see which > 2020Q4 commits exactly replicated the code change from main: > > git cherry -v origin/main origin/branches/2020Q4 > > This little piece of magic looks at the 299 commits in 2020Q4 that are > not in main and compares their code changes to the 4538 commits in main > that are not in 2020Q4. It prints out the 299 2020Q4 commit SHA IDs, > prefixed with either a "- " or a "+ ". The -v appends the commit > message's first line: > > - 394d9746e5eea73f56334b2e7ddbdc8f686d6541 MFH: r550869 > + 1ac9571956759c91d852ee92859a12e52dcbde48 MFH: r550885 r550886 > - fd411bdfda55488b84de75e6b043c513a281abf0 MFH: r551209 > - 533cdaa97457b3318aebcc53f7a1a46ea66721da MFH: r551236 > ...... > > A "-" means that the commit matches the code change made by a commit in > main, while a "+" means that the commit's code change does not *exactly* > match any main commit since commit 5dbe4e5f775ea2. > > So > git cherry -v origin/main origin/branches/2020Q4 | grep ^- > shows us the 234 2020Q4 commits that made the exact same change as a > commit in main. > > And > git cherry -v origin/main origin/branches/2020Q4 | grep ^+ > shows us that there are 41 not-exactly-the-same-change commits in > 2020Q4. Mostly these are ones that combined two or more MFH's into one > commit (e.g. 2020Q4 commit 1ac95719567), or that changed a file in a > slightly different way (see the first patch hunk of 2020Q4 commit > cbd002878f2, compared to its counterpart in main: commit a5d21ea16b6). > Yes. These sorts of issues are why doing merge commits aren't always the right way to go because we're not merging the entire history together (doing a join), but rather just small subsets of it. How to cope with the mostly the same small files tree that is our ports tree in the face of git's guessing which does a poor job on such a tree is an interesting problem to solve. merge commits can help some of the issue, but they can create other issues as well when done incorrectly.... Even so, great hints for how to find cherry picked items. I suspect we'll need to have some tooling that embeds hash(es) into the commit message in some stylized way to allow tracking the non--trivial patch changes that sometimes happen: squashing several cherry picks, necessary differences due to branch drift, etc. It's unclear how we should do this, though, in a way that works well, is reliable and doesn't add undue friction to the process... > Now to your specific question: Given a commit, how can we tell which > branches contain that code change? Let's look at main commit > 6a9a8389d609 which I've determined, through manual spelunking, matches > 2020Q4's commit 02eba4048564. > > At a basic level, "git cherry" can tell us that *something* in 2020Q4 > made the same change as commit 6a9a8389d609. Here I reversed the order > of the branch names in the command: > git cherry origin/branches/2020Q4 origin/main | grep 6a9a8389d609 > This outputs: > - 6a9a8389d609ca0370c8c6eb8f993c1aa4071681 > and the "-" tells me that 6a9a8389d609's code change is *somewhere* in > 2020Q4 unique 299 commits. > > Unfortunately there's no convenient git command that'll tell you *which* > 2020Q4 commit replicated commit 6a9a8389d609. For that, we need to do a > bit of scripting: > > -----8<-----8<-----8<-----8<----- > > #!/bin/sh > > TARGET="6a9a8389d609" > > BASE=`git merge-base origin/branches/2020Q4 origin/main` > > TARGET_PATCH_ID=`git show -p $TARGET | git patch-id --stable | cut -f 1 > -d ' '` > > for REV in `git rev-list $BASE..origin/branches/2020Q4`; do > PATCH_ID=`git show -p $REV | git patch-id --stable | cut -f 1 -d ' '` > if [ "$PATCH_ID" = "$TARGET_PATCH_ID" ]; then > echo "Found a commit that replicated target commit $TARGET:" > echo > git show -s $REV > exit 0 > fi > done > > echo "Did not find any commit that exactly replicated $TARGET." > exit 1 > > ----->8----->8----->8----->8----- > > This only looks at the 2020Q4 branch, but it's easily adapted to look at > a user-specified branch, or multiple branches. (In the above I used > "git patch-id", which is what "git cherry" uses internally to identify a > commit's code changes.) > > I hope all this helps a bit! > It does. I thought I'd had my head deep into git, but hadn't stumbled upon this. It looks useful enough I'll try to add a section to my FAQ. Thanks! Warner From owner-freebsd-git@freebsd.org Thu Nov 19 17:45:16 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 796C4470BAB for ; Thu, 19 Nov 2020 17:45:16 +0000 (UTC) (envelope-from dan@langille.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CcRsS2m3Bz3Bt4; Thu, 19 Nov 2020 17:45:16 +0000 (UTC) (envelope-from dan@langille.org) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 9C2DA5C00A1; Thu, 19 Nov 2020 12:45:15 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Thu, 19 Nov 2020 12:45:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=langille.org; h= content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=fm2; bh=z PJxcpcu0oWIuviMUXc/VuwmAnCzsA1v27vuz18ZTEs=; b=PPn2LbjKyu4vhhS9S N3ftrhSJObokCXUHzw+2rxAeMtcweMwKxoDWwMLs5AAyx5JorYQSY4yn11FRJGeN GqBBFQ5QBT2Fhq1+XJE4eUP7ZC5AloCrr1AVpNW8Ckn1d+G9NxBvF035QPAYfCUS 9JX/fMGHL0eMt3adm0OIQHJB20HPtsVjjlGG+uYsGRswygrx0Ta5C7Dk+SJ41pCq Nhy9GEprXG48FFudqIhd+yCSZY7PUb+Zn+89xgth/EmzMghT+VT0ohoGHnmXp+aT GhNYERGomAY9D4kLZsmHBVGZ02YvxDXW9ZZ3ZzRhL/oZVvilwMGSPgYaplVrhbyf lqrvg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=zPJxcpcu0oWIuviMUXc/VuwmAnCzsA1v27vuz18ZT Es=; b=nyD5SDCv5IaTaYkT5eDRyq969tjlv/QDo1DKc9fvQfIebwz25IMAZIEtT DJIuRt1VWmSEq6FTbfg4g7VexNta28o+QMoaj8uNCJPZW7C5WOp6J+8/kv5+xple r73GlxSU0a096ufP3c1uydfKU4VuJVZhEN+EBwPtwqd/pbOPx3MXUSpP/mxMyzEB 8E+lLxmmWFcFAPBb8JXS7HD2ziymitucN+ktfoXJsGY/6+7mG1lrYJToGCwLT5+R WMcpvXORCQbdASlN4YqWZnYOitg9ts7VD2c13CAInANbyaBKwifueQ8NDp9yFqRN vE6WInzysmMwnIgeI/DmBZKczIZwg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrudefjedguddtfecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurheptggguffhjgffgffkfhfvofesth hqmhdthhdtvdenucfhrhhomhepffgrnhcunfgrnhhgihhllhgvuceouggrnheslhgrnhhg ihhllhgvrdhorhhgqeenucggtffrrghtthgvrhhnpeejtedvgeelfefhveejjeejjeefue euvdejheetjeevkeelhfegudeiieehlefhgfenucfkphepuddtkedrhedvrddvfeejrddu feefnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug grnheslhgrnhhgihhllhgvrdhorhhg X-ME-Proxy: Received: from pro02.wifi.int.unixathome.org (pool-108-52-237-133.phlapa.fios.verizon.net [108.52.237.133]) by mail.messagingengine.com (Postfix) with ESMTPA id 373313064AA7; Thu, 19 Nov 2020 12:45:15 -0500 (EST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.4\)) Subject: Re: Monitoring commits on all branches From: Dan Langille In-Reply-To: <20201119085223.nsrp2e45oyepsvds@aching.in.mat.cc> Date: Thu, 19 Nov 2020 12:45:14 -0500 Cc: freebsd-git@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> <20201119085223.nsrp2e45oyepsvds@aching.in.mat.cc> To: Mathieu Arnold X-Mailer: Apple Mail (2.3608.120.23.2.4) X-Rspamd-Queue-Id: 4CcRsS2m3Bz3Bt4 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 17:45:16 -0000 > On Nov 19, 2020, at 3:52 AM, Mathieu Arnold wrote: >=20 > On Wed, Nov 18, 2020 at 08:49:46PM -0500, Dan Langille wrote: >> How can a repo be monitored for commits on all branches? >>=20 >> I know how to ask a given branch: do you have any commits after = foo_hash? >>=20 >> How do I: >>=20 >> * get a list of all commits since foo_hash >=20 > All commits on the branch foo_hash is: >=20 > git log $foo_hash...branch_name So, branch_name, not HEAD as others have mentioned? e.g. : git log $foo_hash..HEAD >=20 >> * know which branch each of those commits was on (e.g. master, = branches/2020Q4) >=20 > You will need to keep track of the latest commit on each branch > separately, because there is absolutely no relation or ordering > possible between branches. Yes, I agree. FreshPorts is keeping track of the last commit on each = branch. Thank you.= From owner-freebsd-git@freebsd.org Thu Nov 19 19:00:44 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F2CF3472400 for ; Thu, 19 Nov 2020 19:00:44 +0000 (UTC) (envelope-from dan@langille.org) Received: from wout1-smtp.messagingengine.com (wout1-smtp.messagingengine.com [64.147.123.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CcTXW0wnLz3HDB for ; Thu, 19 Nov 2020 19:00:42 +0000 (UTC) (envelope-from dan@langille.org) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 1F73B76A; Thu, 19 Nov 2020 14:00:38 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Thu, 19 Nov 2020 14:00:38 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=langille.org; h= content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=fm2; bh=/ 37srXCWvBumg81HJaaLDO62IrhCpF8Klo6wrSKX9YA=; b=SMcaRRmtb/NO5M4vU lq2GFCgY9RSKr0pPsWt2sVYc0Vx0uI0TJu7xKqwmJa8QoYLWuxT4XdM3W6hzHbW5 /A5O8TSDqhaJpZKQL7mlYZIWqfHab12ROgEx1fq86S+j4tGewzep2XjhZWGhUXlX cD9ProkghILUYjCMhPE3AsGlM7WOT8BtKJ4Z+wa0qzP+hxrJw9KL6TYReZJr/UEy C4AC+qzkEW5L6NGB9/xLVB+Kcu8S/cM5nB/reBglQ6v/puS5r5WyV64NhvK0syng x2yHtujCFvvDBPjCaA12UJVpEPkW/zKisCXfUXdtvp/UdMb8x1T6NC86ecwrwLAF zJikQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=/37srXCWvBumg81HJaaLDO62IrhCpF8Klo6wrSKX9 YA=; b=pfSSuTS0odhuf85glEyiMnXbSTFYpNA/PiCu5mWUwPZEOzX3xiM1hGZvk 2p+nyYRE+NJJC/qqCslmBScu/aFVp3/3cfPsGrHE9Yy7jE5f2QA6rLc8eaxPNp9J oSroZShWEC4LZbypcVXgOclC0420VLVM26IotGjFDtWfc1HKxBoS85ABbZ08OmRV WuUW4Zihfskywr54NxbmtUqRU/QEYY59GcBGimslxs2tv2JpWLDK0JRghzCP0GTR Lhrbv8SevETh2DX6q67Xo+gpSpGLmteXbLn+qzkZPtzJYBRlxfNqwYthv4fpTOZW O9ZvRcsbwUS03CwHNOvpMiU6EZAuA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrudefjedguddukecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurheptggguffhjgffgffkfhfvofesth hqmhdthhdtjeenucfhrhhomhepffgrnhcunfgrnhhgihhllhgvuceouggrnheslhgrnhhg ihhllhgvrdhorhhgqeenucggtffrrghtthgvrhhnpeeggfegfeeukeeigfeiudektefhte ffledugeekfeehjeevteeifeeigefgjeevieenucffohhmrghinheplhgrnhhgihhllhgv rdhorhhgnecukfhppedutdekrdehvddrvdefjedrudeffeenucevlhhushhtvghrufhiii gvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegurghnsehlrghnghhilhhlvgdrohhr gh X-ME-Proxy: Received: from pro02.wifi.int.unixathome.org (pool-108-52-237-133.phlapa.fios.verizon.net [108.52.237.133]) by mail.messagingengine.com (Postfix) with ESMTPA id F0CFD3280065; Thu, 19 Nov 2020 14:00:36 -0500 (EST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.4\)) Subject: Re: Monitoring commits on all branches From: Dan Langille In-Reply-To: <3c9f6285-ae7c-1062-2dd3-42f8c953a230@gmail.com> Date: Thu, 19 Nov 2020 14:00:36 -0500 Cc: freebsd-git@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> <3c9f6285-ae7c-1062-2dd3-42f8c953a230@gmail.com> To: Marc Branchaud X-Mailer: Apple Mail (2.3608.120.23.2.4) X-Rspamd-Queue-Id: 4CcTXW0wnLz3HDB X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=langille.org header.s=fm2 header.b=SMcaRRmt; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=pfSSuTS0; dmarc=pass (policy=none) header.from=langille.org; spf=pass (mx1.freebsd.org: domain of dan@langille.org designates 64.147.123.24 as permitted sender) smtp.mailfrom=dan@langille.org X-Spamd-Result: default: False [-3.60 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.24]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[langille.org:+,messagingengine.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[langille.org,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[gmail.com]; RECEIVED_SPAMHAUS_PBL(0.00)[108.52.237.133:received]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11403, ipnet:64.147.123.0/24, country:US]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.24:from]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[langille.org:s=fm2,messagingengine.com:s=fm1]; FREEFALL_USER(0.00)[dan]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RWL_MAILSPIKE_VERYGOOD(0.00)[64.147.123.24:from]; MAILMAN_DEST(0.00)[freebsd-git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 19:00:45 -0000 > On Nov 19, 2020, at 11:16 AM, Marc Branchaud = wrote: >=20 > On 2020-11-18 8:49 p.m., Dan Langille wrote: >> How can a repo be monitored for commits on all branches? >> I know how to ask a given branch: do you have any commits after = foo_hash? >> How do I: >> * get a list of all commits since foo_hash >=20 > A quick a note about Warner's reply: >=20 >> git log $hash..HEAD >=20 > "HEAD" is just a git nickname for "whatever you have currently = checked-out" (which can be a branch, a tag, or "detached" commit SHA = ID). Mathieu mentioned: git log $foo_hash...branch_name That was the first time I've seen that used. All previous suggestions = were HEAD, not branch_name. But they are all the same in this context? >> * know which branch each of those commits was on (e.g. master, = branches/2020Q4) >=20 > Unfortunately you'll find most normal git advice to be a bit = frustrating with the FreeBSD repos, because FreeBSD doesn't work the way = most people use git. Specifically, the FreeBSD project does not ever = merge branches (in the git sense of the word "merge"). Things would be = very, very much easier if the FreeBSD project were to use git-style = merging. I believe there are discussions underway about adjusting the = whole MFC process for the git world. I admit that part of my motivation = in writing this message is to provide grist for that mill. >=20 > Fortunately even without git-merged branches, there are still git = tools that help, though they're not as precise as one would like. >=20 > Let's look at a concrete example with the beta ports git repo (which I = just cloned), and compare the 2020Q4 and main branches. I'll start with = some overall exploration, then address your specific question. >=20 > There are 298 commits in the 2020Q4 branch. I know this because > git merge-base origin/main origin/branches/2020Q4 > tells me where 2020Q4 branched off of main: commit 5dbe4e5f775ea2. = And > git rev-list 5dbe4e5f775ea2..origin/branches/2020Q4 | wc -l > says "299". (The "rev-list" command is a bare-bones version of "log" = that only lists commit SHA IDs.) [examples snipped] I followed that. I took the merge information as background and good-to-know, because = FreshPorts won't be doing any merging. It just needs a good "git = checkout" working copy. Sorry for such a long reply. * How FreshPorts extracts data FreshPorts is only interested in a snapshot of the repo with respect to = a given commit. It works on the 'repo as a whole' to extract values = from the ports which were affected by that commit. Case in point: a = commit to a parent port might affect any or all of the child ports. All = the child ports need to be refreshed. I am quickly concluding that FreshPorts must decide in advance what git = branches it will pay attention to. At present, it follows all branches. * FreshPorts (without git) uses email to create XML When moving FreshPorts from subversion to git, one of the goals was to = avoid relying on email to know that a commit has occurred. That is how = FreshPorts has always worked. The email (from the CVS commit) was parsed = and XML created. This code was updated for SVN. The XML is then used to = load the commit into the FreshPorts database which then drives the = website contents. When I started the GIT conversion, there was no commit email. "git log = $foo_hash...HEAD" is how FreshPorts knows what commits to process. One positive aspect of email approach: it identified the branch. So far, = I can't see how I can process the repo as a whole and see every commit = and know what branch it was on. * Polling git It is beginning to sound like the FreshPorts git code for detecting = incoming commits will be: Every N minutes, do this: for each repo in REPOS for branch in BRANCHES cd to the directory for that repo git checkout branch git log $branch_last_hash...HEAD for each of those commits process the commit end for end for end for At present, the REPOS and BRANCHES are: * freebsd BRANCHES=3D"master branches/2020Q4 branches/2020Q3 = branches/2020Q2 ...etc" * freebsd-ports BRANCHES=3D"master stable/12 stable/11" * freebsd-doc BRANCHES=3D"master" Some might ask: * Why not just master and latest-quarterly for freebsd-ports? * Because commits to older branches sometimes occur (or at least I = thought I saw one once) Commit hooks might also help, but I'm not sure if that will make things = easier or complicate everything * When new branches arrive It is vital that FreshPorts remain automated as much as it can be. At = present, under SVN, I might have to fix things perhaps 5 or 6 times a = year, usually because a commit did not get processed. Keeping that in mind, I do not yet know how to handle the following = situations: A new branch comes out. * Automation might be possible for ports quarterly branches * FreshPorts has to know there is a new branch * BRANCHES needs to be updated * I don't see that it can be automated for stable/* * need to handle 'git checkout branch' when branch does not exist? * Once branch exists, how do you find out about the commits when you = have no starting point for 'git log'? Right now, a new quarterly branch is noticed when the first commit email = comes through. FreshPorts then does an 'svn co' for that branch. I'm hoping someone has good ideas for my edge cases. Thank you. =E2=80=94=20 Dan Langille http://langille.org/ From owner-freebsd-git@freebsd.org Thu Nov 19 21:29:28 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7EFB44754D0 for ; Thu, 19 Nov 2020 21:29:28 +0000 (UTC) (envelope-from oliver.pntr@gmail.com) Received: from mail-ot1-x329.google.com (mail-ot1-x329.google.com [IPv6:2607:f8b0:4864:20::329]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CcXr73Pplz3RD9 for ; Thu, 19 Nov 2020 21:29:27 +0000 (UTC) (envelope-from oliver.pntr@gmail.com) Received: by mail-ot1-x329.google.com with SMTP id n12so3253880otk.0 for ; Thu, 19 Nov 2020 13:29:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=CEXE3YNFRtXRbagOh7eRz67sH9dgxIN8V5ok+wJ1d7Y=; b=W3mtirgV+9ENqFjnT1qBdieOe9SWc5dgEBOFWDKRcav8ikKSffBxe+OY1rnauAJ1jY XVVeg26Y+zO6gKnR2fgrrDVljVMl18fbfr0bErW0uG2dvY6aggj6fX/xgK97UCjsOz71 RkW6RduSza/tK8uNSilMNXuuK/64DnPbqHkZNeaNFuAy/AxfEKVggLLFDjfR7HL0xQGy lnSFKQioJPeW+Oi2SSFUpCdYJor7oZrV3p9ELsJlwCyO3a5ILzsCKTHJtHCGFW11BY7/ G1Rvw12AMmwwo4cvCqBYBkXzaqtdrDp/cqEkW4H61F0SKU0KWTRKM6iMLODGBJgrbT7q VsCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=CEXE3YNFRtXRbagOh7eRz67sH9dgxIN8V5ok+wJ1d7Y=; b=NXnjrXIWeEo5YVh7IvhfnywN6SOkRLOgKA1DanYgTuFC2kM46mkS5MucYgcJAknX5z A/bB2XaRBF1B1p7SuNUHGGb0ffTLlKgKYDI/JOOadOQxc2D/QiVaGFNdPa1O/oTqsXLT ZaNMEImzKH3B+NBszn8UTcw9RaNINQWhRG7jQRAKTDPUh8ZGuhgaqcmpet0PQEFKZzJR NVRcqD88fBjAPmdixxDxxrjxpFVhCIdMH19sSQUIGK/tZvFTZhD6lpXvisErRRUyVJur a2CLMjaSd3LOGLfiXLOog5nqvbys9c9J9tZkWScKjx4lrrABJk4ijhr0A+uDDBDzHrlR yC1w== X-Gm-Message-State: AOAM533EqFrhoWmFk74GzZ4eyN7gvo2SCZEIh87A85bvO1l9pEu4cR9r 6LbDzoWmhHaL0VYDH9N2vVEbx0wBH6Rvj6+MTJLkVehg X-Google-Smtp-Source: ABdhPJyNLoJ7vnp3+dTAe8/mdmG77h8TnQpOEQy4EbKLsOl4dEnq9GW5lBv8mFl0FQ0C6Dre1JY1n/GkRJUF1cnVj24= X-Received: by 2002:a9d:620d:: with SMTP id g13mr11067008otj.56.1605821366287; Thu, 19 Nov 2020 13:29:26 -0800 (PST) MIME-Version: 1.0 Received: by 2002:a4a:51c1:0:0:0:0:0 with HTTP; Thu, 19 Nov 2020 13:29:25 -0800 (PST) In-Reply-To: References: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> <3c9f6285-ae7c-1062-2dd3-42f8c953a230@gmail.com> From: Oliver Pinter Date: Thu, 19 Nov 2020 22:29:25 +0100 Message-ID: Subject: Re: Monitoring commits on all branches To: Dan Langille Cc: Marc Branchaud , "freebsd-git@freebsd.org" X-Rspamd-Queue-Id: 4CcXr73Pplz3RD9 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=W3mtirgV; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of oliverpntr@gmail.com designates 2607:f8b0:4864:20::329 as permitted sender) smtp.mailfrom=oliverpntr@gmail.com X-Spamd-Result: default: False [-3.46 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.46)[-0.457]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::329:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::329:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::329:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-git]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 21:29:28 -0000 Hi! On Thursday, November 19, 2020, Dan Langille wrote: > > On Nov 19, 2020, at 11:16 AM, Marc Branchaud wrote= : > > > > On 2020-11-18 8:49 p.m., Dan Langille wrote: > >> How can a repo be monitored for commits on all branches? > >> I know how to ask a given branch: do you have any commits after > foo_hash? > >> How do I: > >> * get a list of all commits since foo_hash > > > > A quick a note about Warner's reply: > > > >> git log $hash..HEAD > > > > "HEAD" is just a git nickname for "whatever you have currently > checked-out" (which can be a branch, a tag, or "detached" commit SHA ID). > > Mathieu mentioned: git log $foo_hash...branch_name > > That was the first time I've seen that used. All previous suggestions wer= e > HEAD, not branch_name. But they are all the same in this context? > > >> * know which branch each of those commits was on (e.g. master, > branches/2020Q4) > > > > Unfortunately you'll find most normal git advice to be a bit frustratin= g > with the FreeBSD repos, because FreeBSD doesn't work the way most people > use git. Specifically, the FreeBSD project does not ever merge branches > (in the git sense of the word "merge"). Things would be very, very much > easier if the FreeBSD project were to use git-style merging. I believe > there are discussions underway about adjusting the whole MFC process for > the git world. I admit that part of my motivation in writing this messag= e > is to provide grist for that mill. > > > > Fortunately even without git-merged branches, there are still git tools > that help, though they're not as precise as one would like. > > > > Let's look at a concrete example with the beta ports git repo (which I > just cloned), and compare the 2020Q4 and main branches. I'll start with > some overall exploration, then address your specific question. > > > > There are 298 commits in the 2020Q4 branch. I know this because > > git merge-base origin/main origin/branches/2020Q4 > > tells me where 2020Q4 branched off of main: commit 5dbe4e5f775ea2. And > > git rev-list 5dbe4e5f775ea2..origin/branches/2020Q4 | wc -l > > says "299". (The "rev-list" command is a bare-bones version of "log" > that only lists commit SHA IDs.) > > [examples snipped] > > I followed that. > > I took the merge information as background and good-to-know, because > FreshPorts won't be doing any merging. It just needs a good "git checkout= " > working copy. > > Sorry for such a long reply. > > > * How FreshPorts extracts data > > FreshPorts is only interested in a snapshot of the repo with respect to a > given commit. It works on the 'repo as a whole' to extract values from t= he > ports which were affected by that commit. Case in point: a commit to a > parent port might affect any or all of the child ports. All the child > ports need to be refreshed. > > I am quickly concluding that FreshPorts must decide in advance what git > branches it will pay attention to. At present, it follows all branches. > > > * FreshPorts (without git) uses email to create XML > > When moving FreshPorts from subversion to git, one of the goals was to > avoid relying on email to know that a commit has occurred. That is how > FreshPorts has always worked. The email (from the CVS commit) was parsed > and XML created. This code was updated for SVN. The XML is then used to > load the commit into the FreshPorts database which then drives the websit= e > contents. > > When I started the GIT conversion, there was no commit email. "git log > $foo_hash...HEAD" is how FreshPorts knows what commits to process. > > One positive aspect of email approach: it identified the branch. So far, = I > can't see how I can process the repo as a whole and see every commit and > know what branch it was on. > > > * Polling git > > It is beginning to sound like the FreshPorts git code for detecting > incoming commits will be: > > Every N minutes, do this: > > for each repo in REPOS > for branch in BRANCHES > cd to the directory for that repo > git checkout branch > git log $branch_last_hash...HEAD > for each of those commits > process the commit > end for > end for > end for > > At present, the REPOS and BRANCHES are: > > * freebsd BRANCHES=3D"master branches/2020Q4 branches/2020Q3 branches/202= 0Q2 > ...etc" > * freebsd-ports BRANCHES=3D"master stable/12 stable/11" > * freebsd-doc BRANCHES=3D"master" > > Some might ask: > > * Why not just master and latest-quarterly for freebsd-ports? > * Because commits to older branches sometimes occur (or at least I though= t > I saw one once) > > Commit hooks might also help, but I'm not sure if that will make things > easier or complicate everything > > > * When new branches arrive > > It is vital that FreshPorts remain automated as much as it can be. At > present, under SVN, I might have to fix things perhaps 5 or 6 times a yea= r, > usually because a commit did not get processed. > > Keeping that in mind, I do not yet know how to handle the following > situations: > > A new branch comes out. > > * Automation might be possible for ports quarterly branches > * FreshPorts has to know there is a new branch > * BRANCHES needs to be updated > * I don't see that it can be automated for stable/* > * need to handle 'git checkout branch' when branch does not exist? > * Once branch exists, how do you find out about the commits when you have > no starting point for 'git log'? > > Right now, a new quarterly branch is noticed when the first commit email > comes through. FreshPorts then does an 'svn co' for that branch. > > > > I'm hoping someone has good ideas for my edge cases. Try the 'git branch -rv' command, cache its output in a file, and select the lines with diffed to the new run of previous one. > > Thank you. > > =E2=80=94 > Dan Langille > http://langille.org/ > > > > > _______________________________________________ > freebsd-git@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-git > To unsubscribe, send any mail to "freebsd-git-unsubscribe@freebsd.org" > From owner-freebsd-git@freebsd.org Thu Nov 19 22:30:22 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F03224761AB for ; Thu, 19 Nov 2020 22:30:22 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: from mail-qk1-x72d.google.com (mail-qk1-x72d.google.com [IPv6:2607:f8b0:4864:20::72d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CcZBP1zQ2z3kKH for ; Thu, 19 Nov 2020 22:30:21 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: by mail-qk1-x72d.google.com with SMTP id n132so7169479qke.1 for ; Thu, 19 Nov 2020 14:30:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=WAmHOpsvom22WmDWVPXDXszRa8imX13f+n+w8zxClEo=; b=nkcnbyx+itgq2psOTHVPg58DSHhvfO/1FIpFQ9WZmxrRWWEiiwBnQolCg0QkwfOlDv tKQ4hCJpWF9uP8Kaqz3pRVLbNvGJ2luFsNF+lgrW1iQRdZETktipIq1IFr3Dg1EvHs1h 7ePSr3SSrTa0XM5OkibsANxdwKkB32eEIPh73sb7wpC4GfvK6po2vPCYT7EJX6ab3xNW p8jAmxngrKfmfIhbW2F2xSWhF5VmveJGKRpyPpWgp9ixOp3HGTT44qmaUqA0ahPgrD2y xXI2iQqlHC399HQU21DKEmRJZNHBiO8QEB2EXFz4plGvOB+PXfUySSr+acrq+yba4iZp UHfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=WAmHOpsvom22WmDWVPXDXszRa8imX13f+n+w8zxClEo=; b=rYYd/IaN1zwD9MzjExa768DsECxemCfHiGY8ZteEfEsAF0MAE0r0UvjJwSvkJAZ+Z+ odB526V/9/f1PcYz37QZBO2EaAkggF2VBVEgtweTzx8k/BHNFBPSIYF9cSKpMshmXweQ cnxNctLQ1+i4lvXROym9/yVoJexzPx5aVN8HZ9LpT05VkOPpSANv9TCunDzMhcFjUNS7 TQTH2uDUeG1oQNRNEH+QaV1hpS3yQIuM0Gklj+AgxUGyfoGYINZyFBnEAY2KQ2MeDtmv FxwD7Q/30BIp8qHvo+KNMGF2dpSkLpHdGQb4rEXuC7jlkUeDwA579JVZQunOycxGHfDQ EcYw== X-Gm-Message-State: AOAM531CcojDYJVjZNUSaxJHyWZX5vx7xV0KJCvzVJok1PilIcjY6njl W/mU8u5fSUnsDOHUNowXST3qXt66p0jpJhTf X-Google-Smtp-Source: ABdhPJzYx6swrqSTOGdl/HVvvsoccGLX5hgre1Z+uP+do0r2XcCbmfifZgzbWZ/hFxkT+MdsIRVPow== X-Received: by 2002:a37:5f42:: with SMTP id t63mr14619322qkb.449.1605825019820; Thu, 19 Nov 2020 14:30:19 -0800 (PST) Received: from [192.168.222.18] (192-222-183-158.qc.cable.ebox.net. [192.222.183.158]) by smtp.gmail.com with ESMTPSA id j13sm982474qtc.81.2020.11.19.14.30.18 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 19 Nov 2020 14:30:19 -0800 (PST) Subject: Re: Monitoring commits on all branches To: Dan Langille Cc: freebsd-git@freebsd.org References: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> <3c9f6285-ae7c-1062-2dd3-42f8c953a230@gmail.com> From: Marc Branchaud Message-ID: <93aa2a3e-7e8a-8199-3feb-6e464a590d72@gmail.com> Date: Thu, 19 Nov 2020 17:30:18 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4CcZBP1zQ2z3kKH X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=nkcnbyx+; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of marcnarc@gmail.com designates 2607:f8b0:4864:20::72d as permitted sender) smtp.mailfrom=marcnarc@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::72d:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::72d:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::72d:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 22:30:23 -0000 On 2020-11-19 2:00 p.m., Dan Langille wrote: >> On Nov 19, 2020, at 11:16 AM, Marc Branchaud >> wrote: >> >> On 2020-11-18 8:49 p.m., Dan Langille wrote: >>> How can a repo be monitored for commits on all branches? I know >>> how to ask a given branch: do you have any commits after >>> foo_hash? How do I: * get a list of all commits since foo_hash >> >> A quick a note about Warner's reply: >> >>> git log $hash..HEAD >> >> "HEAD" is just a git nickname for "whatever you have currently >> checked-out" (which can be a branch, a tag, or "detached" commit >> SHA ID). > > Mathieu mentioned: git log $foo_hash...branch_name > > That was the first time I've seen that used. All previous suggestions > were HEAD, not branch_name. But they are all the same in this > context? I hesitate to say yes, if only because I don't know precisely what context you're in. I think the previous suggestions assumed you have checked-out the branch you're interested in. Your email wasn't clear about that, and especially if you're doing "detached" checkouts using commit SHA IDs using "HEAD" might not work as those suggestions suggest. Just remember that the SHA IDs are the fundamental, immutable identifiers for git commits. All other names are just labels/aliases/symbols for a SHA ID, and some of those names will change which SHA ID they represent, depending on the name's type and what operations you do in your repository. "HEAD" is especially mercurial. >>> * know which branch each of those commits was on (e.g. master, >>> branches/2020Q4) >> >> Unfortunately you'll find most normal git advice to be a bit >> frustrating with the FreeBSD repos, because FreeBSD doesn't work >> the way most people use git. Specifically, the FreeBSD project >> does not ever merge branches (in the git sense of the word >> "merge"). Things would be very, very much easier if the FreeBSD >> project were to use git-style merging. I believe there are >> discussions underway about adjusting the whole MFC process for the >> git world. I admit that part of my motivation in writing this >> message is to provide grist for that mill. >> >> Fortunately even without git-merged branches, there are still git >> tools that help, though they're not as precise as one would like. >> >> Let's look at a concrete example with the beta ports git repo >> (which I just cloned), and compare the 2020Q4 and main branches. >> I'll start with some overall exploration, then address your >> specific question. >> >> There are 298 commits in the 2020Q4 branch. I know this because >> git merge-base origin/main origin/branches/2020Q4 tells me where >> 2020Q4 branched off of main: commit 5dbe4e5f775ea2. And git >> rev-list 5dbe4e5f775ea2..origin/branches/2020Q4 | wc -l says "299". >> (The "rev-list" command is a bare-bones version of "log" that only >> lists commit SHA IDs.) > > [examples snipped] > > I followed that. > > I took the merge information as background and good-to-know, because > FreshPorts won't be doing any merging. It just needs a good "git > checkout" working copy. > > Sorry for such a long reply. You're apologizing, after my verbosity? :) > * How FreshPorts extracts data > > FreshPorts is only interested in a snapshot of the repo with respect > to a given commit. It works on the 'repo as a whole' to extract > values from the ports which were affected by that commit. Case in > point: a commit to a parent port might affect any or all of the child > ports. All the child ports need to be refreshed. I'm guessing that typically the child ports' stuff (in the ports tree) does not change when the parent port changes. So you're talking about a build-time or run-time dependency between ports? I imagine you have some kind of ports-dependency database, which you can consult when a particular port is changed. Or are you thinking git might be able to help you here? > I am quickly concluding that FreshPorts must decide in advance what > git branches it will pay attention to. At present, it follows all > branches. > > * FreshPorts (without git) uses email to create XML > > When moving FreshPorts from subversion to git, one of the goals was > to avoid relying on email to know that a commit has occurred. That is > how FreshPorts has always worked. The email (from the CVS commit) was > parsed and XML created. This code was updated for SVN. The XML is > then used to load the commit into the FreshPorts database which then > drives the website contents. > > When I started the GIT conversion, there was no commit email. "git > log $foo_hash...HEAD" is how FreshPorts knows what commits to > process. > > One positive aspect of email approach: it identified the branch. So > far, I can't see how I can process the repo as a whole and see every > commit and know what branch it was on. You're right about that, but you *can* discover all the branches in the repo (see below). > * Polling git > > It is beginning to sound like the FreshPorts git code for detecting > incoming commits will be: > > Every N minutes, do this: > > for each repo in REPOS > for branch in BRANCHES > cd to the directory for that repo > git checkout branch > git log $branch_last_hash...HEAD > for each of those commits > process the commit > end for > end for > end for That loops looks basically right. You'll need to "git fetch" in each repo before processing branches. Watch out with that "git checkout branch" part: This is where a lot of people get tripped up when they move from svn to git. It doesn't help that git tries to do some hand-holding here, but really git users would benefit from simply understanding how branch names are just labels for commit SHA IDs. Without that understanding, people end up going down the rabbit-hole that is the "git pull" command, a wretched hive of scum and villainy if there ever was one. I'll just gloss over a lot of detail (but feel free to ask!) and recommend that you work with the "origin/"-prefixed branch names. These get updated every time you "git fetch" from the remote repo, and basically behave as you'd expect. It's appropriate in your case, because you're not creating any new commits. > At present, the REPOS and BRANCHES are: > > * freebsd BRANCHES="master branches/2020Q4 branches/2020Q3 branches/2020Q2 ...etc" > * freebsd-ports BRANCHES="master stable/12 stable/11" > * freebsd-doc BRANCHES="master" > > Some might ask: > > * Why not just master and latest-quarterly for freebsd-ports? > * Because commits to older branches sometimes occur (or at least I > thought I saw one once) > > Commit hooks might also help, but I'm not sure if that will make > things easier or complicate everything Looking through the set of hooks ("git help hooks"), I don't see anything that would help you. > * When new branches arrive > > It is vital that FreshPorts remain automated as much as it can be. At > present, under SVN, I might have to fix things perhaps 5 or 6 times a > year, usually because a commit did not get processed. > > Keeping that in mind, I do not yet know how to handle the following > situations: > > A new branch comes out. > > * Automation might be possible for ports quarterly branches > * FreshPorts has to know there is a new branch Fundamentally you start by doing a "git fetch" to retrieve updates from the official repository (you probably know that already, but I just want to be pedantic). Then "git branch -r" will list all the remote ("origin/"-prefixed) branches, including whatever new ones were just fetched. You can also do "git for-each-ref refs/remotes/origin" to see which commit SHA ID corresponds to each remote branch name (though the branch names are prefixed with "refs/remotes/origin/" not just "origin/"). You might find its output easier to parse than the equivalent "git branch -rv" suggested by Oliver. > * BRANCHES needs to be updated > * I don't see that it can be automated for stable/* Instead of saving BRANCHES locally, just use the output of one of the above commands every time you run the script. That way the list will always be up to date. > * need to handle 'git checkout branch' when branch does not exist? If you end up doing that, I'd say you're doing something wrong. > * Once branch exists, how do you find out about the commits when you have no > starting point for 'git log'? Use "git merge-base" to find where the new branch split off from the "master" or "main" branch. > Right now, a new quarterly branch is noticed when the first commit > email comes through. FreshPorts then does an 'svn co' for that > branch. > > I'm hoping someone has good ideas for my edge cases. I hope I've been a bit helpful! One last, small idea: Consider using tags in your local repos to track your $branch_last_hash. For example, after you finish processing the 2020Q4 branch, do git tag -f last-2020Q4 origin/branches/2020Q4 Then the next time through the 2020Q4 branch you can start at the "last-2020Q4" tag. For a new branch that doesn't have a tag yet, use "git merge-base" to find the starting point. M. From owner-freebsd-git@freebsd.org Thu Nov 19 22:50:17 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 69C0F476379 for ; Thu, 19 Nov 2020 22:50:17 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: from mail-qt1-x82a.google.com (mail-qt1-x82a.google.com [IPv6:2607:f8b0:4864:20::82a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CcZdN2BsSz3kht for ; Thu, 19 Nov 2020 22:50:16 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: by mail-qt1-x82a.google.com with SMTP id v11so5721492qtq.12 for ; Thu, 19 Nov 2020 14:50:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=yzg7Ye8Rr11T4FI/5zLOBwdyJFldt/vsXly33rNtSn4=; b=iKQWviBixGCgUo1UjAb5syWLuqPQCQQ6aA1zlYv5ba85POEdCvcDlh+/fiohO6NquL +DDnYusjNeGH4vrhka2K6rrSNSvXn8E7yfY0zhp5qDHeV+QKOANsYBXUCN0WPKVrlsGS 0Ow1WhbwwTyX8aDZTSKsWwt1ELPVrtdQf8Hl+Q4B/8IHyoivi3sSVAs2vFAv6+Tkj4n/ hNNC1jWLIZCj8AcC8pgWOC3jkpC7UKQM9nxCrdrLgSyuDz7WVfIuMaa1OJVQEZzpszJ0 Jz5meGhFrd1mFrv6GruKHd1X5U5OafODd0oo5FWlw3ug9tRWhWRU6VF6jBCDZs+4TSUd 78Ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=yzg7Ye8Rr11T4FI/5zLOBwdyJFldt/vsXly33rNtSn4=; b=s89BFsY4E2TqnArJI7IuKIkj/gc3f2i6clxyUdy/5bSANzXCDbbRKWypg7goZvLvwT L0zxOK+eqI3ukt7iCOWFcDMT3xpUEUKV3HRoZSNIefp7U+oGU5oloeF1MaKVUgychnqW xkzv0uhWMbFQV1zoCNwXUeUqsvb2Yjqui8oexqw5ji00NGSmkIHz2S7lcyk0pttMgEK9 BMZKMpomYhmc1kbDSd+Xo0z43H+yRqgS0RlfpRxQY7i8QZ2i9nFERf4aRI/ZvYedVyZ5 zWSNDqWrbtWT6EHhwt3aQxBuNVBO/a7xzn9oppBzTENg0TbheBC1f81HwxvjESyH0ZFA Mrow== X-Gm-Message-State: AOAM532/ptUsgPd1l6Bdu5GrkNpH+U5/aU5SMZ+VTex/pTzEJR2X2QLa H5d1IjlDTdAgllhRcD6DLnw= X-Google-Smtp-Source: ABdhPJzL/E8kFB6LspUVADCl/f5hsyN9DTTirOkQadpxxXgKpHFQpwhDgwCfvaawwE/3uO6MzM9Y0A== X-Received: by 2002:aed:308e:: with SMTP id 14mr12978009qtf.275.1605826214886; Thu, 19 Nov 2020 14:50:14 -0800 (PST) Received: from [192.168.222.18] (192-222-183-158.qc.cable.ebox.net. [192.222.183.158]) by smtp.gmail.com with ESMTPSA id c9sm853800qkm.116.2020.11.19.14.50.14 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 19 Nov 2020 14:50:14 -0800 (PST) Subject: Re: Monitoring commits on all branches To: Warner Losh Cc: Dan Langille , freebsd-git@freebsd.org References: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> <3c9f6285-ae7c-1062-2dd3-42f8c953a230@gmail.com> From: Marc Branchaud Message-ID: <6ead26a8-54e3-ed0e-d1b7-28d69753dea4@gmail.com> Date: Thu, 19 Nov 2020 17:50:13 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4CcZdN2BsSz3kht X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=iKQWviBi; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of marcnarc@gmail.com designates 2607:f8b0:4864:20::82a as permitted sender) smtp.mailfrom=marcnarc@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::82a:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::82a:from:127.0.2.255]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::82a:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 22:50:17 -0000 On 2020-11-19 12:16 p.m., Warner Losh wrote: > > Thanks Marc! This is great advice... more comments below... > > On Thu, Nov 19, 2020 at 9:16 AM Marc Branchaud > wrote: > > On 2020-11-18 8:49 p.m., Dan Langille wrote: > > How can a repo be monitored for commits on all branches? > > > > I know how to ask a given branch: do you have any commits after > foo_hash? > > > > How do I: > > > > * get a list of all commits since foo_hash > > A quick a note about Warner's reply: > > > git log $hash..HEAD > > "HEAD" is just a git nickname for "whatever you have currently > checked-out" (which can be a branch, a tag, or "detached" commit SHA > ID). > > > * know which branch each of those commits was on (e.g. master, > branches/2020Q4) > > Unfortunately you'll find most normal git advice to be a bit > frustrating > with the FreeBSD repos, because FreeBSD doesn't work the way most > people > use git.  Specifically, the FreeBSD project does not ever merge > branches > (in the git sense of the word "merge").  Things would be very, very > much > easier if the FreeBSD project were to use git-style merging.  I believe > there are discussions underway about adjusting the whole MFC process > for > the git world.  I admit that part of my motivation in writing this > message is to provide grist for that mill. > > > FreeBSD src will be doing cherry-picks. There's only pain and suffering > from merge commits in this environment. Git's tools are adequate to cope > with individual and squashed cherry picks. Fair enough. I'm also sure that the git community would welcome patches that help make FreeBSD's workflow a bit smoother. > Fortunately even without git-merged branches, there are still git tools > that help, though they're not as precise as one would like. > > > They are for src. I suspect for ports they might not be. > > Let's look at a concrete example with the beta ports git repo (which I > just cloned), and compare the 2020Q4 and main branches.  I'll start > with > some overall exploration, then address your specific question. > > There are 298 commits in the 2020Q4 branch.  I know this because >      git merge-base origin/main origin/branches/2020Q4 > tells me where 2020Q4 branched off of main: commit 5dbe4e5f775ea2.  And >      git rev-list 5dbe4e5f775ea2..origin/branches/2020Q4 | wc -l > says "299".  (The "rev-list" command is a bare-bones version of "log" > that only lists commit SHA IDs.) > > Meanwhile there have been 4538 commits to the main branch since commit > 5dbe4e5f775ea2. > > As far as git is concerned, those 299 commits in 2020Q4 are *different* > from anything in main.  Even though most of them made the exact same > code changes, they were created at different times, often by different > authors, and they have different commit messages. > > > True. > > But you can still ask git to look at the code-change level to see which > 2020Q4 commits exactly replicated the code change from main: > >      git cherry -v origin/main origin/branches/2020Q4 > > This little piece of magic looks at the 299 commits in 2020Q4 that are > not in main and compares their code changes to the 4538 commits in main > that are not in 2020Q4.  It prints out the 299 2020Q4 commit SHA IDs, > prefixed with either a "- " or a "+ ".  The -v appends the commit > message's first line: > >      - 394d9746e5eea73f56334b2e7ddbdc8f686d6541 MFH: r550869 >      + 1ac9571956759c91d852ee92859a12e52dcbde48 MFH: r550885 r550886 >      - fd411bdfda55488b84de75e6b043c513a281abf0 MFH: r551209 >      - 533cdaa97457b3318aebcc53f7a1a46ea66721da MFH: r551236 >      ...... > > A "-" means that the commit matches the code change made by a commit in > main, while a "+" means that the commit's code change does not > *exactly* > match any main commit since commit 5dbe4e5f775ea2. > > So >      git cherry -v origin/main origin/branches/2020Q4 | grep ^- > shows us the 234 2020Q4 commits that made the exact same change as a > commit in main. > > And >      git cherry -v origin/main origin/branches/2020Q4 | grep ^+ > shows us that there are 41 not-exactly-the-same-change commits in > 2020Q4.  Mostly these are ones that combined two or more MFH's into one > commit (e.g. 2020Q4 commit 1ac95719567), or that changed a file in a > slightly different way (see the first patch hunk of 2020Q4 commit > cbd002878f2, compared to its counterpart in main: commit a5d21ea16b6). > > > Yes. These sorts of issues are why doing merge commits aren't always the > right way to go because we're not merging the entire history together > (doing a join), but rather just small subsets of it. How to cope with > the mostly the same small files tree that is our ports tree in the face > of git's guessing which does a poor job on such a tree is an interesting > problem to solve. merge commits can help some of the issue, but they can > create other issues as well when done incorrectly.... I admit I don't quite follow you there, but I'm particularly ignorant of the ports tree. I have some quite-likely-stupid ideas after having played with it for 10 minutes while composing my earlier message, but even if the ideas are somehow clever I suspect they'd entail too much workflow change to be palatable. > Even so, great hints for how to find cherry picked items. I suspect > we'll need to have some tooling that embeds hash(es) into the commit > message in some stylized way to allow tracking the non--trivial patch > changes that sometimes happen: squashing several cherry picks, necessary > differences due to branch drift, etc. It's unclear how we should do > this, though, in a way that works well, is reliable and doesn't add > undue friction to the process... It's traditional when doing a cherry-pick to add a Cherry-picked-from: line to the commit message. The "cherry-pick" command even has a -x option to automatically add such a line to the new commit's message. (There's also a "git interpret-trailers" command that is a general-purpose tool for manipulating "Foo: blah blah" lines in commit messages.) "git cherry-pick" might actually lead people away from squashing together multiple changes into one commit, because you have to make a bit of an effort to get cherry-pick to squash things up. I personally think the project would benefit from discouraging squashed-together MFC's. > Now to your specific question: Given a commit, how can we tell which > branches contain that code change?  Let's look at main commit > 6a9a8389d609 which I've determined, through manual spelunking, matches > 2020Q4's commit 02eba4048564. > > At a basic level, "git cherry" can tell us that *something* in 2020Q4 > made the same change as commit 6a9a8389d609.  Here I reversed the order > of the branch names in the command: >      git cherry origin/branches/2020Q4 origin/main | grep 6a9a8389d609 > This outputs: >      - 6a9a8389d609ca0370c8c6eb8f993c1aa4071681 > and the "-" tells me that 6a9a8389d609's code change is *somewhere* in > 2020Q4 unique 299 commits. > > Unfortunately there's no convenient git command that'll tell you > *which* > 2020Q4 commit replicated commit 6a9a8389d609.  For that, we need to > do a > bit of scripting: > > -----8<-----8<-----8<-----8<----- > > #!/bin/sh > > TARGET="6a9a8389d609" > > BASE=`git merge-base origin/branches/2020Q4 origin/main` > > TARGET_PATCH_ID=`git show -p $TARGET | git patch-id --stable | cut -f 1 > -d ' '` > > for REV in `git rev-list $BASE..origin/branches/2020Q4`; do >     PATCH_ID=`git show -p $REV | git patch-id --stable | cut -f 1 > -d ' '` >     if [ "$PATCH_ID" = "$TARGET_PATCH_ID" ]; then >        echo "Found a commit that replicated target commit $TARGET:" >        echo >        git show -s $REV >        exit 0 >     fi > done > > echo "Did not find any commit that exactly replicated $TARGET." > exit 1 > > ----->8----->8----->8----->8----- > > This only looks at the 2020Q4 branch, but it's easily adapted to > look at > a user-specified branch, or multiple branches.  (In the above I used > "git patch-id", which is what "git cherry" uses internally to > identify a > commit's code changes.) > > I hope all this helps a bit! > > > It does. I thought I'd had my head deep into git, but hadn't stumbled > upon this. I've been using git for over 10 years, and I still discover new things. This "git cherry" stuff, for example, I've only started using a little bit in the last few months. > It looks useful enough I'll try to add a section to my FAQ. I'm honoured! M. From owner-freebsd-git@freebsd.org Fri Nov 20 11:30:05 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EC21F2C6D11 for ; Fri, 20 Nov 2020 11:30:05 +0000 (UTC) (envelope-from tom@eborcom.com) Received: from eborcom.com (canet.scrubhole.org [IPv6:2a02:1658:1::179:1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CcvV455hwz3DkX for ; Fri, 20 Nov 2020 11:30:04 +0000 (UTC) (envelope-from tom@eborcom.com) Received: from tom by eborcom.com with local (Exim 4.87_1 (FreeBSD)) (envelope-from ) id 1kg4c0-0000mI-9a; Fri, 20 Nov 2020 11:29:56 +0000 Date: Fri, 20 Nov 2020 11:29:56 +0000 From: Tom Hukins To: Dan Langille Cc: freebsd-git@freebsd.org Subject: Re: Monitoring commits on all branches Message-ID: <20201120112955.GO24999@eborcom.com> References: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> <3c9f6285-ae7c-1062-2dd3-42f8c953a230@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.2 (2016-11-26) Sender: Tom Hukins X-Rspamd-Queue-Id: 4CcvV455hwz3DkX X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of tom@eborcom.com designates 2a02:1658:1::179:1 as permitted sender) smtp.mailfrom=tom@eborcom.com X-Spamd-Result: default: False [-3.00 / 15.00]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[tom]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a02:1658:1::179:1/128]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[FreeBSD.org]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a02:1658:1::179:1:from]; SPAMHAUS_ZRD(0.00)[2a02:1658:1::179:1:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FORGED_SENDER(0.30)[tom@FreeBSD.org,tom@eborcom.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:12290, ipnet:2a02:1658::/32, country:GB]; FROM_NEQ_ENVFROM(0.00)[tom@FreeBSD.org,tom@eborcom.com]; MAILMAN_DEST(0.00)[freebsd-git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2020 11:30:06 -0000 On Thu, Nov 19, 2020 at 02:00:36PM -0500, Dan Langille wrote: > Mathieu mentioned: git log $foo_hash...branch_name > > That was the first time I've seen that used. All previous suggestions were HEAD, not branch_name. But they are all the same in this context? As already mentioned, it depends. I found the documentation for "Specifying Revisions" in `git help rev-parse` helpful when I was new to git: https://git-scm.com/docs/git-rev-parse#_specifying_revisions Tom From owner-freebsd-git@freebsd.org Fri Nov 20 17:03:29 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0E2D42EFFCC for ; Fri, 20 Nov 2020 17:03:29 +0000 (UTC) (envelope-from mat@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Cd2tm6wn4z3t43; Fri, 20 Nov 2020 17:03:28 +0000 (UTC) (envelope-from mat@freebsd.org) Received: from mail.j.mat.cc (owncloud.cube.mat.cc [IPv6:2a01:678:4:1::228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.mat.cc", Issuer "Let's Encrypt Authority X3" (verified OK)) (Authenticated sender: mat/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id C62E1206E2; Fri, 20 Nov 2020 17:03:28 +0000 (UTC) (envelope-from mat@freebsd.org) Received: from aching.in.mat.cc (unknown [IPv6:2a01:678:42:0:bdf4:2e83:8ff:e8ad]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: mat@mat.cc) by mail.j.mat.cc (Postfix) with ESMTPSA id A79C7942D84; Fri, 20 Nov 2020 17:03:27 +0000 (UTC) Date: Fri, 20 Nov 2020 18:03:26 +0100 From: Mathieu Arnold To: Dan Langille Cc: freebsd-git@freebsd.org Subject: Re: Monitoring commits on all branches Message-ID: <20201120170326.4k6sj7tt35lrjtaz@aching.in.mat.cc> References: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> <3c9f6285-ae7c-1062-2dd3-42f8c953a230@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="45wtwjfwliaf7fev" Content-Disposition: inline In-Reply-To: X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2020 17:03:29 -0000 --45wtwjfwliaf7fev Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Nov 19, 2020 at 02:00:36PM -0500, Dan Langille wrote: > Mathieu mentioned: git log $foo_hash...branch_name >=20 > That was the first time I've seen that used. All previous suggestions wer= e HEAD, not branch_name. But they are all the same in this context? HEAD is an alias for "whatever is currently checked-out right now". HEAD usally points to a branch that you work on, but if you git checkout some-commit-hash you get in a "detached HEAD" state where HEAD does not point to a branch but to a specific commit, and it can lead to you loosing new commits if you work from there and are not careful.=20 > * Polling git >=20 > It is beginning to sound like the FreshPorts git code for detecting incom= ing commits will be: >=20 > Every N minutes, do this: >=20 > for each repo in REPOS > for branch in BRANCHES > cd to the directory for that repo > git checkout branch > git log $branch_last_hash...HEAD Those two should probably be `git log $branch_last_hash...$branch_name`, you don't actually need to checkout the thing. > for each of those commits > process the commit > end for > end for > end for >=20 > At present, the REPOS and BRANCHES are: >=20 > * freebsd BRANCHES=3D"master branches/2020Q4 branches/2020Q3 branches/202= 0Q2 ...etc" > * freebsd-ports BRANCHES=3D"master stable/12 stable/11" > * freebsd-doc BRANCHES=3D"master" Those should probably not be static but should probably be parsed from, for example, `git branch -a`, possibly with a --format to be easily parsed. --=20 Mathieu Arnold --45wtwjfwliaf7fev Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAABCgB9FiEEVhwchfRfuV0unqO5KesJApEdfgIFAl+39tlfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDU2 MUMxQzg1RjQ1RkI5NUQyRTlFQTNCOTI5RUIwOTAyOTExRDdFMDIACgkQKesJApEd fgIsehAAg7Sk6N6kOOBX8dipQ8xmZbji5h6zon3LaM2uQczvcOqRw3RKfQO0a6ye ngCGvWLT5yacCQb60SjMP1CiySGpoXxV3F7t+6GyQchBdHhtndzKExpPBisKRwtb jUtcro1vTUR21kAeN9XPU6Yp3fFUTqjiofZCzdUD/SMdzzKDH2YyQHxsAIEvWvTP dHLtJ6RdsW0mql/+ERVMkDKfJnIN8xCuonPz9Cww+xAL0oFpxtCPBkww9L/kJneg aY+XJ6o5qN6PU63PzkypH+pcLYdGqiOkvBV84BxTei2YYejmucs5tv6b7egEgdkr cN5QhgJ3Uye5jIgnj+i7tZrpe0sIP2D75TWRHWZZdN5ZUqZsZSh5tPYi3RGtR4pj /uKZqQWL8NI87z6+Ka/vW6k9TRljjg3aH1KmKWKi0GEzcS8sTiLC+uAxqaAPTQd8 hG5ijTypkqqHq+a/8CRY+WsKe80Vb+3/nrgeWStO6c72g06QuNCuX01s1YcrXsjj ENb6KHidaZiUUb8bvxIRh1nZ+DlDwZAZeZsplwkonZ7DtKdDp6cclDcVohdrYgKV H33yqkyQU3H27nIuM6fBqkid7Eme7pQ4wxF5T1ApLPQK6EHvcXIov4ohTqOO/JBT Wy2/mrx5M7p3bxBm0n2FnCBWcwBiTPAF3eJuMOBRnHeBUulP0rU= =YgJs -----END PGP SIGNATURE----- --45wtwjfwliaf7fev-- From owner-freebsd-git@freebsd.org Sat Nov 21 14:12:27 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A2DD14681D4 for ; Sat, 21 Nov 2020 14:12:27 +0000 (UTC) (envelope-from dan@langille.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Cdb2y4Wn3z4mMt for ; Sat, 21 Nov 2020 14:12:26 +0000 (UTC) (envelope-from dan@langille.org) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 14B075C00AE; Sat, 21 Nov 2020 09:12:26 -0500 (EST) Received: from imap36 ([10.202.2.86]) by compute4.internal (MEProxy); Sat, 21 Nov 2020 09:12:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=langille.org; h= mime-version:message-id:in-reply-to:references:date:from:to:cc :subject:content-type; s=fm2; bh=jMkkZHsWiygjx6+r1OWuwJqXCIlUugQ sMIlFiV4FZv4=; b=DzHYv92GGUkHZRFHfpVSVOeV3HgW7sN/kGdWsc40ghjWSVR CJLO95z58Y+pnYwGvV3MdsM0YTG9ukWbF/6HzmAkNuHK0G6tnMlpT83NIGIqk3xw tZN1+UZPIbJRcEdrLSr8Oeth5+AHyhg5/P9q81AMiWEmiTzOh0glABh9KAHAHNzu bspbPw/Y5oawLFv638A9T3VjmLW1oYEEg4NaHwYetkFhcLbrG3NzgfR25L89VMkE UjAawH0NWIHC7F50Uob51q9170sA77Omc/Gugdu++lz4qAXeZwNbk9lrMlSKCaP5 IsuA6BtLsj/AJ3UI6L0LtDarimn0TPmJfeksRvA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=jMkkZH sWiygjx6+r1OWuwJqXCIlUugQsMIlFiV4FZv4=; b=iB4pDSaeRAFtR5Sbb5KOKM +9/iVzLDpF4+hzm8v/hiyEnp7TBmVXuTIh3xB+5YWUs5+zVguN8Z7ikoCUO/Zqwl JiDVa1HwPQ0jzoPVBjnipwNwIaCnFY4Dc2oOm0OktjPIPa6xzMth8pmHfPaNAeo0 tpkO32Wqwdyc9QZEgzRG90srjnbhys4H3w6x8wKC6gq3OWl0YakfrxgHsoYbUHzO egUQBGHxFlt4pNK0VjtAKSRtzpx47ORiNn8pPOM5KTR5S68cci4R8kL96QmnWEkS FyOwZT6eV//h4DGVx6JCbo3MTNjLPtnfuUuOCLlSqXxGVBV8eG1958ORkh4Yd56Q == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrudegvddgiedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsehttd ertderredtnecuhfhrohhmpedfffgrnhcunfgrnhhgihhllhgvfdcuoegurghnsehlrghn ghhilhhlvgdrohhrgheqnecuggftrfgrthhtvghrnhepleejudduvefhjefghedvffejte dvudfhfffghedtiefgueeljeetgfehkeeigeehnecuffhomhgrihhnpehsthgrtghkohhv vghrfhhlohifrdgtohhmpdhlohhnghgrihhrrdhnvghtnecuvehluhhsthgvrhfuihiivg eptdenucfrrghrrghmpehmrghilhhfrhhomhepuggrnheslhgrnhhgihhllhgvrdhorhhg X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id AF8F6188007F; Sat, 21 Nov 2020 09:12:25 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.3.0-622-g4a97c0b-fm-20201115.001-g4a97c0b3 Mime-Version: 1.0 Message-Id: <785f1386-f521-46d2-8e2e-a7c502fd943a@www.fastmail.com> In-Reply-To: <93aa2a3e-7e8a-8199-3feb-6e464a590d72@gmail.com> References: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> <3c9f6285-ae7c-1062-2dd3-42f8c953a230@gmail.com> <93aa2a3e-7e8a-8199-3feb-6e464a590d72@gmail.com> Date: Sat, 21 Nov 2020 09:11:53 -0500 From: "Dan Langille" To: "Marc Branchaud" Cc: freebsd-git@freebsd.org Subject: Re: Monitoring commits on all branches Content-Type: text/plain X-Rspamd-Queue-Id: 4Cdb2y4Wn3z4mMt X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=langille.org header.s=fm2 header.b=DzHYv92G; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=iB4pDSae; dmarc=pass (policy=none) header.from=langille.org; spf=pass (mx1.freebsd.org: domain of dan@langille.org designates 66.111.4.25 as permitted sender) smtp.mailfrom=dan@langille.org X-Spamd-Result: default: False [-3.09 / 15.00]; XM_UA_NO_VERSION(0.01)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.25]; RWL_MAILSPIKE_GOOD(0.00)[66.111.4.25:from]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[langille.org:+,messagingengine.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[langille.org,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.25:from]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[66.111.4.25:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[langille.org:s=fm2,messagingengine.com:s=fm1]; FREEFALL_USER(0.00)[dan]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[66.111.4.25:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_WWW(0.50)[]; MAILMAN_DEST(0.00)[freebsd-git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Nov 2020 14:12:27 -0000 On Thu, Nov 19, 2020, at 5:30 PM, Marc Branchaud wrote: > On 2020-11-19 2:00 p.m., Dan Langille wrote: > > * How FreshPorts extracts data > > > > FreshPorts is only interested in a snapshot of the repo with respect > > to a given commit. It works on the 'repo as a whole' to extract > > values from the ports which were affected by that commit. Case in > > point: a commit to a parent port might affect any or all of the child > > ports. All the child ports need to be refreshed. > > I'm guessing that typically the child ports' stuff (in the ports tree) > does not change when the parent port changes. So you're talking about a > build-time or run-time dependency between ports? I imagine you have > some kind of ports-dependency database, which you can consult when a > particular port is changed. If the parent port change PORTVERSION, then make -V PORTVERSION on the child port probably changes value. The database entry for the child is updated to make sure it reflects what is now declared in the parent. All child ports are updated when a commit affects a parent port. > Or are you thinking git might be able to help you here? This is just background information as to why I need a snapshot (i.e. checkout a commit). > > I am quickly concluding that FreshPorts must decide in advance what > > git branches it will pay attention to. At present, it follows all > > branches. > > > > * FreshPorts (without git) uses email to create XML > > > > When moving FreshPorts from subversion to git, one of the goals was > > to avoid relying on email to know that a commit has occurred. That is > > how FreshPorts has always worked. The email (from the CVS commit) was > > parsed and XML created. This code was updated for SVN. The XML is > > then used to load the commit into the FreshPorts database which then > > drives the website contents. > > > > When I started the GIT conversion, there was no commit email. "git > > log $foo_hash...HEAD" is how FreshPorts knows what commits to > > process. > > > > One positive aspect of email approach: it identified the branch. So > > far, I can't see how I can process the repo as a whole and see every > > commit and know what branch it was on. > > You're right about that, but you *can* discover all the branches in the > repo (see below). > > > * Polling git > > > > It is beginning to sound like the FreshPorts git code for detecting > > incoming commits will be: > > > > Every N minutes, do this: > > > > for each repo in REPOS > > for branch in BRANCHES > > cd to the directory for that repo > git checkout branch > > git log $branch_last_hash...HEAD > > for each of those commits > > process the commit > > end for > end for > > end for > > That loops looks basically right. You'll need to "git fetch" in each > repo before processing branches. I was usually doing a git pull. I've been reading up on 'git pull' vs 'git fetch'. I think I will move to 'git fetch'. To reiterate: FreshPorts basically gets a read-only copy of the repo. It never does local mods. It just needs the files. It is similar to 'svn up' with never changing the files. An 'svn export' would be workable, but in practice, 'svn up' is faster. This is what I was reading: * https://stackoverflow.com/questions/292357/what-is-the-difference-between-git-pull-and-git-fetch * https://longair.net/blog/2009/04/16/git-fetch-and-merge/ > Watch out with that "git checkout branch" part: This is where a lot of > people get tripped up when they move from svn to git. It doesn't help > that git tries to do some hand-holding here, but really git users would > benefit from simply understanding how branch names are just labels for > commit SHA IDs. Without that understanding, people end up going down > the rabbit-hole that is the "git pull" command, a wretched hive of scum > and villainy if there ever was one. I understand the concern and I think I follow. Within the confines of 'only wanting the files for reference, never local modification', pull and fetch might be the same for me. I'll look at fetch more. > I'll just gloss over a lot of detail (but feel free to ask!) and > recommend that you work with the "origin/"-prefixed branch names. These > get updated every time you "git fetch" from the remote repo, and > basically behave as you'd expect. It's appropriate in your case, > because you're not creating any new commits. > > > At present, the REPOS and BRANCHES are: > > > > * freebsd BRANCHES="master branches/2020Q4 branches/2020Q3 branches/2020Q2 ...etc" > > * freebsd-ports BRANCHES="master stable/12 stable/11" > > * freebsd-doc BRANCHES="master" > > > > Some might ask: > > > > * Why not just master and latest-quarterly for freebsd-ports? > > * Because commits to older branches sometimes occur (or at least I > > thought I saw one once) > > > > Commit hooks might also help, but I'm not sure if that will make > > things easier or complicate everything > > Looking through the set of hooks ("git help hooks"), I don't see > anything that would help you. My idea for a hook: FreeBSD tells FreshPorts when a new commit arrives. FreshPorts wakes up and processes it. > > * When new branches arrive > > > > It is vital that FreshPorts remain automated as much as it can be. At > > present, under SVN, I might have to fix things perhaps 5 or 6 times a > > year, usually because a commit did not get processed. > > > > Keeping that in mind, I do not yet know how to handle the following > > situations: > > > > A new branch comes out. > > > > * Automation might be possible for ports quarterly branches > > * FreshPorts has to know there is a new branch > > Fundamentally you start by doing a "git fetch" to retrieve updates from > the official repository (you probably know that already, but I just want > to be pedantic). > > Then "git branch -r" will list all the remote ("origin/"-prefixed) > branches, including whatever new ones were just fetched. > > You can also do "git for-each-ref refs/remotes/origin" to see which > commit SHA ID corresponds to each remote branch name (though the branch > names are prefixed with "refs/remotes/origin/" not just "origin/"). You > might find its output easier to parse than the equivalent "git branch > -rv" suggested by Oliver. Thanks. > > * BRANCHES needs to be updated > > * I don't see that it can be automated for stable/* > > Instead of saving BRANCHES locally, just use the output of one of the > above commands every time you run the script. That way the list will > always be up to date. Yes, I think so too. > > > * need to handle 'git checkout branch' when branch does not exist? > > If you end up doing that, I'd say you're doing something wrong. Agreed. > > * Once branch exists, how do you find out about the commits when you have no > > starting point for 'git log'? > > Use "git merge-base" to find where the new branch split off from the > "master" or "main" branch. That is my starting commit for that branch? > > > Right now, a new quarterly branch is noticed when the first commit > > email comes through. FreshPorts then does an 'svn co' for that > > branch. > > > > I'm hoping someone has good ideas for my edge cases. > > I hope I've been a bit helpful! More than a bit. > One last, small idea: Consider using tags in your local repos to track > your $branch_last_hash. For example, after you finish processing the > 2020Q4 branch, do > git tag -f last-2020Q4 origin/branches/2020Q4 > Then the next time through the 2020Q4 branch you can start at the > "last-2020Q4" tag. For a new branch that doesn't have a tag yet, use > "git merge-base" to find the starting point. Tags in my local repos won't get things upset? If I lose the local repo.... -- Dan Langille dan@langille.org From owner-freebsd-git@freebsd.org Sat Nov 21 19:07:00 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C391746DFF7 for ; Sat, 21 Nov 2020 19:07:00 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: from mail-qt1-x835.google.com (mail-qt1-x835.google.com [IPv6:2607:f8b0:4864:20::835]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CdjZq60Hlz3KMN for ; Sat, 21 Nov 2020 19:06:59 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: by mail-qt1-x835.google.com with SMTP id e60so561921qtd.3 for ; Sat, 21 Nov 2020 11:06:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=6GrDPzZFlol8mHxAHBL9I1vBp5ARf1CmWeDrPCzw/2I=; b=S6KAqnsNFhcT8hpIAL6cFvTjkaUMg5CILEf3fKC8rfdPdmSwJOKJ+ojkVbRkqTUlQ1 8JoQzRB3soUcJueo+zuXyY9hiJtQV9B6rj5d58natSG3gq19dxVsJHDER0f8hGOoiNvw jsCAKbTc2ESz2nMR1a/PAMwhLCFZgQtKS5aAbFg4S7VDk1x8koif9gboN/8B8SBgHluj qkbHcSygxnGX28pWJIFP1y0aHKA7rYg5/3qZjTTrJq1FxRAXS8iFpbvKLDFcUK9KqGrA Pbxy7zSudlY3y2tsxmZlSFpEWu405VaAI9OuTP/J7reOe4aHfwGr7balWsQmfdXKnyVc eQtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=6GrDPzZFlol8mHxAHBL9I1vBp5ARf1CmWeDrPCzw/2I=; b=eKW/2rGwCFTOIf8KibDgATWiBFB+BYN3dEaIs8DX/brfaxto+n3/xgn62xf9kqdvQC vpPyWSrdP3xIfKSEusZBTWu4p49EE1SjnGMtC9CGhsu9nGDXz13S1bCybUTF/7XG4+P0 s9HnTYL6sIDO2QdgUQjr9vXMyOIQWoNVD3+0LTfk1c76dLzD4hNMv7OvQ/UnTkHwQzAk OEqOR474HTUTqDkY6BD2ZL0x1yHeDfzTF7b3R7wc7bJqRl6LYVRIkOW1gjy0nTvJBI44 hd7bjXhOO4W6L15UJvBf2r5sq/Yy4+PYPtSAOGbYV8cScJubTZdc/RVfaazBdgxct1iL wxBw== X-Gm-Message-State: AOAM530e8yHPUXw08DjNA4cJzw25T3bNop3CwWyKA1oTjjpbO+2kRXB2 M9MQnAs5Plqx/Ohh0ioJurg= X-Google-Smtp-Source: ABdhPJx035IcKY19GXS3aLBtkN0G52cUz10xFJpLnsoOndCbBqJZ5LrzGjTEPUDlturCo1Htu2ouZg== X-Received: by 2002:a05:622a:206:: with SMTP id b6mr21701728qtx.285.1605985618647; Sat, 21 Nov 2020 11:06:58 -0800 (PST) Received: from [192.168.222.18] (192-222-183-158.qc.cable.ebox.net. [192.222.183.158]) by smtp.gmail.com with ESMTPSA id l91sm4543824qte.28.2020.11.21.11.06.57 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 21 Nov 2020 11:06:57 -0800 (PST) Subject: Re: Monitoring commits on all branches To: Dan Langille Cc: freebsd-git@freebsd.org References: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> <3c9f6285-ae7c-1062-2dd3-42f8c953a230@gmail.com> <93aa2a3e-7e8a-8199-3feb-6e464a590d72@gmail.com> <785f1386-f521-46d2-8e2e-a7c502fd943a@www.fastmail.com> From: Marc Branchaud Message-ID: Date: Sat, 21 Nov 2020 14:06:56 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: <785f1386-f521-46d2-8e2e-a7c502fd943a@www.fastmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4CdjZq60Hlz3KMN X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=S6KAqnsN; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of marcnarc@gmail.com designates 2607:f8b0:4864:20::835 as permitted sender) smtp.mailfrom=marcnarc@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::835:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::835:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::835:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Nov 2020 19:07:00 -0000 (trimmed some of the quoting) On 2020-11-21 9:11 a.m., Dan Langille wrote: > On Thu, Nov 19, 2020, at 5:30 PM, Marc Branchaud wrote: > >> Watch out with that "git checkout branch" part: This is where a lot of >> people get tripped up when they move from svn to git. It doesn't help >> that git tries to do some hand-holding here, but really git users would >> benefit from simply understanding how branch names are just labels for >> commit SHA IDs. Without that understanding, people end up going down >> the rabbit-hole that is the "git pull" command, a wretched hive of scum >> and villainy if there ever was one. > > I understand the concern and I think I follow. Within the confines of > 'only wanting the files for reference, never local modification', pull and > fetch might be the same for me. I'll look at fetch more. "git fetch" is just the "get updates from the remote repo" part of "git pull". The thing with pull is that it wants to "help" you update your local branches the "right" way. But the "right" way really depends on how you work and how your project works. So most advice to "do a 'git pull'" is only useful for the most general of cases. I consider "git pull" to be one of git's most advanced commands. It's very powerful, and can certainly make life easier -- once you actually understand how git works and how to configure pull to do what you actually want. I always advise git beginners to stay away from using pull until they have a firm grasp of git's branches and of how they want to use them. I personally never use "git pull" because I regularly want to use my local branches in different ways, even within a single project/repo. >> Looking through the set of hooks ("git help hooks"), I don't see >> anything that would help you. > > My idea for a hook: FreeBSD tells FreshPorts when a new commit arrives. > FreshPorts wakes up and processes it. That's certainly doable, though a bit unusual: Most "downstream" projects fetch updates from their upstream. I was assuming you didn't want any special consideration from the FreeBSD repositories. >>> * Once branch exists, how do you find out about the commits when you have no >>> starting point for 'git log'? >> >> Use "git merge-base" to find where the new branch split off from the >> "master" or "main" branch. > > That is my starting commit for that branch? No, it's the *parent* of your starting commit. To list the SHA IDs of all the commits in the branch, do: BASE=`git merge-base master..$BRANCH` git rev-list $BASE..$BRANCH >> One last, small idea: Consider using tags in your local repos to track >> your $branch_last_hash. For example, after you finish processing the >> 2020Q4 branch, do >> git tag -f last-2020Q4 origin/branches/2020Q4 >> Then the next time through the 2020Q4 branch you can start at the >> "last-2020Q4" tag. For a new branch that doesn't have a tag yet, use >> "git merge-base" to find the starting point. > > Tags in my local repos won't get things upset? Not at all: tags are local in git -- just make sure to choose unique tag names, since you don't want to collide with any tags your upstream repo might use. Tags can be pushed to / fetched from other repos, but since you're not updating the upstream FreeBSD repos (or any other public repo, I assume) you can do anything you want with your tags. > If I lose the local repo.... That problem exists no matter how to choose to store state for your scripts. With git it's easy to backup your local repository, including its tags: just "clone --mirror" it. M.