From owner-freebsd-git@freebsd.org Thu Nov 19 16:16:37 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CF8CE46F17D for ; Thu, 19 Nov 2020 16:16:37 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: from mail-qt1-x82b.google.com (mail-qt1-x82b.google.com [IPv6:2607:f8b0:4864:20::82b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CcPv90Ywlz4qkX for ; Thu, 19 Nov 2020 16:16:36 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: by mail-qt1-x82b.google.com with SMTP id g17so4754263qts.5 for ; Thu, 19 Nov 2020 08:16:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-language:content-transfer-encoding; bh=wTiU34XArvVx3zKxyfcCcBjxrOgZBX3pp9QXgBOUJKo=; b=OBggC8lIuCfFzVy4EWbHzweGd9r3F46jMZAj2B09k4WPTW0JNjyR8hHODdlO/jm0bX zLnputAwmU9TjEB+zU62wJjGn/qYEh4y9SfE/oFh/+Fgue3eL0r1fCwMuJzS5w7fwaNC M5HQ8Q3jmvxaaKBmdkF7SP/SFYW1irBZITTHNy7qEoKmOSL3cIJOhCNe8hd/rcfVyT1l HzjhJ0j83rax9Nq2HyPXjlKilLSQMgBKZu8PmWf9MxJHeXIi9lZOrdDIB4IpJxAnjIkQ aby9ufFA6XHLVo1KcgfIs2wP7/jCPumXPzymhGpnyOWtUNvp6JDDFnkLgGwCAgg8h9tN tJKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=wTiU34XArvVx3zKxyfcCcBjxrOgZBX3pp9QXgBOUJKo=; b=VCPQQsMdoVmAtoM0YR5RxYYcrNBz8bDRZ/Q9MJSfysWFaRDv74JCUU/MDaN6v3+5g/ 47pOwp9pvk4Zvn/ddcNDHbFADInn299uQfVIO13MKkG3q2ej9uV2DQFSamUf1haJq2H1 fh0nQKPipSjBm4F9W39T5CQ2hTG7XZpSMbUreiwgHU9SVwWPPfClhOfSjPFO85q2JP03 s4/FkIgDeQT8WieZUHgfqQKGtcHlnh6zmAbNy8aX5qdEuHP/Q2vlYhebzXYbSTfgk5Ju geNI0ZMheKg5nBmS/vQd2sU8LmSquvauAsOP6zUY8dTH3YjxZlSQDVS69iasf9mZHc3y Ap3Q== X-Gm-Message-State: AOAM531CyyfPXzPj4XF2JGarv8H+m+6ZRp5ze7+NIBMQtrotnAWLAeqA 6dNQjwka0tYa6uM9MKqlbcY= X-Google-Smtp-Source: ABdhPJwRePPzvzW6ncK1d+GLHbk33hEO8P599ZPnu3Q+X4FNRz4LOd/ibOIluhIQrhJ2CnMFheglMw== X-Received: by 2002:ac8:5351:: with SMTP id d17mr11461585qto.235.1605802595864; Thu, 19 Nov 2020 08:16:35 -0800 (PST) Received: from [192.168.222.18] (192-222-183-158.qc.cable.ebox.net. [192.222.183.158]) by smtp.gmail.com with ESMTPSA id g9sm192102qtq.21.2020.11.19.08.16.35 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 19 Nov 2020 08:16:35 -0800 (PST) Subject: Re: Monitoring commits on all branches To: Dan Langille , freebsd-git@freebsd.org References: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> From: Marc Branchaud Message-ID: <3c9f6285-ae7c-1062-2dd3-42f8c953a230@gmail.com> Date: Thu, 19 Nov 2020 11:16:34 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4CcPv90Ywlz4qkX X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=OBggC8lI; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of marcnarc@gmail.com designates 2607:f8b0:4864:20::82b as permitted sender) smtp.mailfrom=marcnarc@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::82b:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::82b:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::82b:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 16:16:37 -0000 On 2020-11-18 8:49 p.m., Dan Langille wrote: > How can a repo be monitored for commits on all branches? > > I know how to ask a given branch: do you have any commits after foo_hash? > > How do I: > > * get a list of all commits since foo_hash A quick a note about Warner's reply: > git log $hash..HEAD "HEAD" is just a git nickname for "whatever you have currently checked-out" (which can be a branch, a tag, or "detached" commit SHA ID). > * know which branch each of those commits was on (e.g. master, branches/2020Q4) Unfortunately you'll find most normal git advice to be a bit frustrating with the FreeBSD repos, because FreeBSD doesn't work the way most people use git. Specifically, the FreeBSD project does not ever merge branches (in the git sense of the word "merge"). Things would be very, very much easier if the FreeBSD project were to use git-style merging. I believe there are discussions underway about adjusting the whole MFC process for the git world. I admit that part of my motivation in writing this message is to provide grist for that mill. Fortunately even without git-merged branches, there are still git tools that help, though they're not as precise as one would like. Let's look at a concrete example with the beta ports git repo (which I just cloned), and compare the 2020Q4 and main branches. I'll start with some overall exploration, then address your specific question. There are 298 commits in the 2020Q4 branch. I know this because git merge-base origin/main origin/branches/2020Q4 tells me where 2020Q4 branched off of main: commit 5dbe4e5f775ea2. And git rev-list 5dbe4e5f775ea2..origin/branches/2020Q4 | wc -l says "299". (The "rev-list" command is a bare-bones version of "log" that only lists commit SHA IDs.) Meanwhile there have been 4538 commits to the main branch since commit 5dbe4e5f775ea2. As far as git is concerned, those 299 commits in 2020Q4 are *different* from anything in main. Even though most of them made the exact same code changes, they were created at different times, often by different authors, and they have different commit messages. But you can still ask git to look at the code-change level to see which 2020Q4 commits exactly replicated the code change from main: git cherry -v origin/main origin/branches/2020Q4 This little piece of magic looks at the 299 commits in 2020Q4 that are not in main and compares their code changes to the 4538 commits in main that are not in 2020Q4. It prints out the 299 2020Q4 commit SHA IDs, prefixed with either a "- " or a "+ ". The -v appends the commit message's first line: - 394d9746e5eea73f56334b2e7ddbdc8f686d6541 MFH: r550869 + 1ac9571956759c91d852ee92859a12e52dcbde48 MFH: r550885 r550886 - fd411bdfda55488b84de75e6b043c513a281abf0 MFH: r551209 - 533cdaa97457b3318aebcc53f7a1a46ea66721da MFH: r551236 ...... A "-" means that the commit matches the code change made by a commit in main, while a "+" means that the commit's code change does not *exactly* match any main commit since commit 5dbe4e5f775ea2. So git cherry -v origin/main origin/branches/2020Q4 | grep ^- shows us the 234 2020Q4 commits that made the exact same change as a commit in main. And git cherry -v origin/main origin/branches/2020Q4 | grep ^+ shows us that there are 41 not-exactly-the-same-change commits in 2020Q4. Mostly these are ones that combined two or more MFH's into one commit (e.g. 2020Q4 commit 1ac95719567), or that changed a file in a slightly different way (see the first patch hunk of 2020Q4 commit cbd002878f2, compared to its counterpart in main: commit a5d21ea16b6). Now to your specific question: Given a commit, how can we tell which branches contain that code change? Let's look at main commit 6a9a8389d609 which I've determined, through manual spelunking, matches 2020Q4's commit 02eba4048564. At a basic level, "git cherry" can tell us that *something* in 2020Q4 made the same change as commit 6a9a8389d609. Here I reversed the order of the branch names in the command: git cherry origin/branches/2020Q4 origin/main | grep 6a9a8389d609 This outputs: - 6a9a8389d609ca0370c8c6eb8f993c1aa4071681 and the "-" tells me that 6a9a8389d609's code change is *somewhere* in 2020Q4 unique 299 commits. Unfortunately there's no convenient git command that'll tell you *which* 2020Q4 commit replicated commit 6a9a8389d609. For that, we need to do a bit of scripting: -----8<-----8<-----8<-----8<----- #!/bin/sh TARGET="6a9a8389d609" BASE=`git merge-base origin/branches/2020Q4 origin/main` TARGET_PATCH_ID=`git show -p $TARGET | git patch-id --stable | cut -f 1 -d ' '` for REV in `git rev-list $BASE..origin/branches/2020Q4`; do PATCH_ID=`git show -p $REV | git patch-id --stable | cut -f 1 -d ' '` if [ "$PATCH_ID" = "$TARGET_PATCH_ID" ]; then echo "Found a commit that replicated target commit $TARGET:" echo git show -s $REV exit 0 fi done echo "Did not find any commit that exactly replicated $TARGET." exit 1 ----->8----->8----->8----->8----- This only looks at the 2020Q4 branch, but it's easily adapted to look at a user-specified branch, or multiple branches. (In the above I used "git patch-id", which is what "git cherry" uses internally to identify a commit's code changes.) I hope all this helps a bit! M.