From owner-freebsd-git@freebsd.org Mon Nov 23 19:43:36 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id ADE9646B183 for ; Mon, 23 Nov 2020 19:43:36 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: from mail-qv1-xf2f.google.com (mail-qv1-xf2f.google.com [IPv6:2607:f8b0:4864:20::f2f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CfyJ8036Vz4j7r for ; Mon, 23 Nov 2020 19:43:35 +0000 (UTC) (envelope-from marcnarc@gmail.com) Received: by mail-qv1-xf2f.google.com with SMTP id k3so2551627qvz.4 for ; Mon, 23 Nov 2020 11:43:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=9fDbH+a2Yq9FewmvvPrqw4CWCOihogLy8MOfMEFA2mk=; b=W5rYGR22UGlqlFnbP0HR1rnspgQYzvisX/DmhUJLjeya2z6izdZ/9mGcjx5GRmY7vj wSRQBAZVQZcrBpHYscYMMXlEMaUXJH2lOBgwnj0cX86Qo6ZqOBrqUiXJOMWePDgDobVa H6XjN33eZekZcMXgSEPJfi5nXp3OUnoOosIIK3QGp0L9d1Bh0V96wi1RLT1n2JVZVWUL bIwtpufdAn1FuT9f/Uchkhz0o4Q9y9NR+zu7dd8VBI2e4jKQxeN/3bnxFIZD0O4pdXD3 rVgrDi7oPH3b62S5BIlQ9CQHM3vMqJiy2vB6L25EJ+Upu87M/1EdJ2CYvvZqFbqD1OW9 B67g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=9fDbH+a2Yq9FewmvvPrqw4CWCOihogLy8MOfMEFA2mk=; b=myl4ONby92n9TzwKlFhKpFTZJRFPfXDN6W5me2chvqQoBGkS/DC7p+T5M3gHi+YiWr AyDTumyP6hmL1ud1RjneedvmTVVwBRqMu4aUGt3QJofXx1OjvRqiIbRnkeYpytT3J6X4 GzNjv5zT649Tt0GNRn2M41XNkSu2FgjLqaSuXHAwJcEvvHlpk0EfnYAv0CgYFEBCixys 7TRg2AYYY4koJIeMH54YuPyG4v8E4K6P7wZk8SfnSEQQn0Jvuf+F0eq2CC7+hFQR8G6i XOPxVtogGmON2UuaaLd6ZVk7PG2y4WrQMDehAq/UD4KaywAr5dq111LCun4xw5z74Ef5 6F7w== X-Gm-Message-State: AOAM532KOTzr3/B3vYKvAH/jNfrtvBxECwdzoc8Qwq9FdInw7A1mE/3J oOGwbrWBJqTmc2pKLHSbcXPlv1n4iAdkRsNj X-Google-Smtp-Source: ABdhPJz886ZfKo8ouiBQ32Rp5vv2H44j49XfMzSvZRT6n7tKimYATF8W1TEeyu+7GtqtkPNIXsHNHA== X-Received: by 2002:a0c:c290:: with SMTP id b16mr1008056qvi.53.1606160615174; Mon, 23 Nov 2020 11:43:35 -0800 (PST) Received: from [192.168.222.18] (192-222-183-158.qc.cable.ebox.net. [192.222.183.158]) by smtp.gmail.com with ESMTPSA id y35sm11511446qty.58.2020.11.23.11.43.33 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 23 Nov 2020 11:43:34 -0800 (PST) Subject: Re: Monitoring commits on all branches To: Dan Langille Cc: freebsd-git@freebsd.org References: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> <3c9f6285-ae7c-1062-2dd3-42f8c953a230@gmail.com> <93aa2a3e-7e8a-8199-3feb-6e464a590d72@gmail.com> <785f1386-f521-46d2-8e2e-a7c502fd943a@www.fastmail.com> <6EA93D4C-B399-413D-8C35-224103CC48D6@langille.org> <9B99AE2C-0497-432F-823B-290FDF53233C@langille.org> From: Marc Branchaud Message-ID: Date: Mon, 23 Nov 2020 14:43:33 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: <9B99AE2C-0497-432F-823B-290FDF53233C@langille.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4CfyJ8036Vz4j7r X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=W5rYGR22; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of marcnarc@gmail.com designates 2607:f8b0:4864:20::f2f as permitted sender) smtp.mailfrom=marcnarc@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::f2f:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::f2f:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::f2f:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Nov 2020 19:43:36 -0000 On 2020-11-22 8:47 p.m., Dan Langille wrote: >> On Nov 22, 2020, at 7:58 PM, Dan Langille wrote: >> >> Today I managed to sit down and go through the multiple scripts involved in pulling in a commit. >> >> https://news.freshports.org/2020/11/22/git-commit-processing-how-is-it-done/ >> >> Some changes were made.... >> >>> On Nov 21, 2020, at 2:06 PM, Marc Branchaud wrote: >>> >>> (trimmed some of the quoting) >>> >>> On 2020-11-21 9:11 a.m., Dan Langille wrote: >>>> On Thu, Nov 19, 2020, at 5:30 PM, Marc Branchaud wrote: >>>>> Watch out with that "git checkout branch" part: This is where a lot of >>>>> people get tripped up when they move from svn to git. It doesn't help >>>>> that git tries to do some hand-holding here, but really git users would >>>>> benefit from simply understanding how branch names are just labels for >>>>> commit SHA IDs. Without that understanding, people end up going down >>>>> the rabbit-hole that is the "git pull" command, a wretched hive of scum >>>>> and villainy if there ever was one. >>>> I understand the concern and I think I follow. Within the confines of >>>> 'only wanting the files for reference, never local modification', pull and >>>> fetch might be the same for me. I'll look at fetch more. >>> >>> "git fetch" is just the "get updates from the remote repo" part of "git pull". >>> >>> The thing with pull is that it wants to "help" you update your local branches the "right" way. But the "right" way really depends on how you work and how your project works. So most advice to "do a 'git pull'" is only useful for the most general of cases. >> >> Commits on the quarterly branch are now being automatically processed. Glad to hear you've made some progress! >> Now that the code is not doing 'git checkout master', which made sense on master, things are going better. >> >> git fetch >> git checkout master >> git rebase $REMOTE/master >> >> Now its just doing: >> >> git pull >> git rev-list ..HEAD >> >> This seems to be working. >> >> However, that may change if I start doing local tags. > > I just tried this. > > While I understand the issue around > > [dan@devgit-ingress01:~/src/freebsd/freebsd-ports] $ git fetch > remote: Enumerating objects: 220, done. > remote: Counting objects: 100% (194/194), done. > remote: Compressing objects: 100% (60/60), done. > remote: Total 111 (delta 60), reused 100 (delta 49), pack-reused 0 > Receiving objects: 100% (111/111), 18.38 KiB | 4.59 MiB/s, done. > Resolving deltas: 100% (60/60), completed with 36 local objects. > From https://github.com/freebsd/freebsd-ports > 24a927a0ef5d..bba44fef62c0 master -> origin/master > e3989b2f380e..9944c9cd2c35 branches/2020Q4 -> origin/branches/2020Q4 > 7c20845bd227..430121ca5ac7 svn_head -> origin/svn_head > > > OK, I see how I might be able to use that output, I can see which branches have new commits. > > > [dan@devgit-ingress01:~/src/freebsd/freebsd-ports] $ git merge > Updating 24a927a0ef5d..bba44fef62c0 > Fast-forward FYI, the "Fast-forward" here is git's way of saying "The branch you currently have checked-out did not add any new commits compared to the remote repo's version of the branch, so I'm just going to make your checked-out branch point to the same commit as on the remote repo." A "fast-forward" merge means that git did not do any patch-like operations to combine different sets of code changes. > audio/hpsjam/Makefile | 2 +- > audio/hpsjam/distinfo | 6 +- > devel/Makefile | 1 + > devel/php80-intl/files/patch-icu68 | 222 ---------------------------------------- > devel/py-pytest-datadir/Makefile | 27 +++++ > devel/py-pytest-datadir/distinfo | 3 + > devel/py-pytest-datadir/pkg-descr | 4 + > graphics/tiled/Makefile | 7 +- > graphics/tiled/distinfo | 6 +- > graphics/tiled/pkg-plist | 4 + > irc/irssi/Makefile | 81 +++++++-------- > irc/irssi/distinfo | 6 +- > irc/irssi/files/patch-Makefile.in | 22 ---- > irc/irssi/files/patch-configure.ac | 15 --- > irc/irssi/files/patch-meson.build | 22 ++++ > irc/irssi/files/patch-perl-Makefile | 22 ---- > irc/irssi/pkg-plist | 10 +- > lang/php80/Makefile | 2 +- > lang/php80/distinfo | 6 +- > mail/mutt/Makefile | 2 +- > mail/mutt/distinfo | 6 +- > multimedia/mpv/Makefile | 33 +++--- > multimedia/mpv/distinfo | 10 +- > multimedia/mpv/pkg-plist | 4 +- > net-im/signal-cli/Makefile | 13 +-- > net-im/signal-cli/distinfo | 14 +-- > net-im/signal-cli/pkg-plist | 4 +- > net/fb303/Makefile | 16 ++- > net/fb303/distinfo | 6 +- > net/fb303/files/patch-build_fbcode__builder_CMake_FBThriftLibrary.cmake | 2 +- > net/fb303/files/patch-fb303_thrift_CMakeLists.txt | 22 ---- > net/fb303/pkg-plist | 4 +- > net/geoipupdate/Makefile | 23 +++-- > net/geoipupdate/distinfo | 18 +++- > net/geoipupdate/files/patch-Makefile | 13 ++- > net/netatalk3/Makefile | 2 +- > net/netatalk3/files/patch-etc_afpd_volume.c | 20 ++++ > net/netatalk3/files/patch-libatalk_vfs_extattr.c | 19 ++++ > sysutils/rpi-firmware/Makefile | 4 +- > sysutils/rpi-firmware/distinfo | 6 +- > sysutils/rpi-firmware/pkg-plist | 4 +- > 41 files changed, 265 insertions(+), 448 deletions(-) > delete mode 100644 devel/php80-intl/files/patch-icu68 > create mode 100644 devel/py-pytest-datadir/Makefile > create mode 100644 devel/py-pytest-datadir/distinfo > create mode 100644 devel/py-pytest-datadir/pkg-descr > delete mode 100644 irc/irssi/files/patch-Makefile.in > delete mode 100644 irc/irssi/files/patch-configure.ac > create mode 100644 irc/irssi/files/patch-meson.build > delete mode 100644 irc/irssi/files/patch-perl-Makefile > delete mode 100644 net/fb303/files/patch-fb303_thrift_CMakeLists.txt > create mode 100644 net/netatalk3/files/patch-etc_afpd_volume.c > create mode 100644 net/netatalk3/files/patch-libatalk_vfs_extattr.c > > That output might also be useful for knowing what ports are affected, but really, this is what brings everything in. You can get also that output when comparing any two commits in the repo, with "git diff --stat" (or "git diff --numstat" for a more parsable format): git diff --numstat 24a927a0ef5d origin/branches/2020Q4 > '24a927a0ef5d' is taken from the 'git fetch' output but would also be the LAST_COMMIT for this branch. > > [dan@devgit-ingress01:~/src/freebsd/freebsd-ports] $ git rev-list 24a927a0ef5d..HEAD > bba44fef62c062bf942dd5312128b43dbd813dcf > 6a1e09447b223d078f78f7a7a9072e9842343c76 > dffe7ea4b193785b7309dd161e76e9d626e12b96 > f1a85af087281cfc6cd117301d7932472bd51adc > 2cbf3d698dc5e2d265c8f58447f5dea3e0a9b7ee > e9e1f60c3603041bcca9dda1e95918dbfcd3e480 > 352ce94919051329b591ef3534541f3138e52be7 > 0140eb70289386ad0d2b5a660ae1108a2c6eed45 > e273657ca09b1d2c862b24eb1154789f24e69ea6 > 7a79ec535067cd83fde3b8800509680cf1e60a27 > 182883a91022b14581cd951a733fd7ca1538ba73 > 494e0eafb5c57aadf36e6ef911f922169152cf1a > c1ff2a4159b9fd8d2199bb908a4facfb9b0e6399 > > > Next, I tried a tag, but went over to the freebsd source repo, because it has more recent commits. > > Command output is not shown. > > [dan@devgit-ingress01:~/src/freebsd/freebsd] $ git tag freshports_last_commit > [dan@devgit-ingress01:~/src/freebsd/freebsd] $ git fetch > [dan@devgit-ingress01:~/src/freebsd/freebsd] $ git merge > [dan@devgit-ingress01:~/src/freebsd/freebsd] $ git rev-list freshports_last_commit..HEAD > 5902bf651a44a052969477e3ed4ee0c382dd7b90 > a0e591191ab3d906d2d3d1a6985d424513c69c70 > 8a81e9904d4c3bf58328345f958f5fa5ac3d0801 > 9c6918af8b7927e77d829b7226c456415c029bdf > 6006818f9acd67f5b45d262ec949f0403b34100d > bdbb0128dce50974d8140ad511a1a895db205ece > bf39ccedb1130d2041d937ab6bfa0e6d32f10b68 > df7e32b0f26fd7500b7c1360071ed323fb5b7639 > 236ac76160e57a74747dc4453961fe5bbd18d7fb > 4b194b2e1ccbeb53c7b1a22169fdbcb79ef985ea > e8a0e692d11de506b18de046891293a5d03f15e2 > 45011cba53a65bbcaeb91dba13798d49387f34b3 > 2af377ccd66ed48f2ac8c3677fddfd6ad2608a7b > d97705ee7cb3e6405dd60a1d6f03d9737754593a > dbc3ee07f871a067e0d72d3463992370879a4372 > 7f2206c5a9f89d5e47cce05b79cfeead2b6598f5 > 2241a9b4d5b35fb0fb9ea0eb0f87dc7f0463edc3 > b766790d1f5c8304463537f537a6955f1e0df442 > cbc574bf4544f824705626383f9bef33d9a26089 > 3cc7c8abf47a2558fa0f1179905814ee167257b9 > 5844c0bcff59fd35dc2e0906bb696b17289e5e5c > e6e213967ba710191f3d5d912c9ecf70d7e26b8a > 1dab84551f51bf73e9653559dd7c4aff547719b4 > 50db1dd05447122f5d2c1f2b60e9bab771294bea > > After the last commit is processed, this gets done: > > $ git tag freshports_last_commit > fatal: tag 'freshports_last_commit' already exists > > Oops: > > $ git tag -f freshports_last_commit > Updated tag 'freshports_last_commit' (was 4a90ed0de4e) > > A compromise might be just logging the last commit somewhere so it can be derived easily if all is lost. > > Mind you, this might be the same thing. > > SELECT hash FROM commit_log order by id desc limit 1 > > .. assuming everything gets inserted into the database in the correct order. > > That might work. Just go back a few dozens commits and use that value. If we reprocess commits, no bit deal. They get ignored if already in the database. You should do whatever works for you, of course. Storing your script's state within your git repo is just one option. M.