From owner-freebsd-git@freebsd.org Wed Aug 19 14:36:06 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6A3263C0802 for ; Wed, 19 Aug 2020 14:36:06 +0000 (UTC) (envelope-from dan@langille.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BWr1d4nSQz3dkM for ; Wed, 19 Aug 2020 14:36:05 +0000 (UTC) (envelope-from dan@langille.org) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 702E25C0135 for ; Wed, 19 Aug 2020 10:36:04 -0400 (EDT) Received: from imap36 ([10.202.2.86]) by compute2.internal (MEProxy); Wed, 19 Aug 2020 10:36:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=langille.org; h= mime-version:message-id:in-reply-to:references:date:from:to :subject:content-type; s=fm2; bh=K2T6aifbSGc5MAQZIqz4A/tByidcUNf 4nQoBfia/qSQ=; b=IYrVU0HBAwrndMpmAjZTuW0leEg9nAh1FcueX7kls9Qor6A llMdmi47YfbdY5Vq8xUNobcB808O+ARqf1XdlCvuEFbygeJ6uFEj95n2QxJRw56a X2MDBoIz+hqWTtuMQUZY0SRww3u1ZstXPMSHxqrZyXUgChJJ/wfEwyig7M7hlEBh GUznZPJ7CetPmjxj0rWoxs75+1IkKQvZ1S73SrJMxvHMdNpnfDjYVBN8su4Pszl6 Y+BjJWnzB+LAQ1SrFhOTYUYwrpR51Ww27v9v+A5rU5KCATgJfFsUb5X9aqoll9QR hAQdW8xuYIjmM3JlGFxKrerJGGlHau5Iu66QtJQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=K2T6ai fbSGc5MAQZIqz4A/tByidcUNf4nQoBfia/qSQ=; b=t62lp89CbIAA6NQ0+1DNef 87IWqchwB0ovGp5WFYe4wl8+4+Dp7ebhMBjDcbtyeayPuOd4OcLUik7H4IM3m0UW vtiwJBEaGNyA9Y6ExKAL+PDvjuiKBpqwn2Cj5KPkY9gTVz3gKMu6E0R31Q5D+gjL PsJG63HZuD0pRAI4O9jRmqeHY9aOVbo93n7ua0Im6/O/+5J5Rq26fbWnZu4MG4vi sDrakGbLyWGqYP8x3kh3+1WbXJkbzWeX3KfWEgpnMGqsfNr7qenfeXT5O6konyvc RZmB/Snf8BFqJAdhm2Y339snLpTzUuQQWTSRlfX6ydblscS2OVvH97m7L9eQBzGQ == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduiedruddtkedgheehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsehttd ertderredtnecuhfhrohhmpedfffgrnhcunfgrnhhgihhllhgvfdcuoegurghnsehlrghn ghhilhhlvgdrohhrgheqnecuggftrfgrthhtvghrnhepteejhedtjeefueeujeevgfdvie dvieetueduieeiveeludegheekvdeviefhteegnecuffhomhgrihhnpehgihhthhhusgdr tghomhenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe gurghnsehlrghnghhilhhlvgdrohhrgh X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id E97481880081; Wed, 19 Aug 2020 10:36:03 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.3.0-191-gef79d59-fm-20200818.001-gef79d590 Mime-Version: 1.0 Message-Id: In-Reply-To: References: <3430278.PbzQmE3shN@beastie.bionicmutton.org> <15d19351-3c4c-4ad1-bd1a-7f8f6c2ae211@www.fastmail.com> <20200819124724.yoni3kjxzlboknlq@aching.in.mat.cc> Date: Wed, 19 Aug 2020 10:35:42 -0400 From: "Dan Langille" To: freebsd-git@freebsd.org Subject: Re: FreshPorts & src commits to branches Content-Type: text/plain X-Rspamd-Queue-Id: 4BWr1d4nSQz3dkM X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=langille.org header.s=fm2 header.b=IYrVU0HB; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=t62lp89C; dmarc=pass (policy=none) header.from=langille.org; spf=pass (mx1.freebsd.org: domain of dan@langille.org designates 66.111.4.25 as permitted sender) smtp.mailfrom=dan@langille.org X-Spamd-Result: default: False [-0.55 / 15.00]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[langille.org:s=fm2,messagingengine.com:s=fm3]; XM_UA_NO_VERSION(0.01)[]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[66.111.4.25:from]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FREEFALL_USER(0.00)[dan]; RCVD_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.25]; DKIM_TRACE(0.00)[langille.org:+,messagingengine.com:+]; DMARC_POLICY_ALLOW(-0.50)[langille.org,none]; NEURAL_HAM_SHORT(-0.46)[-0.464]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.25:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; MAILMAN_DEST(0.00)[freebsd-git]; MID_RHS_WWW(0.50)[] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2020 14:36:06 -0000 On Wed, Aug 19, 2020, at 9:06 AM, Dan Langille wrote: > > > On Wed, Aug 19, 2020, at 8:47 AM, Mathieu Arnold wrote: > > On Tue, Aug 18, 2020 at 09:23:03AM -0400, Dan Langille wrote: > > > On Tue, Aug 18, 2020, at 3:44 AM, Adriaan de Groot wrote: > > > > On Tuesday, 18 August 2020 03:19:26 CEST Dan Langille wrote: > > > > > I am quickly realizing that in order for FreshPorts to track src commits it > > > > > needs to query every branch. There does not seem to be any other way to > > > > > know that a commit ocurred on stable/12 without a checkout of stable/12 > > > > > > > > If you have the full repo (e.g. a normal clone) then you might be able to use > > > > something along this line: > > > > > > > > git log --branches --since 2020-08-16 --source --raw | grep ^commit > > > > > > > > (the important bit is --branches to include all of them, and --source to > > > > include the source). I don't have a FreeBSD repo at hand, but my work-repo > > > > (which currently has some other branch checked out) gives me this kind of > > > > output: > > > > > > > > commit 36cefe3be10c707a2cdfaedd14fddee5cda31f50 hotfix-3.2.28.2 > > > > commit eb44d0c6be18c1ced99d5a8c1b1a6a1a0e653be2 hotfix-3.2.28.2 > > > > commit 24efce0c1d5601a77ad2b919cf0cf387231add76 calamares > > > > commit 5db4195b7e3e507a34f84e041b15a4aa86ed32da hotfix-3.2.28.2 > > > > commit 34946ecdee90584caf255fcca732c021e1dc1832 hotfix-3.2.28.2 > > > > commit 1a7b2668dc1e341b493990b99e9a0cc4530d32cd hotfix-3.2.28.2 > > > > > > > > This gives me branches for each commit hash in the given range. The branches > > > > you see mentioned here diverged before the date range given. I can't say > > > > anything about performance, here -- my little test repo only has ~8000 commits > > > > going back six years. > > > > > > > > (Turning this command-line into a collection of libgit calls is left as an > > > > exercise for the reader) > > > > > > FYI, FreshPorts is using devel/py-pygit2 which uses devel/libgit2 :) > > > > > > After a git pull, I same up with this: > > > > > > [dan@devgit-ingress01:~/src/freebsd/freebsd] $ git log --branches --since 2020-08-17 --source --raw | grep ^commit > > > commit ef1942cafc385f2ebc7e4a7a5f5679bb1a4b0d78 stable/12 > > > commit 3889b52f5c73875df9a28180c4b1b4a07bf0042a stable/12 > > > commit 9fc2a101135102074391b6ff28b825b082410ffa master > > > commit a9a7211e27bfe2bec89b08899b977a4677728d97 master > > > ... several lines omitted from this paste > > > commit 5853f38da725897c25a505ce62675e31aca27f26 master > > > commit 1b47d5c7b7fb338daf65ccf05b9df0d9e93d11ae stable/12 > > > > > > Let's try a date time in there: > > > > > > [dan@devgit-ingress01:~/src/freebsd/freebsd] $ git log --branches --since "2020-08-17 23:00:00" --source --raw | grep ^commit > > > commit ef1942cafc385f2ebc7e4a7a5f5679bb1a4b0d78 stable/12 > > > commit 3889b52f5c73875df9a28180c4b1b4a07bf0042a stable/12 > > > > > > Those were both at 23:40 yesterday. > > > > > > The current approach is: I have this commit, show me all the commits after that: hash..HEAD > > > > > > That works on a branch-by-branch basis. > > > > > > > > > This may be a potential issues: The --since & --branches approach requires a date/timestamp. If we always use the timestamp of the last time we asked, will we skip over commits? > > > > Well, I would run something like --since '2 weeks ago' and ignore what > > commits you already have. It will maybe work, or not, it will probably > > depend on wether we allow branches or out of order commits, or... > > > > It would probably be safer to do for each branch, look at the commits in > > the ... range, which would get you all the commits, > > proably even if we allow merge commits. > > > > > Humor me while I contemplate URLs. > > > > > > The 3889b52f5c73875df9a28180c4b1b4a07bf0042a commit translates to: > > > > > > https://github.com/freebsd/freebsd/blob/stable/12/sys/sys/ata.h > > > > Well, nom the 3889b52f5c73875df9a28180c4b1b4a07bf0042a commit translates > > to > > > > https://github.com/freebsd/freebsd/blob/3889b52f5c73875df9a28180c4b1b4a07bf0042a/sys/sys/ata.h > > > > The URL you have is for the head of the stable/12 branch. > > Yes, you are correct. I was trying translates a commit on a branch to > the patch to the file affected by that commit. I think I got that > correct. Ouch. I was trying to translate a commit on a branch to the file path within the repo. -- Dan Langille dan@langille.org