From owner-freebsd-git@freebsd.org Tue Aug 18 13:23:26 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 23F973BBBCB for ; Tue, 18 Aug 2020 13:23:26 +0000 (UTC) (envelope-from dan@langille.org) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BWBSG097pz436j; Tue, 18 Aug 2020 13:23:25 +0000 (UTC) (envelope-from dan@langille.org) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 6F4BA5C00A0; Tue, 18 Aug 2020 09:23:25 -0400 (EDT) Received: from imap36 ([10.202.2.86]) by compute2.internal (MEProxy); Tue, 18 Aug 2020 09:23:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=langille.org; h= mime-version:message-id:in-reply-to:references:date:from:to :subject:content-type; s=fm2; bh=mpLZqZoEWCLWiiGmRCRXu6fBm/B/QOW +YCDHkx++tG0=; b=QZcHwUjsrIH/S4XBQ3cM5mrS0YIy0VLSYRNDTyLSqDrX9Ow 09bvfL1mqw2pQ9bquee8TAQ0Jg3d1Lrx23BJlCREsbpnD1tuoyWCF8ckN7urhMDT RD2E9kvsqh6GvEgntJUdY2q26E2rbeCUDQ2BospTJCnGOGd8fCg7hXCDNNZV2cw6 yyYUKTohj77Vu7WpfbEBykLhe77Sj5mZ6+wX15b9eodiuuWfUua5r77zvM7R96nh acTSbMMnsrgSelsxfxmUF0Uvz5JtplmTAwVe0W75XCVb0lGF+gNo3uUIfh9KocZD h8LCHRJ0wmxVGQyHf9ZJOlzJ7ATZVRJfKZdMyAg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=mpLZqZ oEWCLWiiGmRCRXu6fBm/B/QOW+YCDHkx++tG0=; b=QMPFB5WDerOOtvHgN/hZyb w2meL+rWK+H53+GV7+CbqRjmzI63M/bQYldj0kW8GzhQSj1CaLflVasONBd2q4dp voM3xVxnjm+fqaMjegqBM/YIv9RY3ZGxQmnC+SnBkCYWRbF6E060T1ryKo6iIhEk CtsjQvYOOm+Q3VwKmTvZP3NcUTHSFeIxTiRzOamIpYIsAkCy/YmnZHq4D7PW2+1R 28Ce+g6oKwIoh5MAJ/izpFpO49EX0pt5l0VcDlUlk7NRkrPtw02B+4DxLhkP+ZbT 3WnpS6KoZqkpvL2lYh9iZPBHqkmy5AyN57UZ3rD8qcwlxl35gV9rsEUFJPTQVt7Q == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduiedruddtiedgheejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepofgfggfkjghffffhvffutgesthdtredtreertdenucfhrhhomhepfdffrghn ucfnrghnghhilhhlvgdfuceouggrnheslhgrnhhgihhllhgvrdhorhhgqeenucggtffrrg htthgvrhhnpeetjeehtdejfeeuueejvefgvdeivdeiteeuudeiieevledugeehkedvveei hfetgeenucffohhmrghinhepghhithhhuhgsrdgtohhmnecuvehluhhsthgvrhfuihiivg eptdenucfrrghrrghmpehmrghilhhfrhhomhepuggrnheslhgrnhhgihhllhgvrdhorhhg X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 1B27C188009A; Tue, 18 Aug 2020 09:23:25 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.3.0-191-gef79d59-fm-20200818.001-gef79d590 Mime-Version: 1.0 Message-Id: <15d19351-3c4c-4ad1-bd1a-7f8f6c2ae211@www.fastmail.com> In-Reply-To: <3430278.PbzQmE3shN@beastie.bionicmutton.org> References: <3430278.PbzQmE3shN@beastie.bionicmutton.org> Date: Tue, 18 Aug 2020 09:23:03 -0400 From: "Dan Langille" To: "Adriaan de Groot" , freebsd-git@freebsd.org Subject: Re: FreshPorts & src commits to branches Content-Type: text/plain X-Rspamd-Queue-Id: 4BWBSG097pz436j X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2020 13:23:26 -0000 On Tue, Aug 18, 2020, at 3:44 AM, Adriaan de Groot wrote: > On Tuesday, 18 August 2020 03:19:26 CEST Dan Langille wrote: > > I am quickly realizing that in order for FreshPorts to track src commits it > > needs to query every branch. There does not seem to be any other way to > > know that a commit ocurred on stable/12 without a checkout of stable/12 > > If you have the full repo (e.g. a normal clone) then you might be able to use > something along this line: > > git log --branches --since 2020-08-16 --source --raw | grep ^commit > > (the important bit is --branches to include all of them, and --source to > include the source). I don't have a FreeBSD repo at hand, but my work-repo > (which currently has some other branch checked out) gives me this kind of > output: > > commit 36cefe3be10c707a2cdfaedd14fddee5cda31f50 hotfix-3.2.28.2 > commit eb44d0c6be18c1ced99d5a8c1b1a6a1a0e653be2 hotfix-3.2.28.2 > commit 24efce0c1d5601a77ad2b919cf0cf387231add76 calamares > commit 5db4195b7e3e507a34f84e041b15a4aa86ed32da hotfix-3.2.28.2 > commit 34946ecdee90584caf255fcca732c021e1dc1832 hotfix-3.2.28.2 > commit 1a7b2668dc1e341b493990b99e9a0cc4530d32cd hotfix-3.2.28.2 > > This gives me branches for each commit hash in the given range. The branches > you see mentioned here diverged before the date range given. I can't say > anything about performance, here -- my little test repo only has ~8000 commits > going back six years. > > (Turning this command-line into a collection of libgit calls is left as an > exercise for the reader) FYI, FreshPorts is using devel/py-pygit2 which uses devel/libgit2 :) After a git pull, I same up with this: [dan@devgit-ingress01:~/src/freebsd/freebsd] $ git log --branches --since 2020-08-17 --source --raw | grep ^commit commit ef1942cafc385f2ebc7e4a7a5f5679bb1a4b0d78 stable/12 commit 3889b52f5c73875df9a28180c4b1b4a07bf0042a stable/12 commit 9fc2a101135102074391b6ff28b825b082410ffa master commit a9a7211e27bfe2bec89b08899b977a4677728d97 master ... several lines omitted from this paste commit 5853f38da725897c25a505ce62675e31aca27f26 master commit 1b47d5c7b7fb338daf65ccf05b9df0d9e93d11ae stable/12 Let's try a date time in there: [dan@devgit-ingress01:~/src/freebsd/freebsd] $ git log --branches --since "2020-08-17 23:00:00" --source --raw | grep ^commit commit ef1942cafc385f2ebc7e4a7a5f5679bb1a4b0d78 stable/12 commit 3889b52f5c73875df9a28180c4b1b4a07bf0042a stable/12 Those were both at 23:40 yesterday. The current approach is: I have this commit, show me all the commits after that: hash..HEAD That works on a branch-by-branch basis. This may be a potential issues: The --since & --branches approach requires a date/timestamp. If we always use the timestamp of the last time we asked, will we skip over commits? Humor me while I contemplate URLs. The 3889b52f5c73875df9a28180c4b1b4a07bf0042a commit translates to: https://github.com/freebsd/freebsd/blob/stable/12/sys/sys/ata.h The format seems to be: https://github.com/freebsd/freebsd - URL to repo blob - some abstraction stable/12 - branch sys/sys/ata.h - path to file Compared to head/master: https://github.com/freebsd/freebsd/blob/master/sys/sys/ata.h The only difference: stable/12 -> master : very straight forward. Thank you. -- Dan Langille dan@langille.org