From owner-freebsd-git@freebsd.org Tue Aug 18 01:19:51 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D5EF73A88E4 for ; Tue, 18 Aug 2020 01:19:51 +0000 (UTC) (envelope-from dan@langille.org) Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BVtPL3PBkz4VwZ for ; Tue, 18 Aug 2020 01:19:50 +0000 (UTC) (envelope-from dan@langille.org) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.west.internal (Postfix) with ESMTP id D3F22A05 for ; Mon, 17 Aug 2020 21:19:47 -0400 (EDT) Received: from imap36 ([10.202.2.86]) by compute2.internal (MEProxy); Mon, 17 Aug 2020 21:19:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=langille.org; h= mime-version:message-id:date:from:to:subject:content-type; s= fm2; bh=KS/REdPmXAf4TQkA8wAP+XUvEp773doQhNR1Gztw4PU=; b=EpG8pUHs BPPBODwiYPm9eLcK4QAZeE53/ii0jc/URB9LBuG088qEbz2wwRKUmYw7SWtDuzM6 WgGZwKTwMNWrkCL71RDBmvhBdntKkKtQpNvqaeAWADykRMU3zXktqMoURlm0RYeT QXKMrEYmJjY/RAlAI24t0qjd1vfF66P6J93jiWqldgMh2CpTZE1xFEuZZOB4+fZc r5JMHHm0QaNAwsK3DZCMMNpYf/heAvziow7KiNhjrrFgFUVaPiwsQq5QhJzGH6k9 ILuMqUaZ8Tl31s2k4e8RhxUrFZajblMd/MZFV0oTQnCnFnuQzWXNDVmiyFVfz163 xeApQ6w2vIofmQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:message-id :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm3; bh=KS/REdPmXAf4TQkA8wAP+XUvEp773 doQhNR1Gztw4PU=; b=hupTcdoBCOB59tnUk7crqGZ/3iEQRSEI23xzKbFJDaYLs Y5ze3hiO4+r4diT+4oq+ooLgsMtXczVMDkuC0pSZ2Q/8kczdhD2ifmgYO//Hr10z rQh2ep2oMU5cNwFCOhkkcHoCyCroLuhrAEZdoZI2f+mWEe7LmoqTN5KAwE9dTmvE lnp7/7DRD9B/GIwvBjseo7OmTNI4r8RiQF7pkIM9qRo58b09jjxZ3BpewsUtrgCw QBf62HLee3GBnkqNfNrRk5L+DMlXXdQ2UvfTycADvSp9uScw3Glwp//GkEnScVRQ szp8y2we9qnBnV6yHbtlxR6aKgXSy+9bnQciomk2A== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduiedruddthedggeefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfffhffvufgtsehttdertd erredtnecuhfhrohhmpedfffgrnhcunfgrnhhgihhllhgvfdcuoegurghnsehlrghnghhi lhhlvgdrohhrgheqnecuggftrfgrthhtvghrnhepffdtuddtleejfefgvdeuteevkeettd eihedufffggeethedvudehfeehvefhveetnecuffhomhgrihhnpehfrhgvshhhphhorhht shdrohhrghdpghhithhhuhgsrdgtohhmnecuvehluhhsthgvrhfuihiivgeptdenucfrrg hrrghmpehmrghilhhfrhhomhepuggrnheslhgrnhhgihhllhgvrdhorhhg X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 28CFF188011B; Mon, 17 Aug 2020 21:19:47 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.3.0-192-gd9d7a78-fm-20200816.001-gd9d7a786 Mime-Version: 1.0 Message-Id: Date: Mon, 17 Aug 2020 21:19:26 -0400 From: "Dan Langille" To: freebsd-git@freebsd.org Subject: FreshPorts & src commits to branches Content-Type: text/plain ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1597713591; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: dkim-signature; bh=KS/REdPmXAf4TQkA8wAP+XUvEp773doQhNR1Gztw4PU=; b=vwHWlAXbJi5qnHTs7f7jC0Nx588OpiEr7NQuch8e02U9BQREFLYNZgFbf419l47DhSXY/l zgOLDVBE7lji5kzMZ6V10yXgHrP6bMRKt5yGqYwr6EfucmFh3dTa2oI3WTdfr3ZZhX2s9k LXJS7pYVziIWJuwznBxhhLR0+6q93zchvEu/4m84Ije8GTGIhX7qdUFwa4YjL5xTs9VgzK 4IgFqzL1qDgwvaZex1lodtCD47vVIaNgZspsgTG+jeuKf/PWh1rpCP1TeHyN03yj0w/D0J M3SLcDD/WmlajoizV8itvbSswthRkjn5BunPORF3UmYigDQWic9ovYtiIbrIyA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1597713591; a=rsa-sha256; cv=none; b=eBKRhL2GQnxmPPV9PuduXQAlZUdTjRTPixmcGVq/oFo7ehurxbhzHUYc/psn9vwsIHkfCi xWr1oGIxg42EYYqakVit8lA0KgNNNZrQhlQGZDDZjqqUY8DWumfeWEEpQT/NF4isQyIvKO BzRX+iKBSay98hPhHfdtPS3gZDau7VWTQHFMzsTnzuED0+CnQYIufvCfo9iOZUoiIhSS30 FputxhNQUqT2ri8soEEsWajpss4XSy4x8XSOMTUQ/JTubUVQbEwk8GbYTpthxd2nM+hHxL 7D01Jn9qZbtqev/Oh9hwWOzLDyQiQCgpmEK8ElzvZsvf7cBztI0xiqdgg2Mj1w== ARC-Authentication-Results: i=1; mx1.freebsd.org; dkim=pass header.d=langille.org header.s=fm2 header.b=EpG8pUHs; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=hupTcdoB; spf=pass (mx1.freebsd.org: domain of dan@langille.org designates 64.147.123.19 as permitted sender) smtp.mailfrom=dan@langille.org X-Rspamd-Queue-Id: 4BVtPL3PBkz4VwZ X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=langille.org header.s=fm2 header.b=EpG8pUHs; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=hupTcdoB; dmarc=pass (policy=none) header.from=langille.org; spf=pass (mx1.freebsd.org: domain of dan@langille.org designates 64.147.123.19 as permitted sender) smtp.mailfrom=dan@langille.org X-Spamd-Result: default: False [-2.73 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; R_DKIM_ALLOW(-0.20)[langille.org:s=fm2,messagingengine.com:s=fm3]; XM_UA_NO_VERSION(0.01)[]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[64.147.123.19:from]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.02)[-1.024]; RCVD_COUNT_THREE(0.00)[4]; ARC_SIGNED(0.00)[i=1]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.19]; DKIM_TRACE(0.00)[langille.org:+,messagingengine.com:+]; DMARC_POLICY_ALLOW(-0.50)[langille.org,none]; NEURAL_HAM_SHORT(-0.62)[-0.620]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:11403, ipnet:64.147.123.0/24, country:US]; MID_RHS_WWW(0.50)[]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.19:from] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2020 01:19:51 -0000 Hello, I am quickly realizing that in order for FreshPorts to track src commits it needs to query every branch. There does not seem to be any other way to know that a commit ocurred on stable/12 without a checkout of stable/12 On that note, I'm not sure how far to go back. I see MFH commits into stable7, 8, 9, and 10. Is there a way to know a commit has occurred without having looking at the branch? Yes, use the GitHub API. I have been able to pull that information out, we can get the commit hash, but not the branch name. Regardless, I am hesitant to tie FreshPorts into Github, especially for its vital data: commits. How about those emails? Can we get the commit emails up and running for git? With an email, we can see a commit occurred on stable/7, we have the commit hash. Do we have stable/7 checked out? No, we can checkout that branch and use it. Similarly, a commit hook which passed along the commit hash and the branch, that would be enough. Actually, having both the emails and the commit hook would cover the bases well. For those unaware, FreshPorts has been using commit emails as its primary data point for 20 years.[1] Code has been developed to harvest commit information from git[2]. Knowing the commit hash, we can grab the data, convert it to XML, and load it into the database. But only if we also know the branch. [1] - https://docs.freshports.org/sample_message.1 [2] -https://github.com/FreshPorts/git_proc_commit/blob/master/git-to-freshports/git-to-freshports-xml.py -- Dan Langille dan@langille.org From owner-freebsd-git@freebsd.org Tue Aug 18 07:44:37 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2FE703B243F for ; Tue, 18 Aug 2020 07:44:37 +0000 (UTC) (envelope-from adridg@freebsd.org) Received: from lb2-smtp-cloud9.xs4all.net (lb2-smtp-cloud9.xs4all.net [194.109.24.26]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.xs4all.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BW2xD4p1vz4sT8 for ; Tue, 18 Aug 2020 07:44:32 +0000 (UTC) (envelope-from adridg@freebsd.org) Received: from cust-d4a83f22 ([IPv6:fc0c:c11d:cecc:f58a:eaa1:c0:9d8f:c143]) by smtp-cloud9.xs4all.net with ESMTPA id 7wIBkSJJ8uuXO7wICkaWlP; Tue, 18 Aug 2020 09:44:24 +0200 From: Adriaan de Groot To: freebsd-git@freebsd.org Subject: Re: FreshPorts & src commits to branches Date: Tue, 18 Aug 2020 09:44:17 +0200 Message-ID: <3430278.PbzQmE3shN@beastie.bionicmutton.org> Organization: FreeBSD In-Reply-To: References: MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart3214470.i0SyIzU0eP"; micalg="pgp-sha256"; protocol="application/pgp-signature" X-CMAE-Envelope: MS4wfIBaHpQOKqzrH3r7u4/L9XGrRwfUQVwM+TV70nAHv7OZVJjseE2XKIHx3FXeuXUXRHeFYzJL57uxEJdHKY8xQZQDF9g+dBEH0AKdtLQhPEUgcobgaiuu 42gZJcMqa/AWeed1GhPn2KSUfbJZ1s+vM1cEl3ilFdEKADVVbH50u44o29M4vegpdadnMJlzAKthwWY65nYGTVC6thI0u/scTP0= X-Rspamd-Queue-Id: 4BW2xD4p1vz4sT8 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2020 07:44:37 -0000 --nextPart3214470.i0SyIzU0eP Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="utf-8" On Tuesday, 18 August 2020 03:19:26 CEST Dan Langille wrote: > I am quickly realizing that in order for FreshPorts to track src commits it > needs to query every branch. There does not seem to be any other way to > know that a commit ocurred on stable/12 without a checkout of stable/12 If you have the full repo (e.g. a normal clone) then you might be able to use something along this line: git log --branches --since 2020-08-16 --source --raw | grep ^commit (the important bit is --branches to include all of them, and --source to include the source). I don't have a FreeBSD repo at hand, but my work-repo (which currently has some other branch checked out) gives me this kind of output: commit 36cefe3be10c707a2cdfaedd14fddee5cda31f50 hotfix-3.2.28.2 commit eb44d0c6be18c1ced99d5a8c1b1a6a1a0e653be2 hotfix-3.2.28.2 commit 24efce0c1d5601a77ad2b919cf0cf387231add76 calamares commit 5db4195b7e3e507a34f84e041b15a4aa86ed32da hotfix-3.2.28.2 commit 34946ecdee90584caf255fcca732c021e1dc1832 hotfix-3.2.28.2 commit 1a7b2668dc1e341b493990b99e9a0cc4530d32cd hotfix-3.2.28.2 This gives me branches for each commit hash in the given range. The branches you see mentioned here diverged before the date range given. I can't say anything about performance, here -- my little test repo only has ~8000 commits going back six years. (Turning this command-line into a collection of libgit calls is left as an exercise for the reader) [ade] --nextPart3214470.i0SyIzU0eP Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part. Content-Transfer-Encoding: 7Bit -----BEGIN PGP SIGNATURE----- iQGzBAABCAAdFiEEhrjttu2OP5apuuy1z93JbxKxkVwFAl87htEACgkQz93JbxKx kVy9Twv/T9E4ML2kx2hd081U1ZE6g9xpKMuBd/ufTN22eZhNc4FujjQ0odrr5qy3 XHpkdh7I3eN24koyWuEqvNuHf2JqqPFydBAFetttJ7LvMJ285G6aU9BF5sq3dAjw aH7L8W9F94jQr2Ri9eLhR1SlULQaVzS/iw1shI6EzhLtzsqTn6XVib23T0LbU2V+ RAeab7VZF1tmZQ2v9ymq6JHdxLWN+GctIfAYBZwWvX5rXJ4tYvumHaZHZwG+u9Q8 H+PmhgN3QoTk6LIavwBC1JwauxGPRlJIp7azZlaqt2pnOrkBv/hDwqZ+f/EiQN+R dsAS1KYlZBhd3RQd+COf7kqgqMw/RrIvDzOLal2TFG4y9R6Uq7GpgOdFN4AhW4jE sMePECsWcOLLhWmKZ1EFv3iL+N7fJ9o43QB00jtIr4ewJXh9KArH1JBykFcTJE3s LnxQ9LQgvXiNe7/r3Xo9utO4RBBRS1Hb5icdA+nSevFXqHhs48AOfDPAcmPxI24s fUOIqRGe =MKKb -----END PGP SIGNATURE----- --nextPart3214470.i0SyIzU0eP-- From owner-freebsd-git@freebsd.org Tue Aug 18 13:23:26 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 23F973BBBCB for ; Tue, 18 Aug 2020 13:23:26 +0000 (UTC) (envelope-from dan@langille.org) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BWBSG097pz436j; Tue, 18 Aug 2020 13:23:25 +0000 (UTC) (envelope-from dan@langille.org) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 6F4BA5C00A0; Tue, 18 Aug 2020 09:23:25 -0400 (EDT) Received: from imap36 ([10.202.2.86]) by compute2.internal (MEProxy); Tue, 18 Aug 2020 09:23:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=langille.org; h= mime-version:message-id:in-reply-to:references:date:from:to :subject:content-type; s=fm2; bh=mpLZqZoEWCLWiiGmRCRXu6fBm/B/QOW +YCDHkx++tG0=; b=QZcHwUjsrIH/S4XBQ3cM5mrS0YIy0VLSYRNDTyLSqDrX9Ow 09bvfL1mqw2pQ9bquee8TAQ0Jg3d1Lrx23BJlCREsbpnD1tuoyWCF8ckN7urhMDT RD2E9kvsqh6GvEgntJUdY2q26E2rbeCUDQ2BospTJCnGOGd8fCg7hXCDNNZV2cw6 yyYUKTohj77Vu7WpfbEBykLhe77Sj5mZ6+wX15b9eodiuuWfUua5r77zvM7R96nh acTSbMMnsrgSelsxfxmUF0Uvz5JtplmTAwVe0W75XCVb0lGF+gNo3uUIfh9KocZD h8LCHRJ0wmxVGQyHf9ZJOlzJ7ATZVRJfKZdMyAg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=mpLZqZ oEWCLWiiGmRCRXu6fBm/B/QOW+YCDHkx++tG0=; b=QMPFB5WDerOOtvHgN/hZyb w2meL+rWK+H53+GV7+CbqRjmzI63M/bQYldj0kW8GzhQSj1CaLflVasONBd2q4dp voM3xVxnjm+fqaMjegqBM/YIv9RY3ZGxQmnC+SnBkCYWRbF6E060T1ryKo6iIhEk CtsjQvYOOm+Q3VwKmTvZP3NcUTHSFeIxTiRzOamIpYIsAkCy/YmnZHq4D7PW2+1R 28Ce+g6oKwIoh5MAJ/izpFpO49EX0pt5l0VcDlUlk7NRkrPtw02B+4DxLhkP+ZbT 3WnpS6KoZqkpvL2lYh9iZPBHqkmy5AyN57UZ3rD8qcwlxl35gV9rsEUFJPTQVt7Q == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduiedruddtiedgheejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepofgfggfkjghffffhvffutgesthdtredtreertdenucfhrhhomhepfdffrghn ucfnrghnghhilhhlvgdfuceouggrnheslhgrnhhgihhllhgvrdhorhhgqeenucggtffrrg htthgvrhhnpeetjeehtdejfeeuueejvefgvdeivdeiteeuudeiieevledugeehkedvveei hfetgeenucffohhmrghinhepghhithhhuhgsrdgtohhmnecuvehluhhsthgvrhfuihiivg eptdenucfrrghrrghmpehmrghilhhfrhhomhepuggrnheslhgrnhhgihhllhgvrdhorhhg X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 1B27C188009A; Tue, 18 Aug 2020 09:23:25 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.3.0-191-gef79d59-fm-20200818.001-gef79d590 Mime-Version: 1.0 Message-Id: <15d19351-3c4c-4ad1-bd1a-7f8f6c2ae211@www.fastmail.com> In-Reply-To: <3430278.PbzQmE3shN@beastie.bionicmutton.org> References: <3430278.PbzQmE3shN@beastie.bionicmutton.org> Date: Tue, 18 Aug 2020 09:23:03 -0400 From: "Dan Langille" To: "Adriaan de Groot" , freebsd-git@freebsd.org Subject: Re: FreshPorts & src commits to branches Content-Type: text/plain X-Rspamd-Queue-Id: 4BWBSG097pz436j X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2020 13:23:26 -0000 On Tue, Aug 18, 2020, at 3:44 AM, Adriaan de Groot wrote: > On Tuesday, 18 August 2020 03:19:26 CEST Dan Langille wrote: > > I am quickly realizing that in order for FreshPorts to track src commits it > > needs to query every branch. There does not seem to be any other way to > > know that a commit ocurred on stable/12 without a checkout of stable/12 > > If you have the full repo (e.g. a normal clone) then you might be able to use > something along this line: > > git log --branches --since 2020-08-16 --source --raw | grep ^commit > > (the important bit is --branches to include all of them, and --source to > include the source). I don't have a FreeBSD repo at hand, but my work-repo > (which currently has some other branch checked out) gives me this kind of > output: > > commit 36cefe3be10c707a2cdfaedd14fddee5cda31f50 hotfix-3.2.28.2 > commit eb44d0c6be18c1ced99d5a8c1b1a6a1a0e653be2 hotfix-3.2.28.2 > commit 24efce0c1d5601a77ad2b919cf0cf387231add76 calamares > commit 5db4195b7e3e507a34f84e041b15a4aa86ed32da hotfix-3.2.28.2 > commit 34946ecdee90584caf255fcca732c021e1dc1832 hotfix-3.2.28.2 > commit 1a7b2668dc1e341b493990b99e9a0cc4530d32cd hotfix-3.2.28.2 > > This gives me branches for each commit hash in the given range. The branches > you see mentioned here diverged before the date range given. I can't say > anything about performance, here -- my little test repo only has ~8000 commits > going back six years. > > (Turning this command-line into a collection of libgit calls is left as an > exercise for the reader) FYI, FreshPorts is using devel/py-pygit2 which uses devel/libgit2 :) After a git pull, I same up with this: [dan@devgit-ingress01:~/src/freebsd/freebsd] $ git log --branches --since 2020-08-17 --source --raw | grep ^commit commit ef1942cafc385f2ebc7e4a7a5f5679bb1a4b0d78 stable/12 commit 3889b52f5c73875df9a28180c4b1b4a07bf0042a stable/12 commit 9fc2a101135102074391b6ff28b825b082410ffa master commit a9a7211e27bfe2bec89b08899b977a4677728d97 master ... several lines omitted from this paste commit 5853f38da725897c25a505ce62675e31aca27f26 master commit 1b47d5c7b7fb338daf65ccf05b9df0d9e93d11ae stable/12 Let's try a date time in there: [dan@devgit-ingress01:~/src/freebsd/freebsd] $ git log --branches --since "2020-08-17 23:00:00" --source --raw | grep ^commit commit ef1942cafc385f2ebc7e4a7a5f5679bb1a4b0d78 stable/12 commit 3889b52f5c73875df9a28180c4b1b4a07bf0042a stable/12 Those were both at 23:40 yesterday. The current approach is: I have this commit, show me all the commits after that: hash..HEAD That works on a branch-by-branch basis. This may be a potential issues: The --since & --branches approach requires a date/timestamp. If we always use the timestamp of the last time we asked, will we skip over commits? Humor me while I contemplate URLs. The 3889b52f5c73875df9a28180c4b1b4a07bf0042a commit translates to: https://github.com/freebsd/freebsd/blob/stable/12/sys/sys/ata.h The format seems to be: https://github.com/freebsd/freebsd - URL to repo blob - some abstraction stable/12 - branch sys/sys/ata.h - path to file Compared to head/master: https://github.com/freebsd/freebsd/blob/master/sys/sys/ata.h The only difference: stable/12 -> master : very straight forward. Thank you. -- Dan Langille dan@langille.org From owner-freebsd-git@freebsd.org Wed Aug 19 12:47:28 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8121F3BDC15 for ; Wed, 19 Aug 2020 12:47:28 +0000 (UTC) (envelope-from mat@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BWncJ2tr9z3VpN; Wed, 19 Aug 2020 12:47:28 +0000 (UTC) (envelope-from mat@freebsd.org) Received: from mail.j.mat.cc (owncloud.cube.mat.cc [IPv6:2a01:678:4:1::228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.mat.cc", Issuer "Let's Encrypt Authority X3" (verified OK)) (Authenticated sender: mat/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 153E1150D2; Wed, 19 Aug 2020 12:47:28 +0000 (UTC) (envelope-from mat@freebsd.org) Received: from aching.in.mat.cc (fonera.eqx.absolight.net [79.143.253.217]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: mat@mat.cc) by mail.j.mat.cc (Postfix) with ESMTPSA id CB0AD942D81; Wed, 19 Aug 2020 12:47:25 +0000 (UTC) Date: Wed, 19 Aug 2020 14:47:24 +0200 From: Mathieu Arnold To: Dan Langille Cc: freebsd-git@freebsd.org Subject: Re: FreshPorts & src commits to branches Message-ID: <20200819124724.yoni3kjxzlboknlq@aching.in.mat.cc> References: <3430278.PbzQmE3shN@beastie.bionicmutton.org> <15d19351-3c4c-4ad1-bd1a-7f8f6c2ae211@www.fastmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="r5pkvj7bryq7m7ja" Content-Disposition: inline In-Reply-To: <15d19351-3c4c-4ad1-bd1a-7f8f6c2ae211@www.fastmail.com> X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2020 12:47:28 -0000 --r5pkvj7bryq7m7ja Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Aug 18, 2020 at 09:23:03AM -0400, Dan Langille wrote: > On Tue, Aug 18, 2020, at 3:44 AM, Adriaan de Groot wrote: > > On Tuesday, 18 August 2020 03:19:26 CEST Dan Langille wrote: > > > I am quickly realizing that in order for FreshPorts to track src comm= its it > > > needs to query every branch. There does not seem to be any other way = to > > > know that a commit ocurred on stable/12 without a checkout of stable/= 12 > >=20 > > If you have the full repo (e.g. a normal clone) then you might be able = to use=20 > > something along this line: > >=20 > > git log --branches --since 2020-08-16 --source --raw | grep ^commit > >=20 > > (the important bit is --branches to include all of them, and --source t= o=20 > > include the source). I don't have a FreeBSD repo at hand, but my work-r= epo=20 > > (which currently has some other branch checked out) gives me this kind = of=20 > > output: > >=20 > > commit 36cefe3be10c707a2cdfaedd14fddee5cda31f50 hotfix-3.2.28.2 > > commit eb44d0c6be18c1ced99d5a8c1b1a6a1a0e653be2 hotfix-3.2.28.2 > > commit 24efce0c1d5601a77ad2b919cf0cf387231add76 calamares > > commit 5db4195b7e3e507a34f84e041b15a4aa86ed32da hotfix-3.2.28.2 > > commit 34946ecdee90584caf255fcca732c021e1dc1832 hotfix-3.2.28.2 > > commit 1a7b2668dc1e341b493990b99e9a0cc4530d32cd hotfix-3.2.28.2 > >=20 > > This gives me branches for each commit hash in the given range. The bra= nches=20 > > you see mentioned here diverged before the date range given. I can't sa= y=20 > > anything about performance, here -- my little test repo only has ~8000 = commits=20 > > going back six years. > >=20 > > (Turning this command-line into a collection of libgit calls is left as= an=20 > > exercise for the reader) >=20 > FYI, FreshPorts is using devel/py-pygit2 which uses devel/libgit2 :) >=20 > After a git pull, I same up with this: >=20 > [dan@devgit-ingress01:~/src/freebsd/freebsd] $ git log --branches --since= 2020-08-17 --source --raw | grep ^commit > commit ef1942cafc385f2ebc7e4a7a5f5679bb1a4b0d78 stable/12 > commit 3889b52f5c73875df9a28180c4b1b4a07bf0042a stable/12 > commit 9fc2a101135102074391b6ff28b825b082410ffa master > commit a9a7211e27bfe2bec89b08899b977a4677728d97 master > ... several lines omitted from this paste > commit 5853f38da725897c25a505ce62675e31aca27f26 master > commit 1b47d5c7b7fb338daf65ccf05b9df0d9e93d11ae stable/12 >=20 > Let's try a date time in there: >=20 > [dan@devgit-ingress01:~/src/freebsd/freebsd] $ git log --branches --since= "2020-08-17 23:00:00" --source --raw | grep ^commit > commit ef1942cafc385f2ebc7e4a7a5f5679bb1a4b0d78 stable/12=20 > commit 3889b52f5c73875df9a28180c4b1b4a07bf0042a stable/12 >=20 > Those were both at 23:40 yesterday. >=20 > The current approach is: I have this commit, show me all the commits afte= r that: hash..HEAD >=20 > That works on a branch-by-branch basis. >=20 >=20 > This may be a potential issues: The --since & --branches approach require= s a date/timestamp. If we always use the timestamp of the last time we aske= d, will we skip over commits? Well, I would run something like --since '2 weeks ago' and ignore what commits you already have. It will maybe work, or not, it will probably depend on wether we allow branches or out of order commits, or... It would probably be safer to do for each branch, look at the commits in the ... range, which would get you all the commits, proably even if we allow merge commits. > Humor me while I contemplate URLs. =20 >=20 > The 3889b52f5c73875df9a28180c4b1b4a07bf0042a commit translates to: >=20 > https://github.com/freebsd/freebsd/blob/stable/12/sys/sys/ata.h Well, nom the 3889b52f5c73875df9a28180c4b1b4a07bf0042a commit translates to=20 https://github.com/freebsd/freebsd/blob/3889b52f5c73875df9a28180c4b1b4a07bf= 0042a/sys/sys/ata.h The URL you have is for the head of the stable/12 branch. > The format seems to be: >=20 > https://github.com/freebsd/freebsd - URL to repo >=20 > blob - some abstraction >=20 > stable/12 - branch >=20 > sys/sys/ata.h - path to file >=20 > Compared to head/master: >=20 > https://github.com/freebsd/freebsd/blob/master/sys/sys/ata.h >=20 > The only difference: stable/12 -> master : very straight forward. What do you need the github url for? --=20 Mathieu Arnold --r5pkvj7bryq7m7ja Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAABCgB9FiEEVhwchfRfuV0unqO5KesJApEdfgIFAl89H1dfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDU2 MUMxQzg1RjQ1RkI5NUQyRTlFQTNCOTI5RUIwOTAyOTExRDdFMDIACgkQKesJApEd fgK7gg//ZrQLnvZKycqAoRIo14FQkcHb5nZYals4TCaJh6w2BiCteeQgE++3E++/ B91m9qinEbW3zqkLqd0tFzogA+iOcmMG0JeVNgRfZL+olWOUttKR0y84mxJE4KcR nfLv4xFLmvkdjNlxOeJMOti422t0p8wRnSIJbxYKrKAwpH+cGgHJe0ZxeAFtpM0J It71MRNOgu9hpggbrgei17hKMThU0BWEGnrayGL5rKvI1tN2xKkwcEUBs2YW1Aya wwXpzrOnGqDrCYE/MTn1gmUoxJ6sC7vFvTmecJb5k2XJboQ+YXNst5x0P16+IN72 BcbRfyFhsP1bOUdVOo5LZ5ikpGpryRwcevXDx93ffi+7qg0kaNYnp9hE2aYu4YbR fquKf2/L0xF8y87OAkORvJtiO0E1alOH9iP1IR4Bc/3Dz22lUFNX/LJo/Sfox+Ot GufUZVF3Am4LEo5pWM7+5brrgjtcTYyA7JR/FHTOr/jJBFEHNlOSACVvdxlxuj5O jnais3u7j1ZEwnBv6yGCID7N+Jdc83Jvs35f+2NaLG/opNshHzestj+QT5+TJr87 0CUIF5SdbasR0eZ47n9ooyqFmkUz+Toy6v1hQ3V8gYjoPcstNlyDsIne/6uHcmWe tQb0HVaBpufVfpU60F6gChMDOYuHANJaEPl4suHautK+8jlTTvQ= =2pLX -----END PGP SIGNATURE----- --r5pkvj7bryq7m7ja-- From owner-freebsd-git@freebsd.org Wed Aug 19 13:06:43 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AC9DC3BE230 for ; Wed, 19 Aug 2020 13:06:43 +0000 (UTC) (envelope-from dan@langille.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BWp2V66Txz3Ws0; Wed, 19 Aug 2020 13:06:42 +0000 (UTC) (envelope-from dan@langille.org) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 9164B5C0165; Wed, 19 Aug 2020 09:06:41 -0400 (EDT) Received: from imap36 ([10.202.2.86]) by compute2.internal (MEProxy); Wed, 19 Aug 2020 09:06:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=langille.org; h= mime-version:message-id:in-reply-to:references:date:from:to:cc :subject:content-type; s=fm2; bh=CDGnksufQm7hjL66nv7hSrq2af18OfM OBMQkIFDJoz4=; b=oAjn5PD4cOO4Gq3+LLez/UfMvLlqkNm6mDy29L1/ypAqCfv iSlasrjZLIattubIl8tnK54PMxqs9BzOwRoPZfg52AvQk20CbcFkCb2FmDHTSQkk uFTFGT2+vzRwAV6Qvdi4w2bx5SWv16lnv1efeYGey1F6Z1tvF7FQ7OSzP3h5aFT4 x2u1McD4NiPqaQV+kobNG9catEVXTlSfnNP8DYJpQNYJ29dBXePsg4JQd1xqApH/ ZUctPIHuWIfY/tkJ4ec5mrHWNGwdgmKeRj4qtQFKWHO02GvjFD7Ms6zfOOUoUJka 6l+cV+WAR3MXXZtqAAd9U/oI7ktliC63zgEDyIA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=CDGnks ufQm7hjL66nv7hSrq2af18OfMOBMQkIFDJoz4=; b=qch6Tpa9kCMBMYloW2dpV/ uKCfWgqiXy72+U2HWFdICtnafRTimNikz/qn43JeivZEq0Npm5W6g84UOlfwvG48 C5K6kLXPaAAui6dChCp4fzDGZeNVjA8udSMRuz/MgQdxzLMr2YIxe+PorjRcq1Xd rMIttikZTaVcPXKg8uKVAwAO6mys8Z4883jyTL2cCsdbt4S4qQ4k4IXKnQYysS6i tinfepRDK/95ZIYUs4n0m1DHkvoHnBUiUefBv30IzzTfxxYazyJwrz3ahmnAX4aJ EhoFWjLowMkTN5aqaYG378PccLmiDTwesAU441N2GjAthc0cPL+xkif6mtPk2YvQ == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduiedruddtkedggeefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsehttd ertderredtnecuhfhrohhmpedfffgrnhcunfgrnhhgihhllhgvfdcuoegurghnsehlrghn ghhilhhlvgdrohhrgheqnecuggftrfgrthhtvghrnhepteejhedtjeefueeujeevgfdvie dvieetueduieeiveeludegheekvdeviefhteegnecuffhomhgrihhnpehgihhthhhusgdr tghomhenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe gurghnsehlrghnghhilhhlvgdrohhrgh X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 263CB1880081; Wed, 19 Aug 2020 09:06:40 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.3.0-191-gef79d59-fm-20200818.001-gef79d590 Mime-Version: 1.0 Message-Id: In-Reply-To: <20200819124724.yoni3kjxzlboknlq@aching.in.mat.cc> References: <3430278.PbzQmE3shN@beastie.bionicmutton.org> <15d19351-3c4c-4ad1-bd1a-7f8f6c2ae211@www.fastmail.com> <20200819124724.yoni3kjxzlboknlq@aching.in.mat.cc> Date: Wed, 19 Aug 2020 09:06:20 -0400 From: "Dan Langille" To: "Mathieu Arnold" Cc: freebsd-git@freebsd.org Subject: Re: FreshPorts & src commits to branches Content-Type: text/plain X-Rspamd-Queue-Id: 4BWp2V66Txz3Ws0 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=langille.org header.s=fm2 header.b=oAjn5PD4; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=qch6Tpa9; dmarc=pass (policy=none) header.from=langille.org; spf=pass (mx1.freebsd.org: domain of dan@langille.org designates 66.111.4.25 as permitted sender) smtp.mailfrom=dan@langille.org X-Spamd-Result: default: False [-0.96 / 15.00]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[langille.org:s=fm2,messagingengine.com:s=fm3]; XM_UA_NO_VERSION(0.01)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; FREEFALL_USER(0.00)[dan]; RWL_MAILSPIKE_GOOD(0.00)[66.111.4.25:from]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.25]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[langille.org:+,messagingengine.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[langille.org,none]; NEURAL_HAM_SHORT(-0.87)[-0.873]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.25:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; MID_RHS_WWW(0.50)[]; MAILMAN_DEST(0.00)[freebsd-git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2020 13:06:43 -0000 On Wed, Aug 19, 2020, at 8:47 AM, Mathieu Arnold wrote: > On Tue, Aug 18, 2020 at 09:23:03AM -0400, Dan Langille wrote: > > On Tue, Aug 18, 2020, at 3:44 AM, Adriaan de Groot wrote: > > > On Tuesday, 18 August 2020 03:19:26 CEST Dan Langille wrote: > > > > I am quickly realizing that in order for FreshPorts to track src commits it > > > > needs to query every branch. There does not seem to be any other way to > > > > know that a commit ocurred on stable/12 without a checkout of stable/12 > > > > > > If you have the full repo (e.g. a normal clone) then you might be able to use > > > something along this line: > > > > > > git log --branches --since 2020-08-16 --source --raw | grep ^commit > > > > > > (the important bit is --branches to include all of them, and --source to > > > include the source). I don't have a FreeBSD repo at hand, but my work-repo > > > (which currently has some other branch checked out) gives me this kind of > > > output: > > > > > > commit 36cefe3be10c707a2cdfaedd14fddee5cda31f50 hotfix-3.2.28.2 > > > commit eb44d0c6be18c1ced99d5a8c1b1a6a1a0e653be2 hotfix-3.2.28.2 > > > commit 24efce0c1d5601a77ad2b919cf0cf387231add76 calamares > > > commit 5db4195b7e3e507a34f84e041b15a4aa86ed32da hotfix-3.2.28.2 > > > commit 34946ecdee90584caf255fcca732c021e1dc1832 hotfix-3.2.28.2 > > > commit 1a7b2668dc1e341b493990b99e9a0cc4530d32cd hotfix-3.2.28.2 > > > > > > This gives me branches for each commit hash in the given range. The branches > > > you see mentioned here diverged before the date range given. I can't say > > > anything about performance, here -- my little test repo only has ~8000 commits > > > going back six years. > > > > > > (Turning this command-line into a collection of libgit calls is left as an > > > exercise for the reader) > > > > FYI, FreshPorts is using devel/py-pygit2 which uses devel/libgit2 :) > > > > After a git pull, I same up with this: > > > > [dan@devgit-ingress01:~/src/freebsd/freebsd] $ git log --branches --since 2020-08-17 --source --raw | grep ^commit > > commit ef1942cafc385f2ebc7e4a7a5f5679bb1a4b0d78 stable/12 > > commit 3889b52f5c73875df9a28180c4b1b4a07bf0042a stable/12 > > commit 9fc2a101135102074391b6ff28b825b082410ffa master > > commit a9a7211e27bfe2bec89b08899b977a4677728d97 master > > ... several lines omitted from this paste > > commit 5853f38da725897c25a505ce62675e31aca27f26 master > > commit 1b47d5c7b7fb338daf65ccf05b9df0d9e93d11ae stable/12 > > > > Let's try a date time in there: > > > > [dan@devgit-ingress01:~/src/freebsd/freebsd] $ git log --branches --since "2020-08-17 23:00:00" --source --raw | grep ^commit > > commit ef1942cafc385f2ebc7e4a7a5f5679bb1a4b0d78 stable/12 > > commit 3889b52f5c73875df9a28180c4b1b4a07bf0042a stable/12 > > > > Those were both at 23:40 yesterday. > > > > The current approach is: I have this commit, show me all the commits after that: hash..HEAD > > > > That works on a branch-by-branch basis. > > > > > > This may be a potential issues: The --since & --branches approach requires a date/timestamp. If we always use the timestamp of the last time we asked, will we skip over commits? > > Well, I would run something like --since '2 weeks ago' and ignore what > commits you already have. It will maybe work, or not, it will probably > depend on wether we allow branches or out of order commits, or... > > It would probably be safer to do for each branch, look at the commits in > the ... range, which would get you all the commits, > proably even if we allow merge commits. > > > Humor me while I contemplate URLs. > > > > The 3889b52f5c73875df9a28180c4b1b4a07bf0042a commit translates to: > > > > https://github.com/freebsd/freebsd/blob/stable/12/sys/sys/ata.h > > Well, nom the 3889b52f5c73875df9a28180c4b1b4a07bf0042a commit translates > to > > https://github.com/freebsd/freebsd/blob/3889b52f5c73875df9a28180c4b1b4a07bf0042a/sys/sys/ata.h > > The URL you have is for the head of the stable/12 branch. Yes, you are correct. I was trying translates a commit on a branch to the patch to the file affected by that commit. I think I got that correct. > > > The format seems to be: > > > > https://github.com/freebsd/freebsd - URL to repo > > > > blob - some abstraction > > > > stable/12 - branch > > > > sys/sys/ata.h - path to file > > > > Compared to head/master: > > > > https://github.com/freebsd/freebsd/blob/master/sys/sys/ata.h > > > > The only difference: stable/12 -> master : very straight forward. > > What do you need the github url for? FreshPorts doesn't need the URL. It was for me. Sometimes I want to look at the branch and see if I have missed anything. -- Dan Langille dan@langille.org From owner-freebsd-git@freebsd.org Wed Aug 19 14:36:06 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6A3263C0802 for ; Wed, 19 Aug 2020 14:36:06 +0000 (UTC) (envelope-from dan@langille.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BWr1d4nSQz3dkM for ; Wed, 19 Aug 2020 14:36:05 +0000 (UTC) (envelope-from dan@langille.org) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 702E25C0135 for ; Wed, 19 Aug 2020 10:36:04 -0400 (EDT) Received: from imap36 ([10.202.2.86]) by compute2.internal (MEProxy); Wed, 19 Aug 2020 10:36:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=langille.org; h= mime-version:message-id:in-reply-to:references:date:from:to :subject:content-type; s=fm2; bh=K2T6aifbSGc5MAQZIqz4A/tByidcUNf 4nQoBfia/qSQ=; b=IYrVU0HBAwrndMpmAjZTuW0leEg9nAh1FcueX7kls9Qor6A llMdmi47YfbdY5Vq8xUNobcB808O+ARqf1XdlCvuEFbygeJ6uFEj95n2QxJRw56a X2MDBoIz+hqWTtuMQUZY0SRww3u1ZstXPMSHxqrZyXUgChJJ/wfEwyig7M7hlEBh GUznZPJ7CetPmjxj0rWoxs75+1IkKQvZ1S73SrJMxvHMdNpnfDjYVBN8su4Pszl6 Y+BjJWnzB+LAQ1SrFhOTYUYwrpR51Ww27v9v+A5rU5KCATgJfFsUb5X9aqoll9QR hAQdW8xuYIjmM3JlGFxKrerJGGlHau5Iu66QtJQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=K2T6ai fbSGc5MAQZIqz4A/tByidcUNf4nQoBfia/qSQ=; b=t62lp89CbIAA6NQ0+1DNef 87IWqchwB0ovGp5WFYe4wl8+4+Dp7ebhMBjDcbtyeayPuOd4OcLUik7H4IM3m0UW vtiwJBEaGNyA9Y6ExKAL+PDvjuiKBpqwn2Cj5KPkY9gTVz3gKMu6E0R31Q5D+gjL PsJG63HZuD0pRAI4O9jRmqeHY9aOVbo93n7ua0Im6/O/+5J5Rq26fbWnZu4MG4vi sDrakGbLyWGqYP8x3kh3+1WbXJkbzWeX3KfWEgpnMGqsfNr7qenfeXT5O6konyvc RZmB/Snf8BFqJAdhm2Y339snLpTzUuQQWTSRlfX6ydblscS2OVvH97m7L9eQBzGQ == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduiedruddtkedgheehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsehttd ertderredtnecuhfhrohhmpedfffgrnhcunfgrnhhgihhllhgvfdcuoegurghnsehlrghn ghhilhhlvgdrohhrgheqnecuggftrfgrthhtvghrnhepteejhedtjeefueeujeevgfdvie dvieetueduieeiveeludegheekvdeviefhteegnecuffhomhgrihhnpehgihhthhhusgdr tghomhenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe gurghnsehlrghnghhilhhlvgdrohhrgh X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id E97481880081; Wed, 19 Aug 2020 10:36:03 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.3.0-191-gef79d59-fm-20200818.001-gef79d590 Mime-Version: 1.0 Message-Id: In-Reply-To: References: <3430278.PbzQmE3shN@beastie.bionicmutton.org> <15d19351-3c4c-4ad1-bd1a-7f8f6c2ae211@www.fastmail.com> <20200819124724.yoni3kjxzlboknlq@aching.in.mat.cc> Date: Wed, 19 Aug 2020 10:35:42 -0400 From: "Dan Langille" To: freebsd-git@freebsd.org Subject: Re: FreshPorts & src commits to branches Content-Type: text/plain X-Rspamd-Queue-Id: 4BWr1d4nSQz3dkM X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=langille.org header.s=fm2 header.b=IYrVU0HB; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=t62lp89C; dmarc=pass (policy=none) header.from=langille.org; spf=pass (mx1.freebsd.org: domain of dan@langille.org designates 66.111.4.25 as permitted sender) smtp.mailfrom=dan@langille.org X-Spamd-Result: default: False [-0.55 / 15.00]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[langille.org:s=fm2,messagingengine.com:s=fm3]; XM_UA_NO_VERSION(0.01)[]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[66.111.4.25:from]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FREEFALL_USER(0.00)[dan]; RCVD_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.25]; DKIM_TRACE(0.00)[langille.org:+,messagingengine.com:+]; DMARC_POLICY_ALLOW(-0.50)[langille.org,none]; NEURAL_HAM_SHORT(-0.46)[-0.464]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.25:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; MAILMAN_DEST(0.00)[freebsd-git]; MID_RHS_WWW(0.50)[] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2020 14:36:06 -0000 On Wed, Aug 19, 2020, at 9:06 AM, Dan Langille wrote: > > > On Wed, Aug 19, 2020, at 8:47 AM, Mathieu Arnold wrote: > > On Tue, Aug 18, 2020 at 09:23:03AM -0400, Dan Langille wrote: > > > On Tue, Aug 18, 2020, at 3:44 AM, Adriaan de Groot wrote: > > > > On Tuesday, 18 August 2020 03:19:26 CEST Dan Langille wrote: > > > > > I am quickly realizing that in order for FreshPorts to track src commits it > > > > > needs to query every branch. There does not seem to be any other way to > > > > > know that a commit ocurred on stable/12 without a checkout of stable/12 > > > > > > > > If you have the full repo (e.g. a normal clone) then you might be able to use > > > > something along this line: > > > > > > > > git log --branches --since 2020-08-16 --source --raw | grep ^commit > > > > > > > > (the important bit is --branches to include all of them, and --source to > > > > include the source). I don't have a FreeBSD repo at hand, but my work-repo > > > > (which currently has some other branch checked out) gives me this kind of > > > > output: > > > > > > > > commit 36cefe3be10c707a2cdfaedd14fddee5cda31f50 hotfix-3.2.28.2 > > > > commit eb44d0c6be18c1ced99d5a8c1b1a6a1a0e653be2 hotfix-3.2.28.2 > > > > commit 24efce0c1d5601a77ad2b919cf0cf387231add76 calamares > > > > commit 5db4195b7e3e507a34f84e041b15a4aa86ed32da hotfix-3.2.28.2 > > > > commit 34946ecdee90584caf255fcca732c021e1dc1832 hotfix-3.2.28.2 > > > > commit 1a7b2668dc1e341b493990b99e9a0cc4530d32cd hotfix-3.2.28.2 > > > > > > > > This gives me branches for each commit hash in the given range. The branches > > > > you see mentioned here diverged before the date range given. I can't say > > > > anything about performance, here -- my little test repo only has ~8000 commits > > > > going back six years. > > > > > > > > (Turning this command-line into a collection of libgit calls is left as an > > > > exercise for the reader) > > > > > > FYI, FreshPorts is using devel/py-pygit2 which uses devel/libgit2 :) > > > > > > After a git pull, I same up with this: > > > > > > [dan@devgit-ingress01:~/src/freebsd/freebsd] $ git log --branches --since 2020-08-17 --source --raw | grep ^commit > > > commit ef1942cafc385f2ebc7e4a7a5f5679bb1a4b0d78 stable/12 > > > commit 3889b52f5c73875df9a28180c4b1b4a07bf0042a stable/12 > > > commit 9fc2a101135102074391b6ff28b825b082410ffa master > > > commit a9a7211e27bfe2bec89b08899b977a4677728d97 master > > > ... several lines omitted from this paste > > > commit 5853f38da725897c25a505ce62675e31aca27f26 master > > > commit 1b47d5c7b7fb338daf65ccf05b9df0d9e93d11ae stable/12 > > > > > > Let's try a date time in there: > > > > > > [dan@devgit-ingress01:~/src/freebsd/freebsd] $ git log --branches --since "2020-08-17 23:00:00" --source --raw | grep ^commit > > > commit ef1942cafc385f2ebc7e4a7a5f5679bb1a4b0d78 stable/12 > > > commit 3889b52f5c73875df9a28180c4b1b4a07bf0042a stable/12 > > > > > > Those were both at 23:40 yesterday. > > > > > > The current approach is: I have this commit, show me all the commits after that: hash..HEAD > > > > > > That works on a branch-by-branch basis. > > > > > > > > > This may be a potential issues: The --since & --branches approach requires a date/timestamp. If we always use the timestamp of the last time we asked, will we skip over commits? > > > > Well, I would run something like --since '2 weeks ago' and ignore what > > commits you already have. It will maybe work, or not, it will probably > > depend on wether we allow branches or out of order commits, or... > > > > It would probably be safer to do for each branch, look at the commits in > > the ... range, which would get you all the commits, > > proably even if we allow merge commits. > > > > > Humor me while I contemplate URLs. > > > > > > The 3889b52f5c73875df9a28180c4b1b4a07bf0042a commit translates to: > > > > > > https://github.com/freebsd/freebsd/blob/stable/12/sys/sys/ata.h > > > > Well, nom the 3889b52f5c73875df9a28180c4b1b4a07bf0042a commit translates > > to > > > > https://github.com/freebsd/freebsd/blob/3889b52f5c73875df9a28180c4b1b4a07bf0042a/sys/sys/ata.h > > > > The URL you have is for the head of the stable/12 branch. > > Yes, you are correct. I was trying translates a commit on a branch to > the patch to the file affected by that commit. I think I got that > correct. Ouch. I was trying to translate a commit on a branch to the file path within the repo. -- Dan Langille dan@langille.org