From owner-freebsd-git@freebsd.org Wed Aug 19 13:06:43 2020 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AC9DC3BE230 for ; Wed, 19 Aug 2020 13:06:43 +0000 (UTC) (envelope-from dan@langille.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BWp2V66Txz3Ws0; Wed, 19 Aug 2020 13:06:42 +0000 (UTC) (envelope-from dan@langille.org) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 9164B5C0165; Wed, 19 Aug 2020 09:06:41 -0400 (EDT) Received: from imap36 ([10.202.2.86]) by compute2.internal (MEProxy); Wed, 19 Aug 2020 09:06:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=langille.org; h= mime-version:message-id:in-reply-to:references:date:from:to:cc :subject:content-type; s=fm2; bh=CDGnksufQm7hjL66nv7hSrq2af18OfM OBMQkIFDJoz4=; b=oAjn5PD4cOO4Gq3+LLez/UfMvLlqkNm6mDy29L1/ypAqCfv iSlasrjZLIattubIl8tnK54PMxqs9BzOwRoPZfg52AvQk20CbcFkCb2FmDHTSQkk uFTFGT2+vzRwAV6Qvdi4w2bx5SWv16lnv1efeYGey1F6Z1tvF7FQ7OSzP3h5aFT4 x2u1McD4NiPqaQV+kobNG9catEVXTlSfnNP8DYJpQNYJ29dBXePsg4JQd1xqApH/ ZUctPIHuWIfY/tkJ4ec5mrHWNGwdgmKeRj4qtQFKWHO02GvjFD7Ms6zfOOUoUJka 6l+cV+WAR3MXXZtqAAd9U/oI7ktliC63zgEDyIA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=CDGnks ufQm7hjL66nv7hSrq2af18OfMOBMQkIFDJoz4=; b=qch6Tpa9kCMBMYloW2dpV/ uKCfWgqiXy72+U2HWFdICtnafRTimNikz/qn43JeivZEq0Npm5W6g84UOlfwvG48 C5K6kLXPaAAui6dChCp4fzDGZeNVjA8udSMRuz/MgQdxzLMr2YIxe+PorjRcq1Xd rMIttikZTaVcPXKg8uKVAwAO6mys8Z4883jyTL2cCsdbt4S4qQ4k4IXKnQYysS6i tinfepRDK/95ZIYUs4n0m1DHkvoHnBUiUefBv30IzzTfxxYazyJwrz3ahmnAX4aJ EhoFWjLowMkTN5aqaYG378PccLmiDTwesAU441N2GjAthc0cPL+xkif6mtPk2YvQ == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduiedruddtkedggeefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsehttd ertderredtnecuhfhrohhmpedfffgrnhcunfgrnhhgihhllhgvfdcuoegurghnsehlrghn ghhilhhlvgdrohhrgheqnecuggftrfgrthhtvghrnhepteejhedtjeefueeujeevgfdvie dvieetueduieeiveeludegheekvdeviefhteegnecuffhomhgrihhnpehgihhthhhusgdr tghomhenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe gurghnsehlrghnghhilhhlvgdrohhrgh X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 263CB1880081; Wed, 19 Aug 2020 09:06:40 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.3.0-191-gef79d59-fm-20200818.001-gef79d590 Mime-Version: 1.0 Message-Id: In-Reply-To: <20200819124724.yoni3kjxzlboknlq@aching.in.mat.cc> References: <3430278.PbzQmE3shN@beastie.bionicmutton.org> <15d19351-3c4c-4ad1-bd1a-7f8f6c2ae211@www.fastmail.com> <20200819124724.yoni3kjxzlboknlq@aching.in.mat.cc> Date: Wed, 19 Aug 2020 09:06:20 -0400 From: "Dan Langille" To: "Mathieu Arnold" Cc: freebsd-git@freebsd.org Subject: Re: FreshPorts & src commits to branches Content-Type: text/plain X-Rspamd-Queue-Id: 4BWp2V66Txz3Ws0 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=langille.org header.s=fm2 header.b=oAjn5PD4; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=qch6Tpa9; dmarc=pass (policy=none) header.from=langille.org; spf=pass (mx1.freebsd.org: domain of dan@langille.org designates 66.111.4.25 as permitted sender) smtp.mailfrom=dan@langille.org X-Spamd-Result: default: False [-0.96 / 15.00]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[langille.org:s=fm2,messagingengine.com:s=fm3]; XM_UA_NO_VERSION(0.01)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; FREEFALL_USER(0.00)[dan]; RWL_MAILSPIKE_GOOD(0.00)[66.111.4.25:from]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.25]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[langille.org:+,messagingengine.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[langille.org,none]; NEURAL_HAM_SHORT(-0.87)[-0.873]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.25:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; MID_RHS_WWW(0.50)[]; MAILMAN_DEST(0.00)[freebsd-git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2020 13:06:43 -0000 On Wed, Aug 19, 2020, at 8:47 AM, Mathieu Arnold wrote: > On Tue, Aug 18, 2020 at 09:23:03AM -0400, Dan Langille wrote: > > On Tue, Aug 18, 2020, at 3:44 AM, Adriaan de Groot wrote: > > > On Tuesday, 18 August 2020 03:19:26 CEST Dan Langille wrote: > > > > I am quickly realizing that in order for FreshPorts to track src commits it > > > > needs to query every branch. There does not seem to be any other way to > > > > know that a commit ocurred on stable/12 without a checkout of stable/12 > > > > > > If you have the full repo (e.g. a normal clone) then you might be able to use > > > something along this line: > > > > > > git log --branches --since 2020-08-16 --source --raw | grep ^commit > > > > > > (the important bit is --branches to include all of them, and --source to > > > include the source). I don't have a FreeBSD repo at hand, but my work-repo > > > (which currently has some other branch checked out) gives me this kind of > > > output: > > > > > > commit 36cefe3be10c707a2cdfaedd14fddee5cda31f50 hotfix-3.2.28.2 > > > commit eb44d0c6be18c1ced99d5a8c1b1a6a1a0e653be2 hotfix-3.2.28.2 > > > commit 24efce0c1d5601a77ad2b919cf0cf387231add76 calamares > > > commit 5db4195b7e3e507a34f84e041b15a4aa86ed32da hotfix-3.2.28.2 > > > commit 34946ecdee90584caf255fcca732c021e1dc1832 hotfix-3.2.28.2 > > > commit 1a7b2668dc1e341b493990b99e9a0cc4530d32cd hotfix-3.2.28.2 > > > > > > This gives me branches for each commit hash in the given range. The branches > > > you see mentioned here diverged before the date range given. I can't say > > > anything about performance, here -- my little test repo only has ~8000 commits > > > going back six years. > > > > > > (Turning this command-line into a collection of libgit calls is left as an > > > exercise for the reader) > > > > FYI, FreshPorts is using devel/py-pygit2 which uses devel/libgit2 :) > > > > After a git pull, I same up with this: > > > > [dan@devgit-ingress01:~/src/freebsd/freebsd] $ git log --branches --since 2020-08-17 --source --raw | grep ^commit > > commit ef1942cafc385f2ebc7e4a7a5f5679bb1a4b0d78 stable/12 > > commit 3889b52f5c73875df9a28180c4b1b4a07bf0042a stable/12 > > commit 9fc2a101135102074391b6ff28b825b082410ffa master > > commit a9a7211e27bfe2bec89b08899b977a4677728d97 master > > ... several lines omitted from this paste > > commit 5853f38da725897c25a505ce62675e31aca27f26 master > > commit 1b47d5c7b7fb338daf65ccf05b9df0d9e93d11ae stable/12 > > > > Let's try a date time in there: > > > > [dan@devgit-ingress01:~/src/freebsd/freebsd] $ git log --branches --since "2020-08-17 23:00:00" --source --raw | grep ^commit > > commit ef1942cafc385f2ebc7e4a7a5f5679bb1a4b0d78 stable/12 > > commit 3889b52f5c73875df9a28180c4b1b4a07bf0042a stable/12 > > > > Those were both at 23:40 yesterday. > > > > The current approach is: I have this commit, show me all the commits after that: hash..HEAD > > > > That works on a branch-by-branch basis. > > > > > > This may be a potential issues: The --since & --branches approach requires a date/timestamp. If we always use the timestamp of the last time we asked, will we skip over commits? > > Well, I would run something like --since '2 weeks ago' and ignore what > commits you already have. It will maybe work, or not, it will probably > depend on wether we allow branches or out of order commits, or... > > It would probably be safer to do for each branch, look at the commits in > the ... range, which would get you all the commits, > proably even if we allow merge commits. > > > Humor me while I contemplate URLs. > > > > The 3889b52f5c73875df9a28180c4b1b4a07bf0042a commit translates to: > > > > https://github.com/freebsd/freebsd/blob/stable/12/sys/sys/ata.h > > Well, nom the 3889b52f5c73875df9a28180c4b1b4a07bf0042a commit translates > to > > https://github.com/freebsd/freebsd/blob/3889b52f5c73875df9a28180c4b1b4a07bf0042a/sys/sys/ata.h > > The URL you have is for the head of the stable/12 branch. Yes, you are correct. I was trying translates a commit on a branch to the patch to the file affected by that commit. I think I got that correct. > > > The format seems to be: > > > > https://github.com/freebsd/freebsd - URL to repo > > > > blob - some abstraction > > > > stable/12 - branch > > > > sys/sys/ata.h - path to file > > > > Compared to head/master: > > > > https://github.com/freebsd/freebsd/blob/master/sys/sys/ata.h > > > > The only difference: stable/12 -> master : very straight forward. > > What do you need the github url for? FreshPorts doesn't need the URL. It was for me. Sometimes I want to look at the branch and see if I have missed anything. -- Dan Langille dan@langille.org