From owner-freebsd-git@freebsd.org Mon Apr 12 09:02:50 2021 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 44BCE5EFE96 for ; Mon, 12 Apr 2021 09:02:50 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2a05:fc87:1:5::15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "www.spoerlein.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FJjS94C2Fz3kt0; Mon, 12 Apr 2021 09:02:49 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: from localhost (acme.spoerlein.net [IPv6:2a05:fc87:1:5:0:0:0:15]) by acme.spoerlein.net (8.16.1/8.15.2) with ESMTPS id 13C92co2052030 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 12 Apr 2021 11:02:38 +0200 (CEST) (envelope-from uqs@freebsd.org) Date: Mon, 12 Apr 2021 11:02:38 +0200 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Martin Matuska Cc: Warner Losh , Mateusz Guzik , Ryan Moeller , Alexander Motin , freebsd-git Subject: Re: OpenZFS branch tracking policy Message-ID: References: <21c7313e-315c-ec48-9437-e0a3d4ec14d2@FreeBSD.org> <41924e9d-9d61-6646-6c8f-e4458f94296e@FreeBSD.org> <30f529c1-6087-e704-8cc7-0c48a40b7430@FreeBSD.org> <9679ec9d-4916-92b7-ff70-0050d699875c@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <9679ec9d-4916-92b7-ff70-0050d699875c@FreeBSD.org> User-Agent: Mutt/2.0.3 (2020-12-04) X-Rspamd-Queue-Id: 4FJjS94C2Fz3kt0 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:39540, ipnet:2a05:fc87::/32, country:CH] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Apr 2021 09:02:50 -0000 On Sun, 2021-04-11 at 01:03:30 +0200, Martin Matuska wrote: >Thank you for your comments, Warner. > >What I would like to know is the timing - how much time do we need to >resolve the issues. I can pull in the OpenZFS code up to commit >3522f57b6 the "old" way. This is the last commit common to master and >zfs-2.1-release and can be cherry-picked to stable/13 the "old" way. >This will keep our code on par with openzfs-2.1-rc1 (rc2 is out now) and >I can add a 2-week MFC for stable/13 as usual but there are no >significant changes at all. After that we need to split main and >stable/13 and ideally move to direct tracking of OpenZFS. > >I have added some comments below. I think we should continue with the old way of squashing vendor changes in, for the main reason of bloat and slowdown for our users. Note that unlike SVN, a regular user who builds world will clone all of the git repo including all history. We have many more users than we have developers working on contrib software, so the slight convenience of a few FreeBSD devs comes at the cost of the majority of our users. :( I understand the confusion of a broken `git blame` and I'm wondering if it wouldn't be enough for the folks that want this to fetch the full OpenZFS repo into their FreeBSD repo. Then when the need arises to `git blame foo/bar.c` they see an "unhelpful" commit that says "upstream 01234abcdef was merged" upon which you can run `git blame 01234abcdef -- foo/bar.c` (paths will be different but it all can be hidden behind some script and git alias). Would that ease enough of the developers pain? I wish more stuff would move into ports (llvm, lldb) for reasons of size also. Cheers Uli From owner-freebsd-git@freebsd.org Mon Apr 12 11:09:08 2021 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AE77C5CB663 for ; Mon, 12 Apr 2021 11:09:08 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:191:9029::4]) by mx1.freebsd.org (Postfix) with ESMTP id 4FJmFw2Rh2z3rJ7; Mon, 12 Apr 2021 11:09:07 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 48F7C206838; Mon, 12 Apr 2021 13:09:00 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by mail.vx.sk (amavisd-new, unix socket) with LMTP id hP49P4Jfafxp; Mon, 12 Apr 2021 13:09:00 +0200 (CEST) Received: from [10.9.8.122] (188-167-101-78.dynamic.chello.sk [188.167.101.78]) by mail.vx.sk (Postfix) with ESMTPSA id AECEB206943; Mon, 12 Apr 2021 13:08:59 +0200 (CEST) Subject: Re: OpenZFS branch tracking policy To: =?UTF-8?Q?Ulrich_Sp=c3=b6rlein?= Cc: Warner Losh , Mateusz Guzik , Ryan Moeller , Alexander Motin , freebsd-git References: <21c7313e-315c-ec48-9437-e0a3d4ec14d2@FreeBSD.org> <41924e9d-9d61-6646-6c8f-e4458f94296e@FreeBSD.org> <30f529c1-6087-e704-8cc7-0c48a40b7430@FreeBSD.org> <9679ec9d-4916-92b7-ff70-0050d699875c@FreeBSD.org> From: Martin Matuska Message-ID: Date: Mon, 12 Apr 2021 13:08:59 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4FJmFw2Rh2z3rJ7 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Apr 2021 11:09:08 -0000 If we keep the "old way" than I have an additional question: Wouldn't a "git cherry-pick -Xsubtree=3Dsys/contrib/openzfs" from the=20 vendor branch be a better way to go than "git merge=20 -Xsubtree=3Dsys/contrib/openzfs"? Especially for stable/13, where I have=20 to "merge" in the whole new vendor/openzfs/zfs-2.1-release branch. mm On 12. 4. 2021 11:02, Ulrich Sp=C3=B6rlein wrote: > On Sun, 2021-04-11 at 01:03:30 +0200, Martin Matuska wrote: >> Thank you for your comments, Warner. >> >> What I would like to know is the timing - how much time do we need to >> resolve the issues. I can pull in the OpenZFS code up to commit >> 3522f57b6 the "old" way. This is the last commit common to master and >> zfs-2.1-release and can be cherry-picked to stable/13 the "old" way. >> This will keep our code on par with openzfs-2.1-rc1 (rc2 is out now) a= nd >> I can add a 2-week MFC for stable/13 as usual but there are no >> significant changes at all. After that we need to split main and >> stable/13 and ideally move to direct tracking of OpenZFS. >> >> I have added some comments below. > > I think we should continue with the old way of squashing vendor=20 > changes in, for the main reason of bloat and slowdown for our users.=20 > Note that unlike SVN, a regular user who builds world will clone all=20 > of the git repo including all history. We have many more users than we=20 > have developers working on contrib software, so the slight convenience=20 > of a few FreeBSD devs comes at the cost of the majority of our users. := ( > > I understand the confusion of a broken `git blame` and I'm wondering=20 > if it wouldn't be enough for the folks that want this to fetch the=20 > full OpenZFS repo into their FreeBSD repo. Then when the need arises=20 > to `git blame foo/bar.c` they see an "unhelpful" commit that says=20 > "upstream 01234abcdef was merged" upon which you can run `git blame=20 > 01234abcdef -- foo/bar.c` (paths will be different but it all can be=20 > hidden behind some script and git alias). > > Would that ease enough of the developers pain? > > I wish more stuff would move into ports (llvm, lldb) for reasons of=20 > size also. > > Cheers > Uli From owner-freebsd-git@freebsd.org Tue Apr 13 14:37:17 2021 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B16F15E2AAA for ; Tue, 13 Apr 2021 14:37:17 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2a05:fc87:1:5::15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "www.spoerlein.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FKSqd2MnBz4m0W; Tue, 13 Apr 2021 14:37:17 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: from localhost (acme.spoerlein.net [IPv6:2a05:fc87:1:5:0:0:0:15]) by acme.spoerlein.net (8.16.1/8.15.2) with ESMTPS id 13DEb5PJ082526 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Tue, 13 Apr 2021 16:37:05 +0200 (CEST) (envelope-from uqs@freebsd.org) Date: Tue, 13 Apr 2021 16:37:05 +0200 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Martin Matuska Cc: Warner Losh , Mateusz Guzik , Ryan Moeller , Alexander Motin , freebsd-git Subject: Re: OpenZFS branch tracking policy Message-ID: References: <41924e9d-9d61-6646-6c8f-e4458f94296e@FreeBSD.org> <30f529c1-6087-e704-8cc7-0c48a40b7430@FreeBSD.org> <9679ec9d-4916-92b7-ff70-0050d699875c@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/2.0.3 (2020-12-04) X-Rspamd-Queue-Id: 4FKSqd2MnBz4m0W X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:39540, ipnet:2a05:fc87::/32, country:CH] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Apr 2021 14:37:17 -0000 Hmm, I don't have an opinion on that one really. Cherry-pick of course only works on a single commit and will not record an additional parent, while a merge commit will have (at least) 2 parents. Some vendor branches sometimes have several commits in between a merge into head, so `git merge` is the natural extension of that. So only some folks can use cherry-pick and, as I said, I'm not sure what the recording of 2 parents gives us ... People with more vendor experience should chime in ... Cheers Uli On Mon, 2021-04-12 at 13:08:59 +0200, Martin Matuska wrote: >If we keep the "old way" than I have an additional question: > >Wouldn't a "git cherry-pick -Xsubtree=sys/contrib/openzfs" from the >vendor branch be a better way to go than "git merge >-Xsubtree=sys/contrib/openzfs"? Especially for stable/13, where I have >to "merge" in the whole new vendor/openzfs/zfs-2.1-release branch. > >mm > >On 12. 4. 2021 11:02, Ulrich Spörlein wrote: >> On Sun, 2021-04-11 at 01:03:30 +0200, Martin Matuska wrote: >>> Thank you for your comments, Warner. >>> >>> What I would like to know is the timing - how much time do we need to >>> resolve the issues. I can pull in the OpenZFS code up to commit >>> 3522f57b6 the "old" way. This is the last commit common to master and >>> zfs-2.1-release and can be cherry-picked to stable/13 the "old" way. >>> This will keep our code on par with openzfs-2.1-rc1 (rc2 is out now) and >>> I can add a 2-week MFC for stable/13 as usual but there are no >>> significant changes at all. After that we need to split main and >>> stable/13 and ideally move to direct tracking of OpenZFS. >>> >>> I have added some comments below. >> >> I think we should continue with the old way of squashing vendor >> changes in, for the main reason of bloat and slowdown for our users. >> Note that unlike SVN, a regular user who builds world will clone all >> of the git repo including all history. We have many more users than we >> have developers working on contrib software, so the slight convenience >> of a few FreeBSD devs comes at the cost of the majority of our users. :( >> >> I understand the confusion of a broken `git blame` and I'm wondering >> if it wouldn't be enough for the folks that want this to fetch the >> full OpenZFS repo into their FreeBSD repo. Then when the need arises >> to `git blame foo/bar.c` they see an "unhelpful" commit that says >> "upstream 01234abcdef was merged" upon which you can run `git blame >> 01234abcdef -- foo/bar.c` (paths will be different but it all can be >> hidden behind some script and git alias). >> >> Would that ease enough of the developers pain? >> >> I wish more stuff would move into ports (llvm, lldb) for reasons of >> size also. >> >> Cheers >> Uli From owner-freebsd-git@freebsd.org Tue Apr 13 16:39:57 2021 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 35FC85E5A9A for ; Tue, 13 Apr 2021 16:39:57 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x82d.google.com (mail-qt1-x82d.google.com [IPv6:2607:f8b0:4864:20::82d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FKWY835m3z4s7d for ; Tue, 13 Apr 2021 16:39:55 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x82d.google.com with SMTP id j7so13204175qtx.5 for ; Tue, 13 Apr 2021 09:39:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=JJKpRQjqd9Xbsd8uc7ngOY++solNzXjA4YlY/wn1lz8=; b=d+uQgm2Tu0OvH0Nk0nCjbuF3uZziObb15viYxUM4UFwTQMl8muIrkzwLZC9Rd89e7i +9YzRfmOWoLSD2vNGXlf0sqdZQR8zeUGSy/ffqfP4wMK/+Y9tUvqnN3mOJ2HYpTKnR0x D942aUvidx4xjtmb7790Ha8BmN4XgnjqvQFkUzXJnx28T797pnnyPMps+1cdxdndy0h/ F8Y7i8JjW0Zleh8Y3elVeTk7AIofBqjSrw+kg471/aFWV8o3WNDu7cU6n9LNWZV6ZCcb tZjGZwGvIQ8BrMMDix4ne9bqsFcK0hCyYEkC8ZzRO+JI9TeEAk851DKAvFUvcMDc5hrt 6V2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=JJKpRQjqd9Xbsd8uc7ngOY++solNzXjA4YlY/wn1lz8=; b=FcKM0Bh0A12bGeQ9Eyj+wzQ51toBL5o6V2VF1vbbwVxwOUriA0VHKz3vayu++/q8zW a377sVCOit11EZFZoomBKs/l6fKSfOr5dGVu4OM81fGXDPWwh33I8CpMZi4XgCT2ybvY mLCJzn0GPirdYSCK+WApLUSFiSB/foAJA7yS2jpVhQkfLEiXRX+1daAg85YwRrQ/s7Ya 5szhdjscbu588yqe1XujR9lFWcMYEbPEQ/9ytmizXqD2pJSgU7paIZT0wAhFyUhD5WKF N+A4sz0fY4Uvn0iNb6+AJIwXe4GuIHmBHtrtmQZsmDnr2XOTWeatsI7lcN5IDnsPG+Gb tmHw== X-Gm-Message-State: AOAM532m7qFg/vp9fj5AJZ4Zju2BNMRswjhi3To1daiuMglNHFEDC8Ct 3xEnSiIFj4q40rzJvZFNwpMN7aXmBLkyFlR7RFDfe6vrA6zksA== X-Google-Smtp-Source: ABdhPJzP3B64q3U9zd3y8VrEOSTWPSueH5B2LfmcJ+AUbRKiNvFuTxDwh11rj1cbttQL7al0cu9zgsVU9wUqheRVWVo= X-Received: by 2002:a05:622a:1c5:: with SMTP id t5mr30272458qtw.49.1618331995075; Tue, 13 Apr 2021 09:39:55 -0700 (PDT) MIME-Version: 1.0 References: <41924e9d-9d61-6646-6c8f-e4458f94296e@FreeBSD.org> <30f529c1-6087-e704-8cc7-0c48a40b7430@FreeBSD.org> <9679ec9d-4916-92b7-ff70-0050d699875c@FreeBSD.org> In-Reply-To: From: Warner Losh Date: Tue, 13 Apr 2021 10:39:44 -0600 Message-ID: Subject: Re: OpenZFS branch tracking policy To: =?UTF-8?Q?Ulrich_Sp=C3=B6rlein?= Cc: Martin Matuska , Mateusz Guzik , Ryan Moeller , Alexander Motin , freebsd-git X-Rspamd-Queue-Id: 4FKWY835m3z4s7d X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=d+uQgm2T; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::82d) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-1.00 / 15.00]; ARC_NA(0.00)[]; MAILMAN_DEST(0.00)[freebsd-git]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; RCPT_COUNT_FIVE(0.00)[6]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::82d:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_SPAM_LONG(1.00)[1.000]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::82d:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::82d:from]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_TLS_ALL(0.00)[]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Apr 2021 16:39:57 -0000 On Tue, Apr 13, 2021 at 8:37 AM Ulrich Sp=C3=B6rlein wrot= e: > Hmm, I don't have an opinion on that one really. Cherry-pick of course > only works on a single commit and will not record an additional parent, > while a merge commit will have (at least) 2 parents. > Correct. > Some vendor branches sometimes have several commits in between a merge > into head, so `git merge` is the natural extension of that. So only some > folks can use cherry-pick and, as I said, I'm not sure what the > recording of 2 parents gives us ... > So for normal, low velocity updates, there's little benefit from doing more than what we've done with vendor imports. But for OpenZFS I think there's three primary values from store their branches in our tree and doing merge commits: (1) git blame works (2) it's possible to bisect down to the exact commit (3) Having the merge commits recorded as merge commits makes future commits easier (just like vendor branches). For most things, I agree with Uli: we should have some flavor of 'squash' commit that's not really a merge commit to do this. But for OpenZFS, I think there's enough synergy between the two project that having their branches in our tree would be a net win for both groups. Warner > People with more vendor experience should chime in ... > > Cheers > Uli > > On Mon, 2021-04-12 at 13:08:59 +0200, Martin Matuska wrote: > >If we keep the "old way" than I have an additional question: > > > >Wouldn't a "git cherry-pick -Xsubtree=3Dsys/contrib/openzfs" from the > >vendor branch be a better way to go than "git merge > >-Xsubtree=3Dsys/contrib/openzfs"? Especially for stable/13, where I have > >to "merge" in the whole new vendor/openzfs/zfs-2.1-release branch. > > > >mm > > > >On 12. 4. 2021 11:02, Ulrich Sp=C3=B6rlein wrote: > >> On Sun, 2021-04-11 at 01:03:30 +0200, Martin Matuska wrote: > >>> Thank you for your comments, Warner. > >>> > >>> What I would like to know is the timing - how much time do we need to > >>> resolve the issues. I can pull in the OpenZFS code up to commit > >>> 3522f57b6 the "old" way. This is the last commit common to master and > >>> zfs-2.1-release and can be cherry-picked to stable/13 the "old" way. > >>> This will keep our code on par with openzfs-2.1-rc1 (rc2 is out now) > and > >>> I can add a 2-week MFC for stable/13 as usual but there are no > >>> significant changes at all. After that we need to split main and > >>> stable/13 and ideally move to direct tracking of OpenZFS. > >>> > >>> I have added some comments below. > >> > >> I think we should continue with the old way of squashing vendor > >> changes in, for the main reason of bloat and slowdown for our users. > >> Note that unlike SVN, a regular user who builds world will clone all > >> of the git repo including all history. We have many more users than we > >> have developers working on contrib software, so the slight convenience > >> of a few FreeBSD devs comes at the cost of the majority of our users. = :( > >> > >> I understand the confusion of a broken `git blame` and I'm wondering > >> if it wouldn't be enough for the folks that want this to fetch the > >> full OpenZFS repo into their FreeBSD repo. Then when the need arises > >> to `git blame foo/bar.c` they see an "unhelpful" commit that says > >> "upstream 01234abcdef was merged" upon which you can run `git blame > >> 01234abcdef -- foo/bar.c` (paths will be different but it all can be > >> hidden behind some script and git alias). > >> > >> Would that ease enough of the developers pain? > >> > >> I wish more stuff would move into ports (llvm, lldb) for reasons of > >> size also. > >> > >> Cheers > >> Uli > From owner-freebsd-git@freebsd.org Fri Apr 16 19:24:25 2021 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A70C05DC747 for ; Fri, 16 Apr 2021 19:24:25 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [144.76.20.103]) by mx1.freebsd.org (Postfix) with ESMTP id 4FMR3Y1crSz4l0Q; Fri, 16 Apr 2021 19:24:25 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 106791D8A0A; Fri, 16 Apr 2021 21:24:18 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by mail.vx.sk (amavisd-new, unix socket) with LMTP id uFDBWgb9BK1p; Fri, 16 Apr 2021 21:24:17 +0200 (CEST) Received: from [10.9.8.122] (188-167-101-78.dynamic.chello.sk [188.167.101.78]) by mail.vx.sk (Postfix) with ESMTPSA id 8C2331D88CF; Fri, 16 Apr 2021 21:24:17 +0200 (CEST) To: Warner Losh , =?UTF-8?Q?Ulrich_Sp=c3=b6rlein?= Cc: Mateusz Guzik , Ryan Moeller , Alexander Motin , freebsd-git References: <41924e9d-9d61-6646-6c8f-e4458f94296e@FreeBSD.org> <30f529c1-6087-e704-8cc7-0c48a40b7430@FreeBSD.org> <9679ec9d-4916-92b7-ff70-0050d699875c@FreeBSD.org> From: Martin Matuska Subject: Re: OpenZFS branch tracking policy Message-ID: <2b404ead-d862-c4ba-41cd-4ceb1246ce6f@FreeBSD.org> Date: Fri, 16 Apr 2021 21:24:17 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-Rspamd-Queue-Id: 4FMR3Y1crSz4l0Q X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; ASN(0.00)[asn:24940, ipnet:144.76.0.0/16, country:DE] Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Apr 2021 19:24:25 -0000 Thank you guys for your input. OpenZFS 2.1 has already gone -RC3, eliminating even more diffs the code=20 in our tree. I have merged OpenZFS-2.1 RC1 the old way up to the last common commit.=20 I don't know who or what body is in charge to make a decision on this=20 matter but I would be very happy if a decision is made. I am personally=20 slightly in favor of merging directly from OpenZFS as it makes my work=20 easier and less prone to mistakes but I don't object doing it the "old"=20 way. But even the "old" way is going to be different, as it would mean=20 doing vendor merges into stable/13 what we are not used to. On the other = hand I do "squashed" imports anyway so I could cherry-pick from the new=20 vendor branch into stable/13 as well. One way or another, I would like to continue pushing recent OpenZFS code = to our tree. Martin On 13. 4. 2021 18:39, Warner Losh wrote: > > > On Tue, Apr 13, 2021 at 8:37 AM Ulrich Sp=C3=B6rlein > wrote: > > Hmm, I don't have an opinion on that one really. Cherry-pick of > course > only works on a single commit and will not record an additional > parent, > while a merge commit will have (at least) 2 parents. > > > Correct. > > Some vendor branches sometimes have several commits in between a > merge > into head, so `git merge` is the natural extension of that. So > only some > folks can use cherry-pick and, as I said, I'm not sure what the > recording of 2 parents gives us ... > > > So for normal, low velocity updates, there's little benefit from doing = > more than what we've done with vendor imports. > > But for OpenZFS I think there's three primary values from store their=20 > branches in our tree and doing merge commits: > > (1) git blame works > (2) it's possible to bisect down to the exact commit > (3) Having the merge commits recorded as merge commits makes future=20 > commits easier (just like vendor branches). > > For most things, I agree with Uli: we should have some flavor of=20 > 'squash' commit that's not really a merge commit to do this.=C2=A0 But = for=20 > OpenZFS, I think there's enough synergy between the two project that=20 > having their branches in our tree would be a net win for both groups. > > Warner > > People with more vendor experience should chime in ... > > Cheers > Uli > > On Mon, 2021-04-12 at 13:08:59 +0200, Martin Matuska wrote: > >If we keep the "old way" than I have an additional question: > > > >Wouldn't a "git cherry-pick -Xsubtree=3Dsys/contrib/openzfs" from = the > >vendor branch be a better way to go than "git merge > >-Xsubtree=3Dsys/contrib/openzfs"? Especially for stable/13, where = I > have > >to "merge" in the whole new vendor/openzfs/zfs-2.1-release branch.= > > > >mm > > > >On 12. 4. 2021 11:02, Ulrich Sp=C3=B6rlein wrote: > >> On Sun, 2021-04-11 at 01:03:30 +0200, Martin Matuska wrote: > >>> Thank you for your comments, Warner. > >>> > >>> What I would like to know is the timing - how much time do we > need to > >>> resolve the issues. I can pull in the OpenZFS code up to commit= > >>> 3522f57b6 the "old" way. This is the last commit common to > master and > >>> zfs-2.1-release and can be cherry-picked to stable/13 the > "old" way. > >>> This will keep our code on par with openzfs-2.1-rc1 (rc2 is > out now) and > >>> I can add a 2-week MFC for stable/13 as usual but there are no > >>> significant changes at all. After that we need to split main an= d > >>> stable/13 and ideally move to direct tracking of OpenZFS. > >>> > >>> I have added some comments below. > >> > >> I think we should continue with the old way of squashing vendor > >> changes in, for the main reason of bloat and slowdown for our > users. > >> Note that unlike SVN, a regular user who builds world will > clone all > >> of the git repo including all history. We have many more users > than we > >> have developers working on contrib software, so the slight > convenience > >> of a few FreeBSD devs comes at the cost of the majority of our > users. :( > >> > >> I understand the confusion of a broken `git blame` and I'm > wondering > >> if it wouldn't be enough for the folks that want this to fetch t= he > >> full OpenZFS repo into their FreeBSD repo. Then when the need > arises > >> to `git blame foo/bar.c` they see an "unhelpful" commit that say= s > >> "upstream 01234abcdef was merged" upon which you can run `git bl= ame > >> 01234abcdef -- foo/bar.c` (paths will be different but it all > can be > >> hidden behind some script and git alias). > >> > >> Would that ease enough of the developers pain? > >> > >> I wish more stuff would move into ports (llvm, lldb) for reasons= of > >> size also. > >> > >> Cheers > >> Uli > From owner-freebsd-git@freebsd.org Fri Apr 16 19:27:38 2021 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5F0645DCAE0 for ; Fri, 16 Apr 2021 19:27:38 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x831.google.com (mail-qt1-x831.google.com [IPv6:2607:f8b0:4864:20::831]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FMR7F4sGgz4l4N for ; Fri, 16 Apr 2021 19:27:37 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x831.google.com with SMTP id f12so21624177qtf.2 for ; Fri, 16 Apr 2021 12:27:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=vb4mQNpSezj3Gm5S80MJRzW1uslXBKHX1vk7O5ziviE=; b=Isj2KzlR4hAIX0NqB3giiGTB60eCs9vPmUcVnuQ7PwQ25DykqhPqlK2dJBKm/Pelef xhSKjbSSKddQoXKr950CyYbP5xMOrViTa4vpeUbNHuMkcO/oWh3R4RX6WCYCZgYBSlmr 8v5CPyGLFyMw9LF6CVBCdZTCeIbiBc3WfXxDtDHvSIkuoCCAB/fHI0FqFsjkdPzVygN/ 4KnjPh7jK+LRIOVcUFB2JoD6phTTus8MstA3g2n/6/Jlw7gZXV4pZU/RAxB8I4VVPj4G 97j+vqRID3GixtQZn1YwGJj0tdjclZ15hYuIwsicN8GWyqwMzMIW5NFjwAl2Pq0e2mHU 5nkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=vb4mQNpSezj3Gm5S80MJRzW1uslXBKHX1vk7O5ziviE=; b=CZab6DdNp7IawaWv6mXIxpQdE1T0v8eKMW2ZBNpCq5VJT7eiEK4wAJWQdrnKXT8xBz Ex7eTjOSl3CYjjNMsWhLzg4/7uhM+f/MEPU2BIclCsCm/imlYSsBQFxy3X0t/WPzK0mI 9vbyJJuIK4wSmz3QcwA6uVcnV9iTPGIlrHTBUqDT4+TAteJ5zPK5wSnq1tnagWKHI3US CkTBmJXBEG2EdtJLnvChVMonqmSZyb5dMljS6bqaCSGZSuhSUHPLrUyP/J5/ZgWK3CNd sry9pjVmtTTcDBFKvll+zFitxOETxGMZSTrOGvq+g11QkWLy3F6Q/N4uVt0DgVZlW9o+ EFbw== X-Gm-Message-State: AOAM530zTOFuDMtmxfyA2C8sCE9JbmXVRRc+FPpSiZWEp4VkTe+N0U41 fq6xl680gbOhjYmBTePI2pfT6kyUaYIo0rTeNbrOWg== X-Google-Smtp-Source: ABdhPJyna4v+KSiH0q7oxdJ9Q3e6KviZ7VZSj5bPKM4AEuzQnmRQl6/gHq5bOAAIuZAkqz1DzjqXF/iqykxH1jXfQsk= X-Received: by 2002:ac8:7e95:: with SMTP id w21mr673978qtj.244.1618601256259; Fri, 16 Apr 2021 12:27:36 -0700 (PDT) MIME-Version: 1.0 References: <41924e9d-9d61-6646-6c8f-e4458f94296e@FreeBSD.org> <30f529c1-6087-e704-8cc7-0c48a40b7430@FreeBSD.org> <9679ec9d-4916-92b7-ff70-0050d699875c@FreeBSD.org> <2b404ead-d862-c4ba-41cd-4ceb1246ce6f@FreeBSD.org> In-Reply-To: <2b404ead-d862-c4ba-41cd-4ceb1246ce6f@FreeBSD.org> From: Warner Losh Date: Fri, 16 Apr 2021 13:27:25 -0600 Message-ID: Subject: Re: OpenZFS branch tracking policy To: Martin Matuska Cc: =?UTF-8?Q?Ulrich_Sp=C3=B6rlein?= , Mateusz Guzik , Ryan Moeller , Alexander Motin , freebsd-git X-Rspamd-Queue-Id: 4FMR7F4sGgz4l4N X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=Isj2KzlR; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::831) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-0.99 / 15.00]; ARC_NA(0.00)[]; MAILMAN_DEST(0.00)[freebsd-git]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; RCPT_COUNT_FIVE(0.00)[6]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::831:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-0.99)[-0.991]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::831:from]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::831:from]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_TLS_ALL(0.00)[]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Apr 2021 19:27:38 -0000 On Fri, Apr 16, 2021 at 1:24 PM Martin Matuska wrote: > Thank you guys for your input. > > OpenZFS 2.1 has already gone -RC3, eliminating even more diffs the code i= n > our tree. > > I have merged OpenZFS-2.1 RC1 the old way up to the last common commit. I > don't know who or what body is in charge to make a decision on this matte= r > but I would be very happy if a decision is made. I am personally slightly > in favor of merging directly from OpenZFS as it makes my work easier and > less prone to mistakes but I don't object doing it the "old" way. But eve= n > the "old" way is going to be different, as it would mean doing vendor > merges into stable/13 what we are not used to. On the other hand I do > "squashed" imports anyway so I could cherry-pick from the new vendor bran= ch > into stable/13 as well. > > One way or another, I would like to continue pushing recent OpenZFS code > to our tree. > Hi Martin, I think ultimately it is this group, and since I've been handling it, it's on me to make sure the problem gets to resolution soon. I have one open issue: if you were to bring in the current OpenZFS branch as a vendor branch, would our automation generate a lot of email. I think it would, and need to make sure it wouldn't, or we'd have some way to limit it before moving ahead. I hope to know by Tuesday one way or the other.... Warner > Martin > On 13. 4. 2021 18:39, Warner Losh wrote: > > > > On Tue, Apr 13, 2021 at 8:37 AM Ulrich Sp=C3=B6rlein wr= ote: > >> Hmm, I don't have an opinion on that one really. Cherry-pick of course >> only works on a single commit and will not record an additional parent, >> while a merge commit will have (at least) 2 parents. >> > > Correct. > > >> Some vendor branches sometimes have several commits in between a merge >> into head, so `git merge` is the natural extension of that. So only some >> folks can use cherry-pick and, as I said, I'm not sure what the >> recording of 2 parents gives us ... >> > > So for normal, low velocity updates, there's little benefit from doing > more than what we've done with vendor imports. > > But for OpenZFS I think there's three primary values from store their > branches in our tree and doing merge commits: > > (1) git blame works > (2) it's possible to bisect down to the exact commit > (3) Having the merge commits recorded as merge commits makes future > commits easier (just like vendor branches). > > For most things, I agree with Uli: we should have some flavor of 'squash' > commit that's not really a merge commit to do this. But for OpenZFS, I > think there's enough synergy between the two project that having their > branches in our tree would be a net win for both groups. > > Warner > > >> People with more vendor experience should chime in ... >> >> Cheers >> Uli >> >> On Mon, 2021-04-12 at 13:08:59 +0200, Martin Matuska wrote: >> >If we keep the "old way" than I have an additional question: >> > >> >Wouldn't a "git cherry-pick -Xsubtree=3Dsys/contrib/openzfs" from the >> >vendor branch be a better way to go than "git merge >> >-Xsubtree=3Dsys/contrib/openzfs"? Especially for stable/13, where I hav= e >> >to "merge" in the whole new vendor/openzfs/zfs-2.1-release branch. >> > >> >mm >> > >> >On 12. 4. 2021 11:02, Ulrich Sp=C3=B6rlein wrote: >> >> On Sun, 2021-04-11 at 01:03:30 +0200, Martin Matuska wrote: >> >>> Thank you for your comments, Warner. >> >>> >> >>> What I would like to know is the timing - how much time do we need t= o >> >>> resolve the issues. I can pull in the OpenZFS code up to commit >> >>> 3522f57b6 the "old" way. This is the last commit common to master an= d >> >>> zfs-2.1-release and can be cherry-picked to stable/13 the "old" way. >> >>> This will keep our code on par with openzfs-2.1-rc1 (rc2 is out now) >> and >> >>> I can add a 2-week MFC for stable/13 as usual but there are no >> >>> significant changes at all. After that we need to split main and >> >>> stable/13 and ideally move to direct tracking of OpenZFS. >> >>> >> >>> I have added some comments below. >> >> >> >> I think we should continue with the old way of squashing vendor >> >> changes in, for the main reason of bloat and slowdown for our users. >> >> Note that unlike SVN, a regular user who builds world will clone all >> >> of the git repo including all history. We have many more users than w= e >> >> have developers working on contrib software, so the slight convenienc= e >> >> of a few FreeBSD devs comes at the cost of the majority of our users. >> :( >> >> >> >> I understand the confusion of a broken `git blame` and I'm wondering >> >> if it wouldn't be enough for the folks that want this to fetch the >> >> full OpenZFS repo into their FreeBSD repo. Then when the need arises >> >> to `git blame foo/bar.c` they see an "unhelpful" commit that says >> >> "upstream 01234abcdef was merged" upon which you can run `git blame >> >> 01234abcdef -- foo/bar.c` (paths will be different but it all can be >> >> hidden behind some script and git alias). >> >> >> >> Would that ease enough of the developers pain? >> >> >> >> I wish more stuff would move into ports (llvm, lldb) for reasons of >> >> size also. >> >> >> >> Cheers >> >> Uli >> > From owner-freebsd-git@freebsd.org Sat Apr 17 01:23:41 2021 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1B7EE5E89A7 for ; Sat, 17 Apr 2021 01:23:41 +0000 (UTC) (envelope-from dan@langille.org) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FMb2426MPz3ML2; Sat, 17 Apr 2021 01:23:40 +0000 (UTC) (envelope-from dan@langille.org) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id A798B5C0ECB; Fri, 16 Apr 2021 21:08:27 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 16 Apr 2021 21:08:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=langille.org; h= from:content-type:content-transfer-encoding:mime-version:subject :message-id:date:cc:to; s=fm1; bh=n0tvsXJgORj2QhYly3zbUd9tiHgfWh EafMZ/etkVBK8=; b=fGgLPQlfujyT1XVklRselpmgz2u/kftIVM+3P3ljouXehr zB6ZY09g4Sgw74ZJXNiZkQ0AR7XsqSMu46SOBoqTrdzHLhLVrzfl+uA5ZwrseWu1 nQls+gtSwhKkIwoUz5yPiJeQ7shsN0LGoLxXMM7I301icolX/jMQztRz3Je76aG6 LEMpQL80wCFRauRsLR30ofiPH4nuci/eryCfJBCQjiLiwfFLGOfXQHrDV6AuM8V6 HY0xIFxih0FJvZah4ddfZez1RqEcPbO0B+AA/8xbHSMk8Eso6D4wvzTeoUDJZROD hgOv0twHW8dZFvFtyGxTTf8mLEtqEfA4GThm2WJg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=n0tvsX JgORj2QhYly3zbUd9tiHgfWhEafMZ/etkVBK8=; b=QP6Ki9S3kSyPSDKpVLzmGn f2Q3qXrdV36f7ajTVbcQsO5tPA+3swPXdGGBF7TLVxAFATEvbXWt0ECq78CnU6o+ iUGv+fLU56zrjUwlS6PCNVn80pWdu98BWDQyxDaaCgIc9AdbGph4jD139DidylwV 9spCdpZyAA/XH1RNDq8aTPx2sAYMiZrTJzg24dcajnjnnNc2ZLXsvEvP0T0jO5ut kU0hWkOC0e1+S6SRPYl1v9CKP+hT32+BkPLMApchEmM54PwSVzuJNVGoUw3+w1Du yxpSbA48M9sGj3VnNcVbS7a9x5ItprWHytDIF7XwHerMdQPYy5wF+3XUYkzUS1qg == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudeliedggedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhtgfgggfukfffvffosehtqhhmtd hhtddvnecuhfhrohhmpeffrghnucfnrghnghhilhhlvgcuoegurghnsehlrghnghhilhhl vgdrohhrgheqnecuggftrfgrthhtvghrnhepgfehgfeuheehteefteeifeffjeduieeuue ejvedugfeuteejffdugfduhfekveevnecuffhomhgrihhnpehfrhgvvggsshgurdhorhhg necukfhppeejvddrjeekrdduledtrdduledtnecuvehluhhsthgvrhfuihiivgeptdenuc frrghrrghmpehmrghilhhfrhhomhepuggrnheslhgrnhhgihhllhgvrdhorhhg X-ME-Proxy: Received: from [10.222.78.31] (pool-72-78-190-190.phlapa.fios.verizon.net [72.78.190.190]) by mail.messagingengine.com (Postfix) with ESMTPA id 1C2DA1080064; Fri, 16 Apr 2021 21:08:27 -0400 (EDT) From: Dan Langille Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: FreshPorts - processing branches: help wanted Message-Id: <35888E09-CF1B-42AD-98B4-3792A0458912@langille.org> Date: Fri, 16 Apr 2021 21:08:23 -0400 To: freebsd-git@freebsd.org X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4FMb2426MPz3ML2 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=langille.org header.s=fm1 header.b=fGgLPQlf; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=QP6Ki9S3; dmarc=pass (policy=none) header.from=langille.org; spf=pass (mx1.freebsd.org: domain of dan@langille.org designates 66.111.4.29 as permitted sender) smtp.mailfrom=dan@langille.org X-Spamd-Result: default: False [-1.60 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.29]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[langille.org:+,messagingengine.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[langille.org,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RECEIVED_SPAMHAUS_PBL(0.00)[72.78.190.190:received]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[66.111.4.29:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[langille.org:s=fm1,messagingengine.com:s=fm2]; FREEFALL_USER(0.00)[dan]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; SPAMHAUS_ZRD(0.00)[66.111.4.29:from:127.0.2.255]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.29:from]; MAILMAN_DEST(0.00)[freebsd-git] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Apr 2021 01:23:41 -0000 Back in Nov mat wrote: re: = https://lists.freebsd.org/pipermail/freebsd-git/2020-November/000498.html > =46rom what I understood, freshports never needs to be able to access > files on the top of a branch, it only needs to access the files on > specific commits (the fact that a commit is at the top of the branch = is > only an artefact of the process). So, you never need to checkout any > branches, you only need to checkout commits. So you never have a HEAD > that points to a branch, HEAD is always in a detached state and points > to a commit. Yes. This is the current state on production: [dan@ingress01:/var/db/freshports/ports-jail/var/db/repos/ports] $ git = status HEAD detached at 9d5f4ef1a469 nothing to commit, working tree clean > So what you need to do is, git clone the repository, and then, each = time > the script runs, do (in somewhat shell script): >=20 > ############################# > NAME_OF_REMOTE=3Dorigin > NAME_OF_HEAD=3Dmain >=20 > cd /where/the/repo/is >=20 > git fetch >=20 > # get all references (so, branches, tags, and so one) and keep only > # branches (named commits here) that the remote repository has >=20 If I want to follow a new remote branch, I have to first do: git = checkout ? This makes some sense to me, but this late in the day, I can't follow = it. I'm hoping someone is willing to follow up on this suggestion and = proceed with coding. It doesn't depend upon having a FreshPorts environment at all. Thank you. > git for-each-ref --format '%(objecttype) %(refname)' \ > | sed -n 's/^commit refs\/remotes\///p' > | while read -r type refname > do >=20 > # If we don't have the tag, it means we have not encountered the > # branch yet. > if ! git tag -l freshports/$refname > then >=20 > # get the first commit of that branch and create a tag. > git tag -m "first known commit of $refname" -f freshports/$refname = $(git merge-base $NAME_OF_REMOTE/$NAME_OF_HEAD $refname) > fi >=20 > # Get the list of commits between the last known one and the tip of > # the branch, list may be empty. > git rev-list freshports/$refname..$refname | while read commithash > do > # checkout that commit (with -f so that if some file got changed, = we > # overwrite everything > git checkout -f $commithash >=20 > # process the commit > done >=20 > # Store the last known commit that we just processed. > git tag -m "last known commit of $refname" -f freshports/$refname = $refname > done > ############################# >=20 > That's all, you never need to merge or pull or whatever else. The current solution is doing a 'git fetch'. Now it's time to start doing branches. Thank you.=20 --=20 Dan Langille - BSDCan / PGCon dan@langille.org From owner-freebsd-git@freebsd.org Sat Apr 17 04:49:00 2021 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 00B7B5EE118 for ; Sat, 17 Apr 2021 04:49:00 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x82a.google.com (mail-qt1-x82a.google.com [IPv6:2607:f8b0:4864:20::82a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FMgZz0dCRz3mcF for ; Sat, 17 Apr 2021 04:48:58 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x82a.google.com with SMTP id t11so13816472qtr.8 for ; Fri, 16 Apr 2021 21:48:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=BqIIP8S5utkYYfwdnuGVOXn5HMC5J6+wC0gRS2IZKaw=; b=q2XSXxAeNLGrFr0tBqN4jnVjUKuBAO7742zC/WoB2lI0jaSTx2HwjSmfETYZQdf7G9 xwYnbvv9zTKCpc/eY/eNX7p5MXJdiz9qVcqhN4dZuUfePirEmqp6p+ArnAYzRCvuTJx8 Xv0XuhPoX9klWOOKS59R0UL8ap/hi82yeXlKAQlFxM2mszpdJIm9WFXmmFH8d2vvepC2 w/ZKraePz6x1RxkdaU8d8d3jHdhGOBbT+BcLxgM/e3zWCcztPFqYebzxF0kf5HG7jLkc AOerC2hZ0F1Ax3v5MtGpjH9LyjypsMG+FMz4Y/6s8yRyapf8JQOM8jpcjbRLYNKRDfPI SVrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=BqIIP8S5utkYYfwdnuGVOXn5HMC5J6+wC0gRS2IZKaw=; b=s5Qq034wJDwLfiRxXt8qlX4O7eHTcRKS6FKpTcybgM4y7/935hY97SrNQDBhTAunXT 4LyDK7BNHPvVC5/bYfxdzj6PwJ+hggrZ2UA+dkzIrozuC8Jf8tpVy7sxuHQzmXTqW/BS 1jJ8Zt3y5t0BuSrjBEzjPHhgGmSYn/QbwJpkTCdf3/8oe5c6Rq4EYkVGMIqm9EQ8QGNx w41+nTgn4RparsBO45Pzgy20TUjrkAp18ahkFVt+2ku0nvptJTj2YbExzyl+ubj7BSKQ EmaOs8chk47F7qV21Tt0/8FurViP3km/Vs8nOEHQIOIV+blf7+2iCKeiEjke7Oly2Ykp JYPA== X-Gm-Message-State: AOAM532Ui70ula/TRxJe145XNuoKs9E7vV2KO8/WCMOCIbiYbQs2H7ov jYepxzybHFDLKuxD3Y7WWqQEDcljfpT4kjIw8ElIvVzYScpAFA== X-Google-Smtp-Source: ABdhPJznvjoaiZVEB6A62nwrF0X42n/buok5Oxg+s3fuwsvQfdzi/R2YhSX1l8LPwh10ToA/lYNkUlh5l9QNLmm/LUg= X-Received: by 2002:a05:622a:1748:: with SMTP id l8mr2287586qtk.73.1618634937754; Fri, 16 Apr 2021 21:48:57 -0700 (PDT) MIME-Version: 1.0 References: <35888E09-CF1B-42AD-98B4-3792A0458912@langille.org> In-Reply-To: <35888E09-CF1B-42AD-98B4-3792A0458912@langille.org> From: Warner Losh Date: Fri, 16 Apr 2021 22:48:46 -0600 Message-ID: Subject: Re: FreshPorts - processing branches: help wanted To: Dan Langille Cc: freebsd-git X-Rspamd-Queue-Id: 4FMgZz0dCRz3mcF X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=q2XSXxAe; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::82a) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [1.00 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; RBL_SENDERSCORE_FAIL(0.00)[2607:f8b0:4864:20::82a:server fail]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(1.00)[1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::82a:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::82a:from]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::82a:from]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; MAILMAN_DEST(0.00)[freebsd-git]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Apr 2021 04:49:00 -0000 On Fri, Apr 16, 2021 at 7:23 PM Dan Langille wrote: > Back in Nov mat wrote: > > re: > https://lists.freebsd.org/pipermail/freebsd-git/2020-November/000498.html > > > From what I understood, freshports never needs to be able to access > > files on the top of a branch, it only needs to access the files on > > specific commits (the fact that a commit is at the top of the branch is > > only an artefact of the process). So, you never need to checkout any > > branches, you only need to checkout commits. So you never have a HEAD > > that points to a branch, HEAD is always in a detached state and points > > to a commit. > > Yes. This is the current state on production: > > [dan@ingress01:/var/db/freshports/ports-jail/var/db/repos/ports] $ git > status > HEAD detached at 9d5f4ef1a469 > nothing to commit, working tree clean > So something moved the tip of the main branch, or you checked out a specific version. > > So what you need to do is, git clone the repository, and then, each time > > the script runs, do (in somewhat shell script): > > > > ############################# > > NAME_OF_REMOTE=origin > > NAME_OF_HEAD=main > > > > cd /where/the/repo/is > > > > git fetch > > > > # get all references (so, branches, tags, and so one) and keep only > > # branches (named commits here) that the remote repository has > > > > If I want to follow a new remote branch, I have to first do: git checkout > ? > Yes. > This makes some sense to me, but this late in the day, I can't follow it. > > I'm hoping someone is willing to follow up on this suggestion and proceed > with coding. > It doesn't depend upon having a FreshPorts environment at all. > Yea, I'm out of hours in the day and can just advise on how to proceed. Warner > Thank you. > > > git for-each-ref --format '%(objecttype) %(refname)' \ > > | sed -n 's/^commit refs\/remotes\///p' > > | while read -r type refname > > do > > > > # If we don't have the tag, it means we have not encountered the > > # branch yet. > > if ! git tag -l freshports/$refname > > then > > > > # get the first commit of that branch and create a tag. > > git tag -m "first known commit of $refname" -f freshports/$refname > $(git merge-base $NAME_OF_REMOTE/$NAME_OF_HEAD $refname) > > fi > > > > # Get the list of commits between the last known one and the tip of > > # the branch, list may be empty. > > git rev-list freshports/$refname..$refname | while read commithash > > do > > # checkout that commit (with -f so that if some file got changed, we > > # overwrite everything > > git checkout -f $commithash > > > > # process the commit > > done > > > > # Store the last known commit that we just processed. > > git tag -m "last known commit of $refname" -f freshports/$refname > $refname > > done > > ############################# > > > > That's all, you never need to merge or pull or whatever else. > > The current solution is doing a 'git fetch'. > > Now it's time to start doing branches. > > Thank you. > > -- > Dan Langille - BSDCan / PGCon > dan@langille.org > > > _______________________________________________ > freebsd-git@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-git > To unsubscribe, send any mail to "freebsd-git-unsubscribe@freebsd.org" >