From owner-freebsd-git@freebsd.org Sun Apr 25 14:02:26 2021 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D3733623D3D for ; Sun, 25 Apr 2021 14:02:26 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FSqTt43STz3QBh; Sun, 25 Apr 2021 14:02:26 +0000 (UTC) (envelope-from uqs@freebsd.org) Received: from localhost (acme.spoerlein.net [IPv6:2a05:fc87:1:5::15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: uqs/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 2376D549C; Sun, 25 Apr 2021 14:02:26 +0000 (UTC) (envelope-from uqs@freebsd.org) Date: Sun, 25 Apr 2021 16:02:22 +0200 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Mark Johnston Cc: freebsd-git@freebsd.org Subject: Re: vendor/illumos merges Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/2.0.3 (2020-12-04) X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Apr 2021 14:02:26 -0000 On Sat, 2021-04-24 at 11:08:58 -0400, Mark Johnston wrote: >On Sat, Apr 24, 2021 at 12:44:40PM +0200, Ulrich Spörlein wrote: >> On Fri, 2021-04-23 at 17:26:33 -0400, Mark Johnston wrote: >> >Hi, >> > >> >Now that FreeBSD uses OpenZFS as the upstream for ZFS, vendor/illumos is >> >mostly unused. However, we still use illumos as an upstream for CTF >> >tools and DTrace, though there haven't been any imports in a while. >> > >> >illumos has put a lot of work into their CTF toolchain, and I'd like to >> >import that. There are a couple of snags that I'd appreciate some >> >guidance on. >> > >> >First, I believe I should delete now-unused ZFS code from the vendor >> >branch and merge the result to main. I did this locally and got an >> >empty merge, which is what I'd expect. Is there any problem with this? >> >> Why would you record this empty merge? If you clean up vendor/foo, just >> do that but don't merge a no-op back into main (nothing changed, after >> all). > >Ok, I guess there is no reason to merge that change separately. It >will end up being merged with subsequent imports though. > >> >Second, with Subversion we had both vendor/illumos and >> >vendor-sys/illumos, and now we just have the former, seemingly with >> >sys/* bits imported from vendor-sys. Some of the upstream commits touch >> >both userspace and kernel bits, but the merge targets for these in >> >FreeBSD are different: cddl/contrib/opensolaris vs. >> >sys/cddl/contrib/opensolaris. How should I merge into main in this >> >case? I don't really see any options other than to split each offending >> >upstream commit into two parts, one for userspace and one for the >> >kernel, and merge them separately. >> > >> >If it helps to look at the branch where I staged the upstream commits, >> >I've pushed it to vendor/illumos2 in https://github.com/markjdb/freebsd >> >. >> >> Can you clarify why the merging of the two might be an issue? Note that >> unlike subversion, in git there's no "merge a certain subtree" handling, >> all that is recorded is a tree of some form and then a set of parents or >> ancestor commits. (git is a content tracker, not really a VCS :) >> >> I was under the impression that userland and kernel imports/merges need >> to happen at the same time anyway, so I assume you would import all the >> bits under vendor/foo in 1 commit and then merge them in 1 commit into >> main. Is that not how it goes? > >How can I do that with git subtree merge? Suppose an illumos commit >modifies cmd/dtrace/foo.c (userspace) and uts/common/dtrace/foo.c >(kernel). That maps to cddl/contrib/opensolaris/cmd/dtrace/foo.c and >sys/cddl/contrib/opensolaris/uts/common/dtrace/foo.c in FreeBSD, >respectively. So to do a subtree merge, I need to use distinct prefixes >depending on whether I'm importing userspace or kernel changes. When >they are mixed together, it's not clear to me how I can merge at all. > >I see that for OpenZFS we keep all code, including userspace code, under >sys/contrib/openzfs, so it doesn't have this problem. I don't think you want a subtree merge, especially as things are scattered all over the place. Also note that none of this subtree magic is in any way recorded in the git data, all it does is help you with the 3-way merges (or whatever). So I would do: - import whatever you need into contrib/foo, commit normally. - munge /usr/src to have every kernel and userland stuff (not sure what other merge tools exist, just make sure to copy over file deletions as well :). You could rsync --del two times with the right source/dest pairs, or export a diff/patch from step 1 and apply it under the right prefixes. test, test, test. - write out this tree to git using: git write-tree - then commit this using: git commit-tree -m "my message" -p HEAD -p origin/vendor/illumos - bump main to point to that hash using git update-ref - git log --graph and inspect the hell out of this - git push, then curse that we disallow merge commits and you need to `git pull --rebase` to advance to the latest published head and that might mess up your merge commit pretty bad :( Maybe 2x git subtree merge + then rewriting and squashing them into 1 would work. But I fear it will record 3 parents, not 2 parents. Whatever you do, maybe please push to your private Github clone or our dev repo first and tell us where to look, so we can inspect whether it looks ok. Thanks Uli