Date: Tue, 17 Apr 2012 18:29:50 GMT From: Isabell Long <isabell@issyl0.co.uk> To: freebsd-gnats-submit@FreeBSD.org Subject: docs/167033: [patch] Add the contents of the wiki page about Subversion vendor imports to the committer's guide. Message-ID: <201204171829.q3HITot9033301@red.freebsd.org> Resent-Message-ID: <201204171830.q3HIU4Io040661@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 167033
>Category: docs
>Synopsis: [patch] Add the contents of the wiki page about Subversion vendor imports to the committer's guide.
>Confidential: no
>Severity: non-critical
>Priority: low
>Responsible: freebsd-doc
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: change-request
>Submitter-Id: current-users
>Arrival-Date: Tue Apr 17 18:30:04 UTC 2012
>Closed-Date:
>Last-Modified:
>Originator: Isabell Long
>Release: 9.0.
>Organization:
n/a
>Environment:
>Description:
The separate page about Subversion vendor imports in the wiki's version of the Subversion Primer was not SGMLised as part of Google Code-In. Attached is a diff to go into the committer's guide with all the other Subversion Primer stuff.
>How-To-Repeat:
>Fix:
See the attached diff.
Patch attached with submission follows:
Index: article.sgml
===================================================================
RCS file: /home/isabell/freebsd/doc-cvs/doc/en_US.ISO8859-1/articles/committers-guide/article.sgml,v
retrieving revision 1.317
diff -u -r1.317 article.sgml
--- article.sgml 3 Apr 2012 12:07:47 -0000 1.317
+++ article.sgml 17 Apr 2012 18:04:54 -0000
@@ -2143,6 +2143,265 @@
</sect3>
<sect3>
+ <title>Vendor imports with <acronym>SVN</acronym></title>
+
+ <important>
+ <para>Please read this entire section before starting a vendor
+ import.</para>
+ </important>
+
+ <note>
+ <para>Patches to vendor code fall into two categories:</para>
+
+ <itemizedlist>
+ <listitem>
+ <para>Vendor patches: these are patches that have been
+ issued by the vendor, or that have been extracted from
+ the vendor's version control system, which address
+ issues which in your opinion can't wait until the next
+ vendor release.</para>
+ </listitem>
+ <listitem>
+ <para>&os; patches: these are patches that modify the
+ vendor code to address &os;-specific issues.</para>
+ </listitem>
+ </itemizedlist>
+
+ <para>The nature of a patch dictates where it should be
+ committed:</para>
+
+ <itemizedlist>
+ <listitem>
+ <para>Vendor patches should be committed to the vendor
+ branch, and merged from there to head. If the patch
+ addresses an issue in a new release that is currently
+ being imported, it <emphasis>must not</emphasis> be
+ committed along with the new release: the release must
+ be imported and tagged first, then the patch can be
+ applied and committed. There is no need to re-tag the
+ vendor sources after committing the patch.</para>
+ </listitem>
+ <listitem>
+ <para>&os; patches should be committed directly to
+ head.</para>
+ </listitem>
+ </itemizedlist>
+ </note>
+
+ <sect4>
+ <title>Preparing the tree</title>
+
+ <para>If importing for the first time after the switch to
+ Subversion, flattening and cleaning up the vendor tree is
+ necessary, as well as bootstrapping the merge history in
+ the main tree.</para>
+
+ <sect5>
+ <title>Flattening</title>
+
+ <para>During the conversion from <acronym>CVS</acronym> to
+ Subversion, vendor branches were imported with the same
+ layout as the main tree. This means that the
+ <literal>pf</literal> vendor sources ended up in
+ <filename>vendor/pf/dist/contrib/pf</filename>. The
+ vendor source is best directly in
+ <filename>vendor/pf/dist</filename>.</para>
+
+ <para>To flatten the <literal>pf</literal> tree:</para>
+
+ <screen>&prompt.user; <userinput>cd <replaceable>vendor/pf/dist/contrib/pf</replaceable></userinput>
+&prompt.user; <userinput>svn mv $(svn list) ../..</userinput>
+&prompt.user; <userinput>cd ../..</userinput>
+&prompt.user; <userinput>svn rm contrib</userinput>
+&prompt.user; <userinput>svn propdel -R svn:mergeinfo .</userinput>
+&prompt.user; <userinput>svn commit</userinput></screen>
+
+ <para>The <literal>propdel</literal> bit is necessary
+ because starting with 1.5, Subversion will automatically
+ add <literal>svn:mergeinfo</literal> to any directory
+ that is copied or moved. In this case, as nothing is
+ being merged from the deleted tree, they just get in the
+ way.</para>
+
+ <para>Tags may be flattened as well (3, 4, 3.5 etc.); the
+ procedure is exactly the same, only changing
+ <literal>dist</literal> to <literal>3.5</literal> or
+ similar, and putting the <command>svn commit</command>
+ off until the end of the process.</para>
+ </sect5>
+ <sect5>
+ <title>Cleaning up</title>
+
+ <para>The <literal>dist</literal> tree can be cleaned up
+ as necessary. Disabling keyword expansion is
+ recommended, as it makes no sense on unmodified vendor
+ code and in some cases it can even be harmful.
+ <application>OpenSSH</application>, for example, includes
+ two files that originated with &os; and still contain the
+ original version tags. To do this:</para>
+
+ <screen>&prompt.user; <userinput>svn propdel svn:keywords -R .</userinput>
+&prompt.root; <userinput>svn commit</userinput></screen>
+ </sect5>
+ <sect5>
+ <title>Bootstrapping merge history</title>
+
+ <para>If importing for the first time after the switch to
+ Subversion, bootstrapping
+ <literal>svn:mergeinfo</literal> on the target directory
+ in the main tree to the to the revision that corresponds
+ to the last related change to the vendor tree, prior to
+ importing new sources:</para>
+
+ <screen>&prompt.user; <userinput>cd <replaceable>head/contrib/pf</replaceable></userinput>
+&prompt.user; <userinput>svn merge --record-only svn+ssh://svn.freebsd.org/base/<replaceable>vendor/pf/dist@180876</replaceable> .</userinput>
+&prompt.user; <userinput>svn commit</userinput></screen>
+ </sect5>
+ </sect4>
+
+ <sect4>
+ <title>Importing new sources</title>
+
+ <para>With two commits—one for the import itself and
+ one for the tag—this step can optionally be repeated
+ for every upstream release between the last import and the
+ current import.</para>
+
+ <sect5>
+ <title>Preparing the vendor sources</title>
+
+ <para>Unlike in <acronym>CVS</acronym> where only the needed
+ parts were imported into the vendor tree to avoid bloating
+ the main tree, Subversion is able to store a full
+ distribution in the vendor tree. So, import everything,
+ but merge only what is required.</para>
+
+ <para>A <command>svn add</command> is required to add any
+ files that were added since the last vendor import, and
+ <command>svn rm</command> is required to remove any that
+ were removed since. Preparing sorted lists of the
+ contents of the vendor tree and of the sources that are
+ about to be imported is recommended, to facilitate the
+ process.</para>
+
+ <screen>&prompt.user; <userinput>cd <replaceable>vendor/pf/dist</replaceable></userinput>
+&prompt.user; <userinput>svn list -R | grep -v '/$' | sort >../old</userinput>
+&prompt.user; <userinput>cd <replaceable>../pf-4.3</replaceable></userinput>
+&prompt.user; <userinput>find . -type f | cut -c 3- | sort >../new</userinput></screen>
+
+ <para>With these two files, <command>comm -23 ../old ../new</command>
+ will list removed files (files only in
+ <filename>old</filename>), while <command>comm -13 ../old ../new</command>
+ will list added files only in <filename>new</filename>.</para>
+ </sect5>
+ <sect5>
+ <title>Importing into the vendor tree</title>
+
+ <para>Now, the sources must be copied into
+ <filename><replaceable>dist</replaceable></filename> and
+ the <command>svn add</command> and
+ <command>svn rm</command> commands should be used as
+ needed:</para>
+
+ <screen>&prompt.user; <userinput>cd <replaceable>vendor/pf/pf-4.3</replaceable></userinput>
+&prompt.user; <userinput>tar cf - . | tar xf - -C ../dist</userinput>
+&prompt.user; <userinput>cd <replaceable>../dist</replaceable></userinput>
+&prompt.user; <userinput>comm -23 ../old ../new | xargs svn rm</userinput>
+&prompt.user; <userinput>comm -13 ../old ../new | xargs svn --parents add</userinput></screen>
+
+ <para>If any directories were removed, they will have to be
+ <command>svn rm</command>ed manually. Nothing will break
+ if they are not, but they will remain in the tree.</para>
+
+ <para>Check properties on any new files. All text files
+ should have <literal>svn:eol-style</literal> set to
+ <literal>native</literal>. All binary files should have
+ <literal>svn:mime-type</literal> set to
+ <literal>application/octet-stream</literal> unless there
+ is a more appropriate media type. Executable files should
+ have <literal>svn:executable</literal> set to
+ <literal>*</literal>. No other properties should exist
+ on any file in the tree.</para>
+
+ <para>Committing is now possible, however it is good
+ practice to make sure that everything is OK by using the
+ <command>svn stat</command> and
+ <command>svn diff</command> commands.</para>
+ </sect5>
+ <sect5>
+ <title>Tagging</title>
+
+ <para>Once committed, vendor releases should be tagged for
+ future reference. The best and quickest way to do this
+ is directly in the repository:</para>
+
+ <screen>&prompt.user; <userinput>svn cp svn+ssh://svn.freebsd.org/base/<replaceable>vendor/pf/dist</replaceable> svn+ssh://svn.freebsd.org/base/<replaceable>vendor/pf/4.3</replaceable></userinput></screen>
+
+ <para>Once that is complete, <command>svn up</command> the
+ working copy of
+ <filename><replaceable>vendor/pf</replaceable></filename>
+ to get the new tag, although this is rarely
+ needed.</para>
+
+ <para>If creating the tag in the working copy of the tree,
+ <command>svn:mergeinfo</command> results must be removed:</para>
+
+ <screen>&prompt.user; <userinput>cd <replaceable>vendor/pf</replaceable></userinput>
+&prompt.user; <userinput>svn cp dist 4.3</userinput>
+&prompt.user; <userinput>svn propdel svn:mergeinfo -R 4.3</userinput></screen>
+ </sect5>
+ </sect4>
+ <sect4>
+ <title>Merging to head</title>
+
+ <screen>&prompt.user; <userinput>cd <replaceable>head/contrib/pf</replaceable></userinput>
+&prompt.user; <userinput>svn up</userinput>
+&prompt.user; <userinput>svn merge --accept=postpone svn+ssh://svn.freebsd.org/base/<replaceable>vendor/pf/dist</replaceable> .</userinput></screen>
+
+ <para>The <literal>--accept=postpone</literal> tells
+ Subversion that it shouldn't complain because merge conflicts
+ will be taken care of manually.</para>
+
+ <para>It is necessary to resolve any merge conflicts.
+ This process is the same in <acronym>SVN</acronym> as in
+ <acronym>CVS</acronym>.</para>
+
+ <para>Make sure that any files that were added or removed in
+ the vendor tree have been properly added or removed in the
+ main tree. To check diffs against the vendor branch:</para>
+
+ <screen>&prompt.user; <userinput>svn diff --no-diff-deleted --old=svn+ssh://svn.freebsd.org/base/<replaceable>vendor/pf/dist</replaceable> --new=.</userinput></screen>
+
+ <para>The <literal>--no-diff-deleted</literal> tells
+ Subversion not to complain about files that are in the
+ vendor tree but not in the main tree, i.e. things that
+ would have previously been removed before the vendor
+ import, like for example the like the vendor's makefiles
+ and configure scripts.</para>
+
+ <para>Using <acronym>CVS</acronym>, once a file was off the
+ vendor branch, it was not able to be put back. With
+ Subversion, there is no concept of on or off the vendor
+ branch. If a file that previously had local
+ modifications, to make it not show up in diffs in the
+ vendor tree, all that has to be done is remove any left-over
+ cruft like &os; version tags, which is much easier.</para>
+
+ <para>If any changes are required for the world to build
+ with the new sources, make them now, and keep testing
+ until everything builds and runs perfectly.</para>
+ </sect4>
+ <sect4>
+ <title>Committing the vendor import</title>
+
+ <para>Committing is now possible! Everything must be
+ committed in one go. If done properly, the tree will move
+ from a consistent state with old code, to a consistent
+ state with new code.</para>
+ </sect4>
+ </sect3>
+
+ <sect3>
<title>Reverting a Commit</title>
<para>Reverting a commit to a previous version is fairly
>Release-Note:
>Audit-Trail:
>Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201204171829.q3HITot9033301>