Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 17 Apr 2012 18:29:50 GMT
From:      Isabell Long <isabell@issyl0.co.uk>
To:        freebsd-gnats-submit@FreeBSD.org
Subject:   docs/167033: [patch] Add the contents of the wiki page about Subversion vendor imports to the committer's guide.
Message-ID:  <201204171829.q3HITot9033301@red.freebsd.org>
Resent-Message-ID: <201204171830.q3HIU4Io040661@freefall.freebsd.org>

next in thread | raw e-mail | index | archive | help

>Number:         167033
>Category:       docs
>Synopsis:       [patch] Add the contents of the wiki page about Subversion vendor imports to the committer's guide.
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-doc
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Tue Apr 17 18:30:04 UTC 2012
>Closed-Date:
>Last-Modified:
>Originator:     Isabell Long
>Release:        9.0.
>Organization:
n/a
>Environment:
>Description:
The separate page about Subversion vendor imports in the wiki's version of the Subversion Primer was not SGMLised as part of Google Code-In.  Attached is a diff to go into the committer's guide with all the other Subversion Primer stuff.
>How-To-Repeat:

>Fix:
See the attached diff.

Patch attached with submission follows:

Index: article.sgml
===================================================================
RCS file: /home/isabell/freebsd/doc-cvs/doc/en_US.ISO8859-1/articles/committers-guide/article.sgml,v
retrieving revision 1.317
diff -u -r1.317 article.sgml
--- article.sgml	3 Apr 2012 12:07:47 -0000	1.317
+++ article.sgml	17 Apr 2012 18:04:54 -0000
@@ -2143,6 +2143,265 @@
       </sect3>
 
       <sect3>
+	<title>Vendor imports with <acronym>SVN</acronym></title>
+
+	<important>
+	  <para>Please read this entire section before starting a vendor
+	    import.</para>
+	</important>
+
+	<note>
+	  <para>Patches to vendor code fall into two categories:</para>
+
+	  <itemizedlist>
+	    <listitem>
+	      <para>Vendor patches: these are patches that have been
+		issued by the vendor, or that have been extracted from
+		the vendor's version control system, which address
+		issues which in your opinion can't wait until the next
+		vendor release.</para>
+	    </listitem>
+	    <listitem>
+	      <para>&os; patches: these are patches that modify the
+		vendor code to address &os;-specific issues.</para>
+	    </listitem>
+	  </itemizedlist>
+
+	  <para>The nature of a patch dictates where it should be
+	    committed:</para>
+
+	  <itemizedlist>
+	    <listitem>
+	      <para>Vendor patches should be committed to the vendor
+		branch, and merged from there to head.  If the patch
+		addresses an issue in a new release that is currently
+		being imported, it <emphasis>must not</emphasis> be
+		committed along with the new release: the release must
+		be imported and tagged first, then the patch can be
+		applied and committed.  There is no need to re-tag the
+		vendor sources after committing the patch.</para>
+	    </listitem>
+	    <listitem>
+	      <para>&os; patches should be committed directly to
+		head.</para>
+	    </listitem>
+	  </itemizedlist>
+	</note>
+
+	<sect4>
+	  <title>Preparing the tree</title>
+
+	  <para>If importing for the first time after the switch to
+	    Subversion, flattening and cleaning up the vendor tree is
+	    necessary, as well as bootstrapping the merge history in
+	    the main tree.</para>
+
+	  <sect5>
+	    <title>Flattening</title>
+
+	    <para>During the conversion from <acronym>CVS</acronym> to
+	      Subversion, vendor branches were imported with the same
+	      layout as the main tree.  This means that the
+	      <literal>pf</literal> vendor sources ended up in
+	      <filename>vendor/pf/dist/contrib/pf</filename>.  The
+	      vendor source is best directly in
+	      <filename>vendor/pf/dist</filename>.</para>
+
+	    <para>To flatten the <literal>pf</literal> tree:</para>
+
+	    <screen>&prompt.user; <userinput>cd <replaceable>vendor/pf/dist/contrib/pf</replaceable></userinput>
+&prompt.user; <userinput>svn mv $(svn list) ../..</userinput>
+&prompt.user; <userinput>cd ../..</userinput>
+&prompt.user; <userinput>svn rm contrib</userinput>
+&prompt.user; <userinput>svn propdel -R svn:mergeinfo .</userinput>
+&prompt.user; <userinput>svn commit</userinput></screen>
+
+	    <para>The <literal>propdel</literal> bit is necessary
+	      because starting with 1.5, Subversion will automatically
+	      add <literal>svn:mergeinfo</literal> to any directory
+	      that is copied or moved.  In this case, as nothing is
+	      being merged from the deleted tree, they just get in the
+	      way.</para>
+
+	    <para>Tags may be flattened as well (3, 4, 3.5 etc.); the
+	      procedure is exactly the same, only changing
+	      <literal>dist</literal> to <literal>3.5</literal> or
+	      similar, and putting the <command>svn commit</command>
+	      off until the end of the process.</para>
+	  </sect5>
+	  <sect5>
+	    <title>Cleaning up</title>
+
+	    <para>The <literal>dist</literal> tree can be cleaned up
+	      as necessary.  Disabling keyword expansion is
+	      recommended, as it makes no sense on unmodified vendor
+	      code and in some cases it can even be harmful.
+	      <application>OpenSSH</application>, for example, includes
+	      two files that originated with &os; and still contain the
+	      original version tags.  To do this:</para>
+
+	    <screen>&prompt.user; <userinput>svn propdel svn:keywords -R .</userinput>
+&prompt.root; <userinput>svn commit</userinput></screen>
+	  </sect5>
+	  <sect5>
+	    <title>Bootstrapping merge history</title>
+
+	    <para>If importing for the first time after the switch to
+	      Subversion, bootstrapping
+	      <literal>svn:mergeinfo</literal> on the target directory
+	      in the main tree to the to the revision that corresponds
+	      to the last related change to the vendor tree, prior to
+	      importing new sources:</para>
+
+	    <screen>&prompt.user; <userinput>cd <replaceable>head/contrib/pf</replaceable></userinput>
+&prompt.user; <userinput>svn merge --record-only svn+ssh://svn.freebsd.org/base/<replaceable>vendor/pf/dist@180876</replaceable> .</userinput>
+&prompt.user; <userinput>svn commit</userinput></screen>
+	  </sect5>
+	</sect4>
+
+	<sect4>
+	  <title>Importing new sources</title>
+
+	  <para>With two commits&mdash;one for the import itself and
+	    one for the tag&mdash;this step can optionally be repeated
+	    for every upstream release between the last import and the
+	    current import.</para>
+
+	  <sect5>
+	    <title>Preparing the vendor sources</title>
+
+	    <para>Unlike in <acronym>CVS</acronym> where only the needed
+	      parts were imported into the vendor tree to avoid bloating
+	      the main tree, Subversion is able to store a full
+	      distribution in the vendor tree.  So, import everything,
+	      but merge only what is required.</para>
+
+	    <para>A <command>svn add</command> is required to add any
+	      files that were added since the last vendor import, and
+	      <command>svn rm</command> is required to remove any that
+	      were removed since.  Preparing sorted lists of the
+	      contents of the vendor tree and of the sources that are
+	      about to be imported is recommended, to facilitate the
+	      process.</para>
+
+	    <screen>&prompt.user; <userinput>cd <replaceable>vendor/pf/dist</replaceable></userinput>
+&prompt.user; <userinput>svn list -R | grep -v '/$' | sort >../old</userinput>
+&prompt.user; <userinput>cd <replaceable>../pf-4.3</replaceable></userinput>
+&prompt.user; <userinput>find . -type f | cut -c 3- | sort >../new</userinput></screen>
+
+	    <para>With these two files, <command>comm -23 ../old ../new</command>
+	      will list removed files (files only in
+	      <filename>old</filename>), while <command>comm -13 ../old ../new</command>
+	      will list added files only in <filename>new</filename>.</para>
+	  </sect5>
+	  <sect5>
+	    <title>Importing into the vendor tree</title>
+
+	    <para>Now, the sources must be copied into
+	      <filename><replaceable>dist</replaceable></filename> and
+	      the <command>svn add</command> and
+	      <command>svn rm</command> commands should be used as
+	      needed:</para>
+
+	    <screen>&prompt.user; <userinput>cd <replaceable>vendor/pf/pf-4.3</replaceable></userinput>
+&prompt.user; <userinput>tar cf - . | tar xf - -C ../dist</userinput>
+&prompt.user; <userinput>cd <replaceable>../dist</replaceable></userinput>
+&prompt.user; <userinput>comm -23 ../old ../new | xargs svn rm</userinput>
+&prompt.user; <userinput>comm -13 ../old ../new | xargs svn --parents add</userinput></screen>
+
+	    <para>If any directories were removed, they will have to be
+	      <command>svn rm</command>ed manually.  Nothing will break
+	      if they are not, but they will remain in the tree.</para>
+
+	    <para>Check properties on any new files. All text files
+	      should have <literal>svn:eol-style</literal> set to
+	      <literal>native</literal>.  All binary files should have
+	      <literal>svn:mime-type</literal> set to
+	      <literal>application/octet-stream</literal> unless there
+	      is a more appropriate media type.  Executable files should
+	      have <literal>svn:executable</literal> set to
+	      <literal>*</literal>.  No other properties should exist
+	      on any file in the tree.</para>
+
+	    <para>Committing is now possible, however it is good
+	      practice to make sure that everything is OK by using the
+	      <command>svn stat</command> and
+	      <command>svn diff</command> commands.</para>
+	  </sect5>
+	  <sect5>
+	    <title>Tagging</title>
+
+	    <para>Once committed, vendor releases should be tagged for
+	      future reference.  The best and quickest way to do this
+	      is directly in the repository:</para>
+
+	    <screen>&prompt.user; <userinput>svn cp svn+ssh://svn.freebsd.org/base/<replaceable>vendor/pf/dist</replaceable> svn+ssh://svn.freebsd.org/base/<replaceable>vendor/pf/4.3</replaceable></userinput></screen>
+
+	    <para>Once that is complete, <command>svn up</command> the
+	      working copy of
+	      <filename><replaceable>vendor/pf</replaceable></filename>
+	      to get the new tag, although this is rarely
+	      needed.</para>
+
+	    <para>If creating the tag in the working copy of the tree,
+	      <command>svn:mergeinfo</command> results must be removed:</para>
+
+	    <screen>&prompt.user; <userinput>cd	<replaceable>vendor/pf</replaceable></userinput>
+&prompt.user; <userinput>svn cp dist 4.3</userinput>
+&prompt.user; <userinput>svn propdel svn:mergeinfo -R 4.3</userinput></screen>
+	  </sect5>
+	</sect4>
+	<sect4>
+	  <title>Merging to head</title>
+
+	  <screen>&prompt.user; <userinput>cd <replaceable>head/contrib/pf</replaceable></userinput>
+&prompt.user; <userinput>svn up</userinput>
+&prompt.user; <userinput>svn merge --accept=postpone svn+ssh://svn.freebsd.org/base/<replaceable>vendor/pf/dist</replaceable> .</userinput></screen>
+
+	  <para>The <literal>--accept=postpone</literal> tells
+	    Subversion that it shouldn't complain because merge conflicts
+	    will be taken care of manually.</para>
+
+	  <para>It is necessary to resolve any merge conflicts.
+	    This process is the same in <acronym>SVN</acronym> as in
+	    <acronym>CVS</acronym>.</para>
+
+	  <para>Make sure that any files that were added or removed in
+	    the vendor tree have been properly added or removed in the
+	    main tree.  To check diffs against the vendor branch:</para>
+
+	  <screen>&prompt.user; <userinput>svn diff --no-diff-deleted --old=svn+ssh://svn.freebsd.org/base/<replaceable>vendor/pf/dist</replaceable> --new=.</userinput></screen>
+
+	  <para>The <literal>--no-diff-deleted</literal> tells
+	    Subversion not to complain about files that are in the
+	    vendor tree but not in the main tree, i.e. things that
+	    would have previously been removed before the vendor
+	    import, like for example the like the vendor's makefiles
+	    and configure scripts.</para>
+
+	  <para>Using <acronym>CVS</acronym>, once a file was off the
+	    vendor branch, it was not able to be put back.  With
+	    Subversion, there is no concept of on or off the vendor
+	    branch.  If a file that previously had local
+	    modifications, to make it not show up in diffs in the
+	    vendor tree, all that has to be done is remove any left-over
+	    cruft like &os; version tags, which is much easier.</para>
+
+	  <para>If any changes are required for the world to build
+	    with the new sources, make them now, and keep testing
+	    until everything builds and runs perfectly.</para>
+	</sect4>
+	<sect4>
+	  <title>Committing the vendor import</title>
+
+	  <para>Committing is now possible!  Everything must be
+	    committed in one go.  If done properly, the tree will move
+	    from a consistent state with old code, to a consistent
+	    state with new code.</para>
+	</sect4>
+      </sect3>
+
+      <sect3>
 	<title>Reverting a Commit</title>
 
 	<para>Reverting a commit to a previous version is fairly


>Release-Note:
>Audit-Trail:
>Unformatted:



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201204171829.q3HITot9033301>