Tractive: migrating from Trac to GitHub

1. Ruby

2. Git

3. reposurgeon, which works on Linux (manual compile recommended for the latest versions) and macOS (use Homebrew)

4. trac-admin must be installed in order to extract Trac attachments

Install it via:

1. A GitHub Account as the importer

1. Migrate your Trac-managed SVN repositories to GitHub. (Migrating SVN to GitHub using reposurgeon)

2. Build the SVN Revision to Git Commit RevMap. (Building the Trac Revision to Git Commit RevMap)

3. Bootstrap your Tractive config file. (Boostrapping the Tractive configuration file)

4. Configure mapping in your Tractive config file. (Working with the Tractive configuration file, and mapping)

5. Run Tractive (Running Tractive)

   $ tractive <options>

reposurgeon is the leading tool today (and under active development) that allows conversion among multiple version control systems.

We strongly recommend you to use the excellent reposurgeon for this task.

In the case of Trac to GitHub, we are mostly looking at SVN to Git.

Typically these are the steps to take:

1. In an empty directory, run repotool --initialize {name-of-repo}. Enter that the source VCS is svn, the destination is git. The directory will then be populated with a number of files, including:

   • Makefile includes multiple useful tasks, requires user configuration.

   • {name-of-repo}.lift, a file to store reposurgeon conversion configuration.

   • {name-of-repo}.map, a reposurgeon specific map of SVN users to GitHub users/emails)

   • {name-of-repo}.opts, to enter pre-read reposurgeon options.

2. Customize these lines of the Makefile with the SVN URL and reposurgeon "read options":

   REMOTE_URL = https://svn.ietf.org/svn/tools/mailarch
   READ_OPTIONS = --branchify=trunk:branch/*:personal/*/*:tags/*:*

3. Update the {name-of-repo}.lift with additional reposurgeon commands (see the reference guide) if necessary. e.g.

   branch rename refs/heads/master refs/heads/main
   
   msgin --create <<EOF
   Tag-Name: git-conversion
   Tagger: IETF Bot <noreply@ietf.org> America/Los_Angeles
   
   Marks the spot where this repository was migrated from Subversion to git.
   EOF

4. Download the SVN repository in mirror mode to the {name-of-repo}-mirror/ directory. This is necessary for faster processing and so that reposurgeon won’t attempt to download from the REMOTE_URL.

5. Generate the user map from the SVN repository with make stubmap. Edit the {name-of-repo}.map to correct/fix it if necessary.

6. Run make to generate {name-of-repo}.svn and {name-of-repo}.fo. The resulting Git repository will be at {name-of-repo}-git/.

7. The {name-of-repo}.fo file is a SVN Revision to timestamp mapping. Tractive uses this file to build the SVN Revision to Git Commit SHA hash mapping.

RevMap is a mapping file that contains a one-to-many mapping of an SVN revision number to its corresponding Git commit SHA hash(es).

With the {name-of-repo}.fo file generated by reposurgeon (or any other tool for that matter, as long as the format is identical), we can build the SVN revision to Git commit SHA "RevMap".

A typical {name-of-repo}.fo file looks like this:

The left column is the SVN revision, while the right column provides a timestamp for which you can find with Git. If you enter {name-of-repo}-git/, you can use the git scommit and git scommits commands (defined in the reposurgeon guide) to view the corresponding Git commit(s).

Notice that:

• typically there is a single timestamp per SVN Revision. In this case each timestamp can be identified with one Git commit via git scommit {timestamp}.

• sometimes two SVN revisions have identical timestamps. In this case you will need to use git scommits {timestamp} to see all these commits.

• sometimes one SVN revision maps to multiple Git commits. In this case you will need to use git scommits {timestamp} to see all these commits.

With this information you can now build the RevMap with:

The generated RevMap will be then used in a Tractive run so that the migrated issues and commits will replace references to SVN revisions with the corresponding Git commit SHA, enabling GitHub to expose those linkages in the user interface.

The configuration file contains the following sections.

Thereafter just invoke tractive to read from the default config.yaml path:

By default, tractive assumes the file config.yaml to be in the same directory as the command was run.

You can also specify the configuration file on the command line:

Add the -o flag to only import the tickets that are not in a closed status:

If you want all Trac comments and changes within one Trac ticket to be compiled into a single issue at GitHub:

To resume the migration at a given Trac ticket ID, use -s:

You may want to ensure that the imported Trac ticket IDs are identical to the GitHub Issues ID. This section applies when migrating to an existing or empty GitHub repository,

Tractive can help you create dummy tickets (and close them) for IDs missing in Trac (because they were deleted). This works even if you run it multiple times.

As the process can be interrupted, you can always specify the first ID number to migrate. In this case you need to provide the -s argument for the first ID not available in Github.

By default, Tractive will verify that the created issue numbers match the ticket IDs of the corresponding Trac ticket and error-exit if the numbers do not match.

The fast import option allows you to disable this safe-checking behavior.

In order to utilize this feature, you should also disable user interactions by setting Limit to repository collaborators under your repository settings. Alternatively, when migrating issues to a new repository, import the issues on a test-repository and rename the repository to the final name when the import went satisfactory.

You can disable this check by using the fast option:

In effect your import will be much faster since there is no ID synchronization (but after the script has finished, it can still take some time until the issues are created on github).

If you know that the ticket IDs will not match, e.g. there are existing GitHub issues that are not created by import, using the fast import option is obligatory. In this case, you must specify the ID of the first Trac ticket to be migrated (even if it is 1):

You need the following for wiki migration

• Clone of the github wiki repo.

Then you can run the migration with:

After that open a terminal in the git wiki folder and run git push

Tractive uses the GitHub Issue Import API to create Issues.

While this API is not available via GitHub’s official API bindings (e.g. Octokit), it offers several advantages over the normal Issue creation API:

• will not trigger abuse detection warnings and will not get blocked

• does not send out email notifications on issue changes

• does not increase your contribution count (especially if you attempt multiple import tries)

• faster than with the GitHub v3 Issues API

• allows setting the correct creation/closed date

• creates atomic changes without allowing users to interfere in the creation of a single issue and its comments.

The caveat is that there is still no way of migrating an issue or comment attributing to a third-party user account without using that user’s GitHub account, but this is likely a security concern that will not be addressed by GitHub.