Move Subversion repository without svnadmin

November 12, 2008

Update (2008-11-13)
Okay, so I did actually end up finding a way to move from SF.net for real. It seems they provide read-only rsync access straight to the repository directory. So here is what I did instead.

$ rsync -av blowpass.svn.sourceforge.net::svn/blowpass/* blowpass
$ svnadmin dump blowpass/ > blowpass.dump
$ svnadmin create clearpass
$ svnadmin load clearpass < blowpass.dump

Update (2008-11-12)
Made a small performance change then ran it on the ClearPass repository. Worked flawlessly.

I have been having a terrible time trying to figure out how I can get the Subversion repository for ClearPass out of SourceForge. I could not find a single reference to svnadmin on the SourceForge and no examples of exporting without it. So I took matters into my own hands. Below is a quick and dirty shell script that exports and imports a repository one revision at a time, using common Linux command line tools and the svn command. I'm going to do more testing before using it for real, but so far it has done well. Hope this helps someone else in my position.

Download it: svncrossload

#!/bin/sh

################################################################################
# LICENSE
################################################################################
# Copyright 2008 John Hobbs
################################################################################
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the
# Free Software Foundation, Inc.,
# 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
################################################################################

################################################################################
# ABOUT
################################################################################
#
# Home: http://www.velvetcache.org/
#
# This is a script to cross load subversion repositories (kind of) keeping history
# intact without access to svnadmin.  Import into a FRESH repository only, and
# be sure to do a comprehensive diff at the end.
#
# Also be sure to do this in an empty directory. Temp files get added and removed
# without sincere thought put into them.
#
# Log messages get eaten and re-inserted as shown below. Edit to taste.
#   $ svn log -r 1
#   ------------------------------------------------------------------------
#   r1 | jmhobbs | 2008-11-12 18:19:43 -0600 (Wed, 12 Nov 2008) | 7 lines
#
#   Imported from file:///srv/svn/scs using svncrossload
#
#     |r1 | jmhobbs | 2008-10-27 17:32:44 -0500 (Mon, 27 Oct 2008) | 2 lines
#     |
#     |Initial import.
#     |
#
#   ------------------------------------------------------------------------
#   $

echo "Checking out initial revisions"
svn co $2 importing > /dev/null
svn co -r 0 $1 updateme > /dev/null

echo "Getting most recent revision number"
LATESTREVISION=$(svn info $1 | grep Revision | sed 's/^Revision: *\([0-9]*\)/\1/')

for i in $(seq 1 $LATESTREVISION); do

  echo -e "\nCopying revision $i"

  cd updateme
  svn update -r $i > ../_update
  echo -e "Imported from $1 using svncrossload\n" > ../_log
  # The '\-\-\-\-\...' looks ridiculous, but it works.
  svn log -r $i | grep -v '\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-' | sed 's/\(.*\)/   |\1/'  >> ../_log
  cd ..

  cat _update | grep -E '^A' | sed 's/^A *//' > _update_add
  cat _update | grep -E '^D' | sed 's/^D *//' > _update_del
  cat _update | grep -E '^U' | sed 's/^U *//' > _update_mod

  echo "$(wc -l _update_add | sed 's/^\([0-9]*\).*/\1/') Files To Add"
  echo "$(wc -l _update_mod | sed 's/^\([0-9]*\).*/\1/') Files To Modify"
  echo "$(wc -l _update_del | sed 's/^\([0-9]*\).*/\1/') Files To Delete"

  # Copy
  for j in $(cat _update_add | tr ' ' '@'); do
    if [ -d "updateme/${j//@/ }" ]; then
      mkdir "importing/${j//@/ }"
    else
      cp -f "updateme/${j//@/ }" "importing/${j//@/ }"
    fi
    cd importing
    # We send cerr to null because it warns when we add existing stuff
    svn add "${j//@/ }" 2> /dev/null
    cd ..
  done

  # Modify
  for j in $(cat _update_mod | tr ' ' '@'); do
    if [ -f "updateme/${j//@/ }" ]; then
      cp -f "updateme/${j//@/ }" "importing/${j//@/ }"
    fi
  done

  # Delete
  for j in $(cat _update_del | tr ' ' '@'); do
    cd importing
    svn rm "${j//@/ }"
    cd ..
  done

  echo "Committing"
  cd importing
  svn commit -F ../_log
  cd ..

done

echo "Cleaning up"
rm -rf importing _log _update _update_add _update_del updateme _update_mod
echo "Done!"

Comments

  1. Tom Boutell says:

    Thanks for writing this, it’s very clever even if you did find a way to avoid using it in the long run. It’s the first ‘copy a repository without svnadmin’ script I’ve seen that seems likely to actually do the job.

    I wrote ‘svnforeigncopy’ which doesn’t attempt to copy history, just ignores and externals. It’s good as a way of copying projects with those crucial details intact without copying the entire history, which makes sense if you’re copying a sample project to your own repo to start your own project (something we encourage people to do with our work):

    http://sourceforge.net/projects/svnforeigncopy/

    (Would your version handle externals, ignores, etc. correctly? Looks like it probably wouldn’t, I don’t see anything about handling properties here…)

  2. john says:

    Thanks Tom, it’s been a while since I’ve looked at this, seems brutish to me now, but it did the job.

    I’ve always been an amateur SVN user, so I don’t mess much with props beyond setting execute.

    Thanks for the link, hopefully someone will find that useful if they land here. I’ve never really done PHP on the command line, but that’s pretty cool for only 61 lines. If it works, it works, who cares about language wars :-)

  3. Hey, John. I know it’s an old post, but your tool was very useful at work. I’ve improved it and published it into my blog (http://bubble-tales.blogspot.com) and into github (http://github.com/bolhoso).

    Thank you!

  4. john says:

    Awesome! I love it when old code finds a new use and new life! Thanks for extending it and for sharing!

Leave A Comment

Your email will not be published.