Multiple svn dumps - how to import into SVN

thariqueakbar
Posts: 9
Joined: Tue Oct 19, 2010 1:21 pm

Multiple svn dumps - how to import into SVN

Postby thariqueakbar » Fri May 13, 2011 11:26 am

Hi, I am migrating a huge repo to SVN using Polarion SVNImporter.. I get multiple dump files with approx. size of 900+ mb each say, A.dmp, A.dmp_PART1, A.dmp_PART2.. Although I have kept the dump.file.sizelimit.mb to 5000... Please let me know how to import these multiple dump files into a SVN repo.

************************************************************************************************************************************************************************
****************config.properties************************************************************************************************************************************


# The source repository provider,
# either cvs,cvsrcs,pvcs,mks,cc,vss or st
srcprovider=cvs


###########################################################################
# Import dump settings
#
# import_dump_into_svn - If enabled then after dump creation,
# it will be imported into svn repository via svnadmin
# (you must write proper svn autoimport options)
# If svn repository is not exist it will be created.
#
# existing-svnrepos - if enables and if import enabled by previous
# option then dump will be imported into svn repository ONLY IF REPOSITORY EXISTS
#
# clear_svn_parent_dir - if enabled and import enabled then application will delete all
# existing records in svn's parent directory before import dump to it.
# (affect for full dump only)
#

import_dump_into_svn=yes

existing_svnrepos=no

clear_svn_parent_dir=no

# Option enables that feature:
# "Now the importer always import whole history(with possibilities like trunk-only etc.).
# It should be possible to import only current state as one revision. It is useful for
# the incremental import. In current situation we will never be able to import whole because
# of the size of the dump file."
# But see the new parameter "dump.file.sizelimit.mb" below.
#
# AFFECT FOR FULL DUMP CREATION ONLY!
use_only_last_revision_content=no

# VM systems often allow to set a description on a versioned file (one for all revisions).
# svnimporter is able to migrate it to a svn property. Since there is no predefined
# property key for this purpose in subversion, you can configure it here. If you do
# not give a property key name, the file description will not be migrated.
# NOTE: migration of properties is presently implemented only for PVCS
#
file_description_property_key=description

# If use_file_copy is set to yes, svnimporter uses SVN file copy operations for tags and
# branches. This raises the quality of the import dramatically. On the other hand,
# it works reliably for one-shot imports only. DO NOT SET THIS TO YES if you
# want to make incremental imports to synchronize repositories; otherwise the resulting
# dump files may not be importable to SVN.
#
# If set to no, every branch and tag operation is implemented as a simple file add operation.
# The origin of the tag or branch from the trunk is not recorded.
#
# Presently, this flag is evaluated only by the import from PVCS, CVS, MKS, ClearCase.
#
use_file_copy=yes

#####################################################################################
# FILE SETTINGS
#
# full.dump.file - file name pattern for saving full dump
# incr.dump.file - file name pattern for saving incremental dump
# incr.history.file - file for saving history for incremental dump
# list.files.to - destination file for saving scm's files list
# dump.file.sizelimit.mb - rough maximum dump file size in MegaBytes. See below.

#####################################################################################
full.dump.file=oecvs.dmp
incr.dump.file=incr_dump_%date%.txt
incr.history.file=incr_history.txt
list.files.to=files_%date%.txt

# svnimporter checks the size of the current dump file before dump of each revision. If
# the size (in Megabytes) exceeds this limit, a new dump file is created. For large
# source repositories and/or small size limits, a run of svnimporter will generate a
# sequence of dump files. Their actual sizes will be slightly larger than the limit
# specified here.
#
# Take care not to set the limit too small. The dump file names are distinguished
# by their date part only which has a resolution of one second. Producing one dump file
# should therefore take longer than one second.
#
# Set the value to 0 to switch off this feature.

dump.file.sizelimit.mb=5000

#######################################################################################
# SVN DUMP OPTIONS
#
# trunk_path - location of "trunk" folder. Can be "." if "only_trunk" option is enabled
# branches_path - location of "branches" folder
# tags_path - location of "tags" path
# svnimporter_user_name - name of service user, which create first revision etc.
# only_trunk - if enabled then convert only trunk of repository (skip tags and branches)
#######################################################################################
trunk_path=trunk
branches_path=branches
tags_path=tags
#svnimporter_user_name=SvnImporter
svnimporter_user_name=e1016588
only_trunk=no


#######################################################################################
# SVN AUTOIMPORT OPTIONS
#
# svnadmin.executable - path to svnadmin executable
# svnadmin.repository_path - path to svn repository
# svnadmin.parent_dir - parent dir in svn repository for importing dump (must be created manually)
# svnadmin.tempdir - temp directory for svnadmin
# svnclient.executable - path to svn executable
# svnadmin.import_timeout - The value is length of time to wait in milliseconds,
# if this parameter is set and "svnadmin load" did not finished after specified length of time
# then it's process will be killed and svnimporter execution will be aborted.
# svnadmin.path_not_exist_signature - when importer checks repository path for existing
# it compares output of "svn ls" command with given string. If given string not found
# in command output and command return code is not null then importer cannot determine
# path exist or not, then exception will be thrown.
# If you runs importer not in English locale and log file contains similar as following error:
# EXCEPTION CAUGHT: org.polarion.svnimporter.svnprovider.SvnAdminException:
error during execution 'svn ls' command: svn: URL 'file:///c:/tmp/ImportFromCvs/zzzzz' existiert nicht in dieser Revision
# then you should change signature to "existiert nicht in dieser Revision"
#######################################################################################
svnadmin.executable=C:/Program Files (x86)/CollabNet/Subversion Server/svnadmin.exe
#svnadmin.repository_path=C:/svn_repository/cheena
svnadmin.repository_path=C:/svn_repository/oecvs
svnadmin.parent_dir=.
svnadmin.tempdir=c:/temp/local-oecvs
svnclient.executable=C:/Program Files (x86)/CollabNet/Subversion Server/svn.exe
svnadmin.verbose_exec=yes
#svnadmin.import_timeout=1800000

svnadmin.path_not_exist_signature=non-existent in that revision
#svnadmin.path_not_exist_signature=existiert nicht in dieser Revision

#################################################################################
########################## CVS PROVIDER CONFIGURATION ###########################
#################################################################################

cvs.class=org.polarion.svnimporter.cvsprovider.CvsProvider
cvs.username=d03215
cvs.password=cvspassword02
cvs.hostname=uxap225d.corp.efunds.com
cvs.repository=/repository/oecvs

# 1) Module name can contains filename in tail (ex.: 'test4/dir/file')
# then dump will be built for single file
# 2) If modulename=* (or .) then all modules from the repository will be converted
cvs.modulename=.

cvs.tempdir=c:/temp/local-oecvs


# some repositories uses not standart date format for
# 'log' command. (standart is yyyy/MM/dd HH:mm:ss),
# in that case you must adjust this option to actual value
# For example if you receive error
# 'java.lang.Exception: Couldn't parse date 2003/07/21 13:19:09'
# you must change this option to
#cvs.logdateformat=yyyy-MM-dd HH:mm:ss

#################################################################################
########################## CVSRCS PROVIDER CONFIGURATION ########################
#################################################################################
# This provider works with local CVS repository using RCS tools "rlog" and "co".
# Requirements:
# * You must have CVS repository on your local file system
# * You must have RCS installed
#
# Configuration:
# cvsrcs.repository_path - path to CVS repository
# cvsrcs.rlog_command - path to RCS "rlog" command (can be full path to executable,
# or name of command if it accessible via PATH)
# cvsrcs.co_command - path to RCS "co" command
# cvsrcs.logdateformat - see cvs.logdateformat
# cvsrcs.tempdir - path to temp directory

#cvsrcs.class=org.polarion.svnimporter.cvsprovider.CvsRcsProvider
#cvsrcs.repository_path=/var/cvs/test-file-copy
#cvsrcs.rlog_command=rlog
#cvsrcs.co_command=co
#cvsrcs.tempdir=/tmp
#cvsrcs.logdateformat=yyyy-MM-dd HH:mm:ss

#################################################################################
########################## PVCS PROVIDER CONFIGURATION ##########################
#################################################################################

#pvcs.class=org.polarion.svnimporter.pvcsprovider.PvcsProvider
#pvcs.executable=C:/Program Files (x86)/PVCS/vm/win32/bin/pcli.exe
#pvcs.projectpath=C:/PVCSLocal/Projects/qualifile
#pvcs.subproject=
#pvcs.tempdir=c:/temp/local-qualifile
#pvcs.log.dateformat=MMM dd yyyy HH:mm:ss
#pvcs.log.datelocale=en
#pvcs.log.encoding=Cp1251
#pvcs.log.datetimezone=Europe/Berlin
#pvcs.verbose_exec=yes
#pvcs.username=d12599
#pvcs.username=e1016588
#pvcs.password=

# If you set keep_vlogfile to "yes", and there already exist vlog.tmp and files.tmp
# files in the tempdir from a previous run of svnimporter, these files will not
# be regenerated. This is useful in some special situations, for example when you
# want to make sure you import the same state of the PVCS archive as before.
#
# pvcs.keep_vlogfile=yes

# If import_attributes=yes, svnimporter will try to map the PVCS archive attributes
# to SVN properties. However, it is usually better to use the auto-props feature of Subversion
# to set properties during the import.
#
#pvcs.import_attributes=yes

# The bug was occasionally observed during import of big projects
# (many files/revisions) and heavy PVCS load (probably).
# This bug is actually caused by invalid behaviour of pcli.exe get command,
# which sometimes returns invalid file content (wich less than 0.1% probability)
# if pvcs.validate_checkouts set to "yes" then importer will detect this bug
# and try to fix it.
# It may slow down import process because each checkout will be performed twice (at least)

#pvcs.validate_checkouts=no

#################################################################################
########################## VSS PROVIDER CONFIGURATION ###########################
#################################################################################
#vss.class=org.polarion.svnimporter.vssprovider.VssProvider
#vss.executable=C:\\Program Files\\Microsoft Visual Studio\\VSS\\win32\\ss.exe
#vss.path=\\\\gigaltsov\\vss
#vss.project=$/test2
#vss.username=test
#vss.password=test
#vss.tempdir=c:/temp/local
#vss.log.dateformat=dd.MM.yy HH:mm
#vss.log.datelocale=en
#vss.log.encoding=Cp1251
# if enabled - dump output of vss.executable command to stdout
#vss.verbose_exec=yes


#################################################################################
########################## CLEAR CASE PROVIDER CONFIGURATION ####################
#################################################################################
#cc.class=org.polarion.svnimporter.ccprovider.CCProvider
#cc.executable=cleartool.exe
#cc.projectpath=p:/ccview/sources/test
#cc.log.dateformat=yyyyMMdd.HHmmss
#cc.log.encoding=Cp1251
#cc.tempdir=c:/temp/local
#cc.verbose_exec=yes

#################################################################################
########################## MKS PROVIDER CONFIGURATION ###########################
#################################################################################
#mks.class=org.polarion.svnimporter.mksprovider.MksProvider
#mks.executable=C:/Program Files/MKS/IntegrityClient/bin/si.exe
#mks.project=c:/temp/mks_project/project.pj
#mks.log.dateformat=MMM dd, yyyy - hh:mm aa
#mks.log.datelocale=ru
#mks.log.encoding=ibm866
#mks.tempdir=c:/temp/local
#mks.verbose_exec=yes

#################################################################################
########################## STAR TEAM PROVIDER CONFIGURATION #####################
#################################################################################
#st.class=org.polarion.svnimporter.stprovider.STProvider
### NOTE: The Star Team provider requires the external jar file to access the
### repository. The file has name starteam80.jar and can be found in the Star
### Team installation folder. Copy this jar into the importer lib folder, before
### using the Star Team provider.

#st.url=username:password@server:port/Project/View/Folder
#st.url=username:password@server:port/Project/View
#st.url=username:password@server:port/Project
#st.tempdir=D:/temp/starteam/temp

# you can specify an include and/or exclude regular expression that will be
# matched against the view relative path of an item
# (hint: the path is relative and uses "/" as the path separator)
#st.includes.regex=
#st.excludes.regex

# a property file for translating user names (format: logonname=StarTeam Full Name)
# Note: this us useful to translate verbose StarTeam names to Subversion logon names (without spaces)
# if you set a file here each unhandled user name will be replaced with svnimporter_user_name)
# default value: not set (which means no conversion)
#st.usermappings=usermappings.properties

# can be set to completely skip lables
#st.ignorelables=yes

# should we try separating commits using linked change requests?
# Note: Not all StarTeam projects are suitable for this option.
# this option is likly to create overlapping commits; an exception
# is thrown in the case that this overlapping will break the subversion
# dump. You are requested to deactivate this option in that case.
# default value: no
#st.separatecommitsusingcrs=yes

# a properties file to translate cr numbers (eg. 242=JIRA-142)
# default value: not set (which means no conversion)
#st.crmappings=crmappings.properties

# can be set to automatically attach a list of linked crs to a commit message
# Note: whether or not this is enabled, the list of linked CRs is always set ad
# a svn property; this property just attaches it to the commit message
# default value: no
#st.attachlinkedcrs=yes

# the time span in seconds that should be considered as one single commit
# Note: must be from the same author
# Note2: It might happen that the author changed the same file multiple
# times within the specified timespan. In this case, all changes will appear
# as only ONE single commit and only ONE single change. If you don't want
# this behavior you should not set checkintimespan or set it to 0.
# A message is logged with severity INFO in such cases. Thus, you can check your
# log files afterwards.
# Note3: This can also create overlapping commits. Example:
# st.checkintimespan=300
# 15:01 userA starts committing
# 15:02 userB starts committing
# 15:04 userA finishes committing
# (time when userB finishes committing doesn't matter)
# This maybe because StarTeam was slow ;). Anyway, the importer checks this case
# and issues a WARNING message. It then further checks that the following
# (overlapping) commit does not conflict with the commit. If a conflict
# is detected the conversion process is aborted and you are requested to
# take further actions (eg. reduce checkintimespan).

#st.checkintimespan=300



#################################################################################
########################## LOG4J CONFIGURATION ##################################
#################################################################################
log4j.rootLogger=DEBUG, file

#log4j.logger.cz=DEBUG, file
#log4j.logger.cz=DEBUG, stdout

log4j.logger.historyLogger=DEBUG, historyFile

log4j.appender.file=org.apache.log4j.FileAppender
log4j.appender.file.File=svnimporter.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{HH:mm:ss,SSS} [%t] %5p %c{1}:%L - %m%n

log4j.appender.historyFile=org.apache.log4j.FileAppender
log4j.appender.historyFile.File=history.log
log4j.appender.historyFile.layout=org.apache.log4j.PatternLayout
log4j.appender.historyFile.layout.ConversionPattern=%m%n

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%c{1}:%L %5p - %m%n


#################################################################################
############################### DEBUG OPTIONS ###################################
#################################################################################
#
# do not delete provider's temporary files
#

disable_cleanup=yes

kencorbin
Posts: 197
Joined: Fri Nov 16, 2007 10:30 pm
Location: Corvallis, OR

Re: Multiple svn dumps - how to import into SVN

Postby kencorbin » Mon May 16, 2011 9:29 pm

You load the dump files into an svn repository using the svnadmin program. You have to look up the syntax, from memory it is something like

svnadmin import [repository] < [dumpfile]

It is very important that the dump files be loaded in the correct sequence.
Good luck,
-Ken

kencorbin
Posts: 197
Joined: Fri Nov 16, 2007 10:30 pm
Location: Corvallis, OR

Re: Multiple svn dumps - how to import into SVN

Postby kencorbin » Mon May 16, 2011 9:35 pm

PS. The command is "load", not "import"

thariqueakbar
Posts: 9
Joined: Tue Oct 19, 2010 1:21 pm

Re: Multiple svn dumps - how to import into SVN

Postby thariqueakbar » Mon May 23, 2011 2:18 pm

I agree, but it is for loading/importing one dump file.. I get multiple dump files.. say, abc.dmp_PART1 , abc.dmp_PART2.. I am not sure how do I import them into the SVN.

kencorbin
Posts: 197
Joined: Fri Nov 16, 2007 10:30 pm
Location: Corvallis, OR

Re: Multiple svn dumps - how to import into SVN

Postby kencorbin » Tue Jun 14, 2011 4:24 pm

Just load them one at a time, in the correct order.

Good luck,
-Ken


Return to “Polarion SVN Importer (Repository Converter)”

Who is online

Users browsing this forum: No registered users and 6 guests