svn-importer 1.1-M8: CVSProvider leaving empty files in SVN

jkgentle
Posts: 1
Joined: Fri Aug 17, 2007 2:49 pm

svn-importer 1.1-M8: CVSProvider leaving empty files in SVN

Postby jkgentle » Fri Aug 17, 2007 3:02 pm

I'm attempting to convert a CVSNT repository to SVN for a client. Having had trouble with configurations that included direct import into SVN (failed with Java exception), "use_file_copy=yes" (failed on svnadmin load due to a deleted file being referenced in tags/branches) I finally ran several CVS modules through successfully with "use_file_copy=no".

However, on a recent migration, I'm finding files that have zero length (empty files) in SVN that correspond to files that were deleted in CVSNT. Each of the problem files existed in multiple branches/tags in CVSNT and appear in each of those branches/tags in SVN as an empty file.

An export from CVSNT for one of the tags in question compared with an export from SVN shows that the content is identical - except for these mystery zero-length files.

The configuration file follows - note that the values between the "{}" are substituted at runtime for each CVS module.

Code: Select all

# Provider Settings
srcprovider=cvs

# Import Options
import_dump_into_svn=no
existing_svn_repo=yes
clear_svn_parent_dir=no
use_only_last_revision_content=no

# File Options
file_description_property_key=description
use_file_copy=no
full.dump.file={working-dir}/full_dump_{module-name}.dumpfile
incr.dump.file={working-dir}/incr_dump_{module-name}.dumpfile
incr.history.file={working-dir}/incr_history_{module-name}.txt
list.files.to={working-dir}/files_{module-name}.txt

# SVN dump options
#dump.file.sizelimit.mb=1000
dump.file.sizelimit.mb=0
trunk_path=trunk
branches_path=branches
tags_path=tags
svnimporter_user_name=SvnImporter
only_trunk=no

# SVN load options
svnadmin.executable={svn-home}/svnadmin.exe
svnadmin.repository_path={svn-repos-path}
svnadmin.parent_dir={svn-parent-dir}/{module-name}
svnadmin.tempdir={svn-tempdir}
svnclient.executable={svn-home}/svn.exe
svnadmin.verbose_exec=yes
svnadmin.path_not_exist_signature=non-existent in that revision


# CVS Options
cvs.class=org.polarion.svnimporter.cvsprovider.CvsProvider
cvs.username={cvs-username}
cvs.password={cvs-password}
cvs.hostname={cvs-hostname}
cvs.repository={cvs-repository}
cvs.modulename={module-name}
cvs.tempdir=c:/temp/local

# Logging Options
log4j.rootLogger=DEBUG, file

log4j.logger.historyLogger=DEBUG, historyFile

log4j.appender.file=org.apache.log4j.FileAppender
log4j.appender.file.File={working-dir}/svnimporter-nba-{module-name}.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{HH:mm:ss,SSS} [%t] %5p %c{1}:%L - %m%n

log4j.appender.historyFile=org.apache.log4j.FileAppender
log4j.appender.historyFile.File={working-dir}/history-nba-{module-name}.log
log4j.appender.historyFile.layout=org.apache.log4j.PatternLayout
log4j.appender.historyFile.layout.ConversionPattern=%m%n

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%c{1}:%L %5p - %m%n

# Debug options
disable_cleanup=no

gowen
Posts: 2
Joined: Wed Nov 14, 2007 6:43 pm

Postby gowen » Wed Nov 14, 2007 7:03 pm

I recently encountered the same problems, but I believe I know what is causing them.


The blank files are caused when a file is created a branch in CVSNT and not on HEAD. When this happens CVSNT adds a "dead" version with no content on the head branch. SVN Importer doesn't check for this and will create an empty file on HEAD.


The "File not found" problems seem to be caused by some quirks in CVSNT. In my repository certain files although deleted (ie "dead") will still have branches coming from them. So for example, a file was created at version 1.1, deleted in version 1.2; but a branch is still created at version 1.2. Even though this file doesn't exist on the branch (trying to check it out will give a CVS error) the branch is still created.

SVN Importer will create three revisions for this file: a ADD revision for 1.1, a DELETE for 1.2 and then a COPY for the branch. Since the copy happens after the delete you end up with the "File not found" error.

If anyone is interested (there doesn't appear to be any recent development) I can post a patch here if/when I get it working with these two scenarios

gowen
Posts: 2
Joined: Wed Nov 14, 2007 6:43 pm

[PATCH]

Postby gowen » Tue Nov 20, 2007 7:42 pm

The first patch below should fix the problem with both the deleted file being referenced in a copy and the problem with empty files being left.

However, I have noticed that it can still leave empty directories if all the files have been deleted in a particular directory. But I can live with this for my import, so I haven't tried to fix this. It also leaves some empty revisions in the dump file (ie a revision with no associated actions) but Subversion appears to handle this okay on a load.


The second patch, fixes a problem I had exporting directly from the "cvsrcs" files. It appears my version of CVSNT (2.5.0.3) completely ignored the current directory with when checking out the rcs file, so I change the importer to look in the module directory rather than the temp directory. This really should probably be done via checking out to stdout and then streaming this to the temp directory, but I was under a deadline and needed to get this working quickly. Maybe someone will find this useful and fix it properly.

Code: Select all

Index: org.polarion.svnimporter.cvsprovider/src/org/polarion/svnimporter/cvsprovider/internal/model/CvsBranch.java
===================================================================
--- org.polarion.svnimporter.cvsprovider/src/org/polarion/svnimporter/cvsprovider/internal/model/CvsBranch.java   (revision 17660)
+++ org.polarion.svnimporter.cvsprovider/src/org/polarion/svnimporter/cvsprovider/internal/model/CvsBranch.java   (working copy)
@@ -79,10 +79,23 @@
             throw new CvsModelException("unknown rcs state: " + rcsState);

          }

          if (prev == null) {

-            if(isTrunk())

-               rev.setState(CvsRevisionState.ADD);

-            else

-               rev.setState(CvsRevisionState.CHANGE);

+            if(isTrunk()) {

+               // if this is "dead" don't create this revision (assume DELETE)

+               if ("dead".equals(rcsState)) {

+                  rev.setState(CvsRevisionState.DELETE);

+               } else {

+                  rev.setState(CvsRevisionState.ADD);

+               }

+            } else {

+               // if sprout revision was "dead" there wasn't

+               // a copy/add, therefore this is a new file

+               CvsRevision sprout = (CvsRevision) getSproutRevision();

+               if (sprout == null || "dead".equals(sprout.getRcsState())) {                  

+                  rev.setState(CvsRevisionState.ADD);

+               } else {

+                  rev.setState(CvsRevisionState.CHANGE);

+               }

+            }

          } else {

             if ("dead".equals(rev.getRcsState())) {

                if (prev.getState() == CvsRevisionState.ADD || prev.getState() == CvsRevisionState.CHANGE) {

Index: org.polarion.svnimporter.cvsprovider/src/org/polarion/svnimporter/cvsprovider/internal/model/CvsModel.java
===================================================================
--- org.polarion.svnimporter.cvsprovider/src/org/polarion/svnimporter/cvsprovider/internal/model/CvsModel.java   (revision 17660)
+++ org.polarion.svnimporter.cvsprovider/src/org/polarion/svnimporter/cvsprovider/internal/model/CvsModel.java   (working copy)
@@ -111,6 +111,7 @@
                      CvsBranch branch = new CvsBranch(branchNumber);

                      cvsFile.addBranch(branch);

                      revision.addChildBranch(branch);

+                     branch.setSproutRevision(revision);

                   }

                }

             }

@@ -137,6 +138,7 @@
                      branch = new CvsBranch(branchNumber);

                      cvsFile.addBranch(branch);

                      sproutRevision.addChildBranch(branch);

+                     branch.setSproutRevision(sproutRevision);

                   }

                   branch.setName(name);

                } else { // tag

Index: org.polarion.svnimporter.cvsprovider/src/org/polarion/svnimporter/cvsprovider/internal/CvsTransform.java
===================================================================
--- org.polarion.svnimporter.cvsprovider/src/org/polarion/svnimporter/cvsprovider/internal/CvsTransform.java   (revision 17660)
+++ org.polarion.svnimporter.cvsprovider/src/org/polarion/svnimporter/cvsprovider/internal/CvsTransform.java   (working copy)
@@ -104,21 +104,24 @@
          CvsRevision revision = (CvsRevision) i.next();

          transformRevision(revision);

 

-         if (!isOnlyTrunk()) {

-            //--- record branches ----

-            for (Iterator b = revision.getChildBranches().iterator(); b.hasNext();) {

-               CvsBranch childBranch = (CvsBranch) b.next();

-               if (!childBranches.containsKey((childBranch.getName())))

-                  childBranches.put(childBranch.getName(), new ArrayList());

-               ((Collection) childBranches.get(childBranch.getName())).add(revision);

+         // Don't record tags or branches if this revision is removed

+         if (revision.getState() != CvsRevisionState.DELETE) {

+            if (!isOnlyTrunk()) {

+               //--- record branches ----

+               for (Iterator b = revision.getChildBranches().iterator(); b.hasNext();) {

+                  CvsBranch childBranch = (CvsBranch) b.next();

+                  if (!childBranches.containsKey((childBranch.getName())))

+                     childBranches.put(childBranch.getName(), new ArrayList());

+                  ((Collection) childBranches.get(childBranch.getName())).add(revision);

+               }

+               //--- record tags ---

+               for (Iterator t = revision.getTags().iterator(); t.hasNext();) {

+                  CvsTag childTag = (CvsTag) t.next();

+                  if (!childTags.containsKey(childTag.getName()))

+                     childTags.put(childTag.getName(), new ArrayList());

+                  ((Collection) childTags.get(childTag.getName())).add(revision);

+               }

             }

-            //--- record tags ---

-            for (Iterator t = revision.getTags().iterator(); t.hasNext();) {

-               CvsTag childTag = (CvsTag) t.next();

-               if (!childTags.containsKey(childTag.getName()))

-                  childTags.put(childTag.getName(), new ArrayList());

-               ((Collection) childTags.get(childTag.getName())).add(revision);

-            }

          }

       }

       if (!isOnlyTrunk()) {

@@ -194,6 +197,13 @@
       if (isOnlyTrunk() && !revision.getBranch().isTrunk())

          return;

 

+      // skip creation of empty initial revisions on trunk

+      if (revision.getBranch().isTrunk() &&

+            revision.isFirstRevision() &&

+            revision.getState() == CvsRevisionState.DELETE) {

+         return;

+      }

+         

         SvnProperties p = new SvnProperties();

         p.set(CVS_REVISION_NUMBER, revision.getNumber());



Patch 2 - CVSNT and "cvsrcs"

Code: Select all

Index: org.polarion.svnimporter.cvsprovider/src/org/polarion/svnimporter/cvsprovider/CvsRcsProvider.java
===================================================================
--- org.polarion.svnimporter.cvsprovider/src/org/polarion/svnimporter/cvsprovider/CvsRcsProvider.java   (revision 17660)
+++ org.polarion.svnimporter.cvsprovider/src/org/polarion/svnimporter/cvsprovider/CvsRcsProvider.java   (working copy)
@@ -129,7 +129,7 @@
 
         String path = revision.getModelFile().getPath();
         File rcsFile = (File) path2rcsfile.get(path);
-        File tempFile = new File(tempDir, rcsFile.getName().replaceAll(",v$", ""));
+        File tempFile = new File(rcsFile.getParentFile(), rcsFile.getName().replaceAll(",v$", ""));
         if (tempFile.exists()) tempFile.delete();
 
         Exec exec = new Exec(new String[]{
@@ -139,6 +139,10 @@
         exec.setWorkdir(tempDir);
         exec.setVerboseExec(true);
         exec.exec();
+       
+        if (!tempFile.exists()) {
+           return null;
+        }
         return tempFile;
     }

bryceman
Posts: 2
Joined: Tue May 06, 2008 2:42 pm

Re: svn-importer 1.1-M8: CVSProvider leaving empty files in SVN

Postby bryceman » Fri Sep 05, 2008 3:28 pm

I needed this patch as well - it really should get into a new snapshot release. It was a pain for this java newbie to get set up to use it.

bard
Posts: 1
Joined: Mon Sep 08, 2008 7:41 am

Re: svn-importer 1.1-M8: CVSProvider leaving empty files in SVN

Postby bard » Mon Sep 08, 2008 8:45 am

Have similar issue with files left empty on branch after deletion on the same branch when import used with use_file_copy=no
Even after applayin patch there are empty files. Please, help.

jhaynes
Posts: 3
Joined: Mon Mar 31, 2008 5:07 pm
Location: Colorado
Contact:

Re: svn-importer 1.1-M8: CVSProvider leaving empty files in SVN

Postby jhaynes » Tue Oct 07, 2008 7:34 pm

Hello,

I'm importing from CVSNT to SVN and have been experiencing the same problem with branches being created from the same revision were files were deleted. I implemented the patch and now I am getting "Uknown branch" errors. Anyone else encounter this?

Thanks in advance,

Josh

somanianands
Posts: 6
Joined: Wed May 09, 2012 1:35 pm

Re: svn-importer 1.1-M8: CVSProvider leaving empty files in

Postby somanianands » Wed May 09, 2012 1:43 pm

I'm attempting to convert a CVSNT repository to SVN for a client.

I'm finding files that have zero length (empty files) in SVN that correspond to files that were deleted in CVSNT. Each of the problem files existed in multiple branches/tags in CVSNT and appear in each of those branches/tags in SVN as an empty file.

Is any way to resolve this empty files issue?

somanianands
Posts: 6
Joined: Wed May 09, 2012 1:35 pm

Re: svn-importer 1.1-M8: CVSProvider leaving empty files in

Postby somanianands » Wed May 09, 2012 1:49 pm

I'm attempting to convert a CVSNT repository to SVN for a client.

I'm finding files that have zero length (empty files) in SVN that correspond to files that were deleted in CVSNT. Each of the problem files existed in multiple branches/tags in CVSNT and appear in each of those branches/tags in SVN as an empty file.

Is any way to resolve this empty files issue?

Anand


Return to “Polarion SVN Importer (Repository Converter)”

Who is online

Users browsing this forum: No registered users and 6 guests