ClearCase - importer improvements and bug in ClearTool

[tim_west]

Hi all,

ClearCase 2003 and some versions of 7 suffer from this bug. Basically, cleartool cannot format more than 64k of characters per line (using the -fmt parameter). This means that if you have an old repository with many tags, the ClearCase importer fails badly.

Cleartool prints "Error: Supplied buffer too small." to STDERR when this occurs, but the ClearCase Provider does not actually log STDERR output anywhere.

I have not (yet) found a fix for this yet (other than to patch ClearCase so that the underlying problem does not occur), but the attached patch makes it much easier to identify. It improves the ClearCase importer as follows:

• ClearTool's STDERR is logged to log4j. (Previously, it was ignored). This means the ClearTool error message above is printed.
• Improve multi-line history handling. Presently, once one line is broken, all subsequent lines are likely to get munged too, because they are "grouped" incorrectly. My change means that only the lines with actual problems will fail.

Does anyone know if Polarion is monitoring these forums and so picking up these patches, or has the product been abandoned? If so, perhaps a fork is in order...

Cheers,


-Tim West

[Edit: attach the patch]

[tim_west]

Some further notes (on the off-chance that anyone other than me will ever read this):

Two fields regularly blow out the line length and so push lines over the 64k char (I am guessing this is actually 65 536 chars, but I haven't tested). These are the comments field and the labels field. Our repo regularly has 1700+ labels on files.

An ugly way to get around these problems is to alter the formatter (see fmt_ccase) used to generate the history, for example, the following formatter is unlikely to exceed 64k chars:

Code: Select all

    public static final String HISTORY_FORMAT
            = "%Nd;%En;%Vn;%o;!%.1300Cl;!%a;%m;%u;%0.200Nc;\\n".replaceAll(";", HISTORY_FIELD_SEPARATOR);


(You must also alter the parseLabels method to use the above format, since the "..." appended to the label list must be ignored. See the attached patch file).

In the above, the comments field is limited to 200 chars, and only the first 1300 labels are printed. Obviously, the limitation on labels is far from ideal, particularly since there is no way to print the 1300 most recent labels, rather than the oldest 1300.

So, assuming that the above is unacceptable (which it is, in my case) the ways forward this are:

• Upgrade to ClearCase 7 before converting (haha).
• Avoid using the "-fmt" option to lshistory. For example, we could parse the output of "lshistory -long", which does not have the 64k limit. A simple ANTLR or JavaCC parser wouldn't be too hard to knock up...
• Try a commercial tool like this one which might work and if similarly flawed, at least comes with commerical support.

-Tim

[tim_west]

A follow-up for any hapless souls who are attempting CC to SVN conversion and have stumbled upon this thread: I have ditched the Polarion converter for this Perl script. On further investigation, it is clear that migrating the whole repository is suboptimal (for my organisation, at least).


-Tim

[RandyBonnette]

Hi Tim,

I can't open the source with Perl script. The page is not working. Where else can I find it? Thanks.