A few potential customers asked for a detailed comparisons of our VB Migration Partner with Upgrade Wizard, the conversion tool provided for free inside Microsoft Visual Studio, in terms of compilation errors and warnings. Instead of just providing a series of unverifiable numbers we run both tools against a group of open source code samples which offer a variety of challenges, including rarely-used controls, data-binding, graphic methods, and drag-and-drop.  We benchmarked the Upgrade Wizard installed with Microsoft Visual Studio 2005 and the forthcoming release 1.00.06 of VB Migration Partner, on a Intel Core Duo 7800 @ 2.66GHz running Microsoft Windows Vista 64-bit SP1.

The results are available in a Microsoft Excel spreadsheet for all to see.

A summary of the results:

  • VB Migration Partner generates nearly 5x fewer compilation errors than the Upgrade Wizard before adding a single pragma. In most cases just one or two pragmas are sufficient to generate an error-free application.
  • Considering all the pragmas used in all samples delivers we get an average of one pragma every 430 line of code, but in a real application you need even fewer pragmas because a single pragma can affect all the files and statements in the project.
  • The Upgrade Wizard processes about 46 LOCs per second on the average, VB Migration Partner runs at 187 LOCs per second, i.e. 4 times faster. However, these timings are misleading, because at the end of the process VB Migration Partner generates code statistics and reports that the Upgrade Wizard doesn't.
It should be noted that compilation errors and warnings tell only a part of the story, because even a VB.NET project with zero compilation errors might raise one or more runtime errors. This means that even the samples that show few compilation errors after the migration with the Upgrade Wizard (e.g. CodeLibrary or AldInvaders) might require a lot of additional work to run correctly. By comparison, the column labeled as Number of pragmas indicates how many pragmas were neeed to have a fully functional VB.NET application that raises no runtime errors