jwcolby
jwcolby at colbyconsulting.com
Tue Nov 17 17:04:12 CST 2009
Today I wrote my first C# applet. I have to import CSV files coming back from a processing house into SQL Server. These files had errors in one or more lines in the files. Given that there are tens of millions of lines in each file it isn't possible to just poke around and find the errors. So I wrote an applet that opens the CSV file, reads the header line and counts the " and , characters. Then line by line the applet reads the rest of the lines, comparing the counts to that header line counts. This found lines in the file to be imported where " characters had been dropped, which was causing the import wizard to error out at that point. So read, compare counts, write good lines to output file, write bad lines to an error file. Simple stuff really but it is my first real work done in C#. The speed of this kind of thing is pretty darned good too I thought. I was getting about 300K lines / second read / written. One file had 20.6 million lines, the other had 49 million lines. In the past I would have done this in VBA / Access. Of course it took me a tad longer to write in C# than it would have in VBA. -- John W. Colby www.ColbyConsulting.com