jwcolby
jwcolby at colbyconsulting.com
Sat Apr 26 14:25:56 CDT 2008
I have been setting up an application to read a block of records, perhaps 10K or something. I designed a class that will load a block of records which consist of a name to be parsed, and a set of 6 fields which will be filled in with the parsed data returned from the name parser. Thus as I load the data only the name to parse is filled in. The supervisor iterates this set of records, pulling the name to parse, passing it off to the name parser, then reading the data back from the name parser and updating the fields in the same record. Move to the next record, wash, rinse repeat. I do have a couple of questions though. I am not very familiar with how the ADO.net stuff works. I build a connection string I build a connection object which I leave not connected (for now), but feeding it the connection string. I build a sql statement that represents the fields to pull WHERE PK between StartValue and EndValue. I build a command object which I pass SQL statement and the still unconnected connection object. I build a data adapter which I feed the command object. Notice that we are still not connected to the data store. I build a DataTable object with a table name. I call DataAdapter(DataTable).Fill Does .Net then connect correctly and proceed to fill the data table? Once I have the data table filled I start to iterate the DataTable using the for each DataRow iterator I then call the name splitter passing in the name to be split. Back comes the name pieces. I fill in the row with the name pieces and move to the next row. Once every row has been filled in I need to call an update method of the DataAdapter which is SUPPOSED to write the changes back to the table in SQL Server..... The name splitter handles ~1K names / second, NOT as speedy as I had hoped. At this point I haven't a clue how fast the data adapter will write back to the table in SQL Server. What I am trying to do is write this thing such that each instance of the data class can then be handed off to a thread so that the write back to SQL Server can begin and I can instantiate another data class to load / parse the next "chunk" of data in parallel with the write to SQL Server. I am trying to discover whether I need to close connections to the database while I do the parsing or just not worry about it. Should I just leave all the objects open and close them all down as the data class closes once the write to SQL Server is done? How will I know the write is done? Does the code "hang" at the DataAdapter.Update until the entire update (all the rows) is finished? Am I going about this the right way? Should I become a used car salesman? -- John W. Colby www.ColbyConsulting.com