jwcolby
jwcolby at colbyconsulting.com
Mon May 14 08:06:40 CDT 2007
Gustav, >So you are developing on a 2003 running SQL Server? Yes. This machine is the one I am doing the data transforms on, transforming the dozens of flat file fixed width files into pipe delimited "csv" files. This SQL Server box is the one that then pulls the "CSV" files in to SQL Server using the BULK INSERT, so I was just developing the transform application right on that machine. >That must be slow for you, the user. No, not particularly. It is a dual core AMD x64 3.8GHZ with 4 gb ram, a pretty speedy machine. >Did you check the setting I mentioned? I did, but haven't run the VB.Net program since. I really think that it is more a matter of threading. I am moving to using a worker thread to run the transform process. If I can figure it out that thread will then feed back a progress count to the main form to display in a progress bar. >That said, if you need two boxes for SQL Server, you should add a third for yourself. LOL, right you are. I own 5 different machines. Two are the dual core machines with 4gb ram and terrabyte raid arrays, both run Windows 2003 Standard. Those are slated for use with these big databases being pulled in from the raw files. I already have one database that started as 65 million records / 700+ fields, whittled down to around 50 million records after validating the address. I just finished importing a 97 million record table with 149 fields in it. That is in and I am prepping it for export back out for the address validation software, running on a older machine. I have a third database on the way which I am told is about 80 million records, number of fields unknown. That will have to run through my data transform app and be imported, then exported out again for address validation. So that is what these two "big" machines are for. I have two older machines that are "smaller" AMD single core X64 processors with 2 gb ram. One of them is used for the address validation software mentioned above. The other ATM is just sitting. And then there is my Laptop which is what I normally use for development, email, browsing, remote client access etc. ATM that laptop is suffering heat problems and shutting down occasionally so I am moving all of my critical stuff over to one of the desktops temporarily until that issue is resolved. I am actually considering just buying a new laptop. The old one is 3 years old and the new Intel Core 2 duo laptop chips are now available. I think I can get an HP for about $1600 with 2gb RAM and a large drive. UNFORTUNATELY it comes with Vista which I am not looking forward to. Anyway, I have plenty of machines. I am doing the development on the SQL Server machine simply because that is where the resulting software is going to run, though it doesn't have to. It would be interesting to see how fast the transforms happen on the older single core machines. I have a gigabit switch between all these machines so if it runs fast on one of the older machines I could dedicate the one that is just sitting there to running that transform application. There is going to be plenty of work to do to go around. Of course my power bill goes up every time I turn on another machine. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: accessd-bounces at databaseadvisors.com [mailto:accessd-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Monday, May 14, 2007 8:31 AM To: accessd at databaseadvisors.com Subject: Re: [AccessD] VB.Net - seeing the messagebox Hi John So you are developing on a 2003 running SQL Server? That must be slow for you, the user. Did you check the setting I mentioned? However, if the box is optimized for SQL Server, you should leave it to prioritize background tasks. That said, if you need two boxes for SQL Server, you should add a third for yourself. /gustav