John W. Colby
jwcolby at colbyconsulting.com
Mon Sep 6 12:34:59 CDT 2004
As I mentioned, I have 4 machines simultaneously importing data into a SQL Server database, using DTS from inside of SQL EM. The raw data files are 10gb each, with 3 million records, 660 fields, comma delimited, text files. Neo2 - The server is using 4 data files on separate 250g SATA drives and a log file on a 250g IDE drive. I have set the Server to expand the database in 1gb chunks, unlimited expansion. NEO2 - The server, A64 3ghz with 2gb ram NEO1 - A workstation, A64 3ghz with 2.5gb ram Soltek1 - A workstation, AMD Athlon 2.5ghz with 750mb ram Marys - A workstation, AMD Athlon 1.2ghz with 384mb ram All the machines except the server (NEO2) have attached the SQL Server on NEO2 to their SQL Server Group. Performance (very rough eyeballed) The server is using 1858340k of RAM (1.8g) and 100% of the processor cycles except when it goes out to expand the database files, when the processor usage drops close to 0. So the server is maxed out. Importing 11k records every 30 seconds. Neo1 - Is using 199044k memory and ~35% of the processor cycles, except when the server expands the data files, when the processor usage drops close to 0. Importing 14K records every 30 seconds. Soltek1 - Is using 148476K of ram and ~60% of processor cycles except during file expansion. Notice this processor only has 750m of physical ram which means it is using the swap file somehow. Importing 14k records in 30 seconds. Marys - is using 330884K of ram and 75% of processing power except during file expansion. Notice this machine has only 380M or ram and is using most of it but NOT loading up the swap file. However I haven't done anything else on this machine either. Importing 10K records every 30 seconds. So I am loading 49K records every 30 seconds across all the machines. All the machines are pulling the raw data off a directory on the server. The next set of imports will be pulling the raw data off directories on Neo1. John W. Colby www.ColbyConsulting.com