[dba-VB] Merge rows

Shamil,

The three million is ONLY duplicates.  The main table is 50 million, and there are actually TWO 
fifty million record tables, the fact table and the name / address table.  These always have to be 
joined - the "product" is a list of names / addresses that fit the WHERE.

John W. Colby
www.ColbyConsulting.com


Shamil Salakhetdinov wrote:
> <<<
> However the results should be ... satisfying.  ;)
> Yes, but for 3+ millions records your current system looks more than enough
> from here.
> 
> --
> Shamil
> 
> -----Original Message-----
> From: dba-vb-bounces at databaseadvisors.com
> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby
> Sent: Tuesday, September 22, 2009 6:55 PM
> To: Discussion concerning Visual Basic and related programming issues.
> Subject: Re: [dba-VB] Merge rows
> 
>  >BTW, I really envy you on that "mighty computers you have" subject - with
> white envy :))
> 
> Yes, they are reasonably powerful.  They would be more powerful if I could
> afford...  Top end Intel 
> processors, real server motherboards with more memory slots, multi-chip
> motherboards etc.
> 
> But for what I can afford they do pretty well.
> 
> You ain't seen nothing yet though.  I have every intention of creating RAID0
> arrays of FLASH drives 
> large enough to hold this HSID database.  Probably three or four flash
> drives of 60 gigs each, raid 
> zero.  Remember that most of the time the actual day to day processing of
> this table is read only. 
> Except for the kind of cleanup we are discussing here, the table does not
> change on a month to month 
> basis.
> 
> If I can get the db out on a multi-spindle RAID0 flash drive array, then the
> IOPs will skyrocket, 
> the data transfer will skyrocket and the normal day to day stuff will REALLY
> fly.
> 
> I have read reviews of systems like this and they say it is extremely close
> to having the entire 
> database contained in RAM.
> 
> Right now these flash drives are still quite expensive.  60 gig drives are
> $230, so it would cost me 
> $1000 to do this.  However the results should be ... satisfying.  ;)
> 
> John W. Colby
> www.ColbyConsulting.com
> 
> 
> Shamil Salakhetdinov wrote:
>> <<<
>> you lose the data in the first.
>> John,
>>
>> Our postings crossed - I have posted solution for the above issue in my
>> recent posting.
>>
>> <<<
>> that there will be 50 million records
>> But you say there are "just" 3+ millions when the data are "deduped" -
> that
>> should be not a big issue for the joins on your mighty computers (BTW, I
>> really envy you on that "mighty computers you have" subject - with white
>> envy :))
>>
>> Thank you.
>>
>> --
>> Shamil
> 
> _______________________________________________
> dba-VB mailing list
> dba-VB at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-vb
> http://www.databaseadvisors.com
> 
> 
> __________ Information from ESET NOD32 Antivirus, version of virus signature
> database 4447 (20090922) __________
> 
> The message was checked by ESET NOD32 Antivirus.
> 
> http://www.esetnod32.ru
> 
> 
>  
> 
> __________ Information from ESET NOD32 Antivirus, version of virus signature
> database 4447 (20090922) __________
> 
> The message was checked by ESET NOD32 Antivirus.
> 
> http://www.esetnod32.ru
>  
> 
> _______________________________________________
> dba-VB mailing list
> dba-VB at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-vb
> http://www.databaseadvisors.com
> 
>