Shamil Salakhetdinov
shamil at smsconsulting.spb.ru
Tue Sep 22 11:33:02 CDT 2009
John, Yes, that is huge. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Tuesday, September 22, 2009 8:17 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Merge rows Shamil, The three million is ONLY duplicates. The main table is 50 million, and there are actually TWO fifty million record tables, the fact table and the name / address table. These always have to be joined - the "product" is a list of names / addresses that fit the WHERE. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > <<< > However the results should be ... satisfying. ;) > Yes, but for 3+ millions records your current system looks more than enough > from here. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Tuesday, September 22, 2009 6:55 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Merge rows > > >BTW, I really envy you on that "mighty computers you have" subject - with > white envy :)) > > Yes, they are reasonably powerful. They would be more powerful if I could > afford... Top end Intel > processors, real server motherboards with more memory slots, multi-chip > motherboards etc. > > But for what I can afford they do pretty well. > > You ain't seen nothing yet though. I have every intention of creating RAID0 > arrays of FLASH drives > large enough to hold this HSID database. Probably three or four flash > drives of 60 gigs each, raid > zero. Remember that most of the time the actual day to day processing of > this table is read only. > Except for the kind of cleanup we are discussing here, the table does not > change on a month to month > basis. > > If I can get the db out on a multi-spindle RAID0 flash drive array, then the > IOPs will skyrocket, > the data transfer will skyrocket and the normal day to day stuff will REALLY > fly. > > I have read reviews of systems like this and they say it is extremely close > to having the entire > database contained in RAM. > > Right now these flash drives are still quite expensive. 60 gig drives are > $230, so it would cost me > $1000 to do this. However the results should be ... satisfying. ;) > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> <<< >> you lose the data in the first. >> John, >> >> Our postings crossed - I have posted solution for the above issue in my >> recent posting. >> >> <<< >> that there will be 50 million records >> But you say there are "just" 3+ millions when the data are "deduped" - > that >> should be not a big issue for the joins on your mighty computers (BTW, I >> really envy you on that "mighty computers you have" subject - with white >> envy :)) >> >> Thank you. >> >> -- >> Shamil > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4447 (20090922) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4447 (20090922) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4447 (20090922) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4447 (20090922) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru