[dba-VB] Merge rows

But do you think that the "brute force crunching" will gain more time than is lost in requiring 
SQL Server to pulling all 50 million records from the database, rather than letting SQL 
Server only pull the few thousand that  you are interested in in the first place?

-- 
Stuart


On 23 Sep 2009 at 10:54, Shamil  Salakhetdinov wrote:

> Hi Stuart,
> 
> No problem. "BitArrays population magic" isn't a complicated task for John
> "DB from Hell".
> 
> BTW, I have just run test code for 50 millions 400 bits bit arrays for OR,
> AND and XOR bitwise operations - it takes less than 3 seconds, and for "DB
> from Hell" with 50 million records it will be 50 millions * 400 = 20 billion
> bits = ~2GB - all that can be loaded into memory for 64bits PCs, so, I do
> expect that even "brute force crunching" will produce a kind of
> instantaneous results for the part of query where char "Y/N" columns are
> currently used. And once loaded in memory the info can be kept/updated there
> (and in DB)(and back-up in 2GB+ binary file) as it's not changing that much
> IOW maybe there is no even a need to substitute in "DB from Hell" current
> char "Y/N" fields with Binary(...) or VarBinary(...) fields... etc... 
> 
> Please correct me if you find I'm wrong somewhere in my
> considerations/expectations/evaluations...
> 
> Thank you.
> 
> --
> Shamil
> 
> P.S.
> 
> static void test3()
> {
>     //const long CYCLES_QTY = 1000000000; // 1 billion
>     const long CYCLES_QTY = 50000000; // 50 millions
>     bool[] values1 = (bool[])Array.CreateInstance(typeof(bool), 400);
>     bool[] values2 = (bool[])Array.CreateInstance(typeof(bool), 400);
>     values1[0] = true;
>     values1[225] = true;
>     values1[399] = true;
> 
> 
>     System.Collections.BitArray bitTest1 =
>         new System.Collections.BitArray(values1);
>     System.Collections.BitArray bitTest2 =
>         new System.Collections.BitArray(values2);
> 
>     for (int i = 0; i < 400; i++) bitTest2[i] = true; 
> 
>     Console.WriteLine("{0:HH:mm:ss.fff}", DateTime.Now);
>     for (long l = 1; l < CYCLES_QTY; l++)
>         bitTest1.Xor(bitTest2);
>         //bitTest1.Or(bitTest2);
>         //bitTest1.And(bitTest2);
>     Console.WriteLine("{0:HH:mm:ss.fff}", DateTime.Now);
> 
>     foreach (bool bit in bitTest1)
>         Console.Write("{0}", bit ? "1" : "0");
> 
>     Console.WriteLine();
> 
>     // 50 millions - AND
>     //10:32:56.772
>     //10:32:59.548
>  
> //10000000000000000000000000000000000000000000000000000000000000000000000000
> 000000
>  
> //00000000000000000000000000000000000000000000000000000000000000000000000000
> 000000
>  
> //00000000000000000000000000000000000000000000000000000000000000000100000000
> 000000
>  
> //00000000000000000000000000000000000000000000000000000000000000000000000000
> 000000
>  
> //00000000000000000000000000000000000000000000000000000000000000000000000000
> 000001
> 
>     // 50 millions - OR
>     //10:34:27.644
>     //10:34:30.451
>  
> //11111111111111111111111111111111111111111111111111111111111111111111111111
> 111111
>  
> //11111111111111111111111111111111111111111111111111111111111111111111111111
> 111111
>  
> //11111111111111111111111111111111111111111111111111111111111111111111111111
> 111111
>  
> //11111111111111111111111111111111111111111111111111111111111111111111111111
> 111111
>  
> //11111111111111111111111111111111111111111111111111111111111111111111111111
> 111111
> 
>     // 50 millions - XOR
>     //10:35:27.414
>     //10:35:30.182
>  
> //01111111111111111111111111111111111111111111111111111111111111111111111111
> 111111
>  
> //11111111111111111111111111111111111111111111111111111111111111111111111111
> 111111
>  
> //11111111111111111111111111111111111111111111111111111111111111111011111111
> 111111
>  
> //11111111111111111111111111111111111111111111111111111111111111111111111111
> 111111
>  
> //11111111111111111111111111111111111111111111111111111111111111111111111111
> 111110
>     
>     Console.WriteLine();
>     foreach (bool bit in bitTest2)
>         Console.Write("{0}", bit ? "1" : "0");
> 
>     Console.WriteLine();
> }
> 
> 
> 
> -----Original Message-----
> From: dba-vb-bounces at databaseadvisors.com
> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart McLachlan
> Sent: Wednesday, September 23, 2009 10:19 AM
> To: Discussion concerning Visual Basic and related programming issues.
> Subject: Re: [dba-VB] Merge rows
> 
> Sorry, 
> 
> I was under the impression that we were talking about selecting rows from
> JC's "Database 
> >From Hell", not just manipulating bit arrays which we have somehow
> magically populated 
> without pulling data from SQL Server.
> 
> 
> -- 
> Stuart  
> 
> 
> 
>  
> 
> __________ Information from ESET NOD32 Antivirus, version of virus signature
> database 4448 (20090922) __________
> 
> The message was checked by ESET NOD32 Antivirus.
> 
> http://www.esetnod32.ru
>  
> 
> _______________________________________________
> dba-VB mailing list
> dba-VB at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-vb
> http://www.databaseadvisors.com
>