[dba-VB] A couple of code snippets...

Hi Hans --

For your question: "What Sorting Algorithm Is Used By LINQ “OrderBy”?" please see
http://stackoverflow.com/questions/2792074/what-sorting-algorithm-is-used-by-linq-orderby


I have experimented a bit more with my coding (see P.S.) and for the following test calls:

string p = "{type tab delimited text file fullpath here}";
TestSort5(System.Console.WriteLine, 1, p, 1);
TestSort5(System.Console.WriteLine, 2, p, 10);
TestSort5(System.Console.WriteLine, 3, p, 100);   

I have got:

*** 1. 21297 products processed in 575.231 ms
*** 2. 212970 products processed in 4540.040 ms
*** 3. 2129700 products processed in 58487.657 ms

Test is run on Win7 3+GB on a simple "mere mortals" dual core laptop (produced by DELL in year 2007).

Sorting 2+ million input lines in 58 secs seems to be too slow(?) - is that input file reading code, which influences so badly on overall performance?

Any comments, remarks and corrections are very welcome.

Thank you.

-- Shamil

P.S. Test code sample:


public void TestSort5(Action<string> log, int testNum, string inputFileFullPath, int readCount)
{
    Stopwatch stopWatch = new Stopwatch();
    stopWatch.Start();

    // print sorted products list
    int counter = 0;
    foreach (var product in (from p in GetProducts5(log, testNum, inputFileFullPath, readCount).OrderBy(x => x.Title) select p))
    {
        if (++counter % 100000 == 0)
        {
            //System.Console.WriteLine("{0}. {1}", counter, product);
        }
    }

    stopWatch.Stop();
    double ms = (stopWatch.ElapsedTicks * 1000.0) / Stopwatch.Frequency;

    log(string.Format("*** {0}. {1} products processed in {2:0.000} ms", testNum, counter, ms));
}

public IEnumerable<dynamic> GetProducts5(Action<string> log, int testNum, string inputFileFullPath, int readCount)
{
    const int SKU_FIELD_INDEX = 3;
    const int TITLE_FIELD_INDEX = 4;

    // parse text string and add parsed item to IEnumerable<dynamic> using yield return
    foreach (string textLine in InputLines(inputFileFullPath, readCount))
    {
        string[] fields = textLine.Split(new char[] { '\t' });
        yield return new { SKU = fields[SKU_FIELD_INDEX], Title = fields[TITLE_FIELD_INDEX] };
    }
}

public IEnumerable<string> InputLines(string inputFileFullPath, int readCount)
{
    string[] lines = new string[] { "" };
    for (int i = 1; i <= readCount; i++)
        foreach (string textLine in System.IO.File.ReadAllLines(inputFileFullPath, Encoding.UTF8))
            yield return textLine;
}



Thu, 26 Apr 2012 20:04:50 -0700 от Hans-Christian Andersen <hans.andersen at phulse.com>:
> What sort of sorting algorithm is .orderby using internally?
> 
> - Hans
> 
> Sent from my iPhone
> 
> On 2012-04-26, at 4:28 PM, Salakhetdinov Shamil <mcp2004 at mail.ru> wrote:
> 
> > Hi All --
> > 
> > Just wanted to share a couple of test code samples I've got written here while working on real life development task.
> > Please review, comment, remark, write your own versions.
> > 
> > Test 1
> > =====
> > 
> > public void TestSort()
> > {
> >    // simulate reading products' .csv file into text string
> >    StringBuilder text = new StringBuilder()
> >        .AppendLine("P1|12.34").AppendLine("P2|5.28").AppendLine("P3|21.74")
> >        .AppendLine("P4|67.12").AppendLine("P5|23.98");
> > 
> >    // parse text string and add parsed items n into an array list
> >    ArrayList products = new ArrayList();
> >    foreach (string textLine in text.ToString().Split(new string[] { System.Environment.NewLine }, StringSplitOptions.RemoveEmptyEntries))
> >    {
> >        products.Add(new { Name = textLine.Substring(0, 2), Price = Decimal.Parse(textLine.Substring(3)) });
> >    }
> > 
> >    // print sorted products list
> >    foreach (var product in (from p in products.Cast<dynamic>().OrderBy(x => x.Price) select p))
> >    {
> >        System.Console.WriteLine(product);
> >    }
> > }
> > 
> > Test 2 (Test 1 variation using yield return)
> > ================================
> > 
> > public void TestSort2()
> > {
> >    // print sorted products list
> >    foreach (var product in (from p in GetProducts().OrderBy(x => x.Price) select p))
> >    {
> >        System.Console.WriteLine(product);
> >    }
> > }
> > 
> > public IEnumerable<dynamic> GetProducts()
> > {
> >    // simulate reading products' .csv file into text string
> >    StringBuilder text = new StringBuilder()
> >        .AppendLine("P1|12.34").AppendLine("P2|5.28").AppendLine("P3|21.74")
> >        .AppendLine("P4|67.12").AppendLine("P5|23.98");
> > 
> >    // parse text string and add parsed item to IEnumerable<dynamic> using yield return
> >    foreach (string textLine in text.ToString().Split(new string[] { System.Environment.NewLine }, StringSplitOptions.RemoveEmptyEntries))
> >    {
> >        yield return new { Name = textLine.Substring(0, 2), Price = Decimal.Parse(textLine.Substring(3)) };
> >    }
> > }
> > 
> > Thank you.
> > 
> > -- Shamil
> > 
> > _______________________________________________
> > dba-VB mailing list
> > dba-VB at databaseadvisors.com
> > http://databaseadvisors.com/mailman/listinfo/dba-vb
> > http://www.databaseadvisors.com
> > 
>