[dba-VB] Thread safe list

John  --

<<<
I saw something about locking the object, then doing a deep copy and using
the copy.  Have you ever done this?
>>>
Yes.
You can serialize/deserialize whole collections.
No need to program imperatively such serialization/deserialization usually.
Just Xml attributes and tiny generic serialize/deserialize methods will do
all the tricks.
It would be lighting fast for your collections having under hundred objects.
Folks tend to use JSON for serialization/deserialization these days.
But I haven't used it yet.

Thank you.

--
Shamil
 
-----Original Message-----
From: dba-vb-bounces at databaseadvisors.com
[mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby
Sent: 3 ??????? 2010 ?. 23:16
To: Discussion concerning Visual Basic and related programming issues.
Subject: Re: [dba-VB] Thread safe list

I saw something about locking the object, then doing a deep copy and using
the copy.  Have you ever done this?

My collections are always well under a hundred objects and the objects are
just pointers (and the
key) so it seems like a deep copy of the dictionary should be lightning
fast.  Then I could use the ForEach on the copy and all would be right in
the world.

 > (Advanced) thread-safe multi-threading is "another song" - a way more
complicated than ordinary programming.

We have moved to using threads for every time intensive task, but also using
events to signal that we are finished with a time intensive task.

For example we:

1) Build a database (fast) and then build a table, populate it with (tens
of) millions of records and build a bunch of indexes (SLOW).  The table
build can take minutes to complete.  We run that whole process on a thread
and the very end of the process raises an event.  The event sets a flag and
throws a status object up the chain for display on the form, then exits (the
thread).

2) A thread then creates a chunk table, pulls a set of up to two million
records into a table, creates indexes etc (slow - up to a minute or more)
and then raises an event which sets a flag and throws a status object up the
chain for display on the form.

3) A thread then BCPs out that chunk table (slow - up to a minute or more)
and raises an event which sets a flag, and throws a status object up the
chain for display on the form.

Steps 2 and 3 repeat until all records are BCPd to files on disk.

This all happens in a custom database class which has a collection of
ProcessFile objects (each of which is a chunk).  A supervisor object (owns
the ProcessFile objects) places ProcessFile objects into the collection in
the database object. and the database object is responsible for getting the
data out of SQL Server, getting the chunks, and BCPing them out.  So the
supervisor and the database are both accessing the collection that holds the
ProcessFile objects, each using its own thread.

All of that stuff is just "Stage1 - SQL Server to Staging".  We have other
classes which manage getting these files into a VM for processing and
getting the resulting (processed) files back from the VM and back into SQL
Server.

This whole thing is morphing into a pretty well defined object model where
each object has clearly defined responsibilities.  I say morphing because it
evolved from about 50-80 user stored procedures out on sql server, manually
executed by hand from a list written on paper.  Boy is C# / SQL Server
faster, easier and more reliable than jwc keying stuff into a query window
while reading a check list.  ;)

The key though is these collections where we place supervisors and
ProcessFiles for processing, and these lists (the process file lists) are
hit from two sides by independent threads - one placing items into the lists
and the other getting them out of the lists for processing.  Fun, efficient
but tricky.

John W. Colby
www.ColbyConsulting.com

On 12/3/2010 11:00 AM, Shamil Salakhetdinov wrote:
> Hi John --
>
> 1. Yes, static. Object ref isn't thread-safe AFAIU.
> 2. Yes, foreach isn't good. You can use
>
> int count = getCount(); // getCount should use locking for (int i=0; 
> i<count; i++) {
>     MyClass obj = getObject(i); // getObject should use locking as well as
to
> have try/catch	
> }
>
> Obvisouly while iterating that way you can loose/skip some items, 
> which will be inserted/deleted by parallel threads.
>
> (Advanced) thread-safe multi-threading is "another song" - a way more 
> complicated than ordinary programming.
> I'm not 100% sure my abive sample doesn't have some "hidden glitches".
>
> It' s time to master PLINQ, lamda-expressions, functional programming 
> principles, RESTFul web services,.... to name a few.
>
> The above is a "to do learning list" for myself also :) - I'm not yet
there.
> And I'm not sure I will find good enough time to master all that 
> advanced stuff...
>
> Thank you.
>
> --
> Shamil
_______________________________________________
dba-VB mailing list
dba-VB at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/dba-vb
http://www.databaseadvisors.com