[dba-VB] Parallelism

Hi Jim --

Thank you for your remark.

Yes, I suppose I do realize the difference between multi-threading and
Parallelism. And I suppose I do use both here in my everyday development:

- Parallelism can be implemented by using multi-threading (long running
threads usually performing conceptually different tasks) but it can be also
implemented by running independent processes...

- Multi-threading is usually treated as multiple short-living
threads/-sub-processes running within one process, and often performing
conceptually the same tasks, at least most of the threads of a
multi-threaded application...

Both Parallelism and Multi-threading can be implemented on one CPU provided
independent processes or threads have some waiting/idle time - waiting for
some external events to happen/tasks to be processed - that concept exists
since IBM 360 MFT OS times (early 60-ies?)...

Both Parallelism and Multi-Threading are getting mainstream nowadays when
one process, which can't be "paralleled" on one CPU system because it has no
"external events" to wait, can now be split into several
sub-processes/threads to be scheduled to run on several CPUs/cores available
within one computer - that's new for mass market PCs, but this concept of
Parallelism/multi-threading is rather old coming from supercomputers - CRAY
and CDC there, and Elbrus here (also 60ies AFAIKR)...

Please correct me if I'm wrong.

What's so new with "the new Parallelism" you mentioned?
What am I missing? - Let's define this discussion thread context: I do not
mention "neural networks" and "clouds" here - we are talking within context
of one physical unit having several CPUs/cores and controlled by one
operation system instance, correct?...

Thank you.

-- Shamil 

-----Original Message-----
From: dba-vb-bounces at databaseadvisors.com
[mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence
Sent: Thursday, May 20, 2010 1:50 AM
To: 'Discussion concerning Visual Basic and related programming issues.'
Subject: Re: [dba-VB] Parallelism

Hi Shamil:

Please note that the lecturer gave a very strict line of difference between
multi-threaded applications and the new Parallelism... they are very
different in coding, deployment and performance.

It now appears that the preparation of coding will now take twice as long
but it appears to be the only way to be able to get full performance from
our new multi-cored computers.

I understand that some Linux systems manage the Parallelism within their
kernels so a new application may not see as great as performance advantage
as on a Windows OS... but some List members may have far greater insight, on
this subject, than I do.

I had previously sent an email to Tiberiu Covaci asking for some
clarification on the points and differences and as soon as I hear a reply, I
will share that information.

Jim


 
-----Original Message-----
From: dba-vb-bounces at databaseadvisors.com
[mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil
Salakhetdinov
Sent: Wednesday, May 19, 2010 2:08 PM
To: 'Discussion concerning Visual Basic and related programming issues.'
Subject: Re: [dba-VB] Parallelism

Hi Jim --

Yes, multi-threaded apps development is a very interesting and useful stuff:
I do actively use multi-thereading in my customers' real life 24x7x365 .NET
applications, but I still have a lot to learn/to do (and that learning by
doing promise to last forever :)) - the only thing I can say for sure now is
that multi-threading development is at least a matter of magnitude more
complicated and time consuming than a "mere" conventional single-threaded
apps development. Hopefully the new MS technologies - .NET Framework, C# 4.0
and PLINQ will help to lower this complexity a bit(?)...

Thank you.

--
Shamil

-----Original Message-----
From: dba-vb-bounces at databaseadvisors.com
[mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence
Sent: Wednesday, May 19, 2010 6:55 PM
To: 'Discussion of Hardware and Software issues'; 'Discussion concerning MS
SQL Server'; dba-vb at databaseadvisors.com
Subject: [dba-VB] Parallelism

Hi All:

A couple of nights ago, I went to a series of lectures given by a fellow
named Tiberiu Covaci, Swedish owner of a company called Multi-Core and he is
now doing a tour as a trainer for Microsoft. (His next lecture stop will be
in New Orleans.)

His training lecture was on Parallelism. The first lecture covered the basic
concepts of computers and their new multi-core design, a design made
necessary as single core computers ran into the physical wall as of 2005. He
then went on to describe the attempts by developers to utilize these new
multi-core systems. It has been a slow process as you can imagine.

The rudimentary attempts of threading and hyper-threading were far from
successful. First a single thread will consume up to 200 cycles just to get
substantiated and its deployment still requires the central CPU and to spawn
hundreds of threads, in a pool and then there was the issues of
synchronizing the process results. Even the best developed multi-threaded
applications, like MS SQL 2005 could over-load a computer when challenged by
a heavy duty task. MS even went so far as to apply a system lock when
performing resource heavy jobs... that means all other functionality on the
server would cease... no multi-tasking, until the job was completed... even
then sometimes a process would freeze up a server. (We have all heard of
that exact situation from a number of our DBA members.)

Our lecturer had designed a couple of short programs to showed how
multi-threading was implemented and then demonstrated how these apps when
over-loaded would crash or lock out the system. He used a Quick-sort, demo
while splitting it into multiple pieces, multi-threading it and slowly
increasing the sort data showed a 30 percent gain per thread and a 70
percent gain per core (maximum gain) but it also showed virtually not
performance loss as the data increased (10 to 100 million records) until the
process locked up the CPU. We watched the system progress via the Task
Manger's CPU display.   

Lecture two; enter Parallelism. This process has being developed on the
latest versions of Java and .Net frame work (4 or greater). He demonstrated
two new core features/objects Parallel and Task. Using Parallelism requires
a bit to decompose a problem into its components. He used an example of a
recipe where a number of steps was required from initial preparation, to the
cooking and to the table. The program which ended calculating the time was
initially designed in a standard C# routine.

The app was then recoded using Parallel running tasks. Each task can run
independently, but wait for a process to be completed if required by
scheduling, monitoring a semaphore, event, state and/or wait loops. We also
covered how to handle synchronization, data sharing and how to avoid
deadlocks. By using pararllelism the meal was completed in half the time...
It showed how 6 people could do a job faster than one... now that's obvious
but maybe not so obvious with a computer application.

The most telling observation was when viewing the Task Manger's CPU display.
No matter how heavy the requirements that were imposed on the system, the
CPU demand remained very low and flat and each core showed an even
distribution of utilization.

I was totally impressed. Now I know how such super databases as Cassandra,
using Java Parallelism works. My suggestion to all those of you using MS SQL
2005 or less; "Bail out quick and get you hands on the latest MS SQL...
there will be no comparison in performance."

A couple of recommended books on the subject are:
"Patterns of Parallel Processing" and
"Concurrent Programming on Windows"

The latest .Net has all these features built in...just ready to use.

Jim          

_______________________________________________
dba-VB mailing list
dba-VB at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/dba-vb
http://www.databaseadvisors.com

_______________________________________________
dba-VB mailing list
dba-VB at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/dba-vb
http://www.databaseadvisors.com

_______________________________________________
dba-VB mailing list
dba-VB at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/dba-vb
http://www.databaseadvisors.com