[dba-VB] How I'm approaching the problem

Jim,

Throw in the fact that I have to get real work done (process data for the
client) and it really gets fun.  That is why I have been working from the
bottom up, getting those pieces written that allow me to actually process
the lists, even if I have to manually run them from the click of a button,
with hard coded constants for the parameters.

It is good that I finally have a real paying project that requires SQL
Server and VB.Net.  Yea, having to learn them under the gun is stressful,
but I have not succeeded in learning them in the past because I just didn't
NEED them and I had too much other stuff to do to spend the hundreds of
hours required to figure them out.  Now I NEED them.

I'll tell you, I really love VB.NET.  As a dev language / environment it has
everything that I want.  That was not true before 2.0, but with the latest
version it is complete enough to really do what I need.  Yes, it is 10 times
more complex than anything I have previously encountered, but a lot of the
reason is that it provides so much more "out of the box".  I am a class kind
of guy and a programmer at heart, and this is very much my dream
environment.  It is just a matter of learning it well.


John W. Colby
Colby Consulting
www.ColbyConsulting.com 
-----Original Message-----
From: dba-vb-bounces at databaseadvisors.com
[mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence
Sent: Sunday, July 22, 2007 11:01 PM
To: dba-vb at databaseadvisors.com
Subject: Re: [dba-VB] How I'm approaching the problem

Hi John:

You have taken on a massively complex project single-handedly. When I was
working full-time for a company and a similar sized project appeared I
assign at least 2 people to the project. It seems that 2 people can do the
work of three when they work together.

MS SQL people tend to think their a little better than the standard Access
grunts. Why that is so I have no idea. Considering that MS SQL developers
have the luxury of working with a faster and better product that is much
easier to obtain positive results than from an equally complex project
written totally in Access. That is why I write most of my new apps in a
combination of Access FE and MS SQL BE because I get the best of all worlds.

MS SQL is more rugged than the MDB, handles unbound connections without the
absolute need for a complex locking scheme as MS SQL is designed to work it
this type of environment. It internally handles locking, multi-access to a
single record or group of records. It is a professional level DB and is
actually easier to work with. Unfortunately, ADO is the best connection
protocol for performance and reliability but if you do not know it, it is
just another major item to learn.

If we throw learn .Net from scratch into the mix and you have to hold on
with both hands just to keep your sanity. I am amazed at how far you have
come in such a short time. Nothing like a baptism in fire... If you are a
little stressed, it is to be expected.

Hope your day has gone well.

Regards
Jim
    

-----Original Message-----
From: dba-vb-bounces at databaseadvisors.com
[mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby
Sent: Sunday, July 22, 2007 6:54 PM
To: dba-vb at databaseadvisors.com
Subject: Re: [dba-VB] How I'm approaching the problem

;-)

Sorry if I snapped buddy.  This whole system is just a tad overwhelming.
There are soooo many pieces and steps and things to do.  I am writing a
system in VB.Net to automate the process, where on a form I can specify the
server, name of a new database and table, and a directory where the files
are stored and the software will do the import from all these files into SQL
Server.  I am building another piece that exports a table (or fields in a
table) out to a set of files in a directory, kind of the inverse of the
first piece.  By running those two pieces in order, I can import, export,
address validate, re-import all in one operation.  The I can also schedule
the export / address validate / import on a periodic basis, with luck
completely automated.  All of this has to have process logging so that if
anything fails I can go see what failed, and where in the process.  It also
has to do logging to my billing database so that all this stuff gets billed
to my client automatically, whenever any piece of the process runs.

I am perhaps overly sensitive for a variety of reasons starting with the
fact that I have gotten a lot of flack on the SQL Server list about not
understanding enough SQL Server to do this stuff (true, but when has that
ever stopped me), how the wizards are toys meant for beginners and my needs
far exceed their capabilities (also true) etc.  I am struggling with
learning two entire new systems - SQL Server and VB.Net / ADO.Net AND doing
it on hardware / software that truly is inadequate (or barely adequate) for
the task.  These databases are HUGE by any datasets I have ever encountered
in the past.  I am accustomed to doing systems with hundreds of tables but
under a million records in the largest table.  Here it is a handful of
tables but tens of millions of records in each one.  Desktop machines with
32 bit OS / Sql Server just don't cut it.

On the bright side the quad core machines are out and a price war is on.
The price of memory is dropping like a rock, and I can now build a dual
processor 8 core system with 32 and up to 64 gb of ram for a "reasonable"
price.  It appears that I will be doing so before the end of the year.  I
found 64 bit SQL Server at a price I could afford and now if I can get a
copy of Windows 2003 x64 at a price I can afford (and get it to install and
run - drivers are still an issue) I should finally have a SQL Server system
that will have the oomph to handle my data.

I am a one man show, trying to do a pretty huge job (in my universe anyway)
and I am a little stressed.  But things are finally coming together.

John W. Colby
Colby Consulting