[AccessD] Data Export Spec - Rev1

John Colby jwcolby at ColbyConsulting.com
Thu Oct 20 19:22:22 CDT 2005


Shamil,

I guess I don't understand what you mean.  This is a self contained system.
Place data in the tables, call a function, out pops a text file.  That is
precisely the plan, to make a table driven export SYSTEM, where you place
specifications for exporting data into a set of tables (well defined
interface), instantiate a class, and call a method of the class.  Out pops a
text file.  It is intentionally (or intended to be) that simple.  At the
same time it can be simultaneously used by 1 or a dozen (or a hundred)
different users, exporting different data files, each exporting the files
they choose by selecting which Export record(s) they use.  I run this thing
on a server, automatically, in the middle of the night, but that doesn't
have to be.

>- problem to share the work between team members;

Work on this export wizard?  There are only a handful of classes envisioned,
possibly as few as two.  I am not getting volunteers buy the droves you
might notice, so it does not appear that splitting up the work is going to
be one of the major concerns.

>- problem of duplicated code, which may become "nightmarish" to support;

I guess I just don't understand what you see.  The entire system will likely
be a pair of classes.  One class holds the data and methods for a field, the
other holds the data and methods for the recordset export.

>- problem for future extensions;

Am I missing something?  If you are going to define a program, that performs
a fixed functionality, then you always risk "problem for future extensions".
This is not .net, there is no inheritance.  The best we can do is to open
the discussion up and get as much input and ideas for future expansions as
possible right up front so that they can be planned for.  If additions are
needed later, then you do like you do in any project, you attempt to make
them fit.  This is not Windows however, or SQL Server, or Office, it is a
pair of classes and 4 tables.

>- problem for scaling up....

Well... This one will indeed be a problem.  If you intend to export millions
of records using this method the results will likely be unsatisfactory.
That said however, this method loads the field class instances into memory
and just passes a pointer to the recordset to each one.  We can certainly
run timing analysis per record (per field) but what you get is what you get.
Again, there is no magic involved here.  In fact using DAO instead of ADO
will likely INCREASE the speed rather than decrease it.  It just occurred to
me that you might be able to scale it up using parallel processing, have 10
machines each append to it's own file, then append the 10 files together.

However, if you envision something that needs to export millions of records
at a shot, then a custom highly optimized solution is probably preferred.

>etc.

Sorry, can't really address that one until it is fleshed out a bit.  ;-)

I have a "similar" system actually running.  It is very application
specific, but the concept is virtually identical.  I export 4 files with it
daily.  I can time it to get a feel for performance, but like anything,
performance will always "depend" on a lot of different variables.  Your
Mileage May Vary GREATLY.

My intention here is to attempt to "genericize" a common requirement -
delimited or fixed width text file export.  It is just plain silly to code
this over and over and over (and over).

BTW, I will eventually be porting this to .NET as I have a requirement for
that.

John W. Colby
www.ColbyConsulting.com 

Contribute your unused CPU cycles to a good cause:
http://folding.stanford.edu/

-----Original Message-----
From: accessd-bounces at databaseadvisors.com
[mailto:accessd-bounces at databaseadvisors.com] On Behalf Of Shamil
Salakhetdinov
Sent: Thursday, October 20, 2005 5:29 PM
To: Access Developers discussion and problem solving
Subject: Re: [AccessD] Data Export Spec - Rev1


> are you discussing porting this out of an Access container to 
> something like VB6 or VB.Net?
I just wanted to make "crystal clear" what kind of the data sources/data
source objects are planned to be used for export and transformation....

>  Or is this a "I don't like to reference DAO"
> kind of thing.
No, I like DAO :)

> If the code that does this is contained inside the class, what 
> difference does the choice of DAO vs ADO make
>  - other than having to
> link to the DAO library?
IMO the problem is as far as I see (I can be wrong) that you plan to develop
tightly coupled synchronous solution. This is a problem if you work alone
but this is even a bigger problem when working in a team:

- problem to share the work between team members;
- problem of duplicated code, which may become "nigthmarish" to support;
- problem for future extensions;
- problem for scaling up....

etc.


Shamil

-- 
AccessD mailing list
AccessD at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/accessd
Website: http://www.databaseadvisors.com





More information about the AccessD mailing list