[AccessD] FTP

>Outside of this, so far I haven't found anything that will let me control
file processing if I don't have control of the other end.

And this is a distinct problem.  Most of my stuff I have no control of the
other end, and in fact the other end is resistant to doing ANYTHING to help.
They apparently already have processes in place that say "do this" - give me
a file that contains EXACTLY this information in this column, exactly this
width, starting in this position, the file named this, with this file
extension etc.  IF everything matches, then a mainframe correctly processes
the file.  If ANYTHING is not correct, the mainframe fails to process the
file and kicks out an error report.  We get a call if there is an error
report, else we don't hear anything.

Thus whatever I do has to be geared toward an extremely rigid, already
existing system where I am going to get no help on resolving such issues.

Now there are cases such as my NCOA address processing system where I do
have control over both ends.  In any situation where my C2DbFTP is running
on the remote site, then I can have C2DBFTP use it's own custom protocol
designed however I see fit.  Thus my interest in how you do this.

John W. Colby
Colby Consulting
www.ColbyConsulting.com

-----Original Message-----
From: accessd-bounces at databaseadvisors.com
[mailto:accessd-bounces at databaseadvisors.com] On Behalf Of Jim Dettman
Sent: Thursday, March 08, 2007 9:38 AM
To: 'Access Developers discussion and problem solving'
Subject: Re: [AccessD] FTP

John,

<<I think if I run into that I will need to ensure that the file is locked
(open it for writing?) or add a second file that is written by the FTP
client that says "I'm processing".>>

  I think I might do the latter.  I have been using the technique that if
the file exists, simply don't send another.  That works fine.  What doesn't
work is that I have found that FTP does not lock a file during transmission.

  So on jobs where I have a really tight schedule cycle (say every minute)
and the file is rather large, the transmitting end is still sending the
file, but the receiving end already starts to process the file and ends up
with a partial file.

  A couple of ideas I had to work around that is:

0. A processing flag that says "don't touch anything in this directory".

1. Sending along a second file for each file at the completion of
transmission (A "OK to process" flag for each file).  This avoids locking an
entire directory. Optionally, I could include a CRC checksum in the file.

2. Using a ZIP utility to zip everything (and thus provide a CRC check).

  Right now, I'm leaning towards #1.  The problem with this is you needs to
control both ends.  #2 is the only thing that would work if you don't have
control of the other end.  Outside of this, so far I haven't found anything
that will let me control file processing if I don't have control of the
other end.

Jim.

 

-----Original Message-----
From: accessd-bounces at databaseadvisors.com
[mailto:accessd-bounces at databaseadvisors.com] On Behalf Of JWColby
Sent: Wednesday, March 07, 2007 4:56 PM
To: 'Access Developers discussion and problem solving'
Subject: Re: [AccessD] FTP

>Does the application handle a 'PickupBox' as well as a 'DropBox'

Each drop box by definition has a pickup and drop box.  Which is which is
defined by the direction flag.  I currently use separate directories on each
end if bidirectional transfers are required.  They aren't always required.
For example, we send files to the bank but they never respond in any way

ATM it just sends and deletes behind itself, logging any errors.  This all
started with the objective of sending files for the insurance client.  They
have no method of handling such a thing.  In fact they often require that
the file name be very specific.  The concept generally is a process / delete
kind of thing where if the file is processed it is immediately deleted in
preparation for the next file.  I imagine that from their perspective, the
specific naming conventions allow them to have multiple files in the
directory for / from different clients, DIS' file has DIS_XXX.Dat where XXX
is an account number or something like that.

Obviously there will be issues of timing.  These files only occur daily but
what happens if the requirements push the timing up to where the transfer
can't occur fast enough to move the file before the next drop into the box.
I think if I run into that I will need to ensure that the file is locked
(open it for writing?) or add a second file that is written by the FTP
client that says "I'm processing".

John W. Colby
Colby Consulting
www.ColbyConsulting.com

-----Original Message-----
From: accessd-bounces at databaseadvisors.com
[mailto:accessd-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence
Sent: Wednesday, March 07, 2007 1:25 PM
To: 'Access Developers discussion and problem solving'
Subject: Re: [AccessD] FTP

Hi John:

This sounds really good. Does the application handle a 'PickupBox' as well
as a 'DropBox' and how does it handle files with the same name just
different dates and time?

Jim