JWColby
jwcolby at colbyconsulting.com
Wed Mar 7 12:19:04 CST 2007
So process A writes a start time file and an end time file with the time embedded in the file name? I like it. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: accessd-bounces at databaseadvisors.com [mailto:accessd-bounces at databaseadvisors.com] On Behalf Of Jim Dettman Sent: Wednesday, March 07, 2007 12:53 PM To: 'Access Developers discussion and problem solving' Subject: Re: [AccessD] FTP John, I do almost exactly the same thing, but I've taken it a step further in terms of monitoring/execution. I have one process (process A) like you to monitor the directories and move the FTP files. I have another process (process B) that runs on a separate server which checks for unprocessed files on local disks. In addition, it checks timestamp files written by process A to determine if that process has failed to start or is stalled. Process B also writes its own timestamp files, which process A checks in the same manor. Thus they form a pair, each checking on the other. If a check fails, it sends out an e-mail. I write one timestamp file to disk <appname>_START.txt and delete the end time stamp file when the app starts up. When the app finishes, I write the end file to disk, <appname>_END.txt. By doing so, any app can check on an automated job and tell if it is currently running (Start file but no end file) and if so how long (Current time - start), if it is not running (both time stamp files exist) and how long the last execution took (End - start), or if it's overdue (Current time - End > schedule interval). I use this setup when it is critical that the FTP transfers occur on a timely basis. Jim. -----Original Message----- From: accessd-bounces at databaseadvisors.com [mailto:accessd-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, March 07, 2007 9:59 AM To: 'Access Developers discussion and problem solving' Subject: Re: [AccessD] FTP I have my client purchase the 3DFTP product. I don't have any commercial products, all of my stuff is consulting jobs. I am developing an access C2DbFTP.mda (library / framework) that does drop box monitoring to upload / download files. This application just watches specific directories and if a file appears, moves it to a different directory. Monitor FTP dir / move to local dir, monitor local dir / move to ftp dir. This application does not DO ANYTHING with the files moved, it just moves them, and logs the move. The idea is to allow any application that needs such functionality to just drop a file in a directory and expect that it will be moved to the right place. For example DISCO - my insurance call center client's application - needs to FTP files from insurance companies to the local drive. Once a file appears from the insurer, the file is opened by DISCO and loaded into the database. Likewise the application needs to ftp files back to the insurer and also needs to ftp files to banks. Thus DISCO prepares "reports" (as I discussed in a previous thread) and drops those "reports" into a specific directory on the local hard disk. The C2DbFTP client watches these hard disk directories for files, and simply transfers them up to whatever ftp address is associated with that directory. My C2DbFTP application has a table that says: For tblDropBox: Drop Box PKID 'PKID of a record LocalDir 'Local directory on the hard disk FTP site 'FTP site FTP PW 'FTP Password FTP User 'FTP Username FTP Dir 'Directory on the FTP site Direction of transfer 'Direction of the FTP transfer Log Table 'Table to log results to C2DbFTP loads this table to determine all of the directories / FTP sites to monitor and which direction the transfer has to go. C2DbFTP simply polls each "source" directory for files, and if any files are found, they are moved to the destination directory and deleted in the source directory. The Direction of Transfer tells C2DbFTP whether the transfer is from an FTP site down to a local drive or vv. When any file is transferred, it is logged in the Log Table. The log table is nothing more than the PKID from tblDropBox, the file name, the date and time the move occurred, and an error code in case the FTP failed for some reason. So ANY application can write a record into the tblDropBox telling C2DbFTP "I need a transfer done", drop a file into the directory that is to be monitored (if a send), and watch the log table for the file being sent. It can get files this way, and it can send files this way. If it receiving files, it writes a tblDropBox record to set up the transfers and watches local directory for files. What is done with the files are not the concern of C2DbFTP, just the fact that FTP / local directories need to be monitored for files to move. The application moving files does not have to "know how to do FTP", all it needs to know is how to write a record to the table telling C2DbFTP to move a file, and provide a log table to watch for the move results. The application uses / creates the files, C2DbFTP just moves them. I am using this with another client to NCOA process addresses. The client runs C2DbFTP. He sets up an upload directory to be watched by C2DbFTP and then drops files of addresses to be processed into that upload directory. C2DbFTP watches the directory and uploads them to my FTP site. Another instance of C2DbFTP running on my server here at my office watches the FTP upload directory (for that client) up on my FTP server and if a file exists, downloads it to my local server. My local server runs the file through the NCOA address processing system which drops the result file into a specific directory on my local server hard disk. C2DBFTP is watching that dir and uploads the files back to the FTP site for that client. The C2DbFTP instance running at the client watches for files in that FTP directory on my site and downloads to the client's hard disk. The client's database application watches for files in that local hard disk dir and if one appears, loads the file into the database for processing / merging back into his address table. As you can see, neither my NCOA processing application nor the clients database application needs to understand FTP, all it needs to understand is where to place the files to be NCOA processed and what to do when NCOA processed files appear in a specific directory. Likewise, my NCOA processing application doesn't need to understand FTP, all it needs to do is know where to place files and what to do when files appear in a given directory. BTW, my NCOA processing application also logs the files being processed and generates billing information which is sent to the client, billing them for the NCOA processing. So C2DbFTP can be used by completely different applications to process FTP transfers. I have a wrapper class for the 3DFTP API. And of course each of these applications could be custom programmed to do FTP directly, inside of each application using my wrapper class. And if that becomes necessary I can certainly do that, but a "drop box" metaphor with a log table works quite well much of the time, and can be implemented much more quickly. Drop C2DbFTP in place and run it. Drop records in the table drop box, and start dropping files / watching for files. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: accessd-bounces at databaseadvisors.com [mailto:accessd-bounces at databaseadvisors.com] On Behalf Of Bobby Heid Sent: Wednesday, March 07, 2007 8:53 AM To: 'Access Developers discussion and problem solving' Subject: Re: [AccessD] FTP John, How do you handle licensing with that product? Bobby -----Original Message----- From: accessd-bounces at databaseadvisors.com [mailto:accessd-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, March 07, 2007 5:22 AM To: 'Access Developers discussion and problem solving' Subject: [AccessD] FTP I noticed at least one person discussing monitoring directories and FTPing up/down. For those that do that, what method do you use for implementing the FTP stuff? I have been developing similar capabilities using 3DFTP (which I love and recommend highly) and an API to program to it. John W. Colby Colby Consulting www.ColbyConsulting.com -- AccessD mailing list AccessD at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/accessd Website: http://www.databaseadvisors.com -- AccessD mailing list AccessD at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/accessd Website: http://www.databaseadvisors.com -- AccessD mailing list AccessD at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/accessd Website: http://www.databaseadvisors.com