[dba-SQLServer] Really Interesting puzzle

Arthur Fuller fuller.artful at gmail.com
Fri Oct 3 15:09:54 CDT 2008


Ok I have all the glitches sorted out. It required a change or two on the
incoming side (creation of the source files from the stream) and a change or
two on the destination side, but we're there. On to the next problem...

The c++ program reads the stream and periodically writes an ascii file,
which I then inhale using Bulk Insert. This part is insanely fast, I can't
believe how fast it is. The c++ part creates an in-memory database and then
periodically writes an ascii file. At that point, I grab the file and to the
bulk insert. Although the insert only takes seconds, it could well happen
that more ascii files appear. So the directory may at any moment contain N
files, to be processed in sequence. The current plan is to grab the first
file, inhale it, move it to another directory, and repeat.

The files are going to have names such as xxx_20081103_01.csv. And then 02
and so on.

I'm pondering how to know the next file to open. I suppose that if I read
the directory list sorted by name or date that will give me the earliest
file, and then I can just pass that name to the sproc that calls Bulk
Insert. But before I code it, I thought to ask you wizards if you can see
any holes in this approach.

TIA,
Arthur



More information about the dba-SQLServer mailing list