[AccessD] Can anyone help me with this

Jim Lawrence accessd at shaw.ca
Mon Oct 16 13:59:38 CDT 2017 

That's a tough one. I did a quick scan and found no FS that easily handled huge numbers of small or zero length files.

Are all these small files necessary? When do they get deleted? If these files have a time limit could a CRON or AT batch file be used to spawn a back-ground process to clear the unneeded entries? Could some of these small entries be managed better in a database? Are the small file sequentially named and numbered? Could a "range" entry be used to correctly describe current files?

Many years ago, when an application wrote hundreds of small files, I had to build a system that generated lists of directories each holding numerous files. The FS at the time was only a FAT32 and the number of files in a single directory limit was much lower that today's FSs but the management should be fairly similar.
  
Jim 

----- Original Message -----
From: "Access Developers discussion and problem solving" <accessd at databaseadvisors.com>
To: "Access Developers discussion and problem solving" <accessd at databaseadvisors.com>
Cc: "James Button" <jamesbutton at blueyonder.co.uk>
Sent: Sunday, October 15, 2017 2:37:47 PM
Subject: Re: [AccessD] Can anyone help me with this

My concerns about the NTFS is that ALL the file entries are in the ;hive' as
well as the data from any small files.

I have 'sorted' systems with over 100,000 zero length entries in the MFT -
mostly 'Browser' and other apps marker files. 
Some in the users 'local' and associated folder entries -  appallingly slow
response to the point of hanging and timeouts  while the OS works through the
MFT that is many times larger than the RAM, 

I would not have expected it to be a dying drive as the system runs while doing
things -  maybe a problem with the PSU, or a dying fan!
 
The sleep setting is either manufacturer set or set within the OS - sleep seems
to be the earliest allowed by the shortest setting wherever.
Maybe it would be worth running a chkdsk - report mode - The fix mode can result
in a totally destroyed data store as file entries without the proper links get
deleted.

JimB

-----Original Message-----
From: AccessD [mailto:accessd-bounces at databaseadvisors.com] On Behalf Of Jim
Lawrence
Sent: Sunday, October 15, 2017 9:57 PM
To: Access Developers discussion and problem solving
<accessd at databaseadvisors.com>
Subject: Re: [AccessD] Can anyone help me with this

The latest NTFS 5.x has a very good reputation as extremely reliable. I do not
think MS is directly compatible with any other File Systems, which is too bad as
there are some excellent other FS options available. Very rugged. 

You brought up an interesting point. Maybe the reboot problems are related to
the hard drives...or even a dying harddrive. 

Jim 

----- Original Message -----
From: "Access Developers discussion and problem solving"
<accessd at databaseadvisors.com>
To: "Access Developers discussion and problem solving"
<accessd at databaseadvisors.com>
Cc: "James Button" <jamesbutton at blueyonder.co.uk>
Sent: Sunday, October 15, 2017 10:21:35 AM
Subject: Re: [AccessD] Can anyone help me with this

Maybe check on 
Designed-in "Do this when no user activity processes" features: - 
Write all memory to pagefile - so is the pagefile pre-allocated as  big enough,
and is the allocated space OK for read after rite?
Compress and tidy up the MFT (for NTFS) - so do you have a partition with a
folder that has enormous numbers of empty, or very small files ?
Has an OS update re-instated hibernate ?
Is there a drive that is shutting down  as in sleep after a period ?
I had a WD drive that did that - and caused write failures usually updating the
MFT with entries where the app was not specifying verification of writes - so
the Windoze  just kept the data until it decided it needed to write them to the
drive - usually when it got the "I'm going beddy-byes" notification from the
drive.
And - note the validation required on writes to the drive applies to APPS - NOT
to the windows activities -
 
Nice design feature Microsoft - goes along with the we will be making things
secure opening of comms before starting the firewall in XP


So - I'm getting back to my current annoyance Onedrive - where the upload
progress report removes the destination folder id  from the actions heading
when it gets to the end of a set of actions 
NOTE - I did not say completes the set - and - when you go to see what has
happened to the folders - the report has been totally removed 
And - it being much faster to restart the system (or at least the browser( than
wait for Onedrive "long running script" to finish arguing over which of the 2
threads it uses should get itself back from pagefile into RAM -
Maybe an hour - or more - 2GB with 10GB pagefile to get the upload to usually
run!

  JimB


-----Original Message----- 

From: AccessD [mailto:accessd-bounces at databaseadvisors.com] On Behalf Of John
Colby
Sent: Sunday, October 15, 2017 4:19 PM
To: Access Developers discussion and problem solving
<accessd at databaseadvisors.com>; Jim Lawrence <accessd at shaw.ca>
Subject: Re: [AccessD] Can anyone help me with this

The machine just reboots.  It does not put anything in any log. The log 
as it comes back up says "unexpected shutdown" kind of thing.  So what 
is bizarre is that it will stay running as long as I keep a core running 
something.  NEVER reboots as long as it is kept active.  Which is my 
"solution" at this point.  The reboots are at random times.  Sometimes 
once every few days, normally a few times a day, sometimes 10 times in a 
day.  But if overheating or other "hardware" issue, why will it NOT 
reboot as long as it is kept active.  I run long queries maxing out all 
16 cores for hours and it works just fine.  NEVER reboots if kept 
active.  Let it go idle and it starts rebooting.

Puzzling and frustrating.

In the end, keeping a single core busy keeps it alive so it is no longer 
a problem that I feel pressed to deal with, but I'd like to have my core 
back. ;)

On 10/14/2017 2:20 PM, Jim Lawrence wrote:
> Hi John:
>
> You have probably gone through all these checks but I will post them anyways.
Have you checked your logs? Is there an exact time when this reboot occurs? Is
the reboot spawned by something like an "exceptions error" or a system process?
Considering that all your software is probably up to date could it be a hardware
error? A failing piece of equipment or some new hardware that was introduced?
>
> One comment I have seen is that an active SQL server eventually runs out of
address space and if possible it is quicker and cleaner to just do an "idle
time" reboot than to run the "garbage collection" routine. If that is the case
there should be some option setting that would control the activation and
timing?
>   
> Jim
>
> ----- Original Message -----
> From: "John Colby" <jwcolby at gmail.com>
> To: "Access Developers discussion and problem solving"
<accessd at databaseadvisors.com>
> Sent: Saturday, October 14, 2017 7:45:46 AM
> Subject: [AccessD] Can anyone help me with this
>
> My SQL Server (Windows 2008) reboots periodically.  Periodically implies
> on a schedule so perhaps sporadically is a better description.  This has
> been going on for years.  It will do so several times a day.
>
> What I have discovered is that if I keep one of the cores busy, and I
> use a script in SQL Server to do that, then it never reboots.
>
> I read that Windows 2008 can be led to believe (accidentally in my case)
> that it needs to reboot to finish an install or something of that
> nature, and it will do so when the system is "idle".  By keeping a core
> active, the system never does that reboot thing.  I do not know for sure
> that this is what is going on but it absolutely makes sense.  In he
> meantime I have to do this "keep a core active" thing or my server
> reboots.  If anyone out there understands what is really going on and
> how to tell Windows to knock it off, please contact me.
>
> Thanks,
>

-- 
John W. Colby

-- 
AccessD mailing list
AccessD at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/accessd
Website: http://www.databaseadvisors.com


-- 
AccessD mailing list
AccessD at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/accessd
Website: http://www.databaseadvisors.com
-- 
AccessD mailing list
AccessD at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/accessd
Website: http://www.databaseadvisors.com

-- 
AccessD mailing list
AccessD at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/accessd
Website: http://www.databaseadvisors.com

More information about the AccessD mailing list