From newsgrps at dalyn.co.nz Thu Aug 6 22:00:38 2015 From: newsgrps at dalyn.co.nz (David Emerson) Date: Fri, 7 Aug 2015 15:00:38 +1200 Subject: [dba-SQLServer] Current Connections to SQL from Access Message-ID: <00a101d0d0bd$3da2f330$b8e8d990$@dalyn.co.nz> Hi Listers, How do I find out who is connected to a SQL database using my Access 2010 front end?? Tables are connected using ODBC. Connections done this way: AttachDSNLessTable(stLocalTableName As String, stRemoteTableName As String, stServer As String, stDatabase As String) Dim td As TableDef Dim stConnect As String stConnect = "ODBC;DRIVER=SQL Server;SERVER=" & stServer & ";DATABASE=" & stDatabase & ";Trusted_Connection=Yes" Set td = CurrentDb.CreateTableDef(stLocalTableName, dbAttachSavePWD, stRemoteTableName, stConnect) CurrentDb.TableDefs.Append td I have this code but it only shows me as connected and not the other users connected via Access. SELECT ????DB_NAME(dbid) as DBName, ????COUNT(dbid) as NumberOfConnections, ??? loginame as LoginName FROM ??? sys.sysprocesses WHERE ????dbid > 0 GROUP BY ????dbid, loginame ; Regards David Emerson Dalyn Software Ltd Wellington, New Zealand From newsgrps at dalyn.co.nz Thu Aug 6 23:01:01 2015 From: newsgrps at dalyn.co.nz (David Emerson) Date: Fri, 7 Aug 2015 16:01:01 +1200 Subject: [dba-SQLServer] Current Connections to SQL from Access In-Reply-To: <00a101d0d0bd$3da2f330$b8e8d990$@dalyn.co.nz> References: <00a101d0d0bd$3da2f330$b8e8d990$@dalyn.co.nz> Message-ID: <00e701d0d0c5$ac888ff0$0599afd0$@dalyn.co.nz> Problem solved - I just needed permissions to see other users. -----Original Message----- From: dba-SQLServer [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of David Emerson Sent: Friday, 7 August 2015 3:01 p.m. To: AccessDSQL Subject: [dba-SQLServer] Current Connections to SQL from Access Hi Listers, How do I find out who is connected to a SQL database using my Access 2010 front end?? Tables are connected using ODBC. Connections done this way: AttachDSNLessTable(stLocalTableName As String, stRemoteTableName As String, stServer As String, stDatabase As String) Dim td As TableDef Dim stConnect As String stConnect = "ODBC;DRIVER=SQL Server;SERVER=" & stServer & ";DATABASE=" & stDatabase & ";Trusted_Connection=Yes" Set td = CurrentDb.CreateTableDef(stLocalTableName, dbAttachSavePWD, stRemoteTableName, stConnect) CurrentDb.TableDefs.Append td I have this code but it only shows me as connected and not the other users connected via Access. SELECT ????DB_NAME(dbid) as DBName, ????COUNT(dbid) as NumberOfConnections, ??? loginame as LoginName FROM ??? sys.sysprocesses WHERE ????dbid > 0 GROUP BY ????dbid, loginame ; Regards David Emerson Dalyn Software Ltd Wellington, New Zealand From df.waters at outlook.com Fri Aug 7 10:07:40 2015 From: df.waters at outlook.com (Dan Waters) Date: Fri, 7 Aug 2015 10:07:40 -0500 Subject: [dba-SQLServer] Current Connections to SQL from Access In-Reply-To: <00a101d0d0bd$3da2f330$b8e8d990$@dalyn.co.nz> References: <00a101d0d0bd$3da2f330$b8e8d990$@dalyn.co.nz> Message-ID: Hi David, I use this: https://support.microsoft.com/en-us/kb/198755 It's quite reliable. Good Luck! Dan -----Original Message----- From: dba-SQLServer [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of David Emerson Sent: Thursday, August 06, 2015 10:01 PM To: AccessDSQL Subject: [dba-SQLServer] Current Connections to SQL from Access Hi Listers, How do I find out who is connected to a SQL database using my Access 2010 front end?? Tables are connected using ODBC. Connections done this way: AttachDSNLessTable(stLocalTableName As String, stRemoteTableName As String, stServer As String, stDatabase As String) Dim td As TableDef Dim stConnect As String stConnect = "ODBC;DRIVER=SQL Server;SERVER=" & stServer & ";DATABASE=" & stDatabase & ";Trusted_Connection=Yes" Set td = CurrentDb.CreateTableDef(stLocalTableName, dbAttachSavePWD, stRemoteTableName, stConnect) CurrentDb.TableDefs.Append td I have this code but it only shows me as connected and not the other users connected via Access. SELECT ????DB_NAME(dbid) as DBName, ????COUNT(dbid) as NumberOfConnections, ??? loginame as LoginName FROM ??? sys.sysprocesses WHERE ????dbid > 0 GROUP BY ????dbid, loginame ; Regards David Emerson Dalyn Software Ltd Wellington, New Zealand _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at gmail.com Fri Aug 7 21:20:05 2015 From: jwcolby at gmail.com (John W. Colby) Date: Fri, 7 Aug 2015 22:20:05 -0400 Subject: [dba-SQLServer] A real puzzler Message-ID: <55C56755.9080704@gmail.com> My SQL Server spontaneously reboots. In the past it was once a month, then once week or two. Now it is several times a day. UNLESS SQL Server is exercised heavily, and then it will not reboot. That is counter intuitive. If the issue were a memory issue, SQL Server using all of the memory (and it does) would find and trigger such errors. If it were a heat issue, SQL Server using all of the processors (and it does) would cause more heat. Etc. If I stop the service(s) it still reboots. I am at a loss, not only as to cause, but how to troubleshoot. Any words of wisdom? Dual chip mother board. Two AMD processors each with 8 cores (15 cores total) 80 gigs RAM Windows 2008 SQL Server 2008 -- John W. Colby From Damien.Solodow at harrison.edu Fri Aug 7 21:25:40 2015 From: Damien.Solodow at harrison.edu (Damien Solodow) Date: Sat, 8 Aug 2015 02:25:40 +0000 Subject: [dba-SQLServer] A real puzzler In-Reply-To: <55C56755.9080704@gmail.com> References: <55C56755.9080704@gmail.com> Message-ID: <04E26BCF73774949871E317AB0923ED2BBE75D3B@ENTEXMBX01.ibcschools.edu> No records of BSOD in the Event Log/dumps? Do your baseboard controller logs (iDrac, iLO, etc) say anything about the cause of the reboot? In the Event Log anything about the reboot or around the time? Perhaps some task/process is making it think it *should* reboot? DAMIEN SOLODOW Senior Systems Engineer 317.447.6033 (office) 317.447.6014 (fax) HARRISON COLLEGE ________________________________________ From: dba-SQLServer [dba-sqlserver-bounces at databaseadvisors.com] on behalf of John W. Colby [jwcolby at gmail.com] Sent: Friday, August 07, 2015 10:20 PM To: Discussion concerning MS SQL Server; Access Developers discussion and problem solving Subject: [dba-SQLServer] A real puzzler My SQL Server spontaneously reboots. In the past it was once a month, then once week or two. Now it is several times a day. UNLESS SQL Server is exercised heavily, and then it will not reboot. That is counter intuitive. If the issue were a memory issue, SQL Server using all of the memory (and it does) would find and trigger such errors. If it were a heat issue, SQL Server using all of the processors (and it does) would cause more heat. Etc. If I stop the service(s) it still reboots. I am at a loss, not only as to cause, but how to troubleshoot. Any words of wisdom? Dual chip mother board. Two AMD processors each with 8 cores (15 cores total) 80 gigs RAM Windows 2008 SQL Server 2008 -- John W. Colby _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at gmail.com Fri Aug 7 21:31:57 2015 From: jwcolby at gmail.com (John W. Colby) Date: Fri, 7 Aug 2015 22:31:57 -0400 Subject: [dba-SQLServer] A real puzzler In-Reply-To: <04E26BCF73774949871E317AB0923ED2BBE75D3B@ENTEXMBX01.ibcschools.edu> References: <55C56755.9080704@gmail.com> <04E26BCF73774949871E317AB0923ED2BBE75D3B@ENTEXMBX01.ibcschools.edu> Message-ID: <55C56A1D.9010003@gmail.com> The only log I have managed to find shows the system coming back up and logging an "unexpected reboot". John W. Colby On 8/7/2015 10:25 PM, Damien Solodow wrote: > No records of BSOD in the Event Log/dumps? > Do your baseboard controller logs (iDrac, iLO, etc) say anything about the cause of the reboot? > > In the Event Log anything about the reboot or around the time? Perhaps some task/process is making it think it *should* reboot? > > DAMIEN SOLODOW > Senior Systems Engineer > 317.447.6033 (office) > 317.447.6014 (fax) > HARRISON COLLEGE > > ________________________________________ > From: dba-SQLServer [dba-sqlserver-bounces at databaseadvisors.com] on behalf of John W. Colby [jwcolby at gmail.com] > Sent: Friday, August 07, 2015 10:20 PM > To: Discussion concerning MS SQL Server; Access Developers discussion and problem solving > Subject: [dba-SQLServer] A real puzzler > > My SQL Server spontaneously reboots. In the past it was once a month, > then once week or two. Now it is several times a day. > > UNLESS SQL Server is exercised heavily, and then it will not reboot. > > That is counter intuitive. > > If the issue were a memory issue, SQL Server using all of the memory > (and it does) would find and trigger such errors. > If it were a heat issue, SQL Server using all of the processors (and it > does) would cause more heat. > > Etc. > > If I stop the service(s) it still reboots. > > I am at a loss, not only as to cause, but how to troubleshoot. > > Any words of wisdom? > > Dual chip mother board. > Two AMD processors each with 8 cores (15 cores total) > 80 gigs RAM > Windows 2008 > SQL Server 2008 > > -- > John W. Colby > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From Damien.Solodow at harrison.edu Fri Aug 7 21:34:25 2015 From: Damien.Solodow at harrison.edu (Damien Solodow) Date: Sat, 8 Aug 2015 02:34:25 +0000 Subject: [dba-SQLServer] A real puzzler In-Reply-To: <55C56A1D.9010003@gmail.com> References: <55C56755.9080704@gmail.com> <04E26BCF73774949871E317AB0923ED2BBE75D3B@ENTEXMBX01.ibcschools.edu>, <55C56A1D.9010003@gmail.com> Message-ID: Brand and model of server? Sent from mobile. On Aug 7, 2015 10:32 PM, "John W. Colby" wrote: The only log I have managed to find shows the system coming back up and logging an "unexpected reboot". John W. Colby On 8/7/2015 10:25 PM, Damien Solodow wrote: > No records of BSOD in the Event Log/dumps? > Do your baseboard controller logs (iDrac, iLO, etc) say anything about the cause of the reboot? > > In the Event Log anything about the reboot or around the time? Perhaps some task/process is making it think it *should* reboot? > > DAMIEN SOLODOW > Senior Systems Engineer > 317.447.6033 (office) > 317.447.6014 (fax) > HARRISON COLLEGE > > ________________________________________ > From: dba-SQLServer [dba-sqlserver-bounces at databaseadvisors.com] on behalf of John W. Colby [jwcolby at gmail.com] > Sent: Friday, August 07, 2015 10:20 PM > To: Discussion concerning MS SQL Server; Access Developers discussion and problem solving > Subject: [dba-SQLServer] A real puzzler > > My SQL Server spontaneously reboots. In the past it was once a month, > then once week or two. Now it is several times a day. > > UNLESS SQL Server is exercised heavily, and then it will not reboot. > > That is counter intuitive. > > If the issue were a memory issue, SQL Server using all of the memory > (and it does) would find and trigger such errors. > If it were a heat issue, SQL Server using all of the processors (and it > does) would cause more heat. > > Etc. > > If I stop the service(s) it still reboots. > > I am at a loss, not only as to cause, but how to troubleshoot. > > Any words of wisdom? > > Dual chip mother board. > Two AMD processors each with 8 cores (15 cores total) > 80 gigs RAM > Windows 2008 > SQL Server 2008 > > -- > John W. Colby > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at gmail.com Fri Aug 7 22:08:53 2015 From: jwcolby at gmail.com (John W. Colby) Date: Fri, 7 Aug 2015 23:08:53 -0400 Subject: [dba-SQLServer] A real puzzler In-Reply-To: References: <55C56755.9080704@gmail.com> <04E26BCF73774949871E317AB0923ED2BBE75D3B@ENTEXMBX01.ibcschools.edu> <55C56A1D.9010003@gmail.com> Message-ID: <55C572C5.8090700@gmail.com> >Brand and model of server? Brand: Colby Consulting special Model: 4th generation kinda sorta. :) I built the server because I did not have the $5X that the big guys wanted for their wimpy machines. I have to tell a funny (to me) story. I use a company out in California for a data cleansing service. About five years ago, the owner was talking about how he had just paid $20K for a Dell with a quad core and 32 Gigs of Ram. I had just finished (re) building my server, a dual socket motherboard, single AMD 8 core processor with (at that time) 32 Gigs ram, an Areca RAID 6 controller with 8 tBytes across 4 volumes (using 1 tb WD Blacks), and a TB of flash drive also raid 6. For which I paid $8K. I later added a second chip for 16 total cores and another 64 gigs of RAM for 96 gigs total. And I needed all of that power and more for what I do. I have since put in a new Areca controller, a bunch of 3tb WD drives, and upgraded all of the flash drives because I was running out of drive space. All of my SQL Server log files run out of SSD. All databases and all indexes are compressed (page). Data is loaded into memory compressed, and then uncompressed on-the-fly. SQL Server is pretty darned powerful that way. It needs cores to do that however, and not "virtual" cores. Sadly, as a one man show serving a single client with this machine, I simply cannot convince the client to pay the Dell bucks. John W. Colby On 8/7/2015 10:34 PM, Damien Solodow wrote: > Brand and model of server? > > Sent from mobile. > > On Aug 7, 2015 10:32 PM, "John W. Colby" wrote: > The only log I have managed to find shows the system coming back up and > logging an "unexpected reboot". > > John W. Colby > > On 8/7/2015 10:25 PM, Damien Solodow wrote: >> No records of BSOD in the Event Log/dumps? >> Do your baseboard controller logs (iDrac, iLO, etc) say anything about the cause of the reboot? >> >> In the Event Log anything about the reboot or around the time? Perhaps some task/process is making it think it *should* reboot? >> >> DAMIEN SOLODOW >> Senior Systems Engineer >> 317.447.6033 (office) >> 317.447.6014 (fax) >> HARRISON COLLEGE >> From jwcolby at gmail.com Fri Aug 7 22:32:54 2015 From: jwcolby at gmail.com (John W. Colby) Date: Fri, 7 Aug 2015 23:32:54 -0400 Subject: [dba-SQLServer] PK cast fails Message-ID: <55C57866.4000002@gmail.com> For the first time I ended up with a runtime error trying to insert data into a table. I had an autoincrement PK which was an int type and I stored so many records that the int couldn't hold the value. And so we go to a bigint. It will be interesting to see how this works. There is this thing called a "simmons protocol", which is some kind of demographic analysis used in the industry. My client needs counts from our 225 million record database of how many records are XYZ demographic. I did this "table driven" with a set of stored procedures which calculate about 220 counts from the table. Properly indexed the entire thing takes many hours, and a full day if tied to emails. the table that I built to drive the counts looks like the following, which are just a few of the 220+ counts I have to do. PK SimmonsDefinition Code DB101Field DB101FieldName HowUsed1 SQL1 HowUsed2 SQL2 1 32 MARRIED M 32 Marital_Status NULL NULL NULL NULL 2 32 SINGLE S 32 Marital_Status NULL NULL NULL NULL 3 33 MOVED INTO PRESENT RESIDENCE IN LAST 12 MOS NULL 33 Length_Of_Residence C = '0' NULL NULL 4 39 AGE 18-19 NULL 39 Date_Of_Birth_Year F (DATEPART(yy, GETDATE()) - Date_Of_Birth_Year >= 18 AND DATEPART(yy, GETDATE()) - Date_Of_Birth_Year <= 19) NULL NULL 5 39 AGE 20-21 NULL 39 Date_Of_Birth_Year F (DATEPART(yy, GETDATE()) - Date_Of_Birth_Year >= 20 AND DATEPART(yy, GETDATE()) - Date_Of_Birth_Year <= 21) NULL NULL 6 39 AGE 22 - 24 NULL 39 Date_Of_Birth_Year F (DATEPART(yy, GETDATE()) - Date_Of_Birth_Year >= 22 AND DATEPART(yy, GETDATE()) - Date_Of_Birth_Year <= 24) NULL NULL 18 43 SENIOR ADULTS IN HH (55+) Y 43 Senior_Adult_In_Household NULL NULL NULL NULL 19 44 EMPTY NESTER Y 44 Empty_Nester NULL NULL NULL NULL 20 45 SINGLE PARENT 1 45 Single_Parent NULL NULL NULL NULL I woke up this morning and realized that I needed to select each PKID from DB101 which matched each Simmons selection, and store those with the Simmons PKID. Do this one time and then do my counts off of the stored results instead of recounting each time. The issue I am having is that a) the SQL for doing the selection (or count) is dynamically created, but even worse, the WHERE field is pulled from usually one of hundreds of possible fields in DB101. And it is literally impossible to do indexes on that many fields, and especially across 225 million rows. So I am building a table that looks like: PK (bigint) PKSimmons (int) PKDB101 (int) Select ALL of the DB101 PKs that match each simmons criteria one time and store in a table as above. Select once and store, index the two fields, and then use a join on this table to do further processing, including counts in the future. It appears as I will have many billions of records in this table. I do know that after about 75 selections, I hit the limit of the Int data type to hold my PK (~ 2 billion records), which I had therefore had to change to a BigInt. It will be interesting to see how long it takes to build an index on the PKSimmons and PKDB101 fields for future selects. And of course, how fast PKDB101 values can be pulled out when selected by the PKSimmons field given an index on those two fields - across an index over 8 or 10 billion records. I will report back some results once the table insert(s) finish. -- John W. Colby From jwcolby at gmail.com Sat Aug 8 07:14:47 2015 From: jwcolby at gmail.com (John W. Colby) Date: Sat, 8 Aug 2015 08:14:47 -0400 Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: References: <55C56755.9080704@gmail.com> <1022E1B177C94A4BAC50605960BE881C@XPS> Message-ID: <55C5F2B7.3090806@gmail.com> For the last several days I have been flogging away at the system (performing real work), causing the system to stay active. All cores running, 75 (out of 80) GB used for SQL Server. No reboots during that entire time. And yet: Critical 8/4/2015 6:56:10 AM Kernel-Power 41 (63) Critical 8/4/2015 4:42:07 AM Kernel-Power 41 (63) Critical 8/3/2015 10:04:36 PM Kernel-Power 41 (63) Critical 8/3/2015 5:32:06 PM Kernel-Power 41 (63) Critical 8/3/2015 2:22:15 PM Kernel-Power 41 (63) Critical 8/2/2015 3:29:33 PM Kernel-Power 41 (63) Critical 8/2/2015 11:21:46 AM Kernel-Power 41 (63) Critical 8/2/2015 10:51:30 AM Kernel-Power 41 (63) Critical 8/2/2015 7:17:21 AM Kernel-Power 41 (63) Critical 7/31/2015 11:50:45 PM Kernel-Power 41 (63) Critical 7/31/2015 1:05:39 PM Kernel-Power 41 (63) Critical 7/30/2015 10:11:57 PM Kernel-Power 41 (63) Critical 7/30/2015 4:59:26 PM Kernel-Power 41 (63) Critical 7/29/2015 2:32:50 PM Kernel-Power 41 (63) Critical 7/28/2015 5:20:38 PM Kernel-Power 41 (63) Critical 7/28/2015 12:12:57 PM Kernel-Power 41 (63) Critical 7/28/2015 4:15:38 AM Kernel-Power 41 (63) Critical 7/28/2015 2:47:12 AM Kernel-Power 41 (63) WEIRD!!! Notice no pattern in number of events per day, nor time of day. There is no memory dump being created, the system just reboots as if the power was turned off and back on. When I was in the room with it, (several years ago) the system would beep as it rebooted. Here is the last such event: Log Name: System Source: Microsoft-Windows-Kernel-Power Date: 8/4/2015 6:56:10 AM Event ID: 41 Task Category: (63) Level: Critical Keywords: (2) User: SYSTEM Computer: Azul Description: The system has rebooted without cleanly shutting down first. This error could be caused if the system stopped responding, crashed, or lost power unexpectedly. Event Xml: 41 2 1 63 0 0x8000000000000002 270735 System Azul 0 0x0 0x0 0x0 0x0 false 0 John W. Colby On 8/8/2015 7:57 AM, James Button wrote: > Guest that! > > Re memory diagnostic - I have found with past experiences of accelerating > frequency of shut-downs, that the system didn't get a chance to record any > events. > And memory checks showed no problems - providing the rest of the system wasn't > being stressed. > > One system I found that removing a memory module - any of them stopped the > shutdowns, and I eventually 'bodged' the system by increasing the memory refresh > by a cycle. It was an old system and a 'new' memory module was, being old tech, > horrendously expensive > That worked for several years, and eventually management agreed the system was > too slow - as in users kept complaining about the system's response, so we were > actually allowed to BUY a new one! > > JimB > From jwcolby at gmail.com Sat Aug 8 10:00:12 2015 From: jwcolby at gmail.com (John W. Colby) Date: Sat, 8 Aug 2015 11:00:12 -0400 Subject: [dba-SQLServer] 2.45 billion rows and counting was PK Cast fails Message-ID: <55C6197C.4010609@gmail.com> The Simmons table I am building now has 2.5 billion rows. The stored procedure is running a "loop" iterating through a table of 228 Simmons demographics definitions, pulling PKs from DB101 (225 million possible PKs) one at a time, and inserting them into the junction table mentioned in the previous email. Sometime last night the log file ran out of room (filled up the log disk) and stopped the process, at Simmons Def 70. Luckily I am writing the individual sql statements out as each one is executed so I could see where it stopped. I cleaned a bunch of very large log files off and restarted this morning. While I can see the total storage (records) in the junction table, what I can't do is get a Max() of the Simmons PK in the junction table to see where I am in the process. I was hoping that info would be in metadata somewhere but apparently not, it seems that it has to scan through the table looking for the max() value. While I haven't let it run in a query window, trying to do that in a view timed out for obvious reasons. And so I wait for completion. I would estimate the junction table will contain somewhere north of 10 billion records when done. I have never even come close to this quantity of records in a single table so I am still unsure whether this will even make things faster. Assuming an index on two fields: PKSimmons - PKs for each of 220 Simmons Definitions PKDB101 - DB101 PKs that match the Simmons Definition Will doing a join on PKDB101 for a single value of PKSimmons return a recordset any time in the useful future? I have to join the PKDB101 to another table (and possibly two), then do WHERE stuff on that other table(s), then count the remaining records to determine which DB101 records match the Simmons definition. This essentially pre-selects just exactly the DB101 records that match the simmons definition, storing the selected DB101 PKs in the junction table. The way I do it right now is just dynamically build a SQL statement, joining everything required, doing a pretty complicated where and counting the resulting records. Rinse and repeat 220 times, once for each Simmons definition. The other question is as follows. I use a BigInt as the PK and make it a defined PK, i.e. the table ends up clustered on that PK. Should I have included the PDSimmons and PKDB101 in that PK so that those two fields are already indexed using the PK index? As it is now, I will be creating a separate (non clustered) index on just those two fields when done. Given that the records are being inserted in the table in PKSimmons order, i.e. an entire set of PK101 values inserted each iteration of my code, one set for each PKSimmons value. So had I used at least the PKSimmons in the (clustered) PK would have it made any difference in performance? Or when doing a join, will the server scan the clustered index anyway and just find all the groups by virtue of the fact that the PK is autonumber and all records for each PKSimmons value are contiguous? The things that keep me up at night. -- John W. Colby From accessd at shaw.ca Sat Aug 8 11:39:44 2015 From: accessd at shaw.ca (Jim Lawrence) Date: Sat, 8 Aug 2015 10:39:44 -0600 (MDT) Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: <55C5F2B7.3090806@gmail.com> Message-ID: <707670891.21223295.1439051984140.JavaMail.root@shaw.ca> I have no idea what the issue could be but the server rebooting while not under load gives a suggestion of the cause. It must be power related. Obviously it is not external power so it leaves only the internal power supply and related wiring. IMHO, that is the problem and I would replace that power hardware as soon as convenient. Jim ----- Original Message ----- From: "John W. Colby" To: "Access Developers discussion and problem solving" , "Discussion concerning MS SQL Server" Sent: Saturday, August 8, 2015 5:14:47 AM Subject: Re: [dba-SQLServer] [AccessD] A real puzzler For the last several days I have been flogging away at the system (performing real work), causing the system to stay active. All cores running, 75 (out of 80) GB used for SQL Server. No reboots during that entire time. And yet: Critical 8/4/2015 6:56:10 AM Kernel-Power 41 (63) Critical 8/4/2015 4:42:07 AM Kernel-Power 41 (63) Critical 8/3/2015 10:04:36 PM Kernel-Power 41 (63) Critical 8/3/2015 5:32:06 PM Kernel-Power 41 (63) Critical 8/3/2015 2:22:15 PM Kernel-Power 41 (63) Critical 8/2/2015 3:29:33 PM Kernel-Power 41 (63) Critical 8/2/2015 11:21:46 AM Kernel-Power 41 (63) Critical 8/2/2015 10:51:30 AM Kernel-Power 41 (63) Critical 8/2/2015 7:17:21 AM Kernel-Power 41 (63) Critical 7/31/2015 11:50:45 PM Kernel-Power 41 (63) Critical 7/31/2015 1:05:39 PM Kernel-Power 41 (63) Critical 7/30/2015 10:11:57 PM Kernel-Power 41 (63) Critical 7/30/2015 4:59:26 PM Kernel-Power 41 (63) Critical 7/29/2015 2:32:50 PM Kernel-Power 41 (63) Critical 7/28/2015 5:20:38 PM Kernel-Power 41 (63) Critical 7/28/2015 12:12:57 PM Kernel-Power 41 (63) Critical 7/28/2015 4:15:38 AM Kernel-Power 41 (63) Critical 7/28/2015 2:47:12 AM Kernel-Power 41 (63) WEIRD!!! Notice no pattern in number of events per day, nor time of day. There is no memory dump being created, the system just reboots as if the power was turned off and back on. When I was in the room with it, (several years ago) the system would beep as it rebooted. Here is the last such event: Log Name: System Source: Microsoft-Windows-Kernel-Power Date: 8/4/2015 6:56:10 AM Event ID: 41 Task Category: (63) Level: Critical Keywords: (2) User: SYSTEM Computer: Azul Description: The system has rebooted without cleanly shutting down first. This error could be caused if the system stopped responding, crashed, or lost power unexpectedly. Event Xml: 41 2 1 63 0 0x8000000000000002 270735 System Azul 0 0x0 0x0 0x0 0x0 false 0 John W. Colby On 8/8/2015 7:57 AM, James Button wrote: > Guest that! > > Re memory diagnostic - I have found with past experiences of accelerating > frequency of shut-downs, that the system didn't get a chance to record any > events. > And memory checks showed no problems - providing the rest of the system wasn't > being stressed. > > One system I found that removing a memory module - any of them stopped the > shutdowns, and I eventually 'bodged' the system by increasing the memory refresh > by a cycle. It was an old system and a 'new' memory module was, being old tech, > horrendously expensive > That worked for several years, and eventually management agreed the system was > too slow - as in users kept complaining about the system's response, so we were > actually allowed to BUY a new one! > > JimB > _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From Damien.Solodow at harrison.edu Sat Aug 8 11:42:32 2015 From: Damien.Solodow at harrison.edu (Damien Solodow) Date: Sat, 8 Aug 2015 16:42:32 +0000 Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: <707670891.21223295.1439051984140.JavaMail.root@shaw.ca> References: <55C5F2B7.3090806@gmail.com>, <707670891.21223295.1439051984140.JavaMail.root@shaw.ca> Message-ID: <04E26BCF73774949871E317AB0923ED2BBE76A9C@ENTEXMBX01.ibcschools.edu> Other possibility could be related to CPU power states; when it's under light load it tries to step down and it bounces. Should be able to disable cpu power saving in the bios, and possibly in Windows as well. DAMIEN SOLODOW Senior Systems Engineer 317.447.6033 (office) 317.447.6014 (fax) HARRISON COLLEGE ________________________________________ From: dba-SQLServer [dba-sqlserver-bounces at databaseadvisors.com] on behalf of Jim Lawrence [accessd at shaw.ca] Sent: Saturday, August 08, 2015 12:39 PM To: Discussion concerning MS SQL Server Subject: Re: [dba-SQLServer] [AccessD] A real puzzler I have no idea what the issue could be but the server rebooting while not under load gives a suggestion of the cause. It must be power related. Obviously it is not external power so it leaves only the internal power supply and related wiring. IMHO, that is the problem and I would replace that power hardware as soon as convenient. Jim ----- Original Message ----- From: "John W. Colby" To: "Access Developers discussion and problem solving" , "Discussion concerning MS SQL Server" Sent: Saturday, August 8, 2015 5:14:47 AM Subject: Re: [dba-SQLServer] [AccessD] A real puzzler For the last several days I have been flogging away at the system (performing real work), causing the system to stay active. All cores running, 75 (out of 80) GB used for SQL Server. No reboots during that entire time. And yet: Critical 8/4/2015 6:56:10 AM Kernel-Power 41 (63) Critical 8/4/2015 4:42:07 AM Kernel-Power 41 (63) Critical 8/3/2015 10:04:36 PM Kernel-Power 41 (63) Critical 8/3/2015 5:32:06 PM Kernel-Power 41 (63) Critical 8/3/2015 2:22:15 PM Kernel-Power 41 (63) Critical 8/2/2015 3:29:33 PM Kernel-Power 41 (63) Critical 8/2/2015 11:21:46 AM Kernel-Power 41 (63) Critical 8/2/2015 10:51:30 AM Kernel-Power 41 (63) Critical 8/2/2015 7:17:21 AM Kernel-Power 41 (63) Critical 7/31/2015 11:50:45 PM Kernel-Power 41 (63) Critical 7/31/2015 1:05:39 PM Kernel-Power 41 (63) Critical 7/30/2015 10:11:57 PM Kernel-Power 41 (63) Critical 7/30/2015 4:59:26 PM Kernel-Power 41 (63) Critical 7/29/2015 2:32:50 PM Kernel-Power 41 (63) Critical 7/28/2015 5:20:38 PM Kernel-Power 41 (63) Critical 7/28/2015 12:12:57 PM Kernel-Power 41 (63) Critical 7/28/2015 4:15:38 AM Kernel-Power 41 (63) Critical 7/28/2015 2:47:12 AM Kernel-Power 41 (63) WEIRD!!! Notice no pattern in number of events per day, nor time of day. There is no memory dump being created, the system just reboots as if the power was turned off and back on. When I was in the room with it, (several years ago) the system would beep as it rebooted. Here is the last such event: Log Name: System Source: Microsoft-Windows-Kernel-Power Date: 8/4/2015 6:56:10 AM Event ID: 41 Task Category: (63) Level: Critical Keywords: (2) User: SYSTEM Computer: Azul Description: The system has rebooted without cleanly shutting down first. This error could be caused if the system stopped responding, crashed, or lost power unexpectedly. Event Xml: 41 2 1 63 0 0x8000000000000002 270735 System Azul 0 0x0 0x0 0x0 0x0 false 0 John W. Colby On 8/8/2015 7:57 AM, James Button wrote: > Guest that! > > Re memory diagnostic - I have found with past experiences of accelerating > frequency of shut-downs, that the system didn't get a chance to record any > events. > And memory checks showed no problems - providing the rest of the system wasn't > being stressed. > > One system I found that removing a memory module - any of them stopped the > shutdowns, and I eventually 'bodged' the system by increasing the memory refresh > by a cycle. It was an old system and a 'new' memory module was, being old tech, > horrendously expensive > That worked for several years, and eventually management agreed the system was > too slow - as in users kept complaining about the system's response, so we were > actually allowed to BUY a new one! > > JimB > _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From paul.hartland at googlemail.com Sat Aug 8 11:43:36 2015 From: paul.hartland at googlemail.com (Paul Hartland) Date: Sat, 8 Aug 2015 17:43:36 +0100 Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: <707670891.21223295.1439051984140.JavaMail.root@shaw.ca> References: <55C5F2B7.3090806@gmail.com> <707670891.21223295.1439051984140.JavaMail.root@shaw.ca> Message-ID: Could i put an idea forward that i dont think has been mentioned yet, it looks to be a power problem, have you checked/replaced the leads etc On 8 Aug 2015 17:40, "Jim Lawrence" wrote: > I have no idea what the issue could be but the server rebooting while not > under load gives a suggestion of the cause. > > It must be power related. Obviously it is not external power so it leaves > only the internal power supply and related wiring. IMHO, that is the > problem and I would replace that power hardware as soon as convenient. > > Jim > > ----- Original Message ----- > From: "John W. Colby" > To: "Access Developers discussion and problem solving" < > accessd at databaseadvisors.com>, "Discussion concerning MS SQL Server" < > dba-sqlserver at databaseadvisors.com> > Sent: Saturday, August 8, 2015 5:14:47 AM > Subject: Re: [dba-SQLServer] [AccessD] A real puzzler > > For the last several days I have been flogging away at the system > (performing real work), causing the system to stay active. All cores > running, 75 (out of 80) GB used for SQL Server. No reboots during that > entire time. And yet: > > Critical 8/4/2015 6:56:10 AM Kernel-Power 41 (63) > Critical 8/4/2015 4:42:07 AM Kernel-Power 41 (63) > Critical 8/3/2015 10:04:36 PM Kernel-Power 41 (63) > Critical 8/3/2015 5:32:06 PM Kernel-Power 41 (63) > Critical 8/3/2015 2:22:15 PM Kernel-Power 41 (63) > Critical 8/2/2015 3:29:33 PM Kernel-Power 41 (63) > Critical 8/2/2015 11:21:46 AM Kernel-Power 41 (63) > Critical 8/2/2015 10:51:30 AM Kernel-Power 41 (63) > Critical 8/2/2015 7:17:21 AM Kernel-Power 41 (63) > Critical 7/31/2015 11:50:45 PM Kernel-Power 41 (63) > Critical 7/31/2015 1:05:39 PM Kernel-Power 41 (63) > Critical 7/30/2015 10:11:57 PM Kernel-Power 41 (63) > Critical 7/30/2015 4:59:26 PM Kernel-Power 41 (63) > Critical 7/29/2015 2:32:50 PM Kernel-Power 41 (63) > Critical 7/28/2015 5:20:38 PM Kernel-Power 41 (63) > Critical 7/28/2015 12:12:57 PM Kernel-Power 41 (63) > Critical 7/28/2015 4:15:38 AM Kernel-Power 41 (63) > Critical 7/28/2015 2:47:12 AM Kernel-Power 41 (63) > > WEIRD!!! > > Notice no pattern in number of events per day, nor time of day. There is > no memory dump being created, the system just reboots as if the power > was turned off and back on. When I was in the room with it, (several > years ago) the system would beep as it rebooted. > > Here is the last such event: > > Log Name: System > Source: Microsoft-Windows-Kernel-Power > Date: 8/4/2015 6:56:10 AM > Event ID: 41 > Task Category: (63) > Level: Critical > Keywords: (2) > User: SYSTEM > Computer: Azul > Description: > The system has rebooted without cleanly shutting down first. This error > could be caused if the system stopped responding, crashed, or lost power > unexpectedly. > Event Xml: > > > Guid="{331C3B3A-2005-44C2-AC5E-77220C37D6B4}" /> > 41 > 2 > 1 > 63 > 0 > 0x8000000000000002 > > 270735 > > > System > Azul > > > > 0 > 0x0 > 0x0 > 0x0 > 0x0 > false > 0 > > > > John W. Colby > > On 8/8/2015 7:57 AM, James Button wrote: > > Guest that! > > > > Re memory diagnostic - I have found with past experiences of accelerating > > frequency of shut-downs, that the system didn't get a chance to record > any > > events. > > And memory checks showed no problems - providing the rest of the system > wasn't > > being stressed. > > > > One system I found that removing a memory module - any of them stopped > the > > shutdowns, and I eventually 'bodged' the system by increasing the memory > refresh > > by a cycle. It was an old system and a 'new' memory module was, being > old tech, > > horrendously expensive > > That worked for several years, and eventually management agreed the > system was > > too slow - as in users kept complaining about the system's response, so > we were > > actually allowed to BUY a new one! > > > > JimB > > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From accessd at shaw.ca Sat Aug 8 11:47:56 2015 From: accessd at shaw.ca (Jim Lawrence) Date: Sat, 8 Aug 2015 10:47:56 -0600 (MDT) Subject: [dba-SQLServer] 2.45 billion rows and counting was PK Cast fails In-Reply-To: <55C6197C.4010609@gmail.com> Message-ID: <1467301789.21226471.1439052476171.JavaMail.root@shaw.ca> I think you have crossed the line into "big data" world. Hadoop and friends. Jim ----- Original Message ----- From: "John W. Colby" To: "Access Developers discussion and problem solving" , "Discussion concerning MS SQL Server" Sent: Saturday, August 8, 2015 8:00:12 AM Subject: [dba-SQLServer] 2.45 billion rows and counting was PK Cast fails The Simmons table I am building now has 2.5 billion rows. The stored procedure is running a "loop" iterating through a table of 228 Simmons demographics definitions, pulling PKs from DB101 (225 million possible PKs) one at a time, and inserting them into the junction table mentioned in the previous email. Sometime last night the log file ran out of room (filled up the log disk) and stopped the process, at Simmons Def 70. Luckily I am writing the individual sql statements out as each one is executed so I could see where it stopped. I cleaned a bunch of very large log files off and restarted this morning. While I can see the total storage (records) in the junction table, what I can't do is get a Max() of the Simmons PK in the junction table to see where I am in the process. I was hoping that info would be in metadata somewhere but apparently not, it seems that it has to scan through the table looking for the max() value. While I haven't let it run in a query window, trying to do that in a view timed out for obvious reasons. And so I wait for completion. I would estimate the junction table will contain somewhere north of 10 billion records when done. I have never even come close to this quantity of records in a single table so I am still unsure whether this will even make things faster. Assuming an index on two fields: PKSimmons - PKs for each of 220 Simmons Definitions PKDB101 - DB101 PKs that match the Simmons Definition Will doing a join on PKDB101 for a single value of PKSimmons return a recordset any time in the useful future? I have to join the PKDB101 to another table (and possibly two), then do WHERE stuff on that other table(s), then count the remaining records to determine which DB101 records match the Simmons definition. This essentially pre-selects just exactly the DB101 records that match the simmons definition, storing the selected DB101 PKs in the junction table. The way I do it right now is just dynamically build a SQL statement, joining everything required, doing a pretty complicated where and counting the resulting records. Rinse and repeat 220 times, once for each Simmons definition. The other question is as follows. I use a BigInt as the PK and make it a defined PK, i.e. the table ends up clustered on that PK. Should I have included the PDSimmons and PKDB101 in that PK so that those two fields are already indexed using the PK index? As it is now, I will be creating a separate (non clustered) index on just those two fields when done. Given that the records are being inserted in the table in PKSimmons order, i.e. an entire set of PK101 values inserted each iteration of my code, one set for each PKSimmons value. So had I used at least the PKSimmons in the (clustered) PK would have it made any difference in performance? Or when doing a join, will the server scan the clustered index anyway and just find all the groups by virtue of the fact that the PK is autonumber and all records for each PKSimmons value are contiguous? The things that keep me up at night. -- John W. Colby _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From accessd at shaw.ca Sat Aug 8 12:31:28 2015 From: accessd at shaw.ca (Jim Lawrence) Date: Sat, 8 Aug 2015 11:31:28 -0600 (MDT) Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: <04E26BCF73774949871E317AB0923ED2BBE76A9C@ENTEXMBX01.ibcschools.edu> Message-ID: <1966571744.21246095.1439055088556.JavaMail.root@shaw.ca> Hi Damien: Good call. ...But then what type of CPU are we taking about...there are very limited number of suppliers with 90 plus percent being Intel and AMD. A bad CPU and motherboard combination? IMHO, it is still most likely internal power supply related; like in some recent HP servers. (We are assuming that the computer has not been over-clock as all guarantees, then fly right out the window.) Jim ----- Original Message ----- From: "Damien Solodow" To: "Discussion concerning MS SQL Server" Sent: Saturday, August 8, 2015 9:42:32 AM Subject: Re: [dba-SQLServer] [AccessD] A real puzzler Other possibility could be related to CPU power states; when it's under light load it tries to step down and it bounces. Should be able to disable cpu power saving in the bios, and possibly in Windows as well. DAMIEN SOLODOW Senior Systems Engineer 317.447.6033 (office) 317.447.6014 (fax) HARRISON COLLEGE ________________________________________ From: dba-SQLServer [dba-sqlserver-bounces at databaseadvisors.com] on behalf of Jim Lawrence [accessd at shaw.ca] Sent: Saturday, August 08, 2015 12:39 PM To: Discussion concerning MS SQL Server Subject: Re: [dba-SQLServer] [AccessD] A real puzzler I have no idea what the issue could be but the server rebooting while not under load gives a suggestion of the cause. It must be power related. Obviously it is not external power so it leaves only the internal power supply and related wiring. IMHO, that is the problem and I would replace that power hardware as soon as convenient. Jim ----- Original Message ----- From: "John W. Colby" To: "Access Developers discussion and problem solving" , "Discussion concerning MS SQL Server" Sent: Saturday, August 8, 2015 5:14:47 AM Subject: Re: [dba-SQLServer] [AccessD] A real puzzler For the last several days I have been flogging away at the system (performing real work), causing the system to stay active. All cores running, 75 (out of 80) GB used for SQL Server. No reboots during that entire time. And yet: Critical 8/4/2015 6:56:10 AM Kernel-Power 41 (63) Critical 8/4/2015 4:42:07 AM Kernel-Power 41 (63) Critical 8/3/2015 10:04:36 PM Kernel-Power 41 (63) Critical 8/3/2015 5:32:06 PM Kernel-Power 41 (63) Critical 8/3/2015 2:22:15 PM Kernel-Power 41 (63) Critical 8/2/2015 3:29:33 PM Kernel-Power 41 (63) Critical 8/2/2015 11:21:46 AM Kernel-Power 41 (63) Critical 8/2/2015 10:51:30 AM Kernel-Power 41 (63) Critical 8/2/2015 7:17:21 AM Kernel-Power 41 (63) Critical 7/31/2015 11:50:45 PM Kernel-Power 41 (63) Critical 7/31/2015 1:05:39 PM Kernel-Power 41 (63) Critical 7/30/2015 10:11:57 PM Kernel-Power 41 (63) Critical 7/30/2015 4:59:26 PM Kernel-Power 41 (63) Critical 7/29/2015 2:32:50 PM Kernel-Power 41 (63) Critical 7/28/2015 5:20:38 PM Kernel-Power 41 (63) Critical 7/28/2015 12:12:57 PM Kernel-Power 41 (63) Critical 7/28/2015 4:15:38 AM Kernel-Power 41 (63) Critical 7/28/2015 2:47:12 AM Kernel-Power 41 (63) WEIRD!!! Notice no pattern in number of events per day, nor time of day. There is no memory dump being created, the system just reboots as if the power was turned off and back on. When I was in the room with it, (several years ago) the system would beep as it rebooted. Here is the last such event: Log Name: System Source: Microsoft-Windows-Kernel-Power Date: 8/4/2015 6:56:10 AM Event ID: 41 Task Category: (63) Level: Critical Keywords: (2) User: SYSTEM Computer: Azul Description: The system has rebooted without cleanly shutting down first. This error could be caused if the system stopped responding, crashed, or lost power unexpectedly. Event Xml: 41 2 1 63 0 0x8000000000000002 270735 System Azul 0 0x0 0x0 0x0 0x0 false 0 John W. Colby On 8/8/2015 7:57 AM, James Button wrote: > Guest that! > > Re memory diagnostic - I have found with past experiences of accelerating > frequency of shut-downs, that the system didn't get a chance to record any > events. > And memory checks showed no problems - providing the rest of the system wasn't > being stressed. > > One system I found that removing a memory module - any of them stopped the > shutdowns, and I eventually 'bodged' the system by increasing the memory refresh > by a cycle. It was an old system and a 'new' memory module was, being old tech, > horrendously expensive > That worked for several years, and eventually management agreed the system was > too slow - as in users kept complaining about the system's response, so we were > actually allowed to BUY a new one! > > JimB > _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at gmail.com Sat Aug 8 12:51:35 2015 From: jwcolby at gmail.com (John W. Colby) Date: Sat, 8 Aug 2015 13:51:35 -0400 Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: <04E26BCF73774949871E317AB0923ED2BBE76A9C@ENTEXMBX01.ibcschools.edu> References: <55C5F2B7.3090806@gmail.com> <707670891.21223295.1439051984140.JavaMail.root@shaw.ca> <04E26BCF73774949871E317AB0923ED2BBE76A9C@ENTEXMBX01.ibcschools.edu> Message-ID: <55C641A7.5030405@gmail.com> Interesting idea! I will see if Windows itself can disable it. John W. Colby On 8/8/2015 12:42 PM, Damien Solodow wrote: > Other possibility could be related to CPU power states; when it's under light load it tries to step down and it bounces. Should be able to disable cpu power saving in the bios, and possibly in Windows as well. > > DAMIEN SOLODOW > Senior Systems Engineer > 317.447.6033 (office) > 317.447.6014 (fax) > HARRISON COLLEGE > From jwcolby at gmail.com Sat Aug 8 12:52:34 2015 From: jwcolby at gmail.com (John W. Colby) Date: Sat, 8 Aug 2015 13:52:34 -0400 Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: References: <55C5F2B7.3090806@gmail.com> <707670891.21223295.1439051984140.JavaMail.root@shaw.ca> Message-ID: <55C641E2.5020101@gmail.com> I have replaced the power supply itself (bigger) and the motherboard itself. John W. Colby On 8/8/2015 12:43 PM, Paul Hartland wrote: > Could i put an idea forward that i dont think has been mentioned yet, it > looks to be a power problem, have you checked/replaced the leads etc > On 8 Aug 2015 17:40, "Jim Lawrence" wrote: > >> I have no idea what the issue could be but the server rebooting while not >> under load gives a suggestion of the cause. >> >> It must be power related. Obviously it is not external power so it leaves >> only the internal power supply and related wiring. IMHO, that is the >> problem and I would replace that power hardware as soon as convenient. >> >> Jim From jwcolby at gmail.com Sat Aug 8 13:16:12 2015 From: jwcolby at gmail.com (John W. Colby) Date: Sat, 8 Aug 2015 14:16:12 -0400 Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: <1966571744.21246095.1439055088556.JavaMail.root@shaw.ca> References: <1966571744.21246095.1439055088556.JavaMail.root@shaw.ca> Message-ID: <55C6476C.108@gmail.com> This is a custom built server (by me). The motherboard does not allow overclocking - it is a real server motherboard. John W. Colby On 8/8/2015 1:31 PM, Jim Lawrence wrote: > Hi Damien: > > Good call. > > ...But then what type of CPU are we taking about...there are very limited number of suppliers with 90 plus percent being Intel and AMD. A bad CPU and motherboard combination? > > IMHO, it is still most likely internal power supply related; like in some recent HP servers. (We are assuming that the computer has not been over-clock as all guarantees, then fly right out the window.) > > Jim > > ----- Original Message ----- > From: "Damien Solodow" > To: "Discussion concerning MS SQL Server" > Sent: Saturday, August 8, 2015 9:42:32 AM > Subject: Re: [dba-SQLServer] [AccessD] A real puzzler > > Other possibility could be related to CPU power states; when it's under light load it tries to step down and it bounces. Should be able to disable cpu power saving in the bios, and possibly in Windows as well. > > DAMIEN SOLODOW > Senior Systems Engineer > 317.447.6033 (office) > 317.447.6014 (fax) > HARRISON COLLEGE > > ________________________________________ > From: dba-SQLServer [dba-sqlserver-bounces at databaseadvisors.com] on behalf of Jim Lawrence [accessd at shaw.ca] > Sent: Saturday, August 08, 2015 12:39 PM > To: Discussion concerning MS SQL Server > Subject: Re: [dba-SQLServer] [AccessD] A real puzzler > > I have no idea what the issue could be but the server rebooting while not under load gives a suggestion of the cause. > > It must be power related. Obviously it is not external power so it leaves only the internal power supply and related wiring. IMHO, that is the problem and I would replace that power hardware as soon as convenient. > > Jim > > ----- Original Message ----- > From: "John W. Colby" > To: "Access Developers discussion and problem solving" , "Discussion concerning MS SQL Server" > Sent: Saturday, August 8, 2015 5:14:47 AM > Subject: Re: [dba-SQLServer] [AccessD] A real puzzler > > For the last several days I have been flogging away at the system > (performing real work), causing the system to stay active. All cores > running, 75 (out of 80) GB used for SQL Server. No reboots during that > entire time. And yet: > > Critical 8/4/2015 6:56:10 AM Kernel-Power 41 (63) > Critical 8/4/2015 4:42:07 AM Kernel-Power 41 (63) > Critical 8/3/2015 10:04:36 PM Kernel-Power 41 (63) > Critical 8/3/2015 5:32:06 PM Kernel-Power 41 (63) > Critical 8/3/2015 2:22:15 PM Kernel-Power 41 (63) > Critical 8/2/2015 3:29:33 PM Kernel-Power 41 (63) > Critical 8/2/2015 11:21:46 AM Kernel-Power 41 (63) > Critical 8/2/2015 10:51:30 AM Kernel-Power 41 (63) > Critical 8/2/2015 7:17:21 AM Kernel-Power 41 (63) > Critical 7/31/2015 11:50:45 PM Kernel-Power 41 (63) > Critical 7/31/2015 1:05:39 PM Kernel-Power 41 (63) > Critical 7/30/2015 10:11:57 PM Kernel-Power 41 (63) > Critical 7/30/2015 4:59:26 PM Kernel-Power 41 (63) > Critical 7/29/2015 2:32:50 PM Kernel-Power 41 (63) > Critical 7/28/2015 5:20:38 PM Kernel-Power 41 (63) > Critical 7/28/2015 12:12:57 PM Kernel-Power 41 (63) > Critical 7/28/2015 4:15:38 AM Kernel-Power 41 (63) > Critical 7/28/2015 2:47:12 AM Kernel-Power 41 (63) > > WEIRD!!! > > Notice no pattern in number of events per day, nor time of day. There is > no memory dump being created, the system just reboots as if the power > was turned off and back on. When I was in the room with it, (several > years ago) the system would beep as it rebooted. > > Here is the last such event: > > Log Name: System > Source: Microsoft-Windows-Kernel-Power > Date: 8/4/2015 6:56:10 AM > Event ID: 41 > Task Category: (63) > Level: Critical > Keywords: (2) > User: SYSTEM > Computer: Azul > Description: > The system has rebooted without cleanly shutting down first. This error > could be caused if the system stopped responding, crashed, or lost power > unexpectedly. > Event Xml: > > > Guid="{331C3B3A-2005-44C2-AC5E-77220C37D6B4}" /> > 41 > 2 > 1 > 63 > 0 > 0x8000000000000002 > > 270735 > > > System > Azul > > > > 0 > 0x0 > 0x0 > 0x0 > 0x0 > false > 0 > > > > John W. Colby > > On 8/8/2015 7:57 AM, James Button wrote: >> Guest that! >> >> Re memory diagnostic - I have found with past experiences of accelerating >> frequency of shut-downs, that the system didn't get a chance to record any >> events. >> And memory checks showed no problems - providing the rest of the system wasn't >> being stressed. >> >> One system I found that removing a memory module - any of them stopped the >> shutdowns, and I eventually 'bodged' the system by increasing the memory refresh >> by a cycle. It was an old system and a 'new' memory module was, being old tech, >> horrendously expensive >> That worked for several years, and eventually management agreed the system was >> too slow - as in users kept complaining about the system's response, so we were >> actually allowed to BUY a new one! >> >> JimB >> > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From jwcolby at gmail.com Sat Aug 8 13:25:18 2015 From: jwcolby at gmail.com (John W. Colby) Date: Sat, 8 Aug 2015 14:25:18 -0400 Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: <1966571744.21246095.1439055088556.JavaMail.root@shaw.ca> References: <1966571744.21246095.1439055088556.JavaMail.root@shaw.ca> Message-ID: <55C6498E.1@gmail.com> I have (just now) set the power saving states to keep the processor on max, never shut down disks etc. John W. Colby On 8/8/2015 1:31 PM, Jim Lawrence wrote: > Hi Damien: > > Good call. > > ...But then what type of CPU are we taking about...there are very limited number of suppliers with 90 plus percent being Intel and AMD. A bad CPU and motherboard combination? > > IMHO, it is still most likely internal power supply related; like in some recent HP servers. (We are assuming that the computer has not been over-clock as all guarantees, then fly right out the window.) > > Jim > > ----- Original Message ----- > From: "Damien Solodow" > To: "Discussion concerning MS SQL Server" > Sent: Saturday, August 8, 2015 9:42:32 AM > Subject: Re: [dba-SQLServer] [AccessD] A real puzzler > > Other possibility could be related to CPU power states; when it's under light load it tries to step down and it bounces. Should be able to disable cpu power saving in the bios, and possibly in Windows as well. > > DAMIEN SOLODOW > Senior Systems Engineer > 317.447.6033 (office) > 317.447.6014 (fax) > HARRISON COLLEGE > > ________________________________________ > From: dba-SQLServer [dba-sqlserver-bounces at databaseadvisors.com] on behalf of Jim Lawrence [accessd at shaw.ca] > Sent: Saturday, August 08, 2015 12:39 PM > To: Discussion concerning MS SQL Server > Subject: Re: [dba-SQLServer] [AccessD] A real puzzler > > I have no idea what the issue could be but the server rebooting while not under load gives a suggestion of the cause. > > It must be power related. Obviously it is not external power so it leaves only the internal power supply and related wiring. IMHO, that is the problem and I would replace that power hardware as soon as convenient. > > Jim > > ----- Original Message ----- > From: "John W. Colby" > To: "Access Developers discussion and problem solving" , "Discussion concerning MS SQL Server" > Sent: Saturday, August 8, 2015 5:14:47 AM > Subject: Re: [dba-SQLServer] [AccessD] A real puzzler > > For the last several days I have been flogging away at the system > (performing real work), causing the system to stay active. All cores > running, 75 (out of 80) GB used for SQL Server. No reboots during that > entire time. And yet: > > Critical 8/4/2015 6:56:10 AM Kernel-Power 41 (63) > Critical 8/4/2015 4:42:07 AM Kernel-Power 41 (63) > Critical 8/3/2015 10:04:36 PM Kernel-Power 41 (63) > Critical 8/3/2015 5:32:06 PM Kernel-Power 41 (63) > Critical 8/3/2015 2:22:15 PM Kernel-Power 41 (63) > Critical 8/2/2015 3:29:33 PM Kernel-Power 41 (63) > Critical 8/2/2015 11:21:46 AM Kernel-Power 41 (63) > Critical 8/2/2015 10:51:30 AM Kernel-Power 41 (63) > Critical 8/2/2015 7:17:21 AM Kernel-Power 41 (63) > Critical 7/31/2015 11:50:45 PM Kernel-Power 41 (63) > Critical 7/31/2015 1:05:39 PM Kernel-Power 41 (63) > Critical 7/30/2015 10:11:57 PM Kernel-Power 41 (63) > Critical 7/30/2015 4:59:26 PM Kernel-Power 41 (63) > Critical 7/29/2015 2:32:50 PM Kernel-Power 41 (63) > Critical 7/28/2015 5:20:38 PM Kernel-Power 41 (63) > Critical 7/28/2015 12:12:57 PM Kernel-Power 41 (63) > Critical 7/28/2015 4:15:38 AM Kernel-Power 41 (63) > Critical 7/28/2015 2:47:12 AM Kernel-Power 41 (63) > > WEIRD!!! > > Notice no pattern in number of events per day, nor time of day. There is > no memory dump being created, the system just reboots as if the power > was turned off and back on. When I was in the room with it, (several > years ago) the system would beep as it rebooted. > > Here is the last such event: > > Log Name: System > Source: Microsoft-Windows-Kernel-Power > Date: 8/4/2015 6:56:10 AM > Event ID: 41 > Task Category: (63) > Level: Critical > Keywords: (2) > User: SYSTEM > Computer: Azul > Description: > The system has rebooted without cleanly shutting down first. This error > could be caused if the system stopped responding, crashed, or lost power > unexpectedly. > Event Xml: > > > Guid="{331C3B3A-2005-44C2-AC5E-77220C37D6B4}" /> > 41 > 2 > 1 > 63 > 0 > 0x8000000000000002 > > 270735 > > > System > Azul > > > > 0 > 0x0 > 0x0 > 0x0 > 0x0 > false > 0 > > > > John W. Colby > > On 8/8/2015 7:57 AM, James Button wrote: >> Guest that! >> >> Re memory diagnostic - I have found with past experiences of accelerating >> frequency of shut-downs, that the system didn't get a chance to record any >> events. >> And memory checks showed no problems - providing the rest of the system wasn't >> being stressed. >> >> One system I found that removing a memory module - any of them stopped the >> shutdowns, and I eventually 'bodged' the system by increasing the memory refresh >> by a cycle. It was an old system and a 'new' memory module was, being old tech, >> horrendously expensive >> That worked for several years, and eventually management agreed the system was >> too slow - as in users kept complaining about the system's response, so we were >> actually allowed to BUY a new one! >> >> JimB >> > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From accessd at shaw.ca Sat Aug 8 14:12:06 2015 From: accessd at shaw.ca (Jim Lawrence) Date: Sat, 8 Aug 2015 13:12:06 -0600 (MDT) Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: <55C6476C.108@gmail.com> Message-ID: <538439977.21281534.1439061126354.JavaMail.root@shaw.ca> Just a thought and everything that can be eliminated is a step towards a solution. If a backup server with similar features could be tested on then the possibility of software issues could be either isolated or discovered. Jim ----- Original Message ----- From: "John W. Colby" To: "Discussion concerning MS SQL Server" Sent: Saturday, August 8, 2015 11:16:12 AM Subject: Re: [dba-SQLServer] [AccessD] A real puzzler This is a custom built server (by me). The motherboard does not allow overclocking - it is a real server motherboard. John W. Colby On 8/8/2015 1:31 PM, Jim Lawrence wrote: > Hi Damien: > > Good call. > > ...But then what type of CPU are we taking about...there are very limited number of suppliers with 90 plus percent being Intel and AMD. A bad CPU and motherboard combination? > > IMHO, it is still most likely internal power supply related; like in some recent HP servers. (We are assuming that the computer has not been over-clock as all guarantees, then fly right out the window.) > > Jim > > ----- Original Message ----- > From: "Damien Solodow" > To: "Discussion concerning MS SQL Server" > Sent: Saturday, August 8, 2015 9:42:32 AM > Subject: Re: [dba-SQLServer] [AccessD] A real puzzler > > Other possibility could be related to CPU power states; when it's under light load it tries to step down and it bounces. Should be able to disable cpu power saving in the bios, and possibly in Windows as well. > > DAMIEN SOLODOW > Senior Systems Engineer > 317.447.6033 (office) > 317.447.6014 (fax) > HARRISON COLLEGE > > ________________________________________ > From: dba-SQLServer [dba-sqlserver-bounces at databaseadvisors.com] on behalf of Jim Lawrence [accessd at shaw.ca] > Sent: Saturday, August 08, 2015 12:39 PM > To: Discussion concerning MS SQL Server > Subject: Re: [dba-SQLServer] [AccessD] A real puzzler > > I have no idea what the issue could be but the server rebooting while not under load gives a suggestion of the cause. > > It must be power related. Obviously it is not external power so it leaves only the internal power supply and related wiring. IMHO, that is the problem and I would replace that power hardware as soon as convenient. > > Jim > > ----- Original Message ----- > From: "John W. Colby" > To: "Access Developers discussion and problem solving" , "Discussion concerning MS SQL Server" > Sent: Saturday, August 8, 2015 5:14:47 AM > Subject: Re: [dba-SQLServer] [AccessD] A real puzzler > > For the last several days I have been flogging away at the system > (performing real work), causing the system to stay active. All cores > running, 75 (out of 80) GB used for SQL Server. No reboots during that > entire time. And yet: > > Critical 8/4/2015 6:56:10 AM Kernel-Power 41 (63) > Critical 8/4/2015 4:42:07 AM Kernel-Power 41 (63) > Critical 8/3/2015 10:04:36 PM Kernel-Power 41 (63) > Critical 8/3/2015 5:32:06 PM Kernel-Power 41 (63) > Critical 8/3/2015 2:22:15 PM Kernel-Power 41 (63) > Critical 8/2/2015 3:29:33 PM Kernel-Power 41 (63) > Critical 8/2/2015 11:21:46 AM Kernel-Power 41 (63) > Critical 8/2/2015 10:51:30 AM Kernel-Power 41 (63) > Critical 8/2/2015 7:17:21 AM Kernel-Power 41 (63) > Critical 7/31/2015 11:50:45 PM Kernel-Power 41 (63) > Critical 7/31/2015 1:05:39 PM Kernel-Power 41 (63) > Critical 7/30/2015 10:11:57 PM Kernel-Power 41 (63) > Critical 7/30/2015 4:59:26 PM Kernel-Power 41 (63) > Critical 7/29/2015 2:32:50 PM Kernel-Power 41 (63) > Critical 7/28/2015 5:20:38 PM Kernel-Power 41 (63) > Critical 7/28/2015 12:12:57 PM Kernel-Power 41 (63) > Critical 7/28/2015 4:15:38 AM Kernel-Power 41 (63) > Critical 7/28/2015 2:47:12 AM Kernel-Power 41 (63) > > WEIRD!!! > > Notice no pattern in number of events per day, nor time of day. There is > no memory dump being created, the system just reboots as if the power > was turned off and back on. When I was in the room with it, (several > years ago) the system would beep as it rebooted. > > Here is the last such event: > > Log Name: System > Source: Microsoft-Windows-Kernel-Power > Date: 8/4/2015 6:56:10 AM > Event ID: 41 > Task Category: (63) > Level: Critical > Keywords: (2) > User: SYSTEM > Computer: Azul > Description: > The system has rebooted without cleanly shutting down first. This error > could be caused if the system stopped responding, crashed, or lost power > unexpectedly. > Event Xml: > > > Guid="{331C3B3A-2005-44C2-AC5E-77220C37D6B4}" /> > 41 > 2 > 1 > 63 > 0 > 0x8000000000000002 > > 270735 > > > System > Azul > > > > 0 > 0x0 > 0x0 > 0x0 > 0x0 > false > 0 > > > > John W. Colby > > On 8/8/2015 7:57 AM, James Button wrote: >> Guest that! >> >> Re memory diagnostic - I have found with past experiences of accelerating >> frequency of shut-downs, that the system didn't get a chance to record any >> events. >> And memory checks showed no problems - providing the rest of the system wasn't >> being stressed. >> >> One system I found that removing a memory module - any of them stopped the >> shutdowns, and I eventually 'bodged' the system by increasing the memory refresh >> by a cycle. It was an old system and a 'new' memory module was, being old tech, >> horrendously expensive >> That worked for several years, and eventually management agreed the system was >> too slow - as in users kept complaining about the system's response, so we were >> actually allowed to BUY a new one! >> >> JimB >> > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From accessd at shaw.ca Sat Aug 8 14:22:39 2015 From: accessd at shaw.ca (Jim Lawrence) Date: Sat, 8 Aug 2015 13:22:39 -0600 (MDT) Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: <55C6498E.1@gmail.com> Message-ID: <1476721061.21284731.1439061759933.JavaMail.root@shaw.ca> There are many posts in Google discussing computer crashes due directly or indirectly to sleep and power-saving modes. Resolutions that were found seemed to congregate around issues with particular SS drives and more broadly, around badly written or outdated drivers. Jim ----- Original Message ----- From: "John W. Colby" To: "Discussion concerning MS SQL Server" Sent: Saturday, August 8, 2015 11:25:18 AM Subject: Re: [dba-SQLServer] [AccessD] A real puzzler I have (just now) set the power saving states to keep the processor on max, never shut down disks etc. John W. Colby On 8/8/2015 1:31 PM, Jim Lawrence wrote: > Hi Damien: > > Good call. > > ...But then what type of CPU are we taking about...there are very limited number of suppliers with 90 plus percent being Intel and AMD. A bad CPU and motherboard combination? > > IMHO, it is still most likely internal power supply related; like in some recent HP servers. (We are assuming that the computer has not been over-clock as all guarantees, then fly right out the window.) > > Jim > > ----- Original Message ----- > From: "Damien Solodow" > To: "Discussion concerning MS SQL Server" > Sent: Saturday, August 8, 2015 9:42:32 AM > Subject: Re: [dba-SQLServer] [AccessD] A real puzzler > > Other possibility could be related to CPU power states; when it's under light load it tries to step down and it bounces. Should be able to disable cpu power saving in the bios, and possibly in Windows as well. > > DAMIEN SOLODOW > Senior Systems Engineer > 317.447.6033 (office) > 317.447.6014 (fax) > HARRISON COLLEGE > > ________________________________________ > From: dba-SQLServer [dba-sqlserver-bounces at databaseadvisors.com] on behalf of Jim Lawrence [accessd at shaw.ca] > Sent: Saturday, August 08, 2015 12:39 PM > To: Discussion concerning MS SQL Server > Subject: Re: [dba-SQLServer] [AccessD] A real puzzler > > I have no idea what the issue could be but the server rebooting while not under load gives a suggestion of the cause. > > It must be power related. Obviously it is not external power so it leaves only the internal power supply and related wiring. IMHO, that is the problem and I would replace that power hardware as soon as convenient. > > Jim > > ----- Original Message ----- > From: "John W. Colby" > To: "Access Developers discussion and problem solving" , "Discussion concerning MS SQL Server" > Sent: Saturday, August 8, 2015 5:14:47 AM > Subject: Re: [dba-SQLServer] [AccessD] A real puzzler > > For the last several days I have been flogging away at the system > (performing real work), causing the system to stay active. All cores > running, 75 (out of 80) GB used for SQL Server. No reboots during that > entire time. And yet: > > Critical 8/4/2015 6:56:10 AM Kernel-Power 41 (63) > Critical 8/4/2015 4:42:07 AM Kernel-Power 41 (63) > Critical 8/3/2015 10:04:36 PM Kernel-Power 41 (63) > Critical 8/3/2015 5:32:06 PM Kernel-Power 41 (63) > Critical 8/3/2015 2:22:15 PM Kernel-Power 41 (63) > Critical 8/2/2015 3:29:33 PM Kernel-Power 41 (63) > Critical 8/2/2015 11:21:46 AM Kernel-Power 41 (63) > Critical 8/2/2015 10:51:30 AM Kernel-Power 41 (63) > Critical 8/2/2015 7:17:21 AM Kernel-Power 41 (63) > Critical 7/31/2015 11:50:45 PM Kernel-Power 41 (63) > Critical 7/31/2015 1:05:39 PM Kernel-Power 41 (63) > Critical 7/30/2015 10:11:57 PM Kernel-Power 41 (63) > Critical 7/30/2015 4:59:26 PM Kernel-Power 41 (63) > Critical 7/29/2015 2:32:50 PM Kernel-Power 41 (63) > Critical 7/28/2015 5:20:38 PM Kernel-Power 41 (63) > Critical 7/28/2015 12:12:57 PM Kernel-Power 41 (63) > Critical 7/28/2015 4:15:38 AM Kernel-Power 41 (63) > Critical 7/28/2015 2:47:12 AM Kernel-Power 41 (63) > > WEIRD!!! > > Notice no pattern in number of events per day, nor time of day. There is > no memory dump being created, the system just reboots as if the power > was turned off and back on. When I was in the room with it, (several > years ago) the system would beep as it rebooted. > > Here is the last such event: > > Log Name: System > Source: Microsoft-Windows-Kernel-Power > Date: 8/4/2015 6:56:10 AM > Event ID: 41 > Task Category: (63) > Level: Critical > Keywords: (2) > User: SYSTEM > Computer: Azul > Description: > The system has rebooted without cleanly shutting down first. This error > could be caused if the system stopped responding, crashed, or lost power > unexpectedly. > Event Xml: > > > Guid="{331C3B3A-2005-44C2-AC5E-77220C37D6B4}" /> > 41 > 2 > 1 > 63 > 0 > 0x8000000000000002 > > 270735 > > > System > Azul > > > > 0 > 0x0 > 0x0 > 0x0 > 0x0 > false > 0 > > > > John W. Colby > > On 8/8/2015 7:57 AM, James Button wrote: >> Guest that! >> >> Re memory diagnostic - I have found with past experiences of accelerating >> frequency of shut-downs, that the system didn't get a chance to record any >> events. >> And memory checks showed no problems - providing the rest of the system wasn't >> being stressed. >> >> One system I found that removing a memory module - any of them stopped the >> shutdowns, and I eventually 'bodged' the system by increasing the memory refresh >> by a cycle. It was an old system and a 'new' memory module was, being old tech, >> horrendously expensive >> That worked for several years, and eventually management agreed the system was >> too slow - as in users kept complaining about the system's response, so we were >> actually allowed to BUY a new one! >> >> JimB >> > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From stuart at lexacorp.com.pg Sat Aug 8 14:28:40 2015 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sun, 09 Aug 2015 05:28:40 +1000 Subject: [dba-SQLServer] 2.45 billion rows and counting was PK Cast fails In-Reply-To: <55C6197C.4010609@gmail.com> References: <55C6197C.4010609@gmail.com> Message-ID: <55C65868.32392.1325127B@stuart.lexacorp.com.pg> Have you tried DBCC CHECKIDENT (myHumungousable, NORESEED) On 8 Aug 2015 at 11:00, John W. Colby wrote: > > While I can see the total storage (records) in the junction table, > what I can't do is get a Max() of the Simmons PK in the junction table > to see where I am in the process. I was hoping that info would be in > metadata somewhere but apparently not, it seems that it has to scan > through the table looking for the max() value. While I haven't let it > run in a query window, trying to do that in a view timed out for > obvious reasons. > From jwcolby at gmail.com Sat Aug 8 14:32:14 2015 From: jwcolby at gmail.com (John W. Colby) Date: Sat, 8 Aug 2015 15:32:14 -0400 Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: <538439977.21281534.1439061126354.JavaMail.root@shaw.ca> References: <538439977.21281534.1439061126354.JavaMail.root@shaw.ca> Message-ID: <55C6593E.7000508@gmail.com> This is a one of a kind machine, custom built by me. The closest thing to a "similar machine" is a VM server that I also built, a hex core AMD with consumer grade "everything", 32 gigs ram, which was custom built to run 8 VMs that are also used with this system. The VMs all use 3.5 gigs of ram so there isn't really the memory / cpu horsepower left to even approach the needs of the SQL Server. John W. Colby On 8/8/2015 3:12 PM, Jim Lawrence wrote: > Just a thought and everything that can be eliminated is a step towards a solution. > > If a backup server with similar features could be tested on then the possibility of software issues could be either isolated or discovered. > > Jim > > From jwcolby at gmail.com Sat Aug 8 14:34:13 2015 From: jwcolby at gmail.com (John W. Colby) Date: Sat, 8 Aug 2015 15:34:13 -0400 Subject: [dba-SQLServer] 2.45 billion rows and counting was PK Cast fails In-Reply-To: <55C65868.32392.1325127B@stuart.lexacorp.com.pg> References: <55C6197C.4010609@gmail.com> <55C65868.32392.1325127B@stuart.lexacorp.com.pg> Message-ID: <55C659B5.80206@gmail.com> Nope, never even heard of that one. But isn't the PKID that I need but a matching value of PKSimmons stored in that max record. John W. Colby On 8/8/2015 3:28 PM, Stuart McLachlan wrote: > Have you tried DBCC CHECKIDENT (myHumungousable, NORESEED) > > > On 8 Aug 2015 at 11:00, John W. Colby wrote: > >> While I can see the total storage (records) in the junction table, >> what I can't do is get a Max() of the Simmons PK in the junction table >> to see where I am in the process. I was hoping that info would be in >> metadata somewhere but apparently not, it seems that it has to scan >> through the table looking for the max() value. While I haven't let it >> run in a query window, trying to do that in a view timed out for >> obvious reasons. >> > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Aug 8 14:45:37 2015 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sun, 09 Aug 2015 05:45:37 +1000 Subject: [dba-SQLServer] 2.45 billion rows and counting was PK Cast fails In-Reply-To: <55C659B5.80206@gmail.com> References: <55C6197C.4010609@gmail.com>, <55C65868.32392.1325127B@stuart.lexacorp.com.pg>, <55C659B5.80206@gmail.com> Message-ID: <55C65C61.7389.1334987C@stuart.lexacorp.com.pg> Doh! Didn't read it carefully enough. I just saw Max() and PK and jumped in :( -- Stuart On 8 Aug 2015 at 15:34, John W. Colby wrote: > Nope, never even heard of that one. But isn't the PKID that I need > but a matching value of PKSimmons stored in that max record. > > John W. Colby > > On 8/8/2015 3:28 PM, Stuart McLachlan wrote: > > Have you tried DBCC CHECKIDENT (myHumungousable, NORESEED) > > > > > > On 8 Aug 2015 at 11:00, John W. Colby wrote: > > > >> While I can see the total storage (records) in the junction table, > >> what I can't do is get a Max() of the Simmons PK in the junction > >> table to see where I am in the process. I was hoping that info > >> would be in metadata somewhere but apparently not, it seems that it > >> has to scan through the table looking for the max() value. While I > >> haven't let it run in a query window, trying to do that in a view > >> timed out for obvious reasons. > >> > > _______________________________________________ > > dba-SQLServer mailing list > > dba-SQLServer at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > > http://www.databaseadvisors.com > > > > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From jwcolby at gmail.com Sat Aug 8 16:22:54 2015 From: jwcolby at gmail.com (John W. Colby) Date: Sat, 8 Aug 2015 17:22:54 -0400 Subject: [dba-SQLServer] 2.45 billion rows and counting was PK Cast fails In-Reply-To: <55C65C61.7389.1334987C@stuart.lexacorp.com.pg> References: <55C6197C.4010609@gmail.com> <55C65868.32392.1325127B@stuart.lexacorp.com.pg> <55C659B5.80206@gmail.com> <55C65C61.7389.1334987C@stuart.lexacorp.com.pg> Message-ID: <55C6732E.5000202@gmail.com> I'm just trying to get a feel for "how far done". I know that two billion records was about Simmons record #70 (of 220), but each simmons record has a different number of matching DB101 records so that doesn't say a whole lot. This thing is iterating the Simmons table in code, appending an entire set of PKDB101 / PKSimmons records for each record in the Simmons definition table (220 total such records) - typically tens of millions DB101 PKs per Simmons record (PK). I could just halt the stored procedure but if it was very far into a given append it would have to back out a lot of records before it finally stopped. I figure it will finish sometime tonight. John W. Colby On 8/8/2015 3:45 PM, Stuart McLachlan wrote: > Doh! Didn't read it carefully enough. I just saw Max() and PK and jumped in :( > From bheid at sc.rr.com Sat Aug 8 17:07:14 2015 From: bheid at sc.rr.com (Bobby Heid) Date: Sat, 8 Aug 2015 18:07:14 -0400 Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: <55C6593E.7000508@gmail.com> References: <538439977.21281534.1439061126354.JavaMail.root@shaw.ca> <55C6593E.7000508@gmail.com> Message-ID: <000001d0d226$952d2b60$bf878220$@sc.rr.com> Have you tested RAM? Also, if it is set to automatically reboot on a blue screen, you might can set it to not automatically restart. Bobby -----Original Message----- From: dba-SQLServer [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of John W. Colby Sent: Saturday, August 08, 2015 3:32 PM To: Discussion concerning MS SQL Server Subject: Re: [dba-SQLServer] [AccessD] A real puzzler This is a one of a kind machine, custom built by me. The closest thing to a "similar machine" is a VM server that I also built, a hex core AMD with consumer grade "everything", 32 gigs ram, which was custom built to run 8 VMs that are also used with this system. The VMs all use 3.5 gigs of ram so there isn't really the memory / cpu horsepower left to even approach the needs of the SQL Server. John W. Colby On 8/8/2015 3:12 PM, Jim Lawrence wrote: > Just a thought and everything that can be eliminated is a step towards a solution. > > If a backup server with similar features could be tested on then the possibility of software issues could be either isolated or discovered. > > Jim > > _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From fuller.artful at gmail.com Sun Aug 9 12:00:43 2015 From: fuller.artful at gmail.com (Arthur Fuller) Date: Sun, 9 Aug 2015 13:00:43 -0400 Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: <000001d0d226$952d2b60$bf878220$@sc.rr.com> References: <538439977.21281534.1439061126354.JavaMail.root@shaw.ca> <55C6593E.7000508@gmail.com> <000001d0d226$952d2b60$bf878220$@sc.rr.com> Message-ID: Hi JC, I have no suggestions other than those already presented, but I do have a couple of questions. How large is the database itself? Is it clustered? How large is that one very large table? A. ? From jwcolby at gmail.com Sun Aug 9 18:36:16 2015 From: jwcolby at gmail.com (John W. Colby) Date: Sun, 9 Aug 2015 19:36:16 -0400 Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: References: <538439977.21281534.1439061126354.JavaMail.root@shaw.ca> <55C6593E.7000508@gmail.com> <000001d0d226$952d2b60$bf878220$@sc.rr.com> Message-ID: <55C7E3F0.3030903@gmail.com> The insert finally completed. There are 7043347063 rows in the table. Data space 170 GB, index space 151 GB. The table consists of an identity PK (bigint) which is a literal PK and is therefore clustered on the PK. The only purpose of that PK is to cause the clustering. There are then two other fields, a PKSimmons, values 0-221, and a PKDB101, values 0-225 million. And finally a second index (other than the clustered index) on the two FKs. In hind sight I probably should have not used the identity, instead simply making the two FKs be the PK itself, clustered. It is never clear to me, but I believe that if I want to do that I would need to pull the two PKs sorted (taking a lot of extra time on each select) or the insertion into the clustered index would be a mess, since my understanding is that clustered PKs are required to be inserted in sorted order. An identity is by definition incrementing values and thus slide right in in consecutive order. Not being a SQL Server guru that may be just a serious misunderstanding on my part. John W. Colby On 8/9/2015 1:00 PM, Arthur Fuller wrote: > Hi JC, > > I have no suggestions other than those already presented, but I do have a > couple of questions. How large is the database itself? Is it clustered? How > large is that one very large table? > > A. > ? > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > From jwcolby at gmail.com Sun Aug 9 22:06:26 2015 From: jwcolby at gmail.com (John W. Colby) Date: Sun, 9 Aug 2015 23:06:26 -0400 Subject: [dba-SQLServer] Billions and billions of rows Message-ID: <55C81532.3070905@gmail.com> So the table ended up with 7 billion rows and a combined 320 gb data and index space. I decided to rebuild the table using just the FKs (PKSimmons and PKDB101), and a PK of those two fields combined. I also compressed the index (page). It is unclear how much compression I will get since the data being compressed are two integer FK fields. All just scientific inquiry. :) I can say however that the Index storage is almost nothing. So I ran a count on the first attempt at this junction table, selecting just the second PKSimmons and the time was 1.5 minutes vs just over 2 minutes for doing it the old (hard coded) way. Not a significant savings I must admit. That was with a noncompressed / non clustered index however. We shall see whether a single compressed / clustered index will speed things up. Of course it is likely to take me another day to yank the two FK fields out of the first table and insert into the new table (insert into / select). -- John W. Colby From accessd at shaw.ca Mon Aug 10 03:57:25 2015 From: accessd at shaw.ca (Jim Lawrence) Date: Mon, 10 Aug 2015 02:57:25 -0600 (MDT) Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: <55C6593E.7000508@gmail.com> Message-ID: <1153755338.21898496.1439197045364.JavaMail.root@shaw.ca> You can actually run your database up on a Cloud based system. Before you say it is not possible or far too expensive you should check out DigitalOcean. (https://www.digitalocean.com) It has some very interesting features. To run up a quick small droplet costs only $5.00 per month and with the appropriate code an additional two months are added for free. The system is all SSDs, with tier-1 bandwidth (1 GB per second transfer rate but if a user has access to an ISP unlink (or you live in a community serviced by Google's new communications systems), transfer rates can be 10 GB per second ). Many companies can now afford to use Cloud backup. Basically the data transfer rate is limited to how large of pipe a system's person can get access to: http://tinyurl.com/or3gx38 An example of a pre-set hourly rate would be: $ 0.952 / hour or $ 22.85 / day 64 GB Memory 20 Core Processor 640 GB SSD Disk 9 TB Transfer The above is just a sample but a person can choose any size. Someone can just use their droplet as long as they need it and then just blow it away....no long term contracts: https://www.digitalocean.com/help There are hundreds of tutorials on virtually every subject on how to best use cloud computing: https://www.digitalocean.com/community/tutorials The question is can someone use Windows. Like ALL Cloud systems, they just use KVM containers, just like Microsoft Azure. To get Windows compatibility, a Virtual drive like VMware or Virtualbox (MS uses a custom Hyper-V drive) is installed, on the Linux OS of choice. Just a thought. Jim ----- Original Message ----- From: "John W. Colby" To: "Discussion concerning MS SQL Server" Sent: Saturday, August 8, 2015 12:32:14 PM Subject: Re: [dba-SQLServer] [AccessD] A real puzzler This is a one of a kind machine, custom built by me. The closest thing to a "similar machine" is a VM server that I also built, a hex core AMD with consumer grade "everything", 32 gigs ram, which was custom built to run 8 VMs that are also used with this system. The VMs all use 3.5 gigs of ram so there isn't really the memory / cpu horsepower left to even approach the needs of the SQL Server. John W. Colby On 8/8/2015 3:12 PM, Jim Lawrence wrote: > Just a thought and everything that can be eliminated is a step towards a solution. > > If a backup server with similar features could be tested on then the possibility of software issues could be either isolated or discovered. > > Jim > > _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at gmail.com Mon Aug 10 14:29:43 2015 From: jwcolby at gmail.com (John W. Colby) Date: Mon, 10 Aug 2015 15:29:43 -0400 Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: <1153755338.21898496.1439197045364.JavaMail.root@shaw.ca> References: <1153755338.21898496.1439197045364.JavaMail.root@shaw.ca> Message-ID: <55C8FBA7.2080907@gmail.com> I have always been interested in using a cloud to perform the calculations. However: 1) I have about 600 GB of data in 9 different SQL Server databases. 2) I have 2 years of custom C# written development code in a pair of applications that directly manipulate SQL Server to perform the work that I do. 3) That C# code performs backups to storage local to the C# Server. 4) I keep the database files themselves on SSD, using about 700 GB (of 1 TB) of Raid SSD storage, on the same system that SQL Server runs on 5) I keep the log files on rotating media, again on the same physical machine. 6) I have 16 cores and 80 gb of RAM, and the SQL Server uses it all. 8) I use compression extensively, which requires cores to decompress data in memory as it is used. 9) The custom C# code I run is running on a VM, accessing data in SQL Server, and shoveling CSV files off to a set of 4 (currently) VMs which then run custom third party apps which clean the data, then imports the cleaned data back into SQL Server 10) And finally, I am a one man show. I do not have a million dollar budget and a half dozen flunkies to throw at feasibility studies, nor migration to the cloud. To be quite honest, I consider myself quite lucky and quite skilled to have designed, coded and built the system I have, over a period of many years. It is a very complex system that does a simple job that requires a complex system. The system as it stands uses a pair of pretty darned powerful servers, one running six VMs, the other running a largish instance of SQL Server, all tightly integrated with custom software level virtual networks to allow everything to talk to each other. Every month this system exports about 500 MILLION addresses for cleaning, to the VMs running the third party software, shipping data across the internet from the VMs to their servers in California, then back to the VMs and from there back into the SQL Server. Every two months I have to download a 3 gb install disk from the third party software house, and install that (upgrade the software) on the VMs. I built the servers from parts, installed all software from disk or downloadable images, tweaked to get things running off of SSDs to get the last ounce of speed, created VMs, cloned them, running off of SSDs. I designed the C# systems, and coded about 40% myself, with the remaining done under my supervision by a kid I hired out of a C# class I took. All of this work done in my home office, over a period of about 4 years. And the odd part (to me) is that I am considered unqualified to get a job in the IT world because I don't have a BS(!) degree and a laundry list of silly software experience check boxes. Never mind what you have done, which of these things that you will never actually be required to do can you say you have extensive experience in? Laundry lists of them! :) So could I "run my database up on the cloud?". If anything goes wrong could I fly up into the cloud to fix it? How much of my non-existent million dollar budget could I eat up in tech support because the system is hard down for some reason and no one has a clue. Had the cloud been available when I started this venture I think I would have designed it around the cloud. Now to go back and port it all to the cloud is pretty much not happening. John W. Colby On 8/10/2015 4:57 AM, Jim Lawrence wrote: > You can actually run your database up on a Cloud based system. > > > Just a thought. > > Jim > > From fhtapia at gmail.com Tue Aug 11 08:48:57 2015 From: fhtapia at gmail.com (fhtapia at gmail.com) Date: Tue, 11 Aug 2015 13:48:57 +0000 Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: <55C6498E.1@gmail.com> References: <1966571744.21246095.1439055088556.JavaMail.root@shaw.ca> <55C6498E.1@gmail.com> Message-ID: are you still suffering from intermittent power cycles? or did setting the power saving features to max fix your issues? Regards, Francisco On Sat, Aug 8, 2015 at 11:25 AM John W. Colby wrote: > I have (just now) set the power saving states to keep the processor on > max, never shut down disks etc. > > John W. Colby > > On 8/8/2015 1:31 PM, Jim Lawrence wrote: > > Hi Damien: > > > > Good call. > > > > ...But then what type of CPU are we taking about...there are very > limited number of suppliers with 90 plus percent being Intel and AMD. A bad > CPU and motherboard combination? > > > > IMHO, it is still most likely internal power supply related; like in > some recent HP servers. (We are assuming that the computer has not been > over-clock as all guarantees, then fly right out the window.) > > > > Jim > > > > ----- Original Message ----- > > From: "Damien Solodow" > > To: "Discussion concerning MS SQL Server" < > dba-sqlserver at databaseadvisors.com> > > Sent: Saturday, August 8, 2015 9:42:32 AM > > Subject: Re: [dba-SQLServer] [AccessD] A real puzzler > > > > Other possibility could be related to CPU power states; when it's under > light load it tries to step down and it bounces. Should be able to disable > cpu power saving in the bios, and possibly in Windows as well. > > > > DAMIEN SOLODOW > > Senior Systems Engineer > > 317.447.6033 (office) > > 317.447.6014 (fax) > > HARRISON COLLEGE > > > > ________________________________________ > > From: dba-SQLServer [dba-sqlserver-bounces at databaseadvisors.com] on > behalf of Jim Lawrence [accessd at shaw.ca] > > Sent: Saturday, August 08, 2015 12:39 PM > > To: Discussion concerning MS SQL Server > > Subject: Re: [dba-SQLServer] [AccessD] A real puzzler > > > > I have no idea what the issue could be but the server rebooting while > not under load gives a suggestion of the cause. > > > > It must be power related. Obviously it is not external power so it > leaves only the internal power supply and related wiring. IMHO, that is the > problem and I would replace that power hardware as soon as convenient. > > > > Jim > > > > ----- Original Message ----- > > From: "John W. Colby" > > To: "Access Developers discussion and problem solving" < > accessd at databaseadvisors.com>, "Discussion concerning MS SQL Server" < > dba-sqlserver at databaseadvisors.com> > > Sent: Saturday, August 8, 2015 5:14:47 AM > > Subject: Re: [dba-SQLServer] [AccessD] A real puzzler > > > > For the last several days I have been flogging away at the system > > (performing real work), causing the system to stay active. All cores > > running, 75 (out of 80) GB used for SQL Server. No reboots during that > > entire time. And yet: > > > > Critical 8/4/2015 6:56:10 AM Kernel-Power 41 (63) > > Critical 8/4/2015 4:42:07 AM Kernel-Power 41 (63) > > Critical 8/3/2015 10:04:36 PM Kernel-Power 41 (63) > > Critical 8/3/2015 5:32:06 PM Kernel-Power 41 (63) > > Critical 8/3/2015 2:22:15 PM Kernel-Power 41 (63) > > Critical 8/2/2015 3:29:33 PM Kernel-Power 41 (63) > > Critical 8/2/2015 11:21:46 AM Kernel-Power 41 (63) > > Critical 8/2/2015 10:51:30 AM Kernel-Power 41 (63) > > Critical 8/2/2015 7:17:21 AM Kernel-Power 41 (63) > > Critical 7/31/2015 11:50:45 PM Kernel-Power 41 (63) > > Critical 7/31/2015 1:05:39 PM Kernel-Power 41 (63) > > Critical 7/30/2015 10:11:57 PM Kernel-Power 41 (63) > > Critical 7/30/2015 4:59:26 PM Kernel-Power 41 (63) > > Critical 7/29/2015 2:32:50 PM Kernel-Power 41 (63) > > Critical 7/28/2015 5:20:38 PM Kernel-Power 41 (63) > > Critical 7/28/2015 12:12:57 PM Kernel-Power 41 (63) > > Critical 7/28/2015 4:15:38 AM Kernel-Power 41 (63) > > Critical 7/28/2015 2:47:12 AM Kernel-Power 41 (63) > > > > WEIRD!!! > > > > Notice no pattern in number of events per day, nor time of day. There is > > no memory dump being created, the system just reboots as if the power > > was turned off and back on. When I was in the room with it, (several > > years ago) the system would beep as it rebooted. > > > > Here is the last such event: > > > > Log Name: System > > Source: Microsoft-Windows-Kernel-Power > > Date: 8/4/2015 6:56:10 AM > > Event ID: 41 > > Task Category: (63) > > Level: Critical > > Keywords: (2) > > User: SYSTEM > > Computer: Azul > > Description: > > The system has rebooted without cleanly shutting down first. This error > > could be caused if the system stopped responding, crashed, or lost power > > unexpectedly. > > Event Xml: > > > > > > > Guid="{331C3B3A-2005-44C2-AC5E-77220C37D6B4}" /> > > 41 > > 2 > > 1 > > 63 > > 0 > > 0x8000000000000002 > > > > 270735 > > > > > > System > > Azul > > > > > > > > 0 > > 0x0 > > 0x0 > > 0x0 > > 0x0 > > false > > 0 > > > > > > > > John W. Colby > > > > On 8/8/2015 7:57 AM, James Button wrote: > >> Guest that! > >> > >> Re memory diagnostic - I have found with past experiences of > accelerating > >> frequency of shut-downs, that the system didn't get a chance to record > any > >> events. > >> And memory checks showed no problems - providing the rest of the system > wasn't > >> being stressed. > >> > >> One system I found that removing a memory module - any of them stopped > the > >> shutdowns, and I eventually 'bodged' the system by increasing the > memory refresh > >> by a cycle. It was an old system and a 'new' memory module was, being > old tech, > >> horrendously expensive > >> That worked for several years, and eventually management agreed the > system was > >> too slow - as in users kept complaining about the system's response, so > we were > >> actually allowed to BUY a new one! > >> > >> JimB > >> > > From jwcolby at gmail.com Tue Aug 11 10:18:05 2015 From: jwcolby at gmail.com (John W. Colby) Date: Tue, 11 Aug 2015 11:18:05 -0400 Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: References: <1966571744.21246095.1439055088556.JavaMail.root@shaw.ca> <55C6498E.1@gmail.com> Message-ID: <55CA122D.5070102@gmail.com> >>suffering from intermittent power cycles? What does this mean? I set the properties in the power section to completely stop any and all things which would minimize power usage. Not very much was trying to shut down, specifically (IIRC) the monitor shutting off. The disks weren't. At any rate, the system has been modified to not allow ANY power reduction stuff from occurring. The issue is still occurring. And AFAICT it is unrelated to power minimization. John W. Colby On 8/11/2015 9:48 AM, fhtapia at gmail.com wrote: > are you still suffering from intermittent power cycles? or did setting the > power saving features to max fix your issues? > > Regards, > Francisco > > On Sat, Aug 8, 2015 at 11:25 AM John W. Colby wrote: > >> I have (just now) set the power saving states to keep the processor on >> max, never shut down disks etc. >> From fuller.artful at gmail.com Wed Aug 12 12:46:05 2015 From: fuller.artful at gmail.com (Arthur Fuller) Date: Wed, 12 Aug 2015 13:46:05 -0400 Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: <55CA122D.5070102@gmail.com> References: <1966571744.21246095.1439055088556.JavaMail.root@shaw.ca> <55C6498E.1@gmail.com> <55CA122D.5070102@gmail.com> Message-ID: JWC, a) Given that you get the job done, if not optimally, then All Is Well, sort of. Granted, you have more hardware than I ever imagined owning, and a larger database than I've ever worked on; b) Also given that compared to your knowledge of hardware I am a duffer; I don't understand why so many cores in so relatively few machines is advantageous. Perhaps that's because my electricity costs are included in the rent, and thus essentially frree; but that said, I don't get the point of stacking multiple VMs in one or more boxes, as opposed to creating a cluster with a large number of boxes all visualized as a single computer. Maybe I should Google this and learn why this approach is better. On Tue, Aug 11, 2015 at 11:18 AM, John W. Colby wrote: > >>suffering from intermittent power cycles? > > What does this mean? I set the properties in the power section to > completely stop any and all things which would minimize power usage. Not > very much was trying to shut down, specifically (IIRC) the monitor shutting > off. The disks weren't. At any rate, the system has been modified to not > allow ANY power reduction stuff from occurring. > > The issue is still occurring. And AFAICT it is unrelated to power > minimization. > > John W. Colby > > On 8/11/2015 9:48 AM, fhtapia at gmail.com wrote: > >> are you still suffering from intermittent power cycles? or did setting the >> power saving features to max fix your issues? >> >> Regards, >> Francisco >> >> On Sat, Aug 8, 2015 at 11:25 AM John W. Colby wrote: >> >> I have (just now) set the power saving states to keep the processor on >>> max, never shut down disks etc. >>> >>> > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > -- Arthur From jwcolby at gmail.com Wed Aug 12 13:36:33 2015 From: jwcolby at gmail.com (John W. Colby) Date: Wed, 12 Aug 2015 14:36:33 -0400 Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: References: <1966571744.21246095.1439055088556.JavaMail.root@shaw.ca> <55C6498E.1@gmail.com> <55CA122D.5070102@gmail.com> Message-ID: <55CB9231.1030707@gmail.com> >>I don't understand why so many cores in so relatively few machines is advantageous. SQL Server can break complex queries down and assign pieces to threads. The more cores, the more threads can be used simultaneously. It is fascinating to watch task manager when it does this behavior, I have seen all of my cores (16) maxed. >>I don't get the point of stacking multiple VMs in one or more boxes, as opposed to creating a cluster with a large number of boxes all visualized as a single computer. It turns out that the VMs run faster off of SSDs. I bought an Areca controller, hooked up about five 500gb SSDs, created a raid out of that and then placed the VM files (the files which are the VMs) on that SSD. Having an actual dedicated raid controller allows me to do this "economically" and easily. After that, I got one VM running, then cloned it. It did involve some registry search / replace to get each one thinking it was a different machine, but you get the picture. So I can now "turn on" one or two or six of these machines if I want. I also have a single VM acting as the custom C# program "driver" of the system, and another VM which runs a small SQL Server instance for my own use. I went out and purchased 8 licenses of Windows Home Server when that was going out of style. $30 each IIRC. So I can have 8 different VMs, each with its own legal windows key, no MS complaints etc. I then installed legal copies of third party software on each VM. I could only run a single instance of this software on any given box. Each license gives me (on the VMs) about 1.5 million records / hour data cleaning speed. I have to clean 500 MILLION records per month, plus do other things as well (fill orders). So a single copy of the third party software just wasn't cutting it. I normally run four VMs, thus 4 licenses of this software, and about 6 MILLION records / hour. It still takes a lot of time but 1/4 the time over all (more or less). All of this running on a single reasonably simple "VM server" machine, a hex core AMD with 32 gigs ram, the Areca raid controller and a handful of SSDs. Administering the VMs is easy from the server machine. One physical box, six (normally) VMs running. VMs are incredibly easy to set up and administer. The third party software would in fact run somewhat faster on individual Intel based machines, but the hardware costs would be MUCH higher, including hosting costs at the facility that hosts my servers. And the administration would be much larger. John W. Colby On 8/12/2015 1:46 PM, Arthur Fuller wrote: > JWC, > > a) Given that you get the job done, if not optimally, then All Is Well, > sort of. Granted, you have more hardware than I ever imagined owning, and a > larger database than I've ever worked on; > > b) Also given that compared to your knowledge of hardware I am a duffer; > > I don't understand why so many cores in so relatively few machines is > advantageous. Perhaps that's because my electricity costs are included in > the rent, and thus essentially frree; but that said, I don't get the point > of stacking multiple VMs in one or more boxes, as opposed to creating a > cluster with a large number of boxes all visualized as a single computer. > > Maybe I should Google this and learn why this approach is better. > > From accessd at shaw.ca Wed Aug 12 19:16:47 2015 From: accessd at shaw.ca (Jim Lawrence) Date: Wed, 12 Aug 2015 18:16:47 -0600 (MDT) Subject: [dba-SQLServer] [AccessD] A real puzzler In-Reply-To: <55C8FBA7.2080907@gmail.com> Message-ID: <1897999860.24052649.1439425007157.JavaMail.root@shaw.ca> Hi John: You're not qualified. ;-) If you had a BS and were in your twenties, any other qualifications would have been over-looked. Once over forty, not in a stable and consistent gig, you are either a project manager or not employed. The other reality of the universe, a free-lance tech must be networking continuously...I really have no idea how may workshops, training courses, presentations, cold-call/contract presentations and various other mixures I have been on over the years all in an effort to line up the next contracts. When I retired and stopped doing all the pimping and whoring everyone thought I was dead. I must have done something right as old employers still keep calling from time to time. The truth is that I was totally feed up with Microsoft and Ballmer...both were going nowhere. Any work I do now is on the internet and through Linux...I just love the performance, stability, flexibility, security, no-forced upgrades, rolling release options, leading edge concepts and ideas. I have used Ubuntu for over a decade and upgrades are effortless. There is really nothing more powerful on the desktop. Their enterprise server is the same as the desktop...you can just use what you want...you are limited only by the power of you equipment. I am glad to see that MS has finally embraced Linux. Did you know that MS Asure is really a huge Debian Linux farm, which uses Apache Hive, for expandability, Ubuntu 14.04 as the basic interface and OS, Juju, the Ubuntu Cloud building and management GUI tools and finally on top of that a pretty MS interface. You have a system that has been built using hundreds (thousands) of hours of time and it could not be easily replaced or ever replaced. In your case I was thinking that maybe running up a cloud hourly test station, using a mini-version, might be something you would be interested in...like rent a server for a morning for around $4.00 and then maybe save the droplet and/or just blow it away. :-) Jim ----- Original Message ----- From: "John W. Colby" To: "Discussion concerning MS SQL Server" Sent: Monday, August 10, 2015 12:29:43 PM Subject: Re: [dba-SQLServer] [AccessD] A real puzzler I have always been interested in using a cloud to perform the calculations. However: 1) I have about 600 GB of data in 9 different SQL Server databases. 2) I have 2 years of custom C# written development code in a pair of applications that directly manipulate SQL Server to perform the work that I do. 3) That C# code performs backups to storage local to the C# Server. 4) I keep the database files themselves on SSD, using about 700 GB (of 1 TB) of Raid SSD storage, on the same system that SQL Server runs on 5) I keep the log files on rotating media, again on the same physical machine. 6) I have 16 cores and 80 gb of RAM, and the SQL Server uses it all. 8) I use compression extensively, which requires cores to decompress data in memory as it is used. 9) The custom C# code I run is running on a VM, accessing data in SQL Server, and shoveling CSV files off to a set of 4 (currently) VMs which then run custom third party apps which clean the data, then imports the cleaned data back into SQL Server 10) And finally, I am a one man show. I do not have a million dollar budget and a half dozen flunkies to throw at feasibility studies, nor migration to the cloud. To be quite honest, I consider myself quite lucky and quite skilled to have designed, coded and built the system I have, over a period of many years. It is a very complex system that does a simple job that requires a complex system. The system as it stands uses a pair of pretty darned powerful servers, one running six VMs, the other running a largish instance of SQL Server, all tightly integrated with custom software level virtual networks to allow everything to talk to each other. Every month this system exports about 500 MILLION addresses for cleaning, to the VMs running the third party software, shipping data across the internet from the VMs to their servers in California, then back to the VMs and from there back into the SQL Server. Every two months I have to download a 3 gb install disk from the third party software house, and install that (upgrade the software) on the VMs. I built the servers from parts, installed all software from disk or downloadable images, tweaked to get things running off of SSDs to get the last ounce of speed, created VMs, cloned them, running off of SSDs. I designed the C# systems, and coded about 40% myself, with the remaining done under my supervision by a kid I hired out of a C# class I took. All of this work done in my home office, over a period of about 4 years. And the odd part (to me) is that I am considered unqualified to get a job in the IT world because I don't have a BS(!) degree and a laundry list of silly software experience check boxes. Never mind what you have done, which of these things that you will never actually be required to do can you say you have extensive experience in? Laundry lists of them! :) So could I "run my database up on the cloud?". If anything goes wrong could I fly up into the cloud to fix it? How much of my non-existent million dollar budget could I eat up in tech support because the system is hard down for some reason and no one has a clue. Had the cloud been available when I started this venture I think I would have designed it around the cloud. Now to go back and port it all to the cloud is pretty much not happening. John W. Colby On 8/10/2015 4:57 AM, Jim Lawrence wrote: > You can actually run your database up on a Cloud based system. > > > Just a thought. > > Jim > > _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at gmail.com Fri Aug 21 11:25:30 2015 From: jwcolby at gmail.com (John W. Colby) Date: Fri, 21 Aug 2015 12:25:30 -0400 Subject: [dba-SQLServer] Billions and billions of rows In-Reply-To: <55C81532.3070905@gmail.com> References: <55C81532.3070905@gmail.com> Message-ID: <55D750FA.6010103@gmail.com> Just an update, after rebuilding this as a pair of fields - FKDB101 / FKSimmons (Int), set as the PK (clustered index) and compressed (page), the total storage dropped to just the storage of the data (compressed) and a tiny little amount (a hundred mbytes?) that SQL Server uses for something index related. The time to get counts has dropped by an order of magnitude. Just to recap, previously I would look up in a Simmons Definition table what operations were required to "translate" a given field in DB101 to a corresponding field in Simmons. Simmons has 220 defined "fields" which are "translated from DB101 and mapped into Simmons". Just as an example, I have a single age field. Simmons has (IIRC) 5 year age brackets. So to count people in simmons age bracket 30-35 I would pull all records with an age between 30 and 35 and count them. From that example you can see that to get that specific Simmons count I had to dynamically create SQL and perform a count on the pulled set of records. Given that, the counts for each simmons data point took around 3.5 minutes to define the dynamic sql (not particularly time intensive) and then pull and count the required records against the full DB101 data set (220 million records). And of course the actual time varied all over the map, but that was the average. So what I did was "preselect" the records that fit each simmons data point and stored it in this new junction table a pair of fields - FKDB101 / FKSimmons (Int) So now, while I still have to perform joins on at least one other table, the join is between the already selected PKDB101 set and another table. The results are lightning fast, with each count requiring about 9 seconds (average for junction table) instead of about 3.5 minutes (dynamic sql). I can now do my entire count of all 220 Simmons data points in around 30 minutes instead of days. The junction table contains just over seven billion records. DB101 contains about 220 million records and several hundred fields. The Simmons table contains 228 records, each record mapping one specific DB101 field into a similar Simmons "field". The simmons information is then further filtered on other (non simmons) fields in DB101, for example simmons counts against people with diabetes, or people who are jewish, or people who go cruising. SQL Server is very efficient in determining how to select the various parts of a query to optimize the retrieval of the result set, so it tends to preselect the jewish (for example) records and then only perform the "simmons stuff" against that much smaller result set. What this means is that using either method (old dynamic sql or new junction table) the actual time to get the counts will be reduced by the "other selection" criteria. But the junction table method still wins hands down in time to completion. John W. Colby On 8/9/2015 11:06 PM, John W. Colby wrote: > So the table ended up with 7 billion rows and a combined 320 gb data > and index space. > > I decided to rebuild the table using just the FKs (PKSimmons and > PKDB101), and a PK of those two fields combined. I also compressed > the index (page). It is unclear how much compression I will get since > the data being compressed are two integer FK fields. All just > scientific inquiry. :) > > I can say however that the Index storage is almost nothing. > > So I ran a count on the first attempt at this junction table, > selecting just the second PKSimmons and the time was 1.5 minutes vs > just over 2 minutes for doing it the old (hard coded) way. Not a > significant savings I must admit. That was with a noncompressed / non > clustered index however. We shall see whether a single compressed / > clustered index will speed things up. Of course it is likely to take > me another day to yank the two FK fields out of the first table and > insert into the new table (insert into / select). > From fuller.artful at gmail.com Sat Aug 22 14:09:50 2015 From: fuller.artful at gmail.com (Arthur Fuller) Date: Sat, 22 Aug 2015 15:09:50 -0400 Subject: [dba-SQLServer] Billions and billions of rows In-Reply-To: <55D750FA.6010103@gmail.com> References: <55C81532.3070905@gmail.com> <55D750FA.6010103@gmail.com> Message-ID: Thanks for the insights, JC. I think that you could do even better, but you've already saved so much time that shaving a few seconds off would require much more effort than the payoff would be worth. Congrats! Arthur On Fri, Aug 21, 2015 at 12:25 PM, John W. Colby wrote: > Just an update, after rebuilding this as a pair of fields - FKDB101 / > FKSimmons (Int), set as the PK (clustered index) and compressed (page), the > total storage dropped to just the storage of the data (compressed) and a > tiny little amount (a hundred mbytes?) that SQL Server uses for something > index related. > > The time to get counts has dropped by an order of magnitude. Just to > recap, previously I would look up in a Simmons Definition table what > operations were required to "translate" a given field in DB101 to a > corresponding field in Simmons. Simmons has 220 defined "fields" which are > "translated from DB101 and mapped into Simmons". Just as an example, I have > a single age field. Simmons has (IIRC) 5 year age brackets. So to count > people in simmons age bracket 30-35 I would pull all records with an age > between 30 and 35 and count them. From that example you can see that to > get that specific Simmons count I had to dynamically create SQL and perform > a count on the pulled set of records. > > Given that, the counts for each simmons data point took around 3.5 minutes > to define the dynamic sql (not particularly time intensive) and then pull > and count the required records against the full DB101 data set (220 million > records). And of course the actual time varied all over the map, but that > was the average. > > So what I did was "preselect" the records that fit each simmons data point > and stored it in this new junction table > > a pair of fields - FKDB101 / FKSimmons (Int) > > So now, while I still have to perform joins on at least one other table, > the join is between the already selected PKDB101 set and another table. > > The results are lightning fast, with each count requiring about 9 seconds > (average for junction table) instead of about 3.5 minutes (dynamic sql). I > can now do my entire count of all 220 Simmons data points in around 30 > minutes instead of days. > > The junction table contains just over seven billion records. DB101 > contains about 220 million records and several hundred fields. The Simmons > table contains 228 records, each record mapping one specific DB101 field > into a similar Simmons "field". > > The simmons information is then further filtered on other (non simmons) > fields in DB101, for example simmons counts against people with diabetes, > or people who are jewish, or people who go cruising. > > SQL Server is very efficient in determining how to select the various > parts of a query to optimize the retrieval of the result set, so it tends > to preselect the jewish (for example) records and then only perform the > "simmons stuff" against that much smaller result set. > > What this means is that using either method (old dynamic sql or new > junction table) the actual time to get the counts will be reduced by the > "other selection" criteria. But the junction table method still wins hands > down in time to completion. > > John W. Colby > > On 8/9/2015 11:06 PM, John W. Colby wrote: > >> So the table ended up with 7 billion rows and a combined 320 gb data and >> index space. >> >> I decided to rebuild the table using just the FKs (PKSimmons and >> PKDB101), and a PK of those two fields combined. I also compressed the >> index (page). It is unclear how much compression I will get since the data >> being compressed are two integer FK fields. All just scientific inquiry. >> :) >> >> I can say however that the Index storage is almost nothing. >> >> So I ran a count on the first attempt at this junction table, selecting >> just the second PKSimmons and the time was 1.5 minutes vs just over 2 >> minutes for doing it the old (hard coded) way. Not a significant savings I >> must admit. That was with a noncompressed / non clustered index however. >> We shall see whether a single compressed / clustered index will speed >> things up. Of course it is likely to take me another day to yank the two >> FK fields out of the first table and insert into the new table (insert into >> / select). >> >> > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > -- Arthur From fuller.artful at gmail.com Sat Aug 22 14:44:17 2015 From: fuller.artful at gmail.com (Arthur Fuller) Date: Sat, 22 Aug 2015 15:44:17 -0400 Subject: [dba-SQLServer] How RDBMSs work Message-ID: We are all pretty familiar with the "outer skin" of an RDBMS, but until I read this , I really didn't know much about the innards. Now I know a little more about what goes on under the hood. -- Arthur From accessd at shaw.ca Sat Aug 22 17:46:32 2015 From: accessd at shaw.ca (Jim Lawrence) Date: Sat, 22 Aug 2015 16:46:32 -0600 (MDT) Subject: [dba-SQLServer] How RDBMSs work In-Reply-To: Message-ID: <9987439.31006717.1440283592884.JavaMail.root@shaw.ca> Hi Arthur: That is a very good overview of RDBMS Jim ----- Original Message ----- From: "Arthur Fuller" To: "Discussion concerning MS SQL Server" Sent: Saturday, August 22, 2015 12:44:17 PM Subject: [dba-SQLServer] How RDBMSs work We are all pretty familiar with the "outer skin" of an RDBMS, but until I read this , I really didn't know much about the innards. Now I know a little more about what goes on under the hood. -- Arthur _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From darryl at whittleconsulting.com.au Tue Aug 25 20:23:42 2015 From: darryl at whittleconsulting.com.au (Darryl Collins) Date: Wed, 26 Aug 2015 01:23:42 +0000 Subject: [dba-SQLServer] SQL Server Azure and MS Access 2010 and/or other better options Message-ID: Greetings People, Given you are all way smarter and experienced in the world of SQL Server, I am going to ask your thoughts here. Early days and I am looking for options. In particular I was wondering if anyone has anyone has experience with using SQL Server Azure with a MS Access FE? On the pricing of Azure, it seems you pay for every hour the DB exists, I am not sure how that works? I mean, even if no-one access it for 2 weeks, I want it to 'exist' - or am I misunderstanding this whole SaaS thing? Brief Big Picture: Have multiple global users (about 20) who will need to log in occasionally (about once a week for a couple of hours) to update project forecasting data and view some bespoke reporting. I have built this sort of thing before, but the SQL Server has always been in house and everyone who uses it on the same network. The actual load and demand on the server will be tiny. The SQL Server DB is mostly going to sit there doing bugger all and will spend a lot of time being a data store for reporting. Azure appeals as it can be accessed from anywhere and Access 2010 (which we use) can hook right into it. Frankly am happy to look at all options here. Personally I would be happy to push the whole show out to an external provider (say something like <>) but the boss seems keen to keep the development in house. Which is good for me (I will be doing this). I can think of other potential options - Sharepoint might be one as well? Thanks for your time Cheers Darryl. From scott.marcus at tsstech.com Wed Aug 26 06:42:42 2015 From: scott.marcus at tsstech.com (Scott Marcus) Date: Wed, 26 Aug 2015 07:42:42 -0400 Subject: [dba-SQLServer] SQL Server Azure and MS Access 2010 and/or other better options In-Reply-To: References: Message-ID: Darryl, There are much cheaper options... https://www.arvixe.com/ Don't get confused by the naming of their plans. PersonalClass, BusinessClass, ResellerClass are levels of service and not limits for what you can use them. The PersonalClass would most likely meet your needs. You don't have to use the website portion of the hosting. Just use the MS SQL. Scott Marcus -----Original Message----- From: dba-SQLServer [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Darryl Collins Sent: Tuesday, August 25, 2015 9:24 PM To: Discussion concerning MS SQL Server Subject: [dba-SQLServer] SQL Server Azure and MS Access 2010 and/or other better options Greetings People, Given you are all way smarter and experienced in the world of SQL Server, I am going to ask your thoughts here. Early days and I am looking for options. In particular I was wondering if anyone has anyone has experience with using SQL Server Azure with a MS Access FE? On the pricing of Azure, it seems you pay for every hour the DB exists, I am not sure how that works? I mean, even if no-one access it for 2 weeks, I want it to 'exist' - or am I misunderstanding this whole SaaS thing? Brief Big Picture: Have multiple global users (about 20) who will need to log in occasionally (about once a week for a couple of hours) to update project forecasting data and view some bespoke reporting. I have built this sort of thing before, but the SQL Server has always been in house and everyone who uses it on the same network. The actual load and demand on the server will be tiny. The SQL Server DB is mostly going to sit there doing bugger all and will spend a lot of time being a data store for reporting. Azure appeals as it can be accessed from anywhere and Access 2010 (which we use) can hook right into it. Frankly am happy to look at all options here. Personally I would be happy to push the whole show out to an external provider (say something like <>) but the boss seems keen to keep the development in house. Which is good for me (I will be doing this). I can think of other potential options - Sharepoint might be one as well? Thanks for your time Cheers Darryl. _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com NOTICE: This electronic mail transmission is for the use of the named individual or entity to which it is directed and may contain information that is privileged or confidential. If you are not the intended recipient, any disclosure, copying, distribution or use of the contents of any information contained herein is prohibited. If you have received this electronic mail transmission in error, delete it from your system without copying or forwarding it, and notify the sender of the error by replying via email or calling TSS Technologies at (513) 772-7000, so that our address record can be corrected. Any information included in this email is provided on an "as is" and "where as" basis, and TSS Technologies makes no representations or warranties of any kind with respect to the completeness or accuracy of the information contained in this email. From gustav at cactus.dk Wed Aug 26 07:28:04 2015 From: gustav at cactus.dk (Gustav Brock) Date: Wed, 26 Aug 2015 12:28:04 +0000 Subject: [dba-SQLServer] SQL Server Azure and MS Access 2010 and/or other better options Message-ID: Hi Darryl If all you need is a minor database of a few hundred MB, we use this service: https://en.unoeuro.com/products.php It is cheaper than running it locally where you have to adjust firewalls and so on. Azure is great if you expect to scale it later and includes full backup, but please notice Azure SQL still is slightly different from a normal hosted SQL Server. /gustav -----Oprindelig meddelelse----- Fra: dba-SQLServer [mailto:dba-sqlserver-bounces at databaseadvisors.com] P? vegne af Darryl Collins Sendt: 26. august 2015 03:24 Til: Discussion concerning MS SQL Server Emne: [dba-SQLServer] SQL Server Azure and MS Access 2010 and/or other better options Greetings People, Given you are all way smarter and experienced in the world of SQL Server, I am going to ask your thoughts here. Early days and I am looking for options. In particular I was wondering if anyone has anyone has experience with using SQL Server Azure with a MS Access FE? On the pricing of Azure, it seems you pay for every hour the DB exists, I am not sure how that works? I mean, even if no-one access it for 2 weeks, I want it to 'exist' - or am I misunderstanding this whole SaaS thing? Brief Big Picture: Have multiple global users (about 20) who will need to log in occasionally (about once a week for a couple of hours) to update project forecasting data and view some bespoke reporting. I have built this sort of thing before, but the SQL Server has always been in house and everyone who uses it on the same network. The actual load and demand on the server will be tiny. The SQL Server DB is mostly going to sit there doing bugger all and will spend a lot of time being a data store for reporting. Azure appeals as it can be accessed from anywhere and Access 2010 (which we use) can hook right into it. Frankly am happy to look at all options here. Personally I would be happy to push the whole show out to an external provider (say something like <>) but the boss seems keen to keep the development in house. Which is good for me (I will be doing this). I can think of other potential options - Sharepoint might be one as well? Thanks for your time Cheers Darryl. From accessd at shaw.ca Fri Aug 28 12:42:10 2015 From: accessd at shaw.ca (Jim Lawrence) Date: Fri, 28 Aug 2015 11:42:10 -0600 (MDT) Subject: [dba-SQLServer] Billions and billions of rows In-Reply-To: <55D750FA.6010103@gmail.com> Message-ID: <1385945699.3668758.1440783730746.JavaMail.root@shaw.ca> Impressive as always. You should write an ebook on your discoveries of how to optimize performance on such a huge amount of data, on an MS SQL DB. Jim ----- Original Message ----- From: "John W. Colby" To: "Access Developers discussion and problem solving" , "Discussion concerning MS SQL Server" Sent: Friday, August 21, 2015 9:25:30 AM Subject: Re: [dba-SQLServer] Billions and billions of rows Just an update, after rebuilding this as a pair of fields - FKDB101 / FKSimmons (Int), set as the PK (clustered index) and compressed (page), the total storage dropped to just the storage of the data (compressed) and a tiny little amount (a hundred mbytes?) that SQL Server uses for something index related. The time to get counts has dropped by an order of magnitude. Just to recap, previously I would look up in a Simmons Definition table what operations were required to "translate" a given field in DB101 to a corresponding field in Simmons. Simmons has 220 defined "fields" which are "translated from DB101 and mapped into Simmons". Just as an example, I have a single age field. Simmons has (IIRC) 5 year age brackets. So to count people in simmons age bracket 30-35 I would pull all records with an age between 30 and 35 and count them. From that example you can see that to get that specific Simmons count I had to dynamically create SQL and perform a count on the pulled set of records. Given that, the counts for each simmons data point took around 3.5 minutes to define the dynamic sql (not particularly time intensive) and then pull and count the required records against the full DB101 data set (220 million records). And of course the actual time varied all over the map, but that was the average. So what I did was "preselect" the records that fit each simmons data point and stored it in this new junction table a pair of fields - FKDB101 / FKSimmons (Int) So now, while I still have to perform joins on at least one other table, the join is between the already selected PKDB101 set and another table. The results are lightning fast, with each count requiring about 9 seconds (average for junction table) instead of about 3.5 minutes (dynamic sql). I can now do my entire count of all 220 Simmons data points in around 30 minutes instead of days. The junction table contains just over seven billion records. DB101 contains about 220 million records and several hundred fields. The Simmons table contains 228 records, each record mapping one specific DB101 field into a similar Simmons "field". The simmons information is then further filtered on other (non simmons) fields in DB101, for example simmons counts against people with diabetes, or people who are jewish, or people who go cruising. SQL Server is very efficient in determining how to select the various parts of a query to optimize the retrieval of the result set, so it tends to preselect the jewish (for example) records and then only perform the "simmons stuff" against that much smaller result set. What this means is that using either method (old dynamic sql or new junction table) the actual time to get the counts will be reduced by the "other selection" criteria. But the junction table method still wins hands down in time to completion. John W. Colby On 8/9/2015 11:06 PM, John W. Colby wrote: > So the table ended up with 7 billion rows and a combined 320 gb data > and index space. > > I decided to rebuild the table using just the FKs (PKSimmons and > PKDB101), and a PK of those two fields combined. I also compressed > the index (page). It is unclear how much compression I will get since > the data being compressed are two integer FK fields. All just > scientific inquiry. :) > > I can say however that the Index storage is almost nothing. > > So I ran a count on the first attempt at this junction table, > selecting just the second PKSimmons and the time was 1.5 minutes vs > just over 2 minutes for doing it the old (hard coded) way. Not a > significant savings I must admit. That was with a noncompressed / non > clustered index however. We shall see whether a single compressed / > clustered index will speed things up. Of course it is likely to take > me another day to yank the two FK fields out of the first table and > insert into the new table (insert into / select). > _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From darryl at whittleconsulting.com.au Sun Aug 30 18:35:51 2015 From: darryl at whittleconsulting.com.au (Darryl Collins) Date: Sun, 30 Aug 2015 23:35:51 +0000 Subject: [dba-SQLServer] SQL Server Azure and MS Access 2010 and/or other better options In-Reply-To: References: Message-ID: Thanks folks for these suggestions. Apologies for the sluggish reply - been off sick with a cold for the past few days. Good approach too. I forget we could use our own ISP which provides a similar set up to those suggested. Cheers Darryl. -----Original Message----- From: dba-SQLServer [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Scott Marcus Sent: Wednesday, 26 August 2015 9:43 PM To: 'Discussion concerning MS SQL Server' Subject: Re: [dba-SQLServer] SQL Server Azure and MS Access 2010 and/or other better options Darryl, There are much cheaper options... https://www.arvixe.com/ Don't get confused by the naming of their plans. PersonalClass, BusinessClass, ResellerClass are levels of service and not limits for what you can use them. The PersonalClass would most likely meet your needs. You don't have to use the website portion of the hosting. Just use the MS SQL. Scott Marcus -----Original Message----- From: dba-SQLServer [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Darryl Collins Sent: Tuesday, August 25, 2015 9:24 PM To: Discussion concerning MS SQL Server Subject: [dba-SQLServer] SQL Server Azure and MS Access 2010 and/or other better options Greetings People, Given you are all way smarter and experienced in the world of SQL Server, I am going to ask your thoughts here. Early days and I am looking for options. In particular I was wondering if anyone has anyone has experience with using SQL Server Azure with a MS Access FE? On the pricing of Azure, it seems you pay for every hour the DB exists, I am not sure how that works? I mean, even if no-one access it for 2 weeks, I want it to 'exist' - or am I misunderstanding this whole SaaS thing? Brief Big Picture: Have multiple global users (about 20) who will need to log in occasionally (about once a week for a couple of hours) to update project forecasting data and view some bespoke reporting. I have built this sort of thing before, but the SQL Server has always been in house and everyone who uses it on the same network. The actual load and demand on the server will be tiny. The SQL Server DB is mostly going to sit there doing bugger all and will spend a lot of time being a data store for reporting. Azure appeals as it can be accessed from anywhere and Access 2010 (which we use) can hook right into it. Frankly am happy to look at all options here. Personally I would be happy to push the whole show out to an external provider (say something like <>) but the boss seems keen to keep the development in house. Which is good for me (I will be doing this). I can think of other potential options - Sharepoint might be one as well? Thanks for your time Cheers Darryl. _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com NOTICE: This electronic mail transmission is for the use of the named individual or entity to which it is directed and may contain information that is privileged or confidential. If you are not the intended recipient, any disclosure, copying, distribution or use of the contents of any information contained herein is prohibited. If you have received this electronic mail transmission in error, delete it from your system without copying or forwarding it, and notify the sender of the error by replying via email or calling TSS Technologies at (513) 772-7000, so that our address record can be corrected. Any information included in this email is provided on an "as is" and "where as" basis, and TSS Technologies makes no representations or warranties of any kind with respect to the completeness or accuracy of the information contained in this email. _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com