From jwcolby at colbyconsulting.com  Thu Sep  3 09:32:38 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Thu, 03 Sep 2009 10:32:38 -0400
Subject: [dba-SQLServer] Append only records where some fields not in the
	table already
Message-ID: <4A9FD386.7000407@colbyconsulting.com>

I have a set of 5 tables.  These tables contain names and number of children in age ranges.  As a 
first pass at processing, I need to merge the 5 tables into one table such that the name / address 
only gets into the table one time, IOW every record in the first table, but only the records in 
table 2 where the name / address data is not already in the destination table.

SQL Server 2005.  Clustered index on the name / address / zip5 / zip4 of the source tables.

I have not been able to find an append syntax that will append the records not in the table but 
prevent appending those already in, without aborting the entire append operation.

What is the magic key here?

-- 
John W. Colby
www.ColbyConsulting.com


From jwcolby at colbyconsulting.com  Thu Sep  3 09:42:28 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Thu, 03 Sep 2009 10:42:28 -0400
Subject: [dba-SQLServer] Append only records where some fields not in
 the	table already
In-Reply-To: <4A9FD386.7000407@colbyconsulting.com>
References: <4A9FD386.7000407@colbyconsulting.com>
Message-ID: <4A9FD5D4.5000700@colbyconsulting.com>

I assume that the answer will be a "not in" kind of query.  I am wondering if an index or a 
constraint could prevent an append of individual records while not causing the append to abort?  In 
either case is it going to be faster to have an index on the destination table and put up with the 
delay caused by the update to the index as new records append in, or will it be faster for the logic 
that does the record compare to just do table scans.  Think millions of records in the tables 
appending in, numbers of duplicates unknown.

John W. Colby
www.ColbyConsulting.com


jwcolby wrote:
> I have a set of 5 tables.  These tables contain names and number of children in age ranges.  As a 
> first pass at processing, I need to merge the 5 tables into one table such that the name / address 
> only gets into the table one time, IOW every record in the first table, but only the records in 
> table 2 where the name / address data is not already in the destination table.
> 
> SQL Server 2005.  Clustered index on the name / address / zip5 / zip4 of the source tables.
> 
> I have not been able to find an append syntax that will append the records not in the table but 
> prevent appending those already in, without aborting the entire append operation.
> 
> What is the magic key here?
> 


From Gustav at cactus.dk  Thu Sep  3 09:43:41 2009
From: Gustav at cactus.dk (Gustav Brock)
Date: Thu, 03 Sep 2009 16:43:41 +0200
Subject: [dba-SQLServer] Append only records where some fields not	in
	the table already
Message-ID: <sa9ff256.021@cactus.dk>

Hi John (I have posted this in 2000, 2001, and 2006 - even as a reply to a posting of yours)

This tip from Smart Access is one of my favourites:

<quote>

Update and Append Records with One Query

By Alan Biggs

Did you know that you can use an update query in Access to both update and add records at the same time? This is useful if you have two versions of a table, tblOld and tblNew, and you want to integrate the changes from tblNew into tblOld. 

Follow these steps:

1. Create an update query and add the two tables. Join the two tables by dragging the key field of tblNew onto the matching field of tblOld.

2. Double-click on the relationship and choose the join option that includes all records from tblNew and only those that match from tblOld. 

3. Select all the fields from tblOld and drag them onto the QBE grid.

4. For each field, in the Update To cell type in tblNew.FieldName, where FieldName matches the field name of tblOld.

5. Select Query Properties from the View menu and change Unique Records to False. (This switches off the DISTINCTROW option in the SQL view. If you leave this on you'll get only one blank record in your results, but you want one blank record for each new record to be added to tblOld.) 

6. Run the query and you'll see the changes to tblNew are now in tblOld. 

This will only add records to tblOld that have been added to tblNew.
Records in tblOld that aren't present in tblNew will still remain in tblOld. 

</quote>

I guess it works in SQL Server as well.

/gustav


>>> jwcolby at colbyconsulting.com 03-09-2009 16:32 >>>
I have a set of 5 tables.  These tables contain names and number of children in age ranges.  As a 
first pass at processing, I need to merge the 5 tables into one table such that the name / address 
only gets into the table one time, IOW every record in the first table, but only the records in 
table 2 where the name / address data is not already in the destination table.

SQL Server 2005.  Clustered index on the name / address / zip5 / zip4 of the source tables.

I have not been able to find an append syntax that will append the records not in the table but 
prevent appending those already in, without aborting the entire append operation.

What is the magic key here?

-- 
John W. Colby
www.ColbyConsulting.com 





From paul.hartland at googlemail.com  Thu Sep  3 09:49:36 2009
From: paul.hartland at googlemail.com (Paul Hartland)
Date: Thu, 3 Sep 2009 15:49:36 +0100
Subject: [dba-SQLServer] Append only records where some fields not in
	the table already
In-Reply-To: <sa9ff256.021@cactus.dk>
References: <sa9ff256.021@cactus.dk>
Message-ID: <38c884770909030749u1a446a2p61dc085743945d91@mail.gmail.com>

John,

Not had time to test this as just leaving work, but don't think it will be
too far out, then repeat for tables 3,4,5.

CREATE PROCEDURE MergeTables
AS
BEGIN
 INSERT INTO Table1 (Name, Address, Zip5, Zip4, Age)
 SELECT Name, Address, Zip5, Zip4, Age
 FROM table2 RIGHT OUTER JOIN Table1 ON Table2.Name=Table1.Name AND
Table2.Address=Table1.Address AND Table2.Zip5=Table1.Zip5 AND
Table2.Zip4=Table1.Zip4
 WHERE Table1.Name IS NULL AND Table1.Address IS NULL AND Table1.Zip5 IS
NULL AND Table1.Zip4 IS NULL
END

2009/9/3 Gustav Brock <Gustav at cactus.dk>

> Hi John (I have posted this in 2000, 2001, and 2006 - even as a reply to a
> posting of yours)
>
> This tip from Smart Access is one of my favourites:
>
> <quote>
>
> Update and Append Records with One Query
>
> By Alan Biggs
>
> Did you know that you can use an update query in Access to both update and
> add records at the same time? This is useful if you have two versions of a
> table, tblOld and tblNew, and you want to integrate the changes from tblNew
> into tblOld.
>
> Follow these steps:
>
> 1. Create an update query and add the two tables. Join the two tables by
> dragging the key field of tblNew onto the matching field of tblOld.
>
> 2. Double-click on the relationship and choose the join option that
> includes all records from tblNew and only those that match from tblOld.
>
> 3. Select all the fields from tblOld and drag them onto the QBE grid.
>
> 4. For each field, in the Update To cell type in tblNew.FieldName, where
> FieldName matches the field name of tblOld.
>
> 5. Select Query Properties from the View menu and change Unique Records to
> False. (This switches off the DISTINCTROW option in the SQL view. If you
> leave this on you'll get only one blank record in your results, but you want
> one blank record for each new record to be added to tblOld.)
>
> 6. Run the query and you'll see the changes to tblNew are now in tblOld.
>
> This will only add records to tblOld that have been added to tblNew.
> Records in tblOld that aren't present in tblNew will still remain in
> tblOld.
>
> </quote>
>
> I guess it works in SQL Server as well.
>
> /gustav
>
>
> >>> jwcolby at colbyconsulting.com 03-09-2009 16:32 >>>
> I have a set of 5 tables.  These tables contain names and number of
> children in age ranges.  As a
> first pass at processing, I need to merge the 5 tables into one table such
> that the name / address
> only gets into the table one time, IOW every record in the first table, but
> only the records in
> table 2 where the name / address data is not already in the destination
> table.
>
> SQL Server 2005.  Clustered index on the name / address / zip5 / zip4 of
> the source tables.
>
> I have not been able to find an append syntax that will append the records
> not in the table but
> prevent appending those already in, without aborting the entire append
> operation.
>
> What is the magic key here?
>
> --
> John W. Colby
> www.ColbyConsulting.com <http://www.colbyconsulting.com/>
>
>
>
>  _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>
>


-- 
Paul Hartland
paul.hartland at googlemail.com


From jwcolby at colbyconsulting.com  Thu Sep  3 10:22:57 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Thu, 03 Sep 2009 11:22:57 -0400
Subject: [dba-SQLServer] Append only records where some fields not in
 the table already
In-Reply-To: <38c884770909030749u1a446a2p61dc085743945d91@mail.gmail.com>
References: <sa9ff256.021@cactus.dk>
	<38c884770909030749u1a446a2p61dc085743945d91@mail.gmail.com>
Message-ID: <4A9FDF51.2040704@colbyconsulting.com>

Thanks Paul.  While I did not use that directly it triggered the duhhh... solution

Which is to create a view that pulls just those records.  Now create an append into the table using 
the view as the select part of the append statement.

Duuuhhh...

Actually I had never created a "not in" query like that in SQL Server but it worked like a champ.

John W. Colby
www.ColbyConsulting.com


Paul Hartland wrote:
> John,
> 
> Not had time to test this as just leaving work, but don't think it will be
> too far out, then repeat for tables 3,4,5.
> 
> CREATE PROCEDURE MergeTables
> AS
> BEGIN
>  INSERT INTO Table1 (Name, Address, Zip5, Zip4, Age)
>  SELECT Name, Address, Zip5, Zip4, Age
>  FROM table2 RIGHT OUTER JOIN Table1 ON Table2.Name=Table1.Name AND
> Table2.Address=Table1.Address AND Table2.Zip5=Table1.Zip5 AND
> Table2.Zip4=Table1.Zip4
>  WHERE Table1.Name IS NULL AND Table1.Address IS NULL AND Table1.Zip5 IS
> NULL AND Table1.Zip4 IS NULL
> END
> 
> 2009/9/3 Gustav Brock <Gustav at cactus.dk>
> 
>> Hi John (I have posted this in 2000, 2001, and 2006 - even as a reply to a
>> posting of yours)
>>
>> This tip from Smart Access is one of my favourites:
>>
>> <quote>
>>
>> Update and Append Records with One Query
>>
>> By Alan Biggs
>>
>> Did you know that you can use an update query in Access to both update and
>> add records at the same time? This is useful if you have two versions of a
>> table, tblOld and tblNew, and you want to integrate the changes from tblNew
>> into tblOld.
>>
>> Follow these steps:
>>
>> 1. Create an update query and add the two tables. Join the two tables by
>> dragging the key field of tblNew onto the matching field of tblOld.
>>
>> 2. Double-click on the relationship and choose the join option that
>> includes all records from tblNew and only those that match from tblOld.
>>
>> 3. Select all the fields from tblOld and drag them onto the QBE grid.
>>
>> 4. For each field, in the Update To cell type in tblNew.FieldName, where
>> FieldName matches the field name of tblOld.
>>
>> 5. Select Query Properties from the View menu and change Unique Records to
>> False. (This switches off the DISTINCTROW option in the SQL view. If you
>> leave this on you'll get only one blank record in your results, but you want
>> one blank record for each new record to be added to tblOld.)
>>
>> 6. Run the query and you'll see the changes to tblNew are now in tblOld.
>>
>> This will only add records to tblOld that have been added to tblNew.
>> Records in tblOld that aren't present in tblNew will still remain in
>> tblOld.
>>
>> </quote>
>>
>> I guess it works in SQL Server as well.
>>
>> /gustav
>>
>>
>>>>> jwcolby at colbyconsulting.com 03-09-2009 16:32 >>>
>> I have a set of 5 tables.  These tables contain names and number of
>> children in age ranges.  As a
>> first pass at processing, I need to merge the 5 tables into one table such
>> that the name / address
>> only gets into the table one time, IOW every record in the first table, but
>> only the records in
>> table 2 where the name / address data is not already in the destination
>> table.
>>
>> SQL Server 2005.  Clustered index on the name / address / zip5 / zip4 of
>> the source tables.
>>
>> I have not been able to find an append syntax that will append the records
>> not in the table but
>> prevent appending those already in, without aborting the entire append
>> operation.
>>
>> What is the magic key here?
>>
>> --
>> John W. Colby
>> www.ColbyConsulting.com <http://www.colbyconsulting.com/>
>>
>>
>>
>>  _______________________________________________
>> dba-SQLServer mailing list
>> dba-SQLServer at databaseadvisors.com
>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>> http://www.databaseadvisors.com
>>
>>
> 
> 


From jwcolby at colbyconsulting.com  Thu Sep  3 10:26:01 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Thu, 03 Sep 2009 11:26:01 -0400
Subject: [dba-SQLServer] Append only records where some fields not	in
 the table already
In-Reply-To: <sa9ff256.021@cactus.dk>
References: <sa9ff256.021@cactus.dk>
Message-ID: <4A9FE009.1020203@colbyconsulting.com>

Gustav,

You have a mind like a steel trap.  I on the other hand have a mind like an rusted old iron trap, 
set back in the early 1800s and long since buried under 2 feet of leaves and mud.

Unfortunately I cannot figure out how to do this in SQL Server.  But I did get a solution based on 
Paul's suggestion.

Thanks for the help.

John W. Colby
www.ColbyConsulting.com


Gustav Brock wrote:
> Hi John (I have posted this in 2000, 2001, and 2006 - even as a reply to a posting of yours)
> 
> This tip from Smart Access is one of my favourites:



From Gustav at cactus.dk  Thu Sep  3 10:41:07 2009
From: Gustav at cactus.dk (Gustav Brock)
Date: Thu, 03 Sep 2009 17:41:07 +0200
Subject: [dba-SQLServer] Append only records where some fields not	in
	the table already
Message-ID: <sa9fffd6.022@cactus.dk>

Hi John

You are welcome. 
This is really too easy if you use the SQL query GUI of Access.

/gustav


>>> jwcolby at colbyconsulting.com 03-09-2009 17:26 >>>
Gustav,

You have a mind like a steel trap.  I on the other hand have a mind like an rusted old iron trap, 
set back in the early 1800s and long since buried under 2 feet of leaves and mud.

Unfortunately I cannot figure out how to do this in SQL Server.  But I did get a solution based on 
Paul's suggestion.

Thanks for the help.

John W. Colby
www.ColbyConsulting.com 


Gustav Brock wrote:
> Hi John (I have posted this in 2000, 2001, and 2006 - even as a reply to a posting of yours)
> 
> This tip from Smart Access is one of my favourites:





From jwcolby at colbyconsulting.com  Thu Sep  3 15:17:06 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Thu, 03 Sep 2009 16:17:06 -0400
Subject: [dba-SQLServer] Append only records where some fields not	in
 the table already
In-Reply-To: <sa9fffd6.022@cactus.dk>
References: <sa9fffd6.022@cactus.dk>
Message-ID: <4AA02442.6000608@colbyconsulting.com>

The gui in SQL server is quite similar but not exactly like the Access version.  I have followed 
your directions in Access and it is in fact quite easy.

It turns out that I need to perform about 10 "steps" to get all of the tables appended into one, and 
then go back and update a couple of new fields as required.  I have already started doing this as a 
set of stored procedures so I can just click and run the whole lot.

Thanks again.

John W. Colby
www.ColbyConsulting.com


Gustav Brock wrote:
> Hi John
> 
> You are welcome. 
> This is really too easy if you use the SQL query GUI of Access.
> 
> /gustav
> 
> 
>>>> jwcolby at colbyconsulting.com 03-09-2009 17:26 >>>
> Gustav,
> 
> You have a mind like a steel trap.  I on the other hand have a mind like an rusted old iron trap, 
> set back in the early 1800s and long since buried under 2 feet of leaves and mud.
> 
> Unfortunately I cannot figure out how to do this in SQL Server.  But I did get a solution based on 
> Paul's suggestion.
> 
> Thanks for the help.
> 
> John W. Colby
> www.ColbyConsulting.com 
> 
> 
> Gustav Brock wrote:
>> Hi John (I have posted this in 2000, 2001, and 2006 - even as a reply to a posting of yours)
>>
>> This tip from Smart Access is one of my favourites:
> 
> 
> 
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From Gustav at cactus.dk  Fri Sep  4 10:09:38 2009
From: Gustav at cactus.dk (Gustav Brock)
Date: Fri, 04 Sep 2009 17:09:38 +0200
Subject: [dba-SQLServer] SQL Azure and SSMS
Message-ID: <saa149dd.060@cactus.dk>

Hi all

As you may have noticed, this is up:

---
Welcome to the SQL Azure CTP!  We are excited to have you join us.  
Below are some steps to get you started, as well as links to helpful resources.  
Getting Started

1)     Visit https://sql.azure.com 
2)     Sign in with a valid Windows LiveID
3)     Enter your invitation code
---

What they don't tell is how to connect with SSMS to this bastard. However, this page was a great help for me:

http://english.zachskylesowens.net/2009/08/18/connecting-to-sql-azure/

But surprise: No object browser, no nothing. You are alone in the dark.

So what to do? I'm lazy and not in the mood for writing manual SQL code for anything.
Any tools or smart methods?

/gustav





From jlawrenc1 at shaw.ca  Fri Sep  4 10:28:11 2009
From: jlawrenc1 at shaw.ca (Jim Lawrence)
Date: Fri, 4 Sep 2009 08:28:11 -0700
Subject: [dba-SQLServer] SQL Azure and SSMS
In-Reply-To: <saa149dd.060@cactus.dk>
References: <saa149dd.060@cactus.dk>
Message-ID: <4AD7E8923B6C460E85367E19C66308AD@creativesystemdesigns.com>

Hi Gustav:

Well, that looks like a first step. I like yourself will wait for some
bright boy or girl, with time and foccus, to build a nice gui interface for
the a browser.

Jim


-----Original Message-----
From: dba-sqlserver-bounces at databaseadvisors.com
[mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Gustav
Brock
Sent: Friday, September 04, 2009 8:10 AM
To: dba-sqlserver at databaseadvisors.com
Subject: [dba-SQLServer] SQL Azure and SSMS

Hi all

As you may have noticed, this is up:

---
Welcome to the SQL Azure CTP!  We are excited to have you join us.  
Below are some steps to get you started, as well as links to helpful
resources.  
Getting Started

1)     Visit https://sql.azure.com 
2)     Sign in with a valid Windows LiveID
3)     Enter your invitation code
---

What they don't tell is how to connect with SSMS to this bastard. However,
this page was a great help for me:

http://english.zachskylesowens.net/2009/08/18/connecting-to-sql-azure/

But surprise: No object browser, no nothing. You are alone in the dark.

So what to do? I'm lazy and not in the mood for writing manual SQL code for
anything.
Any tools or smart methods?

/gustav



_______________________________________________
dba-SQLServer mailing list
dba-SQLServer at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
http://www.databaseadvisors.com



From Gustav at cactus.dk  Fri Sep  4 10:44:49 2009
From: Gustav at cactus.dk (Gustav Brock)
Date: Fri, 04 Sep 2009 17:44:49 +0200
Subject: [dba-SQLServer] SQL Azure and SSMS
Message-ID: <saa15231.061@cactus.dk>

Hi Jim

I located this (which I haven't run yet) SQL Azure Manager (alpha):
http://hanssens.org/post/SQL-Azure-Manager.aspx

and this tiny guide:
https://blogs.msdn.com/jimoneil/archive/2009/08/18/sql-azure-ctp-1.aspx 

which I followed, and voila - a select query returned:
(1 row(s) affected)

So it works.

/gustav


>>> jlawrenc1 at shaw.ca 04-09-2009 17:28 >>>
Hi Gustav:

Well, that looks like a first step. I like yourself will wait for some
bright boy or girl, with time and foccus, to build a nice gui interface for
the a browser.

Jim


-----Original Message-----
From: dba-sqlserver-bounces at databaseadvisors.com 
[mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Gustav
Brock
Sent: Friday, September 04, 2009 8:10 AM
To: dba-sqlserver at databaseadvisors.com 
Subject: [dba-SQLServer] SQL Azure and SSMS

Hi all

As you may have noticed, this is up:

---
Welcome to the SQL Azure CTP!  We are excited to have you join us.  
Below are some steps to get you started, as well as links to helpful resources.  
Getting Started

1)     Visit https://sql.azure.com 
2)     Sign in with a valid Windows LiveID
3)     Enter your invitation code
---

What they don't tell is how to connect with SSMS to this bastard. However,
this page was a great help for me:

http://english.zachskylesowens.net/2009/08/18/connecting-to-sql-azure/ 

But surprise: No object browser, no nothing. You are alone in the dark.

So what to do? I'm lazy and not in the mood for writing manual SQL code for anything.
Any tools or smart methods?

/gustav





From jwcolby at colbyconsulting.com  Fri Sep  4 10:55:48 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Fri, 04 Sep 2009 11:55:48 -0400
Subject: [dba-SQLServer] SQL Server 2005 unresponsive
Message-ID: <4AA13884.7000707@colbyconsulting.com>

I am looking for a little information about an issue I have with my SQL Server 2005 instance which 
becomes unresponsive when a long running job from Access occurs.

I have a set of stored procedures that builds a table, imports 2 million record files into an import 
table, appends that to the main table, then when all the 2 million record files are imported, starts 
building up indexes.  Something is happening (one of these index builds I suspect) that causes 
Access to "go away" and when this happens SQL Server "goes away" as well.

Access goes "not responding" which makes sense given the single threading nature of VBA, however the 
server goes unresponsive as well.  In fact I tried to get the database properties for the database 
being worked in and got a blue screen.  I cannot prove that the blue screen was caused by SQL Server 
but... that was the last thing I did before the blue screen.

So fine, I opened another instance of management studio so that I could go build an index in a 
completely different db / table.  That instance of Management studio took a looooooonnnnnggggg time 
even opening.  Expanding the database normally is instantaneous.  ATM it takes many seconds. 
Likewise expanding the tables, etc.  Each step which normally "just happens" takes a much longer time.

Is this normal behavior?  The server is a quad core (AMD) with 16 gigs of ram, with 12 gigs assigned 
to SQL Server.  I would have thought that SQL Server would allow multiple things to occur quickly.

-- 
John W. Colby
www.ColbyConsulting.com


From nancy.lytle at gmail.com  Fri Sep  4 11:26:50 2009
From: nancy.lytle at gmail.com (Nancy Lytle)
Date: Fri, 4 Sep 2009 11:26:50 -0500
Subject: [dba-SQLServer] SQL Server 2005 unresponsive
In-Reply-To: <4AA13884.7000707@colbyconsulting.com>
References: <4AA13884.7000707@colbyconsulting.com>
Message-ID: <BAY134-W1CB1E3484D56C0CF166F184EE0@phx.gbl>


How is your SQL temp db set up, size location, etc? It could be the cause of your issue if it grows out of hand and fills up the drive it is on. Do you use Simple or Bulk logged recovery?  Have you tried running profiler or a trace to capture what is happening?
Just some ideas.


Nancy Lytle 











 EMAILING FOR THE GREATER GOOD
Join me


 
> Date: Fri, 4 Sep 2009 11:55:48 -0400
> From: jwcolby at colbyconsulting.com
> To: dba-sqlserver at databaseadvisors.com
> Subject: [dba-SQLServer] SQL Server 2005 unresponsive
> 
> I am looking for a little information about an issue I have with my SQL Server 2005 instance which 
> becomes unresponsive when a long running job from Access occurs.
> 
> I have a set of stored procedures that builds a table, imports 2 million record files into an import 
> table, appends that to the main table, then when all the 2 million record files are imported, starts 
> building up indexes. Something is happening (one of these index builds I suspect) that causes 
> Access to "go away" and when this happens SQL Server "goes away" as well.
> 
> Access goes "not responding" which makes sense given the single threading nature of VBA, however the 
> server goes unresponsive as well. In fact I tried to get the database properties for the database 
> being worked in and got a blue screen. I cannot prove that the blue screen was caused by SQL Server 
> but... that was the last thing I did before the blue screen.
> 
> So fine, I opened another instance of management studio so that I could go build an index in a 
> completely different db / table. That instance of Management studio took a looooooonnnnnggggg time 
> even opening. Expanding the database normally is instantaneous. ATM it takes many seconds. 
> Likewise expanding the tables, etc. Each step which normally "just happens" takes a much longer time.
> 
> Is this normal behavior? The server is a quad core (AMD) with 16 gigs of ram, with 12 gigs assigned 
> to SQL Server. I would have thought that SQL Server would allow multiple things to occur quickly.
> 
> -- 
> John W. Colby
> www.ColbyConsulting.com
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 


From accessd at shaw.ca  Fri Sep  4 11:37:07 2009
From: accessd at shaw.ca (Jim Lawrence)
Date: Fri, 4 Sep 2009 09:37:07 -0700
Subject: [dba-SQLServer] SQL Server 2005 unresponsive
In-Reply-To: <4AA13884.7000707@colbyconsulting.com>
References: <4AA13884.7000707@colbyconsulting.com>
Message-ID: <8C76543CDDFD4F6FBB8A8C9527EBBA93@creativesystemdesigns.com>

I think the issue is Access related and not MS SQL. Had exactly the same
issues when connected to an Oracle DB. I ended up writing code that would
save my results to a temporary table, in the Oracle DB and then would have
Access just retrieve the table results. 

The single process step of gathering the data and downloading a recordset,
via ADO would sometimes cause a hang up. If I used ODBC, it would always
cause a hold up especially on larger data sets.

Have you tried .Net?

Jim


-----Original Message-----
From: dba-sqlserver-bounces at databaseadvisors.com
[mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby
Sent: Friday, September 04, 2009 8:56 AM
To: Dba-Sqlserver
Subject: [dba-SQLServer] SQL Server 2005 unresponsive

I am looking for a little information about an issue I have with my SQL
Server 2005 instance which 
becomes unresponsive when a long running job from Access occurs.

I have a set of stored procedures that builds a table, imports 2 million
record files into an import 
table, appends that to the main table, then when all the 2 million record
files are imported, starts 
building up indexes.  Something is happening (one of these index builds I
suspect) that causes 
Access to "go away" and when this happens SQL Server "goes away" as well.

Access goes "not responding" which makes sense given the single threading
nature of VBA, however the 
server goes unresponsive as well.  In fact I tried to get the database
properties for the database 
being worked in and got a blue screen.  I cannot prove that the blue screen
was caused by SQL Server 
but... that was the last thing I did before the blue screen.

So fine, I opened another instance of management studio so that I could go
build an index in a 
completely different db / table.  That instance of Management studio took a
looooooonnnnnggggg time 
even opening.  Expanding the database normally is instantaneous.  ATM it
takes many seconds. 
Likewise expanding the tables, etc.  Each step which normally "just happens"
takes a much longer time.

Is this normal behavior?  The server is a quad core (AMD) with 16 gigs of
ram, with 12 gigs assigned 
to SQL Server.  I would have thought that SQL Server would allow multiple
things to occur quickly.

-- 
John W. Colby
www.ColbyConsulting.com
_______________________________________________
dba-SQLServer mailing list
dba-SQLServer at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
http://www.databaseadvisors.com



From jwcolby at colbyconsulting.com  Fri Sep  4 12:06:35 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Fri, 04 Sep 2009 13:06:35 -0400
Subject: [dba-SQLServer] SPAM-LOW:  Re:  SQL Server 2005 unresponsive
In-Reply-To: <BAY134-W1CB1E3484D56C0CF166F184EE0@phx.gbl>
References: <4AA13884.7000707@colbyconsulting.com>
	<BAY134-W1CB1E3484D56C0CF166F184EE0@phx.gbl>
Message-ID: <4AA1491B.5060606@colbyconsulting.com>

Thanks for the reply.

The temp file is going to my C: drive which has about 37 gig used and 250 gig free, so I assume that 
isn't it.

 >Do you use Simple or Bulk logged recovery?

Unfortunately I don't know.  I looked at that stuff ages ago, I think I set it up to use bulk 
because of the massive import / export that I do but at this moment I do not know.

 >Have you tried running profiler or a trace to capture what is happening?

Unfortunately I do not even know how.  Remember that I am a one person shop.  I build my own 
computers, install my own servers, set up my own SQL Server, office, email and everything else that 
I need.  I AM the IT department.  Additionally my REAL work is doing the work that uses these tools, 
writing applications in some cases, processing data in others.

I can only be well versed in so many things and the ins and outs of SQL Server is not one of the 
things that I am well versed in.  Sad but true.

So whenever I ask for assistance I am really asking for more than "have you tried running a 
profiler".  What is a profiler, how do I set one up?  What am I profiling?  Not racial profiling I 
hope (that's a BAD thing).

I am truly in an unfortunate position here in that it takes YEARS for SQL Server administrators 
become good at their jobs and I am not an SQL Server administrator, nor do I want to be one.  I have 
to learn just enough to do exactly what I need and then I have to get back to work on my real job. 
I have two other clients banging on me to get Access work done for them.

Sigh.

So in the end I just put up with problems like this, start it running and go do something else while 
it takes forever to complete.  It is just discouraging since I have spent a lot of money getting 
hardware that should allow SQL Server to fly, and it SOMETIMES seems to limp along like a crippled 
old dog.  Other times it does indeed seem to fly.  Strange.

John W. Colby
www.ColbyConsulting.com


Nancy Lytle wrote:
> How is your SQL temp db set up, size location, etc? It could be the cause of your issue if it grows out of hand and fills up the drive it is on. Do you use Simple or Bulk logged recovery?  Have you tried running profiler or a trace to capture what is happening?
> Just some ideas.
> 
> 
> Nancy Lytle 



From jwcolby at colbyconsulting.com  Fri Sep  4 12:12:16 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Fri, 04 Sep 2009 13:12:16 -0400
Subject: [dba-SQLServer] SQL Server 2005 unresponsive
In-Reply-To: <8C76543CDDFD4F6FBB8A8C9527EBBA93@creativesystemdesigns.com>
References: <4AA13884.7000707@colbyconsulting.com>
	<8C76543CDDFD4F6FBB8A8C9527EBBA93@creativesystemdesigns.com>
Message-ID: <4AA14A70.5020501@colbyconsulting.com>

Jim,

Thanks for the response.  I have to agree, it does seem to be something ODBC / Access related in 
that pretty much everything I do directly inside of SQL Server works just fine.  It is annoying that 
SQL Server would ALLOW an ODBC call from Access to cause it to go into turtle mode though.

 > Have you tried .Net?

Oddly enough I have, but it was some time ago.  I was doing this stuff in VB.Net / ADO, but I was so 
slow at the .Net development that when I needed to "get something done" I just went back to Access. 
  I cannot make any valid comparisons however as back when I was doing the .Net work I also was a 
complete nube at the SQL Server stuff as well, couldn't even write a SP.

I am taking a C# class now and will be moving squarely into C# in the future.  I have pretty much 
made a decision to make that my future.  But in the here and now...

John W. Colby
www.ColbyConsulting.com


Jim Lawrence wrote:
> I think the issue is Access related and not MS SQL. Had exactly the same
> issues when connected to an Oracle DB. I ended up writing code that would
> save my results to a temporary table, in the Oracle DB and then would have
> Access just retrieve the table results. 
> 
> The single process step of gathering the data and downloading a recordset,
> via ADO would sometimes cause a hang up. If I used ODBC, it would always
> cause a hold up especially on larger data sets.
> 
> Have you tried .Net?
> 
> Jim



From fhtapia at gmail.com  Fri Sep  4 12:28:10 2009
From: fhtapia at gmail.com (Francisco Tapia)
Date: Fri, 4 Sep 2009 10:28:10 -0700
Subject: [dba-SQLServer] SQL Server 2005 unresponsive
In-Reply-To: <4AA14A70.5020501@colbyconsulting.com>
References: <4AA13884.7000707@colbyconsulting.com>
	<8C76543CDDFD4F6FBB8A8C9527EBBA93@creativesystemdesigns.com>
	<4AA14A70.5020501@colbyconsulting.com>
Message-ID: <b874372a0909041028t6079917fr5ae75c8939b5555f@mail.gmail.com>

It's not Sql Server ALLOWING the ODBC call to send it to turtle mode, it's
ODBC, that library is inherently slow, and has problems with larger
datasets.

If you are seeing slowness on your SQL Server you'll need to fine tune a few
things


First check the size of of your transaction logs, whenever they exceed 50%
of the data file size, then you can begin running into some performance
problems.  (normally i see this if my transaction log file is 2x or more
than my data file.)

also check the tempdb

When you are MOVING a lot of data around, the way you are, you want to have
each data partition on it's own set of spindle disks, this applies largely
to your transaction logs, because you don't want to loose the data but you
want the system to process extremely quick, you should look into raid 10 if
you can afford it. For TempDB it's possible to squeeze a lot of performance
on the disks, first tempdb is always rebuilt when the server comes back up,
and all data is just temporary, so you could go with a pair of raid 0
drives, possibly 3 spindles if you need the extra speed.  Striping is the
fastest possible solution to speeding up IO,  ideally of course you'll want
to have this on it's own separate i/o card and own channel if possible.

benchmark your system by running it through the paces of the SQLIO
benchmarking tool from MS
http://www.microsoft.com/DOWNLOADS/details.aspx?familyid=9A8B005B-84E4-4F24-8D65-CB53442D9E19&displaylang=en
and benchmark as you move towards faster drives to see the new throughput.
In your case you might be easily impressed by moving the tempdb to a set of
raid 0 drives.


-Francisco
http://sqlthis.blogspot.com | Tsql and More...


On Fri, Sep 4, 2009 at 10:12 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:

> Jim,
>
> Thanks for the response.  I have to agree, it does seem to be something
> ODBC / Access related in
> that pretty much everything I do directly inside of SQL Server works just
> fine.  It is annoying that
> SQL Server would ALLOW an ODBC call from Access to cause it to go into
> turtle mode though.
>
>  > Have you tried .Net?
>
> Oddly enough I have, but it was some time ago.  I was doing this stuff in
> VB.Net / ADO, but I was so
> slow at the .Net development that when I needed to "get something done" I
> just went back to Access.
>  I cannot make any valid comparisons however as back when I was doing the
> .Net work I also was a
> complete nube at the SQL Server stuff as well, couldn't even write a SP.
>
> I am taking a C# class now and will be moving squarely into C# in the
> future.  I have pretty much
> made a decision to make that my future.  But in the here and now...
>
> John W. Colby
> www.ColbyConsulting.com
>
>
> Jim Lawrence wrote:
> > I think the issue is Access related and not MS SQL. Had exactly the same
> > issues when connected to an Oracle DB. I ended up writing code that would
> > save my results to a temporary table, in the Oracle DB and then would
> have
> > Access just retrieve the table results.
> >
> > The single process step of gathering the data and downloading a
> recordset,
> > via ADO would sometimes cause a hang up. If I used ODBC, it would always
> > cause a hold up especially on larger data sets.
> >
> > Have you tried .Net?
> >
> > Jim
>
>
>


From fhtapia at gmail.com  Fri Sep  4 13:33:21 2009
From: fhtapia at gmail.com (Francisco Tapia)
Date: Fri, 4 Sep 2009 11:33:21 -0700
Subject: [dba-SQLServer] SQL Azure and SSMS
In-Reply-To: <saa15231.061@cactus.dk>
References: <saa15231.061@cactus.dk>
Message-ID: <b874372a0909041133x3b384deawd59970d564a99c09@mail.gmail.com>

Wow this works great!,  thanks for the link Gustav

-Francisco
http://sqlthis.blogspot.com | Tsql and More...


On Fri, Sep 4, 2009 at 8:44 AM, Gustav Brock <Gustav at cactus.dk> wrote:

> Hi Jim
>
> I located this (which I haven't run yet) SQL Azure Manager (alpha):
> http://hanssens.org/post/SQL-Azure-Manager.aspx
>
> and this tiny guide:
> https://blogs.msdn.com/jimoneil/archive/2009/08/18/sql-azure-ctp-1.aspx
>
> which I followed, and voila - a select query returned:
> (1 row(s) affected)
>
> So it works.
>
> /gustav
>
>
> >>> jlawrenc1 at shaw.ca 04-09-2009 17:28 >>>
> Hi Gustav:
>
> Well, that looks like a first step. I like yourself will wait for some
> bright boy or girl, with time and foccus, to build a nice gui interface for
> the a browser.
>
> Jim
>
>
> -----Original Message-----
> From: dba-sqlserver-bounces at databaseadvisors.com
> [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Gustav
> Brock
> Sent: Friday, September 04, 2009 8:10 AM
> To: dba-sqlserver at databaseadvisors.com
> Subject: [dba-SQLServer] SQL Azure and SSMS
>
> Hi all
>
> As you may have noticed, this is up:
>
> ---
> Welcome to the SQL Azure CTP!  We are excited to have you join us.
> Below are some steps to get you started, as well as links to helpful
> resources.
> Getting Started
>
> 1)     Visit https://sql.azure.com
> 2)     Sign in with a valid Windows LiveID
> 3)     Enter your invitation code
> ---
>
> What they don't tell is how to connect with SSMS to this bastard. However,
> this page was a great help for me:
>
> http://english.zachskylesowens.net/2009/08/18/connecting-to-sql-azure/
>
> But surprise: No object browser, no nothing. You are alone in the dark.
>
> So what to do? I'm lazy and not in the mood for writing manual SQL code for
> anything.
> Any tools or smart methods?
>
> /gustav
>
>
>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>
>


From jwcolby at colbyconsulting.com  Fri Sep  4 15:26:06 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Fri, 04 Sep 2009 16:26:06 -0400
Subject: [dba-SQLServer] SQL Server 2005 unresponsive
In-Reply-To: <b874372a0909041028t6079917fr5ae75c8939b5555f@mail.gmail.com>
References: <4AA13884.7000707@colbyconsulting.com>	<8C76543CDDFD4F6FBB8A8C9527EBBA93@creativesystemdesigns.com>	<4AA14A70.5020501@colbyconsulting.com>
	<b874372a0909041028t6079917fr5ae75c8939b5555f@mail.gmail.com>
Message-ID: <4AA177DE.8080600@colbyconsulting.com>

Francisco,

Thanks for the suggestions.

 > It's not Sql Server ALLOWING the ODBC call to send it to turtle mode, it's ODBC, that library is 
inherently slow, and has problems with larger datasets.

Yes but - how can it cause SQL server to lock it up?  All that is going on is that Access is running 
Stored procedures.  Now these stored procedures do big operations, but still...

I thought that SQL Server would magically prevent anyone from bringing it to it's knees.  Naive I know.

You may very well be right of course.  I have so much data to store that I have been concentrating 
on size instead of "spindles".  I have some spare drives I can use.  I will throw those on there 
Raid 0.  I doubt seriously that the CURRENT bottle neck is the controller, though it may very well 
change to that if I do a what we are discussing.

Actually I do have a spare card hanging around.  In fact i put it in my other server (just to see if 
it could be used simultaneously) and when I took it back out the raid software puts up a "raid 
missing" message at boot up.  I cannot get rid of that message.

Anyway... I could put that in and throw some extra drives on that controller.  My bigger problem 
really is a another power supply.  That server currently has 11 hard drives in it.

I will definitely do that though just to see what that gets me.  I have to do something!.

John W. Colby
www.ColbyConsulting.com


Francisco Tapia wrote:
> It's not Sql Server ALLOWING the ODBC call to send it to turtle mode, it's
> ODBC, that library is inherently slow, and has problems with larger
> datasets.
> 
> If you are seeing slowness on your SQL Server you'll need to fine tune a few
> things
> 
> 
> First check the size of of your transaction logs, whenever they exceed 50%
> of the data file size, then you can begin running into some performance
> problems.  (normally i see this if my transaction log file is 2x or more
> than my data file.)
> 
> also check the tempdb
> 
> When you are MOVING a lot of data around, the way you are, you want to have
> each data partition on it's own set of spindle disks, this applies largely
> to your transaction logs, because you don't want to loose the data but you
> want the system to process extremely quick, you should look into raid 10 if
> you can afford it. For TempDB it's possible to squeeze a lot of performance
> on the disks, first tempdb is always rebuilt when the server comes back up,
> and all data is just temporary, so you could go with a pair of raid 0
> drives, possibly 3 spindles if you need the extra speed.  Striping is the
> fastest possible solution to speeding up IO,  ideally of course you'll want
> to have this on it's own separate i/o card and own channel if possible.
> 
> benchmark your system by running it through the paces of the SQLIO
> benchmarking tool from MS
> http://www.microsoft.com/DOWNLOADS/details.aspx?familyid=9A8B005B-84E4-4F24-8D65-CB53442D9E19&displaylang=en
> and benchmark as you move towards faster drives to see the new throughput.
> In your case you might be easily impressed by moving the tempdb to a set of
> raid 0 drives.
> 
> 
> -Francisco
> http://sqlthis.blogspot.com | Tsql and More...
> 
> 
> On Fri, Sep 4, 2009 at 10:12 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:
> 
>> Jim,
>>
>> Thanks for the response.  I have to agree, it does seem to be something
>> ODBC / Access related in
>> that pretty much everything I do directly inside of SQL Server works just
>> fine.  It is annoying that
>> SQL Server would ALLOW an ODBC call from Access to cause it to go into
>> turtle mode though.
>>
>>  > Have you tried .Net?
>>
>> Oddly enough I have, but it was some time ago.  I was doing this stuff in
>> VB.Net / ADO, but I was so
>> slow at the .Net development that when I needed to "get something done" I
>> just went back to Access.
>>  I cannot make any valid comparisons however as back when I was doing the
>> .Net work I also was a
>> complete nube at the SQL Server stuff as well, couldn't even write a SP.
>>
>> I am taking a C# class now and will be moving squarely into C# in the
>> future.  I have pretty much
>> made a decision to make that my future.  But in the here and now...
>>
>> John W. Colby
>> www.ColbyConsulting.com
>>
>>
>> Jim Lawrence wrote:
>>> I think the issue is Access related and not MS SQL. Had exactly the same
>>> issues when connected to an Oracle DB. I ended up writing code that would
>>> save my results to a temporary table, in the Oracle DB and then would
>> have
>>> Access just retrieve the table results.
>>>
>>> The single process step of gathering the data and downloading a
>> recordset,
>>> via ADO would sometimes cause a hang up. If I used ODBC, it would always
>>> cause a hold up especially on larger data sets.
>>>
>>> Have you tried .Net?
>>>
>>> Jim
>>
>>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From ebarro at roadrunner.com  Fri Sep  4 16:06:34 2009
From: ebarro at roadrunner.com (Eric Barro)
Date: Fri, 4 Sep 2009 14:06:34 -0700
Subject: [dba-SQLServer] SQL Server 2005 unresponsive
In-Reply-To: <4AA177DE.8080600@colbyconsulting.com>
References: <4AA13884.7000707@colbyconsulting.com>	<8C76543CDDFD4F6FBB8A8C9527EBBA93@creativesystemdesigns.com>	<4AA14A70.5020501@colbyconsulting.com><b874372a0909041028t6079917fr5ae75c8939b5555f@mail.gmail.com>
	<4AA177DE.8080600@colbyconsulting.com>
Message-ID: <1BF060B6F676489398384FADE8B5CF14@advancedinput.com>

John,

Access has to connect to SQL server and SQL server treats the Access
connection as a Linked Server using ODBC. As already mentioned and it is a
well-known fact, ODBC is such a lumbering beast when it comes to large
datasets.

Trust me I've seen this happen many times over and whenever a vendor comes
forward with a solution that involves ODBC connectivity to SQL server I
immediately raise red flags.

Eric 

-----Original Message-----
From: dba-sqlserver-bounces at databaseadvisors.com
[mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby
Sent: Friday, September 04, 2009 1:26 PM
To: Discussion concerning MS SQL Server
Subject: Re: [dba-SQLServer] SQL Server 2005 unresponsive

Francisco,

Thanks for the suggestions.

 > It's not Sql Server ALLOWING the ODBC call to send it to turtle mode,
it's ODBC, that library is inherently slow, and has problems with larger
datasets.

Yes but - how can it cause SQL server to lock it up?  All that is going on
is that Access is running Stored procedures.  Now these stored procedures do
big operations, but still...

I thought that SQL Server would magically prevent anyone from bringing it to
it's knees.  Naive I know.

You may very well be right of course.  I have so much data to store that I
have been concentrating on size instead of "spindles".  I have some spare
drives I can use.  I will throw those on there Raid 0.  I doubt seriously
that the CURRENT bottle neck is the controller, though it may very well
change to that if I do a what we are discussing.

Actually I do have a spare card hanging around.  In fact i put it in my
other server (just to see if it could be used simultaneously) and when I
took it back out the raid software puts up a "raid missing" message at boot
up.  I cannot get rid of that message.

Anyway... I could put that in and throw some extra drives on that
controller.  My bigger problem really is a another power supply.  That
server currently has 11 hard drives in it.

I will definitely do that though just to see what that gets me.  I have to
do something!.

John W. Colby
www.ColbyConsulting.com


Francisco Tapia wrote:
> It's not Sql Server ALLOWING the ODBC call to send it to turtle mode, 
> it's ODBC, that library is inherently slow, and has problems with 
> larger datasets.
> 
> If you are seeing slowness on your SQL Server you'll need to fine tune 
> a few things
> 
> 
> First check the size of of your transaction logs, whenever they exceed 
> 50% of the data file size, then you can begin running into some 
> performance problems.  (normally i see this if my transaction log file 
> is 2x or more than my data file.)
> 
> also check the tempdb
> 
> When you are MOVING a lot of data around, the way you are, you want to 
> have each data partition on it's own set of spindle disks, this 
> applies largely to your transaction logs, because you don't want to 
> loose the data but you want the system to process extremely quick, you 
> should look into raid 10 if you can afford it. For TempDB it's 
> possible to squeeze a lot of performance on the disks, first tempdb is 
> always rebuilt when the server comes back up, and all data is just 
> temporary, so you could go with a pair of raid 0 drives, possibly 3 
> spindles if you need the extra speed.  Striping is the fastest 
> possible solution to speeding up IO,  ideally of course you'll want to
have this on it's own separate i/o card and own channel if possible.
> 
> benchmark your system by running it through the paces of the SQLIO 
> benchmarking tool from MS 
> http://www.microsoft.com/DOWNLOADS/details.aspx?familyid=9A8B005B-84E4
> -4F24-8D65-CB53442D9E19&displaylang=en
> and benchmark as you move towards faster drives to see the new throughput.
> In your case you might be easily impressed by moving the tempdb to a 
> set of raid 0 drives.
> 
> 
> -Francisco
> http://sqlthis.blogspot.com | Tsql and More...
> 
> 
> On Fri, Sep 4, 2009 at 10:12 AM, jwcolby
<jwcolby at colbyconsulting.com>wrote:
> 
>> Jim,
>>
>> Thanks for the response.  I have to agree, it does seem to be 
>> something ODBC / Access related in that pretty much everything I do 
>> directly inside of SQL Server works just fine.  It is annoying that 
>> SQL Server would ALLOW an ODBC call from Access to cause it to go 
>> into turtle mode though.
>>
>>  > Have you tried .Net?
>>
>> Oddly enough I have, but it was some time ago.  I was doing this 
>> stuff in VB.Net / ADO, but I was so slow at the .Net development that 
>> when I needed to "get something done" I just went back to Access.
>>  I cannot make any valid comparisons however as back when I was doing 
>> the .Net work I also was a complete nube at the SQL Server stuff as 
>> well, couldn't even write a SP.
>>
>> I am taking a C# class now and will be moving squarely into C# in the 
>> future.  I have pretty much made a decision to make that my future.  
>> But in the here and now...
>>
>> John W. Colby
>> www.ColbyConsulting.com
>>
>>
>> Jim Lawrence wrote:
>>> I think the issue is Access related and not MS SQL. Had exactly the 
>>> same issues when connected to an Oracle DB. I ended up writing code 
>>> that would save my results to a temporary table, in the Oracle DB 
>>> and then would
>> have
>>> Access just retrieve the table results.
>>>
>>> The single process step of gathering the data and downloading a
>> recordset,
>>> via ADO would sometimes cause a hang up. If I used ODBC, it would 
>>> always cause a hold up especially on larger data sets.
>>>
>>> Have you tried .Net?
>>>
>>> Jim
>>
>>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 
_______________________________________________
dba-SQLServer mailing list
dba-SQLServer at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
http://www.databaseadvisors.com



From fhtapia at gmail.com  Fri Sep  4 16:26:26 2009
From: fhtapia at gmail.com (Francisco Tapia)
Date: Fri, 4 Sep 2009 14:26:26 -0700
Subject: [dba-SQLServer] SQL Server 2005 unresponsive
In-Reply-To: <4AA177DE.8080600@colbyconsulting.com>
References: <4AA13884.7000707@colbyconsulting.com>
	<8C76543CDDFD4F6FBB8A8C9527EBBA93@creativesystemdesigns.com>
	<4AA14A70.5020501@colbyconsulting.com>
	<b874372a0909041028t6079917fr5ae75c8939b5555f@mail.gmail.com>
	<4AA177DE.8080600@colbyconsulting.com>
Message-ID: <b874372a0909041426s594d930bg7f7410a99e755fcb@mail.gmail.com>

>On Fri, Sep 4, 2009 at 1:26 PM, jwcolby <jwcolby at colbyconsulting.com>wrote:
>I thought that SQL Server would magically prevent anyone from bringing it
to it's knees.  Naive I know.

Any badly written query can bring down even the most robust Sql Server to
it's knees.  Someday on your development server create a series of GOTO
statements with cursors... you can program it so that at nth points the
cursor doesn't close correctly, this will cause a run-away cursor, if you
write your procedure to insert data into a table before the commit, you can
lock up the table tight to create a deadlock.  under normal conditions the
system will choose a victor and you will be on your way, but you can spin
this loop many times so that instead of the server simply timing something
out, there is too much memory in use and nothing ever comes back.

so I learned the hard way when I had a DBA whom no longer works here craft
himself a nifty little procedure he found on the web,  only he forgot to
close out his cursors correctly and set this up as a job and had it running
every 15 minutes.... by the afternoon the system was down to a crawl even
though it had plenty of I/O throughput and plenty of CPU and RAM.  we
identified the culprit, but it was difficult as Enterprise Manager would
continue to time out due to the amount of resources that the Agent kept
spinning off.  In the end it was easier to shutdown the server and identify
the culprit on reboot than to try to connect to the server as everything had
become unresponsive.

in your case with ton's of data you need I/O, as you have plenty of memory
and from the examples you've written about you tend to stick to either
actual tables.  Speeding up your transaction log disk and your tempdb disk
will produce the biggest performance gain possible.

(now I mention the transaction log disk, even when you have your recovery
mode to simple, the transaction can get some data written to it temporarily,
but it tends to write back to the main db files as soon as it hits a
checkpoint.  so speeding this disk 2nd to your tempdb will yield noticeable
improvements)

>
>
> -Francisco
> http://sqlthis.blogspot.com | Tsql and More...
>


From jwcolby at colbyconsulting.com  Fri Sep  4 16:36:38 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Fri, 04 Sep 2009 17:36:38 -0400
Subject: [dba-SQLServer] SQL Server 2005 unresponsive
In-Reply-To: <1BF060B6F676489398384FADE8B5CF14@advancedinput.com>
References: <4AA13884.7000707@colbyconsulting.com>	<8C76543CDDFD4F6FBB8A8C9527EBBA93@creativesystemdesigns.com>	<4AA14A70.5020501@colbyconsulting.com><b874372a0909041028t6079917fr5ae75c8939b5555f@mail.gmail.com>	<4AA177DE.8080600@colbyconsulting.com>
	<1BF060B6F676489398384FADE8B5CF14@advancedinput.com>
Message-ID: <4AA18866.4020201@colbyconsulting.com>

 >whenever a vendor comes forward with a solution that involves ODBC connectivity to SQL server I 
immediately raise red flags.

ROTFL.

Some day I will port it all to .Net but it won't be today, or even this month.  I am going to try 
the new array with separate disks.  I'll report back and let you know what that does for me.

John W. Colby
www.ColbyConsulting.com


Eric Barro wrote:
> John,
> 
> Access has to connect to SQL server and SQL server treats the Access
> connection as a Linked Server using ODBC. As already mentioned and it is a
> well-known fact, ODBC is such a lumbering beast when it comes to large
> datasets.
> 
> Trust me I've seen this happen many times over and whenever a vendor comes
> forward with a solution that involves ODBC connectivity to SQL server I
> immediately raise red flags.
> 
> Eric 
> 
> -----Original Message-----
> From: dba-sqlserver-bounces at databaseadvisors.com
> [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby
> Sent: Friday, September 04, 2009 1:26 PM
> To: Discussion concerning MS SQL Server
> Subject: Re: [dba-SQLServer] SQL Server 2005 unresponsive
> 
> Francisco,
> 
> Thanks for the suggestions.
> 
>  > It's not Sql Server ALLOWING the ODBC call to send it to turtle mode,
> it's ODBC, that library is inherently slow, and has problems with larger
> datasets.
> 
> Yes but - how can it cause SQL server to lock it up?  All that is going on
> is that Access is running Stored procedures.  Now these stored procedures do
> big operations, but still...
> 
> I thought that SQL Server would magically prevent anyone from bringing it to
> it's knees.  Naive I know.
> 
> You may very well be right of course.  I have so much data to store that I
> have been concentrating on size instead of "spindles".  I have some spare
> drives I can use.  I will throw those on there Raid 0.  I doubt seriously
> that the CURRENT bottle neck is the controller, though it may very well
> change to that if I do a what we are discussing.
> 
> Actually I do have a spare card hanging around.  In fact i put it in my
> other server (just to see if it could be used simultaneously) and when I
> took it back out the raid software puts up a "raid missing" message at boot
> up.  I cannot get rid of that message.
> 
> Anyway... I could put that in and throw some extra drives on that
> controller.  My bigger problem really is a another power supply.  That
> server currently has 11 hard drives in it.
> 
> I will definitely do that though just to see what that gets me.  I have to
> do something!.
> 
> John W. Colby
> www.ColbyConsulting.com
> 
> 
> Francisco Tapia wrote:
>> It's not Sql Server ALLOWING the ODBC call to send it to turtle mode, 
>> it's ODBC, that library is inherently slow, and has problems with 
>> larger datasets.
>>
>> If you are seeing slowness on your SQL Server you'll need to fine tune 
>> a few things
>>
>>
>> First check the size of of your transaction logs, whenever they exceed 
>> 50% of the data file size, then you can begin running into some 
>> performance problems.  (normally i see this if my transaction log file 
>> is 2x or more than my data file.)
>>
>> also check the tempdb
>>
>> When you are MOVING a lot of data around, the way you are, you want to 
>> have each data partition on it's own set of spindle disks, this 
>> applies largely to your transaction logs, because you don't want to 
>> loose the data but you want the system to process extremely quick, you 
>> should look into raid 10 if you can afford it. For TempDB it's 
>> possible to squeeze a lot of performance on the disks, first tempdb is 
>> always rebuilt when the server comes back up, and all data is just 
>> temporary, so you could go with a pair of raid 0 drives, possibly 3 
>> spindles if you need the extra speed.  Striping is the fastest 
>> possible solution to speeding up IO,  ideally of course you'll want to
> have this on it's own separate i/o card and own channel if possible.
>> benchmark your system by running it through the paces of the SQLIO 
>> benchmarking tool from MS 
>> http://www.microsoft.com/DOWNLOADS/details.aspx?familyid=9A8B005B-84E4
>> -4F24-8D65-CB53442D9E19&displaylang=en
>> and benchmark as you move towards faster drives to see the new throughput.
>> In your case you might be easily impressed by moving the tempdb to a 
>> set of raid 0 drives.
>>
>>
>> -Francisco
>> http://sqlthis.blogspot.com | Tsql and More...
>>
>>
>> On Fri, Sep 4, 2009 at 10:12 AM, jwcolby
> <jwcolby at colbyconsulting.com>wrote:
>>> Jim,
>>>
>>> Thanks for the response.  I have to agree, it does seem to be 
>>> something ODBC / Access related in that pretty much everything I do 
>>> directly inside of SQL Server works just fine.  It is annoying that 
>>> SQL Server would ALLOW an ODBC call from Access to cause it to go 
>>> into turtle mode though.
>>>
>>>  > Have you tried .Net?
>>>
>>> Oddly enough I have, but it was some time ago.  I was doing this 
>>> stuff in VB.Net / ADO, but I was so slow at the .Net development that 
>>> when I needed to "get something done" I just went back to Access.
>>>  I cannot make any valid comparisons however as back when I was doing 
>>> the .Net work I also was a complete nube at the SQL Server stuff as 
>>> well, couldn't even write a SP.
>>>
>>> I am taking a C# class now and will be moving squarely into C# in the 
>>> future.  I have pretty much made a decision to make that my future.  
>>> But in the here and now...
>>>
>>> John W. Colby
>>> www.ColbyConsulting.com
>>>
>>>
>>> Jim Lawrence wrote:
>>>> I think the issue is Access related and not MS SQL. Had exactly the 
>>>> same issues when connected to an Oracle DB. I ended up writing code 
>>>> that would save my results to a temporary table, in the Oracle DB 
>>>> and then would
>>> have
>>>> Access just retrieve the table results.
>>>>
>>>> The single process step of gathering the data and downloading a
>>> recordset,
>>>> via ADO would sometimes cause a hang up. If I used ODBC, it would 
>>>> always cause a hold up especially on larger data sets.
>>>>
>>>> Have you tried .Net?
>>>>
>>>> Jim
>>>
>> _______________________________________________
>> dba-SQLServer mailing list
>> dba-SQLServer at databaseadvisors.com
>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>> http://www.databaseadvisors.com
>>
>>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From jwcolby at colbyconsulting.com  Fri Sep  4 17:11:28 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Fri, 04 Sep 2009 18:11:28 -0400
Subject: [dba-SQLServer] SQL Server 2005 unresponsive
In-Reply-To: <b874372a0909041028t6079917fr5ae75c8939b5555f@mail.gmail.com>
References: <4AA13884.7000707@colbyconsulting.com>	<8C76543CDDFD4F6FBB8A8C9527EBBA93@creativesystemdesigns.com>	<4AA14A70.5020501@colbyconsulting.com>
	<b874372a0909041028t6079917fr5ae75c8939b5555f@mail.gmail.com>
Message-ID: <4AA19090.9060400@colbyconsulting.com>

OK so it appears that I have a raid1 500g drive that I set up previously and I just created a new 
600 gb Raid 0.

So now, how do I move things around?  I do not see a gui way to determine where the temp db goes, 
nor for that matter where the transaction logs go.  I assume that transaction logs go right in the 
database unless you do something specific to split them out?

John W. Colby
www.ColbyConsulting.com


Francisco Tapia wrote:
> It's not Sql Server ALLOWING the ODBC call to send it to turtle mode, it's
> ODBC, that library is inherently slow, and has problems with larger
> datasets.
> 
> If you are seeing slowness on your SQL Server you'll need to fine tune a few
> things
> 
> 
> First check the size of of your transaction logs, whenever they exceed 50%
> of the data file size, then you can begin running into some performance
> problems.  (normally i see this if my transaction log file is 2x or more
> than my data file.)
> 
> also check the tempdb
> 
> When you are MOVING a lot of data around, the way you are, you want to have
> each data partition on it's own set of spindle disks, this applies largely
> to your transaction logs, because you don't want to loose the data but you
> want the system to process extremely quick, you should look into raid 10 if
> you can afford it. For TempDB it's possible to squeeze a lot of performance
> on the disks, first tempdb is always rebuilt when the server comes back up,
> and all data is just temporary, so you could go with a pair of raid 0
> drives, possibly 3 spindles if you need the extra speed.  Striping is the
> fastest possible solution to speeding up IO,  ideally of course you'll want
> to have this on it's own separate i/o card and own channel if possible.
> 
> benchmark your system by running it through the paces of the SQLIO
> benchmarking tool from MS
> http://www.microsoft.com/DOWNLOADS/details.aspx?familyid=9A8B005B-84E4-4F24-8D65-CB53442D9E19&displaylang=en
> and benchmark as you move towards faster drives to see the new throughput.
> In your case you might be easily impressed by moving the tempdb to a set of
> raid 0 drives.
> 
> 
> -Francisco
> http://sqlthis.blogspot.com | Tsql and More...
> 
> 
> On Fri, Sep 4, 2009 at 10:12 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:
> 
>> Jim,
>>
>> Thanks for the response.  I have to agree, it does seem to be something
>> ODBC / Access related in
>> that pretty much everything I do directly inside of SQL Server works just
>> fine.  It is annoying that
>> SQL Server would ALLOW an ODBC call from Access to cause it to go into
>> turtle mode though.
>>
>>  > Have you tried .Net?
>>
>> Oddly enough I have, but it was some time ago.  I was doing this stuff in
>> VB.Net / ADO, but I was so
>> slow at the .Net development that when I needed to "get something done" I
>> just went back to Access.
>>  I cannot make any valid comparisons however as back when I was doing the
>> .Net work I also was a
>> complete nube at the SQL Server stuff as well, couldn't even write a SP.
>>
>> I am taking a C# class now and will be moving squarely into C# in the
>> future.  I have pretty much
>> made a decision to make that my future.  But in the here and now...
>>
>> John W. Colby
>> www.ColbyConsulting.com
>>
>>
>> Jim Lawrence wrote:
>>> I think the issue is Access related and not MS SQL. Had exactly the same
>>> issues when connected to an Oracle DB. I ended up writing code that would
>>> save my results to a temporary table, in the Oracle DB and then would
>> have
>>> Access just retrieve the table results.
>>>
>>> The single process step of gathering the data and downloading a
>> recordset,
>>> via ADO would sometimes cause a hang up. If I used ODBC, it would always
>>> cause a hold up especially on larger data sets.
>>>
>>> Have you tried .Net?
>>>
>>> Jim
>>
>>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From stuart at lexacorp.com.pg  Fri Sep  4 17:37:59 2009
From: stuart at lexacorp.com.pg (Stuart McLachlan)
Date: Sat, 05 Sep 2009 08:37:59 +1000
Subject: [dba-SQLServer] SQL Server 2005 unresponsive
In-Reply-To: <4AA19090.9060400@colbyconsulting.com>
References: <4AA13884.7000707@colbyconsulting.com>,
	<b874372a0909041028t6079917fr5ae75c8939b5555f@mail.gmail.com>,
	<4AA19090.9060400@colbyconsulting.com>
Message-ID: <4AA196C7.2410.96815D2@stuart.lexacorp.com.pg>

Open Management Studio

To change the defaults:
Right click on the Server and select Properties.
Go to the "Database Settings" tab and you can define the locations for the data and log files.

To change the location of data and log files for any normal database. 
Detach the database
Move the data and log files to anywhere you want then
Attach the database and point to the new locations for the files.

To change the location of tempdb:
http://www.tech-recipes.com/rx/2342/sql_server_2005_move_tempdb/

-- 
Stuart
On 4 Sep 2009 at 18:11, jwcolby wrote:

> OK so it appears that I have a raid1 500g drive that I set up previously and I just created a new 
> 600 gb Raid 0.
> 
> So now, how do I move things around?  I do not see a gui way to determine where the temp db goes, 
> nor for that matter where the transaction logs go.  I assume that transaction logs go right in the 
> database unless you do something specific to split them out?
> 
> John W. Colby
> www.ColbyConsulting.com
> 
> 
> Francisco Tapia wrote:
> > It's not Sql Server ALLOWING the ODBC call to send it to turtle mode, it's
> > ODBC, that library is inherently slow, and has problems with larger
> > datasets.
> > 
> > If you are seeing slowness on your SQL Server you'll need to fine tune a few
> > things
> > 
> > 
> > First check the size of of your transaction logs, whenever they exceed 50%
> > of the data file size, then you can begin running into some performance
> > problems.  (normally i see this if my transaction log file is 2x or more
> > than my data file.)
> > 
> > also check the tempdb
> > 
> > When you are MOVING a lot of data around, the way you are, you want to have
> > each data partition on it's own set of spindle disks, this applies largely
> > to your transaction logs, because you don't want to loose the data but you
> > want the system to process extremely quick, you should look into raid 10 if
> > you can afford it. For TempDB it's possible to squeeze a lot of performance
> > on the disks, first tempdb is always rebuilt when the server comes back up,
> > and all data is just temporary, so you could go with a pair of raid 0
> > drives, possibly 3 spindles if you need the extra speed.  Striping is the
> > fastest possible solution to speeding up IO,  ideally of course you'll want
> > to have this on it's own separate i/o card and own channel if possible.
> > 
> > benchmark your system by running it through the paces of the SQLIO
> > benchmarking tool from MS
> > http://www.microsoft.com/DOWNLOADS/details.aspx?familyid=9A8B005B-84E4-4F24-8D65-CB53442D9E19&displaylang=en
> > and benchmark as you move towards faster drives to see the new throughput.
> > In your case you might be easily impressed by moving the tempdb to a set of
> > raid 0 drives.
> > 
> > 
> > -Francisco
> > http://sqlthis.blogspot.com | Tsql and More...
> > 
> > 
> > On Fri, Sep 4, 2009 at 10:12 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:
> > 
> >> Jim,
> >>
> >> Thanks for the response.  I have to agree, it does seem to be something
> >> ODBC / Access related in
> >> that pretty much everything I do directly inside of SQL Server works just
> >> fine.  It is annoying that
> >> SQL Server would ALLOW an ODBC call from Access to cause it to go into
> >> turtle mode though.
> >>
> >>  > Have you tried .Net?
> >>
> >> Oddly enough I have, but it was some time ago.  I was doing this stuff in
> >> VB.Net / ADO, but I was so
> >> slow at the .Net development that when I needed to "get something done" I
> >> just went back to Access.
> >>  I cannot make any valid comparisons however as back when I was doing the
> >> .Net work I also was a
> >> complete nube at the SQL Server stuff as well, couldn't even write a SP.
> >>
> >> I am taking a C# class now and will be moving squarely into C# in the
> >> future.  I have pretty much
> >> made a decision to make that my future.  But in the here and now...
> >>
> >> John W. Colby
> >> www.ColbyConsulting.com
> >>
> >>
> >> Jim Lawrence wrote:
> >>> I think the issue is Access related and not MS SQL. Had exactly the same
> >>> issues when connected to an Oracle DB. I ended up writing code that would
> >>> save my results to a temporary table, in the Oracle DB and then would
> >> have
> >>> Access just retrieve the table results.
> >>>
> >>> The single process step of gathering the data and downloading a
> >> recordset,
> >>> via ADO would sometimes cause a hang up. If I used ODBC, it would always
> >>> cause a hold up especially on larger data sets.
> >>>
> >>> Have you tried .Net?
> >>>
> >>> Jim
> >>
> >>
> > _______________________________________________
> > dba-SQLServer mailing list
> > dba-SQLServer at databaseadvisors.com
> > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> > http://www.databaseadvisors.com
> > 
> > 
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 




From ab-mi at post3.tele.dk  Fri Sep  4 18:38:42 2009
From: ab-mi at post3.tele.dk (Asger Blond)
Date: Sat, 5 Sep 2009 01:38:42 +0200
Subject: [dba-SQLServer] SQL Server 2005 unresponsive
In-Reply-To: <4AA196C7.2410.96815D2@stuart.lexacorp.com.pg>
References: <4AA13884.7000707@colbyconsulting.com>,
	<b874372a0909041028t6079917fr5ae75c8939b5555f@mail.gmail.com>,
	<4AA19090.9060400@colbyconsulting.com>
	<4AA196C7.2410.96815D2@stuart.lexacorp.com.pg>
Message-ID: <49C15838BFF3405984A0BB462B1C15EB@abpc>

Moving the tempdb is a special case. As the tempdb can't be detached you
have to use a sql like this:

ALTER DATABASE tempdb MODIFY FILE 
(name='tempdev', filename='X:\TempDBData\tempdb.mdf')
ALTER DATABASE tempdb MODIFY FILE 
(name='templog', filename='Y:\TempDBLog\templog.ldf')

Then restart sql-server to make the new placement effective. Eventually
delete the old tempdb-files.

Asger

-----Oprindelig meddelelse-----
Fra: dba-sqlserver-bounces at databaseadvisors.com
[mailto:dba-sqlserver-bounces at databaseadvisors.com] P? vegne af Stuart
McLachlan
Sendt: 5. september 2009 00:38
Til: Discussion concerning MS SQL Server
Emne: Re: [dba-SQLServer] SQL Server 2005 unresponsive

Open Management Studio

To change the defaults:
Right click on the Server and select Properties.
Go to the "Database Settings" tab and you can define the locations for the
data and log files.

To change the location of data and log files for any normal database. 
Detach the database
Move the data and log files to anywhere you want then
Attach the database and point to the new locations for the files.

To change the location of tempdb:
http://www.tech-recipes.com/rx/2342/sql_server_2005_move_tempdb/

-- 
Stuart
On 4 Sep 2009 at 18:11, jwcolby wrote:

> OK so it appears that I have a raid1 500g drive that I set up previously
and I just created a new 
> 600 gb Raid 0.
> 
> So now, how do I move things around?  I do not see a gui way to determine
where the temp db goes, 
> nor for that matter where the transaction logs go.  I assume that
transaction logs go right in the 
> database unless you do something specific to split them out?
> 
> John W. Colby
> www.ColbyConsulting.com
> 
> 
> Francisco Tapia wrote:
> > It's not Sql Server ALLOWING the ODBC call to send it to turtle mode,
it's
> > ODBC, that library is inherently slow, and has problems with larger
> > datasets.
> > 
> > If you are seeing slowness on your SQL Server you'll need to fine tune a
few
> > things
> > 
> > 
> > First check the size of of your transaction logs, whenever they exceed
50%
> > of the data file size, then you can begin running into some performance
> > problems.  (normally i see this if my transaction log file is 2x or more
> > than my data file.)
> > 
> > also check the tempdb
> > 
> > When you are MOVING a lot of data around, the way you are, you want to
have
> > each data partition on it's own set of spindle disks, this applies
largely
> > to your transaction logs, because you don't want to loose the data but
you
> > want the system to process extremely quick, you should look into raid 10
if
> > you can afford it. For TempDB it's possible to squeeze a lot of
performance
> > on the disks, first tempdb is always rebuilt when the server comes back
up,
> > and all data is just temporary, so you could go with a pair of raid 0
> > drives, possibly 3 spindles if you need the extra speed.  Striping is
the
> > fastest possible solution to speeding up IO,  ideally of course you'll
want
> > to have this on it's own separate i/o card and own channel if possible.
> > 
> > benchmark your system by running it through the paces of the SQLIO
> > benchmarking tool from MS
> >
http://www.microsoft.com/DOWNLOADS/details.aspx?familyid=9A8B005B-84E4-4F24-
8D65-CB53442D9E19&displaylang=en
> > and benchmark as you move towards faster drives to see the new
throughput.
> > In your case you might be easily impressed by moving the tempdb to a set
of
> > raid 0 drives.
> > 
> > 
> > -Francisco
> > http://sqlthis.blogspot.com | Tsql and More...
> > 
> > 
> > On Fri, Sep 4, 2009 at 10:12 AM, jwcolby
<jwcolby at colbyconsulting.com>wrote:
> > 
> >> Jim,
> >>
> >> Thanks for the response.  I have to agree, it does seem to be something
> >> ODBC / Access related in
> >> that pretty much everything I do directly inside of SQL Server works
just
> >> fine.  It is annoying that
> >> SQL Server would ALLOW an ODBC call from Access to cause it to go into
> >> turtle mode though.
> >>
> >>  > Have you tried .Net?
> >>
> >> Oddly enough I have, but it was some time ago.  I was doing this stuff
in
> >> VB.Net / ADO, but I was so
> >> slow at the .Net development that when I needed to "get something done"
I
> >> just went back to Access.
> >>  I cannot make any valid comparisons however as back when I was doing
the
> >> .Net work I also was a
> >> complete nube at the SQL Server stuff as well, couldn't even write a
SP.
> >>
> >> I am taking a C# class now and will be moving squarely into C# in the
> >> future.  I have pretty much
> >> made a decision to make that my future.  But in the here and now...
> >>
> >> John W. Colby
> >> www.ColbyConsulting.com
> >>
> >>
> >> Jim Lawrence wrote:
> >>> I think the issue is Access related and not MS SQL. Had exactly the
same
> >>> issues when connected to an Oracle DB. I ended up writing code that
would
> >>> save my results to a temporary table, in the Oracle DB and then would
> >> have
> >>> Access just retrieve the table results.
> >>>
> >>> The single process step of gathering the data and downloading a
> >> recordset,
> >>> via ADO would sometimes cause a hang up. If I used ODBC, it would
always
> >>> cause a hold up especially on larger data sets.
> >>>
> >>> Have you tried .Net?
> >>>
> >>> Jim
> >>
> >>
> > _______________________________________________
> > dba-SQLServer mailing list
> > dba-SQLServer at databaseadvisors.com
> > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> > http://www.databaseadvisors.com
> > 
> > 
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 


_______________________________________________
dba-SQLServer mailing list
dba-SQLServer at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
http://www.databaseadvisors.com




From stuart at lexacorp.com.pg  Fri Sep  4 18:53:01 2009
From: stuart at lexacorp.com.pg (Stuart McLachlan)
Date: Sat, 05 Sep 2009 09:53:01 +1000
Subject: [dba-SQLServer] SQL Server 2005 unresponsive
In-Reply-To: <49C15838BFF3405984A0BB462B1C15EB@abpc>
References: <4AA13884.7000707@colbyconsulting.com>,
	<4AA196C7.2410.96815D2@stuart.lexacorp.com.pg>,
	<49C15838BFF3405984A0BB462B1C15EB@abpc>
Message-ID: <4AA1A85D.24665.9ACC86F@stuart.lexacorp.com.pg>

That's essentially what it said on the page I linked to.

-- 
Stuart

On 5 Sep 2009 at 1:38, Asger Blond wrote:

> Moving the tempdb is a special case. As the tempdb can't be detached you
> have to use a sql like this:
> 
> ALTER DATABASE tempdb MODIFY FILE 
> (name='tempdev', filename='X:\TempDBData\tempdb.mdf')
> ALTER DATABASE tempdb MODIFY FILE 
> (name='templog', filename='Y:\TempDBLog\templog.ldf')
> 
> Then restart sql-server to make the new placement effective. Eventually
> delete the old tempdb-files.
> 
> Asger
> 
> -----Oprindelig meddelelse-----
> Fra: dba-sqlserver-bounces at databaseadvisors.com
> [mailto:dba-sqlserver-bounces at databaseadvisors.com] P? vegne af Stuart
> McLachlan
> Sendt: 5. september 2009 00:38
> Til: Discussion concerning MS SQL Server
> Emne: Re: [dba-SQLServer] SQL Server 2005 unresponsive
> 
> Open Management Studio
> 
> To change the defaults:
> Right click on the Server and select Properties.
> Go to the "Database Settings" tab and you can define the locations for the
> data and log files.
> 
> To change the location of data and log files for any normal database. 
> Detach the database
> Move the data and log files to anywhere you want then
> Attach the database and point to the new locations for the files.
> 
> To change the location of tempdb:
> http://www.tech-recipes.com/rx/2342/sql_server_2005_move_tempdb/
> 
> -- 
> Stuart
> On 4 Sep 2009 at 18:11, jwcolby wrote:
> 
> > OK so it appears that I have a raid1 500g drive that I set up previously
> and I just created a new 
> > 600 gb Raid 0.
> > 
> > So now, how do I move things around?  I do not see a gui way to determine
> where the temp db goes, 
> > nor for that matter where the transaction logs go.  I assume that
> transaction logs go right in the 
> > database unless you do something specific to split them out?
> > 
> > John W. Colby
> > www.ColbyConsulting.com
> > 
> > 
> > Francisco Tapia wrote:
> > > It's not Sql Server ALLOWING the ODBC call to send it to turtle mode,
> it's
> > > ODBC, that library is inherently slow, and has problems with larger
> > > datasets.
> > > 
> > > If you are seeing slowness on your SQL Server you'll need to fine tune a
> few
> > > things
> > > 
> > > 
> > > First check the size of of your transaction logs, whenever they exceed
> 50%
> > > of the data file size, then you can begin running into some performance
> > > problems.  (normally i see this if my transaction log file is 2x or more
> > > than my data file.)
> > > 
> > > also check the tempdb
> > > 
> > > When you are MOVING a lot of data around, the way you are, you want to
> have
> > > each data partition on it's own set of spindle disks, this applies
> largely
> > > to your transaction logs, because you don't want to loose the data but
> you
> > > want the system to process extremely quick, you should look into raid 10
> if
> > > you can afford it. For TempDB it's possible to squeeze a lot of
> performance
> > > on the disks, first tempdb is always rebuilt when the server comes back
> up,
> > > and all data is just temporary, so you could go with a pair of raid 0
> > > drives, possibly 3 spindles if you need the extra speed.  Striping is
> the
> > > fastest possible solution to speeding up IO,  ideally of course you'll
> want
> > > to have this on it's own separate i/o card and own channel if possible.
> > > 
> > > benchmark your system by running it through the paces of the SQLIO
> > > benchmarking tool from MS
> > >
> http://www.microsoft.com/DOWNLOADS/details.aspx?familyid=9A8B005B-84E4-4F24-
> 8D65-CB53442D9E19&displaylang=en
> > > and benchmark as you move towards faster drives to see the new
> throughput.
> > > In your case you might be easily impressed by moving the tempdb to a set
> of
> > > raid 0 drives.
> > > 
> > > 
> > > -Francisco
> > > http://sqlthis.blogspot.com | Tsql and More...
> > > 
> > > 
> > > On Fri, Sep 4, 2009 at 10:12 AM, jwcolby
> <jwcolby at colbyconsulting.com>wrote:
> > > 
> > >> Jim,
> > >>
> > >> Thanks for the response.  I have to agree, it does seem to be something
> > >> ODBC / Access related in
> > >> that pretty much everything I do directly inside of SQL Server works
> just
> > >> fine.  It is annoying that
> > >> SQL Server would ALLOW an ODBC call from Access to cause it to go into
> > >> turtle mode though.
> > >>
> > >>  > Have you tried .Net?
> > >>
> > >> Oddly enough I have, but it was some time ago.  I was doing this stuff
> in
> > >> VB.Net / ADO, but I was so
> > >> slow at the .Net development that when I needed to "get something done"
> I
> > >> just went back to Access.
> > >>  I cannot make any valid comparisons however as back when I was doing
> the
> > >> .Net work I also was a
> > >> complete nube at the SQL Server stuff as well, couldn't even write a
> SP.
> > >>
> > >> I am taking a C# class now and will be moving squarely into C# in the
> > >> future.  I have pretty much
> > >> made a decision to make that my future.  But in the here and now...
> > >>
> > >> John W. Colby
> > >> www.ColbyConsulting.com
> > >>
> > >>
> > >> Jim Lawrence wrote:
> > >>> I think the issue is Access related and not MS SQL. Had exactly the
> same
> > >>> issues when connected to an Oracle DB. I ended up writing code that
> would
> > >>> save my results to a temporary table, in the Oracle DB and then would
> > >> have
> > >>> Access just retrieve the table results.
> > >>>
> > >>> The single process step of gathering the data and downloading a
> > >> recordset,
> > >>> via ADO would sometimes cause a hang up. If I used ODBC, it would
> always
> > >>> cause a hold up especially on larger data sets.
> > >>>
> > >>> Have you tried .Net?
> > >>>
> > >>> Jim
> > >>
> > >>
> > > _______________________________________________
> > > dba-SQLServer mailing list
> > > dba-SQLServer at databaseadvisors.com
> > > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> > > http://www.databaseadvisors.com
> > > 
> > > 
> > _______________________________________________
> > dba-SQLServer mailing list
> > dba-SQLServer at databaseadvisors.com
> > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> > http://www.databaseadvisors.com
> > 
> 
> 
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 





From jwcolby at colbyconsulting.com  Fri Sep  4 19:25:50 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Fri, 04 Sep 2009 20:25:50 -0400
Subject: [dba-SQLServer] SQL Server 2005 unresponsive
In-Reply-To: <4AA196C7.2410.96815D2@stuart.lexacorp.com.pg>
References: <4AA13884.7000707@colbyconsulting.com>,
	<b874372a0909041028t6079917fr5ae75c8939b5555f@mail.gmail.com>,
	<4AA19090.9060400@colbyconsulting.com>
	<4AA196C7.2410.96815D2@stuart.lexacorp.com.pg>
Message-ID: <4AA1B00E.9050801@colbyconsulting.com>

I found a way to move the temp db files, which I moved to a raid 0 drive by itself.

The log files are another matter.  Obviously each db has one and AFAICT they have to be moved db by 
db.  I will move the log files for the major databases.  It would be nice if there was a way to just 
say "put all log files here".

John W. Colby
www.ColbyConsulting.com


Stuart McLachlan wrote:
> Open Management Studio
> 
> To change the defaults:
> Right click on the Server and select Properties.
> Go to the "Database Settings" tab and you can define the locations for the data and log files.
> 
> To change the location of data and log files for any normal database. 
> Detach the database
> Move the data and log files to anywhere you want then
> Attach the database and point to the new locations for the files.
> 
> To change the location of tempdb:
> http://www.tech-recipes.com/rx/2342/sql_server_2005_move_tempdb/
> 


From fuller.artful at gmail.com  Sat Sep  5 09:31:29 2009
From: fuller.artful at gmail.com (Arthur Fuller)
Date: Sat, 5 Sep 2009 10:31:29 -0400
Subject: [dba-SQLServer] SQL Server 2005 unresponsive
In-Reply-To: <4AA1B00E.9050801@colbyconsulting.com>
References: <4AA13884.7000707@colbyconsulting.com>
	<b874372a0909041028t6079917fr5ae75c8939b5555f@mail.gmail.com>
	<4AA19090.9060400@colbyconsulting.com>
	<4AA196C7.2410.96815D2@stuart.lexacorp.com.pg>
	<4AA1B00E.9050801@colbyconsulting.com>
Message-ID: <29f585dd0909050731l2d1822c7kee2c146e741c267f@mail.gmail.com>

My first inclination would be to use BULK INSERT for two million rows. It's
insanely fast and I don't see any reason why you couldn't run the command
from the Access app. Have you considered that approach?
Arthur


From jwcolby at colbyconsulting.com  Sat Sep  5 11:51:09 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Sat, 05 Sep 2009 12:51:09 -0400
Subject: [dba-SQLServer] SQL Server 2005 unresponsive
In-Reply-To: <29f585dd0909050731l2d1822c7kee2c146e741c267f@mail.gmail.com>
References: <4AA13884.7000707@colbyconsulting.com>	<b874372a0909041028t6079917fr5ae75c8939b5555f@mail.gmail.com>	<4AA19090.9060400@colbyconsulting.com>	<4AA196C7.2410.96815D2@stuart.lexacorp.com.pg>	<4AA1B00E.9050801@colbyconsulting.com>
	<29f585dd0909050731l2d1822c7kee2c146e741c267f@mail.gmail.com>
Message-ID: <4AA296FD.9030400@colbyconsulting.com>

Arthur,

This is a set of around a dozen or so stored procedures.  The stored procedures are what are run, 
from Access.  One specific stored procedure, which runs just fine from SQl Server, just goes away 
from Access.  Oddly it has worked in the past.  Now it literally hangs SQL Server and in fact will 
cause Windows to hang as well.  I started it running from Access last night, this morning still not 
done, and the server itself (Windows) was completely unresponsive.  And yet when I run all the stuff 
(In Access) but stop the process just before this one specific stored procedure, I can then run that 
SP from inside of SQL Server and it only takes a half hour or so.

The things I put up with!

John W. Colby
www.ColbyConsulting.com


Arthur Fuller wrote:
> My first inclination would be to use BULK INSERT for two million rows. It's
> insanely fast and I don't see any reason why you couldn't run the command
> from the Access app. Have you considered that approach?
> Arthur
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From accessd at shaw.ca  Sun Sep  6 14:19:19 2009
From: accessd at shaw.ca (Jim Lawrence)
Date: Sun, 6 Sep 2009 12:19:19 -0700
Subject: [dba-SQLServer] SQL Server 2005 unresponsive
In-Reply-To: <4AA14A70.5020501@colbyconsulting.com>
References: <4AA13884.7000707@colbyconsulting.com>
	<8C76543CDDFD4F6FBB8A8C9527EBBA93@creativesystemdesigns.com>
	<4AA14A70.5020501@colbyconsulting.com>
Message-ID: <55E9DEEFD45949D59A8EE571ED42EBAE@creativesystemdesigns.com>

Hi John:

I have no idea why these problems are happening when connecting to large
data set but I believe that it is timing issues. I think all SQL DBs, when
transferring data just opens a door and pour.

Using ADO-OLE just removes a few layers of program interface therefore
allowing better and faster flow control but at one point it seems beyond MS
Access's ability to handle. I abandoned ODBC use because of that as there is
just not enough control or features. (I still have a raggedy old book on ADO
2.5 programming (my bible) and would recommend something similar.)

Another thing I have noticed is that complexity of the SQL script
contributed to this hanging especially if the sequel script string was being
passed to server... Sending just parameters to a SP seemed to solve a lot of
problems.

Another thing that will guarantee a crash is attempting to sychronize the
data flow or binding the data. MS Access can only handle limited datasets in
bound mode. 

Another possibility is to limit the amount of data being sent at any one
time. I did not work on any database the size that you have but maybe if you
pass the result data to temp table, not a view and then retrieve the data in
chunks. Ie. "select * from MyTempTable where recnum > 500000 and renumber <
10000000" and then again "select * from MyTempTable where recnum >= 1000000
and renum < 1500000" and you handle the appending at the Access end. That
might/should work.

I am sure what work arounds will work but the best solutions are probably at
the server end.   

Jim


-----Original Message-----
From: dba-sqlserver-bounces at databaseadvisors.com
[mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby
Sent: Friday, September 04, 2009 10:12 AM
To: Discussion concerning MS SQL Server
Subject: Re: [dba-SQLServer] SQL Server 2005 unresponsive

Jim,

Thanks for the response.  I have to agree, it does seem to be something ODBC
/ Access related in 
that pretty much everything I do directly inside of SQL Server works just
fine.  It is annoying that 
SQL Server would ALLOW an ODBC call from Access to cause it to go into
turtle mode though.

 > Have you tried .Net?

Oddly enough I have, but it was some time ago.  I was doing this stuff in
VB.Net / ADO, but I was so 
slow at the .Net development that when I needed to "get something done" I
just went back to Access. 
  I cannot make any valid comparisons however as back when I was doing the
.Net work I also was a 
complete nube at the SQL Server stuff as well, couldn't even write a SP.

I am taking a C# class now and will be moving squarely into C# in the
future.  I have pretty much 
made a decision to make that my future.  But in the here and now...

John W. Colby
www.ColbyConsulting.com


Jim Lawrence wrote:
> I think the issue is Access related and not MS SQL. Had exactly the same
> issues when connected to an Oracle DB. I ended up writing code that would
> save my results to a temporary table, in the Oracle DB and then would have
> Access just retrieve the table results. 
> 
> The single process step of gathering the data and downloading a recordset,
> via ADO would sometimes cause a hang up. If I used ODBC, it would always
> cause a hold up especially on larger data sets.
> 
> Have you tried .Net?
> 
> Jim

_______________________________________________
dba-SQLServer mailing list
dba-SQLServer at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
http://www.databaseadvisors.com



From syafaruddin at trakindo.co.id  Mon Sep  7 12:01:01 2009
From: syafaruddin at trakindo.co.id (syafaruddin at trakindo.co.id)
Date: Tue, 8 Sep 2009 01:01:01 +0800
Subject: [dba-SQLServer] Andi Syafaruddin/MIN/PTTU is out of the office.
Message-ID: <OF0DC033C8.27CCD241-ON4825762A.005D7A4B@trakindo.co.id>

I will be out of the office starting  09/07/2009 and will not return until
10/01/2009.

Should you have any question and/or problem related to IT, please contact
Helpdesk HO.

Any other IT problems, please contact Helpdesk for further assistance.

If  there any urgent matter that i have to involve with, i am able to
reached at my mobile phone: 0811 424 072.


=========================================================================== 

This email is confidential.  If  you are not the  addressee tell the sender

immediately  and destroy this  email without using,  sending or storing it.

Emails are not secure  and may suffer  errors, viruses, delay, interception

and amendment.  The Trakindo Group of Companies do not accept liability for

damage   caused   by   this   email   and   may   monitor   email  traffic.

Unless expressly stated, any opinions are the sender's and are not approved

by  the  Tiara Marga Trakindo  Group of Companies and  this email is not an

offer, solicitation,  recommendation   or   agreement   of    any     kind.

===========================================================================





From fuller.artful at gmail.com  Tue Sep  8 06:59:04 2009
From: fuller.artful at gmail.com (Arthur Fuller)
Date: Tue, 8 Sep 2009 07:59:04 -0400
Subject: [dba-SQLServer] IIS on Windows Server 2008
Message-ID: <29f585dd0909080459p7c1f96aagf918ffe8f0483a4f@mail.gmail.com>

I cannot locate where to turn IIS on in Windows Server 2008. Does anyone
know how to do it? I've looked at Server Manager several times and cannot
see it there.
TIA,
Arthur


From ab-mi at post3.tele.dk  Tue Sep  8 08:13:51 2009
From: ab-mi at post3.tele.dk (Asger Blond)
Date: Tue, 8 Sep 2009 15:13:51 +0200
Subject: [dba-SQLServer] IIS on Windows Server 2008
In-Reply-To: <29f585dd0909080459p7c1f96aagf918ffe8f0483a4f@mail.gmail.com>
References: <29f585dd0909080459p7c1f96aagf918ffe8f0483a4f@mail.gmail.com>
Message-ID: <1778E74DC5C14B48B139C96EA47D5BA7@abpc>

Found this but have not yet tried it:

http://www.lockergnome.com/it/2007/11/05/install-iis-70-on-windows-server-20
08/

Asger

-----Oprindelig meddelelse-----
Fra: dba-sqlserver-bounces at databaseadvisors.com
[mailto:dba-sqlserver-bounces at databaseadvisors.com] P? vegne af Arthur
Fuller
Sendt: 8. september 2009 13:59
Til: Discussion concerning MS SQL Server
Emne: [dba-SQLServer] IIS on Windows Server 2008

I cannot locate where to turn IIS on in Windows Server 2008. Does anyone
know how to do it? I've looked at Server Manager several times and cannot
see it there.
TIA,
Arthur
_______________________________________________
dba-SQLServer mailing list
dba-SQLServer at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
http://www.databaseadvisors.com




From jwcolby at colbyconsulting.com  Thu Sep 10 07:24:49 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Thu, 10 Sep 2009 08:24:49 -0400
Subject: [dba-SQLServer] SQL Server 2005 - Copying a database
Message-ID: <4AA8F011.3080001@colbyconsulting.com>

I need to programmatically copy a database to a new name (make a copy), and specify where the data 
file and log file goes.  Can anyone give me help in figuring that out?  I want to write a stored 
procedure to do this and call that from Access.

-- 
John W. Colby
www.ColbyConsulting.com


From fuller.artful at gmail.com  Thu Sep 10 07:41:54 2009
From: fuller.artful at gmail.com (Arthur Fuller)
Date: Thu, 10 Sep 2009 08:41:54 -0400
Subject: [dba-SQLServer] SQL Server 2005 - Copying a database
In-Reply-To: <4AA8F011.3080001@colbyconsulting.com>
References: <4AA8F011.3080001@colbyconsulting.com>
Message-ID: <29f585dd0909100541y420c76b9j440d7f97b94dbfc6@mail.gmail.com>

I've never had to do this, but I think that this approach would work. The
whole Management Studio UI (and Enterprise Manager before it) are nothing
but front ends that hide T-SQL commands. Whenever I want to know what SSMS
is doing under the covers, I execute the command of interest using the GUI
and then view the query syntax (include query with results).
Arthur

On Thu, Sep 10, 2009 at 8:24 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:

> I need to programmatically copy a database to a new name (make a copy), and
> specify where the data
> file and log file goes.  Can anyone give me help in figuring that out?  I
> want to write a stored
> procedure to do this and call that from Access.
>
>


From jwcolby at colbyconsulting.com  Thu Sep 10 07:50:31 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Thu, 10 Sep 2009 08:50:31 -0400
Subject: [dba-SQLServer] SQL Server 2005 - Copying a database
In-Reply-To: <4AA8F011.3080001@colbyconsulting.com>
References: <4AA8F011.3080001@colbyconsulting.com>
Message-ID: <4AA8F617.3090707@colbyconsulting.com>

While I am at it, I need to shrink the log file after use.  These databases are rarely used after 
they are created but they generate some pretty hefty log files during creation so I need to just 
automatically shrink them back down.

Thanks,

John W. Colby
www.ColbyConsulting.com


jwcolby wrote:
> I need to programmatically copy a database to a new name (make a copy), and specify where the data 
> file and log file goes.  Can anyone give me help in figuring that out?  I want to write a stored 
> procedure to do this and call that from Access.
> 


From jwcolby at colbyconsulting.com  Thu Sep 10 07:53:34 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Thu, 10 Sep 2009 08:53:34 -0400
Subject: [dba-SQLServer] SQL Server 2005 - Copying a database
In-Reply-To: <29f585dd0909100541y420c76b9j440d7f97b94dbfc6@mail.gmail.com>
References: <4AA8F011.3080001@colbyconsulting.com>
	<29f585dd0909100541y420c76b9j440d7f97b94dbfc6@mail.gmail.com>
Message-ID: <4AA8F6CE.9020105@colbyconsulting.com>

Interesting Arthur.

So from the gui I right click on the db of interest, /tasks / copy database.  A wizard opens and I 
do my thing, naming the files, where they should go etc.

Are you saying that SQL Server saves that somewhere as TSQL and I can go look at it?

John W. Colby
www.ColbyConsulting.com


Arthur Fuller wrote:
> I've never had to do this, but I think that this approach would work. The
> whole Management Studio UI (and Enterprise Manager before it) are nothing
> but front ends that hide T-SQL commands. Whenever I want to know what SSMS
> is doing under the covers, I execute the command of interest using the GUI
> and then view the query syntax (include query with results).
> Arthur
> 
> On Thu, Sep 10, 2009 at 8:24 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:
> 
>> I need to programmatically copy a database to a new name (make a copy), and
>> specify where the data
>> file and log file goes.  Can anyone give me help in figuring that out?  I
>> want to write a stored
>> procedure to do this and call that from Access.
>>
>>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From fuller.artful at gmail.com  Thu Sep 10 08:35:52 2009
From: fuller.artful at gmail.com (Arthur Fuller)
Date: Thu, 10 Sep 2009 09:35:52 -0400
Subject: [dba-SQLServer] SQL Server 2005 - Copying a database
In-Reply-To: <4AA8F6CE.9020105@colbyconsulting.com>
References: <4AA8F011.3080001@colbyconsulting.com>
	<29f585dd0909100541y420c76b9j440d7f97b94dbfc6@mail.gmail.com>
	<4AA8F6CE.9020105@colbyconsulting.com>
Message-ID: <29f585dd0909100635t6bcf53baje382fe3ea7bf90b@mail.gmail.com>

To shrink the transaction log from within SSMS, you open a query window and
issue:
dbcc shrinkfile <logfileName> TRUNCATE ONLY.

I've never tried to execute that command from within a stored procedure but
I think that it could work.

As to the earlier question, I tried what I suggested and it didn't work. I'm
still looking into it. I'll get back to you on that in a bit.

Arthur

On Thu, Sep 10, 2009 at 8:53 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:

> Interesting Arthur.
>
> So from the gui I right click on the db of interest, /tasks / copy
> database.  A wizard opens and I
> do my thing, naming the files, where they should go etc.
>
> Are you saying that SQL Server saves that somewhere as TSQL and I can go
> look at it?
>
> John W. Colby
> www.ColbyConsulting.com
>
>
> Arthur Fuller wrote:
> > I've never had to do this, but I think that this approach would work. The
> > whole Management Studio UI (and Enterprise Manager before it) are nothing
> > but front ends that hide T-SQL commands. Whenever I want to know what
> SSMS
> > is doing under the covers, I execute the command of interest using the
> GUI
> > and then view the query syntax (include query with results).
> > Arthur
> >
> > On Thu, Sep 10, 2009 at 8:24 AM, jwcolby <jwcolby at colbyconsulting.com
> >wrote:
> >
> >> I need to programmatically copy a database to a new name (make a copy),
> and
> >> specify where the data
> >> file and log file goes.  Can anyone give me help in figuring that out?
>  I
> >> want to write a stored
> >> procedure to do this and call that from Access.
> >>
> >>
> > _______________________________________________
> > dba-SQLServer mailing list
> > dba-SQLServer at databaseadvisors.com
> > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> > http://www.databaseadvisors.com
> >
> >
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>
>


From fhtapia at gmail.com  Thu Sep 10 10:40:00 2009
From: fhtapia at gmail.com (Francisco Tapia)
Date: Thu, 10 Sep 2009 08:40:00 -0700
Subject: [dba-SQLServer] SQL Server 2005 - Copying a database
In-Reply-To: <4AA8F6CE.9020105@colbyconsulting.com>
References: <4AA8F011.3080001@colbyconsulting.com>
	<29f585dd0909100541y420c76b9j440d7f97b94dbfc6@mail.gmail.com>
	<4AA8F6CE.9020105@colbyconsulting.com>
Message-ID: <b874372a0909100840o2bba31ra38dc2ed43aba9ec@mail.gmail.com>

John,
  It seems that to copy a database it might be easier if you simply backup
and restore a database and it could be much quicker and you avoid the log
issue, I'm wondering if the log file grows to a ridiculous size because you
literally copy data from the existing database to the new database.  If this
is the case doing a restore would allow you to simply restore the last known
good backup + differential, if you don't to this now, you might consider
having your sproc perform a simple step to backup your db as a DIFFERENTIAL
backup, then simply run a restore

--DIFFERENTIAL BACKUP

Declare @Disk AS VARCHAR(1000)
SET @Disk = 'E:\DBBAckups\DIFF_DBName_'+ CAST(YEAR(GETDATE()) AS
VARCHAR(4)) + RIGHT('0'+ CAST(MONTH(GETDATE()) AS  VARCHAR(4)), 2)  +
RIGHT('0' +CAST(DAY(GETDATE()) AS  VARCHAR(4)) ,2)+ '.Bak'
BACKUP DATABASE [DBName] TO DISK = @Disk WITH  INIT ,   NAME = N'DBNameDB
bkup',  NOSKIP WITH DIFFERENTIAL


--Then to restore the database you'll need to Restore the FULL database with
NORECOVERY which puts it into a restore state so you can also apply your
DIFFERENTIAL RESTORE.


CREATE TABLE #tmpFile (PID INT IDENTITY(1,1)NOT NULL, Result VARCHAR(255))
INSERT INTO #tmpFile (Result)
EXEC master.dbo.xp_cmdshell 'DIR E:\DBBackups\\FULL*.sqb /B /O-d'

Declare @RestoreDate AS DateTime
Declare @Disk AS NVARCHAR(1000)
SELECT TOP 1 @Disk = 'E:\DBBackups\' + Result FROM #tmpFile

--FULL DB RESTORE WITH NORECOVERY
SET @Disk = '-SQL "RESTORE DATABASE [DBNAME]  FROM DISK = '''+ at Disk+'''
WITH  NORECOVERY, MOVE ''DataBaseName1'' TO ''D:\Program Files\Microsoft SQL
Server\MSSQL\Data\DatabaseName1.MDF'', MOVE ''DatabaseName2'' TO
''d:\Program Files\Microsoft SQL Server\MSSQL\Data\DataBaseName2.mdf'', MOVE
''DatabaseName_log'' TO ''d:\Program Files\Microsoft SQL
Server\MSSQL\Data\DatabaseName_log.ldf'', REPLACE"'
EXEC master..sqlbackup @Disk

DROP TABLE #tmpFile

--Now Restore the DIFFERENTIAL BACKUP

CREATE TABLE #tmpFile (PID INT IDENTITY(1,1)NOT NULL, Result VARCHAR(255))
INSERT INTO #tmpFile (Result)
EXEC master.dbo.xp_cmdshell 'DIR E:\DBBackups\DIFF*.sqb /B /O-d'

Declare @RestoreDate AS DateTime
Declare @Disk AS NVARCHAR(1000)
SELECT TOP 1 @Disk = 'E:\DBBackups\' + Result FROM #tmpFile

--DIFF DB RESTORE WITH RECOVERY
SET @Disk = '-SQL "RESTORE DATABASE [DBNAME]  FROM DISK = '''+ at Disk+'''
WITH  RECOVERY, MOVE ''DataBaseName1'' TO ''D:\Program Files\Microsoft SQL
Server\MSSQL\Data\DatabaseName1.MDF'', MOVE ''DatabaseName2'' TO
''d:\Program Files\Microsoft SQL Server\MSSQL\Data\DataBaseName2.mdf'', MOVE
''DatabaseName_log'' TO ''d:\Program Files\Microsoft SQL
Server\MSSQL\Data\DatabaseName_log.ldf'', REPLACE"'
EXEC master..sqlbackup @Disk

DROP TABLE #tmpFile




To shrink your log files:
BACKUP LOG DATABASENAME WITH TRUNCATE_ONLY  ---<<< this just clears the
checkpoints from your log file
DBCC SHRINKFILE ('Data_LogFile', 1)  ---- performs the actual shrinking of
the log file which won't happen if you have uncommitted checkpoints in your
log file.





-Francisco
http://sqlthis.blogspot.com | Tsql and More...


On Thu, Sep 10, 2009 at 5:53 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:

> Interesting Arthur.
>
> So from the gui I right click on the db of interest, /tasks / copy
> database.  A wizard opens and I
> do my thing, naming the files, where they should go etc.
>
> Are you saying that SQL Server saves that somewhere as TSQL and I can go
> look at it?
>
> John W. Colby
> www.ColbyConsulting.com
>
>
> Arthur Fuller wrote:
> > I've never had to do this, but I think that this approach would work. The
> > whole Management Studio UI (and Enterprise Manager before it) are nothing
> > but front ends that hide T-SQL commands. Whenever I want to know what
> SSMS
> > is doing under the covers, I execute the command of interest using the
> GUI
> > and then view the query syntax (include query with results).
> > Arthur
> >
> > On Thu, Sep 10, 2009 at 8:24 AM, jwcolby <jwcolby at colbyconsulting.com
> >wrote:
> >
> >> I need to programmatically copy a database to a new name (make a copy),
> and
> >> specify where the data
> >> file and log file goes.  Can anyone give me help in figuring that out?
>  I
> >> want to write a stored
> >> procedure to do this and call that from Access.
> >>
> >>
> > _______________________________________________
> > dba-SQLServer mailing list
> > dba-SQLServer at databaseadvisors.com
> > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> > http://www.databaseadvisors.com
> >
> >
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>
>


From jwcolby at colbyconsulting.com  Thu Sep 10 11:24:56 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Thu, 10 Sep 2009 12:24:56 -0400
Subject: [dba-SQLServer] SQL Server 2005 - Copying a database
In-Reply-To: <b874372a0909100840o2bba31ra38dc2ed43aba9ec@mail.gmail.com>
References: <4AA8F011.3080001@colbyconsulting.com>	<29f585dd0909100541y420c76b9j440d7f97b94dbfc6@mail.gmail.com>	<4AA8F6CE.9020105@colbyconsulting.com>
	<b874372a0909100840o2bba31ra38dc2ed43aba9ec@mail.gmail.com>
Message-ID: <4AA92858.3010907@colbyconsulting.com>

Francisco,

Thanks for the response.  The situation is that I have a small "template" database.  The total thing 
is only a few megabytes, but it has a couple of empty "template tables" as well as about 30 views 
and about 15 stored procedures.

I then copy that database to a new name, the name of an order such as XYZ1209.  I go in and tweak a 
couple of pre-existing views and run a stored procedure which in turns runs about 10 other stored 
procedures.  Those stored procedures build tblOrderData up from scratch with field names specific to 
that order, and populates tblOrderData with data pulled from the "database from hell".

It is that process of populating tblOrderData, and later doing updates to a couple of fields in 
tblOrderData that can cause the log file to swell.

When I am done with this whole process I pretty much just leave that order database alone, in fact 
after a couple of months I usually detach the db.  I MIGHT have to reference tblOrderData in a 
subsequent order to not use records used in a previous order, stuff like that.  But nothing that 
would ever swell the log file again.

So I need to:

1) Copy the template to a new name
2) Get the log file in a different drive ("set of spindles") for performance reasons.
3) Shrink the log file at the very end when I am done with the order.

I need to do this from Access - and eventually from C#.Net.  Right now I have an Access database 
that does all this stuff for me, allows me to execute stored procedures in these order databases 
etc.  Right now I do those three pieces manually, and it would be nice to automate those pieces as well.

I will look at your code to get this done.

Much appreciated.

John W. Colby
www.ColbyConsulting.com


Francisco Tapia wrote:
> John,
>   It seems that to copy a database it might be easier if you simply backup
> and restore a database and it could be much quicker and you avoid the log
> issue, I'm wondering if the log file grows to a ridiculous size because you
> literally copy data from the existing database to the new database.  If this
> is the case doing a restore would allow you to simply restore the last known
> good backup + differential, if you don't to this now, you might consider
> having your sproc perform a simple step to backup your db as a DIFFERENTIAL
> backup, then simply run a restore
> 
> --DIFFERENTIAL BACKUP
> 
> Declare @Disk AS VARCHAR(1000)
> SET @Disk = 'E:\DBBAckups\DIFF_DBName_'+ CAST(YEAR(GETDATE()) AS
> VARCHAR(4)) + RIGHT('0'+ CAST(MONTH(GETDATE()) AS  VARCHAR(4)), 2)  +
> RIGHT('0' +CAST(DAY(GETDATE()) AS  VARCHAR(4)) ,2)+ '.Bak'
> BACKUP DATABASE [DBName] TO DISK = @Disk WITH  INIT ,   NAME = N'DBNameDB
> bkup',  NOSKIP WITH DIFFERENTIAL
> 
> 
> --Then to restore the database you'll need to Restore the FULL database with
> NORECOVERY which puts it into a restore state so you can also apply your
> DIFFERENTIAL RESTORE.
> 
> 
> CREATE TABLE #tmpFile (PID INT IDENTITY(1,1)NOT NULL, Result VARCHAR(255))
> INSERT INTO #tmpFile (Result)
> EXEC master.dbo.xp_cmdshell 'DIR E:\DBBackups\\FULL*.sqb /B /O-d'
> 
> Declare @RestoreDate AS DateTime
> Declare @Disk AS NVARCHAR(1000)
> SELECT TOP 1 @Disk = 'E:\DBBackups\' + Result FROM #tmpFile
> 
> --FULL DB RESTORE WITH NORECOVERY
> SET @Disk = '-SQL "RESTORE DATABASE [DBNAME]  FROM DISK = '''+ at Disk+'''
> WITH  NORECOVERY, MOVE ''DataBaseName1'' TO ''D:\Program Files\Microsoft SQL
> Server\MSSQL\Data\DatabaseName1.MDF'', MOVE ''DatabaseName2'' TO
> ''d:\Program Files\Microsoft SQL Server\MSSQL\Data\DataBaseName2.mdf'', MOVE
> ''DatabaseName_log'' TO ''d:\Program Files\Microsoft SQL
> Server\MSSQL\Data\DatabaseName_log.ldf'', REPLACE"'
> EXEC master..sqlbackup @Disk
> 
> DROP TABLE #tmpFile
> 
> --Now Restore the DIFFERENTIAL BACKUP
> 
> CREATE TABLE #tmpFile (PID INT IDENTITY(1,1)NOT NULL, Result VARCHAR(255))
> INSERT INTO #tmpFile (Result)
> EXEC master.dbo.xp_cmdshell 'DIR E:\DBBackups\DIFF*.sqb /B /O-d'
> 
> Declare @RestoreDate AS DateTime
> Declare @Disk AS NVARCHAR(1000)
> SELECT TOP 1 @Disk = 'E:\DBBackups\' + Result FROM #tmpFile
> 
> --DIFF DB RESTORE WITH RECOVERY
> SET @Disk = '-SQL "RESTORE DATABASE [DBNAME]  FROM DISK = '''+ at Disk+'''
> WITH  RECOVERY, MOVE ''DataBaseName1'' TO ''D:\Program Files\Microsoft SQL
> Server\MSSQL\Data\DatabaseName1.MDF'', MOVE ''DatabaseName2'' TO
> ''d:\Program Files\Microsoft SQL Server\MSSQL\Data\DataBaseName2.mdf'', MOVE
> ''DatabaseName_log'' TO ''d:\Program Files\Microsoft SQL
> Server\MSSQL\Data\DatabaseName_log.ldf'', REPLACE"'
> EXEC master..sqlbackup @Disk
> 
> DROP TABLE #tmpFile
> 
> 
> 
> 
> To shrink your log files:
> BACKUP LOG DATABASENAME WITH TRUNCATE_ONLY  ---<<< this just clears the
> checkpoints from your log file
> DBCC SHRINKFILE ('Data_LogFile', 1)  ---- performs the actual shrinking of
> the log file which won't happen if you have uncommitted checkpoints in your
> log file.
> 
> 
> 
> 
> 
> -Francisco
> http://sqlthis.blogspot.com | Tsql and More...



From robert at webedb.com  Thu Sep 10 15:42:15 2009
From: robert at webedb.com (Robert Stewart)
Date: Thu, 10 Sep 2009 15:42:15 -0500
Subject: [dba-SQLServer] SQL Server 2005 - Copying a database
In-Reply-To: <mailman.3.1252602000.8721.dba-sqlserver@databaseadvisors.c
 om>
References: <mailman.3.1252602000.8721.dba-sqlserver@databaseadvisors.com>
Message-ID: <200909102042.n8AKgT3a007876@databaseadvisors.com>

John,

What you might want to do is shrink the log file by truncating it 
between each of the 10 SPs that you are running. This will keep the 
bloat down to a minimum and in the end, drop it down to it's smallest size.

The simplest way of doing it is to do a backup/restore. Then make the 
changes oyu want to in it.

Robert

At 12:00 PM 9/10/2009, you wrote:
>Date: Thu, 10 Sep 2009 12:24:56 -0400
>From: jwcolby <jwcolby at colbyconsulting.com>
>Subject: Re: [dba-SQLServer] SQL Server 2005 - Copying a database
>To: Discussion concerning MS SQL Server
>         <dba-sqlserver at databaseadvisors.com>
>Message-ID: <4AA92858.3010907 at colbyconsulting.com>
>Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
>Francisco,
>
>Thanks for the response.  The situation is that I have a small 
>"template" database.  The total thing
>is only a few megabytes, but it has a couple of empty "template 
>tables" as well as about 30 views
>and about 15 stored procedures.
>
>I then copy that database to a new name, the name of an order such 
>as XYZ1209.  I go in and tweak a
>couple of pre-existing views and run a stored procedure which in 
>turns runs about 10 other stored
>procedures.  Those stored procedures build tblOrderData up from 
>scratch with field names specific to
>that order, and populates tblOrderData with data pulled from the 
>"database from hell".
>
>It is that process of populating tblOrderData, and later doing 
>updates to a couple of fields in
>tblOrderData that can cause the log file to swell.
>
>When I am done with this whole process I pretty much just leave that 
>order database alone, in fact
>after a couple of months I usually detach the db.  I MIGHT have to 
>reference tblOrderData in a
>subsequent order to not use records used in a previous order, stuff 
>like that.  But nothing that
>would ever swell the log file again.
>
>So I need to:
>
>1) Copy the template to a new name
>2) Get the log file in a different drive ("set of spindles") for 
>performance reasons.
>3) Shrink the log file at the very end when I am done with the order.
>
>I need to do this from Access - and eventually from C#.Net.  Right 
>now I have an Access database
>that does all this stuff for me, allows me to execute stored 
>procedures in these order databases
>etc.  Right now I do those three pieces manually, and it would be 
>nice to automate those pieces as well.
>
>I will look at your code to get this done.
>
>Much appreciated.
>
>John W. Colby
>www.ColbyConsulting.com
>
>
>Francisco Tapia wrote:
> > John,
> >   It seems that to copy a database it might be easier if you simply backup
> > and restore a database and it could be much quicker and you avoid the log
> > issue, I'm wondering if the log file grows to a ridiculous size because you
> > literally copy data from the existing database to the new 
> database.  If this
> > is the case doing a restore would allow you to simply restore the 
> last known
> > good backup + differential, if you don't to this now, you might consider
> > having your sproc perform a simple step to backup your db as a DIFFERENTIAL
> > backup, then simply run a restore
> >
> > --DIFFERENTIAL BACKUP
> >
> > Declare @Disk AS VARCHAR(1000)
> > SET @Disk = 'E:\DBBAckups\DIFF_DBName_'+ CAST(YEAR(GETDATE()) AS
> > VARCHAR(4)) + RIGHT('0'+ CAST(MONTH(GETDATE()) AS  VARCHAR(4)), 2)  +
> > RIGHT('0' +CAST(DAY(GETDATE()) AS  VARCHAR(4)) ,2)+ '.Bak'
> > BACKUP DATABASE [DBName] TO DISK = @Disk WITH  INIT ,   NAME = N'DBNameDB
> > bkup',  NOSKIP WITH DIFFERENTIAL
> >
> >
> > --Then to restore the database you'll need to Restore the FULL 
> database with
> > NORECOVERY which puts it into a restore state so you can also apply your
> > DIFFERENTIAL RESTORE.
> >
> >
> > CREATE TABLE #tmpFile (PID INT IDENTITY(1,1)NOT NULL, Result VARCHAR(255))
> > INSERT INTO #tmpFile (Result)
> > EXEC master.dbo.xp_cmdshell 'DIR E:\DBBackups\\FULL*.sqb /B /O-d'
> >
> > Declare @RestoreDate AS DateTime
> > Declare @Disk AS NVARCHAR(1000)
> > SELECT TOP 1 @Disk = 'E:\DBBackups\' + Result FROM #tmpFile
> >
> > --FULL DB RESTORE WITH NORECOVERY
> > SET @Disk = '-SQL "RESTORE DATABASE [DBNAME]  FROM DISK = '''+ at Disk+'''
> > WITH  NORECOVERY, MOVE ''DataBaseName1'' TO ''D:\Program 
> Files\Microsoft SQL
> > Server\MSSQL\Data\DatabaseName1.MDF'', MOVE ''DatabaseName2'' TO
> > ''d:\Program Files\Microsoft SQL 
> Server\MSSQL\Data\DataBaseName2.mdf'', MOVE
> > ''DatabaseName_log'' TO ''d:\Program Files\Microsoft SQL
> > Server\MSSQL\Data\DatabaseName_log.ldf'', REPLACE"'
> > EXEC master..sqlbackup @Disk
> >
> > DROP TABLE #tmpFile
> >
> > --Now Restore the DIFFERENTIAL BACKUP
> >
> > CREATE TABLE #tmpFile (PID INT IDENTITY(1,1)NOT NULL, Result VARCHAR(255))
> > INSERT INTO #tmpFile (Result)
> > EXEC master.dbo.xp_cmdshell 'DIR E:\DBBackups\DIFF*.sqb /B /O-d'
> >
> > Declare @RestoreDate AS DateTime
> > Declare @Disk AS NVARCHAR(1000)
> > SELECT TOP 1 @Disk = 'E:\DBBackups\' + Result FROM #tmpFile
> >
> > --DIFF DB RESTORE WITH RECOVERY
> > SET @Disk = '-SQL "RESTORE DATABASE [DBNAME]  FROM DISK = '''+ at Disk+'''
> > WITH  RECOVERY, MOVE ''DataBaseName1'' TO ''D:\Program Files\Microsoft SQL
> > Server\MSSQL\Data\DatabaseName1.MDF'', MOVE ''DatabaseName2'' TO
> > ''d:\Program Files\Microsoft SQL 
> Server\MSSQL\Data\DataBaseName2.mdf'', MOVE
> > ''DatabaseName_log'' TO ''d:\Program Files\Microsoft SQL
> > Server\MSSQL\Data\DatabaseName_log.ldf'', REPLACE"'
> > EXEC master..sqlbackup @Disk
> >
> > DROP TABLE #tmpFile
> >
> >
> >
> >
> > To shrink your log files:
> > BACKUP LOG DATABASENAME WITH TRUNCATE_ONLY  ---<<< this just clears the
> > checkpoints from your log file
> > DBCC SHRINKFILE ('Data_LogFile', 1)  ---- performs the actual shrinking of
> > the log file which won't happen if you have uncommitted checkpoints in your
> > log file.
> >


From ab-mi at post3.tele.dk  Thu Sep 10 16:28:06 2009
From: ab-mi at post3.tele.dk (Asger Blond)
Date: Thu, 10 Sep 2009 23:28:06 +0200
Subject: [dba-SQLServer] SQL Server 2005 - Copying a database
In-Reply-To: <200909102042.n8AKgT3a007876@databaseadvisors.com>
References: <mailman.3.1252602000.8721.dba-sqlserver@databaseadvisors.com>
	<200909102042.n8AKgT3a007876@databaseadvisors.com>
Message-ID: <E828F662B3F24411AD42D04B5F287563@abpc>

To prevent the log swelling you could just set the recovery model to simple
before executing the stored procedures then set it back to full:

ALTER DATABASE <dbname> SET RECOVERY SIMPLE
Run stored procedures
ALTER DATABASE <dbname> SET RECOVERY FULL

Asger

-----Oprindelig meddelelse-----
Fra: dba-sqlserver-bounces at databaseadvisors.com
[mailto:dba-sqlserver-bounces at databaseadvisors.com] P? vegne af Robert
Stewart
Sendt: 10. september 2009 22:42
Til: dba-sqlserver at databaseadvisors.com
Emne: Re: [dba-SQLServer] SQL Server 2005 - Copying a database

John,

What you might want to do is shrink the log file by truncating it 
between each of the 10 SPs that you are running. This will keep the 
bloat down to a minimum and in the end, drop it down to it's smallest size.

The simplest way of doing it is to do a backup/restore. Then make the 
changes oyu want to in it.

Robert

At 12:00 PM 9/10/2009, you wrote:
>Date: Thu, 10 Sep 2009 12:24:56 -0400
>From: jwcolby <jwcolby at colbyconsulting.com>
>Subject: Re: [dba-SQLServer] SQL Server 2005 - Copying a database
>To: Discussion concerning MS SQL Server
>         <dba-sqlserver at databaseadvisors.com>
>Message-ID: <4AA92858.3010907 at colbyconsulting.com>
>Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
>Francisco,
>
>Thanks for the response.  The situation is that I have a small 
>"template" database.  The total thing
>is only a few megabytes, but it has a couple of empty "template 
>tables" as well as about 30 views
>and about 15 stored procedures.
>
>I then copy that database to a new name, the name of an order such 
>as XYZ1209.  I go in and tweak a
>couple of pre-existing views and run a stored procedure which in 
>turns runs about 10 other stored
>procedures.  Those stored procedures build tblOrderData up from 
>scratch with field names specific to
>that order, and populates tblOrderData with data pulled from the 
>"database from hell".
>
>It is that process of populating tblOrderData, and later doing 
>updates to a couple of fields in
>tblOrderData that can cause the log file to swell.
>
>When I am done with this whole process I pretty much just leave that 
>order database alone, in fact
>after a couple of months I usually detach the db.  I MIGHT have to 
>reference tblOrderData in a
>subsequent order to not use records used in a previous order, stuff 
>like that.  But nothing that
>would ever swell the log file again.
>
>So I need to:
>
>1) Copy the template to a new name
>2) Get the log file in a different drive ("set of spindles") for 
>performance reasons.
>3) Shrink the log file at the very end when I am done with the order.
>
>I need to do this from Access - and eventually from C#.Net.  Right 
>now I have an Access database
>that does all this stuff for me, allows me to execute stored 
>procedures in these order databases
>etc.  Right now I do those three pieces manually, and it would be 
>nice to automate those pieces as well.
>
>I will look at your code to get this done.
>
>Much appreciated.
>
>John W. Colby
>www.ColbyConsulting.com
>
>
>Francisco Tapia wrote:
> > John,
> >   It seems that to copy a database it might be easier if you simply
backup
> > and restore a database and it could be much quicker and you avoid the
log
> > issue, I'm wondering if the log file grows to a ridiculous size because
you
> > literally copy data from the existing database to the new 
> database.  If this
> > is the case doing a restore would allow you to simply restore the 
> last known
> > good backup + differential, if you don't to this now, you might consider
> > having your sproc perform a simple step to backup your db as a
DIFFERENTIAL
> > backup, then simply run a restore
> >
> > --DIFFERENTIAL BACKUP
> >
> > Declare @Disk AS VARCHAR(1000)
> > SET @Disk = 'E:\DBBAckups\DIFF_DBName_'+ CAST(YEAR(GETDATE()) AS
> > VARCHAR(4)) + RIGHT('0'+ CAST(MONTH(GETDATE()) AS  VARCHAR(4)), 2)  +
> > RIGHT('0' +CAST(DAY(GETDATE()) AS  VARCHAR(4)) ,2)+ '.Bak'
> > BACKUP DATABASE [DBName] TO DISK = @Disk WITH  INIT ,   NAME =
N'DBNameDB
> > bkup',  NOSKIP WITH DIFFERENTIAL
> >
> >
> > --Then to restore the database you'll need to Restore the FULL 
> database with
> > NORECOVERY which puts it into a restore state so you can also apply your
> > DIFFERENTIAL RESTORE.
> >
> >
> > CREATE TABLE #tmpFile (PID INT IDENTITY(1,1)NOT NULL, Result
VARCHAR(255))
> > INSERT INTO #tmpFile (Result)
> > EXEC master.dbo.xp_cmdshell 'DIR E:\DBBackups\\FULL*.sqb /B /O-d'
> >
> > Declare @RestoreDate AS DateTime
> > Declare @Disk AS NVARCHAR(1000)
> > SELECT TOP 1 @Disk = 'E:\DBBackups\' + Result FROM #tmpFile
> >
> > --FULL DB RESTORE WITH NORECOVERY
> > SET @Disk = '-SQL "RESTORE DATABASE [DBNAME]  FROM DISK = '''+ at Disk+'''
> > WITH  NORECOVERY, MOVE ''DataBaseName1'' TO ''D:\Program 
> Files\Microsoft SQL
> > Server\MSSQL\Data\DatabaseName1.MDF'', MOVE ''DatabaseName2'' TO
> > ''d:\Program Files\Microsoft SQL 
> Server\MSSQL\Data\DataBaseName2.mdf'', MOVE
> > ''DatabaseName_log'' TO ''d:\Program Files\Microsoft SQL
> > Server\MSSQL\Data\DatabaseName_log.ldf'', REPLACE"'
> > EXEC master..sqlbackup @Disk
> >
> > DROP TABLE #tmpFile
> >
> > --Now Restore the DIFFERENTIAL BACKUP
> >
> > CREATE TABLE #tmpFile (PID INT IDENTITY(1,1)NOT NULL, Result
VARCHAR(255))
> > INSERT INTO #tmpFile (Result)
> > EXEC master.dbo.xp_cmdshell 'DIR E:\DBBackups\DIFF*.sqb /B /O-d'
> >
> > Declare @RestoreDate AS DateTime
> > Declare @Disk AS NVARCHAR(1000)
> > SELECT TOP 1 @Disk = 'E:\DBBackups\' + Result FROM #tmpFile
> >
> > --DIFF DB RESTORE WITH RECOVERY
> > SET @Disk = '-SQL "RESTORE DATABASE [DBNAME]  FROM DISK = '''+ at Disk+'''
> > WITH  RECOVERY, MOVE ''DataBaseName1'' TO ''D:\Program Files\Microsoft
SQL
> > Server\MSSQL\Data\DatabaseName1.MDF'', MOVE ''DatabaseName2'' TO
> > ''d:\Program Files\Microsoft SQL 
> Server\MSSQL\Data\DataBaseName2.mdf'', MOVE
> > ''DatabaseName_log'' TO ''d:\Program Files\Microsoft SQL
> > Server\MSSQL\Data\DatabaseName_log.ldf'', REPLACE"'
> > EXEC master..sqlbackup @Disk
> >
> > DROP TABLE #tmpFile
> >
> >
> >
> >
> > To shrink your log files:
> > BACKUP LOG DATABASENAME WITH TRUNCATE_ONLY  ---<<< this just clears the
> > checkpoints from your log file
> > DBCC SHRINKFILE ('Data_LogFile', 1)  ---- performs the actual shrinking
of
> > the log file which won't happen if you have uncommitted checkpoints in
your
> > log file.
> >
_______________________________________________
dba-SQLServer mailing list
dba-SQLServer at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
http://www.databaseadvisors.com




From ab-mi at post3.tele.dk  Thu Sep 10 17:19:52 2009
From: ab-mi at post3.tele.dk (Asger Blond)
Date: Fri, 11 Sep 2009 00:19:52 +0200
Subject: [dba-SQLServer] SQL Server 2005 - Copying a database
In-Reply-To: <E828F662B3F24411AD42D04B5F287563@abpc>
References: <mailman.3.1252602000.8721.dba-sqlserver@databaseadvisors.com><200909102042.n8AKgT3a007876@databaseadvisors.com>
	<E828F662B3F24411AD42D04B5F287563@abpc>
Message-ID: <6301A8E896184C60A710C8DEF568112D@abpc>

Just one more point:
Remember to make a full database backup after switching the recovery model.
Otherwise you won't be able to make log-backups.
You could create a main-procedure like this:

CREATE PROCEDURE spMain
AS
ALTER DATABASE <dbname> SET RECOVERY SIMPLE
EXEC sp1
EXEC sp2
EXEC sp3 (etc)
ALTER DATABASE <dbname> SET RECOVERY FULL
BACKUP DATABASE <dbname> TO DISK = 'X:\mybackups\mydb.bak'
GO

Asger

-----Oprindelig meddelelse-----
Fra: dba-sqlserver-bounces at databaseadvisors.com
[mailto:dba-sqlserver-bounces at databaseadvisors.com] P? vegne af Asger Blond
Sendt: 10. september 2009 23:28
Til: 'Discussion concerning MS SQL Server'
Emne: Re: [dba-SQLServer] SQL Server 2005 - Copying a database

To prevent the log swelling you could just set the recovery model to simple
before executing the stored procedures then set it back to full:

ALTER DATABASE <dbname> SET RECOVERY SIMPLE
Run stored procedures
ALTER DATABASE <dbname> SET RECOVERY FULL

Asger

-----Oprindelig meddelelse-----
Fra: dba-sqlserver-bounces at databaseadvisors.com
[mailto:dba-sqlserver-bounces at databaseadvisors.com] P? vegne af Robert
Stewart
Sendt: 10. september 2009 22:42
Til: dba-sqlserver at databaseadvisors.com
Emne: Re: [dba-SQLServer] SQL Server 2005 - Copying a database

John,

What you might want to do is shrink the log file by truncating it 
between each of the 10 SPs that you are running. This will keep the 
bloat down to a minimum and in the end, drop it down to it's smallest size.

The simplest way of doing it is to do a backup/restore. Then make the 
changes oyu want to in it.

Robert

At 12:00 PM 9/10/2009, you wrote:
>Date: Thu, 10 Sep 2009 12:24:56 -0400
>From: jwcolby <jwcolby at colbyconsulting.com>
>Subject: Re: [dba-SQLServer] SQL Server 2005 - Copying a database
>To: Discussion concerning MS SQL Server
>         <dba-sqlserver at databaseadvisors.com>
>Message-ID: <4AA92858.3010907 at colbyconsulting.com>
>Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
>Francisco,
>
>Thanks for the response.  The situation is that I have a small 
>"template" database.  The total thing
>is only a few megabytes, but it has a couple of empty "template 
>tables" as well as about 30 views
>and about 15 stored procedures.
>
>I then copy that database to a new name, the name of an order such 
>as XYZ1209.  I go in and tweak a
>couple of pre-existing views and run a stored procedure which in 
>turns runs about 10 other stored
>procedures.  Those stored procedures build tblOrderData up from 
>scratch with field names specific to
>that order, and populates tblOrderData with data pulled from the 
>"database from hell".
>
>It is that process of populating tblOrderData, and later doing 
>updates to a couple of fields in
>tblOrderData that can cause the log file to swell.
>
>When I am done with this whole process I pretty much just leave that 
>order database alone, in fact
>after a couple of months I usually detach the db.  I MIGHT have to 
>reference tblOrderData in a
>subsequent order to not use records used in a previous order, stuff 
>like that.  But nothing that
>would ever swell the log file again.
>
>So I need to:
>
>1) Copy the template to a new name
>2) Get the log file in a different drive ("set of spindles") for 
>performance reasons.
>3) Shrink the log file at the very end when I am done with the order.
>
>I need to do this from Access - and eventually from C#.Net.  Right 
>now I have an Access database
>that does all this stuff for me, allows me to execute stored 
>procedures in these order databases
>etc.  Right now I do those three pieces manually, and it would be 
>nice to automate those pieces as well.
>
>I will look at your code to get this done.
>
>Much appreciated.
>
>John W. Colby
>www.ColbyConsulting.com
>
>
>Francisco Tapia wrote:
> > John,
> >   It seems that to copy a database it might be easier if you simply
backup
> > and restore a database and it could be much quicker and you avoid the
log
> > issue, I'm wondering if the log file grows to a ridiculous size because
you
> > literally copy data from the existing database to the new 
> database.  If this
> > is the case doing a restore would allow you to simply restore the 
> last known
> > good backup + differential, if you don't to this now, you might consider
> > having your sproc perform a simple step to backup your db as a
DIFFERENTIAL
> > backup, then simply run a restore
> >
> > --DIFFERENTIAL BACKUP
> >
> > Declare @Disk AS VARCHAR(1000)
> > SET @Disk = 'E:\DBBAckups\DIFF_DBName_'+ CAST(YEAR(GETDATE()) AS
> > VARCHAR(4)) + RIGHT('0'+ CAST(MONTH(GETDATE()) AS  VARCHAR(4)), 2)  +
> > RIGHT('0' +CAST(DAY(GETDATE()) AS  VARCHAR(4)) ,2)+ '.Bak'
> > BACKUP DATABASE [DBName] TO DISK = @Disk WITH  INIT ,   NAME =
N'DBNameDB
> > bkup',  NOSKIP WITH DIFFERENTIAL
> >
> >
> > --Then to restore the database you'll need to Restore the FULL 
> database with
> > NORECOVERY which puts it into a restore state so you can also apply your
> > DIFFERENTIAL RESTORE.
> >
> >
> > CREATE TABLE #tmpFile (PID INT IDENTITY(1,1)NOT NULL, Result
VARCHAR(255))
> > INSERT INTO #tmpFile (Result)
> > EXEC master.dbo.xp_cmdshell 'DIR E:\DBBackups\\FULL*.sqb /B /O-d'
> >
> > Declare @RestoreDate AS DateTime
> > Declare @Disk AS NVARCHAR(1000)
> > SELECT TOP 1 @Disk = 'E:\DBBackups\' + Result FROM #tmpFile
> >
> > --FULL DB RESTORE WITH NORECOVERY
> > SET @Disk = '-SQL "RESTORE DATABASE [DBNAME]  FROM DISK = '''+ at Disk+'''
> > WITH  NORECOVERY, MOVE ''DataBaseName1'' TO ''D:\Program 
> Files\Microsoft SQL
> > Server\MSSQL\Data\DatabaseName1.MDF'', MOVE ''DatabaseName2'' TO
> > ''d:\Program Files\Microsoft SQL 
> Server\MSSQL\Data\DataBaseName2.mdf'', MOVE
> > ''DatabaseName_log'' TO ''d:\Program Files\Microsoft SQL
> > Server\MSSQL\Data\DatabaseName_log.ldf'', REPLACE"'
> > EXEC master..sqlbackup @Disk
> >
> > DROP TABLE #tmpFile
> >
> > --Now Restore the DIFFERENTIAL BACKUP
> >
> > CREATE TABLE #tmpFile (PID INT IDENTITY(1,1)NOT NULL, Result
VARCHAR(255))
> > INSERT INTO #tmpFile (Result)
> > EXEC master.dbo.xp_cmdshell 'DIR E:\DBBackups\DIFF*.sqb /B /O-d'
> >
> > Declare @RestoreDate AS DateTime
> > Declare @Disk AS NVARCHAR(1000)
> > SELECT TOP 1 @Disk = 'E:\DBBackups\' + Result FROM #tmpFile
> >
> > --DIFF DB RESTORE WITH RECOVERY
> > SET @Disk = '-SQL "RESTORE DATABASE [DBNAME]  FROM DISK = '''+ at Disk+'''
> > WITH  RECOVERY, MOVE ''DataBaseName1'' TO ''D:\Program Files\Microsoft
SQL
> > Server\MSSQL\Data\DatabaseName1.MDF'', MOVE ''DatabaseName2'' TO
> > ''d:\Program Files\Microsoft SQL 
> Server\MSSQL\Data\DataBaseName2.mdf'', MOVE
> > ''DatabaseName_log'' TO ''d:\Program Files\Microsoft SQL
> > Server\MSSQL\Data\DatabaseName_log.ldf'', REPLACE"'
> > EXEC master..sqlbackup @Disk
> >
> > DROP TABLE #tmpFile
> >
> >
> >
> >
> > To shrink your log files:
> > BACKUP LOG DATABASENAME WITH TRUNCATE_ONLY  ---<<< this just clears the
> > checkpoints from your log file
> > DBCC SHRINKFILE ('Data_LogFile', 1)  ---- performs the actual shrinking
of
> > the log file which won't happen if you have uncommitted checkpoints in
your
> > log file.
> >
_______________________________________________
dba-SQLServer mailing list
dba-SQLServer at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
http://www.databaseadvisors.com


_______________________________________________
dba-SQLServer mailing list
dba-SQLServer at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
http://www.databaseadvisors.com




From ab-mi at post3.tele.dk  Thu Sep 10 18:41:41 2009
From: ab-mi at post3.tele.dk (Asger Blond)
Date: Fri, 11 Sep 2009 01:41:41 +0200
Subject: [dba-SQLServer] OT: missing line breaks in mails to databaseadvisors
Message-ID: <043678F52F224CF2AFAB5DBC91D50BEF@abpc>

Dear listers

 

Why my line breaks in mails to this list are a mess? Often they are just
ignored making code examples quite unreadable.

Don't know if this is some settings in my Outlook which only affects my
reading and not yours.

So could somebody please tell me if my last two postings on this list are
missing my much beloved line breakers?

Maybe the problem is just on my machine. Maybe it's an overall issue. Or
maybe my paranoia just once again proves true.

 

Asger



From stuart at lexacorp.com.pg  Thu Sep 10 18:48:53 2009
From: stuart at lexacorp.com.pg (Stuart McLachlan)
Date: Fri, 11 Sep 2009 09:48:53 +1000
Subject: [dba-SQLServer] OT: missing line breaks in mails to
	databaseadvisors
In-Reply-To: <043678F52F224CF2AFAB5DBC91D50BEF@abpc>
References: <043678F52F224CF2AFAB5DBC91D50BEF@abpc>
Message-ID: <4AA99065.26323.288F2955@stuart.lexacorp.com.pg>

Looks like It's your Outlook display.

Looking at the emails as received here, the last two had appropriate line breaks - the code 
was fine.

-- 
Stuart


On 11 Sep 2009 at 1:41, Asger Blond wrote:

> Dear listers
> 
>  
> 
> Why my line breaks in mails to this list are a mess? Often they are just
> ignored making code examples quite unreadable.
> 
> Don't know if this is some settings in my Outlook which only affects my
> reading and not yours.
> 
> So could somebody please tell me if my last two postings on this list are
> missing my much beloved line breakers?
> 
> Maybe the problem is just on my machine. Maybe it's an overall issue. Or
> maybe my paranoia just once again proves true.
> 
>  
> 
> Asger
> 
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 




From ab-mi at post3.tele.dk  Thu Sep 10 19:04:27 2009
From: ab-mi at post3.tele.dk (Asger Blond)
Date: Fri, 11 Sep 2009 02:04:27 +0200
Subject: [dba-SQLServer] OT: missing line breaks in mails
	todatabaseadvisors
In-Reply-To: <4AA99065.26323.288F2955@stuart.lexacorp.com.pg>
References: <043678F52F224CF2AFAB5DBC91D50BEF@abpc>
	<4AA99065.26323.288F2955@stuart.lexacorp.com.pg>
Message-ID: <D044FD0F4C434398AE270CA3F1358366@abpc>

Thanks Stuart,
So the problem is just here. Which leaves two options: a good advice making
my Outlook perform decent, or ... shrink.
Asger

-----Oprindelig meddelelse-----
Fra: dba-sqlserver-bounces at databaseadvisors.com
[mailto:dba-sqlserver-bounces at databaseadvisors.com] P? vegne af Stuart
McLachlan
Sendt: 11. september 2009 01:49
Til: Discussion concerning MS SQL Server
Emne: Re: [dba-SQLServer] OT: missing line breaks in mails
todatabaseadvisors

Looks like It's your Outlook display.

Looking at the emails as received here, the last two had appropriate line
breaks - the code 
was fine.

-- 
Stuart


On 11 Sep 2009 at 1:41, Asger Blond wrote:

> Dear listers
> 
>  
> 
> Why my line breaks in mails to this list are a mess? Often they are just
> ignored making code examples quite unreadable.
> 
> Don't know if this is some settings in my Outlook which only affects my
> reading and not yours.
> 
> So could somebody please tell me if my last two postings on this list are
> missing my much beloved line breakers?
> 
> Maybe the problem is just on my machine. Maybe it's an overall issue. Or
> maybe my paranoia just once again proves true.
> 
>  
> 
> Asger
> 
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 


_______________________________________________
dba-SQLServer mailing list
dba-SQLServer at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
http://www.databaseadvisors.com




From jwcolby at colbyconsulting.com  Fri Sep 11 09:05:02 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Fri, 11 Sep 2009 10:05:02 -0400
Subject: [dba-SQLServer] SQL Server 2008 compression
Message-ID: <4AAA590E.9060200@colbyconsulting.com>

Does anyone have experience with using compression in 2008?  I understand it is only available in 
the enterprise edition?

-- 
John W. Colby
www.ColbyConsulting.com


From fhtapia at gmail.com  Fri Sep 11 10:38:30 2009
From: fhtapia at gmail.com (Francisco Tapia)
Date: Fri, 11 Sep 2009 08:38:30 -0700
Subject: [dba-SQLServer] SQL Server 2008 compression
In-Reply-To: <4AAA590E.9060200@colbyconsulting.com>
References: <4AAA590E.9060200@colbyconsulting.com>
Message-ID: <b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>

Compression backups?
one thing to remember is that if you run a backup to a backup device you
can't have an uncompressed backup with a compressed backup. as far as cost
it might be cheaper to invest in redgate's backup software wich is fast and
robust we use it here to backup our 2TB databases. for 295 you really can't
beat the price on other solutions... its definitely cheaper than going 2008
enterprise.


more info here: http://msdn.microsoft.com/en-us/library/bb964719.aspx
http://msdn.microsoft.com/en-us/library/ms186865.aspx

alternitavely you may wish to use a free compression product such as:
http://www.idera.com/Products/Free-Tools/SQL-safe-Freeware-Edition/

-Francisco
http://sqlthis.blogspot.com | Tsql and More...


On Fri, Sep 11, 2009 at 7:05 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:

> Does anyone have experience with using compression in 2008?  I understand
> it is only available in
> the enterprise edition?
>
> --
> John W. Colby
> www.ColbyConsulting.com
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>
>


From fuller.artful at gmail.com  Fri Sep 11 11:10:24 2009
From: fuller.artful at gmail.com (Arthur Fuller)
Date: Fri, 11 Sep 2009 12:10:24 -0400
Subject: [dba-SQLServer] SQL Server 2008 compression
In-Reply-To: <b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>
References: <4AAA590E.9060200@colbyconsulting.com>
	<b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>
Message-ID: <29f585dd0909110910gf97d647q8505269bd080ca65@mail.gmail.com>

I couldn't agree more strongly, Francisco! In fact, I have persuaded several
clients to purchase the Red Gate Toolbelt simply by demo'ing SQL Backup from
my laptop, and illustrating that backups take 1/3 the time and occupy 1/3
the space. And now with the latest version, the space is down to 1/4.
Awesome technology. I especially love the feature that lets you copy the
backup to another machine automatically. That is beautiful!
Arthur

On Fri, Sep 11, 2009 at 11:38 AM, Francisco Tapia <fhtapia at gmail.com> wrote:

> Compression backups?
> one thing to remember is that if you run a backup to a backup device you
> can't have an uncompressed backup with a compressed backup. as far as cost
> it might be cheaper to invest in redgate's backup software wich is fast and
> robust we use it here to backup our 2TB databases. for 295 you really can't
> beat the price on other solutions... its definitely cheaper than going 2008
> enterprise.
>
>
> more info here: http://msdn.microsoft.com/en-us/library/bb964719.aspx
> http://msdn.microsoft.com/en-us/library/ms186865.aspx
>
> alternitavely you may wish to use a free compression product such as:
> http://www.idera.com/Products/Free-Tools/SQL-safe-Freeware-Edition/
>
> -Francisco
> http://sqlthis.blogspot.com | Tsql and More...
>
>
> On Fri, Sep 11, 2009 at 7:05 AM, jwcolby <jwcolby at colbyconsulting.com
> >wrote:
>
> > Does anyone have experience with using compression in 2008?  I understand
> > it is only available in
> > the enterprise edition?
> >
> > --
> > John W. Colby
> > www.ColbyConsulting.com
> > _______________________________________________
> > dba-SQLServer mailing list
> > dba-SQLServer at databaseadvisors.com
> > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> > http://www.databaseadvisors.com
> >
> >
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>
>


From jwcolby at colbyconsulting.com  Fri Sep 11 12:28:04 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Fri, 11 Sep 2009 13:28:04 -0400
Subject: [dba-SQLServer] SQL Server 2008 compression
In-Reply-To: <b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>
References: <4AAA590E.9060200@colbyconsulting.com>
	<b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>
Message-ID: <4AAA88A4.8020504@colbyconsulting.com>

Is redgate the same thing as Idera?

John W. Colby
www.ColbyConsulting.com


Francisco Tapia wrote:
> Compression backups?
> one thing to remember is that if you run a backup to a backup device you
> can't have an uncompressed backup with a compressed backup. as far as cost
> it might be cheaper to invest in redgate's backup software wich is fast and
> robust we use it here to backup our 2TB databases. for 295 you really can't
> beat the price on other solutions... its definitely cheaper than going 2008
> enterprise.
> 
> 
> more info here: http://msdn.microsoft.com/en-us/library/bb964719.aspx
> http://msdn.microsoft.com/en-us/library/ms186865.aspx
> 
> alternitavely you may wish to use a free compression product such as:
> http://www.idera.com/Products/Free-Tools/SQL-safe-Freeware-Edition/
> 
> -Francisco
> http://sqlthis.blogspot.com | Tsql and More...
> 
> 
> On Fri, Sep 11, 2009 at 7:05 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:
> 
>> Does anyone have experience with using compression in 2008?  I understand
>> it is only available in
>> the enterprise edition?
>>
>> --
>> John W. Colby
>> www.ColbyConsulting.com
>> _______________________________________________
>> dba-SQLServer mailing list
>> dba-SQLServer at databaseadvisors.com
>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>> http://www.databaseadvisors.com
>>
>>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From fhtapia at gmail.com  Fri Sep 11 12:30:03 2009
From: fhtapia at gmail.com (Francisco Tapia)
Date: Fri, 11 Sep 2009 10:30:03 -0700
Subject: [dba-SQLServer] SQL Server 2008 compression
In-Reply-To: <29f585dd0909110910gf97d647q8505269bd080ca65@mail.gmail.com>
References: <4AAA590E.9060200@colbyconsulting.com>
	<b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>
	<29f585dd0909110910gf97d647q8505269bd080ca65@mail.gmail.com>
Message-ID: <b874372a0909111030h3933b921q1c881a90e36013fc@mail.gmail.com>

at the risk of sounding like a salesman, I couldn't agree more, the copyto
feature has been in the product since sql backup 5, but also one of the cool
new features that I like is the network resilience option, so if your backup
fails you can choose how many retries it does and how long you want the
process to pause for between re-tries.  very nice stuff indeed.

the tightest compression of course comes at a price and that is it takes a
lot of cpu and memory resources so it's good to run when the system is idle,
but if you have a high production system I always schedule log backups to
occur with the highest speed instead, that way a log backup does not tie up
resources for too long.

I haven't had a chance to test sql prompt 4 have you?  from what i've been
reading it's good stuff and faster than sql prompt3 which I do use and love
to use.

recently i moved away from Idera's Diagnostic Manager (an expensive
monitoring tool) and started using RedGates new Sql Response.  It actually
works better for me especially because I can keep track of my fragmented
tables in our SAP systems that literally have over 70k tables you want to
talk a database from hell? look no further.. this thing is a beast.

-Francisco
http://sqlthis.blogspot.com | Tsql and More...


On Fri, Sep 11, 2009 at 9:10 AM, Arthur Fuller <fuller.artful at gmail.com>wrote:

> I couldn't agree more strongly, Francisco! In fact, I have persuaded
> several
> clients to purchase the Red Gate Toolbelt simply by demo'ing SQL Backup
> from
> my laptop, and illustrating that backups take 1/3 the time and occupy 1/3
> the space. And now with the latest version, the space is down to 1/4.
> Awesome technology. I especially love the feature that lets you copy the
> backup to another machine automatically. That is beautiful!
> Arthur
>
> On Fri, Sep 11, 2009 at 11:38 AM, Francisco Tapia <fhtapia at gmail.com>
> wrote:
>
> > Compression backups?
> > one thing to remember is that if you run a backup to a backup device you
> > can't have an uncompressed backup with a compressed backup. as far as
> cost
> > it might be cheaper to invest in redgate's backup software wich is fast
> and
> > robust we use it here to backup our 2TB databases. for 295 you really
> can't
> > beat the price on other solutions... its definitely cheaper than going
> 2008
> > enterprise.
> >
> >
> > more info here: http://msdn.microsoft.com/en-us/library/bb964719.aspx
> > http://msdn.microsoft.com/en-us/library/ms186865.aspx
> >
> > alternitavely you may wish to use a free compression product such as:
> > http://www.idera.com/Products/Free-Tools/SQL-safe-Freeware-Edition/
> >
> > -Francisco
> > http://sqlthis.blogspot.com | Tsql and More...
> >
> >
> > On Fri, Sep 11, 2009 at 7:05 AM, jwcolby <jwcolby at colbyconsulting.com
> > >wrote:
> >
> > > Does anyone have experience with using compression in 2008?  I
> understand
> > > it is only available in
> > > the enterprise edition?
> > >
> > > --
> > > John W. Colby
> > > www.ColbyConsulting.com
> > > _______________________________________________
> > > dba-SQLServer mailing list
> > > dba-SQLServer at databaseadvisors.com
> > > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> > > http://www.databaseadvisors.com
> > >
> > >
> > _______________________________________________
> > dba-SQLServer mailing list
> > dba-SQLServer at databaseadvisors.com
> > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> > http://www.databaseadvisors.com
> >
> >
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>
>


From fhtapia at gmail.com  Fri Sep 11 12:30:51 2009
From: fhtapia at gmail.com (Francisco Tapia)
Date: Fri, 11 Sep 2009 10:30:51 -0700
Subject: [dba-SQLServer] SQL Server 2008 compression
In-Reply-To: <4AAA88A4.8020504@colbyconsulting.com>
References: <4AAA590E.9060200@colbyconsulting.com>
	<b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>
	<4AAA88A4.8020504@colbyconsulting.com>
Message-ID: <b874372a0909111030j7802127bmc429d34e9612239@mail.gmail.com>

Different companies, but at least if you want to try a free compression
product Idera is the way to go, but a good product would be RedGate, that's
just mho, ymmv


-Francisco
http://sqlthis.blogspot.com | Tsql and More...


On Fri, Sep 11, 2009 at 10:28 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:

> Is redgate the same thing as Idera?
>
> John W. Colby
> www.ColbyConsulting.com
>
>
> Francisco Tapia wrote:
> > Compression backups?
> > one thing to remember is that if you run a backup to a backup device you
> > can't have an uncompressed backup with a compressed backup. as far as
> cost
> > it might be cheaper to invest in redgate's backup software wich is fast
> and
> > robust we use it here to backup our 2TB databases. for 295 you really
> can't
> > beat the price on other solutions... its definitely cheaper than going
> 2008
> > enterprise.
> >
> >
> > more info here: http://msdn.microsoft.com/en-us/library/bb964719.aspx
> > http://msdn.microsoft.com/en-us/library/ms186865.aspx
> >
> > alternitavely you may wish to use a free compression product such as:
> > http://www.idera.com/Products/Free-Tools/SQL-safe-Freeware-Edition/
> >
> > -Francisco
> > http://sqlthis.blogspot.com | Tsql and More...
> >
> >
> > On Fri, Sep 11, 2009 at 7:05 AM, jwcolby <jwcolby at colbyconsulting.com
> >wrote:
> >
> >> Does anyone have experience with using compression in 2008?  I
> understand
> >> it is only available in
> >> the enterprise edition?
> >>
> >> --
> >> John W. Colby
> >> www.ColbyConsulting.com
> >> _______________________________________________
> >> dba-SQLServer mailing list
> >> dba-SQLServer at databaseadvisors.com
> >> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> >> http://www.databaseadvisors.com
> >>
> >>
> > _______________________________________________
> > dba-SQLServer mailing list
> > dba-SQLServer at databaseadvisors.com
> > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> > http://www.databaseadvisors.com
> >
> >
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>
>


From jwcolby at colbyconsulting.com  Fri Sep 11 12:31:15 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Fri, 11 Sep 2009 13:31:15 -0400
Subject: [dba-SQLServer] SQL Server 2008 compression
In-Reply-To: <b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>
References: <4AAA590E.9060200@colbyconsulting.com>
	<b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>
Message-ID: <4AAA8963.1020307@colbyconsulting.com>

Francisco,

 From what I am reading, the compression works at all times, but ALSO during the backup.  I am sure 
there is a "cost" in application / database speed.  In the end, AFAICT it is only available in the 
enterprise edition which is way out of my budget range.

I was really just wondering if anyone had ever used it.

Thanks for the links though.  I will go take a look at their free stuff.  I paid about $700 for 
another company's product but I was going to have to pay to move to SQL Server 2005 (and now 2008) 
so I didn't.

I NEED backups!

Again, thanks for the links.

John W. Colby
www.ColbyConsulting.com


Francisco Tapia wrote:
> Compression backups?
> one thing to remember is that if you run a backup to a backup device you
> can't have an uncompressed backup with a compressed backup. as far as cost
> it might be cheaper to invest in redgate's backup software wich is fast and
> robust we use it here to backup our 2TB databases. for 295 you really can't
> beat the price on other solutions... its definitely cheaper than going 2008
> enterprise.
> 
> 
> more info here: http://msdn.microsoft.com/en-us/library/bb964719.aspx
> http://msdn.microsoft.com/en-us/library/ms186865.aspx
> 
> alternitavely you may wish to use a free compression product such as:
> http://www.idera.com/Products/Free-Tools/SQL-safe-Freeware-Edition/
> 
> -Francisco
> http://sqlthis.blogspot.com | Tsql and More...
> 
> 
> On Fri, Sep 11, 2009 at 7:05 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:
> 
>> Does anyone have experience with using compression in 2008?  I understand
>> it is only available in
>> the enterprise edition?
>>
>> --
>> John W. Colby
>> www.ColbyConsulting.com
>> _______________________________________________
>> dba-SQLServer mailing list
>> dba-SQLServer at databaseadvisors.com
>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>> http://www.databaseadvisors.com
>>
>>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From jwcolby at colbyconsulting.com  Fri Sep 11 12:36:52 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Fri, 11 Sep 2009 13:36:52 -0400
Subject: [dba-SQLServer] SQL Server 2008 compression
In-Reply-To: <b874372a0909111030h3933b921q1c881a90e36013fc@mail.gmail.com>
References: <4AAA590E.9060200@colbyconsulting.com>	<b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>	<29f585dd0909110910gf97d647q8505269bd080ca65@mail.gmail.com>
	<b874372a0909111030h3933b921q1c881a90e36013fc@mail.gmail.com>
Message-ID: <4AAA8AB4.6070705@colbyconsulting.com>

SEVENTY THOUSAND TABLES?

8(

How could anyone possibly know what is in there?

John W. Colby
www.ColbyConsulting.com


Francisco Tapia wrote:
> at the risk of sounding like a salesman, I couldn't agree more, the copyto
> feature has been in the product since sql backup 5, but also one of the cool
> new features that I like is the network resilience option, so if your backup
> fails you can choose how many retries it does and how long you want the
> process to pause for between re-tries.  very nice stuff indeed.
> 
> the tightest compression of course comes at a price and that is it takes a
> lot of cpu and memory resources so it's good to run when the system is idle,
> but if you have a high production system I always schedule log backups to
> occur with the highest speed instead, that way a log backup does not tie up
> resources for too long.
> 
> I haven't had a chance to test sql prompt 4 have you?  from what i've been
> reading it's good stuff and faster than sql prompt3 which I do use and love
> to use.
> 
> recently i moved away from Idera's Diagnostic Manager (an expensive
> monitoring tool) and started using RedGates new Sql Response.  It actually
> works better for me especially because I can keep track of my fragmented
> tables in our SAP systems that literally have over 70k tables you want to
> talk a database from hell? look no further.. this thing is a beast.
> 
> -Francisco
> http://sqlthis.blogspot.com | Tsql and More...
> 
> 
> On Fri, Sep 11, 2009 at 9:10 AM, Arthur Fuller <fuller.artful at gmail.com>wrote:
> 
>> I couldn't agree more strongly, Francisco! In fact, I have persuaded
>> several
>> clients to purchase the Red Gate Toolbelt simply by demo'ing SQL Backup
>> from
>> my laptop, and illustrating that backups take 1/3 the time and occupy 1/3
>> the space. And now with the latest version, the space is down to 1/4.
>> Awesome technology. I especially love the feature that lets you copy the
>> backup to another machine automatically. That is beautiful!
>> Arthur
>>
>> On Fri, Sep 11, 2009 at 11:38 AM, Francisco Tapia <fhtapia at gmail.com>
>> wrote:
>>
>>> Compression backups?
>>> one thing to remember is that if you run a backup to a backup device you
>>> can't have an uncompressed backup with a compressed backup. as far as
>> cost
>>> it might be cheaper to invest in redgate's backup software wich is fast
>> and
>>> robust we use it here to backup our 2TB databases. for 295 you really
>> can't
>>> beat the price on other solutions... its definitely cheaper than going
>> 2008
>>> enterprise.
>>>
>>>
>>> more info here: http://msdn.microsoft.com/en-us/library/bb964719.aspx
>>> http://msdn.microsoft.com/en-us/library/ms186865.aspx
>>>
>>> alternitavely you may wish to use a free compression product such as:
>>> http://www.idera.com/Products/Free-Tools/SQL-safe-Freeware-Edition/
>>>
>>> -Francisco
>>> http://sqlthis.blogspot.com | Tsql and More...
>>>
>>>
>>> On Fri, Sep 11, 2009 at 7:05 AM, jwcolby <jwcolby at colbyconsulting.com
>>>> wrote:
>>>> Does anyone have experience with using compression in 2008?  I
>> understand
>>>> it is only available in
>>>> the enterprise edition?
>>>>
>>>> --
>>>> John W. Colby
>>>> www.ColbyConsulting.com
>>>> _______________________________________________
>>>> dba-SQLServer mailing list
>>>> dba-SQLServer at databaseadvisors.com
>>>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>>>> http://www.databaseadvisors.com
>>>>
>>>>
>>> _______________________________________________
>>> dba-SQLServer mailing list
>>> dba-SQLServer at databaseadvisors.com
>>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>>> http://www.databaseadvisors.com
>>>
>>>
>> _______________________________________________
>> dba-SQLServer mailing list
>> dba-SQLServer at databaseadvisors.com
>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>> http://www.databaseadvisors.com
>>
>>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From jwcolby at colbyconsulting.com  Fri Sep 11 12:43:47 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Fri, 11 Sep 2009 13:43:47 -0400
Subject: [dba-SQLServer] SQL Server 2008 compression
In-Reply-To: <b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>
References: <4AAA590E.9060200@colbyconsulting.com>
	<b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>
Message-ID: <4AAA8C53.7020703@colbyconsulting.com>

How do you tell once installed, whether you have the X32 or X64 SQL Server installed?  I assume it 
must be X64 because I can give it 12 gigs of ram on my X64 2003 server.

John W. Colby
www.ColbyConsulting.com


Francisco Tapia wrote:
> Compression backups?
> one thing to remember is that if you run a backup to a backup device you
> can't have an uncompressed backup with a compressed backup. as far as cost
> it might be cheaper to invest in redgate's backup software wich is fast and
> robust we use it here to backup our 2TB databases. for 295 you really can't
> beat the price on other solutions... its definitely cheaper than going 2008
> enterprise.
> 
> 
> more info here: http://msdn.microsoft.com/en-us/library/bb964719.aspx
> http://msdn.microsoft.com/en-us/library/ms186865.aspx
> 
> alternitavely you may wish to use a free compression product such as:
> http://www.idera.com/Products/Free-Tools/SQL-safe-Freeware-Edition/
> 
> -Francisco
> http://sqlthis.blogspot.com | Tsql and More...
> 
> 
> On Fri, Sep 11, 2009 at 7:05 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:
> 
>> Does anyone have experience with using compression in 2008?  I understand
>> it is only available in
>> the enterprise edition?
>>
>> --
>> John W. Colby
>> www.ColbyConsulting.com
>> _______________________________________________
>> dba-SQLServer mailing list
>> dba-SQLServer at databaseadvisors.com
>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>> http://www.databaseadvisors.com
>>
>>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From fhtapia at gmail.com  Fri Sep 11 12:49:16 2009
From: fhtapia at gmail.com (Francisco Tapia)
Date: Fri, 11 Sep 2009 10:49:16 -0700
Subject: [dba-SQLServer] SQL Server 2008 compression
In-Reply-To: <4AAA8C53.7020703@colbyconsulting.com>
References: <4AAA590E.9060200@colbyconsulting.com>
	<b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>
	<4AAA8C53.7020703@colbyconsulting.com>
Message-ID: <b874372a0909111049w5724dac3r84f7c41df2ee5b8f@mail.gmail.com>

SELECT @@VERSION yeilds
Microsoft SQL Server 2005 - 9.00.4035.00 (X64)   Nov 24 2008 16:17:31
Copyright (c) 1988-2005 Microsoft Corporation  Enterprise Edition (64-bit)
on Windows NT 5.2 (Build 3790: Service Pack 2)

-Francisco
http://sqlthis.blogspot.com | Tsql and More...


On Fri, Sep 11, 2009 at 10:43 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:

> How do you tell once installed, whether you have the X32 or X64 SQL Server
> installed?  I assume it
> must be X64 because I can give it 12 gigs of ram on my X64 2003 server.
>
> John W. Colby
> www.ColbyConsulting.com
>
>
> Francisco Tapia wrote:
> > Compression backups?
> > one thing to remember is that if you run a backup to a backup device you
> > can't have an uncompressed backup with a compressed backup. as far as
> cost
> > it might be cheaper to invest in redgate's backup software wich is fast
> and
> > robust we use it here to backup our 2TB databases. for 295 you really
> can't
> > beat the price on other solutions... its definitely cheaper than going
> 2008
> > enterprise.
> >
> >
> > more info here: http://msdn.microsoft.com/en-us/library/bb964719.aspx
> > http://msdn.microsoft.com/en-us/library/ms186865.aspx
> >
> > alternitavely you may wish to use a free compression product such as:
> > http://www.idera.com/Products/Free-Tools/SQL-safe-Freeware-Edition/
> >
> > -Francisco
> > http://sqlthis.blogspot.com | Tsql and More...
> >
> >
> > On Fri, Sep 11, 2009 at 7:05 AM, jwcolby <jwcolby at colbyconsulting.com
> >wrote:
> >
> >> Does anyone have experience with using compression in 2008?  I
> understand
> >> it is only available in
> >> the enterprise edition?
> >>
> >> --
> >> John W. Colby
> >> www.ColbyConsulting.com
> >> _______________________________________________
> >> dba-SQLServer mailing list
> >> dba-SQLServer at databaseadvisors.com
> >> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> >> http://www.databaseadvisors.com
> >>
> >>
> > _______________________________________________
> > dba-SQLServer mailing list
> > dba-SQLServer at databaseadvisors.com
> > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> > http://www.databaseadvisors.com
> >
> >
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>
>


From fhtapia at gmail.com  Fri Sep 11 12:53:49 2009
From: fhtapia at gmail.com (Francisco Tapia)
Date: Fri, 11 Sep 2009 10:53:49 -0700
Subject: [dba-SQLServer] SQL Server 2008 compression
In-Reply-To: <4AAA8AB4.6070705@colbyconsulting.com>
References: <4AAA590E.9060200@colbyconsulting.com>
	<b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>
	<29f585dd0909110910gf97d647q8505269bd080ca65@mail.gmail.com>
	<b874372a0909111030h3933b921q1c881a90e36013fc@mail.gmail.com>
	<4AAA8AB4.6070705@colbyconsulting.com>
Message-ID: <b874372a0909111053u189f3bc2x2b80efc6aa5c1372@mail.gmail.com>

ya can't! literally, this thing is engineered back in the days when database
engines couldn't support this many tables and there are literally tables
within tables using blobs or other techniques, so there are actually more
than 70k tables, but only slightly over 70k physical ones.

going back to your original post, I didn't realize you ment DATA compression
vs Backup compression.  You will want plenty of cores of course since there
will be some overhead on CPU usage.  but because there is less data being
handled you will see some speed increases especially in a redundant database
like yours, it consumes less I/O so you can push more data through it.  It
seems that you would benefit most from what they call ROW compression;

http://blogs.msdn.com/sqlserverstorageengine/archive/2007/11/12/types-of-data-compression-in-sql-server-2008.aspx

-Francisco
http://sqlthis.blogspot.com | Tsql and More...


On Fri, Sep 11, 2009 at 10:36 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:

> SEVENTY THOUSAND TABLES?
>
> 8(
>
> How could anyone possibly know what is in there?
>
> John W. Colby
> www.ColbyConsulting.com
>
>
> Francisco Tapia wrote:
> > at the risk of sounding like a salesman, I couldn't agree more, the
> copyto
> > feature has been in the product since sql backup 5, but also one of the
> cool
> > new features that I like is the network resilience option, so if your
> backup
> > fails you can choose how many retries it does and how long you want the
> > process to pause for between re-tries.  very nice stuff indeed.
> >
> > the tightest compression of course comes at a price and that is it takes
> a
> > lot of cpu and memory resources so it's good to run when the system is
> idle,
> > but if you have a high production system I always schedule log backups to
> > occur with the highest speed instead, that way a log backup does not tie
> up
> > resources for too long.
> >
> > I haven't had a chance to test sql prompt 4 have you?  from what i've
> been
> > reading it's good stuff and faster than sql prompt3 which I do use and
> love
> > to use.
> >
> > recently i moved away from Idera's Diagnostic Manager (an expensive
> > monitoring tool) and started using RedGates new Sql Response.  It
> actually
> > works better for me especially because I can keep track of my fragmented
> > tables in our SAP systems that literally have over 70k tables you want to
> > talk a database from hell? look no further.. this thing is a beast.
> >
> > -Francisco
> > http://sqlthis.blogspot.com | Tsql and More...
> >
> >
> > On Fri, Sep 11, 2009 at 9:10 AM, Arthur Fuller <fuller.artful at gmail.com
> >wrote:
> >
> >> I couldn't agree more strongly, Francisco! In fact, I have persuaded
> >> several
> >> clients to purchase the Red Gate Toolbelt simply by demo'ing SQL Backup
> >> from
> >> my laptop, and illustrating that backups take 1/3 the time and occupy
> 1/3
> >> the space. And now with the latest version, the space is down to 1/4.
> >> Awesome technology. I especially love the feature that lets you copy the
> >> backup to another machine automatically. That is beautiful!
> >> Arthur
> >>
> >> On Fri, Sep 11, 2009 at 11:38 AM, Francisco Tapia <fhtapia at gmail.com>
> >> wrote:
> >>
> >>> Compression backups?
> >>> one thing to remember is that if you run a backup to a backup device
> you
> >>> can't have an uncompressed backup with a compressed backup. as far as
> >> cost
> >>> it might be cheaper to invest in redgate's backup software wich is fast
> >> and
> >>> robust we use it here to backup our 2TB databases. for 295 you really
> >> can't
> >>> beat the price on other solutions... its definitely cheaper than going
> >> 2008
> >>> enterprise.
> >>>
> >>>
> >>> more info here: http://msdn.microsoft.com/en-us/library/bb964719.aspx
> >>> http://msdn.microsoft.com/en-us/library/ms186865.aspx
> >>>
> >>> alternitavely you may wish to use a free compression product such as:
> >>> http://www.idera.com/Products/Free-Tools/SQL-safe-Freeware-Edition/
> >>>
> >>> -Francisco
> >>> http://sqlthis.blogspot.com | Tsql and More...
> >>>
> >>>
> >>> On Fri, Sep 11, 2009 at 7:05 AM, jwcolby <jwcolby at colbyconsulting.com
> >>>> wrote:
> >>>> Does anyone have experience with using compression in 2008?  I
> >> understand
> >>>> it is only available in
> >>>> the enterprise edition?
> >>>>
> >>>> --
> >>>> John W. Colby
> >>>> www.ColbyConsulting.com
> >>>> _______________________________________________
> >>>> dba-SQLServer mailing list
> >>>> dba-SQLServer at databaseadvisors.com
> >>>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> >>>> http://www.databaseadvisors.com
> >>>>
> >>>>
> >>> _______________________________________________
> >>> dba-SQLServer mailing list
> >>> dba-SQLServer at databaseadvisors.com
> >>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> >>> http://www.databaseadvisors.com
> >>>
> >>>
> >> _______________________________________________
> >> dba-SQLServer mailing list
> >> dba-SQLServer at databaseadvisors.com
> >> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> >> http://www.databaseadvisors.com
> >>
> >>
> > _______________________________________________
> > dba-SQLServer mailing list
> > dba-SQLServer at databaseadvisors.com
> > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> > http://www.databaseadvisors.com
> >
> >
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>
>


From jwcolby at colbyconsulting.com  Fri Sep 11 13:29:04 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Fri, 11 Sep 2009 14:29:04 -0400
Subject: [dba-SQLServer] SQL Server 2008 compression
In-Reply-To: <b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>
References: <4AAA590E.9060200@colbyconsulting.com>
	<b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>
Message-ID: <4AAA96F0.7070709@colbyconsulting.com>

Francisco,

I wrote a stored procedure to house the script for the Idera backup TSQL.  Ran it against one of my 
order databases.  The database was 2.972 gb and the backup was 1.165 gb which is just under 40% of 
the original file size.  This will help immensly!  I have databases that are 190 gigs, 170 gigs, 130 
gigs etc.

Getting them down to 40% will allow me to store them on an "external" terabyte drive.

I will now build the restore script and try running it against my other server to see if I can 
restore the db over on the second machine.

Good stuff!  Thanks for the pointer to the free software, it seems to work great.  This will relieve 
the immense pressure of having huge databases and not having a plan for backups.  Depending on a 
raid array (which I understand is NOT a backup plan) only gets you so far!

John W. Colby
www.ColbyConsulting.com


Francisco Tapia wrote:
> Compression backups?
> one thing to remember is that if you run a backup to a backup device you
> can't have an uncompressed backup with a compressed backup. as far as cost
> it might be cheaper to invest in redgate's backup software wich is fast and
> robust we use it here to backup our 2TB databases. for 295 you really can't
> beat the price on other solutions... its definitely cheaper than going 2008
> enterprise.
> 
> 
> more info here: http://msdn.microsoft.com/en-us/library/bb964719.aspx
> http://msdn.microsoft.com/en-us/library/ms186865.aspx
> 
> alternitavely you may wish to use a free compression product such as:
> http://www.idera.com/Products/Free-Tools/SQL-safe-Freeware-Edition/
> 
> -Francisco
> http://sqlthis.blogspot.com | Tsql and More...
> 
> 
> On Fri, Sep 11, 2009 at 7:05 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:
> 
>> Does anyone have experience with using compression in 2008?  I understand
>> it is only available in
>> the enterprise edition?
>>
>> --
>> John W. Colby
>> www.ColbyConsulting.com
>> _______________________________________________
>> dba-SQLServer mailing list
>> dba-SQLServer at databaseadvisors.com
>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>> http://www.databaseadvisors.com
>>
>>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From fuller.artful at gmail.com  Fri Sep 11 17:25:18 2009
From: fuller.artful at gmail.com (Arthur Fuller)
Date: Fri, 11 Sep 2009 18:25:18 -0400
Subject: [dba-SQLServer] SQL Server 2008 compression
In-Reply-To: <4AAA96F0.7070709@colbyconsulting.com>
References: <4AAA590E.9060200@colbyconsulting.com>
	<b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>
	<4AAA96F0.7070709@colbyconsulting.com>
Message-ID: <29f585dd0909111525o258cb875t957fb456076a4c81@mail.gmail.com>

>From my own experiments, I don't think anything compares with RedGate's SQL
Backup, speed-wise or storage-wise. It's at least twice as fast as the
built-in stuff and it takes 1/3 the space required, and it provides the
option to copy said backup to any network drive. Works for me!
A.


From jwcolby at colbyconsulting.com  Fri Sep 11 21:38:17 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Fri, 11 Sep 2009 22:38:17 -0400
Subject: [dba-SQLServer] SQL Server 2008 compression
In-Reply-To: <29f585dd0909111525o258cb875t957fb456076a4c81@mail.gmail.com>
References: <4AAA590E.9060200@colbyconsulting.com>	<b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>	<4AAA96F0.7070709@colbyconsulting.com>
	<29f585dd0909111525o258cb875t957fb456076a4c81@mail.gmail.com>
Message-ID: <4AAB0999.60300@colbyconsulting.com>

Yep, I would but...

It is $800 and I pay for all my own stuff.  I can eventually charge it through but it takes awhile.

In the meantime I got the free Idera stuff working for now.  It is really fast, 80-100 mbytes / 
second and compresses down to anywhere from 25% to 40% of the original size.

It took me about 1/2 hour to take the existing TSQL script that they provide and wrap it into a 
stored procedure which basically passes in a database to back up, the path and the type and off it goes.

That would give me the "gui" and allow me to automate the whole shooting match.

I'm happy with what I have right now.

John W. Colby
www.ColbyConsulting.com


Arthur Fuller wrote:
>>From my own experiments, I don't think anything compares with RedGate's SQL
> Backup, speed-wise or storage-wise. It's at least twice as fast as the
> built-in stuff and it takes 1/3 the space required, and it provides the
> option to copy said backup to any network drive. Works for me!
> A.
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From jwcolby at colbyconsulting.com  Sat Sep 12 08:42:49 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Sat, 12 Sep 2009 09:42:49 -0400
Subject: [dba-SQLServer] Idera Free backup
Message-ID: <4AABA559.1010409@colbyconsulting.com>

I wrote an SP wrapper to the backup script provided with the Idera Free backup.  I then wrote an SP 
wrapper to call my backup SP feeding in the names of specific databases to backup.  Good enough for 
now.

This thing is pretty darned fast!  I get anywhere from 80 mbytes / sec up to 160 mbytes / second. 
Not sure what causes the rather radical spread.  What I can say is that I backed up almost a 
terabyte of raw file size databases down to 234 gigs of backup file size, one file per database 
backed up.  That was a full backup.  I didn't really track the full time but if you assume 100 megs 
/ second that would be around 10,000 seconds or about  166.66 minutes,  or 2.75 hours.

I was happy with that time.

To answer the next question, I have not done a restore yet.  ATM these are on a raid array right on 
the server being backed up.  I will be moving these off to a 1tb drive which will allow me to move 
it easily to the other server.  I will have to install the free backup on the other server, and then 
I will try to restore the databases.  I'll post again when that is done.

-- 
John W. Colby
www.ColbyConsulting.com


From fhtapia at gmail.com  Sat Sep 12 12:33:32 2009
From: fhtapia at gmail.com (Francisco Tapia)
Date: Sat, 12 Sep 2009 10:33:32 -0700
Subject: [dba-SQLServer] SQL Server 2008 compression
In-Reply-To: <4AAB0999.60300@colbyconsulting.com>
References: <4AAA590E.9060200@colbyconsulting.com>
	<b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>
	<4AAA96F0.7070709@colbyconsulting.com>
	<29f585dd0909111525o258cb875t957fb456076a4c81@mail.gmail.com>
	<4AAB0999.60300@colbyconsulting.com>
Message-ID: <b874372a0909121033v315309cax422d001d8e8f2b3d@mail.gmail.com>

In the end when free works you can't really compete,

  but just as Arthur said, as far as enterprise level reliability and
performance, its difficult to beat out redgate, they price their
products very low compared to the other guys so they are friendly on
the pocketbook, which is something my boss likes. In fact as an fyi I
bought a ton of licenses recently it costs me less to allow my license
for iders diagnostic manager (same # of licenses) to expire and buy
redgate backup 6 with sql response.  It would have been more expensive
to renew idera by a noticeable margin. So cost they really compete,
but then I have many sql servers that I manag and I know that can help
with price negotiation





On 9/11/09, jwcolby <jwcolby at colbyconsulting.com> wrote:
> Yep, I would but...
>
> It is $800 and I pay for all my own stuff.  I can eventually charge it
> through but it takes awhile.
>
> In the meantime I got the free Idera stuff working for now.  It is really
> fast, 80-100 mbytes /
> second and compresses down to anywhere from 25% to 40% of the original size.
>
> It took me about 1/2 hour to take the existing TSQL script that they provide
> and wrap it into a
> stored procedure which basically passes in a database to back up, the path
> and the type and off it goes.
>
> That would give me the "gui" and allow me to automate the whole shooting
> match.
>
> I'm happy with what I have right now.
>
> John W. Colby
> www.ColbyConsulting.com
>
>
> Arthur Fuller wrote:
>>>From my own experiments, I don't think anything compares with RedGate's
>> SQL
>> Backup, speed-wise or storage-wise. It's at least twice as fast as the
>> built-in stuff and it takes 1/3 the space required, and it provides the
>> option to copy said backup to any network drive. Works for me!
>> A.
>> _______________________________________________
>> dba-SQLServer mailing list
>> dba-SQLServer at databaseadvisors.com
>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>> http://www.databaseadvisors.com
>>
>>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>
>

-- 
Sent from Gmail for mobile | mobile.google.com

-Francisco
http://sqlthis.blogspot.com | Tsql and More...


From fuller.artful at gmail.com  Sat Sep 12 13:15:23 2009
From: fuller.artful at gmail.com (Arthur Fuller)
Date: Sat, 12 Sep 2009 14:15:23 -0400
Subject: [dba-SQLServer] SQL Server 2008 compression
In-Reply-To: <b874372a0909121033v315309cax422d001d8e8f2b3d@mail.gmail.com>
References: <4AAA590E.9060200@colbyconsulting.com>
	<b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>
	<4AAA96F0.7070709@colbyconsulting.com>
	<29f585dd0909111525o258cb875t957fb456076a4c81@mail.gmail.com>
	<4AAB0999.60300@colbyconsulting.com>
	<b874372a0909121033v315309cax422d001d8e8f2b3d@mail.gmail.com>
Message-ID: <29f585dd0909121115x7217fcd2l992676b538279b0@mail.gmail.com>

I have not a single bad word to say about RedGate. Their stuff is
spectacularly cool and from time to time they publish my writings. And not
least, as a FORG (Friends of Red Gate) I get this wonderful Christmas
package consisting of various cheeses and crackers etc. Totally cool.
A.


From jwcolby at colbyconsulting.com  Sat Sep 12 15:35:49 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Sat, 12 Sep 2009 16:35:49 -0400
Subject: [dba-SQLServer] SQL Server 2008 compression
In-Reply-To: <b874372a0909121033v315309cax422d001d8e8f2b3d@mail.gmail.com>
References: <4AAA590E.9060200@colbyconsulting.com>	<b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>	<4AAA96F0.7070709@colbyconsulting.com>	<29f585dd0909111525o258cb875t957fb456076a4c81@mail.gmail.com>	<4AAB0999.60300@colbyconsulting.com>
	<b874372a0909121033v315309cax422d001d8e8f2b3d@mail.gmail.com>
Message-ID: <4AAC0625.7040706@colbyconsulting.com>

Idera is indeed much more than redgate.

John W. Colby
www.ColbyConsulting.com


Francisco Tapia wrote:
> In the end when free works you can't really compete,
> 
>   but just as Arthur said, as far as enterprise level reliability and
> performance, its difficult to beat out redgate, they price their
> products very low compared to the other guys so they are friendly on
> the pocketbook, which is something my boss likes. In fact as an fyi I
> bought a ton of licenses recently it costs me less to allow my license
> for iders diagnostic manager (same # of licenses) to expire and buy
> redgate backup 6 with sql response.  It would have been more expensive
> to renew idera by a noticeable margin. So cost they really compete,
> but then I have many sql servers that I manag and I know that can help
> with price negotiation
> 
> 
> 
> 
> 
> On 9/11/09, jwcolby <jwcolby at colbyconsulting.com> wrote:
>> Yep, I would but...
>>
>> It is $800 and I pay for all my own stuff.  I can eventually charge it
>> through but it takes awhile.
>>
>> In the meantime I got the free Idera stuff working for now.  It is really
>> fast, 80-100 mbytes /
>> second and compresses down to anywhere from 25% to 40% of the original size.
>>
>> It took me about 1/2 hour to take the existing TSQL script that they provide
>> and wrap it into a
>> stored procedure which basically passes in a database to back up, the path
>> and the type and off it goes.
>>
>> That would give me the "gui" and allow me to automate the whole shooting
>> match.
>>
>> I'm happy with what I have right now.
>>
>> John W. Colby
>> www.ColbyConsulting.com
>>
>>
>> Arthur Fuller wrote:
>>> >From my own experiments, I don't think anything compares with RedGate's
>>> SQL
>>> Backup, speed-wise or storage-wise. It's at least twice as fast as the
>>> built-in stuff and it takes 1/3 the space required, and it provides the
>>> option to copy said backup to any network drive. Works for me!
>>> A.
>>> _______________________________________________
>>> dba-SQLServer mailing list
>>> dba-SQLServer at databaseadvisors.com
>>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>>> http://www.databaseadvisors.com
>>>
>>>
>> _______________________________________________
>> dba-SQLServer mailing list
>> dba-SQLServer at databaseadvisors.com
>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>> http://www.databaseadvisors.com
>>
>>
> 


From fhtapia at gmail.com  Sat Sep 12 20:31:13 2009
From: fhtapia at gmail.com (Francisco Tapia)
Date: Sat, 12 Sep 2009 18:31:13 -0700
Subject: [dba-SQLServer] SQL Server 2008 compression
In-Reply-To: <29f585dd0909121115x7217fcd2l992676b538279b0@mail.gmail.com>
References: <4AAA590E.9060200@colbyconsulting.com>
	<b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>
	<4AAA96F0.7070709@colbyconsulting.com>
	<29f585dd0909111525o258cb875t957fb456076a4c81@mail.gmail.com>
	<4AAB0999.60300@colbyconsulting.com>
	<b874372a0909121033v315309cax422d001d8e8f2b3d@mail.gmail.com>
	<29f585dd0909121115x7217fcd2l992676b538279b0@mail.gmail.com>
Message-ID: <b874372a0909121831h57eb2932m1fe11f03f8890796@mail.gmail.com>

That sounds cool how does one become a friend of redgate, I have drank
plenty of coolaid.



On 9/12/09, Arthur Fuller <fuller.artful at gmail.com> wrote:
> I have not a single bad word to say about RedGate. Their stuff is
> spectacularly cool and from time to time they publish my writings. And not
> least, as a FORG (Friends of Red Gate) I get this wonderful Christmas
> package consisting of various cheeses and crackers etc. Totally cool.
> A.
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>
>

-- 
Sent from Gmail for mobile | mobile.google.com

-Francisco
http://sqlthis.blogspot.com | Tsql and More...


From fuller.artful at gmail.com  Sat Sep 12 23:31:26 2009
From: fuller.artful at gmail.com (Arthur Fuller)
Date: Sun, 13 Sep 2009 00:31:26 -0400
Subject: [dba-SQLServer] SQL Server 2008 compression
In-Reply-To: <b874372a0909121831h57eb2932m1fe11f03f8890796@mail.gmail.com>
References: <4AAA590E.9060200@colbyconsulting.com>
	<b874372a0909110838u571fd78dj68f864c18886e275@mail.gmail.com>
	<4AAA96F0.7070709@colbyconsulting.com>
	<29f585dd0909111525o258cb875t957fb456076a4c81@mail.gmail.com>
	<4AAB0999.60300@colbyconsulting.com>
	<b874372a0909121033v315309cax422d001d8e8f2b3d@mail.gmail.com>
	<29f585dd0909121115x7217fcd2l992676b538279b0@mail.gmail.com>
	<b874372a0909121831h57eb2932m1fe11f03f8890796@mail.gmail.com>
Message-ID: <29f585dd0909122131x2cba2779y903cd52e3d0f3c8f@mail.gmail.com>

In my case, it happened because I sold several clients on their software and
also wrote some stuff for Simple-Talk. I don't know what path you might
follow to get there, but I do know that you have all the credentials. If it
may help in some way vis-a-vis your application, count me in as a reference.
You have helped me surmount numerous problems over the years.
Arthur

On Sat, Sep 12, 2009 at 9:31 PM, Francisco Tapia <fhtapia at gmail.com> wrote:

> That sounds cool how does one become a friend of redgate, I have drank
> plenty of coolaid.
>
>


From jwcolby at colbyconsulting.com  Mon Sep 14 08:07:44 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Mon, 14 Sep 2009 09:07:44 -0400
Subject: [dba-SQLServer] Idera Free - Restore
Message-ID: <4AAE4020.9010208@colbyconsulting.com>

I installed the Idera Free on my other SQL Server machine.  The one thing I had to do to get the 
restores to happen was to recreate the same database file path as on the original machine.  Once I 
did that the backup restored flawlessly on my other server.  Had I needed to do so, there is a "With 
Move" parameter that you can provide to move the database to the location of your choice.  I did not 
discover that until I had already recreated the original file location on the new server.

The speed to do so was 247 mbytes / second for this one specific database.  It restored the database 
to the logical name MyDatabase.Safe where MyDatabase is the name of the db being restored.

For free stuff this is pretty awesome.

Thanks again Francisco for pointing me to this.  It definitely fits my needs for now.

-- 
John W. Colby
www.ColbyConsulting.com


From fhtapia at gmail.com  Mon Sep 14 11:29:07 2009
From: fhtapia at gmail.com (Francisco Tapia)
Date: Mon, 14 Sep 2009 09:29:07 -0700
Subject: [dba-SQLServer] Idera Free - Restore
In-Reply-To: <4AAE4020.9010208@colbyconsulting.com>
References: <4AAE4020.9010208@colbyconsulting.com>
Message-ID: <b874372a0909140929p31fca247w8b976405ec123a5e@mail.gmail.com>

no problem, in the original code i sent you I used the WITH MOVE parameter,
almost all backup utilities will support this, and if they don't, they are
not worth it ;)


-Francisco
http://sqlthis.blogspot.com | Tsql and More...


On Mon, Sep 14, 2009 at 6:07 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:

> I installed the Idera Free on my other SQL Server machine.  The one thing I
> had to do to get the
> restores to happen was to recreate the same database file path as on the
> original machine.  Once I
> did that the backup restored flawlessly on my other server.  Had I needed
> to do so, there is a "With
> Move" parameter that you can provide to move the database to the location
> of your choice.  I did not
> discover that until I had already recreated the original file location on
> the new server.
>
> The speed to do so was 247 mbytes / second for this one specific database.
>  It restored the database
> to the logical name MyDatabase.Safe where MyDatabase is the name of the db
> being restored.
>
> For free stuff this is pretty awesome.
>
> Thanks again Francisco for pointing me to this.  It definitely fits my
> needs for now.
>
> --
> John W. Colby
> www.ColbyConsulting.com
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>
>


From jwcolby at colbyconsulting.com  Mon Sep 14 13:21:31 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Mon, 14 Sep 2009 14:21:31 -0400
Subject: [dba-SQLServer] is restore identical
Message-ID: <4AAE89AB.8080407@colbyconsulting.com>

When you use a utility to backup / restore, is the restore identical to the backup, byte for byte? 
For example would the restore get rid of the "available free space" or would it just restore that 
free space?

-- 
John W. Colby
www.ColbyConsulting.com


From fhtapia at gmail.com  Mon Sep 14 21:54:38 2009
From: fhtapia at gmail.com (Francisco Tapia)
Date: Mon, 14 Sep 2009 19:54:38 -0700
Subject: [dba-SQLServer] is restore identical
In-Reply-To: <4AAE89AB.8080407@colbyconsulting.com>
References: <4AAE89AB.8080407@colbyconsulting.com>
Message-ID: <775934BD-BC63-42BF-98A4-8BB8FDEA0C3F@gmail.com>

it's been my experience that if you restore a database with for  
example 1gb of free extra space, the restore will give you that same  
free space, so if you don't have that much free space in the  
destination drive, then you will want to shrink the data files before  
you try your backup.


On Sep 14, 2009, at 11:21 AM, jwcolby wrote:

> When you use a utility to backup / restore, is the restore identical  
> to the backup, byte for byte?
> For example would the restore get rid of the "available free space"  
> or would it just restore that
> free space?
>
> -- 
> John W. Colby
> www.ColbyConsulting.com
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>



From fuller.artful at gmail.com  Fri Sep 18 00:11:35 2009
From: fuller.artful at gmail.com (Arthur Fuller)
Date: Fri, 18 Sep 2009 01:11:35 -0400
Subject: [dba-SQLServer] Suspect database
Message-ID: <29f585dd0909172211t5c860dddl8f8d9460e2dab193@mail.gmail.com>

What does it mean when a database is identified in SSMS as suspect? One
thing it means is that I can't open it. What is not clear is why the db was
identified as suspect. Bad backup/restore? I have no idea.
Arthur


From jwcolby at colbyconsulting.com  Fri Sep 18 06:50:23 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Fri, 18 Sep 2009 07:50:23 -0400
Subject: [dba-SQLServer] Suspect database
In-Reply-To: <29f585dd0909172211t5c860dddl8f8d9460e2dab193@mail.gmail.com>
References: <29f585dd0909172211t5c860dddl8f8d9460e2dab193@mail.gmail.com>
Message-ID: <4AB373FF.6020101@colbyconsulting.com>

I have had this happen several times.  In my case it usually means that the server lost power as it 
was writing to a table.  In most cases that table was lost but the others were still there.

John W. Colby
www.ColbyConsulting.com


Arthur Fuller wrote:
> What does it mean when a database is identified in SSMS as suspect? One
> thing it means is that I can't open it. What is not clear is why the db was
> identified as suspect. Bad backup/restore? I have no idea.
> Arthur
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From fuller.artful at gmail.com  Fri Sep 18 07:27:25 2009
From: fuller.artful at gmail.com (Arthur Fuller)
Date: Fri, 18 Sep 2009 08:27:25 -0400
Subject: [dba-SQLServer] Suspect database
In-Reply-To: <4AB373FF.6020101@colbyconsulting.com>
References: <29f585dd0909172211t5c860dddl8f8d9460e2dab193@mail.gmail.com>
	<4AB373FF.6020101@colbyconsulting.com>
Message-ID: <29f585dd0909180527p24bf0de9k5b0db0969fe5a3b9@mail.gmail.com>

I must have a more serious problem, since I can't even open the database(s)
in question. Fortunately it happened only to two relatively unimportant
databases.
Thanks,
Arthur

On Fri, Sep 18, 2009 at 7:50 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:

> I have had this happen several times.  In my case it usually means that the
> server lost power as it
> was writing to a table.  In most cases that table was lost but the others
> were still there.
>
> John W. Colby
> www.ColbyConsulting.com
>
>


From jwcolby at colbyconsulting.com  Fri Sep 18 07:38:55 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Fri, 18 Sep 2009 08:38:55 -0400
Subject: [dba-SQLServer] Suspect database
In-Reply-To: <29f585dd0909180527p24bf0de9k5b0db0969fe5a3b9@mail.gmail.com>
References: <29f585dd0909172211t5c860dddl8f8d9460e2dab193@mail.gmail.com>	<4AB373FF.6020101@colbyconsulting.com>
	<29f585dd0909180527p24bf0de9k5b0db0969fe5a3b9@mail.gmail.com>
Message-ID: <4AB37F5F.7000607@colbyconsulting.com>

You can't open them in that state.  I searched Google and found code to "restore them".  Make sure 
you copy the file before doing so as the restore "fixes things", and the fix can in some cases trash 
the database.


John W. Colby
www.ColbyConsulting.com


Arthur Fuller wrote:
> I must have a more serious problem, since I can't even open the database(s)
> in question. Fortunately it happened only to two relatively unimportant
> databases.
> Thanks,
> Arthur
> 
> On Fri, Sep 18, 2009 at 7:50 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:
> 
>> I have had this happen several times.  In my case it usually means that the
>> server lost power as it
>> was writing to a table.  In most cases that table was lost but the others
>> were still there.
>>
>> John W. Colby
>> www.ColbyConsulting.com
>>
>>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From stuart at lexacorp.com.pg  Fri Sep 18 07:41:36 2009
From: stuart at lexacorp.com.pg (Stuart McLachlan)
Date: Fri, 18 Sep 2009 22:41:36 +1000
Subject: [dba-SQLServer] Suspect database
In-Reply-To: <29f585dd0909180527p24bf0de9k5b0db0969fe5a3b9@mail.gmail.com>
References: <29f585dd0909172211t5c860dddl8f8d9460e2dab193@mail.gmail.com>,
	<4AB373FF.6020101@colbyconsulting.com>,
	<29f585dd0909180527p24bf0de9k5b0db0969fe5a3b9@mail.gmail.com>
Message-ID: <4AB38000.1436.70F6FB7@stuart.lexacorp.com.pg>

See

http://windowsitpro.com/article/articleid/14047/my-sql-server-database-has-been-marked-
quotsuspectquot---what-can-i-do.html

and

http://www.devx.com/vb2themax/Tip/18624

-- 
Stuart

On 18 Sep 2009 at 8:27, Arthur Fuller wrote:

> I must have a more serious problem, since I can't even open the database(s)
> in question. Fortunately it happened only to two relatively unimportant
> databases.
> Thanks,
> Arthur
> 
> On Fri, Sep 18, 2009 at 7:50 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:
> 
> > I have had this happen several times.  In my case it usually means that the
> > server lost power as it
> > was writing to a table.  In most cases that table was lost but the others
> > were still there.
> >
> > John W. Colby
> > www.ColbyConsulting.com
> >
> >
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 




From dwaters at usinternet.com  Fri Sep 18 11:30:14 2009
From: dwaters at usinternet.com (Dan Waters)
Date: Fri, 18 Sep 2009 11:30:14 -0500
Subject: [dba-SQLServer] Table Link Refresh Required After Table Design
	Change?
Message-ID: <630F4D937A334F7394FE62DBBE2FA08A@danwaters>

I just added a new field to a table in a SQL Server 2005 database.  The FE
is A2003.  The FE was closed at the time.
 
When I opened the FE, it errored out because it couldn't find the new field.
I opened the table link in the FE and the field didn't show!  Then in code,
I refreshed all the tables, and still the new field didn't show.
 
Then I went through relinking just that table, and now the new field does
show.
 
This does not happen in Access.  So, if I make a change to a table, do I
need to explicitly refresh the table link in the FE every time!
 
Thanks!
Dan


From robert at webedb.com  Fri Sep 18 12:05:46 2009
From: robert at webedb.com (Robert Stewart)
Date: Fri, 18 Sep 2009 12:05:46 -0500
Subject: [dba-SQLServer] Table Link Refresh Required After Table
In-Reply-To: <mailman.5.1253293201.26645.dba-sqlserver@databaseadvisors. com>
References: <mailman.5.1253293201.26645.dba-sqlserver@databaseadvisors.com>
Message-ID: <200909181706.n8IH66ZT027747@databaseadvisors.com>

Absolutely. MS Access caches the table information. You have to refresh it
ANY TIME a change is made to ANY table in the SQL side.

At 12:00 PM 9/18/2009, you wrote:
>Date: Fri, 18 Sep 2009 11:30:14 -0500
>From: "Dan Waters" <dwaters at usinternet.com>
>Subject: [dba-SQLServer] Table Link Refresh Required After Table
>         Design  Change?
>To: "'Discussion concerning MS SQL Server'"
>         <dba-sqlserver at databaseadvisors.com>
>Message-ID: <630F4D937A334F7394FE62DBBE2FA08A at danwaters>
>Content-Type: text/plain;       charset="us-ascii"
>
>I just added a new field to a table in a SQL Server 2005 database.  The FE
>is A2003.  The FE was closed at the time.
>
>When I opened the FE, it errored out because it couldn't find the new field.
>I opened the table link in the FE and the field didn't show!  Then in code,
>I refreshed all the tables, and still the new field didn't show.
>
>Then I went through relinking just that table, and now the new field does
>show.
>
>This does not happen in Access.  So, if I make a change to a table, do I
>need to explicitly refresh the table link in the FE every time!
>
>Thanks!
>Dan


From jwcolby at colbyconsulting.com  Fri Sep 18 12:52:00 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Fri, 18 Sep 2009 13:52:00 -0400
Subject: [dba-SQLServer] Null and DISTINCT
Message-ID: <4AB3C8C0.4060701@colbyconsulting.com>

I have a result set where I have a ton of records that are absolutely identical.  I am trying to do 
a DISTINCT but there is one field that is pulling a NULL.  Is it true that the NULL will prevent the 
DISTINCT from working in this case?

Any way around this?
-- 
John W. Colby
www.ColbyConsulting.com


From jwcolby at colbyconsulting.com  Fri Sep 18 13:03:00 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Fri, 18 Sep 2009 14:03:00 -0400
Subject: [dba-SQLServer] SPAM-LOW:   Null and DISTINCT
In-Reply-To: <4AB3C8C0.4060701@colbyconsulting.com>
References: <4AB3C8C0.4060701@colbyconsulting.com>
Message-ID: <4AB3CB54.6090806@colbyconsulting.com>

Never mind...   LOL.

ISNULL(MyFld,0).

John W. Colby
www.ColbyConsulting.com


jwcolby wrote:
> I have a result set where I have a ton of records that are absolutely identical.  I am trying to do 
> a DISTINCT but there is one field that is pulling a NULL.  Is it true that the NULL will prevent the 
> DISTINCT from working in this case?
> 
> Any way around this?


From davidmcafee at gmail.com  Sat Sep 19 19:24:12 2009
From: davidmcafee at gmail.com (David McAfee)
Date: Sat, 19 Sep 2009 17:24:12 -0700
Subject: [dba-SQLServer] Null and DISTINCT
In-Reply-To: <4AB3C8C0.4060701@colbyconsulting.com>
References: <4AB3C8C0.4060701@colbyconsulting.com>
Message-ID: <8786a4c00909191724v3230ae9bnad0545b35e3e2da0@mail.gmail.com>

use ISNULL(yourfieldwithanullvalue,'someValueHere') in the select
statment. It works like VBA's NZ function

On Fri, Sep 18, 2009 at 10:52 AM, jwcolby <jwcolby at colbyconsulting.com> wrote:
> I have a result set where I have a ton of records that are absolutely identical. ?I am trying to do
> a DISTINCT but there is one field that is pulling a NULL. ?Is it true that the NULL will prevent the
> DISTINCT from working in this case?
>
> Any way around this?
> --
> John W. Colby
> www.ColbyConsulting.com
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>
>



From jwcolby at colbyconsulting.com  Mon Sep 21 12:41:51 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Mon, 21 Sep 2009 13:41:51 -0400
Subject: [dba-SQLServer] Test for empty columns
Message-ID: <4AB7BADF.6010700@colbyconsulting.com>

My database from hell has at least some columns where the column simply contains no data in any 
record.  I am wondering how I would find and list these columns so that I can delete them from my table.

Would I write a stored procedure to get the columns from the database and then just iterate through 
that recordset building a dynamic query?  Is there already something out there that does this? 
Anybody doing this?

TIA,

-- 
John W. Colby
www.ColbyConsulting.com


From Elizabeth.J.Doering at wellsfargo.com  Mon Sep 21 13:07:34 2009
From: Elizabeth.J.Doering at wellsfargo.com (Elizabeth.J.Doering at wellsfargo.com)
Date: Mon, 21 Sep 2009 13:07:34 -0500
Subject: [dba-SQLServer] Test for empty columns
In-Reply-To: <4AB7BADF.6010700@colbyconsulting.com>
References: <4AB7BADF.6010700@colbyconsulting.com>
Message-ID: <4838EC790FF778449985FC3B8A47F87142B772F6EC@MSGCMSV21011.ent.wfb.bank.corp>

This sproc gets at all the columns in all the tables:

 
CREATE  procedure [dbo].[_DataDictionary]
--Create on the fly data dictionary.
as
-build your destination table if necessary
IF NOT EXISTS (Select name FROM sys.objects WHERE object_id = OBJECT_ID(N'[DataDictionary]') AND type in (N'U'))
		BEGIN
		CREATE TABLE [dbo].[DataDictionary](
			[table_qualifier] [varchar](50) NULL,
			[table_owner] [varchar](50) NULL,
			[table_name] [varchar](50) NOT NULL,
			[column_name] [varchar](50) NOT NULL,
			[data_type] [int] NULL,
			[type_name] [varchar](50) NULL,
			[precision] [varchar](50) NULL,
			[length] [varchar](50) NULL,
			[scale] [varchar](50) NULL,
			[radix] [varchar](50) NULL,
			[nullable] [varchar](50) NULL,
			[remarks] [varchar](50) NULL,
			[column_def] [varchar](50) NULL,
			[sql_data_type] [varchar](50) NULL,
			[sql_datetime_sub] [varchar](50) NULL,
			[char_octet_length] [varchar](50) NULL,
			[ordinal_position] [varchar](50) NULL,
			[is_nullable] [varchar](50) NULL,
			[ss_data_type] [varchar](50) NULL,
		 CONSTRAINT [PK_DataDictionary] PRIMARY KEY CLUSTERED 
		(
			[table_name] ASC,
			[column_name] ASC
		)WITH (IGNORE_DUP_KEY = OFF) ON [PRIMARY]
		) ON [PRIMARY]
		END
-but if the destination already exists, just empty it
truncate table datadictionary

-select all the object names in the database into a cursor 

DECLARE tnames_cursor CURSOR
FOR
  --2000 version: select   name, type from sysobjects where (type = 'v' or type = 'u') order by name 
  --2005 version: 
select   name, type from sys.all_objects where is_ms_shipped = 0 and  (type = 'v' or type = 'u') and name <> 'sysdiagrams' order by name 
OPEN tnames_cursor;
DECLARE @tablename varchar(100);
DECLARE @type varchar(5)

-loop through the cursor, pulling off all the column names into your table

FETCH NEXT FROM tnames_cursor INTO @tablename, @type;
WHILE (@@FETCH_STATUS <> -1)
BEGIN
   IF (@@FETCH_STATUS <> -2)
   BEGIN   
      SELECT @tablename = RTRIM(@tablename); 
	  INSERT INTO [dbo].[DataDictionary]
           ([table_qualifier]
           ,[table_owner]
           ,[table_name]
           ,[column_name]
           ,[data_type]
           ,[type_name]
           ,[precision]
           ,[length]
           ,[scale]
           ,[radix]
           ,[nullable]
           ,[remarks]
           ,[column_def]
           ,[sql_data_type]
           ,[sql_datetime_sub]
           ,[char_octet_length]
           ,[ordinal_position]
           ,[is_nullable]
           ,[ss_data_type]) exec sys.sp_columns @tablename
 
   END;
   FETCH NEXT FROM tnames_cursor INTO @tablename, @type;
END;
CLOSE tnames_cursor;
DEALLOCATE tnames_cursor; 






Thanks, 

Liz 


This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation.


-----Original Message-----
From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby
Sent: Monday, September 21, 2009 12:42 PM
To: Dba-Sqlserver
Subject: [dba-SQLServer] Test for empty columns

My database from hell has at least some columns where the column simply contains no data in any 
record.  I am wondering how I would find and list these columns so that I can delete them from my table.

Would I write a stored procedure to get the columns from the database and then just iterate through 
that recordset building a dynamic query?  Is there already something out there that does this? 
Anybody doing this?

TIA,

-- 
John W. Colby
www.ColbyConsulting.com
_______________________________________________
dba-SQLServer mailing list
dba-SQLServer at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
http://www.databaseadvisors.com




From Elizabeth.J.Doering at wellsfargo.com  Mon Sep 21 13:17:52 2009
From: Elizabeth.J.Doering at wellsfargo.com (Elizabeth.J.Doering at wellsfargo.com)
Date: Mon, 21 Sep 2009 13:17:52 -0500
Subject: [dba-SQLServer] Test for empty columns
In-Reply-To: <4838EC790FF778449985FC3B8A47F87142B772F6EC@MSGCMSV21011.ent.wfb.bank.corp>
References: <4AB7BADF.6010700@colbyconsulting.com>
	<4838EC790FF778449985FC3B8A47F87142B772F6EC@MSGCMSV21011.ent.wfb.bank.corp>
Message-ID: <4838EC790FF778449985FC3B8A47F87142B772F7C2@MSGCMSV21011.ent.wfb.bank.corp>

 
I see Outlook differentiated nicely between comments I pasted and comments I typed.  Watch for the difference between '-' and '--' .  Version below is corrected.

Thanks, 

Liz 
 

This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation.


-----Original Message-----
From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Elizabeth.J.Doering at wellsfargo.com
Sent: Monday, September 21, 2009 1:08 PM
To: dba-sqlserver at databaseadvisors.com
Subject: Re: [dba-SQLServer] Test for empty columns

This sproc gets at all the columns in all the tables:

 
CREATE  procedure [dbo].[_DataDictionary]
--Create on the fly data dictionary.
as
--build your destination table if necessary
IF NOT EXISTS (Select name FROM sys.objects WHERE object_id = OBJECT_ID(N'[DataDictionary]') AND type in (N'U'))
		BEGIN
		CREATE TABLE [dbo].[DataDictionary](
			[table_qualifier] [varchar](50) NULL,
			[table_owner] [varchar](50) NULL,
			[table_name] [varchar](50) NOT NULL,
			[column_name] [varchar](50) NOT NULL,
			[data_type] [int] NULL,
			[type_name] [varchar](50) NULL,
			[precision] [varchar](50) NULL,
			[length] [varchar](50) NULL,
			[scale] [varchar](50) NULL,
			[radix] [varchar](50) NULL,
			[nullable] [varchar](50) NULL,
			[remarks] [varchar](50) NULL,
			[column_def] [varchar](50) NULL,
			[sql_data_type] [varchar](50) NULL,
			[sql_datetime_sub] [varchar](50) NULL,
			[char_octet_length] [varchar](50) NULL,
			[ordinal_position] [varchar](50) NULL,
			[is_nullable] [varchar](50) NULL,
			[ss_data_type] [varchar](50) NULL,
		 CONSTRAINT [PK_DataDictionary] PRIMARY KEY CLUSTERED 
		(
			[table_name] ASC,
			[column_name] ASC
		)WITH (IGNORE_DUP_KEY = OFF) ON [PRIMARY]
		) ON [PRIMARY]
		END
--but if the destination already exists, just empty it
truncate table datadictionary

--select all the object names in the database into a cursor 

DECLARE tnames_cursor CURSOR
FOR
  --2000 version: select   name, type from sysobjects where (type = 'v' or type = 'u') order by name 
  --2005 version: 
select   name, type from sys.all_objects where is_ms_shipped = 0 and  (type = 'v' or type = 'u') and name <> 'sysdiagrams' order by name 
OPEN tnames_cursor;
DECLARE @tablename varchar(100);
DECLARE @type varchar(5)

--loop through the cursor, pulling off all the column names into your table

FETCH NEXT FROM tnames_cursor INTO @tablename, @type;
WHILE (@@FETCH_STATUS <> -1)
BEGIN
   IF (@@FETCH_STATUS <> -2)
   BEGIN   
      SELECT @tablename = RTRIM(@tablename); 
	  INSERT INTO [dbo].[DataDictionary]
           ([table_qualifier]
           ,[table_owner]
           ,[table_name]
           ,[column_name]
           ,[data_type]
           ,[type_name]
           ,[precision]
           ,[length]
           ,[scale]
           ,[radix]
           ,[nullable]
           ,[remarks]
           ,[column_def]
           ,[sql_data_type]
           ,[sql_datetime_sub]
           ,[char_octet_length]
           ,[ordinal_position]
           ,[is_nullable]
           ,[ss_data_type]) exec sys.sp_columns @tablename
 
   END;
   FETCH NEXT FROM tnames_cursor INTO @tablename, @type;
END;
CLOSE tnames_cursor;
DEALLOCATE tnames_cursor; 






Thanks, 

Liz 


This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation.


-----Original Message-----
From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby
Sent: Monday, September 21, 2009 12:42 PM
To: Dba-Sqlserver
Subject: [dba-SQLServer] Test for empty columns

My database from hell has at least some columns where the column simply contains no data in any 
record.  I am wondering how I would find and list these columns so that I can delete them from my table.

Would I write a stored procedure to get the columns from the database and then just iterate through 
that recordset building a dynamic query?  Is there already something out there that does this? 
Anybody doing this?

TIA,

-- 
John W. Colby
www.ColbyConsulting.com
_______________________________________________
dba-SQLServer mailing list
dba-SQLServer at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
http://www.databaseadvisors.com


_______________________________________________
dba-SQLServer mailing list
dba-SQLServer at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
http://www.databaseadvisors.com




From jwcolby at colbyconsulting.com  Mon Sep 21 14:18:05 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Mon, 21 Sep 2009 15:18:05 -0400
Subject: [dba-SQLServer] Test for empty columns
In-Reply-To: <4838EC790FF778449985FC3B8A47F87142B772F6EC@MSGCMSV21011.ent.wfb.bank.corp>
References: <4AB7BADF.6010700@colbyconsulting.com>
	<4838EC790FF778449985FC3B8A47F87142B772F6EC@MSGCMSV21011.ent.wfb.bank.corp>
Message-ID: <4AB7D16D.8060605@colbyconsulting.com>

Elizabeth,

That did indeed give me a list of all tables, and fields in the tables.

Now...

The goal is to select a specific table, iterate through all of the fields in that table, and 
discover whether each field has any data, even in just a single row.

The first table I will be doing this to contains 50 million records, and ~640 fields.  Obviously 
there are not indexes on each field, though there are on perhaps a hundred of the 640 fields.  I 
need to store the results such that when the process is finished I have a record of which fields are 
empty (have no data in any record).  This table is completely static to this point, i.e. I never 
update any of the fields.  Thus by dropping any empty fields I can potentially speed up all further 
operations on this table, with no discernible downside.

So is an acceptable strategy to simply create a count query on-the-fly and count() each column? 
Some faster way (remember there will not be an index for most fields).  Would it be faster to make 
an index for one field, count the field, then drop the index?

Or... does the system database somewhere contain this information already?  Which fields contain data.

John W. Colby
www.ColbyConsulting.com


Elizabeth.J.Doering at wellsfargo.com wrote:
> This sproc gets at all the columns in all the tables:
> 
>  
> CREATE  procedure [dbo].[_DataDictionary]
> --Create on the fly data dictionary.
> as
> -build your destination table if necessary
> IF NOT EXISTS (Select name FROM sys.objects WHERE object_id = OBJECT_ID(N'[DataDictionary]') AND type in (N'U'))
> 		BEGIN
> 		CREATE TABLE [dbo].[DataDictionary](
> 			[table_qualifier] [varchar](50) NULL,
> 			[table_owner] [varchar](50) NULL,
> 			[table_name] [varchar](50) NOT NULL,
> 			[column_name] [varchar](50) NOT NULL,
> 			[data_type] [int] NULL,
> 			[type_name] [varchar](50) NULL,
> 			[precision] [varchar](50) NULL,
> 			[length] [varchar](50) NULL,
> 			[scale] [varchar](50) NULL,
> 			[radix] [varchar](50) NULL,
> 			[nullable] [varchar](50) NULL,
> 			[remarks] [varchar](50) NULL,
> 			[column_def] [varchar](50) NULL,
> 			[sql_data_type] [varchar](50) NULL,
> 			[sql_datetime_sub] [varchar](50) NULL,
> 			[char_octet_length] [varchar](50) NULL,
> 			[ordinal_position] [varchar](50) NULL,
> 			[is_nullable] [varchar](50) NULL,
> 			[ss_data_type] [varchar](50) NULL,
> 		 CONSTRAINT [PK_DataDictionary] PRIMARY KEY CLUSTERED 
> 		(
> 			[table_name] ASC,
> 			[column_name] ASC
> 		)WITH (IGNORE_DUP_KEY = OFF) ON [PRIMARY]
> 		) ON [PRIMARY]
> 		END
> -but if the destination already exists, just empty it
> truncate table datadictionary
> 
> -select all the object names in the database into a cursor 
> 
> DECLARE tnames_cursor CURSOR
> FOR
>   --2000 version: select   name, type from sysobjects where (type = 'v' or type = 'u') order by name 
>   --2005 version: 
> select   name, type from sys.all_objects where is_ms_shipped = 0 and  (type = 'v' or type = 'u') and name <> 'sysdiagrams' order by name 
> OPEN tnames_cursor;
> DECLARE @tablename varchar(100);
> DECLARE @type varchar(5)
> 
> -loop through the cursor, pulling off all the column names into your table
> 
> FETCH NEXT FROM tnames_cursor INTO @tablename, @type;
> WHILE (@@FETCH_STATUS <> -1)
> BEGIN
>    IF (@@FETCH_STATUS <> -2)
>    BEGIN   
>       SELECT @tablename = RTRIM(@tablename); 
> 	  INSERT INTO [dbo].[DataDictionary]
>            ([table_qualifier]
>            ,[table_owner]
>            ,[table_name]
>            ,[column_name]
>            ,[data_type]
>            ,[type_name]
>            ,[precision]
>            ,[length]
>            ,[scale]
>            ,[radix]
>            ,[nullable]
>            ,[remarks]
>            ,[column_def]
>            ,[sql_data_type]
>            ,[sql_datetime_sub]
>            ,[char_octet_length]
>            ,[ordinal_position]
>            ,[is_nullable]
>            ,[ss_data_type]) exec sys.sp_columns @tablename
>  
>    END;
>    FETCH NEXT FROM tnames_cursor INTO @tablename, @type;
> END;
> CLOSE tnames_cursor;
> DEALLOCATE tnames_cursor; 
> 
> 
> 
> 
> 
> 
> Thanks, 
> 
> Liz 
> 
> 
> This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation.
> 
> 
> -----Original Message-----
> From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby
> Sent: Monday, September 21, 2009 12:42 PM
> To: Dba-Sqlserver
> Subject: [dba-SQLServer] Test for empty columns
> 
> My database from hell has at least some columns where the column simply contains no data in any 
> record.  I am wondering how I would find and list these columns so that I can delete them from my table.
> 
> Would I write a stored procedure to get the columns from the database and then just iterate through 
> that recordset building a dynamic query?  Is there already something out there that does this? 
> Anybody doing this?
> 
> TIA,
> 


From Elizabeth.J.Doering at wellsfargo.com  Mon Sep 21 14:55:53 2009
From: Elizabeth.J.Doering at wellsfargo.com (Elizabeth.J.Doering at wellsfargo.com)
Date: Mon, 21 Sep 2009 14:55:53 -0500
Subject: [dba-SQLServer] Test for empty columns
In-Reply-To: <4AB7D16D.8060605@colbyconsulting.com>
References: <4AB7BADF.6010700@colbyconsulting.com>
	<4838EC790FF778449985FC3B8A47F87142B772F6EC@MSGCMSV21011.ent.wfb.bank.corp>
	<4AB7D16D.8060605@colbyconsulting.com>
Message-ID: <4838EC790FF778449985FC3B8A47F87142B77300F2@MSGCMSV21011.ent.wfb.bank.corp>

John,

I thank God every day that I don't deal with any thing the magnitude of your db-f-h.

I would probably do the query on the fly and count(*) approach...but it would be all trial and error and I would probably be sorry on account of at least some of the trials.  

You can build a quick and dirty version of the queries on fly like this:

Create PROCEDURE [dbo].[procDataDictionarySearch]
	@TableName as varchar(32),
	@SearchString as varchar(32) = null,
	@LastDate as datetime = null
as
	declare @SQL as varchar(4000);
 

 SELECT @SearchString = '%' + @SearchString + '%'

 DECLARE rsTemp CURSOR
	for 
	Select distinct column_name, data_type from dbo.DataDictionary where table_name = @tablename
	OPEN rsTemp
	DECLARE @column_name varchar(30);
	DECLARE @data_type int

	FETCH NEXT FROM rsTemp INTO @column_name, @data_type;
	if @SearchString is null 
		begin
		set @SQL = 'Select * from ' + @tablename + ' where ('  + @column_name + ' is null'
		end
	else
		begin		
		set @SQL = 'Select * from ' + @tablename + ' where (' + @column_name + ' like ' + char(39) + @SearchString + char(39)
		end

	WHILE @@Fetch_Status =0
	begin
	
	if @SearchString is null 
		begin
		set @SQL = @SQL + ' OR ' + @column_name + ' is null'
		end
	else
		begin				
		set @SQL = @SQL + ' OR ' + @column_name + ' like ' + char(39) + @SearchString + char(39)
		end
	print @SQL
		FETCH NEXT FROM rsTemp INTO @column_name, @data_type;
	end
 
CLOSE rsTemp;
DEALLOCATE rsTemp;

set @sql = @sql + ')'

--print @sql
 execute (@SQL)


This version won't get you counts, just selects.  So you would want to make that improvement before you go on.

Good luck.

Liz



This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation.


-----Original Message-----
From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby
Sent: Monday, September 21, 2009 2:18 PM
To: Discussion concerning MS SQL Server
Subject: Re: [dba-SQLServer] Test for empty columns

Elizabeth,

That did indeed give me a list of all tables, and fields in the tables.

Now...

The goal is to select a specific table, iterate through all of the fields in that table, and 
discover whether each field has any data, even in just a single row.

The first table I will be doing this to contains 50 million records, and ~640 fields.  Obviously 
there are not indexes on each field, though there are on perhaps a hundred of the 640 fields.  I 
need to store the results such that when the process is finished I have a record of which fields are 
empty (have no data in any record).  This table is completely static to this point, i.e. I never 
update any of the fields.  Thus by dropping any empty fields I can potentially speed up all further 
operations on this table, with no discernible downside.

So is an acceptable strategy to simply create a count query on-the-fly and count() each column? 
Some faster way (remember there will not be an index for most fields).  Would it be faster to make 
an index for one field, count the field, then drop the index?

Or... does the system database somewhere contain this information already?  Which fields contain data.

John W. Colby
www.ColbyConsulting.com


Elizabeth.J.Doering at wellsfargo.com wrote:
> This sproc gets at all the columns in all the tables:
> 
>  
> CREATE  procedure [dbo].[_DataDictionary]
> --Create on the fly data dictionary.
> as
> -build your destination table if necessary
> IF NOT EXISTS (Select name FROM sys.objects WHERE object_id = OBJECT_ID(N'[DataDictionary]') AND type in (N'U'))
> 		BEGIN
> 		CREATE TABLE [dbo].[DataDictionary](
> 			[table_qualifier] [varchar](50) NULL,
> 			[table_owner] [varchar](50) NULL,
> 			[table_name] [varchar](50) NOT NULL,
> 			[column_name] [varchar](50) NOT NULL,
> 			[data_type] [int] NULL,
> 			[type_name] [varchar](50) NULL,
> 			[precision] [varchar](50) NULL,
> 			[length] [varchar](50) NULL,
> 			[scale] [varchar](50) NULL,
> 			[radix] [varchar](50) NULL,
> 			[nullable] [varchar](50) NULL,
> 			[remarks] [varchar](50) NULL,
> 			[column_def] [varchar](50) NULL,
> 			[sql_data_type] [varchar](50) NULL,
> 			[sql_datetime_sub] [varchar](50) NULL,
> 			[char_octet_length] [varchar](50) NULL,
> 			[ordinal_position] [varchar](50) NULL,
> 			[is_nullable] [varchar](50) NULL,
> 			[ss_data_type] [varchar](50) NULL,
> 		 CONSTRAINT [PK_DataDictionary] PRIMARY KEY CLUSTERED 
> 		(
> 			[table_name] ASC,
> 			[column_name] ASC
> 		)WITH (IGNORE_DUP_KEY = OFF) ON [PRIMARY]
> 		) ON [PRIMARY]
> 		END
> -but if the destination already exists, just empty it
> truncate table datadictionary
> 
> -select all the object names in the database into a cursor 
> 
> DECLARE tnames_cursor CURSOR
> FOR
>   --2000 version: select   name, type from sysobjects where (type = 'v' or type = 'u') order by name 
>   --2005 version: 
> select   name, type from sys.all_objects where is_ms_shipped = 0 and  (type = 'v' or type = 'u') and name <> 'sysdiagrams' order by name 
> OPEN tnames_cursor;
> DECLARE @tablename varchar(100);
> DECLARE @type varchar(5)
> 
> -loop through the cursor, pulling off all the column names into your table
> 
> FETCH NEXT FROM tnames_cursor INTO @tablename, @type;
> WHILE (@@FETCH_STATUS <> -1)
> BEGIN
>    IF (@@FETCH_STATUS <> -2)
>    BEGIN   
>       SELECT @tablename = RTRIM(@tablename); 
> 	  INSERT INTO [dbo].[DataDictionary]
>            ([table_qualifier]
>            ,[table_owner]
>            ,[table_name]
>            ,[column_name]
>            ,[data_type]
>            ,[type_name]
>            ,[precision]
>            ,[length]
>            ,[scale]
>            ,[radix]
>            ,[nullable]
>            ,[remarks]
>            ,[column_def]
>            ,[sql_data_type]
>            ,[sql_datetime_sub]
>            ,[char_octet_length]
>            ,[ordinal_position]
>            ,[is_nullable]
>            ,[ss_data_type]) exec sys.sp_columns @tablename
>  
>    END;
>    FETCH NEXT FROM tnames_cursor INTO @tablename, @type;
> END;
> CLOSE tnames_cursor;
> DEALLOCATE tnames_cursor; 
> 
> 
> 
> 
> 
> 
> Thanks, 
> 
> Liz 
> 
> 
> This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation.
> 
> 
> -----Original Message-----
> From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby
> Sent: Monday, September 21, 2009 12:42 PM
> To: Dba-Sqlserver
> Subject: [dba-SQLServer] Test for empty columns
> 
> My database from hell has at least some columns where the column simply contains no data in any 
> record.  I am wondering how I would find and list these columns so that I can delete them from my table.
> 
> Would I write a stored procedure to get the columns from the database and then just iterate through 
> that recordset building a dynamic query?  Is there already something out there that does this? 
> Anybody doing this?
> 
> TIA,
> 
_______________________________________________
dba-SQLServer mailing list
dba-SQLServer at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
http://www.databaseadvisors.com




From jwcolby at colbyconsulting.com  Tue Sep 22 21:57:39 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Tue, 22 Sep 2009 22:57:39 -0400
Subject: [dba-SQLServer] I'm baffled
Message-ID: <4AB98EA3.9070806@colbyconsulting.com>

I am trying to return a count of data items in a field of my table.  When I directly execute the 
following I get a count:

SELECT COUNT(PKID) as Cnt
FROM dbo.tblHSIDModified
WHERE (Household_Occupation_code <> '')

The count is 15675589.

When I try to do the "same thing" in a SP (to store the count) the count is equal to the number of 
records in the table.  The SQL is:

DECLARE @DataCnt int

SET @DataCnt = (SELECT count(PKID) as Cnt
		FROM dbo.tblHSIDModified
		WHERE (@fldname <> ''))

PRINT @fldname + ' counted: ' + cast(@DataCnt as varchar) + ' valid data elements'

As you can see everything is the same except @FldName which is being read out of a cursor of field 
names.  It seems obvious that @fldname <> '' evaluates to true every record.  I don't know enough to 
know why.

Somehow I need to "translate" @FldName to the actual name of the field.

What is the syntax to get what I need here?

TIA,

-- 
John W. Colby
www.ColbyConsulting.com


From stuart at lexacorp.com.pg  Wed Sep 23 00:25:53 2009
From: stuart at lexacorp.com.pg (Stuart McLachlan)
Date: Wed, 23 Sep 2009 15:25:53 +1000
Subject: [dba-SQLServer] I'm baffled
In-Reply-To: <4AB98EA3.9070806@colbyconsulting.com>
References: <4AB98EA3.9070806@colbyconsulting.com>
Message-ID: <4AB9B161.16652.1F405BD9@stuart.lexacorp.com.pg>

Household_Occupation_Code is a fieldname (an object)
@fldname is a string

Your SP version evaluates to:

SET @DataCnt = (SELECT count(PKID) as Cnt
		FROM dbo.tblHSIDModified
 		WHERE ('Household_Occupation_Code' <> ''))

rather than

SET @DataCnt = (SELECT count(PKID) as Cnt
		FROM dbo.tblHSIDModified
 		WHERE (Household_Occupation_Code <> ''))


To interpret the variable, you could build your query dynamically and then EXECute it in 
your SP.  Something like:

EXEC 'SET @DataCnt = (SELECT count(PKID) as Cnt ' + 
		'FROM dbo.tblHSIDModified ' + 
 		'WHERE (" + @fldsname + '<> ''''))'


-- 
Stuart

On 22 Sep 2009 at 22:57, jwcolby wrote:

> I am trying to return a count of data items in a field of my table.  When I directly execute the 
> following I get a count:
> 
> SELECT COUNT(PKID) as Cnt
> FROM dbo.tblHSIDModified
> WHERE (Household_Occupation_code <> '')
> 
> The count is 15675589.
> 
> When I try to do the "same thing" in a SP (to store the count) the count is equal to the number of 
> records in the table.  The SQL is:
> 
> DECLARE @DataCnt int
> 
> SET @DataCnt = (SELECT count(PKID) as Cnt
> 		FROM dbo.tblHSIDModified
> 		WHERE (@fldname <> ''))
> 
> PRINT @fldname + ' counted: ' + cast(@DataCnt as varchar) + ' valid data elements'
> 
> As you can see everything is the same except @FldName which is being read out of a cursor of field 
> names.  It seems obvious that @fldname <> '' evaluates to true every record.  I don't know enough to 
> know why.
> 
> Somehow I need to "translate" @FldName to the actual name of the field.
> 
> What is the syntax to get what I need here?
> 
> TIA,
> 
> -- 
> John W. Colby
> www.ColbyConsulting.com
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 




From jwcolby at colbyconsulting.com  Wed Sep 23 07:50:22 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Wed, 23 Sep 2009 08:50:22 -0400
Subject: [dba-SQLServer] [dba-VB]  I'm baffled
In-Reply-To: <4AB9B161.16652.1F405BD9@stuart.lexacorp.com.pg>
References: <4AB98EA3.9070806@colbyconsulting.com>
	<4AB9B161.16652.1F405BD9@stuart.lexacorp.com.pg>
Message-ID: <4ABA198E.8070600@colbyconsulting.com>

Stuart,

When I do that I get an error "must declare the scalar variable @DataCnt"


John W. Colby
www.ColbyConsulting.com


Stuart McLachlan wrote:
> Household_Occupation_Code is a fieldname (an object)
> @fldname is a string
> 
> Your SP version evaluates to:
> 
> SET @DataCnt = (SELECT count(PKID) as Cnt
> 		FROM dbo.tblHSIDModified
>  		WHERE ('Household_Occupation_Code' <> ''))
> 
> rather than
> 
> SET @DataCnt = (SELECT count(PKID) as Cnt
> 		FROM dbo.tblHSIDModified
>  		WHERE (Household_Occupation_Code <> ''))
> 
> 
> To interpret the variable, you could build your query dynamically and then EXECute it in 
> your SP.  Something like:
> 
> EXEC 'SET @DataCnt = (SELECT count(PKID) as Cnt ' + 
> 		'FROM dbo.tblHSIDModified ' + 
>  		'WHERE (" + @fldsname + '<> ''''))'
> 
> 


From jwcolby at colbyconsulting.com  Wed Sep 23 09:41:37 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Wed, 23 Sep 2009 10:41:37 -0400
Subject: [dba-SQLServer] Export wizard hangs
Message-ID: <4ABA33A1.1020600@colbyconsulting.com>

I am trying to export 50 million names / addresses to a CSV file.  I did so once using a simple 
query that pulls from a single table.  I realized I needed to join on the zip from another table (in 
a zip code database) which filters down which zip codes and states I use.  this is normal stuff, I 
do it all the time.

I have a query which I built to do this.  I can do a count (PKID) on this table and get results back 
in just a few seconds.  However the export wizard exports about 50K records and hangs "performing 
operation", with the little circle widget a circlin'.

Indexes applied on all relevant join fields etc.

It has something to do with the join to the ZipCodeWorld out in the other database.  Take that out 
and the thing flies.  50K records / second perhaps.  Put the zip code view back in to the source 
query and the thing slows to a crawl.  It looks like it is doing 10K chunks every 30 seconds or so.

I use this zip code world view all of the time!  Joined to this and other tables.  And the weird 
thing is that I can view the data and it opens instantly and starts displaying data like a mad man. 
  Count the PK from the view and it counts in just a few seconds.  But the export wizard is having 
problems with the view, but only when that zipcode view is joined in.

Any ideas why?

-- 
John W. Colby
www.ColbyConsulting.com


From fuller.artful at gmail.com  Wed Sep 23 09:49:11 2009
From: fuller.artful at gmail.com (Arthur Fuller)
Date: Wed, 23 Sep 2009 10:49:11 -0400
Subject: [dba-SQLServer] Export wizard hangs
In-Reply-To: <4ABA33A1.1020600@colbyconsulting.com>
References: <4ABA33A1.1020600@colbyconsulting.com>
Message-ID: <29f585dd0909230749s7363e357n7852707030cfd2f3@mail.gmail.com>

I forget its name offhand, but there is a switch you can use to control the
size of the batch, as it were.
Arthur

On Wed, Sep 23, 2009 at 10:41 AM, jwcolby <jwcolby at colbyconsulting.com>wrote:

> I am trying to export 50 million names / addresses to a CSV file.  I did so
> once using a simple
> query that pulls from a single table.  I realized I needed to join on the
> zip from another table (in
> a zip code database) which filters down which zip codes and states I use.
>  this is normal stuff, I
> do it all the time.
>
> I have a query which I built to do this.  I can do a count (PKID) on this
> table and get results back
> in just a few seconds.  However the export wizard exports about 50K records
> and hangs "performing
> operation", with the little circle widget a circlin'.
>
> Indexes applied on all relevant join fields etc.
>
> It has something to do with the join to the ZipCodeWorld out in the other
> database.  Take that out
> and the thing flies.  50K records / second perhaps.  Put the zip code view
> back in to the source
> query and the thing slows to a crawl.  It looks like it is doing 10K chunks
> every 30 seconds or so.
>
> I use this zip code world view all of the time!  Joined to this and other
> tables.  And the weird
> thing is that I can view the data and it opens instantly and starts
> displaying data like a mad man.
>  Count the PK from the view and it counts in just a few seconds.  But the
> export wizard is having
> problems with the view, but only when that zipcode view is joined in.
>
> Any ideas why?
>
>


From nancy.lytle at gmail.com  Wed Sep 23 10:01:22 2009
From: nancy.lytle at gmail.com (Nancy Lytle)
Date: Wed, 23 Sep 2009 10:01:22 -0500
Subject: [dba-SQLServer] Export wizard hangs
In-Reply-To: <4ABA33A1.1020600@colbyconsulting.com>
References: <4ABA33A1.1020600@colbyconsulting.com>
Message-ID: <BAY134-W2BA388EB9706DDD8A0EC284DB0@phx.gbl>


Are you doing a BCP out?

Nancy Lytle 
N_Lytle at terpalum.umd.edu









 EMAILING FOR THE GREATER GOOD
Join me
 
> Date: Wed, 23 Sep 2009 10:41:37 -0400
> From: jwcolby at colbyconsulting.com
> To: dba-sqlserver at databaseadvisors.com
> Subject: [dba-SQLServer] Export wizard hangs
> 
> I am trying to export 50 million names / addresses to a CSV file. I did so once using a simple 
> query that pulls from a single table. I realized I needed to join on the zip from another table (in 
> a zip code database) which filters down which zip codes and states I use. this is normal stuff, I 
> do it all the time.
> 
> I have a query which I built to do this. I can do a count (PKID) on this table and get results back 
> in just a few seconds. However the export wizard exports about 50K records and hangs "performing 
> operation", with the little circle widget a circlin'.
> 
> Indexes applied on all relevant join fields etc.
> 
> It has something to do with the join to the ZipCodeWorld out in the other database. Take that out 
> and the thing flies. 50K records / second perhaps. Put the zip code view back in to the source 
> query and the thing slows to a crawl. It looks like it is doing 10K chunks every 30 seconds or so.
> 
> I use this zip code world view all of the time! Joined to this and other tables. And the weird 
> thing is that I can view the data and it opens instantly and starts displaying data like a mad man. 
> Count the PK from the view and it counts in just a few seconds. But the export wizard is having 
> problems with the view, but only when that zipcode view is joined in.
> 
> Any ideas why?
> 
> -- 
> John W. Colby
> www.ColbyConsulting.com
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
 		 	   		  

From jwcolby at colbyconsulting.com  Wed Sep 23 10:09:45 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Wed, 23 Sep 2009 11:09:45 -0400
Subject: [dba-SQLServer] Export wizard hangs
In-Reply-To: <BAY134-W2BA388EB9706DDD8A0EC284DB0@phx.gbl>
References: <4ABA33A1.1020600@colbyconsulting.com>
	<BAY134-W2BA388EB9706DDD8A0EC284DB0@phx.gbl>
Message-ID: <4ABA3A39.1090905@colbyconsulting.com>

Behind the scenes I have no idea.  It is the wizard that is running.

John W. Colby
www.ColbyConsulting.com


Nancy Lytle wrote:
> Are you doing a BCP out?
> 
> Nancy Lytle 
> N_Lytle at terpalum.umd.edu
> 
> 
> 
> 
> 
> 
> 
> 
> 
>  EMAILING FOR THE GREATER GOOD
> Join me
>  
>> Date: Wed, 23 Sep 2009 10:41:37 -0400
>> From: jwcolby at colbyconsulting.com
>> To: dba-sqlserver at databaseadvisors.com
>> Subject: [dba-SQLServer] Export wizard hangs
>>
>> I am trying to export 50 million names / addresses to a CSV file. I did so once using a simple 
>> query that pulls from a single table. I realized I needed to join on the zip from another table (in 
>> a zip code database) which filters down which zip codes and states I use. this is normal stuff, I 
>> do it all the time.
>>
>> I have a query which I built to do this. I can do a count (PKID) on this table and get results back 
>> in just a few seconds. However the export wizard exports about 50K records and hangs "performing 
>> operation", with the little circle widget a circlin'.
>>
>> Indexes applied on all relevant join fields etc.
>>
>> It has something to do with the join to the ZipCodeWorld out in the other database. Take that out 
>> and the thing flies. 50K records / second perhaps. Put the zip code view back in to the source 
>> query and the thing slows to a crawl. It looks like it is doing 10K chunks every 30 seconds or so.
>>
>> I use this zip code world view all of the time! Joined to this and other tables. And the weird 
>> thing is that I can view the data and it opens instantly and starts displaying data like a mad man. 
>> Count the PK from the view and it counts in just a few seconds. But the export wizard is having 
>> problems with the view, but only when that zipcode view is joined in.
>>
>> Any ideas why?
>>
>> -- 
>> John W. Colby
>> www.ColbyConsulting.com
>> _______________________________________________
>> dba-SQLServer mailing list
>> dba-SQLServer at databaseadvisors.com
>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>> http://www.databaseadvisors.com
>>
>  		 	   		  
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From nancy.lytle at gmail.com  Wed Sep 23 10:24:39 2009
From: nancy.lytle at gmail.com (Nancy Lytle)
Date: Wed, 23 Sep 2009 10:24:39 -0500
Subject: [dba-SQLServer] Export wizard hangs
In-Reply-To: <4ABA3A39.1090905@colbyconsulting.com>
References: <4ABA33A1.1020600@colbyconsulting.com>
	<BAY134-W2BA388EB9706DDD8A0EC284DB0@phx.gbl>
Message-ID: <BAY134-W22A49DF3F9DF0733C0223C84DB0@phx.gbl>


BCP is very fast way of getting SQL Server data out and into a text file, check it out in BOL or google it.  Also very easy to use.

Nancy Lytle 
N_Lytle at terpalum.umd.edu









 EMAILING FOR THE GREATER GOOD
Join me
 
> Date: Wed, 23 Sep 2009 11:09:45 -0400
> From: jwcolby at colbyconsulting.com
> To: dba-sqlserver at databaseadvisors.com
> Subject: Re: [dba-SQLServer] Export wizard hangs
> 
> Behind the scenes I have no idea. It is the wizard that is running.
> 
> John W. Colby
> www.ColbyConsulting.com
> 
> 
> Nancy Lytle wrote:
> > Are you doing a BCP out?
> > 
> > Nancy Lytle 
> > N_Lytle at terpalum.umd.edu
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > EMAILING FOR THE GREATER GOOD
> > Join me
> > 
> >> Date: Wed, 23 Sep 2009 10:41:37 -0400
> >> From: jwcolby at colbyconsulting.com
> >> To: dba-sqlserver at databaseadvisors.com
> >> Subject: [dba-SQLServer] Export wizard hangs
> >>
> >> I am trying to export 50 million names / addresses to a CSV file. I did so once using a simple 
> >> query that pulls from a single table. I realized I needed to join on the zip from another table (in 
> >> a zip code database) which filters down which zip codes and states I use. this is normal stuff, I 
> >> do it all the time.
> >>
> >> I have a query which I built to do this. I can do a count (PKID) on this table and get results back 
> >> in just a few seconds. However the export wizard exports about 50K records and hangs "performing 
> >> operation", with the little circle widget a circlin'.
> >>
> >> Indexes applied on all relevant join fields etc.
> >>
> >> It has something to do with the join to the ZipCodeWorld out in the other database. Take that out 
> >> and the thing flies. 50K records / second perhaps. Put the zip code view back in to the source 
> >> query and the thing slows to a crawl. It looks like it is doing 10K chunks every 30 seconds or so.
> >>
> >> I use this zip code world view all of the time! Joined to this and other tables. And the weird 
> >> thing is that I can view the data and it opens instantly and starts displaying data like a mad man. 
> >> Count the PK from the view and it counts in just a few seconds. But the export wizard is having 
> >> problems with the view, but only when that zipcode view is joined in.
> >>
> >> Any ideas why?
> >>
> >> -- 
> >> John W. Colby
> >> www.ColbyConsulting.com
> >> _______________________________________________
> >> dba-SQLServer mailing list
> >> dba-SQLServer at databaseadvisors.com
> >> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> >> http://www.databaseadvisors.com
> >>
> > 
> > _______________________________________________
> > dba-SQLServer mailing list
> > dba-SQLServer at databaseadvisors.com
> > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> > http://www.databaseadvisors.com
> > 
> > 
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
 		 	   		  

From jwcolby at colbyconsulting.com  Thu Sep 24 09:34:24 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Thu, 24 Sep 2009 10:34:24 -0400
Subject: [dba-SQLServer] gracefully shut down a process
Message-ID: <4ABB8370.4020608@colbyconsulting.com>

Is there any way to gracefully shut down a running query QUICKLY?

I have a stored procedure running which really should have completed long ago.  This SP calculates a 
hash value for a set of concatenated text fields and stores the result back into a binary hash field.

I ran this thing 3 times today on other tables of roughly the same number of records and they 
completed in perhaps 15 minutes.  For whatever reason, this one has been running 1.5 hours.  These 
process bogs down the server such that very little real work can get done with it running.

So I need to tell SQL Server to just stop!  Store what you have calculated and stop!

Any way to do this?  Without killing the table being written to.

-- 
John W. Colby
www.ColbyConsulting.com


From jwcolby at colbyconsulting.com  Thu Sep 24 09:56:56 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Thu, 24 Sep 2009 10:56:56 -0400
Subject: [dba-SQLServer] HELP, server completely unresponsive
Message-ID: <4ABB88B8.6060605@colbyconsulting.com>

I have a situation way outside my expertise.  Yea, yea, I know a lot of you would ask what is IN my 
expertise, we can discuss that another day.

Anyway, I run two Windows 2003 X64 servers.  Quad core, 16 gigs ram, RAID6 arrays.  Reasonably 
powerful.  Mostly I run SQL Server on one and VMs on the other.  In BOTH CASES I have told the 
software, SQL Server for instance, to only use X gigs of RAM, to leave 2.5 gigs for the OS and other 
applications.

And yet, SQL Server starts a long running process and the server becomes unresponsive.  In some 
instances for example, I can load a spreadsheet or whatever... but it takes forever to load.  Just 
clicking the start button I have to wait 30 seconds for the start menu to show.

Like that.

I have a query running which is updating a field in a medium size table - 8 million records or so. 
I tried to load another instance of SQL Server.  It loaded, but when I clicked on the databases icon 
to drop down and show me the databases it just put up expanding and locked up the entire system.

I am at a loss to discover what is going on.  It APPEARS to be SQL Server, though there is nothing 
else I run that uses all the memory in this thing so how do I know it isn't just Windows 2003 
flaking out somehow?  I have run memtest-86 for an entire weekend without a single failure, so I 
don't think it is the physical memory.

I have no clue how to troubleshoot this problem.

As I write this, the server is completely locked up.  I cannot switch between tasks, the little task 
manager icon shows about 50% CPU usage but AFAICT that is just what was displayed when it locked up.

In the end I have to do a hard reboot to regain control, and when I do that I corrupt whatever 
database was being worked on.  This is insane!

HELP.

-- 
John W. Colby
www.ColbyConsulting.com


From fhtapia at gmail.com  Thu Sep 24 10:55:48 2009
From: fhtapia at gmail.com (Francisco Tapia)
Date: Thu, 24 Sep 2009 08:55:48 -0700
Subject: [dba-SQLServer] gracefully shut down a process
In-Reply-To: <4ABB8370.4020608@colbyconsulting.com>
References: <4ABB8370.4020608@colbyconsulting.com>
Message-ID: <BC4F0191-8F28-4DFC-9B7E-8BEE0B26F0B6@gmail.com>

You can issue a kill spid for the running process but th

Sent from my mobile

On Sep 24, 2009, at 7:34 AM, jwcolby <jwcolby at colbyconsulting.com>  
wrote:

> Is there any way to gracefully shut down a running query QUICKLY?
>
> I have a stored procedure running which really should have completed  
> long ago.  This SP calculates a
> hash value for a set of concatenated text fields and stores the  
> result back into a binary hash field.
>
> I ran this thing 3 times today on other tables of roughly the same  
> number of records and they
> completed in perhaps 15 minutes.  For whatever reason, this one has  
> been running 1.5 hours.  These
> process bogs down the server such that very little real work can get  
> done with it running.
>
> So I need to tell SQL Server to just stop!  Store what you have  
> calculated and stop!
>
> Any way to do this?  Without killing the table being written to.
>
> -- 
> John W. Colby
> www.ColbyConsulting.com
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>


From fhtapia at gmail.com  Thu Sep 24 10:57:51 2009
From: fhtapia at gmail.com (Francisco Tapia)
Date: Thu, 24 Sep 2009 08:57:51 -0700
Subject: [dba-SQLServer] gracefully shut down a process
In-Reply-To: <4ABB8370.4020608@colbyconsulting.com>
References: <4ABB8370.4020608@colbyconsulting.com>
Message-ID: <19D42E25-411E-4CD1-85E7-D38852AA40B6@gmail.com>

Arg sorry I sent the previous message too early

I was saying if you kill the spid you can stop the process but then  
you might have to wait through the rollback for the changes to be  
undone. Sometimes if what you were running involves a cursor the only  
solution might be to restatt the SQL service.

Sent from my mobile

On Sep 24, 2009, at 7:34 AM, jwcolby <jwcolby at colbyconsulting.com>  
wrote:

> Is there any way to gracefully shut down a running query QUICKLY?
>
> I have a stored procedure running which really should have completed  
> long ago.  This SP calculates a
> hash value for a set of concatenated text fields and stores the  
> result back into a binary hash field.
>
> I ran this thing 3 times today on other tables of roughly the same  
> number of records and they
> completed in perhaps 15 minutes.  For whatever reason, this one has  
> been running 1.5 hours.  These
> process bogs down the server such that very little real work can get  
> done with it running.
>
> So I need to tell SQL Server to just stop!  Store what you have  
> calculated and stop!
>
> Any way to do this?  Without killing the table being written to.
>
> -- 
> John W. Colby
> www.ColbyConsulting.com
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>


From fhtapia at gmail.com  Thu Sep 24 11:01:33 2009
From: fhtapia at gmail.com (Francisco Tapia)
Date: Thu, 24 Sep 2009 09:01:33 -0700
Subject: [dba-SQLServer] HELP, server completely unresponsive
In-Reply-To: <4ABB88B8.6060605@colbyconsulting.com>
References: <4ABB88B8.6060605@colbyconsulting.com>
Message-ID: <3F6B56EB-BE68-4FCE-990F-2D7A54939B32@gmail.com>

In the configuration ( properties) of the server you'll want to leave  
at least on CPU for the os so in your quad setup allow for 3 CPUs for  
SQL but one for the os or if you don't want to do this then you can  
always use the query hint option of MAXDOP=3 so only while the query  
runs you only tie up 3 CPUs and not all of them and this allows you to  
still have a responsive server.

Sent from my mobile

On Sep 24, 2009, at 7:56 AM, jwcolby <jwcolby at colbyconsulting.com>  
wrote:

> I have a situation way outside my expertise.  Yea, yea, I know a lot  
> of you would ask what is IN my
> expertise, we can discuss that another day.
>
> Anyway, I run two Windows 2003 X64 servers.  Quad core, 16 gigs ram,  
> RAID6 arrays.  Reasonably
> powerful.  Mostly I run SQL Server on one and VMs on the other.  In  
> BOTH CASES I have told the
> software, SQL Server for instance, to only use X gigs of RAM, to  
> leave 2.5 gigs for the OS and other
> applications.
>
> And yet, SQL Server starts a long running process and the server  
> becomes unresponsive.  In some
> instances for example, I can load a spreadsheet or whatever... but  
> it takes forever to load.  Just
> clicking the start button I have to wait 30 seconds for the start  
> menu to show.
>
> Like that.
>
> I have a query running which is updating a field in a medium size  
> table - 8 million records or so.
> I tried to load another instance of SQL Server.  It loaded, but when  
> I clicked on the databases icon
> to drop down and show me the databases it just put up expanding and  
> locked up the entire system.
>
> I am at a loss to discover what is going on.  It APPEARS to be SQL  
> Server, though there is nothing
> else I run that uses all the memory in this thing so how do I know  
> it isn't just Windows 2003
> flaking out somehow?  I have run memtest-86 for an entire weekend  
> without a single failure, so I
> don't think it is the physical memory.
>
> I have no clue how to troubleshoot this problem.
>
> As I write this, the server is completely locked up.  I cannot  
> switch between tasks, the little task
> manager icon shows about 50% CPU usage but AFAICT that is just what  
> was displayed when it locked up.
>
> In the end I have to do a hard reboot to regain control, and when I  
> do that I corrupt whatever
> database was being worked on.  This is insane!
>
> HELP.
>
> -- 
> John W. Colby
> www.ColbyConsulting.com
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>


From jwcolby at colbyconsulting.com  Thu Sep 24 11:17:42 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Thu, 24 Sep 2009 12:17:42 -0400
Subject: [dba-SQLServer] HELP, server completely unresponsive
In-Reply-To: <3F6B56EB-BE68-4FCE-990F-2D7A54939B32@gmail.com>
References: <4ABB88B8.6060605@colbyconsulting.com>
	<3F6B56EB-BE68-4FCE-990F-2D7A54939B32@gmail.com>
Message-ID: <4ABB9BA6.90406@colbyconsulting.com>

Is this the setup for Windows 2003 or the setup for SQL Server?

John W. Colby
www.ColbyConsulting.com


Francisco Tapia wrote:
> In the configuration ( properties) of the server you'll want to leave  
> at least on CPU for the os so in your quad setup allow for 3 CPUs for  
> SQL but one for the os or if you don't want to do this then you can  
> always use the query hint option of MAXDOP=3 so only while the query  
> runs you only tie up 3 CPUs and not all of them and this allows you to  
> still have a responsive server.
> 
> Sent from my mobile
> 
> On Sep 24, 2009, at 7:56 AM, jwcolby <jwcolby at colbyconsulting.com>  
> wrote:
> 
>> I have a situation way outside my expertise.  Yea, yea, I know a lot  
>> of you would ask what is IN my
>> expertise, we can discuss that another day.
>>
>> Anyway, I run two Windows 2003 X64 servers.  Quad core, 16 gigs ram,  
>> RAID6 arrays.  Reasonably
>> powerful.  Mostly I run SQL Server on one and VMs on the other.  In  
>> BOTH CASES I have told the
>> software, SQL Server for instance, to only use X gigs of RAM, to  
>> leave 2.5 gigs for the OS and other
>> applications.
>>
>> And yet, SQL Server starts a long running process and the server  
>> becomes unresponsive.  In some
>> instances for example, I can load a spreadsheet or whatever... but  
>> it takes forever to load.  Just
>> clicking the start button I have to wait 30 seconds for the start  
>> menu to show.
>>
>> Like that.
>>
>> I have a query running which is updating a field in a medium size  
>> table - 8 million records or so.
>> I tried to load another instance of SQL Server.  It loaded, but when  
>> I clicked on the databases icon
>> to drop down and show me the databases it just put up expanding and  
>> locked up the entire system.
>>
>> I am at a loss to discover what is going on.  It APPEARS to be SQL  
>> Server, though there is nothing
>> else I run that uses all the memory in this thing so how do I know  
>> it isn't just Windows 2003
>> flaking out somehow?  I have run memtest-86 for an entire weekend  
>> without a single failure, so I
>> don't think it is the physical memory.
>>
>> I have no clue how to troubleshoot this problem.
>>
>> As I write this, the server is completely locked up.  I cannot  
>> switch between tasks, the little task
>> manager icon shows about 50% CPU usage but AFAICT that is just what  
>> was displayed when it locked up.
>>
>> In the end I have to do a hard reboot to regain control, and when I  
>> do that I corrupt whatever
>> database was being worked on.  This is insane!
>>
>> HELP.
>>
>> -- 
>> John W. Colby
>> www.ColbyConsulting.com
>> _______________________________________________
>> dba-SQLServer mailing list
>> dba-SQLServer at databaseadvisors.com
>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>> http://www.databaseadvisors.com
>>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From jwcolby at colbyconsulting.com  Thu Sep 24 12:07:00 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Thu, 24 Sep 2009 13:07:00 -0400
Subject: [dba-SQLServer] HELP, server completely unresponsive
In-Reply-To: <4ABB9BA6.90406@colbyconsulting.com>
References: <4ABB88B8.6060605@colbyconsulting.com>	<3F6B56EB-BE68-4FCE-990F-2D7A54939B32@gmail.com>
	<4ABB9BA6.90406@colbyconsulting.com>
Message-ID: <4ABBA734.5090709@colbyconsulting.com>

Never mind, I found it in the processors property of the SQL Server, server level.

Thanks,

John W. Colby
www.ColbyConsulting.com


jwcolby wrote:
> Is this the setup for Windows 2003 or the setup for SQL Server?
> 
> John W. Colby
> www.ColbyConsulting.com
> 
> 
> Francisco Tapia wrote:
>> In the configuration ( properties) of the server you'll want to leave  
>> at least on CPU for the os so in your quad setup allow for 3 CPUs for  
>> SQL but one for the os or if you don't want to do this then you can  
>> always use the query hint option of MAXDOP=3 so only while the query  
>> runs you only tie up 3 CPUs and not all of them and this allows you to  
>> still have a responsive server.
>>
>> Sent from my mobile
>>
>> On Sep 24, 2009, at 7:56 AM, jwcolby <jwcolby at colbyconsulting.com>  
>> wrote:
>>
>>> I have a situation way outside my expertise.  Yea, yea, I know a lot  
>>> of you would ask what is IN my
>>> expertise, we can discuss that another day.
>>>
>>> Anyway, I run two Windows 2003 X64 servers.  Quad core, 16 gigs ram,  
>>> RAID6 arrays.  Reasonably
>>> powerful.  Mostly I run SQL Server on one and VMs on the other.  In  
>>> BOTH CASES I have told the
>>> software, SQL Server for instance, to only use X gigs of RAM, to  
>>> leave 2.5 gigs for the OS and other
>>> applications.
>>>
>>> And yet, SQL Server starts a long running process and the server  
>>> becomes unresponsive.  In some
>>> instances for example, I can load a spreadsheet or whatever... but  
>>> it takes forever to load.  Just
>>> clicking the start button I have to wait 30 seconds for the start  
>>> menu to show.
>>>
>>> Like that.
>>>
>>> I have a query running which is updating a field in a medium size  
>>> table - 8 million records or so.
>>> I tried to load another instance of SQL Server.  It loaded, but when  
>>> I clicked on the databases icon
>>> to drop down and show me the databases it just put up expanding and  
>>> locked up the entire system.
>>>
>>> I am at a loss to discover what is going on.  It APPEARS to be SQL  
>>> Server, though there is nothing
>>> else I run that uses all the memory in this thing so how do I know  
>>> it isn't just Windows 2003
>>> flaking out somehow?  I have run memtest-86 for an entire weekend  
>>> without a single failure, so I
>>> don't think it is the physical memory.
>>>
>>> I have no clue how to troubleshoot this problem.
>>>
>>> As I write this, the server is completely locked up.  I cannot  
>>> switch between tasks, the little task
>>> manager icon shows about 50% CPU usage but AFAICT that is just what  
>>> was displayed when it locked up.
>>>
>>> In the end I have to do a hard reboot to regain control, and when I  
>>> do that I corrupt whatever
>>> database was being worked on.  This is insane!
>>>
>>> HELP.
>>>
>>> -- 
>>> John W. Colby
>>> www.ColbyConsulting.com
>>> _______________________________________________
>>> dba-SQLServer mailing list
>>> dba-SQLServer at databaseadvisors.com
>>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>>> http://www.databaseadvisors.com
>>>
>> _______________________________________________
>> dba-SQLServer mailing list
>> dba-SQLServer at databaseadvisors.com
>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>> http://www.databaseadvisors.com
>>
>>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From fhtapia at gmail.com  Thu Sep 24 12:19:49 2009
From: fhtapia at gmail.com (Francisco Tapia)
Date: Thu, 24 Sep 2009 10:19:49 -0700
Subject: [dba-SQLServer] HELP, server completely unresponsive
In-Reply-To: <4ABB9BA6.90406@colbyconsulting.com>
References: <4ABB88B8.6060605@colbyconsulting.com>
	<3F6B56EB-BE68-4FCE-990F-2D7A54939B32@gmail.com>
	<4ABB9BA6.90406@colbyconsulting.com>
Message-ID: <1D28B8C7-F3D5-4BB9-B53A-CF2900A5ADF9@gmail.com>

For SQL server

Sent from my mobile

On Sep 24, 2009, at 9:17 AM, jwcolby <jwcolby at colbyconsulting.com>  
wrote:

> Is this the setup for Windows 2003 or the setup for SQL Server?
>
> John W. Colby
> www.ColbyConsulting.com
>
>
> Francisco Tapia wrote:
>> In the configuration ( properties) of the server you'll want to leave
>> at least on CPU for the os so in your quad setup allow for 3 CPUs for
>> SQL but one for the os or if you don't want to do this then you can
>> always use the query hint option of MAXDOP=3 so only while the query
>> runs you only tie up 3 CPUs and not all of them and this allows you  
>> to
>> still have a responsive server.
>>
>> Sent from my mobile
>>
>> On Sep 24, 2009, at 7:56 AM, jwcolby <jwcolby at colbyconsulting.com>
>> wrote:
>>
>>> I have a situation way outside my expertise.  Yea, yea, I know a lot
>>> of you would ask what is IN my
>>> expertise, we can discuss that another day.
>>>
>>> Anyway, I run two Windows 2003 X64 servers.  Quad core, 16 gigs ram,
>>> RAID6 arrays.  Reasonably
>>> powerful.  Mostly I run SQL Server on one and VMs on the other.  In
>>> BOTH CASES I have told the
>>> software, SQL Server for instance, to only use X gigs of RAM, to
>>> leave 2.5 gigs for the OS and other
>>> applications.
>>>
>>> And yet, SQL Server starts a long running process and the server
>>> becomes unresponsive.  In some
>>> instances for example, I can load a spreadsheet or whatever... but
>>> it takes forever to load.  Just
>>> clicking the start button I have to wait 30 seconds for the start
>>> menu to show.
>>>
>>> Like that.
>>>
>>> I have a query running which is updating a field in a medium size
>>> table - 8 million records or so.
>>> I tried to load another instance of SQL Server.  It loaded, but when
>>> I clicked on the databases icon
>>> to drop down and show me the databases it just put up expanding and
>>> locked up the entire system.
>>>
>>> I am at a loss to discover what is going on.  It APPEARS to be SQL
>>> Server, though there is nothing
>>> else I run that uses all the memory in this thing so how do I know
>>> it isn't just Windows 2003
>>> flaking out somehow?  I have run memtest-86 for an entire weekend
>>> without a single failure, so I
>>> don't think it is the physical memory.
>>>
>>> I have no clue how to troubleshoot this problem.
>>>
>>> As I write this, the server is completely locked up.  I cannot
>>> switch between tasks, the little task
>>> manager icon shows about 50% CPU usage but AFAICT that is just what
>>> was displayed when it locked up.
>>>
>>> In the end I have to do a hard reboot to regain control, and when I
>>> do that I corrupt whatever
>>> database was being worked on.  This is insane!
>>>
>>> HELP.
>>>
>>> -- 
>>> John W. Colby
>>> www.ColbyConsulting.com
>>> _______________________________________________
>>> dba-SQLServer mailing list
>>> dba-SQLServer at databaseadvisors.com
>>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>>> http://www.databaseadvisors.com
>>>
>> _______________________________________________
>> dba-SQLServer mailing list
>> dba-SQLServer at databaseadvisors.com
>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>> http://www.databaseadvisors.com
>>
>>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>


From robert at webedb.com  Thu Sep 24 12:55:32 2009
From: robert at webedb.com (Robert Stewart)
Date: Thu, 24 Sep 2009 12:55:32 -0500
Subject: [dba-SQLServer] HELP, server completely unresponsive
In-Reply-To: <mailman.5.1253811601.3940.dba-sqlserver@databaseadvisors.c
 om>
References: <mailman.5.1253811601.3940.dba-sqlserver@databaseadvisors.com>
Message-ID: <200909241755.n8OHtqPB016788@databaseadvisors.com>

SQL Server.

But, it is easier to use the hint and not mess with the settings.


At 12:00 PM 9/24/2009, you wrote:
>Date: Thu, 24 Sep 2009 12:17:42 -0400
>From: jwcolby <jwcolby at colbyconsulting.com>
>Subject: Re: [dba-SQLServer] HELP, server completely unresponsive
>To: Discussion concerning MS SQL Server
>         <dba-sqlserver at databaseadvisors.com>
>Message-ID: <4ABB9BA6.90406 at colbyconsulting.com>
>Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
>Is this the setup for Windows 2003 or the setup for SQL Server?
>
>John W. Colby
>www.ColbyConsulting.com


From jwcolby at colbyconsulting.com  Mon Sep 28 10:19:09 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Mon, 28 Sep 2009 11:19:09 -0400
Subject: [dba-SQLServer] HELP, server completely unresponsive
In-Reply-To: <200909241755.n8OHtqPB016788@databaseadvisors.com>
References: <mailman.5.1253811601.3940.dba-sqlserver@databaseadvisors.com>
	<200909241755.n8OHtqPB016788@databaseadvisors.com>
Message-ID: <4AC0D3ED.2060201@colbyconsulting.com>

Well I thought that assigning three processors to the SQL Server software was the answer, and it in 
fact did make a difference in some cases, but there is still something happening that "locks up" the 
server.  IO started a long running update query going this morning and immediately afterwards I was 
able to move around, look at other stuff.  Then I went away to do other things.  When I came back it 
was "locked up" - with the Management studio application full screen, and the cursor immovable.  Of 
course I thought it was locked up, and I have always believed that it was literally never going to 
come back.

I just happened to to an Alt-Tab, then went off to my lap top to do other stuff.  When I came back I 
was at Windows Explorer, not Management Studio.  So the machine is in fact responding, but 
glacially.  The mouse cursor is locked up, cannot be moved.  I use a KVM switch, and when I move to 
another machine the cursor is moving just fine, when I move back to this machine it is locked up. 
However the cursor changed from the insertion pointer icon to the arrow icon when i switched from 
Management Studio to Windows Explorer, so again the machine is definitely not unresponsive, it is 
just responding glacially.

This time I have SQL Server assigned three processors, and I was observing that SQL Server pegged 
three of the processors when it began processing the query so I am comfortable that it in fact is 
correctly using just three processors.

So what is happening that would so completely freeze up the server that the cursor won't even move?

Weird.

Has anyone else out there ever experienced this?  Found a solution?

John W. Colby
www.ColbyConsulting.com



From jwcolby at colbyconsulting.com  Mon Sep 28 10:44:22 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Mon, 28 Sep 2009 11:44:22 -0400
Subject: [dba-SQLServer] HELP, server completely unresponsive
In-Reply-To: <4AC0D3ED.2060201@colbyconsulting.com>
References: <mailman.5.1253811601.3940.dba-sqlserver@databaseadvisors.com>	<200909241755.n8OHtqPB016788@databaseadvisors.com>
	<4AC0D3ED.2060201@colbyconsulting.com>
Message-ID: <4AC0D9D6.9080600@colbyconsulting.com>

I just managed to Alt-Tab back to Management studio, and that application is updating the screen 
again (though not the cursor movement).  the little "Executing query" icon is rotating, and the 
query timer is counting up the seconds (01:47:08 ATM).

I just discovered (on my laptop) the set IO affinity setting in SQL Server which I did not set.  No 
idea what that does for me but I wonder if that being assigned to SQL Server for all processors 
could be part of the problem.

John W. Colby
www.ColbyConsulting.com


jwcolby wrote:
> Well I thought that assigning three processors to the SQL Server software was the answer, and it in 
> fact did make a difference in some cases, but there is still something happening that "locks up" the 
> server.  IO started a long running update query going this morning and immediately afterwards I was 
> able to move around, look at other stuff.  Then I went away to do other things.  When I came back it 
> was "locked up" - with the Management studio application full screen, and the cursor immovable.  Of 
> course I thought it was locked up, and I have always believed that it was literally never going to 
> come back.
> 
> I just happened to to an Alt-Tab, then went off to my lap top to do other stuff.  When I came back I 
> was at Windows Explorer, not Management Studio.  So the machine is in fact responding, but 
> glacially.  The mouse cursor is locked up, cannot be moved.  I use a KVM switch, and when I move to 
> another machine the cursor is moving just fine, when I move back to this machine it is locked up. 
> However the cursor changed from the insertion pointer icon to the arrow icon when i switched from 
> Management Studio to Windows Explorer, so again the machine is definitely not unresponsive, it is 
> just responding glacially.
> 
> This time I have SQL Server assigned three processors, and I was observing that SQL Server pegged 
> three of the processors when it began processing the query so I am comfortable that it in fact is 
> correctly using just three processors.
> 
> So what is happening that would so completely freeze up the server that the cursor won't even move?
> 
> Weird.
> 
> Has anyone else out there ever experienced this?  Found a solution?
> 
> John W. Colby
> www.ColbyConsulting.com
> 
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From nancy.lytle at gmail.com  Mon Sep 28 14:36:09 2009
From: nancy.lytle at gmail.com (Nancy Lytle)
Date: Mon, 28 Sep 2009 14:36:09 -0500
Subject: [dba-SQLServer] HELP, server completely unresponsive
In-Reply-To: <4AC0D3ED.2060201@colbyconsulting.com>
References: <mailman.5.1253811601.3940.dba-sqlserver@databaseadvisors.com>
	<200909241755.n8OHtqPB016788@databaseadvisors.com>
Message-ID: <BAY134-W21E3D5C110B0AC3F43259284D60@phx.gbl>


What is your MAXDOP set to?  I found that setting mine a little lower than expected actually helped.

We have 2 processors, it was set to use all.  This would cause some queries to just suck the life out of it sometimes so I cut it back to 1 and it runs much better.

Nancy Lytle 
N_Lytle at terpalum.umd.edu











 EMAILING FOR THE GREATER GOOD
Join me


 
> Date: Mon, 28 Sep 2009 11:19:09 -0400
> From: jwcolby at colbyconsulting.com
> To: dba-sqlserver at databaseadvisors.com; dba-vb at databaseadvisors.com; accessd at databaseadvisors.com
> Subject: Re: [dba-SQLServer] HELP, server completely unresponsive
> 
> Well I thought that assigning three processors to the SQL Server software was the answer, and it in 
> fact did make a difference in some cases, but there is still something happening that "locks up" the 
> server. IO started a long running update query going this morning and immediately afterwards I was 
> able to move around, look at other stuff. Then I went away to do other things. When I came back it 
> was "locked up" - with the Management studio application full screen, and the cursor immovable. Of 
> course I thought it was locked up, and I have always believed that it was literally never going to 
> come back.
> 
> I just happened to to an Alt-Tab, then went off to my lap top to do other stuff. When I came back I 
> was at Windows Explorer, not Management Studio. So the machine is in fact responding, but 
> glacially. The mouse cursor is locked up, cannot be moved. I use a KVM switch, and when I move to 
> another machine the cursor is moving just fine, when I move back to this machine it is locked up. 
> However the cursor changed from the insertion pointer icon to the arrow icon when i switched from 
> Management Studio to Windows Explorer, so again the machine is definitely not unresponsive, it is 
> just responding glacially.
> 
> This time I have SQL Server assigned three processors, and I was observing that SQL Server pegged 
> three of the processors when it began processing the query so I am comfortable that it in fact is 
> correctly using just three processors.
> 
> So what is happening that would so completely freeze up the server that the cursor won't even move?
> 
> Weird.
> 
> Has anyone else out there ever experienced this? Found a solution?
> 
> John W. Colby
> www.ColbyConsulting.com
> 
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
 		 	   		  

From jwcolby at colbyconsulting.com  Mon Sep 28 14:53:54 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Mon, 28 Sep 2009 15:53:54 -0400
Subject: [dba-SQLServer] HELP, server completely unresponsive
In-Reply-To: <BAY134-W21E3D5C110B0AC3F43259284D60@phx.gbl>
References: <mailman.5.1253811601.3940.dba-sqlserver@databaseadvisors.com>	<200909241755.n8OHtqPB016788@databaseadvisors.com>
	<BAY134-W21E3D5C110B0AC3F43259284D60@phx.gbl>
Message-ID: <4AC11452.7070804@colbyconsulting.com>

What is MAXDOP?

I have already told SQL Server to only use 12 gigs and 3 processors.

John W. Colby
www.ColbyConsulting.com


Nancy Lytle wrote:
> What is your MAXDOP set to?  I found that setting mine a little lower than expected actually helped.
> 
> We have 2 processors, it was set to use all.  This would cause some queries to just suck the life out of it sometimes so I cut it back to 1 and it runs much better.
> 
> Nancy Lytle 
> N_Lytle at terpalum.umd.edu
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>  EMAILING FOR THE GREATER GOOD
> Join me
> 
> 
>  
>> Date: Mon, 28 Sep 2009 11:19:09 -0400
>> From: jwcolby at colbyconsulting.com
>> To: dba-sqlserver at databaseadvisors.com; dba-vb at databaseadvisors.com; accessd at databaseadvisors.com
>> Subject: Re: [dba-SQLServer] HELP, server completely unresponsive
>>
>> Well I thought that assigning three processors to the SQL Server software was the answer, and it in 
>> fact did make a difference in some cases, but there is still something happening that "locks up" the 
>> server. IO started a long running update query going this morning and immediately afterwards I was 
>> able to move around, look at other stuff. Then I went away to do other things. When I came back it 
>> was "locked up" - with the Management studio application full screen, and the cursor immovable. Of 
>> course I thought it was locked up, and I have always believed that it was literally never going to 
>> come back.
>>
>> I just happened to to an Alt-Tab, then went off to my lap top to do other stuff. When I came back I 
>> was at Windows Explorer, not Management Studio. So the machine is in fact responding, but 
>> glacially. The mouse cursor is locked up, cannot be moved. I use a KVM switch, and when I move to 
>> another machine the cursor is moving just fine, when I move back to this machine it is locked up. 
>> However the cursor changed from the insertion pointer icon to the arrow icon when i switched from 
>> Management Studio to Windows Explorer, so again the machine is definitely not unresponsive, it is 
>> just responding glacially.
>>
>> This time I have SQL Server assigned three processors, and I was observing that SQL Server pegged 
>> three of the processors when it began processing the query so I am comfortable that it in fact is 
>> correctly using just three processors.
>>
>> So what is happening that would so completely freeze up the server that the cursor won't even move?
>>
>> Weird.
>>
>> Has anyone else out there ever experienced this? Found a solution?
>>
>> John W. Colby
>> www.ColbyConsulting.com
>>
>> _______________________________________________
>> dba-SQLServer mailing list
>> dba-SQLServer at databaseadvisors.com
>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>> http://www.databaseadvisors.com
>>
>  		 	   		  
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From Gustav at cactus.dk  Mon Sep 28 15:56:26 2009
From: Gustav at cactus.dk (Gustav Brock)
Date: Mon, 28 Sep 2009 22:56:26 +0200
Subject: [dba-SQLServer] [AccessD] [dba-VB]  HELP,
	server completely	unresponsive
Message-ID: <sac13f38.031@cactus.dk>

Hi John

Perhaps you forgot to give the "applications" (foreground user activities) first priority - that's one of the differences between the default settings of a server OS and a workstation OS. 
My computer, System (advanced), Properties, Advanced, Performance ..

With your tasks I would indeed move _all_ user applications and development tools to a workstation and leave the server applications on one or more servers dedicated that purpose running optimized server OS(s).

/gustav

>>> jwcolby at colbyconsulting.com 28-09-2009 21:29 >>>

Again this is .. a SERVER OS. 




From nancy.lytle at gmail.com  Mon Sep 28 16:15:28 2009
From: nancy.lytle at gmail.com (Nancy Lytle)
Date: Mon, 28 Sep 2009 16:15:28 -0500
Subject: [dba-SQLServer] HELP, server completely unresponsive
In-Reply-To: <4AC11452.7070804@colbyconsulting.com>
References: <mailman.5.1253811601.3940.dba-sqlserver@databaseadvisors.com>
	<200909241755.n8OHtqPB016788@databaseadvisors.com>
	<BAY134-W21E3D5C110B0AC3F43259284D60@phx.gbl>
Message-ID: <BAY134-W13ED5EBAF0E671F63C081584D60@phx.gbl>


It is the Degree of Parallelism you are telling the SQL Server to use for queries, essentially, it is a setting.  It can be set using T-SQL or you can set it in SSMS under Properties, I think Advanced (I'm not at my server right now) and maybe you should throttle it down to 2, and see how that goes.  Of course you can check it out in BOL, but I know how tight you time is, I was given the tip by a someone else and it did work a charm on some of our servers (I manage 1900 servers of various DB sizes, all the same DB though and all with the same hardware/software config) that had the larger DB sizes.  Test it out, see how it goes.  It sounds counter intuitive, but it prevents one query from taking control of all of SQL Server and holding everything else hostage.

Nancy Lytle 
N_Lytle at terpalum.umd.edu









 EMAILING FOR THE GREATER GOOD
Join me
 
> Date: Mon, 28 Sep 2009 15:53:54 -0400
> From: jwcolby at colbyconsulting.com
> To: dba-sqlserver at databaseadvisors.com
> Subject: Re: [dba-SQLServer] HELP, server completely unresponsive
> 
> What is MAXDOP?
> 
> I have already told SQL Server to only use 12 gigs and 3 processors.
> 
> John W. Colby
> www.ColbyConsulting.com
> 
> 
> Nancy Lytle wrote:
> > What is your MAXDOP set to? I found that setting mine a little lower than expected actually helped.
> > 
> > We have 2 processors, it was set to use all. This would cause some queries to just suck the life out of it sometimes so I cut it back to 1 and it runs much better.
> > 
> > Nancy Lytle 
> > N_Lytle at terpalum.umd.edu
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > EMAILING FOR THE GREATER GOOD
> > Join me
> > 
> > 
> > 
> >> Date: Mon, 28 Sep 2009 11:19:09 -0400
> >> From: jwcolby at colbyconsulting.com
> >> To: dba-sqlserver at databaseadvisors.com; dba-vb at databaseadvisors.com; accessd at databaseadvisors.com
> >> Subject: Re: [dba-SQLServer] HELP, server completely unresponsive
> >>
> >> Well I thought that assigning three processors to the SQL Server software was the answer, and it in 
> >> fact did make a difference in some cases, but there is still something happening that "locks up" the 
> >> server. IO started a long running update query going this morning and immediately afterwards I was 
> >> able to move around, look at other stuff. Then I went away to do other things. When I came back it 
> >> was "locked up" - with the Management studio application full screen, and the cursor immovable. Of 
> >> course I thought it was locked up, and I have always believed that it was literally never going to 
> >> come back.
> >>
> >> I just happened to to an Alt-Tab, then went off to my lap top to do other stuff. When I came back I 
> >> was at Windows Explorer, not Management Studio. So the machine is in fact responding, but 
> >> glacially. The mouse cursor is locked up, cannot be moved. I use a KVM switch, and when I move to 
> >> another machine the cursor is moving just fine, when I move back to this machine it is locked up. 
> >> However the cursor changed from the insertion pointer icon to the arrow icon when i switched from 
> >> Management Studio to Windows Explorer, so again the machine is definitely not unresponsive, it is 
> >> just responding glacially.
> >>
> >> This time I have SQL Server assigned three processors, and I was observing that SQL Server pegged 
> >> three of the processors when it began processing the query so I am comfortable that it in fact is 
> >> correctly using just three processors.
> >>
> >> So what is happening that would so completely freeze up the server that the cursor won't even move?
> >>
> >> Weird.
> >>
> >> Has anyone else out there ever experienced this? Found a solution?
> >>
> >> John W. Colby
> >> www.ColbyConsulting.com
> >>
> >> _______________________________________________
> >> dba-SQLServer mailing list
> >> dba-SQLServer at databaseadvisors.com
> >> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> >> http://www.databaseadvisors.com
> >>
> > 
> > _______________________________________________
> > dba-SQLServer mailing list
> > dba-SQLServer at databaseadvisors.com
> > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> > http://www.databaseadvisors.com
> > 
> > 
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
 		 	   		  

From jwcolby at colbyconsulting.com  Mon Sep 28 20:05:27 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Mon, 28 Sep 2009 21:05:27 -0400
Subject: [dba-SQLServer] HELP, server completely unresponsive
In-Reply-To: <BAY134-W13ED5EBAF0E671F63C081584D60@phx.gbl>
References: <mailman.5.1253811601.3940.dba-sqlserver@databaseadvisors.com>	<200909241755.n8OHtqPB016788@databaseadvisors.com>	<BAY134-W21E3D5C110B0AC3F43259284D60@phx.gbl>
	<BAY134-W13ED5EBAF0E671F63C081584D60@phx.gbl>
Message-ID: <4AC15D57.3060603@colbyconsulting.com>

I'll give that a try.

Thanks!

John W. Colby
www.ColbyConsulting.com


Nancy Lytle wrote:
> It is the Degree of Parallelism you are telling the SQL Server to use for queries, essentially, it is a setting.  It can be set using T-SQL or you can set it in SSMS under Properties, I think Advanced (I'm not at my server right now) and maybe you should throttle it down to 2, and see how that goes.  Of course you can check it out in BOL, but I know how tight you time is, I was given the tip by a someone else and it did work a charm on some of our servers (I manage 1900 servers of various DB sizes, all the same DB though and all with the same hardware/software config) that had the larger DB sizes.  Test it out, see how it goes.  It sounds counter intuitive, but it prevents one query from taking control of all of SQL Server and holding everything else hostage.
> 
> Nancy Lytle 
> N_Lytle at terpalum.umd.edu
> 
> 
> 
> 
> 
> 
> 
> 
> 
>  EMAILING FOR THE GREATER GOOD
> Join me
>  
>> Date: Mon, 28 Sep 2009 15:53:54 -0400
>> From: jwcolby at colbyconsulting.com
>> To: dba-sqlserver at databaseadvisors.com
>> Subject: Re: [dba-SQLServer] HELP, server completely unresponsive
>>
>> What is MAXDOP?
>>
>> I have already told SQL Server to only use 12 gigs and 3 processors.
>>
>> John W. Colby
>> www.ColbyConsulting.com
>>
>>
>> Nancy Lytle wrote:
>>> What is your MAXDOP set to? I found that setting mine a little lower than expected actually helped.
>>>
>>> We have 2 processors, it was set to use all. This would cause some queries to just suck the life out of it sometimes so I cut it back to 1 and it runs much better.
>>>
>>> Nancy Lytle 
>>> N_Lytle at terpalum.umd.edu



From jwcolby at colbyconsulting.com  Mon Sep 28 20:22:40 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Mon, 28 Sep 2009 21:22:40 -0400
Subject: [dba-SQLServer] HELP, server completely unresponsive
In-Reply-To: <4AC15D57.3060603@colbyconsulting.com>
References: <mailman.5.1253811601.3940.dba-sqlserver@databaseadvisors.com>	<200909241755.n8OHtqPB016788@databaseadvisors.com>	<BAY134-W21E3D5C110B0AC3F43259284D60@phx.gbl>	<BAY134-W13ED5EBAF0E671F63C081584D60@phx.gbl>
	<4AC15D57.3060603@colbyconsulting.com>
Message-ID: <4AC16160.50208@colbyconsulting.com>

When browsing for network servers from another machine, I normally see about a half dozen servers 
which includes Azul (the SQL Server that is currently locked up).  ATM I can see all of the other 
servers but not AZUL so it is not answering up, at least not in time to be counted.

It is still "processing" the query I started this AM.  11:23:15 and counting.  All of this to set a 
field in 20 million records based on values in about 5 other fields, all of these other fields in an 
index.  The target field is not indexed (yet).  It should never take this long.  This operation 
normally takes about an hour.  But I know of no way to regain control of the server short of 
rebooting, and if I do that I will likely corrupt the database being written to.

John W. Colby
www.ColbyConsulting.com



From michael at ddisolutions.com.au  Mon Sep 28 22:13:21 2009
From: michael at ddisolutions.com.au (Michael Maddison)
Date: Tue, 29 Sep 2009 13:13:21 +1000
Subject: [dba-SQLServer] HELP, server completely unresponsive
References: <mailman.5.1253811601.3940.dba-sqlserver@databaseadvisors.com>	<200909241755.n8OHtqPB016788@databaseadvisors.com>	<BAY134-W21E3D5C110B0AC3F43259284D60@phx.gbl>	<BAY134-W13ED5EBAF0E671F63C081584D60@phx.gbl><4AC15D57.3060603@colbyconsulting.com>
	<4AC16160.50208@colbyconsulting.com>
Message-ID: <59A61174B1F5B54B97FD4ADDE71E7D01582476@ddi-01.DDI.local>

Hi John,

I havn't followed this thread closely but...
Is it possible the tempdb is running out of space?
Or maybe the log or the db itself?

If you can eventually see the server from another server try looking for
any blocking processes, might give you a clue.

Cheers

Michael M


-----Original Message-----
From: dba-sqlserver-bounces at databaseadvisors.com
[mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby
Sent: Tuesday, 29 September 2009 11:23 AM
To: Discussion concerning MS SQL Server
Subject: Re: [dba-SQLServer] HELP, server completely unresponsive

When browsing for network servers from another machine, I normally see
about a half dozen servers 
which includes Azul (the SQL Server that is currently locked up).  ATM I
can see all of the other 
servers but not AZUL so it is not answering up, at least not in time to
be counted.

It is still "processing" the query I started this AM.  11:23:15 and
counting.  All of this to set a 
field in 20 million records based on values in about 5 other fields, all
of these other fields in an 
index.  The target field is not indexed (yet).  It should never take
this long.  This operation 
normally takes about an hour.  But I know of no way to regain control of
the server short of 
rebooting, and if I do that I will likely corrupt the database being
written to.

John W. Colby
www.ColbyConsulting.com

_______________________________________________
dba-SQLServer mailing list
dba-SQLServer at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
http://www.databaseadvisors.com




From fhtapia at gmail.com  Tue Sep 29 00:31:47 2009
From: fhtapia at gmail.com (Francisco Tapia)
Date: Mon, 28 Sep 2009 22:31:47 -0700
Subject: [dba-SQLServer] HELP, server completely unresponsive
In-Reply-To: <4AC16160.50208@colbyconsulting.com>
References: <mailman.5.1253811601.3940.dba-sqlserver@databaseadvisors.com>	<200909241755.n8OHtqPB016788@databaseadvisors.com>	<BAY134-W21E3D5C110B0AC3F43259284D60@phx.gbl>	<BAY134-W13ED5EBAF0E671F63C081584D60@phx.gbl>
	<4AC15D57.3060603@colbyconsulting.com>
	<4AC16160.50208@colbyconsulting.com>
Message-ID: <8692A178-0797-4E57-841F-5F86B0B89FF0@gmail.com>

one thing you can do is to start the sql profiler and look at the sql  
starting statements so you can see what's causing the problem.

--
Francisco
On Sep 28, 2009, at 6:22 PM, jwcolby wrote:

> When browsing for network servers from another machine, I normally  
> see about a half dozen servers
> which includes Azul (the SQL Server that is currently locked up).   
> ATM I can see all of the other
> servers but not AZUL so it is not answering up, at least not in time  
> to be counted.
>
> It is still "processing" the query I started this AM.  11:23:15 and  
> counting.  All of this to set a
> field in 20 million records based on values in about 5 other fields,  
> all of these other fields in an
> index.  The target field is not indexed (yet).  It should never take  
> this long.  This operation
> normally takes about an hour.  But I know of no way to regain  
> control of the server short of
> rebooting, and if I do that I will likely corrupt the database being  
> written to.
>
> John W. Colby
> www.ColbyConsulting.com
>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>



From jwcolby at colbyconsulting.com  Tue Sep 29 11:29:35 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Tue, 29 Sep 2009 12:29:35 -0400
Subject: [dba-SQLServer] SQL Server speed issues - was server locking up.
Message-ID: <4AC235EF.2040002@colbyconsulting.com>

Well, it appears that I have issues with SQL Server, above and beyond the "locking up" issue

When I rebooted the server this morning (it never did finish this update process last night), my 
database being updated was NOT corrupted, indicating no writes were happening to it when I rebooted. 
  Furthermore it did not take off and try and do "gotta finish log file stuff" so it appears that it 
wasn't even doing that stuff any more.  Strange in itself.

I have spent a confusing morning trying to discover what exactly is going on.  The first thing that 
is going on is that just updating a single field To NULL in 21 million records is taking 24 minutes. 
  That is a million records / minute which is not stellar IMHO.  The field is not indexed so it is 
not an "updating the index" kind of problem.

I can tell you that I fed the "update" stored procedure a "Top() one million" kind of query and it 
took only 17 seconds to update one million records, that same field.  If you do the math, 17 seconds 
/ million times 21 million records is only about 6 minutes to update the field for every record.  So 
why does it take 24 minutes to just do a simple "set that field in every record to null"?

This just makes no sense to me, but I am not a SQL Server kind of guy.

-- 
John W. Colby
www.ColbyConsulting.com


From robert at webedb.com  Tue Sep 29 12:44:16 2009
From: robert at webedb.com (Robert Stewart)
Date: Tue, 29 Sep 2009 12:44:16 -0500
Subject: [dba-SQLServer] SQL Server speed issues - was server locking
In-Reply-To: <mailman.5.1254243601.26355.dba-sqlserver@databaseadvisors. com>
References: <mailman.5.1254243601.26355.dba-sqlserver@databaseadvisors.com>
Message-ID: <200909291744.n8THiauN005272@databaseadvisors.com>

How big is your log file?
Have you truncated it?
How big is your space for teh tempdb file?



At 12:00 PM 9/29/2009, you wrote:
>Date: Tue, 29 Sep 2009 12:29:35 -0400
>From: jwcolby <jwcolby at colbyconsulting.com>
>Subject: [dba-SQLServer] SQL Server speed issues - was server locking
>         up.
>To: Dba-Sqlserver <dba-sqlserver at databaseadvisors.com>, Access
>         Developers discussion and problem solving
>         <accessd at databaseadvisors.com>
>Message-ID: <4AC235EF.2040002 at colbyconsulting.com>
>Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
>Well, it appears that I have issues with SQL Server, above and 
>beyond the "locking up" issue
>
>When I rebooted the server this morning (it never did finish this 
>update process last night), my
>database being updated was NOT corrupted, indicating no writes were 
>happening to it when I rebooted.
>   Furthermore it did not take off and try and do "gotta finish log 
> file stuff" so it appears that it
>wasn't even doing that stuff any more.  Strange in itself.
>
>I have spent a confusing morning trying to discover what exactly is 
>going on.  The first thing that
>is going on is that just updating a single field To NULL in 21 
>million records is taking 24 minutes.
>   That is a million records / minute which is not stellar 
> IMHO.  The field is not indexed so it is
>not an "updating the index" kind of problem.
>
>I can tell you that I fed the "update" stored procedure a "Top() one 
>million" kind of query and it
>took only 17 seconds to update one million records, that same 
>field.  If you do the math, 17 seconds
>/ million times 21 million records is only about 6 minutes to update 
>the field for every record.  So
>why does it take 24 minutes to just do a simple "set that field in 
>every record to null"?
>
>This just makes no sense to me, but I am not a SQL Server kind of guy.


From fhtapia at gmail.com  Tue Sep 29 13:16:18 2009
From: fhtapia at gmail.com (Francisco Tapia)
Date: Tue, 29 Sep 2009 11:16:18 -0700
Subject: [dba-SQLServer] SQL Server speed issues - was server locking
In-Reply-To: <200909291744.n8THiauN005272@databaseadvisors.com>
References: <mailman.5.1254243601.26355.dba-sqlserver@databaseadvisors.com>
	<200909291744.n8THiauN005272@databaseadvisors.com>
Message-ID: <b874372a0909291116l1b68b31cj101e4f2862b29b3c@mail.gmail.com>

if your transaction log file is not big enough to house all the data, then
you will end up paging...that's normal, but this brings other issues as well
your log file will want to grow (if you've set it to autogrow) and also if
you're not in simple recovery model. and even in simple recovery model
you'll see the log file grow a good amount based on how much data you push
through your system.  page and working with only 1 million rows is different
than working on all 100 million rows, it's all about resources and how
you've allocated them.  a small tlog fle will require the server to stop
what it was doing in order to grow the tlog first then it will continue on
it's merry way...

I remember you were going to re-organize the location of the tempdb, and the
tlog files... how did that pan out and how much space did you allocate for
it?

--
-Francisco
http://sqlthis.blogspot.com | Tsql and More...


On Tue, Sep 29, 2009 at 10:44 AM, Robert Stewart <robert at webedb.com> wrote:

> How big is your log file?
> Have you truncated it?
> How big is your space for teh tempdb file?
>
>
>
> At 12:00 PM 9/29/2009, you wrote:
> >Date: Tue, 29 Sep 2009 12:29:35 -0400
> >From: jwcolby <jwcolby at colbyconsulting.com>
> >Subject: [dba-SQLServer] SQL Server speed issues - was server locking
> >         up.
> >To: Dba-Sqlserver <dba-sqlserver at databaseadvisors.com>, Access
> >         Developers discussion and problem solving
> >         <accessd at databaseadvisors.com>
> >Message-ID: <4AC235EF.2040002 at colbyconsulting.com>
> >Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> >
> >Well, it appears that I have issues with SQL Server, above and
> >beyond the "locking up" issue
> >
> >When I rebooted the server this morning (it never did finish this
> >update process last night), my
> >database being updated was NOT corrupted, indicating no writes were
> >happening to it when I rebooted.
> >   Furthermore it did not take off and try and do "gotta finish log
> > file stuff" so it appears that it
> >wasn't even doing that stuff any more.  Strange in itself.
> >
> >I have spent a confusing morning trying to discover what exactly is
> >going on.  The first thing that
> >is going on is that just updating a single field To NULL in 21
> >million records is taking 24 minutes.
> >   That is a million records / minute which is not stellar
> > IMHO.  The field is not indexed so it is
> >not an "updating the index" kind of problem.
> >
> >I can tell you that I fed the "update" stored procedure a "Top() one
> >million" kind of query and it
> >took only 17 seconds to update one million records, that same
> >field.  If you do the math, 17 seconds
> >/ million times 21 million records is only about 6 minutes to update
> >the field for every record.  So
> >why does it take 24 minutes to just do a simple "set that field in
> >every record to null"?
> >
> >This just makes no sense to me, but I am not a SQL Server kind of guy.
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>
>


From jwcolby at colbyconsulting.com  Tue Sep 29 15:55:06 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Tue, 29 Sep 2009 16:55:06 -0400
Subject: [dba-SQLServer] SQL Server speed issues - was server locking
In-Reply-To: <b874372a0909291116l1b68b31cj101e4f2862b29b3c@mail.gmail.com>
References: <mailman.5.1254243601.26355.dba-sqlserver@databaseadvisors.com>	<200909291744.n8THiauN005272@databaseadvisors.com>
	<b874372a0909291116l1b68b31cj101e4f2862b29b3c@mail.gmail.com>
Message-ID: <4AC2742A.1020808@colbyconsulting.com>

Francisco,

I did reorganize.  the temp db is out on a single drive IIRC.  The log files are on their own Raid5 
array.  The database itself is on a raid6 array.

The log files are HUGE, as in 45 GIGS.  And more importantly, one of them won't shrink down.  It 
says only 24% is available.  I shrank the log file and sure enough it is now 33 gigs and 0% free 
space.  So what is in the log file?

I have a database that is really nothing but the raw data imported from CSV files.  I created an 
index, and I decided to try updating this same field in it - just to see how the times compare, if 
it was something in one of the databases or is system wide etc.

20 million records, update a field to null, all records.  FOUR HOURS LATER... it still was not 
completed.  However the log file is enormous, 33 gigs.  It was 45 but I could shrink about 25% of 
it.  Is there any way to see what is in the log file that is so almighty important?  And is it 
trying desperately to apply whatever is in the log file to the database?

In the past I would get tremendous sized log files but I could ALWAYS shrink them to just a meg or 
so.  Not this time.

WTF over?

John W. Colby
www.ColbyConsulting.com


Francisco Tapia wrote:
> if your transaction log file is not big enough to house all the data, then
> you will end up paging...that's normal, but this brings other issues as well
> your log file will want to grow (if you've set it to autogrow) and also if
> you're not in simple recovery model. and even in simple recovery model
> you'll see the log file grow a good amount based on how much data you push
> through your system.  page and working with only 1 million rows is different
> than working on all 100 million rows, it's all about resources and how
> you've allocated them.  a small tlog fle will require the server to stop
> what it was doing in order to grow the tlog first then it will continue on
> it's merry way...
> 
> I remember you were going to re-organize the location of the tempdb, and the
> tlog files... how did that pan out and how much space did you allocate for
> it?
> 
> --
> -Francisco
> http://sqlthis.blogspot.com | Tsql and More...
> 
> 
> On Tue, Sep 29, 2009 at 10:44 AM, Robert Stewart <robert at webedb.com> wrote:
> 
>> How big is your log file?
>> Have you truncated it?
>> How big is your space for teh tempdb file?
>>
>>
>>
>> At 12:00 PM 9/29/2009, you wrote:
>>> Date: Tue, 29 Sep 2009 12:29:35 -0400
>>> From: jwcolby <jwcolby at colbyconsulting.com>
>>> Subject: [dba-SQLServer] SQL Server speed issues - was server locking
>>>         up.
>>> To: Dba-Sqlserver <dba-sqlserver at databaseadvisors.com>, Access
>>>         Developers discussion and problem solving
>>>         <accessd at databaseadvisors.com>
>>> Message-ID: <4AC235EF.2040002 at colbyconsulting.com>
>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>
>>> Well, it appears that I have issues with SQL Server, above and
>>> beyond the "locking up" issue
>>>
>>> When I rebooted the server this morning (it never did finish this
>>> update process last night), my
>>> database being updated was NOT corrupted, indicating no writes were
>>> happening to it when I rebooted.
>>>   Furthermore it did not take off and try and do "gotta finish log
>>> file stuff" so it appears that it
>>> wasn't even doing that stuff any more.  Strange in itself.
>>>
>>> I have spent a confusing morning trying to discover what exactly is
>>> going on.  The first thing that
>>> is going on is that just updating a single field To NULL in 21
>>> million records is taking 24 minutes.
>>>   That is a million records / minute which is not stellar
>>> IMHO.  The field is not indexed so it is
>>> not an "updating the index" kind of problem.
>>>
>>> I can tell you that I fed the "update" stored procedure a "Top() one
>>> million" kind of query and it
>>> took only 17 seconds to update one million records, that same
>>> field.  If you do the math, 17 seconds
>>> / million times 21 million records is only about 6 minutes to update
>>> the field for every record.  So
>>> why does it take 24 minutes to just do a simple "set that field in
>>> every record to null"?
>>>
>>> This just makes no sense to me, but I am not a SQL Server kind of guy.
>> _______________________________________________
>> dba-SQLServer mailing list
>> dba-SQLServer at databaseadvisors.com
>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>> http://www.databaseadvisors.com
>>
>>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
> 
> 


From jwcolby at colbyconsulting.com  Wed Sep 30 08:46:29 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Wed, 30 Sep 2009 09:46:29 -0400
Subject: [dba-SQLServer] SQL Server Speed Testing - Setting AddrValid to
	codes
In-Reply-To: <8692A178-0797-4E57-841F-5F86B0B89FF0@gmail.com>
References: <mailman.5.1253811601.3940.dba-sqlserver@databaseadvisors.com>	<200909241755.n8OHtqPB016788@databaseadvisors.com>	<BAY134-W21E3D5C110B0AC3F43259284D60@phx.gbl>	<BAY134-W13ED5EBAF0E671F63C081584D60@phx.gbl>	<4AC15D57.3060603@colbyconsulting.com>	<4AC16160.50208@colbyconsulting.com>
	<8692A178-0797-4E57-841F-5F86B0B89FF0@gmail.com>
Message-ID: <4AC36135.4090902@colbyconsulting.com>

I wrote this email yesterday in the middle of a very frustrating day.  After my testing this morning 
I now realize that there is this quite significant "overhead" for the first pass processing, 
overhead which is not a fixed amount but which increases with the size of the data set.  That 
overhead will likely explain the results I encountered and documented below.  I am sending this 
email because it explains in detail the process and the stored procedure and udf which actually does 
the processing.  I am about to do some testing similar to the last email where I do processing on 
specific size chunks of records in order to see numbers similar to the last email.

The idea is to update an AddrValid Char() field with various codes depending on how the address 
plays out.  PO if the address line has a "PO BOX" in it, "INV" (invalid) if it has a null in the 
zip4, "ANK" if it has a "77" in the ANK field and so forth.

These codes then tell me various things about the addresses and allow me to select the addresses for 
  ANK processing, filter out PO boxes etc.

Awhile back I created a udf into which I passed a set of fields which will be analyzed and which
returns a code up to three characters wide (or null).  I then have a stored procedure which is a
simple update, updating the AddrValid with the results passed back by this udf.

The table being updated is 20 million records (in this case).

I just went through the exercise of passing in a view where I selected the TOP() one million rows
where AddrValid IS NULL.  Processing one million rows takes 17 seconds.  I ran that about five
times, counting the remaining rows with AddrValid IS NULL.  Each time the time to process the
million rows was in fact about 17-18 seconds.  When I got down to 15 million rows, having
successfully processed about 6 million rows, I decided to pass in the table itself instead of the
view with the TOP(1000000) statement, figuring that it would take 15 (million rows remaining) x 17
seconds to complete, or 4.25 minutes.

WRONG.

11:40 later it is still processing.  Sigh.

However ATM I am not locked up!  BTW I manually set the IO affinity mask to the same three cores
which I assigned to SQL Server.  Previously I had told it to do the assignment itself.  I also set
Max Degree of Parallelism to 2 per Nancy Lytle's suggestion.

So as of right now, it is chunking away, 16:28 now.  But I am not locked up.  I am poking around
looking at stuff.  Windows task manager shows a single CPU processing... sometimes.  Mostly it is
flatlining, but it does occasionally spring to life.

It is interesting to me that when I move from "SELECT TOP(1000000) * FROM tblX) as a saved query
passed in to the stored procedure to just passing in tblX itself, I go from one million records
every 17 seconds to "we ain't finishing for some reason".  There are only 20 million records in the
table.  It should have been a quick and easy 4:30 task to finish up the 15 million records where
AddrValid is still null.

With the exception of the field being updated (AddrValid), all of the fields used in the stored
procedure are contained in an index.

The SP itself looks like this:

BEGIN TRY	
   SELECT @SQL = 'UPDATE [' + @DBName + '].[dbo].[' + @TblName + ']
   SET [AddrValid] = _DataMaster.dbo.udf_AddrValid(ank_, Addr, Zip4, dpv_)
   WHERE     (AddrValid IS NULL)'

   exec (@SQL)
   print 'success UPDATING AddrValid on ' + @DBName + '.' + @TblName
END TRY
BEGIN CATCH
   print 'There was an error UPDATING AddrValid ANK '
   print ERROR_MESSAGE()	
END CATCH

And the udf looks like this:

ALTER FUNCTION [dbo].[udf_AddrValid]
(
	-- Add the parameters for the function here
	@ANK varchar(50), @Address varchar(50), @Zip4 varchar(50), @dpv varchar(50)
)
RETURNS char(3)
AS
BEGIN
-- Declare the return variable here
DECLARE @Return char(3)
-- Add the T-SQL statements to compute the return value here
IF @ANK = '77'
SELECT @Return = 'ANK'
ELSE
IF left(@Address,5) = 'MOVED'
SELECT @Return = 'MOV'
ELSE	
if @Zip4 is null
SELECT @Return = 'INV'
else
IF left(@Address,2) = 'PO'
SELECT @Return = 'PO'
ELSE	
IF (@dpv = 'YN')
SELECT @Return = 'V'
ELSE
IF ((@dpv = 'DN') OR (@dpv = 'SN'))
SELECT @Return = 'E'
ELSE
SELECT @Return = 'INV'
		

-- Return the result of the function
RETURN @Return

END


WELL... it just finished.  26:09 to process a 20 million record table.

John W. Colby
www.ColbyConsulting.com



From robert at webedb.com  Wed Sep 30 13:18:27 2009
From: robert at webedb.com (Robert Stewart)
Date: Wed, 30 Sep 2009 13:18:27 -0500
Subject: [dba-SQLServer] SQL Server speed issues - was server  locking
In-Reply-To: <mailman.1.1254330001.2535.dba-sqlserver@databaseadvisors.c
 om>
References: <mailman.1.1254330001.2535.dba-sqlserver@databaseadvisors.com>
Message-ID: <200909301818.n8UIIjhW022522@databaseadvisors.com>

You never answered the questions below...

You said you were shrinking the log files. What you need to do is 
TRUNCATE them.
Do a Books-on-line look up of TRUNCATE LOG.



At 12:00 PM 9/30/2009, you wrote:
>Date: Tue, 29 Sep 2009 12:44:16 -0500
>From: Robert Stewart <robert at webedb.com>
>Subject: Re: [dba-SQLServer] SQL Server speed issues - was server
>         locking
>To: dba-sqlserver at databaseadvisors.com
>Message-ID: <200909291744.n8THiauN005272 at databaseadvisors.com>
>Content-Type: text/plain; charset="us-ascii"; format=flowed
>
>How big is your log file?
>Have you truncated it?
>How big is your space for teh tempdb file?
>
>
>
>At 12:00 PM 9/29/2009, you wrote:
> >Date: Tue, 29 Sep 2009 12:29:35 -0400
> >From: jwcolby <jwcolby at colbyconsulting.com>
> >Subject: [dba-SQLServer] SQL Server speed issues - was server locking
> >         up.
> >To: Dba-Sqlserver <dba-sqlserver at databaseadvisors.com>, Access
> >         Developers discussion and problem solving
> >         <accessd at databaseadvisors.com>
> >Message-ID: <4AC235EF.2040002 at colbyconsulting.com>
> >Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> >
> >Well, it appears that I have issues with SQL Server, above and
> >beyond the "locking up" issue
> >
> >When I rebooted the server this morning (it never did finish this
> >update process last night), my
> >database being updated was NOT corrupted, indicating no writes were
> >happening to it when I rebooted.
> >   Furthermore it did not take off and try and do "gotta finish log
> > file stuff" so it appears that it
> >wasn't even doing that stuff any more.  Strange in itself.
> >
> >I have spent a confusing morning trying to discover what exactly is
> >going on.  The first thing that
> >is going on is that just updating a single field To NULL in 21
> >million records is taking 24 minutes.
> >   That is a million records / minute which is not stellar
> > IMHO.  The field is not indexed so it is
> >not an "updating the index" kind of problem.
> >
> >I can tell you that I fed the "update" stored procedure a "Top() one
> >million" kind of query and it
> >took only 17 seconds to update one million records, that same
> >field.  If you do the math, 17 seconds
> >/ million times 21 million records is only about 6 minutes to update
> >the field for every record.  So
> >why does it take 24 minutes to just do a simple "set that field in
> >every record to null"?
> >
> >This just makes no sense to me, but I am not a SQL Server kind of guy.


From jwcolby at colbyconsulting.com  Wed Sep 30 14:14:14 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Wed, 30 Sep 2009 15:14:14 -0400
Subject: [dba-SQLServer] SQL Server speed issues - was server  locking
In-Reply-To: <200909301818.n8UIIjhW022522@databaseadvisors.com>
References: <mailman.1.1254330001.2535.dba-sqlserver@databaseadvisors.com>
	<200909301818.n8UIIjhW022522@databaseadvisors.com>
Message-ID: <4AC3AE06.7080407@colbyconsulting.com>

Sorry Robert, I didn't answer the questions.

The log files get huge pretty quickly.  In use I don't really care as that is what they are for. 
However as I have mentioned, the application is really static data.  I get tables of addresses, send 
them out for address validation, pull the resulting data (includes but more than the original 
addresses), process the data and then... just use the data for "quite awhile", then send out for 
address validation again, process the results and then... just use the data for "quite awhile". 
There are zero "transactions" in the normal sense.  The resulting data are read-only for weeks or 
months at a time.

So under normal circumstances I want to use the log files, then "do whatever" to get them to as 
close as possible to zero bytes.  Truncate, shrink, doesn't matter to me as long as it works.

In the past the log files were always 99% available and I just used the shrink wizard.  I do need to 
automate this process so that when the address validation process finishes, it TRUNCATES the log 
file as the final process.  The log file will not be used again for weeks or more so why leave it huge.

 >> How big is your log file?

That depends on what is going on, how many records in the table being processed etc.  For this 
specific process for this specific database, for 20 million records, about 15 GB when it is all 
finished.

 >> Have you truncated it?

Not in so many words.  I have never run any code with the TRUNCATE keyword.  I have always used the 
"shrink files" dialog from inside of SS.  My understanding is that the wizard performed a TRUNCATE 
behind the scenes but who am I to say?  I will almost certainly do so eventually.

 >> How big is your space for the tempdb file?

The temp db runs on a separate raid0 array, two 320g drives, apparently waaaayyyy to big for the 
requirements.  The array is 600 gb and the temp db (after all this horsing around) is currently 640 
mb.  The temp db is the only thing on that array.

My log files run on another array.  Raid1, using two 500gb drives for a total of 500gb available. 
That array is used only for the log files, though all of the log files are on that one array.  At 
this instant the total space used for logs is 16 GB, though I have seen a single log file get as big 
as 100 GB under heaving processing as I am setting up a large (100 million record) database.  When 
done I just shrink the log back down though.

The main DATA array is a raid6 array of six 1 tb drives, divided into two 2 tb volumes.  All the 
data files are on one of the two volumes.  The other volume is used for general data storage.  Given 
that both of the two terabyte volumes are on the same array (set of spindles) my understanding is 
that it doesn't matter if they are all on the same volume.

The system drive is on its own array, Raid1, using two 1 tb drives.  That array is divided into two 
volumes, 300g for boot (C:) and 700g for data.  The main SQL Server installation is on that C: 
drive, with the master database etc on that volume.

And then there is a 1 tb backup drive sitting on the end of an ESATA cable in an external enclosure.

All of the RAID arrays are on a single ARECA controller card with 2 gigs of RAM onboard.  One thing 
I want to try is to move the raid arrays for the logs and tempdb off to a second controller card 
which I have sitting on the shelf.  That would put it on a separate co-processor, and a separate 
PCI-Express x8 interface to the processor.  Unfortunately that second controller has only a fixed 
256 MB of memory available to it, and I don't know whether to put the data or the logs on the bigger 
card.  And of course just touching stuff that is working is always a risk.

John W. Colby
www.ColbyConsulting.com


Robert Stewart wrote:
> You never answered the questions below...
> 
> You said you were shrinking the log files. What you need to do is 
> TRUNCATE them.
> Do a Books-on-line look up of TRUNCATE LOG.
> 
> 
> 
> At 12:00 PM 9/30/2009, you wrote:
>> Date: Tue, 29 Sep 2009 12:44:16 -0500
>> From: Robert Stewart <robert at webedb.com>
>> Subject: Re: [dba-SQLServer] SQL Server speed issues - was server
>>         locking
>> To: dba-sqlserver at databaseadvisors.com
>> Message-ID: <200909291744.n8THiauN005272 at databaseadvisors.com>
>> Content-Type: text/plain; charset="us-ascii"; format=flowed
>>
>> How big is your log file?
>> Have you truncated it?
>> How big is your space for teh tempdb file?



From jwcolby at colbyconsulting.com  Wed Sep 30 21:38:56 2009
From: jwcolby at colbyconsulting.com (jwcolby)
Date: Wed, 30 Sep 2009 22:38:56 -0400
Subject: [dba-SQLServer] SQL Server Speed testing - update to real codes.
Message-ID: <4AC41640.4000002@colbyconsulting.com>

OK so this pass will use my stored procedure / udf to perform actual updates of the AddrValid field 
to the appropriate code, dependent upon the address data in other fields of each record.

BTW I went to three as the Max Degree of Parallelism.  Having looked this up it seems that this is 
the maximum number of CPUs (or cores) that SQL Server will use for queries that can be processed in 
parallel. I have not experienced the "lockup" issue since I went from 0 (all available cores) to 
<=3.  However I am not yet ready to declare the problem solved.

So, on to numbers:

500K	8 seconds
500K	8 seconds
1M	16 seconds
1M	16 seconds
2M	33 seconds
2M	35 seconds
4M	68 seconds
4M	77 seconds

So it appears that using the stored procedure takes roughly the "worst case" time from the previous 
timings.  The first set of a direct update of the field to NULL took approximately the same as my 
results here, except that I am not getting an advantage the second time.  This would tend to 
indicate that it is the "compile" time or whatever that is called (execution plan stuff), that is 
taking the extra time, and given that I am running dynamic SQL it is not possible to generate an 
execution plan that can be used the next time.

Maybe.  I am so uneducated in SQL Server that I probably sound like a rambling madman.  ;)

BTW, the log file now sits at 4.524 GB however it is 98% available.  And BTW I did not shrink before 
this set of runs so I did not have to endure as much log file expansion.

So one final run, just passing in the table directly to be updated, rather than a view of Top(N), 
but also no WHERE AddrValid IS NULL.  My expectations would be that it would take about 6 minutes, 
i.e. if 4 megs is ~70 seconds and there are 20M records to process - 70 seconds * 5 = 350 seconds or 
~ 6 minutes.  As you saw from my ramblings in yesterday's email it actually took some 24 minutes 
which is completely unexpected and unexplained.

And the results are in... (drum roll please)....

3:47.  The log file is now 4.524 GB and 98% available.

OK so I shrank the log file.  The results including having to perform a log file expansion...

3:45.  And the log file is not expanded.

OK, something wrong there.

I went back and updated all records to null.  18:54.  Didn't check log file size.  Sorry for the 
lack of science here.

Updated to valid codes:  10:44.  Log file size 17.18 GB, 84% available free space.

Updated back to NULL: 9:52 minutes.  Log file 17.18 GB. 84% free.

Update to valid codes: 9:58.  Log file 18.132.  49% Free.

So there you have it.  The computer is not locking up so far and the updates seem to be happening 
reasonably quickly.  So what was the real issue before?  Another MS mystery.


-- 
John W. Colby
www.ColbyConsulting.com