From jwcolby at colbyconsulting.com Mon Oct 4 13:39:43 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 04 Oct 2010 14:39:43 -0400 Subject: [dba-SQLServer] New Server Message-ID: <4CAA1F6F.3070606@colbyconsulting.com> Last week late and over the weekend I brought up the new server. That is a *lot* of work! The server consists of: 1 Norco 4020 case http://www.newegg.com/Product/Product.aspx?Item=N82E16811219021 1 Corsair 750W modular PS http://www.newegg.com/Product/Product.aspx?Item=N82E16817139010 1 Asus KGPE-D16 Dual Socket G34 motherboard http://www.newegg.com/Product/Product.aspx?Item=N82E16813131643 1 AMD Opteron 6128 8 core processor http://www.newegg.com/Product/Product.aspx?Item=N82E16819105266 2 Kingston 8GB 240-Pin DDR3 SDRAM http://www.newegg.com/Product/Product.aspx?Item=N82E16820139140 3 OCZ Vertex 2 OCZSSD2-2VTXE120G SSD http://www.newegg.com/Product/Product.aspx?Item=N82E16820227551 Windows *Server* 2008 Enterprise ;) SQL Server 2008 Enterprise Visual Studio 2008 Visual SVN One of the SSD drives is the boot drive, the other two will be for specific database files (raid 0). So essentially ATM the server is 8 cores and 16 gigs of RAM. Coming next another 8 core processor and another 16 gigs of RAM, to be followed by a final 32 gigs of RAM. Of course in typical fashion, not thinking about what I was doing, I moved the RAID controller and the drives over to the new server only to discover that I had not detached the databases and they would not attach. So I had to bring the old server back up, move the database files back over, attach and detach them, then move the files back to the new server, whereupon they all attached as expected. So as of this AM, the new server is up and functioning, with my SVN server / repository, and all databases functioning. I am planning on moving a couple of my main databases to the SSDs Raid 0 array. These are read-only databases, I do not write to them under normal circumstances. I will keep a current backup in case the Raid 0 array fails, but will work with them from the Raid 0 array on a daily basis. Having the main working databases on SSDs in a Raid 0 configuration, as well as more cores and more memory should allow me to do some of what I do in a much faster time frame. I do a lot of PK (autonumber) joins between tables, pulling multi-million record sets with data from each of the tables. I am hoping that this kind of processing will be much faster than when the source disks were on rotating media. We shall see. I still have the old server and will use it to run the test on rotating media, while doing an identical test on SSD on the new server. Of course I will not be testing just the effect of the SSD but rather the total speed increase of the entire system. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Mon Oct 4 15:20:08 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 04 Oct 2010 16:20:08 -0400 Subject: [dba-SQLServer] Where does it go? Message-ID: <4CAA36F8.5020402@colbyconsulting.com> I have code which mistakenly allowed me to start a query multiple times. Since fixed. However that does raise the question... what happens inside sql server when this occurs. the code in question was updating a specific field to null when that field was equal to ' ' (space). IOW I am cycling through a bunch of fields where I have ' ' representing no value, to get a consistent null to represent no value. What happens inside SQL Server if I start this same query multiple times on the same field? I use a command object to start the query running. I assume that if I were to set that command object to null back in C#, SQL Server is running the query and continues merrily on its way? So if i do this three times it??? Runs that same query three times? Sequentially (end to end?). Consecutively (simultaneously)? Is there something in SQL Server I can go to see processes or threads that are running and see the sql statement that represents the process? To see what SQL Statements are currently executing? -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Mon Oct 4 16:16:33 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 04 Oct 2010 17:16:33 -0400 Subject: [dba-SQLServer] Apples to Oranges Message-ID: <4CAA4431.5020903@colbyconsulting.com> I On the new server, have the same database on my SSD (two drive) raid 0 and a rotating media (2 drive) Raid 0. There is a clustered index on the PK as the index key. There is no index on the FieldX, forcing a field scan. I did a simple count PK Group By FieldX on both database files. The SSD returned the counts in 1:31 The rotating media returned the counts in 8:58 -- John W. Colby www.ColbyConsulting.com From stuart at lexacorp.com.pg Mon Oct 4 16:32:17 2010 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Tue, 05 Oct 2010 07:32:17 +1000 Subject: [dba-SQLServer] Where does it go? In-Reply-To: <4CAA36F8.5020402@colbyconsulting.com> References: <4CAA36F8.5020402@colbyconsulting.com> Message-ID: <4CAA47E1.9244.215AF870@stuart.lexacorp.com.pg> In the Management Studio, In 2005, it's under Server - Management - Activity monitor. I don't have an instance of 2008 here but IIRC, it's in the same place. You can see details of all running processes -- Stuart On 4 Oct 2010 at 16:20, jwcolby wrote: > I have code which mistakenly allowed me to start a query multiple > times. Since fixed. However that does raise the question... what > happens inside sql server when this occurs. > > the code in question was updating a specific field to null when that > field was equal to ' ' (space). > IOW I am cycling through a bunch of fields where I have ' ' > representing no value, to get a > consistent null to represent no value. > > What happens inside SQL Server if I start this same query multiple > times on the same field? I use a command object to start the query > running. I assume that if I were to set that command object to null > back in C#, SQL Server is running the query and continues merrily on > its way? > > So if i do this three times it??? Runs that same query three times? > Sequentially (end to end?). Consecutively (simultaneously)? > > Is there something in SQL Server I can go to see processes or threads > that are running and see the sql statement that represents the > process? To see what SQL Statements are currently executing? > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Mon Oct 4 16:36:49 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 04 Oct 2010 17:36:49 -0400 Subject: [dba-SQLServer] [AccessD] Apples to Oranges In-Reply-To: <4CAA4431.5020903@colbyconsulting.com> References: <4CAA4431.5020903@colbyconsulting.com> Message-ID: <4CAA48F1.20008@colbyconsulting.com> With an index, both databases took 2 seconds to return the results. John W. Colby www.ColbyConsulting.com On 10/4/2010 5:16 PM, jwcolby wrote: > I On the new server, have the same database on my SSD (two drive) raid 0 and a rotating media (2 > drive) Raid 0. > > There is a clustered index on the PK as the index key. > There is no index on the FieldX, forcing a field scan. > > I did a simple count PK Group By FieldX on both database files. > > The SSD returned the counts in 1:31 > The rotating media returned the counts in 8:58 > From jwcolby at colbyconsulting.com Mon Oct 4 17:05:13 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 04 Oct 2010 18:05:13 -0400 Subject: [dba-SQLServer] [AccessD] Apples to Oranges In-Reply-To: <4CAA48F1.20008@colbyconsulting.com> References: <4CAA4431.5020903@colbyconsulting.com> <4CAA48F1.20008@colbyconsulting.com> Message-ID: <4CAA4F99.8000805@colbyconsulting.com> One pass of the query that updates the ' ' (space) to null value takes about 5 minutes on the SSD, whereas it takes about 30 minutes minutes on rotating media. I am not pursuing actually doing this on the SSD over nagging concerns about hot spot wear. However a actually need to do this for about 540 fields. At 1/2 hour / field... this will be running for the next month. John W. Colby www.ColbyConsulting.com On 10/4/2010 5:36 PM, jwcolby wrote: > With an index, both databases took 2 seconds to return the results. > > John W. Colby > www.ColbyConsulting.com > > On 10/4/2010 5:16 PM, jwcolby wrote: >> I On the new server, have the same database on my SSD (two drive) raid 0 and a rotating media (2 >> drive) Raid 0. >> >> There is a clustered index on the PK as the index key. >> There is no index on the FieldX, forcing a field scan. >> >> I did a simple count PK Group By FieldX on both database files. >> >> The SSD returned the counts in 1:31 >> The rotating media returned the counts in 8:58 >> From fhtapia at gmail.com Mon Oct 4 17:06:34 2010 From: fhtapia at gmail.com (Francisco Tapia) Date: Mon, 4 Oct 2010 15:06:34 -0700 Subject: [dba-SQLServer] Where does it go? In-Reply-To: <4CAA36F8.5020402@colbyconsulting.com> References: <4CAA36F8.5020402@colbyconsulting.com> Message-ID: <5BF1EC11-0F63-46B2-9A70-3246DD12627A@gmail.com> Check out the profiler, expand the tree section under the events so that you can capture the SQL statements starting and completed.... This will show you what is processing, there are also rpc batches events that might be useful, I'm away from my pc at the moment so I can't double check. Check out on google SQL server profiler so you can trace the event. Sent from my mobile On Oct 4, 2010, at 1:20 PM, jwcolby wrote: > I have code which mistakenly allowed me to start a query multiple times. Since fixed. However that > does raise the question... what happens inside sql server when this occurs. > > the code in question was updating a specific field to null when that field was equal to ' ' (space). > IOW I am cycling through a bunch of fields where I have ' ' representing no value, to get a > consistent null to represent no value. > > What happens inside SQL Server if I start this same query multiple times on the same field? I use a > command object to start the query running. I assume that if I were to set that command object to > null back in C#, SQL Server is running the query and continues merrily on its way? > > So if i do this three times it??? Runs that same query three times? Sequentially (end to end?). > Consecutively (simultaneously)? > > Is there something in SQL Server I can go to see processes or threads that are running and see the > sql statement that represents the process? To see what SQL Statements are currently executing? > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > From fhtapia at gmail.com Mon Oct 4 17:14:02 2010 From: fhtapia at gmail.com (Francisco Tapia) Date: Mon, 4 Oct 2010 15:14:02 -0700 Subject: [dba-SQLServer] Where does it go? In-Reply-To: <4CAA47E1.9244.215AF870@stuart.lexacorp.com.pg> References: <4CAA36F8.5020402@colbyconsulting.com> <4CAA47E1.9244.215AF870@stuart.lexacorp.com.pg> Message-ID: <4D0DDD0E-1C49-4D0E-9F5E-3BD70826B0D1@gmail.com> I know jc said monitor but it sounds like he wants to see each execution...activity monitor does show you what is running but more high-level as actual select statements or update statements are not completely visible. Sent from my mobile On Oct 4, 2010, at 2:32 PM, "Stuart McLachlan" wrote: > In the Management Studio, > In 2005, it's under > Server - Management - Activity monitor. > > I don't have an instance of 2008 here but IIRC, it's in the same place. > > You can see details of all running processes > > -- > Stuart > > > On 4 Oct 2010 at 16:20, jwcolby wrote: > >> I have code which mistakenly allowed me to start a query multiple >> times. Since fixed. However that does raise the question... what >> happens inside sql server when this occurs. >> >> the code in question was updating a specific field to null when that >> field was equal to ' ' (space). >> IOW I am cycling through a bunch of fields where I have ' ' >> representing no value, to get a >> consistent null to represent no value. >> >> What happens inside SQL Server if I start this same query multiple >> times on the same field? I use a command object to start the query >> running. I assume that if I were to set that command object to null >> back in C#, SQL Server is running the query and continues merrily on >> its way? >> >> So if i do this three times it??? Runs that same query three times? >> Sequentially (end to end?). Consecutively (simultaneously)? >> >> Is there something in SQL Server I can go to see processes or threads >> that are running and see the sql statement that represents the >> process? To see what SQL Statements are currently executing? >> >> -- >> John W. Colby >> www.ColbyConsulting.com >> _______________________________________________ >> dba-SQLServer mailing list >> dba-SQLServer at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >> http://www.databaseadvisors.com >> >> > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > From stuart at lexacorp.com.pg Mon Oct 4 17:39:23 2010 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Tue, 05 Oct 2010 08:39:23 +1000 Subject: [dba-SQLServer] [AccessD] Apples to Oranges In-Reply-To: <4CAA4F99.8000805@colbyconsulting.com> References: <4CAA4431.5020903@colbyconsulting.com>, <4CAA48F1.20008@colbyconsulting.com>, <4CAA4F99.8000805@colbyconsulting.com> Message-ID: <4CAA579B.32740.219869F9@stuart.lexacorp.com.pg> At 1/2 hour / field, it should only take about 11.25 days :-) But my question is, do you have to do it a single field at a time? can't you update all the fields in a single pass? Something like: Update mytable set field1 = case field1 when " " then Null else field1 end, field2 = case field2 when " " then Null else field2 end, field3 = case field3 when " " then Null else field3 end, ... -- Stuart On 4 Oct 2010 at 18:05, jwcolby wrote: > One pass of the query that updates the ' ' (space) to null value takes > about 5 minutes on the SSD, whereas it takes about 30 minutes minutes > on rotating media. > > I am not pursuing actually doing this on the SSD over nagging concerns > about hot spot wear. > > However a actually need to do this for about 540 fields. At 1/2 hour > / field... this will be running for the next month. > > John W. Colby > www.ColbyConsulting.com > > On 10/4/2010 5:36 PM, jwcolby wrote: > > With an index, both databases took 2 seconds to return the results. > > > > John W. Colby > > www.ColbyConsulting.com > > > > On 10/4/2010 5:16 PM, jwcolby wrote: > >> I On the new server, have the same database on my SSD (two drive) > >> raid 0 and a rotating media (2 drive) Raid 0. > >> > >> There is a clustered index on the PK as the index key. > >> There is no index on the FieldX, forcing a field scan. > >> > >> I did a simple count PK Group By FieldX on both database files. > >> > >> The SSD returned the counts in 1:31 > >> The rotating media returned the counts in 8:58 > >> > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Tue Oct 5 06:12:54 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Tue, 05 Oct 2010 07:12:54 -0400 Subject: [dba-SQLServer] [AccessD] Apples to Oranges In-Reply-To: <4CAA579B.32740.219869F9@stuart.lexacorp.com.pg> References: <4CAA4431.5020903@colbyconsulting.com>, <4CAA48F1.20008@colbyconsulting.com>, <4CAA4F99.8000805@colbyconsulting.com> <4CAA579B.32740.219869F9@stuart.lexacorp.com.pg> Message-ID: <4CAB0836.9030408@colbyconsulting.com> Stuart, Would this really work? Given that it is simultaneously updating 540+ fields for 50 million rows, would it ever finish? I could have my assistant dynamically create this sql statement if you think this is a viable alternative. John W. Colby www.ColbyConsulting.com On 10/4/2010 6:39 PM, Stuart McLachlan wrote: > At 1/2 hour / field, it should only take about 11.25 days :-) > > But my question is, do you have to do it a single field at a time? can't you > update all the fields in a single pass? Something like: > > Update mytable set > field1 = case field1 > when " " then Null > else field1 > end, > field2 = case field2 > when " " then Null > else field2 > end, > field3 = case field3 > when " " then Null > else field3 > end, > ... > From jwcolby at colbyconsulting.com Tue Oct 5 07:04:51 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Tue, 05 Oct 2010 08:04:51 -0400 Subject: [dba-SQLServer] Rename sql server instance Message-ID: <4CAB1463.7030407@colbyconsulting.com> I need to rename my old Azul Server to something else and I need to rename the SQL Server instance running on it to match the server name. I know how to rename the computer, how do I rename the SQL Server instance? TIA, -- John W. Colby www.ColbyConsulting.com From stuart at lexacorp.com.pg Tue Oct 5 07:14:01 2010 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Tue, 05 Oct 2010 22:14:01 +1000 Subject: [dba-SQLServer] [AccessD] Apples to Oranges In-Reply-To: <4CAB0836.9030408@colbyconsulting.com> References: <4CAA4431.5020903@colbyconsulting.com>, <4CAA579B.32740.219869F9@stuart.lexacorp.com.pg>, <4CAB0836.9030408@colbyconsulting.com> Message-ID: <4CAB1689.6576.108216A@stuart.lexacorp.com.pg> Don't know, but my gut feeling is that a single pass has got to be way more efficient that 540 of them. Why not try it on a subset of the records and see how long it takes. -- Stuart On 5 Oct 2010 at 7:12, jwcolby wrote: > Stuart, > > Would this really work? Given that it is simultaneously updating 540+ > fields for 50 million rows, would it ever finish? > > I could have my assistant dynamically create this sql statement if you > think this is a viable alternative. > > John W. Colby > www.ColbyConsulting.com > > On 10/4/2010 6:39 PM, Stuart McLachlan wrote: > > At 1/2 hour / field, it should only take about 11.25 days :-) > > > > But my question is, do you have to do it a single field at a time? > > can't you update all the fields in a single pass? Something like: > > > > Update mytable set > > field1 = case field1 > > when " " then Null > > else field1 > > end, > > field2 = case field2 > > when " " then Null > > else field2 > > end, > > field3 = case field3 > > when " " then Null > > else field3 > > end, > > ... > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Tue Oct 5 07:17:10 2010 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Tue, 05 Oct 2010 22:17:10 +1000 Subject: [dba-SQLServer] Rename sql server instance In-Reply-To: <4CAB1463.7030407@colbyconsulting.com> References: <4CAB1463.7030407@colbyconsulting.com> Message-ID: <4CAB1746.11390.10B036E@stuart.lexacorp.com.pg> In Management Studio, right click on the server and select Properties. On the General tab, type a new "Registered Server name". -- Stuart On 5 Oct 2010 at 8:04, jwcolby wrote: > I need to rename my old Azul Server to something else and I need to > rename the SQL Server instance running on it to match the server name. > I know how to rename the computer, how do I rename the SQL Server > instance? > > TIA, > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From paul.hartland at googlemail.com Tue Oct 5 07:21:02 2010 From: paul.hartland at googlemail.com (Paul Hartland) Date: Tue, 5 Oct 2010 13:21:02 +0100 Subject: [dba-SQLServer] Rename sql server instance In-Reply-To: <4CAB1463.7030407@colbyconsulting.com> References: <4CAB1463.7030407@colbyconsulting.com> Message-ID: John, Never actually done anything like that before, but found something that may be of use to you: http://technet.microsoft.com/en-us/library/ms143799.aspx Paul On 5 October 2010 13:04, jwcolby wrote: > I need to rename my old Azul Server to something else and I need to rename > the SQL Server instance > running on it to match the server name. I know how to rename the computer, > how do I rename the SQL > Server instance? > > TIA, > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > -- Paul Hartland paul.hartland at googlemail.com From jwcolby at colbyconsulting.com Tue Oct 5 07:22:55 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Tue, 05 Oct 2010 08:22:55 -0400 Subject: [dba-SQLServer] Rename sql server instance In-Reply-To: <4CAB1746.11390.10B036E@stuart.lexacorp.com.pg> References: <4CAB1463.7030407@colbyconsulting.com> <4CAB1746.11390.10B036E@stuart.lexacorp.com.pg> Message-ID: <4CAB189F.6000809@colbyconsulting.com> I think I did that. I'll try again. John W. Colby www.ColbyConsulting.com On 10/5/2010 8:17 AM, Stuart McLachlan wrote: > In Management Studio, right click on the server and select Properties. > On the General tab, type a new "Registered Server name". > From mikedorism at verizon.net Tue Oct 5 08:00:04 2010 From: mikedorism at verizon.net (Doris Manning) Date: Tue, 05 Oct 2010 09:00:04 -0400 Subject: [dba-SQLServer] Contract Management template Message-ID: Please forgive the cross post but.Does anyone happen to have a basic Contract Management template that they wouldn't mind sharing or know of a good low cost source for one? We need to be able to track Expiration date, Service Level, Terms, Contacts, Options on renewal, and Cost. Being able to handle or track Document Storage would also be a big plus. I would design it myself but I'm swamped with more requests than I can handle and am just looking for something I can put in place quickly and fix as we go. Back end needs to be SQL Server 2000. Front end can be either desktop or web-based. Thanks, Doris Manning Sr. Developer/Database Administrator Hargrove Inc. www.hargroveinc.com From jwcolby at colbyconsulting.com Tue Oct 5 11:21:33 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Tue, 05 Oct 2010 12:21:33 -0400 Subject: [dba-SQLServer] [AccessD] Apples to Oranges In-Reply-To: <4CAB1689.6576.108216A@stuart.lexacorp.com.pg> References: <4CAA4431.5020903@colbyconsulting.com>, <4CAA579B.32740.219869F9@stuart.lexacorp.com.pg>, <4CAB0836.9030408@colbyconsulting.com> <4CAB1689.6576.108216A@stuart.lexacorp.com.pg> Message-ID: <4CAB508D.1050401@colbyconsulting.com> Stuart, Your gut feeling is correct. We created a dynamic sql statement and updated 100 columns simultaneously using your method and it ran in 33 minutes. I am now going for broke and doing the rest of the columns. We have updated 160 total so far so the next pass will try to do all the remaining columns in one blast. Thanks for recommending this method, you have saved my poor disk drives weeks of punishment. John W. Colby www.ColbyConsulting.com On 10/5/2010 8:14 AM, Stuart McLachlan wrote: > Don't know, but my gut feeling is that a single pass has got to be way more efficient that 540 > of them. > > Why not try it on a subset of the records and see how long it takes. > > From jwcolby at colbyconsulting.com Wed Oct 6 06:49:58 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 06 Oct 2010 07:49:58 -0400 Subject: [dba-SQLServer] Update to null the efficient way In-Reply-To: <4CAB1689.6576.108216A@stuart.lexacorp.com.pg> References: <4CAA4431.5020903@colbyconsulting.com>, <4CAA579B.32740.219869F9@stuart.lexacorp.com.pg>, <4CAB0836.9030408@colbyconsulting.com> <4CAB1689.6576.108216A@stuart.lexacorp.com.pg> Message-ID: <4CAC6266.3010502@colbyconsulting.com> Stuart, We created a dynamic SQL statement to try and do it all in one shot and it timed out after an hour (no big surprise there) but I decided to just break it down into 100 field chunks and do it that way. Each chunk took under an hour. Interestingly, SQL Server threw an error each time, it appears to be related to the length of the string though not certain about that. The SQL statement looks as follows and every time SQL threw an error about the very first case (IBS_IRRITABLE_BOWEL_SYNDROME_IN_HH in this case) however it appears that SQL Server did in fact process the SQL statement, if I go look for spaces in that set of fields I don't find any. UPDATE _DataHSID_UpdSpaces.dbo.tblHSID SET IBS_IRRITABLE_BOWEL_SYNDROME_IN_HH = case IBS_IRRITABLE_BOWEL_SYNDROME_IN_HH when ' ' then NULL else IBS_IRRITABLE_BOWEL_SYNDROME_IN_HH end, IMPOTENCE_ED_IN_HH = case IMPOTENCE_ED_IN_HH when ' ' then NULL else IMPOTENCE_ED_IN_HH end, MENOPAUSE_IN_HH = case MENOPAUSE_IN_HH when ' ' then NULL else MENOPAUSE_IN_HH end, etc. Because of the error being thrown (and apparently unhandled in my code) I am just now processing the last set of fields, however I should be finished in another hour. MUCH better than 10 days. ;) We use SMO everywhere in our programming these days, to get database, table and field names from SQL Server. We use it to back up and restore databases, attach and detach databases etc. Quite a powerful library in C# / .NET. John W. Colby www.ColbyConsulting.com On 10/5/2010 8:14 AM, Stuart McLachlan wrote: > Don't know, but my gut feeling is that a single pass has got to be way more efficient that 540 > of them. > > Why not try it on a subset of the records and see how long it takes. > > From jwcolby at colbyconsulting.com Wed Oct 6 11:23:28 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 06 Oct 2010 12:23:28 -0400 Subject: [dba-SQLServer] Multi-core madness Message-ID: <4CACA280.9070302@colbyconsulting.com> I have never had more than 4 cores at my disposal. On the old (SQL) server I had two cores dedicated to SQL Server. On the new system I currently have 8 cores total and 6 of those dedicated to SQL Server. In the past I would do things like build a multi-field index on a 50 million record table and it would max out the two cores. I pretty much couldn't do anything else. Today I am building multi-field indexes on an "off-line" copy of my database from hell. Task manager tells me it is using about 40% of the total processor power, however the two cores dedicated to the system are not doing much. The other 6 cores are chugging away somewhere (visually) just under 50%. I needed to BCP (using the internal export wizard) about 30 million PKs and email addresses to a csv file. When I started that running, Task manager informed me that I was using just under 60% of the available processor power, but the first two cores (dedicated to Windows) started chugging away, presumably doing file IO and the like. My 6 SQL Server cores jumped up to around 65%. BTW, the export process ripped it out pretty darned fast. I didn't time it but the total took a minute or so. So I was able to get two tasks going, and still had plenty of horsepower left over. I then installed the 64 bit WinRar, which can use multiple threads, and had it compress the resulting text file as SQL Server continued building indexes. All very smooth. If I get no "bandwidth complaints", I will continue to post occasional emails regarding how long it takes to do stuff vs the old server. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Wed Oct 6 12:50:30 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 06 Oct 2010 13:50:30 -0400 Subject: [dba-SQLServer] NearLine and live database Message-ID: <4CACB6E6.3030801@colbyconsulting.com> I am trying to use commercial solid state disks http://www.newegg.com/Product/Product.aspx?Item=N82E16820227551 in Raid 0 to hold my database from hell as well as another database that contains related information. My theory is that under normal use these two databases are read only. As such I can do reads from them all day without worrying about hot spot wear on the SSDs. In theory placing these databases on SSD should provide an enormous performance boost vs rotating media since SSD streaming read values are much higher and the IOPS is enormously higher. My problem is that I know of no way to discover what the write pattern looks like on the disk as SQL Server does (for example) an index build on adjacent fields (let's say 6 adjacent fields) across 50 million records on a table with an existing Clustered Index. Given that MLC SSDs can only handle 10K writes before potentially wearing out the cells, it seems risky to try and do this kind of operations in place on the SSDs. The problem is that I do need to maintain and build new indexes on these databases. I also need to keep a modern backup so that if one of the SSDs die (killing the entire Raid 0 array) I can easily get them back from backup. I am able to do backups easily, however I am struggling with how to easily do maintenance work on the nearline copy of the database and then get that copied to the online location. Basically ATM I have two copies of the database in SQL Server - _DataHSID and _DataNearLine. "NearLine" is just part of the name to remind me that I need to do things like updates and index manipulations in that copy. _DataHSID is physically located on drive H: (the SSD array) whereas its log file is in F:\SQLServerLogFiles (rotating media) which is the default location for SQL Server to build log files. _DataHSIDNearLine is physically located on rotating media J:\ and its log file resides there too. So what I am trying to figure out is the easiest way to copy the NearLine version to the OnLine location. Since both are attached, it seems that I have to detach OnLine, maybe copy (or detach and move) NearLine to OnLine and reattach OnLine. I can use SMO to detach / attach objects programmatically, and C# can do the copy for me, so I can do all this using C# / .Net if that makes sense, and it appears to make sense (to me). I figure I need to 1) Detach both of the databases. 2) Delete both of the log files, OnLine and nearLine 3) Copy the NearLine database file to the SSD 4) Attach both OnLine and nearLine. BTW, HSID is currently about 50.5 million records, 560 fields, has a clustered index and a handful of multi-field non-clustered indexes on it. The file size for _DataHSID is about 47 gigs. It is set to grow by 500 megs at a shot, and has a mere 125 meg available free space after all of the operations done on it recently. It took 6 minutes and 30 seconds to manually copy from rotating to SSD. Another possibility would be to do a backup / restore directly to the SSD. The backup compresses down to about 7.7 gigs, and a backup / restore would no doubt be faster. I just thought I'd check with others out here to see if anyone is doing this kind of thing and has solved the problem. TIA, -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Wed Oct 6 13:03:44 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 06 Oct 2010 14:03:44 -0400 Subject: [dba-SQLServer] HSID database from hell - structure Message-ID: <4CACBA00.7000306@colbyconsulting.com> A while ago it occurred to me that if I have a PKID which is autoincrement and I use that as the key for a unique clustered index, and I insert data into the table in sorted order (on pkid) with the index built before I started, then the database table should grow with minimum fragmentation etc and minimum wasted space. It seems that is in fact the case. I did this, created the table as described. My free space is about 250 megs on a file size of 47.5 gigs, my clustered index has a total fragmentation of .3%, and one of the several indexes (selected at random) I created after creating the table has a total fragmentation of .01%. It appears that as I create new indexes it adds size to the file at the end and stores the new index with virtually no fragmentation and maximum page fullness. So for this specific database / table it seems I have achieved Nirvana. -- John W. Colby www.ColbyConsulting.com From fhtapia at gmail.com Wed Oct 6 13:18:46 2010 From: fhtapia at gmail.com (Francisco Tapia) Date: Wed, 6 Oct 2010 11:18:46 -0700 Subject: [dba-SQLServer] NearLine and live database In-Reply-To: <4CACB6E6.3030801@colbyconsulting.com> References: <4CACB6E6.3030801@colbyconsulting.com> Message-ID: In our environment I have a job schedule to do the backup and restore solution. Off the top of my head I'm not sure why you would want to bother with the detach / attach situation. The backup / restore solution works very well for me, We use red-gate's sql backup which compresses our 1.7 terabyte database down to 282gb, Then we restore the database to our Testing servers. Time is a difficult thing here because for a full backup over a 1gb network, this can take close to 14hrs, So to help minimize data traffic we take a monthly FULL backup with weekly differential backups. This speeds up the entire process by a lot by simply reducing total backup time down to 12 minutes on the restore side, it can take up to 20 hrs to restore the entire contents simply because of the problem with the slow disks. The purpose for doing it this way is that the production database can remain online and continue to get used while the system backs up. We can't afford to have the production system offline for too long and thus we think this is the best option at the moment.. We recently have begun to test filegroup backups. This enables us to take the filegroup backups and restore them onto the testing system and allows you at the testing system to begin logging in and using the new data as soon as possible while the system is still restoring!... very wild... but a few more things need to fall in place before we roll out this solution. Time out your Backup/Restore vs the Copy of files... that should yield what is best for your situation. Everybody here has different hardware and it'd be kinda cool to hear how others have solved their data moving projects. -Francisco http://bit.ly/sqlthis | Tsql and More... On Wed, Oct 6, 2010 at 10:50 AM, jwcolby wrote: > I am trying to use commercial solid state disks > > http://www.newegg.com/Product/Product.aspx?Item=N82E16820227551 > > in Raid 0 to hold my database from hell as well as another database that > contains related > information. My theory is that under normal use these two databases are > read only. As such I can > do reads from them all day without worrying about hot spot wear on the > SSDs. > > In theory placing these databases on SSD should provide an enormous > performance boost vs rotating > media since SSD streaming read values are much higher and the IOPS is > enormously higher. My problem > is that I know of no way to discover what the write pattern looks like on > the disk as SQL Server > does (for example) an index build on adjacent fields (let's say 6 adjacent > fields) across 50 million > records on a table with an existing Clustered Index. Given that MLC SSDs > can only handle 10K writes > before potentially wearing out the cells, it seems risky to try and do this > kind of operations in > place on the SSDs. > > The problem is that I do need to maintain and build new indexes on these > databases. I also need to > keep a modern backup so that if one of the SSDs die (killing the entire > Raid 0 array) I can easily > get them back from backup. > > I am able to do backups easily, however I am struggling with how to easily > do maintenance work on > the nearline copy of the database and then get that copied to the online > location. > > Basically ATM I have two copies of the database in SQL Server - _DataHSID > and _DataNearLine. > "NearLine" is just part of the name to remind me that I need to do things > like updates and index > manipulations in that copy. > > _DataHSID is physically located on drive H: (the SSD array) whereas its log > file is in > F:\SQLServerLogFiles (rotating media) which is the default location for SQL > Server to build log files. > > _DataHSIDNearLine is physically located on rotating media J:\ and its log > file resides there too. > > So what I am trying to figure out is the easiest way to copy the NearLine > version to the OnLine > location. Since both are attached, it seems that I have to detach OnLine, > maybe copy (or detach and > move) NearLine to OnLine and reattach OnLine. > > I can use SMO to detach / attach objects programmatically, and C# can do > the copy for me, so I can > do all this using C# / .Net if that makes sense, and it appears to make > sense (to me). > > I figure I need to > 1) Detach both of the databases. > 2) Delete both of the log files, OnLine and nearLine > 3) Copy the NearLine database file to the SSD > 4) Attach both OnLine and nearLine. > > > BTW, HSID is currently about 50.5 million records, 560 fields, has a > clustered index and a handful > of multi-field non-clustered indexes on it. The file size for _DataHSID is > about 47 gigs. It is > set to grow by 500 megs at a shot, and has a mere 125 meg available free > space after all of the > operations done on it recently. It took 6 minutes and 30 seconds to > manually copy from rotating to SSD. > > Another possibility would be to do a backup / restore directly to the SSD. > The backup compresses > down to about 7.7 gigs, and a backup / restore would no doubt be faster. > > I just thought I'd check with others out here to see if anyone is doing > this kind of thing and has > solved the problem. > > TIA, > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Wed Oct 6 21:20:14 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 06 Oct 2010 22:20:14 -0400 Subject: [dba-SQLServer] Apples to oranges - take 2 Message-ID: <4CAD2E5E.20101@colbyconsulting.com> OK, so I have two databases, each with a single table. BTW, these are two of my main tables, used all of the time in orders. HSID - otherwise known as the database from hell, 51157068 records, ~560 fields. HSIDAllAdults is a database where up to three adult names were lifted out of fields in HSID and placed in a table with a PK_HSID field pointing back to the HSID record from which the information came. Thus HSIDAllAdults is child to HSID in a manner of speaking (has a FK back to the PKID from HSID). HSIDAllAdults has about 67564677 million records, 27 fields. Each table has a PKID which is autonumber and a unique clustered index on the PKID. HSID is demographic information such as income bracket, ChildInAgeGroup_XXX, HasDog etc. and has a handful of indexes on the most commonly used demographics fields. HSIAllAdults has name / address / gender / position (in HSID) fields and has three indexes on it - NameAddr, Hash and one other. So... I have a copy of each of these databases on rotating media. I then backed up the rotating media file and restored on the SSD, so I have a copy of each database in two places. I do this BTW because the SSD is a Raid 0 as well as SSD, and I am worried that if I do too much writing on the SSD I will wear it out - as in hot spot updates due to index updates etc. Thus I will be doing maintenance on the rotating media and just copying the resulting db out to SSD for every day use. Anyway, this allows me to do A/B comparisons of common queries. For the purpose of this test / email, I joined HSID to HSIDAllAdults on the FK in HSIDAllAdults, then did a count of the PK in HSIDAllAdults Group By MOB (mail order buyer, one of the demographics fields in HSID). So the SSD query looks like: SELECT _DataHSID.dbo.tblHSID.Mail_Order_BUYER, COUNT(dbo.tblAllAdultNameAddr.PK) AS Cnt FROM dbo.tblAllAdultNameAddr INNER JOIN _DataHSID.dbo.tblHSID ON dbo.tblAllAdultNameAddr.PKHSID = _DataHSID.dbo.tblHSID.PKID GROUP BY _DataHSID.dbo.tblHSID.Mail_Order_BUYER And runs in 30 seconds, producing the following results: NULL 19702461 1 19422841 2 28439375 The rotating media query looks as follows: SELECT _DataHSID_OffLine.dbo.tblHSID.Mail_Order_BUYER, COUNT(dbo.tblAllAdultNameAddr.PK) AS Cnt FROM dbo.tblAllAdultNameAddr INNER JOIN _DataHSID_OffLine.dbo.tblHSID ON dbo.tblAllAdultNameAddr.PKHSID = _DataHSID_OffLine.dbo.tblHSID.PKID GROUP BY _DataHSID_OffLine.dbo.tblHSID.Mail_Order_BUYER And runs in 1:50, producing the following results: NULL 19702461 1 19422841 2 28439375 The resulting count is identical (as expected), with rotating media taking almost 4 times as long to complete as the SSD. I will be storing these two queries in their respective databases (rotating / SSD) so that I can use them to test again when I add the second physical CPU chip and additional memory. BTW this was a pretty simple query as things go. A more normal query is to pull Name / address and a ValidAddress field out of HSIDAllAdults, filter the ValidAddr using something like In('V','E'), joining that to HSID and pulling out typically 4 to 6 fields from HSID to use in where clauses. So I am typically joining two tables of 50 million and 65 million records and then filtering on 4-8 fields, then actually capturing the resulting names / addresses and writing these into an order table. Depending on the criteria, I will pull anywhere from a 100 K or so up to 5 million or more names into the order table. The order table is created on the fly in an order database created just for that order. The order table will be on rotating media. Or I might eventually go buy another SSD to use specifically for building these order databases. Then if the SSD wears out I can just replace it with another. I currently spend a lot of time, hours at a time building these orders, running the queries, and manipulating the results to get the final export file. I am hoping to radically reduce my time twiddling my thumbs waiting for SQL Server. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Wed Oct 6 21:33:34 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 06 Oct 2010 22:33:34 -0400 Subject: [dba-SQLServer] Apples to oranges take 2a Message-ID: <4CAD317E.6040700@colbyconsulting.com> Same two files, rotating / SSD. I was just curious whether I had any HSIDAllAdult records no longer found in HSID. So this is an outer join where HSID is null, pulling the PK from HSIDAllAdult. The results BTW were an empty set (no records found). The query: SELECT dbo.tblAllAdultNameAddr.PK FROM dbo.tblAllAdultNameAddr LEFT OUTER JOIN _DataHSID_OffLine.dbo.tblHSID ON dbo.tblAllAdultNameAddr.PKHSID = _DataHSID_OffLine.dbo.tblHSID.PKID WHERE (_DataHSID_OffLine.dbo.tblHSID.PKID IS NULL) and a similar one for SSD The SSD finished in 44 seconds. The rotating media in 1:42 -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Thu Oct 7 14:53:12 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 07 Oct 2010 15:53:12 -0400 Subject: [dba-SQLServer] Apples to Oranges - Take 3 Message-ID: <4CAE2528.9@colbyconsulting.com> One thing I do a lot is to export large csv files. Today I have to export ~51 million records to CSV, PK and email fields. I have a table which holds these and two other fields, with a clustered index on PK and a non-clustered index on the email. I exported from rotating media to rotating media and (very rough) got about 10 million records per 45 seconds. I then exported the same job from rotating media to SSD and got roughly the same performance. I then backed up and restored to the SSD (the log file to SSD as well). I then exported the same job from SSD to SSD and got roughly the same performance. I then exported SSD to rotating media and got roughly the same performance. So there ya have it, SSD apparently makes no difference no matter how you play it in this specific task. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Wed Oct 13 21:43:15 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 13 Oct 2010 22:43:15 -0400 Subject: [dba-SQLServer] SSD, Raid 0 and (apparent) bandwidth Message-ID: <4CB66E43.3080505@colbyconsulting.com> I am pulling data out of HSID (the database from hell) on database _DataHSID and writing it into tblHSID in _DataMergePurge. Basically this is an inner join between a PK table in _DataMergePurge and HSID to select the records, then write them into the table in _DataMergePurge. I have both databases on the SSD, along with their respective log files. My temp files are on another 30g SSD. The database SSDs are a pair of disks, raid 0. I don't really have available separate SSDs for the log and data files, and I figured (though by no means certain) that the SSD would be faster even reading / writing both log and data than having the log on rotating media. Disk reads / writes are in Meg BYTES / minute (not second). My read volume is running pretty consistent between 400 and 450 mB/Minute out of _DataHSID and around 3 million bytes / minute out of _DataMergePurge. Response time 1 ms in both cases. All data from Resource Monitor. CPU is running around 25% average with most of that on the 6 cores dedicated to SQL Server. Suddenly a burst of writes to the log file at about 1.8 GBytes / min. 1 hard fault / minute every once in awhile, mostly 0. 14.8 Gig memory private to SQL Server's PID 37 minutes in and it hasn't even begun to write to the destination table. TempDb is up to about 5 gigs. MergePurge_Log about 1.1G atm. Another burst write to tempdb at around 1.8G / Min. Tempdb is a single SSD directly on an SATA port on the motherboard. At about 55 minutes in, data starts to write to the destination table. Fairly massive writes to both the log and data file for mergePurge - anywhere from 500 MBytes / minute up to 1 GByte / minute. Writing about 1.2 GBytes / minute between the log and the data file. Interestingly it is not *reading* at all (right now), it must be dumping from memory? About 930 MBytes / Minute to log, 333 MBytes / min to data. At 1 hour and 10 minutes it is finished. 4890683 row(s) affected (copied from the database from hell into the MP database). BTW the destination table has an existing clustered index (PK) on the People hash and PKID - two fields as the key. The MP data file is about 4.5 gb with 0 free. The MP log file is about 6 gigs with 1.6 gigs free. The tempdb file is about 5 gigs. That was interesting to watch. I sure wish I had more memory to see how that affects the system. ATM I am running 2 dims (channels) on a CPU socket that can use 4 channels, and "only" 16 gigs total available right now. I am supposed to get 64 gigs total and another CPU with 8 cores which would make it 4 dims of 8 gigs each per cpu (all 4 memory channels going). Who knows what that would do. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Thu Oct 14 15:03:47 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 14 Oct 2010 16:03:47 -0400 Subject: [dba-SQLServer] The SSD I am using Message-ID: <4CB76223.2050202@colbyconsulting.com> The following is the link to the SSD on Newegg - the model I purchased for my SQl Server. http://www.newegg.com/Product/Product.aspx?Item=N82E16820227551 This is a review that pretty much says it all. This thing is *fast*. http://benchmarkreviews.com/index.php?option=com_content&task=view&id=585&Itemid=60&limit=1&limitstart=11 -- John W. Colby www.ColbyConsulting.com From accessd at shaw.ca Thu Oct 14 16:27:02 2010 From: accessd at shaw.ca (Jim Lawrence) Date: Thu, 14 Oct 2010 14:27:02 -0700 Subject: [dba-SQLServer] The SSD I am using In-Reply-To: <4CB76223.2050202@colbyconsulting.com> References: <4CB76223.2050202@colbyconsulting.com> Message-ID: Boy, you are sure bragging a lot... not that you don't deserve to. ;-) Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, October 14, 2010 1:04 PM To: Access Developers discussion and problem solving; VBA; Sqlserver-Dba Subject: [dba-SQLServer] The SSD I am using The following is the link to the SSD on Newegg - the model I purchased for my SQl Server. http://www.newegg.com/Product/Product.aspx?Item=N82E16820227551 This is a review that pretty much says it all. This thing is *fast*. http://benchmarkreviews.com/index.php?option=com_content&task=view&id=585&It emid=60&limit=1&limitstart=11 -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Thu Oct 14 21:07:13 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 14 Oct 2010 22:07:13 -0400 Subject: [dba-SQLServer] The SSD I am using In-Reply-To: References: <4CB76223.2050202@colbyconsulting.com> Message-ID: <4CB7B751.6030607@colbyconsulting.com> Hmm... not my intent. I am building a server. Paid for by my client, not mine - though it does reside in my office. I thought there would be interest in how this stuff performs. When I started trying to discover how much this stuff would affect performance I didn't really find much information. I'll be quiet now. John W. Colby www.ColbyConsulting.com On 10/14/2010 5:27 PM, Jim Lawrence wrote: > Boy, you are sure bragging a lot... not that you don't deserve to. ;-) > > Jim > > > > -----Original Message----- > From: dba-sqlserver-bounces at databaseadvisors.com > [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, October 14, 2010 1:04 PM > To: Access Developers discussion and problem solving; VBA; Sqlserver-Dba > Subject: [dba-SQLServer] The SSD I am using > > The following is the link to the SSD on Newegg - the model I purchased for > my SQl Server. > > http://www.newegg.com/Product/Product.aspx?Item=N82E16820227551 > > This is a review that pretty much says it all. This thing is *fast*. > > http://benchmarkreviews.com/index.php?option=com_content&task=view&id=585&It > emid=60&limit=1&limitstart=11 > From jwcolby at colbyconsulting.com Thu Oct 14 22:10:18 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 14 Oct 2010 23:10:18 -0400 Subject: [dba-SQLServer] OK, since you don't want to read about how a modern server functions... Message-ID: <4CB7C61A.7030403@colbyconsulting.com> http://www.thetechrepository.com/showthread.php?t=126 ;) -- John W. Colby www.ColbyConsulting.com From accessd at shaw.ca Thu Oct 14 23:50:22 2010 From: accessd at shaw.ca (Jim Lawrence) Date: Thu, 14 Oct 2010 21:50:22 -0700 Subject: [dba-SQLServer] The SSD I am using In-Reply-To: <4CB7B751.6030607@colbyconsulting.com> References: <4CB76223.2050202@colbyconsulting.com> <4CB7B751.6030607@colbyconsulting.com> Message-ID: I was just fooling with you. ;-) Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, October 14, 2010 7:07 PM To: Discussion concerning MS SQL Server Subject: Re: [dba-SQLServer] The SSD I am using Hmm... not my intent. I am building a server. Paid for by my client, not mine - though it does reside in my office. I thought there would be interest in how this stuff performs. When I started trying to discover how much this stuff would affect performance I didn't really find much information. I'll be quiet now. John W. Colby www.ColbyConsulting.com On 10/14/2010 5:27 PM, Jim Lawrence wrote: > Boy, you are sure bragging a lot... not that you don't deserve to. ;-) > > Jim > > > > -----Original Message----- > From: dba-sqlserver-bounces at databaseadvisors.com > [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, October 14, 2010 1:04 PM > To: Access Developers discussion and problem solving; VBA; Sqlserver-Dba > Subject: [dba-SQLServer] The SSD I am using > > The following is the link to the SSD on Newegg - the model I purchased for > my SQl Server. > > http://www.newegg.com/Product/Product.aspx?Item=N82E16820227551 > > This is a review that pretty much says it all. This thing is *fast*. > > http://benchmarkreviews.com/index.php?option=com_content&task=view&id=585&It > emid=60&limit=1&limitstart=11 > _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From michael at mattysconsulting.com Fri Oct 15 07:41:47 2010 From: michael at mattysconsulting.com (Michael Mattys) Date: Fri, 15 Oct 2010 08:41:47 -0400 Subject: [dba-SQLServer] The SSD I am using In-Reply-To: <4CB7B751.6030607@colbyconsulting.com> References: <4CB76223.2050202@colbyconsulting.com> <4CB7B751.6030607@colbyconsulting.com> Message-ID: <7C68D71DF45D47A2AC9B57E180A9A0ED@Gateway> No, John, don't be quiet! Jim was kidding - he was saying you deserve to brag. My brother and I have to do much of what you're doing and I am interested in all your experiences in this and the other groups. Michael R Mattys Business Process Developers www.mattysconsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, October 14, 2010 10:07 PM To: Discussion concerning MS SQL Server Subject: Re: [dba-SQLServer] The SSD I am using Hmm... not my intent. I am building a server. Paid for by my client, not mine - though it does reside in my office. I thought there would be interest in how this stuff performs. When I started trying to discover how much this stuff would affect performance I didn't really find much information. I'll be quiet now. John W. Colby www.ColbyConsulting.com On 10/14/2010 5:27 PM, Jim Lawrence wrote: > Boy, you are sure bragging a lot... not that you don't deserve to. ;-) > > Jim From df.waters at comcast.net Fri Oct 15 07:46:21 2010 From: df.waters at comcast.net (Dan Waters) Date: Fri, 15 Oct 2010 07:46:21 -0500 Subject: [dba-SQLServer] The SSD I am using In-Reply-To: <7C68D71DF45D47A2AC9B57E180A9A0ED@Gateway> References: <4CB76223.2050202@colbyconsulting.com><4CB7B751.6030607@colbyconsulting.com> <7C68D71DF45D47A2AC9B57E180A9A0ED@Gateway> Message-ID: Seriously John - make some noise! Ya see, I thought you were just a guy who knew how to quickly process millions of records in SQL Server. ;-) Now I know better! :-) Keep going! Dan -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Michael Mattys Sent: Friday, October 15, 2010 7:42 AM To: 'Discussion concerning MS SQL Server' Subject: Re: [dba-SQLServer] The SSD I am using No, John, don't be quiet! Jim was kidding - he was saying you deserve to brag. My brother and I have to do much of what you're doing and I am interested in all your experiences in this and the other groups. Michael R Mattys Business Process Developers www.mattysconsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, October 14, 2010 10:07 PM To: Discussion concerning MS SQL Server Subject: Re: [dba-SQLServer] The SSD I am using Hmm... not my intent. I am building a server. Paid for by my client, not mine - though it does reside in my office. I thought there would be interest in how this stuff performs. When I started trying to discover how much this stuff would affect performance I didn't really find much information. I'll be quiet now. John W. Colby www.ColbyConsulting.com On 10/14/2010 5:27 PM, Jim Lawrence wrote: > Boy, you are sure bragging a lot... not that you don't deserve to. ;-) > > Jim _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Oct 15 08:44:35 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 15 Oct 2010 09:44:35 -0400 Subject: [dba-SQLServer] Powerful servers are not rocket science and pretty darned cheap In-Reply-To: References: <4CB76223.2050202@colbyconsulting.com><4CB7B751.6030607@colbyconsulting.com> <7C68D71DF45D47A2AC9B57E180A9A0ED@Gateway> Message-ID: <4CB85AC3.9080804@colbyconsulting.com> ROTFL. When I started this client / business (Aug 2004 according to my billing records) I built two servers. At that time the Athlon was king and what I could afford was the 3.8 GHz. So they were single core machines, 4 gigs ram, running Windows 2003 X32 and SQL Server X32. You may believe me when I say I wasn't processing *anything* very quickly. I paid (borrowed) about $5000 out of my own pocket to build these two machines in order to get the business. Over the years I just upgraded those servers, dual cores, new motherboard, 8 gigs ram, then 16 gigs ram, then quad core etc. I am now building a brand new server, pretty much from the ground up. This time the client is paying for the upgrade. The only thing I am bringing over from the last server is my Areca 16 port raid controller and the terabyte drives that hold all of the databases and business files. The new server is a dual socket board for the new AMD Opteron 6000 series processor. I selected the 8 core chips because the 2 GHz version is dirt cheap (275 each ATM). I would LOVE to have the 12 core version of the chip but the price for the entry level 12 core is $750 each and I just couldn't justify it (to myself). The point really is that this is not rocket science and it is pretty darned cheap. 450 Motherboard 275 CPU x 1 250 8 gig DIMM x 1 So $1000 for an 8 core machine with 8 gigs of memory. The motherboard has two CPU sockets so you can drop in another CPU. And it has SIXTEEN DIMM sockets so you can drop in up to 128 GIGS of memory if you can afford it. 450 Motherboard 550 CPU X 2 2000 8 gig DIMM X 8 So right around $3K for a 16 core machine with 64 GIGS of memory. This is the configuration I am aiming for. You have to admit that is a pretty reasonable price for the foundation, and the nice part is that you can get in cheap and add more memory and another core as you get the money. The biggest problem I had was finding a reasonably priced chassis for the motherboard. These motherboards are a "server size" and won't physically fit in the average tower. What I am hoping is that the AMD Bulldozer, which is due out in 2011, will allow me to do a cheap processor upgrade down the road. Rumor has it that they will build chips with 16 cores, so in a few years (after the dust settles and prices drop) I could double my core count again if I need to. I have to tell you, just moving from a quad core to an 8 core makes a world of difference. SQL Server expects to "own" the machine but if you are a poor hillbilly in North Carolina your server has to do more. With the quad core I assigned 2 cores to SQL Server and two to the OS. SQL Server would max out the two cores it was assigned, which indicates it needs more horsepower. With 8 cores, SQL Server gets 6 cores now, and there are times when it uses all 6 cores, though I have never seen it max all 6 out. This is a strong indicator that with my current config, six cores is enough. However I will be dropping in more memory, which may remove a bottleneck and allow SQL Server to use the processors more efficiently, potentially maxing them out. If and when I drop in the other CPU, SQL Server could get as many as 14 cores. I am really beginning to doubt that my jobs require that but for an additional $275 for the CPU, it is nice to know I can go there if I need to. John W. Colby www.ColbyConsulting.com On 10/15/2010 8:46 AM, Dan Waters wrote: > Seriously John - make some noise! > > Ya see, I thought you were just a guy who knew how to quickly process > millions of records in SQL Server. ;-) > > Now I know better! :-) > > Keep going! > Dan > > -----Original Message----- > From: dba-sqlserver-bounces at databaseadvisors.com > [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Michael > Mattys > Sent: Friday, October 15, 2010 7:42 AM > To: 'Discussion concerning MS SQL Server' > Subject: Re: [dba-SQLServer] The SSD I am using > > No, John, don't be quiet! > Jim was kidding - he was saying you deserve to brag. > My brother and I have to do much of what you're doing and I am interested in > all your experiences in this and the other groups. > > Michael R Mattys > Business Process Developers > www.mattysconsulting.com > > -----Original Message----- > From: dba-sqlserver-bounces at databaseadvisors.com > [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, October 14, 2010 10:07 PM > To: Discussion concerning MS SQL Server > Subject: Re: [dba-SQLServer] The SSD I am using > > Hmm... not my intent. I am building a server. Paid for by my client, not > mine - though it does > reside in my office. I thought there would be interest in how this stuff > performs. When I started > trying to discover how much this stuff would affect performance I didn't > really find much information. > > I'll be quiet now. > > John W. Colby > www.ColbyConsulting.com > > On 10/14/2010 5:27 PM, Jim Lawrence wrote: >> Boy, you are sure bragging a lot... not that you don't deserve to. ;-) >> >> Jim > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From robert at webedb.com Fri Oct 15 16:14:21 2010 From: robert at webedb.com (Robert) Date: Fri, 15 Oct 2010 16:14:21 -0500 Subject: [dba-SQLServer] Powerful servers are not rocket science In-Reply-To: References: Message-ID: <70F1F7A3-57ED-4E9E-8641-F91B0F08F4CB@holly.arvixe.com> John, To get more detail, can you give us the details on where you got the case, motherboard, memory, and cpu? By the end of the year, I need to build out a similar machine. Robert At 08:44 AM 10/15/2010, you wrote: >Date: Fri, 15 Oct 2010 09:44:35 -0400 >From: jwcolby >Subject: [dba-SQLServer] Powerful servers are not rocket science and > pretty darned cheap >To: Discussion concerning MS SQL Server > >Message-ID: <4CB85AC3.9080804 at colbyconsulting.com> >Content-Type: text/plain; charset=ISO-8859-1; format=flowed > >ROTFL. > >When I started this client / business (Aug 2004 according to my >billing records) I built two >servers. At that time the Athlon was king and what I could afford >was the 3.8 GHz. > >So they were single core machines, 4 gigs ram, running Windows 2003 >X32 and SQL Server X32. > >You may believe me when I say I wasn't processing *anything* very quickly. > >I paid (borrowed) about $5000 out of my own pocket to build these >two machines in order to get the >business. > >Over the years I just upgraded those servers, dual cores, new >motherboard, 8 gigs ram, then 16 gigs >ram, then quad core etc. > >I am now building a brand new server, pretty much from the ground >up. This time the client is >paying for the upgrade. > >The only thing I am bringing over from the last server is my Areca >16 port raid controller and the >terabyte drives that hold all of the databases and business files. > >The new server is a dual socket board for the new AMD Opteron 6000 >series processor. I selected the >8 core chips because the 2 GHz version is dirt cheap (275 each >ATM). I would LOVE to have the 12 >core version of the chip but the price for the entry level 12 core >is $750 each and I just couldn't >justify it (to myself). > >The point really is that this is not rocket science and it is pretty >darned cheap. > >450 Motherboard >275 CPU x 1 >250 8 gig DIMM x 1 > >So $1000 for an 8 core machine with 8 gigs of memory. The >motherboard has two CPU sockets so you >can drop in another CPU. And it has SIXTEEN DIMM sockets so you can >drop in up to 128 GIGS of >memory if you can afford it. > >450 Motherboard >550 CPU X 2 >2000 8 gig DIMM X 8 > >So right around $3K for a 16 core machine with 64 GIGS of >memory. This is the configuration I am >aiming for. You have to admit that is a pretty reasonable price for >the foundation, and the nice >part is that you can get in cheap and add more memory and another >core as you get the money. > >The biggest problem I had was finding a reasonably priced chassis >for the motherboard. These >motherboards are a "server size" and won't physically fit in the >average tower. > >What I am hoping is that the AMD Bulldozer, which is due out in >2011, will allow me to do a cheap >processor upgrade down the road. Rumor has it that they will build >chips with 16 cores, so in a few >years (after the dust settles and prices drop) I could double my >core count again if I need to. > >I have to tell you, just moving from a quad core to an 8 core makes >a world of difference. SQL >Server expects to "own" the machine but if you are a poor hillbilly >in North Carolina your server >has to do more. With the quad core I assigned 2 cores to SQL Server >and two to the OS. SQL Server >would max out the two cores it was assigned, which indicates it >needs more horsepower. > >With 8 cores, SQL Server gets 6 cores now, and there are times when >it uses all 6 cores, though I >have never seen it max all 6 out. This is a strong indicator that >with my current config, six cores >is enough. However I will be dropping in more memory, which may >remove a bottleneck and allow SQL >Server to use the processors more efficiently, potentially maxing them out. > >If and when I drop in the other CPU, SQL Server could get as many as >14 cores. I am really >beginning to doubt that my jobs require that but for an additional >$275 for the CPU, it is nice to >know I can go there if I need to. > >John W. Colby >www.ColbyConsulting.com From jwcolby at colbyconsulting.com Fri Oct 15 21:19:58 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 15 Oct 2010 22:19:58 -0400 Subject: [dba-SQLServer] Powerful servers are not rocket science In-Reply-To: <70F1F7A3-57ED-4E9E-8641-F91B0F08F4CB@holly.arvixe.com> References: <70F1F7A3-57ED-4E9E-8641-F91B0F08F4CB@holly.arvixe.com> Message-ID: <4CB90BCE.9040300@colbyconsulting.com> Robert, > To get more detail, can you give us the details on where you got the case, motherboard, memory, and cpu? Certainly. I buy from NewEgg.com. Motherboard: http://www.newegg.com/Product/Product.aspx?Item=N82E16813131643 CPU: http://www.newegg.com/Product/Product.aspx?Item=N82E16819105266 CPU HS/Fan: http://www.newegg.com/Product/Product.aspx?Item=N82E16835114113 Memory: http://www.newegg.com/Product/Product.aspx?Item=N82E16820139140 Case: http://www.newegg.com/Product/Product.aspx?Item=N82E16811219021&cm_re=4u_rackmount_case-_-11-219-021-_-Product PSU: http://www.newegg.com/Product/Product.aspx?Item=N82E16817139010 Cable splitter: http://www.newegg.com/Product/Product.aspx?Item=N82E16812198019 SSD: http://www.newegg.com/Product/Product.aspx?Item=N82E16820227551 These are the actual components that I purchased. The splitter cable is required for most consumer grade power supplies as the motherboard requires two of these connectors and apparently won't boot without both in place. If you go with a server grade ps it might have two of these connectors. The case is definitely cheap consumer grade, but it works, especially in the latest iteration where they have fixed the early problems. The memory can be bought less expensively if you can go with 4 gig dimms: http://www.newegg.com/Product/Product.aspx?Item=N82E16820134977 I was not paying for the system and decided to spend the extra to get the larger (8 gig) dimms so that I could get 128G in the board if I needed to. This motherboard will accept 1, 2, 4, or 8 DIMMs / CPU socket. Understand that the CPU / socket has 4 memory channels so populating 4 DIMM sockets at a time will give you the best bandwidth. I only have 2 at the moment, for 2 channels / 16 gigs of memory (and one CPU). I will likely populate up to 64 gigs (8 DIMMS) with the one CPU and only populate the other socket if I see my system maxing out the cores assigned to SQL Server. If I do drop in the other CPU I would then just redistribute 1/2 the memory to the other socket (32 gigs / socket). I keep two cores for the OS. These two cores rarely work hard, though I have seen both cores at about 50% when doing heavy writes to system files when SQL Server is working hard. Normally though they just idle along. At this point SQL Server is occasionally using all 6 cores assigned to it and even then only to about 60% capacity (the maximum average I have ever seen), so I still have ~40% of 6 cores left (in terms of available processing power). With my old quad core I would max out the two cores assigned to SQL Server - 100% used - and that in many scenarios. I bought the low end CPU because I couldn't justify the extra expense of the 12 core. $750 for 12 core vs $275 for 8 core was a deal breaker for me. It certainly appears that 8 cores is plenty for my immediate needs, though I do think I could use more memory for the size of my database tables and the stuff I do. If you need the 12 core CPU though: http://www.newegg.com/Product/Product.aspx?Item=N82E16819105267 Supposedly the AMD Bulldozer will be a drop in replacement for these CPUs when it is released next year. I am also using the 16 channel Areca raid controller with 2 gigs ECC disk cache. Highly recommended if you have that kind of need. I have about 12 terabyte drives: http://www.newegg.com/Product/Product.aspx?Item=N82E16822136284 I have been using these for a couple of years and I just had my first failure of these drives. The nice thing about a dedicated raid controller is that you can just have hot spares in place and it will notify you of the failure and automatically rebuild (using the hot spare) in the event of a failure. And you can walk in to Best Buy and buy a replacement. John W. Colby www.ColbyConsulting.com On 10/15/2010 5:14 PM, Robert wrote: > John, > > To get more detail, can you give us the details on where you got the > case, motherboard, memory, and cpu? > > By the end of the year, I need to build out a similar machine. > > > Robert > > At 08:44 AM 10/15/2010, you wrote: >> Date: Fri, 15 Oct 2010 09:44:35 -0400 >> From: jwcolby >> Subject: [dba-SQLServer] Powerful servers are not rocket science and >> pretty darned cheap >> To: Discussion concerning MS SQL Server >> >> Message-ID:<4CB85AC3.9080804 at colbyconsulting.com> >> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >> >> ROTFL. >> >> When I started this client / business (Aug 2004 according to my >> billing records) I built two >> servers. At that time the Athlon was king and what I could afford >> was the 3.8 GHz. >> >> So they were single core machines, 4 gigs ram, running Windows 2003 >> X32 and SQL Server X32. >> >> You may believe me when I say I wasn't processing *anything* very quickly. >> >> I paid (borrowed) about $5000 out of my own pocket to build these >> two machines in order to get the >> business. >> >> Over the years I just upgraded those servers, dual cores, new >> motherboard, 8 gigs ram, then 16 gigs >> ram, then quad core etc. >> >> I am now building a brand new server, pretty much from the ground >> up. This time the client is >> paying for the upgrade. >> >> The only thing I am bringing over from the last server is my Areca >> 16 port raid controller and the >> terabyte drives that hold all of the databases and business files. >> >> The new server is a dual socket board for the new AMD Opteron 6000 >> series processor. I selected the >> 8 core chips because the 2 GHz version is dirt cheap (275 each >> ATM). I would LOVE to have the 12 >> core version of the chip but the price for the entry level 12 core >> is $750 each and I just couldn't >> justify it (to myself). >> >> The point really is that this is not rocket science and it is pretty >> darned cheap. >> >> 450 Motherboard >> 275 CPU x 1 >> 250 8 gig DIMM x 1 >> >> So $1000 for an 8 core machine with 8 gigs of memory. The >> motherboard has two CPU sockets so you >> can drop in another CPU. And it has SIXTEEN DIMM sockets so you can >> drop in up to 128 GIGS of >> memory if you can afford it. >> >> 450 Motherboard >> 550 CPU X 2 >> 2000 8 gig DIMM X 8 >> >> So right around $3K for a 16 core machine with 64 GIGS of >> memory. This is the configuration I am >> aiming for. You have to admit that is a pretty reasonable price for >> the foundation, and the nice >> part is that you can get in cheap and add more memory and another >> core as you get the money. >> >> The biggest problem I had was finding a reasonably priced chassis >> for the motherboard. These >> motherboards are a "server size" and won't physically fit in the >> average tower. >> >> What I am hoping is that the AMD Bulldozer, which is due out in >> 2011, will allow me to do a cheap >> processor upgrade down the road. Rumor has it that they will build >> chips with 16 cores, so in a few >> years (after the dust settles and prices drop) I could double my >> core count again if I need to. >> >> I have to tell you, just moving from a quad core to an 8 core makes >> a world of difference. SQL >> Server expects to "own" the machine but if you are a poor hillbilly >> in North Carolina your server >> has to do more. With the quad core I assigned 2 cores to SQL Server >> and two to the OS. SQL Server >> would max out the two cores it was assigned, which indicates it >> needs more horsepower. >> >> With 8 cores, SQL Server gets 6 cores now, and there are times when >> it uses all 6 cores, though I >> have never seen it max all 6 out. This is a strong indicator that >> with my current config, six cores >> is enough. However I will be dropping in more memory, which may >> remove a bottleneck and allow SQL >> Server to use the processors more efficiently, potentially maxing them out. >> >> If and when I drop in the other CPU, SQL Server could get as many as >> 14 cores. I am really >> beginning to doubt that my jobs require that but for an additional >> $275 for the CPU, it is nice to >> know I can go there if I need to. >> >> John W. Colby >> www.ColbyConsulting.com > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Oct 15 21:35:33 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 15 Oct 2010 22:35:33 -0400 Subject: [dba-SQLServer] Powerful servers are not rocket science In-Reply-To: <4CB90BCE.9040300@colbyconsulting.com> References: <70F1F7A3-57ED-4E9E-8641-F91B0F08F4CB@holly.arvixe.com> <4CB90BCE.9040300@colbyconsulting.com> Message-ID: <4CB90F75.1070507@colbyconsulting.com> For anyone in the market today, Newegg has a bundle with $80 off the motherboard and processor I bought. http://www.newegg.com/Product/ProductCombos.aspx?Item=N82E16813131643&SubCategory=302&SortField=0&PageSize=10&page=1 2nd item down. John W. Colby www.ColbyConsulting.com On 10/15/2010 10:19 PM, jwcolby wrote: > Robert, > > > To get more detail, can you give us the details on where you got the case, motherboard, memory, > and cpu? > > Certainly. I buy from NewEgg.com. > > Motherboard: http://www.newegg.com/Product/Product.aspx?Item=N82E16813131643 > CPU: http://www.newegg.com/Product/Product.aspx?Item=N82E16819105266 > CPU HS/Fan: http://www.newegg.com/Product/Product.aspx?Item=N82E16835114113 > Memory: http://www.newegg.com/Product/Product.aspx?Item=N82E16820139140 > > Case: > http://www.newegg.com/Product/Product.aspx?Item=N82E16811219021&cm_re=4u_rackmount_case-_-11-219-021-_-Product > PSU: http://www.newegg.com/Product/Product.aspx?Item=N82E16817139010 > Cable splitter: http://www.newegg.com/Product/Product.aspx?Item=N82E16812198019 > > SSD: http://www.newegg.com/Product/Product.aspx?Item=N82E16820227551 > > These are the actual components that I purchased. > > The splitter cable is required for most consumer grade power supplies as the motherboard requires > two of these connectors and apparently won't boot without both in place. If you go with a server > grade ps it might have two of these connectors. > > The case is definitely cheap consumer grade, but it works, especially in the latest iteration where > they have fixed the early problems. > > The memory can be bought less expensively if you can go with 4 gig dimms: > > http://www.newegg.com/Product/Product.aspx?Item=N82E16820134977 > > I was not paying for the system and decided to spend the extra to get the larger (8 gig) dimms so > that I could get 128G in the board if I needed to. > > This motherboard will accept 1, 2, 4, or 8 DIMMs / CPU socket. Understand that the CPU / socket has > 4 memory channels so populating 4 DIMM sockets at a time will give you the best bandwidth. I only > have 2 at the moment, for 2 channels / 16 gigs of memory (and one CPU). I will likely populate up > to 64 gigs (8 DIMMS) with the one CPU and only populate the other socket if I see my system maxing > out the cores assigned to SQL Server. If I do drop in the other CPU I would then just redistribute > 1/2 the memory to the other socket (32 gigs / socket). > > I keep two cores for the OS. These two cores rarely work hard, though I have seen both cores at > about 50% when doing heavy writes to system files when SQL Server is working hard. Normally though > they just idle along. > > At this point SQL Server is occasionally using all 6 cores assigned to it and even then only to > about 60% capacity (the maximum average I have ever seen), so I still have ~40% of 6 cores left (in > terms of available processing power). With my old quad core I would max out the two cores assigned > to SQL Server - 100% used - and that in many scenarios. > > I bought the low end CPU because I couldn't justify the extra expense of the 12 core. $750 for 12 > core vs $275 for 8 core was a deal breaker for me. It certainly appears that 8 cores is plenty for > my immediate needs, though I do think I could use more memory for the size of my database tables and > the stuff I do. > > If you need the 12 core CPU though: > > http://www.newegg.com/Product/Product.aspx?Item=N82E16819105267 > > Supposedly the AMD Bulldozer will be a drop in replacement for these CPUs when it is released next year. > > I am also using the 16 channel Areca raid controller with 2 gigs ECC disk cache. Highly recommended > if you have that kind of need. I have about 12 terabyte drives: > > http://www.newegg.com/Product/Product.aspx?Item=N82E16822136284 > > I have been using these for a couple of years and I just had my first failure of these drives. The > nice thing about a dedicated raid controller is that you can just have hot spares in place and it > will notify you of the failure and automatically rebuild (using the hot spare) in the event of a > failure. And you can walk in to Best Buy and buy a replacement. > > John W. Colby > www.ColbyConsulting.com > > On 10/15/2010 5:14 PM, Robert wrote: >> John, >> >> To get more detail, can you give us the details on where you got the >> case, motherboard, memory, and cpu? >> >> By the end of the year, I need to build out a similar machine. >> >> >> Robert >> >> At 08:44 AM 10/15/2010, you wrote: >>> Date: Fri, 15 Oct 2010 09:44:35 -0400 >>> From: jwcolby >>> Subject: [dba-SQLServer] Powerful servers are not rocket science and >>> pretty darned cheap >>> To: Discussion concerning MS SQL Server >>> >>> Message-ID:<4CB85AC3.9080804 at colbyconsulting.com> >>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >>> >>> ROTFL. >>> >>> When I started this client / business (Aug 2004 according to my >>> billing records) I built two >>> servers. At that time the Athlon was king and what I could afford >>> was the 3.8 GHz. >>> >>> So they were single core machines, 4 gigs ram, running Windows 2003 >>> X32 and SQL Server X32. >>> >>> You may believe me when I say I wasn't processing *anything* very quickly. >>> >>> I paid (borrowed) about $5000 out of my own pocket to build these >>> two machines in order to get the >>> business. >>> >>> Over the years I just upgraded those servers, dual cores, new >>> motherboard, 8 gigs ram, then 16 gigs >>> ram, then quad core etc. >>> >>> I am now building a brand new server, pretty much from the ground >>> up. This time the client is >>> paying for the upgrade. >>> >>> The only thing I am bringing over from the last server is my Areca >>> 16 port raid controller and the >>> terabyte drives that hold all of the databases and business files. >>> >>> The new server is a dual socket board for the new AMD Opteron 6000 >>> series processor. I selected the >>> 8 core chips because the 2 GHz version is dirt cheap (275 each >>> ATM). I would LOVE to have the 12 >>> core version of the chip but the price for the entry level 12 core >>> is $750 each and I just couldn't >>> justify it (to myself). >>> >>> The point really is that this is not rocket science and it is pretty >>> darned cheap. >>> >>> 450 Motherboard >>> 275 CPU x 1 >>> 250 8 gig DIMM x 1 >>> >>> So $1000 for an 8 core machine with 8 gigs of memory. The >>> motherboard has two CPU sockets so you >>> can drop in another CPU. And it has SIXTEEN DIMM sockets so you can >>> drop in up to 128 GIGS of >>> memory if you can afford it. >>> >>> 450 Motherboard >>> 550 CPU X 2 >>> 2000 8 gig DIMM X 8 >>> >>> So right around $3K for a 16 core machine with 64 GIGS of >>> memory. This is the configuration I am >>> aiming for. You have to admit that is a pretty reasonable price for >>> the foundation, and the nice >>> part is that you can get in cheap and add more memory and another >>> core as you get the money. >>> >>> The biggest problem I had was finding a reasonably priced chassis >>> for the motherboard. These >>> motherboards are a "server size" and won't physically fit in the >>> average tower. >>> >>> What I am hoping is that the AMD Bulldozer, which is due out in >>> 2011, will allow me to do a cheap >>> processor upgrade down the road. Rumor has it that they will build >>> chips with 16 cores, so in a few >>> years (after the dust settles and prices drop) I could double my >>> core count again if I need to. >>> >>> I have to tell you, just moving from a quad core to an 8 core makes >>> a world of difference. SQL >>> Server expects to "own" the machine but if you are a poor hillbilly >>> in North Carolina your server >>> has to do more. With the quad core I assigned 2 cores to SQL Server >>> and two to the OS. SQL Server >>> would max out the two cores it was assigned, which indicates it >>> needs more horsepower. >>> >>> With 8 cores, SQL Server gets 6 cores now, and there are times when >>> it uses all 6 cores, though I >>> have never seen it max all 6 out. This is a strong indicator that >>> with my current config, six cores >>> is enough. However I will be dropping in more memory, which may >>> remove a bottleneck and allow SQL >>> Server to use the processors more efficiently, potentially maxing them out. >>> >>> If and when I drop in the other CPU, SQL Server could get as many as >>> 14 cores. I am really >>> beginning to doubt that my jobs require that but for an additional >>> $275 for the CPU, it is nice to >>> know I can go there if I need to. >>> >>> John W. Colby >>> www.ColbyConsulting.com >> _______________________________________________ >> dba-SQLServer mailing list >> dba-SQLServer at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Oct 16 22:54:04 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 16 Oct 2010 23:54:04 -0400 Subject: [dba-SQLServer] Can I share a connection object Message-ID: <4CBA735C.8080008@colbyconsulting.com> I am doing a ton of stuff to a database in SQL Server, pulling all the records in a table into a datatable, building out update SQL statements and using a command object to update those records, building out insert sql statements and appending records into another table etc. My question is can I share the connection object between everything that needs to have one? I am going to be doing dozens of update and insert operations / second and if I could just grab and share a connection it seems logical to do so. -- John W. Colby www.ColbyConsulting.com From accessd at shaw.ca Sun Oct 17 12:22:25 2010 From: accessd at shaw.ca (Jim Lawrence) Date: Sun, 17 Oct 2010 10:22:25 -0700 Subject: [dba-SQLServer] Can I share a connection object In-Reply-To: <4CBA735C.8080008@colbyconsulting.com> References: <4CBA735C.8080008@colbyconsulting.com> Message-ID: <72345B52316A4A21A643CA544489D0FE@creativesystemdesigns.com> Hi John: Are we talking about sharing via a web based link? Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, October 16, 2010 8:54 PM To: Sqlserver-Dba Subject: [dba-SQLServer] Can I share a connection object I am doing a ton of stuff to a database in SQL Server, pulling all the records in a table into a datatable, building out update SQL statements and using a command object to update those records, building out insert sql statements and appending records into another table etc. My question is can I share the connection object between everything that needs to have one? I am going to be doing dozens of update and insert operations / second and if I could just grab and share a connection it seems logical to do so. -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Oct 22 15:36:27 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 22 Oct 2010 16:36:27 -0400 Subject: [dba-SQLServer] New system building an order Message-ID: <4CC1F5CB.5050706@colbyconsulting.com> Today I received another 16 GB ram and put in the SQL server. I now have a single CPU / 8 cores and 32 gigs of memory. I assigned 27 Gigs to SQL Server and left 5 to the system. I decided to test the new server building a moderately complex order. This order pulls about 5 million OrderData records from a series of views. _DataAllAdults.tblAllAdultsNameAddr table has a clustered index on the PKID and an NameAddress cover index which covers all of the name / address fields as well as the gender and AddressValid flag. vAllAdults selects records from that table which have specific codes in the ValidAddr vield. vAllFemales selects records from vAllAdults where the gender is 'F'. _DataHSID holds the database from hell, the tblHSID with ~50 million records with ~600 fields. Ut has a variety of cover indexes on it which cover specific groups of fields. vHSIDOrderCriteria joins vAllFemales and the DataHSID.tblHSID on the PKHSID, and uses a where clause which selects specific values from specific fields. Under most circumstances I just edit and save vHSIDOrderCriteria to select the correct HSID fields / values to pull the desired records. I then run an external C# program which dynamically builds the tblOrderData using the field list from vHSIDOrderCriteria (which obviously changes from order to order) and then populates that temp table with the data pulled using vHSIDOrderCriteria. In this case, ~5 million records were selected and stored in tblOrderData. This is the first time ever that I have had more than 2 cores and about 12 gigs of memory to use to run the order. I do not have any timing information for past runs. However what I observed is that processing the order caused all 6 cores assigned to SQL Server to almost max out for the first part of the order process. It wasn't flat line at the top but it was in the 90+ % utilized for all 6 cores, for part of the time - perhaps 40% of the duration. The rest of the time it maxed out a single core. and partly used another. And it used 29+ gigs of memory during use. This order process is the first step in processing an order and takes awhile to complete. I now have _DataAllAdults and _DataHSID on SSD. This is the first time I have ever seen all six cores close to max. No promises, but just out of curiosity I am going to try and "recreate" the old system, with a copy of _DataAllAdults and _DataHSID on rotating media, assign 12 gigs of memory as in the "olden days" and then do a simple stopwatch timing of the two. It really won't be even close to the old system because the old CPU was a quad core at 3 GHZ and this one is 8 cores at 2 GHZ. In the old system I assigned 2 cores to SQL Server, and it would often max out both cores. This one I am assigning 6 cores and it runs about 9% of all six cores at times. None the less it should give a feeling for the relative speed with all the changes. Because of the way I layer the views it should be fairly easy to build a copy database and then modify two specific views to point to the copy of data on rotating media. After that it is really just a matter of running it twice, once for each order database copy. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Oct 23 10:35:44 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 23 Oct 2010 11:35:44 -0400 Subject: [dba-SQLServer] Backing up Windows 2008 Message-ID: <4CC300D0.8060700@colbyconsulting.com> I need to do a system disk backup for Windows. I am probably going with Acronis home for all of my non-server machines, however they want something like $500 / machine for server licenses. While I am sure it is worth it, I just don't have that kind of cash just to get a backup. I have never used the Windows built-in backup but it seems that it does exist. Has anyone here used it? Restored from it? Easy / hard? Potential issues? -- John W. Colby www.ColbyConsulting.com