From jlawrenc1 at shaw.ca Wed Nov 1 10:40:03 2006 From: jlawrenc1 at shaw.ca (Jim Lawrence) Date: Wed, 01 Nov 2006 08:40:03 -0800 Subject: [dba-SQLServer] What is going on In-Reply-To: <002001c6fd76$64323ca0$657aa8c0@m6805> Message-ID: <0J8200BP68WJ64O0@l-daemon> Hi John: Just to confirm: 1. You are using the standard bulk raw import procedures or DTS Text import? 2. No joins, indexes, database logging, datatype conversions, identity fields, etc. etc... just an unattached single staging table... 3. No alerts or other services on... 4. Unlimited unrestricted growth on. Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Tuesday, October 31, 2006 9:27 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Just an update, I started a new import wizard processing the exact same file (a copy on the E: drive) into an empty table and it is running at the expected 3000 records / second, so far. So it appears to have something to do with the number of records already in the main table as opposed to a general SQL Server slow down, raid drive slowdown etc. I am canceling the sloooow import. The import into a clean table should be finished within 15 minutes or so, at which point I will try an append from the smaller table and see if that happens quickly. Something funky is definitely going on though. The cancel takes a long time, and while that is happening, the other data import has slowed to a crawl. This really sucks! John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 12:12 AM To: dba-sqlserver at databaseadvisors.com Subject: [dba-SQLServer] What is going on This afternoon, soon after posting how fast the system is, SQL Server "locked up". I had imported 18 files over the afternoon, and then suddenly the three that were processing just froze. In task manager I would see SQL Server activity, the processors would alternate between no activity and 50% utilization, and when that happened SQL Server was the task using the CPU time. However NO records were importing. I decided to let it run, went away for a meeting, returned several hours later to find NO cpu activity and the exact same number of records imported (still frozen). I aborted, then restarted a single import process. It progressed, but slowly. The next progressed but even more slowly. The last file that needs to be imported is just dragging. As in will take 12 hours or more to do just the one file (20 minutes was the average before). I stopped that last process, moved the file physically out onto the smaller raid to test whether it was a comms issue between machines. No help. My C: (system) drive has about 10g out of 80 used. My D: (SQL system databases there) has about 10g used out of 200 total. My E: drive (the main database file for this database) has used 298 G with 901 gig free (one of two Raid 6 drives) . My F: drive (the main database log file for this database) has used 45 Gb with 330gb free (the second Raid 6 drive). I have huge amounts of free space on the raid drives, well over 700 gb on the one, and 300 gb on the other. The main db file is now up to about 280 mb but by my calcs that is about normal (correct) given the amount of data input so far. At this point I am on the last file to import, sitting at about 1 million records imported out of 3 million to be imported. It is importing them about 1000 records every three seconds. There are about 63 million records in the table. No indexes, no triggers, nothing like that. Just raw data. This thing should be flying. Instead it is crawling. CPU usage is running 12% to 25%, with SQL Server.exe and MMC.exe using that. What the heck is going on? John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Wed Nov 1 11:26:51 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Wed, 1 Nov 2006 12:26:51 -0500 Subject: [dba-SQLServer] What is going on In-Reply-To: <0J8200BP68WJ64O0@l-daemon> Message-ID: <003001c6fdda$ebcbd5e0$657aa8c0@m6805> Yep Yep Yep AFAIK Almost certain but it is in the middle of a shrink file and I can't get control at the moment. Just an update, I am doing a shrink because it told me that the container file was 280 gb but only 100gb used. Speaking of which, is there any way to shrink in the background? That seems like a given but a shrink locks up my database tight which seems rather unhelpful. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Wednesday, November 01, 2006 11:40 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Hi John: Just to confirm: 1. You are using the standard bulk raw import procedures or DTS Text import? 2. No joins, indexes, database logging, datatype conversions, identity fields, etc. etc... just an unattached single staging table... 3. No alerts or other services on... 4. Unlimited unrestricted growth on. Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Tuesday, October 31, 2006 9:27 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Just an update, I started a new import wizard processing the exact same file (a copy on the E: drive) into an empty table and it is running at the expected 3000 records / second, so far. So it appears to have something to do with the number of records already in the main table as opposed to a general SQL Server slow down, raid drive slowdown etc. I am canceling the sloooow import. The import into a clean table should be finished within 15 minutes or so, at which point I will try an append from the smaller table and see if that happens quickly. Something funky is definitely going on though. The cancel takes a long time, and while that is happening, the other data import has slowed to a crawl. This really sucks! John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 12:12 AM To: dba-sqlserver at databaseadvisors.com Subject: [dba-SQLServer] What is going on This afternoon, soon after posting how fast the system is, SQL Server "locked up". I had imported 18 files over the afternoon, and then suddenly the three that were processing just froze. In task manager I would see SQL Server activity, the processors would alternate between no activity and 50% utilization, and when that happened SQL Server was the task using the CPU time. However NO records were importing. I decided to let it run, went away for a meeting, returned several hours later to find NO cpu activity and the exact same number of records imported (still frozen). I aborted, then restarted a single import process. It progressed, but slowly. The next progressed but even more slowly. The last file that needs to be imported is just dragging. As in will take 12 hours or more to do just the one file (20 minutes was the average before). I stopped that last process, moved the file physically out onto the smaller raid to test whether it was a comms issue between machines. No help. My C: (system) drive has about 10g out of 80 used. My D: (SQL system databases there) has about 10g used out of 200 total. My E: drive (the main database file for this database) has used 298 G with 901 gig free (one of two Raid 6 drives) . My F: drive (the main database log file for this database) has used 45 Gb with 330gb free (the second Raid 6 drive). I have huge amounts of free space on the raid drives, well over 700 gb on the one, and 300 gb on the other. The main db file is now up to about 280 mb but by my calcs that is about normal (correct) given the amount of data input so far. At this point I am on the last file to import, sitting at about 1 million records imported out of 3 million to be imported. It is importing them about 1000 records every three seconds. There are about 63 million records in the table. No indexes, no triggers, nothing like that. Just raw data. This thing should be flying. Instead it is crawling. CPU usage is running 12% to 25%, with SQL Server.exe and MMC.exe using that. What the heck is going on? John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jlawrenc1 at shaw.ca Wed Nov 1 11:56:30 2006 From: jlawrenc1 at shaw.ca (Jim Lawrence) Date: Wed, 01 Nov 2006 09:56:30 -0800 Subject: [dba-SQLServer] What is going on In-Reply-To: <003001c6fdda$ebcbd5e0$657aa8c0@m6805> Message-ID: <0J8200B96CFY62S0@l-daemon> Hi John: It does tie up everthing... Here is a step-by-step list for running shrink process that I pulled off a MS SQL site: 1. Create a file which is as large as the data in your primary file (call it "buffer") 2. Empty the primary file (DBCC SHRINKFILE (,EMPTYFILE)) 3. Restart SQL Server Engine 4. Shrink the primary file to the Data size divided by the number of files you're gonna create (DBCC SHRINKFILE(,) 5. Create all the new files with the size of data divided by the number of files 6. Restrict their growth in order to fill the primary file in the next operation 7. Empty the buffer file (DBCC SHRINKFILE(BUFFER,EMPTYFILE)) 8. Delete the buffer file (ALTER DATABASE REMOVE FILE (NAME=BUFFER)) 9. Set final size of data files and unrestrict their growth according to the final configuration needed HTH Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 9:27 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Yep Yep Yep AFAIK Almost certain but it is in the middle of a shrink file and I can't get control at the moment. Just an update, I am doing a shrink because it told me that the container file was 280 gb but only 100gb used. Speaking of which, is there any way to shrink in the background? That seems like a given but a shrink locks up my database tight which seems rather unhelpful. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Wednesday, November 01, 2006 11:40 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Hi John: Just to confirm: 1. You are using the standard bulk raw import procedures or DTS Text import? 2. No joins, indexes, database logging, datatype conversions, identity fields, etc. etc... just an unattached single staging table... 3. No alerts or other services on... 4. Unlimited unrestricted growth on. Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Tuesday, October 31, 2006 9:27 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Just an update, I started a new import wizard processing the exact same file (a copy on the E: drive) into an empty table and it is running at the expected 3000 records / second, so far. So it appears to have something to do with the number of records already in the main table as opposed to a general SQL Server slow down, raid drive slowdown etc. I am canceling the sloooow import. The import into a clean table should be finished within 15 minutes or so, at which point I will try an append from the smaller table and see if that happens quickly. Something funky is definitely going on though. The cancel takes a long time, and while that is happening, the other data import has slowed to a crawl. This really sucks! John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 12:12 AM To: dba-sqlserver at databaseadvisors.com Subject: [dba-SQLServer] What is going on This afternoon, soon after posting how fast the system is, SQL Server "locked up". I had imported 18 files over the afternoon, and then suddenly the three that were processing just froze. In task manager I would see SQL Server activity, the processors would alternate between no activity and 50% utilization, and when that happened SQL Server was the task using the CPU time. However NO records were importing. I decided to let it run, went away for a meeting, returned several hours later to find NO cpu activity and the exact same number of records imported (still frozen). I aborted, then restarted a single import process. It progressed, but slowly. The next progressed but even more slowly. The last file that needs to be imported is just dragging. As in will take 12 hours or more to do just the one file (20 minutes was the average before). I stopped that last process, moved the file physically out onto the smaller raid to test whether it was a comms issue between machines. No help. My C: (system) drive has about 10g out of 80 used. My D: (SQL system databases there) has about 10g used out of 200 total. My E: drive (the main database file for this database) has used 298 G with 901 gig free (one of two Raid 6 drives) . My F: drive (the main database log file for this database) has used 45 Gb with 330gb free (the second Raid 6 drive). I have huge amounts of free space on the raid drives, well over 700 gb on the one, and 300 gb on the other. The main db file is now up to about 280 mb but by my calcs that is about normal (correct) given the amount of data input so far. At this point I am on the last file to import, sitting at about 1 million records imported out of 3 million to be imported. It is importing them about 1000 records every three seconds. There are about 63 million records in the table. No indexes, no triggers, nothing like that. Just raw data. This thing should be flying. Instead it is crawling. CPU usage is running 12% to 25%, with SQL Server.exe and MMC.exe using that. What the heck is going on? John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Wed Nov 1 14:11:49 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Wed, 1 Nov 2006 15:11:49 -0500 Subject: [dba-SQLServer] What is going on In-Reply-To: <0J8200B96CFY62S0@l-daemon> Message-ID: <003d01c6fdf1$f7662510$657aa8c0@m6805> Jim, I don't understand what this is doing for me. I have just one data file ATM. I don't know whether that is a good thing or a bad thing. This sounds like the process for creating and then redistributing data amongst multiple files. If you are saying that it is the existence of one huge file that is causing my slowdown, then I can certainly do that. I have to ask if this is as good as it gets with SQL Server. I understand that I am doing stuff with big tables but keerimeny. I started the build of a PKID field on the big table. It tells me that "ansi nulls was not on and it has to rebuild the table" so I do. EM just locks up tight, won't even redraw the screen (blank white) if you switch away and back again. I believe that it is hard at work but folks, this is the age of threads. So I open another instance of EM to work with another table. EM takes several seconds to expand each tree (server, databases, specific database) then when I try to expand the tables, the second instance of EM locks up tight with an hourglass. I mean c'mon. This is 2006, a monster (desktop) system and EM acts as if it is a DOS app from 1986 running on a '286 with 4 megs and swapping memory. The second EM instance has been trying to open the tables icon just to show me what tables are there for 20 minutes now. To say this is discouraging to work with would be an understatement. If this is the best I am going to get I am going to have to look for another database engine to work with. And the most discouraging part is that my dual proc system is cruising along using (average?) well under 25% of the processor while EM is locked up tight. Does SQl Server 2005 fix any of this? Or should I just go look at MySQL or Oracle personal edition. I have to get work done on this database and it is telling me that one job (building a field / index) on one table is all it can handle. And by the looks of it I will be locked out of this database for the next 12 to 24 hours. Hell, ACCESS can do better than this!!! Well, maybe not but SQL Server's rep is taking a beating here. John W. Colby Colby Consulting www.ColbyConsulting.com From accessd at shaw.ca Wed Nov 1 16:19:25 2006 From: accessd at shaw.ca (Jim Lawrence) Date: Wed, 01 Nov 2006 14:19:25 -0800 Subject: [dba-SQLServer] What is going on In-Reply-To: <003d01c6fdf1$f7662510$657aa8c0@m6805> Message-ID: <0J8200BAYOM464D1@l-daemon> Hi John: I am trying to cover all the bases. The problems initially appears to be with the MS SQL sever setting as they run fairly indendendantly from an OS. If the problem is not in the MS SQL setup, accumulating non responsive results sort of negates the hardware being the issue, then it has to be in the process. If it is absolutely not in the process then it must be in the MS SQL settings. Please tell me you are using server OS as it would be border-line insanity to use a desktop type OS and then I would assure you, without hesitation, that that is the problem. HTH Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 12:12 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Jim, I don't understand what this is doing for me. I have just one data file ATM. I don't know whether that is a good thing or a bad thing. This sounds like the process for creating and then redistributing data amongst multiple files. If you are saying that it is the existence of one huge file that is causing my slowdown, then I can certainly do that. I have to ask if this is as good as it gets with SQL Server. I understand that I am doing stuff with big tables but keerimeny. I started the build of a PKID field on the big table. It tells me that "ansi nulls was not on and it has to rebuild the table" so I do. EM just locks up tight, won't even redraw the screen (blank white) if you switch away and back again. I believe that it is hard at work but folks, this is the age of threads. So I open another instance of EM to work with another table. EM takes several seconds to expand each tree (server, databases, specific database) then when I try to expand the tables, the second instance of EM locks up tight with an hourglass. I mean c'mon. This is 2006, a monster (desktop) system and EM acts as if it is a DOS app from 1986 running on a '286 with 4 megs and swapping memory. The second EM instance has been trying to open the tables icon just to show me what tables are there for 20 minutes now. To say this is discouraging to work with would be an understatement. If this is the best I am going to get I am going to have to look for another database engine to work with. And the most discouraging part is that my dual proc system is cruising along using (average?) well under 25% of the processor while EM is locked up tight. Does SQl Server 2005 fix any of this? Or should I just go look at MySQL or Oracle personal edition. I have to get work done on this database and it is telling me that one job (building a field / index) on one table is all it can handle. And by the looks of it I will be locked out of this database for the next 12 to 24 hours. Hell, ACCESS can do better than this!!! Well, maybe not but SQL Server's rep is taking a beating here. John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Wed Nov 1 16:51:44 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Wed, 1 Nov 2006 17:51:44 -0500 Subject: [dba-SQLServer] What is going on In-Reply-To: <0J8200BAYOM464D1@l-daemon> Message-ID: <003e01c6fe08$4e748390$657aa8c0@m6805> LOL, borderline insanity. I can assure you that I am FULLY insane. Bwaaa haaaa haaaa. I am using SP Pro. SQL Server ran just fine on 2K pro, XP Pro is the 2K Pro base code. Why would it not run on that? I keep hearing that it isn't supposed to, but it does. I can tell you I would be irritated beyond belief if I went to Win 2003 server (which I have but haven't a clue how to install, since this is a desktop machine in the end) and I still had this issue. I actually tried to install 2003 server (in fact I have a disk with the beginnings of the install) and got to questions that implied I was supposed to know what I was doing (and I don't) so how am I supposed to install 2003 so that I can test your theory? I am not a notwork / OS admin, I am a developer. MS themselves push SQL Server lite as a developer tool. If anyone wants to talk me through a 2003 server install I will do that but... John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Wednesday, November 01, 2006 5:19 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Hi John: I am trying to cover all the bases. The problems initially appears to be with the MS SQL sever setting as they run fairly indendendantly from an OS. If the problem is not in the MS SQL setup, accumulating non responsive results sort of negates the hardware being the issue, then it has to be in the process. If it is absolutely not in the process then it must be in the MS SQL settings. Please tell me you are using server OS as it would be border-line insanity to use a desktop type OS and then I would assure you, without hesitation, that that is the problem. HTH Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 12:12 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Jim, I don't understand what this is doing for me. I have just one data file ATM. I don't know whether that is a good thing or a bad thing. This sounds like the process for creating and then redistributing data amongst multiple files. If you are saying that it is the existence of one huge file that is causing my slowdown, then I can certainly do that. I have to ask if this is as good as it gets with SQL Server. I understand that I am doing stuff with big tables but keerimeny. I started the build of a PKID field on the big table. It tells me that "ansi nulls was not on and it has to rebuild the table" so I do. EM just locks up tight, won't even redraw the screen (blank white) if you switch away and back again. I believe that it is hard at work but folks, this is the age of threads. So I open another instance of EM to work with another table. EM takes several seconds to expand each tree (server, databases, specific database) then when I try to expand the tables, the second instance of EM locks up tight with an hourglass. I mean c'mon. This is 2006, a monster (desktop) system and EM acts as if it is a DOS app from 1986 running on a '286 with 4 megs and swapping memory. The second EM instance has been trying to open the tables icon just to show me what tables are there for 20 minutes now. To say this is discouraging to work with would be an understatement. If this is the best I am going to get I am going to have to look for another database engine to work with. And the most discouraging part is that my dual proc system is cruising along using (average?) well under 25% of the processor while EM is locked up tight. Does SQl Server 2005 fix any of this? Or should I just go look at MySQL or Oracle personal edition. I have to get work done on this database and it is telling me that one job (building a field / index) on one table is all it can handle. And by the looks of it I will be locked out of this database for the next 12 to 24 hours. Hell, ACCESS can do better than this!!! Well, maybe not but SQL Server's rep is taking a beating here. John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From michael at ddisolutions.com.au Wed Nov 1 19:06:27 2006 From: michael at ddisolutions.com.au (Michael Maddison) Date: Thu, 2 Nov 2006 12:06:27 +1100 Subject: [dba-SQLServer] What is going on Message-ID: <59A61174B1F5B54B97FD4ADDE71E7D0116AC71@ddi-01.DDI.local> Hi John, You are out of my league but... a couple of ideas come to mind. Set the database recovery Model to simple. Turn off Auto update and auto create statistics. Drop any/all indexes on the table. You can script the drop + create so you can add them back later. If you have calculated fields drop them. (add em back later) Set the database size to double what you think the largest it should be. Turn off auto grow. Shrink the log, it shouldn't grow too much on simple recovery... Check your event log, may be some info there. In short, try and stop as many extra processes during the import as possible... might help. http://www.sswug.org has good info and it own mailing lists. cheers Michael Maddison DDI Solutions Pty Ltd michael at ddisolutions.com.au Bus: 0260400620 Mob: 0412620497 www.ddisolutions.com.au -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Thursday, 2 November 2006 9:52 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on LOL, borderline insanity. I can assure you that I am FULLY insane. Bwaaa haaaa haaaa. I am using SP Pro. SQL Server ran just fine on 2K pro, XP Pro is the 2K Pro base code. Why would it not run on that? I keep hearing that it isn't supposed to, but it does. I can tell you I would be irritated beyond belief if I went to Win 2003 server (which I have but haven't a clue how to install, since this is a desktop machine in the end) and I still had this issue. I actually tried to install 2003 server (in fact I have a disk with the beginnings of the install) and got to questions that implied I was supposed to know what I was doing (and I don't) so how am I supposed to install 2003 so that I can test your theory? I am not a notwork / OS admin, I am a developer. MS themselves push SQL Server lite as a developer tool. If anyone wants to talk me through a 2003 server install I will do that but... John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Wednesday, November 01, 2006 5:19 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Hi John: I am trying to cover all the bases. The problems initially appears to be with the MS SQL sever setting as they run fairly indendendantly from an OS. If the problem is not in the MS SQL setup, accumulating non responsive results sort of negates the hardware being the issue, then it has to be in the process. If it is absolutely not in the process then it must be in the MS SQL settings. Please tell me you are using server OS as it would be border-line insanity to use a desktop type OS and then I would assure you, without hesitation, that that is the problem. HTH Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 12:12 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Jim, I don't understand what this is doing for me. I have just one data file ATM. I don't know whether that is a good thing or a bad thing. This sounds like the process for creating and then redistributing data amongst multiple files. If you are saying that it is the existence of one huge file that is causing my slowdown, then I can certainly do that. I have to ask if this is as good as it gets with SQL Server. I understand that I am doing stuff with big tables but keerimeny. I started the build of a PKID field on the big table. It tells me that "ansi nulls was not on and it has to rebuild the table" so I do. EM just locks up tight, won't even redraw the screen (blank white) if you switch away and back again. I believe that it is hard at work but folks, this is the age of threads. So I open another instance of EM to work with another table. EM takes several seconds to expand each tree (server, databases, specific database) then when I try to expand the tables, the second instance of EM locks up tight with an hourglass. I mean c'mon. This is 2006, a monster (desktop) system and EM acts as if it is a DOS app from 1986 running on a '286 with 4 megs and swapping memory. The second EM instance has been trying to open the tables icon just to show me what tables are there for 20 minutes now. To say this is discouraging to work with would be an understatement. If this is the best I am going to get I am going to have to look for another database engine to work with. And the most discouraging part is that my dual proc system is cruising along using (average?) well under 25% of the processor while EM is locked up tight. Does SQl Server 2005 fix any of this? Or should I just go look at MySQL or Oracle personal edition. I have to get work done on this database and it is telling me that one job (building a field / index) on one table is all it can handle. And by the looks of it I will be locked out of this database for the next 12 to 24 hours. Hell, ACCESS can do better than this!!! Well, maybe not but SQL Server's rep is taking a beating here. John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Wed Nov 1 20:09:26 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Wed, 1 Nov 2006 21:09:26 -0500 Subject: [dba-SQLServer] What is going on In-Reply-To: <59A61174B1F5B54B97FD4ADDE71E7D0116AC71@ddi-01.DDI.local> Message-ID: <003f01c6fe23$ed0becd0$657aa8c0@m6805> At this point the import is finished. After a 12 hour!!! shrink operation the last file came in at the normal speed (25 minutes). I am now trying to add a PKID which is long etc. This looks like it is going to take another 12 hours. Dropping and adding indexes back is a grand idea, but on a 65 million record table when it takes 12 hours to build the index... Somehow I don't think I will drop them once built!!! The log file for this is now up to 118 gbytes and the main data file is back up to 221 gbytes after taking 12 hours to shrink it down to ~100 gbytes. TWELVE HOURS to shrink the damned file!!! Man oh man, I'm 52 years old. I could DIE before I get any real work done here. Once this is built, it is totally a read-only kind of operation. This is just a list of names / addresses and various info fields, which will be queried for counts of things like "count of names in a zip", "count of all people in an income bracket etc. The data still has to be cassed (address validation), ncoa'd, but after that the only thing that will be done is an occasional ncoa operation. I have to believe I am running into a SQL Server "setting" of some sort, where "X% of the data is trying to be cached or something. Since the table is so large it can't. Just a guess but jeeze, I could just about do this in a VBA program faster than SQL Server is handling this thing. There are indications that something isn't right. The page file is maxed out (I guess) at 1.84 gb, unchanging. I actually have 2 gb of RAM in the machine, and the page file is set to 4092 mb but no more or less is being used, it is a flat line at this point in the task manager. The CPU usage is sitting stable at 50% although the cpu usage history shows a varying graph. 8361 handles, 394 threads, 30 processes, all stable as a rock. It is chunking away, but I started this process at 2:30 this afternoon and it is now 8:55 so whatever it is doing it isn't doing very darned fast. Jim seems to think this is an OS issue, why I am not sure. If it ever comes back perhaps I'll go try to install Win 2003 Server again but I have my doubts. I have to admit it is nice knowing that the machine will be busy the whole time I'm asleep though. I didn't put all this money into this thing to see it sit around idle. ;-) John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Michael Maddison Sent: Wednesday, November 01, 2006 8:06 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Hi John, You are out of my league but... a couple of ideas come to mind. Set the database recovery Model to simple. Turn off Auto update and auto create statistics. Drop any/all indexes on the table. You can script the drop + create so you can add them back later. If you have calculated fields drop them. (add em back later) Set the database size to double what you think the largest it should be. Turn off auto grow. Shrink the log, it shouldn't grow too much on simple recovery... Check your event log, may be some info there. In short, try and stop as many extra processes during the import as possible... might help. http://www.sswug.org has good info and it own mailing lists. cheers Michael Maddison DDI Solutions Pty Ltd michael at ddisolutions.com.au Bus: 0260400620 Mob: 0412620497 www.ddisolutions.com.au -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Thursday, 2 November 2006 9:52 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on LOL, borderline insanity. I can assure you that I am FULLY insane. Bwaaa haaaa haaaa. I am using SP Pro. SQL Server ran just fine on 2K pro, XP Pro is the 2K Pro base code. Why would it not run on that? I keep hearing that it isn't supposed to, but it does. I can tell you I would be irritated beyond belief if I went to Win 2003 server (which I have but haven't a clue how to install, since this is a desktop machine in the end) and I still had this issue. I actually tried to install 2003 server (in fact I have a disk with the beginnings of the install) and got to questions that implied I was supposed to know what I was doing (and I don't) so how am I supposed to install 2003 so that I can test your theory? I am not a notwork / OS admin, I am a developer. MS themselves push SQL Server lite as a developer tool. If anyone wants to talk me through a 2003 server install I will do that but... John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Wednesday, November 01, 2006 5:19 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Hi John: I am trying to cover all the bases. The problems initially appears to be with the MS SQL sever setting as they run fairly indendendantly from an OS. If the problem is not in the MS SQL setup, accumulating non responsive results sort of negates the hardware being the issue, then it has to be in the process. If it is absolutely not in the process then it must be in the MS SQL settings. Please tell me you are using server OS as it would be border-line insanity to use a desktop type OS and then I would assure you, without hesitation, that that is the problem. HTH Jim From rl_stewart at highstream.net Wed Nov 1 20:36:53 2006 From: rl_stewart at highstream.net (Robert L. Stewart) Date: Wed, 01 Nov 2006 20:36:53 -0600 Subject: [dba-SQLServer] What is going on In-Reply-To: References: Message-ID: <200611020237.kA22bRq00994@databaseadvisors.com> John, One of the "limitations" you are running into is the OS. Yes, you can do development work in XP. But, running a 68 million record DB is not "development" by any stretch of the imagination. You are trying to run a serious production database on a home OS. I am not sure what setting the install was asking for that were so different from XP. I have installed 2000 and 2003 server a number of times and other than questions about DNS server and such things, which you do not care about because you should say "Stand alone server," the install is the same as XP. You could also spend the $100 or so it would take for a professional to do the install for you. Once you use a server OS, you will see some of this go away. One of the other questions I have not heard anyone ask, is what version of SQL Server are you using? If you are using anything less than standard version, you are hitting another "limitation." You are trying to load a ton of crap into a trailer on a lawn tractor. Robert At 08:09 PM 11/1/2006, you wrote: >Date: Wed, 1 Nov 2006 17:51:44 -0500 >From: "JWColby" >Subject: Re: [dba-SQLServer] What is going on >To: >Message-ID: <003e01c6fe08$4e748390$657aa8c0 at m6805> >Content-Type: text/plain; charset="us-ascii" > >LOL, borderline insanity. I can assure you that I am FULLY insane. Bwaaa >haaaa haaaa. I am using SP Pro. SQL Server ran just fine on 2K pro, XP Pro >is the 2K Pro base code. Why would it not run on that? I keep hearing that >it isn't supposed to, but it does. I can tell you I would be irritated >beyond belief if I went to Win 2003 server (which I have but haven't a clue >how to install, since this is a desktop machine in the end) and I still had >this issue. > >I actually tried to install 2003 server (in fact I have a disk with the >beginnings of the install) and got to questions that implied I was supposed >to know what I was doing (and I don't) so how am I supposed to install 2003 >so that I can test your theory? > >I am not a notwork / OS admin, I am a developer. MS themselves push SQL >Server lite as a developer tool. > >If anyone wants to talk me through a 2003 server install I will do that >but... > >John W. Colby >Colby Consulting >www.ColbyConsulting.com From jwcolby at colbyconsulting.com Wed Nov 1 20:54:22 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Wed, 1 Nov 2006 21:54:22 -0500 Subject: [dba-SQLServer] What is going on In-Reply-To: <200611020237.kA22bRq00994@databaseadvisors.com> Message-ID: <004001c6fe2a$33b09590$657aa8c0@m6805> I am perfectly willing to pay a professional to do the OS install for me, or even talk me through it (more likely scenario given my physical location). Likewise for the SQL Server install. I really just need it to work. I have to tell you though that I have my doubts. Glad to be proven wrong but if a couple of hundred makes this thing usable it is well worth the money. And since when was XP Professional a "home OS"? It is amazing to me how NT was considered a server, 2K Pro was considered a server OS, but XP Pro is considered a "Home OS". John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Robert L. Stewart Sent: Wednesday, November 01, 2006 9:37 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on John, One of the "limitations" you are running into is the OS. Yes, you can do development work in XP. But, running a 68 million record DB is not "development" by any stretch of the imagination. You are trying to run a serious production database on a home OS. I am not sure what setting the install was asking for that were so different from XP. I have installed 2000 and 2003 server a number of times and other than questions about DNS server and such things, which you do not care about because you should say "Stand alone server," the install is the same as XP. You could also spend the $100 or so it would take for a professional to do the install for you. Once you use a server OS, you will see some of this go away. One of the other questions I have not heard anyone ask, is what version of SQL Server are you using? If you are using anything less than standard version, you are hitting another "limitation." You are trying to load a ton of crap into a trailer on a lawn tractor. Robert At 08:09 PM 11/1/2006, you wrote: >Date: Wed, 1 Nov 2006 17:51:44 -0500 >From: "JWColby" >Subject: Re: [dba-SQLServer] What is going on >To: >Message-ID: <003e01c6fe08$4e748390$657aa8c0 at m6805> >Content-Type: text/plain; charset="us-ascii" > >LOL, borderline insanity. I can assure you that I am FULLY insane. >Bwaaa haaaa haaaa. I am using SP Pro. SQL Server ran just fine on 2K >pro, XP Pro is the 2K Pro base code. Why would it not run on that? I >keep hearing that it isn't supposed to, but it does. I can tell you I >would be irritated beyond belief if I went to Win 2003 server (which I >have but haven't a clue how to install, since this is a desktop machine >in the end) and I still had this issue. > >I actually tried to install 2003 server (in fact I have a disk with the >beginnings of the install) and got to questions that implied I was >supposed to know what I was doing (and I don't) so how am I supposed to >install 2003 so that I can test your theory? > >I am not a notwork / OS admin, I am a developer. MS themselves push >SQL Server lite as a developer tool. > >If anyone wants to talk me through a 2003 server install I will do that >but... > >John W. Colby >Colby Consulting >www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jlawrenc1 at shaw.ca Wed Nov 1 21:11:56 2006 From: jlawrenc1 at shaw.ca (Jim Lawrence) Date: Wed, 01 Nov 2006 19:11:56 -0800 Subject: [dba-SQLServer] What is going on In-Reply-To: <003e01c6fe08$4e748390$657aa8c0@m6805> Message-ID: <0J830079Q25MQC10@l-daemon> Hi John: As you are not going, right at the start, into Active Directory (workgroups will work fine), installing Server2003 is as simple as installing XP.. The whole process tends to be a disk jockey nodding off between switch disk and answering such questions as, "What is your product key" and do you want to install IIS. If I can do it so can you. Server2003 can run multiple instances and processes. It will run applications that will over-load any XP desktop box. It has better isolation so if a program misbehaves it does not bring down the system. I understood you were on the MS partner plan and therefore you should have access to all the servers you can stand. Server 2003 standard or developer is just fine unless you are planning to run virtual and exchange server etc. I have taken the luxury to run Server 2003 as my desktop, one that has never gone down... Well only once and that was my fault... But it is really stable. MS SQL and IIS run as background processes and their impact is hardly noticeable. Sorry if this sounds like a lecture but I am sure your problems can be solved by going this route. Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 2:52 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on LOL, borderline insanity. I can assure you that I am FULLY insane. Bwaaa haaaa haaaa. I am using SP Pro. SQL Server ran just fine on 2K pro, XP Pro is the 2K Pro base code. Why would it not run on that? I keep hearing that it isn't supposed to, but it does. I can tell you I would be irritated beyond belief if I went to Win 2003 server (which I have but haven't a clue how to install, since this is a desktop machine in the end) and I still had this issue. I actually tried to install 2003 server (in fact I have a disk with the beginnings of the install) and got to questions that implied I was supposed to know what I was doing (and I don't) so how am I supposed to install 2003 so that I can test your theory? I am not a notwork / OS admin, I am a developer. MS themselves push SQL Server lite as a developer tool. If anyone wants to talk me through a 2003 server install I will do that but... John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Wednesday, November 01, 2006 5:19 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Hi John: I am trying to cover all the bases. The problems initially appears to be with the MS SQL sever setting as they run fairly indendendantly from an OS. If the problem is not in the MS SQL setup, accumulating non responsive results sort of negates the hardware being the issue, then it has to be in the process. If it is absolutely not in the process then it must be in the MS SQL settings. Please tell me you are using server OS as it would be border-line insanity to use a desktop type OS and then I would assure you, without hesitation, that that is the problem. HTH Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 12:12 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Jim, I don't understand what this is doing for me. I have just one data file ATM. I don't know whether that is a good thing or a bad thing. This sounds like the process for creating and then redistributing data amongst multiple files. If you are saying that it is the existence of one huge file that is causing my slowdown, then I can certainly do that. I have to ask if this is as good as it gets with SQL Server. I understand that I am doing stuff with big tables but keerimeny. I started the build of a PKID field on the big table. It tells me that "ansi nulls was not on and it has to rebuild the table" so I do. EM just locks up tight, won't even redraw the screen (blank white) if you switch away and back again. I believe that it is hard at work but folks, this is the age of threads. So I open another instance of EM to work with another table. EM takes several seconds to expand each tree (server, databases, specific database) then when I try to expand the tables, the second instance of EM locks up tight with an hourglass. I mean c'mon. This is 2006, a monster (desktop) system and EM acts as if it is a DOS app from 1986 running on a '286 with 4 megs and swapping memory. The second EM instance has been trying to open the tables icon just to show me what tables are there for 20 minutes now. To say this is discouraging to work with would be an understatement. If this is the best I am going to get I am going to have to look for another database engine to work with. And the most discouraging part is that my dual proc system is cruising along using (average?) well under 25% of the processor while EM is locked up tight. Does SQl Server 2005 fix any of this? Or should I just go look at MySQL or Oracle personal edition. I have to get work done on this database and it is telling me that one job (building a field / index) on one table is all it can handle. And by the looks of it I will be locked out of this database for the next 12 to 24 hours. Hell, ACCESS can do better than this!!! Well, maybe not but SQL Server's rep is taking a beating here. John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Wed Nov 1 21:37:38 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Wed, 1 Nov 2006 22:37:38 -0500 Subject: [dba-SQLServer] What is going on In-Reply-To: <0J830079Q25MQC10@l-daemon> Message-ID: <004101c6fe30$3f25b940$657aa8c0@m6805> Jim, No, I actually appreciate the lecture if it works. I don't have an issue with going there if I am capable. I tried it on my past mb and it wouldn't install for the simple reason that the driver disk did a check and refused to install because it wasn't XP. This MB does not have that problem. All I have to do now is wait for SQL Server to stop. I could reboot the machine but that would just cause more problems down the road I am sure. I have to detach this database before I start the OS install. Tomorrow sounds like fun. 8-( John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Wednesday, November 01, 2006 10:12 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Hi John: As you are not going, right at the start, into Active Directory (workgroups will work fine), installing Server2003 is as simple as installing XP.. The whole process tends to be a disk jockey nodding off between switch disk and answering such questions as, "What is your product key" and do you want to install IIS. If I can do it so can you. Server2003 can run multiple instances and processes. It will run applications that will over-load any XP desktop box. It has better isolation so if a program misbehaves it does not bring down the system. I understood you were on the MS partner plan and therefore you should have access to all the servers you can stand. Server 2003 standard or developer is just fine unless you are planning to run virtual and exchange server etc. I have taken the luxury to run Server 2003 as my desktop, one that has never gone down... Well only once and that was my fault... But it is really stable. MS SQL and IIS run as background processes and their impact is hardly noticeable. Sorry if this sounds like a lecture but I am sure your problems can be solved by going this route. Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 2:52 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on LOL, borderline insanity. I can assure you that I am FULLY insane. Bwaaa haaaa haaaa. I am using SP Pro. SQL Server ran just fine on 2K pro, XP Pro is the 2K Pro base code. Why would it not run on that? I keep hearing that it isn't supposed to, but it does. I can tell you I would be irritated beyond belief if I went to Win 2003 server (which I have but haven't a clue how to install, since this is a desktop machine in the end) and I still had this issue. I actually tried to install 2003 server (in fact I have a disk with the beginnings of the install) and got to questions that implied I was supposed to know what I was doing (and I don't) so how am I supposed to install 2003 so that I can test your theory? I am not a notwork / OS admin, I am a developer. MS themselves push SQL Server lite as a developer tool. If anyone wants to talk me through a 2003 server install I will do that but... John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Wednesday, November 01, 2006 5:19 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Hi John: I am trying to cover all the bases. The problems initially appears to be with the MS SQL sever setting as they run fairly indendendantly from an OS. If the problem is not in the MS SQL setup, accumulating non responsive results sort of negates the hardware being the issue, then it has to be in the process. If it is absolutely not in the process then it must be in the MS SQL settings. Please tell me you are using server OS as it would be border-line insanity to use a desktop type OS and then I would assure you, without hesitation, that that is the problem. HTH Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 12:12 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Jim, I don't understand what this is doing for me. I have just one data file ATM. I don't know whether that is a good thing or a bad thing. This sounds like the process for creating and then redistributing data amongst multiple files. If you are saying that it is the existence of one huge file that is causing my slowdown, then I can certainly do that. I have to ask if this is as good as it gets with SQL Server. I understand that I am doing stuff with big tables but keerimeny. I started the build of a PKID field on the big table. It tells me that "ansi nulls was not on and it has to rebuild the table" so I do. EM just locks up tight, won't even redraw the screen (blank white) if you switch away and back again. I believe that it is hard at work but folks, this is the age of threads. So I open another instance of EM to work with another table. EM takes several seconds to expand each tree (server, databases, specific database) then when I try to expand the tables, the second instance of EM locks up tight with an hourglass. I mean c'mon. This is 2006, a monster (desktop) system and EM acts as if it is a DOS app from 1986 running on a '286 with 4 megs and swapping memory. The second EM instance has been trying to open the tables icon just to show me what tables are there for 20 minutes now. To say this is discouraging to work with would be an understatement. If this is the best I am going to get I am going to have to look for another database engine to work with. And the most discouraging part is that my dual proc system is cruising along using (average?) well under 25% of the processor while EM is locked up tight. Does SQl Server 2005 fix any of this? Or should I just go look at MySQL or Oracle personal edition. I have to get work done on this database and it is telling me that one job (building a field / index) on one table is all it can handle. And by the looks of it I will be locked out of this database for the next 12 to 24 hours. Hell, ACCESS can do better than this!!! Well, maybe not but SQL Server's rep is taking a beating here. John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From mwp.reid at qub.ac.uk Thu Nov 2 02:56:50 2006 From: mwp.reid at qub.ac.uk (Martin Reid) Date: Thu, 2 Nov 2006 08:56:50 -0000 Subject: [dba-SQLServer] What is going on References: <0J8200BAYOM464D1@l-daemon> Message-ID: John I have a friend here who I am sure will help you install Windows Server 2003. He helps me. I have installed it several times with little or no hassle BUT I do have help with areas like you I know nothing about and have no great desire to learn (<: He is a Server Admin in the Univeristy and very very good with all things MS. Gimme a shout if you would like me to ask him. Martin Martin WP Reid Training and Assessment Unit Riddle Hall Belfast tel: 02890 974477 From jwcolby at colbyconsulting.com Thu Nov 2 09:16:06 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Thu, 2 Nov 2006 10:16:06 -0500 Subject: [dba-SQLServer] Installing SBS2003 - was What is going on In-Reply-To: <0J830079Q25MQC10@l-daemon> Message-ID: <005601c6fe91$d27eb4b0$657aa8c0@m6805> Well, nothing ever goes as planned. I own SBS 2003. I started the install which proceeded as smoothly as any of this ever does, which means plenty of "the F6 floppy doesn't work, but it doesn't say that, so spend an hour figuring that out, find the driver on the internet etc.", and "oh I need an active network controller, and what do you mean you have to install a driver disk (right in the middle of the install) to give me that" etc. In the meantime, it gets to some screen where it just informs me that it is installing active directory. >As you are not going, right at the start, into Active Directory (workgroups will work fine), installing Server2003 is as simple as installing XP. So now it appears that I AM installing active directory, whether I need or want it or not. So (from your email) perhaps the install will NOT be as easy as installing XP? I am doing a dual boot, installing SBS2003 to the raid 6 D: drive (might as well use this raid stuff to make the system install more fault tolerant). I just got to a screen where it is asking me what components I want, things like server tools, exchange server, fax service. So what do I need, and will it "just install" or will it ask me questions which I am not prepared to answer? William, you are the one always touting how great SBS is. Any words of wisdom on this? Man I hate this crap! John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Wednesday, November 01, 2006 10:12 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Hi John: As you are not going, right at the start, into Active Directory (workgroups will work fine), installing Server2003 is as simple as installing XP.. The whole process tends to be a disk jockey nodding off between switch disk and answering such questions as, "What is your product key" and do you want to install IIS. If I can do it so can you. Server2003 can run multiple instances and processes. It will run applications that will over-load any XP desktop box. It has better isolation so if a program misbehaves it does not bring down the system. I understood you were on the MS partner plan and therefore you should have access to all the servers you can stand. Server 2003 standard or developer is just fine unless you are planning to run virtual and exchange server etc. I have taken the luxury to run Server 2003 as my desktop, one that has never gone down... Well only once and that was my fault... But it is really stable. MS SQL and IIS run as background processes and their impact is hardly noticeable. Sorry if this sounds like a lecture but I am sure your problems can be solved by going this route. Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 2:52 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on LOL, borderline insanity. I can assure you that I am FULLY insane. Bwaaa haaaa haaaa. I am using SP Pro. SQL Server ran just fine on 2K pro, XP Pro is the 2K Pro base code. Why would it not run on that? I keep hearing that it isn't supposed to, but it does. I can tell you I would be irritated beyond belief if I went to Win 2003 server (which I have but haven't a clue how to install, since this is a desktop machine in the end) and I still had this issue. I actually tried to install 2003 server (in fact I have a disk with the beginnings of the install) and got to questions that implied I was supposed to know what I was doing (and I don't) so how am I supposed to install 2003 so that I can test your theory? I am not a notwork / OS admin, I am a developer. MS themselves push SQL Server lite as a developer tool. If anyone wants to talk me through a 2003 server install I will do that but... John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Wednesday, November 01, 2006 5:19 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Hi John: I am trying to cover all the bases. The problems initially appears to be with the MS SQL sever setting as they run fairly indendendantly from an OS. If the problem is not in the MS SQL setup, accumulating non responsive results sort of negates the hardware being the issue, then it has to be in the process. If it is absolutely not in the process then it must be in the MS SQL settings. Please tell me you are using server OS as it would be border-line insanity to use a desktop type OS and then I would assure you, without hesitation, that that is the problem. HTH Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 12:12 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Jim, I don't understand what this is doing for me. I have just one data file ATM. I don't know whether that is a good thing or a bad thing. This sounds like the process for creating and then redistributing data amongst multiple files. If you are saying that it is the existence of one huge file that is causing my slowdown, then I can certainly do that. I have to ask if this is as good as it gets with SQL Server. I understand that I am doing stuff with big tables but keerimeny. I started the build of a PKID field on the big table. It tells me that "ansi nulls was not on and it has to rebuild the table" so I do. EM just locks up tight, won't even redraw the screen (blank white) if you switch away and back again. I believe that it is hard at work but folks, this is the age of threads. So I open another instance of EM to work with another table. EM takes several seconds to expand each tree (server, databases, specific database) then when I try to expand the tables, the second instance of EM locks up tight with an hourglass. I mean c'mon. This is 2006, a monster (desktop) system and EM acts as if it is a DOS app from 1986 running on a '286 with 4 megs and swapping memory. The second EM instance has been trying to open the tables icon just to show me what tables are there for 20 minutes now. To say this is discouraging to work with would be an understatement. If this is the best I am going to get I am going to have to look for another database engine to work with. And the most discouraging part is that my dual proc system is cruising along using (average?) well under 25% of the processor while EM is locked up tight. Does SQl Server 2005 fix any of this? Or should I just go look at MySQL or Oracle personal edition. I have to get work done on this database and it is telling me that one job (building a field / index) on one table is all it can handle. And by the looks of it I will be locked out of this database for the next 12 to 24 hours. Hell, ACCESS can do better than this!!! Well, maybe not but SQL Server's rep is taking a beating here. John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Thu Nov 2 10:15:01 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Thu, 2 Nov 2006 11:15:01 -0500 Subject: [dba-SQLServer] Installing SBS2003 - was What is going on In-Reply-To: <005601c6fe91$d27eb4b0$657aa8c0@m6805> Message-ID: <005a01c6fe9a$0d2f2510$657aa8c0@m6805> 11:00 am and the saga continues. SBS Server 2003 got to a point where it demanded a Outlook 2003 disk. Hmmmm... Don't have one. I tried the OFFICE 2003 disk, not good enough. I finally said cancel and it told me the installation was aborting and I could continue later. I tried to install the Office 2003 disk, which I have, but it apparently was in the wrong sleeve because the product key failed. Aaaargh. Rebooted, and am up at the "To Do List" where it appears I have an entire afternoon of viewing security practices, activating this, configuring that. LOL, what else could better I possibly have to do on a beautiful and sunny Thursday morning / afternoon / night? John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Thursday, November 02, 2006 10:16 AM To: dba-sqlserver at databaseadvisors.com; 'Access Developers discussion and problem solving' Subject: [dba-SQLServer] Installing SBS2003 - was What is going on Well, nothing ever goes as planned. I own SBS 2003. I started the install which proceeded as smoothly as any of this ever does, which means plenty of "the F6 floppy doesn't work, but it doesn't say that, so spend an hour figuring that out, find the driver on the internet etc.", and "oh I need an active network controller, and what do you mean you have to install a driver disk (right in the middle of the install) to give me that" etc. In the meantime, it gets to some screen where it just informs me that it is installing active directory. >As you are not going, right at the start, into Active Directory >(workgroups will work fine), installing Server2003 is as simple as installing XP. So now it appears that I AM installing active directory, whether I need or want it or not. So (from your email) perhaps the install will NOT be as easy as installing XP? I am doing a dual boot, installing SBS2003 to the raid 6 D: drive (might as well use this raid stuff to make the system install more fault tolerant). I just got to a screen where it is asking me what components I want, things like server tools, exchange server, fax service. So what do I need, and will it "just install" or will it ask me questions which I am not prepared to answer? William, you are the one always touting how great SBS is. Any words of wisdom on this? Man I hate this crap! John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Wednesday, November 01, 2006 10:12 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Hi John: As you are not going, right at the start, into Active Directory (workgroups will work fine), installing Server2003 is as simple as installing XP.. The whole process tends to be a disk jockey nodding off between switch disk and answering such questions as, "What is your product key" and do you want to install IIS. If I can do it so can you. Server2003 can run multiple instances and processes. It will run applications that will over-load any XP desktop box. It has better isolation so if a program misbehaves it does not bring down the system. I understood you were on the MS partner plan and therefore you should have access to all the servers you can stand. Server 2003 standard or developer is just fine unless you are planning to run virtual and exchange server etc. I have taken the luxury to run Server 2003 as my desktop, one that has never gone down... Well only once and that was my fault... But it is really stable. MS SQL and IIS run as background processes and their impact is hardly noticeable. Sorry if this sounds like a lecture but I am sure your problems can be solved by going this route. Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 2:52 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on LOL, borderline insanity. I can assure you that I am FULLY insane. Bwaaa haaaa haaaa. I am using SP Pro. SQL Server ran just fine on 2K pro, XP Pro is the 2K Pro base code. Why would it not run on that? I keep hearing that it isn't supposed to, but it does. I can tell you I would be irritated beyond belief if I went to Win 2003 server (which I have but haven't a clue how to install, since this is a desktop machine in the end) and I still had this issue. I actually tried to install 2003 server (in fact I have a disk with the beginnings of the install) and got to questions that implied I was supposed to know what I was doing (and I don't) so how am I supposed to install 2003 so that I can test your theory? I am not a notwork / OS admin, I am a developer. MS themselves push SQL Server lite as a developer tool. If anyone wants to talk me through a 2003 server install I will do that but... John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Wednesday, November 01, 2006 5:19 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Hi John: I am trying to cover all the bases. The problems initially appears to be with the MS SQL sever setting as they run fairly indendendantly from an OS. If the problem is not in the MS SQL setup, accumulating non responsive results sort of negates the hardware being the issue, then it has to be in the process. If it is absolutely not in the process then it must be in the MS SQL settings. Please tell me you are using server OS as it would be border-line insanity to use a desktop type OS and then I would assure you, without hesitation, that that is the problem. HTH Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 12:12 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Jim, I don't understand what this is doing for me. I have just one data file ATM. I don't know whether that is a good thing or a bad thing. This sounds like the process for creating and then redistributing data amongst multiple files. If you are saying that it is the existence of one huge file that is causing my slowdown, then I can certainly do that. I have to ask if this is as good as it gets with SQL Server. I understand that I am doing stuff with big tables but keerimeny. I started the build of a PKID field on the big table. It tells me that "ansi nulls was not on and it has to rebuild the table" so I do. EM just locks up tight, won't even redraw the screen (blank white) if you switch away and back again. I believe that it is hard at work but folks, this is the age of threads. So I open another instance of EM to work with another table. EM takes several seconds to expand each tree (server, databases, specific database) then when I try to expand the tables, the second instance of EM locks up tight with an hourglass. I mean c'mon. This is 2006, a monster (desktop) system and EM acts as if it is a DOS app from 1986 running on a '286 with 4 megs and swapping memory. The second EM instance has been trying to open the tables icon just to show me what tables are there for 20 minutes now. To say this is discouraging to work with would be an understatement. If this is the best I am going to get I am going to have to look for another database engine to work with. And the most discouraging part is that my dual proc system is cruising along using (average?) well under 25% of the processor while EM is locked up tight. Does SQl Server 2005 fix any of this? Or should I just go look at MySQL or Oracle personal edition. I have to get work done on this database and it is telling me that one job (building a field / index) on one table is all it can handle. And by the looks of it I will be locked out of this database for the next 12 to 24 hours. Hell, ACCESS can do better than this!!! Well, maybe not but SQL Server's rep is taking a beating here. John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From accessd at shaw.ca Thu Nov 2 11:04:01 2006 From: accessd at shaw.ca (Jim Lawrence) Date: Thu, 02 Nov 2006 09:04:01 -0800 Subject: [dba-SQLServer] Installing SBS2003 - was What is going on In-Reply-To: <005601c6fe91$d27eb4b0$657aa8c0@m6805> Message-ID: <0J84009TI4ODK7C0@l-daemon> Hi John: For help for installing Server2003 here is the definitive MS instructional base site location: http://technet2.microsoft.com/windowsserver/en/technologies/default.mspx. >From here you can access a large group of "how-to" manuals for viewing or printing. I had to print out the one on Active directory to get that running... (For me that area was a BEAR but it only took a systems friend 3 minutes to iron everything out... So most of my swags were right.) Here is a good place to start: http://technet2.microsoft.com/WindowsServer/f/?en/library/057105b9-9763-4d17 -ba19-ab1873a7e2961033.mspx As to your message: I just got to a screen where it is asking me what components I want, things like server tools, exchange server, fax service. So what do I need, and will it "just install" or will it ask me questions which I am not prepared to answer? You do not need exchange server (...it is a pig. If you decide to run your own emal server I would suggest the free one called Pegusus...) but fax server is nice but only if you have or intent to receive and send faxes directly from your server... Or from any other station on the network. It is a good idea to go 'dual-boot' or maybe virtual-PC but mostly for testing as like virtual-server require shared resources (translation; everything runs half as fast when they are operational.) This will get you started and individual questions can be answered from there. I will help where I can. Get the terminal admin running. It has to be selected as it does not install by default, then you or anyone else you give permissions to can access your server if any settings need testing and fixing. HTH Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Thursday, November 02, 2006 7:16 AM To: dba-sqlserver at databaseadvisors.com; 'Access Developers discussion and problem solving' Subject: [dba-SQLServer] Installing SBS2003 - was What is going on Well, nothing ever goes as planned. I own SBS 2003. I started the install which proceeded as smoothly as any of this ever does, which means plenty of "the F6 floppy doesn't work, but it doesn't say that, so spend an hour figuring that out, find the driver on the internet etc.", and "oh I need an active network controller, and what do you mean you have to install a driver disk (right in the middle of the install) to give me that" etc. In the meantime, it gets to some screen where it just informs me that it is installing active directory. >As you are not going, right at the start, into Active Directory >(workgroups will work fine), installing Server2003 is as simple as installing XP. So now it appears that I AM installing active directory, whether I need or want it or not. So (from your email) perhaps the install will NOT be as easy as installing XP? I am doing a dual boot, installing SBS2003 to the raid 6 D: drive (might as well use this raid stuff to make the system install more fault tolerant). I just got to a screen where it is asking me what components I want, things like server tools, exchange server, fax service. So what do I need, and will it "just install" or will it ask me questions which I am not prepared to answer? William, you are the one always touting how great SBS is. Any words of wisdom on this? Man I hate this crap! John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Wednesday, November 01, 2006 10:12 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Hi John: As you are not going, right at the start, into Active Directory (workgroups will work fine), installing Server2003 is as simple as installing XP.. The whole process tends to be a disk jockey nodding off between switch disk and answering such questions as, "What is your product key" and do you want to install IIS. If I can do it so can you. Server2003 can run multiple instances and processes. It will run applications that will over-load any XP desktop box. It has better isolation so if a program misbehaves it does not bring down the system. I understood you were on the MS partner plan and therefore you should have access to all the servers you can stand. Server 2003 standard or developer is just fine unless you are planning to run virtual and exchange server etc. I have taken the luxury to run Server 2003 as my desktop, one that has never gone down... Well only once and that was my fault... But it is really stable. MS SQL and IIS run as background processes and their impact is hardly noticeable. Sorry if this sounds like a lecture but I am sure your problems can be solved by going this route. Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 2:52 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on LOL, borderline insanity. I can assure you that I am FULLY insane. Bwaaa haaaa haaaa. I am using SP Pro. SQL Server ran just fine on 2K pro, XP Pro is the 2K Pro base code. Why would it not run on that? I keep hearing that it isn't supposed to, but it does. I can tell you I would be irritated beyond belief if I went to Win 2003 server (which I have but haven't a clue how to install, since this is a desktop machine in the end) and I still had this issue. I actually tried to install 2003 server (in fact I have a disk with the beginnings of the install) and got to questions that implied I was supposed to know what I was doing (and I don't) so how am I supposed to install 2003 so that I can test your theory? I am not a notwork / OS admin, I am a developer. MS themselves push SQL Server lite as a developer tool. If anyone wants to talk me through a 2003 server install I will do that but... John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Wednesday, November 01, 2006 5:19 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Hi John: I am trying to cover all the bases. The problems initially appears to be with the MS SQL sever setting as they run fairly indendendantly from an OS. If the problem is not in the MS SQL setup, accumulating non responsive results sort of negates the hardware being the issue, then it has to be in the process. If it is absolutely not in the process then it must be in the MS SQL settings. Please tell me you are using server OS as it would be border-line insanity to use a desktop type OS and then I would assure you, without hesitation, that that is the problem. HTH Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 12:12 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Jim, I don't understand what this is doing for me. I have just one data file ATM. I don't know whether that is a good thing or a bad thing. This sounds like the process for creating and then redistributing data amongst multiple files. If you are saying that it is the existence of one huge file that is causing my slowdown, then I can certainly do that. I have to ask if this is as good as it gets with SQL Server. I understand that I am doing stuff with big tables but keerimeny. I started the build of a PKID field on the big table. It tells me that "ansi nulls was not on and it has to rebuild the table" so I do. EM just locks up tight, won't even redraw the screen (blank white) if you switch away and back again. I believe that it is hard at work but folks, this is the age of threads. So I open another instance of EM to work with another table. EM takes several seconds to expand each tree (server, databases, specific database) then when I try to expand the tables, the second instance of EM locks up tight with an hourglass. I mean c'mon. This is 2006, a monster (desktop) system and EM acts as if it is a DOS app from 1986 running on a '286 with 4 megs and swapping memory. The second EM instance has been trying to open the tables icon just to show me what tables are there for 20 minutes now. To say this is discouraging to work with would be an understatement. If this is the best I am going to get I am going to have to look for another database engine to work with. And the most discouraging part is that my dual proc system is cruising along using (average?) well under 25% of the processor while EM is locked up tight. Does SQl Server 2005 fix any of this? Or should I just go look at MySQL or Oracle personal edition. I have to get work done on this database and it is telling me that one job (building a field / index) on one table is all it can handle. And by the looks of it I will be locked out of this database for the next 12 to 24 hours. Hell, ACCESS can do better than this!!! Well, maybe not but SQL Server's rep is taking a beating here. John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From accessd at shaw.ca Thu Nov 2 11:08:32 2006 From: accessd at shaw.ca (Jim Lawrence) Date: Thu, 02 Nov 2006 09:08:32 -0800 Subject: [dba-SQLServer] Installing SBS2003 - was What is going on In-Reply-To: <005a01c6fe9a$0d2f2510$657aa8c0@m6805> Message-ID: <0J84009PB4VWK6D0@l-daemon> Hi John: Which version of Server 2003 did you decide to install? I will be installing Miscrosoft Windows Server 2003, standard edition, this morning for a client.... Maybe we can go through this together. Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Thursday, November 02, 2006 8:15 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Installing SBS2003 - was What is going on 11:00 am and the saga continues. SBS Server 2003 got to a point where it demanded a Outlook 2003 disk. Hmmmm... Don't have one. I tried the OFFICE 2003 disk, not good enough. I finally said cancel and it told me the installation was aborting and I could continue later. I tried to install the Office 2003 disk, which I have, but it apparently was in the wrong sleeve because the product key failed. Aaaargh. Rebooted, and am up at the "To Do List" where it appears I have an entire afternoon of viewing security practices, activating this, configuring that. LOL, what else could better I possibly have to do on a beautiful and sunny Thursday morning / afternoon / night? John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Thursday, November 02, 2006 10:16 AM To: dba-sqlserver at databaseadvisors.com; 'Access Developers discussion and problem solving' Subject: [dba-SQLServer] Installing SBS2003 - was What is going on Well, nothing ever goes as planned. I own SBS 2003. I started the install which proceeded as smoothly as any of this ever does, which means plenty of "the F6 floppy doesn't work, but it doesn't say that, so spend an hour figuring that out, find the driver on the internet etc.", and "oh I need an active network controller, and what do you mean you have to install a driver disk (right in the middle of the install) to give me that" etc. In the meantime, it gets to some screen where it just informs me that it is installing active directory. >As you are not going, right at the start, into Active Directory >(workgroups will work fine), installing Server2003 is as simple as installing XP. So now it appears that I AM installing active directory, whether I need or want it or not. So (from your email) perhaps the install will NOT be as easy as installing XP? I am doing a dual boot, installing SBS2003 to the raid 6 D: drive (might as well use this raid stuff to make the system install more fault tolerant). I just got to a screen where it is asking me what components I want, things like server tools, exchange server, fax service. So what do I need, and will it "just install" or will it ask me questions which I am not prepared to answer? William, you are the one always touting how great SBS is. Any words of wisdom on this? Man I hate this crap! John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Wednesday, November 01, 2006 10:12 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Hi John: As you are not going, right at the start, into Active Directory (workgroups will work fine), installing Server2003 is as simple as installing XP.. The whole process tends to be a disk jockey nodding off between switch disk and answering such questions as, "What is your product key" and do you want to install IIS. If I can do it so can you. Server2003 can run multiple instances and processes. It will run applications that will over-load any XP desktop box. It has better isolation so if a program misbehaves it does not bring down the system. I understood you were on the MS partner plan and therefore you should have access to all the servers you can stand. Server 2003 standard or developer is just fine unless you are planning to run virtual and exchange server etc. I have taken the luxury to run Server 2003 as my desktop, one that has never gone down... Well only once and that was my fault... But it is really stable. MS SQL and IIS run as background processes and their impact is hardly noticeable. Sorry if this sounds like a lecture but I am sure your problems can be solved by going this route. Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 2:52 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on LOL, borderline insanity. I can assure you that I am FULLY insane. Bwaaa haaaa haaaa. I am using SP Pro. SQL Server ran just fine on 2K pro, XP Pro is the 2K Pro base code. Why would it not run on that? I keep hearing that it isn't supposed to, but it does. I can tell you I would be irritated beyond belief if I went to Win 2003 server (which I have but haven't a clue how to install, since this is a desktop machine in the end) and I still had this issue. I actually tried to install 2003 server (in fact I have a disk with the beginnings of the install) and got to questions that implied I was supposed to know what I was doing (and I don't) so how am I supposed to install 2003 so that I can test your theory? I am not a notwork / OS admin, I am a developer. MS themselves push SQL Server lite as a developer tool. If anyone wants to talk me through a 2003 server install I will do that but... John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Wednesday, November 01, 2006 5:19 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Hi John: I am trying to cover all the bases. The problems initially appears to be with the MS SQL sever setting as they run fairly indendendantly from an OS. If the problem is not in the MS SQL setup, accumulating non responsive results sort of negates the hardware being the issue, then it has to be in the process. If it is absolutely not in the process then it must be in the MS SQL settings. Please tell me you are using server OS as it would be border-line insanity to use a desktop type OS and then I would assure you, without hesitation, that that is the problem. HTH Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 12:12 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Jim, I don't understand what this is doing for me. I have just one data file ATM. I don't know whether that is a good thing or a bad thing. This sounds like the process for creating and then redistributing data amongst multiple files. If you are saying that it is the existence of one huge file that is causing my slowdown, then I can certainly do that. I have to ask if this is as good as it gets with SQL Server. I understand that I am doing stuff with big tables but keerimeny. I started the build of a PKID field on the big table. It tells me that "ansi nulls was not on and it has to rebuild the table" so I do. EM just locks up tight, won't even redraw the screen (blank white) if you switch away and back again. I believe that it is hard at work but folks, this is the age of threads. So I open another instance of EM to work with another table. EM takes several seconds to expand each tree (server, databases, specific database) then when I try to expand the tables, the second instance of EM locks up tight with an hourglass. I mean c'mon. This is 2006, a monster (desktop) system and EM acts as if it is a DOS app from 1986 running on a '286 with 4 megs and swapping memory. The second EM instance has been trying to open the tables icon just to show me what tables are there for 20 minutes now. To say this is discouraging to work with would be an understatement. If this is the best I am going to get I am going to have to look for another database engine to work with. And the most discouraging part is that my dual proc system is cruising along using (average?) well under 25% of the processor while EM is locked up tight. Does SQl Server 2005 fix any of this? Or should I just go look at MySQL or Oracle personal edition. I have to get work done on this database and it is telling me that one job (building a field / index) on one table is all it can handle. And by the looks of it I will be locked out of this database for the next 12 to 24 hours. Hell, ACCESS can do better than this!!! Well, maybe not but SQL Server's rep is taking a beating here. John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Thu Nov 2 11:54:34 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Thu, 2 Nov 2006 12:54:34 -0500 Subject: [dba-SQLServer] Installing SBS2003 - was What is going on In-Reply-To: <0J84009TI4ODK7C0@l-daemon> Message-ID: <005b01c6fea7$f5eb2620$657aa8c0@m6805> SBS is installed dual boot to XP. SBS is the default in the dual boot menu. SBS seems to run just fine. It is giving me a message "at least one service didn't start" which I still have to investigate. Server 2003 appears to have at least one advantage over XP, which is that it supposedly can really use all of 4 gb (the max I can currently install on this MB). If that is true I will also order another 2g of RAM for this machine since it is up to it's eyeballs in a huge database. I did install all of that remote access stuff. It had terminal admin checked I think. As for actually getting past the firewalls... We shall see. I am in the XP system now detaching the database from SQL Server there. Rebooting into 2003 to attach there. Is there any way to get a status from SQL Server re progress when it is doing something like a shrink operation, building a new field or index etc? Any way to tell how much is done, how long remaining etc? John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Thursday, November 02, 2006 12:04 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Installing SBS2003 - was What is going on Hi John: For help for installing Server2003 here is the definitive MS instructional base site location: http://technet2.microsoft.com/windowsserver/en/technologies/default.mspx. >From here you can access a large group of "how-to" manuals for viewing >or printing. I had to print out the one on Active directory to get that running... (For me that area was a BEAR but it only took a systems friend 3 minutes to iron everything out... So most of my swags were right.) Here is a good place to start: http://technet2.microsoft.com/WindowsServer/f/?en/library/057105b9-9763-4d17 -ba19-ab1873a7e2961033.mspx As to your message: I just got to a screen where it is asking me what components I want, things like server tools, exchange server, fax service. So what do I need, and will it "just install" or will it ask me questions which I am not prepared to answer? You do not need exchange server (...it is a pig. If you decide to run your own emal server I would suggest the free one called Pegusus...) but fax server is nice but only if you have or intent to receive and send faxes directly from your server... Or from any other station on the network. It is a good idea to go 'dual-boot' or maybe virtual-PC but mostly for testing as like virtual-server require shared resources (translation; everything runs half as fast when they are operational.) This will get you started and individual questions can be answered from there. I will help where I can. Get the terminal admin running. It has to be selected as it does not install by default, then you or anyone else you give permissions to can access your server if any settings need testing and fixing. HTH Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Thursday, November 02, 2006 7:16 AM To: dba-sqlserver at databaseadvisors.com; 'Access Developers discussion and problem solving' Subject: [dba-SQLServer] Installing SBS2003 - was What is going on Well, nothing ever goes as planned. I own SBS 2003. I started the install which proceeded as smoothly as any of this ever does, which means plenty of "the F6 floppy doesn't work, but it doesn't say that, so spend an hour figuring that out, find the driver on the internet etc.", and "oh I need an active network controller, and what do you mean you have to install a driver disk (right in the middle of the install) to give me that" etc. In the meantime, it gets to some screen where it just informs me that it is installing active directory. >As you are not going, right at the start, into Active Directory >(workgroups will work fine), installing Server2003 is as simple as installing XP. So now it appears that I AM installing active directory, whether I need or want it or not. So (from your email) perhaps the install will NOT be as easy as installing XP? I am doing a dual boot, installing SBS2003 to the raid 6 D: drive (might as well use this raid stuff to make the system install more fault tolerant). I just got to a screen where it is asking me what components I want, things like server tools, exchange server, fax service. So what do I need, and will it "just install" or will it ask me questions which I am not prepared to answer? William, you are the one always touting how great SBS is. Any words of wisdom on this? Man I hate this crap! John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Wednesday, November 01, 2006 10:12 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Hi John: As you are not going, right at the start, into Active Directory (workgroups will work fine), installing Server2003 is as simple as installing XP.. The whole process tends to be a disk jockey nodding off between switch disk and answering such questions as, "What is your product key" and do you want to install IIS. If I can do it so can you. Server2003 can run multiple instances and processes. It will run applications that will over-load any XP desktop box. It has better isolation so if a program misbehaves it does not bring down the system. I understood you were on the MS partner plan and therefore you should have access to all the servers you can stand. Server 2003 standard or developer is just fine unless you are planning to run virtual and exchange server etc. I have taken the luxury to run Server 2003 as my desktop, one that has never gone down... Well only once and that was my fault... But it is really stable. MS SQL and IIS run as background processes and their impact is hardly noticeable. Sorry if this sounds like a lecture but I am sure your problems can be solved by going this route. Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 2:52 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on LOL, borderline insanity. I can assure you that I am FULLY insane. Bwaaa haaaa haaaa. I am using SP Pro. SQL Server ran just fine on 2K pro, XP Pro is the 2K Pro base code. Why would it not run on that? I keep hearing that it isn't supposed to, but it does. I can tell you I would be irritated beyond belief if I went to Win 2003 server (which I have but haven't a clue how to install, since this is a desktop machine in the end) and I still had this issue. I actually tried to install 2003 server (in fact I have a disk with the beginnings of the install) and got to questions that implied I was supposed to know what I was doing (and I don't) so how am I supposed to install 2003 so that I can test your theory? I am not a notwork / OS admin, I am a developer. MS themselves push SQL Server lite as a developer tool. If anyone wants to talk me through a 2003 server install I will do that but... John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Wednesday, November 01, 2006 5:19 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Hi John: I am trying to cover all the bases. The problems initially appears to be with the MS SQL sever setting as they run fairly indendendantly from an OS. If the problem is not in the MS SQL setup, accumulating non responsive results sort of negates the hardware being the issue, then it has to be in the process. If it is absolutely not in the process then it must be in the MS SQL settings. Please tell me you are using server OS as it would be border-line insanity to use a desktop type OS and then I would assure you, without hesitation, that that is the problem. HTH Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Wednesday, November 01, 2006 12:12 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] What is going on Jim, I don't understand what this is doing for me. I have just one data file ATM. I don't know whether that is a good thing or a bad thing. This sounds like the process for creating and then redistributing data amongst multiple files. If you are saying that it is the existence of one huge file that is causing my slowdown, then I can certainly do that. I have to ask if this is as good as it gets with SQL Server. I understand that I am doing stuff with big tables but keerimeny. I started the build of a PKID field on the big table. It tells me that "ansi nulls was not on and it has to rebuild the table" so I do. EM just locks up tight, won't even redraw the screen (blank white) if you switch away and back again. I believe that it is hard at work but folks, this is the age of threads. So I open another instance of EM to work with another table. EM takes several seconds to expand each tree (server, databases, specific database) then when I try to expand the tables, the second instance of EM locks up tight with an hourglass. I mean c'mon. This is 2006, a monster (desktop) system and EM acts as if it is a DOS app from 1986 running on a '286 with 4 megs and swapping memory. The second EM instance has been trying to open the tables icon just to show me what tables are there for 20 minutes now. To say this is discouraging to work with would be an understatement. If this is the best I am going to get I am going to have to look for another database engine to work with. And the most discouraging part is that my dual proc system is cruising along using (average?) well under 25% of the processor while EM is locked up tight. Does SQl Server 2005 fix any of this? Or should I just go look at MySQL or Oracle personal edition. I have to get work done on this database and it is telling me that one job (building a field / index) on one table is all it can handle. And by the looks of it I will be locked out of this database for the next 12 to 24 hours. Hell, ACCESS can do better than this!!! Well, maybe not but SQL Server's rep is taking a beating here. John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From accessd at shaw.ca Thu Nov 2 12:17:59 2006 From: accessd at shaw.ca (Jim Lawrence) Date: Thu, 02 Nov 2006 10:17:59 -0800 Subject: [dba-SQLServer] Installing SBS2003 - was What is going on In-Reply-To: <005b01c6fea7$f5eb2620$657aa8c0@m6805> Message-ID: <0J84009TL83MK6L0@l-daemon> Well that was fast John... Not as tough as you initially thought, eh??? Maybe this might help: http://sqljunkies.com/Article/7F8518F9-FDAA-4FF3-8FC5-25E8946C8D0C.scuk Regards Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Thursday, November 02, 2006 9:55 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Installing SBS2003 - was What is going on SBS is installed dual boot to XP. SBS is the default in the dual boot menu. SBS seems to run just fine. It is giving me a message "at least one service didn't start" which I still have to investigate. Server 2003 appears to have at least one advantage over XP, which is that it supposedly can really use all of 4 gb (the max I can currently install on this MB). If that is true I will also order another 2g of RAM for this machine since it is up to it's eyeballs in a huge database. I did install all of that remote access stuff. It had terminal admin checked I think. As for actually getting past the firewalls... We shall see. I am in the XP system now detaching the database from SQL Server there. Rebooting into 2003 to attach there. Is there any way to get a status from SQL Server re progress when it is doing something like a shrink operation, building a new field or index etc? Any way to tell how much is done, how long remaining etc? John W. Colby Colby Consulting www.ColbyConsulting.com From jwcolby at colbyconsulting.com Thu Nov 2 12:45:49 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Thu, 2 Nov 2006 13:45:49 -0500 Subject: [dba-SQLServer] Installing SBS2003 - was What is going on In-Reply-To: <0J84009TL83MK6L0@l-daemon> Message-ID: <006501c6feaf$1e33f010$657aa8c0@m6805> LOL, well I never said it was installed correctly. There is a TON of stuff that I just told I did but didn't do. But it seems to run. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Thursday, November 02, 2006 1:18 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Installing SBS2003 - was What is going on Well that was fast John... Not as tough as you initially thought, eh??? Maybe this might help: http://sqljunkies.com/Article/7F8518F9-FDAA-4FF3-8FC5-25E8946C8D0C.scuk Regards Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Thursday, November 02, 2006 9:55 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Installing SBS2003 - was What is going on SBS is installed dual boot to XP. SBS is the default in the dual boot menu. SBS seems to run just fine. It is giving me a message "at least one service didn't start" which I still have to investigate. Server 2003 appears to have at least one advantage over XP, which is that it supposedly can really use all of 4 gb (the max I can currently install on this MB). If that is true I will also order another 2g of RAM for this machine since it is up to it's eyeballs in a huge database. I did install all of that remote access stuff. It had terminal admin checked I think. As for actually getting past the firewalls... We shall see. I am in the XP system now detaching the database from SQL Server there. Rebooting into 2003 to attach there. Is there any way to get a status from SQL Server re progress when it is doing something like a shrink operation, building a new field or index etc? Any way to tell how much is done, how long remaining etc? John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Thu Nov 2 13:19:48 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Thu, 2 Nov 2006 14:19:48 -0500 Subject: [dba-SQLServer] Installing SBS2003 - was What is going on In-Reply-To: <0J84009TL83MK6L0@l-daemon> Message-ID: <006601c6feb3$dd7c4db0$657aa8c0@m6805> Well thank you to all who nagged me incessantly to move this machine to a true server. You all know who you are. I can tell you that several things have happened. Under XP, the load on the two cores of the dual core processor "followed" each other fairly closely in Task Manager, Performance tab. Not identical, but close. Under 2003, they are very independent graphs, one instance of SQL Server EM running a shrink operation is utilizing core 1 but core two remains pretty much unused. Opening another instance of EM happens pretty quickly, burrowing down to the database happens pretty quickly. Opening a smaller (3 million) table containing just one file from the big table happens at a "normal" speed and pegs Core 2 as it is opening. Paging down through the data causes core 2 to work, but does not affect core 1. Just as predicted by you know who you are. Whether the actual shrink operation gets any faster remains to be seen, but at least the database itself remains usable in some sense, which is really fascinating given that one core is in the process of trying to shrink the database file while the other core is trying to (and succeeding) find data in that same database file. So I can't yet say necessarily that any given operation is faster, but at least SQL Server is not locked up tight as it was under XP. In fact it appears that XP's implementation of or usage of dual cores is not as good as it might be given the behavior of the two cores in Task Manager. Thanks again to all the nags. John W. Colby Colby Consulting www.ColbyConsulting.com From stuart at lexacorp.com.pg Thu Nov 2 15:19:29 2006 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Fri, 03 Nov 2006 07:19:29 +1000 Subject: [dba-SQLServer] Installing SBS2003 - was What is going on In-Reply-To: <0J84009TI4ODK7C0@l-daemon> References: <005601c6fe91$d27eb4b0$657aa8c0@m6805>, <0J84009TI4ODK7C0@l-daemon> Message-ID: <454A60E1.23247.179ABB25@stuart.lexacorp.com.pg> On 2 Nov 2006 at 9:04, Jim Lawrence wrote: > > You do not need exchange server (...it is a pig. If you decide to run your > own emal server I would suggest the free one called Pegusus...) Here, here! But it's actually called Mercury :-) Pegasus Mail is the email client, Mercury Mail is the email server. Both written by the same person, David Harris They are available for download from the website http://www.pmail.com. -- Stuart From jwcolby at colbyconsulting.com Thu Nov 2 23:58:00 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Fri, 3 Nov 2006 00:58:00 -0500 Subject: [dba-SQLServer] Any gut feelings? Message-ID: <007b01c6ff0d$0552fba0$657aa8c0@m6805> I have this huge table, 65 million records, ~700 fields. The table has a bunch of now useless fields which I will be deleting, basically old cass and ncoa data from a previous (many years ago) address verification on the table. More importantly, it also has what I will call demographics data, but extended into such things as brand preference for consumables, hobbies, types of electronics purchases, etc. Obviously the sheer size is an issue here. What I would think to do, coming from a small database background, is to create a PK on the big table, then break this big table down into smaller tables, each focused on one "set" of demographics data. My question to you guys is, given the size of the table in terms of sheer number of records, is this a valid strategy? If I have an indexed and identical PK in each table, can I break out the Address into a table, age/income/race/religion into a table, then tables for boating, electronics, medicines, etc. with each sub table only containing records where the demographics info is populated. IOW, not everyone has boating info, not everyone has medical info, electronics info etc. so I end up with a smaller subset of records in each table where there is only a record in that table if the person has data for that demographic set. Having done that can I then expect to be able to join up to a half dozen tables to get inner join subsets, outer join mega-sets etc. in anything approaching near real time (minutes)? I am in the process of doing the very first (two) breakouts, address and boating. Obviously I have to create a new table, append the data in, then build an index on the PK for starters. Having done that I can do a quick and dirty test, but I would like your opinions on the general feasibility of this approach given the limitations of current (but state of the art) desktop hardware, SBS 2003 and SQL Server 2000. Having answered this (assuming that the answer is "it is probably a valid approach", I then need assistance on various methods to determine the existence of data within a given record of valid demographic data. IOW, I am pulling all 65 million records, a subset of fields (typically 10 to 30 fields) into a smaller table. This smaller table will eventually have an indexed PK, but NO indexes on the various fields. One way I can think of to do something like this is to generate a "validity" field where I store a number which is simply a count of the fields with something other than a zero length string. So record 1 has 0 valid data fields, rec 2 has 3 valid data fields, 3 has 12 valid data fields and so forth. IF I could generate such a query to get these counts and store it in a single field (and ever get results) and then apply an index on that field, anything over a zero has valid data and could be used to pull records, or (in reverse) records with a zero in the "validity" field could be deleted from the table to drop the number of records in that table. Other thoughts on how to accomplish this objective? Remember that this main table is simply freakin huge, with probably 20-50 different demographic groupings and some where the boundaries aren't even clear. In order to do this I will have to create that same number of tables, cut out the data and get it into these new tables, index the PK, generate counts of the validity data, delete records with no valid data etc. Some kind of automation for doing this would be nice, but basically, I can if necessary just brute force it once I have the procedure down. Ideas? Objections? "Your crazy to even think about this"? John W. Colby Colby Consulting www.ColbyConsulting.com From jwcolby at colbyconsulting.com Fri Nov 3 00:26:56 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Fri, 3 Nov 2006 01:26:56 -0500 Subject: [dba-SQLServer] Carving out one table Message-ID: <008001c6ff11$0febdb00$657aa8c0@m6805> It took the system 56 minutes to carve just the address fields (and the PK) out into a new table - 64 million records. I have a second query running right now to carve out the boating. More fields. I'm going to bed so I will tell you the time to do this tomorrow. I need to know how to run a query to add an index so that I can see the time required to accomplish this. I have been going in to table design to do this and it "just goes away" and comes back when it is done. I can't really tell how long it takes. Man am I going to learn SQL! Alter table, add constraint, primary key... And I need to find the syntax to make a field non-nullable. Which brings up an interesting point. These tables are being created with a "PK" which is also an "FK", essentially it is a one to one child table to the master address table. I will probably eventually create the relationships, though I am not sure whether they are truly necessary. The master table is never SUPPOSED to have records deleted. Well, stuff for tomorrow. John W. Colby Colby Consulting www.ColbyConsulting.com From jlawrenc1 at shaw.ca Fri Nov 3 00:38:31 2006 From: jlawrenc1 at shaw.ca (Jim Lawrence) Date: Thu, 02 Nov 2006 22:38:31 -0800 Subject: [dba-SQLServer] Any gut feelings? In-Reply-To: <007b01c6ff0d$0552fba0$657aa8c0@m6805> Message-ID: <0J85001PS6DSYF80@l-daemon> Hi John: The only thing that I could suggest seeing that you appear to be on the right path is to leave out actually building the indexes until last. Making them yes but not building them as there will be a speed component in all this. If I was building a huge database system I would be building a small test schema to see it all works first before applying the final design. If there is a mistake or two in the schema it will not be easy to change things when there are 30 plus tables and appropriate joins and 65 million records with indexes to process. Just a thought Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Thursday, November 02, 2006 9:58 PM To: dba-sqlserver at databaseadvisors.com Subject: [dba-SQLServer] Any gut feelings? I have this huge table, 65 million records, ~700 fields. The table has a bunch of now useless fields which I will be deleting, basically old cass and ncoa data from a previous (many years ago) address verification on the table. More importantly, it also has what I will call demographics data, but extended into such things as brand preference for consumables, hobbies, types of electronics purchases, etc. Obviously the sheer size is an issue here. What I would think to do, coming from a small database background, is to create a PK on the big table, then break this big table down into smaller tables, each focused on one "set" of demographics data. My question to you guys is, given the size of the table in terms of sheer number of records, is this a valid strategy? If I have an indexed and identical PK in each table, can I break out the Address into a table, age/income/race/religion into a table, then tables for boating, electronics, medicines, etc. with each sub table only containing records where the demographics info is populated. IOW, not everyone has boating info, not everyone has medical info, electronics info etc. so I end up with a smaller subset of records in each table where there is only a record in that table if the person has data for that demographic set. Having done that can I then expect to be able to join up to a half dozen tables to get inner join subsets, outer join mega-sets etc. in anything approaching near real time (minutes)? I am in the process of doing the very first (two) breakouts, address and boating. Obviously I have to create a new table, append the data in, then build an index on the PK for starters. Having done that I can do a quick and dirty test, but I would like your opinions on the general feasibility of this approach given the limitations of current (but state of the art) desktop hardware, SBS 2003 and SQL Server 2000. Having answered this (assuming that the answer is "it is probably a valid approach", I then need assistance on various methods to determine the existence of data within a given record of valid demographic data. IOW, I am pulling all 65 million records, a subset of fields (typically 10 to 30 fields) into a smaller table. This smaller table will eventually have an indexed PK, but NO indexes on the various fields. One way I can think of to do something like this is to generate a "validity" field where I store a number which is simply a count of the fields with something other than a zero length string. So record 1 has 0 valid data fields, rec 2 has 3 valid data fields, 3 has 12 valid data fields and so forth. IF I could generate such a query to get these counts and store it in a single field (and ever get results) and then apply an index on that field, anything over a zero has valid data and could be used to pull records, or (in reverse) records with a zero in the "validity" field could be deleted from the table to drop the number of records in that table. Other thoughts on how to accomplish this objective? Remember that this main table is simply freakin huge, with probably 20-50 different demographic groupings and some where the boundaries aren't even clear. In order to do this I will have to create that same number of tables, cut out the data and get it into these new tables, index the PK, generate counts of the validity data, delete records with no valid data etc. Some kind of automation for doing this would be nice, but basically, I can if necessary just brute force it once I have the procedure down. Ideas? Objections? "Your crazy to even think about this"? John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From artful at rogers.com Fri Nov 3 03:18:29 2006 From: artful at rogers.com (artful at rogers.com) Date: Fri, 3 Nov 2006 01:18:29 -0800 (PST) Subject: [dba-SQLServer] Any gut feelings? Message-ID: <20061103091829.93377.qmail@web88210.mail.re2.yahoo.com> Given the projects upon which I have worked recently, 65M rows is a smallish table. Mind you, I've been doing everything in SQL Server not Access, but I've been working with situations in which the anticipated growth is 1TB per year. The 700 columns is an issue of a different colour and suggests a serious problem, perhaps not in your final solution but certainly in the source data. Some stakeholder must prioritize these columns and then you must break them out into related tables. (i.e. columns most commonly queried, etc.) 65M rows is not really a large table, except in the Access context. The 700 columns is the locus of your problem, not the rowcount. If you can quantify the probablity of a column being queried, then you can break this big table into several related tables according to probablity of query, then join the related tables if and only if their columns are queried. Just my opinion, but based on working with TB-sized tables. Arthur ----- Original Message ---- From: JWColby To: dba-sqlserver at databaseadvisors.com Sent: Friday, November 3, 2006 12:58:00 AM Subject: [dba-SQLServer] Any gut feelings? I have this huge table, 65 million records, ~700 fields. The table has a bunch of now useless fields which I will be deleting, basically old cass and ncoa data from a previous (many years ago) address verification on the table. More importantly, it also has what I will call demographics data, but extended into such things as brand preference for consumables, hobbies, types of electronics purchases, etc. Obviously the sheer size is an issue here. What I would think to do, coming from a small database background, is to create a PK on the big table, then break this big table down into smaller tables, each focused on one "set" of demographics data. My question to you guys is, given the size of the table in terms of sheer number of records, is this a valid strategy? If I have an indexed and identical PK in each table, can I break out the Address into a table, age/income/race/religion into a table, then tables for boating, electronics, medicines, etc. with each sub table only containing records where the demographics info is populated. IOW, not everyone has boating info, not everyone has medical info, electronics info etc. so I end up with a smaller subset of records in each table where there is only a record in that table if the person has data for that demographic set. Having done that can I then expect to be able to join up to a half dozen tables to get inner join subsets, outer join mega-sets etc. in anything approaching near real time (minutes)? I am in the process of doing the very first (two) breakouts, address and boating. Obviously I have to create a new table, append the data in, then build an index on the PK for starters. Having done that I can do a quick and dirty test, but I would like your opinions on the general feasibility of this approach given the limitations of current (but state of the art) desktop hardware, SBS 2003 and SQL Server 2000. Having answered this (assuming that the answer is "it is probably a valid approach", I then need assistance on various methods to determine the existence of data within a given record of valid demographic data. IOW, I am pulling all 65 million records, a subset of fields (typically 10 to 30 fields) into a smaller table. This smaller table will eventually have an indexed PK, but NO indexes on the various fields. One way I can think of to do something like this is to generate a "validity" field where I store a number which is simply a count of the fields with something other than a zero length string. So record 1 has 0 valid data fields, rec 2 has 3 valid data fields, 3 has 12 valid data fields and so forth. IF I could generate such a query to get these counts and store it in a single field (and ever get results) and then apply an index on that field, anything over a zero has valid data and could be used to pull records, or (in reverse) records with a zero in the "validity" field could be deleted from the table to drop the number of records in that table. Other thoughts on how to accomplish this objective? Remember that this main table is simply freakin huge, with probably 20-50 different demographic groupings and some where the boundaries aren't even clear. In order to do this I will have to create that same number of tables, cut out the data and get it into these new tables, index the PK, generate counts of the validity data, delete records with no valid data etc. Some kind of automation for doing this would be nice, but basically, I can if necessary just brute force it once I have the procedure down. Ideas? Objections? "Your crazy to even think about this"? John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Nov 3 06:19:12 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Fri, 3 Nov 2006 07:19:12 -0500 Subject: [dba-SQLServer] Any gut feelings? In-Reply-To: <20061103091829.93377.qmail@web88210.mail.re2.yahoo.com> Message-ID: <008c01c6ff42$46441830$657aa8c0@m6805> Arthur, That is exactly what I need to know. This is a SQL Server table, but so far (granted, based on an XP Pro OS) the time to query or do ANYTHING on the big table is overnight. I can't have that if this is going to work. My guess is that the original table came from a company with big iron machines with dozens of processors running huge analysis programs. They had the brute horsepower to do this, I don't. I have to do it different, or not at all. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Friday, November 03, 2006 4:18 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? Given the projects upon which I have worked recently, 65M rows is a smallish table. Mind you, I've been doing everything in SQL Server not Access, but I've been working with situations in which the anticipated growth is 1TB per year. The 700 columns is an issue of a different colour and suggests a serious problem, perhaps not in your final solution but certainly in the source data. Some stakeholder must prioritize these columns and then you must break them out into related tables. (i.e. columns most commonly queried, etc.) 65M rows is not really a large table, except in the Access context. The 700 columns is the locus of your problem, not the rowcount. If you can quantify the probablity of a column being queried, then you can break this big table into several related tables according to probablity of query, then join the related tables if and only if their columns are queried. Just my opinion, but based on working with TB-sized tables. Arthur ----- Original Message ---- From: JWColby To: dba-sqlserver at databaseadvisors.com Sent: Friday, November 3, 2006 12:58:00 AM Subject: [dba-SQLServer] Any gut feelings? I have this huge table, 65 million records, ~700 fields. The table has a bunch of now useless fields which I will be deleting, basically old cass and ncoa data from a previous (many years ago) address verification on the table. More importantly, it also has what I will call demographics data, but extended into such things as brand preference for consumables, hobbies, types of electronics purchases, etc. Obviously the sheer size is an issue here. What I would think to do, coming from a small database background, is to create a PK on the big table, then break this big table down into smaller tables, each focused on one "set" of demographics data. My question to you guys is, given the size of the table in terms of sheer number of records, is this a valid strategy? If I have an indexed and identical PK in each table, can I break out the Address into a table, age/income/race/religion into a table, then tables for boating, electronics, medicines, etc. with each sub table only containing records where the demographics info is populated. IOW, not everyone has boating info, not everyone has medical info, electronics info etc. so I end up with a smaller subset of records in each table where there is only a record in that table if the person has data for that demographic set. Having done that can I then expect to be able to join up to a half dozen tables to get inner join subsets, outer join mega-sets etc. in anything approaching near real time (minutes)? I am in the process of doing the very first (two) breakouts, address and boating. Obviously I have to create a new table, append the data in, then build an index on the PK for starters. Having done that I can do a quick and dirty test, but I would like your opinions on the general feasibility of this approach given the limitations of current (but state of the art) desktop hardware, SBS 2003 and SQL Server 2000. Having answered this (assuming that the answer is "it is probably a valid approach", I then need assistance on various methods to determine the existence of data within a given record of valid demographic data. IOW, I am pulling all 65 million records, a subset of fields (typically 10 to 30 fields) into a smaller table. This smaller table will eventually have an indexed PK, but NO indexes on the various fields. One way I can think of to do something like this is to generate a "validity" field where I store a number which is simply a count of the fields with something other than a zero length string. So record 1 has 0 valid data fields, rec 2 has 3 valid data fields, 3 has 12 valid data fields and so forth. IF I could generate such a query to get these counts and store it in a single field (and ever get results) and then apply an index on that field, anything over a zero has valid data and could be used to pull records, or (in reverse) records with a zero in the "validity" field could be deleted from the table to drop the number of records in that table. Other thoughts on how to accomplish this objective? Remember that this main table is simply freakin huge, with probably 20-50 different demographic groupings and some where the boundaries aren't even clear. In order to do this I will have to create that same number of tables, cut out the data and get it into these new tables, index the PK, generate counts of the validity data, delete records with no valid data etc. Some kind of automation for doing this would be nice, but basically, I can if necessary just brute force it once I have the procedure down. Ideas? Objections? "Your crazy to even think about this"? John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Nov 3 08:03:11 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Fri, 3 Nov 2006 09:03:11 -0500 Subject: [dba-SQLServer] Any gut feelings? In-Reply-To: <20061103091829.93377.qmail@web88210.mail.re2.yahoo.com> Message-ID: <009201c6ff50$cd130480$657aa8c0@m6805> Arthur, The other question I have given your experience with these big databases, is what kind of hardware were they using? Was this database you mention running on 1/2 proc machines with 2/4 gig of memory, or was it on processor arrays with a dozen or two or three processors and a dozen or two or three gigabytes of main memory? Running windows (what flavor)? SQL Server 2K (what flavor)? John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Friday, November 03, 2006 4:18 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? Given the projects upon which I have worked recently, 65M rows is a smallish table. Mind you, I've been doing everything in SQL Server not Access, but I've been working with situations in which the anticipated growth is 1TB per year. The 700 columns is an issue of a different colour and suggests a serious problem, perhaps not in your final solution but certainly in the source data. Some stakeholder must prioritize these columns and then you must break them out into related tables. (i.e. columns most commonly queried, etc.) 65M rows is not really a large table, except in the Access context. The 700 columns is the locus of your problem, not the rowcount. If you can quantify the probablity of a column being queried, then you can break this big table into several related tables according to probablity of query, then join the related tables if and only if their columns are queried. Just my opinion, but based on working with TB-sized tables. Arthur ----- Original Message ---- From: JWColby To: dba-sqlserver at databaseadvisors.com Sent: Friday, November 3, 2006 12:58:00 AM Subject: [dba-SQLServer] Any gut feelings? I have this huge table, 65 million records, ~700 fields. The table has a bunch of now useless fields which I will be deleting, basically old cass and ncoa data from a previous (many years ago) address verification on the table. More importantly, it also has what I will call demographics data, but extended into such things as brand preference for consumables, hobbies, types of electronics purchases, etc. Obviously the sheer size is an issue here. What I would think to do, coming from a small database background, is to create a PK on the big table, then break this big table down into smaller tables, each focused on one "set" of demographics data. My question to you guys is, given the size of the table in terms of sheer number of records, is this a valid strategy? If I have an indexed and identical PK in each table, can I break out the Address into a table, age/income/race/religion into a table, then tables for boating, electronics, medicines, etc. with each sub table only containing records where the demographics info is populated. IOW, not everyone has boating info, not everyone has medical info, electronics info etc. so I end up with a smaller subset of records in each table where there is only a record in that table if the person has data for that demographic set. Having done that can I then expect to be able to join up to a half dozen tables to get inner join subsets, outer join mega-sets etc. in anything approaching near real time (minutes)? I am in the process of doing the very first (two) breakouts, address and boating. Obviously I have to create a new table, append the data in, then build an index on the PK for starters. Having done that I can do a quick and dirty test, but I would like your opinions on the general feasibility of this approach given the limitations of current (but state of the art) desktop hardware, SBS 2003 and SQL Server 2000. Having answered this (assuming that the answer is "it is probably a valid approach", I then need assistance on various methods to determine the existence of data within a given record of valid demographic data. IOW, I am pulling all 65 million records, a subset of fields (typically 10 to 30 fields) into a smaller table. This smaller table will eventually have an indexed PK, but NO indexes on the various fields. One way I can think of to do something like this is to generate a "validity" field where I store a number which is simply a count of the fields with something other than a zero length string. So record 1 has 0 valid data fields, rec 2 has 3 valid data fields, 3 has 12 valid data fields and so forth. IF I could generate such a query to get these counts and store it in a single field (and ever get results) and then apply an index on that field, anything over a zero has valid data and could be used to pull records, or (in reverse) records with a zero in the "validity" field could be deleted from the table to drop the number of records in that table. Other thoughts on how to accomplish this objective? Remember that this main table is simply freakin huge, with probably 20-50 different demographic groupings and some where the boundaries aren't even clear. In order to do this I will have to create that same number of tables, cut out the data and get it into these new tables, index the PK, generate counts of the validity data, delete records with no valid data etc. Some kind of automation for doing this would be nice, but basically, I can if necessary just brute force it once I have the procedure down. Ideas? Objections? "Your crazy to even think about this"? John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From rl_stewart at highstream.net Fri Nov 3 08:23:55 2006 From: rl_stewart at highstream.net (Robert L. Stewart) Date: Fri, 03 Nov 2006 08:23:55 -0600 Subject: [dba-SQLServer] Carving out one table In-Reply-To: References: Message-ID: <200611031428.kA3ES1q07743@databaseadvisors.com> John, For creating an index... CREATE INDEX tblRatingScoreRange_IX01 ON dbo.tblRatingScoreRange ( LowScore ASC ) go Primary key constraint... ALTER TABLE dbo.tblRatingScoreRange ADD CONSTRAINT tblRatingScoreRange_PK PRIMARY KEY CLUSTERED ( RatingScoreRangeID ASC) go Unique index... CREATE UNIQUE INDEX tblRatingScoreRange_UI01 ON dbo.tblRatingScoreRange ( LowScore ASC, HighScore ASC, RatingScoreText ASC ) go Allow null... ALTER TABLE dbo.tblRatingScoreRange LowScore tinyint NULL (I think this is correct. Try it on a small table first.) As far as carving out the data, yes it is a great idea. It is what data warehouses are made out of. If you want help modeling it, let me know. At Waste Management, I was modeling databases that were between 5 and 10 TB for their data warehouse. Robert At 03:18 AM 11/3/2006, you wrote: >Date: Fri, 3 Nov 2006 01:26:56 -0500 >From: "JWColby" >Subject: [dba-SQLServer] Carving out one table >To: >Message-ID: <008001c6ff11$0febdb00$657aa8c0 at m6805> >Content-Type: text/plain; charset="us-ascii" > >It took the system 56 minutes to carve just the address fields (and the PK) >out into a new table - 64 million records. I have a second query running >right now to carve out the boating. More fields. I'm going to bed so I >will tell you the time to do this tomorrow. > >I need to know how to run a query to add an index so that I can see the time >required to accomplish this. I have been going in to table design to do >this and it "just goes away" and comes back when it is done. I can't really >tell how long it takes. > >Man am I going to learn SQL! Alter table, add constraint, primary key... >And I need to find the syntax to make a field non-nullable. > >Which brings up an interesting point. These tables are being created with a >"PK" which is also an "FK", essentially it is a one to one child table to >the master address table. I will probably eventually create the >relationships, though I am not sure whether they are truly necessary. The >master table is never SUPPOSED to have records deleted. > >Well, stuff for tomorrow. > >John W. Colby >Colby Consulting >www.ColbyConsulting.com From askolits at ot.com Fri Nov 3 08:42:47 2006 From: askolits at ot.com (John Skolits) Date: Fri, 3 Nov 2006 09:42:47 -0500 Subject: [dba-SQLServer] ODBC Connection error In-Reply-To: <009201c6ff50$cd130480$657aa8c0@m6805> Message-ID: <000401c6ff56$57ea96e0$800101df@officexp> I'm having this ODBC connection error that's driving me nuts. The application is written in Access 2000. The User's Pc is Access 2003. MDAC 2.8 is his latest version. There have been times where the application has run fine, then I make a few changes and the user can't connect to the tables. The only way to make it work is to go in an manually relink every table on his PC. If I do it on my development PC, or any other PC for that matter, he can't open the tables. I ran a Pass thru query on his PC with a new DSN I created and a connection string like: ODBC;DRIVER=SQL Server;SERVER=otto;UID=User;PWD=User;DATABASE=ssa_erpdb I then received an error message like : ODBC error ..... ODBC;DRIVER=SQL Server;SERVER=otto;UID=reports;PWD=repo_' Notice how it truncated the string. So it looks like It's a problem on his PC. I tried a myriad of things. Is there any Service releases or anything anyone has heard of that may fix this issue before I spend weeks trying to find a work around. John From askolits at ot.com Fri Nov 3 09:09:46 2006 From: askolits at ot.com (John Skolits) Date: Fri, 3 Nov 2006 10:09:46 -0500 Subject: [dba-SQLServer] ODBC Connection error - More Info In-Reply-To: <000401c6ff56$57ea96e0$800101df@officexp> Message-ID: <007101c6ff5a$1c604ad0$6501a8c0@LaptopXP> I'm also getting the following error when trying to run a few queries. Connection Failed: SQL State '28000' Sql Server Error: 18452 {Microssoft][Sql Server Driver][Sql Server]Logon Failed for user '(null)' Reason: Not associated with a trusted cSql Server Conenction. -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of John Skolits Sent: Friday, November 03, 2006 9:43 AM To: dba-sqlserver at databaseadvisors.com Subject: [dba-SQLServer] ODBC Connection error I'm having this ODBC connection error that's driving me nuts. The application is written in Access 2000. The User's Pc is Access 2003. MDAC 2.8 is his latest version. There have been times where the application has run fine, then I make a few changes and the user can't connect to the tables. The only way to make it work is to go in an manually relink every table on his PC. If I do it on my development PC, or any other PC for that matter, he can't open the tables. I ran a Pass thru query on his PC with a new DSN I created and a connection string like: ODBC;DRIVER=SQL Server;SERVER=otto;UID=User;PWD=User;DATABASE=ssa_erpdb I then received an error message like : ODBC error ..... ODBC;DRIVER=SQL Server;SERVER=otto;UID=reports;PWD=repo_' Notice how it truncated the string. So it looks like It's a problem on his PC. I tried a myriad of things. Is there any Service releases or anything anyone has heard of that may fix this issue before I spend weeks trying to find a work around. John _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Nov 3 10:15:27 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Fri, 3 Nov 2006 11:15:27 -0500 Subject: [dba-SQLServer] Tell Query analyzer where to store queries Message-ID: <009301c6ff63$475f9750$657aa8c0@m6805> Is there any way to tell query analyzer where to save queries I am building, preferably on a database level? It is just "offering" to save them in my documents. Not useful to have to navigate over to another drive / directory structure every time I want to save a query. Also not helpful to have a thousand queries all stuffed in my documents. John W. Colby Colby Consulting www.ColbyConsulting.com From jlawrenc1 at shaw.ca Fri Nov 3 10:50:38 2006 From: jlawrenc1 at shaw.ca (Jim Lawrence) Date: Fri, 03 Nov 2006 08:50:38 -0800 Subject: [dba-SQLServer] ODBC Connection error - More Info In-Reply-To: <007101c6ff5a$1c604ad0$6501a8c0@LaptopXP> Message-ID: <0J8500LAPYPYSDA0@l-daemon> Hi John: Would his password have a special character in it that is not allowed? Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of John Skolits Sent: Friday, November 03, 2006 7:10 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] ODBC Connection error - More Info I'm also getting the following error when trying to run a few queries. Connection Failed: SQL State '28000' Sql Server Error: 18452 {Microssoft][Sql Server Driver][Sql Server]Logon Failed for user '(null)' Reason: Not associated with a trusted cSql Server Conenction. -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of John Skolits Sent: Friday, November 03, 2006 9:43 AM To: dba-sqlserver at databaseadvisors.com Subject: [dba-SQLServer] ODBC Connection error I'm having this ODBC connection error that's driving me nuts. The application is written in Access 2000. The User's Pc is Access 2003. MDAC 2.8 is his latest version. There have been times where the application has run fine, then I make a few changes and the user can't connect to the tables. The only way to make it work is to go in an manually relink every table on his PC. If I do it on my development PC, or any other PC for that matter, he can't open the tables. I ran a Pass thru query on his PC with a new DSN I created and a connection string like: ODBC;DRIVER=SQL Server;SERVER=otto;UID=User;PWD=User;DATABASE=ssa_erpdb I then received an error message like : ODBC error ..... ODBC;DRIVER=SQL Server;SERVER=otto;UID=reports;PWD=repo_' Notice how it truncated the string. So it looks like It's a problem on his PC. I tried a myriad of things. Is there any Service releases or anything anyone has heard of that may fix this issue before I spend weeks trying to find a work around. John _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From rl_stewart at highstream.net Fri Nov 3 10:58:56 2006 From: rl_stewart at highstream.net (Robert L. Stewart) Date: Fri, 03 Nov 2006 10:58:56 -0600 Subject: [dba-SQLServer] Tell Query analyzer where to store queries In-Reply-To: References: Message-ID: <200611031702.kA3H27q30075@databaseadvisors.com> John, If you are repeating them, store them as stored procedures. For example: CREATE PROCEDURE uspGetPostalCodeCount @PostalCode varchar(5) AS SELECT COUNT(PostalCode) FROM dbo.tblNameAddress WHERE PostalCode = @PostalCOde Robert At 10:49 AM 11/3/2006, you wrote: >Date: Fri, 3 Nov 2006 11:15:27 -0500 >From: "JWColby" >Subject: [dba-SQLServer] Tell Query analyzer where to store queries >To: >Message-ID: <009301c6ff63$475f9750$657aa8c0 at m6805> >Content-Type: text/plain; charset="us-ascii" > >Is there any way to tell query analyzer where to save queries I am building, >preferably on a database level? It is just "offering" to save them in my >documents. Not useful to have to navigate over to another drive / directory >structure every time I want to save a query. Also not helpful to have a >thousand queries all stuffed in my documents. > >John W. Colby >Colby Consulting >www.ColbyConsulting.com > From askolits at ot.com Fri Nov 3 11:26:35 2006 From: askolits at ot.com (John Skolits) Date: Fri, 3 Nov 2006 12:26:35 -0500 Subject: [dba-SQLServer] ODBC Connection error - More Info In-Reply-To: <0J8500LAPYPYSDA0@l-daemon> Message-ID: <00a801c6ff6d$397dc620$6501a8c0@LaptopXP> No, not really. I was just on the phone with him and it seems there is a compile issue. I have two, almost identical functions. One works from the immediate window and the other gives him a compile error. Although the program compiles correctly. I'll first have him uncompile\recompile and repair. Yet, I'm running the exact same version on an Access 2000 version with no problems. The other option would be to have him reinstall Access 2003. As far as the ODBC error, I'm finding that when he manually reconnects the Trusted Connection is automatically set to True. When I relink through code that I wrote, I always set the trusted connection to False and have never had a problem on the many applications I have written. Except on his PC. For now, I think it's his installation, although not positive. -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Friday, November 03, 2006 11:51 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] ODBC Connection error - More Info Hi John: Would his password have a special character in it that is not allowed? Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of John Skolits Sent: Friday, November 03, 2006 7:10 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] ODBC Connection error - More Info I'm also getting the following error when trying to run a few queries. Connection Failed: SQL State '28000' Sql Server Error: 18452 {Microssoft][Sql Server Driver][Sql Server]Logon Failed for user '(null)' Reason: Not associated with a trusted cSql Server Conenction. -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of John Skolits Sent: Friday, November 03, 2006 9:43 AM To: dba-sqlserver at databaseadvisors.com Subject: [dba-SQLServer] ODBC Connection error I'm having this ODBC connection error that's driving me nuts. The application is written in Access 2000. The User's Pc is Access 2003. MDAC 2.8 is his latest version. There have been times where the application has run fine, then I make a few changes and the user can't connect to the tables. The only way to make it work is to go in an manually relink every table on his PC. If I do it on my development PC, or any other PC for that matter, he can't open the tables. I ran a Pass thru query on his PC with a new DSN I created and a connection string like: ODBC;DRIVER=SQL Server;SERVER=otto;UID=User;PWD=User;DATABASE=ssa_erpdb I then received an error message like : ODBC error ..... ODBC;DRIVER=SQL Server;SERVER=otto;UID=reports;PWD=repo_' Notice how it truncated the string. So it looks like It's a problem on his PC. I tried a myriad of things. Is there any Service releases or anything anyone has heard of that may fix this issue before I spend weeks trying to find a work around. John _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From markamatte at hotmail.com Fri Nov 3 12:22:37 2006 From: markamatte at hotmail.com (Mark A Matte) Date: Fri, 03 Nov 2006 18:22:37 +0000 Subject: [dba-SQLServer] Tell Query analyzer where to store queries Message-ID: John, I am using an older version(SQL7)...and the only thing I've found with query analyzer is 'open' or 'save as' always goes to the last folder I saved in. I couldn't find any setting to pick a specific folder. If you figure it out...please share. Thanks, Mark A. Matte >From: "JWColby" >Reply-To: dba-sqlserver at databaseadvisors.com >To: >Subject: [dba-SQLServer] Tell Query analyzer where to store queries >Date: Fri, 3 Nov 2006 11:15:27 -0500 > >Is there any way to tell query analyzer where to save queries I am >building, >preferably on a database level? It is just "offering" to save them in my >documents. Not useful to have to navigate over to another drive / >directory >structure every time I want to save a query. Also not helpful to have a >thousand queries all stuffed in my documents. > >John W. Colby >Colby Consulting >www.ColbyConsulting.com > >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > _________________________________________________________________ Try Search Survival Kits: Fix up your home and better handle your cash with Live Search! http://imagine-windowslive.com/search/kits/default.aspx?kit=improve&locale=en-US&source=hmtagline From artful at rogers.com Fri Nov 3 12:31:32 2006 From: artful at rogers.com (artful at rogers.com) Date: Fri, 3 Nov 2006 10:31:32 -0800 (PST) Subject: [dba-SQLServer] Any gut feelings? Message-ID: <20061103183133.58538.qmail@web88210.mail.re2.yahoo.com> It is split across 8 servers, each running two processors and each having > 2 GB of RAM. In part this arrangement was due to security concerns, which I won't go into, but also the sheer size of the database made it impossible to put it on one conventional server. All the servers run Windows Server 2003. The first iteration ran on SQL 2000, but the conversion to SQL 2005 is underway. The fact of 8 databases, each on a different server, exposed a serious down side as compared to a similar implementation in Oracle -- MS SQL cannot do referential integrity constraints across databases, so a bunch of additional code + duplicate lookup tables were required. In Oracle, one could simply declare tablespaces then place them on different servers, and it would still remain a single database. Until this project, I had never been involved with anything so large, so the issue never arose. ----- Original Message ---- From: JWColby To: dba-sqlserver at databaseadvisors.com Sent: Friday, November 3, 2006 9:03:11 AM Subject: Re: [dba-SQLServer] Any gut feelings? Arthur, The other question I have given your experience with these big databases, is what kind of hardware were they using? Was this database you mention running on 1/2 proc machines with 2/4 gig of memory, or was it on processor arrays with a dozen or two or three processors and a dozen or two or three gigabytes of main memory? Running windows (what flavor)? SQL Server 2K (what flavor)? John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Friday, November 03, 2006 4:18 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? Given the projects upon which I have worked recently, 65M rows is a smallish table. Mind you, I've been doing everything in SQL Server not Access, but I've been working with situations in which the anticipated growth is 1TB per year. The 700 columns is an issue of a different colour and suggests a serious problem, perhaps not in your final solution but certainly in the source data. Some stakeholder must prioritize these columns and then you must break them out into related tables. (i.e. columns most commonly queried, etc.) 65M rows is not really a large table, except in the Access context. The 700 columns is the locus of your problem, not the rowcount. If you can quantify the probablity of a column being queried, then you can break this big table into several related tables according to probablity of query, then join the related tables if and only if their columns are queried. Just my opinion, but based on working with TB-sized tables. Arthur ----- Original Message ---- From: JWColby To: dba-sqlserver at databaseadvisors.com Sent: Friday, November 3, 2006 12:58:00 AM Subject: [dba-SQLServer] Any gut feelings? I have this huge table, 65 million records, ~700 fields. The table has a bunch of now useless fields which I will be deleting, basically old cass and ncoa data from a previous (many years ago) address verification on the table. More importantly, it also has what I will call demographics data, but extended into such things as brand preference for consumables, hobbies, types of electronics purchases, etc. Obviously the sheer size is an issue here. What I would think to do, coming from a small database background, is to create a PK on the big table, then break this big table down into smaller tables, each focused on one "set" of demographics data. My question to you guys is, given the size of the table in terms of sheer number of records, is this a valid strategy? If I have an indexed and identical PK in each table, can I break out the Address into a table, age/income/race/religion into a table, then tables for boating, electronics, medicines, etc. with each sub table only containing records where the demographics info is populated. IOW, not everyone has boating info, not everyone has medical info, electronics info etc. so I end up with a smaller subset of records in each table where there is only a record in that table if the person has data for that demographic set. Having done that can I then expect to be able to join up to a half dozen tables to get inner join subsets, outer join mega-sets etc. in anything approaching near real time (minutes)? I am in the process of doing the very first (two) breakouts, address and boating. Obviously I have to create a new table, append the data in, then build an index on the PK for starters. Having done that I can do a quick and dirty test, but I would like your opinions on the general feasibility of this approach given the limitations of current (but state of the art) desktop hardware, SBS 2003 and SQL Server 2000. Having answered this (assuming that the answer is "it is probably a valid approach", I then need assistance on various methods to determine the existence of data within a given record of valid demographic data. IOW, I am pulling all 65 million records, a subset of fields (typically 10 to 30 fields) into a smaller table. This smaller table will eventually have an indexed PK, but NO indexes on the various fields. One way I can think of to do something like this is to generate a "validity" field where I store a number which is simply a count of the fields with something other than a zero length string. So record 1 has 0 valid data fields, rec 2 has 3 valid data fields, 3 has 12 valid data fields and so forth. IF I could generate such a query to get these counts and store it in a single field (and ever get results) and then apply an index on that field, anything over a zero has valid data and could be used to pull records, or (in reverse) records with a zero in the "validity" field could be deleted from the table to drop the number of records in that table. Other thoughts on how to accomplish this objective? Remember that this main table is simply freakin huge, with probably 20-50 different demographic groupings and some where the boundaries aren't even clear. In order to do this I will have to create that same number of tables, cut out the data and get it into these new tables, index the PK, generate counts of the validity data, delete records with no valid data etc. Some kind of automation for doing this would be nice, but basically, I can if necessary just brute force it once I have the procedure down. Ideas? Objections? "Your crazy to even think about this"? John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From martyconnelly at shaw.ca Fri Nov 3 12:53:10 2006 From: martyconnelly at shaw.ca (MartyConnelly) Date: Fri, 03 Nov 2006 10:53:10 -0800 Subject: [dba-SQLServer] ODBC Connection error - More Info In-Reply-To: <00a801c6ff6d$397dc620$6501a8c0@LaptopXP> References: <00a801c6ff6d$397dc620$6501a8c0@LaptopXP> Message-ID: <454B9016.3010306@shaw.ca> Are you both running the same Jet Service Pack See http://www.microsoft.com/data John Skolits wrote: >No, not really. > >I was just on the phone with him and it seems there is a compile issue. > >I have two, almost identical functions. One works from the immediate window >and the other gives him a compile error. Although the program compiles >correctly. >I'll first have him uncompile\recompile and repair. Yet, I'm running the >exact same version on an Access 2000 version with no problems. > >The other option would be to have him reinstall Access 2003. > > >As far as the ODBC error, I'm finding that when he manually reconnects the >Trusted Connection is automatically set to True. When I relink through code >that I wrote, I always set the trusted connection to False and have never >had a problem on the many applications I have written. Except on his PC. > >For now, I think it's his installation, although not positive. > > > > >-----Original Message----- >From: dba-sqlserver-bounces at databaseadvisors.com >[mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim >Lawrence >Sent: Friday, November 03, 2006 11:51 AM >To: dba-sqlserver at databaseadvisors.com >Subject: Re: [dba-SQLServer] ODBC Connection error - More Info > >Hi John: > >Would his password have a special character in it that is not allowed? > >Jim > >-----Original Message----- >From: dba-sqlserver-bounces at databaseadvisors.com >[mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of John >Skolits >Sent: Friday, November 03, 2006 7:10 AM >To: dba-sqlserver at databaseadvisors.com >Subject: Re: [dba-SQLServer] ODBC Connection error - More Info > >I'm also getting the following error when trying to run a few queries. > >Connection Failed: >SQL State '28000' >Sql Server Error: 18452 >{Microssoft][Sql Server Driver][Sql Server]Logon Failed for user '(null)' >Reason: Not associated with a trusted cSql Server Conenction. > > >-----Original Message----- >From: dba-sqlserver-bounces at databaseadvisors.com >[mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of John >Skolits >Sent: Friday, November 03, 2006 9:43 AM >To: dba-sqlserver at databaseadvisors.com >Subject: [dba-SQLServer] ODBC Connection error > >I'm having this ODBC connection error that's driving me nuts. > >The application is written in Access 2000. The User's Pc is Access 2003. >MDAC 2.8 is his latest version. > >There have been times where the application has run fine, then I make a few >changes and the user can't connect to the tables. >The only way to make it work is to go in an manually relink every table on >his PC. If I do it on my development PC, or any other PC for that matter, he >can't open the tables. > >I ran a Pass thru query on his PC with a new DSN I created and a connection >string like: >ODBC;DRIVER=SQL Server;SERVER=otto;UID=User;PWD=User;DATABASE=ssa_erpdb > >I then received an error message like : >ODBC error ..... ODBC;DRIVER=SQL Server;SERVER=otto;UID=reports;PWD=repo_' > >Notice how it truncated the string. > >So it looks like It's a problem on his PC. > >I tried a myriad of things. Is there any Service releases or anything anyone >has heard of that may fix this issue before I spend weeks trying to find a >work around. > >John > > >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > > > >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > > > >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > > > > > -- Marty Connelly Victoria, B.C. Canada From jwcolby at colbyconsulting.com Fri Nov 3 13:12:42 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Fri, 3 Nov 2006 14:12:42 -0500 Subject: [dba-SQLServer] Any gut feelings? In-Reply-To: <20061103183133.58538.qmail@web88210.mail.re2.yahoo.com> Message-ID: <00b501c6ff7c$0a078390$657aa8c0@m6805> Do you (or anyone reading) have any feelings for performance of SQl Server 2005 vs 2000? I have 2005 but had trouble getting imports happening as easily and so never really explored it. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Friday, November 03, 2006 1:32 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? It is split across 8 servers, each running two processors and each having > 2 GB of RAM. In part this arrangement was due to security concerns, which I won't go into, but also the sheer size of the database made it impossible to put it on one conventional server. All the servers run Windows Server 2003. The first iteration ran on SQL 2000, but the conversion to SQL 2005 is underway. The fact of 8 databases, each on a different server, exposed a serious down side as compared to a similar implementation in Oracle -- MS SQL cannot do referential integrity constraints across databases, so a bunch of additional code + duplicate lookup tables were required. In Oracle, one could simply declare tablespaces then place them on different servers, and it would still remain a single database. Until this project, I had never been involved with anything so large, so the issue never arose. ----- Original Message ---- From: JWColby To: dba-sqlserver at databaseadvisors.com Sent: Friday, November 3, 2006 9:03:11 AM Subject: Re: [dba-SQLServer] Any gut feelings? Arthur, The other question I have given your experience with these big databases, is what kind of hardware were they using? Was this database you mention running on 1/2 proc machines with 2/4 gig of memory, or was it on processor arrays with a dozen or two or three processors and a dozen or two or three gigabytes of main memory? Running windows (what flavor)? SQL Server 2K (what flavor)? John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Friday, November 03, 2006 4:18 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? Given the projects upon which I have worked recently, 65M rows is a smallish table. Mind you, I've been doing everything in SQL Server not Access, but I've been working with situations in which the anticipated growth is 1TB per year. The 700 columns is an issue of a different colour and suggests a serious problem, perhaps not in your final solution but certainly in the source data. Some stakeholder must prioritize these columns and then you must break them out into related tables. (i.e. columns most commonly queried, etc.) 65M rows is not really a large table, except in the Access context. The 700 columns is the locus of your problem, not the rowcount. If you can quantify the probablity of a column being queried, then you can break this big table into several related tables according to probablity of query, then join the related tables if and only if their columns are queried. Just my opinion, but based on working with TB-sized tables. Arthur ----- Original Message ---- From: JWColby To: dba-sqlserver at databaseadvisors.com Sent: Friday, November 3, 2006 12:58:00 AM Subject: [dba-SQLServer] Any gut feelings? I have this huge table, 65 million records, ~700 fields. The table has a bunch of now useless fields which I will be deleting, basically old cass and ncoa data from a previous (many years ago) address verification on the table. More importantly, it also has what I will call demographics data, but extended into such things as brand preference for consumables, hobbies, types of electronics purchases, etc. Obviously the sheer size is an issue here. What I would think to do, coming from a small database background, is to create a PK on the big table, then break this big table down into smaller tables, each focused on one "set" of demographics data. My question to you guys is, given the size of the table in terms of sheer number of records, is this a valid strategy? If I have an indexed and identical PK in each table, can I break out the Address into a table, age/income/race/religion into a table, then tables for boating, electronics, medicines, etc. with each sub table only containing records where the demographics info is populated. IOW, not everyone has boating info, not everyone has medical info, electronics info etc. so I end up with a smaller subset of records in each table where there is only a record in that table if the person has data for that demographic set. Having done that can I then expect to be able to join up to a half dozen tables to get inner join subsets, outer join mega-sets etc. in anything approaching near real time (minutes)? I am in the process of doing the very first (two) breakouts, address and boating. Obviously I have to create a new table, append the data in, then build an index on the PK for starters. Having done that I can do a quick and dirty test, but I would like your opinions on the general feasibility of this approach given the limitations of current (but state of the art) desktop hardware, SBS 2003 and SQL Server 2000. Having answered this (assuming that the answer is "it is probably a valid approach", I then need assistance on various methods to determine the existence of data within a given record of valid demographic data. IOW, I am pulling all 65 million records, a subset of fields (typically 10 to 30 fields) into a smaller table. This smaller table will eventually have an indexed PK, but NO indexes on the various fields. One way I can think of to do something like this is to generate a "validity" field where I store a number which is simply a count of the fields with something other than a zero length string. So record 1 has 0 valid data fields, rec 2 has 3 valid data fields, 3 has 12 valid data fields and so forth. IF I could generate such a query to get these counts and store it in a single field (and ever get results) and then apply an index on that field, anything over a zero has valid data and could be used to pull records, or (in reverse) records with a zero in the "validity" field could be deleted from the table to drop the number of records in that table. Other thoughts on how to accomplish this objective? Remember that this main table is simply freakin huge, with probably 20-50 different demographic groupings and some where the boundaries aren't even clear. In order to do this I will have to create that same number of tables, cut out the data and get it into these new tables, index the PK, generate counts of the validity data, delete records with no valid data etc. Some kind of automation for doing this would be nice, but basically, I can if necessary just brute force it once I have the procedure down. Ideas? Objections? "Your crazy to even think about this"? John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From James at fcidms.com Fri Nov 3 13:18:09 2006 From: James at fcidms.com (James Barash) Date: Fri, 3 Nov 2006 14:18:09 -0500 Subject: [dba-SQLServer] Tell Query analyzer where to store queries In-Reply-To: Message-ID: <008001c6ff7c$cc7b0640$800101df@fci.local> John and Mark: For SQL 2000 at least In Query Analyzer: Tools -> Options -> General->Query File Directory That should do what you want. You can also save different configurations if you want to get fancy. James Barash -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Mark A Matte Sent: Friday, November 03, 2006 1:23 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Tell Query analyzer where to store queries John, I am using an older version(SQL7)...and the only thing I've found with query analyzer is 'open' or 'save as' always goes to the last folder I saved in. I couldn't find any setting to pick a specific folder. If you figure it out...please share. Thanks, Mark A. Matte >From: "JWColby" >Reply-To: dba-sqlserver at databaseadvisors.com >To: >Subject: [dba-SQLServer] Tell Query analyzer where to store queries >Date: Fri, 3 Nov 2006 11:15:27 -0500 > >Is there any way to tell query analyzer where to save queries I am >building, >preferably on a database level? It is just "offering" to save them in my >documents. Not useful to have to navigate over to another drive / >directory >structure every time I want to save a query. Also not helpful to have a >thousand queries all stuffed in my documents. > >John W. Colby >Colby Consulting >www.ColbyConsulting.com > >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > _________________________________________________________________ Try Search Survival Kits: Fix up your home and better handle your cash with Live Search! http://imagine-windowslive.com/search/kits/default.aspx?kit=improve&locale=e n-US&source=hmtagline _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Nov 3 13:38:10 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Fri, 3 Nov 2006 14:38:10 -0500 Subject: [dba-SQLServer] Tell Query analyzer where to store queries In-Reply-To: <008001c6ff7c$cc7b0640$800101df@fci.local> Message-ID: <00b601c6ff7f$9884b040$657aa8c0@m6805> James, You da man! John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of James Barash Sent: Friday, November 03, 2006 2:18 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Tell Query analyzer where to store queries John and Mark: For SQL 2000 at least In Query Analyzer: Tools -> Options -> General->Query File Directory That should do what you want. You can also save different configurations if you want to get fancy. James Barash -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Mark A Matte Sent: Friday, November 03, 2006 1:23 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Tell Query analyzer where to store queries John, I am using an older version(SQL7)...and the only thing I've found with query analyzer is 'open' or 'save as' always goes to the last folder I saved in. I couldn't find any setting to pick a specific folder. If you figure it out...please share. Thanks, Mark A. Matte >From: "JWColby" >Reply-To: dba-sqlserver at databaseadvisors.com >To: >Subject: [dba-SQLServer] Tell Query analyzer where to store queries >Date: Fri, 3 Nov 2006 11:15:27 -0500 > >Is there any way to tell query analyzer where to save queries I am >building, preferably on a database level? It is just "offering" to >save them in my documents. Not useful to have to navigate over to >another drive / directory structure every time I want to save a query. >Also not helpful to have a thousand queries all stuffed in my >documents. > >John W. Colby >Colby Consulting >www.ColbyConsulting.com > >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > _________________________________________________________________ Try Search Survival Kits: Fix up your home and better handle your cash with Live Search! http://imagine-windowslive.com/search/kits/default.aspx?kit=improve&locale=e n-US&source=hmtagline _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From accessd at shaw.ca Fri Nov 3 13:58:43 2006 From: accessd at shaw.ca (Jim Lawrence) Date: Fri, 03 Nov 2006 11:58:43 -0800 Subject: [dba-SQLServer] Any gut feelings? In-Reply-To: <00b501c6ff7c$0a078390$657aa8c0@m6805> Message-ID: <0J8600AIK7FE0TC0@l-daemon> Hi John: To the best of my knowledge: 1. SQL 2005 has a wonderful lot of new features like reporting services. 2. grows easier... 3. they both perform at the same speed. 4. SQL 2000 is definitely more intuitive in its use. 5. SQL 2005 can be a real PITA... I.e.. I have not figured out how to store a new or even a modified SP as it defaults to dumping the changes in some unassociated directory off somewhere irrelevant. I had to resort to making modes in SQL 2000 and importing them in. (A bit of reading will solve that but it should be straight forward...). It will not directly import and convert SQL2000 DTS files but relegates them to some sub-directory for legacy files.... Found out where after someone on this list so nicely pointed it out... Running this beast correctly will require a course or lot of reading as nothing is obvious, unlike SQL 2000 which I picked up on the fly and have been sucessfully using it for almost 6 years now. My current opinion will change in time but as a newbie to this application it is a long verticle climb. Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Friday, November 03, 2006 11:13 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? Do you (or anyone reading) have any feelings for performance of SQl Server 2005 vs 2000? I have 2005 but had trouble getting imports happening as easily and so never really explored it. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Friday, November 03, 2006 1:32 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? It is split across 8 servers, each running two processors and each having > 2 GB of RAM. In part this arrangement was due to security concerns, which I won't go into, but also the sheer size of the database made it impossible to put it on one conventional server. All the servers run Windows Server 2003. The first iteration ran on SQL 2000, but the conversion to SQL 2005 is underway. The fact of 8 databases, each on a different server, exposed a serious down side as compared to a similar implementation in Oracle -- MS SQL cannot do referential integrity constraints across databases, so a bunch of additional code + duplicate lookup tables were required. In Oracle, one could simply declare tablespaces then place them on different servers, and it would still remain a single database. Until this project, I had never been involved with anything so large, so the issue never arose. ----- Original Message ---- From: JWColby To: dba-sqlserver at databaseadvisors.com Sent: Friday, November 3, 2006 9:03:11 AM Subject: Re: [dba-SQLServer] Any gut feelings? Arthur, The other question I have given your experience with these big databases, is what kind of hardware were they using? Was this database you mention running on 1/2 proc machines with 2/4 gig of memory, or was it on processor arrays with a dozen or two or three processors and a dozen or two or three gigabytes of main memory? Running windows (what flavor)? SQL Server 2K (what flavor)? John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Friday, November 03, 2006 4:18 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? Given the projects upon which I have worked recently, 65M rows is a smallish table. Mind you, I've been doing everything in SQL Server not Access, but I've been working with situations in which the anticipated growth is 1TB per year. The 700 columns is an issue of a different colour and suggests a serious problem, perhaps not in your final solution but certainly in the source data. Some stakeholder must prioritize these columns and then you must break them out into related tables. (i.e. columns most commonly queried, etc.) 65M rows is not really a large table, except in the Access context. The 700 columns is the locus of your problem, not the rowcount. If you can quantify the probablity of a column being queried, then you can break this big table into several related tables according to probablity of query, then join the related tables if and only if their columns are queried. Just my opinion, but based on working with TB-sized tables. Arthur ----- Original Message ---- From: JWColby To: dba-sqlserver at databaseadvisors.com Sent: Friday, November 3, 2006 12:58:00 AM Subject: [dba-SQLServer] Any gut feelings? I have this huge table, 65 million records, ~700 fields. The table has a bunch of now useless fields which I will be deleting, basically old cass and ncoa data from a previous (many years ago) address verification on the table. More importantly, it also has what I will call demographics data, but extended into such things as brand preference for consumables, hobbies, types of electronics purchases, etc. Obviously the sheer size is an issue here. What I would think to do, coming from a small database background, is to create a PK on the big table, then break this big table down into smaller tables, each focused on one "set" of demographics data. My question to you guys is, given the size of the table in terms of sheer number of records, is this a valid strategy? If I have an indexed and identical PK in each table, can I break out the Address into a table, age/income/race/religion into a table, then tables for boating, electronics, medicines, etc. with each sub table only containing records where the demographics info is populated. IOW, not everyone has boating info, not everyone has medical info, electronics info etc. so I end up with a smaller subset of records in each table where there is only a record in that table if the person has data for that demographic set. Having done that can I then expect to be able to join up to a half dozen tables to get inner join subsets, outer join mega-sets etc. in anything approaching near real time (minutes)? I am in the process of doing the very first (two) breakouts, address and boating. Obviously I have to create a new table, append the data in, then build an index on the PK for starters. Having done that I can do a quick and dirty test, but I would like your opinions on the general feasibility of this approach given the limitations of current (but state of the art) desktop hardware, SBS 2003 and SQL Server 2000. Having answered this (assuming that the answer is "it is probably a valid approach", I then need assistance on various methods to determine the existence of data within a given record of valid demographic data. IOW, I am pulling all 65 million records, a subset of fields (typically 10 to 30 fields) into a smaller table. This smaller table will eventually have an indexed PK, but NO indexes on the various fields. One way I can think of to do something like this is to generate a "validity" field where I store a number which is simply a count of the fields with something other than a zero length string. So record 1 has 0 valid data fields, rec 2 has 3 valid data fields, 3 has 12 valid data fields and so forth. IF I could generate such a query to get these counts and store it in a single field (and ever get results) and then apply an index on that field, anything over a zero has valid data and could be used to pull records, or (in reverse) records with a zero in the "validity" field could be deleted from the table to drop the number of records in that table. Other thoughts on how to accomplish this objective? Remember that this main table is simply freakin huge, with probably 20-50 different demographic groupings and some where the boundaries aren't even clear. In order to do this I will have to create that same number of tables, cut out the data and get it into these new tables, index the PK, generate counts of the validity data, delete records with no valid data etc. Some kind of automation for doing this would be nice, but basically, I can if necessary just brute force it once I have the procedure down. Ideas? Objections? "Your crazy to even think about this"? John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From askolits at ot.com Fri Nov 3 14:21:04 2006 From: askolits at ot.com (John Skolits) Date: Fri, 3 Nov 2006 15:21:04 -0500 Subject: [dba-SQLServer] ODBC Connection error - More Info In-Reply-To: <454B9016.3010306@shaw.ca> Message-ID: <00fb01c6ff85$999f8760$6501a8c0@LaptopXP> I assume you mean the latest MDAC? Or is there something else that would make sure we had the same one. -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of MartyConnelly Sent: Friday, November 03, 2006 1:53 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] ODBC Connection error - More Info Are you both running the same Jet Service Pack See http://www.microsoft.com/data John Skolits wrote: >No, not really. > >I was just on the phone with him and it seems there is a compile issue. >I have two, almost identical functions. One works from the immediate window >and the other gives him a compile error. Although the program compiles >correctly. >I'll first have him uncompile\recompile and repair. Yet, I'm running the >exact same version on an Access 2000 version with no problems. > >The other option would be to have him reinstall Access 2003. > > >As far as the ODBC error, I'm finding that when he manually reconnects the >Trusted Connection is automatically set to True. When I relink through code >that I wrote, I always set the trusted connection to False and have never >had a problem on the many applications I have written. Except on his PC. > >For now, I think it's his installation, although not positive. > > > > >-----Original Message----- >From: dba-sqlserver-bounces at databaseadvisors.com >[mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim >Lawrence >Sent: Friday, November 03, 2006 11:51 AM >To: dba-sqlserver at databaseadvisors.com >Subject: Re: [dba-SQLServer] ODBC Connection error - More Info > >Hi John: > >Would his password have a special character in it that is not allowed? > >Jim > >-----Original Message----- >From: dba-sqlserver-bounces at databaseadvisors.com >[mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of John >Skolits >Sent: Friday, November 03, 2006 7:10 AM >To: dba-sqlserver at databaseadvisors.com >Subject: Re: [dba-SQLServer] ODBC Connection error - More Info > >I'm also getting the following error when trying to run a few queries. > >Connection Failed: >SQL State '28000' >Sql Server Error: 18452 >{Microssoft][Sql Server Driver][Sql Server]Logon Failed for user '(null)' >Reason: Not associated with a trusted cSql Server Conenction. > > >-----Original Message----- >From: dba-sqlserver-bounces at databaseadvisors.com >[mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of John >Skolits >Sent: Friday, November 03, 2006 9:43 AM >To: dba-sqlserver at databaseadvisors.com >Subject: [dba-SQLServer] ODBC Connection error > >I'm having this ODBC connection error that's driving me nuts. > >The application is written in Access 2000. The User's Pc is Access 2003. >MDAC 2.8 is his latest version. > >There have been times where the application has run fine, then I make a few >changes and the user can't connect to the tables. >The only way to make it work is to go in an manually relink every table on >his PC. If I do it on my development PC, or any other PC for that matter, he >can't open the tables. > >I ran a Pass thru query on his PC with a new DSN I created and a connection >string like: >ODBC;DRIVER=SQL Server;SERVER=otto;UID=User;PWD=User;DATABASE=ssa_erpdb > >I then received an error message like : >ODBC error ..... ODBC;DRIVER=SQL Server;SERVER=otto;UID=reports;PWD=repo_' > >Notice how it truncated the string. > >So it looks like It's a problem on his PC. > >I tried a myriad of things. Is there any Service releases or anything anyone >has heard of that may fix this issue before I spend weeks trying to find a >work around. > >John > > >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > > > >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > > > >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > > > > > -- Marty Connelly Victoria, B.C. Canada _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Nov 3 14:51:58 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Fri, 3 Nov 2006 15:51:58 -0500 Subject: [dba-SQLServer] Any gut feelings? In-Reply-To: <0J8600AIK7FE0TC0@l-daemon> Message-ID: <00bb01c6ff89$e812b3a0$657aa8c0@m6805> >My current opinion will change in time but as a newbie to this application it is a long vertical climb. LOL. Not what I need at this very point in time. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Friday, November 03, 2006 2:59 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? Hi John: To the best of my knowledge: 1. SQL 2005 has a wonderful lot of new features like reporting services. 2. grows easier... 3. they both perform at the same speed. 4. SQL 2000 is definitely more intuitive in its use. 5. SQL 2005 can be a real PITA... I.e.. I have not figured out how to store a new or even a modified SP as it defaults to dumping the changes in some unassociated directory off somewhere irrelevant. I had to resort to making modes in SQL 2000 and importing them in. (A bit of reading will solve that but it should be straight forward...). It will not directly import and convert SQL2000 DTS files but relegates them to some sub-directory for legacy files.... Found out where after someone on this list so nicely pointed it out... Running this beast correctly will require a course or lot of reading as nothing is obvious, unlike SQL 2000 which I picked up on the fly and have been sucessfully using it for almost 6 years now. My current opinion will change in time but as a newbie to this application it is a long verticle climb. Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Friday, November 03, 2006 11:13 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? Do you (or anyone reading) have any feelings for performance of SQl Server 2005 vs 2000? I have 2005 but had trouble getting imports happening as easily and so never really explored it. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Friday, November 03, 2006 1:32 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? It is split across 8 servers, each running two processors and each having > 2 GB of RAM. In part this arrangement was due to security concerns, which I won't go into, but also the sheer size of the database made it impossible to put it on one conventional server. All the servers run Windows Server 2003. The first iteration ran on SQL 2000, but the conversion to SQL 2005 is underway. The fact of 8 databases, each on a different server, exposed a serious down side as compared to a similar implementation in Oracle -- MS SQL cannot do referential integrity constraints across databases, so a bunch of additional code + duplicate lookup tables were required. In Oracle, one could simply declare tablespaces then place them on different servers, and it would still remain a single database. Until this project, I had never been involved with anything so large, so the issue never arose. ----- Original Message ---- From: JWColby To: dba-sqlserver at databaseadvisors.com Sent: Friday, November 3, 2006 9:03:11 AM Subject: Re: [dba-SQLServer] Any gut feelings? Arthur, The other question I have given your experience with these big databases, is what kind of hardware were they using? Was this database you mention running on 1/2 proc machines with 2/4 gig of memory, or was it on processor arrays with a dozen or two or three processors and a dozen or two or three gigabytes of main memory? Running windows (what flavor)? SQL Server 2K (what flavor)? John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Friday, November 03, 2006 4:18 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? Given the projects upon which I have worked recently, 65M rows is a smallish table. Mind you, I've been doing everything in SQL Server not Access, but I've been working with situations in which the anticipated growth is 1TB per year. The 700 columns is an issue of a different colour and suggests a serious problem, perhaps not in your final solution but certainly in the source data. Some stakeholder must prioritize these columns and then you must break them out into related tables. (i.e. columns most commonly queried, etc.) 65M rows is not really a large table, except in the Access context. The 700 columns is the locus of your problem, not the rowcount. If you can quantify the probablity of a column being queried, then you can break this big table into several related tables according to probablity of query, then join the related tables if and only if their columns are queried. Just my opinion, but based on working with TB-sized tables. Arthur ----- Original Message ---- From: JWColby To: dba-sqlserver at databaseadvisors.com Sent: Friday, November 3, 2006 12:58:00 AM Subject: [dba-SQLServer] Any gut feelings? I have this huge table, 65 million records, ~700 fields. The table has a bunch of now useless fields which I will be deleting, basically old cass and ncoa data from a previous (many years ago) address verification on the table. More importantly, it also has what I will call demographics data, but extended into such things as brand preference for consumables, hobbies, types of electronics purchases, etc. Obviously the sheer size is an issue here. What I would think to do, coming from a small database background, is to create a PK on the big table, then break this big table down into smaller tables, each focused on one "set" of demographics data. My question to you guys is, given the size of the table in terms of sheer number of records, is this a valid strategy? If I have an indexed and identical PK in each table, can I break out the Address into a table, age/income/race/religion into a table, then tables for boating, electronics, medicines, etc. with each sub table only containing records where the demographics info is populated. IOW, not everyone has boating info, not everyone has medical info, electronics info etc. so I end up with a smaller subset of records in each table where there is only a record in that table if the person has data for that demographic set. Having done that can I then expect to be able to join up to a half dozen tables to get inner join subsets, outer join mega-sets etc. in anything approaching near real time (minutes)? I am in the process of doing the very first (two) breakouts, address and boating. Obviously I have to create a new table, append the data in, then build an index on the PK for starters. Having done that I can do a quick and dirty test, but I would like your opinions on the general feasibility of this approach given the limitations of current (but state of the art) desktop hardware, SBS 2003 and SQL Server 2000. Having answered this (assuming that the answer is "it is probably a valid approach", I then need assistance on various methods to determine the existence of data within a given record of valid demographic data. IOW, I am pulling all 65 million records, a subset of fields (typically 10 to 30 fields) into a smaller table. This smaller table will eventually have an indexed PK, but NO indexes on the various fields. One way I can think of to do something like this is to generate a "validity" field where I store a number which is simply a count of the fields with something other than a zero length string. So record 1 has 0 valid data fields, rec 2 has 3 valid data fields, 3 has 12 valid data fields and so forth. IF I could generate such a query to get these counts and store it in a single field (and ever get results) and then apply an index on that field, anything over a zero has valid data and could be used to pull records, or (in reverse) records with a zero in the "validity" field could be deleted from the table to drop the number of records in that table. Other thoughts on how to accomplish this objective? Remember that this main table is simply freakin huge, with probably 20-50 different demographic groupings and some where the boundaries aren't even clear. In order to do this I will have to create that same number of tables, cut out the data and get it into these new tables, index the PK, generate counts of the validity data, delete records with no valid data etc. Some kind of automation for doing this would be nice, but basically, I can if necessary just brute force it once I have the procedure down. Ideas? Objections? "Your crazy to even think about this"? John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From martyconnelly at shaw.ca Fri Nov 3 14:54:33 2006 From: martyconnelly at shaw.ca (MartyConnelly) Date: Fri, 03 Nov 2006 12:54:33 -0800 Subject: [dba-SQLServer] ODBC Connection error - More Info In-Reply-To: <00fb01c6ff85$999f8760$6501a8c0@LaptopXP> References: <00fb01c6ff85$999f8760$6501a8c0@LaptopXP> Message-ID: <454BAC89.4090602@shaw.ca> Jet Service Packs were seperated from the MDAC install after MDAC 2.5 Jet 8 SP available as seperate install from here http://msdn.microsoft.com/data/ref/mdac/downloads/ Various OS install a set version of MDAC ie WinXP SP2 installs MDAC 2.8 However Access 2000 installs Jet SP4 and Access 2003 JetSP7 and Access 2007 Jet SP8 You might want to check if he is running under Admin or User privileges. Also for further info How to keep a Jet 4.0 database in top working condition http://support.microsoft.com/default.aspx?scid=kb;en-us;Q303528 John Skolits wrote: >I assume you mean the latest MDAC? Or is there something else that would >make sure we had the same one. > >-----Original Message----- >From: dba-sqlserver-bounces at databaseadvisors.com >[mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of >MartyConnelly >Sent: Friday, November 03, 2006 1:53 PM >To: dba-sqlserver at databaseadvisors.com >Subject: Re: [dba-SQLServer] ODBC Connection error - More Info > >Are you both running the same Jet Service Pack >See >http://www.microsoft.com/data > >John Skolits wrote: > > > >>No, not really. >> >>I was just on the phone with him and it seems there is a compile issue. >> >> > > > >>I have two, almost identical functions. One works from the immediate window >>and the other gives him a compile error. Although the program compiles >>correctly. >>I'll first have him uncompile\recompile and repair. Yet, I'm running the >>exact same version on an Access 2000 version with no problems. >> >>The other option would be to have him reinstall Access 2003. >> >> >>As far as the ODBC error, I'm finding that when he manually reconnects the >>Trusted Connection is automatically set to True. When I relink through code >>that I wrote, I always set the trusted connection to False and have never >>had a problem on the many applications I have written. Except on his PC. >> >>For now, I think it's his installation, although not positive. >> >> >> >> >>-----Original Message----- >>From: dba-sqlserver-bounces at databaseadvisors.com >>[mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim >>Lawrence >>Sent: Friday, November 03, 2006 11:51 AM >>To: dba-sqlserver at databaseadvisors.com >>Subject: Re: [dba-SQLServer] ODBC Connection error - More Info >> >>Hi John: >> >>Would his password have a special character in it that is not allowed? >> >>Jim >> >>-----Original Message----- >>From: dba-sqlserver-bounces at databaseadvisors.com >>[mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of John >>Skolits >>Sent: Friday, November 03, 2006 7:10 AM >>To: dba-sqlserver at databaseadvisors.com >>Subject: Re: [dba-SQLServer] ODBC Connection error - More Info >> >>I'm also getting the following error when trying to run a few queries. >> >>Connection Failed: >>SQL State '28000' >>Sql Server Error: 18452 >>{Microssoft][Sql Server Driver][Sql Server]Logon Failed for user '(null)' >>Reason: Not associated with a trusted cSql Server Conenction. >> >> >>-----Original Message----- >>From: dba-sqlserver-bounces at databaseadvisors.com >>[mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of John >>Skolits >>Sent: Friday, November 03, 2006 9:43 AM >>To: dba-sqlserver at databaseadvisors.com >>Subject: [dba-SQLServer] ODBC Connection error >> >>I'm having this ODBC connection error that's driving me nuts. >> >>The application is written in Access 2000. The User's Pc is Access 2003. >>MDAC 2.8 is his latest version. >> >>There have been times where the application has run fine, then I make a few >>changes and the user can't connect to the tables. >>The only way to make it work is to go in an manually relink every table on >>his PC. If I do it on my development PC, or any other PC for that matter, >> >> >he > > >>can't open the tables. >> >>I ran a Pass thru query on his PC with a new DSN I created and a connection >>string like: >>ODBC;DRIVER=SQL Server;SERVER=otto;UID=User;PWD=User;DATABASE=ssa_erpdb >> >>I then received an error message like : >>ODBC error ..... ODBC;DRIVER=SQL Server;SERVER=otto;UID=reports;PWD=repo_' >> >>Notice how it truncated the string. >> >>So it looks like It's a problem on his PC. >> >>I tried a myriad of things. Is there any Service releases or anything >> >> >anyone > > >>has heard of that may fix this issue before I spend weeks trying to find a >>work around. >> >>John >> >> >> >> -- Marty Connelly Victoria, B.C. Canada From jwcolby at colbyconsulting.com Fri Nov 3 14:55:10 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Fri, 3 Nov 2006 15:55:10 -0500 Subject: [dba-SQLServer] Any gut feelings? In-Reply-To: <20061103183133.58538.qmail@web88210.mail.re2.yahoo.com> Message-ID: <00bc01c6ff8a$5a96a670$657aa8c0@m6805> So question: Is it possible to spread tables out across databases and use queries that join tables in other databases (inside the same SQL Server instance)? The reason I ask is that I have all of these raw data tables, which I need, need to keep around, but don't need active. It would be handy (for size of database files reasons) to have the raw source tables out in their own database and the finished processed, indexed tables in another table. Getting the raw data into the finished data table is the issue. Exporting 65 million records, even a few fields, to a csv file or the like is not exactly dba friendly. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Friday, November 03, 2006 1:32 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? It is split across 8 servers, each running two processors and each having > 2 GB of RAM. In part this arrangement was due to security concerns, which I won't go into, but also the sheer size of the database made it impossible to put it on one conventional server. All the servers run Windows Server 2003. The first iteration ran on SQL 2000, but the conversion to SQL 2005 is underway. The fact of 8 databases, each on a different server, exposed a serious down side as compared to a similar implementation in Oracle -- MS SQL cannot do referential integrity constraints across databases, so a bunch of additional code + duplicate lookup tables were required. In Oracle, one could simply declare tablespaces then place them on different servers, and it would still remain a single database. Until this project, I had never been involved with anything so large, so the issue never arose. ----- Original Message ---- From: JWColby To: dba-sqlserver at databaseadvisors.com Sent: Friday, November 3, 2006 9:03:11 AM Subject: Re: [dba-SQLServer] Any gut feelings? Arthur, The other question I have given your experience with these big databases, is what kind of hardware were they using? Was this database you mention running on 1/2 proc machines with 2/4 gig of memory, or was it on processor arrays with a dozen or two or three processors and a dozen or two or three gigabytes of main memory? Running windows (what flavor)? SQL Server 2K (what flavor)? John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Friday, November 03, 2006 4:18 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? Given the projects upon which I have worked recently, 65M rows is a smallish table. Mind you, I've been doing everything in SQL Server not Access, but I've been working with situations in which the anticipated growth is 1TB per year. The 700 columns is an issue of a different colour and suggests a serious problem, perhaps not in your final solution but certainly in the source data. Some stakeholder must prioritize these columns and then you must break them out into related tables. (i.e. columns most commonly queried, etc.) 65M rows is not really a large table, except in the Access context. The 700 columns is the locus of your problem, not the rowcount. If you can quantify the probablity of a column being queried, then you can break this big table into several related tables according to probablity of query, then join the related tables if and only if their columns are queried. Just my opinion, but based on working with TB-sized tables. Arthur ----- Original Message ---- From: JWColby To: dba-sqlserver at databaseadvisors.com Sent: Friday, November 3, 2006 12:58:00 AM Subject: [dba-SQLServer] Any gut feelings? I have this huge table, 65 million records, ~700 fields. The table has a bunch of now useless fields which I will be deleting, basically old cass and ncoa data from a previous (many years ago) address verification on the table. More importantly, it also has what I will call demographics data, but extended into such things as brand preference for consumables, hobbies, types of electronics purchases, etc. Obviously the sheer size is an issue here. What I would think to do, coming from a small database background, is to create a PK on the big table, then break this big table down into smaller tables, each focused on one "set" of demographics data. My question to you guys is, given the size of the table in terms of sheer number of records, is this a valid strategy? If I have an indexed and identical PK in each table, can I break out the Address into a table, age/income/race/religion into a table, then tables for boating, electronics, medicines, etc. with each sub table only containing records where the demographics info is populated. IOW, not everyone has boating info, not everyone has medical info, electronics info etc. so I end up with a smaller subset of records in each table where there is only a record in that table if the person has data for that demographic set. Having done that can I then expect to be able to join up to a half dozen tables to get inner join subsets, outer join mega-sets etc. in anything approaching near real time (minutes)? I am in the process of doing the very first (two) breakouts, address and boating. Obviously I have to create a new table, append the data in, then build an index on the PK for starters. Having done that I can do a quick and dirty test, but I would like your opinions on the general feasibility of this approach given the limitations of current (but state of the art) desktop hardware, SBS 2003 and SQL Server 2000. Having answered this (assuming that the answer is "it is probably a valid approach", I then need assistance on various methods to determine the existence of data within a given record of valid demographic data. IOW, I am pulling all 65 million records, a subset of fields (typically 10 to 30 fields) into a smaller table. This smaller table will eventually have an indexed PK, but NO indexes on the various fields. One way I can think of to do something like this is to generate a "validity" field where I store a number which is simply a count of the fields with something other than a zero length string. So record 1 has 0 valid data fields, rec 2 has 3 valid data fields, 3 has 12 valid data fields and so forth. IF I could generate such a query to get these counts and store it in a single field (and ever get results) and then apply an index on that field, anything over a zero has valid data and could be used to pull records, or (in reverse) records with a zero in the "validity" field could be deleted from the table to drop the number of records in that table. Other thoughts on how to accomplish this objective? Remember that this main table is simply freakin huge, with probably 20-50 different demographic groupings and some where the boundaries aren't even clear. In order to do this I will have to create that same number of tables, cut out the data and get it into these new tables, index the PK, generate counts of the validity data, delete records with no valid data etc. Some kind of automation for doing this would be nice, but basically, I can if necessary just brute force it once I have the procedure down. Ideas? Objections? "Your crazy to even think about this"? John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From mikedorism at verizon.net Fri Nov 3 15:17:42 2006 From: mikedorism at verizon.net (Doris Manning) Date: Fri, 03 Nov 2006 16:17:42 -0500 Subject: [dba-SQLServer] Any gut feelings? In-Reply-To: <00bc01c6ff8a$5a96a670$657aa8c0@m6805> Message-ID: <002f01c6ff8d$803f6df0$2f01a8c0@Kermit> John, Yes, it is possible. Same server instance...Database.dbo.Table.Field Separate server instance... Server.Database.dbo.Table.Field Doris Manning Hargrove Inc. -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Friday, November 03, 2006 3:55 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? So question: Is it possible to spread tables out across databases and use queries that join tables in other databases (inside the same SQL Server instance)? The reason I ask is that I have all of these raw data tables, which I need, need to keep around, but don't need active. It would be handy (for size of database files reasons) to have the raw source tables out in their own database and the finished processed, indexed tables in another table. Getting the raw data into the finished data table is the issue. Exporting 65 million records, even a few fields, to a csv file or the like is not exactly dba friendly. John W. Colby Colby Consulting www.ColbyConsulting.com From jwcolby at colbyconsulting.com Fri Nov 3 15:27:37 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Fri, 3 Nov 2006 16:27:37 -0500 Subject: [dba-SQLServer] Any gut feelings? In-Reply-To: <00bc01c6ff8a$5a96a670$657aa8c0@m6805> Message-ID: <00c001c6ff8e$e33d6b90$657aa8c0@m6805> I guess I wrote this a little too quickly. What I meant is, does SQL have anything like Access where you can with a query build a new table based on a handful of fields fro the source table, and having done that, have the destination field retain at least the same datatypes / data sizes in the destination table. And further can the source tables be out in one SQL Server database, and destination table be inside of a second SQL Server database. I am having to cut and paste fields to the windows paste buffer, then switch to a different (new) table and paste them in in order to get the same fields and same field structures. Even then I am doing this inside of a common database because I don't know how to reference tables across different databases. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Friday, November 03, 2006 3:55 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? So question: Is it possible to spread tables out across databases and use queries that join tables in other databases (inside the same SQL Server instance)? The reason I ask is that I have all of these raw data tables, which I need, need to keep around, but don't need active. It would be handy (for size of database files reasons) to have the raw source tables out in their own database and the finished processed, indexed tables in another table. Getting the raw data into the finished data table is the issue. Exporting 65 million records, even a few fields, to a csv file or the like is not exactly dba friendly. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Friday, November 03, 2006 1:32 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? It is split across 8 servers, each running two processors and each having > 2 GB of RAM. In part this arrangement was due to security concerns, which I won't go into, but also the sheer size of the database made it impossible to put it on one conventional server. All the servers run Windows Server 2003. The first iteration ran on SQL 2000, but the conversion to SQL 2005 is underway. The fact of 8 databases, each on a different server, exposed a serious down side as compared to a similar implementation in Oracle -- MS SQL cannot do referential integrity constraints across databases, so a bunch of additional code + duplicate lookup tables were required. In Oracle, one could simply declare tablespaces then place them on different servers, and it would still remain a single database. Until this project, I had never been involved with anything so large, so the issue never arose. From jwcolby at colbyconsulting.com Fri Nov 3 16:09:30 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Fri, 3 Nov 2006 17:09:30 -0500 Subject: [dba-SQLServer] Any gut feelings? In-Reply-To: <002f01c6ff8d$803f6df0$2f01a8c0@Kermit> Message-ID: <00c401c6ff94$bd39c3c0$657aa8c0@m6805> Whoa, cool. Now you da man! See how easy it is to be da man? Boy am I out of my league here. 8( Now I just gotta try it. If I can do this, then I can actually use that other server machine I built for something useful. Next YouDaMan prize goes to the one who provides the syntax to transfer an entire table - structure, data and indexes intact, from one db to another. That ought to slow you folks down! John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Doris Manning Sent: Friday, November 03, 2006 4:18 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? John, Yes, it is possible. Same server instance...Database.dbo.Table.Field Separate server instance... Server.Database.dbo.Table.Field Doris Manning Hargrove Inc. -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Friday, November 03, 2006 3:55 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? So question: Is it possible to spread tables out across databases and use queries that join tables in other databases (inside the same SQL Server instance)? The reason I ask is that I have all of these raw data tables, which I need, need to keep around, but don't need active. It would be handy (for size of database files reasons) to have the raw source tables out in their own database and the finished processed, indexed tables in another table. Getting the raw data into the finished data table is the issue. Exporting 65 million records, even a few fields, to a csv file or the like is not exactly dba friendly. John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From mikedorism at verizon.net Fri Nov 3 16:37:19 2006 From: mikedorism at verizon.net (Doris Manning) Date: Fri, 03 Nov 2006 17:37:19 -0500 Subject: [dba-SQLServer] Any gut feelings? In-Reply-To: <00c401c6ff94$bd39c3c0$657aa8c0@m6805> Message-ID: <003001c6ff98$9f7cb5a0$2f01a8c0@Kermit> John, Can I claim this one too? Next YouDaMan prize goes to the one who provides the syntax to transfer an entire table - structure, data and indexes intact, from one db to another. That ought to slow you folks down! Check out Data Transformation Service (DTS) in Enterprise Manager. There is a wizard that walks you through it. You set up the from connection (server and database) and to connection (server and database). You pick the objects you want to transfer and off they go. Doris Manning Hargrove Inc. From stuart at lexacorp.com.pg Fri Nov 3 17:17:47 2006 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 04 Nov 2006 09:17:47 +1000 Subject: [dba-SQLServer] Any gut feelings? In-Reply-To: <00c001c6ff8e$e33d6b90$657aa8c0@m6805> References: <00bc01c6ff8a$5a96a670$657aa8c0@m6805>, <00c001c6ff8e$e33d6b90$657aa8c0@m6805> Message-ID: <454BCE1B.30739.1D2D5EEF@stuart.lexacorp.com.pg> On 3 Nov 2006 at 16:27, JWColby wrote: > I guess I wrote this a little too quickly. What I meant is, does SQL have > anything like Access where you can with a query build a new table based on a > handful of fields fro the source table, and having done that, have the > destination field retain at least the same datatypes / data sizes in the > destination table. "SELECT RecordPK, carpref1, carpref2 FROM MainTable INTO NewTable" -- Stuart From jwcolby at colbyconsulting.com Fri Nov 3 17:18:17 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Fri, 3 Nov 2006 18:18:17 -0500 Subject: [dba-SQLServer] Any gut feelings? In-Reply-To: <003001c6ff98$9f7cb5a0$2f01a8c0@Kermit> Message-ID: <00cb01c6ff9e$5908ea70$657aa8c0@m6805> Maybe that was back in the day when I couldn't see the servers from each other. I THINK I have solved that issue so maybe it will work for me now. But... Does it actually transfer the whole shebang - field type, data, indexes etc? Or just a generic "here's your data, now you fix it back up"? So far I have only successfully used DTS to export CSV files which of course promptly loses all the good stuff. Well, my problems aren't completely solved. I can see the database on the new SBS server from the other old server, but not VV. Whatever. I switch to the old server and run the DTS wizard and the table does indeed transfer, DAMNED quickly too! PKs and indexes did NOT move however. So I am now trying the "copy objects" wizard which apparently allows you to copy the whole shebang. It appears to be working, though the speed is not so wonderful. This is a smallish table, ~6 million records, ~15 fields, each having an index. Should be interesting to see how long it takes. But thanks for mentioning this, it will make my job easier if I can master this. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Doris Manning Sent: Friday, November 03, 2006 5:37 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? John, Can I claim this one too? Next YouDaMan prize goes to the one who provides the syntax to transfer an entire table - structure, data and indexes intact, from one db to another. That ought to slow you folks down! Check out Data Transformation Service (DTS) in Enterprise Manager. There is a wizard that walks you through it. You set up the from connection (server and database) and to connection (server and database). You pick the objects you want to transfer and off they go. Doris Manning Hargrove Inc. _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From artful at rogers.com Fri Nov 3 19:02:29 2006 From: artful at rogers.com (artful at rogers.com) Date: Fri, 3 Nov 2006 17:02:29 -0800 (PST) Subject: [dba-SQLServer] Any gut feelings? Message-ID: <20061104010229.58744.qmail@web88203.mail.re2.yahoo.com> Of course you can, JC. Assuming it's a repetitive operation, it's smarter to build the table first, but if not then you can build it dynamically. For the former, the paradigm is the same as Access: INSERT INTO NewTable SELECT this, that, theOther FROM T1 INNER JOIN T1.PK = T2.FK You might have to qualify said statement with column names and the VALUES predicate. If you need to create the table, then SELECT INTO NewTable is the approach. HTH, Arthur ----- Original Message ---- From: JWColby To: dba-sqlserver at databaseadvisors.com Sent: Friday, November 3, 2006 4:27:37 PM Subject: Re: [dba-SQLServer] Any gut feelings? I guess I wrote this a little too quickly. What I meant is, does SQL have anything like Access where you can with a query build a new table based on a handful of fields fro the source table, and having done that, have the destination field retain at least the same datatypes / data sizes in the destination table. And further can the source tables be out in one SQL Server database, and destination table be inside of a second SQL Server database. I am having to cut and paste fields to the windows paste buffer, then switch to a different (new) table and paste them in in order to get the same fields and same field structures. Even then I am doing this inside of a common database because I don't know how to reference tables across different databases. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Friday, November 03, 2006 3:55 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? So question: Is it possible to spread tables out across databases and use queries that join tables in other databases (inside the same SQL Server instance)? The reason I ask is that I have all of these raw data tables, which I need, need to keep around, but don't need active. It would be handy (for size of database files reasons) to have the raw source tables out in their own database and the finished processed, indexed tables in another table. Getting the raw data into the finished data table is the issue. Exporting 65 million records, even a few fields, to a csv file or the like is not exactly dba friendly. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Friday, November 03, 2006 1:32 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Any gut feelings? It is split across 8 servers, each running two processors and each having > 2 GB of RAM. In part this arrangement was due to security concerns, which I won't go into, but also the sheer size of the database made it impossible to put it on one conventional server. All the servers run Windows Server 2003. The first iteration ran on SQL 2000, but the conversion to SQL 2005 is underway. The fact of 8 databases, each on a different server, exposed a serious down side as compared to a similar implementation in Oracle -- MS SQL cannot do referential integrity constraints across databases, so a bunch of additional code + duplicate lookup tables were required. In Oracle, one could simply declare tablespaces then place them on different servers, and it would still remain a single database. Until this project, I had never been involved with anything so large, so the issue never arose. _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Nov 4 22:42:31 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Sat, 4 Nov 2006 23:42:31 -0500 Subject: [dba-SQLServer] Dual cores running 100% Message-ID: <000701c70094$d4860060$657aa8c0@m6805> LOL, well I have nirvana. Well... sorta. I am in the process of CASS validating my 65 million record database using Accuzip software, a rather kludgy process which uses FoxPro to validate ~3 million record chunks. I have two instances running on two different files. As you guys probably know by now, my system is now running Windows 2003 SBS, a dual core AMD X2 at 2.8 GHz and 2 gbytes and a raid 6 array with two largish volumes. Running just one instance of Accuzip the system processes about 6 million records / hour. Starting the second instance, leaves the first still processing about 6 million records per hour but the second is processing about 2.4 million records per hour. I find the inequality odd. My experience in the past (using XP Pro) is that it slowed down the first instance and both instances would run at roughly the same speed. Whatever. So basically I get an additional 40+ % processing using the second instance of Accuzip. Boy would it be nice to have a 16x processor array right now, but this is usable. I have to process ~22 files, each file would take ~30 minutes if processed by itself. Once the processing is finished, I delete the undeliverables and pack the FoxPro database, then start the next file importing. Once all 22 files are processed, I then have to re-import them into SQL Server and process them further there. At that point after the import I will have deliverable addresses, I am deleting all undeliverables. Since the exported / reimported records have a PK that matches my original (big) table, I will then have the ability to create a master list of completely valid deliverable ADDRESSES (not necessarily people at those addresses). It appears so far the non-deliverables are running around 14% which means I will end up with 86% of 64 million addresses in the deliverable master address table. This is only the address validation side of the picture. The fun begins when I normalize the big table into a bunch of smaller tables as discussed in an earlier email. I have actually done that for one specific job (a boating database) in order to do matching on a small (50K) database of names of yacht owners that they got from a client. The biggest problem is that in order to match names (or even addresses) you have to compare apples to apples and neither database was in CASSed form, and neither database provided "match codes" do do the comparison on. Much work quickly to make this happen. John W. Colby Colby Consulting www.ColbyConsulting.com From artful at rogers.com Sat Nov 11 11:43:02 2006 From: artful at rogers.com (artful at rogers.com) Date: Sat, 11 Nov 2006 09:43:02 -0800 (PST) Subject: [dba-SQLServer] Front-End or Back-End or Middle-Tier Solutions Message-ID: <20061111174302.88491.qmail@web88205.mail.re2.yahoo.com> As my birthday approaches, I like to revisit one or two cherished assumptions and call them into question. Sometimes I learn something new, sometimes I just confirm the assumptions. For as long as I've been involved with databases that allow sprocs and UDFs, I have held the cherished belief that everything the back-end can do, the back-end should do. But I am aware that a lot of folks don't agree with this, and it's not because they're fools. I even know one seasoned DBA who lives at the other end of the spectrum on this argument -- he designs databases to contain the absolute minimum required. He uses Referential Integrity of course, and stored procedures, but anything remotely resembling business logic (that is, anything other than select, insert, delete, update) he believes should reside in either the middle tier or the front end, and anything remotely resembling presentation should reside in the front-end. As an example of the latter, he says crosstabs (or PIVOTs in SQL 2005 lingo) should not be done in the back end, but rather left up to the front end. I'm wondering where you listers stand on this spectrum. Incidentally, my 59th birthday is November 15. Large stacks of currency will be gratefully received. Arthur From mwp.reid at qub.ac.uk Sat Nov 11 12:32:50 2006 From: mwp.reid at qub.ac.uk (Martin Reid) Date: Sat, 11 Nov 2006 18:32:50 -0000 Subject: [dba-SQLServer] Front-End or Back-End or Middle-Tier Solutions References: <20061111174302.88491.qmail@web88205.mail.re2.yahoo.com> Message-ID: Happy Birthday Arthur. As you know someoen with 6 children would never see a stack of notes so unable to grant your wish (<: Funny just working on a design and for the first time I will be using three tiers. Classes for the Data (Thanks to JC) a business logic tier and .NET presentation. Well actually .NET for the whole thing. Should be interesting as I have done none of this before. But in general I would agree with you. As much as possible on the server in the form of SPs and UDFs etc. Martin Martin WP Reid Training and Assessment Unit Riddle Hall Belfast tel: 02890 974477 ________________________________ From: dba-sqlserver-bounces at databaseadvisors.com on behalf of artful at rogers.com Sent: Sat 11/11/2006 17:43 To: dba-SQLServer Subject: [dba-SQLServer] Front-End or Back-End or Middle-Tier Solutions As my birthday approaches, I like to revisit one or two cherished assumptions and call them into question. Sometimes I learn something new, sometimes I just confirm the assumptions. For as long as I've been involved with databases that allow sprocs and UDFs, I have held the cherished belief that everything the back-end can do, the back-end should do. But I am aware that a lot of folks don't agree with this, and it's not because they're fools. I even know one seasoned DBA who lives at the other end of the spectrum on this argument -- he designs databases to contain the absolute minimum required. He uses Referential Integrity of course, and stored procedures, but anything remotely resembling business logic (that is, anything other than select, insert, delete, update) he believes should reside in either the middle tier or the front end, and anything remotely resembling presentation should reside in the front-end. As an example of the latter, he says crosstabs (or PIVOTs in SQL 2005 lingo) should not be done in the back end, but rather left up to the front end. I'm wondering where you listers stand on this spectrum. Incidentally, my 59th birthday is November 15. Large stacks of currency will be gratefully received. Arthur _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From lizd1 at charter.net Sat Nov 11 13:16:32 2006 From: lizd1 at charter.net (Liz Doering) Date: Sat, 11 Nov 2006 13:16:32 -0600 Subject: [dba-SQLServer] Job Change, and a Question In-Reply-To: <20061111174302.88491.qmail@web88205.mail.re2.yahoo.com> Message-ID: <000501c705c5$e838ef10$6601a8c0@Lillian> Dear SQL Server List, It is no doubt due to lurking on this list that I have now landed myself in the kettle of soup I'm in. Two weeks ago, I was a all-Access developer at a tiny almost-all-Access shop. Now I am the SQL Server DBA on a new project internal to Wells Fargo. That I was able to stumble through the interview satisfactorily was due to you folks having put enough concepts in my head so that I could avoid sounding like a complete idiot. Of course, now that I've been on the job for a week, the veneer of knowledge is looking thinner and thinner, and the number of questions I can't answer is growing daily. Here's the most recent poser: They're using SmallTalk for this project, which is actually a major extension of another application which has been in use for 10+ years. They are devoted to using Windows authentication. The developers would like SmallTalk to be able to run SQL statements directly from their code, however, they can only do that if the logged-in user has such rights. Which raises the specter of a savvy user running random SQL statements directly against the database. The solution has been to disallow running any "unapproved" SQL statements by requiring that only sprocs can be run, but the development team isn't happy with that solution, so they are asking me for alternatives. I'm getting more confused as I write this, so I'll guarantee you that there are questions I don't know enough to ask. Can you recommend some reading for me? Do any of you have any specific ideas for this problem? Thanks so much! Hopefully I'll be wise enough to contribute here eventually! Liz From accessd at shaw.ca Sat Nov 11 13:26:51 2006 From: accessd at shaw.ca (Jim Lawrence) Date: Sat, 11 Nov 2006 11:26:51 -0800 Subject: [dba-SQLServer] Front-End or Back-End or Middle-Tier Solutions In-Reply-To: <20061111174302.88491.qmail@web88205.mail.re2.yahoo.com> Message-ID: <0J8K0031UZ9A6IG1@l-daemon> Hi Art: The move towards web based applications is pushing towards tight integration with the data and application all being controlled and ran from the server. The leaders in this design are definitely Oracle and IBM and the one detractor is Microsoft who has a huge investment in desktop applications. I am half way in between. With everything running off the server or servers (more like it) the process can become very expensive for the host... that is just what IBM and Oracle want as they are not interested in individuals or small companies, they are only interested in where the big easy money is. I think that a system should be as redundant and distributive as possible without exposing the data or the business logic. Clients should do their share of the base computing and leave the all important data management up to the server. There should not be any tight integration and the data receipt and requests should be made using XML. Server based queues should manage the linked data requests... On the other had, I think most applications should run as web based; using ASP.Net instead of the desktop equivalent. AJAX/Atlas is a great technology and I have been designing all my new apps in that direction. On a slightly different note, do you think LINUX servers and even stations (desktops like Susi and Ubuntu are so simple to install and use.... anyone who can install Window is set and they have all the bells and whistles.) will overtake our Microsoft world? Our generation of programmers has a vested interest but I have noticed that the younger systems people are becoming skilled in the open-source products. A student can not afford M$ products so they become skilled at what they have access to.... and they are the system admin, managers, application developers and even company owners of our future. The only thing that is holding this trend at bay is a really super kick-ass database product that is either free or very cheap. Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Saturday, November 11, 2006 9:43 AM To: dba-SQLServer Subject: [dba-SQLServer] Front-End or Back-End or Middle-Tier Solutions As my birthday approaches, I like to revisit one or two cherished assumptions and call them into question. Sometimes I learn something new, sometimes I just confirm the assumptions. For as long as I've been involved with databases that allow sprocs and UDFs, I have held the cherished belief that everything the back-end can do, the back-end should do. But I am aware that a lot of folks don't agree with this, and it's not because they're fools. I even know one seasoned DBA who lives at the other end of the spectrum on this argument -- he designs databases to contain the absolute minimum required. He uses Referential Integrity of course, and stored procedures, but anything remotely resembling business logic (that is, anything other than select, insert, delete, update) he believes should reside in either the middle tier or the front end, and anything remotely resembling presentation should reside in the front-end. As an example of the latter, he says crosstabs (or PIVOTs in SQL 2005 lingo) should not be done in the back end, but rather left up to the front end. I'm wondering where you listers stand on this spectrum. Incidentally, my 59th birthday is November 15. Large stacks of currency will be gratefully received. Arthur _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From accessd at shaw.ca Sat Nov 11 13:29:10 2006 From: accessd at shaw.ca (Jim Lawrence) Date: Sat, 11 Nov 2006 11:29:10 -0800 Subject: [dba-SQLServer] Happy Birthday In-Reply-To: <20061111174302.88491.qmail@web88205.mail.re2.yahoo.com> Message-ID: <0J8K00DWZZD5IHA1@l-daemon> Well Arthur congratulations :-) I am right behind you with my 57 birthday on the 24th of November... Best regards for the day. Jim From jwcolby at colbyconsulting.com Sat Nov 11 13:50:10 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Sat, 11 Nov 2006 14:50:10 -0500 Subject: [dba-SQLServer] Front-End or Back-End or Middle-Tier Solutions In-Reply-To: <20061111174302.88491.qmail@web88205.mail.re2.yahoo.com> Message-ID: <000901c705ca$99d7ae10$d60f390a@m6805> Happy bday. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Saturday, November 11, 2006 12:43 PM To: dba-SQLServer Subject: [dba-SQLServer] Front-End or Back-End or Middle-Tier Solutions As my birthday approaches, I like to revisit one or two cherished assumptions and call them into question. Sometimes I learn something new, sometimes I just confirm the assumptions. For as long as I've been involved with databases that allow sprocs and UDFs, I have held the cherished belief that everything the back-end can do, the back-end should do. But I am aware that a lot of folks don't agree with this, and it's not because they're fools. I even know one seasoned DBA who lives at the other end of the spectrum on this argument -- he designs databases to contain the absolute minimum required. He uses Referential Integrity of course, and stored procedures, but anything remotely resembling business logic (that is, anything other than select, insert, delete, update) he believes should reside in either the middle tier or the front end, and anything remotely resembling presentation should reside in the front-end. As an example of the latter, he says crosstabs (or PIVOTs in SQL 2005 lingo) should not be done in the back end, but rather left up to the front end. I'm wondering where you listers stand on this spectrum. Incidentally, my 59th birthday is November 15. Large stacks of currency will be gratefully received. Arthur _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Nov 11 13:53:34 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Sat, 11 Nov 2006 14:53:34 -0500 Subject: [dba-SQLServer] Happy Birthday In-Reply-To: <0J8K00DWZZD5IHA1@l-daemon> Message-ID: <000a01c705cb$13793900$d60f390a@m6805> And happy bday to you too. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Jim Lawrence Sent: Saturday, November 11, 2006 2:29 PM To: dba-sqlserver at databaseadvisors.com Subject: [dba-SQLServer] Happy Birthday Well Arthur congratulations :-) I am right behind you with my 57 birthday on the 24th of November... Best regards for the day. Jim _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From ssharkins at setel.com Sat Nov 11 14:03:02 2006 From: ssharkins at setel.com (Susan Harkins) Date: Sat, 11 Nov 2006 15:03:02 -0500 Subject: [dba-SQLServer] Happy Birthday In-Reply-To: <000a01c705cb$13793900$d60f390a@m6805> Message-ID: <001501c705cc$65b338a0$e5b62ad1@SUSANONE> Oh my gosh -- so many Sags -- it must mean something -- Nov 27 Susan H. And happy bday to you too. Well Arthur congratulations :-) I am right behind you with my 57 birthday on the 24th of November... Best regards for the day. Jim From artful at rogers.com Sat Nov 11 14:21:35 2006 From: artful at rogers.com (artful at rogers.com) Date: Sat, 11 Nov 2006 12:21:35 -0800 (PST) Subject: [dba-SQLServer] Job Change, and a Question Message-ID: <20061111202135.71369.qmail@web88212.mail.re2.yahoo.com> I understand your dilemma, Liz, but I feel most compelled to protest your developers' insistence on running SQL directly, for at least two reasons: SQL injection is difficult to defend against when wild-ass SQL statements are coming in, and this method implies no clearly-thought-out security (roles/users/etc.). In your position, I would resolutely stand my ground in favor of sprocs/UDFs/views, all of which have been locked down with roles. A few really basic guidelines, not intended to be exhaustive: 1. NOBODY gets direct access to tables, except you (and your team, and even then, think carefully about the possible damage someone could do -- the damage might be accidental or might stem from a disgruntled employee). 2. For each table, there should be 4 sprocs -- insert, update, delete and select. NOBODY should be able to touch a table without going through your sprocs. (These, incidentally, can be generated, although you might need to tweak the code.) 3. Clearly define the roles that various users and employees and managers and developers might occupy. A role means, oversimplifying, that Role A can do this and that, Role B can do everything Role A can plus some other things, and Role C can do everything Role B can, plus some other things. Once you have a clear handle on the roles, you can define access to the various sprocs. To take just one example, in my last app only three people (all in one role) had the right to insert, delete or update the Countries table. Similarly for the States/Provinces table (it is rare that Canada adds a province, USA adds a state, Switzerland adds a canton, etc. On the other hand, the Cities table will probably not be exhaustive, so any given salesperson should be able to enter one. Fight the developers tooth and nail on their preference for dynamic SQL. Be politically savvy about it, and ask them how to fire a stored procedure in their language. Learn as much as you can about the intricacies therein. Find out if their language lets them fire UDFs and views, too. But guide them toward relying on the back end, and gracefully point out the risks inherent in dynamic SQL. (If you need info on this, google SQL Injection.) This is perhaps more information than you wanted. Arthur Dear SQL Server List, It is no doubt due to lurking on this list that I have now landed myself in the kettle of soup I'm in. Two weeks ago, I was a all-Access developer at a tiny almost-all-Access shop. Now I am the SQL Server DBA on a new project internal to Wells Fargo. That I was able to stumble through the interview satisfactorily was due to you folks having put enough concepts in my head so that I could avoid sounding like a complete idiot. f course, now that I've been on the job for a week, the veneer of knowledge is looking thinner and thinner, and the number of questions I can't answer is growing daily. Here's the most recent poser: They're using SmallTalk for this project, which is actually a major extension of another application which has been in use for 10+ years. They are devoted to using Windows authentication. The developers would like SmallTalk to be able to run SQL statements directly from their code, however, they can only do that if the logged-in user has such rights. Which raises the specter of a savvy user running random SQL statements directly against the database. The solution has been to disallow running any "unapproved" SQL statements by requiring that only sprocs can be run, but the development team isn't happy with that solution, so they are asking me for alternatives. I'm getting more confused as I write this, so I'll guarantee you that there are questions I don't know enough to ask. Can you recommend some reading for me? Do any of you have any specific ideas for this problem? Thanks so much! Hopefully I'll be wise enough to contribute here eventually! Liz From mwp.reid at qub.ac.uk Sat Nov 11 14:28:03 2006 From: mwp.reid at qub.ac.uk (Martin Reid) Date: Sat, 11 Nov 2006 20:28:03 -0000 Subject: [dba-SQLServer] Job Change, and a Question References: <20061111202135.71369.qmail@web88212.mail.re2.yahoo.com> Message-ID: This is perhaps more information than you wanted. But good advice. Martin From ssharkins at setel.com Sat Nov 11 14:35:14 2006 From: ssharkins at setel.com (Susan Harkins) Date: Sat, 11 Nov 2006 15:35:14 -0500 Subject: [dba-SQLServer] Job Change, and a Question In-Reply-To: Message-ID: <002701c705d0$f00e6480$e5b62ad1@SUSANONE> I agree Liz and I'm no expert on this subject -- but part of your job is to maintain security and sometimes that's going to put you at odds with the development teams. Although, diplomacy can go a long way. :) In the end, you're actually doing them a favor -- you're protecting their data and designs -- their livelihoods in other words. :) Susan H. This is perhaps more information than you wanted. But good advice. Martin From martyconnelly at shaw.ca Sat Nov 11 17:21:00 2006 From: martyconnelly at shaw.ca (MartyConnelly) Date: Sat, 11 Nov 2006 15:21:00 -0800 Subject: [dba-SQLServer] Job Change, and a Question In-Reply-To: <000501c705c5$e838ef10$6601a8c0@Lillian> References: <000501c705c5$e838ef10$6601a8c0@Lillian> Message-ID: <45565ADC.20400@shaw.ca> Could you not give them a development or sandbox copy of the database for them to bang away at. Or even a MSDE 2.0 personal copy. When their code goes to production tests convert the SQL to SProcs. Liz Doering wrote: >Dear SQL Server List, > >It is no doubt due to lurking on this list that I have now landed myself in >the kettle of soup I'm in. > >Two weeks ago, I was a all-Access developer at a tiny almost-all-Access >shop. Now I am the SQL Server DBA on a new project internal to Wells Fargo. >That I was able to stumble through the interview satisfactorily was due to >you folks having put enough concepts in my head so that I could avoid >sounding like a complete idiot. > >Of course, now that I've been on the job for a week, the veneer of knowledge >is looking thinner and thinner, and the number of questions I can't answer >is growing daily. > >Here's the most recent poser: They're using SmallTalk for this project, >which is actually a major extension of another application which has been in >use for 10+ years. They are devoted to using Windows authentication. The >developers would like SmallTalk to be able to run SQL statements directly >from their code, however, they can only do that if the logged-in user has >such rights. Which raises the specter of a savvy user running random SQL >statements directly against the database. The solution has been to disallow >running any "unapproved" SQL statements by requiring that only sprocs can be >run, but the development team isn't happy with that solution, so they are >asking me for alternatives. > >I'm getting more confused as I write this, so I'll guarantee you that there >are questions I don't know enough to ask. Can you recommend some reading for >me? Do any of you have any specific ideas for this problem? > >Thanks so much! Hopefully I'll be wise enough to contribute here >eventually! > > >Liz > > > >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > > > > > -- Marty Connelly Victoria, B.C. Canada From artful at rogers.com Sun Nov 12 02:47:03 2006 From: artful at rogers.com (artful at rogers.com) Date: Sun, 12 Nov 2006 00:47:03 -0800 (PST) Subject: [dba-SQLServer] Job Change, and a Question Message-ID: <20061112084703.8197.qmail@web88203.mail.re2.yahoo.com> I definitely agree with the sandbox but disagree with the idea of letting them run dynamic SQL even against a sandbox DB. Far better to inspect their requirements, with their help, and to design sprocs and UDFs that attempt to fulfill them, IMO. These can be refined in the sandbox. For example, their UI presents three combo-boxes, and you write sprocs to deliver what is required to populate them, perhaps returning an OUT parameter so it's easy for them to grab the selected value. You work through the UI with their guidance and deliver sprocs that do what they need. From what you wrote, it appears that this project is FE-driven. That in itself is not a bad thing, but you must keep them constantly aware of the costs and risks of dynamic SQL. You could even prove this point by a) comparing the speed of a dynamic statement vs. an equivalent sproc; and b) demonstrating a SQL injection that destroys all the rows in some table that is critical to the operation. The first is easily constructed. Suppose that the statement accepts three variables and then constructs a statment along the lines of: SELECT * FROM SomeTables WHERE Column1 = Var1 AND Column2 = Var2 and Column3 = Var3 Admittedly a simplistic example, but the logic extends. All this would require is a sproc that accepts 3 variables and does the equivalent Select. This takes advantage of the compiler and the optimizer, while the dynamic statement can never do either. Against a large number of rows, the difference won't even require a timer -- you can see it. The second requires a little imagination, but goes approximately like this. The UI wants the name of the customer. The user types in: Joe Blow'; Go; Delete Customers -- This string gets piped into a dynamic statement along the lines of: Insert Into Customers 'Joe Blow'; Go; Delete Customers --' Note a couple of things in this improvised example. The user is knowledgable enough to anticipate that the argument will be wrapped in single quotes, so includes one at the alleged end of the data-entry part of the string, but then the rest of the string gets plugged in as standard and legitimate SQL, and executes (absent countermeasures, of course). The simplest countermeasure is a sproc that accepts the parameter(s). It's 3am and this was not a great example of SQL injection, but for more info and better examples, google SQL Injection and you will find lots. It is also true that some situations demand dynamic SQL, for example when a screen allows the user to select among 20 or 30 controls, leaving most blank (null) and choosing a random 5 to fill in. It is difficult (but not impossible) to write the sproc that can deal with such a situation. I emphasize that it is NOT impossible, but I realize that the techniques required are beyond the ken of many SQL developers. Once you adopt the sproc-route, you need 4 sprocs per table, plus sprocs/UDFs to populate various listboxes and combo-boxes etc. throughout the interface. The UI guys' responsibility is to gather the user input, then call the appropriate sprocs. If the back-end is solid, then even if the front-end has logical holes, the worst that can happen is an ugly message indicating failure. I can live with that. A. ----- Original Message ---- From: MartyConnelly To: dba-sqlserver at databaseadvisors.com Sent: Saturday, November 11, 2006 6:21:00 PM Subject: Re: [dba-SQLServer] Job Change, and a Question Could you not give them a development or sandbox copy of the database for them to bang away at. Or even a MSDE 2.0 personal copy. When their code goes to production tests convert the SQL to SProcs. Liz Doering wrote: >Dear SQL Server List, > >It is no doubt due to lurking on this list that I have now landed myself in >the kettle of soup I'm in. > >Two weeks ago, I was a all-Access developer at a tiny almost-all-Access >shop. Now I am the SQL Server DBA on a new project internal to Wells Fargo. >That I was able to stumble through the interview satisfactorily was due to >you folks having put enough concepts in my head so that I could avoid >sounding like a complete idiot. > >Of course, now that I've been on the job for a week, the veneer of knowledge >is looking thinner and thinner, and the number of questions I can't answer >is growing daily. > >Here's the most recent poser: They're using SmallTalk for this project, >which is actually a major extension of another application which has been in >use for 10+ years. They are devoted to using Windows authentication. The >developers would like SmallTalk to be able to run SQL statements directly >from their code, however, they can only do that if the logged-in user has >such rights. Which raises the specter of a savvy user running random SQL >statements directly against the database. The solution has been to disallow >running any "unapproved" SQL statements by requiring that only sprocs can be >run, but the development team isn't happy with that solution, so they are >asking me for alternatives. > >I'm getting more confused as I write this, so I'll guarantee you that there >are questions I don't know enough to ask. Can you recommend some reading for >me? Do any of you have any specific ideas for this problem? > >Thanks so much! Hopefully I'll be wise enough to contribute here >eventually! > > >Liz > > > >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > > > > > -- Marty Connelly Victoria, B.C. Canada _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From ssharkins at setel.com Sun Nov 12 09:11:10 2006 From: ssharkins at setel.com (Susan Harkins) Date: Sun, 12 Nov 2006 10:11:10 -0500 Subject: [dba-SQLServer] Job Change, and a Question In-Reply-To: <20061112084703.8197.qmail@web88203.mail.re2.yahoo.com> Message-ID: <003001c7066c$ce4fa830$c632fad1@SUSANONE> If you run a google on SQL Injection, you'll get a ton of information. I had a white paper on protecting yourself from the problem, and it was really good, but I no longer have it. I apologize, but I'm sure it's still out there. Susan H. I definitely agree with the sandbox but disagree with the idea of letting them run dynamic SQL even against a sandbox DB. Far better to inspect their requirements, with their help, and to design sprocs and UDFs that attempt to fulfill them, IMO. These can be refined in the sandbox. For example, their UI presents three combo-boxes, and you write sprocs to deliver what is required to populate them, perhaps returning an OUT parameter so it's easy for them to grab the selected value. You work through the UI with their guidance and deliver sprocs that do what they need. From michael at ddisolutions.com.au Sun Nov 12 15:54:54 2006 From: michael at ddisolutions.com.au (Michael Maddison) Date: Mon, 13 Nov 2006 08:54:54 +1100 Subject: [dba-SQLServer] Job Change, and a Question Message-ID: <59A61174B1F5B54B97FD4ADDE71E7D0116ACA1@ddi-01.DDI.local> I basically agree with Arthur but would question the statement about sprocs being faster. This is a generalisation of course, but, SQL + Access (IIRC) even DO store execution plans for dynamic SQL. Another argument to use sprocs is that you the dba can then optimise the SQL within without recompiling the exe/dll's of the app. cheers Michael Maddison DDI Solutions Pty Ltd michael at ddisolutions.com.au Bus: 0260400620 Mob: 0412620497 www.ddisolutions.com.au -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Sunday, 12 November 2006 7:47 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Job Change, and a Question I definitely agree with the sandbox but disagree with the idea of letting them run dynamic SQL even against a sandbox DB. Far better to inspect their requirements, with their help, and to design sprocs and UDFs that attempt to fulfill them, IMO. These can be refined in the sandbox. For example, their UI presents three combo-boxes, and you write sprocs to deliver what is required to populate them, perhaps returning an OUT parameter so it's easy for them to grab the selected value. You work through the UI with their guidance and deliver sprocs that do what they need. >From what you wrote, it appears that this project is FE-driven. That in itself is not a bad thing, but you must keep them constantly aware of the costs and risks of dynamic SQL. You could even prove this point by a) comparing the speed of a dynamic statement vs. an equivalent sproc; and b) demonstrating a SQL injection that destroys all the rows in some table that is critical to the operation. The first is easily constructed. Suppose that the statement accepts three variables and then constructs a statment along the lines of: SELECT * FROM SomeTables WHERE Column1 = Var1 AND Column2 = Var2 and Column3 = Var3 Admittedly a simplistic example, but the logic extends. All this would require is a sproc that accepts 3 variables and does the equivalent Select. This takes advantage of the compiler and the optimizer, while the dynamic statement can never do either. Against a large number of rows, the difference won't even require a timer -- you can see it. The second requires a little imagination, but goes approximately like this. The UI wants the name of the customer. The user types in: Joe Blow'; Go; Delete Customers -- This string gets piped into a dynamic statement along the lines of: Insert Into Customers 'Joe Blow'; Go; Delete Customers --' Note a couple of things in this improvised example. The user is knowledgable enough to anticipate that the argument will be wrapped in single quotes, so includes one at the alleged end of the data-entry part of the string, but then the rest of the string gets plugged in as standard and legitimate SQL, and executes (absent countermeasures, of course). The simplest countermeasure is a sproc that accepts the parameter(s). It's 3am and this was not a great example of SQL injection, but for more info and better examples, google SQL Injection and you will find lots. It is also true that some situations demand dynamic SQL, for example when a screen allows the user to select among 20 or 30 controls, leaving most blank (null) and choosing a random 5 to fill in. It is difficult (but not impossible) to write the sproc that can deal with such a situation. I emphasize that it is NOT impossible, but I realize that the techniques required are beyond the ken of many SQL developers. Once you adopt the sproc-route, you need 4 sprocs per table, plus sprocs/UDFs to populate various listboxes and combo-boxes etc. throughout the interface. The UI guys' responsibility is to gather the user input, then call the appropriate sprocs. If the back-end is solid, then even if the front-end has logical holes, the worst that can happen is an ugly message indicating failure. I can live with that. A. ----- Original Message ---- From: MartyConnelly To: dba-sqlserver at databaseadvisors.com Sent: Saturday, November 11, 2006 6:21:00 PM Subject: Re: [dba-SQLServer] Job Change, and a Question Could you not give them a development or sandbox copy of the database for them to bang away at. Or even a MSDE 2.0 personal copy. When their code goes to production tests convert the SQL to SProcs. Liz Doering wrote: >Dear SQL Server List, > >It is no doubt due to lurking on this list that I have now landed >myself in the kettle of soup I'm in. > >Two weeks ago, I was a all-Access developer at a tiny almost-all-Access >shop. Now I am the SQL Server DBA on a new project internal to Wells Fargo. >That I was able to stumble through the interview satisfactorily was due >to you folks having put enough concepts in my head so that I could >avoid sounding like a complete idiot. > >Of course, now that I've been on the job for a week, the veneer of >knowledge is looking thinner and thinner, and the number of questions I >can't answer is growing daily. > >Here's the most recent poser: They're using SmallTalk for this >project, which is actually a major extension of another application >which has been in use for 10+ years. They are devoted to using Windows >authentication. The developers would like SmallTalk to be able to run >SQL statements directly from their code, however, they can only do that >if the logged-in user has such rights. Which raises the specter of a >savvy user running random SQL statements directly against the database. >The solution has been to disallow running any "unapproved" SQL >statements by requiring that only sprocs can be run, but the >development team isn't happy with that solution, so they are asking me for alternatives. > >I'm getting more confused as I write this, so I'll guarantee you that >there are questions I don't know enough to ask. Can you recommend some >reading for me? Do any of you have any specific ideas for this problem? > >Thanks so much! Hopefully I'll be wise enough to contribute here >eventually! > > >Liz > > > >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > > > > > -- Marty Connelly Victoria, B.C. Canada _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From stuart at lexacorp.com.pg Sun Nov 12 15:59:38 2006 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Mon, 13 Nov 2006 07:59:38 +1000 Subject: [dba-SQLServer] Job Change, and a Question In-Reply-To: <59A61174B1F5B54B97FD4ADDE71E7D0116ACA1@ddi-01.DDI.local> References: <59A61174B1F5B54B97FD4ADDE71E7D0116ACA1@ddi-01.DDI.local> Message-ID: <4557994A.5557.911B30A@stuart.lexacorp.com.pg> On 13 Nov 2006 at 8:54, Michael Maddison wrote: > Another argument to use sprocs is that you the dba can then optimise > the SQL within without > recompiling the exe/dll's of the app. Not just the SQL, you can change the database itself if you need to without breaking the exe, the FE doesn't need to know anything abou how the BE is structured as long as the sprocs still work the same. -- Stuart From accessd at shaw.ca Sun Nov 12 16:53:05 2006 From: accessd at shaw.ca (Jim Lawrence) Date: Sun, 12 Nov 2006 14:53:05 -0800 Subject: [dba-SQLServer] Job Change, and a Question In-Reply-To: <59A61174B1F5B54B97FD4ADDE71E7D0116ACA1@ddi-01.DDI.local> Message-ID: <0J8N00CXN3GVX1Q3@l-daemon> Michael... Just a note... To the best of my knowledge SPs are faster at least the second time they are run and this is usually the case when a site is active. The SP when ran first is compiled and optimized and every other immediate access uses the compiled version in memory. When only sending parameters to an SP a lot less data is needed to be sent to DB and this should also produce a speed increase of course virtually un-measurable. Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Michael Maddison Sent: Sunday, November 12, 2006 1:55 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Job Change, and a Question I basically agree with Arthur but would question the statement about sprocs being faster. This is a generalisation of course, but, SQL + Access (IIRC) even DO store execution plans for dynamic SQL. Another argument to use sprocs is that you the dba can then optimise the SQL within without recompiling the exe/dll's of the app. cheers Michael Maddison DDI Solutions Pty Ltd michael at ddisolutions.com.au Bus: 0260400620 Mob: 0412620497 www.ddisolutions.com.au -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Sunday, 12 November 2006 7:47 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Job Change, and a Question I definitely agree with the sandbox but disagree with the idea of letting them run dynamic SQL even against a sandbox DB. Far better to inspect their requirements, with their help, and to design sprocs and UDFs that attempt to fulfill them, IMO. These can be refined in the sandbox. For example, their UI presents three combo-boxes, and you write sprocs to deliver what is required to populate them, perhaps returning an OUT parameter so it's easy for them to grab the selected value. You work through the UI with their guidance and deliver sprocs that do what they need. >From what you wrote, it appears that this project is FE-driven. That in itself is not a bad thing, but you must keep them constantly aware of the costs and risks of dynamic SQL. You could even prove this point by a) comparing the speed of a dynamic statement vs. an equivalent sproc; and b) demonstrating a SQL injection that destroys all the rows in some table that is critical to the operation. The first is easily constructed. Suppose that the statement accepts three variables and then constructs a statment along the lines of: SELECT * FROM SomeTables WHERE Column1 = Var1 AND Column2 = Var2 and Column3 = Var3 Admittedly a simplistic example, but the logic extends. All this would require is a sproc that accepts 3 variables and does the equivalent Select. This takes advantage of the compiler and the optimizer, while the dynamic statement can never do either. Against a large number of rows, the difference won't even require a timer -- you can see it. The second requires a little imagination, but goes approximately like this. The UI wants the name of the customer. The user types in: Joe Blow'; Go; Delete Customers -- This string gets piped into a dynamic statement along the lines of: Insert Into Customers 'Joe Blow'; Go; Delete Customers --' Note a couple of things in this improvised example. The user is knowledgable enough to anticipate that the argument will be wrapped in single quotes, so includes one at the alleged end of the data-entry part of the string, but then the rest of the string gets plugged in as standard and legitimate SQL, and executes (absent countermeasures, of course). The simplest countermeasure is a sproc that accepts the parameter(s). It's 3am and this was not a great example of SQL injection, but for more info and better examples, google SQL Injection and you will find lots. It is also true that some situations demand dynamic SQL, for example when a screen allows the user to select among 20 or 30 controls, leaving most blank (null) and choosing a random 5 to fill in. It is difficult (but not impossible) to write the sproc that can deal with such a situation. I emphasize that it is NOT impossible, but I realize that the techniques required are beyond the ken of many SQL developers. Once you adopt the sproc-route, you need 4 sprocs per table, plus sprocs/UDFs to populate various listboxes and combo-boxes etc. throughout the interface. The UI guys' responsibility is to gather the user input, then call the appropriate sprocs. If the back-end is solid, then even if the front-end has logical holes, the worst that can happen is an ugly message indicating failure. I can live with that. A. ----- Original Message ---- From: MartyConnelly To: dba-sqlserver at databaseadvisors.com Sent: Saturday, November 11, 2006 6:21:00 PM Subject: Re: [dba-SQLServer] Job Change, and a Question Could you not give them a development or sandbox copy of the database for them to bang away at. Or even a MSDE 2.0 personal copy. When their code goes to production tests convert the SQL to SProcs. Liz Doering wrote: >Dear SQL Server List, > >It is no doubt due to lurking on this list that I have now landed >myself in the kettle of soup I'm in. > >Two weeks ago, I was a all-Access developer at a tiny almost-all-Access >shop. Now I am the SQL Server DBA on a new project internal to Wells Fargo. >That I was able to stumble through the interview satisfactorily was due >to you folks having put enough concepts in my head so that I could >avoid sounding like a complete idiot. > >Of course, now that I've been on the job for a week, the veneer of >knowledge is looking thinner and thinner, and the number of questions I >can't answer is growing daily. > >Here's the most recent poser: They're using SmallTalk for this >project, which is actually a major extension of another application >which has been in use for 10+ years. They are devoted to using Windows >authentication. The developers would like SmallTalk to be able to run >SQL statements directly from their code, however, they can only do that >if the logged-in user has such rights. Which raises the specter of a >savvy user running random SQL statements directly against the database. >The solution has been to disallow running any "unapproved" SQL >statements by requiring that only sprocs can be run, but the >development team isn't happy with that solution, so they are asking me for alternatives. > >I'm getting more confused as I write this, so I'll guarantee you that >there are questions I don't know enough to ask. Can you recommend some >reading for me? Do any of you have any specific ideas for this problem? > >Thanks so much! Hopefully I'll be wise enough to contribute here >eventually! > > >Liz > > > >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > > > > > -- Marty Connelly Victoria, B.C. Canada _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From artful at rogers.com Tue Nov 14 12:20:41 2006 From: artful at rogers.com (artful at rogers.com) Date: Tue, 14 Nov 2006 10:20:41 -0800 (PST) Subject: [dba-SQLServer] Useful Lists Message-ID: <20061114182042.41237.qmail@web88209.mail.re2.yahoo.com> On the basis of the thread about airport lists, I proposed to Jim Lawrence that we accumulate a bunch of similar lists. As my father said when I joined the CDN Air Force at 17, "Don't volunteer for anything." O well. Jim has invited me to spearhead the acquisition of said lists, and I have accepted the assignment. So, to all of you listers, three questions: 1. What lists do you possess that you think would be useful to your fellow AccessD/dba-SQL listers? 2. What lists do you lack that you desire? (I couldn't resist the old programmer's joke. There are three types of programmers -- those who can count and those who can't.) Chances are that more than one of you may volunteer to contribute similar or identical lists. The latter is easy to deal with. The former is a little tougher, since it might require UNIONing several lists, but that's ok. I have one list ready to go, which is the list of words corresponding to letters that travel agents use to spell names. I think it is identical to the similar list from the military, but perhaps not. (In case you don't know what I mean, Able Baker Charlie etc.) I also have another list of cities within North America, and states and provinces to correspond. It is not ready to go, there are some duplicates, but I could prune the dupes. The list consists only of those cities into which the company I was formerly associated with sold products, but it numbers about 5,000 cities, give or take 3. Far from exhaustive, but a good start, and similar lists could easily be UNIONed. That leaves out all the listers uninterested in cities in North America, but listers residing elsewhere might be able to contribute more cities. On this one, there is a difficulty. Within North America, there are states and provinces. In Switzerland, there are cantons. On this subject, I have just done some searches in dictionary.com and come up with some hilarious definitions: City -- an important town Town -- a large village Village -- a small community or group of houses in a rural area, larger than a hamlet and usually smaller than a town, and sometimes (as in parts of the U.S.) incorporated as a municipality. Hamlet -- British. a village without a church of its own, belonging to the parish of another village or town. County (the richest by far) -- 1.the largest administrative division of a U.S. state: Miami, Florida, is in Dade County. 2.one of the chief administrative divisions of a country or state, as in Great Britain and Ireland. 3.one of the larger divisions for purposes of local administration, as in Canada and New Zealand. 4.the territory of a county, esp. its rural areas, as in: "We farmed out in the county before moving to town.". 5.the inhabitants of a county, as in, "It was supposed to be a secret, but you told the whole county.". 6.the domain of a count or earl. All these years I have been under the (clearly false) impression that precise population-numbers defined these terms. Apparently I have wrong, lo these decades. I have just taken a local poll (only 4 people) and the agreement here is that a city is 100,000 people or more; a town is 999,999 people or fewer; a village is 2,000 people or fewer; a hamlet is 500 people or fewer. We four Canadians readily agreed on these numbers, but that might be something we picked up in school that has no relation to the larger world. Arthur From fhtapia at gmail.com Tue Nov 14 17:34:11 2006 From: fhtapia at gmail.com (Francisco Tapia) Date: Tue, 14 Nov 2006 15:34:11 -0800 Subject: [dba-SQLServer] Happy Birthday In-Reply-To: <001501c705cc$65b338a0$e5b62ad1@SUSANONE> References: <000a01c705cb$13793900$d60f390a@m6805> <001501c705cc$65b338a0$e5b62ad1@SUSANONE> Message-ID: SAGs? On 11/11/06, Susan Harkins wrote: > Oh my gosh -- so many Sags -- it must mean something -- Nov 27 > > Susan H. > > And happy bday to you too. > > Well Arthur congratulations :-) > > I am right behind you with my 57 birthday on the 24th of November... > > Best regards for the day. > Jim > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > -- -Francisco http://sqlthis.blogspot.com | Tsql and More... From stuart at lexacorp.com.pg Tue Nov 14 18:01:18 2006 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Wed, 15 Nov 2006 10:01:18 +1000 Subject: [dba-SQLServer] Happy Birthday In-Reply-To: References: <000a01c705cb$13793900$d60f390a@m6805>, <001501c705cc$65b338a0$e5b62ad1@SUSANONE>, Message-ID: <455AE56E.25782.6FD6C74F@stuart.lexacorp.com.pg> Star signs - Sagitarians? On 14 Nov 2006 at 15:34, Francisco Tapia wrote: > SAGs? > > > On 11/11/06, Susan Harkins wrote: > > Oh my gosh -- so many Sags -- it must mean something -- Nov 27 > > > > Susan H. > > > > And happy bday to you too. > > > > Well Arthur congratulations :-) > > > > I am right behind you with my 57 birthday on the 24th of November... > > > > Best regards for the day. > > Jim > > > > _______________________________________________ > > dba-SQLServer mailing list > > dba-SQLServer at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > > http://www.databaseadvisors.com > > > > > > > -- > -Francisco > http://sqlthis.blogspot.com | Tsql and More... > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > From artful at rogers.com Tue Nov 14 18:14:44 2006 From: artful at rogers.com (artful at rogers.com) Date: Tue, 14 Nov 2006 16:14:44 -0800 (PST) Subject: [dba-SQLServer] Happy Birthday Message-ID: <20061115001444.50422.qmail@web88203.mail.re2.yahoo.com> Could be, but if so, incorrect. According to this ancient assignation, I'm a Scorpion. ----- Original Message ---- From: Stuart McLachlan To: dba-sqlserver at databaseadvisors.com Sent: Tuesday, November 14, 2006 7:01:18 PM Subject: Re: [dba-SQLServer] Happy Birthday Star signs - Sagitarians? On 14 Nov 2006 at 15:34, Francisco Tapia wrote: > SAGs? From artful at rogers.com Tue Nov 14 18:18:54 2006 From: artful at rogers.com (artful at rogers.com) Date: Tue, 14 Nov 2006 16:18:54 -0800 (PST) Subject: [dba-SQLServer] Happy Birthday Message-ID: <20061115001855.29257.qmail@web88214.mail.re2.yahoo.com> How happy shall it be? I'm awaiting Fedex with large parcels of small denominations of currency. ----- Original Message ---- From: Stuart McLachlan To: dba-sqlserver at databaseadvisors.com Sent: Tuesday, November 14, 2006 7:01:18 PM Subject: Re: [dba-SQLServer] Happy Birthday Star signs - Sagitarians? From stuart at lexacorp.com.pg Tue Nov 14 18:19:07 2006 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Wed, 15 Nov 2006 10:19:07 +1000 Subject: [dba-SQLServer] Happy Birthday In-Reply-To: <20061115001444.50422.qmail@web88203.mail.re2.yahoo.com> References: <20061115001444.50422.qmail@web88203.mail.re2.yahoo.com> Message-ID: <455AE99B.1944.6FE71784@stuart.lexacorp.com.pg> Just guessing, I never bothered to learnt what star signs are when. On 14 Nov 2006 at 16:14, artful at rogers.com wrote: > Could be, but if so, incorrect. According to this ancient assignation, I'm a Scorpion. > > ----- Original Message ---- > From: Stuart McLachlan > To: dba-sqlserver at databaseadvisors.com > Sent: Tuesday, November 14, 2006 7:01:18 PM > Subject: Re: [dba-SQLServer] Happy Birthday > > Star signs - Sagitarians? > > On 14 Nov 2006 at 15:34, Francisco Tapia wrote: > > > SAGs? > > > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > From rl_stewart at highstream.net Wed Nov 15 13:46:26 2006 From: rl_stewart at highstream.net (Robert L. Stewart) Date: Wed, 15 Nov 2006 13:46:26 -0600 Subject: [dba-SQLServer] Useful Lists In-Reply-To: References: Message-ID: <200611151951.kAFJpMq11089@databaseadvisors.com> Oh Well, I tried their postal code web service for the USA. Mine is 77418. The area code changed over 5 years ago from 409 to 979. Their service is still returning 409 as the area code. Art and all, Since Art cross posted this, I will put the offer here also. I am willing to host the databases in SQL Server for all to connect to for the data. As long as we all behave and keep the list up to date as well as not put junk into them. I have a couple, like the postal code list, some of the ISO standards lists for country codes, SIC and NAICS code, and so on that I can start it with it any one else is interested. I also have some of the TIGER data tables for the USA. Robert At 08:27 AM 11/15/2006, you wrote: >Date: Tue, 14 Nov 2006 21:08:15 -0800 >From: MartyConnelly >Subject: Re: [AccessD] Useful Lists >To: Access Developers discussion and problem solving > >Message-ID: <455AA0BF.7050407 at shaw.ca> >Content-Type: text/plain; charset=windows-1252; format=flowed > >In addition to lists there are public web services that provide things like >those listed below from places like >http://www.webservicex.net/WS/default.aspx At 12:00 PM 11/15/2006, you wrote: >Date: Tue, 14 Nov 2006 10:20:41 -0800 (PST) >From: artful at rogers.com >Subject: [dba-SQLServer] Useful Lists >To: "AccessD at databaseadvisors. com" , > dba-SQLServer >Message-ID: <20061114182042.41237.qmail at web88209.mail.re2.yahoo.com> >Content-Type: text/plain; charset=ascii > >On the basis of the thread about airport lists, I proposed to Jim >Lawrence that we accumulate a bunch of similar lists. As my father >said when I joined the CDN Air Force at 17, "Don't volunteer for >anything." O well. Jim has invited me to spearhead the acquisition >of said lists, and I have accepted the assignment. > >So, to all of you listers, three questions: > >1. What lists do you possess that you think would be useful to your >fellow AccessD/dba-SQL listers? >2. What lists do you lack that you desire? > >(I couldn't resist the old programmer's joke. There are three types >of programmers -- those who can count and those who can't.) > >Chances are that more than one of you may volunteer to contribute >similar or identical lists. The latter is easy to deal with. The >former is a little tougher, since it might require UNIONing several >lists, but that's ok. > >I have one list ready to go, which is the list of words >corresponding to letters that travel agents use to spell names. I >think it is identical to the similar list from the military, but >perhaps not. (In case you don't know what I mean, Able Baker Charlie etc.) > >I also have another list of cities within North America, and states >and provinces to correspond. It is not ready to go, there are some >duplicates, but I could prune the dupes. The list consists only of >those cities into which the company I was formerly associated with >sold products, but it numbers about 5,000 cities, give or take 3. >Far from exhaustive, but a good start, and similar lists could >easily be UNIONed. That leaves out all the listers uninterested in >cities in North America, but listers residing elsewhere might be >able to contribute more cities. On this one, there is a difficulty. >Within North America, there are states and provinces. In >Switzerland, there are cantons. > >On this subject, I have just done some searches in dictionary.com >and come up with some hilarious definitions: > >City -- an important town >Town -- a large village >Village -- a small community or group of houses in a rural area, larger than a >hamlet and usually smaller than a town, and sometimes (as in parts of >the U.S.) incorporated as a municipality. >Hamlet -- British. a village without a church of its own, belonging >to the parish of another village or town. >County (the richest by far) -- >1.the largest administrative division of a U.S. state: Miami, >Florida, is in Dade County. > >2.one of the chief administrative divisions of a country or state, >as in Great Britain and Ireland. > >3.one of the larger divisions for purposes of local administration, >as in Canada and New Zealand. > >4.the territory of a county, esp. its rural areas, as in: "We farmed >out in the county before moving to town.". > >5.the inhabitants of a county, as in, "It was supposed to be a >secret, but you told the whole county.". > >6.the domain of a count or earl. > > >All these years I have been under the (clearly false) impression >that precise population-numbers defined these terms. Apparently I >have wrong, lo these decades. I have just taken a local poll (only 4 >people) and the agreement here is that a city is 100,000 people or >more; a town is 999,999 people or fewer; a village is 2,000 people >or fewer; a hamlet is 500 people or fewer. We four Canadians readily >agreed on these numbers, but that might be something we picked up in >school that has no relation to the larger world. > >Arthur > From accessd at shaw.ca Wed Nov 15 15:38:59 2006 From: accessd at shaw.ca (Jim Lawrence) Date: Wed, 15 Nov 2006 13:38:59 -0800 Subject: [dba-SQLServer] Useful Lists In-Reply-To: <200611151951.kAFJpMq11089@databaseadvisors.com> Message-ID: <0J8S0035WK2O2350@l-daemon> Hi Robert: We still have room on DBA server for files and we have a SQL server as well... but if you want I could set up a re-direct to download lists from you server. Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Robert L. Stewart Sent: Wednesday, November 15, 2006 11:46 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Useful Lists Oh Well, I tried their postal code web service for the USA. Mine is 77418. The area code changed over 5 years ago from 409 to 979. Their service is still returning 409 as the area code. Art and all, Since Art cross posted this, I will put the offer here also. I am willing to host the databases in SQL Server for all to connect to for the data. As long as we all behave and keep the list up to date as well as not put junk into them. I have a couple, like the postal code list, some of the ISO standards lists for country codes, SIC and NAICS code, and so on that I can start it with it any one else is interested. I also have some of the TIGER data tables for the USA. Robert At 08:27 AM 11/15/2006, you wrote: >Date: Tue, 14 Nov 2006 21:08:15 -0800 >From: MartyConnelly >Subject: Re: [AccessD] Useful Lists >To: Access Developers discussion and problem solving > >Message-ID: <455AA0BF.7050407 at shaw.ca> >Content-Type: text/plain; charset=windows-1252; format=flowed > >In addition to lists there are public web services that provide things like >those listed below from places like >http://www.webservicex.net/WS/default.aspx At 12:00 PM 11/15/2006, you wrote: >Date: Tue, 14 Nov 2006 10:20:41 -0800 (PST) >From: artful at rogers.com >Subject: [dba-SQLServer] Useful Lists >To: "AccessD at databaseadvisors. com" , > dba-SQLServer >Message-ID: <20061114182042.41237.qmail at web88209.mail.re2.yahoo.com> >Content-Type: text/plain; charset=ascii > >On the basis of the thread about airport lists, I proposed to Jim >Lawrence that we accumulate a bunch of similar lists. As my father >said when I joined the CDN Air Force at 17, "Don't volunteer for >anything." O well. Jim has invited me to spearhead the acquisition >of said lists, and I have accepted the assignment. > >So, to all of you listers, three questions: > >1. What lists do you possess that you think would be useful to your >fellow AccessD/dba-SQL listers? >2. What lists do you lack that you desire? > >(I couldn't resist the old programmer's joke. There are three types >of programmers -- those who can count and those who can't.) > >Chances are that more than one of you may volunteer to contribute >similar or identical lists. The latter is easy to deal with. The >former is a little tougher, since it might require UNIONing several >lists, but that's ok. > >I have one list ready to go, which is the list of words >corresponding to letters that travel agents use to spell names. I >think it is identical to the similar list from the military, but >perhaps not. (In case you don't know what I mean, Able Baker Charlie etc.) > >I also have another list of cities within North America, and states >and provinces to correspond. It is not ready to go, there are some >duplicates, but I could prune the dupes. The list consists only of >those cities into which the company I was formerly associated with >sold products, but it numbers about 5,000 cities, give or take 3. >Far from exhaustive, but a good start, and similar lists could >easily be UNIONed. That leaves out all the listers uninterested in >cities in North America, but listers residing elsewhere might be >able to contribute more cities. On this one, there is a difficulty. >Within North America, there are states and provinces. In >Switzerland, there are cantons. > >On this subject, I have just done some searches in dictionary.com >and come up with some hilarious definitions: > >City -- an important town >Town -- a large village >Village -- a small community or group of houses in a rural area, larger than a >hamlet and usually smaller than a town, and sometimes (as in parts of >the U.S.) incorporated as a municipality. >Hamlet -- British. a village without a church of its own, belonging >to the parish of another village or town. >County (the richest by far) -- >1.the largest administrative division of a U.S. state: Miami, >Florida, is in Dade County. > >2.one of the chief administrative divisions of a country or state, >as in Great Britain and Ireland. > >3.one of the larger divisions for purposes of local administration, >as in Canada and New Zealand. > >4.the territory of a county, esp. its rural areas, as in: "We farmed >out in the county before moving to town.". > >5.the inhabitants of a county, as in, "It was supposed to be a >secret, but you told the whole county.". > >6.the domain of a count or earl. > > >All these years I have been under the (clearly false) impression >that precise population-numbers defined these terms. Apparently I >have wrong, lo these decades. I have just taken a local poll (only 4 >people) and the agreement here is that a city is 100,000 people or >more; a town is 999,999 people or fewer; a village is 2,000 people >or fewer; a hamlet is 500 people or fewer. We four Canadians readily >agreed on these numbers, but that might be something we picked up in >school that has no relation to the larger world. > >Arthur > _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From artful at rogers.com Wed Nov 15 19:58:34 2006 From: artful at rogers.com (artful at rogers.com) Date: Wed, 15 Nov 2006 17:58:34 -0800 (PST) Subject: [dba-SQLServer] Useful Lists Message-ID: <20061116015834.16327.qmail@web88201.mail.re2.yahoo.com> What a cool offer! I will immediately convert the Access DBs that I have so far to MS-SQL then zip the backups and send them to you for restoration. Arthur ----- Original Message ---- From: Robert L. Stewart To: dba-sqlserver at databaseadvisors.com Sent: Wednesday, November 15, 2006 2:46:26 PM Subject: Re: [dba-SQLServer] Useful Lists Oh Well, I tried their postal code web service for the USA. Mine is 77418. The area code changed over 5 years ago from 409 to 979. Their service is still returning 409 as the area code. Art and all, Since Art cross posted this, I will put the offer here also. I am willing to host the databases in SQL Server for all to connect to for the data. As long as we all behave and keep the list up to date as well as not put junk into them. I have a couple, like the postal code list, some of the ISO standards lists for country codes, SIC and NAICS code, and so on that I can start it with it any one else is interested. I also have some of the TIGER data tables for the USA. Robert At 08:27 AM 11/15/2006, you wrote: >Date: Tue, 14 Nov 2006 21:08:15 -0800 >From: MartyConnelly >Subject: Re: [AccessD] Useful Lists >To: Access Developers discussion and problem solving > >Message-ID: <455AA0BF.7050407 at shaw.ca> >Content-Type: text/plain; charset=windows-1252; format=flowed > >In addition to lists there are public web services that provide things like >those listed below from places like >http://www.webservicex.net/WS/default.aspx At 12:00 PM 11/15/2006, you wrote: >Date: Tue, 14 Nov 2006 10:20:41 -0800 (PST) >From: artful at rogers.com >Subject: [dba-SQLServer] Useful Lists >To: "AccessD at databaseadvisors. com" , > dba-SQLServer >Message-ID: <20061114182042.41237.qmail at web88209.mail.re2.yahoo.com> >Content-Type: text/plain; charset=ascii > >On the basis of the thread about airport lists, I proposed to Jim >Lawrence that we accumulate a bunch of similar lists. As my father >said when I joined the CDN Air Force at 17, "Don't volunteer for >anything." O well. Jim has invited me to spearhead the acquisition >of said lists, and I have accepted the assignment. > >So, to all of you listers, three questions: > >1. What lists do you possess that you think would be useful to your >fellow AccessD/dba-SQL listers? >2. What lists do you lack that you desire? > >(I couldn't resist the old programmer's joke. There are three types >of programmers -- those who can count and those who can't.) > >Chances are that more than one of you may volunteer to contribute >similar or identical lists. The latter is easy to deal with. The >former is a little tougher, since it might require UNIONing several >lists, but that's ok. > >I have one list ready to go, which is the list of words >corresponding to letters that travel agents use to spell names. I >think it is identical to the similar list from the military, but >perhaps not. (In case you don't know what I mean, Able Baker Charlie etc.) > >I also have another list of cities within North America, and states >and provinces to correspond. It is not ready to go, there are some >duplicates, but I could prune the dupes. The list consists only of >those cities into which the company I was formerly associated with >sold products, but it numbers about 5,000 cities, give or take 3. >Far from exhaustive, but a good start, and similar lists could >easily be UNIONed. That leaves out all the listers uninterested in >cities in North America, but listers residing elsewhere might be >able to contribute more cities. On this one, there is a difficulty. >Within North America, there are states and provinces. In >Switzerland, there are cantons. > >On this subject, I have just done some searches in dictionary.com >and come up with some hilarious definitions: > >City -- an important town >Town -- a large village >Village -- a small community or group of houses in a rural area, larger than a >hamlet and usually smaller than a town, and sometimes (as in parts of >the U.S.) incorporated as a municipality. >Hamlet -- British. a village without a church of its own, belonging >to the parish of another village or town. >County (the richest by far) -- >1.the largest administrative division of a U.S. state: Miami, >Florida, is in Dade County. > >2.one of the chief administrative divisions of a country or state, >as in Great Britain and Ireland. > >3.one of the larger divisions for purposes of local administration, >as in Canada and New Zealand. > >4.the territory of a county, esp. its rural areas, as in: "We farmed >out in the county before moving to town.". > >5.the inhabitants of a county, as in, "It was supposed to be a >secret, but you told the whole county.". > >6.the domain of a count or earl. > > >All these years I have been under the (clearly false) impression >that precise population-numbers defined these terms. Apparently I >have wrong, lo these decades. I have just taken a local poll (only 4 >people) and the agreement here is that a city is 100,000 people or >more; a town is 999,999 people or fewer; a village is 2,000 people >or fewer; a hamlet is 500 people or fewer. We four Canadians readily >agreed on these numbers, but that might be something we picked up in >school that has no relation to the larger world. > >Arthur > _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From rl_stewart at highstream.net Thu Nov 16 13:22:07 2006 From: rl_stewart at highstream.net (Robert L. Stewart) Date: Thu, 16 Nov 2006 13:22:07 -0600 Subject: [dba-SQLServer] dba-SQLServer Digest, Vol 45, Issue 19 In-Reply-To: References: Message-ID: <200611161922.kAGJMjq11670@databaseadvisors.com> Arthur, Just detach them, zip, and send them. I hate having to do a restore to a different machine. :-) All, I was also looking at working on the GUI for keeping it up to date and being able to download the information. Any takers on helping on it in ASP.net/VB.net 2005 or perhaps a Smart Client installation? Robert At 12:00 PM 11/16/2006, you wrote: >Date: Wed, 15 Nov 2006 17:58:34 -0800 (PST) >From: artful at rogers.com >Subject: Re: [dba-SQLServer] Useful Lists >To: dba-sqlserver at databaseadvisors.com >Message-ID: <20061116015834.16327.qmail at web88201.mail.re2.yahoo.com> >Content-Type: text/plain; charset=ascii > >What a cool offer! I will immediately convert the Access DBs that I >have so far to MS-SQL then zip the backups and send them to you for >restoration. > >Arthur From askolits at ot.com Wed Nov 22 10:54:03 2006 From: askolits at ot.com (askolits at ot.com) Date: Wed, 22 Nov 2006 11:54:03 -0500 Subject: [dba-SQLServer] ODBC error when Server is down with unused linked table Message-ID: <003001c70e56$d42613b0$3d00a8c0@LaptopXP> I ran into a problem where a database had an ODBC errors when an old server was shut down. This ODBC linked table was 'not' being used by the database. It was just left over from an old system. My customer shut down the server and the MDB timed out with an ODBC error even though the table is not used in anywhere in the database. At least I haven't found any use. I plan to do more testing but in the meantime, has anyone else seen anything like this before? John From artful at rogers.com Wed Nov 22 11:43:27 2006 From: artful at rogers.com (artful at rogers.com) Date: Wed, 22 Nov 2006 09:43:27 -0800 (PST) Subject: [dba-SQLServer] ODBC error when Server is down with unused linked table Message-ID: <20061122174327.21633.qmail@web88209.mail.re2.yahoo.com> A linked table will cause this, even if it is never used. How is the app to know that the table is never used? If you're using ODBC then you're linking to tables and therefore bad references will create this problem. The simplest solution, I think, is to fire up the Linked Table Manager and delete the references to obsolete tables. Arthur ----- Original Message ---- From: "askolits at ot.com" To: dba-SQLServer at databaseadvisors.com Sent: Wednesday, November 22, 2006 11:54:03 AM Subject: [dba-SQLServer] ODBC error when Server is down with unused linked table I ran into a problem where a database had an ODBC errors when an old server was shut down. This ODBC linked table was 'not' being used by the database. It was just left over from an old system. My customer shut down the server and the MDB timed out with an ODBC error even though the table is not used in anywhere in the database. At least I haven't found any use. From jwcolby at colbyconsulting.com Thu Nov 23 21:11:16 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Thu, 23 Nov 2006 22:11:16 -0500 Subject: [dba-SQLServer] The challenge - not a big one ;-) Message-ID: <008101c70f76$354a5550$647aa8c0@m6805> Given a table of 64 million records and growing, I need to do the following: Export sets of records out into csv files. The CSV files need to have specific field names, some of them unused which will be filled in by an NCOA (change of address) process outside of SQL Server. The sets need to be ~5 million records though it may be bigger with tuning. The table which this data comes from has an incrementing integer PK but about 15% of the records (PKs) have been "deleted". Thus the count can not depend on the PK. The table will grow, thus the number of exported csv files will grow. The CSV files need to be dropped into a specific "in" directory. The CSV files placed in the "NCOA IN" directory will be sensed by a program that performs NCOA processing on them, dropping files that have been NCOAd back into an "NCOA OUT" directory. The processing of each 5 million record CSV file by the NCOA program takes approximately 3 hours or more, depending on time of day. NCOA processing will take place weekly and must be automated so that it just happens. The presence of files in the NCOA "out" directory needs to trigger an import back into SQL Server and processing inside of SQL Server, i.e. affecting other tables. The process inside of SQL Server basically consists of sensing which records have been NCOAd, generate a "match code" from the leading N characters of the Address, city, zip5 and zip4. If that match code does not exist in a Master Address table, a new address record must be created in the master "address" table, and a FK updated in a m-m Person/Address table to point to the new address. IOW track addresses as a person moves around. I would like to do this from within SQL Server. I can and will if necessary do this from Access until I can come up to speed on VB.Net and do it from there. My question to you folks is whether this is possible completely inside of SQL Server? I suspect not but thought I would ask the masters. BTW, while I cannot say I am getting even so much as proficient with SQL Server, I do have a stable server up now, up to the task of doing this processing, with almost a terabyte of space for the main database and another 800g of disk for log files and the like, all running on a very fast raid6 drive. I am using Windows 2003 Server Standard edition and SQL Server 2000. Given what I am trying to accomplish, I am looking at moving towards SQL Server 2005 and VBA.Net 2005. Can anyone doing anything remotely like this comment on whether this should be split into parts, with the NCOA process running off on another server ( I have one available, though not as powerful) which would run the VB.Net stuff, do the NCOA processing, just manipulating the data over in the SQL Server. Also can SQL Server 2005 and SQL Server 2000 coexist on the same machine? If so is there anything I need to know before attempting to do the 2005 install on the SQL Server 2000 server? John W. Colby Colby Consulting www.ColbyConsulting.com From artful at rogers.com Thu Nov 23 21:45:37 2006 From: artful at rogers.com (artful at rogers.com) Date: Thu, 23 Nov 2006 19:45:37 -0800 (PST) Subject: [dba-SQLServer] The challenge - not a big one ;-) Message-ID: <20061124034537.51266.qmail@web88213.mail.re2.yahoo.com> Everything you want to do can be done from within SQL Server. Before installing 2005, either back up everything or choose another box on which to install 2005. CYA, I say, otay? Do NOT trust the OS + the MS stuff to save your bacon. Save your bacon first, then try the install, but make certain that you can undo it. This comes from experience. Been there, ruined that, spent days fixing it. Trust me on this, JC. CYA first and foremost. I've presented faulty arguments to you perhaps, but never lied to you. Trust me. Proceed with caution. Make sure that you can undo your last adventure. This is particularly true given the size of the tables with which you are dealing. Arthur ----- Original Message ---- From: JWColby To: dba-sqlserver at databaseadvisors.com Sent: Thursday, November 23, 2006 10:11:16 PM Subject: [dba-SQLServer] The challenge - not a big one ;-) Given a table of 64 million records and growing, I need to do the following: Export sets of records out into csv files. The CSV files need to have specific field names, some of them unused which will be filled in by an NCOA (change of address) process outside of SQL Server. The sets need to be ~5 million records though it may be bigger with tuning. The table which this data comes from has an incrementing integer PK but about 15% of the records (PKs) have been "deleted". Thus the count can not depend on the PK. The table will grow, thus the number of exported csv files will grow. The CSV files need to be dropped into a specific "in" directory. The CSV files placed in the "NCOA IN" directory will be sensed by a program that performs NCOA processing on them, dropping files that have been NCOAd back into an "NCOA OUT" directory. The processing of each 5 million record CSV file by the NCOA program takes approximately 3 hours or more, depending on time of day. NCOA processing will take place weekly and must be automated so that it just happens. The presence of files in the NCOA "out" directory needs to trigger an import back into SQL Server and processing inside of SQL Server, i.e. affecting other tables. The process inside of SQL Server basically consists of sensing which records have been NCOAd, generate a "match code" from the leading N characters of the Address, city, zip5 and zip4. If that match code does not exist in a Master Address table, a new address record must be created in the master "address" table, and a FK updated in a m-m Person/Address table to point to the new address. IOW track addresses as a person moves around. I would like to do this from within SQL Server. I can and will if necessary do this from Access until I can come up to speed on VB.Net and do it from there. My question to you folks is whether this is possible completely inside of SQL Server? I suspect not but thought I would ask the masters. BTW, while I cannot say I am getting even so much as proficient with SQL Server, I do have a stable server up now, up to the task of doing this processing, with almost a terabyte of space for the main database and another 800g of disk for log files and the like, all running on a very fast raid6 drive. I am using Windows 2003 Server Standard edition and SQL Server 2000. Given what I am trying to accomplish, I am looking at moving towards SQL Server 2005 and VBA.Net 2005. Can anyone doing anything remotely like this comment on whether this should be split into parts, with the NCOA process running off on another server ( I have one available, though not as powerful) which would run the VB.Net stuff, do the NCOA processing, just manipulating the data over in the SQL Server. Also can SQL Server 2005 and SQL Server 2000 coexist on the same machine? If so is there anything I need to know before attempting to do the 2005 install on the SQL Server 2000 server? John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Nov 24 08:59:00 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Fri, 24 Nov 2006 09:59:00 -0500 Subject: [dba-SQLServer] Query Analyzer hogs machine Message-ID: <009401c70fd9$13e40920$647aa8c0@m6805> Here's a fascinating one. In query Analyzer I built a simple query joining two tables with a PKID on the PKID, pulling one of the PKIDs out. It was taking awhile to run so I switched away to run a process over in Access. This Access process displays a progress meter and normally processes about 100K records per second. With the query running in Query Analyzer , Access was only processing about 10K records per second! The CPU Utilization was almost nil!!! When I aborted Query Analyzer, processor utilization shot up to 50% and my Access process went to full speed. Has anyone seen anything like this? I must be hitting some kind of limit somewhere but it isn't CPU Cycles. John W. Colby Colby Consulting www.ColbyConsulting.com From jwcolby at colbyconsulting.com Fri Nov 24 09:18:34 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Fri, 24 Nov 2006 10:18:34 -0500 Subject: [dba-SQLServer] Query Analyzer hogs machine In-Reply-To: <009401c70fd9$13e40920$647aa8c0@m6805> Message-ID: <009901c70fdb$cf3af330$647aa8c0@m6805> Well, the wall appears to be physical memory. I went into task manager to look at memory used by process and sql server is sitting at 1.7 gb. This is with Query analyzer and EM closed / unloaded. The only way to get it back is to stop sql server, as soon as I do that the memory is returned to Windows. To do this processing I created a view of the two tables joined on the PKID and pulling just the PKID out. When I try to open the view, the memory usage climbs until it hits 1.7gb, then a 10-15 seconds later I get an ODBC SQL Server Driver error - Timeout Expired. Pretty fascinating to me. I would have thought that it would continue to process, grabbing chunks of page file but that doesn't appear to be the case. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Friday, November 24, 2006 9:59 AM To: dba-sqlserver at databaseadvisors.com Subject: [dba-SQLServer] Query Analyzer hogs machine Here's a fascinating one. In query Analyzer I built a simple query joining two tables with a PKID on the PKID, pulling one of the PKIDs out. It was taking awhile to run so I switched away to run a process over in Access. This Access process displays a progress meter and normally processes about 100K records per second. With the query running in Query Analyzer , Access was only processing about 10K records per second! The CPU Utilization was almost nil!!! When I aborted Query Analyzer, processor utilization shot up to 50% and my Access process went to full speed. Has anyone seen anything like this? I must be hitting some kind of limit somewhere but it isn't CPU Cycles. John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From askolits at ot.com Sat Nov 25 07:32:27 2006 From: askolits at ot.com (John Skolits) Date: Sat, 25 Nov 2006 08:32:27 -0500 Subject: [dba-SQLServer] ODBC error when Server is down with unusedlinked table In-Reply-To: <20061122174327.21633.qmail@web88209.mail.re2.yahoo.com> Message-ID: <003701c71096$285079d0$6702a8c0@LaptopXP> Thanks for the info. Not exactly sure the best way top handle things but just using the link manager may be the best way to go. Thanks! -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Wednesday, November 22, 2006 12:43 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] ODBC error when Server is down with unusedlinked table A linked table will cause this, even if it is never used. How is the app to know that the table is never used? If you're using ODBC then you're linking to tables and therefore bad references will create this problem. The simplest solution, I think, is to fire up the Linked Table Manager and delete the references to obsolete tables. Arthur ----- Original Message ---- From: "askolits at ot.com" To: dba-SQLServer at databaseadvisors.com Sent: Wednesday, November 22, 2006 11:54:03 AM Subject: [dba-SQLServer] ODBC error when Server is down with unused linked table I ran into a problem where a database had an ODBC errors when an old server was shut down. This ODBC linked table was 'not' being used by the database. It was just left over from an old system. My customer shut down the server and the MDB timed out with an ODBC error even though the table is not used in anywhere in the database. At least I haven't found any use. _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From michael at ddisolutions.com.au Sat Nov 25 23:16:21 2006 From: michael at ddisolutions.com.au (Michael Maddison) Date: Sun, 26 Nov 2006 16:16:21 +1100 Subject: [dba-SQLServer] Query Analyzer hogs machine Message-ID: <59A61174B1F5B54B97FD4ADDE71E7D0116AD07@ddi-01.DDI.local> Hi John, SQL will as you have discovered use and hold as much memory as the OS will allow. This is by design. You can tinker with this but there are limitations with diff combos of SQL and OS. It's not actually QA that hogs memory, it uses bugger all IIRC, but SQL, another reason it really should be run on its own server, WITH NO OTHER APPS COMPETING. Rather then stopping the whole service when you have a runaway process try killing the process itself. In QA --This example shows how to terminate SPID 53. KILL 53 Or Kill the process in EM if it hasn't locked up. I've never seen QA lock up BTW. This will return the used memory as and when SQL needs it. SQL uses RAM to cache data++ so it can perform queries more efficiently. In your case with the size of your datasets you probably don't have enough RAM to make much difference... lol In a way SQL treats RAM similarly to how Access treats CPU, it will take 100% but give back when required. You don't say where you are running the query when you get the timeout? I'll assume its either in Access or EM? By default QA has no timeout set, use that always IMO. I'm a bit confused as to what you expect to happen when you run large queries concurrently. SQL has a lot of tools that can help to show where your bottlenecks are, unfortunately there is not likely to be a simple 1 answer fits all response as to how your server hardware + and SQL should be configured to work together. http://www.sql-server-performance.com/articles_performance.asp start reading ;-))) cheers Michael Maddison DDI Solutions Pty Ltd michael at ddisolutions.com.au Bus: 0260400620 Mob: 0412620497 www.ddisolutions.com.au -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Saturday, 25 November 2006 2:19 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Query Analyzer hogs machine Well, the wall appears to be physical memory. I went into task manager to look at memory used by process and sql server is sitting at 1.7 gb. This is with Query analyzer and EM closed / unloaded. The only way to get it back is to stop sql server, as soon as I do that the memory is returned to Windows. To do this processing I created a view of the two tables joined on the PKID and pulling just the PKID out. When I try to open the view, the memory usage climbs until it hits 1.7gb, then a 10-15 seconds later I get an ODBC SQL Server Driver error - Timeout Expired. Pretty fascinating to me. I would have thought that it would continue to process, grabbing chunks of page file but that doesn't appear to be the case. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Friday, November 24, 2006 9:59 AM To: dba-sqlserver at databaseadvisors.com Subject: [dba-SQLServer] Query Analyzer hogs machine Here's a fascinating one. In query Analyzer I built a simple query joining two tables with a PKID on the PKID, pulling one of the PKIDs out. It was taking awhile to run so I switched away to run a process over in Access. This Access process displays a progress meter and normally processes about 100K records per second. With the query running in Query Analyzer , Access was only processing about 10K records per second! The CPU Utilization was almost nil!!! When I aborted Query Analyzer, processor utilization shot up to 50% and my Access process went to full speed. Has anyone seen anything like this? I must be hitting some kind of limit somewhere but it isn't CPU Cycles. John W. Colby Colby Consulting www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From artful at rogers.com Sun Nov 26 04:35:59 2006 From: artful at rogers.com (artful at rogers.com) Date: Sun, 26 Nov 2006 02:35:59 -0800 (PST) Subject: [dba-SQLServer] Query Analyzer hogs machine Message-ID: <20061126103559.65638.qmail@web88208.mail.re2.yahoo.com> I most definitely agree with this: "another reason it really should be run on its own server, WITH NO OTHER APPS COMPETING." It's one thing to consume instruction-space on a development machine, but even then, boxes are so cheap now that one ought not run SQL (any version) on the same box that you deem your workstation. That is asinine. SQL was meant to occupy its own box, and be spoken to by other boxes. Yes, you can fake it by placing everything on one box, but that is to pervert its intended implantation. The rule is: SQL lives on one box; clients live on other boxes. Depending on your situation, that box might need to be firewalled so that nobody can get to it without the requisite credentials. In situations I have built, the SQL box was insulated from everyone save me and the CEO. Every other person in the firm could only get there by invoking the Access app that connected to it. No one else, no time, ever. And the only reason the CEO could get there was in case I got killed in a traffic accident, in which case he had specific instructions supplied by me long beforehand. Call this anal-retentive (or any other derogatory appellation you prefer) if you wish, but as I see it, my job is to protect the firm's data first and foremost -- from outside intruders, from disgruntled employees, from kids with too much time on their hands. I spend a LOT of time making sure that nobody can get to the machine, let alone the database, without credentials. And I go further: even assuming that you have credentials, I slot you into a role which is precisely delimited so you can do a, b and c, but not d, e or f. Arthur ----- Original Message ---- From: Michael Maddison To: dba-sqlserver at databaseadvisors.com Sent: Sunday, November 26, 2006 12:16:21 AM Subject: Re: [dba-SQLServer] Query Analyzer hogs machine Hi John, SQL will as you have discovered use and hold as much memory as the OS will allow. This is by design. You can tinker with this but there are limitations with diff combos of SQL and OS. From jwcolby at colbyconsulting.com Sun Nov 26 08:35:18 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Sun, 26 Nov 2006 09:35:18 -0500 Subject: [dba-SQLServer] Query Analyzer hogs machine In-Reply-To: <20061126103559.65638.qmail@web88208.mail.re2.yahoo.com> Message-ID: <001e01c71168$194742c0$657aa8c0@m6805> LOL. This is a server in my home office. Seriously though, if I must (and it appears that I must), then I must. Cheap it ain't though. I spent well over 3K of my own money for this machine. Unlike others perhaps, I do not have a big company handing me a budget for hardware. But no one is addr5essing why the thing gives an error and stops when I try to run a view and main memory runs out. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Sunday, November 26, 2006 5:36 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Query Analyzer hogs machine I most definitely agree with this: "another reason it really should be run on its own server, WITH NO OTHER APPS COMPETING." It's one thing to consume instruction-space on a development machine, but even then, boxes are so cheap now that one ought not run SQL (any version) on the same box that you deem your workstation. That is asinine. SQL was meant to occupy its own box, and be spoken to by other boxes. Yes, you can fake it by placing everything on one box, but that is to pervert its intended implantation. The rule is: SQL lives on one box; clients live on other boxes. Depending on your situation, that box might need to be firewalled so that nobody can get to it without the requisite credentials. In situations I have built, the SQL box was insulated from everyone save me and the CEO. Every other person in the firm could only get there by invoking the Access app that connected to it. No one else, no time, ever. And the only reason the CEO could get there was in case I got killed in a traffic accident, in which case he had specific instructions supplied by me long beforehand. Call this anal-retentive (or any other derogatory appellation you prefer) if you wish, but as I see it, my job is to protect the firm's data first and foremost -- from outside intruders, from disgruntled employees, from kids with too much time on their hands. I spend a LOT of time making sure that nobody can get to the machine, let alone the database, without credentials. And I go further: even assuming that you have credentials, I slot you into a role which is precisely delimited so you can do a, b and c, but not d, e or f. Arthur ----- Original Message ---- From: Michael Maddison To: dba-sqlserver at databaseadvisors.com Sent: Sunday, November 26, 2006 12:16:21 AM Subject: Re: [dba-SQLServer] Query Analyzer hogs machine Hi John, SQL will as you have discovered use and hold as much memory as the OS will allow. This is by design. You can tinker with this but there are limitations with diff combos of SQL and OS. _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From artful at rogers.com Sun Nov 26 09:34:26 2006 From: artful at rogers.com (artful at rogers.com) Date: Sun, 26 Nov 2006 07:34:26 -0800 (PST) Subject: [dba-SQLServer] Query Analyzer hogs machine Message-ID: <20061126153426.27178.qmail@web88215.mail.re2.yahoo.com> Send me a sample (or the entire) db and the query, and I will investigate what happens on my box. The only way (without evidence) that I can see this happening is that your indexes are out of whack and that you're forcing table-scans. Even then, it shouldn't consume so many resources. Arthur ----- Original Message ---- From: JWColby To: dba-sqlserver at databaseadvisors.com Sent: Sunday, November 26, 2006 9:35:18 AM Subject: Re: [dba-SQLServer] Query Analyzer hogs machine LOL. This is a server in my home office. Seriously though, if I must (and it appears that I must), then I must. Cheap it ain't though. I spent well over 3K of my own money for this machine. Unlike others perhaps, I do not have a big company handing me a budget for hardware. From jwcolby at colbyconsulting.com Sun Nov 26 12:22:09 2006 From: jwcolby at colbyconsulting.com (JWColby) Date: Sun, 26 Nov 2006 13:22:09 -0500 Subject: [dba-SQLServer] Query Analyzer hogs machine In-Reply-To: <20061126153426.27178.qmail@web88215.mail.re2.yahoo.com> Message-ID: <001f01c71187$ca110a40$657aa8c0@m6805> Arthur, There is only one index on each table, that being the PK itself. The PK is a long (4 byte). The view is a join of a table with ~65 million records, to another table with about 25 million records, PK to PK, pilling one PK out to display. As for sending you the entire thing, you must be joking. The first table consumes close to 100 gbytes inside of SQL Server. The raw text came to me on 21 4 gb DVDs, each of which contained a zipped text file which when unzipped was roughly 10 gbytes. The second table is being reconstructed from 13 CSV files, each of which is about 400 mbytes unzipped. The SQL Server database file is about 120 gbytes at the moment. I have never tried to zip it but I can tell you it wouldn't fit on a dvd, nor will it upload to my pitiful 3 gbytes of FTP space. John W. Colby Colby Consulting www.ColbyConsulting.com -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of artful at rogers.com Sent: Sunday, November 26, 2006 10:34 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] Query Analyzer hogs machine Send me a sample (or the entire) db and the query, and I will investigate what happens on my box. The only way (without evidence) that I can see this happening is that your indexes are out of whack and that you're forcing table-scans. Even then, it shouldn't consume so many resources. Arthur ----- Original Message ---- From: JWColby To: dba-sqlserver at databaseadvisors.com Sent: Sunday, November 26, 2006 9:35:18 AM Subject: Re: [dba-SQLServer] Query Analyzer hogs machine LOL. This is a server in my home office. Seriously though, if I must (and it appears that I must), then I must. Cheap it ain't though. I spent well over 3K of my own money for this machine. Unlike others perhaps, I do not have a big company handing me a budget for hardware. _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From michael at ddisolutions.com.au Sun Nov 26 16:34:39 2006 From: michael at ddisolutions.com.au (Michael Maddison) Date: Mon, 27 Nov 2006 09:34:39 +1100 Subject: [dba-SQLServer] Query Analyzer hogs machine Message-ID: <59A61174B1F5B54B97FD4ADDE71E7D0116AD08@ddi-01.DDI.local> John, "But no one is addr5essing why the thing gives an error and stops when I try to run a view and main memory runs out." It's not an error from SQL, at least not on the surface, its whatever client you are using that is timing out. AFAIK SQL will attempt to run a query forever... unless limits are placed on it. The limits can come from clients OR from SQL. Check the database properties and see if you set the timeout at some stage. IIRC 0 means run forever. Grab the execution plan from QA for the query and post it, it may tell us something. cheers Michael Maddison DDI Solutions Pty Ltd michael at ddisolutions.com.au Bus: 0260400620 Mob: 0412620497 www.ddisolutions.com.au From markamatte at hotmail.com Thu Nov 30 13:13:44 2006 From: markamatte at hotmail.com (Mark A Matte) Date: Thu, 30 Nov 2006 19:13:44 +0000 Subject: [dba-SQLServer] SQL in SP failing In-Reply-To: <59A61174B1F5B54B97FD4ADDE71E7D0116AD08@ddi-01.DDI.local> Message-ID: Hello All, Any idea why the following SQL will fail in an SP? 'insert into tblBrita_ID(case_id,Criteria) SELECT case_id, "test" As Criteria FROM tblCase where case_id=1' The error is "Invalid column name 'test'." It doesn't always fail...normally 'test' is a parameter passed to the SP...I have modified it to the above for demo purposes...but using just the word 'test' as above still gives the same error. Thanks, Mark A. Matte _________________________________________________________________ Get the latest Windows Live Messenger 8.1 Beta version.?Join now. http://ideas.live.com From DElam at jenkens.com Thu Nov 30 13:23:00 2006 From: DElam at jenkens.com (Elam, Debbie) Date: Thu, 30 Nov 2006 13:23:00 -0600 Subject: [dba-SQLServer] SQL in SP failing Message-ID: <7B1961ED924D1A459E378C9B1BB22B4C068DE9EE@natexch.jenkens.com> Use single quotes around test. Double quotes will cause problems. Debbie -----Original Message----- From: Mark A Matte [mailto:markamatte at hotmail.com] Sent: Thursday, November 30, 2006 1:14 PM To: dba-sqlserver at databaseadvisors.com Subject: [dba-SQLServer] SQL in SP failing Hello All, Any idea why the following SQL will fail in an SP? 'insert into tblBrita_ID(case_id,Criteria) SELECT case_id, "test" As Criteria FROM tblCase where case_id=1' The error is "Invalid column name 'test'." It doesn't always fail...normally 'test' is a parameter passed to the SP...I have modified it to the above for demo purposes...but using just the word 'test' as above still gives the same error. Thanks, Mark A. Matte _________________________________________________________________ Get the latest Windows Live Messenger 8.1 Beta version.?Join now. http://ideas.live.com - JENKENS & GILCHRIST E-MAIL NOTICE - This transmission may be: (1) subject to the Attorney-Client Privilege, (2) an attorney work product, or (3) strictly confidential. If you are not the intended recipient of this message, you may not disclose, print, copy or disseminate this information. If you have received this in error, please reply and notify the sender (only) and delete the message. Unauthorized interception of this e-mail is a violation of federal criminal law. This communication does not reflect an intention by the sender or the sender's client or principal to conduct a transaction or make any agreement by electronic means. Nothing contained in this message or in any attachment shall satisfy the requirements for a writing, and nothing contained herein shall constitute a contract or electronic signature under the Electronic Signatures in Global and National Commerce Act, any version of the Uniform Electronic Transactions Act or any other statute governing electronic transactions. From markamatte at hotmail.com Thu Nov 30 13:26:01 2006 From: markamatte at hotmail.com (Mark A Matte) Date: Thu, 30 Nov 2006 19:26:01 +0000 Subject: [dba-SQLServer] SQL in SP failing In-Reply-To: <7B1961ED924D1A459E378C9B1BB22B4C068DE9EE@natexch.jenkens.com> Message-ID: If I use single quotes...SQL says it is a syntax error before I even run it...won't even let me save it that way. Thanks, Mark A. Matte >From: "Elam, Debbie" >Reply-To: dba-sqlserver at databaseadvisors.com >To: "'dba-sqlserver at databaseadvisors.com'" > >Subject: Re: [dba-SQLServer] SQL in SP failing >Date: Thu, 30 Nov 2006 13:23:00 -0600 > >Use single quotes around test. Double quotes will cause problems. > >Debbie > >-----Original Message----- >From: Mark A Matte [mailto:markamatte at hotmail.com] >Sent: Thursday, November 30, 2006 1:14 PM >To: dba-sqlserver at databaseadvisors.com >Subject: [dba-SQLServer] SQL in SP failing > > >Hello All, > >Any idea why the following SQL will fail in an SP? > >'insert into tblBrita_ID(case_id,Criteria) SELECT case_id, "test" As >Criteria FROM tblCase where case_id=1' > >The error is "Invalid column name 'test'." It doesn't always >fail...normally 'test' is a parameter passed to the SP...I have modified it >to the above for demo purposes...but using just the word 'test' as above >still gives the same error. > >Thanks, > >Mark A. Matte > >_________________________________________________________________ >Get the latest Windows Live Messenger 8.1 Beta version.?Join now. >http://ideas.live.com > >- JENKENS & GILCHRIST E-MAIL NOTICE - This transmission may be: (1) subject >to the Attorney-Client Privilege, (2) an attorney work product, or (3) >strictly confidential. If you are not the intended recipient of this >message, you may not disclose, print, copy or disseminate this information. >If you have received this in error, please reply and notify the sender >(only) and delete the message. Unauthorized interception of this e-mail is >a >violation of federal criminal law. >This communication does not reflect an intention by the sender or the >sender's client or principal to conduct a transaction or make any agreement >by electronic means. Nothing contained in this message or in any >attachment >shall satisfy the requirements for a writing, and nothing contained herein >shall constitute a contract or electronic signature under the Electronic >Signatures in Global and National Commerce Act, any version of the Uniform >Electronic Transactions Act or any other statute governing electronic >transactions. >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > _________________________________________________________________ Stay up-to-date with your friends through the Windows Live Spaces friends list. http://clk.atdmt.com/MSN/go/msnnkwsp0070000001msn/direct/01/?href=http://spaces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mk From DElam at jenkens.com Thu Nov 30 13:34:58 2006 From: DElam at jenkens.com (Elam, Debbie) Date: Thu, 30 Nov 2006 13:34:58 -0600 Subject: [dba-SQLServer] SQL in SP failing Message-ID: <7B1961ED924D1A459E378C9B1BB22B4C068DE9F0@natexch.jenkens.com> Hmm, I can get just the select statement to work correctly with single quotes. Does the select work OK? When I try to just run the select statement with double quotes, I get that test is an invalid column, so that sounds like the problem. Debbie -----Original Message----- From: Mark A Matte [mailto:markamatte at hotmail.com] Sent: Thursday, November 30, 2006 1:26 PM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] SQL in SP failing If I use single quotes...SQL says it is a syntax error before I even run it...won't even let me save it that way. Thanks, Mark A. Matte >From: "Elam, Debbie" >Reply-To: dba-sqlserver at databaseadvisors.com >To: "'dba-sqlserver at databaseadvisors.com'" > >Subject: Re: [dba-SQLServer] SQL in SP failing >Date: Thu, 30 Nov 2006 13:23:00 -0600 > >Use single quotes around test. Double quotes will cause problems. > >Debbie > >-----Original Message----- >From: Mark A Matte [mailto:markamatte at hotmail.com] >Sent: Thursday, November 30, 2006 1:14 PM >To: dba-sqlserver at databaseadvisors.com >Subject: [dba-SQLServer] SQL in SP failing > > >Hello All, > >Any idea why the following SQL will fail in an SP? > >'insert into tblBrita_ID(case_id,Criteria) SELECT case_id, "test" As >Criteria FROM tblCase where case_id=1' > >The error is "Invalid column name 'test'." It doesn't always >fail...normally 'test' is a parameter passed to the SP...I have modified it >to the above for demo purposes...but using just the word 'test' as above >still gives the same error. > >Thanks, > >Mark A. Matte > >_________________________________________________________________ >Get the latest Windows Live Messenger 8.1 Beta version.?Join now. >http://ideas.live.com > >- JENKENS & GILCHRIST E-MAIL NOTICE - This transmission may be: (1) subject >to the Attorney-Client Privilege, (2) an attorney work product, or (3) >strictly confidential. If you are not the intended recipient of this >message, you may not disclose, print, copy or disseminate this information. >If you have received this in error, please reply and notify the sender >(only) and delete the message. Unauthorized interception of this e-mail is >a >violation of federal criminal law. >This communication does not reflect an intention by the sender or the >sender's client or principal to conduct a transaction or make any agreement >by electronic means. Nothing contained in this message or in any >attachment >shall satisfy the requirements for a writing, and nothing contained herein >shall constitute a contract or electronic signature under the Electronic >Signatures in Global and National Commerce Act, any version of the Uniform >Electronic Transactions Act or any other statute governing electronic >transactions. >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > _________________________________________________________________ Stay up-to-date with your friends through the Windows Live Spaces friends list. http://clk.atdmt.com/MSN/go/msnnkwsp0070000001msn/direct/01/?href=http://spa ces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mk - JENKENS & GILCHRIST E-MAIL NOTICE - This transmission may be: (1) subject to the Attorney-Client Privilege, (2) an attorney work product, or (3) strictly confidential. If you are not the intended recipient of this message, you may not disclose, print, copy or disseminate this information. If you have received this in error, please reply and notify the sender (only) and delete the message. Unauthorized interception of this e-mail is a violation of federal criminal law. This communication does not reflect an intention by the sender or the sender's client or principal to conduct a transaction or make any agreement by electronic means. Nothing contained in this message or in any attachment shall satisfy the requirements for a writing, and nothing contained herein shall constitute a contract or electronic signature under the Electronic Signatures in Global and National Commerce Act, any version of the Uniform Electronic Transactions Act or any other statute governing electronic transactions. From ebarro at verizon.net Thu Nov 30 13:40:25 2006 From: ebarro at verizon.net (Eric Barro) Date: Thu, 30 Nov 2006 11:40:25 -0800 Subject: [dba-SQLServer] SQL in SP failing In-Reply-To: Message-ID: <0J9K003NE6NCKE8S@vms048.mailsrvcs.net> Enclose test in brackets like this [test] -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Mark A Matte Sent: Thursday, November 30, 2006 11:26 AM To: dba-sqlserver at databaseadvisors.com Subject: Re: [dba-SQLServer] SQL in SP failing If I use single quotes...SQL says it is a syntax error before I even run it...won't even let me save it that way. Thanks, Mark A. Matte >From: "Elam, Debbie" >Reply-To: dba-sqlserver at databaseadvisors.com >To: "'dba-sqlserver at databaseadvisors.com'" > >Subject: Re: [dba-SQLServer] SQL in SP failing >Date: Thu, 30 Nov 2006 13:23:00 -0600 > >Use single quotes around test. Double quotes will cause problems. > >Debbie > >-----Original Message----- >From: Mark A Matte [mailto:markamatte at hotmail.com] >Sent: Thursday, November 30, 2006 1:14 PM >To: dba-sqlserver at databaseadvisors.com >Subject: [dba-SQLServer] SQL in SP failing > > >Hello All, > >Any idea why the following SQL will fail in an SP? > >'insert into tblBrita_ID(case_id,Criteria) SELECT case_id, "test" As >Criteria FROM tblCase where case_id=1' > >The error is "Invalid column name 'test'." It doesn't always >fail...normally 'test' is a parameter passed to the SP...I have >modified it to the above for demo purposes...but using just the word >'test' as above still gives the same error. > >Thanks, > >Mark A. Matte > >_________________________________________________________________ >Get the latest Windows Live Messenger 8.1 Beta version. Join now. >http://ideas.live.com > >- JENKENS & GILCHRIST E-MAIL NOTICE - This transmission may be: (1) >subject to the Attorney-Client Privilege, (2) an attorney work product, >or (3) strictly confidential. If you are not the intended recipient of >this message, you may not disclose, print, copy or disseminate this information. >If you have received this in error, please reply and notify the sender >(only) and delete the message. Unauthorized interception of this e-mail >is a violation of federal criminal law. >This communication does not reflect an intention by the sender or the >sender's client or principal to conduct a transaction or make any >agreement by electronic means. Nothing contained in this message or in >any attachment shall satisfy the requirements for a writing, and >nothing contained herein shall constitute a contract or electronic >signature under the Electronic Signatures in Global and National >Commerce Act, any version of the Uniform Electronic Transactions Act or >any other statute governing electronic transactions. >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > _________________________________________________________________ Stay up-to-date with your friends through the Windows Live Spaces friends list. http://clk.atdmt.com/MSN/go/msnnkwsp0070000001msn/direct/01/?href=http://spa ces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mk -- No virus found in this incoming message. Checked by AVG Free Edition. Version: 7.1.409 / Virus Database: 268.15.2/559 - Release Date: 11/30/2006 From markamatte at hotmail.com Thu Nov 30 13:45:39 2006 From: markamatte at hotmail.com (Mark A Matte) Date: Thu, 30 Nov 2006 19:45:39 +0000 Subject: [dba-SQLServer] SQL in SP failing In-Reply-To: <7B1961ED924D1A459E378C9B1BB22B4C068DE9F0@natexch.jenkens.com> Message-ID: The real mystery is that it ran the first few times after I created it. When it started failing I would change things, save, run, fail,change things back,save, and it would run correctly. Like just now...I got it to run...here is the actual sql: set @sql='insert into tblBrita_ID(case_id,Criteria) SELECT case_id, "'+ at ID+'" As Criteria FROM tblCase where creation_date > "1/1/2006" and' + @ID And this was what I started with that would NOT run...and what I finished with that did run(SAME CODE). To clarify...The @ID is actually criteria used to isolate records. I need to know what criteria picked a record...that is why I save it in the new table. I got it to run...but am very confused as to why it does...sometimes? Any and all feedback is appreciated. Thanks, Mark A. Matte >From: "Elam, Debbie" >Reply-To: dba-sqlserver at databaseadvisors.com >To: "'dba-sqlserver at databaseadvisors.com'" > >Subject: Re: [dba-SQLServer] SQL in SP failing >Date: Thu, 30 Nov 2006 13:34:58 -0600 > >Hmm, I can get just the select statement to work correctly with single >quotes. Does the select work OK? When I try to just run the select >statement with double quotes, I get that test is an invalid column, so that >sounds like the problem. > >Debbie > >-----Original Message----- >From: Mark A Matte [mailto:markamatte at hotmail.com] >Sent: Thursday, November 30, 2006 1:26 PM >To: dba-sqlserver at databaseadvisors.com >Subject: Re: [dba-SQLServer] SQL in SP failing > > >If I use single quotes...SQL says it is a syntax error before I even run >it...won't even let me save it that way. > >Thanks, > >Mark A. Matte > > > >From: "Elam, Debbie" > >Reply-To: dba-sqlserver at databaseadvisors.com > >To: "'dba-sqlserver at databaseadvisors.com'" > > > >Subject: Re: [dba-SQLServer] SQL in SP failing > >Date: Thu, 30 Nov 2006 13:23:00 -0600 > > > >Use single quotes around test. Double quotes will cause problems. > > > >Debbie > > > >-----Original Message----- > >From: Mark A Matte [mailto:markamatte at hotmail.com] > >Sent: Thursday, November 30, 2006 1:14 PM > >To: dba-sqlserver at databaseadvisors.com > >Subject: [dba-SQLServer] SQL in SP failing > > > > > >Hello All, > > > >Any idea why the following SQL will fail in an SP? > > > >'insert into tblBrita_ID(case_id,Criteria) SELECT case_id, "test" As > >Criteria FROM tblCase where case_id=1' > > > >The error is "Invalid column name 'test'." It doesn't always > >fail...normally 'test' is a parameter passed to the SP...I have modified >it > >to the above for demo purposes...but using just the word 'test' as above > >still gives the same error. > > > >Thanks, > > > >Mark A. Matte > > > >_________________________________________________________________ > >Get the latest Windows Live Messenger 8.1 Beta version.?Join now. > >http://ideas.live.com > > > >- JENKENS & GILCHRIST E-MAIL NOTICE - This transmission may be: (1) >subject > >to the Attorney-Client Privilege, (2) an attorney work product, or (3) > >strictly confidential. If you are not the intended recipient of this > >message, you may not disclose, print, copy or disseminate this >information. > >If you have received this in error, please reply and notify the sender > >(only) and delete the message. Unauthorized interception of this e-mail >is > >a > >violation of federal criminal law. > >This communication does not reflect an intention by the sender or the > >sender's client or principal to conduct a transaction or make any >agreement > >by electronic means. Nothing contained in this message or in any > >attachment > >shall satisfy the requirements for a writing, and nothing contained >herein > >shall constitute a contract or electronic signature under the Electronic > >Signatures in Global and National Commerce Act, any version of the >Uniform > >Electronic Transactions Act or any other statute governing electronic > >transactions. > >_______________________________________________ > >dba-SQLServer mailing list > >dba-SQLServer at databaseadvisors.com > >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > >http://www.databaseadvisors.com > > > >_________________________________________________________________ >Stay up-to-date with your friends through the Windows Live Spaces friends >list. >http://clk.atdmt.com/MSN/go/msnnkwsp0070000001msn/direct/01/?href=http://spa >ces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mk > >- JENKENS & GILCHRIST E-MAIL NOTICE - This transmission may be: (1) subject >to the Attorney-Client Privilege, (2) an attorney work product, or (3) >strictly confidential. If you are not the intended recipient of this >message, you may not disclose, print, copy or disseminate this information. >If you have received this in error, please reply and notify the sender >(only) and delete the message. Unauthorized interception of this e-mail is >a >violation of federal criminal law. >This communication does not reflect an intention by the sender or the >sender's client or principal to conduct a transaction or make any agreement >by electronic means. Nothing contained in this message or in any >attachment >shall satisfy the requirements for a writing, and nothing contained herein >shall constitute a contract or electronic signature under the Electronic >Signatures in Global and National Commerce Act, any version of the Uniform >Electronic Transactions Act or any other statute governing electronic >transactions. >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > _________________________________________________________________ Stay up-to-date with your friends through the Windows Live Spaces friends list. http://clk.atdmt.com/MSN/go/msnnkwsp0070000001msn/direct/01/?href=http://spaces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mk From markamatte at hotmail.com Thu Nov 30 14:07:42 2006 From: markamatte at hotmail.com (Mark A Matte) Date: Thu, 30 Nov 2006 20:07:42 +0000 Subject: [dba-SQLServer] SQL in SP failing In-Reply-To: <0J9K003NE6NCKE8S@vms048.mailsrvcs.net> Message-ID: Wouldn't that make it think TEST was a column name...when I want it to be the value in each row? Thanks, Mark A. Matte >From: "Eric Barro" >Reply-To: dba-sqlserver at databaseadvisors.com >To: >Subject: Re: [dba-SQLServer] SQL in SP failing >Date: Thu, 30 Nov 2006 11:40:25 -0800 > >Enclose test in brackets like this > >[test] > >-----Original Message----- >From: dba-sqlserver-bounces at databaseadvisors.com >[mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Mark A >Matte >Sent: Thursday, November 30, 2006 11:26 AM >To: dba-sqlserver at databaseadvisors.com >Subject: Re: [dba-SQLServer] SQL in SP failing > >If I use single quotes...SQL says it is a syntax error before I even run >it...won't even let me save it that way. > >Thanks, > >Mark A. Matte > > > >From: "Elam, Debbie" > >Reply-To: dba-sqlserver at databaseadvisors.com > >To: "'dba-sqlserver at databaseadvisors.com'" > > > >Subject: Re: [dba-SQLServer] SQL in SP failing > >Date: Thu, 30 Nov 2006 13:23:00 -0600 > > > >Use single quotes around test. Double quotes will cause problems. > > > >Debbie > > > >-----Original Message----- > >From: Mark A Matte [mailto:markamatte at hotmail.com] > >Sent: Thursday, November 30, 2006 1:14 PM > >To: dba-sqlserver at databaseadvisors.com > >Subject: [dba-SQLServer] SQL in SP failing > > > > > >Hello All, > > > >Any idea why the following SQL will fail in an SP? > > > >'insert into tblBrita_ID(case_id,Criteria) SELECT case_id, "test" As > >Criteria FROM tblCase where case_id=1' > > > >The error is "Invalid column name 'test'." It doesn't always > >fail...normally 'test' is a parameter passed to the SP...I have > >modified it to the above for demo purposes...but using just the word > >'test' as above still gives the same error. > > > >Thanks, > > > >Mark A. Matte > > > >_________________________________________________________________ > >Get the latest Windows Live Messenger 8.1 Beta version. Join now. > >http://ideas.live.com > > > >- JENKENS & GILCHRIST E-MAIL NOTICE - This transmission may be: (1) > >subject to the Attorney-Client Privilege, (2) an attorney work product, > >or (3) strictly confidential. If you are not the intended recipient of > >this message, you may not disclose, print, copy or disseminate this >information. > >If you have received this in error, please reply and notify the sender > >(only) and delete the message. Unauthorized interception of this e-mail > >is a violation of federal criminal law. > >This communication does not reflect an intention by the sender or the > >sender's client or principal to conduct a transaction or make any > >agreement by electronic means. Nothing contained in this message or in > >any attachment shall satisfy the requirements for a writing, and > >nothing contained herein shall constitute a contract or electronic > >signature under the Electronic Signatures in Global and National > >Commerce Act, any version of the Uniform Electronic Transactions Act or > >any other statute governing electronic transactions. > >_______________________________________________ > >dba-SQLServer mailing list > >dba-SQLServer at databaseadvisors.com > >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > >http://www.databaseadvisors.com > > > >_________________________________________________________________ >Stay up-to-date with your friends through the Windows Live Spaces friends >list. >http://clk.atdmt.com/MSN/go/msnnkwsp0070000001msn/direct/01/?href=http://spa >ces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mk > > >-- >No virus found in this incoming message. >Checked by AVG Free Edition. >Version: 7.1.409 / Virus Database: 268.15.2/559 - Release Date: 11/30/2006 > > > >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > _________________________________________________________________ Fixing up the home? Live Search can help http://imagine-windowslive.com/search/kits/default.aspx?kit=improve&locale=en-US&source=hmemailtaglinenov06&FORM=WLMTAG From rl_stewart at highstream.net Thu Nov 30 14:20:41 2006 From: rl_stewart at highstream.net (Robert L. Stewart) Date: Thu, 30 Nov 2006 14:20:41 -0600 Subject: [dba-SQLServer] SQL in SP failing In-Reply-To: References: Message-ID: <200611302022.kAUKMlq11946@databaseadvisors.com> Take the AS Criteria out. SQL does not care about it and it is probably confusing it. At 02:07 PM 11/30/2006, you wrote: >Date: Thu, 30 Nov 2006 20:07:42 +0000 >From: "Mark A Matte" >Subject: Re: [dba-SQLServer] SQL in SP failing >To: dba-sqlserver at databaseadvisors.com >Message-ID: >Content-Type: text/plain; format=flowed > >Wouldn't that make it think TEST was a column name...when I want it to be >the value in each row? > >Thanks, > >Mark A. Matte > > > >From: "Eric Barro" > >Reply-To: dba-sqlserver at databaseadvisors.com > >To: > >Subject: Re: [dba-SQLServer] SQL in SP failing > >Date: Thu, 30 Nov 2006 11:40:25 -0800 > > > >Enclose test in brackets like this > > > >[test] > > > >-----Original Message----- > >From: dba-sqlserver-bounces at databaseadvisors.com > >[mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Mark A > >Matte > >Sent: Thursday, November 30, 2006 11:26 AM > >To: dba-sqlserver at databaseadvisors.com > >Subject: Re: [dba-SQLServer] SQL in SP failing > > > >If I use single quotes...SQL says it is a syntax error before I even run > >it...won't even let me save it that way. > > > >Thanks, > > > >Mark A. Matte > > > > From markamatte at hotmail.com Thu Nov 30 14:31:50 2006 From: markamatte at hotmail.com (Mark A Matte) Date: Thu, 30 Nov 2006 20:31:50 +0000 Subject: [dba-SQLServer] SQL in SP failing In-Reply-To: <200611302022.kAUKMlq11946@databaseadvisors.com> Message-ID: Robert, I think that did it...thank you very much...I'm still not sure why it would fail...but after modifying and changing back it would run succesfully with the same syntax that originally failed. Thanks Again, Mark A. Matte >From: "Robert L. Stewart" >Reply-To: dba-sqlserver at databaseadvisors.com >To: dba-sqlserver at databaseadvisors.com >Subject: Re: [dba-SQLServer] SQL in SP failing >Date: Thu, 30 Nov 2006 14:20:41 -0600 > >Take the AS Criteria out. >SQL does not care about it and it is probably confusing it. > >At 02:07 PM 11/30/2006, you wrote: > >Date: Thu, 30 Nov 2006 20:07:42 +0000 > >From: "Mark A Matte" > >Subject: Re: [dba-SQLServer] SQL in SP failing > >To: dba-sqlserver at databaseadvisors.com > >Message-ID: > >Content-Type: text/plain; format=flowed > > > >Wouldn't that make it think TEST was a column name...when I want it to be > >the value in each row? > > > >Thanks, > > > >Mark A. Matte > > > > > > >From: "Eric Barro" > > >Reply-To: dba-sqlserver at databaseadvisors.com > > >To: > > >Subject: Re: [dba-SQLServer] SQL in SP failing > > >Date: Thu, 30 Nov 2006 11:40:25 -0800 > > > > > >Enclose test in brackets like this > > > > > >[test] > > > > > >-----Original Message----- > > >From: dba-sqlserver-bounces at databaseadvisors.com > > >[mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of Mark A > > >Matte > > >Sent: Thursday, November 30, 2006 11:26 AM > > >To: dba-sqlserver at databaseadvisors.com > > >Subject: Re: [dba-SQLServer] SQL in SP failing > > > > > >If I use single quotes...SQL says it is a syntax error before I even >run > > >it...won't even let me save it that way. > > > > > >Thanks, > > > > > >Mark A. Matte > > > > > > > > >_______________________________________________ >dba-SQLServer mailing list >dba-SQLServer at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >http://www.databaseadvisors.com > _________________________________________________________________ Get free, personalized commercial-free online radio with MSN Radio powered by Pandora http://radio.msn.com/?icid=T002MSN03A07001 From DavidL at sierranevada.com Thu Nov 30 14:44:44 2006 From: DavidL at sierranevada.com (David Lewis) Date: Thu, 30 Nov 2006 12:44:44 -0800 Subject: [dba-SQLServer] dba-SQLServer Digest, Vol 45, Issue 26 In-Reply-To: Message-ID: <00101736F13D774F88C54058CB2663C8E02F0F@celebration.sierranevada.corp> I had something very similar, if not identical, happen to me a few weeks ago. In the end I ended QA and shut down the computer, then restarted everything and it then worked. I could come up with no reasonable explanation. I haven't come across anything like that ever before. Go Figure. D. Lewis Message: 6 Date: Thu, 30 Nov 2006 19:45:39 +0000 From: "Mark A Matte" Subject: Re: [dba-SQLServer] SQL in SP failing To: dba-sqlserver at databaseadvisors.com Message-ID: Content-Type: text/plain; format=flowed The real mystery is that it ran the first few times after I created it. When it started failing I would change things, save, run, fail,change things back,save, and it would run correctly. Like just now...I got it to run...here is the actual sql: set @sql='insert into tblBrita_ID(case_id,Criteria) SELECT case_id, "'+ at ID+'" As Criteria FROM tblCase where creation_date > "1/1/2006" and' + @ID And this was what I started with that would NOT run...and what I finished with that did run(SAME CODE). To clarify...The @ID is actually criteria used to isolate records. I need to know what criteria picked a record...that is why I save it in the new table. I got it to run...but am very confused as to why it does...sometimes? Any and all feedback is appreciated. Thanks, Mark A. Matte