From jwcolby at colbyconsulting.com Tue Jan 5 09:57:35 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Tue, 05 Jan 2010 10:57:35 -0500 Subject: [dba-SQLServer] Microsoft partner Message-ID: <4B43616F.5050509@colbyconsulting.com> I am getting calls from Microsoft re my partnership. The caller today gave me a free certificate for a training course on server virtualization. It is for the Server 2008 Hyper V virtualization stuff. I just clicked through to see what it is and it looks interesting. This is the only certificate they are giving away today, not sure WHY they are giving it to me, other than perhaps to convince me how valuable my membership is. My membership is expiring this month. If you have a membership and are interested in such stuff, you might want to call and ask if you are eligible for this. BTW, if anyone receiving this is interested in forming a virtual organization for the purpose of obtaining the credentials for partner companies, please speak up. As a "sole proprietor" it is difficult for me to get enough of these credentials to qualify. However I am told that we as a group could form a virtual organization and do this. Each member would need to get one (or more) credentials, then we would need to band together and present as an organization. It might be worth doing. -- John W. Colby www.ColbyConsulting.com From garykjos at gmail.com Tue Jan 5 10:26:52 2010 From: garykjos at gmail.com (Gary Kjos) Date: Tue, 5 Jan 2010 10:26:52 -0600 Subject: [dba-SQLServer] Microsoft partner In-Reply-To: <4B43616F.5050509@colbyconsulting.com> References: <4B43616F.5050509@colbyconsulting.com> Message-ID: I'm a Microsoft partner on my own too. Had to come up with a company name the last time. Gary Kjos Consulting Services. I guess I would be interested in at least looking at the idea of combining with some DBA people. GK On Tue, Jan 5, 2010 at 9:57 AM, jwcolby wrote: > I am getting calls from Microsoft re my partnership. ?The caller today gave me a free certificate > for a training course on server virtualization. ?It is for the Server 2008 Hyper V virtualization > stuff. ?I just clicked through to see what it is and it looks interesting. ?This is the only > certificate they are giving away today, not sure WHY they are giving it to me, other than perhaps to > convince me how valuable my membership is. ?My membership is expiring this month. > > If you have a membership and are interested in such stuff, you might want to call and ask if you are > eligible for this. > > BTW, if anyone receiving this is interested in forming a virtual organization for the purpose of > obtaining the credentials for partner companies, please speak up. ?As a "sole proprietor" it is > difficult for me to get enough of these credentials to qualify. ?However I am told that we as a > group could form a virtual organization and do this. ?Each member would need to get one (or more) > credentials, then we would need to band together and present as an organization. > > It might be worth doing. > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > -- Gary Kjos garykjos at gmail.com From accessd at shaw.ca Tue Jan 5 12:45:15 2010 From: accessd at shaw.ca (Jim Lawrence) Date: Tue, 5 Jan 2010 10:45:15 -0800 Subject: [dba-SQLServer] Microsoft partner In-Reply-To: <4B43616F.5050509@colbyconsulting.com> References: <4B43616F.5050509@colbyconsulting.com> Message-ID: <488B4D1C7CC64FB680657DFB38F31A85@creativesystemdesigns.com> Hi John: On a different subject and related to a problem you noted a while back; an article has appeared, on the process of handle a million(s) or so records in a timely manner using the Row_Number() function...very fast processing according to the article. If you have not already discovered this feature it might be of some help: http://www.sqlservercentral.com/articles/Large+Data+Sets/68930 A year ago I would have been interested in this but I have now found myself with a full roster of steady clients and at this time just can not take on anymore. Jim -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Tuesday, January 05, 2010 7:58 AM To: Access Developers discussion and problem solving; VBA; Dba-Sqlserver Subject: [dba-SQLServer] Microsoft partner I am getting calls from Microsoft re my partnership. The caller today gave me a free certificate for a training course on server virtualization. It is for the Server 2008 Hyper V virtualization stuff. I just clicked through to see what it is and it looks interesting. This is the only certificate they are giving away today, not sure WHY they are giving it to me, other than perhaps to convince me how valuable my membership is. My membership is expiring this month. If you have a membership and are interested in such stuff, you might want to call and ask if you are eligible for this. BTW, if anyone receiving this is interested in forming a virtual organization for the purpose of obtaining the credentials for partner companies, please speak up. As a "sole proprietor" it is difficult for me to get enough of these credentials to qualify. However I am told that we as a group could form a virtual organization and do this. Each member would need to get one (or more) credentials, then we would need to band together and present as an organization. It might be worth doing. -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Wed Jan 6 07:28:11 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 06 Jan 2010 08:28:11 -0500 Subject: [dba-SQLServer] [AccessD] Microsoft partner In-Reply-To: <034a01ca8e5a$131eb1e0$0701a8c0@MSIMMSWS> References: <4B43616F.5050509@colbyconsulting.com> <034a01ca8e5a$131eb1e0$0701a8c0@MSIMMSWS> Message-ID: <4B448FEB.70602@colbyconsulting.com> Mark, Understand that I am NOT a very active "partner", in fact as far as I can tell I am more an "associate". I have never made any effort to do the qualifications etc. From what I can tell, for various reasons it is not a good idea to approach the "partnership" from a "how many referrals" angle. While MS does have a "referral" process, if you are going to pursue the partnership it would probably work best to approach it from a "what can I learn" and "what software can I get" all for a reduced price. If you ever get any referrals that is a bonus. All of that said with the understanding that I am not really active in their program. I would like to be, but more in order to get trained up a little. There are areas that I would like to know more about. Personally, I have never had much use for Microsoft's "credentials", but many people do get them. I have always been about "what do I know" more than "what strings of characters can I put after my name". OTOH if I had spent 10 years getting a "doctor" title that would probably be important to me. In terms of my proposal however I think forming a "virtual consulting company" would be about referrals, however the referrals would likely come from other members of the company. If there are 10 guys from this list, we would undoubtedly have a wide range of backgrounds and expertise. If someone asks if I can do virtualization, I would be able to say "I'll get back to you" and go see if any of you are strong in that area. Or maybe even have a spreadsheet or word document that would tell me "Member XYZ claims to be an 'expert' in these areas". That kind of thing. AND... those "strings of characters" are important to some clients. Thus being able to put a whole string of Microsoft credentials on my web site might attract business. I would not necessarily perform the work, it might go to another partner in the company. More and more, where you live affects your ability to perform your work very little. We can remote in to do programming, database design or maintenance and so forth. If you don't have to physically touch something then it can often be done remotely. A "virtual" company consisting of highly skilled people scattered around the world could work well. John W. Colby www.ColbyConsulting.com Mark Simms wrote: > John - I'm just kind of curious about the MSFT Partnership program.... > Do you get any decent leads or referals from them ? > > > From jwcolby at colbyconsulting.com Wed Jan 6 07:31:38 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 06 Jan 2010 08:31:38 -0500 Subject: [dba-SQLServer] Transfer database schema objects and data to another server/database with SMO Message-ID: <4B4490BA.6050006@colbyconsulting.com> http://www.mssqltips.com/tip.asp?tip=1910&home -- John W. Colby www.ColbyConsulting.com From ab-mi at post3.tele.dk Mon Jan 11 16:48:39 2010 From: ab-mi at post3.tele.dk (Asger Blond) Date: Mon, 11 Jan 2010 23:48:39 +0100 Subject: [dba-SQLServer] SSMS unable to create/open solutions Message-ID: <4036C8A769C645DF821DC6568296F93B@abpc> Dear list, On one of my boxes SSMS is suddenly unable to create or open solutions. Using File|New and File|Open I no longer see the option for Project/Solution. And if I try to open a solution-file I get the message ?Make sure the application for the project type (.ssmssqlproj) is installed?. Any clues? TIA Asger From fhtapia at gmail.com Tue Jan 12 12:28:16 2010 From: fhtapia at gmail.com (Francisco Tapia) Date: Tue, 12 Jan 2010 10:28:16 -0800 Subject: [dba-SQLServer] SSMS unable to create/open solutions In-Reply-To: <4036C8A769C645DF821DC6568296F93B@abpc> References: <4036C8A769C645DF821DC6568296F93B@abpc> Message-ID: What operating system are you using? did you add any updates when you last saw this working? (.net 3 or greater??) -Francisco On Mon, Jan 11, 2010 at 2:48 PM, Asger Blond wrote: > Dear list, > > On one of my boxes SSMS is suddenly unable to create or open solutions. > > Using File|New and File|Open I no longer see the option for > Project/Solution. > > And if I try to open a solution-file I get the message ?Make sure the > application for the project type (.ssmssqlproj) is installed?. > > Any clues? > > TIA > > Asger > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From ab-mi at post3.tele.dk Tue Jan 12 12:47:03 2010 From: ab-mi at post3.tele.dk (Asger Blond) Date: Tue, 12 Jan 2010 19:47:03 +0100 Subject: [dba-SQLServer] SSMS unable to create/open solutions In-Reply-To: References: <4036C8A769C645DF821DC6568296F93B@abpc> Message-ID: <0B62FCC735B54DA399B5208B0754F239@abpc> Windows Server 2008 Enterprise run in a virtual pc - .Net Framework 3.5 with SP 1. No updates added since I saw it working (about two month ago). Just all of a sudden SSMS lost the ability to handle solutions - I'm puzzled. Asger -----Oprindelig meddelelse----- Fra: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] P? vegne af Francisco Tapia Sendt: 12. januar 2010 19:28 Til: Discussion concerning MS SQL Server Emne: Re: [dba-SQLServer] SSMS unable to create/open solutions What operating system are you using? did you add any updates when you last saw this working? (.net 3 or greater??) -Francisco On Mon, Jan 11, 2010 at 2:48 PM, Asger Blond wrote: > Dear list, > > On one of my boxes SSMS is suddenly unable to create or open solutions. > > Using File|New and File|Open I no longer see the option for > Project/Solution. > > And if I try to open a solution-file I get the message ?Make sure the > application for the project type (.ssmssqlproj) is installed?. > > Any clues? > > TIA > > Asger > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From fhtapia at gmail.com Tue Jan 12 15:17:27 2010 From: fhtapia at gmail.com (Francisco Tapia) Date: Tue, 12 Jan 2010 13:17:27 -0800 Subject: [dba-SQLServer] SSMS unable to create/open solutions In-Reply-To: <0B62FCC735B54DA399B5208B0754F239@abpc> References: <4036C8A769C645DF821DC6568296F93B@abpc> <0B62FCC735B54DA399B5208B0754F239@abpc> Message-ID: Were these solutions a part of a project? if so, does it work when you add the project first?? -Francisco On Tue, Jan 12, 2010 at 10:47 AM, Asger Blond wrote: > Windows Server 2008 Enterprise run in a virtual pc - .Net Framework 3.5 > with SP 1. No updates added since I saw it working (about two month ago). > Just all of a sudden SSMS lost the ability to handle solutions - I'm > puzzled. > Asger > > -----Oprindelig meddelelse----- > Fra: dba-sqlserver-bounces at databaseadvisors.com [mailto: > dba-sqlserver-bounces at databaseadvisors.com] P? vegne af Francisco Tapia > Sendt: 12. januar 2010 19:28 > Til: Discussion concerning MS SQL Server > Emne: Re: [dba-SQLServer] SSMS unable to create/open solutions > > What operating system are you using? did you add any updates when you last > saw this working? (.net 3 or greater??) > -Francisco > > > On Mon, Jan 11, 2010 at 2:48 PM, Asger Blond wrote: > > > Dear list, > > > > On one of my boxes SSMS is suddenly unable to create or open solutions. > > > > Using File|New and File|Open I no longer see the option for > > Project/Solution. > > > > And if I try to open a solution-file I get the message ?Make sure the > > application for the project type (.ssmssqlproj) is installed?. > > > > Any clues? > > > > TIA > > > > Asger > > > > _______________________________________________ > > dba-SQLServer mailing list > > dba-SQLServer at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > > http://www.databaseadvisors.com > > > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From fhtapia at gmail.com Tue Jan 12 15:22:35 2010 From: fhtapia at gmail.com (Francisco Tapia) Date: Tue, 12 Jan 2010 13:22:35 -0800 Subject: [dba-SQLServer] SSMS unable to create/open solutions In-Reply-To: <0B62FCC735B54DA399B5208B0754F239@abpc> References: <4036C8A769C645DF821DC6568296F93B@abpc> <0B62FCC735B54DA399B5208B0754F239@abpc> Message-ID: I found this on google, maybe it will help you out from http://social.msdn.microsoft.com/Forums/en-US/tfsgeneral/thread/becf015a-38e3-4acd-96f8-836a980cc2ce After dealing with numerous installation issues like management studio not beign installed and then trying to install Client tools and getting rediculous errors, I found the MSI that sorted it all out for me. In Enterprise edition, go to your DVD directory and find the following: \ENGLISH\SQL2005\ENTERPRISE\32BIT\Tools\Setup\SqlRun_Tools.msi When you run this, you will be able to force the installation of all the client tools which will put Management Studio in, as well as add the project types for Visual Studio. ========== -Francisco http://sqlthis.blogspot.com | Tsql and More... On Tue, Jan 12, 2010 at 10:47 AM, Asger Blond wrote: > Windows Server 2008 Enterprise run in a virtual pc - .Net Framework 3.5 > with SP 1. No updates added since I saw it working (about two month ago). > Just all of a sudden SSMS lost the ability to handle solutions - I'm > puzzled. > Asger > > -----Oprindelig meddelelse----- > Fra: dba-sqlserver-bounces at databaseadvisors.com [mailto: > dba-sqlserver-bounces at databaseadvisors.com] P? vegne af Francisco Tapia > Sendt: 12. januar 2010 19:28 > Til: Discussion concerning MS SQL Server > Emne: Re: [dba-SQLServer] SSMS unable to create/open solutions > > What operating system are you using? did you add any updates when you last > saw this working? (.net 3 or greater??) > -Francisco > > > On Mon, Jan 11, 2010 at 2:48 PM, Asger Blond wrote: > > > Dear list, > > > > On one of my boxes SSMS is suddenly unable to create or open solutions. > > > > Using File|New and File|Open I no longer see the option for > > Project/Solution. > > > > And if I try to open a solution-file I get the message ?Make sure the > > application for the project type (.ssmssqlproj) is installed?. > > > > Any clues? > > > > TIA > > > > Asger > > > > _______________________________________________ > > dba-SQLServer mailing list > > dba-SQLServer at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > > http://www.databaseadvisors.com > > > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From ab-mi at post3.tele.dk Tue Jan 12 16:30:00 2010 From: ab-mi at post3.tele.dk (Asger Blond) Date: Tue, 12 Jan 2010 23:30:00 +0100 Subject: [dba-SQLServer] SSMS unable to create/open solutions In-Reply-To: References: <4036C8A769C645DF821DC6568296F93B@abpc> <0B62FCC735B54DA399B5208B0754F239@abpc> Message-ID: Hi Francisco, I can't add a project in SSMS since it no longer gives me this opportunity. The solutions are just a bunch of scripts, and I managed to export them to the host and then import them to another virtual pc where they work ok. I'll try the hint from your second posting and report back if it solves the problem. Thanks Asger -----Oprindelig meddelelse----- Fra: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] P? vegne af Francisco Tapia Sendt: 12. januar 2010 22:17 Til: Discussion concerning MS SQL Server Emne: Re: [dba-SQLServer] SSMS unable to create/open solutions Were these solutions a part of a project? if so, does it work when you add the project first?? -Francisco On Tue, Jan 12, 2010 at 10:47 AM, Asger Blond wrote: > Windows Server 2008 Enterprise run in a virtual pc - .Net Framework 3.5 > with SP 1. No updates added since I saw it working (about two month ago). > Just all of a sudden SSMS lost the ability to handle solutions - I'm > puzzled. > Asger > > -----Oprindelig meddelelse----- > Fra: dba-sqlserver-bounces at databaseadvisors.com [mailto: > dba-sqlserver-bounces at databaseadvisors.com] P? vegne af Francisco Tapia > Sendt: 12. januar 2010 19:28 > Til: Discussion concerning MS SQL Server > Emne: Re: [dba-SQLServer] SSMS unable to create/open solutions > > What operating system are you using? did you add any updates when you last > saw this working? (.net 3 or greater??) > -Francisco > > > On Mon, Jan 11, 2010 at 2:48 PM, Asger Blond wrote: > > > Dear list, > > > > On one of my boxes SSMS is suddenly unable to create or open solutions. > > > > Using File|New and File|Open I no longer see the option for > > Project/Solution. > > > > And if I try to open a solution-file I get the message ?Make sure the > > application for the project type (.ssmssqlproj) is installed?. > > > > Any clues? > > > > TIA > > > > Asger > > > > _______________________________________________ > > dba-SQLServer mailing list > > dba-SQLServer at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > > http://www.databaseadvisors.com > > > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Mon Jan 18 14:40:19 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 18 Jan 2010 15:40:19 -0500 Subject: [dba-SQLServer] 2008 backup / restore scripts Message-ID: <4B54C733.5060006@colbyconsulting.com> I obtained a developer copy of SQL Server 2008, which has full backup / restore with compression if I understand correctly. I had been using Idera's SQL Safe free version but that is no longer free. Of course now all of my backups are locked away in a version of a program that I can no longer use. Free is only free until it's not, whereupon it is a liability. Sigh. Anyway... I am wondering if anyone has a stored procedure that will use the 2008 built-in backup to do a full backup with compression. I built a C# program to use the Idera scripts to do my backups and restores. I am hoping that I can get a similar native 2008 stored procedure that I can call to do the same thing. -- John W. Colby www.ColbyConsulting.com From ab-mi at post3.tele.dk Mon Jan 18 15:20:11 2010 From: ab-mi at post3.tele.dk (Asger Blond) Date: Mon, 18 Jan 2010 22:20:11 +0100 Subject: [dba-SQLServer] 2008 backup / restore scripts In-Reply-To: <4B54C733.5060006@colbyconsulting.com> References: <4B54C733.5060006@colbyconsulting.com> Message-ID: <12F9B2F44EB5424BA33ED6276D183A37@abpc> BACKUP DATABASE TO DISK = '' WITH COMPRESSION Asger -----Oprindelig meddelelse----- Fra: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] P? vegne af jwcolby Sendt: 18. januar 2010 21:40 Til: Dba-Sqlserver; VBA Emne: [dba-SQLServer] 2008 backup / restore scripts I obtained a developer copy of SQL Server 2008, which has full backup / restore with compression if I understand correctly. I had been using Idera's SQL Safe free version but that is no longer free. Of course now all of my backups are locked away in a version of a program that I can no longer use. Free is only free until it's not, whereupon it is a liability. Sigh. Anyway... I am wondering if anyone has a stored procedure that will use the 2008 built-in backup to do a full backup with compression. I built a C# program to use the Idera scripts to do my backups and restores. I am hoping that I can get a similar native 2008 stored procedure that I can call to do the same thing. -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From bheid at sc.rr.com Mon Jan 18 20:10:38 2010 From: bheid at sc.rr.com (Bobby Heid) Date: Mon, 18 Jan 2010 21:10:38 -0500 Subject: [dba-SQLServer] 2008 backup / restore scripts In-Reply-To: <4B54C733.5060006@colbyconsulting.com> References: <4B54C733.5060006@colbyconsulting.com> Message-ID: <000301ca98ac$9844c100$c8ce4300$@rr.com> John, BACKUP DATABASE [db] TO DISK = N'C:\Path\db.bak' WITH NOFORMAT, NOINIT, NAME = N'Some description', SKIP, NOREWIND, NOUNLOAD, COMPRESSION, STATS = 10 GO I let SSMS 2008 generate this for me. What I did was: 1) Right-click on the db you want to backup 2) Choose Tasks\Backup... 3) Under the options (list on left hand side), select the 'Set backup compression' (lower right) and change the dropdown to compression. 4) Then at the top of the backup window, you'll see a dropdown that says 'Script'. Select 'Script action to new query window' (or one of the other options) and it will create the script for you. This can be done with many of the options in SSMS. Bobby -----Original Message----- From: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, January 18, 2010 3:40 PM To: Dba-Sqlserver; VBA Subject: [dba-SQLServer] 2008 backup / restore scripts I obtained a developer copy of SQL Server 2008, which has full backup / restore with compression if I understand correctly. I had been using Idera's SQL Safe free version but that is no longer free. Of course now all of my backups are locked away in a version of a program that I can no longer use. Free is only free until it's not, whereupon it is a liability. Sigh. Anyway... I am wondering if anyone has a stored procedure that will use the 2008 built-in backup to do a full backup with compression. I built a C# program to use the Idera scripts to do my backups and restores. I am hoping that I can get a similar native 2008 stored procedure that I can call to do the same thing. -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Tue Jan 19 08:05:51 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Tue, 19 Jan 2010 09:05:51 -0500 Subject: [dba-SQLServer] SQL Server: An interesting data load article Message-ID: <4B55BC3F.20304@colbyconsulting.com> http://msdn.microsoft.com/en-us/library/dd537533.aspx -- John W. Colby www.ColbyConsulting.com From mmattys at rochester.rr.com Tue Jan 19 10:36:52 2010 From: mmattys at rochester.rr.com (Mike Mattys) Date: Tue, 19 Jan 2010 11:36:52 -0500 Subject: [dba-SQLServer] SQL Server: An interesting data load article References: <4B55BC3F.20304@colbyconsulting.com> Message-ID: <19D46979D955437FAB7CC7A7AC266178@Mattys> Cool. This is an important niche. I am pondering the levels where the setup requires the hash table for evenly distibuting threads I just read about vs partitioning plus NUMA. - Michael R Mattys MapPoint and Database Dev www.mattysconsulting.com - ----- Original Message ----- From: "jwcolby" To: "Access Developers discussion and problem solving" ; "VBA" ; "Dba-Sqlserver" Sent: Tuesday, January 19, 2010 9:05 AM Subject: [dba-SQLServer] SQL Server: An interesting data load article > > http://msdn.microsoft.com/en-us/library/dd537533.aspx > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Tue Jan 19 10:44:38 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Tue, 19 Jan 2010 11:44:38 -0500 Subject: [dba-SQLServer] SQL Server: An interesting data load article In-Reply-To: <19D46979D955437FAB7CC7A7AC266178@Mattys> References: <4B55BC3F.20304@colbyconsulting.com> <19D46979D955437FAB7CC7A7AC266178@Mattys> Message-ID: <4B55E176.6090104@colbyconsulting.com> I found the concept of loading a temp table and then using that table as a partition of the final table to be pretty darned cool. And of course, as for the server... I WANT ONE! ;) John W. Colby www.ColbyConsulting.com Mike Mattys wrote: > Cool. This is an important niche. > > I am pondering the levels where the setup requires the > hash table for evenly distibuting threads I just read about > vs partitioning plus NUMA. > > - > Michael R Mattys > MapPoint and Database Dev > www.mattysconsulting.com > - > ----- Original Message ----- > From: "jwcolby" > To: "Access Developers discussion and problem solving" > ; "VBA" ; > "Dba-Sqlserver" > Sent: Tuesday, January 19, 2010 9:05 AM > Subject: [dba-SQLServer] SQL Server: An interesting data load article > > >> http://msdn.microsoft.com/en-us/library/dd537533.aspx >> >> -- >> John W. Colby >> www.ColbyConsulting.com >> _______________________________________________ >> dba-SQLServer mailing list >> dba-SQLServer at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >> http://www.databaseadvisors.com >> >> > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Thu Jan 21 07:58:41 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 21 Jan 2010 08:58:41 -0500 Subject: [dba-SQLServer] How unique are SQL Server error numbers? Message-ID: <4B585D91.4010604@colbyconsulting.com> I am calling stored procedures and passing back the error number and error string to my C# program. I am doing things like "create field, create index" etc. If I am trying to create a field and it already exists I get an error: pErrorDesc "Column names in each table must be unique. Column name 'AddrValid' in table '_DataDepotEmailNew.dbo.AZImportFrom' is specified more than once." And pErrorNo "2705" So my question is, can I count on a 2705 being this "column name must be unique"? If so, in my C# program, I can count that as a successful field creation rather than a failure of the stored procedure. ATM I am saying that a failure occurred and not performing the next step, whereas I really can do the next step (update that field). TIA for your assistance. -- John W. Colby www.ColbyConsulting.com From davidmcafee at gmail.com Thu Jan 21 08:41:37 2010 From: davidmcafee at gmail.com (David McAfee) Date: Thu, 21 Jan 2010 06:41:37 -0800 Subject: [dba-SQLServer] [dba-VB] How unique are SQL Server error numbers? In-Reply-To: <8786a4c01001210640g6bc427a7w11bcf56afb912d96@mail.gmail.com> References: <4B585D91.4010604@colbyconsulting.com> <8786a4c01001210640g6bc427a7w11bcf56afb912d96@mail.gmail.com> Message-ID: <8786a4c01001210641t6eec982n5e4ecbc7735e8d94@mail.gmail.com> John, you can trap for built in errors (such as 2705) in TSQL, or you can raise your own if/when certain conditions are met. Sent from my Droid phone. On Jan 21, 2010 6:00 AM, "jwcolby" wrote: I am calling stored procedures and passing back the error number and error string to my C# program. I am doing things like "create field, create index" etc. If I am trying to create a field and it already exists I get an error: pErrorDesc "Column names in each table must be unique. Column name 'AddrValid' in table '_DataDepotEmailNew.dbo.AZImportFrom' is specified more than once." And pErrorNo "2705" So my question is, can I count on a 2705 being this "column name must be unique"? If so, in my C# program, I can count that as a successful field creation rather than a failure of the stored procedure. ATM I am saying that a failure occurred and not performing the next step, whereas I really can do the next step (update that field). TIA for your assistance. -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From ab-mi at post3.tele.dk Thu Jan 21 08:44:12 2010 From: ab-mi at post3.tele.dk (Asger Blond) Date: Thu, 21 Jan 2010 15:44:12 +0100 Subject: [dba-SQLServer] How unique are SQL Server error numbers? In-Reply-To: <4B585D91.4010604@colbyconsulting.com> References: <4B585D91.4010604@colbyconsulting.com> Message-ID: <09F46A688CEA47CD90D7C58C371C71AE@abpc> Select master.sys.messages where message_id=2705 --> only gives the said error message. So it seems to be unique. Asger -----Oprindelig meddelelse----- Fra: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] P? vegne af jwcolby Sendt: 21. januar 2010 14:59 Til: VBA; Sqlserver-Dba Emne: [dba-SQLServer] How unique are SQL Server error numbers? I am calling stored procedures and passing back the error number and error string to my C# program. I am doing things like "create field, create index" etc. If I am trying to create a field and it already exists I get an error: pErrorDesc "Column names in each table must be unique. Column name 'AddrValid' in table '_DataDepotEmailNew.dbo.AZImportFrom' is specified more than once." And pErrorNo "2705" So my question is, can I count on a 2705 being this "column name must be unique"? If so, in my C# program, I can count that as a successful field creation rather than a failure of the stored procedure. ATM I am saying that a failure occurred and not performing the next step, whereas I really can do the next step (update that field). TIA for your assistance. -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Thu Jan 21 10:42:07 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 21 Jan 2010 11:42:07 -0500 Subject: [dba-SQLServer] How unique are SQL Server error numbers? In-Reply-To: <09F46A688CEA47CD90D7C58C371C71AE@abpc> References: <4B585D91.4010604@colbyconsulting.com> <09F46A688CEA47CD90D7C58C371C71AE@abpc> Message-ID: <4B5883DF.8040701@colbyconsulting.com> Asger, > Select master.sys.messages where message_id=XXXXX That is a good thing to know. Thanks! John W. Colby www.ColbyConsulting.com Asger Blond wrote: > Select master.sys.messages where message_id=2705 --> only gives the said error message. So it seems to be unique. > Asger > > -----Oprindelig meddelelse----- > Fra: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] P? vegne af jwcolby > Sendt: 21. januar 2010 14:59 > Til: VBA; Sqlserver-Dba > Emne: [dba-SQLServer] How unique are SQL Server error numbers? > > I am calling stored procedures and passing back the error number and error string to my C# program. > I am doing things like "create field, create index" etc. If I am trying to create a field and it > already exists I get an error: > > pErrorDesc "Column names in each table must be unique. Column name 'AddrValid' in table > '_DataDepotEmailNew.dbo.AZImportFrom' is specified more than once." > > And > > pErrorNo "2705" > > So my question is, can I count on a 2705 being this "column name must be unique"? If so, in my C# > program, I can count that as a successful field creation rather than a failure of the stored procedure. > > ATM I am saying that a failure occurred and not performing the next step, whereas I really can do > the next step (update that field). > > TIA for your assistance. > From jwcolby at colbyconsulting.com Fri Jan 22 07:56:31 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 22 Jan 2010 08:56:31 -0500 Subject: [dba-SQLServer] Free SQL Server tools Message-ID: <4B59AE8F.2030206@colbyconsulting.com> For all the small shops out there, this just in: http://www.sqlservercentral.com/articles/Tools/64908/ -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Fri Jan 22 10:11:29 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 22 Jan 2010 11:11:29 -0500 Subject: [dba-SQLServer] Is it true? Message-ID: <4B59CE31.7090200@colbyconsulting.com> My understanding is that if you have a clustered index then you do not need to include the key fields in other indexes because all other indexes include that key field data automatically behind the scenes. True? -- John W. Colby www.ColbyConsulting.com From ab-mi at post3.tele.dk Fri Jan 22 13:42:06 2010 From: ab-mi at post3.tele.dk (Asger Blond) Date: Fri, 22 Jan 2010 20:42:06 +0100 Subject: [dba-SQLServer] Is it true? In-Reply-To: <4B59CE31.7090200@colbyconsulting.com> References: <4B59CE31.7090200@colbyconsulting.com> Message-ID: <4F97725A47BD4BD6B2EA43384BBA2A9E@abpc> True. Asger -----Oprindelig meddelelse----- Fra: dba-sqlserver-bounces at databaseadvisors.com [mailto:dba-sqlserver-bounces at databaseadvisors.com] P? vegne af jwcolby Sendt: 22. januar 2010 17:11 Til: Sqlserver-Dba Emne: [dba-SQLServer] Is it true? My understanding is that if you have a clustered index then you do not need to include the key fields in other indexes because all other indexes include that key field data automatically behind the scenes. True? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-SQLServer mailing list dba-SQLServer at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-sqlserver http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Jan 29 22:44:15 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 29 Jan 2010 23:44:15 -0500 Subject: [dba-SQLServer] BCP import spec Message-ID: <4B63B91F.8070803@colbyconsulting.com> I have a situation where I have to import into SQL Server millions of records, fixed width, same format, different data, over and over again. The format is given to me in a spreadsheet looking like this: FROM TO LENGTH MODE DESCRIPTION 1 40 40 C FULL NAME (PARSED NAME IN POS 421-482) 41 80 40 C COMPANY 81 120 40 C ADDRESS 1 121 160 40 C ADDRESS 2 161 190 30 C CITY 191 192 2 C STATE 193 197 5 C ZIP 198 198 1 C ZIP 4 - MANY more columns. I need to turn import this into a table. I have done this once, brute force, using the import wizard. The bottom line is that I now have an existing table in SQL Server that "fits" the spec. The fields are all nvarchar() with the correct number of characters in each field. One problem is that I do NOT want the trailing spaces which pad the data to the fixed width. I do have a udf which strips off padding but it is going to be a PITA to apply that to all of the fields for millions of records (though I am capable of doing that if necessary). Is there any way to tell SQL Server to import the data into this table. Preferably in a stored procedure that I can execute from my existing C# code base which knows how to execute stored procedures. I have to do this many times in the future, and I do NOT want to do this manually every time. Somebody mentioned on time doing something with the wizard and then peeking at the code that the wizard generated. I don't know how to do that but if I could, I could then cut that into a SP and execute the SP as needed. Something like that. Anybody have any ideas? TIA -- John W. Colby www.ColbyConsulting.com From marklbreen at gmail.com Sat Jan 30 06:06:48 2010 From: marklbreen at gmail.com (Mark Breen) Date: Sat, 30 Jan 2010 12:06:48 +0000 Subject: [dba-SQLServer] BCP import spec In-Reply-To: <4B63B91F.8070803@colbyconsulting.com> References: <4B63B91F.8070803@colbyconsulting.com> Message-ID: Hello John, Forgive me if you have already deep dived into SSIS (2005) which used to be DTS (2000 + SQL 7.0), which used to be BCP (SQL 6.5) which you thought me about in 1997. I just looked around my SQL 2008 installation here and could not find SSIS, but I do have Business Intelligence here so that must be what they are calling it in SQL 2008. I only explored SSIS a little a few years ago, but is seemed to be a simple, almost script based system that is specificially designed for this type of work. I believe that data warehouse people refer to this work as ETL - Extraction, Transforming and Loading. On one project I working on there was a whole ETL department. As we all know, you can easily code this by hand, but I guess that the SQL team now that for data warehouses they need a set ot ETL tools and BCP was version 1.0, but now with SQL 2008, we must have a more sophisticated set of tools. >From my own, limited use of SSIS, I know that you can load a file, perform validation, modification on a line by line basis, and then push that along, to either a good data or bad data branch, and finally place that good data in a new table, in what ever form you wish. Would it be faster than loading a single record, or even a 1000 records at a time - who knows, but the SQL tools for ETL are designed for this type of work. >From this link, I have copied the following Overview Microsoft SQL Server 2008 provides a scalable enterprise data integration platform with exceptional Extract, Transform, Load (ETL) and integration capabilities, enabling organizations to more easily manage data from a wide array of data sources. Top New Features - Create Script tasks by using Microsoft Visual C# and Microsoft Visual Basic .NET. - Use ADO.NET for tasks as well as for source and destination components. - Improve scalability with thread pooling and enhanced lookup transformations. - Perform more functional and scalable data transfers with the improved SQL Server Import and Export Wizard. - New SSIS connectors for SAP BW, Oracle and Teradata. I hope that is some help, and does not frustrate you to say "I know that stuff, but it is a hassle to take the deep dive" Mark On 30 January 2010 04:44, jwcolby wrote: > I have a situation where I have to import into SQL Server millions of > records, fixed width, same > format, different data, over and over again. > > > The format is given to me in a spreadsheet looking like this: > > FROM TO LENGTH MODE DESCRIPTION > 1 40 40 C FULL NAME (PARSED NAME IN POS 421-482) > 41 80 40 C COMPANY > 81 120 40 C ADDRESS 1 > 121 160 40 C ADDRESS 2 > 161 190 30 C CITY > 191 192 2 C STATE > 193 197 5 C ZIP > 198 198 1 C ZIP 4 - > > MANY more columns. > > I need to turn import this into a table. I have done this once, brute > force, using the import wizard. > > The bottom line is that I now have an existing table in SQL Server that > "fits" the spec. The fields > are all nvarchar() with the correct number of characters in each field. > One problem is that I do > NOT want the trailing spaces which pad the data to the fixed width. I do > have a udf which strips > off padding but it is going to be a PITA to apply that to all of the fields > for millions of records > (though I am capable of doing that if necessary). > > Is there any way to tell SQL Server to import the data into this table. > Preferably in a stored > procedure that I can execute from my existing C# code base which knows how > to execute stored procedures. > > I have to do this many times in the future, and I do NOT want to do this > manually every time. > Somebody mentioned on time doing something with the wizard and then peeking > at the code that the > wizard generated. I don't know how to do that but if I could, I could then > cut that into a SP and > execute the SP as needed. > > Something like that. > > Anybody have any ideas? > > TIA > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Jan 30 08:04:32 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 30 Jan 2010 09:04:32 -0500 Subject: [dba-SQLServer] BCP of fixed width files Message-ID: <4B643C70.7050706@colbyconsulting.com> Amigos, I have to import a fixed width file periodically. It is fairly complex, containing perhaps 50-70 fields. I performed this import one time "manually" using the import wizard, but manually defining the field names / widths every time is not happening! So, I have a "template" table that has the field names and sizes (all nvarchar() ). I have a stored procedure I created last night which will create that table for me. I routinely use BCP to import and export "CSV" type files (delimited), and I have a C# class which allows me to quickly and easily set up a stored procedure object, set up parameters, get back error codes etc. What I need to do (it it is even possible) is to learn how to use BCP with a fixed width file. My understanding is that you use a format file in which you define the fields, data types etc. and then you pass that to BCP. Does anyone out there do this thing, any words of advice. Any alternatives which I as a SQL Server novice could possibly make happen? TIA for any assistance. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Jan 30 08:30:55 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 30 Jan 2010 09:30:55 -0500 Subject: [dba-SQLServer] BCP import spec In-Reply-To: References: <4B63B91F.8070803@colbyconsulting.com> Message-ID: <4B64429F.6020002@colbyconsulting.com> Thanks for the reply Mark, Whatever I do has to be executable fro C#. Additionally I have to be able to figure it out. That last is a pretty severe constraint. ;) BCP is extremely fast, and I already automate that in C#. I have a couple of BCP input and output stored procedures which build dynamic TSQL to perform the BCP. I then execute those stored procedures using a Stored Procedure class I wrote which essentially wraps a command object and implements a set of standard SP parameters that I use all of the time. So... if I can do it with BCP that is preferred, simply because I have struggled 1/2 way up the learning curve already. I am open to other tools, and I know that I really should learn SSIS (or whatever they are calling it now). John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > Forgive me if you have already deep dived into SSIS (2005) which used to be > DTS (2000 + SQL 7.0), which used to be BCP (SQL 6.5) which you thought me > about in 1997. > > I just looked around my SQL 2008 installation here and could not find SSIS, > but I do have Business Intelligence here so that must be what they are > calling it in SQL 2008. > > I only explored SSIS a little a few years ago, but is seemed to be a simple, > almost script based system that is specificially designed for this type of > work. I believe that data warehouse people refer to this work as ETL - > Extraction, Transforming and Loading. On one project I working on there was > a whole ETL department. > > As we all know, you can easily code this by hand, but I guess that the SQL > team now that for data warehouses they need a set ot ETL tools and BCP was > version 1.0, but now with SQL 2008, we must have a more sophisticated set of > tools. > >>From my own, limited use of SSIS, I know that you can load a file, perform > validation, modification on a line by line basis, and then push that along, > to either a good data or bad data branch, and finally place that good data > in a new table, in what ever form you wish. > > Would it be faster than loading a single record, or even a 1000 records at a > time - who knows, but the SQL tools for ETL are designed for this type of > work. > > >>From this link, > I have copied the following > > Overview > > Microsoft SQL Server 2008 provides a scalable enterprise data integration > platform with exceptional Extract, Transform, Load (ETL) and integration > capabilities, enabling organizations to more easily manage data from a wide > array of data sources. > Top New Features > > - > > Create Script tasks by using Microsoft Visual C# and Microsoft Visual > Basic .NET. > - > > Use ADO.NET for tasks as well as for source and destination components. > - > > Improve scalability with thread pooling and enhanced lookup > transformations. > - > > Perform more functional and scalable data transfers with the improved SQL > Server Import and Export Wizard. > - > > New SSIS connectors for SAP BW, Oracle and Teradata. > > > > > > I hope that is some help, and does not frustrate you to say "I know that > stuff, but it is a hassle to take the deep dive" > > Mark > > > > > On 30 January 2010 04:44, jwcolby wrote: > >> I have a situation where I have to import into SQL Server millions of >> records, fixed width, same >> format, different data, over and over again. >> >> >> The format is given to me in a spreadsheet looking like this: >> >> FROM TO LENGTH MODE DESCRIPTION >> 1 40 40 C FULL NAME (PARSED NAME IN POS 421-482) >> 41 80 40 C COMPANY >> 81 120 40 C ADDRESS 1 >> 121 160 40 C ADDRESS 2 >> 161 190 30 C CITY >> 191 192 2 C STATE >> 193 197 5 C ZIP >> 198 198 1 C ZIP 4 - >> >> MANY more columns. >> >> I need to turn import this into a table. I have done this once, brute >> force, using the import wizard. >> >> The bottom line is that I now have an existing table in SQL Server that >> "fits" the spec. The fields >> are all nvarchar() with the correct number of characters in each field. >> One problem is that I do >> NOT want the trailing spaces which pad the data to the fixed width. I do >> have a udf which strips >> off padding but it is going to be a PITA to apply that to all of the fields >> for millions of records >> (though I am capable of doing that if necessary). >> >> Is there any way to tell SQL Server to import the data into this table. >> Preferably in a stored >> procedure that I can execute from my existing C# code base which knows how >> to execute stored procedures. >> >> I have to do this many times in the future, and I do NOT want to do this >> manually every time. >> Somebody mentioned on time doing something with the wizard and then peeking >> at the code that the >> wizard generated. I don't know how to do that but if I could, I could then >> cut that into a SP and >> execute the SP as needed. >> >> Something like that. >> >> Anybody have any ideas? >> >> TIA >> >> -- >> John W. Colby >> www.ColbyConsulting.com >> _______________________________________________ >> dba-SQLServer mailing list >> dba-SQLServer at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From marklbreen at gmail.com Sat Jan 30 13:56:48 2010 From: marklbreen at gmail.com (Mark Breen) Date: Sat, 30 Jan 2010 19:56:48 +0000 Subject: [dba-SQLServer] BCP of fixed width files In-Reply-To: <4B643C70.7050706@colbyconsulting.com> References: <4B643C70.7050706@colbyconsulting.com> Message-ID: Hello John, the way that I learnt to create a format file for bcp use was to perform an export from the table that i intend to import to, and then once I have the export working well, I could save that as a format file. Once you have the format file, the import will work well also. Did you teach me that trick in 1997/8 ? Having said all that, I still encourage you to spend eight hours with SSIS, you are performing ETL, and SSIS is the ETL tool that MS supply. It is designed for volumes like you are using. SSIS must be as fast as bcp, just a better GUI and much more control. You can run scripts on a column by column or record by record basis. *Extract* from your text file, *transform * the data and then when you have clean data *load* it into SQL Server tables. With your knowledge of your particular data, and your programming abilities, you could have SSIS really singing in a week or two, and I think you would not look back. Additionally, SSIS and ETL are very marketable toolsets to be expert in. Will not hurt the CV. Once you have an SSIS package, you can call that from an sproc, and probably even more sophisticated ways to manipulate it. PS, I have used fixed width files in the past and hate them, once change in column seven and you have to re-do every column. For even small tables, I think it is worth while building a string concatenation tool in Excel, so that I just name the columns and sizes and Excel automatically calculates the start and end points of the data. For you, this must be a must. If you have not already discovered that, you will soon enough. Remapping by hand the second time is no fun. Thanks Mark On 30 January 2010 14:04, jwcolby wrote: > Amigos, > > I have to import a fixed width file periodically. It is fairly complex, > containing perhaps 50-70 > fields. I performed this import one time "manually" using the import > wizard, but manually defining > the field names / widths every time is not happening! > > So, I have a "template" table that has the field names and sizes (all > nvarchar() ). I have a stored > procedure I created last night which will create that table for me. > > I routinely use BCP to import and export "CSV" type files (delimited), and > I have a C# class which > allows me to quickly and easily set up a stored procedure object, set up > parameters, get back error > codes etc. > > What I need to do (it it is even possible) is to learn how to use BCP with > a fixed width file. My > understanding is that you use a format file in which you define the fields, > data types etc. and then > you pass that to BCP. > > Does anyone out there do this thing, any words of advice. Any alternatives > which I as a SQL Server > novice could possibly make happen? > > TIA for any assistance. > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Jan 30 15:27:09 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 30 Jan 2010 16:27:09 -0500 Subject: [dba-SQLServer] Bulk insert failing Message-ID: <4B64A42D.7020403@colbyconsulting.com> I have created an XML format file for a table using BCP to create the format file. I have placed that in a known path. I have created a stored procedure which builds a dynamic SQL Statement to perform a bulk insert using the format file. My TSQL is as follows: BULK INSERT [_DataDepotEmail].[dbo].[ANK_Recovered_Raw] FROM 'D:\UD37_M12_NCOA_OP.TXT' WITH (FORMATFILE = 'D:\ANK_Recovered_Raw.xml') I received the following error: There was a BCP error IMPORTING data into _DataDepotEmail Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)". Unfortunately this is a generic "something went wrong" error message that SQL Server is so famous for. It just occurred to me to cut and paste the dynamic SQL above into a query window and run it and I get a further error message: "The bulk load failed. The column is too long in the data file for row 1, column 1. Verify that the field terminator and row terminator are specified correctly." So apparently it is attempting to use the format file to open the file. I guess. The format file says the first field should be 80 characters, which it is. I am pretty much stalled at this point. If anyone has any ideas I would be most grateful. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Jan 30 15:42:11 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 30 Jan 2010 16:42:11 -0500 Subject: [dba-SQLServer] BCP of fixed width files In-Reply-To: References: <4B643C70.7050706@colbyconsulting.com> Message-ID: <4B64A7B3.3030308@colbyconsulting.com> Mark, Sigh. The problem here is that I don't have the table I intend to import to. Or more correctly I thought I did but they changed the format. Dropped one field for sure, and who knows what else. I found a way to cause BCP to export the spec if you have the table, but if you don't have the table (importing something new...) then you have to do it once manually in order to get that table the first time. This thing is 670 characters wide and about 80 or so fields. This sucks so bad! John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > the way that I learnt to create a format file for bcp use was to perform an > export from the table that i intend to import to, and then once I have the > export working well, I could save that as a format file. Once you have the > format file, the import will work well also. Did you teach me that trick in > 1997/8 ? > > Having said all that, I still encourage you to spend eight hours with SSIS, > you are performing ETL, and SSIS is the ETL tool that MS supply. It is > designed for volumes like you are using. SSIS must be as fast as bcp, just > a better GUI and much more control. You can run scripts on a column by > column or record by record basis. *Extract* from your text file, *transform > * the data and then when you have clean data *load* it into SQL Server > tables. > > With your knowledge of your particular data, and your programming abilities, > you could have SSIS really singing in a week or two, and I think you would > not look back. > > Additionally, SSIS and ETL are very marketable toolsets to be expert in. > Will not hurt the CV. > > Once you have an SSIS package, you can call that from an sproc, and probably > even more sophisticated ways to manipulate it. > > PS, I have used fixed width files in the past and hate them, once change in > column seven and you have to re-do every column. For even small tables, I > think it is worth while building a string concatenation tool in Excel, so > that I just name the columns and sizes and Excel automatically calculates > the start and end points of the data. For you, this must be a must. If you > have not already discovered that, you will soon enough. Remapping by hand > the second time is no fun. > > Thanks > > Mark > > > > > > > On 30 January 2010 14:04, jwcolby wrote: > >> Amigos, >> >> I have to import a fixed width file periodically. It is fairly complex, >> containing perhaps 50-70 >> fields. I performed this import one time "manually" using the import >> wizard, but manually defining >> the field names / widths every time is not happening! >> >> So, I have a "template" table that has the field names and sizes (all >> nvarchar() ). I have a stored >> procedure I created last night which will create that table for me. >> >> I routinely use BCP to import and export "CSV" type files (delimited), and >> I have a C# class which >> allows me to quickly and easily set up a stored procedure object, set up >> parameters, get back error >> codes etc. >> >> What I need to do (it it is even possible) is to learn how to use BCP with >> a fixed width file. My >> understanding is that you use a format file in which you define the fields, >> data types etc. and then >> you pass that to BCP. >> >> Does anyone out there do this thing, any words of advice. Any alternatives >> which I as a SQL Server >> novice could possibly make happen? >> >> TIA for any assistance. >> >> -- >> John W. Colby >> www.ColbyConsulting.com >> _______________________________________________ >> dba-SQLServer mailing list >> dba-SQLServer at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From marklbreen at gmail.com Sun Jan 31 04:24:05 2010 From: marklbreen at gmail.com (Mark Breen) Date: Sun, 31 Jan 2010 10:24:05 +0000 Subject: [dba-SQLServer] BCP of fixed width files In-Reply-To: <4B64A7B3.3030308@colbyconsulting.com> References: <4B643C70.7050706@colbyconsulting.com> <4B64A7B3.3030308@colbyconsulting.com> Message-ID: Hello John, Sorry to hear that, do you have the column mappings? If not, how can you interrogate the fixed width file, if you then... if it is only 80 columns can you not a) make that table manually, then use it to generate the format file (probably 80 minutes work) b) if you have an existing similar table then modify that, add the new columns, and use that to generate a format file. Good luck, Mark On 30 January 2010 21:42, jwcolby wrote: > Mark, > > Sigh. > > The problem here is that I don't have the table I intend to import to. Or > more correctly I thought > I did but they changed the format. Dropped one field for sure, and who > knows what else. I found a > way to cause BCP to export the spec if you have the table, but if you don't > have the table > (importing something new...) then you have to do it once manually in order > to get that table the > first time. This thing is 670 characters wide and about 80 or so fields. > > This sucks so bad! > > John W. Colby > www.ColbyConsulting.com > > > Mark Breen wrote: > > Hello John, > > > > the way that I learnt to create a format file for bcp use was to perform > an > > export from the table that i intend to import to, and then once I have > the > > export working well, I could save that as a format file. Once you have > the > > format file, the import will work well also. Did you teach me that trick > in > > 1997/8 ? > > > > Having said all that, I still encourage you to spend eight hours with > SSIS, > > you are performing ETL, and SSIS is the ETL tool that MS supply. It is > > designed for volumes like you are using. SSIS must be as fast as bcp, > just > > a better GUI and much more control. You can run scripts on a column by > > column or record by record basis. *Extract* from your text file, > *transform > > * the data and then when you have clean data *load* it into SQL Server > > tables. > > > > With your knowledge of your particular data, and your programming > abilities, > > you could have SSIS really singing in a week or two, and I think you > would > > not look back. > > > > Additionally, SSIS and ETL are very marketable toolsets to be expert in. > > Will not hurt the CV. > > > > Once you have an SSIS package, you can call that from an sproc, and > probably > > even more sophisticated ways to manipulate it. > > > > PS, I have used fixed width files in the past and hate them, once change > in > > column seven and you have to re-do every column. For even small tables, > I > > think it is worth while building a string concatenation tool in Excel, so > > that I just name the columns and sizes and Excel automatically calculates > > the start and end points of the data. For you, this must be a must. If > you > > have not already discovered that, you will soon enough. Remapping by > hand > > the second time is no fun. > > > > Thanks > > > > Mark > > > > > > > > > > > > > > On 30 January 2010 14:04, jwcolby wrote: > > > >> Amigos, > >> > >> I have to import a fixed width file periodically. It is fairly complex, > >> containing perhaps 50-70 > >> fields. I performed this import one time "manually" using the import > >> wizard, but manually defining > >> the field names / widths every time is not happening! > >> > >> So, I have a "template" table that has the field names and sizes (all > >> nvarchar() ). I have a stored > >> procedure I created last night which will create that table for me. > >> > >> I routinely use BCP to import and export "CSV" type files (delimited), > and > >> I have a C# class which > >> allows me to quickly and easily set up a stored procedure object, set up > >> parameters, get back error > >> codes etc. > >> > >> What I need to do (it it is even possible) is to learn how to use BCP > with > >> a fixed width file. My > >> understanding is that you use a format file in which you define the > fields, > >> data types etc. and then > >> you pass that to BCP. > >> > >> Does anyone out there do this thing, any words of advice. Any > alternatives > >> which I as a SQL Server > >> novice could possibly make happen? > >> > >> TIA for any assistance. > >> > >> -- > >> John W. Colby > >> www.ColbyConsulting.com > >> _______________________________________________ > >> dba-SQLServer mailing list > >> dba-SQLServer at databaseadvisors.com > >> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > >> http://www.databaseadvisors.com > >> > >> > > _______________________________________________ > > dba-SQLServer mailing list > > dba-SQLServer at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > > http://www.databaseadvisors.com > > > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > From gustav at cactus.dk Sun Jan 31 06:06:34 2010 From: gustav at cactus.dk (Gustav Brock) Date: Sun, 31 Jan 2010 13:06:34 +0100 Subject: [dba-SQLServer] UDT: User Defined Type Message-ID: Hi all I noticed that you can declare and use user defined types in SQL Server for the data type of fields. Are any of you using this feature? Any drawbacks or traps? I guess it would not be possible to read such a column in, say, Access? /gustav From jwcolby at colbyconsulting.com Sun Jan 31 07:17:35 2010 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 31 Jan 2010 08:17:35 -0500 Subject: [dba-SQLServer] BCP of fixed width files In-Reply-To: References: <4B643C70.7050706@colbyconsulting.com> <4B64A7B3.3030308@colbyconsulting.com> Message-ID: <4B6582EF.2050300@colbyconsulting.com> I do have the column definitions. In fact I pretty much don't care about most of the columns. It occurred to me last night that I can just define the columns I care about which makes it just a few minutes work. Sometimes you can't see the table for the fields, so to speak. John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > Sorry to hear that, do you have the column mappings? If not, how can you > interrogate the fixed width file, if you then... > > if it is only 80 columns can you not > > a) make that table manually, then use it to generate the format file > (probably 80 minutes work) > b) if you have an existing similar table then modify that, add the new > columns, and use that to generate a format file. > > Good luck, > > Mark > > > > On 30 January 2010 21:42, jwcolby wrote: > >> Mark, >> >> Sigh. >> >> The problem here is that I don't have the table I intend to import to. Or >> more correctly I thought >> I did but they changed the format. Dropped one field for sure, and who >> knows what else. I found a >> way to cause BCP to export the spec if you have the table, but if you don't >> have the table >> (importing something new...) then you have to do it once manually in order >> to get that table the >> first time. This thing is 670 characters wide and about 80 or so fields. >> >> This sucks so bad! >> >> John W. Colby >> www.ColbyConsulting.com >> >> >> Mark Breen wrote: >>> Hello John, >>> >>> the way that I learnt to create a format file for bcp use was to perform >> an >>> export from the table that i intend to import to, and then once I have >> the >>> export working well, I could save that as a format file. Once you have >> the >>> format file, the import will work well also. Did you teach me that trick >> in >>> 1997/8 ? >>> >>> Having said all that, I still encourage you to spend eight hours with >> SSIS, >>> you are performing ETL, and SSIS is the ETL tool that MS supply. It is >>> designed for volumes like you are using. SSIS must be as fast as bcp, >> just >>> a better GUI and much more control. You can run scripts on a column by >>> column or record by record basis. *Extract* from your text file, >> *transform >>> * the data and then when you have clean data *load* it into SQL Server >>> tables. >>> >>> With your knowledge of your particular data, and your programming >> abilities, >>> you could have SSIS really singing in a week or two, and I think you >> would >>> not look back. >>> >>> Additionally, SSIS and ETL are very marketable toolsets to be expert in. >>> Will not hurt the CV. >>> >>> Once you have an SSIS package, you can call that from an sproc, and >> probably >>> even more sophisticated ways to manipulate it. >>> >>> PS, I have used fixed width files in the past and hate them, once change >> in >>> column seven and you have to re-do every column. For even small tables, >> I >>> think it is worth while building a string concatenation tool in Excel, so >>> that I just name the columns and sizes and Excel automatically calculates >>> the start and end points of the data. For you, this must be a must. If >> you >>> have not already discovered that, you will soon enough. Remapping by >> hand >>> the second time is no fun. >>> >>> Thanks >>> >>> Mark >>> >>> >>> >>> >>> >>> >>> On 30 January 2010 14:04, jwcolby wrote: >>> >>>> Amigos, >>>> >>>> I have to import a fixed width file periodically. It is fairly complex, >>>> containing perhaps 50-70 >>>> fields. I performed this import one time "manually" using the import >>>> wizard, but manually defining >>>> the field names / widths every time is not happening! >>>> >>>> So, I have a "template" table that has the field names and sizes (all >>>> nvarchar() ). I have a stored >>>> procedure I created last night which will create that table for me. >>>> >>>> I routinely use BCP to import and export "CSV" type files (delimited), >> and >>>> I have a C# class which >>>> allows me to quickly and easily set up a stored procedure object, set up >>>> parameters, get back error >>>> codes etc. >>>> >>>> What I need to do (it it is even possible) is to learn how to use BCP >> with >>>> a fixed width file. My >>>> understanding is that you use a format file in which you define the >> fields, >>>> data types etc. and then >>>> you pass that to BCP. >>>> >>>> Does anyone out there do this thing, any words of advice. Any >> alternatives >>>> which I as a SQL Server >>>> novice could possibly make happen? >>>> >>>> TIA for any assistance. >>>> >>>> -- >>>> John W. Colby >>>> www.ColbyConsulting.com >>>> _______________________________________________ >>>> dba-SQLServer mailing list >>>> dba-SQLServer at databaseadvisors.com >>>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >>>> http://www.databaseadvisors.com >>>> >>>> >>> _______________________________________________ >>> dba-SQLServer mailing list >>> dba-SQLServer at databaseadvisors.com >>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >>> http://www.databaseadvisors.com >>> >>> >> _______________________________________________ >> dba-SQLServer mailing list >> dba-SQLServer at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > >