From jwcolby at colbyconsulting.com Mon Nov 2 07:58:56 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 02 Nov 2009 08:58:56 -0500 Subject: [dba-VB] SQL Server 2005: Running stored procedures from C# / VBA Message-ID: <4AEEE5A0.4040901@colbyconsulting.com> I am embarking on the task of running the "DB from hell" from C#. I have several dozen stored procedures that perform various tasks and I currently run many of them from Access using a single procedure provided by Charlotte. I need a similar function written in C# (preferably) or VB.Net. ATM I have an Access database which has two specific forms, bound to two tables. These two forms each execute a set of SPs which build temp tables of data to be exported to an external process, build indexes to make the processes faster, BCP the data out to CSV files, and then the reverse on the way back in. I have another Access database that creates an order for the same client. Same kind of thing, temp tables created, indexes, fields added and so forth. My intention is to port each of these processes to C#. The reason is simply that C# has a couple of huge advantages over VBA, such as child threads and built-in capabilities in the framework that I have to really scrounge to make work in VBA. The first thing I need though is code that can execute a stored procedure, pass parameters to it, retrieve parameters back etc. With demo code preferably. Also I would like to open a conversation about how using .Net from inside of SQL Server (2005) works. Does anyone on the list use C# or VB.Net from inside of SQL Server? Where is the code stored? How to do you call it? What do you use it for? TIA, -- John W. Colby www.ColbyConsulting.com From marklbreen at gmail.com Mon Nov 2 10:18:13 2009 From: marklbreen at gmail.com (Mark Breen) Date: Tue, 3 Nov 2009 00:18:13 +0800 Subject: [dba-VB] SQL Server 2005: Running stored procedures from C# / VBA In-Reply-To: <4AEEE5A0.4040901@colbyconsulting.com> References: <4AEEE5A0.4040901@colbyconsulting.com> Message-ID: Hello John, First of all, good luck. Secondly, you mentioned a few days ago that you referred to yourself as an SQL Wannabe, well given the work you are currently doing, I think you can no longer refer to yourself as a wannabe, you are using SQL, and no doubt will be pushing it to it's limits. *SPROCs* There are loads of samples of code and I have nothing to hand here that will assist much, but I think you will be surprised how easy it is. You will get better exmaples, but as an overview you just need to create a connection - usually referred to as cnn. create a cmd to 'hold' the sproc you wish to call join the cnn to the sproc specify the type of the cmd (no results returned, or records returned or a few other types) then you add objects which are parameters, you need one each for each param in the sproc, you can even add a param that will be referred to as OUTPUT instead of INPUT - these output params can be used to pass back single values in additional or instead of, the actual result set. you can also receive back the result code of the sproc. then you just execute the cmd object. Once you get a few lines of code that handle the above, you can copy and paste for the next five years. There are a few other things that I am not as accustomed to, but when you know that you need to read in a result set, you can use a data reader. Someone else can advise on that. when you create the cnn object, usually people pull the actual string from a app.config file, so watch out for that snipit also. BTW, FWIW, other than for binding to grids or combo's, I never used a dataset in code. *BCP* You mentioned BCP, well I have no used that since SQL 6.5, but what I have used in the SSIS in SQL 2005. It is worth taking a look, you may find that you can do a lot of your work in SSIS. It can branch and call sprocs and import and export all as a package. It can be incredibly fast and would be worth take an hour or two and read up on what you can do with SSIS. *Embeding C#* Lastly, other than an article I read in MSDN magazine, where a guy wanted to do some hashing or encryption in side SQL Server I never heard of anyone using / executing code in side SQL Serer. For performance that you need, I would, with my limited skills, keep as much of the work in TSQL - IE Sprocs, and by doing so, you will keep the mass movement of data within the db. I would love to see if anyone here actually ever used the feature of including C# Code within the db. I know that this is not samples, but you can do this, and once you get a set of lines of code that 1) create the cnn 2) create the cmd 3) add the params 4) run the sproc you will not look back. Just keep as much work as possible inside sql server. *Scheduled Tasks* Finally, once last thing to mention, if you have a smtp server, it is so, so each to create scheduled tasks, that run at predefined times and then mail you when they are finished. Running the heavy jobs nightly, can take a lot of the hard work from your daily routines. Again you can manage all this with very little actually code and have working routines in minutes. HTH. Mark 2009/11/2 jwcolby > I am embarking on the task of running the "DB from hell" from C#. I have > several dozen stored > procedures that perform various tasks and I currently run many of them from > Access using a single > procedure provided by Charlotte. I need a similar function written in C# > (preferably) or VB.Net. > > ATM I have an Access database which has two specific forms, bound to two > tables. These two forms > each execute a set of SPs which build temp tables of data to be exported to > an external process, > build indexes to make the processes faster, BCP the data out to CSV files, > and then the reverse on > the way back in. > > I have another Access database that creates an order for the same client. > Same kind of thing, temp > tables created, indexes, fields added and so forth. > > My intention is to port each of these processes to C#. The reason is > simply that C# has a couple of > huge advantages over VBA, such as child threads and built-in capabilities > in the framework that I > have to really scrounge to make work in VBA. > > The first thing I need though is code that can execute a stored procedure, > pass parameters to it, > retrieve parameters back etc. With demo code preferably. > > Also I would like to open a conversation about how using .Net from inside > of SQL Server (2005) > works. Does anyone on the list use C# or VB.Net from inside of SQL Server? > Where is the code > stored? How to do you call it? What do you use it for? > > TIA, > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Mon Nov 2 11:51:47 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 02 Nov 2009 12:51:47 -0500 Subject: [dba-VB] SQL Server 2005: Running stored procedures from C# / VBA In-Reply-To: References: <4AEEE5A0.4040901@colbyconsulting.com> Message-ID: <4AEF1C33.1020904@colbyconsulting.com> Mark, > First of all, good luck. Uh-Ohhh... is that a warning? ;) I refer to myself as a wannabe simply because to this point SQL Server has been a small part of my day. As a sole proprietor my day contains so many different kinds of work that I find it hard to be an expert in something as complex as SQL Server. There are so many people on these lists that spend all day every day in SQL Server that I will truly never be anything other than a wannabe - from their perspective. I have always been more programmer than anything. I come at the world from the perspective of "how can I write code to make my job easier". Luckily I have enough programming background that I can generally accomplish that. Right now I am trying to switch my environment of choice from Access to C#, and BOY IS THAT TOUGH. Programming is always about spending enough time doing it that the syntax becomes second nature. C# is nowhere close to second nature yet. I can taste the power though, so tantalizing. ;) I am using the Idera free backup stuff. Very powerful, recommended. It took me a couple of hours to install on my two servers and write some stored procedures to make them work. Now I really want to be able to select one specific database to backup or restore. To do that I have to be able to: See/select a database on a server. Run a stored procedure that does the backup. How simple is that eh? I just spent the morning learning about the DMO object, the listview control and writing the code to read out the databases and display them in the listview. I still have to find and get working the code that executes the stored procedure. The problem is not that this is hard, it is that I am not fluent in C# yet. But I am getting there, sloooowly but surely. John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > First of all, good luck. > > Secondly, you mentioned a few days ago that you referred to yourself as an > SQL Wannabe, well given the work you are currently doing, I think you can no > longer refer to yourself as a wannabe, you are using SQL, and no doubt will > be pushing it to it's limits. > > *SPROCs* > > There are loads of samples of code and I have nothing to hand here that will > assist much, but I think you will be surprised how easy it is. > > You will get better exmaples, but as an overview you just need to > > create a connection - usually referred to as cnn. > create a cmd to 'hold' the sproc you wish to call > join the cnn to the sproc > specify the type of the cmd (no results returned, or records returned or a > few other types) > then you add objects which are parameters, you need one each for each param > in the sproc, > you can even add a param that will be referred to as OUTPUT instead of INPUT > - these output params can be used to pass back single values in additional > or instead of, the actual result set. > you can also receive back the result code of the sproc. > then you just execute the cmd object. > > Once you get a few lines of code that handle the above, you can copy and > paste for the next five years. > > There are a few other things that I am not as accustomed to, but when you > know that you need to read in a result set, you can use a data reader. > Someone else can advise on that. > > when you create the cnn object, usually people pull the actual string from a > app.config file, so watch out for that snipit also. > > BTW, FWIW, other than for binding to grids or combo's, I never used a > dataset in code. > > > *BCP* > You mentioned BCP, well I have no used that since SQL 6.5, but what I have > used in the SSIS in SQL 2005. It is worth taking a look, you may find that > you can do a lot of your work in SSIS. It can branch and call sprocs and > import and export all as a package. It can be incredibly fast and would be > worth take an hour or two and read up on what you can do with SSIS. > > *Embeding C#* > Lastly, other than an article I read in MSDN magazine, where a guy wanted to > do some hashing or encryption in side SQL Server I never heard of anyone > using / executing code in side SQL Serer. For performance that you need, I > would, with my limited skills, keep as much of the work in TSQL - IE Sprocs, > and by doing so, you will keep the mass movement of data within the db. I > would love to see if anyone here actually ever used the feature of including > C# Code within the db. > > > I know that this is not samples, but you can do this, and once you get a set > of lines of code that > 1) create the cnn > 2) create the cmd > 3) add the params > 4) run the sproc > > you will not look back. Just keep as much work as possible inside sql > server. > > *Scheduled Tasks* > Finally, once last thing to mention, if you have a smtp server, it is so, so > each to create scheduled tasks, that run at predefined times and then mail > you when they are finished. Running the heavy jobs nightly, can take a lot > of the hard work from your daily routines. Again you can manage all this > with very little actually code and have working routines in minutes. > > HTH. > > Mark > > > > > 2009/11/2 jwcolby > >> I am embarking on the task of running the "DB from hell" from C#. I have >> several dozen stored >> procedures that perform various tasks and I currently run many of them from >> Access using a single >> procedure provided by Charlotte. I need a similar function written in C# >> (preferably) or VB.Net. >> >> ATM I have an Access database which has two specific forms, bound to two >> tables. These two forms >> each execute a set of SPs which build temp tables of data to be exported to >> an external process, >> build indexes to make the processes faster, BCP the data out to CSV files, >> and then the reverse on >> the way back in. >> >> I have another Access database that creates an order for the same client. >> Same kind of thing, temp >> tables created, indexes, fields added and so forth. >> >> My intention is to port each of these processes to C#. The reason is >> simply that C# has a couple of >> huge advantages over VBA, such as child threads and built-in capabilities >> in the framework that I >> have to really scrounge to make work in VBA. >> >> The first thing I need though is code that can execute a stored procedure, >> pass parameters to it, >> retrieve parameters back etc. With demo code preferably. >> >> Also I would like to open a conversation about how using .Net from inside >> of SQL Server (2005) >> works. Does anyone on the list use C# or VB.Net from inside of SQL Server? >> Where is the code >> stored? How to do you call it? What do you use it for? >> >> TIA, >> >> -- >> John W. Colby >> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Mon Nov 2 12:41:24 2009 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Tue, 03 Nov 2009 04:41:24 +1000 Subject: [dba-VB] SQL Server 2005: Running stored procedures from C# / VBA In-Reply-To: References: <4AEEE5A0.4040901@colbyconsulting.com>, Message-ID: <4AEF27D4.29660.1C94540E@stuart.lexacorp.com.pg> If you don't havr an SMTP Server, grab the tiny one that Max put us on to a while ago: miniRelay. You can download it from here (site in Spanish): http://www.netvicious.com/miniRelay/miniRelay.zip English documentation and FAQ here (but download link is out of date) http://www.blat.net/miniRelay/ -- Stuart On 3 Nov 2009 at 0:18, Mark Breen wrote: > > *Scheduled Tasks* > Finally, once last thing to mention, if you have a smtp server, it is so, so > each to create scheduled tasks, that run at predefined times and then mail > you when they are finished. Running the heavy jobs nightly, can take a lot > of the hard work from your daily routines. Again you can manage all this > with very little actually code and have working routines in minutes. > From jwcolby at colbyconsulting.com Tue Nov 3 21:12:23 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Tue, 03 Nov 2009 22:12:23 -0500 Subject: [dba-VB] How do you run SQL Server processes Message-ID: <4AF0F117.20209@colbyconsulting.com> I have processes that I run which use sets of stored procedures. For example two of the processes export / import records from large tables for address validation. The addresses need to be sorted in Zip5/Zip4/Address order in order to make the external validation process as fast as possible so I create a table that pulls millions of records, often tens of millions of records, into a table (created on-the-fly) in sorted order, then pulls out 2 million records at a time and BCPs them out for processing in an external program. The inverse occurs when the records finish processing, 2 million record chunks are imported back into a temp table, then each 2 million record chunk is appended to a main table. Once back in SQL Server I have processes that build clustered and non-clustered indexes, run sha-1 hash functions to generate hash keys, update address valid codes etc. I will run this entire project at least once a month (to keep the addresses valid as people move), and I do so for many tables - a table of about 65 million addresses, another of 21 million, another of 11 million etc. I store all of my stored procedures in a dedicated database that pretty much just contains the SPs and a few UDFs as well as a couple of logging type tables. At the moment I use an Access database to run the individual stored procedures in specific order. I am just embarking on a project to move that functioning but not very efficient Access database to C#. I want to be able to have better monitoring of SP completion, records processed (counts), time to complete each SP etc. I found a pretty nice set of C# classes that allow creation of "stored procedure" objects, with parameter objects, then stored into a collection as the SPs (class instances) are assembled and finally the whole collection of SPs executed. I created a test database of a million names / addresses and today I started testing my existing SPs on this small data set. In the next few days I expect to build out a preliminary "port" of the existing Access VBA code. I am hoping to eventually make the entire process "table driven" where I can store (in a table) the name of the sp, the parameters in a child table etc. and then have a supervisor pull out the records and execute the SPs based on what is in the table. This would allow me to create the SPs and then use a C# application (data entry forms) to organize the SPs, add / delete SPs from the process and so forth. That is down the road of course, for now I will have hard coded sequences - as I already do in supervisor SPs as well as in hard coded Access VBA modules. As a programmer, doing this supervisor level in C# feels more natural to me (than doing it inside of SQL Server), and gives me the flexibility to do what I want in code external to the SQL Server database being manipulated. I want to be able to do things external to the database processes, things like monitor the directories that the BCP processes export to / from, move files popping out of a bcp process off to a VM on a different server, monitor those VMs for files coming out of their processes, moving the files back over to the SQL Server directories, and trigger BCP processes that import the data back in to SQL server. I need to be able to FTP order and data files to clients, and get (FTP) files from process bureaus. So it FEELS to me like using a language such as C# to build a system outside of SQL Server is the way to go. BTW I stumbled across something called SMO (SQL Server Management Objects) which makes it really easy to get at the SQL Server object model from C#. I can apparently see the entire tree structure that is a server/database/tables/etc. and do I know not what (yet) with those objects. Looks pretty cool though. The database is a large part of what I do but it is not the whole picture and I have been handcuffed by my lack of ability to effectively do these external parts. But I am a programmer at heart. I want to know how you do this kind of stuff. I know that running such "systems" of stored procedures tied to external processes must be quite common and I am wondering what you guys do for supervising your processes. If anyone doing anything remotely similar would take the time to explain how you do it, I would be most appreciative. Thanks, -- John W. Colby www.ColbyConsulting.com From raibeart at gmail.com Wed Nov 4 12:37:58 2009 From: raibeart at gmail.com (Robert Stewart) Date: Wed, 04 Nov 2009 12:37:58 -0600 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: References: Message-ID: <4af1ca07.0637560a.20ef.1362@mx.google.com> Sorry, but using C# to do what you would and could do simply using SQL Server is adding a middle layer you do not need. You can do exactly the same thing using SQL Server and stored procedures and writing to a log table about what the process is doing and keeping track of it that way. And, not have to use a middle layer language like C# to do it. I am not sure why writing T-SQL code is not like real programming to you. But, it is. And, if you are going to continue using SQL Server, you need to learn what it can do for you instead of always fighting learning it. This is coming from someone that does T-SQL, C#, VB.Net and Access VBA. So, I think I have a fair perspective of how they all work. At 12:00 PM 11/4/2009, you wrote: >Date: Tue, 03 Nov 2009 22:12:23 -0500 >From: jwcolby >Subject: [dba-VB] How do you run SQL Server processes >To: VBA , Dba-Sqlserver > >Message-ID: <4AF0F117.20209 at colbyconsulting.com> >Content-Type: text/plain; charset=ISO-8859-1; format=flowed > >I have processes that I run which use sets of stored procedures. > >For example two of the processes export / import records from large >tables for address validation. >The addresses need to be sorted in Zip5/Zip4/Address order in order >to make the external validation >process as fast as possible so I create a table that pulls millions >of records, often tens of >millions of records, into a table (created on-the-fly) in sorted >order, then pulls out 2 million >records at a time and BCPs them out for processing in an external program. > >The inverse occurs when the records finish processing, 2 million >record chunks are imported back >into a temp table, then each 2 million record chunk is appended to a >main table. Once back in SQL >Server I have processes that build clustered and non-clustered >indexes, run sha-1 hash functions to >generate hash keys, update address valid codes etc. I will run this >entire project at least once a >month (to keep the addresses valid as people move), and I do so for >many tables - a table of about >65 million addresses, another of 21 million, another of 11 million etc. > >I store all of my stored procedures in a dedicated database that >pretty much just contains the SPs >and a few UDFs as well as a couple of logging type tables. At the >moment I use an Access database >to run the individual stored procedures in specific order. > >I am just embarking on a project to move that functioning but not >very efficient Access database to >C#. I want to be able to have better monitoring of SP completion, >records processed (counts), time >to complete each SP etc. I found a pretty nice set of C# classes >that allow creation of "stored >procedure" objects, with parameter objects, then stored into a >collection as the SPs (class >instances) are assembled and finally the whole collection of SPs >executed. I created a test >database of a million names / addresses and today I started testing >my existing SPs on this small >data set. In the next few days I expect to build out a preliminary >"port" of the existing Access >VBA code. > >I am hoping to eventually make the entire process "table driven" >where I can store (in a table) the >name of the sp, the parameters in a child table etc. and then have a >supervisor pull out the records >and execute the SPs based on what is in the table. This would allow >me to create the SPs and then >use a C# application (data entry forms) to organize the SPs, add / >delete SPs from the process and >so forth. > >That is down the road of course, for now I will have hard coded >sequences - as I already do in >supervisor SPs as well as in hard coded Access VBA modules. > >As a programmer, doing this supervisor level in C# feels more >natural to me (than doing it inside of >SQL Server), and gives me the flexibility to do what I want in code >external to the SQL Server >database being manipulated. I want to be able to do things external >to the database processes, >things like monitor the directories that the BCP processes export to >/ from, move files popping out >of a bcp process off to a VM on a different server, monitor those >VMs for files coming out of their >processes, moving the files back over to the SQL Server directories, >and trigger BCP processes that >import the data back in to SQL server. I need to be able to FTP >order and data files to clients, >and get (FTP) files from process bureaus. So it FEELS to me like >using a language such as C# to >build a system outside of SQL Server is the way to go. > >BTW I stumbled across something called SMO (SQL Server Management >Objects) which makes it really >easy to get at the SQL Server object model from C#. I can >apparently see the entire tree structure >that is a server/database/tables/etc. and do I know not what (yet) >with those objects. Looks pretty >cool though. > >The database is a large part of what I do but it is not the whole >picture and I have been handcuffed >by my lack of ability to effectively do these external parts. But I >am a programmer at heart. I >want to know how you do this kind of stuff. > >I know that running such "systems" of stored procedures tied to >external processes must be quite >common and I am wondering what you guys do for supervising your >processes. If anyone doing anything >remotely similar would take the time to explain how you do it, I >would be most appreciative. > >Thanks, >-- >John W. Colby >www.ColbyConsulting.com > > >------------------------------ > >_______________________________________________ >dba-VB mailing list >dba-VB at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-vb > > >End of dba-VB Digest, Vol 73, Issue 3 >************************************* From jwcolby at colbyconsulting.com Wed Nov 4 15:11:58 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 04 Nov 2009 16:11:58 -0500 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <4af1ca07.0637560a.20ef.1362@mx.google.com> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> Message-ID: <4AF1EE1E.9030406@colbyconsulting.com> Robert, I don't really want to get in an argument about this. I asked a serious question and got a somewhat brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY answer I have received so that is not particularly encouraging. 8( TSQL, at least as written from inside of SSMS is about like writing GBasic code back in 1982. Well, maybe a tiny step up but not much. You are right, I probably just don't know how, but in what I will call "a real language" I can set breakpoints, I can see a call stack, I have watch statements, I have... simple very basic DEBUG capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in SSMS). I can have forms with controls displaying data, buttons that cause things to happen. I can FTP files, monitor directories for activities. Believe me, if this exists in SSMS I will jump on that! Just don't tell me to go spend thousands to buy a tool that will do some of what C# can already do for me. Modern languages, and the .Net environment in particular have capabilities that (AFAICT) TSQL can not even dream of. TSQL dreams of manipulating data, C# dreams of manipulating ... the world. Robert I am not "fighting learning it", I am light years beyond where I was two years ago, and I learn more TSQL and SQL Server every day. However from what I have seen so far, TSQL will NEVER be what I need for the supervisor. And answers like you just provided aren't really much help. I am a consultant. I am a programmer, not a DBA, or a SQL Server administrator, or a SQL Server jock at a company somewhere. I need C# for automating my business, which is NOT being a DBA (I already said that I think). I will write C# applications for clients that use SQL Server, and others that do not use SQL Server. SQL Server is used by exactly ONE of my clients, so I cannot spend the time to do whatever it is that you envision SQL Server doing because I cannot amortize the learning over other clients. Now... if you want to answer my question, I am all ears. The first thing to answer (If I am to do all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. Let's take a concrete, real world example. I have a set of virtual machines which run a third party application. I place files into an "inbox" directory, and I wait for files to pop out of the running application into the "outbox" directory. Just an aside, the files placed into the inbox are created by a set of about 5 stored procedures, which can operate on any of dozens of databases in my server. It is the case that the third party application occasionally hangs. If it does I need to rename a file in a specific directory to cause that application to restart processing that file. It is my intention to use C# to execute the SPs to create the files, move the files into the VM, then monitor files dropped into the inbox, and files popping out into the outbox. Look for "no new files in X minutes" and if that condition exists, rename the file. Log files dropped into the in-box, files popping out into the out-box, and the time it took for the files to process through the application. Log any "hangs" restarted. Supervisor stuff. Explain to me how (and more importantly WHY) I would use TSQL to do that? Other than a "when TSQL is all you have" (which is NOT true for me) mentality. A SMALL part of my business is to perform some moderately complex processes using SQL Server and SPs as a tool. I envision C# as a supervisor, not a "middle layer". I envision an application (written in C#) that runs on my desktop which starts, then monitors SQL Server processes, displays errors, restarts processes if necessary, creates databases, tables etc, backs them up and detaches them when done, logging everything that happens, showing it all to me me in real time in a Windows application. Possibly even automatically generating invoices for the work done and emailing them to the client. Explain to me one more time (with feeling) how (and more importantly WHY) I would use TSQL to do all of that stuff? John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > Sorry, but using C# to do what you would and could do simply using > SQL Server is adding a middle layer you do not need. You can do > exactly the same thing using SQL Server and stored procedures and > writing to a log table about what the process is doing and keeping > track of it that way. And, not have to use a middle layer language > like C# to do it. > > I am not sure why writing T-SQL code is not like real programming to > you. But, it is. And, if you are going to continue using SQL > Server, you need to learn what it can do for you instead of always > fighting learning it. > > This is coming from someone that does T-SQL, C#, VB.Net and Access > VBA. So, I think I have a fair perspective of how they all work. From Gustav at cactus.dk Wed Nov 4 16:21:22 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 04 Nov 2009 23:21:22 +0100 Subject: [dba-VB] How do you run SQL Server processes Message-ID: Hi John I have no answer to you but if it can bring you some relief, I feel much the same. The database is for storing data, a business layer and/or a front end is for the logic and interface. That said, nothing is perfect, and in the real world you will have to write some T-SQL or similar but I try to keep it at minimum, first because of the trouble of maintenance (for example, no version control I know of), second because I feel old when I write "GBasic style" code. And I'm not in a big corporation where I can get every expensive tool I can point at, and tools for SQL Server seem all to be enterprise-priced. So, writing views and T-SQL is because I have to, writing C# (and VBA for that matter) is because I like to. I guess Robert is a very skilled DBA and can write T-SQL with his left foot and that's very good - then we know where to ask for advice as I'll never be more than a decent low-level DBA. /gustav >>> jwcolby at colbyconsulting.com 04-11-2009 22:11 >>> Robert, I don't really want to get in an argument about this. I asked a serious question and got a somewhat brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY answer I have received so that is not particularly encouraging. 8( TSQL, at least as written from inside of SSMS is about like writing GBasic code back in 1982. Well, maybe a tiny step up but not much. You are right, I probably just don't know how, but in what I will call "a real language" I can set breakpoints, I can see a call stack, I have watch statements, I have... simple very basic DEBUG capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in SSMS). I can have forms with controls displaying data, buttons that cause things to happen. I can FTP files, monitor directories for activities. Believe me, if this exists in SSMS I will jump on that! Just don't tell me to go spend thousands to buy a tool that will do some of what C# can already do for me. Modern languages, and the .Net environment in particular have capabilities that (AFAICT) TSQL can not even dream of. TSQL dreams of manipulating data, C# dreams of manipulating ... the world. Robert I am not "fighting learning it", I am light years beyond where I was two years ago, and I learn more TSQL and SQL Server every day. However from what I have seen so far, TSQL will NEVER be what I need for the supervisor. And answers like you just provided aren't really much help. I am a consultant. I am a programmer, not a DBA, or a SQL Server administrator, or a SQL Server jock at a company somewhere. I need C# for automating my business, which is NOT being a DBA (I already said that I think). I will write C# applications for clients that use SQL Server, and others that do not use SQL Server. SQL Server is used by exactly ONE of my clients, so I cannot spend the time to do whatever it is that you envision SQL Server doing because I cannot amortize the learning over other clients. Now... if you want to answer my question, I am all ears. The first thing to answer (If I am to do all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. Let's take a concrete, real world example. I have a set of virtual machines which run a third party application. I place files into an "inbox" directory, and I wait for files to pop out of the running application into the "outbox" directory. Just an aside, the files placed into the inbox are created by a set of about 5 stored procedures, which can operate on any of dozens of databases in my server. It is the case that the third party application occasionally hangs. If it does I need to rename a file in a specific directory to cause that application to restart processing that file. It is my intention to use C# to execute the SPs to create the files, move the files into the VM, then monitor files dropped into the inbox, and files popping out into the outbox. Look for "no new files in X minutes" and if that condition exists, rename the file. Log files dropped into the in-box, files popping out into the out-box, and the time it took for the files to process through the application. Log any "hangs" restarted. Supervisor stuff. Explain to me how (and more importantly WHY) I would use TSQL to do that? Other than a "when TSQL is all you have" (which is NOT true for me) mentality. A SMALL part of my business is to perform some moderately complex processes using SQL Server and SPs as a tool. I envision C# as a supervisor, not a "middle layer". I envision an application (written in C#) that runs on my desktop which starts, then monitors SQL Server processes, displays errors, restarts processes if necessary, creates databases, tables etc, backs them up and detaches them when done, logging everything that happens, showing it all to me me in real time in a Windows application. Possibly even automatically generating invoices for the work done and emailing them to the client. Explain to me one more time (with feeling) how (and more importantly WHY) I would use TSQL to do all of that stuff? John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > Sorry, but using C# to do what you would and could do simply using > SQL Server is adding a middle layer you do not need. You can do > exactly the same thing using SQL Server and stored procedures and > writing to a log table about what the process is doing and keeping > track of it that way. And, not have to use a middle layer language > like C# to do it. > > I am not sure why writing T-SQL code is not like real programming to > you. But, it is. And, if you are going to continue using SQL > Server, you need to learn what it can do for you instead of always > fighting learning it. > > This is coming from someone that does T-SQL, C#, VB.Net and Access > VBA. So, I think I have a fair perspective of how they all work. _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Wed Nov 4 16:48:32 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 01:48:32 +0300 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <4AF1EE1E.9030406@colbyconsulting.com> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> <4AF1EE1E.9030406@colbyconsulting.com> Message-ID: <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> Hi John, <<< The first thing to answer (If I am to do all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. >>> Overview of T-SQL and CLR debugging in SQL Server 2005 http://blogs.msdn.com/sqlclr/archive/2006/06/29/651644.aspx How to: Enable SQL Server 2005 Debugging http://msdn.microsoft.com/en-us/library/s0fk6z6e(VS.80).aspx HTH, --Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 12:12 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes Robert, I don't really want to get in an argument about this. I asked a serious question and got a somewhat brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY answer I have received so that is not particularly encouraging. 8( TSQL, at least as written from inside of SSMS is about like writing GBasic code back in 1982. Well, maybe a tiny step up but not much. You are right, I probably just don't know how, but in what I will call "a real language" I can set breakpoints, I can see a call stack, I have watch statements, I have... simple very basic DEBUG capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in SSMS). I can have forms with controls displaying data, buttons that cause things to happen. I can FTP files, monitor directories for activities. Believe me, if this exists in SSMS I will jump on that! Just don't tell me to go spend thousands to buy a tool that will do some of what C# can already do for me. Modern languages, and the .Net environment in particular have capabilities that (AFAICT) TSQL can not even dream of. TSQL dreams of manipulating data, C# dreams of manipulating ... the world. Robert I am not "fighting learning it", I am light years beyond where I was two years ago, and I learn more TSQL and SQL Server every day. However from what I have seen so far, TSQL will NEVER be what I need for the supervisor. And answers like you just provided aren't really much help. I am a consultant. I am a programmer, not a DBA, or a SQL Server administrator, or a SQL Server jock at a company somewhere. I need C# for automating my business, which is NOT being a DBA (I already said that I think). I will write C# applications for clients that use SQL Server, and others that do not use SQL Server. SQL Server is used by exactly ONE of my clients, so I cannot spend the time to do whatever it is that you envision SQL Server doing because I cannot amortize the learning over other clients. Now... if you want to answer my question, I am all ears. The first thing to answer (If I am to do all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. Let's take a concrete, real world example. I have a set of virtual machines which run a third party application. I place files into an "inbox" directory, and I wait for files to pop out of the running application into the "outbox" directory. Just an aside, the files placed into the inbox are created by a set of about 5 stored procedures, which can operate on any of dozens of databases in my server. It is the case that the third party application occasionally hangs. If it does I need to rename a file in a specific directory to cause that application to restart processing that file. It is my intention to use C# to execute the SPs to create the files, move the files into the VM, then monitor files dropped into the inbox, and files popping out into the outbox. Look for "no new files in X minutes" and if that condition exists, rename the file. Log files dropped into the in-box, files popping out into the out-box, and the time it took for the files to process through the application. Log any "hangs" restarted. Supervisor stuff. Explain to me how (and more importantly WHY) I would use TSQL to do that? Other than a "when TSQL is all you have" (which is NOT true for me) mentality. A SMALL part of my business is to perform some moderately complex processes using SQL Server and SPs as a tool. I envision C# as a supervisor, not a "middle layer". I envision an application (written in C#) that runs on my desktop which starts, then monitors SQL Server processes, displays errors, restarts processes if necessary, creates databases, tables etc, backs them up and detaches them when done, logging everything that happens, showing it all to me me in real time in a Windows application. Possibly even automatically generating invoices for the work done and emailing them to the client. Explain to me one more time (with feeling) how (and more importantly WHY) I would use TSQL to do all of that stuff? John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > Sorry, but using C# to do what you would and could do simply using > SQL Server is adding a middle layer you do not need. You can do > exactly the same thing using SQL Server and stored procedures and > writing to a log table about what the process is doing and keeping > track of it that way. And, not have to use a middle layer language > like C# to do it. > > I am not sure why writing T-SQL code is not like real programming to > you. But, it is. And, if you are going to continue using SQL > Server, you need to learn what it can do for you instead of always > fighting learning it. > > This is coming from someone that does T-SQL, C#, VB.Net and Access > VBA. So, I think I have a fair perspective of how they all work. _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4574 (20091104) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4574 (20091104) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Wed Nov 4 17:01:33 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 04 Nov 2009 18:01:33 -0500 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: References: Message-ID: <4AF207CD.2060307@colbyconsulting.com> Gustav, >as I'll never be more than a decent low-level DBA. Likewise. And that is precisely my problem. I think when you spend all day in TSQL it is easy to lose sight of just how crude the environment is. I have finally gotten to the point where I can write a SP, debug it, get it functioning etc but it is just hard. Not that C# isn't hard, but C# is a general purpose language where the effort spent is useful for other things. And with the debugging tools in .Net, debugging things is really pretty easy, very similar to many other environments. I can go weeks without doing SQL development and when I go back it is a PITA to pick back up. What it does, it does well, what it doesn't do... it doesn't do quite spectacularly. > I guess Robert is a very skilled DBA and can write T-SQL with his left foot It certainly seems so. But with my current understanding of SQL Server and TSQL I cannot imagine doing the big picture of my work strictly with TSQL. With my current understanding, TSQL will only ever be a hand full of processes, 30% of the whole job. Don't get me wrong, there is no alternative to the power of SQL Server, but in the end it is just a place to store data and some tools for getting it out and back in to tables. Everything else is out in the real world. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > I have no answer to you but if it can bring you some relief, I feel much the same. The database is for storing data, a business layer and/or a front end is for the logic and interface. > That said, nothing is perfect, and in the real world you will have to write some T-SQL or similar but I try to keep it at minimum, first because of the trouble of maintenance (for example, no version control I know of), second because I feel old when I write "GBasic style" code. And I'm not in a big corporation where I can get every expensive tool I can point at, and tools for SQL Server seem all to be enterprise-priced. So, writing views and T-SQL is because I have to, writing C# (and VBA for that matter) is because I like to. > > I guess Robert is a very skilled DBA and can write T-SQL with his left foot and that's very good - then we know where to ask for advice as I'll never be more than a decent low-level DBA. > > /gustav > > >>>> jwcolby at colbyconsulting.com 04-11-2009 22:11 >>> > Robert, > > I don't really want to get in an argument about this. I asked a serious question and got a somewhat > brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY answer I have received > so that is not particularly encouraging. > > 8( > > TSQL, at least as written from inside of SSMS is about like writing GBasic code back in 1982. Well, > maybe a tiny step up but not much. > > You are right, I probably just don't know how, but in what I will call "a real language" I can set > breakpoints, I can see a call stack, I have watch statements, I have... simple very basic DEBUG > capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in SSMS). I can have forms > with controls displaying data, buttons that cause things to happen. I can FTP files, monitor > directories for activities. > > Believe me, if this exists in SSMS I will jump on that! Just don't tell me to go spend thousands to > buy a tool that will do some of what C# can already do for me. > > Modern languages, and the .Net environment in particular have capabilities that (AFAICT) TSQL can > not even dream of. TSQL dreams of manipulating data, C# dreams of manipulating ... the world. > > Robert I am not "fighting learning it", I am light years beyond where I was two years ago, and I > learn more TSQL and SQL Server every day. However from what I have seen so far, TSQL will NEVER be > what I need for the supervisor. And answers like you just provided aren't really much help. > > I am a consultant. I am a programmer, not a DBA, or a SQL Server administrator, or a SQL Server > jock at a company somewhere. I need C# for automating my business, which is NOT being a DBA (I > already said that I think). I will write C# applications for clients that use SQL Server, and > others that do not use SQL Server. SQL Server is used by exactly ONE of my clients, so I cannot > spend the time to do whatever it is that you envision SQL Server doing because I cannot amortize the > learning over other clients. > > Now... if you want to answer my question, I am all ears. The first thing to answer (If I am to do > all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. > > Let's take a concrete, real world example. I have a set of virtual machines which run a third party > application. I place files into an "inbox" directory, and I wait for files to pop out of the > running application into the "outbox" directory. Just an aside, the files placed into the inbox are > created by a set of about 5 stored procedures, which can operate on any of dozens of databases in my > server. > > It is the case that the third party application occasionally hangs. If it does I need to rename a > file in a specific directory to cause that application to restart processing that file. > > It is my intention to use C# to execute the SPs to create the files, move the files into the VM, > then monitor files dropped into the inbox, and files popping out into the outbox. Look for "no new > files in X minutes" and if that condition exists, rename the file. Log files dropped into the > in-box, files popping out into the out-box, and the time it took for the files to process through > the application. Log any "hangs" restarted. Supervisor stuff. > > Explain to me how (and more importantly WHY) I would use TSQL to do that? Other than a "when TSQL > is all you have" (which is NOT true for me) mentality. > > A SMALL part of my business is to perform some moderately complex processes using SQL Server and SPs > as a tool. I envision C# as a supervisor, not a "middle layer". I envision an application (written > in C#) that runs on my desktop which starts, then monitors SQL Server processes, displays errors, > restarts processes if necessary, creates databases, tables etc, backs them up and detaches them when > done, logging everything that happens, showing it all to me me in real time in a Windows > application. Possibly even automatically generating invoices for the work done and emailing them to > the client. > > Explain to me one more time (with feeling) how (and more importantly WHY) I would use TSQL to do all > of that stuff? > > John W. Colby > www.ColbyConsulting.com > > > Robert Stewart wrote: >> Sorry, but using C# to do what you would and could do simply using >> SQL Server is adding a middle layer you do not need. You can do >> exactly the same thing using SQL Server and stored procedures and >> writing to a log table about what the process is doing and keeping >> track of it that way. And, not have to use a middle layer language >> like C# to do it. >> >> I am not sure why writing T-SQL code is not like real programming to >> you. But, it is. And, if you are going to continue using SQL >> Server, you need to learn what it can do for you instead of always >> fighting learning it. >> >> This is coming from someone that does T-SQL, C#, VB.Net and Access >> VBA. So, I think I have a fair perspective of how they all work. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From davidmcafee at gmail.com Wed Nov 4 17:06:37 2009 From: davidmcafee at gmail.com (David McAfee) Date: Wed, 4 Nov 2009 15:06:37 -0800 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> <4AF1EE1E.9030406@colbyconsulting.com> <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> Message-ID: <8786a4c00911041506o49c88352kee08cbbb64fd62b@mail.gmail.com> In SQL 2000 (QA) you would simply right click on the Sproc in the Object explorer and choose debug. Early beta versions of SSMS 2005 had the same ability (right click and choose "step into") but it was removed and moved to Visual Studio instead. :( On Wed, Nov 4, 2009 at 2:48 PM, Shamil Salakhetdinov wrote: > Hi John, > > <<< > The first thing to answer (If I am to do > all this in TSQL) is how do I get a modern tool with all > of the debug capabilities to write TSQL code. >>>> > > Overview of T-SQL and CLR debugging in SQL Server 2005 > http://blogs.msdn.com/sqlclr/archive/2006/06/29/651644.aspx > > How to: Enable SQL Server 2005 Debugging > http://msdn.microsoft.com/en-us/library/s0fk6z6e(VS.80).aspx > > HTH, > > --Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 12:12 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes > > Robert, > > I don't really want to get in an argument about this. ?I asked a serious > question and got a somewhat > brusque "answer" that isn't an answer at all. ?OTOH it is (so far) the ONLY > answer I have received > so that is not particularly encouraging. > > 8( > > TSQL, at least as written from inside of SSMS is about like writing GBasic > code back in 1982. ?Well, > maybe a tiny step up but not much. > > You are right, I probably just don't know how, but in what I will call "a > real language" I can set > breakpoints, I can see a call stack, I have watch statements, ?I have... > simple very basic DEBUG > capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in > SSMS). ?I can have forms > with controls displaying data, buttons that cause things to happen. ?I can > FTP files, monitor > directories for activities. > > Believe me, if this exists in SSMS I will jump on that! ?Just don't tell me > to go spend thousands to > buy a tool that will do some of what C# can already do for me. > > Modern languages, and the .Net environment in particular have capabilities > that (AFAICT) TSQL can > not even dream of. ?TSQL dreams of manipulating data, C# dreams of > manipulating ... the world. > > Robert I am not "fighting learning it", I am light years beyond where I was > two years ago, and I > learn more TSQL and SQL Server every day. ?However from what I have seen so > far, TSQL will NEVER be > what I need for the supervisor. ?And answers like you just provided aren't > really much help. > > I am a consultant. ?I am a programmer, not a DBA, or a SQL Server > administrator, or a SQL Server > jock at a company somewhere. ?I need C# for automating my business, which is > NOT being a DBA (I > already said that I think). ?I will write C# applications for clients that > use SQL Server, and > others that do not use SQL Server. ?SQL Server is used by exactly ONE of my > clients, so I cannot > spend the time to do whatever it is that you envision SQL Server doing > because I cannot amortize the > learning over other clients. > > Now... if you want to answer my question, I am all ears. ?The first thing to > answer (If I am to do > all this in TSQL) is how do I get a modern tool with all of the debug > capabilities to write TSQL code. > > Let's take a concrete, real world example. ?I have a set of virtual machines > which run a third party > application. ?I place files into an "inbox" directory, and I wait for files > to pop out of the > running application into the "outbox" directory. ?Just an aside, the files > placed into the inbox are > created by a set of about 5 stored procedures, which can operate on any of > dozens of databases in my > server. > > It is the case that the third party application occasionally hangs. ?If it > does I need to rename a > file in a specific directory to cause that application to restart processing > that file. > > It is my intention to use C# to execute the SPs to create the files, move > the files into the VM, > then monitor files dropped into the inbox, and files popping out into the > outbox. ?Look for "no new > files in X minutes" and if that condition exists, rename the file. ?Log > files dropped into the > in-box, files popping out into the out-box, and the time it took for the > files to process through > the application. ?Log any "hangs" restarted. ?Supervisor stuff. > > Explain to me how (and more importantly WHY) I would use TSQL to do that? > Other than a "when TSQL > is all you have" (which is NOT true for me) mentality. > > A SMALL part of my business is to perform some moderately complex processes > using SQL Server and SPs > as a tool. ?I envision C# as a supervisor, not a "middle layer". ?I envision > an application (written > in C#) that runs on my desktop which starts, then monitors SQL Server > processes, displays errors, > restarts processes if necessary, creates databases, tables etc, backs them > up and detaches them when > done, logging everything that happens, showing it all to me me in real time > in a Windows > application. ?Possibly even automatically generating invoices for the work > done and emailing them to > the client. > > Explain to me one more time (with feeling) how (and more importantly WHY) I > would use TSQL to do all > of that stuff? > > John W. Colby > www.ColbyConsulting.com > > > Robert Stewart wrote: >> Sorry, but using C# to do what you would and could do simply using >> SQL Server is adding a middle layer you do not need. You can do >> exactly the same thing using SQL Server and stored procedures and >> writing to a log table about what the process is doing and keeping >> track of it that way. And, not have to use a middle layer language >> like C# to do it. >> >> I am not sure why writing T-SQL code is not like real programming to >> you. ?But, it is. ?And, if you are going to continue using SQL >> Server, you need to learn what it can do for you instead of always >> fighting learning it. >> >> This is coming from someone that does T-SQL, C#, VB.Net and Access >> VBA. ?So, I think I have a fair perspective of how they all work. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Wed Nov 4 17:08:49 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 04 Nov 2009 18:08:49 -0500 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> <4AF1EE1E.9030406@colbyconsulting.com> <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> Message-ID: <4AF20981.3060804@colbyconsulting.com> Thanks for that. I guess I will need to install my professional edition license to go there. Worthwhile though I am guessing. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > <<< > The first thing to answer (If I am to do > all this in TSQL) is how do I get a modern tool with all > of the debug capabilities to write TSQL code. > > Overview of T-SQL and CLR debugging in SQL Server 2005 > http://blogs.msdn.com/sqlclr/archive/2006/06/29/651644.aspx > > How to: Enable SQL Server 2005 Debugging > http://msdn.microsoft.com/en-us/library/s0fk6z6e(VS.80).aspx > > HTH, > > --Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 12:12 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes > > Robert, > > I don't really want to get in an argument about this. I asked a serious > question and got a somewhat > brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY > answer I have received > so that is not particularly encouraging. > > 8( > > TSQL, at least as written from inside of SSMS is about like writing GBasic > code back in 1982. Well, > maybe a tiny step up but not much. > > You are right, I probably just don't know how, but in what I will call "a > real language" I can set > breakpoints, I can see a call stack, I have watch statements, I have... > simple very basic DEBUG > capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in > SSMS). I can have forms > with controls displaying data, buttons that cause things to happen. I can > FTP files, monitor > directories for activities. > > Believe me, if this exists in SSMS I will jump on that! Just don't tell me > to go spend thousands to > buy a tool that will do some of what C# can already do for me. > > Modern languages, and the .Net environment in particular have capabilities > that (AFAICT) TSQL can > not even dream of. TSQL dreams of manipulating data, C# dreams of > manipulating ... the world. > > Robert I am not "fighting learning it", I am light years beyond where I was > two years ago, and I > learn more TSQL and SQL Server every day. However from what I have seen so > far, TSQL will NEVER be > what I need for the supervisor. And answers like you just provided aren't > really much help. > > I am a consultant. I am a programmer, not a DBA, or a SQL Server > administrator, or a SQL Server > jock at a company somewhere. I need C# for automating my business, which is > NOT being a DBA (I > already said that I think). I will write C# applications for clients that > use SQL Server, and > others that do not use SQL Server. SQL Server is used by exactly ONE of my > clients, so I cannot > spend the time to do whatever it is that you envision SQL Server doing > because I cannot amortize the > learning over other clients. > > Now... if you want to answer my question, I am all ears. The first thing to > answer (If I am to do > all this in TSQL) is how do I get a modern tool with all of the debug > capabilities to write TSQL code. > > Let's take a concrete, real world example. I have a set of virtual machines > which run a third party > application. I place files into an "inbox" directory, and I wait for files > to pop out of the > running application into the "outbox" directory. Just an aside, the files > placed into the inbox are > created by a set of about 5 stored procedures, which can operate on any of > dozens of databases in my > server. > > It is the case that the third party application occasionally hangs. If it > does I need to rename a > file in a specific directory to cause that application to restart processing > that file. > > It is my intention to use C# to execute the SPs to create the files, move > the files into the VM, > then monitor files dropped into the inbox, and files popping out into the > outbox. Look for "no new > files in X minutes" and if that condition exists, rename the file. Log > files dropped into the > in-box, files popping out into the out-box, and the time it took for the > files to process through > the application. Log any "hangs" restarted. Supervisor stuff. > > Explain to me how (and more importantly WHY) I would use TSQL to do that? > Other than a "when TSQL > is all you have" (which is NOT true for me) mentality. > > A SMALL part of my business is to perform some moderately complex processes > using SQL Server and SPs > as a tool. I envision C# as a supervisor, not a "middle layer". I envision > an application (written > in C#) that runs on my desktop which starts, then monitors SQL Server > processes, displays errors, > restarts processes if necessary, creates databases, tables etc, backs them > up and detaches them when > done, logging everything that happens, showing it all to me me in real time > in a Windows > application. Possibly even automatically generating invoices for the work > done and emailing them to > the client. > > Explain to me one more time (with feeling) how (and more importantly WHY) I > would use TSQL to do all > of that stuff? > > John W. Colby > www.ColbyConsulting.com > > > Robert Stewart wrote: >> Sorry, but using C# to do what you would and could do simply using >> SQL Server is adding a middle layer you do not need. You can do >> exactly the same thing using SQL Server and stored procedures and >> writing to a log table about what the process is doing and keeping >> track of it that way. And, not have to use a middle layer language >> like C# to do it. >> >> I am not sure why writing T-SQL code is not like real programming to >> you. But, it is. And, if you are going to continue using SQL >> Server, you need to learn what it can do for you instead of always >> fighting learning it. >> >> This is coming from someone that does T-SQL, C#, VB.Net and Access >> VBA. So, I think I have a fair perspective of how they all work. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From marklbreen at gmail.com Thu Nov 5 01:43:49 2009 From: marklbreen at gmail.com (Mark Breen) Date: Thu, 5 Nov 2009 15:43:49 +0800 Subject: [dba-VB] Fwd: How do you run SQL Server processes In-Reply-To: References: <4AF207CD.2060307@colbyconsulting.com> Message-ID: Hello John, FWIW, I agree with you that you need a rich environment to do the work that you plan to do. However, I think that Robert is correct in suggestion to try, as much as possible to not cross the technology stack with volumes of data when you do not have to. IOW, do all the heavy lifting in the database, and by all means return as many parameters or log results back to the front end you are building in C#, but try to avoid returning your data when ever possible. I guess I was suggesting the same with with SSIS. Where you have to export data based on decisions, SSIS is an extremely fast option. Have you already used EXECUTE command to call an sproc within an sproc? This is a useful means of creating an sproc, that running a set of sprocs, and can insert / update a log table as it progresses. For data only actvity, this might be effective. For all the front end work however, I guess you will make C# sing, like only you can, so work away, and let TSQL do what it is good at, all the rest should not involve enormous volumes of data, so the technology bridge will be irrelevant. Good luck, you know you are lucky. Most Database people never get to read about a db as big as what you have :) On last thing, in SQL Server, Cursors are a dirty word, I never see them used. However, when you have low volumes of iteration (thousands or a few tens of thousands), cursors are a very powerful way of enumerating tasks. Eg, lets imagine you had a table of states, and you needed to perform ten seperate sprocs for each state. You could use a cursor to enumerate the states table and pass in the state name to the ten different sprocs. You could log between each iteration, and you could have the whole thing running in 30 minutes. In this sort of context, I agree with Robert, that TSQL can give you Raw power quickly and easily. C# could be watching your log table and your 'Status Update' table and simply returning 1 or 2 records from the db to C# per minute, whereas within your sproc that is enumerating it might be handling millions of records each two minutes. Those millions would never cross the stack divide of DB Server to .Net. Again this sharing of processing power is probably what Robert was referring to. BTW I recently started using FileZilla and find it super. Good Luck, Mark Mark From shamil at smsconsulting.spb.ru Thu Nov 5 07:01:18 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 16:01:18 +0300 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <4AF20981.3060804@colbyconsulting.com> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> <4AF1EE1E.9030406@colbyconsulting.com> <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> <4AF20981.3060804@colbyconsulting.com> Message-ID: <00da01ca5e18$119f5080$34ddf180$@spb.ru> Hi John, I just posted links as you asked about this feature but I must note I have used T-SQL debugging just once when I used T-SQL cursors (and that was my mistake I suppose) - IOW I doubt that cursors/T-SQL debugging will be ever needed for you as you plan to use C# to drive your T-SQL's SPs... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 2:09 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes Thanks for that. I guess I will need to install my professional edition license to go there. Worthwhile though I am guessing. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > <<< > The first thing to answer (If I am to do > all this in TSQL) is how do I get a modern tool with all > of the debug capabilities to write TSQL code. > > Overview of T-SQL and CLR debugging in SQL Server 2005 > http://blogs.msdn.com/sqlclr/archive/2006/06/29/651644.aspx > > How to: Enable SQL Server 2005 Debugging > http://msdn.microsoft.com/en-us/library/s0fk6z6e(VS.80).aspx > > HTH, > > --Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 12:12 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes > > Robert, > > I don't really want to get in an argument about this. I asked a serious > question and got a somewhat > brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY > answer I have received > so that is not particularly encouraging. > > 8( > > TSQL, at least as written from inside of SSMS is about like writing GBasic > code back in 1982. Well, > maybe a tiny step up but not much. > > You are right, I probably just don't know how, but in what I will call "a > real language" I can set > breakpoints, I can see a call stack, I have watch statements, I have... > simple very basic DEBUG > capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in > SSMS). I can have forms > with controls displaying data, buttons that cause things to happen. I can > FTP files, monitor > directories for activities. > > Believe me, if this exists in SSMS I will jump on that! Just don't tell me > to go spend thousands to > buy a tool that will do some of what C# can already do for me. > > Modern languages, and the .Net environment in particular have capabilities > that (AFAICT) TSQL can > not even dream of. TSQL dreams of manipulating data, C# dreams of > manipulating ... the world. > > Robert I am not "fighting learning it", I am light years beyond where I was > two years ago, and I > learn more TSQL and SQL Server every day. However from what I have seen so > far, TSQL will NEVER be > what I need for the supervisor. And answers like you just provided aren't > really much help. > > I am a consultant. I am a programmer, not a DBA, or a SQL Server > administrator, or a SQL Server > jock at a company somewhere. I need C# for automating my business, which is > NOT being a DBA (I > already said that I think). I will write C# applications for clients that > use SQL Server, and > others that do not use SQL Server. SQL Server is used by exactly ONE of my > clients, so I cannot > spend the time to do whatever it is that you envision SQL Server doing > because I cannot amortize the > learning over other clients. > > Now... if you want to answer my question, I am all ears. The first thing to > answer (If I am to do > all this in TSQL) is how do I get a modern tool with all of the debug > capabilities to write TSQL code. > > Let's take a concrete, real world example. I have a set of virtual machines > which run a third party > application. I place files into an "inbox" directory, and I wait for files > to pop out of the > running application into the "outbox" directory. Just an aside, the files > placed into the inbox are > created by a set of about 5 stored procedures, which can operate on any of > dozens of databases in my > server. > > It is the case that the third party application occasionally hangs. If it > does I need to rename a > file in a specific directory to cause that application to restart processing > that file. > > It is my intention to use C# to execute the SPs to create the files, move > the files into the VM, > then monitor files dropped into the inbox, and files popping out into the > outbox. Look for "no new > files in X minutes" and if that condition exists, rename the file. Log > files dropped into the > in-box, files popping out into the out-box, and the time it took for the > files to process through > the application. Log any "hangs" restarted. Supervisor stuff. > > Explain to me how (and more importantly WHY) I would use TSQL to do that? > Other than a "when TSQL > is all you have" (which is NOT true for me) mentality. > > A SMALL part of my business is to perform some moderately complex processes > using SQL Server and SPs > as a tool. I envision C# as a supervisor, not a "middle layer". I envision > an application (written > in C#) that runs on my desktop which starts, then monitors SQL Server > processes, displays errors, > restarts processes if necessary, creates databases, tables etc, backs them > up and detaches them when > done, logging everything that happens, showing it all to me me in real time > in a Windows > application. Possibly even automatically generating invoices for the work > done and emailing them to > the client. > > Explain to me one more time (with feeling) how (and more importantly WHY) I > would use TSQL to do all > of that stuff? > > John W. Colby > www.ColbyConsulting.com > > > Robert Stewart wrote: >> Sorry, but using C# to do what you would and could do simply using >> SQL Server is adding a middle layer you do not need. You can do >> exactly the same thing using SQL Server and stored procedures and >> writing to a log table about what the process is doing and keeping >> track of it that way. And, not have to use a middle layer language >> like C# to do it. >> >> I am not sure why writing T-SQL code is not like real programming to >> you. But, it is. And, if you are going to continue using SQL >> Server, you need to learn what it can do for you instead of always >> fighting learning it. >> >> This is coming from someone that does T-SQL, C#, VB.Net and Access >> VBA. So, I think I have a fair perspective of how they all work. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4575 (20091105) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Thu Nov 5 07:52:53 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 05 Nov 2009 08:52:53 -0500 Subject: [dba-VB] Fwd: How do you run SQL Server processes In-Reply-To: References: <4AF207CD.2060307@colbyconsulting.com> Message-ID: <4AF2D8B5.1030507@colbyconsulting.com> Mark, > However, I think that Robert is correct in suggestion to try, as much as possible to not cross the technology stack with volumes of data when you do not have to. I have never had a problem with letting each tool do what it does best. In my case (or for this specific client) the database is decidedly not "normal". I have many different databases, each with a few tables, and ZERO (yes ZERO) parent / child tables. I keep tables of names / addresses. That's it. So I am not doing transactions, I am not doing reports, I am not doing data entry, I am not doing any of the "normal" things that people do with databases. What I have to do is get the entire table of name / address out to an external process and then back in to the same database, and I have to do this at least monthly. We are talking 20, 50, 80 million names exported into 2 million record files, processed through this external program, and then imported back in again. This is a "huge" process, not in terms of sheer SQL complexity but rather in terms of the steps involved, the monitoring that has to occur, and the time it takes. And I have to do this on a half dozen large databases, monthly. The external program processes about 2 million records an hour start to finish. Do the math and 60 million records turns into 30 hours of external processing. I use three virtual machines to get three times the records / hour but you are still talking 10 hours and that doesn't count the time to get the records out and back in to SQL Server again. My thought was to start a discussion about that "periphery processing" so that I could get ideas from people who already do it. I am getting almost no response however with actual "I do that using..." responses, and I am wondering why. Perhaps because I deal with small clients, the cost of manual labor to do those kinds of things by hand is too high so they hire me to automate it. Perhaps in a big organization they simply throw people at the problem? Believe me, when you get down to a "one person company" (me!) you can't do this stuff manually for very long, you MUST automate it or you will spend you entire day doing stuff that should be done by the computer. My original question was aimed at discovering how other people handle this kind of processing. I have another client (in Access in this case) that routinely exports data using very specialized export functions (that I wrote) dictated by mainframe interface specs. The data has to be pulled, exported, formatted, sometimes encrypted, sometimes zipped, sometimes FTPd, sometimes emailed, (the "sometimes" part depending on the recipient's preferences), and emails sent announcing the data is on the way etc. I do all of this in Access currently but it is ugly in Access because VBA does not have the built-in tools to do ftp/zip/encrypt etc. C# has some of that built-in and you can usually find C# routines to do the rest. BTW, as I have mentioned, I am taking a C# class at the local community college. I met a Russian gentleman, 60ish, who has been in programming / IT all of his life, recently unemployed. He and I are the only two in the class that have actually worked in the industry (other than the teacher). We struck up a conversation and he asked me if I would like to bid with him on a couple of small C# jobs on a jobs board "just to get some real world experience". I counter-offered that he join me in automating some of my real-world tasks. He came over yesterday and we are going to be working together on the kinds of tasks I described above. He has a LOT of C# experience but almost no database experience beyond simple select queries. We should each learn a lot from the other, it should be fun, and hopefully I can get some of this automation done! John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > FWIW, I agree with you that you need a rich environment to do the work that > you plan to do. > > However, I think that Robert is correct in suggestion to try, as much as > possible to not cross the technology stack with volumes of data when you do > not have to. > > IOW, do all the heavy lifting in the database, and by all means return as > many parameters or log results back to the front end you are building in C#, > but try to avoid returning your data when ever possible. > > I guess I was suggesting the same with with SSIS. Where you have to export > data based on decisions, SSIS is an extremely fast option. > > Have you already used EXECUTE command to call an sproc within an sproc? > This is a useful means of creating an sproc, that running a set of sprocs, > and can insert / update a log table as it progresses. For data only > actvity, this might be effective. > > For all the front end work however, I guess you will make C# sing, like only > you can, so work away, and let TSQL do what it is good at, all the rest > should not involve enormous volumes of data, so the technology bridge will > be irrelevant. > > Good luck, you know you are lucky. Most Database people never get to read > about a db as big as what you have :) > > On last thing, in SQL Server, Cursors are a dirty word, I never see them > used. However, when you have low volumes of iteration (thousands or a few > tens of thousands), cursors are a very powerful way of enumerating tasks. > > Eg, lets imagine you had a table of states, and you needed to perform ten > seperate sprocs for each state. You could use a cursor to enumerate the > states table and pass in the state name to the ten different sprocs. > > You could log between each iteration, and you could have the whole thing > running in 30 minutes. In this sort of context, I agree with Robert, that > TSQL can give you Raw power quickly and easily. > > C# could be watching your log table and your 'Status Update' table and > simply returning 1 or 2 records from the db to C# per minute, whereas within > your sproc that is enumerating it might be handling millions of records each > two minutes. Those millions would never cross the stack divide of DB Server > to .Net. Again this sharing of processing power is probably what Robert was > referring to. > > BTW I recently started using FileZilla and find it super. > > Good Luck, > > Mark > > > > Mark > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From raibeart at gmail.com Thu Nov 5 09:16:01 2009 From: raibeart at gmail.com (Robert Stewart) Date: Thu, 05 Nov 2009 09:16:01 -0600 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: References: Message-ID: <4af2ec32.0305560a.75b1.0901@mx.google.com> John, Mark has it exactly correct. Use SQL Server to the max of what it can do for you before you go off creating something in c# to do essentially the same things. Data is what it does, C# is the GUI. Very similar to what we have all done with MS Access over the years. Now, lets get down to what you are doing... Running code out of a database table is not efficient. The optimizer has to work on it before it can be run. So, the processing will be slower. It would be better to have the code one or more stored procedures with all the parameters and run them against small data sets to get them optimized. Then they would be ready for the larger chunks of data. Each SP can write to a log table and your C# GUI can display the progress by reading data out of the log tables. By using the log tables, you can pass start times, records processed and the time at predetermined numbers of records and which stored proc is doing the work at the time, end times, and just about any other information you may want, Including things that cause errors. SSIS is extremely powerful and the real tool that should probably be used for what you want to do. It can do all of the logging, error trapping and handle passing errored records into a different table for analysis. The C# GUI could be used to read the output from the SSIS package to do the monitoring. It is also relatively simple to sopy SSIS packages and change the parameters around on them to allow them to be used for different extractions and builds. Robert At 01:43 AM 11/5/2009, you wrote: >Date: Thu, 5 Nov 2009 15:43:49 +0800 >From: Mark Breen >Subject: [dba-VB] Fwd: How do you run SQL Server processes >To: dba-vb at databaseadvisors.com >Message-ID: > >Content-Type: text/plain; charset=ISO-8859-1 > >Hello John, > >FWIW, I agree with you that you need a rich environment to do the work that >you plan to do. > >However, I think that Robert is correct in suggestion to try, as much as >possible to not cross the technology stack with volumes of data when you do >not have to. > >IOW, do all the heavy lifting in the database, and by all means return as >many parameters or log results back to the front end you are building in C#, >but try to avoid returning your data when ever possible. > >I guess I was suggesting the same with with SSIS. Where you have to export >data based on decisions, SSIS is an extremely fast option. > >Have you already used EXECUTE command to call an sproc within an sproc? > This is a useful means of creating an sproc, that running a set of sprocs, >and can insert / update a log table as it progresses. For data only >actvity, this might be effective. > >For all the front end work however, I guess you will make C# sing, like only >you can, so work away, and let TSQL do what it is good at, all the rest >should not involve enormous volumes of data, so the technology bridge will >be irrelevant. > >Good luck, you know you are lucky. Most Database people never get to read >about a db as big as what you have :) > >On last thing, in SQL Server, Cursors are a dirty word, I never see them >used. However, when you have low volumes of iteration (thousands or a few >tens of thousands), cursors are a very powerful way of enumerating tasks. > >Eg, lets imagine you had a table of states, and you needed to perform ten >seperate sprocs for each state. You could use a cursor to enumerate the >states table and pass in the state name to the ten different sprocs. > >You could log between each iteration, and you could have the whole thing >running in 30 minutes. In this sort of context, I agree with Robert, that >TSQL can give you Raw power quickly and easily. > >C# could be watching your log table and your 'Status Update' table and >simply returning 1 or 2 records from the db to C# per minute, whereas within >your sproc that is enumerating it might be handling millions of records each >two minutes. Those millions would never cross the stack divide of DB Server >to .Net. Again this sharing of processing power is probably what Robert was >referring to. > >BTW I recently started using FileZilla and find it super. > >Good Luck, > >Mark > > > >Mark From shamil at smsconsulting.spb.ru Thu Nov 5 09:22:02 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 18:22:02 +0300 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <4AF0F117.20209@colbyconsulting.com> References: <4AF0F117.20209@colbyconsulting.com> Message-ID: <00db01ca5e2b$bad004c0$30700e40$@spb.ru> Hi John, <<< I am hoping to eventually make the entire process "table driven" where I can store (in a table) the name of the sp, the parameters in a child table etc. >>> Using table-driven process could be not quite right in C#/.Net world, which proposes many other options to implement generic solutions - I'd just start with "hardcoded" solutions using calls to a set of small "generic" classes/methods executing custom (MS SQL) database processing, when that set of methods/functions stabilizes I'd try to generalize/"generic-ize" it if that will be needed at all... What is the purpose of your "going generic" for this sets of tasks/customer? Isn't it "just for fun" of generic programming? Would that "going generic" make your programming more effective? Developing/debugging/testing generic custom "table-driven"/"other-metadata media driven" solutions is always (much) more time consuming than "hardcoded" ones, and supporting "table-driven" generic custom solutions, especially in the context of constantly changing customer requirements also promise to be rather time consuming - I mean generic "table-"/metadata-driven solutions could only(?) pay back well in the case they are used on many deployment sites by many people. I can be wrong, just talking from my experience, which is not universal of course... I'd better propose to use metadata descriptions/specs (stored in db tables or (XML) files), if you're so bored to write repetitive code, to generate "hardcoded" solutions, then make quick fixes to this generated hardcode to satisfy your customers' current requests, then (in free time) adjust code generator, then when everything will stabilize (will that ever happen?) develop truly generic "table-driven" solution. You can also consider using F# and Windows Workflow Foundation (WF)... The following presentation seems to be useful to get true picture of "generic vs. specific" solutions trade-offs: http://www.infoq.com/presentations/Generic-Specific-Tradeoffs-Stefan-Tilkov Thank you. -- Shamil P.S. FYI: AFAIKR MS has got created Domain Specific Language (DSL) team to develop DSL tools somewhere in year 2004 and they planned to come with first release in one(?) year but they aren't yet there (?)... -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 04, 2009 6:12 AM To: VBA; Dba-Sqlserver Subject: [dba-VB] How do you run SQL Server processes I have processes that I run which use sets of stored procedures. For example two of the processes export / import records from large tables for address validation. The addresses need to be sorted in Zip5/Zip4/Address order in order to make the external validation process as fast as possible so I create a table that pulls millions of records, often tens of millions of records, into a table (created on-the-fly) in sorted order, then pulls out 2 million records at a time and BCPs them out for processing in an external program. The inverse occurs when the records finish processing, 2 million record chunks are imported back into a temp table, then each 2 million record chunk is appended to a main table. Once back in SQL Server I have processes that build clustered and non-clustered indexes, run sha-1 hash functions to generate hash keys, update address valid codes etc. I will run this entire project at least once a month (to keep the addresses valid as people move), and I do so for many tables - a table of about 65 million addresses, another of 21 million, another of 11 million etc. I store all of my stored procedures in a dedicated database that pretty much just contains the SPs and a few UDFs as well as a couple of logging type tables. At the moment I use an Access database to run the individual stored procedures in specific order. I am just embarking on a project to move that functioning but not very efficient Access database to C#. I want to be able to have better monitoring of SP completion, records processed (counts), time to complete each SP etc. I found a pretty nice set of C# classes that allow creation of "stored procedure" objects, with parameter objects, then stored into a collection as the SPs (class instances) are assembled and finally the whole collection of SPs executed. I created a test database of a million names / addresses and today I started testing my existing SPs on this small data set. In the next few days I expect to build out a preliminary "port" of the existing Access VBA code. I am hoping to eventually make the entire process "table driven" where I can store (in a table) the name of the sp, the parameters in a child table etc. and then have a supervisor pull out the records and execute the SPs based on what is in the table. This would allow me to create the SPs and then use a C# application (data entry forms) to organize the SPs, add / delete SPs from the process and so forth. That is down the road of course, for now I will have hard coded sequences - as I already do in supervisor SPs as well as in hard coded Access VBA modules. As a programmer, doing this supervisor level in C# feels more natural to me (than doing it inside of SQL Server), and gives me the flexibility to do what I want in code external to the SQL Server database being manipulated. I want to be able to do things external to the database processes, things like monitor the directories that the BCP processes export to / from, move files popping out of a bcp process off to a VM on a different server, monitor those VMs for files coming out of their processes, moving the files back over to the SQL Server directories, and trigger BCP processes that import the data back in to SQL server. I need to be able to FTP order and data files to clients, and get (FTP) files from process bureaus. So it FEELS to me like using a language such as C# to build a system outside of SQL Server is the way to go. BTW I stumbled across something called SMO (SQL Server Management Objects) which makes it really easy to get at the SQL Server object model from C#. I can apparently see the entire tree structure that is a server/database/tables/etc. and do I know not what (yet) with those objects. Looks pretty cool though. The database is a large part of what I do but it is not the whole picture and I have been handcuffed by my lack of ability to effectively do these external parts. But I am a programmer at heart. I want to know how you do this kind of stuff. I know that running such "systems" of stored procedures tied to external processes must be quite common and I am wondering what you guys do for supervising your processes. If anyone doing anything remotely similar would take the time to explain how you do it, I would be most appreciative. Thanks, -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4573 (20091104) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From shamil at smsconsulting.spb.ru Thu Nov 5 09:39:05 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 18:39:05 +0300 Subject: [dba-VB] Recommended watching: "Thoughts on the Generic vs. Specific Tradeoff" Message-ID: <00e801ca5e2e$1c7f93f0$557ebbd0$@spb.ru> Hi All, The following presentation is a one from a wise man talking from his real life experience - it's fun and so useful to watch, recommended: http://www.infoq.com/presentations/Generic-Specific-Tradeoffs-Stefan-Tilkov Thank you. -- Shamil From jwcolby at colbyconsulting.com Thu Nov 5 10:08:36 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 05 Nov 2009 11:08:36 -0500 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <4af2ec32.0305560a.75b1.0901@mx.google.com> References: <4af2ec32.0305560a.75b1.0901@mx.google.com> Message-ID: <4AF2F884.2040903@colbyconsulting.com> Robert, Thanks for the reply. The issue here is that I have to run the same code against a large set of databases / tables. Fir this discussion, the "process" is exporting millions of records to CSV and later importing the resulting CSVs back in again. For example I have about 6 different databases, each with it's own set of tables. Additionally I have to do this exact same process on "Orders" where I am selecting anywhere from a few thousand to a few million addresses to fill an order. The client wants those orders processed as well. Understand that I am not claiming that the following is the most efficient, just "what I know how to do". So my solution so far has been to build a set of stored procedures where I pass in the database and the source table (for the export process) and dynamically build up SQL statements in the SP. The generic process is to: 1) SP1: Build a big table to hold the entire set of records, sorted on Zip5/Zip4, with an autonumber for selection of 2 million record chunks. Move the records from TblSrc (or ViewSrc) to TblBig. Build an index on autonumber and the fields to export to make the subsequent processing faster. Params DB, TblNameSrc. 2) SP2: Build a chunk table. Move a chunk from TblBig to TblChunk Params: DBName, StartPKID, EndPKID 3) SP3: BCP TblChunk to CSV file Params: DBName, Destination Dir This is not an exact representation of what is going on but gets the point across, which is that I have to do this on a "random" database, probably a dozen a month. Initially I did this in stored procedures stored inside of each database but the maintenance of the stored procedures became a nightmare. As I would figure out a better way, a different way, fix a bug... I would be constantly trying to modify my code in each database. NOT FUN!!! Eventually I moved to the parameterized stored procedure with dynamically constructed SQL executed inside of try/catch blocks etc. I store all of these SPs out in a "management" database and execute them from there, passing in the database name, table name and any other parameters required to make each SP work. As I have said, I don't know enough to determine whether this is "the best way", I have enough problems just getting this much to work. I have in fact started using logging from inside of each SP. I designed a standard log table with fields for the SP name being processed, record counts, error codes if any, memo field so I can write little explanation of each step and so forth. I JUST started doing that. I created a SP (again parameterized, stored in the management db) that performs the write to the log, then I call that SP whenever I want to log anything. The thing everyone following this thread needs to understand is that I have no resource other than this list and Google to learn how to do this stuff. I work alone, I do what I have to, as quickly as I reasonably can, because my job (for this client) is to process data, NOT write code. I am a consultant providing a service, NOT a DBA. That said, I have no one but myself to assist me in automating this stuff so that I do not do every step manually, and believe me, when I started I did EVERY STEP manually. Everything that is now a SP I did manually, sometimes dozens of times over the period I have worked for this client. It was not only UGLY, it was DAMNED UGLY. I am a programmer by trade, a DBA/Programmer by necessity, a consultant/ business analyst/ DBA/ Programmer by job description. I view ALL of this from "How can I most quickly, with the least effort, make my job easier". I don't have the luxury of learning anything that I don't HAVE to know, right now, to do my job. I also don't have the luxury of spending weeks working on one specific problem. Do a task for client A, move on to the task for client B. At the end of the month, bill for my time. Thanks for the hint to use SSIS. That is one of those "haven't had the time to learn it" things but I have always known I probably should. When you are up to your ass in alligators, it is hard to STOP... and study SSIS. John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > John, > > Mark has it exactly correct. > > Use SQL Server to the max of what it can do for you before you go off > creating something in c# to do essentially the same things. Data is what > it does, C# is the GUI. Very similar to what we have all done with MS > Access over the years. > > Now, lets get down to what you are doing... > > Running code out of a database table is not efficient. The optimizer has to > work on it before it can be run. So, the processing will be slower. > > It would be better to have the code one or more stored procedures with all the > parameters and run them against small data sets to get them > optimized. Then they > would be ready for the larger chunks of data. > > Each SP can write to a log table and your C# GUI can display the progress by > reading data out of the log tables. > > By using the log tables, you can pass start times, records processed and the > time at predetermined numbers of records and which stored proc is doing the > work at the time, end times, and just about any other information you may want, > Including things that cause errors. > > SSIS is extremely powerful and the real tool that should probably be used for > what you want to do. It can do all of the logging, error trapping and handle > passing errored records into a different table for analysis. The C# GUI could > be used to read the output from the SSIS package to do the monitoring. It is > also relatively simple to sopy SSIS packages and change the parameters around > on them to allow them to be used for different extractions and builds. > > Robert From jwcolby at colbyconsulting.com Thu Nov 5 10:33:38 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 05 Nov 2009 11:33:38 -0500 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <00db01ca5e2b$bad004c0$30700e40$@spb.ru> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> Message-ID: <4AF2FE62.2050501@colbyconsulting.com> Shamil, > Using table-driven process could be not quite right in C#/.Net world, which proposes many other options to implement generic solutions - I'd just start with "hardcoded" solutions using calls to a set of small "generic" That is precisely what I do right now, and what I will do for the near future. The table driven thing is simply that I write "processes" as discussed in this thread. A process might be: Export To Accuzip: "Export dbX TblY to files in DirectoryZ". However that breaks down onto a set of a handful of stored procedures, with passed in parameters. Now, the process "Export To Accuzip" is applied against 6 or 8 different databases every month, but ALSO against random databases which are orders. Orders come in every month and they are a whole nother discussion, but PART of an order is the process Export to Accuzip. DatabaseA exports files to its own directory, DatabaseB to its own directory, OrderAA to its own directory and so forth. But the EXPORT itself is a very standard process. So I am examining the concept of storing the process in a table, so that I can simply select (from a form: Process X: Use SP1, params use SP4, Params use SP9, Params Process Y: Use SP1, Params Use SP2, Params Use SP5, Params Process Z: Use... Table driven (to the extent possible). Then a supervisor application could execute a process (set of SPs), read out of a table, passing in only the database name and possibly a directory on disk. Everything else is selected and set up in advance. To see what a process is, just open a table, to modify the process, select different stored procedures. This stuff is VERY standard, with just a few things that change from DB to db (the database name and where files are going) however I do occasionally shufffle around the interior processes (indexes created, fields used etc. That stuff is internal to the SP and if I change that it is in the SP itself. Then that change is applied to every database from then on. I am not saying that this will work but as my understand and knowledge changes I am spending a LOT of time rewriting stuff. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > <<< > I am hoping to eventually make the entire process "table driven" where I can > store (in a table) the > name of the sp, the parameters in a child table etc. > Using table-driven process could be not quite right in C#/.Net world, which > proposes many other options to implement generic solutions - I'd just start > with "hardcoded" solutions using calls to a set of small "generic" > classes/methods executing custom (MS SQL) database processing, when that set > of methods/functions stabilizes I'd try to generalize/"generic-ize" it if > that will be needed at all... > > What is the purpose of your "going generic" for this sets of tasks/customer? > > Isn't it "just for fun" of generic programming? > Would that "going generic" make your programming more effective? > > Developing/debugging/testing generic custom "table-driven"/"other-metadata > media driven" solutions is always (much) more time consuming than > "hardcoded" ones, and supporting "table-driven" generic custom solutions, > especially in the context of constantly changing customer requirements also > promise to be rather time consuming - I mean generic > "table-"/metadata-driven solutions could only(?) pay back well in the case > they are used on many deployment sites by many people. I can be wrong, just > talking from my experience, which is not universal of course... > > I'd better propose to use metadata descriptions/specs (stored in db tables > or (XML) files), if you're so bored to write repetitive code, to generate > "hardcoded" solutions, then make quick fixes to this generated hardcode to > satisfy your customers' current requests, then (in free time) adjust code > generator, then when everything will stabilize (will that ever happen?) > develop truly generic "table-driven" solution. > > You can also consider using F# and Windows Workflow Foundation (WF)... > > The following presentation seems to be useful to get true picture of > "generic vs. specific" solutions trade-offs: > > http://www.infoq.com/presentations/Generic-Specific-Tradeoffs-Stefan-Tilkov > > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Thu Nov 5 11:13:22 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 20:13:22 +0300 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <4AF2FE62.2050501@colbyconsulting.com> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> <4AF2FE62.2050501@colbyconsulting.com> Message-ID: <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> Hi John, If your "Process X", "Process Y",... calls to stored procedures do have constant parameters' values then I do not see (I can be missing them) clear reasons to use table-driven approach - I'd just have hardcoded ProcessX, ProcessY, ... calling C# wrapper classes' methods to execute SPs. And the latter wrapper classes can inherit from some generic base class(es) to keep executing context, collect statistics etc... As I noted previously WF (Windows Workflow Foundation) could be a good candidate for your subject area but WF has a steep learning curve so you will not be able to effectively apply it right now but with small dedicated classes, which you're writing right now (and delivering your custom solutions right now) you'll be able to relatively easy migrate to WF based solutions in the future. And if you will spend time working on generic custom table-driven solution you'll probably have to throw it away later when migrating to WF... Before you get comfortable with WF you can code you ProcessX, ProcessY, ... steps as "meta-process" text or xml files keeping fully qualified class names for your dedicated classes. And your "generic execution engine" will be just a simple code reading "meta-process" description, instantiating custom classes using .NET Reflection, calling their generic context/initialization methods, running SP by calling generic Run() method (parameters setting is inside your custom classes), then calling generic termination methods and iterating to the next step/finishing when "meta-process" file description ends. That could be it. Thank you. --Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 7:34 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] How do you run SQL Server processes Shamil, > Using table-driven process could be not quite right in C#/.Net world, which proposes many other options to implement generic solutions - I'd just start with "hardcoded" solutions using calls to a set of small "generic" That is precisely what I do right now, and what I will do for the near future. The table driven thing is simply that I write "processes" as discussed in this thread. A process might be: Export To Accuzip: "Export dbX TblY to files in DirectoryZ". However that breaks down onto a set of a handful of stored procedures, with passed in parameters. Now, the process "Export To Accuzip" is applied against 6 or 8 different databases every month, but ALSO against random databases which are orders. Orders come in every month and they are a whole nother discussion, but PART of an order is the process Export to Accuzip. DatabaseA exports files to its own directory, DatabaseB to its own directory, OrderAA to its own directory and so forth. But the EXPORT itself is a very standard process. So I am examining the concept of storing the process in a table, so that I can simply select (from a form: Process X: Use SP1, params use SP4, Params use SP9, Params Process Y: Use SP1, Params Use SP2, Params Use SP5, Params Process Z: Use... Table driven (to the extent possible). Then a supervisor application could execute a process (set of SPs), read out of a table, passing in only the database name and possibly a directory on disk. Everything else is selected and set up in advance. To see what a process is, just open a table, to modify the process, select different stored procedures. This stuff is VERY standard, with just a few things that change from DB to db (the database name and where files are going) however I do occasionally shufffle around the interior processes (indexes created, fields used etc. That stuff is internal to the SP and if I change that it is in the SP itself. Then that change is applied to every database from then on. I am not saying that this will work but as my understand and knowledge changes I am spending a LOT of time rewriting stuff. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > <<< > I am hoping to eventually make the entire process "table driven" where I can > store (in a table) the > name of the sp, the parameters in a child table etc. > Using table-driven process could be not quite right in C#/.Net world, which > proposes many other options to implement generic solutions - I'd just start > with "hardcoded" solutions using calls to a set of small "generic" > classes/methods executing custom (MS SQL) database processing, when that set > of methods/functions stabilizes I'd try to generalize/"generic-ize" it if > that will be needed at all... > > What is the purpose of your "going generic" for this sets of tasks/customer? > > Isn't it "just for fun" of generic programming? > Would that "going generic" make your programming more effective? > > Developing/debugging/testing generic custom "table-driven"/"other-metadata > media driven" solutions is always (much) more time consuming than > "hardcoded" ones, and supporting "table-driven" generic custom solutions, > especially in the context of constantly changing customer requirements also > promise to be rather time consuming - I mean generic > "table-"/metadata-driven solutions could only(?) pay back well in the case > they are used on many deployment sites by many people. I can be wrong, just > talking from my experience, which is not universal of course... > > I'd better propose to use metadata descriptions/specs (stored in db tables > or (XML) files), if you're so bored to write repetitive code, to generate > "hardcoded" solutions, then make quick fixes to this generated hardcode to > satisfy your customers' current requests, then (in free time) adjust code > generator, then when everything will stabilize (will that ever happen?) > develop truly generic "table-driven" solution. > > You can also consider using F# and Windows Workflow Foundation (WF)... > > The following presentation seems to be useful to get true picture of > "generic vs. specific" solutions trade-offs: > > http://www.infoq.com/presentations/Generic-Specific-Tradeoffs-Stefan-Tilkov > > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4576 (20091105) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Thu Nov 5 11:32:51 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 05 Nov 2009 12:32:51 -0500 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> <4AF2FE62.2050501@colbyconsulting.com> <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> Message-ID: <4AF30C43.8010508@colbyconsulting.com> Shamil, Listen to this... > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... steps as "meta-process" text or xml files keeping fully qualified class names for your dedicated classes. And your "generic execution engine" will be just a simple code reading "meta-process" description, instantiating custom classes using .NET Reflection, calling their generic context/initialization methods, running SP by calling generic Run() method (parameters setting is inside your custom classes), then calling generic termination methods and iterating to the next step/finishing when "meta-process" file description ends. Now read this. Open form Select stored procedure, select parameter, select parameter. click save. Select stored procedure, select parameter, select parameter. click save. Continue till done. ;) Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? Read your solution again. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > If your "Process X", "Process Y",... calls to stored procedures do have > constant parameters' values then I do not see (I can be missing them) clear > reasons to use table-driven approach - I'd just have hardcoded ProcessX, > ProcessY, ... calling C# wrapper classes' methods to execute SPs. And the > latter wrapper classes can inherit from some generic base class(es) to keep > executing context, collect statistics etc... > > As I noted previously WF (Windows Workflow Foundation) could be a good > candidate for your subject area but WF has a steep learning curve so you > will not be able to effectively apply it right now but with small dedicated > classes, which you're writing right now (and delivering your custom > solutions right now) you'll be able to relatively easy migrate to WF based > solutions in the future. And if you will spend time working on generic > custom table-driven solution you'll probably have to throw it away later > when migrating to WF... > > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... > steps as "meta-process" text or xml files keeping fully qualified class > names for your dedicated classes. And your "generic execution engine" will > be just a simple code reading "meta-process" description, instantiating > custom classes using .NET Reflection, calling their generic > context/initialization methods, running SP by calling generic Run() method > (parameters setting is inside your custom classes), then calling generic > termination methods and iterating to the next step/finishing when > "meta-process" file description ends. That could be it. > > Thank you. > > --Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 7:34 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] How do you run SQL Server processes > > Shamil, From shamil at smsconsulting.spb.ru Thu Nov 5 13:00:15 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 22:00:15 +0300 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <4AF30C43.8010508@colbyconsulting.com> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> <4AF2FE62.2050501@colbyconsulting.com> <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> <4AF30C43.8010508@colbyconsulting.com> Message-ID: <010c01ca5e4a$3735e430$a5a1ac90$@spb.ru> Hi John, <<< Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? >>> Yes, that is also what I mean: developing generic table-driven procedure for you currently could force you to learn more C#/.NET development stuff than when implementing "meta-process description" interpreter, which will consist of several C# code lines. But even the latter is not needed probably as you can hardcode your solutions for every custom task "shuffling" your parameterized wrappers classes/methods calling SPs, and instead of "Select stored procedure, select parameter,..." use "Create C# project, type C# code with calls to the wrapper classes/methods with Stored Procedures names and parameter names, types and values, then compile and run the custom solution". And it should be more suitable to debug/trace latter solution than "table-driven" or "meta-process description driven" ones... OK? (Am I still missing something here?) -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 8:33 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] How do you run SQL Server processes Shamil, Listen to this... > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... steps as "meta-process" text or xml files keeping fully qualified class names for your dedicated classes. And your "generic execution engine" will be just a simple code reading "meta-process" description, instantiating custom classes using .NET Reflection, calling their generic context/initialization methods, running SP by calling generic Run() method (parameters setting is inside your custom classes), then calling generic termination methods and iterating to the next step/finishing when "meta-process" file description ends. Now read this. Open form Select stored procedure, select parameter, select parameter. click save. Select stored procedure, select parameter, select parameter. click save. Continue till done. ;) Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? Read your solution again. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > If your "Process X", "Process Y",... calls to stored procedures do have > constant parameters' values then I do not see (I can be missing them) clear > reasons to use table-driven approach - I'd just have hardcoded ProcessX, > ProcessY, ... calling C# wrapper classes' methods to execute SPs. And the > latter wrapper classes can inherit from some generic base class(es) to keep > executing context, collect statistics etc... > > As I noted previously WF (Windows Workflow Foundation) could be a good > candidate for your subject area but WF has a steep learning curve so you > will not be able to effectively apply it right now but with small dedicated > classes, which you're writing right now (and delivering your custom > solutions right now) you'll be able to relatively easy migrate to WF based > solutions in the future. And if you will spend time working on generic > custom table-driven solution you'll probably have to throw it away later > when migrating to WF... > > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... > steps as "meta-process" text or xml files keeping fully qualified class > names for your dedicated classes. And your "generic execution engine" will > be just a simple code reading "meta-process" description, instantiating > custom classes using .NET Reflection, calling their generic > context/initialization methods, running SP by calling generic Run() method > (parameters setting is inside your custom classes), then calling generic > termination methods and iterating to the next step/finishing when > "meta-process" file description ends. That could be it. > > Thank you. > > --Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 7:34 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] How do you run SQL Server processes > > Shamil, _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4576 (20091105) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4576 (20091105) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From shamil at smsconsulting.spb.ru Sat Nov 7 05:28:32 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 7 Nov 2009 14:28:32 +0300 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <010c01ca5e4a$3735e430$a5a1ac90$@spb.ru> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> <4AF2FE62.2050501@colbyconsulting.com> <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> <4AF30C43.8010508@colbyconsulting.com> <010c01ca5e4a$3735e430$a5a1ac90$@spb.ru> Message-ID: <018c01ca5f9d$71389c00$53a9d400$@spb.ru> Hi John, Here is a .NET Reflection sample: using System; namespace NetReflectionSample { class Program { static void Main(string[] args) { string[] spRunnersNames = { "NetReflectionSample.SP1", "NetReflectionSample.SP2" }; foreach (string spRunnerName in spRunnersNames) { Type spRunnerType = Type.GetType(spRunnerName); StoredProcedureRunner runner = Activator.CreateInstance(spRunnerType) as StoredProcedureRunner; runner.Init(); runner.Run(); runner.CollectStats(); Console.WriteLine(); } } } public abstract class StoredProcedureRunner { public void Init() { Console.WriteLine("Initializing {0}...", this.GetType().ToString()); } public abstract void Run(); public void CollectStats() { Console.WriteLine("Collecting stats {0}...", this.GetType().ToString()); } } public class SP1 : StoredProcedureRunner { public override void Run() { Console.WriteLine("Running {0}...", this.GetType().ToString()); } } public class SP2 : StoredProcedureRunner { public override void Run() { Console.WriteLine("Running {0}...", this.GetType().ToString()); } } } Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: Thursday, November 05, 2009 10:00 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] How do you run SQL Server processes Hi John, <<< Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? >>> Yes, that is also what I mean: developing generic table-driven procedure for you currently could force you to learn more C#/.NET development stuff than when implementing "meta-process description" interpreter, which will consist of several C# code lines. But even the latter is not needed probably as you can hardcode your solutions for every custom task "shuffling" your parameterized wrappers classes/methods calling SPs, and instead of "Select stored procedure, select parameter,..." use "Create C# project, type C# code with calls to the wrapper classes/methods with Stored Procedures names and parameter names, types and values, then compile and run the custom solution". And it should be more suitable to debug/trace latter solution than "table-driven" or "meta-process description driven" ones... OK? (Am I still missing something here?) -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 8:33 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] How do you run SQL Server processes Shamil, Listen to this... > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... steps as "meta-process" text or xml files keeping fully qualified class names for your dedicated classes. And your "generic execution engine" will be just a simple code reading "meta-process" description, instantiating custom classes using .NET Reflection, calling their generic context/initialization methods, running SP by calling generic Run() method (parameters setting is inside your custom classes), then calling generic termination methods and iterating to the next step/finishing when "meta-process" file description ends. Now read this. Open form Select stored procedure, select parameter, select parameter. click save. Select stored procedure, select parameter, select parameter. click save. Continue till done. ;) Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? Read your solution again. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > If your "Process X", "Process Y",... calls to stored procedures do have > constant parameters' values then I do not see (I can be missing them) clear > reasons to use table-driven approach - I'd just have hardcoded ProcessX, > ProcessY, ... calling C# wrapper classes' methods to execute SPs. And the > latter wrapper classes can inherit from some generic base class(es) to keep > executing context, collect statistics etc... > > As I noted previously WF (Windows Workflow Foundation) could be a good > candidate for your subject area but WF has a steep learning curve so you > will not be able to effectively apply it right now but with small dedicated > classes, which you're writing right now (and delivering your custom > solutions right now) you'll be able to relatively easy migrate to WF based > solutions in the future. And if you will spend time working on generic > custom table-driven solution you'll probably have to throw it away later > when migrating to WF... > > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... > steps as "meta-process" text or xml files keeping fully qualified class > names for your dedicated classes. And your "generic execution engine" will > be just a simple code reading "meta-process" description, instantiating > custom classes using .NET Reflection, calling their generic > context/initialization methods, running SP by calling generic Run() method > (parameters setting is inside your custom classes), then calling generic > termination methods and iterating to the next step/finishing when > "meta-process" file description ends. That could be it. > > Thank you. > > --Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 7:34 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] How do you run SQL Server processes > > Shamil, From shamil at smsconsulting.spb.ru Sat Nov 7 08:19:11 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 7 Nov 2009 17:19:11 +0300 Subject: [dba-VB] NetReflectionSample - Part I of IV Message-ID: <000601ca5fb5$49b79dd0$dd26d970$@spb.ru> Hi John and all, Just wanted to post a bit more advanced sample on .NET reflection and how it can be used to instantiate custom classes and to execute their parameterized methods based on text definition: using System; using System.Reflection; #region Test Program namespace NetReflectionSample { /// /// Test class /// class Program { static void Main(string[] args) { // List of custom classes (with parameters) to instantiate and // call their Run(...) method string[] spRunnersCalls = { "NetReflectionSample.SP1", "NetReflectionSample.SP2|503|Test String|3.14|11/7/2009", "NetReflectionSample.SP3|123|Test String2" }; foreach (string spRunnerCall in spRunnersCalls) { // Parse current SP call string definition and // return StoredProcedureRunner instance StoredProcedureRunner runner = DynamicMethodCallParser.Parse(spRunnerCall); runner.Init(); // If there are calling parameters use CustomRunMethodInfo to // execute SP call wrapper method, else use default Run() method CustomRunMethodInfo customRunner = runner as CustomRunMethodInfo; if (customRunner.Parameters != null) { customRunner.ParameterizedRunMethodInfo.Invoke( customRunner, customRunner.Parameters); } else runner.Run(); // collect execution stats runner.CollectStats(); Console.WriteLine(); } } } } #endregion Thank you. --Shamil From shamil at smsconsulting.spb.ru Sat Nov 7 08:20:44 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 7 Nov 2009 17:20:44 +0300 Subject: [dba-VB] NetReflectionSample - Part II of IV Message-ID: <000b01ca5fb5$7f9438a0$7ebca9e0$@spb.ru> #region Custom business functionality namespace NetReflectionSample { /// /// SP1 wrapper class simulator /// public class SP1 : StoredProcedureRunner { public override void Run() { Console.WriteLine("Running {0}.Run...", this.GetType().ToString()); } } /// /// SP2 wrapper class simulator /// public class SP2 : StoredProcedureRunner { public void Run( int prm1, string prm2, decimal prm3, DateTime prm4) { Console.WriteLine("Running {0}.Run...", this.GetType().ToString()); Console.WriteLine("Parameters: prm1={0}, prm2={1}, prm3={2}, prm4={3}", prm1, prm2, prm3, prm4); } } /// /// SP3 wrapper class simulator /// public class SP3 : StoredProcedureRunner { public void Run( int prm1, string prm2) { Console.WriteLine("Running {0}.Run...", this.GetType().ToString()); Console.WriteLine("Parameters: prm1={0}, prm2={1}", prm1, prm2); } } } #endregion --Shamil From shamil at smsconsulting.spb.ru Sat Nov 7 08:37:01 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 7 Nov 2009 17:37:01 +0300 Subject: [dba-VB] NetReflectionSample - full code text Message-ID: <001a01ca5fb7$c5cab310$51601930$@spb.ru> Hi All, Here is a full text of the subject sample: http://smsconsulting.spb.ru/samples/NetReflectionSample.txt dba-VB robot currently blocks to post two parts of my four parts posting because they are longer than 20KB - dba-VB robot writes that those two postings are awaiting moderators approval but AFAIHF this approval never happens (at least that was for several my past postings with code) - therefore I'm posting the above link to get the whole code text available. Here are the results of the test program execution: === Initializing NetReflectionSample.SP1... Running NetReflectionSample.SP1.Run... Collecting stats NetReflectionSample.SP1... Initializing NetReflectionSample.SP2... Running NetReflectionSample.SP2.Run... Parameters: prm1=503, prm2=Test String, prm3=3.14, prm4=11/7/2009 12:00:00 AM Collecting stats NetReflectionSample.SP2... Initializing NetReflectionSample.SP3... Running NetReflectionSample.SP3.Run... Parameters: prm1=123, prm2=Test String2 Collecting stats NetReflectionSample.SP3... === Note: posted code is a "quick & dirty" first version, if you have time and desire to make a better version of this sample code you're welcome to do that. Any code review remarks and comments are very welcome. Thank you. -- Shamil From Gustav at cactus.dk Sat Nov 7 14:00:22 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Sat, 07 Nov 2009 21:00:22 +0100 Subject: [dba-VB] NetReflectionSample - full code text Message-ID: Hi Shamil Thanks! Good example. /gustav >>> shamil at smsconsulting.spb.ru 07-11-2009 15:37 >>> Any code review remarks and comments are very welcome. From shamil at smsconsulting.spb.ru Sun Nov 8 15:47:44 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 9 Nov 2009 00:47:44 +0300 Subject: [dba-VB] NetReflectionSample - full code text In-Reply-To: References: Message-ID: <000301ca60bd$1c0d9ca0$5428d5e0$@spb.ru> Thank you, Gustav. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Saturday, November 07, 2009 11:00 PM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] NetReflectionSample - full code text Hi Shamil Thanks! Good example. /gustav >>> shamil at smsconsulting.spb.ru 07-11-2009 15:37 >>> Any code review remarks and comments are very welcome. _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4585 (20091108) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sun Nov 8 20:20:20 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 08 Nov 2009 21:20:20 -0500 Subject: [dba-VB] NetReflectionSample - full code text In-Reply-To: References: Message-ID: <4AF77C64.5090707@colbyconsulting.com> But what does it DO? John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi Shamil > > Thanks! Good example. > > /gustav > >>>> shamil at smsconsulting.spb.ru 07-11-2009 15:37 >>> > > Any code review remarks and comments are very welcome. > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Mon Nov 9 00:11:28 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 9 Nov 2009 09:11:28 +0300 Subject: [dba-VB] NetReflectionSample - full code text In-Reply-To: <4AF77C64.5090707@colbyconsulting.com> References: <4AF77C64.5090707@colbyconsulting.com> Message-ID: <000901ca6103$7adfbd80$709f3880$@spb.ru> Hi John, Basically, this sample code demonstrates how to interpret a simple scripting "birds language" into a sequence of calls to the parameterized methods of custom classes. Substitute text script with meta-table(s) - and you'll get table-driven interpreter. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, November 09, 2009 5:20 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] NetReflectionSample - full code text But what does it DO? John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi Shamil > > Thanks! Good example. > > /gustav > >>>> shamil at smsconsulting.spb.ru 07-11-2009 15:37 >>> > > Any code review remarks and comments are very welcome. > > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4586 (20091108) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From shamil at smsconsulting.spb.ru Tue Nov 10 23:49:05 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Wed, 11 Nov 2009 08:49:05 +0300 Subject: [dba-VB] FYI: ORM, EDM, ESQL, Entity Framework, LINQ to SQL, LINQ to Entities - Confused? Message-ID: <003601ca6292$af1581d0$0d408570$@spb.ru> "ORM, EDM, ESQL, Entity Framework, LINQ to SQL, LINQ to Entities - Confused?" http://www.infoq.com/presentations/ORM-LINQ-Entity-Framework-Eric-Nelson --Shamil From Gustav at cactus.dk Wed Nov 11 12:52:48 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 11 Nov 2009 19:52:48 +0100 Subject: [dba-VB] Microsoft WebsiteSpark Program Message-ID: Hi all Have you heard of Microsoft? WebsiteSpark*? http://www.microsoft.com/web/websitespark/Support.aspx?tab=ProgramDetails It is designed to ignite the success of professional Web developers and designers. The program enables you to get software, support and business opportunities from Microsoft at no upfront costs for 3 years or exit from the program. WebSiteSpark provides access to a technology offering for development and production hosting of websites. Once enrolled, you can download the following software from Microsoft: For design, development, testing and demonstration of new websites * for a total of up to three users per Web design and development company: *Visual Studio Professional *Expression Studio 3 (1 user) and Expression Web 3 (up to 2 users) *Windows Web Server 2008 R2 *SQL Server 2008 Web Edition For production use * that is, to deploy and host new websites developed using Program software * using a total of up to four processors per Web design and development company, of the following (physical or virtual) dedicated servers: *Windows Web Server 2008 R2 *SQL Server 2008 Web Edition So if you consider "something else" than Access, this might help you. /gustav From jwcolby at colbyconsulting.com Wed Nov 11 13:08:24 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 11 Nov 2009 14:08:24 -0500 Subject: [dba-VB] SPAM-LOW: Microsoft WebsiteSpark Program In-Reply-To: References: Message-ID: <4AFB0BA8.8020105@colbyconsulting.com> This appears to be one part of this: http://www.microsoftstartupzone.com/pages/home.aspx which I posted about a month or so ago. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi all > > Have you heard of Microsoft? WebsiteSpark*? > > http://www.microsoft.com/web/websitespark/Support.aspx?tab=ProgramDetails > > > > It is designed to ignite the success of professional Web developers and > designers. The program enables you to get software, support and business > opportunities from Microsoft at no upfront costs for 3 years or exit > from the program. > > WebSiteSpark provides access to a technology offering for development > and production hosting of websites. Once enrolled, you can download the > following software from Microsoft: > > For design, development, testing and demonstration of new websites * > for a total of up to three users per Web design and development company: > > > *Visual Studio Professional > *Expression Studio 3 (1 user) and Expression Web 3 (up to 2 users) > *Windows Web Server 2008 R2 > *SQL Server 2008 Web Edition > > For production use * that is, to deploy and host new websites > developed using Program software * using a total of up to four > processors per Web design and development company, of the following > (physical or virtual) dedicated servers: > > *Windows Web Server 2008 R2 > *SQL Server 2008 Web Edition > > > > So if you consider "something else" than Access, this might help you. > > /gustav > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Thu Nov 12 11:22:55 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 12 Nov 2009 12:22:55 -0500 Subject: [dba-VB] What to do, what to do? Message-ID: <4AFC446F.6070207@colbyconsulting.com> I am writing an application in C# to merge records (previously discussed). It looks pretty simple, all things considered. Get data into a dataset object which will contain about 4 tables. Then use the records and fields collections of the recordset to iterate through the fields building up a record (or modifying the first record of a small set of records - the dupes). Copy a record from one table into another table inside of the dataset. Delete records in a table in the dataset. Etc. Etc all without going back out to the actual data tables on SQL Server until the very end when I will do an update back to the live data. Am I correct in assuming here that I can programmatically append records from one table in the dataset to another record in the dataset? Delete records in the tables in the dataset? Update records in tables in the dataset? All "in memory" without connecting back to the live data in the database? Is there anything I need to know before I launch into this? -- John W. Colby www.ColbyConsulting.com From cfoust at infostatsystems.com Thu Nov 12 11:29:43 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Thu, 12 Nov 2009 09:29:43 -0800 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFC446F.6070207@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> Message-ID: I think that depends on the objects you're using. .Net tends to complain that a record already belongs to another table when you try to import it. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 12, 2009 9:23 AM To: VBA Subject: [dba-VB] What to do, what to do? I am writing an application in C# to merge records (previously discussed). It looks pretty simple, all things considered. Get data into a dataset object which will contain about 4 tables. Then use the records and fields collections of the recordset to iterate through the fields building up a record (or modifying the first record of a small set of records - the dupes). Copy a record from one table into another table inside of the dataset. Delete records in a table in the dataset. Etc. Etc all without going back out to the actual data tables on SQL Server until the very end when I will do an update back to the live data. Am I correct in assuming here that I can programmatically append records from one table in the dataset to another record in the dataset? Delete records in the tables in the dataset? Update records in tables in the dataset? All "in memory" without connecting back to the live data in the database? Is there anything I need to know before I launch into this? -- John W. Colby www.ColbyConsulting.com _______________________________________________ From shamil at smsconsulting.spb.ru Fri Nov 13 00:04:33 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 09:04:33 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFC446F.6070207@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> Message-ID: <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> Hi John -- Here is a sample: using System; using System.Collections.Generic; using System.Text; using System.Data; using System.Data.SqlClient; namespace TestConsole.Samples { public class DataSetTests { public static void Run() { string connectionString = "Data Source=HAMBURG\\SQL2005;"+ "Initial Catalog=Northwind;User Id=sa;Password=N/A"; using (SqlConnection connection = new SqlConnection(connectionString)) { connection.Open(); string sql = "select * from [Shippers]"; string sql1 = "select * from [NewShippers] where(1=0)"; SqlCommand command = new SqlCommand(sql, connection); SqlDataAdapter adapter = new SqlDataAdapter(command); DataSet dataSet = new DataSet("Shippers"); adapter.Fill(dataSet); command = new SqlCommand(sql1, connection); adapter = new SqlDataAdapter(command); DataSet newDataSet = new DataSet("NewShippers"); adapter.Fill(newDataSet); Console.WriteLine("*** newDataSet ***"); foreach (DataRow dataRow in newDataSet.Tables[0].Rows) { Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); } Console.WriteLine("*** dataSet ***"); foreach (DataRow dataRow in dataSet.Tables[0].Rows) { // This row already belongs to another table. //newDataSet.Tables[0].Rows.Add(dataRow); object[] items = { dataRow[0], dataRow[1], dataRow[2] }; newDataSet.Tables[0].Rows.Add(items); Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); } Console.WriteLine("*** newDataSet ***"); foreach (DataRow dataRow in newDataSet.Tables[0].Rows) { Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); } newDataSet.Tables[0].TableName = "NewShippers"; SqlCommandBuilder builder = new SqlCommandBuilder(adapter); adapter.InsertCommand = builder.GetInsertCommand(true); adapter.Update(newDataSet.Tables[0]); } } } } Thank you. --Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 12, 2009 8:23 PM To: VBA Subject: [dba-VB] What to do, what to do? I am writing an application in C# to merge records (previously discussed). It looks pretty simple, all things considered. Get data into a dataset object which will contain about 4 tables. Then use the records and fields collections of the recordset to iterate through the fields building up a record (or modifying the first record of a small set of records - the dupes). Copy a record from one table into another table inside of the dataset. Delete records in a table in the dataset. Etc. Etc all without going back out to the actual data tables on SQL Server until the very end when I will do an update back to the live data. Am I correct in assuming here that I can programmatically append records from one table in the dataset to another record in the dataset? Delete records in the tables in the dataset? Update records in tables in the dataset? All "in memory" without connecting back to the live data in the database? Is there anything I need to know before I launch into this? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4600 (20091112) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From marklbreen at gmail.com Fri Nov 13 00:58:16 2009 From: marklbreen at gmail.com (Mark Breen) Date: Fri, 13 Nov 2009 14:58:16 +0800 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFC446F.6070207@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> Message-ID: Hello John, Just curious, how many records are you talking about - a few hundred or millions? Second question, other than for purely academic reasons, why are you bring the data into C#, to concatenate and insert into another table can so easily be done in one sproc, even if you need a temporary holding place, you would still use a temp table or even a local table. Of course if it is to learn datasets then thats another story, Mark 2009/11/13 jwcolby > I am writing an application in C# to merge records (previously discussed). > It looks pretty simple, > all things considered. Get data into a dataset object which will contain > about 4 tables. Then use > the records and fields collections of the recordset to iterate through the > fields building up a > record (or modifying the first record of a small set of records - the > dupes). Copy a record from > one table into another table inside of the dataset. Delete records in a > table in the dataset. Etc. > Etc all without going back out to the actual data tables on SQL Server > until the very end when I > will do an update back to the live data. > > Am I correct in assuming here that I can programmatically append records > from one table in the > dataset to another record in the dataset? Delete records in the tables in > the dataset? Update > records in tables in the dataset? All "in memory" without connecting back > to the live data in the > database? > > Is there anything I need to know before I launch into this? > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Fri Nov 13 03:27:08 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Fri, 13 Nov 2009 10:27:08 +0100 Subject: [dba-VB] What to do, what to do? Message-ID: Hi John Shamil knows the SqlClient stuff! As he knows, I prefer the datatable route - a matter of taste I think, but I admit working with SqlCommand may allow for some fine tuning not possible with the datatableadapters. Here's an example from an app that updates (syncs) one table from another. Not exactly what you wish to do but it should contain the building blocks. The tricky part here is that the unique id (autonumber) should not be compared. However, the id of a source row is stored in the target row as SyncId for later reference: private void ConvertOrder() { // DataTableAdapter to copy from. VrsCloneTableAdapters.OrderTableAdapter sourceDataTableAdapter = new VrsCloneTableAdapters.OrderTableAdapter(); VrsClone.OrderDataTable sourceDataTable; // DataTableAdapter to copy to. KarneliaTableAdapters.SyncOrderTableAdapter targetDataTableAdapter = new KarneliaTableAdapters.SyncOrderTableAdapter(); Karnelia.SyncOrderDataTable targetDataTable; sourceDataTableAdapter.Connection = ConnectionApplyPassword(sourceDataTableAdapter.Connection.ConnectionString); targetDataTableAdapter.Connection = ConnectionApplyPassword(targetDataTableAdapter.Connection.ConnectionString); // DataTable for source. sourceDataTable = sourceDataTableAdapter.GetDataSync(); // DataTable for target. targetDataTable = targetDataTableAdapter.GetData(); // Read each row from the source rows. // Update those found in the target table. // Append those not found in the target table. // Variable to hold primary key to locate. int key; // Row of target DataTable to be updated. Karnelia.SyncOrderRow targetRow = null; // Variables to hold hash codes of row contents. int sourceHash; int targetHash; int colCount = sourceDataTable.Columns.Count; int colItem; int colOrdinal; string colName; for (colItem = 0; colItem < colCount; colItem++) { colName = sourceDataTable.Columns[colItem].ColumnName; colOrdinal = sourceDataTable.Columns[colItem].Ordinal; targetDataTable.Columns[colName].SetOrdinal(colOrdinal + 1); } foreach (VrsClone.OrderRow sourceRow in sourceDataTable) { // Value of key to locate. key = sourceRow.SyncId; // Find a matching row in the target. targetRow = targetDataTable.FindBySyncId(key); if (targetRow == null) { // Row is not found. targetRow = targetDataTable.NewSyncOrderRow(); targetRow.BeginEdit(); targetRow.VrsOrderId = sourceRow.VrsOrderId; targetRow.ClientId = sourceRow.ClientId; targetRow.StylistId = sourceRow.StylistId; targetRow.DateCreated = sourceRow.DateCreated; targetRow.SyncId = sourceRow.SyncId; targetRow.EndEdit(); targetDataTable.AddSyncOrderRow(targetRow); Console.WriteLine("Insert " + targetDataTable.ToString() + ": " + sourceRow.VrsOrderId.ToString() + " - " + targetRow.RowState.ToString()); } else { // Row is found. // Calculate hash code to see if source row and target row match. // Id of targetrow (first column) is not present in sourcerow, thus // concatenate first column of targetrow with the columns of sourcerow. sourceHash = String.Concat(targetRow.ItemArray[0], String.Concat(sourceRow.ItemArray)).GetHashCode(); targetHash = String.Concat(targetRow.ItemArray).GetHashCode(); if (sourceHash != targetHash) { // The source row is updated. // Copy full content from source row to target row. // This sets RowState of the target row to Modified. targetRow.BeginEdit(); targetRow.VrsOrderId = sourceRow.VrsOrderId; targetRow.ClientId = sourceRow.ClientId; targetRow.StylistId = sourceRow.StylistId; targetRow.DateCreated = sourceRow.DateCreated; targetRow.EndEdit(); // Do NOT call AcceptChanges here as this would clear property RowState. Console.WriteLine("Update " + targetDataTable.ToString() + ": " + sourceRow.VrsOrderId.ToString() + " - " + targetRow.RowState.ToString()); } } } // Write back to the database table the updated target DataTable. targetDataTableAdapter.Update(targetDataTable); // Call AcceptChanges if needed for further processing. //targetDataTable.AcceptChanges(); } This database contains records by the thousands. How code like this will perform for millions of records has to be tested. /gustav >>> jwcolby at colbyconsulting.com 12-11-2009 18:22 >>> I am writing an application in C# to merge records (previously discussed). It looks pretty simple, all things considered. Get data into a dataset object which will contain about 4 tables. Then use the records and fields collections of the recordset to iterate through the fields building up a record (or modifying the first record of a small set of records - the dupes). Copy a record from one table into another table inside of the dataset. Delete records in a table in the dataset. Etc. Etc all without going back out to the actual data tables on SQL Server until the very end when I will do an update back to the live data. Am I correct in assuming here that I can programmatically append records from one table in the dataset to another record in the dataset? Delete records in the tables in the dataset? Update records in tables in the dataset? All "in memory" without connecting back to the live data in the database? Is there anything I need to know before I launch into this? -- John W. Colby www.ColbyConsulting.com From Gustav at cactus.dk Fri Nov 13 04:48:24 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Fri, 13 Nov 2009 11:48:24 +0100 Subject: [dba-VB] AWS (Amazon Web Services) SDK for .NET and VS Message-ID: Hi all This has just been announced for Visual Studio 2008: The AWS SDK for .NET makes it even easier for Windows developers to build .NET applications that tap into the cost-effective, scalable, and reliable AWS cloud. Using the SDK, developers will be able to build solutions for AWS infrastructure services, including Amazon Simple Storage Service (Amazon S3), Amazon Elastic Compute Cloud (Amazon EC2), and Amazon SimpleDB. With the AWS SDK for .NET, developers get started in minutes with a single, downloadable package complete with Visual Studio project templates, the AWS .NET library, C# code samples, and documentation. http://aws.amazon.com/sdkfornet/ /gustav From jwcolby at colbyconsulting.com Fri Nov 13 06:51:36 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 07:51:36 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> Message-ID: <4AFD5658.7000703@colbyconsulting.com> Shamil, That is precisely what I have in mind, with variations. Then at the end, write all updates back to SQL Server. Thanks for the demo code, always helpful. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Here is a sample: > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class DataSetTests > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;"+ > "Initial Catalog=Northwind;User Id=sa;Password=N/A"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { > connection.Open(); > string sql = "select * from [Shippers]"; > string sql1 = "select * from [NewShippers] where(1=0)"; > SqlCommand command = new SqlCommand(sql, connection); > SqlDataAdapter adapter = new SqlDataAdapter(command); > > DataSet dataSet = new DataSet("Shippers"); > adapter.Fill(dataSet); > > command = new SqlCommand(sql1, connection); > adapter = new SqlDataAdapter(command); > DataSet newDataSet = new DataSet("NewShippers"); > adapter.Fill(newDataSet); > > Console.WriteLine("*** newDataSet ***"); > foreach (DataRow dataRow in newDataSet.Tables[0].Rows) > { > Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); > } > > Console.WriteLine("*** dataSet ***"); > foreach (DataRow dataRow in dataSet.Tables[0].Rows) > { > // This row already belongs to another table. > //newDataSet.Tables[0].Rows.Add(dataRow); > > object[] items = { dataRow[0], dataRow[1], dataRow[2] }; > newDataSet.Tables[0].Rows.Add(items); > > Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); > } > > Console.WriteLine("*** newDataSet ***"); > foreach (DataRow dataRow in newDataSet.Tables[0].Rows) > { > Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); > } > > newDataSet.Tables[0].TableName = "NewShippers"; > SqlCommandBuilder builder = new SqlCommandBuilder(adapter); > adapter.InsertCommand = builder.GetInsertCommand(true); > adapter.Update(newDataSet.Tables[0]); > } > } > > } > } > > > Thank you. > > --Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 12, 2009 8:23 PM > To: VBA > Subject: [dba-VB] What to do, what to do? > > I am writing an application in C# to merge records (previously discussed). > It looks pretty simple, > all things considered. Get data into a dataset object which will contain > about 4 tables. Then use > the records and fields collections of the recordset to iterate through the > fields building up a > record (or modifying the first record of a small set of records - the > dupes). Copy a record from > one table into another table inside of the dataset. Delete records in a > table in the dataset. Etc. > Etc all without going back out to the actual data tables on SQL Server until > the very end when I > will do an update back to the live data. > > Am I correct in assuming here that I can programmatically append records > from one table in the > dataset to another record in the dataset? Delete records in the tables in > the dataset? Update > records in tables in the dataset? All "in memory" without connecting back > to the live data in the > database? > > Is there anything I need to know before I launch into this? > From shamil at smsconsulting.spb.ru Fri Nov 13 08:19:39 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 17:19:39 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD5658.7000703@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> Message-ID: <004801ca646c$5758e690$060ab3b0$@spb.ru> Hi John -- You can also consider using SqlBulkLoad - below is just a simple sample, and you can develop very flexible variations of this approach. SqlBulkLoad should be as quick as bcp maybe even quicker. In fact the source for SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this IDataReader to have all the data to bulkload then your programs will become as streamlined as they probably have never been, FYI: IDataReader can be built in code with data sources being everything as e.g. .csv files etc. (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) using System; using System.Collections.Generic; using System.Text; using System.Data; using System.Data.SqlClient; namespace TestConsole.Samples { public class SqlBulkLoadTest { public static void Run() { string connectionString = "Data Source=HAMBURG\\SQL2005;" + "Initial Catalog=Northwind;User Id=sa;Password=shms"; using (SqlConnection connection = new SqlConnection(connectionString)) { connection.Open(); string sql = "select * from [Shippers]"; SqlCommand command = new SqlCommand(sql, connection); SqlDataAdapter adapter = new SqlDataAdapter(command); DataSet dataSet = new DataSet("Shippers"); adapter.Fill(dataSet); Console.WriteLine("*** dataSet ***"); foreach (DataRow dataRow in dataSet.Tables[0].Rows) { object[] items = { dataRow[0], dataRow[1], dataRow[2] }; Console.WriteLine("{0} {1} {2}", dataRow[0], dataRow[1], dataRow[2]); } (new SqlCommand("delete from [NewShippers]",connection)) .ExecuteNonQuery(); SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, SqlBulkCopyOptions.KeepIdentity); bulkCopy.DestinationTableName = "NewShippers"; bulkCopy.WriteToServer(dataSet.Tables[0]); } } } } -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 13, 2009 3:52 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? Shamil, That is precisely what I have in mind, with variations. Then at the end, write all updates back to SQL Server. Thanks for the demo code, always helpful. John W. Colby www.ColbyConsulting.com <<< snip >> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4603 (20091113) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 13 08:30:34 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 09:30:34 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <004801ca646c$5758e690$060ab3b0$@spb.ru> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> <004801ca646c$5758e690$060ab3b0$@spb.ru> Message-ID: <4AFD6D8A.3060306@colbyconsulting.com> In fact this is a one shot deal. I need to do this one time, to one table. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > You can also consider using SqlBulkLoad - below is just a simple sample, and > you can develop very flexible variations of this approach. SqlBulkLoad > should be as quick as bcp maybe even quicker. In fact the source for > SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this > IDataReader to have all the data to bulkload then your programs will become > as streamlined as they probably have never been, FYI: IDataReader can be > built in code with data sources being everything as e.g. .csv files etc. > (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) > > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class SqlBulkLoadTest > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;" + > "Initial Catalog=Northwind;User Id=sa;Password=shms"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { > connection.Open(); > string sql = "select * from [Shippers]"; > SqlCommand command = new SqlCommand(sql, connection); > SqlDataAdapter adapter = new SqlDataAdapter(command); > > DataSet dataSet = new DataSet("Shippers"); > adapter.Fill(dataSet); > > Console.WriteLine("*** dataSet ***"); > foreach (DataRow dataRow in dataSet.Tables[0].Rows) > { > object[] items = { dataRow[0], dataRow[1], dataRow[2] }; > > Console.WriteLine("{0} {1} {2}", > dataRow[0], dataRow[1], dataRow[2]); > } > > > (new SqlCommand("delete from [NewShippers]",connection)) > .ExecuteNonQuery(); > > SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, > SqlBulkCopyOptions.KeepIdentity); > bulkCopy.DestinationTableName = "NewShippers"; > bulkCopy.WriteToServer(dataSet.Tables[0]); > } > } > > } > } > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 13, 2009 3:52 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > Shamil, > > That is precisely what I have in mind, with variations. Then at the end, > write all updates back to > SQL Server. > > Thanks for the demo code, always helpful. > > John W. Colby > www.ColbyConsulting.com > > > <<< snip >> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4603 (20091113) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 13 08:30:42 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 09:30:42 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: References: <4AFC446F.6070207@colbyconsulting.com> Message-ID: <4AFD6D92.50807@colbyconsulting.com> Mark, > Just curious, how many records are you talking about - a few hundred or millions? Millions, however I am talking about updating sets of records, from 2 to < 30 rows. Imagine a table where Mark Breen has 22 records in the table, John Colby has 17 records in the table, Shamil has 3 records etc. Each of Mark's records have data in a "random" handful of 525 fields. The objective is to get all of Mark's data into one record. Once Mark's records are processed, move on to John's records. Then process William's records. There are 6,514,623 records to be processed. These 6.5 million records represent perhaps about 3 million actual people. I am creating tables right now to store the hash field and a count of records that hash field represents. As an example I have 9 actual people with more than 10 records to be merged, 4,725 people with 5 records, 270,0839 people with 3 records to merge, and 2,771,885 people with two records to merge. The nice part is that the table I am discussing is not constantly changing, in fact this data is entirely static. And there is no time frame for getting it done. I can start the process running and if it takes all night or all week I don't care. I suspect that with efficient programming I can do this at hundreds of records per second. I intend to do it out in tables dedicated to this process in a database dedicated to this process. All of the updates between fields in different records will be taking place entirely in memory. I will probably pull in all of the records for a thousand people, do the merges, then update those records back to SQL Server and pull another chunk of a thousand records. > Second question, other than for purely academic reasons, why are you bring the data into C# Mostly because I know programming, and TSQL is far enough from my expertise as to be unusable in the immediate future. To be honest I am not sure it is even possible to do this easily in TSQL. That said I am sure some TSQL GURU will pop up with the answer. But I am not a TSQL Guru (nor a Guru of any type for that matter). But the thought of programming this kind of algorithm in TSQL sends shivers down my spine. I need to do the following: R1F1 F1F2 R1F3... R1F525 R2F1 R2F2 R2F3... R2F525 R3F1 R3F2 R3F3... R3F525 The lines above represent Rows R1, R2 and R3, Fields F1,F2,F3...F525 Start with R1 and R2 Start at column 1 If R1F1 is null AND R2F1 is NOT NULL THEN Search through sequential records, same field until data found endif Move to column 2, continue to all columns Move to R1 and R3 In English: There is a set of records for a given person. If R1Col(N) has data, do not update. If R1Col(N) has no data, get data from R2Col(N) If R2Col(N) has no data, get data from R3Col(N) Start at N = (column) 1. Search until data found, update R1Col(N), then abort search and move to next column C# has a wonderful representation of data inside of the data adapter. Each table is a collection of rows AND a collection of columns. So I can iterate through the collection of columns, and once I have a collection of fields in a single column, iterate through that looking at the data in row 1, row 2 row 3 until data is found. This data representation just immediately does 1/2 the work for me by presenting me with my data already loaded into these collections. > Of course if it is to learn datasets then thats another story It is to get work done that needs doing. Along the way I expect to learn datasets and get that much more comfortable with C#. Doesn't this sound like FUN programming? I am a programmer at heart, I love doing this kind of stuff, and C# is where I am going, so why not do it there? John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > Just curious, how many records are you talking about - a few hundred or > millions? > > Second question, other than for purely academic reasons, why are you bring > the data into C#, to concatenate and insert into another table can so easily > be done in one sproc, even if you need a temporary holding place, you would > still use a temp table or even a local table. > > Of course if it is to learn datasets then thats another story, > > Mark From shamil at smsconsulting.spb.ru Fri Nov 13 08:36:06 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 17:36:06 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD6D8A.3060306@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> <004801ca646c$5758e690$060ab3b0$@spb.ru> <4AFD6D8A.3060306@colbyconsulting.com> Message-ID: <004a01ca646e$a3b24570$eb16d050$@spb.ru> Hi John, OK, if you're adding several millions rows then SqlBulkLoad can make your "shot deal" in one shot :) -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 13, 2009 5:31 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? In fact this is a one shot deal. I need to do this one time, to one table. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > You can also consider using SqlBulkLoad - below is just a simple sample, and > you can develop very flexible variations of this approach. SqlBulkLoad > should be as quick as bcp maybe even quicker. In fact the source for > SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this > IDataReader to have all the data to bulkload then your programs will become > as streamlined as they probably have never been, FYI: IDataReader can be > built in code with data sources being everything as e.g. .csv files etc. > (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) > > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class SqlBulkLoadTest > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;" + > "Initial Catalog=Northwind;User Id=sa;Password=N/A"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { > connection.Open(); > string sql = "select * from [Shippers]"; > SqlCommand command = new SqlCommand(sql, connection); > SqlDataAdapter adapter = new SqlDataAdapter(command); > > DataSet dataSet = new DataSet("Shippers"); > adapter.Fill(dataSet); > > Console.WriteLine("*** dataSet ***"); > foreach (DataRow dataRow in dataSet.Tables[0].Rows) > { > object[] items = { dataRow[0], dataRow[1], dataRow[2] }; > > Console.WriteLine("{0} {1} {2}", > dataRow[0], dataRow[1], dataRow[2]); > } > > > (new SqlCommand("delete from [NewShippers]",connection)) > .ExecuteNonQuery(); > > SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, > SqlBulkCopyOptions.KeepIdentity); > bulkCopy.DestinationTableName = "NewShippers"; > bulkCopy.WriteToServer(dataSet.Tables[0]); > } > } > > } > } > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 13, 2009 3:52 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > Shamil, > > That is precisely what I have in mind, with variations. Then at the end, > write all updates back to > SQL Server. > > Thanks for the demo code, always helpful. > > John W. Colby > www.ColbyConsulting.com > > > <<< snip >> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4603 (20091113) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4603 (20091113) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 13 08:40:25 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 09:40:25 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <004a01ca646e$a3b24570$eb16d050$@spb.ru> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> <004801ca646c$5758e690$060ab3b0$@spb.ru> <4AFD6D8A.3060306@colbyconsulting.com> <004a01ca646e$a3b24570$eb16d050$@spb.ru> Message-ID: <4AFD6FD9.2040902@colbyconsulting.com> See my reply to Mark Breen coming up. This is not a bulk load thing. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > OK, if you're adding several millions rows then SqlBulkLoad can make your > "shot deal" in one shot :) > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 13, 2009 5:31 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > In fact this is a one shot deal. I need to do this one time, to one table. > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> You can also consider using SqlBulkLoad - below is just a simple sample, > and >> you can develop very flexible variations of this approach. SqlBulkLoad >> should be as quick as bcp maybe even quicker. In fact the source for >> SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this >> IDataReader to have all the data to bulkload then your programs will > become >> as streamlined as they probably have never been, FYI: IDataReader can be >> built in code with data sources being everything as e.g. .csv files etc. >> (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) >> >> >> using System; >> using System.Collections.Generic; >> using System.Text; >> >> using System.Data; >> using System.Data.SqlClient; >> >> namespace TestConsole.Samples >> { >> public class SqlBulkLoadTest >> { >> public static void Run() >> { >> string connectionString = "Data Source=HAMBURG\\SQL2005;" + >> "Initial Catalog=Northwind;User Id=sa;Password=N/A"; >> using (SqlConnection connection = >> new SqlConnection(connectionString)) >> { >> connection.Open(); >> string sql = "select * from [Shippers]"; >> SqlCommand command = new SqlCommand(sql, connection); >> SqlDataAdapter adapter = new SqlDataAdapter(command); >> >> DataSet dataSet = new DataSet("Shippers"); >> adapter.Fill(dataSet); >> >> Console.WriteLine("*** dataSet ***"); >> foreach (DataRow dataRow in dataSet.Tables[0].Rows) >> { >> object[] items = { dataRow[0], dataRow[1], dataRow[2] > }; >> Console.WriteLine("{0} {1} {2}", >> dataRow[0], dataRow[1], dataRow[2]); >> } >> >> >> (new SqlCommand("delete from [NewShippers]",connection)) >> .ExecuteNonQuery(); >> >> SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, >> SqlBulkCopyOptions.KeepIdentity); >> bulkCopy.DestinationTableName = "NewShippers"; >> bulkCopy.WriteToServer(dataSet.Tables[0]); >> } >> } >> >> } >> } >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: Friday, November 13, 2009 3:52 PM >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] What to do, what to do? >> >> Shamil, >> >> That is precisely what I have in mind, with variations. Then at the end, >> write all updates back to >> SQL Server. >> >> Thanks for the demo code, always helpful. >> >> John W. Colby >> www.ColbyConsulting.com >> >> >> <<< snip >> >> >> >> __________ Information from ESET NOD32 Antivirus, version of virus > signature >> database 4603 (20091113) __________ >> >> The message was checked by ESET NOD32 Antivirus. >> >> http://www.esetnod32.ru >> >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4603 (20091113) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Nov 13 08:54:16 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 17:54:16 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD6FD9.2040902@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> <004801ca646c$5758e690$060ab3b0$@spb.ru> <4AFD6D8A.3060306@colbyconsulting.com> <004a01ca646e$a3b24570$eb16d050$@spb.ru> <4AFD6FD9.2040902@colbyconsulting.com> Message-ID: <004b01ca6471$2d3f5dd0$87be1970$@spb.ru> OK, I see this is "update thing" not insert/bulkload. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 13, 2009 5:40 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? See my reply to Mark Breen coming up. This is not a bulk load thing. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > OK, if you're adding several millions rows then SqlBulkLoad can make your > "shot deal" in one shot :) > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 13, 2009 5:31 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > In fact this is a one shot deal. I need to do this one time, to one table. > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> You can also consider using SqlBulkLoad - below is just a simple sample, > and >> you can develop very flexible variations of this approach. SqlBulkLoad >> should be as quick as bcp maybe even quicker. In fact the source for >> SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this >> IDataReader to have all the data to bulkload then your programs will > become >> as streamlined as they probably have never been, FYI: IDataReader can be >> built in code with data sources being everything as e.g. .csv files etc. >> (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) >> >> >> using System; >> using System.Collections.Generic; >> using System.Text; >> >> using System.Data; >> using System.Data.SqlClient; >> >> namespace TestConsole.Samples >> { >> public class SqlBulkLoadTest >> { >> public static void Run() >> { >> string connectionString = "Data Source=HAMBURG\\SQL2005;" + >> "Initial Catalog=Northwind;User Id=sa;Password=N/A"; >> using (SqlConnection connection = >> new SqlConnection(connectionString)) >> { >> connection.Open(); >> string sql = "select * from [Shippers]"; >> SqlCommand command = new SqlCommand(sql, connection); >> SqlDataAdapter adapter = new SqlDataAdapter(command); >> >> DataSet dataSet = new DataSet("Shippers"); >> adapter.Fill(dataSet); >> >> Console.WriteLine("*** dataSet ***"); >> foreach (DataRow dataRow in dataSet.Tables[0].Rows) >> { >> object[] items = { dataRow[0], dataRow[1], dataRow[2] > }; >> Console.WriteLine("{0} {1} {2}", >> dataRow[0], dataRow[1], dataRow[2]); >> } >> >> >> (new SqlCommand("delete from [NewShippers]",connection)) >> .ExecuteNonQuery(); >> >> SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, >> SqlBulkCopyOptions.KeepIdentity); >> bulkCopy.DestinationTableName = "NewShippers"; >> bulkCopy.WriteToServer(dataSet.Tables[0]); >> } >> } >> >> } >> } >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: Friday, November 13, 2009 3:52 PM >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] What to do, what to do? >> >> Shamil, >> >> That is precisely what I have in mind, with variations. Then at the end, >> write all updates back to >> SQL Server. >> >> Thanks for the demo code, always helpful. >> >> John W. Colby >> www.ColbyConsulting.com >> >> >> <<< snip >> >> From jwcolby at colbyconsulting.com Fri Nov 13 10:39:14 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 11:39:14 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> Message-ID: <4AFD8BB2.5030203@colbyconsulting.com> Shamil, Given what you now know about what I want to do... Why does each table get created in it's own DataSet. I thought a data set could have multiple tables. I envisioned the following structure out in SQL Server: tblHSID - contains 6.4 million records. Source of records for tblMergeSrc. tblMergeSrc - Contains X thousand records for Y hundred people. Working table in memory. tblMergedRecs - the single record merged into will be created here (or copied here). Working table in memory. tblPurgedRecs - The multiple records merged FROM will be archived here. Working table in memory. tblHashPK - Hash for a person / address, PKs for person / address. There is one hash for each person / hash, there are multiple PKs for each person / address. One record for each record in tblHSID. tblHashCnt - Group by Hash, Cnt PK. Server two purposes, provides a "DISTINCT" hash for each person being processed as well as a count of how many records will be merged down. So... tblHSID is the source of the records to be processed. Contains ONLY people who actually have "duplicates", most people don't. The Top(N) records from tblHashCnt will select the Top(N) PEOPLE by number of records to be merged. Merge the ones with the most records first. That will be joined to tblHashPK to pull all of the PKs for the Top(N) people about to be processed. The PKs from that will be joined to tblHSID to pull a set of records to be merged. Copied into tblMergeSrc. This is the first time HSID records (records to be merged) will be moved into a table in a dataset. tblMergedRecs is empty but exists in the dataset. tblPurgedRecs is empty but exists in the dataset. The merge will most likely happen in tblMergeSrc although a new record could be created in tblMergedRecs to work in. Not decided yet. In the end, the merge happens in the dataset in memory. The actual ONE merged record for each person ends up in tblMergedRecs. The actual (multiple) records merged into other records are moved into tblPurgedRecs. This is an archive. The records in tblPurgedRecs are then deleted from tblMergedSrc, and eventually deleted out of tblHSID back in the database. The records for the next person are processed. Merged records moved to tblMergedRecs. "Purged" records moved to tblPurgedRecs. Deleted from tblMergeSrc. Wash, rinse repeat. Now... I don't care how many data sets I have but I just question why I would create a new dataset for each table above. It seems that having them all in a single dataset would minimize confusion and stuff. Logically it seems that working with multiple tables inside of a single DataSet object MIGHT be faster than moving data back and forth between tables in different DataSet objects, though maybe not. And of course... my book talks all about connections, then all about commands, and then all about parameters, then... How to work with a data reader. WTF over? What happened to my dataset and table objects? Not a WORD about those two objects. Sigh. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Here is a sample: > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class DataSetTests > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;"+ > "Initial Catalog=Northwind;User Id=sa;Password=N/A"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { From shamil at smsconsulting.spb.ru Fri Nov 13 11:22:31 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 20:22:31 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD8BB2.5030203@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD8BB2.5030203@colbyconsulting.com> Message-ID: <005b01ca6485$e42c31d0$ac849570$@spb.ru> John, Yes, of course you can have multiple DataTable(s) in a DataSet - I have just posted a very simple sample - one of a myriad possible combinations, the simplest one probably. And of course you can have your DataSet(s) structure created statically in VS designer. Yes, I think your plan is OK. You probably do not need a DataReader but if you'll find you need one - in P.S. is a simple sample. In fact Gustav knows DataSets better than I do - please address your questions to him also and to everybody else here - I will not have time to reply any more postings today... Thank you. -- Shamil P.S. DataReader sample: using System; using System.Collections.Generic; using System.Text; using System.Data; using System.Data.SqlClient; namespace TestConsole.Samples { public class SqlDataReaderTest { public static void Run() { string connectionString = "Data Source=HAMBURG\\SQL2005;" + "Initial Catalog=Northwind;User Id=sa;Password=N/A"; using (SqlConnection connection = new SqlConnection(connectionString)) { connection.Open(); // Create empty [NewShippers] dataset in memory string sql1 = "select * from [NewShippers] where(1=0)"; SqlCommand command = new SqlCommand(sql1, connection); SqlDataAdapter adapter = new SqlDataAdapter(command); DataSet newDataSet = new DataSet("NewShippers"); adapter.Fill(newDataSet); // Load source [Shippers] in memory string sql = "select * from [Shippers]"; command = new SqlCommand(sql, connection); SqlDataReader dataReader = command.ExecuteReader(); // Add loaded [Shippers] to [NewShippers] im memory Console.WriteLine("*** dataReader ***"); while (dataReader.Read()) { object[] items = { dataReader[0], dataReader[1], dataReader[2] }; newDataSet.Tables[0].Rows.Add(items); Console.WriteLine("{0} {1} {2}", items[0], items[1], items[2]); } dataReader.Close(); // ! always close DataReader // Insert [NewShippers] into db (new SqlCommand("delete from [NewShippers]",connection)) .ExecuteNonQuery(); SqlCommandBuilder builder = new SqlCommandBuilder(adapter); adapter.InsertCommand = builder.GetInsertCommand(true); adapter.Update(newDataSet.Tables[0]); // Check that [NewShippers] were inserted OK Console.WriteLine("*** newDataSet ***"); newDataSet.Clear(); adapter = new SqlDataAdapter( new SqlCommand("select * from [NewShippers]",connection)); adapter.Fill(newDataSet); foreach (DataRow dataRow in newDataSet.Tables[0].Rows) { Console.WriteLine("{0} {1} {2}", dataRow[0], dataRow[1], dataRow[2]); } } } } } <<< snip to get through dba-VB robot >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4604 (20091113) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From cfoust at infostatsystems.com Fri Nov 13 12:18:07 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Fri, 13 Nov 2009 10:18:07 -0800 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD8BB2.5030203@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com><002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD8BB2.5030203@colbyconsulting.com> Message-ID: You can have all sorts of tables in a dataset, John. But if you want to act on multiple non-cascading tables, then you probably need to validate multiple tables. If you have relationships set so you can cascade updates, it isn't too bad. Otherwise, you have to write code to individually handle the gotchas in each table before you update any of them. Without that, you'd wind up with a partial update. Some of my biggest headaches are the one rich typed dataset we use to allow the users to enter data into 3 or 4 different tables from a single form. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 13, 2009 8:39 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? Shamil, Given what you now know about what I want to do... Why does each table get created in it's own DataSet. I thought a data set could have multiple tables. I envisioned the following structure out in SQL Server: tblHSID - contains 6.4 million records. Source of records for tblMergeSrc. tblMergeSrc - Contains X thousand records for Y hundred people. Working table in memory. tblMergedRecs - the single record merged into will be created here (or copied here). Working table in memory. tblPurgedRecs - The multiple records merged FROM will be archived here. Working table in memory. tblHashPK - Hash for a person / address, PKs for person / address. There is one hash for each person / hash, there are multiple PKs for each person / address. One record for each record in tblHSID. tblHashCnt - Group by Hash, Cnt PK. Server two purposes, provides a "DISTINCT" hash for each person being processed as well as a count of how many records will be merged down. So... tblHSID is the source of the records to be processed. Contains ONLY people who actually have "duplicates", most people don't. The Top(N) records from tblHashCnt will select the Top(N) PEOPLE by number of records to be merged. Merge the ones with the most records first. That will be joined to tblHashPK to pull all of the PKs for the Top(N) people about to be processed. The PKs from that will be joined to tblHSID to pull a set of records to be merged. Copied into tblMergeSrc. This is the first time HSID records (records to be merged) will be moved into a table in a dataset. tblMergedRecs is empty but exists in the dataset. tblPurgedRecs is empty but exists in the dataset. The merge will most likely happen in tblMergeSrc although a new record could be created in tblMergedRecs to work in. Not decided yet. In the end, the merge happens in the dataset in memory. The actual ONE merged record for each person ends up in tblMergedRecs. The actual (multiple) records merged into other records are moved into tblPurgedRecs. This is an archive. The records in tblPurgedRecs are then deleted from tblMergedSrc, and eventually deleted out of tblHSID back in the database. The records for the next person are processed. Merged records moved to tblMergedRecs. "Purged" records moved to tblPurgedRecs. Deleted from tblMergeSrc. Wash, rinse repeat. Now... I don't care how many data sets I have but I just question why I would create a new dataset for each table above. It seems that having them all in a single dataset would minimize confusion and stuff. Logically it seems that working with multiple tables inside of a single DataSet object MIGHT be faster than moving data back and forth between tables in different DataSet objects, though maybe not. And of course... my book talks all about connections, then all about commands, and then all about parameters, then... How to work with a data reader. WTF over? What happened to my dataset and table objects? Not a WORD about those two objects. Sigh. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Here is a sample: > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class DataSetTests > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;"+ > "Initial Catalog=Northwind;User Id=sa;Password=N/A"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From raibeart at gmail.com Fri Nov 13 22:46:18 2009 From: raibeart at gmail.com (Robert Stewart) Date: Fri, 13 Nov 2009 22:46:18 -0600 Subject: [dba-VB] What to do, what to do? In-Reply-To: References: Message-ID: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert At 12:00 PM 11/13/2009, you wrote: >Date: Fri, 13 Nov 2009 11:39:14 -0500 >From: jwcolby >Subject: Re: [dba-VB] What to do, what to do? >To: "Discussion concerning Visual Basic and related programming > issues." >Message-ID: <4AFD8BB2.5030203 at colbyconsulting.com> >Content-Type: text/plain; charset=ISO-8859-1; format=flowed > >Shamil, > >Given what you now know about what I want to do... > >Why does each table get created in it's own DataSet. I thought a >data set could have multiple >tables. I envisioned the following structure out in SQL Server: > >tblHSID - contains 6.4 million records. Source of records for tblMergeSrc. >tblMergeSrc - Contains X thousand records for Y hundred >people. Working table in memory. >tblMergedRecs - the single record merged into will be created here >(or copied here). Working table >in memory. >tblPurgedRecs - The multiple records merged FROM will be archived >here. Working table in memory. > >tblHashPK - Hash for a person / address, PKs for person / >address. There is one hash for each >person / hash, there are multiple PKs for each person / >address. One record for each record in >tblHSID. > >tblHashCnt - Group by Hash, Cnt PK. Server two purposes, provides a >"DISTINCT" hash for each person >being processed as well as a count of how many records will be merged down. > >So... tblHSID is the source of the records to be >processed. Contains ONLY people who actually have >"duplicates", most people don't. > >The Top(N) records from tblHashCnt will select the Top(N) PEOPLE by >number of records to be merged. > Merge the ones with the most records first. > >That will be joined to tblHashPK to pull all of the PKs for the >Top(N) people about to be processed. > >The PKs from that will be joined to tblHSID to pull a set of records >to be merged. Copied into >tblMergeSrc. This is the first time HSID records (records to be >merged) will be moved into a table >in a dataset. > >tblMergedRecs is empty but exists in the dataset. >tblPurgedRecs is empty but exists in the dataset. > >The merge will most likely happen in tblMergeSrc although a new >record could be created in >tblMergedRecs to work in. Not decided yet. > >In the end, the merge happens in the dataset in memory. The actual >ONE merged record for each >person ends up in tblMergedRecs. > >The actual (multiple) records merged into other records are moved >into tblPurgedRecs. This is an >archive. > >The records in tblPurgedRecs are then deleted from tblMergedSrc, and >eventually deleted out of >tblHSID back in the database. > >The records for the next person are processed. Merged records moved >to tblMergedRecs. "Purged" >records moved to tblPurgedRecs. > >Deleted from tblMergeSrc. > >Wash, rinse repeat. > >Now... I don't care how many data sets I have but I just question >why I would create a new dataset >for each table above. It seems that having them all in a single >dataset would minimize confusion >and stuff. Logically it seems that working with multiple tables >inside of a single DataSet object >MIGHT be faster than moving data back and forth between tables in >different DataSet objects, though >maybe not. > >And of course... my book talks all about connections, then all about >commands, and then all about >parameters, then... > >How to work with a data reader. WTF over? What happened to my >dataset and table objects? Not a >WORD about those two objects. > >Sigh. > >John W. Colby From shamil at smsconsulting.spb.ru Sat Nov 14 04:40:43 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 14 Nov 2009 13:40:43 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> Message-ID: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Hi Robert, Yes, DataSets are mainly targeted for working with small number of records in memory - so the following is more a "theoretical" calculation: - JC has 64bit PC which allow to load in memory "practically unlimited" volume of data - 64bit logical (process) address space could be as large as 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 (http://en.wikipedia.org/wiki/Exbibyte). - the speed of data transfer for 64bit Intel's processor - "the 200 MHz McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); ----- So "theoretically" with modern speedy harddisks JC can load 50 millions of records into memory within minutes - within half a minute in the near future when large enough flash-memory disks will become available and relatively inexpensive? To load such large data volume it would be better to use SqlDataReader as it keeps loaded data in a very compact form. And 50 million records long SqlDataReader should be probably better split into several chunks... Again - this is just a "theoretical" consideration - in practice JC can process his 50 million records long data table in chunks as he has a hash field (tblHashPK), which can be used to load related records into memory: I mean he can split tblHashPK table keeping hash values into several groups, e.g.: 1 - 100,000 100,001 - 200,000 ... and process each group joining its records to source 50 million records... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart Sent: Saturday, November 14, 2009 7:46 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] What to do, what to do? John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From max.wanadoo at gmail.com Sat Nov 14 05:30:41 2009 From: max.wanadoo at gmail.com (Max Wanadoo) Date: Sat, 14 Nov 2009 11:30:41 -0000 Subject: [dba-VB] What to do, what to do? In-Reply-To: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Message-ID: <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> Very interesting Shamil What happens with all the other stuff that memory is taking up though, OS, Programs, etc and the physical memory he has placed in the machine? Your limits below presumably are saying that his physical memory is loaded to...what? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 10:41 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Robert, Yes, DataSets are mainly targeted for working with small number of records in memory - so the following is more a "theoretical" calculation: - JC has 64bit PC which allow to load in memory "practically unlimited" volume of data - 64bit logical (process) address space could be as large as 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 (http://en.wikipedia.org/wiki/Exbibyte). - the speed of data transfer for 64bit Intel's processor - "the 200 MHz McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); ----- So "theoretically" with modern speedy harddisks JC can load 50 millions of records into memory within minutes - within half a minute in the near future when large enough flash-memory disks will become available and relatively inexpensive? To load such large data volume it would be better to use SqlDataReader as it keeps loaded data in a very compact form. And 50 million records long SqlDataReader should be probably better split into several chunks... Again - this is just a "theoretical" consideration - in practice JC can process his 50 million records long data table in chunks as he has a hash field (tblHashPK), which can be used to load related records into memory: I mean he can split tblHashPK table keeping hash values into several groups, e.g.: 1 - 100,000 100,001 - 200,000 ... and process each group joining its records to source 50 million records... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart Sent: Saturday, November 14, 2009 7:46 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] What to do, what to do? John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Nov 14 06:30:35 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 14 Nov 2009 07:30:35 -0500 Subject: [dba-VB] SPAM-LOW: Re: What to do, what to do? In-Reply-To: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> Message-ID: <4AFEA2EB.5050507@colbyconsulting.com> Read the discussion carefully Robert. I am discussing doing chunks of thousands. John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > John, > > I don't think your machine, or any for that matter, have the memory > to load 50 million records, extract 40 million, transform them and > then load them back to the 50 million and update the database from > the recordset. Datasets are for working with a small number of > records in memory, not millions. > > Robert > > > At 12:00 PM 11/13/2009, you wrote: >> Date: Fri, 13 Nov 2009 11:39:14 -0500 >> From: jwcolby >> Subject: Re: [dba-VB] What to do, what to do? >> To: "Discussion concerning Visual Basic and related programming >> issues." >> Message-ID: <4AFD8BB2.5030203 at colbyconsulting.com> >> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >> >> Shamil, >> >> Given what you now know about what I want to do... >> >> Why does each table get created in it's own DataSet. I thought a >> data set could have multiple >> tables. I envisioned the following structure out in SQL Server: >> >> tblHSID - contains 6.4 million records. Source of records for tblMergeSrc. >> tblMergeSrc - Contains X thousand records for Y hundred >> people. Working table in memory. >> tblMergedRecs - the single record merged into will be created here >> (or copied here). Working table >> in memory. >> tblPurgedRecs - The multiple records merged FROM will be archived >> here. Working table in memory. >> >> tblHashPK - Hash for a person / address, PKs for person / >> address. There is one hash for each >> person / hash, there are multiple PKs for each person / >> address. One record for each record in >> tblHSID. >> >> tblHashCnt - Group by Hash, Cnt PK. Server two purposes, provides a >> "DISTINCT" hash for each person >> being processed as well as a count of how many records will be merged down. >> >> So... tblHSID is the source of the records to be >> processed. Contains ONLY people who actually have >> "duplicates", most people don't. >> >> The Top(N) records from tblHashCnt will select the Top(N) PEOPLE by >> number of records to be merged. >> Merge the ones with the most records first. >> >> That will be joined to tblHashPK to pull all of the PKs for the >> Top(N) people about to be processed. >> >> The PKs from that will be joined to tblHSID to pull a set of records >> to be merged. Copied into >> tblMergeSrc. This is the first time HSID records (records to be >> merged) will be moved into a table >> in a dataset. >> >> tblMergedRecs is empty but exists in the dataset. >> tblPurgedRecs is empty but exists in the dataset. >> >> The merge will most likely happen in tblMergeSrc although a new >> record could be created in >> tblMergedRecs to work in. Not decided yet. >> >> In the end, the merge happens in the dataset in memory. The actual >> ONE merged record for each >> person ends up in tblMergedRecs. >> >> The actual (multiple) records merged into other records are moved >> into tblPurgedRecs. This is an >> archive. >> >> The records in tblPurgedRecs are then deleted from tblMergedSrc, and >> eventually deleted out of >> tblHSID back in the database. >> >> The records for the next person are processed. Merged records moved >> to tblMergedRecs. "Purged" >> records moved to tblPurgedRecs. >> >> Deleted from tblMergeSrc. >> >> Wash, rinse repeat. >> >> Now... I don't care how many data sets I have but I just question >> why I would create a new dataset >> for each table above. It seems that having them all in a single >> dataset would minimize confusion >> and stuff. Logically it seems that working with multiple tables >> inside of a single DataSet object >> MIGHT be faster than moving data back and forth between tables in >> different DataSet objects, though >> maybe not. >> >> And of course... my book talks all about connections, then all about >> commands, and then all about >> parameters, then... >> >> How to work with a data reader. WTF over? What happened to my >> dataset and table objects? Not a >> WORD about those two objects. >> >> Sigh. >> >> John W. Colby > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Nov 14 07:46:21 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 14 Nov 2009 16:46:21 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> Message-ID: <008201ca6530$daf14140$90d3c3c0$@spb.ru> Hi Max, But OS, programs etc. - they all use their own logical address space, which is mapped to (shared (using OS directed swapping)) physical address space: with JC 64bit PC OS image/runtime will be kept in its own physical memory segments - and no swapping will occur, MS SQL image/runtime/data will be (mainly) kept in its own physical memory segments - and almost no swapping will occur, JC's program will have occupied its own physical memory segments - and no swapping will occur... 50 million records assuming every of them is equal to approx. 1KB will consume ~50GB - 64bit PC can have(?) that much RAM allocated for one process, and it all can be addresses directly within this process logical address space. And I have noted that it's currently a "theoretical consideration" but it may become practically achievable in not that far future(?) Correct/wrong? (I must say I didn't get checked how large can be logical address space for a Win7 process running on 64bit PC). Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Max Wanadoo Sent: Saturday, November 14, 2009 2:31 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Very interesting Shamil What happens with all the other stuff that memory is taking up though, OS, Programs, etc and the physical memory he has placed in the machine? Your limits below presumably are saying that his physical memory is loaded to...what? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 10:41 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Robert, Yes, DataSets are mainly targeted for working with small number of records in memory - so the following is more a "theoretical" calculation: - JC has 64bit PC which allow to load in memory "practically unlimited" volume of data - 64bit logical (process) address space could be as large as 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 (http://en.wikipedia.org/wiki/Exbibyte). - the speed of data transfer for 64bit Intel's processor - "the 200 MHz McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); ----- So "theoretically" with modern speedy harddisks JC can load 50 millions of records into memory within minutes - within half a minute in the near future when large enough flash-memory disks will become available and relatively inexpensive? To load such large data volume it would be better to use SqlDataReader as it keeps loaded data in a very compact form. And 50 million records long SqlDataReader should be probably better split into several chunks... Again - this is just a "theoretical" consideration - in practice JC can process his 50 million records long data table in chunks as he has a hash field (tblHashPK), which can be used to load related records into memory: I mean he can split tblHashPK table keeping hash values into several groups, e.g.: 1 - 100,000 100,001 - 200,000 ... and process each group joining its records to source 50 million records... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart Sent: Saturday, November 14, 2009 7:46 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] What to do, what to do? John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 14 08:23:42 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 14 Nov 2009 09:23:42 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Message-ID: <4AFEBD6E.1020709@colbyconsulting.com> All of which is beside the point. Divide and conquer. A principal discovered by military strategists thousands of years ago, and applied to problems of all sorts today. I can EASILY work with sets of thousands, and even hundreds of thousands of records. In real life, in my computer. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Robert, > > Yes, DataSets are mainly targeted for working with small number of records > in memory - so the following is more a "theoretical" calculation: > > - JC has 64bit PC which allow to load in memory "practically unlimited" > volume of data - 64bit logical (process) address space could be as large as > 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 > (http://en.wikipedia.org/wiki/Exbibyte). > > - the speed of data transfer for 64bit Intel's processor - "the 200 MHz > McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers > 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); > > ----- > So "theoretically" with modern speedy harddisks JC can load 50 millions of > records into memory within minutes - within half a minute in the near future > when large enough flash-memory disks will become available and relatively > inexpensive? > > To load such large data volume it would be better to use SqlDataReader as it > keeps loaded data in a very compact form. And 50 million records long > SqlDataReader should be probably better split into several chunks... > > Again - this is just a "theoretical" consideration - in practice JC can > process his 50 million records long data table in chunks as he has a hash > field (tblHashPK), which can be used to load related records into memory: I > mean he can split tblHashPK table keeping hash values into several groups, > e.g.: > > 1 - 100,000 > 100,001 - 200,000 > ... > > and process each group joining its records to source 50 million records... > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart > Sent: Saturday, November 14, 2009 7:46 AM > To: dba-vb at databaseadvisors.com > Subject: Re: [dba-VB] What to do, what to do? > > John, > > I don't think your machine, or any for that matter, have the memory > to load 50 million records, extract 40 million, transform them and > then load them back to the 50 million and update the database from > the recordset. Datasets are for working with a small number of > records in memory, not millions. > > Robert > > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4606 (20091114) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Nov 14 08:44:11 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 14 Nov 2009 17:44:11 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFEBD6E.1020709@colbyconsulting.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4AFEBD6E.1020709@colbyconsulting.com> Message-ID: <008301ca6538$eeef8910$ccce9b30$@spb.ru> Hi John, Yes, I have just "theoretically" noted that in the near future you'll be able to EVEN MORE EASILY work with sets of millions (and billions?) records by just loading them into memory, "crunching", and saving back updated... ... it will be just 1,2,3 instead of your today's (1,2,3) (1,2,3)... (1,2,3) (multi-threaded)... ... the time just to develop (1,2,3) (1,2,3)... (1,2,3)... will be more expensive than time to develop and run 1,2,3 on the near future hardware (hardware costs included)... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 14, 2009 5:24 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? All of which is beside the point. Divide and conquer. A principal discovered by military strategists thousands of years ago, and applied to problems of all sorts today. I can EASILY work with sets of thousands, and even hundreds of thousands of records. In real life, in my computer. John W. Colby www.ColbyConsulting.com <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4607 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From max.wanadoo at gmail.com Sat Nov 14 09:38:39 2009 From: max.wanadoo at gmail.com (Max Wanadoo) Date: Sat, 14 Nov 2009 15:38:39 -0000 Subject: [dba-VB] What to do, what to do? In-Reply-To: <008201ca6530$daf14140$90d3c3c0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> <008201ca6530$daf14140$90d3c3c0$@spb.ru> Message-ID: <4afecf2a.0702d00a.3bd1.4040@mx.google.com> Hi Shamil, I understand the logical size bit but for that to be mapped to physical memory requires physical memory to be present. Current there are limits to how much can be installed on the motherboard. 64Bit memory can address the sizes you mentioned but the physical memory doesn't exist, so (presumably) it maps it to the HD, in which case it is not in memory. Have I misunderstood something (it has been known to happen)? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 13:46 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Max, But OS, programs etc. - they all use their own logical address space, which is mapped to (shared (using OS directed swapping)) physical address space: with JC 64bit PC OS image/runtime will be kept in its own physical memory segments - and no swapping will occur, MS SQL image/runtime/data will be (mainly) kept in its own physical memory segments - and almost no swapping will occur, JC's program will have occupied its own physical memory segments - and no swapping will occur... 50 million records assuming every of them is equal to approx. 1KB will consume ~50GB - 64bit PC can have(?) that much RAM allocated for one process, and it all can be addresses directly within this process logical address space. And I have noted that it's currently a "theoretical consideration" but it may become practically achievable in not that far future(?) Correct/wrong? (I must say I didn't get checked how large can be logical address space for a Win7 process running on 64bit PC). Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Max Wanadoo Sent: Saturday, November 14, 2009 2:31 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Very interesting Shamil What happens with all the other stuff that memory is taking up though, OS, Programs, etc and the physical memory he has placed in the machine? Your limits below presumably are saying that his physical memory is loaded to...what? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 10:41 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Robert, Yes, DataSets are mainly targeted for working with small number of records in memory - so the following is more a "theoretical" calculation: - JC has 64bit PC which allow to load in memory "practically unlimited" volume of data - 64bit logical (process) address space could be as large as 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 (http://en.wikipedia.org/wiki/Exbibyte). - the speed of data transfer for 64bit Intel's processor - "the 200 MHz McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); ----- So "theoretically" with modern speedy harddisks JC can load 50 millions of records into memory within minutes - within half a minute in the near future when large enough flash-memory disks will become available and relatively inexpensive? To load such large data volume it would be better to use SqlDataReader as it keeps loaded data in a very compact form. And 50 million records long SqlDataReader should be probably better split into several chunks... Again - this is just a "theoretical" consideration - in practice JC can process his 50 million records long data table in chunks as he has a hash field (tblHashPK), which can be used to load related records into memory: I mean he can split tblHashPK table keeping hash values into several groups, e.g.: 1 - 100,000 100,001 - 200,000 ... and process each group joining its records to source 50 million records... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart Sent: Saturday, November 14, 2009 7:46 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] What to do, what to do? John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Sat Nov 14 10:08:26 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 14 Nov 2009 19:08:26 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4afecf2a.0702d00a.3bd1.4040@mx.google.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> <008201ca6530$daf14140$90d3c3c0$@spb.ru> <4afecf2a.0702d00a.3bd1.4040@mx.google.com> Message-ID: <008401ca6544$b4290890$1c7b19b0$@spb.ru> Hi Max, Here are theoretical limits I have found info about: Address space 64-bit Windows 32-bit Windows Virtual memory 16 Tb 4 Gb Swap file 512 Tb 16 Tb System cache 1 Tb 1 Gb In practice, mass market motherboards can already handle 64GB(?) - Not enough for JC tasks but this max RAM size limitations should be removed in the near future AFAICG, and Win7 is ready(?) to handle effectively that much RAM.... http://www.nextag.com/64-bit-pci-motherboard/search-html http://www.viva64.com/content/articles/64-bit-development/?f=Application_por t_to_64-bit_platforms_or_never_cackle_till_your_egg_is_laid.html&lang=en&con tent=64-bit-development http://blogs.zdnet.com/hardware/?p=4254 http://www.tomshardware.com/reviews/vista-workshop,1775-3.html Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Max Wanadoo Sent: Saturday, November 14, 2009 6:39 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Shamil, I understand the logical size bit but for that to be mapped to physical memory requires physical memory to be present. Current there are limits to how much can be installed on the motherboard. 64Bit memory can address the sizes you mentioned but the physical memory doesn't exist, so (presumably) it maps it to the HD, in which case it is not in memory. Have I misunderstood something (it has been known to happen)? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 13:46 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Max, But OS, programs etc. - they all use their own logical address space, which is mapped to (shared (using OS directed swapping)) physical address space: with JC 64bit PC OS image/runtime will be kept in its own physical memory segments - and no swapping will occur, MS SQL image/runtime/data will be (mainly) kept in its own physical memory segments - and almost no swapping will occur, JC's program will have occupied its own physical memory segments - and no swapping will occur... 50 million records assuming every of them is equal to approx. 1KB will consume ~50GB - 64bit PC can have(?) that much RAM allocated for one process, and it all can be addresses directly within this process logical address space. And I have noted that it's currently a "theoretical consideration" but it may become practically achievable in not that far future(?) Correct/wrong? (I must say I didn't get checked how large can be logical address space for a Win7 process running on 64bit PC). Thank you. -- Shamil <<< snipped >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4607 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From max.wanadoo at gmail.com Sat Nov 14 10:27:00 2009 From: max.wanadoo at gmail.com (Max Wanadoo) Date: Sat, 14 Nov 2009 16:27:00 -0000 Subject: [dba-VB] What to do, what to do? In-Reply-To: <008401ca6544$b4290890$1c7b19b0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> <008201ca6530$daf14140$90d3c3c0$@spb.ru> <4afecf2a.0702d00a.3bd1.4040@mx.google.com> <008401ca6544$b4290890$1c7b19b0$@spb.ru> Message-ID: <4afeda84.0508d00a.3cd0.ffffe9aa@mx.google.com> Ok, I thought you were referring to ram available now on common-or-garden PCs. So, we can look forward to physical ram being cheap enough to enable in-vitro processing for millions of records. Hurrah! And goodbye to latency... Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 16:08 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Max, Here are theoretical limits I have found info about: Address space 64-bit Windows 32-bit Windows Virtual memory 16 Tb 4 Gb Swap file 512 Tb 16 Tb System cache 1 Tb 1 Gb In practice, mass market motherboards can already handle 64GB(?) - Not enough for JC tasks but this max RAM size limitations should be removed in the near future AFAICG, and Win7 is ready(?) to handle effectively that much RAM.... http://www.nextag.com/64-bit-pci-motherboard/search-html http://www.viva64.com/content/articles/64-bit-development/?f=Application_por t_to_64-bit_platforms_or_never_cackle_till_your_egg_is_laid.html&lang=en&con tent=64-bit-development http://blogs.zdnet.com/hardware/?p=4254 http://www.tomshardware.com/reviews/vista-workshop,1775-3.html Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Max Wanadoo Sent: Saturday, November 14, 2009 6:39 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Shamil, I understand the logical size bit but for that to be mapped to physical memory requires physical memory to be present. Current there are limits to how much can be installed on the motherboard. 64Bit memory can address the sizes you mentioned but the physical memory doesn't exist, so (presumably) it maps it to the HD, in which case it is not in memory. Have I misunderstood something (it has been known to happen)? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 13:46 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Max, But OS, programs etc. - they all use their own logical address space, which is mapped to (shared (using OS directed swapping)) physical address space: with JC 64bit PC OS image/runtime will be kept in its own physical memory segments - and no swapping will occur, MS SQL image/runtime/data will be (mainly) kept in its own physical memory segments - and almost no swapping will occur, JC's program will have occupied its own physical memory segments - and no swapping will occur... 50 million records assuming every of them is equal to approx. 1KB will consume ~50GB - 64bit PC can have(?) that much RAM allocated for one process, and it all can be addresses directly within this process logical address space. And I have noted that it's currently a "theoretical consideration" but it may become practically achievable in not that far future(?) Correct/wrong? (I must say I didn't get checked how large can be logical address space for a Win7 process running on 64bit PC). Thank you. -- Shamil <<< snipped >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4607 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Nov 14 10:28:14 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 14 Nov 2009 11:28:14 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <008301ca6538$eeef8910$ccce9b30$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4AFEBD6E.1020709@colbyconsulting.com> <008301ca6538$eeef8910$ccce9b30$@spb.ru> Message-ID: <4AFEDA9E.9050101@colbyconsulting.com> And believe me I look forward to the day. And it will indeed be soon, of course the data sets get bigger as well. ;) I already have 16 gigs of RAM in my servers, and quad cores. I could easily have 64 gigs and 16 CPUs today if I had the bucks. And of course you are correct as well, the Flash drives are wicked fast for working with large data sets. They still have real issues when writing to them bot for reading they are an order of magnitude faster than rotating media. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > Yes, I have just "theoretically" noted that in the near future you'll be > able to EVEN MORE EASILY work with sets of millions (and billions?) records > by just loading them into memory, "crunching", and saving back updated... > > ... it will be just 1,2,3 instead of your today's (1,2,3) (1,2,3)... (1,2,3) > (multi-threaded)... > > ... the time just to develop (1,2,3) (1,2,3)... (1,2,3)... will be more > expensive than time to develop and run 1,2,3 on the near future hardware > (hardware costs included)... > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 14, 2009 5:24 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > All of which is beside the point. > > Divide and conquer. A principal discovered by military strategists > thousands of years ago, and > applied to problems of all sorts today. > > I can EASILY work with sets of thousands, and even hundreds of thousands of > records. In real life, > in my computer. > > John W. Colby > www.ColbyConsulting.com > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4607 (20091114) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Nov 14 16:36:57 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 14 Nov 2009 17:36:57 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Message-ID: <4AFF3109.40003@colbyconsulting.com> To get back to reality, I really need to process under 6 million records, and the vast majority will be merging two records into one. I will almost certainly pull about 10,000 records at a time, so I will be processing 5,000 people at a time. Merge two records, merge two more, do that ~ 5000 times, write back to SQL Server, load the next set of 10,000 records. Do that 600 times. I will see how many records can actually be loaded, perhaps 100,000 at a time would work and do that 60 times. Maybe break the process into threads. One thread watches the ADO recordsets in memory and keeps them filled, another thread performs the merge, another thread writes back to SQL Server. In any case there is simply no need to load all 6 million records at once. To get back to the original point however, while TSQL might be able to do what I need, I have serious doubts. Even if it could, trying to debug this kind of program written in TSQL would be a nightmare for me. Furthermore even TSQL could not do this on 6 million records at once. No sane person claims that any specific language is the best for everything, and for this kind of thing ADO and C# is probably a good choice. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Robert, > > Yes, DataSets are mainly targeted for working with small number of records > in memory - so the following is more a "theoretical" calculation: > From jwcolby at colbyconsulting.com Sat Nov 14 23:11:48 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 15 Nov 2009 00:11:48 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Message-ID: <4AFF8D94.8010409@colbyconsulting.com> http://arstechnica.com/hardware/news/2009/11/biography-solid-state-disk.ars John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Robert, > > Yes, DataSets are mainly targeted for working with small number of records > in memory - so the following is more a "theoretical" calculation: > > - JC has 64bit PC which allow to load in memory "practically unlimited" > volume of data - 64bit logical (process) address space could be as large as > 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 > (http://en.wikipedia.org/wiki/Exbibyte). > > - the speed of data transfer for 64bit Intel's processor - "the 200 MHz > McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers > 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); > > ----- > So "theoretically" with modern speedy harddisks JC can load 50 millions of > records into memory within minutes - within half a minute in the near future > when large enough flash-memory disks will become available and relatively > inexpensive? > > To load such large data volume it would be better to use SqlDataReader as it > keeps loaded data in a very compact form. And 50 million records long > SqlDataReader should be probably better split into several chunks... > > Again - this is just a "theoretical" consideration - in practice JC can > process his 50 million records long data table in chunks as he has a hash > field (tblHashPK), which can be used to load related records into memory: I > mean he can split tblHashPK table keeping hash values into several groups, > e.g.: > > 1 - 100,000 > 100,001 - 200,000 > ... > > and process each group joining its records to source 50 million records... > > -- > Shamil From jwcolby at colbyconsulting.com Mon Nov 16 07:42:52 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 16 Nov 2009 08:42:52 -0500 Subject: [dba-VB] Archives down Message-ID: <4B0156DC.5000407@colbyconsulting.com> I am unable to get at the archives for DBA-VB. Does anyone here use the ADO.Net Entity Framework? Why? Does anyone here specifically AVOID using the ADO.Net Entity Framework? Why? Thanks, -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Mon Nov 16 07:59:09 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 16 Nov 2009 16:59:09 +0300 Subject: [dba-VB] Archives down In-Reply-To: <4B0156DC.5000407@colbyconsulting.com> References: <4B0156DC.5000407@colbyconsulting.com> Message-ID: <00d901ca66c4$f94089e0$ebc19da0$@spb.ru> Hi John -- ADO.NET EF is a great thing. But maybe it will be wise to wait to use it in production GUI when VS2010 will be released then it will be possible to have natively bound WinForms and WPF Forms (and ASP.NET controls? And SilverLight controls), and where ADO.NET EF entities' mapping to db objects will be more flexible, and where it will be possible to have entity model driven(?) (database model) development... I have recently posted here a link to the following presentation: "ORM, EDM, ESQL, Entity Framework, LINQ to SQL, LINQ to Entities - Confused?" http://www.infoq.com/presentations/ORM-LINQ-Entity-Framework-Eric-Nelson I suppose it will answer most of your questions. BTW, our http://northwind.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=26600 project has a couple of ADO.NET EF samples - SPJWinFormsForEF_VS2008.zip source code, 838K, uploaded Jul 23 Application SPJWinFormsForEF_Executables.zip application, 203K, uploaded Jul 23 Thank you. -- Shamil P.S. BTW, I have mainly skipped "LINQ to SQL" - and I'm now happy (:)) I will not need to rewrite my applications as "LINQ to SQL" development is stopped there at MS AFAIU... -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, November 16, 2009 4:43 PM To: VBA Subject: [dba-VB] Archives down I am unable to get at the archives for DBA-VB. Does anyone here use the ADO.Net Entity Framework? Why? Does anyone here specifically AVOID using the ADO.Net Entity Framework? Why? Thanks, -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4611 (20091116) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From Gustav at cactus.dk Mon Nov 16 09:24:01 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 16 Nov 2009 16:24:01 +0100 Subject: [dba-VB] Archives down Message-ID: Hi John I like the concept and the design features, but have only used it for experiments and testing. It takes some time to get around it - I haven't concluded the tour yet ... /gustav >>> jwcolby at colbyconsulting.com 16-11-2009 14:42 >>> I am unable to get at the archives for DBA-VB. Does anyone here use the ADO.Net Entity Framework? Why? Does anyone here specifically AVOID using the ADO.Net Entity Framework? Why? Thanks, -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Tue Nov 17 17:04:12 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Tue, 17 Nov 2009 18:04:12 -0500 Subject: [dba-VB] First little applet Message-ID: <4B032BEC.9050508@colbyconsulting.com> Today I wrote my first C# applet. I have to import CSV files coming back from a processing house into SQL Server. These files had errors in one or more lines in the files. Given that there are tens of millions of lines in each file it isn't possible to just poke around and find the errors. So I wrote an applet that opens the CSV file, reads the header line and counts the " and , characters. Then line by line the applet reads the rest of the lines, comparing the counts to that header line counts. This found lines in the file to be imported where " characters had been dropped, which was causing the import wizard to error out at that point. So read, compare counts, write good lines to output file, write bad lines to an error file. Simple stuff really but it is my first real work done in C#. The speed of this kind of thing is pretty darned good too I thought. I was getting about 300K lines / second read / written. One file had 20.6 million lines, the other had 49 million lines. In the past I would have done this in VBA / Access. Of course it took me a tad longer to write in C# than it would have in VBA. -- John W. Colby www.ColbyConsulting.com From marklbreen at gmail.com Wed Nov 18 02:54:19 2009 From: marklbreen at gmail.com (Mark Breen) Date: Wed, 18 Nov 2009 08:54:19 +0000 Subject: [dba-VB] First little applet In-Reply-To: <4B032BEC.9050508@colbyconsulting.com> References: <4B032BEC.9050508@colbyconsulting.com> Message-ID: Congrats to you John, I suspect once you get a few more utils in your library, you will be flying from here on. good for you, Mark 2009/11/17 jwcolby > Today I wrote my first C# applet. I have to import CSV files coming back > from a processing house > into SQL Server. These files had errors in one or more lines in the files. > Given that there are > tens of millions of lines in each file it isn't possible to just poke > around and find the errors. > > So I wrote an applet that opens the CSV file, reads the header line and > counts the " and , > characters. Then line by line the applet reads the rest of the lines, > comparing the counts to that > header line counts. > > This found lines in the file to be imported where " characters had been > dropped, which was causing > the import wizard to error out at that point. > > So read, compare counts, write good lines to output file, write bad lines > to an error file. > > Simple stuff really but it is my first real work done in C#. > > The speed of this kind of thing is pretty darned good too I thought. I was > getting about 300K lines > / second read / written. One file had 20.6 million lines, the other had 49 > million lines. > > In the past I would have done this in VBA / Access. Of course it took me a > tad longer to write in > C# than it would have in VBA. > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Wed Nov 18 03:11:43 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 18 Nov 2009 10:11:43 +0100 Subject: [dba-VB] First little applet Message-ID: Hi John Great! The good thing is that code from this little applet can easily be incorporated in another project. /gustav >>> jwcolby at colbyconsulting.com 18-11-2009 00:04 >>> Today I wrote my first C# applet. I have to import CSV files coming back from a processing house into SQL Server. These files had errors in one or more lines in the files. Given that there are tens of millions of lines in each file it isn't possible to just poke around and find the errors. So I wrote an applet that opens the CSV file, reads the header line and counts the " and , characters. Then line by line the applet reads the rest of the lines, comparing the counts to that header line counts. This found lines in the file to be imported where " characters had been dropped, which was causing the import wizard to error out at that point. So read, compare counts, write good lines to output file, write bad lines to an error file. Simple stuff really but it is my first real work done in C#. The speed of this kind of thing is pretty darned good too I thought. I was getting about 300K lines / second read / written. One file had 20.6 million lines, the other had 49 million lines. In the past I would have done this in VBA / Access. Of course it took me a tad longer to write in C# than it would have in VBA. -- John W. Colby www.ColbyConsulting.com From andy at minstersystems.co.uk Wed Nov 18 05:23:43 2009 From: andy at minstersystems.co.uk (Andy Lacey) Date: Wed, 18 Nov 2009 12:23:43 +0100 Subject: [dba-VB] First little applet Message-ID: <200911181223.nAICNl4W018133@databaseadvisors.com> Congratulations John. Hope father and little applet are both doing well. Andy --------- Original Message -------- From: "Discussion concerning Visual Basic and related programming issues." To: "Discussion concerning Visual Basic and related programming issues." Subject: Re: [dba-VB] First little applet Date: 18/11/09 08:55 Congrats to you John, I suspect once you get a few more utils in your library, you will be flying from here on. good for you, Mark 2009/11/17 jwcolby > Today I wrote my first C# applet. I have to import CSV files coming back > from a processing house > into SQL Server. These files had errors in one or more lines in the files. > Given that there are > tens of millions of lines in each file it isn't possible to just poke > around and find the errors. > > So I wrote an applet that opens the CSV file, reads the header line and > counts the " and , > characters. Then line by line the applet reads the rest of the lines, > comparing the counts to that > header line counts. > > This found lines in the file to be imported where " characters had been > dropped, which was causing > the import wizard to error out at that point. > > So read, compare counts, write good lines to output file, write bad lines > to an error file. > > Simple stuff really but it is my first real work done in C#. > > The speed of this kind of thing is pretty darned good too I thought. I was > getting about 300K lines > / second read / written. One file had 20.6 million lines, the other had 49 > million lines. > > In the past I would have done this in VBA / Access. Of course it took me a > tad longer to write in > C# than it would have in VBA. > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com ________________________________________________ Message sent using UebiMiau 2.7.2 From jwcolby at colbyconsulting.com Wed Nov 18 06:45:16 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 07:45:16 -0500 Subject: [dba-VB] First little applet In-Reply-To: <200911181223.nAICNl4W018133@databaseadvisors.com> References: <200911181223.nAICNl4W018133@databaseadvisors.com> Message-ID: <4B03EC5C.4090101@colbyconsulting.com> ROTFL. Father is so proud of little applet. ;) John W. Colby www.ColbyConsulting.com Andy Lacey wrote: > Congratulations John. Hope father and little applet are both doing well. > > Andy > > > --------- Original Message -------- > From: "Discussion concerning Visual Basic and related programming issues." > > To: "Discussion concerning Visual Basic and related programming issues." > > Subject: Re: [dba-VB] First little applet > Date: 18/11/09 08:55 > > > Congrats to you John, > > I suspect once you get a few more utils in your library, you will be flying > from here on. > > good for you, > > Mark > > > > 2009/11/17 jwcolby > >> Today I wrote my first C# applet. I have to import CSV files coming back >> from a processing house >> into SQL Server. These files had errors in one or more lines in the files. >> Given that there are >> tens of millions of lines in each file it isn't possible to just poke >> around and find the errors. >> >> So I wrote an applet that opens the CSV file, reads the header line and >> counts the " and , >> characters. Then line by line the applet reads the rest of the lines, >> comparing the counts to that >> header line counts. >> >> This found lines in the file to be imported where " characters had been >> dropped, which was causing >> the import wizard to error out at that point. >> >> So read, compare counts, write good lines to output file, write bad lines >> to an error file. >> >> Simple stuff really but it is my first real work done in C#. >> >> The speed of this kind of thing is pretty darned good too I thought. I was >> getting about 300K lines >> / second read / written. One file had 20.6 million lines, the other had 49 >> million lines. >> >> In the past I would have done this in VBA / Access. Of course it took me a >> tad longer to write in >> C# than it would have in VBA. >> >> -- >> John W. Colby >> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > ________________________________________________ > Message sent using UebiMiau 2.7.2 > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Wed Nov 18 06:50:01 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 07:50:01 -0500 Subject: [dba-VB] SPAM-LOW: Re: First little applet In-Reply-To: References: Message-ID: <4B03ED79.5020704@colbyconsulting.com> I do imports from CSV for large files all of the time, and I have had similar failures in the past, so I will be keeping this applet for future use. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > Great! The good thing is that code from this little applet can easily be incorporated in another project. > > /gustav > > >>>> jwcolby at colbyconsulting.com 18-11-2009 00:04 >>> > Today I wrote my first C# applet. I have to import CSV files coming back from a processing house > into SQL Server. These files had errors in one or more lines in the files. Given that there are > tens of millions of lines in each file it isn't possible to just poke around and find the errors. > > So I wrote an applet that opens the CSV file, reads the header line and counts the " and , > characters. Then line by line the applet reads the rest of the lines, comparing the counts to that > header line counts. > > This found lines in the file to be imported where " characters had been dropped, which was causing > the import wizard to error out at that point. > > So read, compare counts, write good lines to output file, write bad lines to an error file. > > Simple stuff really but it is my first real work done in C#. > > The speed of this kind of thing is pretty darned good too I thought. I was getting about 300K lines > / second read / written. One file had 20.6 million lines, the other had 49 million lines. > > In the past I would have done this in VBA / Access. Of course it took me a tad longer to write in > C# than it would have in VBA. > From jwcolby at colbyconsulting.com Wed Nov 18 07:06:44 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 08:06:44 -0500 Subject: [dba-VB] Projects vs Solutions Message-ID: <4B03F164.6090601@colbyconsulting.com> I am hazy on how things are organized in .Net once you start building pieces of code. What is a project, what is a solution and how are they tied together. My role for the "DBFH" client involves lots of different tasks, for which there will be many different pieces. I would hope to tie the pieces together in one big application. For example I just wrote the little applet to cleanup CSV files, which will have to be used again. I do exports to Accuzip, imports from Accuzip, these two are closely related obviously. I build orders. I have to export data from tables for processing by external processing houses, which involves a very similar export / zip / FTP process. So as I build these processes in C#, how are they physically organized on the disk, and how are they tied back together into one program. From what I am reading, a Solution is the big picture, a Project is a small part of the Solution. But when I open VS2008 there is no "create a solution" or at least I don't see one. So how does one create a solution, and then look at just one project at a time inside of that solution? -- John W. Colby www.ColbyConsulting.com From Gustav at cactus.dk Wed Nov 18 07:17:08 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 18 Nov 2009 14:17:08 +0100 Subject: [dba-VB] Projects vs Solutions Message-ID: Hi John On the New Project pane at the bottom select the name for the (new) solution and mark Create directory for solution. That folder holds your projects of this solution. All projects of a solution are listed in the treeview (typically the right panel). Open only those projects you wish to. /gustav >>> jwcolby at colbyconsulting.com 18-11-2009 14:06 >>> I am hazy on how things are organized in .Net once you start building pieces of code. What is a project, what is a solution and how are they tied together. My role for the "DBFH" client involves lots of different tasks, for which there will be many different pieces. I would hope to tie the pieces together in one big application. For example I just wrote the little applet to cleanup CSV files, which will have to be used again. I do exports to Accuzip, imports from Accuzip, these two are closely related obviously. I build orders. I have to export data from tables for processing by external processing houses, which involves a very similar export / zip / FTP process. So as I build these processes in C#, how are they physically organized on the disk, and how are they tied back together into one program. From what I am reading, a Solution is the big picture, a Project is a small part of the Solution. But when I open VS2008 there is no "create a solution" or at least I don't see one. So how does one create a solution, and then look at just one project at a time inside of that solution? -- John W. Colby www.ColbyConsulting.com From cfoust at infostatsystems.com Wed Nov 18 10:12:41 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 10:12:41 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B03F164.6090601@colbyconsulting.com> References: <4B03F164.6090601@colbyconsulting.com> Message-ID: Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 5:07 AM To: VBA Subject: [dba-VB] Projects vs Solutions I am hazy on how things are organized in .Net once you start building pieces of code. What is a project, what is a solution and how are they tied together. My role for the "DBFH" client involves lots of different tasks, for which there will be many different pieces. I would hope to tie the pieces together in one big application. For example I just wrote the little applet to cleanup CSV files, which will have to be used again. I do exports to Accuzip, imports from Accuzip, these two are closely related obviously. I build orders. I have to export data from tables for processing by external processing houses, which involves a very similar export / zip / FTP process. So as I build these processes in C#, how are they physically organized on the disk, and how are they tied back together into one program. From what I am reading, a Solution is the big picture, a Project is a small part of the Solution. But when I open VS2008 there is no "create a solution" or at least I don't see one. So how does one create a solution, and then look at just one project at a time inside of that solution? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Wed Nov 18 11:57:56 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 12:57:56 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: <4B03F164.6090601@colbyconsulting.com> Message-ID: <4B0435A4.3050804@colbyconsulting.com> OK, this immediately causes issues referencing objects in another project. For example I have a "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I then need to be able to open a form in the other project where I gather information about the file to fix. The objects in that other project aren't visible in intellisense. "Normal" opening of a form is done with something like MyFormName Somename = new MyFormNname(); SomeName.Show(); Well... MyFormName is not valid in project main. While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. > > Charlotte Foust > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, November 18, 2009 5:07 AM > To: VBA > Subject: [dba-VB] Projects vs Solutions From cfoust at infostatsystems.com Wed Nov 18 12:16:55 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 12:16:55 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B0435A4.3050804@colbyconsulting.com> References: <4B03F164.6090601@colbyconsulting.com> <4B0435A4.3050804@colbyconsulting.com> Message-ID: That's where references and "Imports" come in. You need to be careful about project level references to avoid circular messes, so our UI and Data projects have references to the Configuration project, but not the other way around, and the UI has a reference to the Data project but not the other way around. You can always fully reference a class in another project in code starting with the Namespace, so if I need to use something in a project that isn't referenced in the current one, I can use Infostat.RIMDrill.Configuration.RIMConstants (for example) and have access to the methods and properties of RIMConstants even without a reference in the current project. Alternatively, I could use Imports Infostat.RIMDrill.Configuration in the declarations above my current class and get the same thing. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 9:58 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions OK, this immediately causes issues referencing objects in another project. For example I have a "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I then need to be able to open a form in the other project where I gather information about the file to fix. The objects in that other project aren't visible in intellisense. "Normal" opening of a form is done with something like MyFormName Somename = new MyFormNname(); SomeName.Show(); Well... MyFormName is not valid in project main. While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. > > Charlotte Foust > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, November 18, 2009 5:07 AM > To: VBA > Subject: [dba-VB] Projects vs Solutions _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From Gustav at cactus.dk Wed Nov 18 12:27:23 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 18 Nov 2009 19:27:23 +0100 Subject: [dba-VB] Projects vs Solutions Message-ID: Hi John In addition to the comments from Charlotte: Also, you can either add in your project Using ProjectOther; or state this explicitly ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); someName.Show(); /gustav >>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> OK, this immediately causes issues referencing objects in another project. For example I have a "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I then need to be able to open a form in the other project where I gather information about the file to fix. The objects in that other project aren't visible in intellisense. "Normal" opening of a form is done with something like MyFormName Somename = new MyFormNname(); SomeName.Show(); Well... MyFormName is not valid in project main. While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. > > Charlotte Foust From jwcolby at colbyconsulting.com Wed Nov 18 12:54:47 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 13:54:47 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: Message-ID: <4B0442F7.2070805@colbyconsulting.com> Using projOther doesn't work. When doing this is the project allowed to be a part of the solution? Is it even SUPPOSED TO BE a part of the solution. IOW are you supposed to create projects as separate entities and then reference them from other projects, or can you create SolutionMain / ProjMain / projOther1 / projOther2 etc. I have the latter situation (now). When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. This whole area seems poorly documented and rather messy. Lots of chatter on Google about how difficult it is to move a project and so forth. This is the kind of stuff I hate to even get involved in. I am trying to get work done but get sucked off into areas where it certainly looks like I could easily wreak havoc on working code. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > In addition to the comments from Charlotte: > > Also, you can either add in your project > > Using ProjectOther; > > or state this explicitly > > ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); > someName.Show(); > > /gustav > >>>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> > OK, this immediately causes issues referencing objects in another project. For example I have a > "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I > then need to be able to open a form in the other project where I gather information about the file > to fix. > > The objects in that other project aren't visible in intellisense. > > "Normal" opening of a form is done with something like > > MyFormName Somename = new MyFormNname(); > SomeName.Show(); > > Well... MyFormName is not valid in project main. > > While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. >> >> Charlotte Foust > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From cfoust at infostatsystems.com Wed Nov 18 13:03:55 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 13:03:55 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B0442F7.2070805@colbyconsulting.com> References: <4B0442F7.2070805@colbyconsulting.com> Message-ID: A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 10:55 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions Using projOther doesn't work. When doing this is the project allowed to be a part of the solution? Is it even SUPPOSED TO BE a part of the solution. IOW are you supposed to create projects as separate entities and then reference them from other projects, or can you create SolutionMain / ProjMain / projOther1 / projOther2 etc. I have the latter situation (now). When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. This whole area seems poorly documented and rather messy. Lots of chatter on Google about how difficult it is to move a project and so forth. This is the kind of stuff I hate to even get involved in. I am trying to get work done but get sucked off into areas where it certainly looks like I could easily wreak havoc on working code. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > In addition to the comments from Charlotte: > > Also, you can either add in your project > > Using ProjectOther; > > or state this explicitly > > ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); > someName.Show(); > > /gustav > >>>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> > OK, this immediately causes issues referencing objects in another > project. For example I have a "main" project, and a "CSVRepair" > project. The Main project form opens when I run the project but I > then need to be able to open a form in the other project where I gather information about the file to fix. > > The objects in that other project aren't visible in intellisense. > > "Normal" opening of a form is done with something like > > MyFormName Somename = new MyFormNname(); SomeName.Show(); > > Well... MyFormName is not valid in project main. > > While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. >> >> Charlotte Foust > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From Gustav at cactus.dk Wed Nov 18 13:30:04 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 18 Nov 2009 20:30:04 +0100 Subject: [dba-VB] Projects vs Solutions Message-ID: Hi John Sorry, I read it as if ProjectOther was in the same (current) solution. If not, as Charlotte explains, then in the current solution reference the project from the other solution where it is situated. You will be given the choice to either leave the files of the project or to import them. What to chose is a question of maintenance. /gustav >>> jwcolby at colbyconsulting.com 18-11-2009 19:54 >>> Using projOther doesn't work. When doing this is the project allowed to be a part of the solution? Is it even SUPPOSED TO BE a part of the solution. IOW are you supposed to create projects as separate entities and then reference them from other projects, or can you create SolutionMain / ProjMain / projOther1 / projOther2 etc. I have the latter situation (now). When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. This whole area seems poorly documented and rather messy. Lots of chatter on Google about how difficult it is to move a project and so forth. This is the kind of stuff I hate to even get involved in. I am trying to get work done but get sucked off into areas where it certainly looks like I could easily wreak havoc on working code. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > In addition to the comments from Charlotte: > > Also, you can either add in your project > > Using ProjectOther; > > or state this explicitly > > ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); > someName.Show(); > > /gustav > >>>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> > OK, this immediately causes issues referencing objects in another project. For example I have a > "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I > then need to be able to open a form in the other project where I gather information about the file > to fix. > > The objects in that other project aren't visible in intellisense. > > "Normal" opening of a form is done with something like > > MyFormName Somename = new MyFormNname(); > SomeName.Show(); > > Well... MyFormName is not valid in project main. > > While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. >> >> Charlotte Foust From shamil at smsconsulting.spb.ru Wed Nov 18 13:35:05 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Wed, 18 Nov 2009 22:35:05 +0300 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B0442F7.2070805@colbyconsulting.com> References: <4B0442F7.2070805@colbyconsulting.com> Message-ID: <000501ca6886$3c5fd420$b51f7c60$@spb.ru> Hi John -- In fact referencing other projects works very well in .NET, and it's clean and very useful: just note that 'using' directive is to refer to *namespaces* not to projects. If you right-click a project in Solution Explorer, and then select 'Properties' you can find that every project has 'Default Namespace', which is usually the same as project name but this is not necessary. When I'm developing my solution and I need "to get work done" I do sometimes postpone creating new projects within a solution - I just create subfolders in my main project's treeview and I put my code to be later moved to other projects into those subfolders, and I keep namespace for those subfolders different from the default project's namespace - check it out... You'll soon get fluent with that namespaces and projects stuff and then you'll be "cooking" new projects seamlessly - it's usual story to have 10+ projects within .NET solutions... <<< When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. >>> Do you have 'ProjOther' within your solution? Did you set reference to 'ProjOther'? Does 'ProjOther''s default namespace is 'ProjOther'? 'using' directive should be started with lowercase letter.... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 9:55 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions Using projOther doesn't work. When doing this is the project allowed to be a part of the solution? Is it even SUPPOSED TO BE a part of the solution. IOW are you supposed to create projects as separate entities and then reference them from other projects, or can you create SolutionMain / ProjMain / projOther1 / projOther2 etc. I have the latter situation (now). When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. This whole area seems poorly documented and rather messy. Lots of chatter on Google about how difficult it is to move a project and so forth. This is the kind of stuff I hate to even get involved in. I am trying to get work done but get sucked off into areas where it certainly looks like I could easily wreak havoc on working code. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > In addition to the comments from Charlotte: > > Also, you can either add in your project > > Using ProjectOther; > > or state this explicitly > > ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); > someName.Show(); > > /gustav > >>>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> > OK, this immediately causes issues referencing objects in another project. For example I have a > "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I > then need to be able to open a form in the other project where I gather information about the file > to fix. > > The objects in that other project aren't visible in intellisense. > > "Normal" opening of a form is done with something like > > MyFormName Somename = new MyFormNname(); > SomeName.Show(); > > Well... MyFormName is not valid in project main. > > While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. >> >> Charlotte Foust > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4618 (20091118) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4618 (20091118) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Wed Nov 18 13:40:52 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 14:40:52 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: <4B0442F7.2070805@colbyconsulting.com> Message-ID: <4B044DC4.2050304@colbyconsulting.com> OK, let's back up a minute. I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". Except that: a) I really want the Solution to be InfoEngine. b) I would PREFER that the other projects be physically stored under InfoEngine. c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. d) Each of these "projects" contains a solution file. So what is it, a solution or a project? Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. > > Charlotte Foust From shamil at smsconsulting.spb.ru Wed Nov 18 14:04:58 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Wed, 18 Nov 2009 23:04:58 +0300 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B044DC4.2050304@colbyconsulting.com> References: <4B0442F7.2070805@colbyconsulting.com> <4B044DC4.2050304@colbyconsulting.com> Message-ID: <000c01ca688a$68c16200$3a442600$@spb.ru> John -- Just make an experiment with a test solution from scratch: - start VS; - create C# Console project - TestConsole - that will create a project TestConsole and a solution - TestConsole; - File->New->Project... - create C# Class Library project - MyClassLib1; - File->New->Project... - create C# Class Library project - MyClassLib2; - ... - Right-Click TestConsole project in Solution Explorer -> Add Reference -> [Projects] (third tab) -> Select MyClassLib1; - ... - now in TestConsole Program.cs you can type: using MyClassLib1; and you'll have Class1 (from MyClassLib1 - MyclassLib1.Class1) available to Intellisense.... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 10:41 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions OK, let's back up a minute. I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". Except that: a) I really want the Solution to be InfoEngine. b) I would PREFER that the other projects be physically stored under InfoEngine. c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. d) Each of these "projects" contains a solution file. So what is it, a solution or a project? Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. > > Charlotte Foust __________ Information from ESET NOD32 Antivirus, version of virus signature database 4618 (20091118) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From cfoust at infostatsystems.com Wed Nov 18 14:49:22 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 14:49:22 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B044DC4.2050304@colbyconsulting.com> References: <4B0442F7.2070805@colbyconsulting.com> <4B044DC4.2050304@colbyconsulting.com> Message-ID: John, Susan and I ran into this when we wrote a beginner's article. In tools -> Options under Project and Solutions, make sure the Always Show Solution checkbox is checked. Otherwise, you only see a solution and you keep creating separate solutions for each project. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 11:41 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions OK, let's back up a minute. I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". Except that: a) I really want the Solution to be InfoEngine. b) I would PREFER that the other projects be physically stored under InfoEngine. c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. d) Each of these "projects" contains a solution file. So what is it, a solution or a project? Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. > > Charlotte Foust _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From cfoust at infostatsystems.com Wed Nov 18 18:21:50 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 18:21:50 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: <4B0442F7.2070805@colbyconsulting.com> <4B044DC4.2050304@colbyconsulting.com> Message-ID: Oops, I meant you only see a PROJECT unles the Always Show Solution checkbox is checked! Sorry!! Charlotte -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Charlotte Foust Sent: Wednesday, November 18, 2009 12:49 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions John, Susan and I ran into this when we wrote a beginner's article. In tools -> Options under Project and Solutions, make sure the Always Show Solution checkbox is checked. Otherwise, you only see a solution and you keep creating separate solutions for each project. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 11:41 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions OK, let's back up a minute. I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". Except that: a) I really want the Solution to be InfoEngine. b) I would PREFER that the other projects be physically stored under InfoEngine. c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. d) Each of these "projects" contains a solution file. So what is it, a solution or a project? Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. > > Charlotte Foust _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Wed Nov 18 20:14:37 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 21:14:37 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: <4B0442F7.2070805@colbyconsulting.com> <4B044DC4.2050304@colbyconsulting.com> Message-ID: <4B04AA0D.5030107@colbyconsulting.com> Creating a reference to the other project solved the immediate problem of referencing objects in the other project. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > Oops, I meant you only see a PROJECT unles the Always Show Solution checkbox is checked! Sorry!! > > Charlotte > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Charlotte Foust > Sent: Wednesday, November 18, 2009 12:49 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Projects vs Solutions > > John, > > Susan and I ran into this when we wrote a beginner's article. In tools -> Options under Project and Solutions, make sure the Always Show Solution checkbox is checked. Otherwise, you only see a solution and you keep creating separate solutions for each project. > > Charlotte Foust > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, November 18, 2009 11:41 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Projects vs Solutions > > OK, let's back up a minute. > > I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. > > Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". > > Except that: > > a) I really want the Solution to be InfoEngine. > b) I would PREFER that the other projects be physically stored under InfoEngine. > c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. > d) Each of these "projects" contains a solution file. So what is it, a solution or a project? > > Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? > Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? > > By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". > > And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. >> >> Charlotte Foust > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Thu Nov 19 03:41:50 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Thu, 19 Nov 2009 10:41:50 +0100 Subject: [dba-VB] Projects vs Solutions Message-ID: Hi John This raises the question if your coming applet will be the new little applet or will become a little bigger little applet from day one. Does a little applet grow up? /gustav >>> jwcolby at colbyconsulting.com 19-11-2009 03:14 >>> Creating a reference to the other project solved the immediate problem of referencing objects in the other project. John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Thu Nov 19 07:10:03 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 19 Nov 2009 08:10:03 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: Message-ID: <4B0543AB.5080700@colbyconsulting.com> Gustav, Well, the point is that I am doing my job and suddenly I have to do something for the client that I have no solution for. This latest was a good example. I tried to import three files coming back from a processing vendor and one imported, the other two failed. So I write an applet that scans through the file sorting lines of the file into two output files, good and bad. When I am done the remaining "good" file now imports. What do I do with this applet? One thing I can do is to just leave it where it is and hope I don't lose it, because sure as the sun rises I will need to do this again. Or do I create a bigger application, build a menu in it, and place this applet on the menu? Does this Fixit applet grow? Conceivably, if I find some file that isn't fixed (or problem found) as easily as these files were, then this applet could grow to include different tests. This applet has its own form with a set of text boxes to display paths / files, buttons to select folders and files and a button to perform the cleanup. I probably will add another button to open the resulting "bad" file in an editor so that I can see and fix the problems in the data. Perhaps a button to append the fixed lines back to the end of the "good" file before the import. Stuff like that. So applets can grow, and the main application grows as I write new applets to perform various processing tasks, and those new applets get added into the main application. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > This raises the question if your coming applet will be the new little applet or will become a little bigger little applet from day one. Does a little applet grow up? > > /gustav > > >>>> jwcolby at colbyconsulting.com 19-11-2009 03:14 >>> > Creating a reference to the other project solved the immediate problem of referencing objects in the > other project. > > John W. Colby > www.ColbyConsulting.com From marklbreen at gmail.com Fri Nov 20 03:05:11 2009 From: marklbreen at gmail.com (Mark Breen) Date: Fri, 20 Nov 2009 09:05:11 +0000 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B0543AB.5080700@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> Message-ID: Hello John, While I am not qualified to comment really on this subject, I am waiting / hoping that someone will let you now how you can move 90% of your applet to a DLL project, and then from your main program you will just reference the DLL and consume the functionality within that DLL. With that approach, you could start your library of the new Colby C# Framework, albeit a humble beginning. Am I correct in assuming that with this approach you would compile your applet to to a DLL, and then you main program, which could be a winform or console app, could call applet.InputFilePathSet, applet.OutPutFileSet, and then applet.ParseFile etc etc. Let I say, I am not qualified to advise on this, but perhaps some others would like to, thanks Mark 2009/11/19 jwcolby > Gustav, > > Well, the point is that I am doing my job and suddenly I have to do > something for the client that I > have no solution for. This latest was a good example. I tried to import > three files coming back > from a processing vendor and one imported, the other two failed. So I > write an applet that scans > through the file sorting lines of the file into two output files, good and > bad. When I am done the > remaining "good" file now imports. > > What do I do with this applet? One thing I can do is to just leave it > where it is and hope I don't > lose it, because sure as the sun rises I will need to do this again. Or do > I create a bigger > application, build a menu in it, and place this applet on the menu? > > Does this Fixit applet grow? Conceivably, if I find some file that isn't > fixed (or problem found) > as easily as these files were, then this applet could grow to include > different tests. This applet > has its own form with a set of text boxes to display paths / files, buttons > to select folders and > files and a button to perform the cleanup. I probably will add another > button to open the resulting > "bad" file in an editor so that I can see and fix the problems in the data. > Perhaps a button to > append the fixed lines back to the end of the "good" file before the > import. Stuff like that. > > So applets can grow, and the main application grows as I write new applets > to perform various > processing tasks, and those new applets get added into the main > application. > > John W. Colby > www.ColbyConsulting.com > > > Gustav Brock wrote: > > Hi John > > > > This raises the question if your coming applet will be the new little > applet or will become a little bigger little applet from day one. Does a > little applet grow up? > > > > /gustav > > > > > >>>> jwcolby at colbyconsulting.com 19-11-2009 03:14 >>> > > Creating a reference to the other project solved the immediate problem of > referencing objects in the > > other project. > > > > John W. Colby > > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 20 06:26:10 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 20 Nov 2009 07:26:10 -0500 Subject: [dba-VB] SPAM-LOW: Re: Projects vs Solutions In-Reply-To: References: <4B0543AB.5080700@colbyconsulting.com> Message-ID: <4B068AE2.9060301@colbyconsulting.com> The blind leading the blind here. 1) I built a main application 2) I referenced the existing file repair applet from the main application (project). 3) I set a using statement. It appears that you have to both reference it and then use the "using" statement. 4) I can now open forms out in the file repair applet from the main application. 5) I physically moved the file repair applet underneath the main application directory. 6) I changed the directory for the applet and it just worked. That was fairly easy. From this point on I "Add Project" to the main solution. I have added a class project to wrap the DMO. In case you haven't discovered it, the DMO is a real cool SQL Server Management Object API that allows you to see and manage database objects. I am just getting into it but it allows me to reference a server object, then see the database collection. The each database object has a table collection, the table object has a fields collection etc. Everything you can see and manage in the SQL Server management studio you can (apparently) see and manage from the SMO from C#. An example of what this does for me is allows me to see all of the databases in a server, and thus populate a combo with their names. Selecting a database from the combo I can see and fill a combo with the names of the tables. Selecting a database and a specific table I can then can then run my stored procedures that export that table in that database to CSV files. That kind of stuff is what I do a lot of and what the big application will manage. John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > While I am not qualified to comment really on this subject, I am waiting / > hoping that someone will let you now how you can move 90% of your applet to > a DLL project, and then from your main program you will just reference the > DLL and consume the functionality within that DLL. > > With that approach, you could start your library of the new Colby C# > Framework, albeit a humble beginning. > > Am I correct in assuming that with this approach you would compile your > applet to to a DLL, and then you main program, which could be a winform or > console app, could call applet.InputFilePathSet, applet.OutPutFileSet, and > then applet.ParseFile etc etc. > > Let I say, I am not qualified to advise on this, but perhaps some others > would like to, > > thanks > > Mark > > > 2009/11/19 jwcolby > >> Gustav, >> >> Well, the point is that I am doing my job and suddenly I have to do >> something for the client that I >> have no solution for. This latest was a good example. I tried to import >> three files coming back >> from a processing vendor and one imported, the other two failed. So I >> write an applet that scans >> through the file sorting lines of the file into two output files, good and >> bad. When I am done the >> remaining "good" file now imports. >> >> What do I do with this applet? One thing I can do is to just leave it >> where it is and hope I don't >> lose it, because sure as the sun rises I will need to do this again. Or do >> I create a bigger >> application, build a menu in it, and place this applet on the menu? >> >> Does this Fixit applet grow? Conceivably, if I find some file that isn't >> fixed (or problem found) >> as easily as these files were, then this applet could grow to include >> different tests. This applet >> has its own form with a set of text boxes to display paths / files, buttons >> to select folders and >> files and a button to perform the cleanup. I probably will add another >> button to open the resulting >> "bad" file in an editor so that I can see and fix the problems in the data. >> Perhaps a button to >> append the fixed lines back to the end of the "good" file before the >> import. Stuff like that. >> >> So applets can grow, and the main application grows as I write new applets >> to perform various >> processing tasks, and those new applets get added into the main >> application. >> >> John W. Colby >> www.ColbyConsulting.com >> >> >> Gustav Brock wrote: >>> Hi John >>> >>> This raises the question if your coming applet will be the new little >> applet or will become a little bigger little applet from day one. Does a >> little applet grow up? >>> /gustav >>> >>> >>>>>> jwcolby at colbyconsulting.com 19-11-2009 03:14 >>> >>> Creating a reference to the other project solved the immediate problem of >> referencing objects in the >>> other project. >>> >>> John W. Colby >>> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 20 09:35:42 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 20 Nov 2009 10:35:42 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B068AE2.9060301@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> Message-ID: <4B06B74E.9080909@colbyconsulting.com> The object I am referring to is the SMO or SQL Server Management object. In order to use it you have to add several references: Microsoft.SQLServer.SMO Microsoft.SQLServer.SMOEnum Microsoft.SQLServer.SQLEnum Microsoft.SQLServer.ConnectionInfo then in the classes using the SMO you have to do using Microsoft.SqlServer.Management.Smo; After that you can do things like: Server Svr; Svr = new Server("MyServerName") foreach (Database in Svr.Databases) { //Etc. // } This allows you to iterate collections of database objects, using them directly or just pulling the names out (as I did) to populate lists, combos, collections etc. As I mentioned, once you have a database object you can manipulate it. I am just starting to learn what I can do with this API but it looks pretty powerful. John W. Colby www.ColbyConsulting.com jwcolby wrote: > The blind leading the blind here. > > 1) I built a main application > 2) I referenced the existing file repair applet from the main application (project). > 3) I set a using statement. It appears that you have to both reference it and then use the "using" > statement. > 4) I can now open forms out in the file repair applet from the main application. > 5) I physically moved the file repair applet underneath the main application directory. > 6) I changed the directory for the applet and it just worked. That was fairly easy. > > From this point on I "Add Project" to the main solution. I have added a class project to wrap the > DMO. In case you haven't discovered it, the DMO is a real cool SQL Server Management Object API > that allows you to see and manage database objects. I am just getting into it but it allows me to > reference a server object, then see the database collection. The each database object has a table > collection, the table object has a fields collection etc. Everything you can see and manage in the > SQL Server management studio you can (apparently) see and manage from the SMO from C#. > > An example of what this does for me is allows me to see all of the databases in a server, and thus > populate a combo with their names. Selecting a database from the combo I can see and fill a combo > with the names of the tables. Selecting a database and a specific table I can then can then run my > stored procedures that export that table in that database to CSV files. > > That kind of stuff is what I do a lot of and what the big application will manage. > > John W. Colby > www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Fri Nov 20 14:18:40 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 20 Nov 2009 23:18:40 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B06B74E.9080909@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> Message-ID: <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> Hi John -- I'm wondering what's the use of that Microsoft.SQLServer.* when you have to have your customer tasks done first of all? Why not just use (static) custom settings to point to different SQL servers etc.? I suppose Microsoft.SQLServer.* is good for companies like http://www.red-gate.com/ for them to develop their tools used worldwide, and I wonder what customers' business tasks can be solved by using Microsoft.SQLServer.* ? Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 20, 2009 6:36 PM To: Discussion concerning Visual Basic and related programming issues. Subject: [dba-VB] SMO was Projects vs Solutions The object I am referring to is the SMO or SQL Server Management object. In order to use it you have to add several references: Microsoft.SQLServer.SMO Microsoft.SQLServer.SMOEnum Microsoft.SQLServer.SQLEnum Microsoft.SQLServer.ConnectionInfo then in the classes using the SMO you have to do using Microsoft.SqlServer.Management.Smo; After that you can do things like: Server Svr; Svr = new Server("MyServerName") foreach (Database in Svr.Databases) { //Etc. // } This allows you to iterate collections of database objects, using them directly or just pulling the names out (as I did) to populate lists, combos, collections etc. As I mentioned, once you have a database object you can manipulate it. I am just starting to learn what I can do with this API but it looks pretty powerful. John W. Colby www.ColbyConsulting.com jwcolby wrote: > The blind leading the blind here. > > 1) I built a main application > 2) I referenced the existing file repair applet from the main application (project). > 3) I set a using statement. It appears that you have to both reference it and then use the "using" > statement. > 4) I can now open forms out in the file repair applet from the main application. > 5) I physically moved the file repair applet underneath the main application directory. > 6) I changed the directory for the applet and it just worked. That was fairly easy. > > From this point on I "Add Project" to the main solution. I have added a class project to wrap the > DMO. In case you haven't discovered it, the DMO is a real cool SQL Server Management Object API > that allows you to see and manage database objects. I am just getting into it but it allows me to > reference a server object, then see the database collection. The each database object has a table > collection, the table object has a fields collection etc. Everything you can see and manage in the > SQL Server management studio you can (apparently) see and manage from the SMO from C#. > > An example of what this does for me is allows me to see all of the databases in a server, and thus > populate a combo with their names. Selecting a database from the combo I can see and fill a combo > with the names of the tables. Selecting a database and a specific table I can then can then run my > stored procedures that export that table in that database to CSV files. > > That kind of stuff is what I do a lot of and what the big application will manage. > > John W. Colby > www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4624 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4625 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 20 14:30:47 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 20 Nov 2009 15:30:47 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> Message-ID: <4B06FC77.6040008@colbyconsulting.com> I don't understand the question. What do you mean by Microsoft.SQLServer.* I do have multiple servers running SQL Server. The default is Azul but I may need to reference Stonehenge. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > I'm wondering what's the use of that Microsoft.SQLServer.* when you have to > have your customer tasks done first of all? > Why not just use (static) custom settings to point to different SQL servers > etc.? > > I suppose Microsoft.SQLServer.* is good for companies like > http://www.red-gate.com/ for them to develop their tools used worldwide, and > I wonder what customers' business tasks can be solved by using > Microsoft.SQLServer.* ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 20, 2009 6:36 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: [dba-VB] SMO was Projects vs Solutions > > The object I am referring to is the SMO or SQL Server Management object. In > order to use it you > have to add several references: > > Microsoft.SQLServer.SMO > Microsoft.SQLServer.SMOEnum > Microsoft.SQLServer.SQLEnum > Microsoft.SQLServer.ConnectionInfo > > then in the classes using the SMO you have to do > > using Microsoft.SqlServer.Management.Smo; > > After that you can do things like: > > Server Svr; > Svr = new Server("MyServerName") > > foreach (Database in Svr.Databases) > { > //Etc. > // > } > > This allows you to iterate collections of database objects, using them > directly or just pulling the > names out (as I did) to populate lists, combos, collections etc. > > As I mentioned, once you have a database object you can manipulate it. I am > just starting to learn > what I can do with this API but it looks pretty powerful. > > John W. Colby > www.ColbyConsulting.com > > > jwcolby wrote: >> The blind leading the blind here. >> >> 1) I built a main application >> 2) I referenced the existing file repair applet from the main application > (project). >> 3) I set a using statement. It appears that you have to both reference it > and then use the "using" >> statement. >> 4) I can now open forms out in the file repair applet from the main > application. >> 5) I physically moved the file repair applet underneath the main > application directory. >> 6) I changed the directory for the applet and it just worked. That was > fairly easy. >> From this point on I "Add Project" to the main solution. I have added a > class project to wrap the >> DMO. In case you haven't discovered it, the DMO is a real cool SQL Server > Management Object API >> that allows you to see and manage database objects. I am just getting > into it but it allows me to >> reference a server object, then see the database collection. The each > database object has a table >> collection, the table object has a fields collection etc. Everything you > can see and manage in the >> SQL Server management studio you can (apparently) see and manage from the > SMO from C#. >> An example of what this does for me is allows me to see all of the > databases in a server, and thus >> populate a combo with their names. Selecting a database from the combo I > can see and fill a combo >> with the names of the tables. Selecting a database and a specific table I > can then can then run my >> stored procedures that export that table in that database to CSV files. >> >> That kind of stuff is what I do a lot of and what the big application will > manage. >> John W. Colby >> www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4624 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4625 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Nov 20 14:50:35 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 20 Nov 2009 23:50:35 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B06FC77.6040008@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> Message-ID: <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> Hi John, By Microsoft.SQLServer.* I mean all the namespaces you mentioned > Microsoft.SQLServer.SMO > Microsoft.SQLServer.SMOEnum > Microsoft.SQLServer.SQLEnum > Microsoft.SQLServer.ConnectionInfo As well as their classes and those classes enumerations, methods, properties, events etc. <<< I do have multiple servers running SQL Server. The default is Azul but I may need to reference Stonehenge. >>> But do you need to change the reference(/connection string information) to SQL Server after you have your program started? Do you need to enumerate SQL Server objects to solve your customers' tasks? Or maybe you have developed a kind of code generator, which does need to enumerate MS SQL Server objects and some of their properties to generate some custom code? - I can understand the latter - if you have to develop a lot of repetitive custom code then it's often useful to generate it... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 20, 2009 11:31 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SMO was Projects vs Solutions I don't understand the question. What do you mean by Microsoft.SQLServer.* I do have multiple servers running SQL Server. The default is Azul but I may need to reference Stonehenge. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > I'm wondering what's the use of that Microsoft.SQLServer.* when you have to > have your customer tasks done first of all? > Why not just use (static) custom settings to point to different SQL servers > etc.? > > I suppose Microsoft.SQLServer.* is good for companies like > http://www.red-gate.com/ for them to develop their tools used worldwide, and > I wonder what customers' business tasks can be solved by using > Microsoft.SQLServer.* ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 20, 2009 6:36 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: [dba-VB] SMO was Projects vs Solutions > > The object I am referring to is the SMO or SQL Server Management object. In > order to use it you > have to add several references: > > Microsoft.SQLServer.SMO > Microsoft.SQLServer.SMOEnum > Microsoft.SQLServer.SQLEnum > Microsoft.SQLServer.ConnectionInfo > > then in the classes using the SMO you have to do > > using Microsoft.SqlServer.Management.Smo; > > After that you can do things like: > > Server Svr; > Svr = new Server("MyServerName") > > foreach (Database in Svr.Databases) > { > //Etc. > // > } > > This allows you to iterate collections of database objects, using them > directly or just pulling the > names out (as I did) to populate lists, combos, collections etc. > > As I mentioned, once you have a database object you can manipulate it. I am > just starting to learn > what I can do with this API but it looks pretty powerful. > > John W. Colby > www.ColbyConsulting.com > > > jwcolby wrote: >> The blind leading the blind here. >> >> 1) I built a main application >> 2) I referenced the existing file repair applet from the main application > (project). >> 3) I set a using statement. It appears that you have to both reference it > and then use the "using" >> statement. >> 4) I can now open forms out in the file repair applet from the main > application. >> 5) I physically moved the file repair applet underneath the main > application directory. >> 6) I changed the directory for the applet and it just worked. That was > fairly easy. >> From this point on I "Add Project" to the main solution. I have added a > class project to wrap the >> DMO. In case you haven't discovered it, the DMO is a real cool SQL Server > Management Object API >> that allows you to see and manage database objects. I am just getting > into it but it allows me to >> reference a server object, then see the database collection. The each > database object has a table >> collection, the table object has a fields collection etc. Everything you > can see and manage in the >> SQL Server management studio you can (apparently) see and manage from the > SMO from C#. >> An example of what this does for me is allows me to see all of the > databases in a server, and thus >> populate a combo with their names. Selecting a database from the combo I > can see and fill a combo >> with the names of the tables. Selecting a database and a specific table I > can then can then run my >> stored procedures that export that table in that database to CSV files. >> >> That kind of stuff is what I do a lot of and what the big application will > manage. >> John W. Colby >> www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4625 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 20 20:05:57 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 20 Nov 2009 21:05:57 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> Message-ID: <4B074B05.6000406@colbyconsulting.com> That is used for manipulating the SMO object and SQL Server. I am just learning this stuff and do what the demos tell me to do. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > By Microsoft.SQLServer.* I mean all the namespaces you mentioned > >> Microsoft.SQLServer.SMO >> Microsoft.SQLServer.SMOEnum >> Microsoft.SQLServer.SQLEnum >> Microsoft.SQLServer.ConnectionInfo > > As well as their classes and those classes enumerations, methods, > properties, events etc. > > <<< > I do have multiple servers running SQL Server. > The default is Azul but I may need to reference > Stonehenge. > But do you need to change the reference(/connection string information) to > SQL Server after you have your program started? > Do you need to enumerate SQL Server objects to solve your customers' tasks? > Or maybe you have developed a kind of code generator, which does need to > enumerate MS SQL Server objects and some of their properties to generate > some custom code? - I can understand the latter - if you have to develop a > lot of repetitive custom code then it's often useful to generate it... > > Thank you. > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 20, 2009 11:31 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > I don't understand the question. > > What do you mean by Microsoft.SQLServer.* > > I do have multiple servers running SQL Server. The default is Azul but I > may need to reference > Stonehenge. > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> I'm wondering what's the use of that Microsoft.SQLServer.* when you have > to >> have your customer tasks done first of all? >> Why not just use (static) custom settings to point to different SQL > servers >> etc.? >> >> I suppose Microsoft.SQLServer.* is good for companies like >> http://www.red-gate.com/ for them to develop their tools used worldwide, > and >> I wonder what customers' business tasks can be solved by using >> Microsoft.SQLServer.* ? >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: Friday, November 20, 2009 6:36 PM >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: [dba-VB] SMO was Projects vs Solutions >> >> The object I am referring to is the SMO or SQL Server Management object. > In >> order to use it you >> have to add several references: >> >> Microsoft.SQLServer.SMO >> Microsoft.SQLServer.SMOEnum >> Microsoft.SQLServer.SQLEnum >> Microsoft.SQLServer.ConnectionInfo >> >> then in the classes using the SMO you have to do >> >> using Microsoft.SqlServer.Management.Smo; >> >> After that you can do things like: >> >> Server Svr; >> Svr = new Server("MyServerName") >> >> foreach (Database in Svr.Databases) >> { >> //Etc. >> // >> } >> >> This allows you to iterate collections of database objects, using them >> directly or just pulling the >> names out (as I did) to populate lists, combos, collections etc. >> >> As I mentioned, once you have a database object you can manipulate it. I > am >> just starting to learn >> what I can do with this API but it looks pretty powerful. >> >> John W. Colby >> www.ColbyConsulting.com >> >> >> jwcolby wrote: >>> The blind leading the blind here. >>> >>> 1) I built a main application >>> 2) I referenced the existing file repair applet from the main application >> (project). >>> 3) I set a using statement. It appears that you have to both reference > it >> and then use the "using" >>> statement. >>> 4) I can now open forms out in the file repair applet from the main >> application. >>> 5) I physically moved the file repair applet underneath the main >> application directory. >>> 6) I changed the directory for the applet and it just worked. That was >> fairly easy. >>> From this point on I "Add Project" to the main solution. I have added a >> class project to wrap the >>> DMO. In case you haven't discovered it, the DMO is a real cool SQL > Server >> Management Object API >>> that allows you to see and manage database objects. I am just getting >> into it but it allows me to >>> reference a server object, then see the database collection. The each >> database object has a table >>> collection, the table object has a fields collection etc. Everything you >> can see and manage in the >>> SQL Server management studio you can (apparently) see and manage from the >> SMO from C#. >>> An example of what this does for me is allows me to see all of the >> databases in a server, and thus >>> populate a combo with their names. Selecting a database from the combo I >> can see and fill a combo >>> with the names of the tables. Selecting a database and a specific table > I >> can then can then run my >>> stored procedures that export that table in that database to CSV files. >>> >>> That kind of stuff is what I do a lot of and what the big application > will >> manage. >>> John W. Colby >>> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4625 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Nov 21 08:58:31 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 21 Nov 2009 17:58:31 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B074B05.6000406@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> Message-ID: <002001ca6abb$186807e0$493817a0$@spb.ru> Hi John -- Yes, I understand that. I'm wondering what for do you learning this stuff? - Just out of curiosity or do you plan to implement some custom applications using Microsoft.SQLServer.* ? Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 21, 2009 5:06 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SMO was Projects vs Solutions That is used for manipulating the SMO object and SQL Server. I am just learning this stuff and do what the demos tell me to do. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > By Microsoft.SQLServer.* I mean all the namespaces you mentioned > >> Microsoft.SQLServer.SMO >> Microsoft.SQLServer.SMOEnum >> Microsoft.SQLServer.SQLEnum >> Microsoft.SQLServer.ConnectionInfo > > As well as their classes and those classes enumerations, methods, > properties, events etc. > > <<< > I do have multiple servers running SQL Server. > The default is Azul but I may need to reference > Stonehenge. > But do you need to change the reference(/connection string information) to > SQL Server after you have your program started? > Do you need to enumerate SQL Server objects to solve your customers' tasks? > Or maybe you have developed a kind of code generator, which does need to > enumerate MS SQL Server objects and some of their properties to generate > some custom code? - I can understand the latter - if you have to develop a > lot of repetitive custom code then it's often useful to generate it... > > Thank you. > > -- > Shamil > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4626 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 21 09:42:12 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 10:42:12 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <002001ca6abb$186807e0$493817a0$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> Message-ID: <4B080A54.4030609@colbyconsulting.com> > or do you plan to implement some custom applications using Microsoft.SQLServer.* ? Exactly. My work for one of my clients is heavy manipulation of SQL Server. I routinely export files to an external application running on virtual machines on another server, then wait for the files to process, then import them back in to SQL server. These two processes together take about 10 stored procedures. The export / import has to occur on (currently) a half dozen large databases monthly. However as I obtain more lists that number goes up. Each process exports to it's own directory path, then imports back from a parallel directory path. While the stored procedures eliminate much of the manual labor, there is plenty left. Plus logging of everything etc. I also routinely process orders. These orders take about a dozen stored procedures and also use the exact same set of stored procedures as the above process, i.e. the export / import process is also used on each and every order. So there is just a ton of very repetitive work that has to be done, but which database / table varies from run to run. To this point I have used Access / VBA to automate these tasks. Now I am moving to C# for this automation work. SMO allows me to manipulate SQL Server from C# and so it is an API that I will use to assist me in getting this program working. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I understand that. > I'm wondering what for do you learning this stuff? - Just out of curiosity > or do you plan to implement some custom applications using > Microsoft.SQLServer.* ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 21, 2009 5:06 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > That is used for manipulating the SMO object and SQL Server. > > I am just learning this stuff and do what the demos tell me to do. > > John W. Colby > www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sat Nov 21 10:27:15 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 21 Nov 2009 19:27:15 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B080A54.4030609@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> Message-ID: <002601ca6ac7$7dcfb180$796f1480$@spb.ru> Hi John -- OK, but why not just use connections strings and System.Data.SqlClient classes, .... ? Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 21, 2009 6:42 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SMO was Projects vs Solutions > or do you plan to implement some custom applications using Microsoft.SQLServer.* ? Exactly. My work for one of my clients is heavy manipulation of SQL Server. I routinely export files to an external application running on virtual machines on another server, then wait for the files to process, then import them back in to SQL server. These two processes together take about 10 stored procedures. The export / import has to occur on (currently) a half dozen large databases monthly. However as I obtain more lists that number goes up. Each process exports to it's own directory path, then imports back from a parallel directory path. While the stored procedures eliminate much of the manual labor, there is plenty left. Plus logging of everything etc. I also routinely process orders. These orders take about a dozen stored procedures and also use the exact same set of stored procedures as the above process, i.e. the export / import process is also used on each and every order. So there is just a ton of very repetitive work that has to be done, but which database / table varies from run to run. To this point I have used Access / VBA to automate these tasks. Now I am moving to C# for this automation work. SMO allows me to manipulate SQL Server from C# and so it is an API that I will use to assist me in getting this program working. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I understand that. > I'm wondering what for do you learning this stuff? - Just out of curiosity > or do you plan to implement some custom applications using > Microsoft.SQLServer.* ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 21, 2009 5:06 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > That is used for manipulating the SMO object and SQL Server. > > I am just learning this stuff and do what the demos tell me to do. > > John W. Colby > www.ColbyConsulting.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4626 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 21 11:05:43 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 12:05:43 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <002601ca6ac7$7dcfb180$796f1480$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> <002601ca6ac7$7dcfb180$796f1480$@spb.ru> Message-ID: <4B081DE7.7060203@colbyconsulting.com> Shamil, As far as I can tell they are quite different things, and perform different jobs. I do use SQLClient. SQLClient appears to be about getting at data, and I do use that for the ADO side of things. http://msdn.microsoft.com/en-us/library/system.data.sqlclient.aspx SMO appears to be about getting at and manipulating the objects in the database. http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.aspx SMO appears to be similar to the DAO object where you can see and manipulate the table (not the data IN the table) the fields, indexes and so forth. It is a collection based object where you can drill down into a server and see the objects underneath the database, then (for example) drill down into a table and see the objects under the table. Just as an example, using SMO in a few lines of code I got a list of all of the databases in the database collection of my server. I used that to populate a database combo box. When I select the database that I want to export data from, I then use SMO (again a couple of more lines of code) to get a list of the tables in the selected database and use that to populate a combo of tables. I have no idea the relative merit of one way vs the other. To me it is just a tool. In this particular case I use it to quickly and easily get at the names of objects in the database. I understand that using SMO you can do other maintenance kinds of things as well. One of the things I have to do is copy an "order template" database to a new name in preparing to fill an order. SMO appears to have built-in methods for doing this programmatically from C# - as opposed to running TSQL or using some other method. Understand I am not an expert on any of this, in fact quite the opposite. I stumbled across SMO and saw an example of how easy it was to get at the database STRUCTURE information and decided to use it for that purpose, when I had that need. Is there another way to do this? Almost certainly, given that there is always a dozen ways to do anything in programming. Given your persistent questioning of my motives I guess now I have to ask whether I am causing myself problems doing this? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > OK, but why not just use connections strings and System.Data.SqlClient > classes, .... ? > > Thank you. From jwcolby at colbyconsulting.com Sat Nov 21 11:40:33 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 12:40:33 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <002601ca6ac7$7dcfb180$796f1480$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> <002601ca6ac7$7dcfb180$796f1480$@spb.ru> Message-ID: <4B082611.4040401@colbyconsulting.com> Shamil, By get an instance of the Server object you can see a collection of Database objects. The following is what you can do with the database object. So I can manipulate a given database as shown below. Several of those things look potentially useful, such as running TSQL statements, truncating log files and so forth. http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.database.aspx However the database object has a tables collection. I can get lists of the names of tables, select a specific table etc. http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.table.aspx http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.table_methods.aspx The point is that it is just a programming model that is available if you want to learn it, for manipulating a database from C#. I will almost certainly never use 99.9% of it, just as most people (myself included) rarely use 99.9% of the DAO object model. OTOH the .1% that I do use is pretty darned cool. You can find a lot of little articles about using SMO out on Google. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > OK, but why not just use connections strings and System.Data.SqlClient > classes, .... ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 21, 2009 6:42 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > > or do you plan to implement some custom applications using > Microsoft.SQLServer.* ? > > Exactly. My work for one of my clients is heavy manipulation of SQL Server. > I routinely export > files to an external application running on virtual machines on another > server, then wait for the > files to process, then import them back in to SQL server. These two > processes together take about > 10 stored procedures. The export / import has to occur on (currently) a > half dozen large databases > monthly. However as I obtain more lists that number goes up. Each process > exports to it's own > directory path, then imports back from a parallel directory path. While the > stored procedures > eliminate much of the manual labor, there is plenty left. Plus logging of > everything etc. > > I also routinely process orders. These orders take about a dozen stored > procedures and also use the > exact same set of stored procedures as the above process, i.e. the export / > import process is also > used on each and every order. > > So there is just a ton of very repetitive work that has to be done, but > which database / table > varies from run to run. To this point I have used Access / VBA to automate > these tasks. Now I am > moving to C# for this automation work. > > SMO allows me to manipulate SQL Server from C# and so it is an API that I > will use to assist me in > getting this program working. > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> Yes, I understand that. >> I'm wondering what for do you learning this stuff? - Just out of curiosity >> or do you plan to implement some custom applications using >> Microsoft.SQLServer.* ? >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: Saturday, November 21, 2009 5:06 AM >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] SMO was Projects vs Solutions >> >> That is used for manipulating the SMO object and SQL Server. >> >> I am just learning this stuff and do what the demos tell me to do. >> >> John W. Colby >> www.ColbyConsulting.com > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4626 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Nov 21 11:47:27 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 12:47:27 -0500 Subject: [dba-VB] Loss leader (or just plain loss) Message-ID: <4B0827AF.6040608@colbyconsulting.com> A friend has approached me about building an application for a small chain of stores to manage inventory. "Very simple". I haven't yet talked to her about the requirements. She did say they were quoted (and rejected) a bid of $4000 to do the job. If I am going to do this at all it will have to be very quick and dirty, and would be just to help a friend. Is Access 2007 runtime stable? Easy or difficult to use / install / implement on site? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sat Nov 21 11:55:29 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 21 Nov 2009 20:55:29 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B081DE7.7060203@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> <002601ca6ac7$7dcfb180$796f1480$@spb.ru> <4B081DE7.7060203@colbyconsulting.com> Message-ID: <002c01ca6ad3$d1e37930$75aa6b90$@spb.ru> Hi John -- <<< SMO appears to be similar to the DAO object where you can see and manipulate the table... >>> Yes, I understand/know that. <<< When I select the database that I want to export data from, I then use SMO (again a couple of more lines of code) to get a list of the tables in the selected database and use that to populate a combo of tables. >>> Yes, but how often would you need to do that? Why not just use (relatively static) program settings files, or utility program execution command string parameters - I mean you anyway have to do manual selection (in the case you'll use SMO) or manual editing and saving (in the case you'll use program setting files...) - what for to spend time on learning SMO if you very probably (am I wrong?) can solve your customer tasks without using SMO, and applying relatively the same efforts in both cases to implement custom logic (but in the case of using SMO you'll also have to spend time on learning SMO)... <<< Given your persistent questioning of my motives I guess now I have to ask whether I am causing myself problems doing this? >>> Just maybe by spending time on learning SMO when you don't need it to implement required custom logic? Not sure (I didn't check so I can be wrong) but SMO might need special installation procedures for customers' PCs without full MS SQL Version (standard, prof., enterprise) so if you'll develop some custom code to be reused on customers' PC then you might need to be prepared to develop custom setup - not a big issue just be prepared - if I'm not wrong in guessing that that custom setup would be needed for some customers system... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 21, 2009 8:06 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SMO was Projects vs Solutions Shamil, As far as I can tell they are quite different things, and perform different jobs. I do use SQLClient. SQLClient appears to be about getting at data, and I do use that for the ADO side of things. http://msdn.microsoft.com/en-us/library/system.data.sqlclient.aspx SMO appears to be about getting at and manipulating the objects in the database. http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.a spx SMO appears to be similar to the DAO object where you can see and manipulate the table (not the data IN the table) the fields, indexes and so forth. It is a collection based object where you can drill down into a server and see the objects underneath the database, then (for example) drill down into a table and see the objects under the table. Just as an example, using SMO in a few lines of code I got a list of all of the databases in the database collection of my server. I used that to populate a database combo box. When I select the database that I want to export data from, I then use SMO (again a couple of more lines of code) to get a list of the tables in the selected database and use that to populate a combo of tables. I have no idea the relative merit of one way vs the other. To me it is just a tool. In this particular case I use it to quickly and easily get at the names of objects in the database. I understand that using SMO you can do other maintenance kinds of things as well. One of the things I have to do is copy an "order template" database to a new name in preparing to fill an order. SMO appears to have built-in methods for doing this programmatically from C# - as opposed to running TSQL or using some other method. Understand I am not an expert on any of this, in fact quite the opposite. I stumbled across SMO and saw an example of how easy it was to get at the database STRUCTURE information and decided to use it for that purpose, when I had that need. Is there another way to do this? Almost certainly, given that there is always a dozen ways to do anything in programming. Given your persistent questioning of my motives I guess now I have to ask whether I am causing myself problems doing this? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > OK, but why not just use connections strings and System.Data.SqlClient > classes, .... ? > > Thank you. __________ Information from ESET NOD32 Antivirus, version of virus signature database 4626 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From dbdoug at gmail.com Sat Nov 21 12:26:06 2009 From: dbdoug at gmail.com (Doug Steele) Date: Sat, 21 Nov 2009 10:26:06 -0800 Subject: [dba-VB] Loss leader (or just plain loss) In-Reply-To: <4B0827AF.6040608@colbyconsulting.com> References: <4B0827AF.6040608@colbyconsulting.com> Message-ID: <4dd71a0c0911211026y79e4d583t99d6ca1dfb4a10f1@mail.gmail.com> Hi John: I've only ever used the Access 2003 runtime/installer, and that was for one job. Getting the runtime code installed, running, and generating/testing a proper installation package took the best part of a day. It was straightforward enough, just fiddly. After that, updates only consisted of recreating the .mde file and sending it to the client. From what I've read, the Access 2007 runtime is pretty stable now (it started off very badly) and it's the right price. My experience with projects that started out to be 'quick and dirty' has been consistently horrible. Not quick, and very dirty! Doug On Sat, Nov 21, 2009 at 9:47 AM, jwcolby wrote: > A friend has approached me about building an application for a small chain > of stores to manage > inventory. "Very simple". I haven't yet talked to her about the > requirements. She did say they > were quoted (and rejected) a bid of $4000 to do the job. > > If I am going to do this at all it will have to be very quick and dirty, > and would be just to help a > friend. > > Is Access 2007 runtime stable? Easy or difficult to use / install / > implement on site? > > > From shamil at smsconsulting.spb.ru Sat Nov 21 12:40:36 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 21 Nov 2009 21:40:36 +0300 Subject: [dba-VB] Loss leader (or just plain loss) In-Reply-To: <4dd71a0c0911211026y79e4d583t99d6ca1dfb4a10f1@mail.gmail.com> References: <4B0827AF.6040608@colbyconsulting.com> <4dd71a0c0911211026y79e4d583t99d6ca1dfb4a10f1@mail.gmail.com> Message-ID: <002d01ca6ada$1ecb5fa0$5c621ee0$@spb.ru> <<< Not quick, and very dirty! >>> Yes. And a high opportunity to lose a friend... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Doug Steele Sent: Saturday, November 21, 2009 9:26 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Loss leader (or just plain loss) Hi John: I've only ever used the Access 2003 runtime/installer, and that was for one job. Getting the runtime code installed, running, and generating/testing a proper installation package took the best part of a day. It was straightforward enough, just fiddly. After that, updates only consisted of recreating the .mde file and sending it to the client. From what I've read, the Access 2007 runtime is pretty stable now (it started off very badly) and it's the right price. My experience with projects that started out to be 'quick and dirty' has been consistently horrible. Not quick, and very dirty! Doug On Sat, Nov 21, 2009 at 9:47 AM, jwcolby wrote: > A friend has approached me about building an application for a small chain > of stores to manage > inventory. "Very simple". I haven't yet talked to her about the > requirements. She did say they > were quoted (and rejected) a bid of $4000 to do the job. > > If I am going to do this at all it will have to be very quick and dirty, > and would be just to help a > friend. > > Is Access 2007 runtime stable? Easy or difficult to use / install / > implement on site? > > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4626 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 21 13:30:39 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 14:30:39 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <002c01ca6ad3$d1e37930$75aa6b90$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> <002601ca6ac7$7dcfb180$796f1480$@spb.ru> <4B081DE7.7060203@colbyconsulting.com> <002c01ca6ad3$d1e37930$75aa6b90$@spb.ru> Message-ID: <4B083FDF.505@colbyconsulting.com> There is no "customer PC". This is my server in my office, running an application I will write, which I will operate. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > <<< > SMO appears to be similar to the DAO object > where you can see and manipulate the table... > Yes, I understand/know that. > > <<< > When I select the > database that I want to export data from, I then > use SMO (again a couple of more lines of code) to > get a list of the tables in the selected database > and use that to populate a combo of tables. > Yes, but how often would you need to do that? > Why not just use (relatively static) program settings files, or utility > program execution command string parameters - I mean you anyway have to do > manual selection (in the case you'll use SMO) or manual editing and saving > (in the case you'll use program setting files...) - what for to spend time > on learning SMO if you very probably (am I wrong?) can solve your customer > tasks without using SMO, and applying relatively the same efforts in both > cases to implement custom logic (but in the case of using SMO you'll also > have to spend time on learning SMO)... > > <<< > Given your persistent questioning of my motives > I guess now I have to ask whether I am causing > myself problems doing this? > Just maybe by spending time on learning SMO when you don't need it to > implement required custom logic? > > Not sure (I didn't check so I can be wrong) but SMO might need special > installation procedures for customers' PCs without full MS SQL Version > (standard, prof., enterprise) so if you'll develop some custom code to be > reused on customers' PC then you might need to be prepared to develop custom > setup - not a big issue just be prepared - if I'm not wrong in guessing that > that custom setup would be needed for some customers system... > > Thank you. > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 21, 2009 8:06 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > Shamil, > > As far as I can tell they are quite different things, and perform different > jobs. I do use > SQLClient. SQLClient appears to be about getting at data, and I do use that > for the ADO side of > things. > > http://msdn.microsoft.com/en-us/library/system.data.sqlclient.aspx > > SMO appears to be about getting at and manipulating the objects in the > database. > > http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.a > spx > > SMO appears to be similar to the DAO object where you can see and manipulate > the table (not the data > IN the table) the fields, indexes and so forth. It is a collection based > object where you can drill > down into a server and see the objects underneath the database, then (for > example) drill down into a > table and see the objects under the table. > > Just as an example, using SMO in a few lines of code I got a list of all of > the databases in the > database collection of my server. I used that to populate a database combo > box. When I select the > database that I want to export data from, I then use SMO (again a couple of > more lines of code) to > get a list of the tables in the selected database and use that to populate a > combo of tables. > > I have no idea the relative merit of one way vs the other. To me it is just > a tool. In this > particular case I use it to quickly and easily get at the names of objects > in the database. I > understand that using SMO you can do other maintenance kinds of things as > well. One of the things I > have to do is copy an "order template" database to a new name in preparing > to fill an order. SMO > appears to have built-in methods for doing this programmatically from C# - > as opposed to running > TSQL or using some other method. > > Understand I am not an expert on any of this, in fact quite the opposite. I > stumbled across SMO and > saw an example of how easy it was to get at the database STRUCTURE > information and decided to use it > for that purpose, when I had that need. Is there another way to do this? > Almost certainly, given > that there is always a dozen ways to do anything in programming. > > Given your persistent questioning of my motives I guess now I have to ask > whether I am causing > myself problems doing this? > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> OK, but why not just use connections strings and System.Data.SqlClient >> classes, .... ? >> >> Thank you. > > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4626 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sun Nov 22 22:46:38 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 22 Nov 2009 23:46:38 -0500 Subject: [dba-VB] Ya know you're in trouble when... Message-ID: <4B0A13AE.1030404@colbyconsulting.com> http://www.hanselman.com/blog/WhatGreatNETDevelopersOughtToKnowMoreNETInterviewQuestions.aspx -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Mon Nov 23 07:57:41 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 23 Nov 2009 08:57:41 -0500 Subject: [dba-VB] XNA game development Message-ID: <4B0A94D5.3010601@colbyconsulting.com> Have you ever wanted to write your own little windows game. Microsoft provides a free game development framework called XNA. It runs in C# express and everything is free free free. It is designed to get you writing games for the XBox BUT... it can write games for Windows as well, and in fact they are apparently interchangeable, once written they can be played on the XBox (but not for free). The best part is that they have a very basic cannon / flying saucer demo game with videos for how to develop this game, all of the source code, sprites etc. I have stepped through the first few chapters and it really does start at ground zero. They claim you don't even need to know C# to follow the tutorial. As we all know, writing a project that holds your interest is the best way to get involved and learn a language. If writing a game is your interest, this might be just the ticket to getting into C#. http://creators.xna.com/en-US/education/gettingstarted/bg2d/chapter1 -- John W. Colby www.ColbyConsulting.com From mmattys at rochester.rr.com Mon Nov 23 09:41:38 2009 From: mmattys at rochester.rr.com (Mike Mattys) Date: Mon, 23 Nov 2009 10:41:38 -0500 Subject: [dba-VB] Ya know you're in trouble when... References: <4B0A13AE.1030404@colbyconsulting.com> Message-ID: I've already come up against a great many of these Some of them I look forward to getting the answers for. If I went to an interview for all of it, I would hope it to be "open book" - Michael R Mattys MapPoint and Database Dev www.mattysconsulting.com - ----- Original Message ----- From: "jwcolby" To: "VBA" Sent: Sunday, November 22, 2009 11:46 PM Subject: [dba-VB] Ya know you're in trouble when... > > http://www.hanselman.com/blog/WhatGreatNETDevelopersOughtToKnowMoreNETInterviewQuestions.aspx > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > From dwaters at usinternet.com Mon Nov 23 12:22:19 2009 From: dwaters at usinternet.com (Dan Waters) Date: Mon, 23 Nov 2009 12:22:19 -0600 Subject: [dba-VB] Ya know you're in trouble when... In-Reply-To: References: <4B0A13AE.1030404@colbyconsulting.com> Message-ID: Hey! I knew one of these! What's the difference between late binding and early binding? But that's it. :-( Dan -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Mike Mattys Sent: Monday, November 23, 2009 9:42 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Ya know you're in trouble when... I've already come up against a great many of these Some of them I look forward to getting the answers for. If I went to an interview for all of it, I would hope it to be "open book" - Michael R Mattys MapPoint and Database Dev www.mattysconsulting.com - ----- Original Message ----- From: "jwcolby" To: "VBA" Sent: Sunday, November 22, 2009 11:46 PM Subject: [dba-VB] Ya know you're in trouble when... > > http://www.hanselman.com/blog/WhatGreatNETDevelopersOughtToKnowMoreNETInterv iewQuestions.aspx > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Mon Nov 23 15:57:12 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 23 Nov 2009 16:57:12 -0500 Subject: [dba-VB] XNA is darned fun Message-ID: <4B0B0538.1020904@colbyconsulting.com> I have to say that this is darned fun. I have worked through much of the tutorial I posted a link to. I have ships flying and cannonballs firing. Woohoo! Very very cool! I know what I will be doing over the Thanksgiving holiday. -- John W. Colby www.ColbyConsulting.com From michael at ddisolutions.com.au Mon Nov 23 23:38:16 2009 From: michael at ddisolutions.com.au (Michael Maddison) Date: Tue, 24 Nov 2009 16:38:16 +1100 Subject: [dba-VB] HTTPWebRequest blues Message-ID: <59A61174B1F5B54B97FD4ADDE71E7D01582726@ddi-01.DDI.local> Hi guys, I may be clutching at straws here but has anyone done much with HttpWebRequest? My process goes like this, GET /ExpressBet/Standard/Default.aspx?State=2 Store the cookie container for re use GET /ExpressBet/LoginFrame.aspx?State=2&ExpressBetMode=Standard Store viewState for re use in next POST POST /Login/LoginUser.aspx?State=2&ReturnURL=http://www.tab.com.au/ExpressBet/LoginFrame.aspx?State=2&ExpressBetMode=Standard Does login, this succeeds GET /ExpressBet/Aggregator.aspx?State=2&Controls=STD_MAIN&RacingCode=R&MeetingCode=M&FromDate=2009-11-24T00:00:00&RaceNumber=07&BetType=WinPlace&ExpressBetMode=Standard Store viewState for re use GET /Betting/BetTicket/DisplayClientSideBetTicket.aspx?State=2&ExpressBet=true&RacingCode=R&MeetingCode=M&FromDate=2009-11-24T00:00:00&BetType=WinPlace&amount=1.00&Selections=%3csc+id%3d%220%22+f%3d%22false%22+m%3d%220%22+s%3d%228%22+t%3d%22%22+k%3d%2207%22+%2f%3e+ The above redirects to GET /Betting/BetTicket/PlaceRaceBetConfirmation.aspx?State=2&CurrentBet=2901d195-c59a-4c1b-8b0a-c476fc274f06&ExpressBet=True Result looks ok in Fiddler, But response is unreadable in plain text. Anyone know what this is? I think I need to get the viewstate from it somehow. Snipped because of size constraints. _???2??-w?sF??'15??1??_i???i??_l?___???_?__?S??fY?d?R_adZ?Ri??__ 2?#??4+?='+??N!??_n^???????nQ^?`????_?_____?0???_?h??X????_???dvTH??`??[?k?Sh?SDp??[h?c?OH???k?K?{??3??@?'13OW7????0W??8,????r2????_=}A???{?@??? i???)?5__F_?[?S?'4N?__C]}?M?}?Af???s?H??K?-H?? ?u+N ?s?r,? The Last step I need to do is a final POST, This returns an error page, dammit. POST /Betting/BetTicket/PlaceRaceBetConfirmation.aspx?State=2&CurrentBet=2901d195-c59a-4c1b-8b0a-c476fc274f06&ExpressBet=True Redirects to an error.aspx page. The code for the last POST is the same for the successful calls as the failed call. Both are HTTPS, I am handling the certificates. Does this make sense to anyone? J Any ideas? Cheers Michael M From Gustav at cactus.dk Tue Nov 24 04:50:40 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Tue, 24 Nov 2009 11:50:40 +0100 Subject: [dba-VB] This is hot: WCF RIA Services Message-ID: Hi all RIA: Rich Internet Applications Microsoft WCF RIA Services simplifies the traditional n-tier application pattern by bringing together the ASP.NET and Silverlight platforms. RIA Services provides a pattern to write application logic that runs on the mid-tier and controls access to data for queries, changes and custom operations. It also provides end-to-end support for common tasks such as data validation, authentication and roles by integrating with Silverlight components on the client and ASP.NET on the mid-tier Though not explained at a place to find, what it does is automatically to set up for you a web service which your Silverlight frontend communicates with, and to establish the client side code that handles this communication. http://silverlight.net/getstarted/riaservices/ Watch the 1-hour video "Introduction to WCF RIA Services" and combine this with the racing speed at which Silverlight evolves, and it will erase any doubt you still may have about browser-based applications. Note too, that while this works in VS2008 and with Silverlight 3 it is really aimed at VS2010 and Silverlight 4 both currently available as betas. I attended a developer meeting a couple of days ago where we ran a similar walk-through, and it was very convincing. This is exactly what I for years have been looking for. /gustav From fuller.artful at gmail.com Tue Nov 24 10:26:24 2009 From: fuller.artful at gmail.com (Arthur Fuller) Date: Tue, 24 Nov 2009 11:26:24 -0500 Subject: [dba-VB] XNA is darned fun In-Reply-To: <4B0B0538.1020904@colbyconsulting.com> References: <4B0B0538.1020904@colbyconsulting.com> Message-ID: <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> When is American Thanksgiving? We already had ours last month. What is XNA, anyway? Arthur On Mon, Nov 23, 2009 at 4:57 PM, jwcolby wrote: > I have to say that this is darned fun. I have worked through much of the > tutorial I posted a link > to. I have ships flying and cannonballs firing. Woohoo! > > Very very cool! I know what I will be doing over the Thanksgiving holiday. > From cfoust at infostatsystems.com Tue Nov 24 10:48:12 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Tue, 24 Nov 2009 10:48:12 -0600 Subject: [dba-VB] XNA is darned fun In-Reply-To: <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> References: <4B0B0538.1020904@colbyconsulting.com> <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> Message-ID: Ours is Thursday. Charlotte -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Arthur Fuller Sent: Tuesday, November 24, 2009 8:26 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] XNA is darned fun When is American Thanksgiving? We already had ours last month. What is XNA, anyway? Arthur On Mon, Nov 23, 2009 at 4:57 PM, jwcolby wrote: > I have to say that this is darned fun. I have worked through much of > the tutorial I posted a link to. I have ships flying and cannonballs > firing. Woohoo! > > Very very cool! I know what I will be doing over the Thanksgiving holiday. > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Tue Nov 24 10:53:21 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Tue, 24 Nov 2009 11:53:21 -0500 Subject: [dba-VB] XNA is darned fun In-Reply-To: <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> References: <4B0B0538.1020904@colbyconsulting.com> <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> Message-ID: <4B0C0F81.9070806@colbyconsulting.com> > When is American Thanksgiving? We already had ours last month. Not sure exactly. IIRC it is the last Thursday in November so it is not a fixed date. > What is XNA, anyway? It is a Microsoft gaming framework. VERY COOL if you like to (or always wanted to) write a windows game. It runs under C# in Visual Studio, even the express version. The framework provides a game class that has a method for loading the "content" - video, sound, sprites etc. It has a method automatically called X times / second that is then used to perform updates and display the content. And it has a draw method that is called to draw the game. LOTS of game specific classes including math classes, 2d and 3d classes for doing game specific kinds of things, sprite classes, point classes, rectangle classes. You get the picture. REALLY really cool for the child in all of us. If you happen to have an XBox, you can write games that can run on the XBox as well as Windows (XP, Vista and Windows 7). I don't own an XBox so I am just running the game on Windows, but any game CAN be written to run on either, and apparently SENSE which it is running on. You can even use the XBox joystick / button controller on your Windows machine! There is a step by step video series that walks you through building a "space invaders" game, with cannon, cannon balls, UFOs. All graphics provided, all code stepped through in short videos. I did that and I have to say it is really easy. It was so cool as each step comes together, displaying the background, displaying the flying saucers moving across the screen, the cannon swiveling, then cannon balls firing, and finally cannon balls destroying the UFOs. Of course doing it all by myself would be not so easy but it gives a great feeling for what can be done and how it is done. And of course there are after market books on it. I have always wanted to write my own "Empire" replacement, and have even started doing so. I played Empire all of my life (still do) but it is long in the tooth and not updated any more. Plus the multi-player stuff never really worked very well. This will set me back to ground zero on that attempt but it will also provide a framework for doing it the right way. I just don't have enough to do with my time, as I am sure you can understand! John W. Colby www.ColbyConsulting.com Arthur Fuller wrote: > When is American Thanksgiving? We already had ours last month. > > What is XNA, anyway? > > Arthur > > On Mon, Nov 23, 2009 at 4:57 PM, jwcolby wrote: > >> I have to say that this is darned fun. I have worked through much of the >> tutorial I posted a link >> to. I have ships flying and cannonballs firing. Woohoo! >> >> Very very cool! I know what I will be doing over the Thanksgiving holiday. >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Wed Nov 25 08:24:39 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 09:24:39 -0500 Subject: [dba-VB] Part 1: An Introduction to Data Structures Message-ID: <4B0D3E27.9040601@colbyconsulting.com> A quick overview: http://creators.xna.com/en-US/article/datastructures A more detailed overview: http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Wed Nov 25 08:51:50 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Wed, 25 Nov 2009 17:51:50 +0300 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: <4B0D3E27.9040601@colbyconsulting.com> References: <4B0D3E27.9040601@colbyconsulting.com> Message-ID: <006e01ca6dde$d31de700$7959b500$@spb.ru> Hi John -- You might also find interesting: http://en.wikipedia.org/wiki/Microsoft_Robotics_Developer_Studio e.g., for programming: http://en.wikipedia.org/wiki/Lego_Mindstorms_NXT using C#... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 25, 2009 5:25 PM To: VBA; Nikolai Vitsyn; Judy Hawley Subject: [dba-VB] Part 1: An Introduction to Data Structures A quick overview: http://creators.xna.com/en-US/article/datastructures A more detailed overview: http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4636 (20091125) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From Gustav at cactus.dk Wed Nov 25 08:54:57 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 25 Nov 2009 15:54:57 +0100 Subject: [dba-VB] Part 1: An Introduction to Data Structures Message-ID: Hi John Thanks. "collision code" ... I love that. /gustav >>> jwcolby at colbyconsulting.com 25-11-2009 15:24 >>> A quick overview: http://creators.xna.com/en-US/article/datastructures A more detailed overview: http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Wed Nov 25 09:23:22 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 10:23:22 -0500 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: References: Message-ID: <4B0D4BEA.2050603@colbyconsulting.com> > Thanks. "collision code" ... I love that. LOL. Obviously this was aimed at gamers. A collision is when one rectangle (sprite) intersects another rectangle. BTW, check out this video of a game. http://www.youtube.com/watch?v=Lz-TQcdXDJU&feature=related LOTS of collisions going on here. ;) I am going to work on writing this game using the XNA gaming framework. The XNA framework is just awesome, and I am going to get my 8 year old son involved in helping me to write this thing. A fun, interactive way of teaching him C#. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > Thanks. "collision code" ... I love that. > > /gustav > > >>>> jwcolby at colbyconsulting.com 25-11-2009 15:24 >>> > A quick overview: > http://creators.xna.com/en-US/article/datastructures > > A more detailed overview: > http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx From jwcolby at colbyconsulting.com Wed Nov 25 09:24:15 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 10:24:15 -0500 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: <006e01ca6dde$d31de700$7959b500$@spb.ru> References: <4B0D3E27.9040601@colbyconsulting.com> <006e01ca6dde$d31de700$7959b500$@spb.ru> Message-ID: <4B0D4C1F.9050103@colbyconsulting.com> I have this thing. I haven't looked at it yet though. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > You might also find interesting: > > http://en.wikipedia.org/wiki/Microsoft_Robotics_Developer_Studio > > e.g., for programming: > > http://en.wikipedia.org/wiki/Lego_Mindstorms_NXT > > using C#... > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, November 25, 2009 5:25 PM > To: VBA; Nikolai Vitsyn; Judy Hawley > Subject: [dba-VB] Part 1: An Introduction to Data Structures > > A quick overview: > http://creators.xna.com/en-US/article/datastructures > > A more detailed overview: > http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx From stuart at lexacorp.com.pg Wed Nov 25 15:31:48 2009 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Thu, 26 Nov 2009 07:31:48 +1000 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: <4B0D4BEA.2050603@colbyconsulting.com> References: , <4B0D4BEA.2050603@colbyconsulting.com> Message-ID: <4B0DA244.1838.3178D240@stuart.lexacorp.com.pg> Gee you guys are taking me back - I used to write games for the Amiga twenty odd years ago - It all revolved around sprites and collisions. -- Stuart On 25 Nov 2009 at 10:23, jwcolby wrote: > > Thanks. "collision code" ... I love that. > > LOL. Obviously this was aimed at gamers. A collision is when one rectangle (sprite) intersects > another rectangle. > > BTW, check out this video of a game. > > http://www.youtube.com/watch?v=Lz-TQcdXDJU&feature=related > > LOTS of collisions going on here. ;) > > I am going to work on writing this game using the XNA gaming framework. The XNA framework is just > awesome, and I am going to get my 8 year old son involved in helping me to write this thing. A fun, > interactive way of teaching him C#. > > John W. Colby > www.ColbyConsulting.com > > > Gustav Brock wrote: > > Hi John > > > > Thanks. "collision code" ... I love that. > > > > /gustav > > > > > >>>> jwcolby at colbyconsulting.com 25-11-2009 15:24 >>> > > A quick overview: > > http://creators.xna.com/en-US/article/datastructures > > > > A more detailed overview: > > http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > From Gustav at cactus.dk Wed Nov 25 15:50:53 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 25 Nov 2009 22:50:53 +0100 Subject: [dba-VB] Part 1: An Introduction to Data Structures Message-ID: Hi John Oh my, will this never stop? I guess that is what this poor guy thinks, in real trouble as he is ... /gustav >>> jwcolby at colbyconsulting.com 25-11-2009 16:23 >>> > Thanks. "collision code" ... I love that. LOL. Obviously this was aimed at gamers. A collision is when one rectangle (sprite) intersects another rectangle. BTW, check out this video of a game. http://www.youtube.com/watch?v=Lz-TQcdXDJU&feature=related LOTS of collisions going on here. ;) I am going to work on writing this game using the XNA gaming framework. The XNA framework is just awesome, and I am going to get my 8 year old son involved in helping me to write this thing. A fun, interactive way of teaching him C#. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > Thanks. "collision code" ... I love that. > > /gustav > > >>>> jwcolby at colbyconsulting.com 25-11-2009 15:24 >>> > A quick overview: > http://creators.xna.com/en-US/article/datastructures > > A more detailed overview: > http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx From jwcolby at colbyconsulting.com Wed Nov 25 15:52:30 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 16:52:30 -0500 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: <4B0DA244.1838.3178D240@stuart.lexacorp.com.pg> References: , <4B0D4BEA.2050603@colbyconsulting.com> <4B0DA244.1838.3178D240@stuart.lexacorp.com.pg> Message-ID: <4B0DA71E.3030105@colbyconsulting.com> > Gee you guys are taking me back - I used to write games for the Amiga twenty odd years ago - It all revolved around sprites and collisions. LOL. Give it a whirl. I bet you find it much easier in this day and age. One of the reasons I am doing this is that I want to get really good at C#. I do have real work that I am starting to do in C#, but writing games is fun and will keep me in it on a daily / weekly basis whereas my work could be weeks without having to write any C# code. Plus I am hoping to slowly suck my son into the programming. He was up in my office changing the constants that determine how many space ships will be created in the shoot-em-up demo I was studying. He now knows how to edit that constant, how to run it, how to close the game, he knows now that he can't put any number he wants in an integer variable (he tried to run with 20000000000000 spaceships;). We'll see. John W. Colby www.ColbyConsulting.com Stuart McLachlan wrote: > Gee you guys are taking me back - I used to write games for the Amiga twenty odd years > ago - It all revolved around sprites and collisions. > From jwcolby at colbyconsulting.com Wed Nov 25 20:24:52 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 21:24:52 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. Message-ID: <4B0DE6F4.3070605@colbyconsulting.com> If a parameter in a stored procedure is declared as an output (returns a value) will the parameter object in C# / ADO contain that value when the stored procedure finishes? I haven't used output parameters yet in stored procedures. Can values be passed in via an output parameter or can it only be set from inside of the stored procedure? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Thu Nov 26 00:02:40 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 26 Nov 2009 09:02:40 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0DE6F4.3070605@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> Message-ID: <00a101ca6e5e$111e9ac0$335bd040$@spb.ru> Hi John, Yes. Set System.Data.ParameterDirection.Output for .Direction property of output parameters. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 26, 2009 5:25 AM To: VBA Subject: [dba-VB] Parameter objects in C# and ADO. If a parameter in a stored procedure is declared as an output (returns a value) will the parameter object in C# / ADO contain that value when the stored procedure finishes? I haven't used output parameters yet in stored procedures. Can values be passed in via an output parameter or can it only be set from inside of the stored procedure? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4637 (20091125) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4637 (20091125) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From marklbreen at gmail.com Thu Nov 26 03:04:54 2009 From: marklbreen at gmail.com (Mark Breen) Date: Thu, 26 Nov 2009 09:04:54 +0000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0DE6F4.3070605@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> Message-ID: Hello John, There are three ways to bring back values from an sproc, 1 the result set 2 the output parameters, of which there can be many 3 the return code afaik, a param can be input (the default direction) or output. I have not heard of an inputoutput param so unless anyone here says different, assume that you cannot have bi-directional. Now you have the ability to standardize on a return code which may relate to error codes, a complete recordset of data, and also a few params that passed back logging codes, temp counts etc etc etc. In fact, if the sproc is doing multiple stages, you will love the ability to store staged log codes and messages and then bring the whole lot back when it is finished, and still being 1M records as well ! Thanks Mark 2009/11/26 jwcolby > If a parameter in a stored procedure is declared as an output (returns a > value) will the parameter > object in C# / ADO contain that value when the stored procedure finishes? > > I haven't used output parameters yet in stored procedures. Can values be > passed in via an output > parameter or can it only be set from inside of the stored procedure? > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Thu Nov 26 04:26:58 2009 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Thu, 26 Nov 2009 20:26:58 +1000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: <4B0DE6F4.3070605@colbyconsulting.com>, Message-ID: <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> SP with a parameter that is both Input and Output In some situations, there is no need to store the output into an extra variable because the output must replace the input. CREATE PROCEDURE cap2 (@name NVARCHAR(20) OUT ) --Capitalizes first char, changes the rest to lower case --SP with one input/output parameter AS SET @name=UPPER(LEFT(@name,1))+LOWER(RIGHT(@name,LEN(@name)-1)) or for you .net types, see http://fabioscagliola.spaces.live.com/blog/cns!919F8FCDE3DC9AC4!121.entry -- Stuart On 26 Nov 2009 at 9:04, Mark Breen wrote: > > afaik, a param can be input (the default direction) or output. I have not > heard of an inputoutput param so unless anyone here says different, assume > that you cannot have bi-directional. > From shamil at smsconsulting.spb.ru Thu Nov 26 05:03:27 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 26 Nov 2009 14:03:27 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> References: <4B0DE6F4.3070605@colbyconsulting.com>, <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> Message-ID: <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> Hi Mark, > assume that you cannot have bi-directional. You can: System.Data.ParameterDirection.InputOutput -- Shamil On 26 Nov 2009 at 9:04, Mark Breen wrote: > > afaik, a param can be input (the default direction) or output. I have not > heard of an inputoutput param so unless anyone here says different, assume > that you cannot have bi-directional. > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4638 (20091126) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Thu Nov 26 06:55:59 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 26 Nov 2009 07:55:59 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: <4B0DE6F4.3070605@colbyconsulting.com> Message-ID: <4B0E7ADF.5010504@colbyconsulting.com> Thanks for the reply Mark. I am looking forward to having the flexibility of C# to handle errors appropriately. John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > There are three ways to bring back values from an sproc, > > 1 the result set > 2 the output parameters, of which there can be many > 3 the return code > > afaik, a param can be input (the default direction) or output. I have not > heard of an inputoutput param so unless anyone here says different, assume > that you cannot have bi-directional. > > Now you have the ability to standardize on a return code which may relate to > error codes, a complete recordset of data, and also a few params that passed > back logging codes, temp counts etc etc etc. > > In fact, if the sproc is doing multiple stages, you will love the ability to > store staged log codes and messages and then bring the whole lot back when > it is finished, and still being 1M records as well ! > > Thanks > > Mark > > > > > 2009/11/26 jwcolby > >> If a parameter in a stored procedure is declared as an output (returns a >> value) will the parameter >> object in C# / ADO contain that value when the stored procedure finishes? >> >> I haven't used output parameters yet in stored procedures. Can values be >> passed in via an output >> parameter or can it only be set from inside of the stored procedure? >> >> -- >> John W. Colby >> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From marklbreen at gmail.com Thu Nov 26 09:59:22 2009 From: marklbreen at gmail.com (Mark Breen) Date: Thu, 26 Nov 2009 15:59:22 +0000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> Message-ID: Hello Shamil and Stuart, I never knew params could be bi-directional, thanks a lot for that, Mark 2009/11/26 Shamil Salakhetdinov > Hi Mark, > > > assume that you cannot have bi-directional. > You can: > > System.Data.ParameterDirection.InputOutput > > -- > Shamil > > On 26 Nov 2009 at 9:04, Mark Breen wrote: > > > > afaik, a param can be input (the default direction) or output. I have > not > > heard of an inputoutput param so unless anyone here says different, > assume > > that you cannot have bi-directional. > > > > > > > > __________ Information from ESET NOD32 Antivirus, version of virus > signature > database 4638 (20091126) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Thu Nov 26 10:04:07 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 26 Nov 2009 11:04:07 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> References: <4B0DE6F4.3070605@colbyconsulting.com>, <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> Message-ID: <4B0EA6F7.4050600@colbyconsulting.com> Thanks Stuart. John W. Colby www.ColbyConsulting.com Stuart McLachlan wrote: > SP with a parameter that is both Input and Output > > In some situations, there is no need to store the output into an extra variable because the > output must replace the input. > > CREATE PROCEDURE cap2 (@name NVARCHAR(20) OUT ) > --Capitalizes first char, changes the rest to lower case > --SP with one input/output parameter > AS > SET @name=UPPER(LEFT(@name,1))+LOWER(RIGHT(@name,LEN(@name)-1)) > > or for you .net types, see > http://fabioscagliola.spaces.live.com/blog/cns!919F8FCDE3DC9AC4!121.entry > From jwcolby at colbyconsulting.com Thu Nov 26 10:05:27 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 26 Nov 2009 11:05:27 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> References: <4B0DE6F4.3070605@colbyconsulting.com>, <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> Message-ID: <4B0EA747.4020506@colbyconsulting.com> > System.Data.ParameterDirection.InputOutput Thanks Shamil, What do you do with this? Set it to true? Set something else equal to this (is this a constant)? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Mark, > >> assume that you cannot have bi-directional. > You can: > > System.Data.ParameterDirection.InputOutput > > -- > Shamil > > On 26 Nov 2009 at 9:04, Mark Breen wrote: >> afaik, a param can be input (the default direction) or output. I have > not >> heard of an inputoutput param so unless anyone here says different, assume >> that you cannot have bi-directional. >> > > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4638 (20091126) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Thu Nov 26 10:54:43 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 26 Nov 2009 19:54:43 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0EA747.4020506@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com>, <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> <4B0EA747.4020506@colbyconsulting.com> Message-ID: <00de01ca6eb9$287b8830$79729890$@spb.ru> Hi John -- This is enumeration value - have a look here how to use it (near to the bottom): http://stackoverflow.com/questions/1637646/need-a-return-value-from-a-stored -procedure-in-c -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 26, 2009 7:05 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. > System.Data.ParameterDirection.InputOutput Thanks Shamil, What do you do with this? Set it to true? Set something else equal to this (is this a constant)? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Mark, > >> assume that you cannot have bi-directional. > You can: > > System.Data.ParameterDirection.InputOutput > > -- > Shamil > > On 26 Nov 2009 at 9:04, Mark Breen wrote: >> afaik, a param can be input (the default direction) or output. I have > not >> heard of an inputoutput param so unless anyone here says different, assume >> that you cannot have bi-directional. >> > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4639 (20091126) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From marklbreen at gmail.com Fri Nov 27 04:00:01 2009 From: marklbreen at gmail.com (Mark Breen) Date: Fri, 27 Nov 2009 10:00:01 +0000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0EA747.4020506@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> <4B0EA747.4020506@colbyconsulting.com> Message-ID: Hello John, that Shamil has said is that you can use one param to pass in a value, and use the same param to accept back a value, which may be different to when it was passes in. You can pass in ProductId, and get back, ProductCode, or pass in username and get back fullname, or pass in product code and get back zero if the product code does not exist, else you get back the code you passed in or pass in productid and get back qty in stock, all with one param. My examples are not great, but there are plenty of times where you need to somehow inspect a value and it may not be ideal to have two params named paramOriginal paramNewValue with inputout you can have just one param named paramToBeVerified Thanks again to Shamil, Stuart and Mr Colby for asking the question. Mark 2009/11/26 jwcolby > > System.Data.ParameterDirection.InputOutput > > Thanks Shamil, > > What do you do with this? Set it to true? Set something else equal to > this (is this a constant)? > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: > > Hi Mark, > > > >> assume that you cannot have bi-directional. > > You can: > > > > System.Data.ParameterDirection.InputOutput > > > > -- > > Shamil > > > > On 26 Nov 2009 at 9:04, Mark Breen wrote: > >> afaik, a param can be input (the default direction) or output. I have > > not > >> heard of an inputoutput param so unless anyone here says different, > assume > >> that you cannot have bi-directional. > >> > > > > > > > > > > > > __________ Information from ESET NOD32 Antivirus, version of virus > signature > > database 4638 (20091126) __________ > > > > The message was checked by ESET NOD32 Antivirus. > > > > http://www.esetnod32.ru > > > > > > _______________________________________________ > > dba-VB mailing list > > dba-VB at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > http://www.databaseadvisors.com > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Fri Nov 27 04:44:46 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Fri, 27 Nov 2009 11:44:46 +0100 Subject: [dba-VB] Parameter objects in C# and ADO. Message-ID: Hi Mark et al What's the big deal? Are parameters costly? Hardly in money, but in resources? /gustav >>> marklbreen at gmail.com 27-11-2009 11:00 >>> Hello John, that Shamil has said is that you can use one param to pass in a value, and use the same param to accept back a value, which may be different to when it was passes in. You can pass in ProductId, and get back, ProductCode, or pass in username and get back fullname, or pass in product code and get back zero if the product code does not exist, else you get back the code you passed in or pass in productid and get back qty in stock, all with one param. My examples are not great, but there are plenty of times where you need to somehow inspect a value and it may not be ideal to have two params named paramOriginal paramNewValue with inputout you can have just one param named paramToBeVerified Thanks again to Shamil, Stuart and Mr Colby for asking the question. Mark From shamil at smsconsulting.spb.ru Fri Nov 27 06:43:24 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 27 Nov 2009 15:43:24 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: Message-ID: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> Hi Gustav at all, BTW, have you ever seen the following below error handling in CRUD SPs as I have found in one of my customers' databases? Isn't an overkill? Or do you use even more detailed/elaborated error handling in your CRUD SPs? How do you organize error handling in your CRUD SPs? (I usually just return error code in RETURN statement). CREATE procedure [dbo].[GettblUserByUsername] @Username varchar(20), @ErrorDesc varchar(100) output, @ErrorNo int output, @RowsAffected int output AS DECLARE @lErrorNo INTEGER DECLARE @lRowsAffected INTEGER Begin SELECT tblUser.tblUserId, tblUser.tblUserGroupId, tblUserGroup.groupCode as UserGroup, tblUser.Username, tblUser.tblUserLevelId FROM tblUser left outer join tblUserGroup on tblUser.tblUserGroupId = tblUserGroup.tblUserGroupId WHERE tblUser.Username = @username and tblUser.Authorised = 1 select @lErrorNo = @@ERROR, @lRowsAffected = @@ROWCOUNT IF @lErrorNo > 0 Begin set @ErrorDesc = 'error with get of tblUser' set @ErrorNo = @lErrorNo set @RowsAffected = 0 RETURN End Else Begin set @ErrorDesc = 'Get was Successful for tblUser' set @ErrorNo = 0 set @RowsAffected = @lRowsAffected RETURN End end Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Friday, November 27, 2009 1:45 PM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] Parameter objects in C# and ADO. Hi Mark et al What's the big deal? Are parameters costly? Hardly in money, but in resources? /gustav <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4641 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 27 08:11:29 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 09:11:29 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> References: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> Message-ID: <4B0FDE11.9070108@colbyconsulting.com> RowsAffected is not part of the error stuff and I definitely want that info in most cases. I haven't done this yet but I don't think that getting back the error code and description is excessive. If it is possible to easily and unambiguously look up the error code and get the exact description, each and every time, then passing back the description is not necessary. This is exactly the kind of thing I will be doing in the future. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Gustav at all, > > BTW, have you ever seen the following below error handling in CRUD SPs as I > have found in one of my customers' databases? > Isn't an overkill? > Or do you use even more detailed/elaborated error handling in your CRUD SPs? > How do you organize error handling in your CRUD SPs? (I usually just return > error code in RETURN statement). > > CREATE procedure [dbo].[GettblUserByUsername] > @Username varchar(20), > @ErrorDesc varchar(100) output, > @ErrorNo int output, > @RowsAffected int output > AS > DECLARE @lErrorNo INTEGER > DECLARE @lRowsAffected INTEGER > Begin > > SELECT tblUser.tblUserId, > tblUser.tblUserGroupId, > tblUserGroup.groupCode as UserGroup, > tblUser.Username, > tblUser.tblUserLevelId > FROM tblUser > left outer join tblUserGroup > on tblUser.tblUserGroupId = tblUserGroup.tblUserGroupId > WHERE tblUser.Username = @username > and tblUser.Authorised = 1 > > select @lErrorNo = @@ERROR, @lRowsAffected = @@ROWCOUNT > > IF @lErrorNo > 0 > Begin > set @ErrorDesc = 'error with get of tblUser' > set @ErrorNo = @lErrorNo > set @RowsAffected = 0 > RETURN > End > Else > Begin > set @ErrorDesc = 'Get was Successful for tblUser' > set @ErrorNo = 0 > set @RowsAffected = @lRowsAffected > RETURN > End > end > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, November 27, 2009 1:45 PM > To: dba-vb at databaseadvisors.com > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > Hi Mark et al > > What's the big deal? Are parameters costly? Hardly in money, but in > resources? > > /gustav > > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4641 (20091127) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From marklbreen at gmail.com Fri Nov 27 08:33:52 2009 From: marklbreen at gmail.com (Mark Breen) Date: Fri, 27 Nov 2009 14:33:52 +0000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: Message-ID: Hello Gustav, I presume they are not costly in terms of resources, and in fact, I would not dream of re-using a param for two different purposes. I think my delight was just that I never know you could share directions. I can imagine that Mr Colby will find a use for a standard piece of his arsenal to have a in-out param and once he does, he will wonder how we ever existing without utilising that aspect of params. Perhaps lastupdated as a field would be an appropriate use, where the last updated value is passed in to the sproc, and then passed back again with the new value after an update has been made, after checking for concurrency errors first. Thanks Mark 2009/11/27 Gustav Brock > Hi Mark et al > > What's the big deal? Are parameters costly? Hardly in money, but in > resources? > > /gustav > > > >>> marklbreen at gmail.com 27-11-2009 11:00 >>> > Hello John, > > that Shamil has said is that you can use one param to pass in a value, and > use the same param to accept back a value, which may be different to when > it > was passes in. > > You can pass in ProductId, and get back, ProductCode, > or pass in username and get back fullname, > > or pass in product code and get back zero if the product code does not > exist, else you get back the code you passed in > or pass in productid and get back qty in stock, > > all with one param. > > My examples are not great, but there are plenty of times where you need to > somehow inspect a value and it may not be ideal to have two params named > paramOriginal > paramNewValue > > with inputout you can have just one param named paramToBeVerified > > Thanks again to Shamil, Stuart and Mr Colby for asking the question. > > Mark > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 27 08:44:16 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 09:44:16 -0500 Subject: [dba-VB] SPAM-LOW: Re: Parameter objects in C# and ADO. In-Reply-To: References: Message-ID: <4B0FE5C0.5030409@colbyconsulting.com> > I can imagine that Mr Colby will find a use for a standard piece of his arsenal to have a in-out param and once he does, he will wonder how we ever existing without utilizing that aspect of params. LOL. Your confidence is encouraging. ;) While I use "by reference" values in normal programming, it is rare that I actually modify the passed in value. But it is something that can theoretically be useful and so it is nice to know it is available. > "Mr Colby" I assume you are either feeling particularly young this week or you discovered how old I actually am? ;) All of my kid's friends call me "Mr. Colby". Have you been hanging out with Robbie? John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello Gustav, > > I presume they are not costly in terms of resources, and in fact, I would > not dream of re-using a param for two different purposes. > > I think my delight was just that I never know you could share directions. > > I can imagine that Mr Colby will find a use for a standard piece of his > arsenal to have a in-out param and once he does, he will wonder how we ever > existing without utilising that aspect of params. > > Perhaps lastupdated as a field would be an appropriate use, where the last > updated value is passed in to the sproc, and then passed back again with the > new value after an update has been made, after checking for concurrency > errors first. > > Thanks > > Mark From shamil at smsconsulting.spb.ru Fri Nov 27 08:59:16 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 27 Nov 2009 17:59:16 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0FDE11.9070108@colbyconsulting.com> References: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> <4B0FDE11.9070108@colbyconsulting.com> Message-ID: <000d01ca6f72$31f97f70$95ec7e50$@spb.ru> Hi John -- This kind of "excessive" execution result/error information is useful in my opinion for non-CRUD SPs. But for CRUD ones one can just use SPs return value: - if it's zero or positive - all is OK and returned value is @@ROWCOUNT; - if it's negative - then this is @@ERROR (if actual @@ERROR value will be positive just return it as negative number). If some kinds of runtime errors happen in SPs then they can be trapped by calling C# code try/catch block and actual error message can be obtained from System.Data.SqlClient.SqlException. Well, above is what I'd call a "lightweight" error handling approach - and it works well in many real life systems AFAIK... But for long running (set of) SPs one can use "heavyweight" error handling approach using T-SQLs TRY...CATCH features as described in MS SQL books online (local link): ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/tsqlref9/html/248df62a-7334-4bca-8262- 235a28f4b07f.htm I could be missing something. Please correct me where I'm wrong, please write about your approach to T-SQL (and calling C# code) error handling. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 27, 2009 5:11 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. RowsAffected is not part of the error stuff and I definitely want that info in most cases. I haven't done this yet but I don't think that getting back the error code and description is excessive. If it is possible to easily and unambiguously look up the error code and get the exact description, each and every time, then passing back the description is not necessary. This is exactly the kind of thing I will be doing in the future. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Gustav at all, > > BTW, have you ever seen the following below error handling in CRUD SPs as I > have found in one of my customers' databases? > Isn't an overkill? > Or do you use even more detailed/elaborated error handling in your CRUD SPs? > How do you organize error handling in your CRUD SPs? (I usually just return > error code in RETURN statement). > > CREATE procedure [dbo].[GettblUserByUsername] > @Username varchar(20), > @ErrorDesc varchar(100) output, > @ErrorNo int output, > @RowsAffected int output > AS > DECLARE @lErrorNo INTEGER > DECLARE @lRowsAffected INTEGER > Begin > > SELECT tblUser.tblUserId, > tblUser.tblUserGroupId, > tblUserGroup.groupCode as UserGroup, > tblUser.Username, > tblUser.tblUserLevelId > FROM tblUser > left outer join tblUserGroup > on tblUser.tblUserGroupId = tblUserGroup.tblUserGroupId > WHERE tblUser.Username = @username > and tblUser.Authorised = 1 > > select @lErrorNo = @@ERROR, @lRowsAffected = @@ROWCOUNT > > IF @lErrorNo > 0 > Begin > set @ErrorDesc = 'error with get of tblUser' > set @ErrorNo = @lErrorNo > set @RowsAffected = 0 > RETURN > End > Else > Begin > set @ErrorDesc = 'Get was Successful for tblUser' > set @ErrorNo = 0 > set @RowsAffected = @lRowsAffected > RETURN > End > end > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, November 27, 2009 1:45 PM > To: dba-vb at databaseadvisors.com > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > Hi Mark et al > > What's the big deal? Are parameters costly? Hardly in money, but in > resources? > > /gustav > > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4641 (20091127) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4641 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4641 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 27 09:17:22 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 10:17:22 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: Message-ID: <4B0FED82.3080608@colbyconsulting.com> Over the last year or so I have built up an arsenal of stored procedures which form a system for performing big picture tasks. A set of SPs create a set of temp tables, move data into them, index them, the export chunks into large CSV files for address validation. Similar functionality on the way back in, create a chunk temp table, import the data from a CSV file, copy to a big temp table, create indexes on them, update hash codes and address valid flags, create more indexes on those fields etc. To this point I have executed these stored procedures from an Access database, simply because I could write Access VBA, however VBA is less than ideal for this functionality, not to mention that I had no ability to get data back from the SP. Thus the SP had to write to a log file which I could then read from Access. Again, less than ideal. Now that I am (slooooowwwwly) coming up to speed on C#, I have embarked on rewriting all of that control logic into C#. Once I figure out how to read back parameters from the SP so that I can see things like RecordsAffected, error numbers and messages etc. I will have made a giant step forward in fine grained control of my big picture processes. While I am at it I have to modify the stored procedures to pass back things like recordcount affected, error codes etc. Since I could not (did not know how to) get at them from VBA I hadn't bothered to add that kind of stuff to the SPs. John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello Gustav, > > I presume they are not costly in terms of resources, and in fact, I would > not dream of re-using a param for two different purposes. > > I think my delight was just that I never know you could share directions. > > I can imagine that Mr Colby will find a use for a standard piece of his > arsenal to have a in-out param and once he does, he will wonder how we ever > existing without utilising that aspect of params. > > Perhaps lastupdated as a field would be an appropriate use, where the last > updated value is passed in to the sproc, and then passed back again with the > new value after an update has been made, after checking for concurrency > errors first. > > Thanks > > Mark From jwcolby at colbyconsulting.com Fri Nov 27 09:24:15 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 10:24:15 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <000d01ca6f72$31f97f70$95ec7e50$@spb.ru> References: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> <4B0FDE11.9070108@colbyconsulting.com> <000d01ca6f72$31f97f70$95ec7e50$@spb.ru> Message-ID: <4B0FEF1F.8060905@colbyconsulting.com> Shamil, I am looking for guidance in this area myself. As you know by now, the particular application I am working on is not CRUD based objects but rather specific sets of SPs to perform specific operations in my business. To this point, I have used VBA and a function for executing a stored procedure provided by Charlotte. I could never figure out how to use that function to get at returned values so I have to this point not used that functionality at all. Now I expect to be able to do so, and as a result I will be expanding my processing to include such error handling. It will be a long time before I am correcting anyone on this stuff! John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > This kind of "excessive" execution result/error information is useful in my > opinion for non-CRUD SPs. > But for CRUD ones one can just use SPs return value: > > - if it's zero or positive - all is OK and returned value is @@ROWCOUNT; > - if it's negative - then this is @@ERROR (if actual @@ERROR value will be > positive just return it as negative number). > > If some kinds of runtime errors happen in SPs then they can be trapped by > calling C# code try/catch block and actual error message can be obtained > from System.Data.SqlClient.SqlException. > > Well, above is what I'd call a "lightweight" error handling approach - and > it works well in many real life systems AFAIK... > > But for long running (set of) SPs one can use "heavyweight" error handling > approach using T-SQLs > > TRY...CATCH > > features as described in MS SQL books online (local link): > > ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/tsqlref9/html/248df62a-7334-4bca-8262- > 235a28f4b07f.htm > > I could be missing something. > Please correct me where I'm wrong, please write about your approach to T-SQL > (and calling C# code) error handling. > > Thank you. > > -- > Shamil From accessd at shaw.ca Fri Nov 27 11:50:56 2009 From: accessd at shaw.ca (Jim Lawrence) Date: Fri, 27 Nov 2009 09:50:56 -0800 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0FDE11.9070108@colbyconsulting.com> References: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> <4B0FDE11.9070108@colbyconsulting.com> Message-ID: <805FE569738A4777BD83069AB8C812CE@creativesystemdesigns.com> Hi John: ADO always returns the number of rows affected in MS SQL SP call. It just has to be retrieved. Here is the code that I used as a sample/base before. http://msdn.microsoft.com/en-us/library/aa302325.aspx Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 27, 2009 6:11 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. RowsAffected is not part of the error stuff and I definitely want that info in most cases. I haven't done this yet but I don't think that getting back the error code and description is excessive. If it is possible to easily and unambiguously look up the error code and get the exact description, each and every time, then passing back the description is not necessary. This is exactly the kind of thing I will be doing in the future. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Gustav at all, > > BTW, have you ever seen the following below error handling in CRUD SPs as I > have found in one of my customers' databases? > Isn't an overkill? > Or do you use even more detailed/elaborated error handling in your CRUD SPs? > How do you organize error handling in your CRUD SPs? (I usually just return > error code in RETURN statement). > > CREATE procedure [dbo].[GettblUserByUsername] > @Username varchar(20), > @ErrorDesc varchar(100) output, > @ErrorNo int output, > @RowsAffected int output > AS > DECLARE @lErrorNo INTEGER > DECLARE @lRowsAffected INTEGER > Begin > > SELECT tblUser.tblUserId, > tblUser.tblUserGroupId, > tblUserGroup.groupCode as UserGroup, > tblUser.Username, > tblUser.tblUserLevelId > FROM tblUser > left outer join tblUserGroup > on tblUser.tblUserGroupId = tblUserGroup.tblUserGroupId > WHERE tblUser.Username = @username > and tblUser.Authorised = 1 > > select @lErrorNo = @@ERROR, @lRowsAffected = @@ROWCOUNT > > IF @lErrorNo > 0 > Begin > set @ErrorDesc = 'error with get of tblUser' > set @ErrorNo = @lErrorNo > set @RowsAffected = 0 > RETURN > End > Else > Begin > set @ErrorDesc = 'Get was Successful for tblUser' > set @ErrorNo = 0 > set @RowsAffected = @lRowsAffected > RETURN > End > end > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, November 27, 2009 1:45 PM > To: dba-vb at databaseadvisors.com > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > Hi Mark et al > > What's the big deal? Are parameters costly? Hardly in money, but in > resources? > > /gustav > > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4641 (20091127) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Nov 27 14:26:11 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 15:26:11 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0DE6F4.3070605@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> Message-ID: <4B1035E3.7000003@colbyconsulting.com> > If a parameter in a stored procedure is declared as an output (returns a value) will the parameter object in C# / ADO contain that value when the stored procedure finishes? This part of the question got lost, not directly answered AFAICT. I am directly creating parameter objects and adding them to the parameter collection of the command object. Code execution will halt until the stored procedure finishes executing and then the parameter objects in the command object's parameters collection can be looked at for the returned values? John W. Colby www.ColbyConsulting.com jwcolby wrote: > If a parameter in a stored procedure is declared as an output (returns a value) will the parameter > object in C# / ADO contain that value when the stored procedure finishes? > > I haven't used output parameters yet in stored procedures. Can values be passed in via an output > parameter or can it only be set from inside of the stored procedure? > From shamil at smsconsulting.spb.ru Fri Nov 27 15:14:53 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 00:14:53 +0300 Subject: [dba-VB] Calling SP with params from C# sample Message-ID: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> Hi John - Here is a sample for you: public static void TestSP() { string connectionString = "{{Your connection string here}}"; //CREATE Procedure [dbo].[TestParams] // @Username varchar(20), // @ErrorDesc varchar(100) output, // @ErrorNo int output, // @RowsAffected int output AS //DECLARE @lErrorNo INTEGER //DECLARE @lRowsAffected INTEGER //Begin // set @ErrorDesc = 'SUCCESS' + ', UserName = ' + IsNull(@USerName,'?') // set @ErrorNo = 1 -- OK // set @RowsAffected = 503 // RETURN 1234 //end try { using (SqlConnection connection = new SqlConnection(connectionString)) { connection.Open(); SqlCommand command = new SqlCommand("TestParams", connection); command.CommandType = System.Data.CommandType.StoredProcedure; // @Username varchar(20), command.Parameters.Add( new SqlParameter("@Username", System.Data.SqlDbType.VarChar, 20)); command.Parameters["@UserName"].Value = "SP_PARAMS_TEST"; // @ErrorDesc varchar(100) output, command.Parameters.Add( new SqlParameter("@ErrorDesc", System.Data.SqlDbType.VarChar, 100)); command.Parameters["@ErrorDesc"].Direction = System.Data.ParameterDirection.Output; // @ErrorNo int output, command.Parameters.Add( new SqlParameter("@ErrorNo", System.Data.SqlDbType.Int, 4)); command.Parameters["@ErrorNo"].Direction = System.Data.ParameterDirection.Output; // @RowsAffected int output AS command.Parameters.Add( new SqlParameter("@RowsAffected", System.Data.SqlDbType.Int, 4)); command.Parameters["@RowsAffected"].Direction = System.Data.ParameterDirection.Output; // RETURN command.Parameters.Add( new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int, 4)); command.Parameters["@RETURN_VALUE"].Direction = System.Data.ParameterDirection.ReturnValue; int retValue = command.ExecuteNonQuery(); Console.WriteLine( "retValue = {0}\n" + "@RETURN_VALUE = {1}\n" + "@ErrorDesc = '{2}'\n" + "@ErrorNo = {3}\n" + "@RowsAffected = {4}" , retValue, command.Parameters["@RETURN_VALUE"].Value.ToString(), command.Parameters["@ErrorDesc"].Value.ToString(), command.Parameters["@ErrorNo"].Value.ToString(), command.Parameters["@RowsAffected"].Value.ToString() ); } } catch (SqlException ex) { Console.WriteLine("T-SQL: {0}", ex.Message); } catch (Exception ex) { Console.WriteLine("{0}", ex.Message); } } Result: retValue = -1 @RETURN_VALUE = 1234 @ErrorDesc = 'SUCCESS, UserName = SP_PARAMS_TEST' @ErrorNo = 1 @RowsAffected = 503 -- Shamil From shamil at smsconsulting.spb.ru Fri Nov 27 15:31:02 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 00:31:02 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B1035E3.7000003@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B1035E3.7000003@colbyconsulting.com> Message-ID: <009501ca6fa8$ecaf22d0$c60d6870$@spb.ru> Hi John -- I have just posted here a sample for synchronous execution of SP with params. You can also run SPs from C# asynchronously - e.g. look MSDN for BeginExecuteNonQuery method of System.Data.SqlClient.SqlCommand class. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 27, 2009 11:26 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. > If a parameter in a stored procedure is declared as an output (returns a value) will the parameter object in C# / ADO contain that value when the stored procedure finishes? This part of the question got lost, not directly answered AFAICT. I am directly creating parameter objects and adding them to the parameter collection of the command object. Code execution will halt until the stored procedure finishes executing and then the parameter objects in the command object's parameters collection can be looked at for the returned values? John W. Colby www.ColbyConsulting.com jwcolby wrote: > If a parameter in a stored procedure is declared as an output (returns a value) will the parameter > object in C# / ADO contain that value when the stored procedure finishes? > > I haven't used output parameters yet in stored procedures. Can values be passed in via an output > parameter or can it only be set from inside of the stored procedure? > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4643 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 27 15:40:45 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 16:40:45 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <009501ca6fa8$ecaf22d0$c60d6870$@spb.ru> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B1035E3.7000003@colbyconsulting.com> <009501ca6fa8$ecaf22d0$c60d6870$@spb.ru> Message-ID: <4B10475D.8040102@colbyconsulting.com> Very cool! I am rewriting the code I found on the internet to allow me to pass in the direction as well as manipulate the param object when it returns. Thanks again for the example code. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > I have just posted here a sample for synchronous execution of SP with > params. > You can also run SPs from C# asynchronously - e.g. look MSDN for > BeginExecuteNonQuery method of System.Data.SqlClient.SqlCommand class. > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 27, 2009 11:26 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > > If a parameter in a stored procedure is declared as an output (returns a > value) will the > parameter object in C# / ADO contain that value when the stored procedure > finishes? > > This part of the question got lost, not directly answered AFAICT. > > I am directly creating parameter objects and adding them to the parameter > collection of the command > object. Code execution will halt until the stored procedure finishes > executing and then the > parameter objects in the command object's parameters collection can be > looked at for the returned > values? > > John W. Colby > www.ColbyConsulting.com > > > jwcolby wrote: >> If a parameter in a stored procedure is declared as an output (returns a > value) will the parameter >> object in C# / ADO contain that value when the stored procedure finishes? >> >> I haven't used output parameters yet in stored procedures. Can values be > passed in via an output >> parameter or can it only be set from inside of the stored procedure? >> > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4643 (20091127) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Nov 27 16:02:17 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 01:02:17 +0300 Subject: [dba-VB] Async calling SP with params from C# sample In-Reply-To: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> Message-ID: <009601ca6fad$4a2737f0$de75a7d0$@spb.ru> Another sample - async call (partially used MSDN sample): public static void runTest() { SPTester o = new SPTester(); o.TestSP(); while (!o.Completed) System.Threading.Thread.Sleep(1000); } public class SPTester { public bool Completed { get; set; } private SqlConnection _connection; public void TestSP() { Completed = false; string connectionString = "Data Source=HAMBURG\\SQL2005;"+ "Initial Catalog=X;"+ "User Id=sa;Password=Y;"+ "Asynchronous Processing=true"; //CREATE Procedure [dbo].[TestParams] // @Username varchar(20), // @ErrorDesc varchar(100) output, // @ErrorNo int output, // @RowsAffected int output AS //DECLARE @lErrorNo INTEGER //DECLARE @lRowsAffected INTEGER //Begin // set @ErrorDesc = 'SUCCESS' + ', UserName = ' + IsNull(@USerName,'?') // set @ErrorNo = 1 -- OK // set @RowsAffected = 503 // RETURN 1234 //end try { _connection = new SqlConnection(connectionString); { _connection.Open(); SqlCommand command = new SqlCommand("TestParams", _connection); command.CommandType = System.Data.CommandType.StoredProcedure; // @Username varchar(20), command.Parameters.Add( new SqlParameter("@Username", System.Data.SqlDbType.VarChar, 20)); command.Parameters["@UserName"].Value = "SP_PARAMS_TEST"; // @ErrorDesc varchar(100) output, command.Parameters.Add( new SqlParameter("@ErrorDesc", System.Data.SqlDbType.VarChar, 100)); command.Parameters["@ErrorDesc"].Direction = System.Data.ParameterDirection.Output; // @ErrorNo int output, command.Parameters.Add( new SqlParameter("@ErrorNo", System.Data.SqlDbType.Int, 4)); command.Parameters["@ErrorNo"].Direction = System.Data.ParameterDirection.Output; // @RowsAffected int output AS command.Parameters.Add( new SqlParameter("@RowsAffected", System.Data.SqlDbType.Int, 4)); command.Parameters["@RowsAffected"].Direction = System.Data.ParameterDirection.Output; // RETURN command.Parameters.Add( new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int, 4)); command.Parameters["@RETURN_VALUE"].Direction = System.Data.ParameterDirection.ReturnValue; AsyncCallback callback = new AsyncCallback(handleCallback); command.BeginExecuteNonQuery(callback, command); } } catch (Exception ex) { Console.WriteLine(ex.Message); if (_connection != null) { _connection.Close(); _connection = null; } } } private void handleCallback(IAsyncResult result) { try { SqlCommand command = (SqlCommand)result.AsyncState; int rowCount = command.EndExecuteNonQuery(result); Console.WriteLine( "rowCount = {0}\n" + "@RETURN_VALUE = {1}\n" + "@ErrorDesc = '{2}'\n" + "@ErrorNo = {3}\n" + "@RowsAffected = {4}" , rowCount, command.Parameters["@RETURN_VALUE"].Value.ToString(), command.Parameters["@ErrorDesc"].Value.ToString(), command.Parameters["@ErrorNo"].Value.ToString(), command.Parameters["@RowsAffected"].Value.ToString() ); } catch (SqlException ex) { Console.WriteLine("T-SQL: {0}", ex.Message); } catch (Exception ex) { Console.WriteLine("{0}", ex.Message); } finally { Completed = true; if (_connection != null) { _connection.Close(); _connection = null; } } } } -- Shamil From shamil at smsconsulting.spb.ru Fri Nov 27 16:09:52 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 01:09:52 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B10475D.8040102@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B1035E3.7000003@colbyconsulting.com> <009501ca6fa8$ecaf22d0$c60d6870$@spb.ru> <4B10475D.8040102@colbyconsulting.com> Message-ID: <009701ca6fae$591f3310$0b5d9930$@spb.ru> Hi John -- You're welcome! I have just posted another sample code where SP call from C# is processed asynchronously. -- Shamil P.S. There is an issue with the code I mentioned above - Completed flag should be also set to true in the catch() block of the calling method: public void TestSP() -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 28, 2009 12:41 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. Very cool! I am rewriting the code I found on the internet to allow me to pass in the direction as well as manipulate the param object when it returns. Thanks again for the example code. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > I have just posted here a sample for synchronous execution of SP with > params. > You can also run SPs from C# asynchronously - e.g. look MSDN for > BeginExecuteNonQuery method of System.Data.SqlClient.SqlCommand class. > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 27, 2009 11:26 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > > If a parameter in a stored procedure is declared as an output (returns a > value) will the > parameter object in C# / ADO contain that value when the stored procedure > finishes? > > This part of the question got lost, not directly answered AFAICT. > > I am directly creating parameter objects and adding them to the parameter > collection of the command > object. Code execution will halt until the stored procedure finishes > executing and then the > parameter objects in the command object's parameters collection can be > looked at for the returned > values? > > John W. Colby > www.ColbyConsulting.com > > > jwcolby wrote: >> If a parameter in a stored procedure is declared as an output (returns a > value) will the parameter >> object in C# / ADO contain that value when the stored procedure finishes? >> >> I haven't used output parameters yet in stored procedures. Can values be > passed in via an output >> parameter or can it only be set from inside of the stored procedure? >> > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4643 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 27 21:15:28 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 22:15:28 -0500 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> Message-ID: <4B1095D0.2050804@colbyconsulting.com> Hi Shamil, I assume that the parameters on the C# side have to be in the same order / name / datatype / size as the parameters in the stored procedure? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John - > > > > Here is a sample for you: > > > > public static void TestSP() > > { > > string connectionString = "{{Your connection string here}}"; > > > > //CREATE Procedure [dbo].[TestParams] > > // @Username varchar(20), > > // @ErrorDesc varchar(100) output, > > // @ErrorNo int output, > > // @RowsAffected int output AS > > //DECLARE @lErrorNo INTEGER > > //DECLARE @lRowsAffected INTEGER > > //Begin > > // set @ErrorDesc = 'SUCCESS' + ', UserName = ' + IsNull(@USerName,'?') > > // set @ErrorNo = 1 -- OK > > // set @RowsAffected = 503 > > // RETURN 1234 > > //end > > > > try > > { > > using (SqlConnection connection = new > SqlConnection(connectionString)) > > { > > connection.Open(); > > > > SqlCommand command = new SqlCommand("TestParams", connection); > > command.CommandType = System.Data.CommandType.StoredProcedure; > > // @Username varchar(20), > > command.Parameters.Add( > > new SqlParameter("@Username", System.Data.SqlDbType.VarChar, > 20)); > > command.Parameters["@UserName"].Value = "SP_PARAMS_TEST"; > > // @ErrorDesc varchar(100) output, > > command.Parameters.Add( > > new SqlParameter("@ErrorDesc", > System.Data.SqlDbType.VarChar, 100)); > > command.Parameters["@ErrorDesc"].Direction = > System.Data.ParameterDirection.Output; > > // @ErrorNo int output, > > command.Parameters.Add( > > new SqlParameter("@ErrorNo", System.Data.SqlDbType.Int, 4)); > > command.Parameters["@ErrorNo"].Direction = > System.Data.ParameterDirection.Output; > > // @RowsAffected int output AS > > command.Parameters.Add( > > new SqlParameter("@RowsAffected", System.Data.SqlDbType.Int, > 4)); > > command.Parameters["@RowsAffected"].Direction = > System.Data.ParameterDirection.Output; > > // RETURN > > command.Parameters.Add( > > new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int, > 4)); > > command.Parameters["@RETURN_VALUE"].Direction = > System.Data.ParameterDirection.ReturnValue; > > > > int retValue = command.ExecuteNonQuery(); > > > > Console.WriteLine( > > "retValue = {0}\n" + > > "@RETURN_VALUE = {1}\n" + > > "@ErrorDesc = '{2}'\n" + > > "@ErrorNo = {3}\n" + > > "@RowsAffected = {4}" > > , > > retValue, > > command.Parameters["@RETURN_VALUE"].Value.ToString(), > > command.Parameters["@ErrorDesc"].Value.ToString(), > > command.Parameters["@ErrorNo"].Value.ToString(), > > command.Parameters["@RowsAffected"].Value.ToString() > > ); > > } > > } > > catch (SqlException ex) > > { > > Console.WriteLine("T-SQL: {0}", ex.Message); > > } > > catch (Exception ex) > > { > > Console.WriteLine("{0}", ex.Message); > > } > > } > > > > Result: > > > > retValue = -1 > > @RETURN_VALUE = 1234 > > @ErrorDesc = 'SUCCESS, UserName = SP_PARAMS_TEST' > > @ErrorNo = 1 > > @RowsAffected = 503 > > > > -- > > Shamil > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 27 21:45:19 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 22:45:19 -0500 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> Message-ID: <4B109CCF.5020107@colbyconsulting.com> Shamil, I executed my first SP using your example code (first light as they say in the astronomy world). Man oh man is it ugly though. Tons of code to get one lonely little SP to execute (NOT your fault!!!). Thank you so much for the examples, I can definitely take it from here. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John - > > > > Here is a sample for you: > > > > public static void TestSP() > > { > > string connectionString = "{{Your connection string here}}"; > > > > //CREATE Procedure [dbo].[TestParams] > > // @Username varchar(20), > > // @ErrorDesc varchar(100) output, > > // @ErrorNo int output, > > // @RowsAffected int output AS > > //DECLARE @lErrorNo INTEGER > > //DECLARE @lRowsAffected INTEGER > > //Begin > > // set @ErrorDesc = 'SUCCESS' + ', UserName = ' + IsNull(@USerName,'?') > > // set @ErrorNo = 1 -- OK > > // set @RowsAffected = 503 > > // RETURN 1234 > > //end > > > > try > > { > > using (SqlConnection connection = new > SqlConnection(connectionString)) > > { > > connection.Open(); > > > > SqlCommand command = new SqlCommand("TestParams", connection); > > command.CommandType = System.Data.CommandType.StoredProcedure; > > // @Username varchar(20), > > command.Parameters.Add( > > new SqlParameter("@Username", System.Data.SqlDbType.VarChar, > 20)); > > command.Parameters["@UserName"].Value = "SP_PARAMS_TEST"; > > // @ErrorDesc varchar(100) output, > > command.Parameters.Add( > > new SqlParameter("@ErrorDesc", > System.Data.SqlDbType.VarChar, 100)); > > command.Parameters["@ErrorDesc"].Direction = > System.Data.ParameterDirection.Output; > > // @ErrorNo int output, > > command.Parameters.Add( > > new SqlParameter("@ErrorNo", System.Data.SqlDbType.Int, 4)); > > command.Parameters["@ErrorNo"].Direction = > System.Data.ParameterDirection.Output; > > // @RowsAffected int output AS > > command.Parameters.Add( > > new SqlParameter("@RowsAffected", System.Data.SqlDbType.Int, > 4)); > > command.Parameters["@RowsAffected"].Direction = > System.Data.ParameterDirection.Output; > > // RETURN > > command.Parameters.Add( > > new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int, > 4)); > > command.Parameters["@RETURN_VALUE"].Direction = > System.Data.ParameterDirection.ReturnValue; > > > > int retValue = command.ExecuteNonQuery(); > > > > Console.WriteLine( > > "retValue = {0}\n" + > > "@RETURN_VALUE = {1}\n" + > > "@ErrorDesc = '{2}'\n" + > > "@ErrorNo = {3}\n" + > > "@RowsAffected = {4}" > > , > > retValue, > > command.Parameters["@RETURN_VALUE"].Value.ToString(), > > command.Parameters["@ErrorDesc"].Value.ToString(), > > command.Parameters["@ErrorNo"].Value.ToString(), > > command.Parameters["@RowsAffected"].Value.ToString() > > ); > > } > > } > > catch (SqlException ex) > > { > > Console.WriteLine("T-SQL: {0}", ex.Message); > > } > > catch (Exception ex) > > { > > Console.WriteLine("{0}", ex.Message); > > } > > } > > > > Result: > > > > retValue = -1 > > @RETURN_VALUE = 1234 > > @ErrorDesc = 'SUCCESS, UserName = SP_PARAMS_TEST' > > @ErrorNo = 1 > > @RowsAffected = 503 > > > > -- > > Shamil > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Nov 28 03:19:35 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 12:19:35 +0300 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <4B1095D0.2050804@colbyconsulting.com> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B1095D0.2050804@colbyconsulting.com> Message-ID: <00a201ca700b$e98b6ba0$bca242e0$@spb.ru> Hi John, The order of parameters' creation on C# side doesn't matter, name and datatype and size do matter. There could be some variations for the latter two but better keep them in strict correspondence. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 28, 2009 6:15 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Calling SP with params from C# sample Hi Shamil, I assume that the parameters on the C# side have to be in the same order / name / datatype / size as the parameters in the stored procedure? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John - > > > > Here is a sample for you: > > > > public static void TestSP() > <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4643 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From shamil at smsconsulting.spb.ru Sat Nov 28 03:19:35 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 12:19:35 +0300 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <4B109CCF.5020107@colbyconsulting.com> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B109CCF.5020107@colbyconsulting.com> Message-ID: <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> John -- Tons of code for C# (or VB.NET) is not an issue at all. And for C#/VB.Net programmers also. You'll get adapted to that reality soon, I believe. I have projects with literally thousands of custom classes - they work in production for several years without any issues. And they recompile from cleaned solution with 10+ projects in 10+ seconds. And when you're working on such solution and change code/recompile/run then recompiling and restarting takes <1-2 seconds as VS recompiles/rebuilds only changed projects. Have a look e.g. on DotNetNuke(DNN) sources - and if you'll try to "dig them out" you'll find how "ugly" DNN sources are sometimes... For your case I'd use: - plain "dumb" manual coding if you have relatively static set of SPs and their parameters; - LINQ for SQL (now); - ADO.NET Entity Framework (VS2010); - statically generated custom classes to call SPs and process SPs' results. I'd not use dynamically generated SqlCommand and SqlParameters as this approach could result in too much and never ending "plumbing programming"... And in all the cases you can use the power of true OOP C# provides to minimize "copy and paste" coding approach... BTW, C# 4.0 provides some very powerful dynamic programming features - more powerful than late binding(?) - I must say I have seen some samples of that new C# 4.0 features usage but I didn't get how to use them :) - maybe they can help you to minimize "ugliness" of your C# coding but be careful to not spend too much time on "plumbing programming"... http://stackoverflow.com/questions/244302/what-do-you-think-of-the-new-c-4-0 -dynamic-keyword http://geekswithblogs.net/sdorman/archive/2008/11/16/c-4.0-dynamic-programmi ng.aspx -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 28, 2009 6:45 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Calling SP with params from C# sample Shamil, I executed my first SP using your example code (first light as they say in the astronomy world). Man oh man is it ugly though. Tons of code to get one lonely little SP to execute (NOT your fault!!!). Thank you so much for the examples, I can definitely take it from here. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John - > > > > Here is a sample for you: > > > > public static void TestSP() > > { <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4643 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From marklbreen at gmail.com Sat Nov 28 08:27:07 2009 From: marklbreen at gmail.com (Mark Breen) Date: Sat, 28 Nov 2009 14:27:07 +0000 Subject: [dba-VB] SPAM-LOW: Re: Parameter objects in C# and ADO. In-Reply-To: <4B0FE5C0.5030409@colbyconsulting.com> References: <4B0FE5C0.5030409@colbyconsulting.com> Message-ID: Haha, My kids call some people by their first name and I still call the same people Mr Dowling, sounds funny to hear it. Mark 2009/11/27 jwcolby > > I can imagine that Mr Colby will find a use for a standard piece of his > arsenal to have a in-out > param and once he does, he will wonder how we ever existing without > utilizing that aspect of params. > > LOL. Your confidence is encouraging. ;) > > While I use "by reference" values in normal programming, it is rare that I > actually modify the > passed in value. But it is something that can theoretically be useful and > so it is nice to know it > is available. > > > "Mr Colby" > > I assume you are either feeling particularly young this week or you > discovered how old I actually > am? ;) > > All of my kid's friends call me "Mr. Colby". Have you been hanging out > with Robbie? > > John W. Colby > www.ColbyConsulting.com > > > Mark Breen wrote: > > Hello Gustav, > > > > I presume they are not costly in terms of resources, and in fact, I would > > not dream of re-using a param for two different purposes. > > > > I think my delight was just that I never know you could share directions. > > > > I can imagine that Mr Colby will find a use for a standard piece of his > > arsenal to have a in-out param and once he does, he will wonder how we > ever > > existing without utilising that aspect of params. > > > > Perhaps lastupdated as a field would be an appropriate use, where the > last > > updated value is passed in to the sproc, and then passed back again with > the > > new value after an update has been made, after checking for concurrency > > errors first. > > > > Thanks > > > > Mark > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Nov 28 16:29:17 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 28 Nov 2009 17:29:17 -0500 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B109CCF.5020107@colbyconsulting.com> <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> Message-ID: <4B11A43D.70104@colbyconsulting.com> Shamil, I ended up creating a "stored procedure class". This class hosts a sCmd command object that can be programmed with the name of the SP, connection etc but then included wrapping the lines of code to create specific often used parameters into functions. /// /// The following code creates standard parameters used all of the time in my stored procedures /// Basically just wrapper code so that I can call a function to create specific parameters as needed /// /// It also allows me to completely standardize the names, data type and length of parameters in the SPs etc. /// /// public void paramDBName(string lstrDBName) { sCmd.Parameters.Add(new SqlParameter( "@DBName", SqlDbType.VarChar, 50)); sCmd.Parameters["@DBName"].Value = lstrDBName; } public void paramTblName(string lstrTblName) { sCmd.Parameters.Add(new SqlParameter("@TblName", SqlDbType.VarChar, 50)); sCmd.Parameters["@TblName"].Value = lstrTblName; } public void paramErrorDesc() { sCmd.Parameters.Add(new SqlParameter("@ErrorDesc", System.Data.SqlDbType.VarChar, 4000)); sCmd.Parameters["@ErrorDesc"].Direction = System.Data.ParameterDirection.Output; } Then I can do things like: // //sp_AZOut_AZExportToOrdered_Append // sp = new StoredProcedure(myConnection, "_aDataMaster.dbo.sp_AZOut_AZExportToOrdered_Append"); sp.paramDBName(lstrDBName); //@DBName varchar(50) input sp.paramTblName(lstrTblName); //@TblName varchar(50) input sp.paramErrorDesc(); //@ErrorDesc varchar(100) output sp.paramErrorNo(); //@ErrorNo int output, sp.paramReturnValue(); //RETURN retValue = sp.ExecuteSP(); I believe I am on my way now. Thanks again, John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > John -- > > Tons of code for C# (or VB.NET) is not an issue at all. And for C#/VB.Net > programmers also. You'll get adapted to that reality soon, I believe. From shamil at smsconsulting.spb.ru Sat Nov 28 17:12:11 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sun, 29 Nov 2009 02:12:11 +0300 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <4B11A43D.70104@colbyconsulting.com> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B109CCF.5020107@colbyconsulting.com> <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> <4B11A43D.70104@colbyconsulting.com> Message-ID: <00ba01ca7080$392fd000$ab8f7000$@spb.ru> John, Yes, that looks good with me. AFAIS you are getting what is usually declared as abstract class, and it can be inherited by custom classes, which would use their base abstract class's "services"/utilities methods. I suppose you can go even a bit further by combining sets of your standard parameters assignments into methods of your base abstract class. BTW, all the methods/properties of your abstract class, which are not planned to be called from outside of custom classes can be declared as 'protected', all the local class level common variables (of all kinds/types) can be also put with 'protected' declaration into your abstract class... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Sunday, November 29, 2009 1:29 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Calling SP with params from C# sample Shamil, I ended up creating a "stored procedure class". This class hosts a sCmd command object that can be programmed with the name of the SP, connection etc but then included wrapping the lines of code to create specific often used parameters into functions. /// /// The following code creates standard parameters used all of the time in my stored procedures /// Basically just wrapper code so that I can call a function to create specific parameters as needed /// /// It also allows me to completely standardize the names, data type and length of parameters in the SPs etc. /// /// public void paramDBName(string lstrDBName) { sCmd.Parameters.Add(new SqlParameter( "@DBName", SqlDbType.VarChar, 50)); sCmd.Parameters["@DBName"].Value = lstrDBName; } public void paramTblName(string lstrTblName) { sCmd.Parameters.Add(new SqlParameter("@TblName", SqlDbType.VarChar, 50)); sCmd.Parameters["@TblName"].Value = lstrTblName; } public void paramErrorDesc() { sCmd.Parameters.Add(new SqlParameter("@ErrorDesc", System.Data.SqlDbType.VarChar, 4000)); sCmd.Parameters["@ErrorDesc"].Direction = System.Data.ParameterDirection.Output; } Then I can do things like: // //sp_AZOut_AZExportToOrdered_Append // sp = new StoredProcedure(myConnection, "_aDataMaster.dbo.sp_AZOut_AZExportToOrdered_Append"); sp.paramDBName(lstrDBName); //@DBName varchar(50) input sp.paramTblName(lstrTblName); //@TblName varchar(50) input sp.paramErrorDesc(); //@ErrorDesc varchar(100) output sp.paramErrorNo(); //@ErrorNo int output, sp.paramReturnValue(); //RETURN retValue = sp.ExecuteSP(); I believe I am on my way now. Thanks again, John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > John -- > > Tons of code for C# (or VB.NET) is not an issue at all. And for C#/VB.Net > programmers also. You'll get adapted to that reality soon, I believe. __________ Information from ESET NOD32 Antivirus, version of virus signature database 4645 (20091128) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 28 17:19:19 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 28 Nov 2009 18:19:19 -0500 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <00ba01ca7080$392fd000$ab8f7000$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B109CCF.5020107@colbyconsulting.com> <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> <4B11A43D.70104@colbyconsulting.com> <00ba01ca7080$392fd000$ab8f7000$@spb.ru> Message-ID: <4B11AFF7.8010706@colbyconsulting.com> Perhaps I will do the inheritance thing once I get rolling. It certainly seems like a logical extension. For now I am just instantiating the SP class and manipulating methods of that class. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > John, > > Yes, that looks good with me. > AFAIS you are getting what is usually declared as abstract class, and it can > be inherited by custom classes, which would use their base abstract class's > "services"/utilities methods. I suppose you can go even a bit further by > combining sets of your standard parameters assignments into methods of your > base abstract class. BTW, all the methods/properties of your abstract class, > which are not planned to be called from outside of custom classes can be > declared as 'protected', all the local class level common variables (of all > kinds/types) can be also put with 'protected' declaration into your abstract > class... > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Sunday, November 29, 2009 1:29 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Calling SP with params from C# sample > > Shamil, > > I ended up creating a "stored procedure class". This class hosts a sCmd > command object that can be > programmed with the name of the SP, connection etc but then included > wrapping the lines of code to > create specific often used parameters into functions. > > /// > /// The following code creates standard parameters used all of the > time in my stored procedures > /// Basically just wrapper code so that I can call a function to > create specific parameters > as needed > /// > /// It also allows me to completely standardize the names, data > type and length of > parameters in the SPs etc. > /// > /// > public void paramDBName(string lstrDBName) > { > sCmd.Parameters.Add(new SqlParameter( "@DBName", > SqlDbType.VarChar, 50)); > sCmd.Parameters["@DBName"].Value = lstrDBName; > } > public void paramTblName(string lstrTblName) > { > sCmd.Parameters.Add(new SqlParameter("@TblName", > SqlDbType.VarChar, 50)); > sCmd.Parameters["@TblName"].Value = lstrTblName; > } > public void paramErrorDesc() > { > sCmd.Parameters.Add(new SqlParameter("@ErrorDesc", > System.Data.SqlDbType.VarChar, 4000)); > sCmd.Parameters["@ErrorDesc"].Direction = > System.Data.ParameterDirection.Output; > } > > Then I can do things like: > > // > //sp_AZOut_AZExportToOrdered_Append > // > sp = new StoredProcedure(myConnection, > "_aDataMaster.dbo.sp_AZOut_AZExportToOrdered_Append"); > sp.paramDBName(lstrDBName); //@DBName varchar(50) input > sp.paramTblName(lstrTblName); //@TblName varchar(50) input > sp.paramErrorDesc(); //@ErrorDesc varchar(100) > output > sp.paramErrorNo(); //@ErrorNo int output, > sp.paramReturnValue(); //RETURN > retValue = sp.ExecuteSP(); > > I believe I am on my way now. > > Thanks again, > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> John -- >> >> Tons of code for C# (or VB.NET) is not an issue at all. And for C#/VB.Net >> programmers also. You'll get adapted to that reality soon, I believe. > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4645 (20091128) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sun Nov 29 12:51:14 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 29 Nov 2009 13:51:14 -0500 Subject: [dba-VB] C# - Presenting process status Message-ID: <4B12C2A2.1010001@colbyconsulting.com> Just a general question about how you go about presenting process status back from classes. In Access I tended to use Withevents in my classes, then sink the class events in the form and use the event to pass in values to the form, and in the event sink display the status in a text box control on the form. For example the process that exports data from SQL Server to Accuzip (address validation) would display the name of the file just created and the PKFrom / PKTo pk range. I am also looking for tips on how to report status from a thread. I have vague recollections that a thread cannot directly update controls on a form (or something like that). So how do you folks display status information in a form? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sun Nov 29 15:33:05 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 30 Nov 2009 00:33:05 +0300 Subject: [dba-VB] C# - Presenting process status In-Reply-To: <4B12C2A2.1010001@colbyconsulting.com> References: <4B12C2A2.1010001@colbyconsulting.com> Message-ID: <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> Hi John, Here is one of the methods I use (there are zillion combinations) - I dunno where it comes from, did I borrowed it from somewhere or I got it elaborated myself, is it recommended way or not - but it just works, and I have "cooked" it for you here from scratch - this is codebehind for Windows Form named Form1: using System; using System.Windows.Forms; namespace WinFormsSample { public partial class Form1 : Form { // Create Windows Form Form1 with three controls: // // 1. Button: cmdTest1 -> cmdTest1_Click // 2. Button: cmdTest2 -> cmdTest2_Click // 3. TextBox: txtLog (Multiline = true) #region Test class public class Test { public delegate void ProgressStatus(string message); private static int _id; private System.Threading.Thread _thread; public void Run(ProgressStatus progressStatus) { _testId = ++_id; _progressStatus = progressStatus; System.Threading.ThreadStart ts = new System.Threading.ThreadStart(runThread); _thread = new System.Threading.Thread(ts); _thread.IsBackground = true; _thread.Start(); } private int _testId; private ProgressStatus _progressStatus; private void runThread() { for (int i = 1; i < 10; i++) { _progressStatus(string.Format("Test#{0}, cycle#{1}", _testId, i)); System.Threading.Thread.Sleep(500); } } } #endregion // Test class public Form1() { InitializeComponent(); } private Test _test1; private void cmdTest1_Click(object sender, EventArgs e) { _test1 = new Test(); _test1.Run(progressStatus); } private Test _test2; private void cmdTest2_Click(object sender, EventArgs e) { _test2 = new Test(); _test2.Run(progressStatus); } private void progressStatus(string message) { if (this.InvokeRequired) { object[] args = { message }; this.Invoke( new Test.ProgressStatus( progressStatusInThisFormThread), args); } else progressStatusInThisFormThread(message); } private static object _statusMessageOutputLocker = new object(); private void progressStatusInThisFormThread(string message) { lock (_statusMessageOutputLocker) { txtLog.Text = String.Format("{0:hh:mm:ss.fff}: {1}", DateTime.Now, message) + System.Environment.NewLine + txtLog.Text; } } } } I will try to post about another one I use in the coming days... Thank you. -- Shamil -----Original Message----- So how do you folks display status information in a form? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sun Nov 29 15:51:48 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 30 Nov 2009 00:51:48 +0300 Subject: [dba-VB] C# - Presenting process status In-Reply-To: <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> References: <4B12C2A2.1010001@colbyconsulting.com> <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> Message-ID: <00d301ca713e$28009bc0$7801d340$@spb.ru> OK, here is another sample more close to VB6/VBA WithEvents approach: using System; using System.Windows.Forms; namespace WinFormsSample { public partial class Form2 : Form { // Create Windows Form Form2 with three controls: // // 1. Button: cmdTest1 -> cmdTest1_Click // 2. Button: cmdTest2 -> cmdTest2_Click // 3. TextBox: txtLog (Multiline = true) #region Test class public class Test { public class ProgressStatusEventArgs : EventArgs { public string Message { get; set; } } public event EventHandler ProgressStatus; private static int _id; private System.Threading.Thread _thread; public void Run() { _testId = ++_id; System.Threading.ThreadStart ts = new System.Threading.ThreadStart(runThread); _thread = new System.Threading.Thread(ts); _thread.IsBackground = true; _thread.Start(); } private int _testId; private void runThread() { for (int i = 1; i < 10; i++) { string message = string.Format("Test#{0}, cycle#{1}", _testId, i); ProgressStatusEventArgs e = new ProgressStatusEventArgs(); e.Message = message; // raise event if (ProgressStatus != null) ProgressStatus(this, e); System.Threading.Thread.Sleep(500); } } } #endregion // Test class public Form2() { InitializeComponent(); } private Test _test1; private void cmdTest1_Click(object sender, EventArgs e) { _test1 = new Test(); _test1.ProgressStatus += new EventHandler(progressStatus); _test1.Run(); } private Test _test2; private void cmdTest2_Click(object sender, EventArgs e) { _test2 = new Test(); _test2.ProgressStatus += new EventHandler(progressStatus); _test2.Run(); } public delegate void ProgressStatusDelegate(string message); private void progressStatus(object sender, Test.ProgressStatusEventArgs e) { if (this.InvokeRequired) { object[] args = { e.Message }; this.Invoke( new ProgressStatusDelegate( progressStatusInThisFormThread), args); } else progressStatusInThisFormThread(e.Message); } private static object _statusMessageOutputLocker = new object(); private void progressStatusInThisFormThread(string message) { lock (_statusMessageOutputLocker) { txtLog.Text = String.Format("{0:hh:mm:ss.fff}: {1}", DateTime.Now, message) + System.Environment.NewLine + txtLog.Text; } } } } -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: Monday, November 30, 2009 12:33 AM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] C# - Presenting process status <<< I will try to post about another one I use in the coming days... >>> From jwcolby at colbyconsulting.com Sun Nov 29 22:09:43 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 29 Nov 2009 23:09:43 -0500 Subject: [dba-VB] C# - Presenting process status In-Reply-To: <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> References: <4B12C2A2.1010001@colbyconsulting.com> <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> Message-ID: <4B134587.9010005@colbyconsulting.com> Shamil, Thanks for this. I will need to study it. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > Here is one of the methods I use (there are zillion combinations) - I dunno > where it comes from, did I borrowed it from somewhere or I got it elaborated > myself, is it recommended way or not - but it just works, and I have > "cooked" it for you here from scratch - this is codebehind for Windows Form > named Form1: > > using System; > using System.Windows.Forms; > > namespace WinFormsSample > { > public partial class Form1 : Form > { > // Create Windows Form Form1 with three controls: > // > // 1. Button: cmdTest1 -> cmdTest1_Click > // 2. Button: cmdTest2 -> cmdTest2_Click > // 3. TextBox: txtLog (Multiline = true) > > #region Test class > public class Test > { > public delegate void ProgressStatus(string message); > > private static int _id; > private System.Threading.Thread _thread; > public void Run(ProgressStatus progressStatus) > { > _testId = ++_id; > _progressStatus = progressStatus; > > System.Threading.ThreadStart ts = > new System.Threading.ThreadStart(runThread); > _thread = new System.Threading.Thread(ts); > _thread.IsBackground = true; > _thread.Start(); > } > > private int _testId; > private ProgressStatus _progressStatus; > private void runThread() > { > for (int i = 1; i < 10; i++) > { > _progressStatus(string.Format("Test#{0}, cycle#{1}", > _testId, i)); > System.Threading.Thread.Sleep(500); > } > } > } > #endregion // Test class > > public Form1() > { > InitializeComponent(); > } > > private Test _test1; > private void cmdTest1_Click(object sender, EventArgs e) > { > _test1 = new Test(); > _test1.Run(progressStatus); > } > > private Test _test2; > private void cmdTest2_Click(object sender, EventArgs e) > { > _test2 = new Test(); > _test2.Run(progressStatus); > } > > private void progressStatus(string message) > { > if (this.InvokeRequired) > { > object[] args = { message }; > this.Invoke( > new Test.ProgressStatus( > progressStatusInThisFormThread), > args); > } > else > progressStatusInThisFormThread(message); > > } > > private static object _statusMessageOutputLocker = new object(); > private void progressStatusInThisFormThread(string message) > { > lock (_statusMessageOutputLocker) > { > txtLog.Text = > String.Format("{0:hh:mm:ss.fff}: {1}", > DateTime.Now, > message) + > System.Environment.NewLine + > txtLog.Text; > } > } > } > } > > I will try to post about another one I use in the coming days... > > Thank you. > > -- > Shamil > > -----Original Message----- > So how do you folks display status information in a form? From jwcolby at colbyconsulting.com Mon Nov 30 11:44:39 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 12:44:39 -0500 Subject: [dba-VB] C# - return value for stored procedures Message-ID: <4B140487.1030905@colbyconsulting.com> While I am getting back values in my "output" parameters for the SPs, I am getting a constant -1 for the return value. In the stored procedure I have tried just using RETURN 1234 and also creating an int variable, setting that to 1234 and returning that. declare @Ret int select @ret = 1234 RETURN @ret In all cases I get a -1 value which leads me to believe that the problem is on the C# side of things. My c# code looks like: try { if (sCmd.Connection.State != ConnectionState.Open) sCmd.Connection.Open(); int retValue = sCmd.ExecuteNonQuery(); return retValue; } catch (SqlException sqlEx) { throw sqlEx; } I am not throwing an error, the SP executes, but retValue is always -1. Any ideas? Do I need to add a SqlParameter to the Parameters collection for the return value? -- John W. Colby www.ColbyConsulting.com From james at fcidms.com Mon Nov 30 11:56:47 2009 From: james at fcidms.com (James Barash) Date: Mon, 30 Nov 2009 12:56:47 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <4B140487.1030905@colbyconsulting.com> Message-ID: <12D2F894654B53498DEF840AE12B7BB36B4B5797@fciexchange.fcidms.com> John: You do, indeed, need to define a Parameter for the return value. From the Microsoft documentation: -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, November 30, 2009 12:45 PM To: VBA Subject: [dba-VB] C# - return value for stored procedures While I am getting back values in my "output" parameters for the SPs, I am getting a constant -1 for the return value. In the stored procedure I have tried just using RETURN 1234 and also creating an int variable, setting that to 1234 and returning that. declare @Ret int select @ret = 1234 RETURN @ret In all cases I get a -1 value which leads me to believe that the problem is on the C# side of things. My c# code looks like: try { if (sCmd.Connection.State != ConnectionState.Open) sCmd.Connection.Open(); int retValue = sCmd.ExecuteNonQuery(); return retValue; } catch (SqlException sqlEx) { throw sqlEx; } I am not throwing an error, the SP executes, but retValue is always -1. Any ideas? Do I need to add a SqlParameter to the Parameters collection for the return value? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From james at fcidms.com Mon Nov 30 12:06:48 2009 From: james at fcidms.com (James Barash) Date: Mon, 30 Nov 2009 13:06:48 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <4B140487.1030905@colbyconsulting.com> Message-ID: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com> John: You do, indeed, need to define a Parameter for the return value. From the Microsoft documentation: SqlCommand.ExecuteNonQuery Method Executes a Transact-SQL statement against the connection and returns the number of rows affected. To add a Return Value parameter: SqlParameter ret = new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int); ret.Direction = ParameterDirection.ReturnValue; cmd.Parameters.Add(ret); Hope this helps. James Barash -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, November 30, 2009 12:45 PM To: VBA Subject: [dba-VB] C# - return value for stored procedures While I am getting back values in my "output" parameters for the SPs, I am getting a constant -1 for the return value. In the stored procedure I have tried just using RETURN 1234 and also creating an int variable, setting that to 1234 and returning that. declare @Ret int select @ret = 1234 RETURN @ret In all cases I get a -1 value which leads me to believe that the problem is on the C# side of things. My c# code looks like: try { if (sCmd.Connection.State != ConnectionState.Open) sCmd.Connection.Open(); int retValue = sCmd.ExecuteNonQuery(); return retValue; } catch (SqlException sqlEx) { throw sqlEx; } I am not throwing an error, the SP executes, but retValue is always -1. Any ideas? Do I need to add a SqlParameter to the Parameters collection for the return value? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Mon Nov 30 12:08:16 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 13:08:16 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <12D2F894654B53498DEF840AE12B7BB36B4B5797@fciexchange.fcidms.com> References: <12D2F894654B53498DEF840AE12B7BB36B4B5797@fciexchange.fcidms.com> Message-ID: <4B140A10.4020409@colbyconsulting.com> Thanks James. Your response did not provide a link or anything but knowing that I have to do this, I will go a Googling. Thanks, John W. Colby www.ColbyConsulting.com James Barash wrote: > John: > > You do, indeed, need to define a Parameter for the return value. From the Microsoft documentation: > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Monday, November 30, 2009 12:45 PM > To: VBA > Subject: [dba-VB] C# - return value for stored procedures > > While I am getting back values in my "output" parameters for the SPs, I am getting a constant -1 for the return value. > > In the stored procedure I have tried just using > > RETURN 1234 > > and also creating an int variable, setting that to 1234 and returning that. > > declare @Ret int > select @ret = 1234 > RETURN @ret > > In all cases I get a -1 value which leads me to believe that the problem is on the C# side of things. > > My c# code looks like: > > try > { > if (sCmd.Connection.State != ConnectionState.Open) > sCmd.Connection.Open(); > int retValue = sCmd.ExecuteNonQuery(); > return retValue; > } > catch (SqlException sqlEx) > { > throw sqlEx; > } > > I am not throwing an error, the SP executes, but retValue is always -1. > > Any ideas? Do I need to add a SqlParameter to the Parameters collection for the return value? > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Mon Nov 30 12:25:27 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 13:25:27 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com> References: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com> Message-ID: <4B140E17.3050400@colbyconsulting.com> James, So what does this do for me? int retValue = sCmd.ExecuteNonQuery(); From my Googling it seems like I have to do something like: retValue = Int32.Parse(sCmd.Parameters["@Return_Value"].Value.ToString()); which is working just fine. scmd.ExecuteNonQuery() returns a -1 (not sure why) and then the @Return_Value is parsed and returned. So that part is now working as well. Thanks again for the response James. John W. Colby www.ColbyConsulting.com James Barash wrote: > John: > > You do, indeed, need to define a Parameter for the return value. From the Microsoft documentation: > > SqlCommand.ExecuteNonQuery Method > > Executes a Transact-SQL statement against the connection and returns the number of rows affected. > > To add a Return Value parameter: > > SqlParameter ret = new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int); > ret.Direction = ParameterDirection.ReturnValue; > cmd.Parameters.Add(ret); > > Hope this helps. From jwcolby at colbyconsulting.com Mon Nov 30 13:55:11 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 14:55:11 -0500 Subject: [dba-VB] C# SQLCommand.Timeout Message-ID: <4B14231F.3020905@colbyconsulting.com> Any words of wisdom on setting the command.Timeout property? I have stored procedures that execute instantly, and others that can take an hour or more (large appends). How can I discover how long a stored procedure takes to run so that I can set this property dynamically? As an example I have cases where I am appending 65 million rows into a table of 6 fields. This can take a long time (at the very least perhaps 20 minutes) but if I could get a "RecordsAffected" count for such queries as well as the time it took to execute, then I could start to discover that it takes "X seconds / million" records, and set the timeout to a reasonable value based on the records to be appended. This doesn't have to be any exact time value. I assume a simple pair of time variables, then "stop time - start time"? Or should I just discover worst case and set it to that? -- John W. Colby www.ColbyConsulting.com From stuart at lexacorp.com.pg Mon Nov 30 15:07:43 2009 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Tue, 01 Dec 2009 07:07:43 +1000 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <4B140E17.3050400@colbyconsulting.com> References: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com>, <4B140E17.3050400@colbyconsulting.com> Message-ID: <4B14341F.21088.149E29F8@stuart.lexacorp.com.pg> -1 = True, meanimg that the sp executed with no errors? -- Stuart On 30 Nov 2009 at 13:25, jwcolby wrote: > which is working just fine. scmd.ExecuteNonQuery() returns a -1 (not sure why) and then the > @Return_Value is parsed and returned. > From jwcolby at colbyconsulting.com Mon Nov 30 15:35:18 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 16:35:18 -0500 Subject: [dba-VB] C# - Stored procedures working! Message-ID: <4B143A96.8080905@colbyconsulting.com> I would like to thank all of you for your assistance in getting my stored procedures working from C#. My strategy was to design a StoredProcedure class which contained all of the data and code required to run any of the stored procedures that I have written so far. We shall see whether I succeeded in that however I have a set of seven stored procedures that are required to export large tables in chunks to CSV files and I am successfully running all of those seven stored procedures and obtaining CSV files. This StoredProcedure class has a command object and a set of about eight "Parameter" methods which wrap the code for creating standard parameters used over and over in my various stored procedures, parameters like DBName, TblName, PKStart, PKend etc, as well as the return value and errorno and errormsg parameters. I then created an AccuzipExport class that "supervises" the Accuzip export process, i.e. holds all of the code to DEFINE these seven stored procedures, i.e. set up the required stored procedure class mentioned above for each of these seven SPs, and the specific parameters that each SP uses. This class then has a method that calls three of the stored procedures to create a standard named table, fill it with sorted data and index it in preparation for creating chunk files. After that the method sits in a while loop calling the remaining four stored procedures which create the chunk table, append a chunk of records to that, index it and then BCP it out. All of this stuff now works. This stuff was what the old Access database did, so I have essentially ported the Accuzip Export process to C#. However I have successfully returned a success/fail return value as well as an error code / message in the C# version, whereas I never had that ability in the Access version so I am now in some respects ahead of where I was in Access. There is more to do of course. Now that I have a robust dev platform at my fingertips I can log the process from beginning to end, errors, times for each SP to execute, stuff like that. And of course get the process status displaying on the form I use to start this process. Eventually I need to get this export process running in a thread so that I can use the big picture application for other things while this export process runs. But next comes the matching AccuzipInput class, which will pull the CSV files coming back from Accuzip back into SQL Server. With all of the grunt work done for executing Stored Procedures it should be relatively quick and easy to get the matching input class up. At any rate I wanted to say thanks to everyone for all the help and express how jazzed I am to be where I am today. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Mon Nov 30 15:37:43 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 16:37:43 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <4B14341F.21088.149E29F8@stuart.lexacorp.com.pg> References: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com>, <4B140E17.3050400@colbyconsulting.com> <4B14341F.21088.149E29F8@stuart.lexacorp.com.pg> Message-ID: <4B143B27.9050902@colbyconsulting.com> Stuart, I don't think so as I have had failures and I *think* I got a -1 regardless. I will have to pursue this one. John W. Colby www.ColbyConsulting.com Stuart McLachlan wrote: > -1 = True, meanimg that the sp executed with no errors? > From jwcolby at colbyconsulting.com Mon Nov 2 07:58:56 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 02 Nov 2009 08:58:56 -0500 Subject: [dba-VB] SQL Server 2005: Running stored procedures from C# / VBA Message-ID: <4AEEE5A0.4040901@colbyconsulting.com> I am embarking on the task of running the "DB from hell" from C#. I have several dozen stored procedures that perform various tasks and I currently run many of them from Access using a single procedure provided by Charlotte. I need a similar function written in C# (preferably) or VB.Net. ATM I have an Access database which has two specific forms, bound to two tables. These two forms each execute a set of SPs which build temp tables of data to be exported to an external process, build indexes to make the processes faster, BCP the data out to CSV files, and then the reverse on the way back in. I have another Access database that creates an order for the same client. Same kind of thing, temp tables created, indexes, fields added and so forth. My intention is to port each of these processes to C#. The reason is simply that C# has a couple of huge advantages over VBA, such as child threads and built-in capabilities in the framework that I have to really scrounge to make work in VBA. The first thing I need though is code that can execute a stored procedure, pass parameters to it, retrieve parameters back etc. With demo code preferably. Also I would like to open a conversation about how using .Net from inside of SQL Server (2005) works. Does anyone on the list use C# or VB.Net from inside of SQL Server? Where is the code stored? How to do you call it? What do you use it for? TIA, -- John W. Colby www.ColbyConsulting.com From marklbreen at gmail.com Mon Nov 2 10:18:13 2009 From: marklbreen at gmail.com (Mark Breen) Date: Tue, 3 Nov 2009 00:18:13 +0800 Subject: [dba-VB] SQL Server 2005: Running stored procedures from C# / VBA In-Reply-To: <4AEEE5A0.4040901@colbyconsulting.com> References: <4AEEE5A0.4040901@colbyconsulting.com> Message-ID: Hello John, First of all, good luck. Secondly, you mentioned a few days ago that you referred to yourself as an SQL Wannabe, well given the work you are currently doing, I think you can no longer refer to yourself as a wannabe, you are using SQL, and no doubt will be pushing it to it's limits. *SPROCs* There are loads of samples of code and I have nothing to hand here that will assist much, but I think you will be surprised how easy it is. You will get better exmaples, but as an overview you just need to create a connection - usually referred to as cnn. create a cmd to 'hold' the sproc you wish to call join the cnn to the sproc specify the type of the cmd (no results returned, or records returned or a few other types) then you add objects which are parameters, you need one each for each param in the sproc, you can even add a param that will be referred to as OUTPUT instead of INPUT - these output params can be used to pass back single values in additional or instead of, the actual result set. you can also receive back the result code of the sproc. then you just execute the cmd object. Once you get a few lines of code that handle the above, you can copy and paste for the next five years. There are a few other things that I am not as accustomed to, but when you know that you need to read in a result set, you can use a data reader. Someone else can advise on that. when you create the cnn object, usually people pull the actual string from a app.config file, so watch out for that snipit also. BTW, FWIW, other than for binding to grids or combo's, I never used a dataset in code. *BCP* You mentioned BCP, well I have no used that since SQL 6.5, but what I have used in the SSIS in SQL 2005. It is worth taking a look, you may find that you can do a lot of your work in SSIS. It can branch and call sprocs and import and export all as a package. It can be incredibly fast and would be worth take an hour or two and read up on what you can do with SSIS. *Embeding C#* Lastly, other than an article I read in MSDN magazine, where a guy wanted to do some hashing or encryption in side SQL Server I never heard of anyone using / executing code in side SQL Serer. For performance that you need, I would, with my limited skills, keep as much of the work in TSQL - IE Sprocs, and by doing so, you will keep the mass movement of data within the db. I would love to see if anyone here actually ever used the feature of including C# Code within the db. I know that this is not samples, but you can do this, and once you get a set of lines of code that 1) create the cnn 2) create the cmd 3) add the params 4) run the sproc you will not look back. Just keep as much work as possible inside sql server. *Scheduled Tasks* Finally, once last thing to mention, if you have a smtp server, it is so, so each to create scheduled tasks, that run at predefined times and then mail you when they are finished. Running the heavy jobs nightly, can take a lot of the hard work from your daily routines. Again you can manage all this with very little actually code and have working routines in minutes. HTH. Mark 2009/11/2 jwcolby > I am embarking on the task of running the "DB from hell" from C#. I have > several dozen stored > procedures that perform various tasks and I currently run many of them from > Access using a single > procedure provided by Charlotte. I need a similar function written in C# > (preferably) or VB.Net. > > ATM I have an Access database which has two specific forms, bound to two > tables. These two forms > each execute a set of SPs which build temp tables of data to be exported to > an external process, > build indexes to make the processes faster, BCP the data out to CSV files, > and then the reverse on > the way back in. > > I have another Access database that creates an order for the same client. > Same kind of thing, temp > tables created, indexes, fields added and so forth. > > My intention is to port each of these processes to C#. The reason is > simply that C# has a couple of > huge advantages over VBA, such as child threads and built-in capabilities > in the framework that I > have to really scrounge to make work in VBA. > > The first thing I need though is code that can execute a stored procedure, > pass parameters to it, > retrieve parameters back etc. With demo code preferably. > > Also I would like to open a conversation about how using .Net from inside > of SQL Server (2005) > works. Does anyone on the list use C# or VB.Net from inside of SQL Server? > Where is the code > stored? How to do you call it? What do you use it for? > > TIA, > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Mon Nov 2 11:51:47 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 02 Nov 2009 12:51:47 -0500 Subject: [dba-VB] SQL Server 2005: Running stored procedures from C# / VBA In-Reply-To: References: <4AEEE5A0.4040901@colbyconsulting.com> Message-ID: <4AEF1C33.1020904@colbyconsulting.com> Mark, > First of all, good luck. Uh-Ohhh... is that a warning? ;) I refer to myself as a wannabe simply because to this point SQL Server has been a small part of my day. As a sole proprietor my day contains so many different kinds of work that I find it hard to be an expert in something as complex as SQL Server. There are so many people on these lists that spend all day every day in SQL Server that I will truly never be anything other than a wannabe - from their perspective. I have always been more programmer than anything. I come at the world from the perspective of "how can I write code to make my job easier". Luckily I have enough programming background that I can generally accomplish that. Right now I am trying to switch my environment of choice from Access to C#, and BOY IS THAT TOUGH. Programming is always about spending enough time doing it that the syntax becomes second nature. C# is nowhere close to second nature yet. I can taste the power though, so tantalizing. ;) I am using the Idera free backup stuff. Very powerful, recommended. It took me a couple of hours to install on my two servers and write some stored procedures to make them work. Now I really want to be able to select one specific database to backup or restore. To do that I have to be able to: See/select a database on a server. Run a stored procedure that does the backup. How simple is that eh? I just spent the morning learning about the DMO object, the listview control and writing the code to read out the databases and display them in the listview. I still have to find and get working the code that executes the stored procedure. The problem is not that this is hard, it is that I am not fluent in C# yet. But I am getting there, sloooowly but surely. John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > First of all, good luck. > > Secondly, you mentioned a few days ago that you referred to yourself as an > SQL Wannabe, well given the work you are currently doing, I think you can no > longer refer to yourself as a wannabe, you are using SQL, and no doubt will > be pushing it to it's limits. > > *SPROCs* > > There are loads of samples of code and I have nothing to hand here that will > assist much, but I think you will be surprised how easy it is. > > You will get better exmaples, but as an overview you just need to > > create a connection - usually referred to as cnn. > create a cmd to 'hold' the sproc you wish to call > join the cnn to the sproc > specify the type of the cmd (no results returned, or records returned or a > few other types) > then you add objects which are parameters, you need one each for each param > in the sproc, > you can even add a param that will be referred to as OUTPUT instead of INPUT > - these output params can be used to pass back single values in additional > or instead of, the actual result set. > you can also receive back the result code of the sproc. > then you just execute the cmd object. > > Once you get a few lines of code that handle the above, you can copy and > paste for the next five years. > > There are a few other things that I am not as accustomed to, but when you > know that you need to read in a result set, you can use a data reader. > Someone else can advise on that. > > when you create the cnn object, usually people pull the actual string from a > app.config file, so watch out for that snipit also. > > BTW, FWIW, other than for binding to grids or combo's, I never used a > dataset in code. > > > *BCP* > You mentioned BCP, well I have no used that since SQL 6.5, but what I have > used in the SSIS in SQL 2005. It is worth taking a look, you may find that > you can do a lot of your work in SSIS. It can branch and call sprocs and > import and export all as a package. It can be incredibly fast and would be > worth take an hour or two and read up on what you can do with SSIS. > > *Embeding C#* > Lastly, other than an article I read in MSDN magazine, where a guy wanted to > do some hashing or encryption in side SQL Server I never heard of anyone > using / executing code in side SQL Serer. For performance that you need, I > would, with my limited skills, keep as much of the work in TSQL - IE Sprocs, > and by doing so, you will keep the mass movement of data within the db. I > would love to see if anyone here actually ever used the feature of including > C# Code within the db. > > > I know that this is not samples, but you can do this, and once you get a set > of lines of code that > 1) create the cnn > 2) create the cmd > 3) add the params > 4) run the sproc > > you will not look back. Just keep as much work as possible inside sql > server. > > *Scheduled Tasks* > Finally, once last thing to mention, if you have a smtp server, it is so, so > each to create scheduled tasks, that run at predefined times and then mail > you when they are finished. Running the heavy jobs nightly, can take a lot > of the hard work from your daily routines. Again you can manage all this > with very little actually code and have working routines in minutes. > > HTH. > > Mark > > > > > 2009/11/2 jwcolby > >> I am embarking on the task of running the "DB from hell" from C#. I have >> several dozen stored >> procedures that perform various tasks and I currently run many of them from >> Access using a single >> procedure provided by Charlotte. I need a similar function written in C# >> (preferably) or VB.Net. >> >> ATM I have an Access database which has two specific forms, bound to two >> tables. These two forms >> each execute a set of SPs which build temp tables of data to be exported to >> an external process, >> build indexes to make the processes faster, BCP the data out to CSV files, >> and then the reverse on >> the way back in. >> >> I have another Access database that creates an order for the same client. >> Same kind of thing, temp >> tables created, indexes, fields added and so forth. >> >> My intention is to port each of these processes to C#. The reason is >> simply that C# has a couple of >> huge advantages over VBA, such as child threads and built-in capabilities >> in the framework that I >> have to really scrounge to make work in VBA. >> >> The first thing I need though is code that can execute a stored procedure, >> pass parameters to it, >> retrieve parameters back etc. With demo code preferably. >> >> Also I would like to open a conversation about how using .Net from inside >> of SQL Server (2005) >> works. Does anyone on the list use C# or VB.Net from inside of SQL Server? >> Where is the code >> stored? How to do you call it? What do you use it for? >> >> TIA, >> >> -- >> John W. Colby >> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Mon Nov 2 12:41:24 2009 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Tue, 03 Nov 2009 04:41:24 +1000 Subject: [dba-VB] SQL Server 2005: Running stored procedures from C# / VBA In-Reply-To: References: <4AEEE5A0.4040901@colbyconsulting.com>, Message-ID: <4AEF27D4.29660.1C94540E@stuart.lexacorp.com.pg> If you don't havr an SMTP Server, grab the tiny one that Max put us on to a while ago: miniRelay. You can download it from here (site in Spanish): http://www.netvicious.com/miniRelay/miniRelay.zip English documentation and FAQ here (but download link is out of date) http://www.blat.net/miniRelay/ -- Stuart On 3 Nov 2009 at 0:18, Mark Breen wrote: > > *Scheduled Tasks* > Finally, once last thing to mention, if you have a smtp server, it is so, so > each to create scheduled tasks, that run at predefined times and then mail > you when they are finished. Running the heavy jobs nightly, can take a lot > of the hard work from your daily routines. Again you can manage all this > with very little actually code and have working routines in minutes. > From jwcolby at colbyconsulting.com Tue Nov 3 21:12:23 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Tue, 03 Nov 2009 22:12:23 -0500 Subject: [dba-VB] How do you run SQL Server processes Message-ID: <4AF0F117.20209@colbyconsulting.com> I have processes that I run which use sets of stored procedures. For example two of the processes export / import records from large tables for address validation. The addresses need to be sorted in Zip5/Zip4/Address order in order to make the external validation process as fast as possible so I create a table that pulls millions of records, often tens of millions of records, into a table (created on-the-fly) in sorted order, then pulls out 2 million records at a time and BCPs them out for processing in an external program. The inverse occurs when the records finish processing, 2 million record chunks are imported back into a temp table, then each 2 million record chunk is appended to a main table. Once back in SQL Server I have processes that build clustered and non-clustered indexes, run sha-1 hash functions to generate hash keys, update address valid codes etc. I will run this entire project at least once a month (to keep the addresses valid as people move), and I do so for many tables - a table of about 65 million addresses, another of 21 million, another of 11 million etc. I store all of my stored procedures in a dedicated database that pretty much just contains the SPs and a few UDFs as well as a couple of logging type tables. At the moment I use an Access database to run the individual stored procedures in specific order. I am just embarking on a project to move that functioning but not very efficient Access database to C#. I want to be able to have better monitoring of SP completion, records processed (counts), time to complete each SP etc. I found a pretty nice set of C# classes that allow creation of "stored procedure" objects, with parameter objects, then stored into a collection as the SPs (class instances) are assembled and finally the whole collection of SPs executed. I created a test database of a million names / addresses and today I started testing my existing SPs on this small data set. In the next few days I expect to build out a preliminary "port" of the existing Access VBA code. I am hoping to eventually make the entire process "table driven" where I can store (in a table) the name of the sp, the parameters in a child table etc. and then have a supervisor pull out the records and execute the SPs based on what is in the table. This would allow me to create the SPs and then use a C# application (data entry forms) to organize the SPs, add / delete SPs from the process and so forth. That is down the road of course, for now I will have hard coded sequences - as I already do in supervisor SPs as well as in hard coded Access VBA modules. As a programmer, doing this supervisor level in C# feels more natural to me (than doing it inside of SQL Server), and gives me the flexibility to do what I want in code external to the SQL Server database being manipulated. I want to be able to do things external to the database processes, things like monitor the directories that the BCP processes export to / from, move files popping out of a bcp process off to a VM on a different server, monitor those VMs for files coming out of their processes, moving the files back over to the SQL Server directories, and trigger BCP processes that import the data back in to SQL server. I need to be able to FTP order and data files to clients, and get (FTP) files from process bureaus. So it FEELS to me like using a language such as C# to build a system outside of SQL Server is the way to go. BTW I stumbled across something called SMO (SQL Server Management Objects) which makes it really easy to get at the SQL Server object model from C#. I can apparently see the entire tree structure that is a server/database/tables/etc. and do I know not what (yet) with those objects. Looks pretty cool though. The database is a large part of what I do but it is not the whole picture and I have been handcuffed by my lack of ability to effectively do these external parts. But I am a programmer at heart. I want to know how you do this kind of stuff. I know that running such "systems" of stored procedures tied to external processes must be quite common and I am wondering what you guys do for supervising your processes. If anyone doing anything remotely similar would take the time to explain how you do it, I would be most appreciative. Thanks, -- John W. Colby www.ColbyConsulting.com From raibeart at gmail.com Wed Nov 4 12:37:58 2009 From: raibeart at gmail.com (Robert Stewart) Date: Wed, 04 Nov 2009 12:37:58 -0600 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: References: Message-ID: <4af1ca07.0637560a.20ef.1362@mx.google.com> Sorry, but using C# to do what you would and could do simply using SQL Server is adding a middle layer you do not need. You can do exactly the same thing using SQL Server and stored procedures and writing to a log table about what the process is doing and keeping track of it that way. And, not have to use a middle layer language like C# to do it. I am not sure why writing T-SQL code is not like real programming to you. But, it is. And, if you are going to continue using SQL Server, you need to learn what it can do for you instead of always fighting learning it. This is coming from someone that does T-SQL, C#, VB.Net and Access VBA. So, I think I have a fair perspective of how they all work. At 12:00 PM 11/4/2009, you wrote: >Date: Tue, 03 Nov 2009 22:12:23 -0500 >From: jwcolby >Subject: [dba-VB] How do you run SQL Server processes >To: VBA , Dba-Sqlserver > >Message-ID: <4AF0F117.20209 at colbyconsulting.com> >Content-Type: text/plain; charset=ISO-8859-1; format=flowed > >I have processes that I run which use sets of stored procedures. > >For example two of the processes export / import records from large >tables for address validation. >The addresses need to be sorted in Zip5/Zip4/Address order in order >to make the external validation >process as fast as possible so I create a table that pulls millions >of records, often tens of >millions of records, into a table (created on-the-fly) in sorted >order, then pulls out 2 million >records at a time and BCPs them out for processing in an external program. > >The inverse occurs when the records finish processing, 2 million >record chunks are imported back >into a temp table, then each 2 million record chunk is appended to a >main table. Once back in SQL >Server I have processes that build clustered and non-clustered >indexes, run sha-1 hash functions to >generate hash keys, update address valid codes etc. I will run this >entire project at least once a >month (to keep the addresses valid as people move), and I do so for >many tables - a table of about >65 million addresses, another of 21 million, another of 11 million etc. > >I store all of my stored procedures in a dedicated database that >pretty much just contains the SPs >and a few UDFs as well as a couple of logging type tables. At the >moment I use an Access database >to run the individual stored procedures in specific order. > >I am just embarking on a project to move that functioning but not >very efficient Access database to >C#. I want to be able to have better monitoring of SP completion, >records processed (counts), time >to complete each SP etc. I found a pretty nice set of C# classes >that allow creation of "stored >procedure" objects, with parameter objects, then stored into a >collection as the SPs (class >instances) are assembled and finally the whole collection of SPs >executed. I created a test >database of a million names / addresses and today I started testing >my existing SPs on this small >data set. In the next few days I expect to build out a preliminary >"port" of the existing Access >VBA code. > >I am hoping to eventually make the entire process "table driven" >where I can store (in a table) the >name of the sp, the parameters in a child table etc. and then have a >supervisor pull out the records >and execute the SPs based on what is in the table. This would allow >me to create the SPs and then >use a C# application (data entry forms) to organize the SPs, add / >delete SPs from the process and >so forth. > >That is down the road of course, for now I will have hard coded >sequences - as I already do in >supervisor SPs as well as in hard coded Access VBA modules. > >As a programmer, doing this supervisor level in C# feels more >natural to me (than doing it inside of >SQL Server), and gives me the flexibility to do what I want in code >external to the SQL Server >database being manipulated. I want to be able to do things external >to the database processes, >things like monitor the directories that the BCP processes export to >/ from, move files popping out >of a bcp process off to a VM on a different server, monitor those >VMs for files coming out of their >processes, moving the files back over to the SQL Server directories, >and trigger BCP processes that >import the data back in to SQL server. I need to be able to FTP >order and data files to clients, >and get (FTP) files from process bureaus. So it FEELS to me like >using a language such as C# to >build a system outside of SQL Server is the way to go. > >BTW I stumbled across something called SMO (SQL Server Management >Objects) which makes it really >easy to get at the SQL Server object model from C#. I can >apparently see the entire tree structure >that is a server/database/tables/etc. and do I know not what (yet) >with those objects. Looks pretty >cool though. > >The database is a large part of what I do but it is not the whole >picture and I have been handcuffed >by my lack of ability to effectively do these external parts. But I >am a programmer at heart. I >want to know how you do this kind of stuff. > >I know that running such "systems" of stored procedures tied to >external processes must be quite >common and I am wondering what you guys do for supervising your >processes. If anyone doing anything >remotely similar would take the time to explain how you do it, I >would be most appreciative. > >Thanks, >-- >John W. Colby >www.ColbyConsulting.com > > >------------------------------ > >_______________________________________________ >dba-VB mailing list >dba-VB at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-vb > > >End of dba-VB Digest, Vol 73, Issue 3 >************************************* From jwcolby at colbyconsulting.com Wed Nov 4 15:11:58 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 04 Nov 2009 16:11:58 -0500 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <4af1ca07.0637560a.20ef.1362@mx.google.com> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> Message-ID: <4AF1EE1E.9030406@colbyconsulting.com> Robert, I don't really want to get in an argument about this. I asked a serious question and got a somewhat brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY answer I have received so that is not particularly encouraging. 8( TSQL, at least as written from inside of SSMS is about like writing GBasic code back in 1982. Well, maybe a tiny step up but not much. You are right, I probably just don't know how, but in what I will call "a real language" I can set breakpoints, I can see a call stack, I have watch statements, I have... simple very basic DEBUG capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in SSMS). I can have forms with controls displaying data, buttons that cause things to happen. I can FTP files, monitor directories for activities. Believe me, if this exists in SSMS I will jump on that! Just don't tell me to go spend thousands to buy a tool that will do some of what C# can already do for me. Modern languages, and the .Net environment in particular have capabilities that (AFAICT) TSQL can not even dream of. TSQL dreams of manipulating data, C# dreams of manipulating ... the world. Robert I am not "fighting learning it", I am light years beyond where I was two years ago, and I learn more TSQL and SQL Server every day. However from what I have seen so far, TSQL will NEVER be what I need for the supervisor. And answers like you just provided aren't really much help. I am a consultant. I am a programmer, not a DBA, or a SQL Server administrator, or a SQL Server jock at a company somewhere. I need C# for automating my business, which is NOT being a DBA (I already said that I think). I will write C# applications for clients that use SQL Server, and others that do not use SQL Server. SQL Server is used by exactly ONE of my clients, so I cannot spend the time to do whatever it is that you envision SQL Server doing because I cannot amortize the learning over other clients. Now... if you want to answer my question, I am all ears. The first thing to answer (If I am to do all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. Let's take a concrete, real world example. I have a set of virtual machines which run a third party application. I place files into an "inbox" directory, and I wait for files to pop out of the running application into the "outbox" directory. Just an aside, the files placed into the inbox are created by a set of about 5 stored procedures, which can operate on any of dozens of databases in my server. It is the case that the third party application occasionally hangs. If it does I need to rename a file in a specific directory to cause that application to restart processing that file. It is my intention to use C# to execute the SPs to create the files, move the files into the VM, then monitor files dropped into the inbox, and files popping out into the outbox. Look for "no new files in X minutes" and if that condition exists, rename the file. Log files dropped into the in-box, files popping out into the out-box, and the time it took for the files to process through the application. Log any "hangs" restarted. Supervisor stuff. Explain to me how (and more importantly WHY) I would use TSQL to do that? Other than a "when TSQL is all you have" (which is NOT true for me) mentality. A SMALL part of my business is to perform some moderately complex processes using SQL Server and SPs as a tool. I envision C# as a supervisor, not a "middle layer". I envision an application (written in C#) that runs on my desktop which starts, then monitors SQL Server processes, displays errors, restarts processes if necessary, creates databases, tables etc, backs them up and detaches them when done, logging everything that happens, showing it all to me me in real time in a Windows application. Possibly even automatically generating invoices for the work done and emailing them to the client. Explain to me one more time (with feeling) how (and more importantly WHY) I would use TSQL to do all of that stuff? John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > Sorry, but using C# to do what you would and could do simply using > SQL Server is adding a middle layer you do not need. You can do > exactly the same thing using SQL Server and stored procedures and > writing to a log table about what the process is doing and keeping > track of it that way. And, not have to use a middle layer language > like C# to do it. > > I am not sure why writing T-SQL code is not like real programming to > you. But, it is. And, if you are going to continue using SQL > Server, you need to learn what it can do for you instead of always > fighting learning it. > > This is coming from someone that does T-SQL, C#, VB.Net and Access > VBA. So, I think I have a fair perspective of how they all work. From Gustav at cactus.dk Wed Nov 4 16:21:22 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 04 Nov 2009 23:21:22 +0100 Subject: [dba-VB] How do you run SQL Server processes Message-ID: Hi John I have no answer to you but if it can bring you some relief, I feel much the same. The database is for storing data, a business layer and/or a front end is for the logic and interface. That said, nothing is perfect, and in the real world you will have to write some T-SQL or similar but I try to keep it at minimum, first because of the trouble of maintenance (for example, no version control I know of), second because I feel old when I write "GBasic style" code. And I'm not in a big corporation where I can get every expensive tool I can point at, and tools for SQL Server seem all to be enterprise-priced. So, writing views and T-SQL is because I have to, writing C# (and VBA for that matter) is because I like to. I guess Robert is a very skilled DBA and can write T-SQL with his left foot and that's very good - then we know where to ask for advice as I'll never be more than a decent low-level DBA. /gustav >>> jwcolby at colbyconsulting.com 04-11-2009 22:11 >>> Robert, I don't really want to get in an argument about this. I asked a serious question and got a somewhat brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY answer I have received so that is not particularly encouraging. 8( TSQL, at least as written from inside of SSMS is about like writing GBasic code back in 1982. Well, maybe a tiny step up but not much. You are right, I probably just don't know how, but in what I will call "a real language" I can set breakpoints, I can see a call stack, I have watch statements, I have... simple very basic DEBUG capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in SSMS). I can have forms with controls displaying data, buttons that cause things to happen. I can FTP files, monitor directories for activities. Believe me, if this exists in SSMS I will jump on that! Just don't tell me to go spend thousands to buy a tool that will do some of what C# can already do for me. Modern languages, and the .Net environment in particular have capabilities that (AFAICT) TSQL can not even dream of. TSQL dreams of manipulating data, C# dreams of manipulating ... the world. Robert I am not "fighting learning it", I am light years beyond where I was two years ago, and I learn more TSQL and SQL Server every day. However from what I have seen so far, TSQL will NEVER be what I need for the supervisor. And answers like you just provided aren't really much help. I am a consultant. I am a programmer, not a DBA, or a SQL Server administrator, or a SQL Server jock at a company somewhere. I need C# for automating my business, which is NOT being a DBA (I already said that I think). I will write C# applications for clients that use SQL Server, and others that do not use SQL Server. SQL Server is used by exactly ONE of my clients, so I cannot spend the time to do whatever it is that you envision SQL Server doing because I cannot amortize the learning over other clients. Now... if you want to answer my question, I am all ears. The first thing to answer (If I am to do all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. Let's take a concrete, real world example. I have a set of virtual machines which run a third party application. I place files into an "inbox" directory, and I wait for files to pop out of the running application into the "outbox" directory. Just an aside, the files placed into the inbox are created by a set of about 5 stored procedures, which can operate on any of dozens of databases in my server. It is the case that the third party application occasionally hangs. If it does I need to rename a file in a specific directory to cause that application to restart processing that file. It is my intention to use C# to execute the SPs to create the files, move the files into the VM, then monitor files dropped into the inbox, and files popping out into the outbox. Look for "no new files in X minutes" and if that condition exists, rename the file. Log files dropped into the in-box, files popping out into the out-box, and the time it took for the files to process through the application. Log any "hangs" restarted. Supervisor stuff. Explain to me how (and more importantly WHY) I would use TSQL to do that? Other than a "when TSQL is all you have" (which is NOT true for me) mentality. A SMALL part of my business is to perform some moderately complex processes using SQL Server and SPs as a tool. I envision C# as a supervisor, not a "middle layer". I envision an application (written in C#) that runs on my desktop which starts, then monitors SQL Server processes, displays errors, restarts processes if necessary, creates databases, tables etc, backs them up and detaches them when done, logging everything that happens, showing it all to me me in real time in a Windows application. Possibly even automatically generating invoices for the work done and emailing them to the client. Explain to me one more time (with feeling) how (and more importantly WHY) I would use TSQL to do all of that stuff? John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > Sorry, but using C# to do what you would and could do simply using > SQL Server is adding a middle layer you do not need. You can do > exactly the same thing using SQL Server and stored procedures and > writing to a log table about what the process is doing and keeping > track of it that way. And, not have to use a middle layer language > like C# to do it. > > I am not sure why writing T-SQL code is not like real programming to > you. But, it is. And, if you are going to continue using SQL > Server, you need to learn what it can do for you instead of always > fighting learning it. > > This is coming from someone that does T-SQL, C#, VB.Net and Access > VBA. So, I think I have a fair perspective of how they all work. _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Wed Nov 4 16:48:32 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 01:48:32 +0300 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <4AF1EE1E.9030406@colbyconsulting.com> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> <4AF1EE1E.9030406@colbyconsulting.com> Message-ID: <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> Hi John, <<< The first thing to answer (If I am to do all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. >>> Overview of T-SQL and CLR debugging in SQL Server 2005 http://blogs.msdn.com/sqlclr/archive/2006/06/29/651644.aspx How to: Enable SQL Server 2005 Debugging http://msdn.microsoft.com/en-us/library/s0fk6z6e(VS.80).aspx HTH, --Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 12:12 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes Robert, I don't really want to get in an argument about this. I asked a serious question and got a somewhat brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY answer I have received so that is not particularly encouraging. 8( TSQL, at least as written from inside of SSMS is about like writing GBasic code back in 1982. Well, maybe a tiny step up but not much. You are right, I probably just don't know how, but in what I will call "a real language" I can set breakpoints, I can see a call stack, I have watch statements, I have... simple very basic DEBUG capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in SSMS). I can have forms with controls displaying data, buttons that cause things to happen. I can FTP files, monitor directories for activities. Believe me, if this exists in SSMS I will jump on that! Just don't tell me to go spend thousands to buy a tool that will do some of what C# can already do for me. Modern languages, and the .Net environment in particular have capabilities that (AFAICT) TSQL can not even dream of. TSQL dreams of manipulating data, C# dreams of manipulating ... the world. Robert I am not "fighting learning it", I am light years beyond where I was two years ago, and I learn more TSQL and SQL Server every day. However from what I have seen so far, TSQL will NEVER be what I need for the supervisor. And answers like you just provided aren't really much help. I am a consultant. I am a programmer, not a DBA, or a SQL Server administrator, or a SQL Server jock at a company somewhere. I need C# for automating my business, which is NOT being a DBA (I already said that I think). I will write C# applications for clients that use SQL Server, and others that do not use SQL Server. SQL Server is used by exactly ONE of my clients, so I cannot spend the time to do whatever it is that you envision SQL Server doing because I cannot amortize the learning over other clients. Now... if you want to answer my question, I am all ears. The first thing to answer (If I am to do all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. Let's take a concrete, real world example. I have a set of virtual machines which run a third party application. I place files into an "inbox" directory, and I wait for files to pop out of the running application into the "outbox" directory. Just an aside, the files placed into the inbox are created by a set of about 5 stored procedures, which can operate on any of dozens of databases in my server. It is the case that the third party application occasionally hangs. If it does I need to rename a file in a specific directory to cause that application to restart processing that file. It is my intention to use C# to execute the SPs to create the files, move the files into the VM, then monitor files dropped into the inbox, and files popping out into the outbox. Look for "no new files in X minutes" and if that condition exists, rename the file. Log files dropped into the in-box, files popping out into the out-box, and the time it took for the files to process through the application. Log any "hangs" restarted. Supervisor stuff. Explain to me how (and more importantly WHY) I would use TSQL to do that? Other than a "when TSQL is all you have" (which is NOT true for me) mentality. A SMALL part of my business is to perform some moderately complex processes using SQL Server and SPs as a tool. I envision C# as a supervisor, not a "middle layer". I envision an application (written in C#) that runs on my desktop which starts, then monitors SQL Server processes, displays errors, restarts processes if necessary, creates databases, tables etc, backs them up and detaches them when done, logging everything that happens, showing it all to me me in real time in a Windows application. Possibly even automatically generating invoices for the work done and emailing them to the client. Explain to me one more time (with feeling) how (and more importantly WHY) I would use TSQL to do all of that stuff? John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > Sorry, but using C# to do what you would and could do simply using > SQL Server is adding a middle layer you do not need. You can do > exactly the same thing using SQL Server and stored procedures and > writing to a log table about what the process is doing and keeping > track of it that way. And, not have to use a middle layer language > like C# to do it. > > I am not sure why writing T-SQL code is not like real programming to > you. But, it is. And, if you are going to continue using SQL > Server, you need to learn what it can do for you instead of always > fighting learning it. > > This is coming from someone that does T-SQL, C#, VB.Net and Access > VBA. So, I think I have a fair perspective of how they all work. _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4574 (20091104) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4574 (20091104) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Wed Nov 4 17:01:33 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 04 Nov 2009 18:01:33 -0500 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: References: Message-ID: <4AF207CD.2060307@colbyconsulting.com> Gustav, >as I'll never be more than a decent low-level DBA. Likewise. And that is precisely my problem. I think when you spend all day in TSQL it is easy to lose sight of just how crude the environment is. I have finally gotten to the point where I can write a SP, debug it, get it functioning etc but it is just hard. Not that C# isn't hard, but C# is a general purpose language where the effort spent is useful for other things. And with the debugging tools in .Net, debugging things is really pretty easy, very similar to many other environments. I can go weeks without doing SQL development and when I go back it is a PITA to pick back up. What it does, it does well, what it doesn't do... it doesn't do quite spectacularly. > I guess Robert is a very skilled DBA and can write T-SQL with his left foot It certainly seems so. But with my current understanding of SQL Server and TSQL I cannot imagine doing the big picture of my work strictly with TSQL. With my current understanding, TSQL will only ever be a hand full of processes, 30% of the whole job. Don't get me wrong, there is no alternative to the power of SQL Server, but in the end it is just a place to store data and some tools for getting it out and back in to tables. Everything else is out in the real world. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > I have no answer to you but if it can bring you some relief, I feel much the same. The database is for storing data, a business layer and/or a front end is for the logic and interface. > That said, nothing is perfect, and in the real world you will have to write some T-SQL or similar but I try to keep it at minimum, first because of the trouble of maintenance (for example, no version control I know of), second because I feel old when I write "GBasic style" code. And I'm not in a big corporation where I can get every expensive tool I can point at, and tools for SQL Server seem all to be enterprise-priced. So, writing views and T-SQL is because I have to, writing C# (and VBA for that matter) is because I like to. > > I guess Robert is a very skilled DBA and can write T-SQL with his left foot and that's very good - then we know where to ask for advice as I'll never be more than a decent low-level DBA. > > /gustav > > >>>> jwcolby at colbyconsulting.com 04-11-2009 22:11 >>> > Robert, > > I don't really want to get in an argument about this. I asked a serious question and got a somewhat > brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY answer I have received > so that is not particularly encouraging. > > 8( > > TSQL, at least as written from inside of SSMS is about like writing GBasic code back in 1982. Well, > maybe a tiny step up but not much. > > You are right, I probably just don't know how, but in what I will call "a real language" I can set > breakpoints, I can see a call stack, I have watch statements, I have... simple very basic DEBUG > capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in SSMS). I can have forms > with controls displaying data, buttons that cause things to happen. I can FTP files, monitor > directories for activities. > > Believe me, if this exists in SSMS I will jump on that! Just don't tell me to go spend thousands to > buy a tool that will do some of what C# can already do for me. > > Modern languages, and the .Net environment in particular have capabilities that (AFAICT) TSQL can > not even dream of. TSQL dreams of manipulating data, C# dreams of manipulating ... the world. > > Robert I am not "fighting learning it", I am light years beyond where I was two years ago, and I > learn more TSQL and SQL Server every day. However from what I have seen so far, TSQL will NEVER be > what I need for the supervisor. And answers like you just provided aren't really much help. > > I am a consultant. I am a programmer, not a DBA, or a SQL Server administrator, or a SQL Server > jock at a company somewhere. I need C# for automating my business, which is NOT being a DBA (I > already said that I think). I will write C# applications for clients that use SQL Server, and > others that do not use SQL Server. SQL Server is used by exactly ONE of my clients, so I cannot > spend the time to do whatever it is that you envision SQL Server doing because I cannot amortize the > learning over other clients. > > Now... if you want to answer my question, I am all ears. The first thing to answer (If I am to do > all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. > > Let's take a concrete, real world example. I have a set of virtual machines which run a third party > application. I place files into an "inbox" directory, and I wait for files to pop out of the > running application into the "outbox" directory. Just an aside, the files placed into the inbox are > created by a set of about 5 stored procedures, which can operate on any of dozens of databases in my > server. > > It is the case that the third party application occasionally hangs. If it does I need to rename a > file in a specific directory to cause that application to restart processing that file. > > It is my intention to use C# to execute the SPs to create the files, move the files into the VM, > then monitor files dropped into the inbox, and files popping out into the outbox. Look for "no new > files in X minutes" and if that condition exists, rename the file. Log files dropped into the > in-box, files popping out into the out-box, and the time it took for the files to process through > the application. Log any "hangs" restarted. Supervisor stuff. > > Explain to me how (and more importantly WHY) I would use TSQL to do that? Other than a "when TSQL > is all you have" (which is NOT true for me) mentality. > > A SMALL part of my business is to perform some moderately complex processes using SQL Server and SPs > as a tool. I envision C# as a supervisor, not a "middle layer". I envision an application (written > in C#) that runs on my desktop which starts, then monitors SQL Server processes, displays errors, > restarts processes if necessary, creates databases, tables etc, backs them up and detaches them when > done, logging everything that happens, showing it all to me me in real time in a Windows > application. Possibly even automatically generating invoices for the work done and emailing them to > the client. > > Explain to me one more time (with feeling) how (and more importantly WHY) I would use TSQL to do all > of that stuff? > > John W. Colby > www.ColbyConsulting.com > > > Robert Stewart wrote: >> Sorry, but using C# to do what you would and could do simply using >> SQL Server is adding a middle layer you do not need. You can do >> exactly the same thing using SQL Server and stored procedures and >> writing to a log table about what the process is doing and keeping >> track of it that way. And, not have to use a middle layer language >> like C# to do it. >> >> I am not sure why writing T-SQL code is not like real programming to >> you. But, it is. And, if you are going to continue using SQL >> Server, you need to learn what it can do for you instead of always >> fighting learning it. >> >> This is coming from someone that does T-SQL, C#, VB.Net and Access >> VBA. So, I think I have a fair perspective of how they all work. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From davidmcafee at gmail.com Wed Nov 4 17:06:37 2009 From: davidmcafee at gmail.com (David McAfee) Date: Wed, 4 Nov 2009 15:06:37 -0800 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> <4AF1EE1E.9030406@colbyconsulting.com> <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> Message-ID: <8786a4c00911041506o49c88352kee08cbbb64fd62b@mail.gmail.com> In SQL 2000 (QA) you would simply right click on the Sproc in the Object explorer and choose debug. Early beta versions of SSMS 2005 had the same ability (right click and choose "step into") but it was removed and moved to Visual Studio instead. :( On Wed, Nov 4, 2009 at 2:48 PM, Shamil Salakhetdinov wrote: > Hi John, > > <<< > The first thing to answer (If I am to do > all this in TSQL) is how do I get a modern tool with all > of the debug capabilities to write TSQL code. >>>> > > Overview of T-SQL and CLR debugging in SQL Server 2005 > http://blogs.msdn.com/sqlclr/archive/2006/06/29/651644.aspx > > How to: Enable SQL Server 2005 Debugging > http://msdn.microsoft.com/en-us/library/s0fk6z6e(VS.80).aspx > > HTH, > > --Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 12:12 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes > > Robert, > > I don't really want to get in an argument about this. ?I asked a serious > question and got a somewhat > brusque "answer" that isn't an answer at all. ?OTOH it is (so far) the ONLY > answer I have received > so that is not particularly encouraging. > > 8( > > TSQL, at least as written from inside of SSMS is about like writing GBasic > code back in 1982. ?Well, > maybe a tiny step up but not much. > > You are right, I probably just don't know how, but in what I will call "a > real language" I can set > breakpoints, I can see a call stack, I have watch statements, ?I have... > simple very basic DEBUG > capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in > SSMS). ?I can have forms > with controls displaying data, buttons that cause things to happen. ?I can > FTP files, monitor > directories for activities. > > Believe me, if this exists in SSMS I will jump on that! ?Just don't tell me > to go spend thousands to > buy a tool that will do some of what C# can already do for me. > > Modern languages, and the .Net environment in particular have capabilities > that (AFAICT) TSQL can > not even dream of. ?TSQL dreams of manipulating data, C# dreams of > manipulating ... the world. > > Robert I am not "fighting learning it", I am light years beyond where I was > two years ago, and I > learn more TSQL and SQL Server every day. ?However from what I have seen so > far, TSQL will NEVER be > what I need for the supervisor. ?And answers like you just provided aren't > really much help. > > I am a consultant. ?I am a programmer, not a DBA, or a SQL Server > administrator, or a SQL Server > jock at a company somewhere. ?I need C# for automating my business, which is > NOT being a DBA (I > already said that I think). ?I will write C# applications for clients that > use SQL Server, and > others that do not use SQL Server. ?SQL Server is used by exactly ONE of my > clients, so I cannot > spend the time to do whatever it is that you envision SQL Server doing > because I cannot amortize the > learning over other clients. > > Now... if you want to answer my question, I am all ears. ?The first thing to > answer (If I am to do > all this in TSQL) is how do I get a modern tool with all of the debug > capabilities to write TSQL code. > > Let's take a concrete, real world example. ?I have a set of virtual machines > which run a third party > application. ?I place files into an "inbox" directory, and I wait for files > to pop out of the > running application into the "outbox" directory. ?Just an aside, the files > placed into the inbox are > created by a set of about 5 stored procedures, which can operate on any of > dozens of databases in my > server. > > It is the case that the third party application occasionally hangs. ?If it > does I need to rename a > file in a specific directory to cause that application to restart processing > that file. > > It is my intention to use C# to execute the SPs to create the files, move > the files into the VM, > then monitor files dropped into the inbox, and files popping out into the > outbox. ?Look for "no new > files in X minutes" and if that condition exists, rename the file. ?Log > files dropped into the > in-box, files popping out into the out-box, and the time it took for the > files to process through > the application. ?Log any "hangs" restarted. ?Supervisor stuff. > > Explain to me how (and more importantly WHY) I would use TSQL to do that? > Other than a "when TSQL > is all you have" (which is NOT true for me) mentality. > > A SMALL part of my business is to perform some moderately complex processes > using SQL Server and SPs > as a tool. ?I envision C# as a supervisor, not a "middle layer". ?I envision > an application (written > in C#) that runs on my desktop which starts, then monitors SQL Server > processes, displays errors, > restarts processes if necessary, creates databases, tables etc, backs them > up and detaches them when > done, logging everything that happens, showing it all to me me in real time > in a Windows > application. ?Possibly even automatically generating invoices for the work > done and emailing them to > the client. > > Explain to me one more time (with feeling) how (and more importantly WHY) I > would use TSQL to do all > of that stuff? > > John W. Colby > www.ColbyConsulting.com > > > Robert Stewart wrote: >> Sorry, but using C# to do what you would and could do simply using >> SQL Server is adding a middle layer you do not need. You can do >> exactly the same thing using SQL Server and stored procedures and >> writing to a log table about what the process is doing and keeping >> track of it that way. And, not have to use a middle layer language >> like C# to do it. >> >> I am not sure why writing T-SQL code is not like real programming to >> you. ?But, it is. ?And, if you are going to continue using SQL >> Server, you need to learn what it can do for you instead of always >> fighting learning it. >> >> This is coming from someone that does T-SQL, C#, VB.Net and Access >> VBA. ?So, I think I have a fair perspective of how they all work. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Wed Nov 4 17:08:49 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 04 Nov 2009 18:08:49 -0500 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> <4AF1EE1E.9030406@colbyconsulting.com> <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> Message-ID: <4AF20981.3060804@colbyconsulting.com> Thanks for that. I guess I will need to install my professional edition license to go there. Worthwhile though I am guessing. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > <<< > The first thing to answer (If I am to do > all this in TSQL) is how do I get a modern tool with all > of the debug capabilities to write TSQL code. > > Overview of T-SQL and CLR debugging in SQL Server 2005 > http://blogs.msdn.com/sqlclr/archive/2006/06/29/651644.aspx > > How to: Enable SQL Server 2005 Debugging > http://msdn.microsoft.com/en-us/library/s0fk6z6e(VS.80).aspx > > HTH, > > --Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 12:12 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes > > Robert, > > I don't really want to get in an argument about this. I asked a serious > question and got a somewhat > brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY > answer I have received > so that is not particularly encouraging. > > 8( > > TSQL, at least as written from inside of SSMS is about like writing GBasic > code back in 1982. Well, > maybe a tiny step up but not much. > > You are right, I probably just don't know how, but in what I will call "a > real language" I can set > breakpoints, I can see a call stack, I have watch statements, I have... > simple very basic DEBUG > capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in > SSMS). I can have forms > with controls displaying data, buttons that cause things to happen. I can > FTP files, monitor > directories for activities. > > Believe me, if this exists in SSMS I will jump on that! Just don't tell me > to go spend thousands to > buy a tool that will do some of what C# can already do for me. > > Modern languages, and the .Net environment in particular have capabilities > that (AFAICT) TSQL can > not even dream of. TSQL dreams of manipulating data, C# dreams of > manipulating ... the world. > > Robert I am not "fighting learning it", I am light years beyond where I was > two years ago, and I > learn more TSQL and SQL Server every day. However from what I have seen so > far, TSQL will NEVER be > what I need for the supervisor. And answers like you just provided aren't > really much help. > > I am a consultant. I am a programmer, not a DBA, or a SQL Server > administrator, or a SQL Server > jock at a company somewhere. I need C# for automating my business, which is > NOT being a DBA (I > already said that I think). I will write C# applications for clients that > use SQL Server, and > others that do not use SQL Server. SQL Server is used by exactly ONE of my > clients, so I cannot > spend the time to do whatever it is that you envision SQL Server doing > because I cannot amortize the > learning over other clients. > > Now... if you want to answer my question, I am all ears. The first thing to > answer (If I am to do > all this in TSQL) is how do I get a modern tool with all of the debug > capabilities to write TSQL code. > > Let's take a concrete, real world example. I have a set of virtual machines > which run a third party > application. I place files into an "inbox" directory, and I wait for files > to pop out of the > running application into the "outbox" directory. Just an aside, the files > placed into the inbox are > created by a set of about 5 stored procedures, which can operate on any of > dozens of databases in my > server. > > It is the case that the third party application occasionally hangs. If it > does I need to rename a > file in a specific directory to cause that application to restart processing > that file. > > It is my intention to use C# to execute the SPs to create the files, move > the files into the VM, > then monitor files dropped into the inbox, and files popping out into the > outbox. Look for "no new > files in X minutes" and if that condition exists, rename the file. Log > files dropped into the > in-box, files popping out into the out-box, and the time it took for the > files to process through > the application. Log any "hangs" restarted. Supervisor stuff. > > Explain to me how (and more importantly WHY) I would use TSQL to do that? > Other than a "when TSQL > is all you have" (which is NOT true for me) mentality. > > A SMALL part of my business is to perform some moderately complex processes > using SQL Server and SPs > as a tool. I envision C# as a supervisor, not a "middle layer". I envision > an application (written > in C#) that runs on my desktop which starts, then monitors SQL Server > processes, displays errors, > restarts processes if necessary, creates databases, tables etc, backs them > up and detaches them when > done, logging everything that happens, showing it all to me me in real time > in a Windows > application. Possibly even automatically generating invoices for the work > done and emailing them to > the client. > > Explain to me one more time (with feeling) how (and more importantly WHY) I > would use TSQL to do all > of that stuff? > > John W. Colby > www.ColbyConsulting.com > > > Robert Stewart wrote: >> Sorry, but using C# to do what you would and could do simply using >> SQL Server is adding a middle layer you do not need. You can do >> exactly the same thing using SQL Server and stored procedures and >> writing to a log table about what the process is doing and keeping >> track of it that way. And, not have to use a middle layer language >> like C# to do it. >> >> I am not sure why writing T-SQL code is not like real programming to >> you. But, it is. And, if you are going to continue using SQL >> Server, you need to learn what it can do for you instead of always >> fighting learning it. >> >> This is coming from someone that does T-SQL, C#, VB.Net and Access >> VBA. So, I think I have a fair perspective of how they all work. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From marklbreen at gmail.com Thu Nov 5 01:43:49 2009 From: marklbreen at gmail.com (Mark Breen) Date: Thu, 5 Nov 2009 15:43:49 +0800 Subject: [dba-VB] Fwd: How do you run SQL Server processes In-Reply-To: References: <4AF207CD.2060307@colbyconsulting.com> Message-ID: Hello John, FWIW, I agree with you that you need a rich environment to do the work that you plan to do. However, I think that Robert is correct in suggestion to try, as much as possible to not cross the technology stack with volumes of data when you do not have to. IOW, do all the heavy lifting in the database, and by all means return as many parameters or log results back to the front end you are building in C#, but try to avoid returning your data when ever possible. I guess I was suggesting the same with with SSIS. Where you have to export data based on decisions, SSIS is an extremely fast option. Have you already used EXECUTE command to call an sproc within an sproc? This is a useful means of creating an sproc, that running a set of sprocs, and can insert / update a log table as it progresses. For data only actvity, this might be effective. For all the front end work however, I guess you will make C# sing, like only you can, so work away, and let TSQL do what it is good at, all the rest should not involve enormous volumes of data, so the technology bridge will be irrelevant. Good luck, you know you are lucky. Most Database people never get to read about a db as big as what you have :) On last thing, in SQL Server, Cursors are a dirty word, I never see them used. However, when you have low volumes of iteration (thousands or a few tens of thousands), cursors are a very powerful way of enumerating tasks. Eg, lets imagine you had a table of states, and you needed to perform ten seperate sprocs for each state. You could use a cursor to enumerate the states table and pass in the state name to the ten different sprocs. You could log between each iteration, and you could have the whole thing running in 30 minutes. In this sort of context, I agree with Robert, that TSQL can give you Raw power quickly and easily. C# could be watching your log table and your 'Status Update' table and simply returning 1 or 2 records from the db to C# per minute, whereas within your sproc that is enumerating it might be handling millions of records each two minutes. Those millions would never cross the stack divide of DB Server to .Net. Again this sharing of processing power is probably what Robert was referring to. BTW I recently started using FileZilla and find it super. Good Luck, Mark Mark From shamil at smsconsulting.spb.ru Thu Nov 5 07:01:18 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 16:01:18 +0300 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <4AF20981.3060804@colbyconsulting.com> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> <4AF1EE1E.9030406@colbyconsulting.com> <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> <4AF20981.3060804@colbyconsulting.com> Message-ID: <00da01ca5e18$119f5080$34ddf180$@spb.ru> Hi John, I just posted links as you asked about this feature but I must note I have used T-SQL debugging just once when I used T-SQL cursors (and that was my mistake I suppose) - IOW I doubt that cursors/T-SQL debugging will be ever needed for you as you plan to use C# to drive your T-SQL's SPs... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 2:09 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes Thanks for that. I guess I will need to install my professional edition license to go there. Worthwhile though I am guessing. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > <<< > The first thing to answer (If I am to do > all this in TSQL) is how do I get a modern tool with all > of the debug capabilities to write TSQL code. > > Overview of T-SQL and CLR debugging in SQL Server 2005 > http://blogs.msdn.com/sqlclr/archive/2006/06/29/651644.aspx > > How to: Enable SQL Server 2005 Debugging > http://msdn.microsoft.com/en-us/library/s0fk6z6e(VS.80).aspx > > HTH, > > --Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 12:12 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes > > Robert, > > I don't really want to get in an argument about this. I asked a serious > question and got a somewhat > brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY > answer I have received > so that is not particularly encouraging. > > 8( > > TSQL, at least as written from inside of SSMS is about like writing GBasic > code back in 1982. Well, > maybe a tiny step up but not much. > > You are right, I probably just don't know how, but in what I will call "a > real language" I can set > breakpoints, I can see a call stack, I have watch statements, I have... > simple very basic DEBUG > capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in > SSMS). I can have forms > with controls displaying data, buttons that cause things to happen. I can > FTP files, monitor > directories for activities. > > Believe me, if this exists in SSMS I will jump on that! Just don't tell me > to go spend thousands to > buy a tool that will do some of what C# can already do for me. > > Modern languages, and the .Net environment in particular have capabilities > that (AFAICT) TSQL can > not even dream of. TSQL dreams of manipulating data, C# dreams of > manipulating ... the world. > > Robert I am not "fighting learning it", I am light years beyond where I was > two years ago, and I > learn more TSQL and SQL Server every day. However from what I have seen so > far, TSQL will NEVER be > what I need for the supervisor. And answers like you just provided aren't > really much help. > > I am a consultant. I am a programmer, not a DBA, or a SQL Server > administrator, or a SQL Server > jock at a company somewhere. I need C# for automating my business, which is > NOT being a DBA (I > already said that I think). I will write C# applications for clients that > use SQL Server, and > others that do not use SQL Server. SQL Server is used by exactly ONE of my > clients, so I cannot > spend the time to do whatever it is that you envision SQL Server doing > because I cannot amortize the > learning over other clients. > > Now... if you want to answer my question, I am all ears. The first thing to > answer (If I am to do > all this in TSQL) is how do I get a modern tool with all of the debug > capabilities to write TSQL code. > > Let's take a concrete, real world example. I have a set of virtual machines > which run a third party > application. I place files into an "inbox" directory, and I wait for files > to pop out of the > running application into the "outbox" directory. Just an aside, the files > placed into the inbox are > created by a set of about 5 stored procedures, which can operate on any of > dozens of databases in my > server. > > It is the case that the third party application occasionally hangs. If it > does I need to rename a > file in a specific directory to cause that application to restart processing > that file. > > It is my intention to use C# to execute the SPs to create the files, move > the files into the VM, > then monitor files dropped into the inbox, and files popping out into the > outbox. Look for "no new > files in X minutes" and if that condition exists, rename the file. Log > files dropped into the > in-box, files popping out into the out-box, and the time it took for the > files to process through > the application. Log any "hangs" restarted. Supervisor stuff. > > Explain to me how (and more importantly WHY) I would use TSQL to do that? > Other than a "when TSQL > is all you have" (which is NOT true for me) mentality. > > A SMALL part of my business is to perform some moderately complex processes > using SQL Server and SPs > as a tool. I envision C# as a supervisor, not a "middle layer". I envision > an application (written > in C#) that runs on my desktop which starts, then monitors SQL Server > processes, displays errors, > restarts processes if necessary, creates databases, tables etc, backs them > up and detaches them when > done, logging everything that happens, showing it all to me me in real time > in a Windows > application. Possibly even automatically generating invoices for the work > done and emailing them to > the client. > > Explain to me one more time (with feeling) how (and more importantly WHY) I > would use TSQL to do all > of that stuff? > > John W. Colby > www.ColbyConsulting.com > > > Robert Stewart wrote: >> Sorry, but using C# to do what you would and could do simply using >> SQL Server is adding a middle layer you do not need. You can do >> exactly the same thing using SQL Server and stored procedures and >> writing to a log table about what the process is doing and keeping >> track of it that way. And, not have to use a middle layer language >> like C# to do it. >> >> I am not sure why writing T-SQL code is not like real programming to >> you. But, it is. And, if you are going to continue using SQL >> Server, you need to learn what it can do for you instead of always >> fighting learning it. >> >> This is coming from someone that does T-SQL, C#, VB.Net and Access >> VBA. So, I think I have a fair perspective of how they all work. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4575 (20091105) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Thu Nov 5 07:52:53 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 05 Nov 2009 08:52:53 -0500 Subject: [dba-VB] Fwd: How do you run SQL Server processes In-Reply-To: References: <4AF207CD.2060307@colbyconsulting.com> Message-ID: <4AF2D8B5.1030507@colbyconsulting.com> Mark, > However, I think that Robert is correct in suggestion to try, as much as possible to not cross the technology stack with volumes of data when you do not have to. I have never had a problem with letting each tool do what it does best. In my case (or for this specific client) the database is decidedly not "normal". I have many different databases, each with a few tables, and ZERO (yes ZERO) parent / child tables. I keep tables of names / addresses. That's it. So I am not doing transactions, I am not doing reports, I am not doing data entry, I am not doing any of the "normal" things that people do with databases. What I have to do is get the entire table of name / address out to an external process and then back in to the same database, and I have to do this at least monthly. We are talking 20, 50, 80 million names exported into 2 million record files, processed through this external program, and then imported back in again. This is a "huge" process, not in terms of sheer SQL complexity but rather in terms of the steps involved, the monitoring that has to occur, and the time it takes. And I have to do this on a half dozen large databases, monthly. The external program processes about 2 million records an hour start to finish. Do the math and 60 million records turns into 30 hours of external processing. I use three virtual machines to get three times the records / hour but you are still talking 10 hours and that doesn't count the time to get the records out and back in to SQL Server again. My thought was to start a discussion about that "periphery processing" so that I could get ideas from people who already do it. I am getting almost no response however with actual "I do that using..." responses, and I am wondering why. Perhaps because I deal with small clients, the cost of manual labor to do those kinds of things by hand is too high so they hire me to automate it. Perhaps in a big organization they simply throw people at the problem? Believe me, when you get down to a "one person company" (me!) you can't do this stuff manually for very long, you MUST automate it or you will spend you entire day doing stuff that should be done by the computer. My original question was aimed at discovering how other people handle this kind of processing. I have another client (in Access in this case) that routinely exports data using very specialized export functions (that I wrote) dictated by mainframe interface specs. The data has to be pulled, exported, formatted, sometimes encrypted, sometimes zipped, sometimes FTPd, sometimes emailed, (the "sometimes" part depending on the recipient's preferences), and emails sent announcing the data is on the way etc. I do all of this in Access currently but it is ugly in Access because VBA does not have the built-in tools to do ftp/zip/encrypt etc. C# has some of that built-in and you can usually find C# routines to do the rest. BTW, as I have mentioned, I am taking a C# class at the local community college. I met a Russian gentleman, 60ish, who has been in programming / IT all of his life, recently unemployed. He and I are the only two in the class that have actually worked in the industry (other than the teacher). We struck up a conversation and he asked me if I would like to bid with him on a couple of small C# jobs on a jobs board "just to get some real world experience". I counter-offered that he join me in automating some of my real-world tasks. He came over yesterday and we are going to be working together on the kinds of tasks I described above. He has a LOT of C# experience but almost no database experience beyond simple select queries. We should each learn a lot from the other, it should be fun, and hopefully I can get some of this automation done! John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > FWIW, I agree with you that you need a rich environment to do the work that > you plan to do. > > However, I think that Robert is correct in suggestion to try, as much as > possible to not cross the technology stack with volumes of data when you do > not have to. > > IOW, do all the heavy lifting in the database, and by all means return as > many parameters or log results back to the front end you are building in C#, > but try to avoid returning your data when ever possible. > > I guess I was suggesting the same with with SSIS. Where you have to export > data based on decisions, SSIS is an extremely fast option. > > Have you already used EXECUTE command to call an sproc within an sproc? > This is a useful means of creating an sproc, that running a set of sprocs, > and can insert / update a log table as it progresses. For data only > actvity, this might be effective. > > For all the front end work however, I guess you will make C# sing, like only > you can, so work away, and let TSQL do what it is good at, all the rest > should not involve enormous volumes of data, so the technology bridge will > be irrelevant. > > Good luck, you know you are lucky. Most Database people never get to read > about a db as big as what you have :) > > On last thing, in SQL Server, Cursors are a dirty word, I never see them > used. However, when you have low volumes of iteration (thousands or a few > tens of thousands), cursors are a very powerful way of enumerating tasks. > > Eg, lets imagine you had a table of states, and you needed to perform ten > seperate sprocs for each state. You could use a cursor to enumerate the > states table and pass in the state name to the ten different sprocs. > > You could log between each iteration, and you could have the whole thing > running in 30 minutes. In this sort of context, I agree with Robert, that > TSQL can give you Raw power quickly and easily. > > C# could be watching your log table and your 'Status Update' table and > simply returning 1 or 2 records from the db to C# per minute, whereas within > your sproc that is enumerating it might be handling millions of records each > two minutes. Those millions would never cross the stack divide of DB Server > to .Net. Again this sharing of processing power is probably what Robert was > referring to. > > BTW I recently started using FileZilla and find it super. > > Good Luck, > > Mark > > > > Mark > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From raibeart at gmail.com Thu Nov 5 09:16:01 2009 From: raibeart at gmail.com (Robert Stewart) Date: Thu, 05 Nov 2009 09:16:01 -0600 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: References: Message-ID: <4af2ec32.0305560a.75b1.0901@mx.google.com> John, Mark has it exactly correct. Use SQL Server to the max of what it can do for you before you go off creating something in c# to do essentially the same things. Data is what it does, C# is the GUI. Very similar to what we have all done with MS Access over the years. Now, lets get down to what you are doing... Running code out of a database table is not efficient. The optimizer has to work on it before it can be run. So, the processing will be slower. It would be better to have the code one or more stored procedures with all the parameters and run them against small data sets to get them optimized. Then they would be ready for the larger chunks of data. Each SP can write to a log table and your C# GUI can display the progress by reading data out of the log tables. By using the log tables, you can pass start times, records processed and the time at predetermined numbers of records and which stored proc is doing the work at the time, end times, and just about any other information you may want, Including things that cause errors. SSIS is extremely powerful and the real tool that should probably be used for what you want to do. It can do all of the logging, error trapping and handle passing errored records into a different table for analysis. The C# GUI could be used to read the output from the SSIS package to do the monitoring. It is also relatively simple to sopy SSIS packages and change the parameters around on them to allow them to be used for different extractions and builds. Robert At 01:43 AM 11/5/2009, you wrote: >Date: Thu, 5 Nov 2009 15:43:49 +0800 >From: Mark Breen >Subject: [dba-VB] Fwd: How do you run SQL Server processes >To: dba-vb at databaseadvisors.com >Message-ID: > >Content-Type: text/plain; charset=ISO-8859-1 > >Hello John, > >FWIW, I agree with you that you need a rich environment to do the work that >you plan to do. > >However, I think that Robert is correct in suggestion to try, as much as >possible to not cross the technology stack with volumes of data when you do >not have to. > >IOW, do all the heavy lifting in the database, and by all means return as >many parameters or log results back to the front end you are building in C#, >but try to avoid returning your data when ever possible. > >I guess I was suggesting the same with with SSIS. Where you have to export >data based on decisions, SSIS is an extremely fast option. > >Have you already used EXECUTE command to call an sproc within an sproc? > This is a useful means of creating an sproc, that running a set of sprocs, >and can insert / update a log table as it progresses. For data only >actvity, this might be effective. > >For all the front end work however, I guess you will make C# sing, like only >you can, so work away, and let TSQL do what it is good at, all the rest >should not involve enormous volumes of data, so the technology bridge will >be irrelevant. > >Good luck, you know you are lucky. Most Database people never get to read >about a db as big as what you have :) > >On last thing, in SQL Server, Cursors are a dirty word, I never see them >used. However, when you have low volumes of iteration (thousands or a few >tens of thousands), cursors are a very powerful way of enumerating tasks. > >Eg, lets imagine you had a table of states, and you needed to perform ten >seperate sprocs for each state. You could use a cursor to enumerate the >states table and pass in the state name to the ten different sprocs. > >You could log between each iteration, and you could have the whole thing >running in 30 minutes. In this sort of context, I agree with Robert, that >TSQL can give you Raw power quickly and easily. > >C# could be watching your log table and your 'Status Update' table and >simply returning 1 or 2 records from the db to C# per minute, whereas within >your sproc that is enumerating it might be handling millions of records each >two minutes. Those millions would never cross the stack divide of DB Server >to .Net. Again this sharing of processing power is probably what Robert was >referring to. > >BTW I recently started using FileZilla and find it super. > >Good Luck, > >Mark > > > >Mark From shamil at smsconsulting.spb.ru Thu Nov 5 09:22:02 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 18:22:02 +0300 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <4AF0F117.20209@colbyconsulting.com> References: <4AF0F117.20209@colbyconsulting.com> Message-ID: <00db01ca5e2b$bad004c0$30700e40$@spb.ru> Hi John, <<< I am hoping to eventually make the entire process "table driven" where I can store (in a table) the name of the sp, the parameters in a child table etc. >>> Using table-driven process could be not quite right in C#/.Net world, which proposes many other options to implement generic solutions - I'd just start with "hardcoded" solutions using calls to a set of small "generic" classes/methods executing custom (MS SQL) database processing, when that set of methods/functions stabilizes I'd try to generalize/"generic-ize" it if that will be needed at all... What is the purpose of your "going generic" for this sets of tasks/customer? Isn't it "just for fun" of generic programming? Would that "going generic" make your programming more effective? Developing/debugging/testing generic custom "table-driven"/"other-metadata media driven" solutions is always (much) more time consuming than "hardcoded" ones, and supporting "table-driven" generic custom solutions, especially in the context of constantly changing customer requirements also promise to be rather time consuming - I mean generic "table-"/metadata-driven solutions could only(?) pay back well in the case they are used on many deployment sites by many people. I can be wrong, just talking from my experience, which is not universal of course... I'd better propose to use metadata descriptions/specs (stored in db tables or (XML) files), if you're so bored to write repetitive code, to generate "hardcoded" solutions, then make quick fixes to this generated hardcode to satisfy your customers' current requests, then (in free time) adjust code generator, then when everything will stabilize (will that ever happen?) develop truly generic "table-driven" solution. You can also consider using F# and Windows Workflow Foundation (WF)... The following presentation seems to be useful to get true picture of "generic vs. specific" solutions trade-offs: http://www.infoq.com/presentations/Generic-Specific-Tradeoffs-Stefan-Tilkov Thank you. -- Shamil P.S. FYI: AFAIKR MS has got created Domain Specific Language (DSL) team to develop DSL tools somewhere in year 2004 and they planned to come with first release in one(?) year but they aren't yet there (?)... -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 04, 2009 6:12 AM To: VBA; Dba-Sqlserver Subject: [dba-VB] How do you run SQL Server processes I have processes that I run which use sets of stored procedures. For example two of the processes export / import records from large tables for address validation. The addresses need to be sorted in Zip5/Zip4/Address order in order to make the external validation process as fast as possible so I create a table that pulls millions of records, often tens of millions of records, into a table (created on-the-fly) in sorted order, then pulls out 2 million records at a time and BCPs them out for processing in an external program. The inverse occurs when the records finish processing, 2 million record chunks are imported back into a temp table, then each 2 million record chunk is appended to a main table. Once back in SQL Server I have processes that build clustered and non-clustered indexes, run sha-1 hash functions to generate hash keys, update address valid codes etc. I will run this entire project at least once a month (to keep the addresses valid as people move), and I do so for many tables - a table of about 65 million addresses, another of 21 million, another of 11 million etc. I store all of my stored procedures in a dedicated database that pretty much just contains the SPs and a few UDFs as well as a couple of logging type tables. At the moment I use an Access database to run the individual stored procedures in specific order. I am just embarking on a project to move that functioning but not very efficient Access database to C#. I want to be able to have better monitoring of SP completion, records processed (counts), time to complete each SP etc. I found a pretty nice set of C# classes that allow creation of "stored procedure" objects, with parameter objects, then stored into a collection as the SPs (class instances) are assembled and finally the whole collection of SPs executed. I created a test database of a million names / addresses and today I started testing my existing SPs on this small data set. In the next few days I expect to build out a preliminary "port" of the existing Access VBA code. I am hoping to eventually make the entire process "table driven" where I can store (in a table) the name of the sp, the parameters in a child table etc. and then have a supervisor pull out the records and execute the SPs based on what is in the table. This would allow me to create the SPs and then use a C# application (data entry forms) to organize the SPs, add / delete SPs from the process and so forth. That is down the road of course, for now I will have hard coded sequences - as I already do in supervisor SPs as well as in hard coded Access VBA modules. As a programmer, doing this supervisor level in C# feels more natural to me (than doing it inside of SQL Server), and gives me the flexibility to do what I want in code external to the SQL Server database being manipulated. I want to be able to do things external to the database processes, things like monitor the directories that the BCP processes export to / from, move files popping out of a bcp process off to a VM on a different server, monitor those VMs for files coming out of their processes, moving the files back over to the SQL Server directories, and trigger BCP processes that import the data back in to SQL server. I need to be able to FTP order and data files to clients, and get (FTP) files from process bureaus. So it FEELS to me like using a language such as C# to build a system outside of SQL Server is the way to go. BTW I stumbled across something called SMO (SQL Server Management Objects) which makes it really easy to get at the SQL Server object model from C#. I can apparently see the entire tree structure that is a server/database/tables/etc. and do I know not what (yet) with those objects. Looks pretty cool though. The database is a large part of what I do but it is not the whole picture and I have been handcuffed by my lack of ability to effectively do these external parts. But I am a programmer at heart. I want to know how you do this kind of stuff. I know that running such "systems" of stored procedures tied to external processes must be quite common and I am wondering what you guys do for supervising your processes. If anyone doing anything remotely similar would take the time to explain how you do it, I would be most appreciative. Thanks, -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4573 (20091104) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From shamil at smsconsulting.spb.ru Thu Nov 5 09:39:05 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 18:39:05 +0300 Subject: [dba-VB] Recommended watching: "Thoughts on the Generic vs. Specific Tradeoff" Message-ID: <00e801ca5e2e$1c7f93f0$557ebbd0$@spb.ru> Hi All, The following presentation is a one from a wise man talking from his real life experience - it's fun and so useful to watch, recommended: http://www.infoq.com/presentations/Generic-Specific-Tradeoffs-Stefan-Tilkov Thank you. -- Shamil From jwcolby at colbyconsulting.com Thu Nov 5 10:08:36 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 05 Nov 2009 11:08:36 -0500 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <4af2ec32.0305560a.75b1.0901@mx.google.com> References: <4af2ec32.0305560a.75b1.0901@mx.google.com> Message-ID: <4AF2F884.2040903@colbyconsulting.com> Robert, Thanks for the reply. The issue here is that I have to run the same code against a large set of databases / tables. Fir this discussion, the "process" is exporting millions of records to CSV and later importing the resulting CSVs back in again. For example I have about 6 different databases, each with it's own set of tables. Additionally I have to do this exact same process on "Orders" where I am selecting anywhere from a few thousand to a few million addresses to fill an order. The client wants those orders processed as well. Understand that I am not claiming that the following is the most efficient, just "what I know how to do". So my solution so far has been to build a set of stored procedures where I pass in the database and the source table (for the export process) and dynamically build up SQL statements in the SP. The generic process is to: 1) SP1: Build a big table to hold the entire set of records, sorted on Zip5/Zip4, with an autonumber for selection of 2 million record chunks. Move the records from TblSrc (or ViewSrc) to TblBig. Build an index on autonumber and the fields to export to make the subsequent processing faster. Params DB, TblNameSrc. 2) SP2: Build a chunk table. Move a chunk from TblBig to TblChunk Params: DBName, StartPKID, EndPKID 3) SP3: BCP TblChunk to CSV file Params: DBName, Destination Dir This is not an exact representation of what is going on but gets the point across, which is that I have to do this on a "random" database, probably a dozen a month. Initially I did this in stored procedures stored inside of each database but the maintenance of the stored procedures became a nightmare. As I would figure out a better way, a different way, fix a bug... I would be constantly trying to modify my code in each database. NOT FUN!!! Eventually I moved to the parameterized stored procedure with dynamically constructed SQL executed inside of try/catch blocks etc. I store all of these SPs out in a "management" database and execute them from there, passing in the database name, table name and any other parameters required to make each SP work. As I have said, I don't know enough to determine whether this is "the best way", I have enough problems just getting this much to work. I have in fact started using logging from inside of each SP. I designed a standard log table with fields for the SP name being processed, record counts, error codes if any, memo field so I can write little explanation of each step and so forth. I JUST started doing that. I created a SP (again parameterized, stored in the management db) that performs the write to the log, then I call that SP whenever I want to log anything. The thing everyone following this thread needs to understand is that I have no resource other than this list and Google to learn how to do this stuff. I work alone, I do what I have to, as quickly as I reasonably can, because my job (for this client) is to process data, NOT write code. I am a consultant providing a service, NOT a DBA. That said, I have no one but myself to assist me in automating this stuff so that I do not do every step manually, and believe me, when I started I did EVERY STEP manually. Everything that is now a SP I did manually, sometimes dozens of times over the period I have worked for this client. It was not only UGLY, it was DAMNED UGLY. I am a programmer by trade, a DBA/Programmer by necessity, a consultant/ business analyst/ DBA/ Programmer by job description. I view ALL of this from "How can I most quickly, with the least effort, make my job easier". I don't have the luxury of learning anything that I don't HAVE to know, right now, to do my job. I also don't have the luxury of spending weeks working on one specific problem. Do a task for client A, move on to the task for client B. At the end of the month, bill for my time. Thanks for the hint to use SSIS. That is one of those "haven't had the time to learn it" things but I have always known I probably should. When you are up to your ass in alligators, it is hard to STOP... and study SSIS. John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > John, > > Mark has it exactly correct. > > Use SQL Server to the max of what it can do for you before you go off > creating something in c# to do essentially the same things. Data is what > it does, C# is the GUI. Very similar to what we have all done with MS > Access over the years. > > Now, lets get down to what you are doing... > > Running code out of a database table is not efficient. The optimizer has to > work on it before it can be run. So, the processing will be slower. > > It would be better to have the code one or more stored procedures with all the > parameters and run them against small data sets to get them > optimized. Then they > would be ready for the larger chunks of data. > > Each SP can write to a log table and your C# GUI can display the progress by > reading data out of the log tables. > > By using the log tables, you can pass start times, records processed and the > time at predetermined numbers of records and which stored proc is doing the > work at the time, end times, and just about any other information you may want, > Including things that cause errors. > > SSIS is extremely powerful and the real tool that should probably be used for > what you want to do. It can do all of the logging, error trapping and handle > passing errored records into a different table for analysis. The C# GUI could > be used to read the output from the SSIS package to do the monitoring. It is > also relatively simple to sopy SSIS packages and change the parameters around > on them to allow them to be used for different extractions and builds. > > Robert From jwcolby at colbyconsulting.com Thu Nov 5 10:33:38 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 05 Nov 2009 11:33:38 -0500 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <00db01ca5e2b$bad004c0$30700e40$@spb.ru> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> Message-ID: <4AF2FE62.2050501@colbyconsulting.com> Shamil, > Using table-driven process could be not quite right in C#/.Net world, which proposes many other options to implement generic solutions - I'd just start with "hardcoded" solutions using calls to a set of small "generic" That is precisely what I do right now, and what I will do for the near future. The table driven thing is simply that I write "processes" as discussed in this thread. A process might be: Export To Accuzip: "Export dbX TblY to files in DirectoryZ". However that breaks down onto a set of a handful of stored procedures, with passed in parameters. Now, the process "Export To Accuzip" is applied against 6 or 8 different databases every month, but ALSO against random databases which are orders. Orders come in every month and they are a whole nother discussion, but PART of an order is the process Export to Accuzip. DatabaseA exports files to its own directory, DatabaseB to its own directory, OrderAA to its own directory and so forth. But the EXPORT itself is a very standard process. So I am examining the concept of storing the process in a table, so that I can simply select (from a form: Process X: Use SP1, params use SP4, Params use SP9, Params Process Y: Use SP1, Params Use SP2, Params Use SP5, Params Process Z: Use... Table driven (to the extent possible). Then a supervisor application could execute a process (set of SPs), read out of a table, passing in only the database name and possibly a directory on disk. Everything else is selected and set up in advance. To see what a process is, just open a table, to modify the process, select different stored procedures. This stuff is VERY standard, with just a few things that change from DB to db (the database name and where files are going) however I do occasionally shufffle around the interior processes (indexes created, fields used etc. That stuff is internal to the SP and if I change that it is in the SP itself. Then that change is applied to every database from then on. I am not saying that this will work but as my understand and knowledge changes I am spending a LOT of time rewriting stuff. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > <<< > I am hoping to eventually make the entire process "table driven" where I can > store (in a table) the > name of the sp, the parameters in a child table etc. > Using table-driven process could be not quite right in C#/.Net world, which > proposes many other options to implement generic solutions - I'd just start > with "hardcoded" solutions using calls to a set of small "generic" > classes/methods executing custom (MS SQL) database processing, when that set > of methods/functions stabilizes I'd try to generalize/"generic-ize" it if > that will be needed at all... > > What is the purpose of your "going generic" for this sets of tasks/customer? > > Isn't it "just for fun" of generic programming? > Would that "going generic" make your programming more effective? > > Developing/debugging/testing generic custom "table-driven"/"other-metadata > media driven" solutions is always (much) more time consuming than > "hardcoded" ones, and supporting "table-driven" generic custom solutions, > especially in the context of constantly changing customer requirements also > promise to be rather time consuming - I mean generic > "table-"/metadata-driven solutions could only(?) pay back well in the case > they are used on many deployment sites by many people. I can be wrong, just > talking from my experience, which is not universal of course... > > I'd better propose to use metadata descriptions/specs (stored in db tables > or (XML) files), if you're so bored to write repetitive code, to generate > "hardcoded" solutions, then make quick fixes to this generated hardcode to > satisfy your customers' current requests, then (in free time) adjust code > generator, then when everything will stabilize (will that ever happen?) > develop truly generic "table-driven" solution. > > You can also consider using F# and Windows Workflow Foundation (WF)... > > The following presentation seems to be useful to get true picture of > "generic vs. specific" solutions trade-offs: > > http://www.infoq.com/presentations/Generic-Specific-Tradeoffs-Stefan-Tilkov > > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Thu Nov 5 11:13:22 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 20:13:22 +0300 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <4AF2FE62.2050501@colbyconsulting.com> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> <4AF2FE62.2050501@colbyconsulting.com> Message-ID: <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> Hi John, If your "Process X", "Process Y",... calls to stored procedures do have constant parameters' values then I do not see (I can be missing them) clear reasons to use table-driven approach - I'd just have hardcoded ProcessX, ProcessY, ... calling C# wrapper classes' methods to execute SPs. And the latter wrapper classes can inherit from some generic base class(es) to keep executing context, collect statistics etc... As I noted previously WF (Windows Workflow Foundation) could be a good candidate for your subject area but WF has a steep learning curve so you will not be able to effectively apply it right now but with small dedicated classes, which you're writing right now (and delivering your custom solutions right now) you'll be able to relatively easy migrate to WF based solutions in the future. And if you will spend time working on generic custom table-driven solution you'll probably have to throw it away later when migrating to WF... Before you get comfortable with WF you can code you ProcessX, ProcessY, ... steps as "meta-process" text or xml files keeping fully qualified class names for your dedicated classes. And your "generic execution engine" will be just a simple code reading "meta-process" description, instantiating custom classes using .NET Reflection, calling their generic context/initialization methods, running SP by calling generic Run() method (parameters setting is inside your custom classes), then calling generic termination methods and iterating to the next step/finishing when "meta-process" file description ends. That could be it. Thank you. --Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 7:34 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] How do you run SQL Server processes Shamil, > Using table-driven process could be not quite right in C#/.Net world, which proposes many other options to implement generic solutions - I'd just start with "hardcoded" solutions using calls to a set of small "generic" That is precisely what I do right now, and what I will do for the near future. The table driven thing is simply that I write "processes" as discussed in this thread. A process might be: Export To Accuzip: "Export dbX TblY to files in DirectoryZ". However that breaks down onto a set of a handful of stored procedures, with passed in parameters. Now, the process "Export To Accuzip" is applied against 6 or 8 different databases every month, but ALSO against random databases which are orders. Orders come in every month and they are a whole nother discussion, but PART of an order is the process Export to Accuzip. DatabaseA exports files to its own directory, DatabaseB to its own directory, OrderAA to its own directory and so forth. But the EXPORT itself is a very standard process. So I am examining the concept of storing the process in a table, so that I can simply select (from a form: Process X: Use SP1, params use SP4, Params use SP9, Params Process Y: Use SP1, Params Use SP2, Params Use SP5, Params Process Z: Use... Table driven (to the extent possible). Then a supervisor application could execute a process (set of SPs), read out of a table, passing in only the database name and possibly a directory on disk. Everything else is selected and set up in advance. To see what a process is, just open a table, to modify the process, select different stored procedures. This stuff is VERY standard, with just a few things that change from DB to db (the database name and where files are going) however I do occasionally shufffle around the interior processes (indexes created, fields used etc. That stuff is internal to the SP and if I change that it is in the SP itself. Then that change is applied to every database from then on. I am not saying that this will work but as my understand and knowledge changes I am spending a LOT of time rewriting stuff. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > <<< > I am hoping to eventually make the entire process "table driven" where I can > store (in a table) the > name of the sp, the parameters in a child table etc. > Using table-driven process could be not quite right in C#/.Net world, which > proposes many other options to implement generic solutions - I'd just start > with "hardcoded" solutions using calls to a set of small "generic" > classes/methods executing custom (MS SQL) database processing, when that set > of methods/functions stabilizes I'd try to generalize/"generic-ize" it if > that will be needed at all... > > What is the purpose of your "going generic" for this sets of tasks/customer? > > Isn't it "just for fun" of generic programming? > Would that "going generic" make your programming more effective? > > Developing/debugging/testing generic custom "table-driven"/"other-metadata > media driven" solutions is always (much) more time consuming than > "hardcoded" ones, and supporting "table-driven" generic custom solutions, > especially in the context of constantly changing customer requirements also > promise to be rather time consuming - I mean generic > "table-"/metadata-driven solutions could only(?) pay back well in the case > they are used on many deployment sites by many people. I can be wrong, just > talking from my experience, which is not universal of course... > > I'd better propose to use metadata descriptions/specs (stored in db tables > or (XML) files), if you're so bored to write repetitive code, to generate > "hardcoded" solutions, then make quick fixes to this generated hardcode to > satisfy your customers' current requests, then (in free time) adjust code > generator, then when everything will stabilize (will that ever happen?) > develop truly generic "table-driven" solution. > > You can also consider using F# and Windows Workflow Foundation (WF)... > > The following presentation seems to be useful to get true picture of > "generic vs. specific" solutions trade-offs: > > http://www.infoq.com/presentations/Generic-Specific-Tradeoffs-Stefan-Tilkov > > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4576 (20091105) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Thu Nov 5 11:32:51 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 05 Nov 2009 12:32:51 -0500 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> <4AF2FE62.2050501@colbyconsulting.com> <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> Message-ID: <4AF30C43.8010508@colbyconsulting.com> Shamil, Listen to this... > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... steps as "meta-process" text or xml files keeping fully qualified class names for your dedicated classes. And your "generic execution engine" will be just a simple code reading "meta-process" description, instantiating custom classes using .NET Reflection, calling their generic context/initialization methods, running SP by calling generic Run() method (parameters setting is inside your custom classes), then calling generic termination methods and iterating to the next step/finishing when "meta-process" file description ends. Now read this. Open form Select stored procedure, select parameter, select parameter. click save. Select stored procedure, select parameter, select parameter. click save. Continue till done. ;) Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? Read your solution again. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > If your "Process X", "Process Y",... calls to stored procedures do have > constant parameters' values then I do not see (I can be missing them) clear > reasons to use table-driven approach - I'd just have hardcoded ProcessX, > ProcessY, ... calling C# wrapper classes' methods to execute SPs. And the > latter wrapper classes can inherit from some generic base class(es) to keep > executing context, collect statistics etc... > > As I noted previously WF (Windows Workflow Foundation) could be a good > candidate for your subject area but WF has a steep learning curve so you > will not be able to effectively apply it right now but with small dedicated > classes, which you're writing right now (and delivering your custom > solutions right now) you'll be able to relatively easy migrate to WF based > solutions in the future. And if you will spend time working on generic > custom table-driven solution you'll probably have to throw it away later > when migrating to WF... > > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... > steps as "meta-process" text or xml files keeping fully qualified class > names for your dedicated classes. And your "generic execution engine" will > be just a simple code reading "meta-process" description, instantiating > custom classes using .NET Reflection, calling their generic > context/initialization methods, running SP by calling generic Run() method > (parameters setting is inside your custom classes), then calling generic > termination methods and iterating to the next step/finishing when > "meta-process" file description ends. That could be it. > > Thank you. > > --Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 7:34 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] How do you run SQL Server processes > > Shamil, From shamil at smsconsulting.spb.ru Thu Nov 5 13:00:15 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 22:00:15 +0300 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <4AF30C43.8010508@colbyconsulting.com> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> <4AF2FE62.2050501@colbyconsulting.com> <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> <4AF30C43.8010508@colbyconsulting.com> Message-ID: <010c01ca5e4a$3735e430$a5a1ac90$@spb.ru> Hi John, <<< Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? >>> Yes, that is also what I mean: developing generic table-driven procedure for you currently could force you to learn more C#/.NET development stuff than when implementing "meta-process description" interpreter, which will consist of several C# code lines. But even the latter is not needed probably as you can hardcode your solutions for every custom task "shuffling" your parameterized wrappers classes/methods calling SPs, and instead of "Select stored procedure, select parameter,..." use "Create C# project, type C# code with calls to the wrapper classes/methods with Stored Procedures names and parameter names, types and values, then compile and run the custom solution". And it should be more suitable to debug/trace latter solution than "table-driven" or "meta-process description driven" ones... OK? (Am I still missing something here?) -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 8:33 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] How do you run SQL Server processes Shamil, Listen to this... > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... steps as "meta-process" text or xml files keeping fully qualified class names for your dedicated classes. And your "generic execution engine" will be just a simple code reading "meta-process" description, instantiating custom classes using .NET Reflection, calling their generic context/initialization methods, running SP by calling generic Run() method (parameters setting is inside your custom classes), then calling generic termination methods and iterating to the next step/finishing when "meta-process" file description ends. Now read this. Open form Select stored procedure, select parameter, select parameter. click save. Select stored procedure, select parameter, select parameter. click save. Continue till done. ;) Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? Read your solution again. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > If your "Process X", "Process Y",... calls to stored procedures do have > constant parameters' values then I do not see (I can be missing them) clear > reasons to use table-driven approach - I'd just have hardcoded ProcessX, > ProcessY, ... calling C# wrapper classes' methods to execute SPs. And the > latter wrapper classes can inherit from some generic base class(es) to keep > executing context, collect statistics etc... > > As I noted previously WF (Windows Workflow Foundation) could be a good > candidate for your subject area but WF has a steep learning curve so you > will not be able to effectively apply it right now but with small dedicated > classes, which you're writing right now (and delivering your custom > solutions right now) you'll be able to relatively easy migrate to WF based > solutions in the future. And if you will spend time working on generic > custom table-driven solution you'll probably have to throw it away later > when migrating to WF... > > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... > steps as "meta-process" text or xml files keeping fully qualified class > names for your dedicated classes. And your "generic execution engine" will > be just a simple code reading "meta-process" description, instantiating > custom classes using .NET Reflection, calling their generic > context/initialization methods, running SP by calling generic Run() method > (parameters setting is inside your custom classes), then calling generic > termination methods and iterating to the next step/finishing when > "meta-process" file description ends. That could be it. > > Thank you. > > --Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 7:34 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] How do you run SQL Server processes > > Shamil, _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4576 (20091105) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4576 (20091105) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From shamil at smsconsulting.spb.ru Sat Nov 7 05:28:32 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 7 Nov 2009 14:28:32 +0300 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <010c01ca5e4a$3735e430$a5a1ac90$@spb.ru> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> <4AF2FE62.2050501@colbyconsulting.com> <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> <4AF30C43.8010508@colbyconsulting.com> <010c01ca5e4a$3735e430$a5a1ac90$@spb.ru> Message-ID: <018c01ca5f9d$71389c00$53a9d400$@spb.ru> Hi John, Here is a .NET Reflection sample: using System; namespace NetReflectionSample { class Program { static void Main(string[] args) { string[] spRunnersNames = { "NetReflectionSample.SP1", "NetReflectionSample.SP2" }; foreach (string spRunnerName in spRunnersNames) { Type spRunnerType = Type.GetType(spRunnerName); StoredProcedureRunner runner = Activator.CreateInstance(spRunnerType) as StoredProcedureRunner; runner.Init(); runner.Run(); runner.CollectStats(); Console.WriteLine(); } } } public abstract class StoredProcedureRunner { public void Init() { Console.WriteLine("Initializing {0}...", this.GetType().ToString()); } public abstract void Run(); public void CollectStats() { Console.WriteLine("Collecting stats {0}...", this.GetType().ToString()); } } public class SP1 : StoredProcedureRunner { public override void Run() { Console.WriteLine("Running {0}...", this.GetType().ToString()); } } public class SP2 : StoredProcedureRunner { public override void Run() { Console.WriteLine("Running {0}...", this.GetType().ToString()); } } } Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: Thursday, November 05, 2009 10:00 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] How do you run SQL Server processes Hi John, <<< Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? >>> Yes, that is also what I mean: developing generic table-driven procedure for you currently could force you to learn more C#/.NET development stuff than when implementing "meta-process description" interpreter, which will consist of several C# code lines. But even the latter is not needed probably as you can hardcode your solutions for every custom task "shuffling" your parameterized wrappers classes/methods calling SPs, and instead of "Select stored procedure, select parameter,..." use "Create C# project, type C# code with calls to the wrapper classes/methods with Stored Procedures names and parameter names, types and values, then compile and run the custom solution". And it should be more suitable to debug/trace latter solution than "table-driven" or "meta-process description driven" ones... OK? (Am I still missing something here?) -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 8:33 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] How do you run SQL Server processes Shamil, Listen to this... > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... steps as "meta-process" text or xml files keeping fully qualified class names for your dedicated classes. And your "generic execution engine" will be just a simple code reading "meta-process" description, instantiating custom classes using .NET Reflection, calling their generic context/initialization methods, running SP by calling generic Run() method (parameters setting is inside your custom classes), then calling generic termination methods and iterating to the next step/finishing when "meta-process" file description ends. Now read this. Open form Select stored procedure, select parameter, select parameter. click save. Select stored procedure, select parameter, select parameter. click save. Continue till done. ;) Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? Read your solution again. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > If your "Process X", "Process Y",... calls to stored procedures do have > constant parameters' values then I do not see (I can be missing them) clear > reasons to use table-driven approach - I'd just have hardcoded ProcessX, > ProcessY, ... calling C# wrapper classes' methods to execute SPs. And the > latter wrapper classes can inherit from some generic base class(es) to keep > executing context, collect statistics etc... > > As I noted previously WF (Windows Workflow Foundation) could be a good > candidate for your subject area but WF has a steep learning curve so you > will not be able to effectively apply it right now but with small dedicated > classes, which you're writing right now (and delivering your custom > solutions right now) you'll be able to relatively easy migrate to WF based > solutions in the future. And if you will spend time working on generic > custom table-driven solution you'll probably have to throw it away later > when migrating to WF... > > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... > steps as "meta-process" text or xml files keeping fully qualified class > names for your dedicated classes. And your "generic execution engine" will > be just a simple code reading "meta-process" description, instantiating > custom classes using .NET Reflection, calling their generic > context/initialization methods, running SP by calling generic Run() method > (parameters setting is inside your custom classes), then calling generic > termination methods and iterating to the next step/finishing when > "meta-process" file description ends. That could be it. > > Thank you. > > --Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 7:34 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] How do you run SQL Server processes > > Shamil, From shamil at smsconsulting.spb.ru Sat Nov 7 08:19:11 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 7 Nov 2009 17:19:11 +0300 Subject: [dba-VB] NetReflectionSample - Part I of IV Message-ID: <000601ca5fb5$49b79dd0$dd26d970$@spb.ru> Hi John and all, Just wanted to post a bit more advanced sample on .NET reflection and how it can be used to instantiate custom classes and to execute their parameterized methods based on text definition: using System; using System.Reflection; #region Test Program namespace NetReflectionSample { /// /// Test class /// class Program { static void Main(string[] args) { // List of custom classes (with parameters) to instantiate and // call their Run(...) method string[] spRunnersCalls = { "NetReflectionSample.SP1", "NetReflectionSample.SP2|503|Test String|3.14|11/7/2009", "NetReflectionSample.SP3|123|Test String2" }; foreach (string spRunnerCall in spRunnersCalls) { // Parse current SP call string definition and // return StoredProcedureRunner instance StoredProcedureRunner runner = DynamicMethodCallParser.Parse(spRunnerCall); runner.Init(); // If there are calling parameters use CustomRunMethodInfo to // execute SP call wrapper method, else use default Run() method CustomRunMethodInfo customRunner = runner as CustomRunMethodInfo; if (customRunner.Parameters != null) { customRunner.ParameterizedRunMethodInfo.Invoke( customRunner, customRunner.Parameters); } else runner.Run(); // collect execution stats runner.CollectStats(); Console.WriteLine(); } } } } #endregion Thank you. --Shamil From shamil at smsconsulting.spb.ru Sat Nov 7 08:20:44 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 7 Nov 2009 17:20:44 +0300 Subject: [dba-VB] NetReflectionSample - Part II of IV Message-ID: <000b01ca5fb5$7f9438a0$7ebca9e0$@spb.ru> #region Custom business functionality namespace NetReflectionSample { /// /// SP1 wrapper class simulator /// public class SP1 : StoredProcedureRunner { public override void Run() { Console.WriteLine("Running {0}.Run...", this.GetType().ToString()); } } /// /// SP2 wrapper class simulator /// public class SP2 : StoredProcedureRunner { public void Run( int prm1, string prm2, decimal prm3, DateTime prm4) { Console.WriteLine("Running {0}.Run...", this.GetType().ToString()); Console.WriteLine("Parameters: prm1={0}, prm2={1}, prm3={2}, prm4={3}", prm1, prm2, prm3, prm4); } } /// /// SP3 wrapper class simulator /// public class SP3 : StoredProcedureRunner { public void Run( int prm1, string prm2) { Console.WriteLine("Running {0}.Run...", this.GetType().ToString()); Console.WriteLine("Parameters: prm1={0}, prm2={1}", prm1, prm2); } } } #endregion --Shamil From shamil at smsconsulting.spb.ru Sat Nov 7 08:37:01 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 7 Nov 2009 17:37:01 +0300 Subject: [dba-VB] NetReflectionSample - full code text Message-ID: <001a01ca5fb7$c5cab310$51601930$@spb.ru> Hi All, Here is a full text of the subject sample: http://smsconsulting.spb.ru/samples/NetReflectionSample.txt dba-VB robot currently blocks to post two parts of my four parts posting because they are longer than 20KB - dba-VB robot writes that those two postings are awaiting moderators approval but AFAIHF this approval never happens (at least that was for several my past postings with code) - therefore I'm posting the above link to get the whole code text available. Here are the results of the test program execution: === Initializing NetReflectionSample.SP1... Running NetReflectionSample.SP1.Run... Collecting stats NetReflectionSample.SP1... Initializing NetReflectionSample.SP2... Running NetReflectionSample.SP2.Run... Parameters: prm1=503, prm2=Test String, prm3=3.14, prm4=11/7/2009 12:00:00 AM Collecting stats NetReflectionSample.SP2... Initializing NetReflectionSample.SP3... Running NetReflectionSample.SP3.Run... Parameters: prm1=123, prm2=Test String2 Collecting stats NetReflectionSample.SP3... === Note: posted code is a "quick & dirty" first version, if you have time and desire to make a better version of this sample code you're welcome to do that. Any code review remarks and comments are very welcome. Thank you. -- Shamil From Gustav at cactus.dk Sat Nov 7 14:00:22 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Sat, 07 Nov 2009 21:00:22 +0100 Subject: [dba-VB] NetReflectionSample - full code text Message-ID: Hi Shamil Thanks! Good example. /gustav >>> shamil at smsconsulting.spb.ru 07-11-2009 15:37 >>> Any code review remarks and comments are very welcome. From shamil at smsconsulting.spb.ru Sun Nov 8 15:47:44 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 9 Nov 2009 00:47:44 +0300 Subject: [dba-VB] NetReflectionSample - full code text In-Reply-To: References: Message-ID: <000301ca60bd$1c0d9ca0$5428d5e0$@spb.ru> Thank you, Gustav. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Saturday, November 07, 2009 11:00 PM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] NetReflectionSample - full code text Hi Shamil Thanks! Good example. /gustav >>> shamil at smsconsulting.spb.ru 07-11-2009 15:37 >>> Any code review remarks and comments are very welcome. _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4585 (20091108) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sun Nov 8 20:20:20 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 08 Nov 2009 21:20:20 -0500 Subject: [dba-VB] NetReflectionSample - full code text In-Reply-To: References: Message-ID: <4AF77C64.5090707@colbyconsulting.com> But what does it DO? John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi Shamil > > Thanks! Good example. > > /gustav > >>>> shamil at smsconsulting.spb.ru 07-11-2009 15:37 >>> > > Any code review remarks and comments are very welcome. > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Mon Nov 9 00:11:28 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 9 Nov 2009 09:11:28 +0300 Subject: [dba-VB] NetReflectionSample - full code text In-Reply-To: <4AF77C64.5090707@colbyconsulting.com> References: <4AF77C64.5090707@colbyconsulting.com> Message-ID: <000901ca6103$7adfbd80$709f3880$@spb.ru> Hi John, Basically, this sample code demonstrates how to interpret a simple scripting "birds language" into a sequence of calls to the parameterized methods of custom classes. Substitute text script with meta-table(s) - and you'll get table-driven interpreter. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, November 09, 2009 5:20 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] NetReflectionSample - full code text But what does it DO? John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi Shamil > > Thanks! Good example. > > /gustav > >>>> shamil at smsconsulting.spb.ru 07-11-2009 15:37 >>> > > Any code review remarks and comments are very welcome. > > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4586 (20091108) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From shamil at smsconsulting.spb.ru Tue Nov 10 23:49:05 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Wed, 11 Nov 2009 08:49:05 +0300 Subject: [dba-VB] FYI: ORM, EDM, ESQL, Entity Framework, LINQ to SQL, LINQ to Entities - Confused? Message-ID: <003601ca6292$af1581d0$0d408570$@spb.ru> "ORM, EDM, ESQL, Entity Framework, LINQ to SQL, LINQ to Entities - Confused?" http://www.infoq.com/presentations/ORM-LINQ-Entity-Framework-Eric-Nelson --Shamil From Gustav at cactus.dk Wed Nov 11 12:52:48 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 11 Nov 2009 19:52:48 +0100 Subject: [dba-VB] Microsoft WebsiteSpark Program Message-ID: Hi all Have you heard of Microsoft? WebsiteSpark*? http://www.microsoft.com/web/websitespark/Support.aspx?tab=ProgramDetails It is designed to ignite the success of professional Web developers and designers. The program enables you to get software, support and business opportunities from Microsoft at no upfront costs for 3 years or exit from the program. WebSiteSpark provides access to a technology offering for development and production hosting of websites. Once enrolled, you can download the following software from Microsoft: For design, development, testing and demonstration of new websites * for a total of up to three users per Web design and development company: *Visual Studio Professional *Expression Studio 3 (1 user) and Expression Web 3 (up to 2 users) *Windows Web Server 2008 R2 *SQL Server 2008 Web Edition For production use * that is, to deploy and host new websites developed using Program software * using a total of up to four processors per Web design and development company, of the following (physical or virtual) dedicated servers: *Windows Web Server 2008 R2 *SQL Server 2008 Web Edition So if you consider "something else" than Access, this might help you. /gustav From jwcolby at colbyconsulting.com Wed Nov 11 13:08:24 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 11 Nov 2009 14:08:24 -0500 Subject: [dba-VB] SPAM-LOW: Microsoft WebsiteSpark Program In-Reply-To: References: Message-ID: <4AFB0BA8.8020105@colbyconsulting.com> This appears to be one part of this: http://www.microsoftstartupzone.com/pages/home.aspx which I posted about a month or so ago. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi all > > Have you heard of Microsoft? WebsiteSpark*? > > http://www.microsoft.com/web/websitespark/Support.aspx?tab=ProgramDetails > > > > It is designed to ignite the success of professional Web developers and > designers. The program enables you to get software, support and business > opportunities from Microsoft at no upfront costs for 3 years or exit > from the program. > > WebSiteSpark provides access to a technology offering for development > and production hosting of websites. Once enrolled, you can download the > following software from Microsoft: > > For design, development, testing and demonstration of new websites * > for a total of up to three users per Web design and development company: > > > *Visual Studio Professional > *Expression Studio 3 (1 user) and Expression Web 3 (up to 2 users) > *Windows Web Server 2008 R2 > *SQL Server 2008 Web Edition > > For production use * that is, to deploy and host new websites > developed using Program software * using a total of up to four > processors per Web design and development company, of the following > (physical or virtual) dedicated servers: > > *Windows Web Server 2008 R2 > *SQL Server 2008 Web Edition > > > > So if you consider "something else" than Access, this might help you. > > /gustav > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Thu Nov 12 11:22:55 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 12 Nov 2009 12:22:55 -0500 Subject: [dba-VB] What to do, what to do? Message-ID: <4AFC446F.6070207@colbyconsulting.com> I am writing an application in C# to merge records (previously discussed). It looks pretty simple, all things considered. Get data into a dataset object which will contain about 4 tables. Then use the records and fields collections of the recordset to iterate through the fields building up a record (or modifying the first record of a small set of records - the dupes). Copy a record from one table into another table inside of the dataset. Delete records in a table in the dataset. Etc. Etc all without going back out to the actual data tables on SQL Server until the very end when I will do an update back to the live data. Am I correct in assuming here that I can programmatically append records from one table in the dataset to another record in the dataset? Delete records in the tables in the dataset? Update records in tables in the dataset? All "in memory" without connecting back to the live data in the database? Is there anything I need to know before I launch into this? -- John W. Colby www.ColbyConsulting.com From cfoust at infostatsystems.com Thu Nov 12 11:29:43 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Thu, 12 Nov 2009 09:29:43 -0800 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFC446F.6070207@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> Message-ID: I think that depends on the objects you're using. .Net tends to complain that a record already belongs to another table when you try to import it. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 12, 2009 9:23 AM To: VBA Subject: [dba-VB] What to do, what to do? I am writing an application in C# to merge records (previously discussed). It looks pretty simple, all things considered. Get data into a dataset object which will contain about 4 tables. Then use the records and fields collections of the recordset to iterate through the fields building up a record (or modifying the first record of a small set of records - the dupes). Copy a record from one table into another table inside of the dataset. Delete records in a table in the dataset. Etc. Etc all without going back out to the actual data tables on SQL Server until the very end when I will do an update back to the live data. Am I correct in assuming here that I can programmatically append records from one table in the dataset to another record in the dataset? Delete records in the tables in the dataset? Update records in tables in the dataset? All "in memory" without connecting back to the live data in the database? Is there anything I need to know before I launch into this? -- John W. Colby www.ColbyConsulting.com _______________________________________________ From shamil at smsconsulting.spb.ru Fri Nov 13 00:04:33 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 09:04:33 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFC446F.6070207@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> Message-ID: <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> Hi John -- Here is a sample: using System; using System.Collections.Generic; using System.Text; using System.Data; using System.Data.SqlClient; namespace TestConsole.Samples { public class DataSetTests { public static void Run() { string connectionString = "Data Source=HAMBURG\\SQL2005;"+ "Initial Catalog=Northwind;User Id=sa;Password=N/A"; using (SqlConnection connection = new SqlConnection(connectionString)) { connection.Open(); string sql = "select * from [Shippers]"; string sql1 = "select * from [NewShippers] where(1=0)"; SqlCommand command = new SqlCommand(sql, connection); SqlDataAdapter adapter = new SqlDataAdapter(command); DataSet dataSet = new DataSet("Shippers"); adapter.Fill(dataSet); command = new SqlCommand(sql1, connection); adapter = new SqlDataAdapter(command); DataSet newDataSet = new DataSet("NewShippers"); adapter.Fill(newDataSet); Console.WriteLine("*** newDataSet ***"); foreach (DataRow dataRow in newDataSet.Tables[0].Rows) { Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); } Console.WriteLine("*** dataSet ***"); foreach (DataRow dataRow in dataSet.Tables[0].Rows) { // This row already belongs to another table. //newDataSet.Tables[0].Rows.Add(dataRow); object[] items = { dataRow[0], dataRow[1], dataRow[2] }; newDataSet.Tables[0].Rows.Add(items); Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); } Console.WriteLine("*** newDataSet ***"); foreach (DataRow dataRow in newDataSet.Tables[0].Rows) { Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); } newDataSet.Tables[0].TableName = "NewShippers"; SqlCommandBuilder builder = new SqlCommandBuilder(adapter); adapter.InsertCommand = builder.GetInsertCommand(true); adapter.Update(newDataSet.Tables[0]); } } } } Thank you. --Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 12, 2009 8:23 PM To: VBA Subject: [dba-VB] What to do, what to do? I am writing an application in C# to merge records (previously discussed). It looks pretty simple, all things considered. Get data into a dataset object which will contain about 4 tables. Then use the records and fields collections of the recordset to iterate through the fields building up a record (or modifying the first record of a small set of records - the dupes). Copy a record from one table into another table inside of the dataset. Delete records in a table in the dataset. Etc. Etc all without going back out to the actual data tables on SQL Server until the very end when I will do an update back to the live data. Am I correct in assuming here that I can programmatically append records from one table in the dataset to another record in the dataset? Delete records in the tables in the dataset? Update records in tables in the dataset? All "in memory" without connecting back to the live data in the database? Is there anything I need to know before I launch into this? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4600 (20091112) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From marklbreen at gmail.com Fri Nov 13 00:58:16 2009 From: marklbreen at gmail.com (Mark Breen) Date: Fri, 13 Nov 2009 14:58:16 +0800 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFC446F.6070207@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> Message-ID: Hello John, Just curious, how many records are you talking about - a few hundred or millions? Second question, other than for purely academic reasons, why are you bring the data into C#, to concatenate and insert into another table can so easily be done in one sproc, even if you need a temporary holding place, you would still use a temp table or even a local table. Of course if it is to learn datasets then thats another story, Mark 2009/11/13 jwcolby > I am writing an application in C# to merge records (previously discussed). > It looks pretty simple, > all things considered. Get data into a dataset object which will contain > about 4 tables. Then use > the records and fields collections of the recordset to iterate through the > fields building up a > record (or modifying the first record of a small set of records - the > dupes). Copy a record from > one table into another table inside of the dataset. Delete records in a > table in the dataset. Etc. > Etc all without going back out to the actual data tables on SQL Server > until the very end when I > will do an update back to the live data. > > Am I correct in assuming here that I can programmatically append records > from one table in the > dataset to another record in the dataset? Delete records in the tables in > the dataset? Update > records in tables in the dataset? All "in memory" without connecting back > to the live data in the > database? > > Is there anything I need to know before I launch into this? > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Fri Nov 13 03:27:08 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Fri, 13 Nov 2009 10:27:08 +0100 Subject: [dba-VB] What to do, what to do? Message-ID: Hi John Shamil knows the SqlClient stuff! As he knows, I prefer the datatable route - a matter of taste I think, but I admit working with SqlCommand may allow for some fine tuning not possible with the datatableadapters. Here's an example from an app that updates (syncs) one table from another. Not exactly what you wish to do but it should contain the building blocks. The tricky part here is that the unique id (autonumber) should not be compared. However, the id of a source row is stored in the target row as SyncId for later reference: private void ConvertOrder() { // DataTableAdapter to copy from. VrsCloneTableAdapters.OrderTableAdapter sourceDataTableAdapter = new VrsCloneTableAdapters.OrderTableAdapter(); VrsClone.OrderDataTable sourceDataTable; // DataTableAdapter to copy to. KarneliaTableAdapters.SyncOrderTableAdapter targetDataTableAdapter = new KarneliaTableAdapters.SyncOrderTableAdapter(); Karnelia.SyncOrderDataTable targetDataTable; sourceDataTableAdapter.Connection = ConnectionApplyPassword(sourceDataTableAdapter.Connection.ConnectionString); targetDataTableAdapter.Connection = ConnectionApplyPassword(targetDataTableAdapter.Connection.ConnectionString); // DataTable for source. sourceDataTable = sourceDataTableAdapter.GetDataSync(); // DataTable for target. targetDataTable = targetDataTableAdapter.GetData(); // Read each row from the source rows. // Update those found in the target table. // Append those not found in the target table. // Variable to hold primary key to locate. int key; // Row of target DataTable to be updated. Karnelia.SyncOrderRow targetRow = null; // Variables to hold hash codes of row contents. int sourceHash; int targetHash; int colCount = sourceDataTable.Columns.Count; int colItem; int colOrdinal; string colName; for (colItem = 0; colItem < colCount; colItem++) { colName = sourceDataTable.Columns[colItem].ColumnName; colOrdinal = sourceDataTable.Columns[colItem].Ordinal; targetDataTable.Columns[colName].SetOrdinal(colOrdinal + 1); } foreach (VrsClone.OrderRow sourceRow in sourceDataTable) { // Value of key to locate. key = sourceRow.SyncId; // Find a matching row in the target. targetRow = targetDataTable.FindBySyncId(key); if (targetRow == null) { // Row is not found. targetRow = targetDataTable.NewSyncOrderRow(); targetRow.BeginEdit(); targetRow.VrsOrderId = sourceRow.VrsOrderId; targetRow.ClientId = sourceRow.ClientId; targetRow.StylistId = sourceRow.StylistId; targetRow.DateCreated = sourceRow.DateCreated; targetRow.SyncId = sourceRow.SyncId; targetRow.EndEdit(); targetDataTable.AddSyncOrderRow(targetRow); Console.WriteLine("Insert " + targetDataTable.ToString() + ": " + sourceRow.VrsOrderId.ToString() + " - " + targetRow.RowState.ToString()); } else { // Row is found. // Calculate hash code to see if source row and target row match. // Id of targetrow (first column) is not present in sourcerow, thus // concatenate first column of targetrow with the columns of sourcerow. sourceHash = String.Concat(targetRow.ItemArray[0], String.Concat(sourceRow.ItemArray)).GetHashCode(); targetHash = String.Concat(targetRow.ItemArray).GetHashCode(); if (sourceHash != targetHash) { // The source row is updated. // Copy full content from source row to target row. // This sets RowState of the target row to Modified. targetRow.BeginEdit(); targetRow.VrsOrderId = sourceRow.VrsOrderId; targetRow.ClientId = sourceRow.ClientId; targetRow.StylistId = sourceRow.StylistId; targetRow.DateCreated = sourceRow.DateCreated; targetRow.EndEdit(); // Do NOT call AcceptChanges here as this would clear property RowState. Console.WriteLine("Update " + targetDataTable.ToString() + ": " + sourceRow.VrsOrderId.ToString() + " - " + targetRow.RowState.ToString()); } } } // Write back to the database table the updated target DataTable. targetDataTableAdapter.Update(targetDataTable); // Call AcceptChanges if needed for further processing. //targetDataTable.AcceptChanges(); } This database contains records by the thousands. How code like this will perform for millions of records has to be tested. /gustav >>> jwcolby at colbyconsulting.com 12-11-2009 18:22 >>> I am writing an application in C# to merge records (previously discussed). It looks pretty simple, all things considered. Get data into a dataset object which will contain about 4 tables. Then use the records and fields collections of the recordset to iterate through the fields building up a record (or modifying the first record of a small set of records - the dupes). Copy a record from one table into another table inside of the dataset. Delete records in a table in the dataset. Etc. Etc all without going back out to the actual data tables on SQL Server until the very end when I will do an update back to the live data. Am I correct in assuming here that I can programmatically append records from one table in the dataset to another record in the dataset? Delete records in the tables in the dataset? Update records in tables in the dataset? All "in memory" without connecting back to the live data in the database? Is there anything I need to know before I launch into this? -- John W. Colby www.ColbyConsulting.com From Gustav at cactus.dk Fri Nov 13 04:48:24 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Fri, 13 Nov 2009 11:48:24 +0100 Subject: [dba-VB] AWS (Amazon Web Services) SDK for .NET and VS Message-ID: Hi all This has just been announced for Visual Studio 2008: The AWS SDK for .NET makes it even easier for Windows developers to build .NET applications that tap into the cost-effective, scalable, and reliable AWS cloud. Using the SDK, developers will be able to build solutions for AWS infrastructure services, including Amazon Simple Storage Service (Amazon S3), Amazon Elastic Compute Cloud (Amazon EC2), and Amazon SimpleDB. With the AWS SDK for .NET, developers get started in minutes with a single, downloadable package complete with Visual Studio project templates, the AWS .NET library, C# code samples, and documentation. http://aws.amazon.com/sdkfornet/ /gustav From jwcolby at colbyconsulting.com Fri Nov 13 06:51:36 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 07:51:36 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> Message-ID: <4AFD5658.7000703@colbyconsulting.com> Shamil, That is precisely what I have in mind, with variations. Then at the end, write all updates back to SQL Server. Thanks for the demo code, always helpful. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Here is a sample: > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class DataSetTests > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;"+ > "Initial Catalog=Northwind;User Id=sa;Password=N/A"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { > connection.Open(); > string sql = "select * from [Shippers]"; > string sql1 = "select * from [NewShippers] where(1=0)"; > SqlCommand command = new SqlCommand(sql, connection); > SqlDataAdapter adapter = new SqlDataAdapter(command); > > DataSet dataSet = new DataSet("Shippers"); > adapter.Fill(dataSet); > > command = new SqlCommand(sql1, connection); > adapter = new SqlDataAdapter(command); > DataSet newDataSet = new DataSet("NewShippers"); > adapter.Fill(newDataSet); > > Console.WriteLine("*** newDataSet ***"); > foreach (DataRow dataRow in newDataSet.Tables[0].Rows) > { > Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); > } > > Console.WriteLine("*** dataSet ***"); > foreach (DataRow dataRow in dataSet.Tables[0].Rows) > { > // This row already belongs to another table. > //newDataSet.Tables[0].Rows.Add(dataRow); > > object[] items = { dataRow[0], dataRow[1], dataRow[2] }; > newDataSet.Tables[0].Rows.Add(items); > > Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); > } > > Console.WriteLine("*** newDataSet ***"); > foreach (DataRow dataRow in newDataSet.Tables[0].Rows) > { > Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); > } > > newDataSet.Tables[0].TableName = "NewShippers"; > SqlCommandBuilder builder = new SqlCommandBuilder(adapter); > adapter.InsertCommand = builder.GetInsertCommand(true); > adapter.Update(newDataSet.Tables[0]); > } > } > > } > } > > > Thank you. > > --Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 12, 2009 8:23 PM > To: VBA > Subject: [dba-VB] What to do, what to do? > > I am writing an application in C# to merge records (previously discussed). > It looks pretty simple, > all things considered. Get data into a dataset object which will contain > about 4 tables. Then use > the records and fields collections of the recordset to iterate through the > fields building up a > record (or modifying the first record of a small set of records - the > dupes). Copy a record from > one table into another table inside of the dataset. Delete records in a > table in the dataset. Etc. > Etc all without going back out to the actual data tables on SQL Server until > the very end when I > will do an update back to the live data. > > Am I correct in assuming here that I can programmatically append records > from one table in the > dataset to another record in the dataset? Delete records in the tables in > the dataset? Update > records in tables in the dataset? All "in memory" without connecting back > to the live data in the > database? > > Is there anything I need to know before I launch into this? > From shamil at smsconsulting.spb.ru Fri Nov 13 08:19:39 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 17:19:39 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD5658.7000703@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> Message-ID: <004801ca646c$5758e690$060ab3b0$@spb.ru> Hi John -- You can also consider using SqlBulkLoad - below is just a simple sample, and you can develop very flexible variations of this approach. SqlBulkLoad should be as quick as bcp maybe even quicker. In fact the source for SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this IDataReader to have all the data to bulkload then your programs will become as streamlined as they probably have never been, FYI: IDataReader can be built in code with data sources being everything as e.g. .csv files etc. (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) using System; using System.Collections.Generic; using System.Text; using System.Data; using System.Data.SqlClient; namespace TestConsole.Samples { public class SqlBulkLoadTest { public static void Run() { string connectionString = "Data Source=HAMBURG\\SQL2005;" + "Initial Catalog=Northwind;User Id=sa;Password=shms"; using (SqlConnection connection = new SqlConnection(connectionString)) { connection.Open(); string sql = "select * from [Shippers]"; SqlCommand command = new SqlCommand(sql, connection); SqlDataAdapter adapter = new SqlDataAdapter(command); DataSet dataSet = new DataSet("Shippers"); adapter.Fill(dataSet); Console.WriteLine("*** dataSet ***"); foreach (DataRow dataRow in dataSet.Tables[0].Rows) { object[] items = { dataRow[0], dataRow[1], dataRow[2] }; Console.WriteLine("{0} {1} {2}", dataRow[0], dataRow[1], dataRow[2]); } (new SqlCommand("delete from [NewShippers]",connection)) .ExecuteNonQuery(); SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, SqlBulkCopyOptions.KeepIdentity); bulkCopy.DestinationTableName = "NewShippers"; bulkCopy.WriteToServer(dataSet.Tables[0]); } } } } -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 13, 2009 3:52 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? Shamil, That is precisely what I have in mind, with variations. Then at the end, write all updates back to SQL Server. Thanks for the demo code, always helpful. John W. Colby www.ColbyConsulting.com <<< snip >> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4603 (20091113) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 13 08:30:34 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 09:30:34 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <004801ca646c$5758e690$060ab3b0$@spb.ru> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> <004801ca646c$5758e690$060ab3b0$@spb.ru> Message-ID: <4AFD6D8A.3060306@colbyconsulting.com> In fact this is a one shot deal. I need to do this one time, to one table. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > You can also consider using SqlBulkLoad - below is just a simple sample, and > you can develop very flexible variations of this approach. SqlBulkLoad > should be as quick as bcp maybe even quicker. In fact the source for > SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this > IDataReader to have all the data to bulkload then your programs will become > as streamlined as they probably have never been, FYI: IDataReader can be > built in code with data sources being everything as e.g. .csv files etc. > (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) > > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class SqlBulkLoadTest > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;" + > "Initial Catalog=Northwind;User Id=sa;Password=shms"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { > connection.Open(); > string sql = "select * from [Shippers]"; > SqlCommand command = new SqlCommand(sql, connection); > SqlDataAdapter adapter = new SqlDataAdapter(command); > > DataSet dataSet = new DataSet("Shippers"); > adapter.Fill(dataSet); > > Console.WriteLine("*** dataSet ***"); > foreach (DataRow dataRow in dataSet.Tables[0].Rows) > { > object[] items = { dataRow[0], dataRow[1], dataRow[2] }; > > Console.WriteLine("{0} {1} {2}", > dataRow[0], dataRow[1], dataRow[2]); > } > > > (new SqlCommand("delete from [NewShippers]",connection)) > .ExecuteNonQuery(); > > SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, > SqlBulkCopyOptions.KeepIdentity); > bulkCopy.DestinationTableName = "NewShippers"; > bulkCopy.WriteToServer(dataSet.Tables[0]); > } > } > > } > } > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 13, 2009 3:52 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > Shamil, > > That is precisely what I have in mind, with variations. Then at the end, > write all updates back to > SQL Server. > > Thanks for the demo code, always helpful. > > John W. Colby > www.ColbyConsulting.com > > > <<< snip >> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4603 (20091113) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 13 08:30:42 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 09:30:42 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: References: <4AFC446F.6070207@colbyconsulting.com> Message-ID: <4AFD6D92.50807@colbyconsulting.com> Mark, > Just curious, how many records are you talking about - a few hundred or millions? Millions, however I am talking about updating sets of records, from 2 to < 30 rows. Imagine a table where Mark Breen has 22 records in the table, John Colby has 17 records in the table, Shamil has 3 records etc. Each of Mark's records have data in a "random" handful of 525 fields. The objective is to get all of Mark's data into one record. Once Mark's records are processed, move on to John's records. Then process William's records. There are 6,514,623 records to be processed. These 6.5 million records represent perhaps about 3 million actual people. I am creating tables right now to store the hash field and a count of records that hash field represents. As an example I have 9 actual people with more than 10 records to be merged, 4,725 people with 5 records, 270,0839 people with 3 records to merge, and 2,771,885 people with two records to merge. The nice part is that the table I am discussing is not constantly changing, in fact this data is entirely static. And there is no time frame for getting it done. I can start the process running and if it takes all night or all week I don't care. I suspect that with efficient programming I can do this at hundreds of records per second. I intend to do it out in tables dedicated to this process in a database dedicated to this process. All of the updates between fields in different records will be taking place entirely in memory. I will probably pull in all of the records for a thousand people, do the merges, then update those records back to SQL Server and pull another chunk of a thousand records. > Second question, other than for purely academic reasons, why are you bring the data into C# Mostly because I know programming, and TSQL is far enough from my expertise as to be unusable in the immediate future. To be honest I am not sure it is even possible to do this easily in TSQL. That said I am sure some TSQL GURU will pop up with the answer. But I am not a TSQL Guru (nor a Guru of any type for that matter). But the thought of programming this kind of algorithm in TSQL sends shivers down my spine. I need to do the following: R1F1 F1F2 R1F3... R1F525 R2F1 R2F2 R2F3... R2F525 R3F1 R3F2 R3F3... R3F525 The lines above represent Rows R1, R2 and R3, Fields F1,F2,F3...F525 Start with R1 and R2 Start at column 1 If R1F1 is null AND R2F1 is NOT NULL THEN Search through sequential records, same field until data found endif Move to column 2, continue to all columns Move to R1 and R3 In English: There is a set of records for a given person. If R1Col(N) has data, do not update. If R1Col(N) has no data, get data from R2Col(N) If R2Col(N) has no data, get data from R3Col(N) Start at N = (column) 1. Search until data found, update R1Col(N), then abort search and move to next column C# has a wonderful representation of data inside of the data adapter. Each table is a collection of rows AND a collection of columns. So I can iterate through the collection of columns, and once I have a collection of fields in a single column, iterate through that looking at the data in row 1, row 2 row 3 until data is found. This data representation just immediately does 1/2 the work for me by presenting me with my data already loaded into these collections. > Of course if it is to learn datasets then thats another story It is to get work done that needs doing. Along the way I expect to learn datasets and get that much more comfortable with C#. Doesn't this sound like FUN programming? I am a programmer at heart, I love doing this kind of stuff, and C# is where I am going, so why not do it there? John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > Just curious, how many records are you talking about - a few hundred or > millions? > > Second question, other than for purely academic reasons, why are you bring > the data into C#, to concatenate and insert into another table can so easily > be done in one sproc, even if you need a temporary holding place, you would > still use a temp table or even a local table. > > Of course if it is to learn datasets then thats another story, > > Mark From shamil at smsconsulting.spb.ru Fri Nov 13 08:36:06 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 17:36:06 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD6D8A.3060306@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> <004801ca646c$5758e690$060ab3b0$@spb.ru> <4AFD6D8A.3060306@colbyconsulting.com> Message-ID: <004a01ca646e$a3b24570$eb16d050$@spb.ru> Hi John, OK, if you're adding several millions rows then SqlBulkLoad can make your "shot deal" in one shot :) -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 13, 2009 5:31 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? In fact this is a one shot deal. I need to do this one time, to one table. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > You can also consider using SqlBulkLoad - below is just a simple sample, and > you can develop very flexible variations of this approach. SqlBulkLoad > should be as quick as bcp maybe even quicker. In fact the source for > SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this > IDataReader to have all the data to bulkload then your programs will become > as streamlined as they probably have never been, FYI: IDataReader can be > built in code with data sources being everything as e.g. .csv files etc. > (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) > > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class SqlBulkLoadTest > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;" + > "Initial Catalog=Northwind;User Id=sa;Password=N/A"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { > connection.Open(); > string sql = "select * from [Shippers]"; > SqlCommand command = new SqlCommand(sql, connection); > SqlDataAdapter adapter = new SqlDataAdapter(command); > > DataSet dataSet = new DataSet("Shippers"); > adapter.Fill(dataSet); > > Console.WriteLine("*** dataSet ***"); > foreach (DataRow dataRow in dataSet.Tables[0].Rows) > { > object[] items = { dataRow[0], dataRow[1], dataRow[2] }; > > Console.WriteLine("{0} {1} {2}", > dataRow[0], dataRow[1], dataRow[2]); > } > > > (new SqlCommand("delete from [NewShippers]",connection)) > .ExecuteNonQuery(); > > SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, > SqlBulkCopyOptions.KeepIdentity); > bulkCopy.DestinationTableName = "NewShippers"; > bulkCopy.WriteToServer(dataSet.Tables[0]); > } > } > > } > } > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 13, 2009 3:52 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > Shamil, > > That is precisely what I have in mind, with variations. Then at the end, > write all updates back to > SQL Server. > > Thanks for the demo code, always helpful. > > John W. Colby > www.ColbyConsulting.com > > > <<< snip >> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4603 (20091113) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4603 (20091113) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 13 08:40:25 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 09:40:25 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <004a01ca646e$a3b24570$eb16d050$@spb.ru> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> <004801ca646c$5758e690$060ab3b0$@spb.ru> <4AFD6D8A.3060306@colbyconsulting.com> <004a01ca646e$a3b24570$eb16d050$@spb.ru> Message-ID: <4AFD6FD9.2040902@colbyconsulting.com> See my reply to Mark Breen coming up. This is not a bulk load thing. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > OK, if you're adding several millions rows then SqlBulkLoad can make your > "shot deal" in one shot :) > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 13, 2009 5:31 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > In fact this is a one shot deal. I need to do this one time, to one table. > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> You can also consider using SqlBulkLoad - below is just a simple sample, > and >> you can develop very flexible variations of this approach. SqlBulkLoad >> should be as quick as bcp maybe even quicker. In fact the source for >> SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this >> IDataReader to have all the data to bulkload then your programs will > become >> as streamlined as they probably have never been, FYI: IDataReader can be >> built in code with data sources being everything as e.g. .csv files etc. >> (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) >> >> >> using System; >> using System.Collections.Generic; >> using System.Text; >> >> using System.Data; >> using System.Data.SqlClient; >> >> namespace TestConsole.Samples >> { >> public class SqlBulkLoadTest >> { >> public static void Run() >> { >> string connectionString = "Data Source=HAMBURG\\SQL2005;" + >> "Initial Catalog=Northwind;User Id=sa;Password=N/A"; >> using (SqlConnection connection = >> new SqlConnection(connectionString)) >> { >> connection.Open(); >> string sql = "select * from [Shippers]"; >> SqlCommand command = new SqlCommand(sql, connection); >> SqlDataAdapter adapter = new SqlDataAdapter(command); >> >> DataSet dataSet = new DataSet("Shippers"); >> adapter.Fill(dataSet); >> >> Console.WriteLine("*** dataSet ***"); >> foreach (DataRow dataRow in dataSet.Tables[0].Rows) >> { >> object[] items = { dataRow[0], dataRow[1], dataRow[2] > }; >> Console.WriteLine("{0} {1} {2}", >> dataRow[0], dataRow[1], dataRow[2]); >> } >> >> >> (new SqlCommand("delete from [NewShippers]",connection)) >> .ExecuteNonQuery(); >> >> SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, >> SqlBulkCopyOptions.KeepIdentity); >> bulkCopy.DestinationTableName = "NewShippers"; >> bulkCopy.WriteToServer(dataSet.Tables[0]); >> } >> } >> >> } >> } >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: Friday, November 13, 2009 3:52 PM >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] What to do, what to do? >> >> Shamil, >> >> That is precisely what I have in mind, with variations. Then at the end, >> write all updates back to >> SQL Server. >> >> Thanks for the demo code, always helpful. >> >> John W. Colby >> www.ColbyConsulting.com >> >> >> <<< snip >> >> >> >> __________ Information from ESET NOD32 Antivirus, version of virus > signature >> database 4603 (20091113) __________ >> >> The message was checked by ESET NOD32 Antivirus. >> >> http://www.esetnod32.ru >> >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4603 (20091113) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Nov 13 08:54:16 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 17:54:16 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD6FD9.2040902@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> <004801ca646c$5758e690$060ab3b0$@spb.ru> <4AFD6D8A.3060306@colbyconsulting.com> <004a01ca646e$a3b24570$eb16d050$@spb.ru> <4AFD6FD9.2040902@colbyconsulting.com> Message-ID: <004b01ca6471$2d3f5dd0$87be1970$@spb.ru> OK, I see this is "update thing" not insert/bulkload. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 13, 2009 5:40 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? See my reply to Mark Breen coming up. This is not a bulk load thing. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > OK, if you're adding several millions rows then SqlBulkLoad can make your > "shot deal" in one shot :) > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 13, 2009 5:31 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > In fact this is a one shot deal. I need to do this one time, to one table. > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> You can also consider using SqlBulkLoad - below is just a simple sample, > and >> you can develop very flexible variations of this approach. SqlBulkLoad >> should be as quick as bcp maybe even quicker. In fact the source for >> SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this >> IDataReader to have all the data to bulkload then your programs will > become >> as streamlined as they probably have never been, FYI: IDataReader can be >> built in code with data sources being everything as e.g. .csv files etc. >> (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) >> >> >> using System; >> using System.Collections.Generic; >> using System.Text; >> >> using System.Data; >> using System.Data.SqlClient; >> >> namespace TestConsole.Samples >> { >> public class SqlBulkLoadTest >> { >> public static void Run() >> { >> string connectionString = "Data Source=HAMBURG\\SQL2005;" + >> "Initial Catalog=Northwind;User Id=sa;Password=N/A"; >> using (SqlConnection connection = >> new SqlConnection(connectionString)) >> { >> connection.Open(); >> string sql = "select * from [Shippers]"; >> SqlCommand command = new SqlCommand(sql, connection); >> SqlDataAdapter adapter = new SqlDataAdapter(command); >> >> DataSet dataSet = new DataSet("Shippers"); >> adapter.Fill(dataSet); >> >> Console.WriteLine("*** dataSet ***"); >> foreach (DataRow dataRow in dataSet.Tables[0].Rows) >> { >> object[] items = { dataRow[0], dataRow[1], dataRow[2] > }; >> Console.WriteLine("{0} {1} {2}", >> dataRow[0], dataRow[1], dataRow[2]); >> } >> >> >> (new SqlCommand("delete from [NewShippers]",connection)) >> .ExecuteNonQuery(); >> >> SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, >> SqlBulkCopyOptions.KeepIdentity); >> bulkCopy.DestinationTableName = "NewShippers"; >> bulkCopy.WriteToServer(dataSet.Tables[0]); >> } >> } >> >> } >> } >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: Friday, November 13, 2009 3:52 PM >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] What to do, what to do? >> >> Shamil, >> >> That is precisely what I have in mind, with variations. Then at the end, >> write all updates back to >> SQL Server. >> >> Thanks for the demo code, always helpful. >> >> John W. Colby >> www.ColbyConsulting.com >> >> >> <<< snip >> >> From jwcolby at colbyconsulting.com Fri Nov 13 10:39:14 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 11:39:14 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> Message-ID: <4AFD8BB2.5030203@colbyconsulting.com> Shamil, Given what you now know about what I want to do... Why does each table get created in it's own DataSet. I thought a data set could have multiple tables. I envisioned the following structure out in SQL Server: tblHSID - contains 6.4 million records. Source of records for tblMergeSrc. tblMergeSrc - Contains X thousand records for Y hundred people. Working table in memory. tblMergedRecs - the single record merged into will be created here (or copied here). Working table in memory. tblPurgedRecs - The multiple records merged FROM will be archived here. Working table in memory. tblHashPK - Hash for a person / address, PKs for person / address. There is one hash for each person / hash, there are multiple PKs for each person / address. One record for each record in tblHSID. tblHashCnt - Group by Hash, Cnt PK. Server two purposes, provides a "DISTINCT" hash for each person being processed as well as a count of how many records will be merged down. So... tblHSID is the source of the records to be processed. Contains ONLY people who actually have "duplicates", most people don't. The Top(N) records from tblHashCnt will select the Top(N) PEOPLE by number of records to be merged. Merge the ones with the most records first. That will be joined to tblHashPK to pull all of the PKs for the Top(N) people about to be processed. The PKs from that will be joined to tblHSID to pull a set of records to be merged. Copied into tblMergeSrc. This is the first time HSID records (records to be merged) will be moved into a table in a dataset. tblMergedRecs is empty but exists in the dataset. tblPurgedRecs is empty but exists in the dataset. The merge will most likely happen in tblMergeSrc although a new record could be created in tblMergedRecs to work in. Not decided yet. In the end, the merge happens in the dataset in memory. The actual ONE merged record for each person ends up in tblMergedRecs. The actual (multiple) records merged into other records are moved into tblPurgedRecs. This is an archive. The records in tblPurgedRecs are then deleted from tblMergedSrc, and eventually deleted out of tblHSID back in the database. The records for the next person are processed. Merged records moved to tblMergedRecs. "Purged" records moved to tblPurgedRecs. Deleted from tblMergeSrc. Wash, rinse repeat. Now... I don't care how many data sets I have but I just question why I would create a new dataset for each table above. It seems that having them all in a single dataset would minimize confusion and stuff. Logically it seems that working with multiple tables inside of a single DataSet object MIGHT be faster than moving data back and forth between tables in different DataSet objects, though maybe not. And of course... my book talks all about connections, then all about commands, and then all about parameters, then... How to work with a data reader. WTF over? What happened to my dataset and table objects? Not a WORD about those two objects. Sigh. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Here is a sample: > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class DataSetTests > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;"+ > "Initial Catalog=Northwind;User Id=sa;Password=N/A"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { From shamil at smsconsulting.spb.ru Fri Nov 13 11:22:31 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 20:22:31 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD8BB2.5030203@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD8BB2.5030203@colbyconsulting.com> Message-ID: <005b01ca6485$e42c31d0$ac849570$@spb.ru> John, Yes, of course you can have multiple DataTable(s) in a DataSet - I have just posted a very simple sample - one of a myriad possible combinations, the simplest one probably. And of course you can have your DataSet(s) structure created statically in VS designer. Yes, I think your plan is OK. You probably do not need a DataReader but if you'll find you need one - in P.S. is a simple sample. In fact Gustav knows DataSets better than I do - please address your questions to him also and to everybody else here - I will not have time to reply any more postings today... Thank you. -- Shamil P.S. DataReader sample: using System; using System.Collections.Generic; using System.Text; using System.Data; using System.Data.SqlClient; namespace TestConsole.Samples { public class SqlDataReaderTest { public static void Run() { string connectionString = "Data Source=HAMBURG\\SQL2005;" + "Initial Catalog=Northwind;User Id=sa;Password=N/A"; using (SqlConnection connection = new SqlConnection(connectionString)) { connection.Open(); // Create empty [NewShippers] dataset in memory string sql1 = "select * from [NewShippers] where(1=0)"; SqlCommand command = new SqlCommand(sql1, connection); SqlDataAdapter adapter = new SqlDataAdapter(command); DataSet newDataSet = new DataSet("NewShippers"); adapter.Fill(newDataSet); // Load source [Shippers] in memory string sql = "select * from [Shippers]"; command = new SqlCommand(sql, connection); SqlDataReader dataReader = command.ExecuteReader(); // Add loaded [Shippers] to [NewShippers] im memory Console.WriteLine("*** dataReader ***"); while (dataReader.Read()) { object[] items = { dataReader[0], dataReader[1], dataReader[2] }; newDataSet.Tables[0].Rows.Add(items); Console.WriteLine("{0} {1} {2}", items[0], items[1], items[2]); } dataReader.Close(); // ! always close DataReader // Insert [NewShippers] into db (new SqlCommand("delete from [NewShippers]",connection)) .ExecuteNonQuery(); SqlCommandBuilder builder = new SqlCommandBuilder(adapter); adapter.InsertCommand = builder.GetInsertCommand(true); adapter.Update(newDataSet.Tables[0]); // Check that [NewShippers] were inserted OK Console.WriteLine("*** newDataSet ***"); newDataSet.Clear(); adapter = new SqlDataAdapter( new SqlCommand("select * from [NewShippers]",connection)); adapter.Fill(newDataSet); foreach (DataRow dataRow in newDataSet.Tables[0].Rows) { Console.WriteLine("{0} {1} {2}", dataRow[0], dataRow[1], dataRow[2]); } } } } } <<< snip to get through dba-VB robot >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4604 (20091113) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From cfoust at infostatsystems.com Fri Nov 13 12:18:07 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Fri, 13 Nov 2009 10:18:07 -0800 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD8BB2.5030203@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com><002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD8BB2.5030203@colbyconsulting.com> Message-ID: You can have all sorts of tables in a dataset, John. But if you want to act on multiple non-cascading tables, then you probably need to validate multiple tables. If you have relationships set so you can cascade updates, it isn't too bad. Otherwise, you have to write code to individually handle the gotchas in each table before you update any of them. Without that, you'd wind up with a partial update. Some of my biggest headaches are the one rich typed dataset we use to allow the users to enter data into 3 or 4 different tables from a single form. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 13, 2009 8:39 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? Shamil, Given what you now know about what I want to do... Why does each table get created in it's own DataSet. I thought a data set could have multiple tables. I envisioned the following structure out in SQL Server: tblHSID - contains 6.4 million records. Source of records for tblMergeSrc. tblMergeSrc - Contains X thousand records for Y hundred people. Working table in memory. tblMergedRecs - the single record merged into will be created here (or copied here). Working table in memory. tblPurgedRecs - The multiple records merged FROM will be archived here. Working table in memory. tblHashPK - Hash for a person / address, PKs for person / address. There is one hash for each person / hash, there are multiple PKs for each person / address. One record for each record in tblHSID. tblHashCnt - Group by Hash, Cnt PK. Server two purposes, provides a "DISTINCT" hash for each person being processed as well as a count of how many records will be merged down. So... tblHSID is the source of the records to be processed. Contains ONLY people who actually have "duplicates", most people don't. The Top(N) records from tblHashCnt will select the Top(N) PEOPLE by number of records to be merged. Merge the ones with the most records first. That will be joined to tblHashPK to pull all of the PKs for the Top(N) people about to be processed. The PKs from that will be joined to tblHSID to pull a set of records to be merged. Copied into tblMergeSrc. This is the first time HSID records (records to be merged) will be moved into a table in a dataset. tblMergedRecs is empty but exists in the dataset. tblPurgedRecs is empty but exists in the dataset. The merge will most likely happen in tblMergeSrc although a new record could be created in tblMergedRecs to work in. Not decided yet. In the end, the merge happens in the dataset in memory. The actual ONE merged record for each person ends up in tblMergedRecs. The actual (multiple) records merged into other records are moved into tblPurgedRecs. This is an archive. The records in tblPurgedRecs are then deleted from tblMergedSrc, and eventually deleted out of tblHSID back in the database. The records for the next person are processed. Merged records moved to tblMergedRecs. "Purged" records moved to tblPurgedRecs. Deleted from tblMergeSrc. Wash, rinse repeat. Now... I don't care how many data sets I have but I just question why I would create a new dataset for each table above. It seems that having them all in a single dataset would minimize confusion and stuff. Logically it seems that working with multiple tables inside of a single DataSet object MIGHT be faster than moving data back and forth between tables in different DataSet objects, though maybe not. And of course... my book talks all about connections, then all about commands, and then all about parameters, then... How to work with a data reader. WTF over? What happened to my dataset and table objects? Not a WORD about those two objects. Sigh. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Here is a sample: > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class DataSetTests > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;"+ > "Initial Catalog=Northwind;User Id=sa;Password=N/A"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From raibeart at gmail.com Fri Nov 13 22:46:18 2009 From: raibeart at gmail.com (Robert Stewart) Date: Fri, 13 Nov 2009 22:46:18 -0600 Subject: [dba-VB] What to do, what to do? In-Reply-To: References: Message-ID: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert At 12:00 PM 11/13/2009, you wrote: >Date: Fri, 13 Nov 2009 11:39:14 -0500 >From: jwcolby >Subject: Re: [dba-VB] What to do, what to do? >To: "Discussion concerning Visual Basic and related programming > issues." >Message-ID: <4AFD8BB2.5030203 at colbyconsulting.com> >Content-Type: text/plain; charset=ISO-8859-1; format=flowed > >Shamil, > >Given what you now know about what I want to do... > >Why does each table get created in it's own DataSet. I thought a >data set could have multiple >tables. I envisioned the following structure out in SQL Server: > >tblHSID - contains 6.4 million records. Source of records for tblMergeSrc. >tblMergeSrc - Contains X thousand records for Y hundred >people. Working table in memory. >tblMergedRecs - the single record merged into will be created here >(or copied here). Working table >in memory. >tblPurgedRecs - The multiple records merged FROM will be archived >here. Working table in memory. > >tblHashPK - Hash for a person / address, PKs for person / >address. There is one hash for each >person / hash, there are multiple PKs for each person / >address. One record for each record in >tblHSID. > >tblHashCnt - Group by Hash, Cnt PK. Server two purposes, provides a >"DISTINCT" hash for each person >being processed as well as a count of how many records will be merged down. > >So... tblHSID is the source of the records to be >processed. Contains ONLY people who actually have >"duplicates", most people don't. > >The Top(N) records from tblHashCnt will select the Top(N) PEOPLE by >number of records to be merged. > Merge the ones with the most records first. > >That will be joined to tblHashPK to pull all of the PKs for the >Top(N) people about to be processed. > >The PKs from that will be joined to tblHSID to pull a set of records >to be merged. Copied into >tblMergeSrc. This is the first time HSID records (records to be >merged) will be moved into a table >in a dataset. > >tblMergedRecs is empty but exists in the dataset. >tblPurgedRecs is empty but exists in the dataset. > >The merge will most likely happen in tblMergeSrc although a new >record could be created in >tblMergedRecs to work in. Not decided yet. > >In the end, the merge happens in the dataset in memory. The actual >ONE merged record for each >person ends up in tblMergedRecs. > >The actual (multiple) records merged into other records are moved >into tblPurgedRecs. This is an >archive. > >The records in tblPurgedRecs are then deleted from tblMergedSrc, and >eventually deleted out of >tblHSID back in the database. > >The records for the next person are processed. Merged records moved >to tblMergedRecs. "Purged" >records moved to tblPurgedRecs. > >Deleted from tblMergeSrc. > >Wash, rinse repeat. > >Now... I don't care how many data sets I have but I just question >why I would create a new dataset >for each table above. It seems that having them all in a single >dataset would minimize confusion >and stuff. Logically it seems that working with multiple tables >inside of a single DataSet object >MIGHT be faster than moving data back and forth between tables in >different DataSet objects, though >maybe not. > >And of course... my book talks all about connections, then all about >commands, and then all about >parameters, then... > >How to work with a data reader. WTF over? What happened to my >dataset and table objects? Not a >WORD about those two objects. > >Sigh. > >John W. Colby From shamil at smsconsulting.spb.ru Sat Nov 14 04:40:43 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 14 Nov 2009 13:40:43 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> Message-ID: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Hi Robert, Yes, DataSets are mainly targeted for working with small number of records in memory - so the following is more a "theoretical" calculation: - JC has 64bit PC which allow to load in memory "practically unlimited" volume of data - 64bit logical (process) address space could be as large as 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 (http://en.wikipedia.org/wiki/Exbibyte). - the speed of data transfer for 64bit Intel's processor - "the 200 MHz McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); ----- So "theoretically" with modern speedy harddisks JC can load 50 millions of records into memory within minutes - within half a minute in the near future when large enough flash-memory disks will become available and relatively inexpensive? To load such large data volume it would be better to use SqlDataReader as it keeps loaded data in a very compact form. And 50 million records long SqlDataReader should be probably better split into several chunks... Again - this is just a "theoretical" consideration - in practice JC can process his 50 million records long data table in chunks as he has a hash field (tblHashPK), which can be used to load related records into memory: I mean he can split tblHashPK table keeping hash values into several groups, e.g.: 1 - 100,000 100,001 - 200,000 ... and process each group joining its records to source 50 million records... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart Sent: Saturday, November 14, 2009 7:46 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] What to do, what to do? John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From max.wanadoo at gmail.com Sat Nov 14 05:30:41 2009 From: max.wanadoo at gmail.com (Max Wanadoo) Date: Sat, 14 Nov 2009 11:30:41 -0000 Subject: [dba-VB] What to do, what to do? In-Reply-To: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Message-ID: <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> Very interesting Shamil What happens with all the other stuff that memory is taking up though, OS, Programs, etc and the physical memory he has placed in the machine? Your limits below presumably are saying that his physical memory is loaded to...what? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 10:41 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Robert, Yes, DataSets are mainly targeted for working with small number of records in memory - so the following is more a "theoretical" calculation: - JC has 64bit PC which allow to load in memory "practically unlimited" volume of data - 64bit logical (process) address space could be as large as 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 (http://en.wikipedia.org/wiki/Exbibyte). - the speed of data transfer for 64bit Intel's processor - "the 200 MHz McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); ----- So "theoretically" with modern speedy harddisks JC can load 50 millions of records into memory within minutes - within half a minute in the near future when large enough flash-memory disks will become available and relatively inexpensive? To load such large data volume it would be better to use SqlDataReader as it keeps loaded data in a very compact form. And 50 million records long SqlDataReader should be probably better split into several chunks... Again - this is just a "theoretical" consideration - in practice JC can process his 50 million records long data table in chunks as he has a hash field (tblHashPK), which can be used to load related records into memory: I mean he can split tblHashPK table keeping hash values into several groups, e.g.: 1 - 100,000 100,001 - 200,000 ... and process each group joining its records to source 50 million records... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart Sent: Saturday, November 14, 2009 7:46 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] What to do, what to do? John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Nov 14 06:30:35 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 14 Nov 2009 07:30:35 -0500 Subject: [dba-VB] SPAM-LOW: Re: What to do, what to do? In-Reply-To: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> Message-ID: <4AFEA2EB.5050507@colbyconsulting.com> Read the discussion carefully Robert. I am discussing doing chunks of thousands. John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > John, > > I don't think your machine, or any for that matter, have the memory > to load 50 million records, extract 40 million, transform them and > then load them back to the 50 million and update the database from > the recordset. Datasets are for working with a small number of > records in memory, not millions. > > Robert > > > At 12:00 PM 11/13/2009, you wrote: >> Date: Fri, 13 Nov 2009 11:39:14 -0500 >> From: jwcolby >> Subject: Re: [dba-VB] What to do, what to do? >> To: "Discussion concerning Visual Basic and related programming >> issues." >> Message-ID: <4AFD8BB2.5030203 at colbyconsulting.com> >> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >> >> Shamil, >> >> Given what you now know about what I want to do... >> >> Why does each table get created in it's own DataSet. I thought a >> data set could have multiple >> tables. I envisioned the following structure out in SQL Server: >> >> tblHSID - contains 6.4 million records. Source of records for tblMergeSrc. >> tblMergeSrc - Contains X thousand records for Y hundred >> people. Working table in memory. >> tblMergedRecs - the single record merged into will be created here >> (or copied here). Working table >> in memory. >> tblPurgedRecs - The multiple records merged FROM will be archived >> here. Working table in memory. >> >> tblHashPK - Hash for a person / address, PKs for person / >> address. There is one hash for each >> person / hash, there are multiple PKs for each person / >> address. One record for each record in >> tblHSID. >> >> tblHashCnt - Group by Hash, Cnt PK. Server two purposes, provides a >> "DISTINCT" hash for each person >> being processed as well as a count of how many records will be merged down. >> >> So... tblHSID is the source of the records to be >> processed. Contains ONLY people who actually have >> "duplicates", most people don't. >> >> The Top(N) records from tblHashCnt will select the Top(N) PEOPLE by >> number of records to be merged. >> Merge the ones with the most records first. >> >> That will be joined to tblHashPK to pull all of the PKs for the >> Top(N) people about to be processed. >> >> The PKs from that will be joined to tblHSID to pull a set of records >> to be merged. Copied into >> tblMergeSrc. This is the first time HSID records (records to be >> merged) will be moved into a table >> in a dataset. >> >> tblMergedRecs is empty but exists in the dataset. >> tblPurgedRecs is empty but exists in the dataset. >> >> The merge will most likely happen in tblMergeSrc although a new >> record could be created in >> tblMergedRecs to work in. Not decided yet. >> >> In the end, the merge happens in the dataset in memory. The actual >> ONE merged record for each >> person ends up in tblMergedRecs. >> >> The actual (multiple) records merged into other records are moved >> into tblPurgedRecs. This is an >> archive. >> >> The records in tblPurgedRecs are then deleted from tblMergedSrc, and >> eventually deleted out of >> tblHSID back in the database. >> >> The records for the next person are processed. Merged records moved >> to tblMergedRecs. "Purged" >> records moved to tblPurgedRecs. >> >> Deleted from tblMergeSrc. >> >> Wash, rinse repeat. >> >> Now... I don't care how many data sets I have but I just question >> why I would create a new dataset >> for each table above. It seems that having them all in a single >> dataset would minimize confusion >> and stuff. Logically it seems that working with multiple tables >> inside of a single DataSet object >> MIGHT be faster than moving data back and forth between tables in >> different DataSet objects, though >> maybe not. >> >> And of course... my book talks all about connections, then all about >> commands, and then all about >> parameters, then... >> >> How to work with a data reader. WTF over? What happened to my >> dataset and table objects? Not a >> WORD about those two objects. >> >> Sigh. >> >> John W. Colby > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Nov 14 07:46:21 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 14 Nov 2009 16:46:21 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> Message-ID: <008201ca6530$daf14140$90d3c3c0$@spb.ru> Hi Max, But OS, programs etc. - they all use their own logical address space, which is mapped to (shared (using OS directed swapping)) physical address space: with JC 64bit PC OS image/runtime will be kept in its own physical memory segments - and no swapping will occur, MS SQL image/runtime/data will be (mainly) kept in its own physical memory segments - and almost no swapping will occur, JC's program will have occupied its own physical memory segments - and no swapping will occur... 50 million records assuming every of them is equal to approx. 1KB will consume ~50GB - 64bit PC can have(?) that much RAM allocated for one process, and it all can be addresses directly within this process logical address space. And I have noted that it's currently a "theoretical consideration" but it may become practically achievable in not that far future(?) Correct/wrong? (I must say I didn't get checked how large can be logical address space for a Win7 process running on 64bit PC). Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Max Wanadoo Sent: Saturday, November 14, 2009 2:31 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Very interesting Shamil What happens with all the other stuff that memory is taking up though, OS, Programs, etc and the physical memory he has placed in the machine? Your limits below presumably are saying that his physical memory is loaded to...what? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 10:41 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Robert, Yes, DataSets are mainly targeted for working with small number of records in memory - so the following is more a "theoretical" calculation: - JC has 64bit PC which allow to load in memory "practically unlimited" volume of data - 64bit logical (process) address space could be as large as 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 (http://en.wikipedia.org/wiki/Exbibyte). - the speed of data transfer for 64bit Intel's processor - "the 200 MHz McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); ----- So "theoretically" with modern speedy harddisks JC can load 50 millions of records into memory within minutes - within half a minute in the near future when large enough flash-memory disks will become available and relatively inexpensive? To load such large data volume it would be better to use SqlDataReader as it keeps loaded data in a very compact form. And 50 million records long SqlDataReader should be probably better split into several chunks... Again - this is just a "theoretical" consideration - in practice JC can process his 50 million records long data table in chunks as he has a hash field (tblHashPK), which can be used to load related records into memory: I mean he can split tblHashPK table keeping hash values into several groups, e.g.: 1 - 100,000 100,001 - 200,000 ... and process each group joining its records to source 50 million records... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart Sent: Saturday, November 14, 2009 7:46 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] What to do, what to do? John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 14 08:23:42 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 14 Nov 2009 09:23:42 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Message-ID: <4AFEBD6E.1020709@colbyconsulting.com> All of which is beside the point. Divide and conquer. A principal discovered by military strategists thousands of years ago, and applied to problems of all sorts today. I can EASILY work with sets of thousands, and even hundreds of thousands of records. In real life, in my computer. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Robert, > > Yes, DataSets are mainly targeted for working with small number of records > in memory - so the following is more a "theoretical" calculation: > > - JC has 64bit PC which allow to load in memory "practically unlimited" > volume of data - 64bit logical (process) address space could be as large as > 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 > (http://en.wikipedia.org/wiki/Exbibyte). > > - the speed of data transfer for 64bit Intel's processor - "the 200 MHz > McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers > 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); > > ----- > So "theoretically" with modern speedy harddisks JC can load 50 millions of > records into memory within minutes - within half a minute in the near future > when large enough flash-memory disks will become available and relatively > inexpensive? > > To load such large data volume it would be better to use SqlDataReader as it > keeps loaded data in a very compact form. And 50 million records long > SqlDataReader should be probably better split into several chunks... > > Again - this is just a "theoretical" consideration - in practice JC can > process his 50 million records long data table in chunks as he has a hash > field (tblHashPK), which can be used to load related records into memory: I > mean he can split tblHashPK table keeping hash values into several groups, > e.g.: > > 1 - 100,000 > 100,001 - 200,000 > ... > > and process each group joining its records to source 50 million records... > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart > Sent: Saturday, November 14, 2009 7:46 AM > To: dba-vb at databaseadvisors.com > Subject: Re: [dba-VB] What to do, what to do? > > John, > > I don't think your machine, or any for that matter, have the memory > to load 50 million records, extract 40 million, transform them and > then load them back to the 50 million and update the database from > the recordset. Datasets are for working with a small number of > records in memory, not millions. > > Robert > > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4606 (20091114) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Nov 14 08:44:11 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 14 Nov 2009 17:44:11 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFEBD6E.1020709@colbyconsulting.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4AFEBD6E.1020709@colbyconsulting.com> Message-ID: <008301ca6538$eeef8910$ccce9b30$@spb.ru> Hi John, Yes, I have just "theoretically" noted that in the near future you'll be able to EVEN MORE EASILY work with sets of millions (and billions?) records by just loading them into memory, "crunching", and saving back updated... ... it will be just 1,2,3 instead of your today's (1,2,3) (1,2,3)... (1,2,3) (multi-threaded)... ... the time just to develop (1,2,3) (1,2,3)... (1,2,3)... will be more expensive than time to develop and run 1,2,3 on the near future hardware (hardware costs included)... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 14, 2009 5:24 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? All of which is beside the point. Divide and conquer. A principal discovered by military strategists thousands of years ago, and applied to problems of all sorts today. I can EASILY work with sets of thousands, and even hundreds of thousands of records. In real life, in my computer. John W. Colby www.ColbyConsulting.com <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4607 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From max.wanadoo at gmail.com Sat Nov 14 09:38:39 2009 From: max.wanadoo at gmail.com (Max Wanadoo) Date: Sat, 14 Nov 2009 15:38:39 -0000 Subject: [dba-VB] What to do, what to do? In-Reply-To: <008201ca6530$daf14140$90d3c3c0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> <008201ca6530$daf14140$90d3c3c0$@spb.ru> Message-ID: <4afecf2a.0702d00a.3bd1.4040@mx.google.com> Hi Shamil, I understand the logical size bit but for that to be mapped to physical memory requires physical memory to be present. Current there are limits to how much can be installed on the motherboard. 64Bit memory can address the sizes you mentioned but the physical memory doesn't exist, so (presumably) it maps it to the HD, in which case it is not in memory. Have I misunderstood something (it has been known to happen)? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 13:46 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Max, But OS, programs etc. - they all use their own logical address space, which is mapped to (shared (using OS directed swapping)) physical address space: with JC 64bit PC OS image/runtime will be kept in its own physical memory segments - and no swapping will occur, MS SQL image/runtime/data will be (mainly) kept in its own physical memory segments - and almost no swapping will occur, JC's program will have occupied its own physical memory segments - and no swapping will occur... 50 million records assuming every of them is equal to approx. 1KB will consume ~50GB - 64bit PC can have(?) that much RAM allocated for one process, and it all can be addresses directly within this process logical address space. And I have noted that it's currently a "theoretical consideration" but it may become practically achievable in not that far future(?) Correct/wrong? (I must say I didn't get checked how large can be logical address space for a Win7 process running on 64bit PC). Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Max Wanadoo Sent: Saturday, November 14, 2009 2:31 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Very interesting Shamil What happens with all the other stuff that memory is taking up though, OS, Programs, etc and the physical memory he has placed in the machine? Your limits below presumably are saying that his physical memory is loaded to...what? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 10:41 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Robert, Yes, DataSets are mainly targeted for working with small number of records in memory - so the following is more a "theoretical" calculation: - JC has 64bit PC which allow to load in memory "practically unlimited" volume of data - 64bit logical (process) address space could be as large as 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 (http://en.wikipedia.org/wiki/Exbibyte). - the speed of data transfer for 64bit Intel's processor - "the 200 MHz McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); ----- So "theoretically" with modern speedy harddisks JC can load 50 millions of records into memory within minutes - within half a minute in the near future when large enough flash-memory disks will become available and relatively inexpensive? To load such large data volume it would be better to use SqlDataReader as it keeps loaded data in a very compact form. And 50 million records long SqlDataReader should be probably better split into several chunks... Again - this is just a "theoretical" consideration - in practice JC can process his 50 million records long data table in chunks as he has a hash field (tblHashPK), which can be used to load related records into memory: I mean he can split tblHashPK table keeping hash values into several groups, e.g.: 1 - 100,000 100,001 - 200,000 ... and process each group joining its records to source 50 million records... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart Sent: Saturday, November 14, 2009 7:46 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] What to do, what to do? John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Sat Nov 14 10:08:26 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 14 Nov 2009 19:08:26 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4afecf2a.0702d00a.3bd1.4040@mx.google.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> <008201ca6530$daf14140$90d3c3c0$@spb.ru> <4afecf2a.0702d00a.3bd1.4040@mx.google.com> Message-ID: <008401ca6544$b4290890$1c7b19b0$@spb.ru> Hi Max, Here are theoretical limits I have found info about: Address space 64-bit Windows 32-bit Windows Virtual memory 16 Tb 4 Gb Swap file 512 Tb 16 Tb System cache 1 Tb 1 Gb In practice, mass market motherboards can already handle 64GB(?) - Not enough for JC tasks but this max RAM size limitations should be removed in the near future AFAICG, and Win7 is ready(?) to handle effectively that much RAM.... http://www.nextag.com/64-bit-pci-motherboard/search-html http://www.viva64.com/content/articles/64-bit-development/?f=Application_por t_to_64-bit_platforms_or_never_cackle_till_your_egg_is_laid.html&lang=en&con tent=64-bit-development http://blogs.zdnet.com/hardware/?p=4254 http://www.tomshardware.com/reviews/vista-workshop,1775-3.html Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Max Wanadoo Sent: Saturday, November 14, 2009 6:39 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Shamil, I understand the logical size bit but for that to be mapped to physical memory requires physical memory to be present. Current there are limits to how much can be installed on the motherboard. 64Bit memory can address the sizes you mentioned but the physical memory doesn't exist, so (presumably) it maps it to the HD, in which case it is not in memory. Have I misunderstood something (it has been known to happen)? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 13:46 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Max, But OS, programs etc. - they all use their own logical address space, which is mapped to (shared (using OS directed swapping)) physical address space: with JC 64bit PC OS image/runtime will be kept in its own physical memory segments - and no swapping will occur, MS SQL image/runtime/data will be (mainly) kept in its own physical memory segments - and almost no swapping will occur, JC's program will have occupied its own physical memory segments - and no swapping will occur... 50 million records assuming every of them is equal to approx. 1KB will consume ~50GB - 64bit PC can have(?) that much RAM allocated for one process, and it all can be addresses directly within this process logical address space. And I have noted that it's currently a "theoretical consideration" but it may become practically achievable in not that far future(?) Correct/wrong? (I must say I didn't get checked how large can be logical address space for a Win7 process running on 64bit PC). Thank you. -- Shamil <<< snipped >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4607 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From max.wanadoo at gmail.com Sat Nov 14 10:27:00 2009 From: max.wanadoo at gmail.com (Max Wanadoo) Date: Sat, 14 Nov 2009 16:27:00 -0000 Subject: [dba-VB] What to do, what to do? In-Reply-To: <008401ca6544$b4290890$1c7b19b0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> <008201ca6530$daf14140$90d3c3c0$@spb.ru> <4afecf2a.0702d00a.3bd1.4040@mx.google.com> <008401ca6544$b4290890$1c7b19b0$@spb.ru> Message-ID: <4afeda84.0508d00a.3cd0.ffffe9aa@mx.google.com> Ok, I thought you were referring to ram available now on common-or-garden PCs. So, we can look forward to physical ram being cheap enough to enable in-vitro processing for millions of records. Hurrah! And goodbye to latency... Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 16:08 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Max, Here are theoretical limits I have found info about: Address space 64-bit Windows 32-bit Windows Virtual memory 16 Tb 4 Gb Swap file 512 Tb 16 Tb System cache 1 Tb 1 Gb In practice, mass market motherboards can already handle 64GB(?) - Not enough for JC tasks but this max RAM size limitations should be removed in the near future AFAICG, and Win7 is ready(?) to handle effectively that much RAM.... http://www.nextag.com/64-bit-pci-motherboard/search-html http://www.viva64.com/content/articles/64-bit-development/?f=Application_por t_to_64-bit_platforms_or_never_cackle_till_your_egg_is_laid.html&lang=en&con tent=64-bit-development http://blogs.zdnet.com/hardware/?p=4254 http://www.tomshardware.com/reviews/vista-workshop,1775-3.html Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Max Wanadoo Sent: Saturday, November 14, 2009 6:39 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Shamil, I understand the logical size bit but for that to be mapped to physical memory requires physical memory to be present. Current there are limits to how much can be installed on the motherboard. 64Bit memory can address the sizes you mentioned but the physical memory doesn't exist, so (presumably) it maps it to the HD, in which case it is not in memory. Have I misunderstood something (it has been known to happen)? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 13:46 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Max, But OS, programs etc. - they all use their own logical address space, which is mapped to (shared (using OS directed swapping)) physical address space: with JC 64bit PC OS image/runtime will be kept in its own physical memory segments - and no swapping will occur, MS SQL image/runtime/data will be (mainly) kept in its own physical memory segments - and almost no swapping will occur, JC's program will have occupied its own physical memory segments - and no swapping will occur... 50 million records assuming every of them is equal to approx. 1KB will consume ~50GB - 64bit PC can have(?) that much RAM allocated for one process, and it all can be addresses directly within this process logical address space. And I have noted that it's currently a "theoretical consideration" but it may become practically achievable in not that far future(?) Correct/wrong? (I must say I didn't get checked how large can be logical address space for a Win7 process running on 64bit PC). Thank you. -- Shamil <<< snipped >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4607 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Nov 14 10:28:14 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 14 Nov 2009 11:28:14 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <008301ca6538$eeef8910$ccce9b30$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4AFEBD6E.1020709@colbyconsulting.com> <008301ca6538$eeef8910$ccce9b30$@spb.ru> Message-ID: <4AFEDA9E.9050101@colbyconsulting.com> And believe me I look forward to the day. And it will indeed be soon, of course the data sets get bigger as well. ;) I already have 16 gigs of RAM in my servers, and quad cores. I could easily have 64 gigs and 16 CPUs today if I had the bucks. And of course you are correct as well, the Flash drives are wicked fast for working with large data sets. They still have real issues when writing to them bot for reading they are an order of magnitude faster than rotating media. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > Yes, I have just "theoretically" noted that in the near future you'll be > able to EVEN MORE EASILY work with sets of millions (and billions?) records > by just loading them into memory, "crunching", and saving back updated... > > ... it will be just 1,2,3 instead of your today's (1,2,3) (1,2,3)... (1,2,3) > (multi-threaded)... > > ... the time just to develop (1,2,3) (1,2,3)... (1,2,3)... will be more > expensive than time to develop and run 1,2,3 on the near future hardware > (hardware costs included)... > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 14, 2009 5:24 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > All of which is beside the point. > > Divide and conquer. A principal discovered by military strategists > thousands of years ago, and > applied to problems of all sorts today. > > I can EASILY work with sets of thousands, and even hundreds of thousands of > records. In real life, > in my computer. > > John W. Colby > www.ColbyConsulting.com > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4607 (20091114) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Nov 14 16:36:57 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 14 Nov 2009 17:36:57 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Message-ID: <4AFF3109.40003@colbyconsulting.com> To get back to reality, I really need to process under 6 million records, and the vast majority will be merging two records into one. I will almost certainly pull about 10,000 records at a time, so I will be processing 5,000 people at a time. Merge two records, merge two more, do that ~ 5000 times, write back to SQL Server, load the next set of 10,000 records. Do that 600 times. I will see how many records can actually be loaded, perhaps 100,000 at a time would work and do that 60 times. Maybe break the process into threads. One thread watches the ADO recordsets in memory and keeps them filled, another thread performs the merge, another thread writes back to SQL Server. In any case there is simply no need to load all 6 million records at once. To get back to the original point however, while TSQL might be able to do what I need, I have serious doubts. Even if it could, trying to debug this kind of program written in TSQL would be a nightmare for me. Furthermore even TSQL could not do this on 6 million records at once. No sane person claims that any specific language is the best for everything, and for this kind of thing ADO and C# is probably a good choice. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Robert, > > Yes, DataSets are mainly targeted for working with small number of records > in memory - so the following is more a "theoretical" calculation: > From jwcolby at colbyconsulting.com Sat Nov 14 23:11:48 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 15 Nov 2009 00:11:48 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Message-ID: <4AFF8D94.8010409@colbyconsulting.com> http://arstechnica.com/hardware/news/2009/11/biography-solid-state-disk.ars John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Robert, > > Yes, DataSets are mainly targeted for working with small number of records > in memory - so the following is more a "theoretical" calculation: > > - JC has 64bit PC which allow to load in memory "practically unlimited" > volume of data - 64bit logical (process) address space could be as large as > 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 > (http://en.wikipedia.org/wiki/Exbibyte). > > - the speed of data transfer for 64bit Intel's processor - "the 200 MHz > McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers > 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); > > ----- > So "theoretically" with modern speedy harddisks JC can load 50 millions of > records into memory within minutes - within half a minute in the near future > when large enough flash-memory disks will become available and relatively > inexpensive? > > To load such large data volume it would be better to use SqlDataReader as it > keeps loaded data in a very compact form. And 50 million records long > SqlDataReader should be probably better split into several chunks... > > Again - this is just a "theoretical" consideration - in practice JC can > process his 50 million records long data table in chunks as he has a hash > field (tblHashPK), which can be used to load related records into memory: I > mean he can split tblHashPK table keeping hash values into several groups, > e.g.: > > 1 - 100,000 > 100,001 - 200,000 > ... > > and process each group joining its records to source 50 million records... > > -- > Shamil From jwcolby at colbyconsulting.com Mon Nov 16 07:42:52 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 16 Nov 2009 08:42:52 -0500 Subject: [dba-VB] Archives down Message-ID: <4B0156DC.5000407@colbyconsulting.com> I am unable to get at the archives for DBA-VB. Does anyone here use the ADO.Net Entity Framework? Why? Does anyone here specifically AVOID using the ADO.Net Entity Framework? Why? Thanks, -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Mon Nov 16 07:59:09 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 16 Nov 2009 16:59:09 +0300 Subject: [dba-VB] Archives down In-Reply-To: <4B0156DC.5000407@colbyconsulting.com> References: <4B0156DC.5000407@colbyconsulting.com> Message-ID: <00d901ca66c4$f94089e0$ebc19da0$@spb.ru> Hi John -- ADO.NET EF is a great thing. But maybe it will be wise to wait to use it in production GUI when VS2010 will be released then it will be possible to have natively bound WinForms and WPF Forms (and ASP.NET controls? And SilverLight controls), and where ADO.NET EF entities' mapping to db objects will be more flexible, and where it will be possible to have entity model driven(?) (database model) development... I have recently posted here a link to the following presentation: "ORM, EDM, ESQL, Entity Framework, LINQ to SQL, LINQ to Entities - Confused?" http://www.infoq.com/presentations/ORM-LINQ-Entity-Framework-Eric-Nelson I suppose it will answer most of your questions. BTW, our http://northwind.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=26600 project has a couple of ADO.NET EF samples - SPJWinFormsForEF_VS2008.zip source code, 838K, uploaded Jul 23 Application SPJWinFormsForEF_Executables.zip application, 203K, uploaded Jul 23 Thank you. -- Shamil P.S. BTW, I have mainly skipped "LINQ to SQL" - and I'm now happy (:)) I will not need to rewrite my applications as "LINQ to SQL" development is stopped there at MS AFAIU... -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, November 16, 2009 4:43 PM To: VBA Subject: [dba-VB] Archives down I am unable to get at the archives for DBA-VB. Does anyone here use the ADO.Net Entity Framework? Why? Does anyone here specifically AVOID using the ADO.Net Entity Framework? Why? Thanks, -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4611 (20091116) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From Gustav at cactus.dk Mon Nov 16 09:24:01 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 16 Nov 2009 16:24:01 +0100 Subject: [dba-VB] Archives down Message-ID: Hi John I like the concept and the design features, but have only used it for experiments and testing. It takes some time to get around it - I haven't concluded the tour yet ... /gustav >>> jwcolby at colbyconsulting.com 16-11-2009 14:42 >>> I am unable to get at the archives for DBA-VB. Does anyone here use the ADO.Net Entity Framework? Why? Does anyone here specifically AVOID using the ADO.Net Entity Framework? Why? Thanks, -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Tue Nov 17 17:04:12 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Tue, 17 Nov 2009 18:04:12 -0500 Subject: [dba-VB] First little applet Message-ID: <4B032BEC.9050508@colbyconsulting.com> Today I wrote my first C# applet. I have to import CSV files coming back from a processing house into SQL Server. These files had errors in one or more lines in the files. Given that there are tens of millions of lines in each file it isn't possible to just poke around and find the errors. So I wrote an applet that opens the CSV file, reads the header line and counts the " and , characters. Then line by line the applet reads the rest of the lines, comparing the counts to that header line counts. This found lines in the file to be imported where " characters had been dropped, which was causing the import wizard to error out at that point. So read, compare counts, write good lines to output file, write bad lines to an error file. Simple stuff really but it is my first real work done in C#. The speed of this kind of thing is pretty darned good too I thought. I was getting about 300K lines / second read / written. One file had 20.6 million lines, the other had 49 million lines. In the past I would have done this in VBA / Access. Of course it took me a tad longer to write in C# than it would have in VBA. -- John W. Colby www.ColbyConsulting.com From marklbreen at gmail.com Wed Nov 18 02:54:19 2009 From: marklbreen at gmail.com (Mark Breen) Date: Wed, 18 Nov 2009 08:54:19 +0000 Subject: [dba-VB] First little applet In-Reply-To: <4B032BEC.9050508@colbyconsulting.com> References: <4B032BEC.9050508@colbyconsulting.com> Message-ID: Congrats to you John, I suspect once you get a few more utils in your library, you will be flying from here on. good for you, Mark 2009/11/17 jwcolby > Today I wrote my first C# applet. I have to import CSV files coming back > from a processing house > into SQL Server. These files had errors in one or more lines in the files. > Given that there are > tens of millions of lines in each file it isn't possible to just poke > around and find the errors. > > So I wrote an applet that opens the CSV file, reads the header line and > counts the " and , > characters. Then line by line the applet reads the rest of the lines, > comparing the counts to that > header line counts. > > This found lines in the file to be imported where " characters had been > dropped, which was causing > the import wizard to error out at that point. > > So read, compare counts, write good lines to output file, write bad lines > to an error file. > > Simple stuff really but it is my first real work done in C#. > > The speed of this kind of thing is pretty darned good too I thought. I was > getting about 300K lines > / second read / written. One file had 20.6 million lines, the other had 49 > million lines. > > In the past I would have done this in VBA / Access. Of course it took me a > tad longer to write in > C# than it would have in VBA. > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Wed Nov 18 03:11:43 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 18 Nov 2009 10:11:43 +0100 Subject: [dba-VB] First little applet Message-ID: Hi John Great! The good thing is that code from this little applet can easily be incorporated in another project. /gustav >>> jwcolby at colbyconsulting.com 18-11-2009 00:04 >>> Today I wrote my first C# applet. I have to import CSV files coming back from a processing house into SQL Server. These files had errors in one or more lines in the files. Given that there are tens of millions of lines in each file it isn't possible to just poke around and find the errors. So I wrote an applet that opens the CSV file, reads the header line and counts the " and , characters. Then line by line the applet reads the rest of the lines, comparing the counts to that header line counts. This found lines in the file to be imported where " characters had been dropped, which was causing the import wizard to error out at that point. So read, compare counts, write good lines to output file, write bad lines to an error file. Simple stuff really but it is my first real work done in C#. The speed of this kind of thing is pretty darned good too I thought. I was getting about 300K lines / second read / written. One file had 20.6 million lines, the other had 49 million lines. In the past I would have done this in VBA / Access. Of course it took me a tad longer to write in C# than it would have in VBA. -- John W. Colby www.ColbyConsulting.com From andy at minstersystems.co.uk Wed Nov 18 05:23:43 2009 From: andy at minstersystems.co.uk (Andy Lacey) Date: Wed, 18 Nov 2009 12:23:43 +0100 Subject: [dba-VB] First little applet Message-ID: <200911181223.nAICNl4W018133@databaseadvisors.com> Congratulations John. Hope father and little applet are both doing well. Andy --------- Original Message -------- From: "Discussion concerning Visual Basic and related programming issues." To: "Discussion concerning Visual Basic and related programming issues." Subject: Re: [dba-VB] First little applet Date: 18/11/09 08:55 Congrats to you John, I suspect once you get a few more utils in your library, you will be flying from here on. good for you, Mark 2009/11/17 jwcolby > Today I wrote my first C# applet. I have to import CSV files coming back > from a processing house > into SQL Server. These files had errors in one or more lines in the files. > Given that there are > tens of millions of lines in each file it isn't possible to just poke > around and find the errors. > > So I wrote an applet that opens the CSV file, reads the header line and > counts the " and , > characters. Then line by line the applet reads the rest of the lines, > comparing the counts to that > header line counts. > > This found lines in the file to be imported where " characters had been > dropped, which was causing > the import wizard to error out at that point. > > So read, compare counts, write good lines to output file, write bad lines > to an error file. > > Simple stuff really but it is my first real work done in C#. > > The speed of this kind of thing is pretty darned good too I thought. I was > getting about 300K lines > / second read / written. One file had 20.6 million lines, the other had 49 > million lines. > > In the past I would have done this in VBA / Access. Of course it took me a > tad longer to write in > C# than it would have in VBA. > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com ________________________________________________ Message sent using UebiMiau 2.7.2 From jwcolby at colbyconsulting.com Wed Nov 18 06:45:16 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 07:45:16 -0500 Subject: [dba-VB] First little applet In-Reply-To: <200911181223.nAICNl4W018133@databaseadvisors.com> References: <200911181223.nAICNl4W018133@databaseadvisors.com> Message-ID: <4B03EC5C.4090101@colbyconsulting.com> ROTFL. Father is so proud of little applet. ;) John W. Colby www.ColbyConsulting.com Andy Lacey wrote: > Congratulations John. Hope father and little applet are both doing well. > > Andy > > > --------- Original Message -------- > From: "Discussion concerning Visual Basic and related programming issues." > > To: "Discussion concerning Visual Basic and related programming issues." > > Subject: Re: [dba-VB] First little applet > Date: 18/11/09 08:55 > > > Congrats to you John, > > I suspect once you get a few more utils in your library, you will be flying > from here on. > > good for you, > > Mark > > > > 2009/11/17 jwcolby > >> Today I wrote my first C# applet. I have to import CSV files coming back >> from a processing house >> into SQL Server. These files had errors in one or more lines in the files. >> Given that there are >> tens of millions of lines in each file it isn't possible to just poke >> around and find the errors. >> >> So I wrote an applet that opens the CSV file, reads the header line and >> counts the " and , >> characters. Then line by line the applet reads the rest of the lines, >> comparing the counts to that >> header line counts. >> >> This found lines in the file to be imported where " characters had been >> dropped, which was causing >> the import wizard to error out at that point. >> >> So read, compare counts, write good lines to output file, write bad lines >> to an error file. >> >> Simple stuff really but it is my first real work done in C#. >> >> The speed of this kind of thing is pretty darned good too I thought. I was >> getting about 300K lines >> / second read / written. One file had 20.6 million lines, the other had 49 >> million lines. >> >> In the past I would have done this in VBA / Access. Of course it took me a >> tad longer to write in >> C# than it would have in VBA. >> >> -- >> John W. Colby >> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > ________________________________________________ > Message sent using UebiMiau 2.7.2 > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Wed Nov 18 06:50:01 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 07:50:01 -0500 Subject: [dba-VB] SPAM-LOW: Re: First little applet In-Reply-To: References: Message-ID: <4B03ED79.5020704@colbyconsulting.com> I do imports from CSV for large files all of the time, and I have had similar failures in the past, so I will be keeping this applet for future use. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > Great! The good thing is that code from this little applet can easily be incorporated in another project. > > /gustav > > >>>> jwcolby at colbyconsulting.com 18-11-2009 00:04 >>> > Today I wrote my first C# applet. I have to import CSV files coming back from a processing house > into SQL Server. These files had errors in one or more lines in the files. Given that there are > tens of millions of lines in each file it isn't possible to just poke around and find the errors. > > So I wrote an applet that opens the CSV file, reads the header line and counts the " and , > characters. Then line by line the applet reads the rest of the lines, comparing the counts to that > header line counts. > > This found lines in the file to be imported where " characters had been dropped, which was causing > the import wizard to error out at that point. > > So read, compare counts, write good lines to output file, write bad lines to an error file. > > Simple stuff really but it is my first real work done in C#. > > The speed of this kind of thing is pretty darned good too I thought. I was getting about 300K lines > / second read / written. One file had 20.6 million lines, the other had 49 million lines. > > In the past I would have done this in VBA / Access. Of course it took me a tad longer to write in > C# than it would have in VBA. > From jwcolby at colbyconsulting.com Wed Nov 18 07:06:44 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 08:06:44 -0500 Subject: [dba-VB] Projects vs Solutions Message-ID: <4B03F164.6090601@colbyconsulting.com> I am hazy on how things are organized in .Net once you start building pieces of code. What is a project, what is a solution and how are they tied together. My role for the "DBFH" client involves lots of different tasks, for which there will be many different pieces. I would hope to tie the pieces together in one big application. For example I just wrote the little applet to cleanup CSV files, which will have to be used again. I do exports to Accuzip, imports from Accuzip, these two are closely related obviously. I build orders. I have to export data from tables for processing by external processing houses, which involves a very similar export / zip / FTP process. So as I build these processes in C#, how are they physically organized on the disk, and how are they tied back together into one program. From what I am reading, a Solution is the big picture, a Project is a small part of the Solution. But when I open VS2008 there is no "create a solution" or at least I don't see one. So how does one create a solution, and then look at just one project at a time inside of that solution? -- John W. Colby www.ColbyConsulting.com From Gustav at cactus.dk Wed Nov 18 07:17:08 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 18 Nov 2009 14:17:08 +0100 Subject: [dba-VB] Projects vs Solutions Message-ID: Hi John On the New Project pane at the bottom select the name for the (new) solution and mark Create directory for solution. That folder holds your projects of this solution. All projects of a solution are listed in the treeview (typically the right panel). Open only those projects you wish to. /gustav >>> jwcolby at colbyconsulting.com 18-11-2009 14:06 >>> I am hazy on how things are organized in .Net once you start building pieces of code. What is a project, what is a solution and how are they tied together. My role for the "DBFH" client involves lots of different tasks, for which there will be many different pieces. I would hope to tie the pieces together in one big application. For example I just wrote the little applet to cleanup CSV files, which will have to be used again. I do exports to Accuzip, imports from Accuzip, these two are closely related obviously. I build orders. I have to export data from tables for processing by external processing houses, which involves a very similar export / zip / FTP process. So as I build these processes in C#, how are they physically organized on the disk, and how are they tied back together into one program. From what I am reading, a Solution is the big picture, a Project is a small part of the Solution. But when I open VS2008 there is no "create a solution" or at least I don't see one. So how does one create a solution, and then look at just one project at a time inside of that solution? -- John W. Colby www.ColbyConsulting.com From cfoust at infostatsystems.com Wed Nov 18 10:12:41 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 10:12:41 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B03F164.6090601@colbyconsulting.com> References: <4B03F164.6090601@colbyconsulting.com> Message-ID: Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 5:07 AM To: VBA Subject: [dba-VB] Projects vs Solutions I am hazy on how things are organized in .Net once you start building pieces of code. What is a project, what is a solution and how are they tied together. My role for the "DBFH" client involves lots of different tasks, for which there will be many different pieces. I would hope to tie the pieces together in one big application. For example I just wrote the little applet to cleanup CSV files, which will have to be used again. I do exports to Accuzip, imports from Accuzip, these two are closely related obviously. I build orders. I have to export data from tables for processing by external processing houses, which involves a very similar export / zip / FTP process. So as I build these processes in C#, how are they physically organized on the disk, and how are they tied back together into one program. From what I am reading, a Solution is the big picture, a Project is a small part of the Solution. But when I open VS2008 there is no "create a solution" or at least I don't see one. So how does one create a solution, and then look at just one project at a time inside of that solution? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Wed Nov 18 11:57:56 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 12:57:56 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: <4B03F164.6090601@colbyconsulting.com> Message-ID: <4B0435A4.3050804@colbyconsulting.com> OK, this immediately causes issues referencing objects in another project. For example I have a "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I then need to be able to open a form in the other project where I gather information about the file to fix. The objects in that other project aren't visible in intellisense. "Normal" opening of a form is done with something like MyFormName Somename = new MyFormNname(); SomeName.Show(); Well... MyFormName is not valid in project main. While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. > > Charlotte Foust > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, November 18, 2009 5:07 AM > To: VBA > Subject: [dba-VB] Projects vs Solutions From cfoust at infostatsystems.com Wed Nov 18 12:16:55 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 12:16:55 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B0435A4.3050804@colbyconsulting.com> References: <4B03F164.6090601@colbyconsulting.com> <4B0435A4.3050804@colbyconsulting.com> Message-ID: That's where references and "Imports" come in. You need to be careful about project level references to avoid circular messes, so our UI and Data projects have references to the Configuration project, but not the other way around, and the UI has a reference to the Data project but not the other way around. You can always fully reference a class in another project in code starting with the Namespace, so if I need to use something in a project that isn't referenced in the current one, I can use Infostat.RIMDrill.Configuration.RIMConstants (for example) and have access to the methods and properties of RIMConstants even without a reference in the current project. Alternatively, I could use Imports Infostat.RIMDrill.Configuration in the declarations above my current class and get the same thing. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 9:58 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions OK, this immediately causes issues referencing objects in another project. For example I have a "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I then need to be able to open a form in the other project where I gather information about the file to fix. The objects in that other project aren't visible in intellisense. "Normal" opening of a form is done with something like MyFormName Somename = new MyFormNname(); SomeName.Show(); Well... MyFormName is not valid in project main. While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. > > Charlotte Foust > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, November 18, 2009 5:07 AM > To: VBA > Subject: [dba-VB] Projects vs Solutions _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From Gustav at cactus.dk Wed Nov 18 12:27:23 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 18 Nov 2009 19:27:23 +0100 Subject: [dba-VB] Projects vs Solutions Message-ID: Hi John In addition to the comments from Charlotte: Also, you can either add in your project Using ProjectOther; or state this explicitly ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); someName.Show(); /gustav >>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> OK, this immediately causes issues referencing objects in another project. For example I have a "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I then need to be able to open a form in the other project where I gather information about the file to fix. The objects in that other project aren't visible in intellisense. "Normal" opening of a form is done with something like MyFormName Somename = new MyFormNname(); SomeName.Show(); Well... MyFormName is not valid in project main. While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. > > Charlotte Foust From jwcolby at colbyconsulting.com Wed Nov 18 12:54:47 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 13:54:47 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: Message-ID: <4B0442F7.2070805@colbyconsulting.com> Using projOther doesn't work. When doing this is the project allowed to be a part of the solution? Is it even SUPPOSED TO BE a part of the solution. IOW are you supposed to create projects as separate entities and then reference them from other projects, or can you create SolutionMain / ProjMain / projOther1 / projOther2 etc. I have the latter situation (now). When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. This whole area seems poorly documented and rather messy. Lots of chatter on Google about how difficult it is to move a project and so forth. This is the kind of stuff I hate to even get involved in. I am trying to get work done but get sucked off into areas where it certainly looks like I could easily wreak havoc on working code. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > In addition to the comments from Charlotte: > > Also, you can either add in your project > > Using ProjectOther; > > or state this explicitly > > ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); > someName.Show(); > > /gustav > >>>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> > OK, this immediately causes issues referencing objects in another project. For example I have a > "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I > then need to be able to open a form in the other project where I gather information about the file > to fix. > > The objects in that other project aren't visible in intellisense. > > "Normal" opening of a form is done with something like > > MyFormName Somename = new MyFormNname(); > SomeName.Show(); > > Well... MyFormName is not valid in project main. > > While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. >> >> Charlotte Foust > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From cfoust at infostatsystems.com Wed Nov 18 13:03:55 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 13:03:55 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B0442F7.2070805@colbyconsulting.com> References: <4B0442F7.2070805@colbyconsulting.com> Message-ID: A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 10:55 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions Using projOther doesn't work. When doing this is the project allowed to be a part of the solution? Is it even SUPPOSED TO BE a part of the solution. IOW are you supposed to create projects as separate entities and then reference them from other projects, or can you create SolutionMain / ProjMain / projOther1 / projOther2 etc. I have the latter situation (now). When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. This whole area seems poorly documented and rather messy. Lots of chatter on Google about how difficult it is to move a project and so forth. This is the kind of stuff I hate to even get involved in. I am trying to get work done but get sucked off into areas where it certainly looks like I could easily wreak havoc on working code. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > In addition to the comments from Charlotte: > > Also, you can either add in your project > > Using ProjectOther; > > or state this explicitly > > ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); > someName.Show(); > > /gustav > >>>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> > OK, this immediately causes issues referencing objects in another > project. For example I have a "main" project, and a "CSVRepair" > project. The Main project form opens when I run the project but I > then need to be able to open a form in the other project where I gather information about the file to fix. > > The objects in that other project aren't visible in intellisense. > > "Normal" opening of a form is done with something like > > MyFormName Somename = new MyFormNname(); SomeName.Show(); > > Well... MyFormName is not valid in project main. > > While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. >> >> Charlotte Foust > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From Gustav at cactus.dk Wed Nov 18 13:30:04 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 18 Nov 2009 20:30:04 +0100 Subject: [dba-VB] Projects vs Solutions Message-ID: Hi John Sorry, I read it as if ProjectOther was in the same (current) solution. If not, as Charlotte explains, then in the current solution reference the project from the other solution where it is situated. You will be given the choice to either leave the files of the project or to import them. What to chose is a question of maintenance. /gustav >>> jwcolby at colbyconsulting.com 18-11-2009 19:54 >>> Using projOther doesn't work. When doing this is the project allowed to be a part of the solution? Is it even SUPPOSED TO BE a part of the solution. IOW are you supposed to create projects as separate entities and then reference them from other projects, or can you create SolutionMain / ProjMain / projOther1 / projOther2 etc. I have the latter situation (now). When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. This whole area seems poorly documented and rather messy. Lots of chatter on Google about how difficult it is to move a project and so forth. This is the kind of stuff I hate to even get involved in. I am trying to get work done but get sucked off into areas where it certainly looks like I could easily wreak havoc on working code. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > In addition to the comments from Charlotte: > > Also, you can either add in your project > > Using ProjectOther; > > or state this explicitly > > ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); > someName.Show(); > > /gustav > >>>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> > OK, this immediately causes issues referencing objects in another project. For example I have a > "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I > then need to be able to open a form in the other project where I gather information about the file > to fix. > > The objects in that other project aren't visible in intellisense. > > "Normal" opening of a form is done with something like > > MyFormName Somename = new MyFormNname(); > SomeName.Show(); > > Well... MyFormName is not valid in project main. > > While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. >> >> Charlotte Foust From shamil at smsconsulting.spb.ru Wed Nov 18 13:35:05 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Wed, 18 Nov 2009 22:35:05 +0300 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B0442F7.2070805@colbyconsulting.com> References: <4B0442F7.2070805@colbyconsulting.com> Message-ID: <000501ca6886$3c5fd420$b51f7c60$@spb.ru> Hi John -- In fact referencing other projects works very well in .NET, and it's clean and very useful: just note that 'using' directive is to refer to *namespaces* not to projects. If you right-click a project in Solution Explorer, and then select 'Properties' you can find that every project has 'Default Namespace', which is usually the same as project name but this is not necessary. When I'm developing my solution and I need "to get work done" I do sometimes postpone creating new projects within a solution - I just create subfolders in my main project's treeview and I put my code to be later moved to other projects into those subfolders, and I keep namespace for those subfolders different from the default project's namespace - check it out... You'll soon get fluent with that namespaces and projects stuff and then you'll be "cooking" new projects seamlessly - it's usual story to have 10+ projects within .NET solutions... <<< When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. >>> Do you have 'ProjOther' within your solution? Did you set reference to 'ProjOther'? Does 'ProjOther''s default namespace is 'ProjOther'? 'using' directive should be started with lowercase letter.... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 9:55 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions Using projOther doesn't work. When doing this is the project allowed to be a part of the solution? Is it even SUPPOSED TO BE a part of the solution. IOW are you supposed to create projects as separate entities and then reference them from other projects, or can you create SolutionMain / ProjMain / projOther1 / projOther2 etc. I have the latter situation (now). When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. This whole area seems poorly documented and rather messy. Lots of chatter on Google about how difficult it is to move a project and so forth. This is the kind of stuff I hate to even get involved in. I am trying to get work done but get sucked off into areas where it certainly looks like I could easily wreak havoc on working code. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > In addition to the comments from Charlotte: > > Also, you can either add in your project > > Using ProjectOther; > > or state this explicitly > > ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); > someName.Show(); > > /gustav > >>>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> > OK, this immediately causes issues referencing objects in another project. For example I have a > "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I > then need to be able to open a form in the other project where I gather information about the file > to fix. > > The objects in that other project aren't visible in intellisense. > > "Normal" opening of a form is done with something like > > MyFormName Somename = new MyFormNname(); > SomeName.Show(); > > Well... MyFormName is not valid in project main. > > While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. >> >> Charlotte Foust > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4618 (20091118) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4618 (20091118) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Wed Nov 18 13:40:52 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 14:40:52 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: <4B0442F7.2070805@colbyconsulting.com> Message-ID: <4B044DC4.2050304@colbyconsulting.com> OK, let's back up a minute. I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". Except that: a) I really want the Solution to be InfoEngine. b) I would PREFER that the other projects be physically stored under InfoEngine. c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. d) Each of these "projects" contains a solution file. So what is it, a solution or a project? Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. > > Charlotte Foust From shamil at smsconsulting.spb.ru Wed Nov 18 14:04:58 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Wed, 18 Nov 2009 23:04:58 +0300 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B044DC4.2050304@colbyconsulting.com> References: <4B0442F7.2070805@colbyconsulting.com> <4B044DC4.2050304@colbyconsulting.com> Message-ID: <000c01ca688a$68c16200$3a442600$@spb.ru> John -- Just make an experiment with a test solution from scratch: - start VS; - create C# Console project - TestConsole - that will create a project TestConsole and a solution - TestConsole; - File->New->Project... - create C# Class Library project - MyClassLib1; - File->New->Project... - create C# Class Library project - MyClassLib2; - ... - Right-Click TestConsole project in Solution Explorer -> Add Reference -> [Projects] (third tab) -> Select MyClassLib1; - ... - now in TestConsole Program.cs you can type: using MyClassLib1; and you'll have Class1 (from MyClassLib1 - MyclassLib1.Class1) available to Intellisense.... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 10:41 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions OK, let's back up a minute. I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". Except that: a) I really want the Solution to be InfoEngine. b) I would PREFER that the other projects be physically stored under InfoEngine. c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. d) Each of these "projects" contains a solution file. So what is it, a solution or a project? Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. > > Charlotte Foust __________ Information from ESET NOD32 Antivirus, version of virus signature database 4618 (20091118) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From cfoust at infostatsystems.com Wed Nov 18 14:49:22 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 14:49:22 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B044DC4.2050304@colbyconsulting.com> References: <4B0442F7.2070805@colbyconsulting.com> <4B044DC4.2050304@colbyconsulting.com> Message-ID: John, Susan and I ran into this when we wrote a beginner's article. In tools -> Options under Project and Solutions, make sure the Always Show Solution checkbox is checked. Otherwise, you only see a solution and you keep creating separate solutions for each project. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 11:41 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions OK, let's back up a minute. I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". Except that: a) I really want the Solution to be InfoEngine. b) I would PREFER that the other projects be physically stored under InfoEngine. c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. d) Each of these "projects" contains a solution file. So what is it, a solution or a project? Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. > > Charlotte Foust _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From cfoust at infostatsystems.com Wed Nov 18 18:21:50 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 18:21:50 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: <4B0442F7.2070805@colbyconsulting.com> <4B044DC4.2050304@colbyconsulting.com> Message-ID: Oops, I meant you only see a PROJECT unles the Always Show Solution checkbox is checked! Sorry!! Charlotte -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Charlotte Foust Sent: Wednesday, November 18, 2009 12:49 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions John, Susan and I ran into this when we wrote a beginner's article. In tools -> Options under Project and Solutions, make sure the Always Show Solution checkbox is checked. Otherwise, you only see a solution and you keep creating separate solutions for each project. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 11:41 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions OK, let's back up a minute. I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". Except that: a) I really want the Solution to be InfoEngine. b) I would PREFER that the other projects be physically stored under InfoEngine. c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. d) Each of these "projects" contains a solution file. So what is it, a solution or a project? Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. > > Charlotte Foust _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Wed Nov 18 20:14:37 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 21:14:37 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: <4B0442F7.2070805@colbyconsulting.com> <4B044DC4.2050304@colbyconsulting.com> Message-ID: <4B04AA0D.5030107@colbyconsulting.com> Creating a reference to the other project solved the immediate problem of referencing objects in the other project. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > Oops, I meant you only see a PROJECT unles the Always Show Solution checkbox is checked! Sorry!! > > Charlotte > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Charlotte Foust > Sent: Wednesday, November 18, 2009 12:49 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Projects vs Solutions > > John, > > Susan and I ran into this when we wrote a beginner's article. In tools -> Options under Project and Solutions, make sure the Always Show Solution checkbox is checked. Otherwise, you only see a solution and you keep creating separate solutions for each project. > > Charlotte Foust > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, November 18, 2009 11:41 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Projects vs Solutions > > OK, let's back up a minute. > > I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. > > Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". > > Except that: > > a) I really want the Solution to be InfoEngine. > b) I would PREFER that the other projects be physically stored under InfoEngine. > c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. > d) Each of these "projects" contains a solution file. So what is it, a solution or a project? > > Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? > Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? > > By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". > > And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. >> >> Charlotte Foust > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Thu Nov 19 03:41:50 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Thu, 19 Nov 2009 10:41:50 +0100 Subject: [dba-VB] Projects vs Solutions Message-ID: Hi John This raises the question if your coming applet will be the new little applet or will become a little bigger little applet from day one. Does a little applet grow up? /gustav >>> jwcolby at colbyconsulting.com 19-11-2009 03:14 >>> Creating a reference to the other project solved the immediate problem of referencing objects in the other project. John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Thu Nov 19 07:10:03 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 19 Nov 2009 08:10:03 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: Message-ID: <4B0543AB.5080700@colbyconsulting.com> Gustav, Well, the point is that I am doing my job and suddenly I have to do something for the client that I have no solution for. This latest was a good example. I tried to import three files coming back from a processing vendor and one imported, the other two failed. So I write an applet that scans through the file sorting lines of the file into two output files, good and bad. When I am done the remaining "good" file now imports. What do I do with this applet? One thing I can do is to just leave it where it is and hope I don't lose it, because sure as the sun rises I will need to do this again. Or do I create a bigger application, build a menu in it, and place this applet on the menu? Does this Fixit applet grow? Conceivably, if I find some file that isn't fixed (or problem found) as easily as these files were, then this applet could grow to include different tests. This applet has its own form with a set of text boxes to display paths / files, buttons to select folders and files and a button to perform the cleanup. I probably will add another button to open the resulting "bad" file in an editor so that I can see and fix the problems in the data. Perhaps a button to append the fixed lines back to the end of the "good" file before the import. Stuff like that. So applets can grow, and the main application grows as I write new applets to perform various processing tasks, and those new applets get added into the main application. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > This raises the question if your coming applet will be the new little applet or will become a little bigger little applet from day one. Does a little applet grow up? > > /gustav > > >>>> jwcolby at colbyconsulting.com 19-11-2009 03:14 >>> > Creating a reference to the other project solved the immediate problem of referencing objects in the > other project. > > John W. Colby > www.ColbyConsulting.com From marklbreen at gmail.com Fri Nov 20 03:05:11 2009 From: marklbreen at gmail.com (Mark Breen) Date: Fri, 20 Nov 2009 09:05:11 +0000 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B0543AB.5080700@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> Message-ID: Hello John, While I am not qualified to comment really on this subject, I am waiting / hoping that someone will let you now how you can move 90% of your applet to a DLL project, and then from your main program you will just reference the DLL and consume the functionality within that DLL. With that approach, you could start your library of the new Colby C# Framework, albeit a humble beginning. Am I correct in assuming that with this approach you would compile your applet to to a DLL, and then you main program, which could be a winform or console app, could call applet.InputFilePathSet, applet.OutPutFileSet, and then applet.ParseFile etc etc. Let I say, I am not qualified to advise on this, but perhaps some others would like to, thanks Mark 2009/11/19 jwcolby > Gustav, > > Well, the point is that I am doing my job and suddenly I have to do > something for the client that I > have no solution for. This latest was a good example. I tried to import > three files coming back > from a processing vendor and one imported, the other two failed. So I > write an applet that scans > through the file sorting lines of the file into two output files, good and > bad. When I am done the > remaining "good" file now imports. > > What do I do with this applet? One thing I can do is to just leave it > where it is and hope I don't > lose it, because sure as the sun rises I will need to do this again. Or do > I create a bigger > application, build a menu in it, and place this applet on the menu? > > Does this Fixit applet grow? Conceivably, if I find some file that isn't > fixed (or problem found) > as easily as these files were, then this applet could grow to include > different tests. This applet > has its own form with a set of text boxes to display paths / files, buttons > to select folders and > files and a button to perform the cleanup. I probably will add another > button to open the resulting > "bad" file in an editor so that I can see and fix the problems in the data. > Perhaps a button to > append the fixed lines back to the end of the "good" file before the > import. Stuff like that. > > So applets can grow, and the main application grows as I write new applets > to perform various > processing tasks, and those new applets get added into the main > application. > > John W. Colby > www.ColbyConsulting.com > > > Gustav Brock wrote: > > Hi John > > > > This raises the question if your coming applet will be the new little > applet or will become a little bigger little applet from day one. Does a > little applet grow up? > > > > /gustav > > > > > >>>> jwcolby at colbyconsulting.com 19-11-2009 03:14 >>> > > Creating a reference to the other project solved the immediate problem of > referencing objects in the > > other project. > > > > John W. Colby > > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 20 06:26:10 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 20 Nov 2009 07:26:10 -0500 Subject: [dba-VB] SPAM-LOW: Re: Projects vs Solutions In-Reply-To: References: <4B0543AB.5080700@colbyconsulting.com> Message-ID: <4B068AE2.9060301@colbyconsulting.com> The blind leading the blind here. 1) I built a main application 2) I referenced the existing file repair applet from the main application (project). 3) I set a using statement. It appears that you have to both reference it and then use the "using" statement. 4) I can now open forms out in the file repair applet from the main application. 5) I physically moved the file repair applet underneath the main application directory. 6) I changed the directory for the applet and it just worked. That was fairly easy. From this point on I "Add Project" to the main solution. I have added a class project to wrap the DMO. In case you haven't discovered it, the DMO is a real cool SQL Server Management Object API that allows you to see and manage database objects. I am just getting into it but it allows me to reference a server object, then see the database collection. The each database object has a table collection, the table object has a fields collection etc. Everything you can see and manage in the SQL Server management studio you can (apparently) see and manage from the SMO from C#. An example of what this does for me is allows me to see all of the databases in a server, and thus populate a combo with their names. Selecting a database from the combo I can see and fill a combo with the names of the tables. Selecting a database and a specific table I can then can then run my stored procedures that export that table in that database to CSV files. That kind of stuff is what I do a lot of and what the big application will manage. John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > While I am not qualified to comment really on this subject, I am waiting / > hoping that someone will let you now how you can move 90% of your applet to > a DLL project, and then from your main program you will just reference the > DLL and consume the functionality within that DLL. > > With that approach, you could start your library of the new Colby C# > Framework, albeit a humble beginning. > > Am I correct in assuming that with this approach you would compile your > applet to to a DLL, and then you main program, which could be a winform or > console app, could call applet.InputFilePathSet, applet.OutPutFileSet, and > then applet.ParseFile etc etc. > > Let I say, I am not qualified to advise on this, but perhaps some others > would like to, > > thanks > > Mark > > > 2009/11/19 jwcolby > >> Gustav, >> >> Well, the point is that I am doing my job and suddenly I have to do >> something for the client that I >> have no solution for. This latest was a good example. I tried to import >> three files coming back >> from a processing vendor and one imported, the other two failed. So I >> write an applet that scans >> through the file sorting lines of the file into two output files, good and >> bad. When I am done the >> remaining "good" file now imports. >> >> What do I do with this applet? One thing I can do is to just leave it >> where it is and hope I don't >> lose it, because sure as the sun rises I will need to do this again. Or do >> I create a bigger >> application, build a menu in it, and place this applet on the menu? >> >> Does this Fixit applet grow? Conceivably, if I find some file that isn't >> fixed (or problem found) >> as easily as these files were, then this applet could grow to include >> different tests. This applet >> has its own form with a set of text boxes to display paths / files, buttons >> to select folders and >> files and a button to perform the cleanup. I probably will add another >> button to open the resulting >> "bad" file in an editor so that I can see and fix the problems in the data. >> Perhaps a button to >> append the fixed lines back to the end of the "good" file before the >> import. Stuff like that. >> >> So applets can grow, and the main application grows as I write new applets >> to perform various >> processing tasks, and those new applets get added into the main >> application. >> >> John W. Colby >> www.ColbyConsulting.com >> >> >> Gustav Brock wrote: >>> Hi John >>> >>> This raises the question if your coming applet will be the new little >> applet or will become a little bigger little applet from day one. Does a >> little applet grow up? >>> /gustav >>> >>> >>>>>> jwcolby at colbyconsulting.com 19-11-2009 03:14 >>> >>> Creating a reference to the other project solved the immediate problem of >> referencing objects in the >>> other project. >>> >>> John W. Colby >>> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 20 09:35:42 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 20 Nov 2009 10:35:42 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B068AE2.9060301@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> Message-ID: <4B06B74E.9080909@colbyconsulting.com> The object I am referring to is the SMO or SQL Server Management object. In order to use it you have to add several references: Microsoft.SQLServer.SMO Microsoft.SQLServer.SMOEnum Microsoft.SQLServer.SQLEnum Microsoft.SQLServer.ConnectionInfo then in the classes using the SMO you have to do using Microsoft.SqlServer.Management.Smo; After that you can do things like: Server Svr; Svr = new Server("MyServerName") foreach (Database in Svr.Databases) { //Etc. // } This allows you to iterate collections of database objects, using them directly or just pulling the names out (as I did) to populate lists, combos, collections etc. As I mentioned, once you have a database object you can manipulate it. I am just starting to learn what I can do with this API but it looks pretty powerful. John W. Colby www.ColbyConsulting.com jwcolby wrote: > The blind leading the blind here. > > 1) I built a main application > 2) I referenced the existing file repair applet from the main application (project). > 3) I set a using statement. It appears that you have to both reference it and then use the "using" > statement. > 4) I can now open forms out in the file repair applet from the main application. > 5) I physically moved the file repair applet underneath the main application directory. > 6) I changed the directory for the applet and it just worked. That was fairly easy. > > From this point on I "Add Project" to the main solution. I have added a class project to wrap the > DMO. In case you haven't discovered it, the DMO is a real cool SQL Server Management Object API > that allows you to see and manage database objects. I am just getting into it but it allows me to > reference a server object, then see the database collection. The each database object has a table > collection, the table object has a fields collection etc. Everything you can see and manage in the > SQL Server management studio you can (apparently) see and manage from the SMO from C#. > > An example of what this does for me is allows me to see all of the databases in a server, and thus > populate a combo with their names. Selecting a database from the combo I can see and fill a combo > with the names of the tables. Selecting a database and a specific table I can then can then run my > stored procedures that export that table in that database to CSV files. > > That kind of stuff is what I do a lot of and what the big application will manage. > > John W. Colby > www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Fri Nov 20 14:18:40 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 20 Nov 2009 23:18:40 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B06B74E.9080909@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> Message-ID: <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> Hi John -- I'm wondering what's the use of that Microsoft.SQLServer.* when you have to have your customer tasks done first of all? Why not just use (static) custom settings to point to different SQL servers etc.? I suppose Microsoft.SQLServer.* is good for companies like http://www.red-gate.com/ for them to develop their tools used worldwide, and I wonder what customers' business tasks can be solved by using Microsoft.SQLServer.* ? Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 20, 2009 6:36 PM To: Discussion concerning Visual Basic and related programming issues. Subject: [dba-VB] SMO was Projects vs Solutions The object I am referring to is the SMO or SQL Server Management object. In order to use it you have to add several references: Microsoft.SQLServer.SMO Microsoft.SQLServer.SMOEnum Microsoft.SQLServer.SQLEnum Microsoft.SQLServer.ConnectionInfo then in the classes using the SMO you have to do using Microsoft.SqlServer.Management.Smo; After that you can do things like: Server Svr; Svr = new Server("MyServerName") foreach (Database in Svr.Databases) { //Etc. // } This allows you to iterate collections of database objects, using them directly or just pulling the names out (as I did) to populate lists, combos, collections etc. As I mentioned, once you have a database object you can manipulate it. I am just starting to learn what I can do with this API but it looks pretty powerful. John W. Colby www.ColbyConsulting.com jwcolby wrote: > The blind leading the blind here. > > 1) I built a main application > 2) I referenced the existing file repair applet from the main application (project). > 3) I set a using statement. It appears that you have to both reference it and then use the "using" > statement. > 4) I can now open forms out in the file repair applet from the main application. > 5) I physically moved the file repair applet underneath the main application directory. > 6) I changed the directory for the applet and it just worked. That was fairly easy. > > From this point on I "Add Project" to the main solution. I have added a class project to wrap the > DMO. In case you haven't discovered it, the DMO is a real cool SQL Server Management Object API > that allows you to see and manage database objects. I am just getting into it but it allows me to > reference a server object, then see the database collection. The each database object has a table > collection, the table object has a fields collection etc. Everything you can see and manage in the > SQL Server management studio you can (apparently) see and manage from the SMO from C#. > > An example of what this does for me is allows me to see all of the databases in a server, and thus > populate a combo with their names. Selecting a database from the combo I can see and fill a combo > with the names of the tables. Selecting a database and a specific table I can then can then run my > stored procedures that export that table in that database to CSV files. > > That kind of stuff is what I do a lot of and what the big application will manage. > > John W. Colby > www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4624 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4625 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 20 14:30:47 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 20 Nov 2009 15:30:47 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> Message-ID: <4B06FC77.6040008@colbyconsulting.com> I don't understand the question. What do you mean by Microsoft.SQLServer.* I do have multiple servers running SQL Server. The default is Azul but I may need to reference Stonehenge. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > I'm wondering what's the use of that Microsoft.SQLServer.* when you have to > have your customer tasks done first of all? > Why not just use (static) custom settings to point to different SQL servers > etc.? > > I suppose Microsoft.SQLServer.* is good for companies like > http://www.red-gate.com/ for them to develop their tools used worldwide, and > I wonder what customers' business tasks can be solved by using > Microsoft.SQLServer.* ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 20, 2009 6:36 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: [dba-VB] SMO was Projects vs Solutions > > The object I am referring to is the SMO or SQL Server Management object. In > order to use it you > have to add several references: > > Microsoft.SQLServer.SMO > Microsoft.SQLServer.SMOEnum > Microsoft.SQLServer.SQLEnum > Microsoft.SQLServer.ConnectionInfo > > then in the classes using the SMO you have to do > > using Microsoft.SqlServer.Management.Smo; > > After that you can do things like: > > Server Svr; > Svr = new Server("MyServerName") > > foreach (Database in Svr.Databases) > { > //Etc. > // > } > > This allows you to iterate collections of database objects, using them > directly or just pulling the > names out (as I did) to populate lists, combos, collections etc. > > As I mentioned, once you have a database object you can manipulate it. I am > just starting to learn > what I can do with this API but it looks pretty powerful. > > John W. Colby > www.ColbyConsulting.com > > > jwcolby wrote: >> The blind leading the blind here. >> >> 1) I built a main application >> 2) I referenced the existing file repair applet from the main application > (project). >> 3) I set a using statement. It appears that you have to both reference it > and then use the "using" >> statement. >> 4) I can now open forms out in the file repair applet from the main > application. >> 5) I physically moved the file repair applet underneath the main > application directory. >> 6) I changed the directory for the applet and it just worked. That was > fairly easy. >> From this point on I "Add Project" to the main solution. I have added a > class project to wrap the >> DMO. In case you haven't discovered it, the DMO is a real cool SQL Server > Management Object API >> that allows you to see and manage database objects. I am just getting > into it but it allows me to >> reference a server object, then see the database collection. The each > database object has a table >> collection, the table object has a fields collection etc. Everything you > can see and manage in the >> SQL Server management studio you can (apparently) see and manage from the > SMO from C#. >> An example of what this does for me is allows me to see all of the > databases in a server, and thus >> populate a combo with their names. Selecting a database from the combo I > can see and fill a combo >> with the names of the tables. Selecting a database and a specific table I > can then can then run my >> stored procedures that export that table in that database to CSV files. >> >> That kind of stuff is what I do a lot of and what the big application will > manage. >> John W. Colby >> www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4624 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4625 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Nov 20 14:50:35 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 20 Nov 2009 23:50:35 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B06FC77.6040008@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> Message-ID: <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> Hi John, By Microsoft.SQLServer.* I mean all the namespaces you mentioned > Microsoft.SQLServer.SMO > Microsoft.SQLServer.SMOEnum > Microsoft.SQLServer.SQLEnum > Microsoft.SQLServer.ConnectionInfo As well as their classes and those classes enumerations, methods, properties, events etc. <<< I do have multiple servers running SQL Server. The default is Azul but I may need to reference Stonehenge. >>> But do you need to change the reference(/connection string information) to SQL Server after you have your program started? Do you need to enumerate SQL Server objects to solve your customers' tasks? Or maybe you have developed a kind of code generator, which does need to enumerate MS SQL Server objects and some of their properties to generate some custom code? - I can understand the latter - if you have to develop a lot of repetitive custom code then it's often useful to generate it... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 20, 2009 11:31 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SMO was Projects vs Solutions I don't understand the question. What do you mean by Microsoft.SQLServer.* I do have multiple servers running SQL Server. The default is Azul but I may need to reference Stonehenge. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > I'm wondering what's the use of that Microsoft.SQLServer.* when you have to > have your customer tasks done first of all? > Why not just use (static) custom settings to point to different SQL servers > etc.? > > I suppose Microsoft.SQLServer.* is good for companies like > http://www.red-gate.com/ for them to develop their tools used worldwide, and > I wonder what customers' business tasks can be solved by using > Microsoft.SQLServer.* ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 20, 2009 6:36 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: [dba-VB] SMO was Projects vs Solutions > > The object I am referring to is the SMO or SQL Server Management object. In > order to use it you > have to add several references: > > Microsoft.SQLServer.SMO > Microsoft.SQLServer.SMOEnum > Microsoft.SQLServer.SQLEnum > Microsoft.SQLServer.ConnectionInfo > > then in the classes using the SMO you have to do > > using Microsoft.SqlServer.Management.Smo; > > After that you can do things like: > > Server Svr; > Svr = new Server("MyServerName") > > foreach (Database in Svr.Databases) > { > //Etc. > // > } > > This allows you to iterate collections of database objects, using them > directly or just pulling the > names out (as I did) to populate lists, combos, collections etc. > > As I mentioned, once you have a database object you can manipulate it. I am > just starting to learn > what I can do with this API but it looks pretty powerful. > > John W. Colby > www.ColbyConsulting.com > > > jwcolby wrote: >> The blind leading the blind here. >> >> 1) I built a main application >> 2) I referenced the existing file repair applet from the main application > (project). >> 3) I set a using statement. It appears that you have to both reference it > and then use the "using" >> statement. >> 4) I can now open forms out in the file repair applet from the main > application. >> 5) I physically moved the file repair applet underneath the main > application directory. >> 6) I changed the directory for the applet and it just worked. That was > fairly easy. >> From this point on I "Add Project" to the main solution. I have added a > class project to wrap the >> DMO. In case you haven't discovered it, the DMO is a real cool SQL Server > Management Object API >> that allows you to see and manage database objects. I am just getting > into it but it allows me to >> reference a server object, then see the database collection. The each > database object has a table >> collection, the table object has a fields collection etc. Everything you > can see and manage in the >> SQL Server management studio you can (apparently) see and manage from the > SMO from C#. >> An example of what this does for me is allows me to see all of the > databases in a server, and thus >> populate a combo with their names. Selecting a database from the combo I > can see and fill a combo >> with the names of the tables. Selecting a database and a specific table I > can then can then run my >> stored procedures that export that table in that database to CSV files. >> >> That kind of stuff is what I do a lot of and what the big application will > manage. >> John W. Colby >> www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4625 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 20 20:05:57 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 20 Nov 2009 21:05:57 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> Message-ID: <4B074B05.6000406@colbyconsulting.com> That is used for manipulating the SMO object and SQL Server. I am just learning this stuff and do what the demos tell me to do. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > By Microsoft.SQLServer.* I mean all the namespaces you mentioned > >> Microsoft.SQLServer.SMO >> Microsoft.SQLServer.SMOEnum >> Microsoft.SQLServer.SQLEnum >> Microsoft.SQLServer.ConnectionInfo > > As well as their classes and those classes enumerations, methods, > properties, events etc. > > <<< > I do have multiple servers running SQL Server. > The default is Azul but I may need to reference > Stonehenge. > But do you need to change the reference(/connection string information) to > SQL Server after you have your program started? > Do you need to enumerate SQL Server objects to solve your customers' tasks? > Or maybe you have developed a kind of code generator, which does need to > enumerate MS SQL Server objects and some of their properties to generate > some custom code? - I can understand the latter - if you have to develop a > lot of repetitive custom code then it's often useful to generate it... > > Thank you. > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 20, 2009 11:31 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > I don't understand the question. > > What do you mean by Microsoft.SQLServer.* > > I do have multiple servers running SQL Server. The default is Azul but I > may need to reference > Stonehenge. > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> I'm wondering what's the use of that Microsoft.SQLServer.* when you have > to >> have your customer tasks done first of all? >> Why not just use (static) custom settings to point to different SQL > servers >> etc.? >> >> I suppose Microsoft.SQLServer.* is good for companies like >> http://www.red-gate.com/ for them to develop their tools used worldwide, > and >> I wonder what customers' business tasks can be solved by using >> Microsoft.SQLServer.* ? >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: Friday, November 20, 2009 6:36 PM >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: [dba-VB] SMO was Projects vs Solutions >> >> The object I am referring to is the SMO or SQL Server Management object. > In >> order to use it you >> have to add several references: >> >> Microsoft.SQLServer.SMO >> Microsoft.SQLServer.SMOEnum >> Microsoft.SQLServer.SQLEnum >> Microsoft.SQLServer.ConnectionInfo >> >> then in the classes using the SMO you have to do >> >> using Microsoft.SqlServer.Management.Smo; >> >> After that you can do things like: >> >> Server Svr; >> Svr = new Server("MyServerName") >> >> foreach (Database in Svr.Databases) >> { >> //Etc. >> // >> } >> >> This allows you to iterate collections of database objects, using them >> directly or just pulling the >> names out (as I did) to populate lists, combos, collections etc. >> >> As I mentioned, once you have a database object you can manipulate it. I > am >> just starting to learn >> what I can do with this API but it looks pretty powerful. >> >> John W. Colby >> www.ColbyConsulting.com >> >> >> jwcolby wrote: >>> The blind leading the blind here. >>> >>> 1) I built a main application >>> 2) I referenced the existing file repair applet from the main application >> (project). >>> 3) I set a using statement. It appears that you have to both reference > it >> and then use the "using" >>> statement. >>> 4) I can now open forms out in the file repair applet from the main >> application. >>> 5) I physically moved the file repair applet underneath the main >> application directory. >>> 6) I changed the directory for the applet and it just worked. That was >> fairly easy. >>> From this point on I "Add Project" to the main solution. I have added a >> class project to wrap the >>> DMO. In case you haven't discovered it, the DMO is a real cool SQL > Server >> Management Object API >>> that allows you to see and manage database objects. I am just getting >> into it but it allows me to >>> reference a server object, then see the database collection. The each >> database object has a table >>> collection, the table object has a fields collection etc. Everything you >> can see and manage in the >>> SQL Server management studio you can (apparently) see and manage from the >> SMO from C#. >>> An example of what this does for me is allows me to see all of the >> databases in a server, and thus >>> populate a combo with their names. Selecting a database from the combo I >> can see and fill a combo >>> with the names of the tables. Selecting a database and a specific table > I >> can then can then run my >>> stored procedures that export that table in that database to CSV files. >>> >>> That kind of stuff is what I do a lot of and what the big application > will >> manage. >>> John W. Colby >>> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4625 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Nov 21 08:58:31 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 21 Nov 2009 17:58:31 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B074B05.6000406@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> Message-ID: <002001ca6abb$186807e0$493817a0$@spb.ru> Hi John -- Yes, I understand that. I'm wondering what for do you learning this stuff? - Just out of curiosity or do you plan to implement some custom applications using Microsoft.SQLServer.* ? Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 21, 2009 5:06 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SMO was Projects vs Solutions That is used for manipulating the SMO object and SQL Server. I am just learning this stuff and do what the demos tell me to do. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > By Microsoft.SQLServer.* I mean all the namespaces you mentioned > >> Microsoft.SQLServer.SMO >> Microsoft.SQLServer.SMOEnum >> Microsoft.SQLServer.SQLEnum >> Microsoft.SQLServer.ConnectionInfo > > As well as their classes and those classes enumerations, methods, > properties, events etc. > > <<< > I do have multiple servers running SQL Server. > The default is Azul but I may need to reference > Stonehenge. > But do you need to change the reference(/connection string information) to > SQL Server after you have your program started? > Do you need to enumerate SQL Server objects to solve your customers' tasks? > Or maybe you have developed a kind of code generator, which does need to > enumerate MS SQL Server objects and some of their properties to generate > some custom code? - I can understand the latter - if you have to develop a > lot of repetitive custom code then it's often useful to generate it... > > Thank you. > > -- > Shamil > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4626 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 21 09:42:12 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 10:42:12 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <002001ca6abb$186807e0$493817a0$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> Message-ID: <4B080A54.4030609@colbyconsulting.com> > or do you plan to implement some custom applications using Microsoft.SQLServer.* ? Exactly. My work for one of my clients is heavy manipulation of SQL Server. I routinely export files to an external application running on virtual machines on another server, then wait for the files to process, then import them back in to SQL server. These two processes together take about 10 stored procedures. The export / import has to occur on (currently) a half dozen large databases monthly. However as I obtain more lists that number goes up. Each process exports to it's own directory path, then imports back from a parallel directory path. While the stored procedures eliminate much of the manual labor, there is plenty left. Plus logging of everything etc. I also routinely process orders. These orders take about a dozen stored procedures and also use the exact same set of stored procedures as the above process, i.e. the export / import process is also used on each and every order. So there is just a ton of very repetitive work that has to be done, but which database / table varies from run to run. To this point I have used Access / VBA to automate these tasks. Now I am moving to C# for this automation work. SMO allows me to manipulate SQL Server from C# and so it is an API that I will use to assist me in getting this program working. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I understand that. > I'm wondering what for do you learning this stuff? - Just out of curiosity > or do you plan to implement some custom applications using > Microsoft.SQLServer.* ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 21, 2009 5:06 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > That is used for manipulating the SMO object and SQL Server. > > I am just learning this stuff and do what the demos tell me to do. > > John W. Colby > www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sat Nov 21 10:27:15 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 21 Nov 2009 19:27:15 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B080A54.4030609@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> Message-ID: <002601ca6ac7$7dcfb180$796f1480$@spb.ru> Hi John -- OK, but why not just use connections strings and System.Data.SqlClient classes, .... ? Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 21, 2009 6:42 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SMO was Projects vs Solutions > or do you plan to implement some custom applications using Microsoft.SQLServer.* ? Exactly. My work for one of my clients is heavy manipulation of SQL Server. I routinely export files to an external application running on virtual machines on another server, then wait for the files to process, then import them back in to SQL server. These two processes together take about 10 stored procedures. The export / import has to occur on (currently) a half dozen large databases monthly. However as I obtain more lists that number goes up. Each process exports to it's own directory path, then imports back from a parallel directory path. While the stored procedures eliminate much of the manual labor, there is plenty left. Plus logging of everything etc. I also routinely process orders. These orders take about a dozen stored procedures and also use the exact same set of stored procedures as the above process, i.e. the export / import process is also used on each and every order. So there is just a ton of very repetitive work that has to be done, but which database / table varies from run to run. To this point I have used Access / VBA to automate these tasks. Now I am moving to C# for this automation work. SMO allows me to manipulate SQL Server from C# and so it is an API that I will use to assist me in getting this program working. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I understand that. > I'm wondering what for do you learning this stuff? - Just out of curiosity > or do you plan to implement some custom applications using > Microsoft.SQLServer.* ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 21, 2009 5:06 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > That is used for manipulating the SMO object and SQL Server. > > I am just learning this stuff and do what the demos tell me to do. > > John W. Colby > www.ColbyConsulting.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4626 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 21 11:05:43 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 12:05:43 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <002601ca6ac7$7dcfb180$796f1480$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> <002601ca6ac7$7dcfb180$796f1480$@spb.ru> Message-ID: <4B081DE7.7060203@colbyconsulting.com> Shamil, As far as I can tell they are quite different things, and perform different jobs. I do use SQLClient. SQLClient appears to be about getting at data, and I do use that for the ADO side of things. http://msdn.microsoft.com/en-us/library/system.data.sqlclient.aspx SMO appears to be about getting at and manipulating the objects in the database. http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.aspx SMO appears to be similar to the DAO object where you can see and manipulate the table (not the data IN the table) the fields, indexes and so forth. It is a collection based object where you can drill down into a server and see the objects underneath the database, then (for example) drill down into a table and see the objects under the table. Just as an example, using SMO in a few lines of code I got a list of all of the databases in the database collection of my server. I used that to populate a database combo box. When I select the database that I want to export data from, I then use SMO (again a couple of more lines of code) to get a list of the tables in the selected database and use that to populate a combo of tables. I have no idea the relative merit of one way vs the other. To me it is just a tool. In this particular case I use it to quickly and easily get at the names of objects in the database. I understand that using SMO you can do other maintenance kinds of things as well. One of the things I have to do is copy an "order template" database to a new name in preparing to fill an order. SMO appears to have built-in methods for doing this programmatically from C# - as opposed to running TSQL or using some other method. Understand I am not an expert on any of this, in fact quite the opposite. I stumbled across SMO and saw an example of how easy it was to get at the database STRUCTURE information and decided to use it for that purpose, when I had that need. Is there another way to do this? Almost certainly, given that there is always a dozen ways to do anything in programming. Given your persistent questioning of my motives I guess now I have to ask whether I am causing myself problems doing this? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > OK, but why not just use connections strings and System.Data.SqlClient > classes, .... ? > > Thank you. From jwcolby at colbyconsulting.com Sat Nov 21 11:40:33 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 12:40:33 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <002601ca6ac7$7dcfb180$796f1480$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> <002601ca6ac7$7dcfb180$796f1480$@spb.ru> Message-ID: <4B082611.4040401@colbyconsulting.com> Shamil, By get an instance of the Server object you can see a collection of Database objects. The following is what you can do with the database object. So I can manipulate a given database as shown below. Several of those things look potentially useful, such as running TSQL statements, truncating log files and so forth. http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.database.aspx However the database object has a tables collection. I can get lists of the names of tables, select a specific table etc. http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.table.aspx http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.table_methods.aspx The point is that it is just a programming model that is available if you want to learn it, for manipulating a database from C#. I will almost certainly never use 99.9% of it, just as most people (myself included) rarely use 99.9% of the DAO object model. OTOH the .1% that I do use is pretty darned cool. You can find a lot of little articles about using SMO out on Google. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > OK, but why not just use connections strings and System.Data.SqlClient > classes, .... ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 21, 2009 6:42 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > > or do you plan to implement some custom applications using > Microsoft.SQLServer.* ? > > Exactly. My work for one of my clients is heavy manipulation of SQL Server. > I routinely export > files to an external application running on virtual machines on another > server, then wait for the > files to process, then import them back in to SQL server. These two > processes together take about > 10 stored procedures. The export / import has to occur on (currently) a > half dozen large databases > monthly. However as I obtain more lists that number goes up. Each process > exports to it's own > directory path, then imports back from a parallel directory path. While the > stored procedures > eliminate much of the manual labor, there is plenty left. Plus logging of > everything etc. > > I also routinely process orders. These orders take about a dozen stored > procedures and also use the > exact same set of stored procedures as the above process, i.e. the export / > import process is also > used on each and every order. > > So there is just a ton of very repetitive work that has to be done, but > which database / table > varies from run to run. To this point I have used Access / VBA to automate > these tasks. Now I am > moving to C# for this automation work. > > SMO allows me to manipulate SQL Server from C# and so it is an API that I > will use to assist me in > getting this program working. > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> Yes, I understand that. >> I'm wondering what for do you learning this stuff? - Just out of curiosity >> or do you plan to implement some custom applications using >> Microsoft.SQLServer.* ? >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: Saturday, November 21, 2009 5:06 AM >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] SMO was Projects vs Solutions >> >> That is used for manipulating the SMO object and SQL Server. >> >> I am just learning this stuff and do what the demos tell me to do. >> >> John W. Colby >> www.ColbyConsulting.com > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4626 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Nov 21 11:47:27 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 12:47:27 -0500 Subject: [dba-VB] Loss leader (or just plain loss) Message-ID: <4B0827AF.6040608@colbyconsulting.com> A friend has approached me about building an application for a small chain of stores to manage inventory. "Very simple". I haven't yet talked to her about the requirements. She did say they were quoted (and rejected) a bid of $4000 to do the job. If I am going to do this at all it will have to be very quick and dirty, and would be just to help a friend. Is Access 2007 runtime stable? Easy or difficult to use / install / implement on site? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sat Nov 21 11:55:29 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 21 Nov 2009 20:55:29 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B081DE7.7060203@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> <002601ca6ac7$7dcfb180$796f1480$@spb.ru> <4B081DE7.7060203@colbyconsulting.com> Message-ID: <002c01ca6ad3$d1e37930$75aa6b90$@spb.ru> Hi John -- <<< SMO appears to be similar to the DAO object where you can see and manipulate the table... >>> Yes, I understand/know that. <<< When I select the database that I want to export data from, I then use SMO (again a couple of more lines of code) to get a list of the tables in the selected database and use that to populate a combo of tables. >>> Yes, but how often would you need to do that? Why not just use (relatively static) program settings files, or utility program execution command string parameters - I mean you anyway have to do manual selection (in the case you'll use SMO) or manual editing and saving (in the case you'll use program setting files...) - what for to spend time on learning SMO if you very probably (am I wrong?) can solve your customer tasks without using SMO, and applying relatively the same efforts in both cases to implement custom logic (but in the case of using SMO you'll also have to spend time on learning SMO)... <<< Given your persistent questioning of my motives I guess now I have to ask whether I am causing myself problems doing this? >>> Just maybe by spending time on learning SMO when you don't need it to implement required custom logic? Not sure (I didn't check so I can be wrong) but SMO might need special installation procedures for customers' PCs without full MS SQL Version (standard, prof., enterprise) so if you'll develop some custom code to be reused on customers' PC then you might need to be prepared to develop custom setup - not a big issue just be prepared - if I'm not wrong in guessing that that custom setup would be needed for some customers system... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 21, 2009 8:06 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SMO was Projects vs Solutions Shamil, As far as I can tell they are quite different things, and perform different jobs. I do use SQLClient. SQLClient appears to be about getting at data, and I do use that for the ADO side of things. http://msdn.microsoft.com/en-us/library/system.data.sqlclient.aspx SMO appears to be about getting at and manipulating the objects in the database. http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.a spx SMO appears to be similar to the DAO object where you can see and manipulate the table (not the data IN the table) the fields, indexes and so forth. It is a collection based object where you can drill down into a server and see the objects underneath the database, then (for example) drill down into a table and see the objects under the table. Just as an example, using SMO in a few lines of code I got a list of all of the databases in the database collection of my server. I used that to populate a database combo box. When I select the database that I want to export data from, I then use SMO (again a couple of more lines of code) to get a list of the tables in the selected database and use that to populate a combo of tables. I have no idea the relative merit of one way vs the other. To me it is just a tool. In this particular case I use it to quickly and easily get at the names of objects in the database. I understand that using SMO you can do other maintenance kinds of things as well. One of the things I have to do is copy an "order template" database to a new name in preparing to fill an order. SMO appears to have built-in methods for doing this programmatically from C# - as opposed to running TSQL or using some other method. Understand I am not an expert on any of this, in fact quite the opposite. I stumbled across SMO and saw an example of how easy it was to get at the database STRUCTURE information and decided to use it for that purpose, when I had that need. Is there another way to do this? Almost certainly, given that there is always a dozen ways to do anything in programming. Given your persistent questioning of my motives I guess now I have to ask whether I am causing myself problems doing this? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > OK, but why not just use connections strings and System.Data.SqlClient > classes, .... ? > > Thank you. __________ Information from ESET NOD32 Antivirus, version of virus signature database 4626 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From dbdoug at gmail.com Sat Nov 21 12:26:06 2009 From: dbdoug at gmail.com (Doug Steele) Date: Sat, 21 Nov 2009 10:26:06 -0800 Subject: [dba-VB] Loss leader (or just plain loss) In-Reply-To: <4B0827AF.6040608@colbyconsulting.com> References: <4B0827AF.6040608@colbyconsulting.com> Message-ID: <4dd71a0c0911211026y79e4d583t99d6ca1dfb4a10f1@mail.gmail.com> Hi John: I've only ever used the Access 2003 runtime/installer, and that was for one job. Getting the runtime code installed, running, and generating/testing a proper installation package took the best part of a day. It was straightforward enough, just fiddly. After that, updates only consisted of recreating the .mde file and sending it to the client. From what I've read, the Access 2007 runtime is pretty stable now (it started off very badly) and it's the right price. My experience with projects that started out to be 'quick and dirty' has been consistently horrible. Not quick, and very dirty! Doug On Sat, Nov 21, 2009 at 9:47 AM, jwcolby wrote: > A friend has approached me about building an application for a small chain > of stores to manage > inventory. "Very simple". I haven't yet talked to her about the > requirements. She did say they > were quoted (and rejected) a bid of $4000 to do the job. > > If I am going to do this at all it will have to be very quick and dirty, > and would be just to help a > friend. > > Is Access 2007 runtime stable? Easy or difficult to use / install / > implement on site? > > > From shamil at smsconsulting.spb.ru Sat Nov 21 12:40:36 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 21 Nov 2009 21:40:36 +0300 Subject: [dba-VB] Loss leader (or just plain loss) In-Reply-To: <4dd71a0c0911211026y79e4d583t99d6ca1dfb4a10f1@mail.gmail.com> References: <4B0827AF.6040608@colbyconsulting.com> <4dd71a0c0911211026y79e4d583t99d6ca1dfb4a10f1@mail.gmail.com> Message-ID: <002d01ca6ada$1ecb5fa0$5c621ee0$@spb.ru> <<< Not quick, and very dirty! >>> Yes. And a high opportunity to lose a friend... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Doug Steele Sent: Saturday, November 21, 2009 9:26 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Loss leader (or just plain loss) Hi John: I've only ever used the Access 2003 runtime/installer, and that was for one job. Getting the runtime code installed, running, and generating/testing a proper installation package took the best part of a day. It was straightforward enough, just fiddly. After that, updates only consisted of recreating the .mde file and sending it to the client. From what I've read, the Access 2007 runtime is pretty stable now (it started off very badly) and it's the right price. My experience with projects that started out to be 'quick and dirty' has been consistently horrible. Not quick, and very dirty! Doug On Sat, Nov 21, 2009 at 9:47 AM, jwcolby wrote: > A friend has approached me about building an application for a small chain > of stores to manage > inventory. "Very simple". I haven't yet talked to her about the > requirements. She did say they > were quoted (and rejected) a bid of $4000 to do the job. > > If I am going to do this at all it will have to be very quick and dirty, > and would be just to help a > friend. > > Is Access 2007 runtime stable? Easy or difficult to use / install / > implement on site? > > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4626 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 21 13:30:39 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 14:30:39 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <002c01ca6ad3$d1e37930$75aa6b90$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> <002601ca6ac7$7dcfb180$796f1480$@spb.ru> <4B081DE7.7060203@colbyconsulting.com> <002c01ca6ad3$d1e37930$75aa6b90$@spb.ru> Message-ID: <4B083FDF.505@colbyconsulting.com> There is no "customer PC". This is my server in my office, running an application I will write, which I will operate. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > <<< > SMO appears to be similar to the DAO object > where you can see and manipulate the table... > Yes, I understand/know that. > > <<< > When I select the > database that I want to export data from, I then > use SMO (again a couple of more lines of code) to > get a list of the tables in the selected database > and use that to populate a combo of tables. > Yes, but how often would you need to do that? > Why not just use (relatively static) program settings files, or utility > program execution command string parameters - I mean you anyway have to do > manual selection (in the case you'll use SMO) or manual editing and saving > (in the case you'll use program setting files...) - what for to spend time > on learning SMO if you very probably (am I wrong?) can solve your customer > tasks without using SMO, and applying relatively the same efforts in both > cases to implement custom logic (but in the case of using SMO you'll also > have to spend time on learning SMO)... > > <<< > Given your persistent questioning of my motives > I guess now I have to ask whether I am causing > myself problems doing this? > Just maybe by spending time on learning SMO when you don't need it to > implement required custom logic? > > Not sure (I didn't check so I can be wrong) but SMO might need special > installation procedures for customers' PCs without full MS SQL Version > (standard, prof., enterprise) so if you'll develop some custom code to be > reused on customers' PC then you might need to be prepared to develop custom > setup - not a big issue just be prepared - if I'm not wrong in guessing that > that custom setup would be needed for some customers system... > > Thank you. > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 21, 2009 8:06 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > Shamil, > > As far as I can tell they are quite different things, and perform different > jobs. I do use > SQLClient. SQLClient appears to be about getting at data, and I do use that > for the ADO side of > things. > > http://msdn.microsoft.com/en-us/library/system.data.sqlclient.aspx > > SMO appears to be about getting at and manipulating the objects in the > database. > > http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.a > spx > > SMO appears to be similar to the DAO object where you can see and manipulate > the table (not the data > IN the table) the fields, indexes and so forth. It is a collection based > object where you can drill > down into a server and see the objects underneath the database, then (for > example) drill down into a > table and see the objects under the table. > > Just as an example, using SMO in a few lines of code I got a list of all of > the databases in the > database collection of my server. I used that to populate a database combo > box. When I select the > database that I want to export data from, I then use SMO (again a couple of > more lines of code) to > get a list of the tables in the selected database and use that to populate a > combo of tables. > > I have no idea the relative merit of one way vs the other. To me it is just > a tool. In this > particular case I use it to quickly and easily get at the names of objects > in the database. I > understand that using SMO you can do other maintenance kinds of things as > well. One of the things I > have to do is copy an "order template" database to a new name in preparing > to fill an order. SMO > appears to have built-in methods for doing this programmatically from C# - > as opposed to running > TSQL or using some other method. > > Understand I am not an expert on any of this, in fact quite the opposite. I > stumbled across SMO and > saw an example of how easy it was to get at the database STRUCTURE > information and decided to use it > for that purpose, when I had that need. Is there another way to do this? > Almost certainly, given > that there is always a dozen ways to do anything in programming. > > Given your persistent questioning of my motives I guess now I have to ask > whether I am causing > myself problems doing this? > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> OK, but why not just use connections strings and System.Data.SqlClient >> classes, .... ? >> >> Thank you. > > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4626 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sun Nov 22 22:46:38 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 22 Nov 2009 23:46:38 -0500 Subject: [dba-VB] Ya know you're in trouble when... Message-ID: <4B0A13AE.1030404@colbyconsulting.com> http://www.hanselman.com/blog/WhatGreatNETDevelopersOughtToKnowMoreNETInterviewQuestions.aspx -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Mon Nov 23 07:57:41 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 23 Nov 2009 08:57:41 -0500 Subject: [dba-VB] XNA game development Message-ID: <4B0A94D5.3010601@colbyconsulting.com> Have you ever wanted to write your own little windows game. Microsoft provides a free game development framework called XNA. It runs in C# express and everything is free free free. It is designed to get you writing games for the XBox BUT... it can write games for Windows as well, and in fact they are apparently interchangeable, once written they can be played on the XBox (but not for free). The best part is that they have a very basic cannon / flying saucer demo game with videos for how to develop this game, all of the source code, sprites etc. I have stepped through the first few chapters and it really does start at ground zero. They claim you don't even need to know C# to follow the tutorial. As we all know, writing a project that holds your interest is the best way to get involved and learn a language. If writing a game is your interest, this might be just the ticket to getting into C#. http://creators.xna.com/en-US/education/gettingstarted/bg2d/chapter1 -- John W. Colby www.ColbyConsulting.com From mmattys at rochester.rr.com Mon Nov 23 09:41:38 2009 From: mmattys at rochester.rr.com (Mike Mattys) Date: Mon, 23 Nov 2009 10:41:38 -0500 Subject: [dba-VB] Ya know you're in trouble when... References: <4B0A13AE.1030404@colbyconsulting.com> Message-ID: I've already come up against a great many of these Some of them I look forward to getting the answers for. If I went to an interview for all of it, I would hope it to be "open book" - Michael R Mattys MapPoint and Database Dev www.mattysconsulting.com - ----- Original Message ----- From: "jwcolby" To: "VBA" Sent: Sunday, November 22, 2009 11:46 PM Subject: [dba-VB] Ya know you're in trouble when... > > http://www.hanselman.com/blog/WhatGreatNETDevelopersOughtToKnowMoreNETInterviewQuestions.aspx > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > From dwaters at usinternet.com Mon Nov 23 12:22:19 2009 From: dwaters at usinternet.com (Dan Waters) Date: Mon, 23 Nov 2009 12:22:19 -0600 Subject: [dba-VB] Ya know you're in trouble when... In-Reply-To: References: <4B0A13AE.1030404@colbyconsulting.com> Message-ID: Hey! I knew one of these! What's the difference between late binding and early binding? But that's it. :-( Dan -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Mike Mattys Sent: Monday, November 23, 2009 9:42 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Ya know you're in trouble when... I've already come up against a great many of these Some of them I look forward to getting the answers for. If I went to an interview for all of it, I would hope it to be "open book" - Michael R Mattys MapPoint and Database Dev www.mattysconsulting.com - ----- Original Message ----- From: "jwcolby" To: "VBA" Sent: Sunday, November 22, 2009 11:46 PM Subject: [dba-VB] Ya know you're in trouble when... > > http://www.hanselman.com/blog/WhatGreatNETDevelopersOughtToKnowMoreNETInterv iewQuestions.aspx > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Mon Nov 23 15:57:12 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 23 Nov 2009 16:57:12 -0500 Subject: [dba-VB] XNA is darned fun Message-ID: <4B0B0538.1020904@colbyconsulting.com> I have to say that this is darned fun. I have worked through much of the tutorial I posted a link to. I have ships flying and cannonballs firing. Woohoo! Very very cool! I know what I will be doing over the Thanksgiving holiday. -- John W. Colby www.ColbyConsulting.com From michael at ddisolutions.com.au Mon Nov 23 23:38:16 2009 From: michael at ddisolutions.com.au (Michael Maddison) Date: Tue, 24 Nov 2009 16:38:16 +1100 Subject: [dba-VB] HTTPWebRequest blues Message-ID: <59A61174B1F5B54B97FD4ADDE71E7D01582726@ddi-01.DDI.local> Hi guys, I may be clutching at straws here but has anyone done much with HttpWebRequest? My process goes like this, GET /ExpressBet/Standard/Default.aspx?State=2 Store the cookie container for re use GET /ExpressBet/LoginFrame.aspx?State=2&ExpressBetMode=Standard Store viewState for re use in next POST POST /Login/LoginUser.aspx?State=2&ReturnURL=http://www.tab.com.au/ExpressBet/LoginFrame.aspx?State=2&ExpressBetMode=Standard Does login, this succeeds GET /ExpressBet/Aggregator.aspx?State=2&Controls=STD_MAIN&RacingCode=R&MeetingCode=M&FromDate=2009-11-24T00:00:00&RaceNumber=07&BetType=WinPlace&ExpressBetMode=Standard Store viewState for re use GET /Betting/BetTicket/DisplayClientSideBetTicket.aspx?State=2&ExpressBet=true&RacingCode=R&MeetingCode=M&FromDate=2009-11-24T00:00:00&BetType=WinPlace&amount=1.00&Selections=%3csc+id%3d%220%22+f%3d%22false%22+m%3d%220%22+s%3d%228%22+t%3d%22%22+k%3d%2207%22+%2f%3e+ The above redirects to GET /Betting/BetTicket/PlaceRaceBetConfirmation.aspx?State=2&CurrentBet=2901d195-c59a-4c1b-8b0a-c476fc274f06&ExpressBet=True Result looks ok in Fiddler, But response is unreadable in plain text. Anyone know what this is? I think I need to get the viewstate from it somehow. Snipped because of size constraints. _???2??-w?sF??'15??1??_i???i??_l?___???_?__?S??fY?d?R_adZ?Ri??__ 2?#??4+?='+??N!??_n^???????nQ^?`????_?_____?0???_?h??X????_???dvTH??`??[?k?Sh?SDp??[h?c?OH???k?K?{??3??@?'13OW7????0W??8,????r2????_=}A???{?@??? i???)?5__F_?[?S?'4N?__C]}?M?}?Af???s?H??K?-H?? ?u+N ?s?r,? The Last step I need to do is a final POST, This returns an error page, dammit. POST /Betting/BetTicket/PlaceRaceBetConfirmation.aspx?State=2&CurrentBet=2901d195-c59a-4c1b-8b0a-c476fc274f06&ExpressBet=True Redirects to an error.aspx page. The code for the last POST is the same for the successful calls as the failed call. Both are HTTPS, I am handling the certificates. Does this make sense to anyone? J Any ideas? Cheers Michael M From Gustav at cactus.dk Tue Nov 24 04:50:40 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Tue, 24 Nov 2009 11:50:40 +0100 Subject: [dba-VB] This is hot: WCF RIA Services Message-ID: Hi all RIA: Rich Internet Applications Microsoft WCF RIA Services simplifies the traditional n-tier application pattern by bringing together the ASP.NET and Silverlight platforms. RIA Services provides a pattern to write application logic that runs on the mid-tier and controls access to data for queries, changes and custom operations. It also provides end-to-end support for common tasks such as data validation, authentication and roles by integrating with Silverlight components on the client and ASP.NET on the mid-tier Though not explained at a place to find, what it does is automatically to set up for you a web service which your Silverlight frontend communicates with, and to establish the client side code that handles this communication. http://silverlight.net/getstarted/riaservices/ Watch the 1-hour video "Introduction to WCF RIA Services" and combine this with the racing speed at which Silverlight evolves, and it will erase any doubt you still may have about browser-based applications. Note too, that while this works in VS2008 and with Silverlight 3 it is really aimed at VS2010 and Silverlight 4 both currently available as betas. I attended a developer meeting a couple of days ago where we ran a similar walk-through, and it was very convincing. This is exactly what I for years have been looking for. /gustav From fuller.artful at gmail.com Tue Nov 24 10:26:24 2009 From: fuller.artful at gmail.com (Arthur Fuller) Date: Tue, 24 Nov 2009 11:26:24 -0500 Subject: [dba-VB] XNA is darned fun In-Reply-To: <4B0B0538.1020904@colbyconsulting.com> References: <4B0B0538.1020904@colbyconsulting.com> Message-ID: <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> When is American Thanksgiving? We already had ours last month. What is XNA, anyway? Arthur On Mon, Nov 23, 2009 at 4:57 PM, jwcolby wrote: > I have to say that this is darned fun. I have worked through much of the > tutorial I posted a link > to. I have ships flying and cannonballs firing. Woohoo! > > Very very cool! I know what I will be doing over the Thanksgiving holiday. > From cfoust at infostatsystems.com Tue Nov 24 10:48:12 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Tue, 24 Nov 2009 10:48:12 -0600 Subject: [dba-VB] XNA is darned fun In-Reply-To: <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> References: <4B0B0538.1020904@colbyconsulting.com> <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> Message-ID: Ours is Thursday. Charlotte -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Arthur Fuller Sent: Tuesday, November 24, 2009 8:26 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] XNA is darned fun When is American Thanksgiving? We already had ours last month. What is XNA, anyway? Arthur On Mon, Nov 23, 2009 at 4:57 PM, jwcolby wrote: > I have to say that this is darned fun. I have worked through much of > the tutorial I posted a link to. I have ships flying and cannonballs > firing. Woohoo! > > Very very cool! I know what I will be doing over the Thanksgiving holiday. > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Tue Nov 24 10:53:21 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Tue, 24 Nov 2009 11:53:21 -0500 Subject: [dba-VB] XNA is darned fun In-Reply-To: <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> References: <4B0B0538.1020904@colbyconsulting.com> <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> Message-ID: <4B0C0F81.9070806@colbyconsulting.com> > When is American Thanksgiving? We already had ours last month. Not sure exactly. IIRC it is the last Thursday in November so it is not a fixed date. > What is XNA, anyway? It is a Microsoft gaming framework. VERY COOL if you like to (or always wanted to) write a windows game. It runs under C# in Visual Studio, even the express version. The framework provides a game class that has a method for loading the "content" - video, sound, sprites etc. It has a method automatically called X times / second that is then used to perform updates and display the content. And it has a draw method that is called to draw the game. LOTS of game specific classes including math classes, 2d and 3d classes for doing game specific kinds of things, sprite classes, point classes, rectangle classes. You get the picture. REALLY really cool for the child in all of us. If you happen to have an XBox, you can write games that can run on the XBox as well as Windows (XP, Vista and Windows 7). I don't own an XBox so I am just running the game on Windows, but any game CAN be written to run on either, and apparently SENSE which it is running on. You can even use the XBox joystick / button controller on your Windows machine! There is a step by step video series that walks you through building a "space invaders" game, with cannon, cannon balls, UFOs. All graphics provided, all code stepped through in short videos. I did that and I have to say it is really easy. It was so cool as each step comes together, displaying the background, displaying the flying saucers moving across the screen, the cannon swiveling, then cannon balls firing, and finally cannon balls destroying the UFOs. Of course doing it all by myself would be not so easy but it gives a great feeling for what can be done and how it is done. And of course there are after market books on it. I have always wanted to write my own "Empire" replacement, and have even started doing so. I played Empire all of my life (still do) but it is long in the tooth and not updated any more. Plus the multi-player stuff never really worked very well. This will set me back to ground zero on that attempt but it will also provide a framework for doing it the right way. I just don't have enough to do with my time, as I am sure you can understand! John W. Colby www.ColbyConsulting.com Arthur Fuller wrote: > When is American Thanksgiving? We already had ours last month. > > What is XNA, anyway? > > Arthur > > On Mon, Nov 23, 2009 at 4:57 PM, jwcolby wrote: > >> I have to say that this is darned fun. I have worked through much of the >> tutorial I posted a link >> to. I have ships flying and cannonballs firing. Woohoo! >> >> Very very cool! I know what I will be doing over the Thanksgiving holiday. >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Wed Nov 25 08:24:39 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 09:24:39 -0500 Subject: [dba-VB] Part 1: An Introduction to Data Structures Message-ID: <4B0D3E27.9040601@colbyconsulting.com> A quick overview: http://creators.xna.com/en-US/article/datastructures A more detailed overview: http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Wed Nov 25 08:51:50 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Wed, 25 Nov 2009 17:51:50 +0300 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: <4B0D3E27.9040601@colbyconsulting.com> References: <4B0D3E27.9040601@colbyconsulting.com> Message-ID: <006e01ca6dde$d31de700$7959b500$@spb.ru> Hi John -- You might also find interesting: http://en.wikipedia.org/wiki/Microsoft_Robotics_Developer_Studio e.g., for programming: http://en.wikipedia.org/wiki/Lego_Mindstorms_NXT using C#... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 25, 2009 5:25 PM To: VBA; Nikolai Vitsyn; Judy Hawley Subject: [dba-VB] Part 1: An Introduction to Data Structures A quick overview: http://creators.xna.com/en-US/article/datastructures A more detailed overview: http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4636 (20091125) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From Gustav at cactus.dk Wed Nov 25 08:54:57 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 25 Nov 2009 15:54:57 +0100 Subject: [dba-VB] Part 1: An Introduction to Data Structures Message-ID: Hi John Thanks. "collision code" ... I love that. /gustav >>> jwcolby at colbyconsulting.com 25-11-2009 15:24 >>> A quick overview: http://creators.xna.com/en-US/article/datastructures A more detailed overview: http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Wed Nov 25 09:23:22 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 10:23:22 -0500 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: References: Message-ID: <4B0D4BEA.2050603@colbyconsulting.com> > Thanks. "collision code" ... I love that. LOL. Obviously this was aimed at gamers. A collision is when one rectangle (sprite) intersects another rectangle. BTW, check out this video of a game. http://www.youtube.com/watch?v=Lz-TQcdXDJU&feature=related LOTS of collisions going on here. ;) I am going to work on writing this game using the XNA gaming framework. The XNA framework is just awesome, and I am going to get my 8 year old son involved in helping me to write this thing. A fun, interactive way of teaching him C#. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > Thanks. "collision code" ... I love that. > > /gustav > > >>>> jwcolby at colbyconsulting.com 25-11-2009 15:24 >>> > A quick overview: > http://creators.xna.com/en-US/article/datastructures > > A more detailed overview: > http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx From jwcolby at colbyconsulting.com Wed Nov 25 09:24:15 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 10:24:15 -0500 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: <006e01ca6dde$d31de700$7959b500$@spb.ru> References: <4B0D3E27.9040601@colbyconsulting.com> <006e01ca6dde$d31de700$7959b500$@spb.ru> Message-ID: <4B0D4C1F.9050103@colbyconsulting.com> I have this thing. I haven't looked at it yet though. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > You might also find interesting: > > http://en.wikipedia.org/wiki/Microsoft_Robotics_Developer_Studio > > e.g., for programming: > > http://en.wikipedia.org/wiki/Lego_Mindstorms_NXT > > using C#... > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, November 25, 2009 5:25 PM > To: VBA; Nikolai Vitsyn; Judy Hawley > Subject: [dba-VB] Part 1: An Introduction to Data Structures > > A quick overview: > http://creators.xna.com/en-US/article/datastructures > > A more detailed overview: > http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx From stuart at lexacorp.com.pg Wed Nov 25 15:31:48 2009 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Thu, 26 Nov 2009 07:31:48 +1000 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: <4B0D4BEA.2050603@colbyconsulting.com> References: , <4B0D4BEA.2050603@colbyconsulting.com> Message-ID: <4B0DA244.1838.3178D240@stuart.lexacorp.com.pg> Gee you guys are taking me back - I used to write games for the Amiga twenty odd years ago - It all revolved around sprites and collisions. -- Stuart On 25 Nov 2009 at 10:23, jwcolby wrote: > > Thanks. "collision code" ... I love that. > > LOL. Obviously this was aimed at gamers. A collision is when one rectangle (sprite) intersects > another rectangle. > > BTW, check out this video of a game. > > http://www.youtube.com/watch?v=Lz-TQcdXDJU&feature=related > > LOTS of collisions going on here. ;) > > I am going to work on writing this game using the XNA gaming framework. The XNA framework is just > awesome, and I am going to get my 8 year old son involved in helping me to write this thing. A fun, > interactive way of teaching him C#. > > John W. Colby > www.ColbyConsulting.com > > > Gustav Brock wrote: > > Hi John > > > > Thanks. "collision code" ... I love that. > > > > /gustav > > > > > >>>> jwcolby at colbyconsulting.com 25-11-2009 15:24 >>> > > A quick overview: > > http://creators.xna.com/en-US/article/datastructures > > > > A more detailed overview: > > http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > From Gustav at cactus.dk Wed Nov 25 15:50:53 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 25 Nov 2009 22:50:53 +0100 Subject: [dba-VB] Part 1: An Introduction to Data Structures Message-ID: Hi John Oh my, will this never stop? I guess that is what this poor guy thinks, in real trouble as he is ... /gustav >>> jwcolby at colbyconsulting.com 25-11-2009 16:23 >>> > Thanks. "collision code" ... I love that. LOL. Obviously this was aimed at gamers. A collision is when one rectangle (sprite) intersects another rectangle. BTW, check out this video of a game. http://www.youtube.com/watch?v=Lz-TQcdXDJU&feature=related LOTS of collisions going on here. ;) I am going to work on writing this game using the XNA gaming framework. The XNA framework is just awesome, and I am going to get my 8 year old son involved in helping me to write this thing. A fun, interactive way of teaching him C#. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > Thanks. "collision code" ... I love that. > > /gustav > > >>>> jwcolby at colbyconsulting.com 25-11-2009 15:24 >>> > A quick overview: > http://creators.xna.com/en-US/article/datastructures > > A more detailed overview: > http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx From jwcolby at colbyconsulting.com Wed Nov 25 15:52:30 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 16:52:30 -0500 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: <4B0DA244.1838.3178D240@stuart.lexacorp.com.pg> References: , <4B0D4BEA.2050603@colbyconsulting.com> <4B0DA244.1838.3178D240@stuart.lexacorp.com.pg> Message-ID: <4B0DA71E.3030105@colbyconsulting.com> > Gee you guys are taking me back - I used to write games for the Amiga twenty odd years ago - It all revolved around sprites and collisions. LOL. Give it a whirl. I bet you find it much easier in this day and age. One of the reasons I am doing this is that I want to get really good at C#. I do have real work that I am starting to do in C#, but writing games is fun and will keep me in it on a daily / weekly basis whereas my work could be weeks without having to write any C# code. Plus I am hoping to slowly suck my son into the programming. He was up in my office changing the constants that determine how many space ships will be created in the shoot-em-up demo I was studying. He now knows how to edit that constant, how to run it, how to close the game, he knows now that he can't put any number he wants in an integer variable (he tried to run with 20000000000000 spaceships;). We'll see. John W. Colby www.ColbyConsulting.com Stuart McLachlan wrote: > Gee you guys are taking me back - I used to write games for the Amiga twenty odd years > ago - It all revolved around sprites and collisions. > From jwcolby at colbyconsulting.com Wed Nov 25 20:24:52 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 21:24:52 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. Message-ID: <4B0DE6F4.3070605@colbyconsulting.com> If a parameter in a stored procedure is declared as an output (returns a value) will the parameter object in C# / ADO contain that value when the stored procedure finishes? I haven't used output parameters yet in stored procedures. Can values be passed in via an output parameter or can it only be set from inside of the stored procedure? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Thu Nov 26 00:02:40 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 26 Nov 2009 09:02:40 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0DE6F4.3070605@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> Message-ID: <00a101ca6e5e$111e9ac0$335bd040$@spb.ru> Hi John, Yes. Set System.Data.ParameterDirection.Output for .Direction property of output parameters. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 26, 2009 5:25 AM To: VBA Subject: [dba-VB] Parameter objects in C# and ADO. If a parameter in a stored procedure is declared as an output (returns a value) will the parameter object in C# / ADO contain that value when the stored procedure finishes? I haven't used output parameters yet in stored procedures. Can values be passed in via an output parameter or can it only be set from inside of the stored procedure? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4637 (20091125) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4637 (20091125) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From marklbreen at gmail.com Thu Nov 26 03:04:54 2009 From: marklbreen at gmail.com (Mark Breen) Date: Thu, 26 Nov 2009 09:04:54 +0000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0DE6F4.3070605@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> Message-ID: Hello John, There are three ways to bring back values from an sproc, 1 the result set 2 the output parameters, of which there can be many 3 the return code afaik, a param can be input (the default direction) or output. I have not heard of an inputoutput param so unless anyone here says different, assume that you cannot have bi-directional. Now you have the ability to standardize on a return code which may relate to error codes, a complete recordset of data, and also a few params that passed back logging codes, temp counts etc etc etc. In fact, if the sproc is doing multiple stages, you will love the ability to store staged log codes and messages and then bring the whole lot back when it is finished, and still being 1M records as well ! Thanks Mark 2009/11/26 jwcolby > If a parameter in a stored procedure is declared as an output (returns a > value) will the parameter > object in C# / ADO contain that value when the stored procedure finishes? > > I haven't used output parameters yet in stored procedures. Can values be > passed in via an output > parameter or can it only be set from inside of the stored procedure? > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Thu Nov 26 04:26:58 2009 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Thu, 26 Nov 2009 20:26:58 +1000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: <4B0DE6F4.3070605@colbyconsulting.com>, Message-ID: <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> SP with a parameter that is both Input and Output In some situations, there is no need to store the output into an extra variable because the output must replace the input. CREATE PROCEDURE cap2 (@name NVARCHAR(20) OUT ) --Capitalizes first char, changes the rest to lower case --SP with one input/output parameter AS SET @name=UPPER(LEFT(@name,1))+LOWER(RIGHT(@name,LEN(@name)-1)) or for you .net types, see http://fabioscagliola.spaces.live.com/blog/cns!919F8FCDE3DC9AC4!121.entry -- Stuart On 26 Nov 2009 at 9:04, Mark Breen wrote: > > afaik, a param can be input (the default direction) or output. I have not > heard of an inputoutput param so unless anyone here says different, assume > that you cannot have bi-directional. > From shamil at smsconsulting.spb.ru Thu Nov 26 05:03:27 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 26 Nov 2009 14:03:27 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> References: <4B0DE6F4.3070605@colbyconsulting.com>, <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> Message-ID: <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> Hi Mark, > assume that you cannot have bi-directional. You can: System.Data.ParameterDirection.InputOutput -- Shamil On 26 Nov 2009 at 9:04, Mark Breen wrote: > > afaik, a param can be input (the default direction) or output. I have not > heard of an inputoutput param so unless anyone here says different, assume > that you cannot have bi-directional. > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4638 (20091126) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Thu Nov 26 06:55:59 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 26 Nov 2009 07:55:59 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: <4B0DE6F4.3070605@colbyconsulting.com> Message-ID: <4B0E7ADF.5010504@colbyconsulting.com> Thanks for the reply Mark. I am looking forward to having the flexibility of C# to handle errors appropriately. John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > There are three ways to bring back values from an sproc, > > 1 the result set > 2 the output parameters, of which there can be many > 3 the return code > > afaik, a param can be input (the default direction) or output. I have not > heard of an inputoutput param so unless anyone here says different, assume > that you cannot have bi-directional. > > Now you have the ability to standardize on a return code which may relate to > error codes, a complete recordset of data, and also a few params that passed > back logging codes, temp counts etc etc etc. > > In fact, if the sproc is doing multiple stages, you will love the ability to > store staged log codes and messages and then bring the whole lot back when > it is finished, and still being 1M records as well ! > > Thanks > > Mark > > > > > 2009/11/26 jwcolby > >> If a parameter in a stored procedure is declared as an output (returns a >> value) will the parameter >> object in C# / ADO contain that value when the stored procedure finishes? >> >> I haven't used output parameters yet in stored procedures. Can values be >> passed in via an output >> parameter or can it only be set from inside of the stored procedure? >> >> -- >> John W. Colby >> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From marklbreen at gmail.com Thu Nov 26 09:59:22 2009 From: marklbreen at gmail.com (Mark Breen) Date: Thu, 26 Nov 2009 15:59:22 +0000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> Message-ID: Hello Shamil and Stuart, I never knew params could be bi-directional, thanks a lot for that, Mark 2009/11/26 Shamil Salakhetdinov > Hi Mark, > > > assume that you cannot have bi-directional. > You can: > > System.Data.ParameterDirection.InputOutput > > -- > Shamil > > On 26 Nov 2009 at 9:04, Mark Breen wrote: > > > > afaik, a param can be input (the default direction) or output. I have > not > > heard of an inputoutput param so unless anyone here says different, > assume > > that you cannot have bi-directional. > > > > > > > > __________ Information from ESET NOD32 Antivirus, version of virus > signature > database 4638 (20091126) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Thu Nov 26 10:04:07 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 26 Nov 2009 11:04:07 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> References: <4B0DE6F4.3070605@colbyconsulting.com>, <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> Message-ID: <4B0EA6F7.4050600@colbyconsulting.com> Thanks Stuart. John W. Colby www.ColbyConsulting.com Stuart McLachlan wrote: > SP with a parameter that is both Input and Output > > In some situations, there is no need to store the output into an extra variable because the > output must replace the input. > > CREATE PROCEDURE cap2 (@name NVARCHAR(20) OUT ) > --Capitalizes first char, changes the rest to lower case > --SP with one input/output parameter > AS > SET @name=UPPER(LEFT(@name,1))+LOWER(RIGHT(@name,LEN(@name)-1)) > > or for you .net types, see > http://fabioscagliola.spaces.live.com/blog/cns!919F8FCDE3DC9AC4!121.entry > From jwcolby at colbyconsulting.com Thu Nov 26 10:05:27 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 26 Nov 2009 11:05:27 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> References: <4B0DE6F4.3070605@colbyconsulting.com>, <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> Message-ID: <4B0EA747.4020506@colbyconsulting.com> > System.Data.ParameterDirection.InputOutput Thanks Shamil, What do you do with this? Set it to true? Set something else equal to this (is this a constant)? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Mark, > >> assume that you cannot have bi-directional. > You can: > > System.Data.ParameterDirection.InputOutput > > -- > Shamil > > On 26 Nov 2009 at 9:04, Mark Breen wrote: >> afaik, a param can be input (the default direction) or output. I have > not >> heard of an inputoutput param so unless anyone here says different, assume >> that you cannot have bi-directional. >> > > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4638 (20091126) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Thu Nov 26 10:54:43 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 26 Nov 2009 19:54:43 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0EA747.4020506@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com>, <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> <4B0EA747.4020506@colbyconsulting.com> Message-ID: <00de01ca6eb9$287b8830$79729890$@spb.ru> Hi John -- This is enumeration value - have a look here how to use it (near to the bottom): http://stackoverflow.com/questions/1637646/need-a-return-value-from-a-stored -procedure-in-c -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 26, 2009 7:05 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. > System.Data.ParameterDirection.InputOutput Thanks Shamil, What do you do with this? Set it to true? Set something else equal to this (is this a constant)? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Mark, > >> assume that you cannot have bi-directional. > You can: > > System.Data.ParameterDirection.InputOutput > > -- > Shamil > > On 26 Nov 2009 at 9:04, Mark Breen wrote: >> afaik, a param can be input (the default direction) or output. I have > not >> heard of an inputoutput param so unless anyone here says different, assume >> that you cannot have bi-directional. >> > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4639 (20091126) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From marklbreen at gmail.com Fri Nov 27 04:00:01 2009 From: marklbreen at gmail.com (Mark Breen) Date: Fri, 27 Nov 2009 10:00:01 +0000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0EA747.4020506@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> <4B0EA747.4020506@colbyconsulting.com> Message-ID: Hello John, that Shamil has said is that you can use one param to pass in a value, and use the same param to accept back a value, which may be different to when it was passes in. You can pass in ProductId, and get back, ProductCode, or pass in username and get back fullname, or pass in product code and get back zero if the product code does not exist, else you get back the code you passed in or pass in productid and get back qty in stock, all with one param. My examples are not great, but there are plenty of times where you need to somehow inspect a value and it may not be ideal to have two params named paramOriginal paramNewValue with inputout you can have just one param named paramToBeVerified Thanks again to Shamil, Stuart and Mr Colby for asking the question. Mark 2009/11/26 jwcolby > > System.Data.ParameterDirection.InputOutput > > Thanks Shamil, > > What do you do with this? Set it to true? Set something else equal to > this (is this a constant)? > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: > > Hi Mark, > > > >> assume that you cannot have bi-directional. > > You can: > > > > System.Data.ParameterDirection.InputOutput > > > > -- > > Shamil > > > > On 26 Nov 2009 at 9:04, Mark Breen wrote: > >> afaik, a param can be input (the default direction) or output. I have > > not > >> heard of an inputoutput param so unless anyone here says different, > assume > >> that you cannot have bi-directional. > >> > > > > > > > > > > > > __________ Information from ESET NOD32 Antivirus, version of virus > signature > > database 4638 (20091126) __________ > > > > The message was checked by ESET NOD32 Antivirus. > > > > http://www.esetnod32.ru > > > > > > _______________________________________________ > > dba-VB mailing list > > dba-VB at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > http://www.databaseadvisors.com > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Fri Nov 27 04:44:46 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Fri, 27 Nov 2009 11:44:46 +0100 Subject: [dba-VB] Parameter objects in C# and ADO. Message-ID: Hi Mark et al What's the big deal? Are parameters costly? Hardly in money, but in resources? /gustav >>> marklbreen at gmail.com 27-11-2009 11:00 >>> Hello John, that Shamil has said is that you can use one param to pass in a value, and use the same param to accept back a value, which may be different to when it was passes in. You can pass in ProductId, and get back, ProductCode, or pass in username and get back fullname, or pass in product code and get back zero if the product code does not exist, else you get back the code you passed in or pass in productid and get back qty in stock, all with one param. My examples are not great, but there are plenty of times where you need to somehow inspect a value and it may not be ideal to have two params named paramOriginal paramNewValue with inputout you can have just one param named paramToBeVerified Thanks again to Shamil, Stuart and Mr Colby for asking the question. Mark From shamil at smsconsulting.spb.ru Fri Nov 27 06:43:24 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 27 Nov 2009 15:43:24 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: Message-ID: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> Hi Gustav at all, BTW, have you ever seen the following below error handling in CRUD SPs as I have found in one of my customers' databases? Isn't an overkill? Or do you use even more detailed/elaborated error handling in your CRUD SPs? How do you organize error handling in your CRUD SPs? (I usually just return error code in RETURN statement). CREATE procedure [dbo].[GettblUserByUsername] @Username varchar(20), @ErrorDesc varchar(100) output, @ErrorNo int output, @RowsAffected int output AS DECLARE @lErrorNo INTEGER DECLARE @lRowsAffected INTEGER Begin SELECT tblUser.tblUserId, tblUser.tblUserGroupId, tblUserGroup.groupCode as UserGroup, tblUser.Username, tblUser.tblUserLevelId FROM tblUser left outer join tblUserGroup on tblUser.tblUserGroupId = tblUserGroup.tblUserGroupId WHERE tblUser.Username = @username and tblUser.Authorised = 1 select @lErrorNo = @@ERROR, @lRowsAffected = @@ROWCOUNT IF @lErrorNo > 0 Begin set @ErrorDesc = 'error with get of tblUser' set @ErrorNo = @lErrorNo set @RowsAffected = 0 RETURN End Else Begin set @ErrorDesc = 'Get was Successful for tblUser' set @ErrorNo = 0 set @RowsAffected = @lRowsAffected RETURN End end Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Friday, November 27, 2009 1:45 PM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] Parameter objects in C# and ADO. Hi Mark et al What's the big deal? Are parameters costly? Hardly in money, but in resources? /gustav <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4641 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 27 08:11:29 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 09:11:29 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> References: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> Message-ID: <4B0FDE11.9070108@colbyconsulting.com> RowsAffected is not part of the error stuff and I definitely want that info in most cases. I haven't done this yet but I don't think that getting back the error code and description is excessive. If it is possible to easily and unambiguously look up the error code and get the exact description, each and every time, then passing back the description is not necessary. This is exactly the kind of thing I will be doing in the future. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Gustav at all, > > BTW, have you ever seen the following below error handling in CRUD SPs as I > have found in one of my customers' databases? > Isn't an overkill? > Or do you use even more detailed/elaborated error handling in your CRUD SPs? > How do you organize error handling in your CRUD SPs? (I usually just return > error code in RETURN statement). > > CREATE procedure [dbo].[GettblUserByUsername] > @Username varchar(20), > @ErrorDesc varchar(100) output, > @ErrorNo int output, > @RowsAffected int output > AS > DECLARE @lErrorNo INTEGER > DECLARE @lRowsAffected INTEGER > Begin > > SELECT tblUser.tblUserId, > tblUser.tblUserGroupId, > tblUserGroup.groupCode as UserGroup, > tblUser.Username, > tblUser.tblUserLevelId > FROM tblUser > left outer join tblUserGroup > on tblUser.tblUserGroupId = tblUserGroup.tblUserGroupId > WHERE tblUser.Username = @username > and tblUser.Authorised = 1 > > select @lErrorNo = @@ERROR, @lRowsAffected = @@ROWCOUNT > > IF @lErrorNo > 0 > Begin > set @ErrorDesc = 'error with get of tblUser' > set @ErrorNo = @lErrorNo > set @RowsAffected = 0 > RETURN > End > Else > Begin > set @ErrorDesc = 'Get was Successful for tblUser' > set @ErrorNo = 0 > set @RowsAffected = @lRowsAffected > RETURN > End > end > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, November 27, 2009 1:45 PM > To: dba-vb at databaseadvisors.com > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > Hi Mark et al > > What's the big deal? Are parameters costly? Hardly in money, but in > resources? > > /gustav > > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4641 (20091127) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From marklbreen at gmail.com Fri Nov 27 08:33:52 2009 From: marklbreen at gmail.com (Mark Breen) Date: Fri, 27 Nov 2009 14:33:52 +0000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: Message-ID: Hello Gustav, I presume they are not costly in terms of resources, and in fact, I would not dream of re-using a param for two different purposes. I think my delight was just that I never know you could share directions. I can imagine that Mr Colby will find a use for a standard piece of his arsenal to have a in-out param and once he does, he will wonder how we ever existing without utilising that aspect of params. Perhaps lastupdated as a field would be an appropriate use, where the last updated value is passed in to the sproc, and then passed back again with the new value after an update has been made, after checking for concurrency errors first. Thanks Mark 2009/11/27 Gustav Brock > Hi Mark et al > > What's the big deal? Are parameters costly? Hardly in money, but in > resources? > > /gustav > > > >>> marklbreen at gmail.com 27-11-2009 11:00 >>> > Hello John, > > that Shamil has said is that you can use one param to pass in a value, and > use the same param to accept back a value, which may be different to when > it > was passes in. > > You can pass in ProductId, and get back, ProductCode, > or pass in username and get back fullname, > > or pass in product code and get back zero if the product code does not > exist, else you get back the code you passed in > or pass in productid and get back qty in stock, > > all with one param. > > My examples are not great, but there are plenty of times where you need to > somehow inspect a value and it may not be ideal to have two params named > paramOriginal > paramNewValue > > with inputout you can have just one param named paramToBeVerified > > Thanks again to Shamil, Stuart and Mr Colby for asking the question. > > Mark > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 27 08:44:16 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 09:44:16 -0500 Subject: [dba-VB] SPAM-LOW: Re: Parameter objects in C# and ADO. In-Reply-To: References: Message-ID: <4B0FE5C0.5030409@colbyconsulting.com> > I can imagine that Mr Colby will find a use for a standard piece of his arsenal to have a in-out param and once he does, he will wonder how we ever existing without utilizing that aspect of params. LOL. Your confidence is encouraging. ;) While I use "by reference" values in normal programming, it is rare that I actually modify the passed in value. But it is something that can theoretically be useful and so it is nice to know it is available. > "Mr Colby" I assume you are either feeling particularly young this week or you discovered how old I actually am? ;) All of my kid's friends call me "Mr. Colby". Have you been hanging out with Robbie? John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello Gustav, > > I presume they are not costly in terms of resources, and in fact, I would > not dream of re-using a param for two different purposes. > > I think my delight was just that I never know you could share directions. > > I can imagine that Mr Colby will find a use for a standard piece of his > arsenal to have a in-out param and once he does, he will wonder how we ever > existing without utilising that aspect of params. > > Perhaps lastupdated as a field would be an appropriate use, where the last > updated value is passed in to the sproc, and then passed back again with the > new value after an update has been made, after checking for concurrency > errors first. > > Thanks > > Mark From shamil at smsconsulting.spb.ru Fri Nov 27 08:59:16 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 27 Nov 2009 17:59:16 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0FDE11.9070108@colbyconsulting.com> References: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> <4B0FDE11.9070108@colbyconsulting.com> Message-ID: <000d01ca6f72$31f97f70$95ec7e50$@spb.ru> Hi John -- This kind of "excessive" execution result/error information is useful in my opinion for non-CRUD SPs. But for CRUD ones one can just use SPs return value: - if it's zero or positive - all is OK and returned value is @@ROWCOUNT; - if it's negative - then this is @@ERROR (if actual @@ERROR value will be positive just return it as negative number). If some kinds of runtime errors happen in SPs then they can be trapped by calling C# code try/catch block and actual error message can be obtained from System.Data.SqlClient.SqlException. Well, above is what I'd call a "lightweight" error handling approach - and it works well in many real life systems AFAIK... But for long running (set of) SPs one can use "heavyweight" error handling approach using T-SQLs TRY...CATCH features as described in MS SQL books online (local link): ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/tsqlref9/html/248df62a-7334-4bca-8262- 235a28f4b07f.htm I could be missing something. Please correct me where I'm wrong, please write about your approach to T-SQL (and calling C# code) error handling. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 27, 2009 5:11 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. RowsAffected is not part of the error stuff and I definitely want that info in most cases. I haven't done this yet but I don't think that getting back the error code and description is excessive. If it is possible to easily and unambiguously look up the error code and get the exact description, each and every time, then passing back the description is not necessary. This is exactly the kind of thing I will be doing in the future. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Gustav at all, > > BTW, have you ever seen the following below error handling in CRUD SPs as I > have found in one of my customers' databases? > Isn't an overkill? > Or do you use even more detailed/elaborated error handling in your CRUD SPs? > How do you organize error handling in your CRUD SPs? (I usually just return > error code in RETURN statement). > > CREATE procedure [dbo].[GettblUserByUsername] > @Username varchar(20), > @ErrorDesc varchar(100) output, > @ErrorNo int output, > @RowsAffected int output > AS > DECLARE @lErrorNo INTEGER > DECLARE @lRowsAffected INTEGER > Begin > > SELECT tblUser.tblUserId, > tblUser.tblUserGroupId, > tblUserGroup.groupCode as UserGroup, > tblUser.Username, > tblUser.tblUserLevelId > FROM tblUser > left outer join tblUserGroup > on tblUser.tblUserGroupId = tblUserGroup.tblUserGroupId > WHERE tblUser.Username = @username > and tblUser.Authorised = 1 > > select @lErrorNo = @@ERROR, @lRowsAffected = @@ROWCOUNT > > IF @lErrorNo > 0 > Begin > set @ErrorDesc = 'error with get of tblUser' > set @ErrorNo = @lErrorNo > set @RowsAffected = 0 > RETURN > End > Else > Begin > set @ErrorDesc = 'Get was Successful for tblUser' > set @ErrorNo = 0 > set @RowsAffected = @lRowsAffected > RETURN > End > end > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, November 27, 2009 1:45 PM > To: dba-vb at databaseadvisors.com > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > Hi Mark et al > > What's the big deal? Are parameters costly? Hardly in money, but in > resources? > > /gustav > > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4641 (20091127) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4641 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4641 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 27 09:17:22 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 10:17:22 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: Message-ID: <4B0FED82.3080608@colbyconsulting.com> Over the last year or so I have built up an arsenal of stored procedures which form a system for performing big picture tasks. A set of SPs create a set of temp tables, move data into them, index them, the export chunks into large CSV files for address validation. Similar functionality on the way back in, create a chunk temp table, import the data from a CSV file, copy to a big temp table, create indexes on them, update hash codes and address valid flags, create more indexes on those fields etc. To this point I have executed these stored procedures from an Access database, simply because I could write Access VBA, however VBA is less than ideal for this functionality, not to mention that I had no ability to get data back from the SP. Thus the SP had to write to a log file which I could then read from Access. Again, less than ideal. Now that I am (slooooowwwwly) coming up to speed on C#, I have embarked on rewriting all of that control logic into C#. Once I figure out how to read back parameters from the SP so that I can see things like RecordsAffected, error numbers and messages etc. I will have made a giant step forward in fine grained control of my big picture processes. While I am at it I have to modify the stored procedures to pass back things like recordcount affected, error codes etc. Since I could not (did not know how to) get at them from VBA I hadn't bothered to add that kind of stuff to the SPs. John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello Gustav, > > I presume they are not costly in terms of resources, and in fact, I would > not dream of re-using a param for two different purposes. > > I think my delight was just that I never know you could share directions. > > I can imagine that Mr Colby will find a use for a standard piece of his > arsenal to have a in-out param and once he does, he will wonder how we ever > existing without utilising that aspect of params. > > Perhaps lastupdated as a field would be an appropriate use, where the last > updated value is passed in to the sproc, and then passed back again with the > new value after an update has been made, after checking for concurrency > errors first. > > Thanks > > Mark From jwcolby at colbyconsulting.com Fri Nov 27 09:24:15 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 10:24:15 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <000d01ca6f72$31f97f70$95ec7e50$@spb.ru> References: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> <4B0FDE11.9070108@colbyconsulting.com> <000d01ca6f72$31f97f70$95ec7e50$@spb.ru> Message-ID: <4B0FEF1F.8060905@colbyconsulting.com> Shamil, I am looking for guidance in this area myself. As you know by now, the particular application I am working on is not CRUD based objects but rather specific sets of SPs to perform specific operations in my business. To this point, I have used VBA and a function for executing a stored procedure provided by Charlotte. I could never figure out how to use that function to get at returned values so I have to this point not used that functionality at all. Now I expect to be able to do so, and as a result I will be expanding my processing to include such error handling. It will be a long time before I am correcting anyone on this stuff! John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > This kind of "excessive" execution result/error information is useful in my > opinion for non-CRUD SPs. > But for CRUD ones one can just use SPs return value: > > - if it's zero or positive - all is OK and returned value is @@ROWCOUNT; > - if it's negative - then this is @@ERROR (if actual @@ERROR value will be > positive just return it as negative number). > > If some kinds of runtime errors happen in SPs then they can be trapped by > calling C# code try/catch block and actual error message can be obtained > from System.Data.SqlClient.SqlException. > > Well, above is what I'd call a "lightweight" error handling approach - and > it works well in many real life systems AFAIK... > > But for long running (set of) SPs one can use "heavyweight" error handling > approach using T-SQLs > > TRY...CATCH > > features as described in MS SQL books online (local link): > > ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/tsqlref9/html/248df62a-7334-4bca-8262- > 235a28f4b07f.htm > > I could be missing something. > Please correct me where I'm wrong, please write about your approach to T-SQL > (and calling C# code) error handling. > > Thank you. > > -- > Shamil From accessd at shaw.ca Fri Nov 27 11:50:56 2009 From: accessd at shaw.ca (Jim Lawrence) Date: Fri, 27 Nov 2009 09:50:56 -0800 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0FDE11.9070108@colbyconsulting.com> References: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> <4B0FDE11.9070108@colbyconsulting.com> Message-ID: <805FE569738A4777BD83069AB8C812CE@creativesystemdesigns.com> Hi John: ADO always returns the number of rows affected in MS SQL SP call. It just has to be retrieved. Here is the code that I used as a sample/base before. http://msdn.microsoft.com/en-us/library/aa302325.aspx Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 27, 2009 6:11 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. RowsAffected is not part of the error stuff and I definitely want that info in most cases. I haven't done this yet but I don't think that getting back the error code and description is excessive. If it is possible to easily and unambiguously look up the error code and get the exact description, each and every time, then passing back the description is not necessary. This is exactly the kind of thing I will be doing in the future. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Gustav at all, > > BTW, have you ever seen the following below error handling in CRUD SPs as I > have found in one of my customers' databases? > Isn't an overkill? > Or do you use even more detailed/elaborated error handling in your CRUD SPs? > How do you organize error handling in your CRUD SPs? (I usually just return > error code in RETURN statement). > > CREATE procedure [dbo].[GettblUserByUsername] > @Username varchar(20), > @ErrorDesc varchar(100) output, > @ErrorNo int output, > @RowsAffected int output > AS > DECLARE @lErrorNo INTEGER > DECLARE @lRowsAffected INTEGER > Begin > > SELECT tblUser.tblUserId, > tblUser.tblUserGroupId, > tblUserGroup.groupCode as UserGroup, > tblUser.Username, > tblUser.tblUserLevelId > FROM tblUser > left outer join tblUserGroup > on tblUser.tblUserGroupId = tblUserGroup.tblUserGroupId > WHERE tblUser.Username = @username > and tblUser.Authorised = 1 > > select @lErrorNo = @@ERROR, @lRowsAffected = @@ROWCOUNT > > IF @lErrorNo > 0 > Begin > set @ErrorDesc = 'error with get of tblUser' > set @ErrorNo = @lErrorNo > set @RowsAffected = 0 > RETURN > End > Else > Begin > set @ErrorDesc = 'Get was Successful for tblUser' > set @ErrorNo = 0 > set @RowsAffected = @lRowsAffected > RETURN > End > end > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, November 27, 2009 1:45 PM > To: dba-vb at databaseadvisors.com > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > Hi Mark et al > > What's the big deal? Are parameters costly? Hardly in money, but in > resources? > > /gustav > > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4641 (20091127) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Nov 27 14:26:11 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 15:26:11 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0DE6F4.3070605@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> Message-ID: <4B1035E3.7000003@colbyconsulting.com> > If a parameter in a stored procedure is declared as an output (returns a value) will the parameter object in C# / ADO contain that value when the stored procedure finishes? This part of the question got lost, not directly answered AFAICT. I am directly creating parameter objects and adding them to the parameter collection of the command object. Code execution will halt until the stored procedure finishes executing and then the parameter objects in the command object's parameters collection can be looked at for the returned values? John W. Colby www.ColbyConsulting.com jwcolby wrote: > If a parameter in a stored procedure is declared as an output (returns a value) will the parameter > object in C# / ADO contain that value when the stored procedure finishes? > > I haven't used output parameters yet in stored procedures. Can values be passed in via an output > parameter or can it only be set from inside of the stored procedure? > From shamil at smsconsulting.spb.ru Fri Nov 27 15:14:53 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 00:14:53 +0300 Subject: [dba-VB] Calling SP with params from C# sample Message-ID: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> Hi John - Here is a sample for you: public static void TestSP() { string connectionString = "{{Your connection string here}}"; //CREATE Procedure [dbo].[TestParams] // @Username varchar(20), // @ErrorDesc varchar(100) output, // @ErrorNo int output, // @RowsAffected int output AS //DECLARE @lErrorNo INTEGER //DECLARE @lRowsAffected INTEGER //Begin // set @ErrorDesc = 'SUCCESS' + ', UserName = ' + IsNull(@USerName,'?') // set @ErrorNo = 1 -- OK // set @RowsAffected = 503 // RETURN 1234 //end try { using (SqlConnection connection = new SqlConnection(connectionString)) { connection.Open(); SqlCommand command = new SqlCommand("TestParams", connection); command.CommandType = System.Data.CommandType.StoredProcedure; // @Username varchar(20), command.Parameters.Add( new SqlParameter("@Username", System.Data.SqlDbType.VarChar, 20)); command.Parameters["@UserName"].Value = "SP_PARAMS_TEST"; // @ErrorDesc varchar(100) output, command.Parameters.Add( new SqlParameter("@ErrorDesc", System.Data.SqlDbType.VarChar, 100)); command.Parameters["@ErrorDesc"].Direction = System.Data.ParameterDirection.Output; // @ErrorNo int output, command.Parameters.Add( new SqlParameter("@ErrorNo", System.Data.SqlDbType.Int, 4)); command.Parameters["@ErrorNo"].Direction = System.Data.ParameterDirection.Output; // @RowsAffected int output AS command.Parameters.Add( new SqlParameter("@RowsAffected", System.Data.SqlDbType.Int, 4)); command.Parameters["@RowsAffected"].Direction = System.Data.ParameterDirection.Output; // RETURN command.Parameters.Add( new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int, 4)); command.Parameters["@RETURN_VALUE"].Direction = System.Data.ParameterDirection.ReturnValue; int retValue = command.ExecuteNonQuery(); Console.WriteLine( "retValue = {0}\n" + "@RETURN_VALUE = {1}\n" + "@ErrorDesc = '{2}'\n" + "@ErrorNo = {3}\n" + "@RowsAffected = {4}" , retValue, command.Parameters["@RETURN_VALUE"].Value.ToString(), command.Parameters["@ErrorDesc"].Value.ToString(), command.Parameters["@ErrorNo"].Value.ToString(), command.Parameters["@RowsAffected"].Value.ToString() ); } } catch (SqlException ex) { Console.WriteLine("T-SQL: {0}", ex.Message); } catch (Exception ex) { Console.WriteLine("{0}", ex.Message); } } Result: retValue = -1 @RETURN_VALUE = 1234 @ErrorDesc = 'SUCCESS, UserName = SP_PARAMS_TEST' @ErrorNo = 1 @RowsAffected = 503 -- Shamil From shamil at smsconsulting.spb.ru Fri Nov 27 15:31:02 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 00:31:02 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B1035E3.7000003@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B1035E3.7000003@colbyconsulting.com> Message-ID: <009501ca6fa8$ecaf22d0$c60d6870$@spb.ru> Hi John -- I have just posted here a sample for synchronous execution of SP with params. You can also run SPs from C# asynchronously - e.g. look MSDN for BeginExecuteNonQuery method of System.Data.SqlClient.SqlCommand class. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 27, 2009 11:26 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. > If a parameter in a stored procedure is declared as an output (returns a value) will the parameter object in C# / ADO contain that value when the stored procedure finishes? This part of the question got lost, not directly answered AFAICT. I am directly creating parameter objects and adding them to the parameter collection of the command object. Code execution will halt until the stored procedure finishes executing and then the parameter objects in the command object's parameters collection can be looked at for the returned values? John W. Colby www.ColbyConsulting.com jwcolby wrote: > If a parameter in a stored procedure is declared as an output (returns a value) will the parameter > object in C# / ADO contain that value when the stored procedure finishes? > > I haven't used output parameters yet in stored procedures. Can values be passed in via an output > parameter or can it only be set from inside of the stored procedure? > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4643 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 27 15:40:45 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 16:40:45 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <009501ca6fa8$ecaf22d0$c60d6870$@spb.ru> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B1035E3.7000003@colbyconsulting.com> <009501ca6fa8$ecaf22d0$c60d6870$@spb.ru> Message-ID: <4B10475D.8040102@colbyconsulting.com> Very cool! I am rewriting the code I found on the internet to allow me to pass in the direction as well as manipulate the param object when it returns. Thanks again for the example code. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > I have just posted here a sample for synchronous execution of SP with > params. > You can also run SPs from C# asynchronously - e.g. look MSDN for > BeginExecuteNonQuery method of System.Data.SqlClient.SqlCommand class. > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 27, 2009 11:26 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > > If a parameter in a stored procedure is declared as an output (returns a > value) will the > parameter object in C# / ADO contain that value when the stored procedure > finishes? > > This part of the question got lost, not directly answered AFAICT. > > I am directly creating parameter objects and adding them to the parameter > collection of the command > object. Code execution will halt until the stored procedure finishes > executing and then the > parameter objects in the command object's parameters collection can be > looked at for the returned > values? > > John W. Colby > www.ColbyConsulting.com > > > jwcolby wrote: >> If a parameter in a stored procedure is declared as an output (returns a > value) will the parameter >> object in C# / ADO contain that value when the stored procedure finishes? >> >> I haven't used output parameters yet in stored procedures. Can values be > passed in via an output >> parameter or can it only be set from inside of the stored procedure? >> > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4643 (20091127) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Nov 27 16:02:17 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 01:02:17 +0300 Subject: [dba-VB] Async calling SP with params from C# sample In-Reply-To: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> Message-ID: <009601ca6fad$4a2737f0$de75a7d0$@spb.ru> Another sample - async call (partially used MSDN sample): public static void runTest() { SPTester o = new SPTester(); o.TestSP(); while (!o.Completed) System.Threading.Thread.Sleep(1000); } public class SPTester { public bool Completed { get; set; } private SqlConnection _connection; public void TestSP() { Completed = false; string connectionString = "Data Source=HAMBURG\\SQL2005;"+ "Initial Catalog=X;"+ "User Id=sa;Password=Y;"+ "Asynchronous Processing=true"; //CREATE Procedure [dbo].[TestParams] // @Username varchar(20), // @ErrorDesc varchar(100) output, // @ErrorNo int output, // @RowsAffected int output AS //DECLARE @lErrorNo INTEGER //DECLARE @lRowsAffected INTEGER //Begin // set @ErrorDesc = 'SUCCESS' + ', UserName = ' + IsNull(@USerName,'?') // set @ErrorNo = 1 -- OK // set @RowsAffected = 503 // RETURN 1234 //end try { _connection = new SqlConnection(connectionString); { _connection.Open(); SqlCommand command = new SqlCommand("TestParams", _connection); command.CommandType = System.Data.CommandType.StoredProcedure; // @Username varchar(20), command.Parameters.Add( new SqlParameter("@Username", System.Data.SqlDbType.VarChar, 20)); command.Parameters["@UserName"].Value = "SP_PARAMS_TEST"; // @ErrorDesc varchar(100) output, command.Parameters.Add( new SqlParameter("@ErrorDesc", System.Data.SqlDbType.VarChar, 100)); command.Parameters["@ErrorDesc"].Direction = System.Data.ParameterDirection.Output; // @ErrorNo int output, command.Parameters.Add( new SqlParameter("@ErrorNo", System.Data.SqlDbType.Int, 4)); command.Parameters["@ErrorNo"].Direction = System.Data.ParameterDirection.Output; // @RowsAffected int output AS command.Parameters.Add( new SqlParameter("@RowsAffected", System.Data.SqlDbType.Int, 4)); command.Parameters["@RowsAffected"].Direction = System.Data.ParameterDirection.Output; // RETURN command.Parameters.Add( new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int, 4)); command.Parameters["@RETURN_VALUE"].Direction = System.Data.ParameterDirection.ReturnValue; AsyncCallback callback = new AsyncCallback(handleCallback); command.BeginExecuteNonQuery(callback, command); } } catch (Exception ex) { Console.WriteLine(ex.Message); if (_connection != null) { _connection.Close(); _connection = null; } } } private void handleCallback(IAsyncResult result) { try { SqlCommand command = (SqlCommand)result.AsyncState; int rowCount = command.EndExecuteNonQuery(result); Console.WriteLine( "rowCount = {0}\n" + "@RETURN_VALUE = {1}\n" + "@ErrorDesc = '{2}'\n" + "@ErrorNo = {3}\n" + "@RowsAffected = {4}" , rowCount, command.Parameters["@RETURN_VALUE"].Value.ToString(), command.Parameters["@ErrorDesc"].Value.ToString(), command.Parameters["@ErrorNo"].Value.ToString(), command.Parameters["@RowsAffected"].Value.ToString() ); } catch (SqlException ex) { Console.WriteLine("T-SQL: {0}", ex.Message); } catch (Exception ex) { Console.WriteLine("{0}", ex.Message); } finally { Completed = true; if (_connection != null) { _connection.Close(); _connection = null; } } } } -- Shamil From shamil at smsconsulting.spb.ru Fri Nov 27 16:09:52 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 01:09:52 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B10475D.8040102@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B1035E3.7000003@colbyconsulting.com> <009501ca6fa8$ecaf22d0$c60d6870$@spb.ru> <4B10475D.8040102@colbyconsulting.com> Message-ID: <009701ca6fae$591f3310$0b5d9930$@spb.ru> Hi John -- You're welcome! I have just posted another sample code where SP call from C# is processed asynchronously. -- Shamil P.S. There is an issue with the code I mentioned above - Completed flag should be also set to true in the catch() block of the calling method: public void TestSP() -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 28, 2009 12:41 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. Very cool! I am rewriting the code I found on the internet to allow me to pass in the direction as well as manipulate the param object when it returns. Thanks again for the example code. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > I have just posted here a sample for synchronous execution of SP with > params. > You can also run SPs from C# asynchronously - e.g. look MSDN for > BeginExecuteNonQuery method of System.Data.SqlClient.SqlCommand class. > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 27, 2009 11:26 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > > If a parameter in a stored procedure is declared as an output (returns a > value) will the > parameter object in C# / ADO contain that value when the stored procedure > finishes? > > This part of the question got lost, not directly answered AFAICT. > > I am directly creating parameter objects and adding them to the parameter > collection of the command > object. Code execution will halt until the stored procedure finishes > executing and then the > parameter objects in the command object's parameters collection can be > looked at for the returned > values? > > John W. Colby > www.ColbyConsulting.com > > > jwcolby wrote: >> If a parameter in a stored procedure is declared as an output (returns a > value) will the parameter >> object in C# / ADO contain that value when the stored procedure finishes? >> >> I haven't used output parameters yet in stored procedures. Can values be > passed in via an output >> parameter or can it only be set from inside of the stored procedure? >> > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4643 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 27 21:15:28 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 22:15:28 -0500 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> Message-ID: <4B1095D0.2050804@colbyconsulting.com> Hi Shamil, I assume that the parameters on the C# side have to be in the same order / name / datatype / size as the parameters in the stored procedure? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John - > > > > Here is a sample for you: > > > > public static void TestSP() > > { > > string connectionString = "{{Your connection string here}}"; > > > > //CREATE Procedure [dbo].[TestParams] > > // @Username varchar(20), > > // @ErrorDesc varchar(100) output, > > // @ErrorNo int output, > > // @RowsAffected int output AS > > //DECLARE @lErrorNo INTEGER > > //DECLARE @lRowsAffected INTEGER > > //Begin > > // set @ErrorDesc = 'SUCCESS' + ', UserName = ' + IsNull(@USerName,'?') > > // set @ErrorNo = 1 -- OK > > // set @RowsAffected = 503 > > // RETURN 1234 > > //end > > > > try > > { > > using (SqlConnection connection = new > SqlConnection(connectionString)) > > { > > connection.Open(); > > > > SqlCommand command = new SqlCommand("TestParams", connection); > > command.CommandType = System.Data.CommandType.StoredProcedure; > > // @Username varchar(20), > > command.Parameters.Add( > > new SqlParameter("@Username", System.Data.SqlDbType.VarChar, > 20)); > > command.Parameters["@UserName"].Value = "SP_PARAMS_TEST"; > > // @ErrorDesc varchar(100) output, > > command.Parameters.Add( > > new SqlParameter("@ErrorDesc", > System.Data.SqlDbType.VarChar, 100)); > > command.Parameters["@ErrorDesc"].Direction = > System.Data.ParameterDirection.Output; > > // @ErrorNo int output, > > command.Parameters.Add( > > new SqlParameter("@ErrorNo", System.Data.SqlDbType.Int, 4)); > > command.Parameters["@ErrorNo"].Direction = > System.Data.ParameterDirection.Output; > > // @RowsAffected int output AS > > command.Parameters.Add( > > new SqlParameter("@RowsAffected", System.Data.SqlDbType.Int, > 4)); > > command.Parameters["@RowsAffected"].Direction = > System.Data.ParameterDirection.Output; > > // RETURN > > command.Parameters.Add( > > new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int, > 4)); > > command.Parameters["@RETURN_VALUE"].Direction = > System.Data.ParameterDirection.ReturnValue; > > > > int retValue = command.ExecuteNonQuery(); > > > > Console.WriteLine( > > "retValue = {0}\n" + > > "@RETURN_VALUE = {1}\n" + > > "@ErrorDesc = '{2}'\n" + > > "@ErrorNo = {3}\n" + > > "@RowsAffected = {4}" > > , > > retValue, > > command.Parameters["@RETURN_VALUE"].Value.ToString(), > > command.Parameters["@ErrorDesc"].Value.ToString(), > > command.Parameters["@ErrorNo"].Value.ToString(), > > command.Parameters["@RowsAffected"].Value.ToString() > > ); > > } > > } > > catch (SqlException ex) > > { > > Console.WriteLine("T-SQL: {0}", ex.Message); > > } > > catch (Exception ex) > > { > > Console.WriteLine("{0}", ex.Message); > > } > > } > > > > Result: > > > > retValue = -1 > > @RETURN_VALUE = 1234 > > @ErrorDesc = 'SUCCESS, UserName = SP_PARAMS_TEST' > > @ErrorNo = 1 > > @RowsAffected = 503 > > > > -- > > Shamil > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 27 21:45:19 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 22:45:19 -0500 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> Message-ID: <4B109CCF.5020107@colbyconsulting.com> Shamil, I executed my first SP using your example code (first light as they say in the astronomy world). Man oh man is it ugly though. Tons of code to get one lonely little SP to execute (NOT your fault!!!). Thank you so much for the examples, I can definitely take it from here. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John - > > > > Here is a sample for you: > > > > public static void TestSP() > > { > > string connectionString = "{{Your connection string here}}"; > > > > //CREATE Procedure [dbo].[TestParams] > > // @Username varchar(20), > > // @ErrorDesc varchar(100) output, > > // @ErrorNo int output, > > // @RowsAffected int output AS > > //DECLARE @lErrorNo INTEGER > > //DECLARE @lRowsAffected INTEGER > > //Begin > > // set @ErrorDesc = 'SUCCESS' + ', UserName = ' + IsNull(@USerName,'?') > > // set @ErrorNo = 1 -- OK > > // set @RowsAffected = 503 > > // RETURN 1234 > > //end > > > > try > > { > > using (SqlConnection connection = new > SqlConnection(connectionString)) > > { > > connection.Open(); > > > > SqlCommand command = new SqlCommand("TestParams", connection); > > command.CommandType = System.Data.CommandType.StoredProcedure; > > // @Username varchar(20), > > command.Parameters.Add( > > new SqlParameter("@Username", System.Data.SqlDbType.VarChar, > 20)); > > command.Parameters["@UserName"].Value = "SP_PARAMS_TEST"; > > // @ErrorDesc varchar(100) output, > > command.Parameters.Add( > > new SqlParameter("@ErrorDesc", > System.Data.SqlDbType.VarChar, 100)); > > command.Parameters["@ErrorDesc"].Direction = > System.Data.ParameterDirection.Output; > > // @ErrorNo int output, > > command.Parameters.Add( > > new SqlParameter("@ErrorNo", System.Data.SqlDbType.Int, 4)); > > command.Parameters["@ErrorNo"].Direction = > System.Data.ParameterDirection.Output; > > // @RowsAffected int output AS > > command.Parameters.Add( > > new SqlParameter("@RowsAffected", System.Data.SqlDbType.Int, > 4)); > > command.Parameters["@RowsAffected"].Direction = > System.Data.ParameterDirection.Output; > > // RETURN > > command.Parameters.Add( > > new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int, > 4)); > > command.Parameters["@RETURN_VALUE"].Direction = > System.Data.ParameterDirection.ReturnValue; > > > > int retValue = command.ExecuteNonQuery(); > > > > Console.WriteLine( > > "retValue = {0}\n" + > > "@RETURN_VALUE = {1}\n" + > > "@ErrorDesc = '{2}'\n" + > > "@ErrorNo = {3}\n" + > > "@RowsAffected = {4}" > > , > > retValue, > > command.Parameters["@RETURN_VALUE"].Value.ToString(), > > command.Parameters["@ErrorDesc"].Value.ToString(), > > command.Parameters["@ErrorNo"].Value.ToString(), > > command.Parameters["@RowsAffected"].Value.ToString() > > ); > > } > > } > > catch (SqlException ex) > > { > > Console.WriteLine("T-SQL: {0}", ex.Message); > > } > > catch (Exception ex) > > { > > Console.WriteLine("{0}", ex.Message); > > } > > } > > > > Result: > > > > retValue = -1 > > @RETURN_VALUE = 1234 > > @ErrorDesc = 'SUCCESS, UserName = SP_PARAMS_TEST' > > @ErrorNo = 1 > > @RowsAffected = 503 > > > > -- > > Shamil > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Nov 28 03:19:35 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 12:19:35 +0300 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <4B1095D0.2050804@colbyconsulting.com> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B1095D0.2050804@colbyconsulting.com> Message-ID: <00a201ca700b$e98b6ba0$bca242e0$@spb.ru> Hi John, The order of parameters' creation on C# side doesn't matter, name and datatype and size do matter. There could be some variations for the latter two but better keep them in strict correspondence. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 28, 2009 6:15 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Calling SP with params from C# sample Hi Shamil, I assume that the parameters on the C# side have to be in the same order / name / datatype / size as the parameters in the stored procedure? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John - > > > > Here is a sample for you: > > > > public static void TestSP() > <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4643 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From shamil at smsconsulting.spb.ru Sat Nov 28 03:19:35 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 12:19:35 +0300 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <4B109CCF.5020107@colbyconsulting.com> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B109CCF.5020107@colbyconsulting.com> Message-ID: <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> John -- Tons of code for C# (or VB.NET) is not an issue at all. And for C#/VB.Net programmers also. You'll get adapted to that reality soon, I believe. I have projects with literally thousands of custom classes - they work in production for several years without any issues. And they recompile from cleaned solution with 10+ projects in 10+ seconds. And when you're working on such solution and change code/recompile/run then recompiling and restarting takes <1-2 seconds as VS recompiles/rebuilds only changed projects. Have a look e.g. on DotNetNuke(DNN) sources - and if you'll try to "dig them out" you'll find how "ugly" DNN sources are sometimes... For your case I'd use: - plain "dumb" manual coding if you have relatively static set of SPs and their parameters; - LINQ for SQL (now); - ADO.NET Entity Framework (VS2010); - statically generated custom classes to call SPs and process SPs' results. I'd not use dynamically generated SqlCommand and SqlParameters as this approach could result in too much and never ending "plumbing programming"... And in all the cases you can use the power of true OOP C# provides to minimize "copy and paste" coding approach... BTW, C# 4.0 provides some very powerful dynamic programming features - more powerful than late binding(?) - I must say I have seen some samples of that new C# 4.0 features usage but I didn't get how to use them :) - maybe they can help you to minimize "ugliness" of your C# coding but be careful to not spend too much time on "plumbing programming"... http://stackoverflow.com/questions/244302/what-do-you-think-of-the-new-c-4-0 -dynamic-keyword http://geekswithblogs.net/sdorman/archive/2008/11/16/c-4.0-dynamic-programmi ng.aspx -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 28, 2009 6:45 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Calling SP with params from C# sample Shamil, I executed my first SP using your example code (first light as they say in the astronomy world). Man oh man is it ugly though. Tons of code to get one lonely little SP to execute (NOT your fault!!!). Thank you so much for the examples, I can definitely take it from here. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John - > > > > Here is a sample for you: > > > > public static void TestSP() > > { <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4643 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From marklbreen at gmail.com Sat Nov 28 08:27:07 2009 From: marklbreen at gmail.com (Mark Breen) Date: Sat, 28 Nov 2009 14:27:07 +0000 Subject: [dba-VB] SPAM-LOW: Re: Parameter objects in C# and ADO. In-Reply-To: <4B0FE5C0.5030409@colbyconsulting.com> References: <4B0FE5C0.5030409@colbyconsulting.com> Message-ID: Haha, My kids call some people by their first name and I still call the same people Mr Dowling, sounds funny to hear it. Mark 2009/11/27 jwcolby > > I can imagine that Mr Colby will find a use for a standard piece of his > arsenal to have a in-out > param and once he does, he will wonder how we ever existing without > utilizing that aspect of params. > > LOL. Your confidence is encouraging. ;) > > While I use "by reference" values in normal programming, it is rare that I > actually modify the > passed in value. But it is something that can theoretically be useful and > so it is nice to know it > is available. > > > "Mr Colby" > > I assume you are either feeling particularly young this week or you > discovered how old I actually > am? ;) > > All of my kid's friends call me "Mr. Colby". Have you been hanging out > with Robbie? > > John W. Colby > www.ColbyConsulting.com > > > Mark Breen wrote: > > Hello Gustav, > > > > I presume they are not costly in terms of resources, and in fact, I would > > not dream of re-using a param for two different purposes. > > > > I think my delight was just that I never know you could share directions. > > > > I can imagine that Mr Colby will find a use for a standard piece of his > > arsenal to have a in-out param and once he does, he will wonder how we > ever > > existing without utilising that aspect of params. > > > > Perhaps lastupdated as a field would be an appropriate use, where the > last > > updated value is passed in to the sproc, and then passed back again with > the > > new value after an update has been made, after checking for concurrency > > errors first. > > > > Thanks > > > > Mark > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Nov 28 16:29:17 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 28 Nov 2009 17:29:17 -0500 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B109CCF.5020107@colbyconsulting.com> <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> Message-ID: <4B11A43D.70104@colbyconsulting.com> Shamil, I ended up creating a "stored procedure class". This class hosts a sCmd command object that can be programmed with the name of the SP, connection etc but then included wrapping the lines of code to create specific often used parameters into functions. /// /// The following code creates standard parameters used all of the time in my stored procedures /// Basically just wrapper code so that I can call a function to create specific parameters as needed /// /// It also allows me to completely standardize the names, data type and length of parameters in the SPs etc. /// /// public void paramDBName(string lstrDBName) { sCmd.Parameters.Add(new SqlParameter( "@DBName", SqlDbType.VarChar, 50)); sCmd.Parameters["@DBName"].Value = lstrDBName; } public void paramTblName(string lstrTblName) { sCmd.Parameters.Add(new SqlParameter("@TblName", SqlDbType.VarChar, 50)); sCmd.Parameters["@TblName"].Value = lstrTblName; } public void paramErrorDesc() { sCmd.Parameters.Add(new SqlParameter("@ErrorDesc", System.Data.SqlDbType.VarChar, 4000)); sCmd.Parameters["@ErrorDesc"].Direction = System.Data.ParameterDirection.Output; } Then I can do things like: // //sp_AZOut_AZExportToOrdered_Append // sp = new StoredProcedure(myConnection, "_aDataMaster.dbo.sp_AZOut_AZExportToOrdered_Append"); sp.paramDBName(lstrDBName); //@DBName varchar(50) input sp.paramTblName(lstrTblName); //@TblName varchar(50) input sp.paramErrorDesc(); //@ErrorDesc varchar(100) output sp.paramErrorNo(); //@ErrorNo int output, sp.paramReturnValue(); //RETURN retValue = sp.ExecuteSP(); I believe I am on my way now. Thanks again, John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > John -- > > Tons of code for C# (or VB.NET) is not an issue at all. And for C#/VB.Net > programmers also. You'll get adapted to that reality soon, I believe. From shamil at smsconsulting.spb.ru Sat Nov 28 17:12:11 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sun, 29 Nov 2009 02:12:11 +0300 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <4B11A43D.70104@colbyconsulting.com> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B109CCF.5020107@colbyconsulting.com> <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> <4B11A43D.70104@colbyconsulting.com> Message-ID: <00ba01ca7080$392fd000$ab8f7000$@spb.ru> John, Yes, that looks good with me. AFAIS you are getting what is usually declared as abstract class, and it can be inherited by custom classes, which would use their base abstract class's "services"/utilities methods. I suppose you can go even a bit further by combining sets of your standard parameters assignments into methods of your base abstract class. BTW, all the methods/properties of your abstract class, which are not planned to be called from outside of custom classes can be declared as 'protected', all the local class level common variables (of all kinds/types) can be also put with 'protected' declaration into your abstract class... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Sunday, November 29, 2009 1:29 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Calling SP with params from C# sample Shamil, I ended up creating a "stored procedure class". This class hosts a sCmd command object that can be programmed with the name of the SP, connection etc but then included wrapping the lines of code to create specific often used parameters into functions. /// /// The following code creates standard parameters used all of the time in my stored procedures /// Basically just wrapper code so that I can call a function to create specific parameters as needed /// /// It also allows me to completely standardize the names, data type and length of parameters in the SPs etc. /// /// public void paramDBName(string lstrDBName) { sCmd.Parameters.Add(new SqlParameter( "@DBName", SqlDbType.VarChar, 50)); sCmd.Parameters["@DBName"].Value = lstrDBName; } public void paramTblName(string lstrTblName) { sCmd.Parameters.Add(new SqlParameter("@TblName", SqlDbType.VarChar, 50)); sCmd.Parameters["@TblName"].Value = lstrTblName; } public void paramErrorDesc() { sCmd.Parameters.Add(new SqlParameter("@ErrorDesc", System.Data.SqlDbType.VarChar, 4000)); sCmd.Parameters["@ErrorDesc"].Direction = System.Data.ParameterDirection.Output; } Then I can do things like: // //sp_AZOut_AZExportToOrdered_Append // sp = new StoredProcedure(myConnection, "_aDataMaster.dbo.sp_AZOut_AZExportToOrdered_Append"); sp.paramDBName(lstrDBName); //@DBName varchar(50) input sp.paramTblName(lstrTblName); //@TblName varchar(50) input sp.paramErrorDesc(); //@ErrorDesc varchar(100) output sp.paramErrorNo(); //@ErrorNo int output, sp.paramReturnValue(); //RETURN retValue = sp.ExecuteSP(); I believe I am on my way now. Thanks again, John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > John -- > > Tons of code for C# (or VB.NET) is not an issue at all. And for C#/VB.Net > programmers also. You'll get adapted to that reality soon, I believe. __________ Information from ESET NOD32 Antivirus, version of virus signature database 4645 (20091128) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 28 17:19:19 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 28 Nov 2009 18:19:19 -0500 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <00ba01ca7080$392fd000$ab8f7000$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B109CCF.5020107@colbyconsulting.com> <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> <4B11A43D.70104@colbyconsulting.com> <00ba01ca7080$392fd000$ab8f7000$@spb.ru> Message-ID: <4B11AFF7.8010706@colbyconsulting.com> Perhaps I will do the inheritance thing once I get rolling. It certainly seems like a logical extension. For now I am just instantiating the SP class and manipulating methods of that class. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > John, > > Yes, that looks good with me. > AFAIS you are getting what is usually declared as abstract class, and it can > be inherited by custom classes, which would use their base abstract class's > "services"/utilities methods. I suppose you can go even a bit further by > combining sets of your standard parameters assignments into methods of your > base abstract class. BTW, all the methods/properties of your abstract class, > which are not planned to be called from outside of custom classes can be > declared as 'protected', all the local class level common variables (of all > kinds/types) can be also put with 'protected' declaration into your abstract > class... > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Sunday, November 29, 2009 1:29 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Calling SP with params from C# sample > > Shamil, > > I ended up creating a "stored procedure class". This class hosts a sCmd > command object that can be > programmed with the name of the SP, connection etc but then included > wrapping the lines of code to > create specific often used parameters into functions. > > /// > /// The following code creates standard parameters used all of the > time in my stored procedures > /// Basically just wrapper code so that I can call a function to > create specific parameters > as needed > /// > /// It also allows me to completely standardize the names, data > type and length of > parameters in the SPs etc. > /// > /// > public void paramDBName(string lstrDBName) > { > sCmd.Parameters.Add(new SqlParameter( "@DBName", > SqlDbType.VarChar, 50)); > sCmd.Parameters["@DBName"].Value = lstrDBName; > } > public void paramTblName(string lstrTblName) > { > sCmd.Parameters.Add(new SqlParameter("@TblName", > SqlDbType.VarChar, 50)); > sCmd.Parameters["@TblName"].Value = lstrTblName; > } > public void paramErrorDesc() > { > sCmd.Parameters.Add(new SqlParameter("@ErrorDesc", > System.Data.SqlDbType.VarChar, 4000)); > sCmd.Parameters["@ErrorDesc"].Direction = > System.Data.ParameterDirection.Output; > } > > Then I can do things like: > > // > //sp_AZOut_AZExportToOrdered_Append > // > sp = new StoredProcedure(myConnection, > "_aDataMaster.dbo.sp_AZOut_AZExportToOrdered_Append"); > sp.paramDBName(lstrDBName); //@DBName varchar(50) input > sp.paramTblName(lstrTblName); //@TblName varchar(50) input > sp.paramErrorDesc(); //@ErrorDesc varchar(100) > output > sp.paramErrorNo(); //@ErrorNo int output, > sp.paramReturnValue(); //RETURN > retValue = sp.ExecuteSP(); > > I believe I am on my way now. > > Thanks again, > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> John -- >> >> Tons of code for C# (or VB.NET) is not an issue at all. And for C#/VB.Net >> programmers also. You'll get adapted to that reality soon, I believe. > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4645 (20091128) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sun Nov 29 12:51:14 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 29 Nov 2009 13:51:14 -0500 Subject: [dba-VB] C# - Presenting process status Message-ID: <4B12C2A2.1010001@colbyconsulting.com> Just a general question about how you go about presenting process status back from classes. In Access I tended to use Withevents in my classes, then sink the class events in the form and use the event to pass in values to the form, and in the event sink display the status in a text box control on the form. For example the process that exports data from SQL Server to Accuzip (address validation) would display the name of the file just created and the PKFrom / PKTo pk range. I am also looking for tips on how to report status from a thread. I have vague recollections that a thread cannot directly update controls on a form (or something like that). So how do you folks display status information in a form? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sun Nov 29 15:33:05 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 30 Nov 2009 00:33:05 +0300 Subject: [dba-VB] C# - Presenting process status In-Reply-To: <4B12C2A2.1010001@colbyconsulting.com> References: <4B12C2A2.1010001@colbyconsulting.com> Message-ID: <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> Hi John, Here is one of the methods I use (there are zillion combinations) - I dunno where it comes from, did I borrowed it from somewhere or I got it elaborated myself, is it recommended way or not - but it just works, and I have "cooked" it for you here from scratch - this is codebehind for Windows Form named Form1: using System; using System.Windows.Forms; namespace WinFormsSample { public partial class Form1 : Form { // Create Windows Form Form1 with three controls: // // 1. Button: cmdTest1 -> cmdTest1_Click // 2. Button: cmdTest2 -> cmdTest2_Click // 3. TextBox: txtLog (Multiline = true) #region Test class public class Test { public delegate void ProgressStatus(string message); private static int _id; private System.Threading.Thread _thread; public void Run(ProgressStatus progressStatus) { _testId = ++_id; _progressStatus = progressStatus; System.Threading.ThreadStart ts = new System.Threading.ThreadStart(runThread); _thread = new System.Threading.Thread(ts); _thread.IsBackground = true; _thread.Start(); } private int _testId; private ProgressStatus _progressStatus; private void runThread() { for (int i = 1; i < 10; i++) { _progressStatus(string.Format("Test#{0}, cycle#{1}", _testId, i)); System.Threading.Thread.Sleep(500); } } } #endregion // Test class public Form1() { InitializeComponent(); } private Test _test1; private void cmdTest1_Click(object sender, EventArgs e) { _test1 = new Test(); _test1.Run(progressStatus); } private Test _test2; private void cmdTest2_Click(object sender, EventArgs e) { _test2 = new Test(); _test2.Run(progressStatus); } private void progressStatus(string message) { if (this.InvokeRequired) { object[] args = { message }; this.Invoke( new Test.ProgressStatus( progressStatusInThisFormThread), args); } else progressStatusInThisFormThread(message); } private static object _statusMessageOutputLocker = new object(); private void progressStatusInThisFormThread(string message) { lock (_statusMessageOutputLocker) { txtLog.Text = String.Format("{0:hh:mm:ss.fff}: {1}", DateTime.Now, message) + System.Environment.NewLine + txtLog.Text; } } } } I will try to post about another one I use in the coming days... Thank you. -- Shamil -----Original Message----- So how do you folks display status information in a form? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sun Nov 29 15:51:48 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 30 Nov 2009 00:51:48 +0300 Subject: [dba-VB] C# - Presenting process status In-Reply-To: <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> References: <4B12C2A2.1010001@colbyconsulting.com> <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> Message-ID: <00d301ca713e$28009bc0$7801d340$@spb.ru> OK, here is another sample more close to VB6/VBA WithEvents approach: using System; using System.Windows.Forms; namespace WinFormsSample { public partial class Form2 : Form { // Create Windows Form Form2 with three controls: // // 1. Button: cmdTest1 -> cmdTest1_Click // 2. Button: cmdTest2 -> cmdTest2_Click // 3. TextBox: txtLog (Multiline = true) #region Test class public class Test { public class ProgressStatusEventArgs : EventArgs { public string Message { get; set; } } public event EventHandler ProgressStatus; private static int _id; private System.Threading.Thread _thread; public void Run() { _testId = ++_id; System.Threading.ThreadStart ts = new System.Threading.ThreadStart(runThread); _thread = new System.Threading.Thread(ts); _thread.IsBackground = true; _thread.Start(); } private int _testId; private void runThread() { for (int i = 1; i < 10; i++) { string message = string.Format("Test#{0}, cycle#{1}", _testId, i); ProgressStatusEventArgs e = new ProgressStatusEventArgs(); e.Message = message; // raise event if (ProgressStatus != null) ProgressStatus(this, e); System.Threading.Thread.Sleep(500); } } } #endregion // Test class public Form2() { InitializeComponent(); } private Test _test1; private void cmdTest1_Click(object sender, EventArgs e) { _test1 = new Test(); _test1.ProgressStatus += new EventHandler(progressStatus); _test1.Run(); } private Test _test2; private void cmdTest2_Click(object sender, EventArgs e) { _test2 = new Test(); _test2.ProgressStatus += new EventHandler(progressStatus); _test2.Run(); } public delegate void ProgressStatusDelegate(string message); private void progressStatus(object sender, Test.ProgressStatusEventArgs e) { if (this.InvokeRequired) { object[] args = { e.Message }; this.Invoke( new ProgressStatusDelegate( progressStatusInThisFormThread), args); } else progressStatusInThisFormThread(e.Message); } private static object _statusMessageOutputLocker = new object(); private void progressStatusInThisFormThread(string message) { lock (_statusMessageOutputLocker) { txtLog.Text = String.Format("{0:hh:mm:ss.fff}: {1}", DateTime.Now, message) + System.Environment.NewLine + txtLog.Text; } } } } -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: Monday, November 30, 2009 12:33 AM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] C# - Presenting process status <<< I will try to post about another one I use in the coming days... >>> From jwcolby at colbyconsulting.com Sun Nov 29 22:09:43 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 29 Nov 2009 23:09:43 -0500 Subject: [dba-VB] C# - Presenting process status In-Reply-To: <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> References: <4B12C2A2.1010001@colbyconsulting.com> <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> Message-ID: <4B134587.9010005@colbyconsulting.com> Shamil, Thanks for this. I will need to study it. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > Here is one of the methods I use (there are zillion combinations) - I dunno > where it comes from, did I borrowed it from somewhere or I got it elaborated > myself, is it recommended way or not - but it just works, and I have > "cooked" it for you here from scratch - this is codebehind for Windows Form > named Form1: > > using System; > using System.Windows.Forms; > > namespace WinFormsSample > { > public partial class Form1 : Form > { > // Create Windows Form Form1 with three controls: > // > // 1. Button: cmdTest1 -> cmdTest1_Click > // 2. Button: cmdTest2 -> cmdTest2_Click > // 3. TextBox: txtLog (Multiline = true) > > #region Test class > public class Test > { > public delegate void ProgressStatus(string message); > > private static int _id; > private System.Threading.Thread _thread; > public void Run(ProgressStatus progressStatus) > { > _testId = ++_id; > _progressStatus = progressStatus; > > System.Threading.ThreadStart ts = > new System.Threading.ThreadStart(runThread); > _thread = new System.Threading.Thread(ts); > _thread.IsBackground = true; > _thread.Start(); > } > > private int _testId; > private ProgressStatus _progressStatus; > private void runThread() > { > for (int i = 1; i < 10; i++) > { > _progressStatus(string.Format("Test#{0}, cycle#{1}", > _testId, i)); > System.Threading.Thread.Sleep(500); > } > } > } > #endregion // Test class > > public Form1() > { > InitializeComponent(); > } > > private Test _test1; > private void cmdTest1_Click(object sender, EventArgs e) > { > _test1 = new Test(); > _test1.Run(progressStatus); > } > > private Test _test2; > private void cmdTest2_Click(object sender, EventArgs e) > { > _test2 = new Test(); > _test2.Run(progressStatus); > } > > private void progressStatus(string message) > { > if (this.InvokeRequired) > { > object[] args = { message }; > this.Invoke( > new Test.ProgressStatus( > progressStatusInThisFormThread), > args); > } > else > progressStatusInThisFormThread(message); > > } > > private static object _statusMessageOutputLocker = new object(); > private void progressStatusInThisFormThread(string message) > { > lock (_statusMessageOutputLocker) > { > txtLog.Text = > String.Format("{0:hh:mm:ss.fff}: {1}", > DateTime.Now, > message) + > System.Environment.NewLine + > txtLog.Text; > } > } > } > } > > I will try to post about another one I use in the coming days... > > Thank you. > > -- > Shamil > > -----Original Message----- > So how do you folks display status information in a form? From jwcolby at colbyconsulting.com Mon Nov 30 11:44:39 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 12:44:39 -0500 Subject: [dba-VB] C# - return value for stored procedures Message-ID: <4B140487.1030905@colbyconsulting.com> While I am getting back values in my "output" parameters for the SPs, I am getting a constant -1 for the return value. In the stored procedure I have tried just using RETURN 1234 and also creating an int variable, setting that to 1234 and returning that. declare @Ret int select @ret = 1234 RETURN @ret In all cases I get a -1 value which leads me to believe that the problem is on the C# side of things. My c# code looks like: try { if (sCmd.Connection.State != ConnectionState.Open) sCmd.Connection.Open(); int retValue = sCmd.ExecuteNonQuery(); return retValue; } catch (SqlException sqlEx) { throw sqlEx; } I am not throwing an error, the SP executes, but retValue is always -1. Any ideas? Do I need to add a SqlParameter to the Parameters collection for the return value? -- John W. Colby www.ColbyConsulting.com From james at fcidms.com Mon Nov 30 11:56:47 2009 From: james at fcidms.com (James Barash) Date: Mon, 30 Nov 2009 12:56:47 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <4B140487.1030905@colbyconsulting.com> Message-ID: <12D2F894654B53498DEF840AE12B7BB36B4B5797@fciexchange.fcidms.com> John: You do, indeed, need to define a Parameter for the return value. From the Microsoft documentation: -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, November 30, 2009 12:45 PM To: VBA Subject: [dba-VB] C# - return value for stored procedures While I am getting back values in my "output" parameters for the SPs, I am getting a constant -1 for the return value. In the stored procedure I have tried just using RETURN 1234 and also creating an int variable, setting that to 1234 and returning that. declare @Ret int select @ret = 1234 RETURN @ret In all cases I get a -1 value which leads me to believe that the problem is on the C# side of things. My c# code looks like: try { if (sCmd.Connection.State != ConnectionState.Open) sCmd.Connection.Open(); int retValue = sCmd.ExecuteNonQuery(); return retValue; } catch (SqlException sqlEx) { throw sqlEx; } I am not throwing an error, the SP executes, but retValue is always -1. Any ideas? Do I need to add a SqlParameter to the Parameters collection for the return value? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From james at fcidms.com Mon Nov 30 12:06:48 2009 From: james at fcidms.com (James Barash) Date: Mon, 30 Nov 2009 13:06:48 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <4B140487.1030905@colbyconsulting.com> Message-ID: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com> John: You do, indeed, need to define a Parameter for the return value. From the Microsoft documentation: SqlCommand.ExecuteNonQuery Method Executes a Transact-SQL statement against the connection and returns the number of rows affected. To add a Return Value parameter: SqlParameter ret = new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int); ret.Direction = ParameterDirection.ReturnValue; cmd.Parameters.Add(ret); Hope this helps. James Barash -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, November 30, 2009 12:45 PM To: VBA Subject: [dba-VB] C# - return value for stored procedures While I am getting back values in my "output" parameters for the SPs, I am getting a constant -1 for the return value. In the stored procedure I have tried just using RETURN 1234 and also creating an int variable, setting that to 1234 and returning that. declare @Ret int select @ret = 1234 RETURN @ret In all cases I get a -1 value which leads me to believe that the problem is on the C# side of things. My c# code looks like: try { if (sCmd.Connection.State != ConnectionState.Open) sCmd.Connection.Open(); int retValue = sCmd.ExecuteNonQuery(); return retValue; } catch (SqlException sqlEx) { throw sqlEx; } I am not throwing an error, the SP executes, but retValue is always -1. Any ideas? Do I need to add a SqlParameter to the Parameters collection for the return value? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Mon Nov 30 12:08:16 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 13:08:16 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <12D2F894654B53498DEF840AE12B7BB36B4B5797@fciexchange.fcidms.com> References: <12D2F894654B53498DEF840AE12B7BB36B4B5797@fciexchange.fcidms.com> Message-ID: <4B140A10.4020409@colbyconsulting.com> Thanks James. Your response did not provide a link or anything but knowing that I have to do this, I will go a Googling. Thanks, John W. Colby www.ColbyConsulting.com James Barash wrote: > John: > > You do, indeed, need to define a Parameter for the return value. From the Microsoft documentation: > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Monday, November 30, 2009 12:45 PM > To: VBA > Subject: [dba-VB] C# - return value for stored procedures > > While I am getting back values in my "output" parameters for the SPs, I am getting a constant -1 for the return value. > > In the stored procedure I have tried just using > > RETURN 1234 > > and also creating an int variable, setting that to 1234 and returning that. > > declare @Ret int > select @ret = 1234 > RETURN @ret > > In all cases I get a -1 value which leads me to believe that the problem is on the C# side of things. > > My c# code looks like: > > try > { > if (sCmd.Connection.State != ConnectionState.Open) > sCmd.Connection.Open(); > int retValue = sCmd.ExecuteNonQuery(); > return retValue; > } > catch (SqlException sqlEx) > { > throw sqlEx; > } > > I am not throwing an error, the SP executes, but retValue is always -1. > > Any ideas? Do I need to add a SqlParameter to the Parameters collection for the return value? > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Mon Nov 30 12:25:27 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 13:25:27 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com> References: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com> Message-ID: <4B140E17.3050400@colbyconsulting.com> James, So what does this do for me? int retValue = sCmd.ExecuteNonQuery(); From my Googling it seems like I have to do something like: retValue = Int32.Parse(sCmd.Parameters["@Return_Value"].Value.ToString()); which is working just fine. scmd.ExecuteNonQuery() returns a -1 (not sure why) and then the @Return_Value is parsed and returned. So that part is now working as well. Thanks again for the response James. John W. Colby www.ColbyConsulting.com James Barash wrote: > John: > > You do, indeed, need to define a Parameter for the return value. From the Microsoft documentation: > > SqlCommand.ExecuteNonQuery Method > > Executes a Transact-SQL statement against the connection and returns the number of rows affected. > > To add a Return Value parameter: > > SqlParameter ret = new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int); > ret.Direction = ParameterDirection.ReturnValue; > cmd.Parameters.Add(ret); > > Hope this helps. From jwcolby at colbyconsulting.com Mon Nov 30 13:55:11 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 14:55:11 -0500 Subject: [dba-VB] C# SQLCommand.Timeout Message-ID: <4B14231F.3020905@colbyconsulting.com> Any words of wisdom on setting the command.Timeout property? I have stored procedures that execute instantly, and others that can take an hour or more (large appends). How can I discover how long a stored procedure takes to run so that I can set this property dynamically? As an example I have cases where I am appending 65 million rows into a table of 6 fields. This can take a long time (at the very least perhaps 20 minutes) but if I could get a "RecordsAffected" count for such queries as well as the time it took to execute, then I could start to discover that it takes "X seconds / million" records, and set the timeout to a reasonable value based on the records to be appended. This doesn't have to be any exact time value. I assume a simple pair of time variables, then "stop time - start time"? Or should I just discover worst case and set it to that? -- John W. Colby www.ColbyConsulting.com From stuart at lexacorp.com.pg Mon Nov 30 15:07:43 2009 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Tue, 01 Dec 2009 07:07:43 +1000 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <4B140E17.3050400@colbyconsulting.com> References: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com>, <4B140E17.3050400@colbyconsulting.com> Message-ID: <4B14341F.21088.149E29F8@stuart.lexacorp.com.pg> -1 = True, meanimg that the sp executed with no errors? -- Stuart On 30 Nov 2009 at 13:25, jwcolby wrote: > which is working just fine. scmd.ExecuteNonQuery() returns a -1 (not sure why) and then the > @Return_Value is parsed and returned. > From jwcolby at colbyconsulting.com Mon Nov 30 15:35:18 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 16:35:18 -0500 Subject: [dba-VB] C# - Stored procedures working! Message-ID: <4B143A96.8080905@colbyconsulting.com> I would like to thank all of you for your assistance in getting my stored procedures working from C#. My strategy was to design a StoredProcedure class which contained all of the data and code required to run any of the stored procedures that I have written so far. We shall see whether I succeeded in that however I have a set of seven stored procedures that are required to export large tables in chunks to CSV files and I am successfully running all of those seven stored procedures and obtaining CSV files. This StoredProcedure class has a command object and a set of about eight "Parameter" methods which wrap the code for creating standard parameters used over and over in my various stored procedures, parameters like DBName, TblName, PKStart, PKend etc, as well as the return value and errorno and errormsg parameters. I then created an AccuzipExport class that "supervises" the Accuzip export process, i.e. holds all of the code to DEFINE these seven stored procedures, i.e. set up the required stored procedure class mentioned above for each of these seven SPs, and the specific parameters that each SP uses. This class then has a method that calls three of the stored procedures to create a standard named table, fill it with sorted data and index it in preparation for creating chunk files. After that the method sits in a while loop calling the remaining four stored procedures which create the chunk table, append a chunk of records to that, index it and then BCP it out. All of this stuff now works. This stuff was what the old Access database did, so I have essentially ported the Accuzip Export process to C#. However I have successfully returned a success/fail return value as well as an error code / message in the C# version, whereas I never had that ability in the Access version so I am now in some respects ahead of where I was in Access. There is more to do of course. Now that I have a robust dev platform at my fingertips I can log the process from beginning to end, errors, times for each SP to execute, stuff like that. And of course get the process status displaying on the form I use to start this process. Eventually I need to get this export process running in a thread so that I can use the big picture application for other things while this export process runs. But next comes the matching AccuzipInput class, which will pull the CSV files coming back from Accuzip back into SQL Server. With all of the grunt work done for executing Stored Procedures it should be relatively quick and easy to get the matching input class up. At any rate I wanted to say thanks to everyone for all the help and express how jazzed I am to be where I am today. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Mon Nov 30 15:37:43 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 16:37:43 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <4B14341F.21088.149E29F8@stuart.lexacorp.com.pg> References: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com>, <4B140E17.3050400@colbyconsulting.com> <4B14341F.21088.149E29F8@stuart.lexacorp.com.pg> Message-ID: <4B143B27.9050902@colbyconsulting.com> Stuart, I don't think so as I have had failures and I *think* I got a -1 regardless. I will have to pursue this one. John W. Colby www.ColbyConsulting.com Stuart McLachlan wrote: > -1 = True, meanimg that the sp executed with no errors? > From jwcolby at colbyconsulting.com Mon Nov 2 07:58:56 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 02 Nov 2009 08:58:56 -0500 Subject: [dba-VB] SQL Server 2005: Running stored procedures from C# / VBA Message-ID: <4AEEE5A0.4040901@colbyconsulting.com> I am embarking on the task of running the "DB from hell" from C#. I have several dozen stored procedures that perform various tasks and I currently run many of them from Access using a single procedure provided by Charlotte. I need a similar function written in C# (preferably) or VB.Net. ATM I have an Access database which has two specific forms, bound to two tables. These two forms each execute a set of SPs which build temp tables of data to be exported to an external process, build indexes to make the processes faster, BCP the data out to CSV files, and then the reverse on the way back in. I have another Access database that creates an order for the same client. Same kind of thing, temp tables created, indexes, fields added and so forth. My intention is to port each of these processes to C#. The reason is simply that C# has a couple of huge advantages over VBA, such as child threads and built-in capabilities in the framework that I have to really scrounge to make work in VBA. The first thing I need though is code that can execute a stored procedure, pass parameters to it, retrieve parameters back etc. With demo code preferably. Also I would like to open a conversation about how using .Net from inside of SQL Server (2005) works. Does anyone on the list use C# or VB.Net from inside of SQL Server? Where is the code stored? How to do you call it? What do you use it for? TIA, -- John W. Colby www.ColbyConsulting.com From marklbreen at gmail.com Mon Nov 2 10:18:13 2009 From: marklbreen at gmail.com (Mark Breen) Date: Tue, 3 Nov 2009 00:18:13 +0800 Subject: [dba-VB] SQL Server 2005: Running stored procedures from C# / VBA In-Reply-To: <4AEEE5A0.4040901@colbyconsulting.com> References: <4AEEE5A0.4040901@colbyconsulting.com> Message-ID: Hello John, First of all, good luck. Secondly, you mentioned a few days ago that you referred to yourself as an SQL Wannabe, well given the work you are currently doing, I think you can no longer refer to yourself as a wannabe, you are using SQL, and no doubt will be pushing it to it's limits. *SPROCs* There are loads of samples of code and I have nothing to hand here that will assist much, but I think you will be surprised how easy it is. You will get better exmaples, but as an overview you just need to create a connection - usually referred to as cnn. create a cmd to 'hold' the sproc you wish to call join the cnn to the sproc specify the type of the cmd (no results returned, or records returned or a few other types) then you add objects which are parameters, you need one each for each param in the sproc, you can even add a param that will be referred to as OUTPUT instead of INPUT - these output params can be used to pass back single values in additional or instead of, the actual result set. you can also receive back the result code of the sproc. then you just execute the cmd object. Once you get a few lines of code that handle the above, you can copy and paste for the next five years. There are a few other things that I am not as accustomed to, but when you know that you need to read in a result set, you can use a data reader. Someone else can advise on that. when you create the cnn object, usually people pull the actual string from a app.config file, so watch out for that snipit also. BTW, FWIW, other than for binding to grids or combo's, I never used a dataset in code. *BCP* You mentioned BCP, well I have no used that since SQL 6.5, but what I have used in the SSIS in SQL 2005. It is worth taking a look, you may find that you can do a lot of your work in SSIS. It can branch and call sprocs and import and export all as a package. It can be incredibly fast and would be worth take an hour or two and read up on what you can do with SSIS. *Embeding C#* Lastly, other than an article I read in MSDN magazine, where a guy wanted to do some hashing or encryption in side SQL Server I never heard of anyone using / executing code in side SQL Serer. For performance that you need, I would, with my limited skills, keep as much of the work in TSQL - IE Sprocs, and by doing so, you will keep the mass movement of data within the db. I would love to see if anyone here actually ever used the feature of including C# Code within the db. I know that this is not samples, but you can do this, and once you get a set of lines of code that 1) create the cnn 2) create the cmd 3) add the params 4) run the sproc you will not look back. Just keep as much work as possible inside sql server. *Scheduled Tasks* Finally, once last thing to mention, if you have a smtp server, it is so, so each to create scheduled tasks, that run at predefined times and then mail you when they are finished. Running the heavy jobs nightly, can take a lot of the hard work from your daily routines. Again you can manage all this with very little actually code and have working routines in minutes. HTH. Mark 2009/11/2 jwcolby > I am embarking on the task of running the "DB from hell" from C#. I have > several dozen stored > procedures that perform various tasks and I currently run many of them from > Access using a single > procedure provided by Charlotte. I need a similar function written in C# > (preferably) or VB.Net. > > ATM I have an Access database which has two specific forms, bound to two > tables. These two forms > each execute a set of SPs which build temp tables of data to be exported to > an external process, > build indexes to make the processes faster, BCP the data out to CSV files, > and then the reverse on > the way back in. > > I have another Access database that creates an order for the same client. > Same kind of thing, temp > tables created, indexes, fields added and so forth. > > My intention is to port each of these processes to C#. The reason is > simply that C# has a couple of > huge advantages over VBA, such as child threads and built-in capabilities > in the framework that I > have to really scrounge to make work in VBA. > > The first thing I need though is code that can execute a stored procedure, > pass parameters to it, > retrieve parameters back etc. With demo code preferably. > > Also I would like to open a conversation about how using .Net from inside > of SQL Server (2005) > works. Does anyone on the list use C# or VB.Net from inside of SQL Server? > Where is the code > stored? How to do you call it? What do you use it for? > > TIA, > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Mon Nov 2 11:51:47 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 02 Nov 2009 12:51:47 -0500 Subject: [dba-VB] SQL Server 2005: Running stored procedures from C# / VBA In-Reply-To: References: <4AEEE5A0.4040901@colbyconsulting.com> Message-ID: <4AEF1C33.1020904@colbyconsulting.com> Mark, > First of all, good luck. Uh-Ohhh... is that a warning? ;) I refer to myself as a wannabe simply because to this point SQL Server has been a small part of my day. As a sole proprietor my day contains so many different kinds of work that I find it hard to be an expert in something as complex as SQL Server. There are so many people on these lists that spend all day every day in SQL Server that I will truly never be anything other than a wannabe - from their perspective. I have always been more programmer than anything. I come at the world from the perspective of "how can I write code to make my job easier". Luckily I have enough programming background that I can generally accomplish that. Right now I am trying to switch my environment of choice from Access to C#, and BOY IS THAT TOUGH. Programming is always about spending enough time doing it that the syntax becomes second nature. C# is nowhere close to second nature yet. I can taste the power though, so tantalizing. ;) I am using the Idera free backup stuff. Very powerful, recommended. It took me a couple of hours to install on my two servers and write some stored procedures to make them work. Now I really want to be able to select one specific database to backup or restore. To do that I have to be able to: See/select a database on a server. Run a stored procedure that does the backup. How simple is that eh? I just spent the morning learning about the DMO object, the listview control and writing the code to read out the databases and display them in the listview. I still have to find and get working the code that executes the stored procedure. The problem is not that this is hard, it is that I am not fluent in C# yet. But I am getting there, sloooowly but surely. John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > First of all, good luck. > > Secondly, you mentioned a few days ago that you referred to yourself as an > SQL Wannabe, well given the work you are currently doing, I think you can no > longer refer to yourself as a wannabe, you are using SQL, and no doubt will > be pushing it to it's limits. > > *SPROCs* > > There are loads of samples of code and I have nothing to hand here that will > assist much, but I think you will be surprised how easy it is. > > You will get better exmaples, but as an overview you just need to > > create a connection - usually referred to as cnn. > create a cmd to 'hold' the sproc you wish to call > join the cnn to the sproc > specify the type of the cmd (no results returned, or records returned or a > few other types) > then you add objects which are parameters, you need one each for each param > in the sproc, > you can even add a param that will be referred to as OUTPUT instead of INPUT > - these output params can be used to pass back single values in additional > or instead of, the actual result set. > you can also receive back the result code of the sproc. > then you just execute the cmd object. > > Once you get a few lines of code that handle the above, you can copy and > paste for the next five years. > > There are a few other things that I am not as accustomed to, but when you > know that you need to read in a result set, you can use a data reader. > Someone else can advise on that. > > when you create the cnn object, usually people pull the actual string from a > app.config file, so watch out for that snipit also. > > BTW, FWIW, other than for binding to grids or combo's, I never used a > dataset in code. > > > *BCP* > You mentioned BCP, well I have no used that since SQL 6.5, but what I have > used in the SSIS in SQL 2005. It is worth taking a look, you may find that > you can do a lot of your work in SSIS. It can branch and call sprocs and > import and export all as a package. It can be incredibly fast and would be > worth take an hour or two and read up on what you can do with SSIS. > > *Embeding C#* > Lastly, other than an article I read in MSDN magazine, where a guy wanted to > do some hashing or encryption in side SQL Server I never heard of anyone > using / executing code in side SQL Serer. For performance that you need, I > would, with my limited skills, keep as much of the work in TSQL - IE Sprocs, > and by doing so, you will keep the mass movement of data within the db. I > would love to see if anyone here actually ever used the feature of including > C# Code within the db. > > > I know that this is not samples, but you can do this, and once you get a set > of lines of code that > 1) create the cnn > 2) create the cmd > 3) add the params > 4) run the sproc > > you will not look back. Just keep as much work as possible inside sql > server. > > *Scheduled Tasks* > Finally, once last thing to mention, if you have a smtp server, it is so, so > each to create scheduled tasks, that run at predefined times and then mail > you when they are finished. Running the heavy jobs nightly, can take a lot > of the hard work from your daily routines. Again you can manage all this > with very little actually code and have working routines in minutes. > > HTH. > > Mark > > > > > 2009/11/2 jwcolby > >> I am embarking on the task of running the "DB from hell" from C#. I have >> several dozen stored >> procedures that perform various tasks and I currently run many of them from >> Access using a single >> procedure provided by Charlotte. I need a similar function written in C# >> (preferably) or VB.Net. >> >> ATM I have an Access database which has two specific forms, bound to two >> tables. These two forms >> each execute a set of SPs which build temp tables of data to be exported to >> an external process, >> build indexes to make the processes faster, BCP the data out to CSV files, >> and then the reverse on >> the way back in. >> >> I have another Access database that creates an order for the same client. >> Same kind of thing, temp >> tables created, indexes, fields added and so forth. >> >> My intention is to port each of these processes to C#. The reason is >> simply that C# has a couple of >> huge advantages over VBA, such as child threads and built-in capabilities >> in the framework that I >> have to really scrounge to make work in VBA. >> >> The first thing I need though is code that can execute a stored procedure, >> pass parameters to it, >> retrieve parameters back etc. With demo code preferably. >> >> Also I would like to open a conversation about how using .Net from inside >> of SQL Server (2005) >> works. Does anyone on the list use C# or VB.Net from inside of SQL Server? >> Where is the code >> stored? How to do you call it? What do you use it for? >> >> TIA, >> >> -- >> John W. Colby >> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Mon Nov 2 12:41:24 2009 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Tue, 03 Nov 2009 04:41:24 +1000 Subject: [dba-VB] SQL Server 2005: Running stored procedures from C# / VBA In-Reply-To: References: <4AEEE5A0.4040901@colbyconsulting.com>, Message-ID: <4AEF27D4.29660.1C94540E@stuart.lexacorp.com.pg> If you don't havr an SMTP Server, grab the tiny one that Max put us on to a while ago: miniRelay. You can download it from here (site in Spanish): http://www.netvicious.com/miniRelay/miniRelay.zip English documentation and FAQ here (but download link is out of date) http://www.blat.net/miniRelay/ -- Stuart On 3 Nov 2009 at 0:18, Mark Breen wrote: > > *Scheduled Tasks* > Finally, once last thing to mention, if you have a smtp server, it is so, so > each to create scheduled tasks, that run at predefined times and then mail > you when they are finished. Running the heavy jobs nightly, can take a lot > of the hard work from your daily routines. Again you can manage all this > with very little actually code and have working routines in minutes. > From jwcolby at colbyconsulting.com Tue Nov 3 21:12:23 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Tue, 03 Nov 2009 22:12:23 -0500 Subject: [dba-VB] How do you run SQL Server processes Message-ID: <4AF0F117.20209@colbyconsulting.com> I have processes that I run which use sets of stored procedures. For example two of the processes export / import records from large tables for address validation. The addresses need to be sorted in Zip5/Zip4/Address order in order to make the external validation process as fast as possible so I create a table that pulls millions of records, often tens of millions of records, into a table (created on-the-fly) in sorted order, then pulls out 2 million records at a time and BCPs them out for processing in an external program. The inverse occurs when the records finish processing, 2 million record chunks are imported back into a temp table, then each 2 million record chunk is appended to a main table. Once back in SQL Server I have processes that build clustered and non-clustered indexes, run sha-1 hash functions to generate hash keys, update address valid codes etc. I will run this entire project at least once a month (to keep the addresses valid as people move), and I do so for many tables - a table of about 65 million addresses, another of 21 million, another of 11 million etc. I store all of my stored procedures in a dedicated database that pretty much just contains the SPs and a few UDFs as well as a couple of logging type tables. At the moment I use an Access database to run the individual stored procedures in specific order. I am just embarking on a project to move that functioning but not very efficient Access database to C#. I want to be able to have better monitoring of SP completion, records processed (counts), time to complete each SP etc. I found a pretty nice set of C# classes that allow creation of "stored procedure" objects, with parameter objects, then stored into a collection as the SPs (class instances) are assembled and finally the whole collection of SPs executed. I created a test database of a million names / addresses and today I started testing my existing SPs on this small data set. In the next few days I expect to build out a preliminary "port" of the existing Access VBA code. I am hoping to eventually make the entire process "table driven" where I can store (in a table) the name of the sp, the parameters in a child table etc. and then have a supervisor pull out the records and execute the SPs based on what is in the table. This would allow me to create the SPs and then use a C# application (data entry forms) to organize the SPs, add / delete SPs from the process and so forth. That is down the road of course, for now I will have hard coded sequences - as I already do in supervisor SPs as well as in hard coded Access VBA modules. As a programmer, doing this supervisor level in C# feels more natural to me (than doing it inside of SQL Server), and gives me the flexibility to do what I want in code external to the SQL Server database being manipulated. I want to be able to do things external to the database processes, things like monitor the directories that the BCP processes export to / from, move files popping out of a bcp process off to a VM on a different server, monitor those VMs for files coming out of their processes, moving the files back over to the SQL Server directories, and trigger BCP processes that import the data back in to SQL server. I need to be able to FTP order and data files to clients, and get (FTP) files from process bureaus. So it FEELS to me like using a language such as C# to build a system outside of SQL Server is the way to go. BTW I stumbled across something called SMO (SQL Server Management Objects) which makes it really easy to get at the SQL Server object model from C#. I can apparently see the entire tree structure that is a server/database/tables/etc. and do I know not what (yet) with those objects. Looks pretty cool though. The database is a large part of what I do but it is not the whole picture and I have been handcuffed by my lack of ability to effectively do these external parts. But I am a programmer at heart. I want to know how you do this kind of stuff. I know that running such "systems" of stored procedures tied to external processes must be quite common and I am wondering what you guys do for supervising your processes. If anyone doing anything remotely similar would take the time to explain how you do it, I would be most appreciative. Thanks, -- John W. Colby www.ColbyConsulting.com From raibeart at gmail.com Wed Nov 4 12:37:58 2009 From: raibeart at gmail.com (Robert Stewart) Date: Wed, 04 Nov 2009 12:37:58 -0600 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: References: Message-ID: <4af1ca07.0637560a.20ef.1362@mx.google.com> Sorry, but using C# to do what you would and could do simply using SQL Server is adding a middle layer you do not need. You can do exactly the same thing using SQL Server and stored procedures and writing to a log table about what the process is doing and keeping track of it that way. And, not have to use a middle layer language like C# to do it. I am not sure why writing T-SQL code is not like real programming to you. But, it is. And, if you are going to continue using SQL Server, you need to learn what it can do for you instead of always fighting learning it. This is coming from someone that does T-SQL, C#, VB.Net and Access VBA. So, I think I have a fair perspective of how they all work. At 12:00 PM 11/4/2009, you wrote: >Date: Tue, 03 Nov 2009 22:12:23 -0500 >From: jwcolby >Subject: [dba-VB] How do you run SQL Server processes >To: VBA , Dba-Sqlserver > >Message-ID: <4AF0F117.20209 at colbyconsulting.com> >Content-Type: text/plain; charset=ISO-8859-1; format=flowed > >I have processes that I run which use sets of stored procedures. > >For example two of the processes export / import records from large >tables for address validation. >The addresses need to be sorted in Zip5/Zip4/Address order in order >to make the external validation >process as fast as possible so I create a table that pulls millions >of records, often tens of >millions of records, into a table (created on-the-fly) in sorted >order, then pulls out 2 million >records at a time and BCPs them out for processing in an external program. > >The inverse occurs when the records finish processing, 2 million >record chunks are imported back >into a temp table, then each 2 million record chunk is appended to a >main table. Once back in SQL >Server I have processes that build clustered and non-clustered >indexes, run sha-1 hash functions to >generate hash keys, update address valid codes etc. I will run this >entire project at least once a >month (to keep the addresses valid as people move), and I do so for >many tables - a table of about >65 million addresses, another of 21 million, another of 11 million etc. > >I store all of my stored procedures in a dedicated database that >pretty much just contains the SPs >and a few UDFs as well as a couple of logging type tables. At the >moment I use an Access database >to run the individual stored procedures in specific order. > >I am just embarking on a project to move that functioning but not >very efficient Access database to >C#. I want to be able to have better monitoring of SP completion, >records processed (counts), time >to complete each SP etc. I found a pretty nice set of C# classes >that allow creation of "stored >procedure" objects, with parameter objects, then stored into a >collection as the SPs (class >instances) are assembled and finally the whole collection of SPs >executed. I created a test >database of a million names / addresses and today I started testing >my existing SPs on this small >data set. In the next few days I expect to build out a preliminary >"port" of the existing Access >VBA code. > >I am hoping to eventually make the entire process "table driven" >where I can store (in a table) the >name of the sp, the parameters in a child table etc. and then have a >supervisor pull out the records >and execute the SPs based on what is in the table. This would allow >me to create the SPs and then >use a C# application (data entry forms) to organize the SPs, add / >delete SPs from the process and >so forth. > >That is down the road of course, for now I will have hard coded >sequences - as I already do in >supervisor SPs as well as in hard coded Access VBA modules. > >As a programmer, doing this supervisor level in C# feels more >natural to me (than doing it inside of >SQL Server), and gives me the flexibility to do what I want in code >external to the SQL Server >database being manipulated. I want to be able to do things external >to the database processes, >things like monitor the directories that the BCP processes export to >/ from, move files popping out >of a bcp process off to a VM on a different server, monitor those >VMs for files coming out of their >processes, moving the files back over to the SQL Server directories, >and trigger BCP processes that >import the data back in to SQL server. I need to be able to FTP >order and data files to clients, >and get (FTP) files from process bureaus. So it FEELS to me like >using a language such as C# to >build a system outside of SQL Server is the way to go. > >BTW I stumbled across something called SMO (SQL Server Management >Objects) which makes it really >easy to get at the SQL Server object model from C#. I can >apparently see the entire tree structure >that is a server/database/tables/etc. and do I know not what (yet) >with those objects. Looks pretty >cool though. > >The database is a large part of what I do but it is not the whole >picture and I have been handcuffed >by my lack of ability to effectively do these external parts. But I >am a programmer at heart. I >want to know how you do this kind of stuff. > >I know that running such "systems" of stored procedures tied to >external processes must be quite >common and I am wondering what you guys do for supervising your >processes. If anyone doing anything >remotely similar would take the time to explain how you do it, I >would be most appreciative. > >Thanks, >-- >John W. Colby >www.ColbyConsulting.com > > >------------------------------ > >_______________________________________________ >dba-VB mailing list >dba-VB at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-vb > > >End of dba-VB Digest, Vol 73, Issue 3 >************************************* From jwcolby at colbyconsulting.com Wed Nov 4 15:11:58 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 04 Nov 2009 16:11:58 -0500 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <4af1ca07.0637560a.20ef.1362@mx.google.com> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> Message-ID: <4AF1EE1E.9030406@colbyconsulting.com> Robert, I don't really want to get in an argument about this. I asked a serious question and got a somewhat brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY answer I have received so that is not particularly encouraging. 8( TSQL, at least as written from inside of SSMS is about like writing GBasic code back in 1982. Well, maybe a tiny step up but not much. You are right, I probably just don't know how, but in what I will call "a real language" I can set breakpoints, I can see a call stack, I have watch statements, I have... simple very basic DEBUG capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in SSMS). I can have forms with controls displaying data, buttons that cause things to happen. I can FTP files, monitor directories for activities. Believe me, if this exists in SSMS I will jump on that! Just don't tell me to go spend thousands to buy a tool that will do some of what C# can already do for me. Modern languages, and the .Net environment in particular have capabilities that (AFAICT) TSQL can not even dream of. TSQL dreams of manipulating data, C# dreams of manipulating ... the world. Robert I am not "fighting learning it", I am light years beyond where I was two years ago, and I learn more TSQL and SQL Server every day. However from what I have seen so far, TSQL will NEVER be what I need for the supervisor. And answers like you just provided aren't really much help. I am a consultant. I am a programmer, not a DBA, or a SQL Server administrator, or a SQL Server jock at a company somewhere. I need C# for automating my business, which is NOT being a DBA (I already said that I think). I will write C# applications for clients that use SQL Server, and others that do not use SQL Server. SQL Server is used by exactly ONE of my clients, so I cannot spend the time to do whatever it is that you envision SQL Server doing because I cannot amortize the learning over other clients. Now... if you want to answer my question, I am all ears. The first thing to answer (If I am to do all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. Let's take a concrete, real world example. I have a set of virtual machines which run a third party application. I place files into an "inbox" directory, and I wait for files to pop out of the running application into the "outbox" directory. Just an aside, the files placed into the inbox are created by a set of about 5 stored procedures, which can operate on any of dozens of databases in my server. It is the case that the third party application occasionally hangs. If it does I need to rename a file in a specific directory to cause that application to restart processing that file. It is my intention to use C# to execute the SPs to create the files, move the files into the VM, then monitor files dropped into the inbox, and files popping out into the outbox. Look for "no new files in X minutes" and if that condition exists, rename the file. Log files dropped into the in-box, files popping out into the out-box, and the time it took for the files to process through the application. Log any "hangs" restarted. Supervisor stuff. Explain to me how (and more importantly WHY) I would use TSQL to do that? Other than a "when TSQL is all you have" (which is NOT true for me) mentality. A SMALL part of my business is to perform some moderately complex processes using SQL Server and SPs as a tool. I envision C# as a supervisor, not a "middle layer". I envision an application (written in C#) that runs on my desktop which starts, then monitors SQL Server processes, displays errors, restarts processes if necessary, creates databases, tables etc, backs them up and detaches them when done, logging everything that happens, showing it all to me me in real time in a Windows application. Possibly even automatically generating invoices for the work done and emailing them to the client. Explain to me one more time (with feeling) how (and more importantly WHY) I would use TSQL to do all of that stuff? John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > Sorry, but using C# to do what you would and could do simply using > SQL Server is adding a middle layer you do not need. You can do > exactly the same thing using SQL Server and stored procedures and > writing to a log table about what the process is doing and keeping > track of it that way. And, not have to use a middle layer language > like C# to do it. > > I am not sure why writing T-SQL code is not like real programming to > you. But, it is. And, if you are going to continue using SQL > Server, you need to learn what it can do for you instead of always > fighting learning it. > > This is coming from someone that does T-SQL, C#, VB.Net and Access > VBA. So, I think I have a fair perspective of how they all work. From Gustav at cactus.dk Wed Nov 4 16:21:22 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 04 Nov 2009 23:21:22 +0100 Subject: [dba-VB] How do you run SQL Server processes Message-ID: Hi John I have no answer to you but if it can bring you some relief, I feel much the same. The database is for storing data, a business layer and/or a front end is for the logic and interface. That said, nothing is perfect, and in the real world you will have to write some T-SQL or similar but I try to keep it at minimum, first because of the trouble of maintenance (for example, no version control I know of), second because I feel old when I write "GBasic style" code. And I'm not in a big corporation where I can get every expensive tool I can point at, and tools for SQL Server seem all to be enterprise-priced. So, writing views and T-SQL is because I have to, writing C# (and VBA for that matter) is because I like to. I guess Robert is a very skilled DBA and can write T-SQL with his left foot and that's very good - then we know where to ask for advice as I'll never be more than a decent low-level DBA. /gustav >>> jwcolby at colbyconsulting.com 04-11-2009 22:11 >>> Robert, I don't really want to get in an argument about this. I asked a serious question and got a somewhat brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY answer I have received so that is not particularly encouraging. 8( TSQL, at least as written from inside of SSMS is about like writing GBasic code back in 1982. Well, maybe a tiny step up but not much. You are right, I probably just don't know how, but in what I will call "a real language" I can set breakpoints, I can see a call stack, I have watch statements, I have... simple very basic DEBUG capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in SSMS). I can have forms with controls displaying data, buttons that cause things to happen. I can FTP files, monitor directories for activities. Believe me, if this exists in SSMS I will jump on that! Just don't tell me to go spend thousands to buy a tool that will do some of what C# can already do for me. Modern languages, and the .Net environment in particular have capabilities that (AFAICT) TSQL can not even dream of. TSQL dreams of manipulating data, C# dreams of manipulating ... the world. Robert I am not "fighting learning it", I am light years beyond where I was two years ago, and I learn more TSQL and SQL Server every day. However from what I have seen so far, TSQL will NEVER be what I need for the supervisor. And answers like you just provided aren't really much help. I am a consultant. I am a programmer, not a DBA, or a SQL Server administrator, or a SQL Server jock at a company somewhere. I need C# for automating my business, which is NOT being a DBA (I already said that I think). I will write C# applications for clients that use SQL Server, and others that do not use SQL Server. SQL Server is used by exactly ONE of my clients, so I cannot spend the time to do whatever it is that you envision SQL Server doing because I cannot amortize the learning over other clients. Now... if you want to answer my question, I am all ears. The first thing to answer (If I am to do all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. Let's take a concrete, real world example. I have a set of virtual machines which run a third party application. I place files into an "inbox" directory, and I wait for files to pop out of the running application into the "outbox" directory. Just an aside, the files placed into the inbox are created by a set of about 5 stored procedures, which can operate on any of dozens of databases in my server. It is the case that the third party application occasionally hangs. If it does I need to rename a file in a specific directory to cause that application to restart processing that file. It is my intention to use C# to execute the SPs to create the files, move the files into the VM, then monitor files dropped into the inbox, and files popping out into the outbox. Look for "no new files in X minutes" and if that condition exists, rename the file. Log files dropped into the in-box, files popping out into the out-box, and the time it took for the files to process through the application. Log any "hangs" restarted. Supervisor stuff. Explain to me how (and more importantly WHY) I would use TSQL to do that? Other than a "when TSQL is all you have" (which is NOT true for me) mentality. A SMALL part of my business is to perform some moderately complex processes using SQL Server and SPs as a tool. I envision C# as a supervisor, not a "middle layer". I envision an application (written in C#) that runs on my desktop which starts, then monitors SQL Server processes, displays errors, restarts processes if necessary, creates databases, tables etc, backs them up and detaches them when done, logging everything that happens, showing it all to me me in real time in a Windows application. Possibly even automatically generating invoices for the work done and emailing them to the client. Explain to me one more time (with feeling) how (and more importantly WHY) I would use TSQL to do all of that stuff? John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > Sorry, but using C# to do what you would and could do simply using > SQL Server is adding a middle layer you do not need. You can do > exactly the same thing using SQL Server and stored procedures and > writing to a log table about what the process is doing and keeping > track of it that way. And, not have to use a middle layer language > like C# to do it. > > I am not sure why writing T-SQL code is not like real programming to > you. But, it is. And, if you are going to continue using SQL > Server, you need to learn what it can do for you instead of always > fighting learning it. > > This is coming from someone that does T-SQL, C#, VB.Net and Access > VBA. So, I think I have a fair perspective of how they all work. _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Wed Nov 4 16:48:32 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 01:48:32 +0300 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <4AF1EE1E.9030406@colbyconsulting.com> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> <4AF1EE1E.9030406@colbyconsulting.com> Message-ID: <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> Hi John, <<< The first thing to answer (If I am to do all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. >>> Overview of T-SQL and CLR debugging in SQL Server 2005 http://blogs.msdn.com/sqlclr/archive/2006/06/29/651644.aspx How to: Enable SQL Server 2005 Debugging http://msdn.microsoft.com/en-us/library/s0fk6z6e(VS.80).aspx HTH, --Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 12:12 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes Robert, I don't really want to get in an argument about this. I asked a serious question and got a somewhat brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY answer I have received so that is not particularly encouraging. 8( TSQL, at least as written from inside of SSMS is about like writing GBasic code back in 1982. Well, maybe a tiny step up but not much. You are right, I probably just don't know how, but in what I will call "a real language" I can set breakpoints, I can see a call stack, I have watch statements, I have... simple very basic DEBUG capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in SSMS). I can have forms with controls displaying data, buttons that cause things to happen. I can FTP files, monitor directories for activities. Believe me, if this exists in SSMS I will jump on that! Just don't tell me to go spend thousands to buy a tool that will do some of what C# can already do for me. Modern languages, and the .Net environment in particular have capabilities that (AFAICT) TSQL can not even dream of. TSQL dreams of manipulating data, C# dreams of manipulating ... the world. Robert I am not "fighting learning it", I am light years beyond where I was two years ago, and I learn more TSQL and SQL Server every day. However from what I have seen so far, TSQL will NEVER be what I need for the supervisor. And answers like you just provided aren't really much help. I am a consultant. I am a programmer, not a DBA, or a SQL Server administrator, or a SQL Server jock at a company somewhere. I need C# for automating my business, which is NOT being a DBA (I already said that I think). I will write C# applications for clients that use SQL Server, and others that do not use SQL Server. SQL Server is used by exactly ONE of my clients, so I cannot spend the time to do whatever it is that you envision SQL Server doing because I cannot amortize the learning over other clients. Now... if you want to answer my question, I am all ears. The first thing to answer (If I am to do all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. Let's take a concrete, real world example. I have a set of virtual machines which run a third party application. I place files into an "inbox" directory, and I wait for files to pop out of the running application into the "outbox" directory. Just an aside, the files placed into the inbox are created by a set of about 5 stored procedures, which can operate on any of dozens of databases in my server. It is the case that the third party application occasionally hangs. If it does I need to rename a file in a specific directory to cause that application to restart processing that file. It is my intention to use C# to execute the SPs to create the files, move the files into the VM, then monitor files dropped into the inbox, and files popping out into the outbox. Look for "no new files in X minutes" and if that condition exists, rename the file. Log files dropped into the in-box, files popping out into the out-box, and the time it took for the files to process through the application. Log any "hangs" restarted. Supervisor stuff. Explain to me how (and more importantly WHY) I would use TSQL to do that? Other than a "when TSQL is all you have" (which is NOT true for me) mentality. A SMALL part of my business is to perform some moderately complex processes using SQL Server and SPs as a tool. I envision C# as a supervisor, not a "middle layer". I envision an application (written in C#) that runs on my desktop which starts, then monitors SQL Server processes, displays errors, restarts processes if necessary, creates databases, tables etc, backs them up and detaches them when done, logging everything that happens, showing it all to me me in real time in a Windows application. Possibly even automatically generating invoices for the work done and emailing them to the client. Explain to me one more time (with feeling) how (and more importantly WHY) I would use TSQL to do all of that stuff? John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > Sorry, but using C# to do what you would and could do simply using > SQL Server is adding a middle layer you do not need. You can do > exactly the same thing using SQL Server and stored procedures and > writing to a log table about what the process is doing and keeping > track of it that way. And, not have to use a middle layer language > like C# to do it. > > I am not sure why writing T-SQL code is not like real programming to > you. But, it is. And, if you are going to continue using SQL > Server, you need to learn what it can do for you instead of always > fighting learning it. > > This is coming from someone that does T-SQL, C#, VB.Net and Access > VBA. So, I think I have a fair perspective of how they all work. _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4574 (20091104) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4574 (20091104) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Wed Nov 4 17:01:33 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 04 Nov 2009 18:01:33 -0500 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: References: Message-ID: <4AF207CD.2060307@colbyconsulting.com> Gustav, >as I'll never be more than a decent low-level DBA. Likewise. And that is precisely my problem. I think when you spend all day in TSQL it is easy to lose sight of just how crude the environment is. I have finally gotten to the point where I can write a SP, debug it, get it functioning etc but it is just hard. Not that C# isn't hard, but C# is a general purpose language where the effort spent is useful for other things. And with the debugging tools in .Net, debugging things is really pretty easy, very similar to many other environments. I can go weeks without doing SQL development and when I go back it is a PITA to pick back up. What it does, it does well, what it doesn't do... it doesn't do quite spectacularly. > I guess Robert is a very skilled DBA and can write T-SQL with his left foot It certainly seems so. But with my current understanding of SQL Server and TSQL I cannot imagine doing the big picture of my work strictly with TSQL. With my current understanding, TSQL will only ever be a hand full of processes, 30% of the whole job. Don't get me wrong, there is no alternative to the power of SQL Server, but in the end it is just a place to store data and some tools for getting it out and back in to tables. Everything else is out in the real world. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > I have no answer to you but if it can bring you some relief, I feel much the same. The database is for storing data, a business layer and/or a front end is for the logic and interface. > That said, nothing is perfect, and in the real world you will have to write some T-SQL or similar but I try to keep it at minimum, first because of the trouble of maintenance (for example, no version control I know of), second because I feel old when I write "GBasic style" code. And I'm not in a big corporation where I can get every expensive tool I can point at, and tools for SQL Server seem all to be enterprise-priced. So, writing views and T-SQL is because I have to, writing C# (and VBA for that matter) is because I like to. > > I guess Robert is a very skilled DBA and can write T-SQL with his left foot and that's very good - then we know where to ask for advice as I'll never be more than a decent low-level DBA. > > /gustav > > >>>> jwcolby at colbyconsulting.com 04-11-2009 22:11 >>> > Robert, > > I don't really want to get in an argument about this. I asked a serious question and got a somewhat > brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY answer I have received > so that is not particularly encouraging. > > 8( > > TSQL, at least as written from inside of SSMS is about like writing GBasic code back in 1982. Well, > maybe a tiny step up but not much. > > You are right, I probably just don't know how, but in what I will call "a real language" I can set > breakpoints, I can see a call stack, I have watch statements, I have... simple very basic DEBUG > capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in SSMS). I can have forms > with controls displaying data, buttons that cause things to happen. I can FTP files, monitor > directories for activities. > > Believe me, if this exists in SSMS I will jump on that! Just don't tell me to go spend thousands to > buy a tool that will do some of what C# can already do for me. > > Modern languages, and the .Net environment in particular have capabilities that (AFAICT) TSQL can > not even dream of. TSQL dreams of manipulating data, C# dreams of manipulating ... the world. > > Robert I am not "fighting learning it", I am light years beyond where I was two years ago, and I > learn more TSQL and SQL Server every day. However from what I have seen so far, TSQL will NEVER be > what I need for the supervisor. And answers like you just provided aren't really much help. > > I am a consultant. I am a programmer, not a DBA, or a SQL Server administrator, or a SQL Server > jock at a company somewhere. I need C# for automating my business, which is NOT being a DBA (I > already said that I think). I will write C# applications for clients that use SQL Server, and > others that do not use SQL Server. SQL Server is used by exactly ONE of my clients, so I cannot > spend the time to do whatever it is that you envision SQL Server doing because I cannot amortize the > learning over other clients. > > Now... if you want to answer my question, I am all ears. The first thing to answer (If I am to do > all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. > > Let's take a concrete, real world example. I have a set of virtual machines which run a third party > application. I place files into an "inbox" directory, and I wait for files to pop out of the > running application into the "outbox" directory. Just an aside, the files placed into the inbox are > created by a set of about 5 stored procedures, which can operate on any of dozens of databases in my > server. > > It is the case that the third party application occasionally hangs. If it does I need to rename a > file in a specific directory to cause that application to restart processing that file. > > It is my intention to use C# to execute the SPs to create the files, move the files into the VM, > then monitor files dropped into the inbox, and files popping out into the outbox. Look for "no new > files in X minutes" and if that condition exists, rename the file. Log files dropped into the > in-box, files popping out into the out-box, and the time it took for the files to process through > the application. Log any "hangs" restarted. Supervisor stuff. > > Explain to me how (and more importantly WHY) I would use TSQL to do that? Other than a "when TSQL > is all you have" (which is NOT true for me) mentality. > > A SMALL part of my business is to perform some moderately complex processes using SQL Server and SPs > as a tool. I envision C# as a supervisor, not a "middle layer". I envision an application (written > in C#) that runs on my desktop which starts, then monitors SQL Server processes, displays errors, > restarts processes if necessary, creates databases, tables etc, backs them up and detaches them when > done, logging everything that happens, showing it all to me me in real time in a Windows > application. Possibly even automatically generating invoices for the work done and emailing them to > the client. > > Explain to me one more time (with feeling) how (and more importantly WHY) I would use TSQL to do all > of that stuff? > > John W. Colby > www.ColbyConsulting.com > > > Robert Stewart wrote: >> Sorry, but using C# to do what you would and could do simply using >> SQL Server is adding a middle layer you do not need. You can do >> exactly the same thing using SQL Server and stored procedures and >> writing to a log table about what the process is doing and keeping >> track of it that way. And, not have to use a middle layer language >> like C# to do it. >> >> I am not sure why writing T-SQL code is not like real programming to >> you. But, it is. And, if you are going to continue using SQL >> Server, you need to learn what it can do for you instead of always >> fighting learning it. >> >> This is coming from someone that does T-SQL, C#, VB.Net and Access >> VBA. So, I think I have a fair perspective of how they all work. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From davidmcafee at gmail.com Wed Nov 4 17:06:37 2009 From: davidmcafee at gmail.com (David McAfee) Date: Wed, 4 Nov 2009 15:06:37 -0800 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> <4AF1EE1E.9030406@colbyconsulting.com> <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> Message-ID: <8786a4c00911041506o49c88352kee08cbbb64fd62b@mail.gmail.com> In SQL 2000 (QA) you would simply right click on the Sproc in the Object explorer and choose debug. Early beta versions of SSMS 2005 had the same ability (right click and choose "step into") but it was removed and moved to Visual Studio instead. :( On Wed, Nov 4, 2009 at 2:48 PM, Shamil Salakhetdinov wrote: > Hi John, > > <<< > The first thing to answer (If I am to do > all this in TSQL) is how do I get a modern tool with all > of the debug capabilities to write TSQL code. >>>> > > Overview of T-SQL and CLR debugging in SQL Server 2005 > http://blogs.msdn.com/sqlclr/archive/2006/06/29/651644.aspx > > How to: Enable SQL Server 2005 Debugging > http://msdn.microsoft.com/en-us/library/s0fk6z6e(VS.80).aspx > > HTH, > > --Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 12:12 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes > > Robert, > > I don't really want to get in an argument about this. ?I asked a serious > question and got a somewhat > brusque "answer" that isn't an answer at all. ?OTOH it is (so far) the ONLY > answer I have received > so that is not particularly encouraging. > > 8( > > TSQL, at least as written from inside of SSMS is about like writing GBasic > code back in 1982. ?Well, > maybe a tiny step up but not much. > > You are right, I probably just don't know how, but in what I will call "a > real language" I can set > breakpoints, I can see a call stack, I have watch statements, ?I have... > simple very basic DEBUG > capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in > SSMS). ?I can have forms > with controls displaying data, buttons that cause things to happen. ?I can > FTP files, monitor > directories for activities. > > Believe me, if this exists in SSMS I will jump on that! ?Just don't tell me > to go spend thousands to > buy a tool that will do some of what C# can already do for me. > > Modern languages, and the .Net environment in particular have capabilities > that (AFAICT) TSQL can > not even dream of. ?TSQL dreams of manipulating data, C# dreams of > manipulating ... the world. > > Robert I am not "fighting learning it", I am light years beyond where I was > two years ago, and I > learn more TSQL and SQL Server every day. ?However from what I have seen so > far, TSQL will NEVER be > what I need for the supervisor. ?And answers like you just provided aren't > really much help. > > I am a consultant. ?I am a programmer, not a DBA, or a SQL Server > administrator, or a SQL Server > jock at a company somewhere. ?I need C# for automating my business, which is > NOT being a DBA (I > already said that I think). ?I will write C# applications for clients that > use SQL Server, and > others that do not use SQL Server. ?SQL Server is used by exactly ONE of my > clients, so I cannot > spend the time to do whatever it is that you envision SQL Server doing > because I cannot amortize the > learning over other clients. > > Now... if you want to answer my question, I am all ears. ?The first thing to > answer (If I am to do > all this in TSQL) is how do I get a modern tool with all of the debug > capabilities to write TSQL code. > > Let's take a concrete, real world example. ?I have a set of virtual machines > which run a third party > application. ?I place files into an "inbox" directory, and I wait for files > to pop out of the > running application into the "outbox" directory. ?Just an aside, the files > placed into the inbox are > created by a set of about 5 stored procedures, which can operate on any of > dozens of databases in my > server. > > It is the case that the third party application occasionally hangs. ?If it > does I need to rename a > file in a specific directory to cause that application to restart processing > that file. > > It is my intention to use C# to execute the SPs to create the files, move > the files into the VM, > then monitor files dropped into the inbox, and files popping out into the > outbox. ?Look for "no new > files in X minutes" and if that condition exists, rename the file. ?Log > files dropped into the > in-box, files popping out into the out-box, and the time it took for the > files to process through > the application. ?Log any "hangs" restarted. ?Supervisor stuff. > > Explain to me how (and more importantly WHY) I would use TSQL to do that? > Other than a "when TSQL > is all you have" (which is NOT true for me) mentality. > > A SMALL part of my business is to perform some moderately complex processes > using SQL Server and SPs > as a tool. ?I envision C# as a supervisor, not a "middle layer". ?I envision > an application (written > in C#) that runs on my desktop which starts, then monitors SQL Server > processes, displays errors, > restarts processes if necessary, creates databases, tables etc, backs them > up and detaches them when > done, logging everything that happens, showing it all to me me in real time > in a Windows > application. ?Possibly even automatically generating invoices for the work > done and emailing them to > the client. > > Explain to me one more time (with feeling) how (and more importantly WHY) I > would use TSQL to do all > of that stuff? > > John W. Colby > www.ColbyConsulting.com > > > Robert Stewart wrote: >> Sorry, but using C# to do what you would and could do simply using >> SQL Server is adding a middle layer you do not need. You can do >> exactly the same thing using SQL Server and stored procedures and >> writing to a log table about what the process is doing and keeping >> track of it that way. And, not have to use a middle layer language >> like C# to do it. >> >> I am not sure why writing T-SQL code is not like real programming to >> you. ?But, it is. ?And, if you are going to continue using SQL >> Server, you need to learn what it can do for you instead of always >> fighting learning it. >> >> This is coming from someone that does T-SQL, C#, VB.Net and Access >> VBA. ?So, I think I have a fair perspective of how they all work. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Wed Nov 4 17:08:49 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 04 Nov 2009 18:08:49 -0500 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> <4AF1EE1E.9030406@colbyconsulting.com> <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> Message-ID: <4AF20981.3060804@colbyconsulting.com> Thanks for that. I guess I will need to install my professional edition license to go there. Worthwhile though I am guessing. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > <<< > The first thing to answer (If I am to do > all this in TSQL) is how do I get a modern tool with all > of the debug capabilities to write TSQL code. > > Overview of T-SQL and CLR debugging in SQL Server 2005 > http://blogs.msdn.com/sqlclr/archive/2006/06/29/651644.aspx > > How to: Enable SQL Server 2005 Debugging > http://msdn.microsoft.com/en-us/library/s0fk6z6e(VS.80).aspx > > HTH, > > --Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 12:12 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes > > Robert, > > I don't really want to get in an argument about this. I asked a serious > question and got a somewhat > brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY > answer I have received > so that is not particularly encouraging. > > 8( > > TSQL, at least as written from inside of SSMS is about like writing GBasic > code back in 1982. Well, > maybe a tiny step up but not much. > > You are right, I probably just don't know how, but in what I will call "a > real language" I can set > breakpoints, I can see a call stack, I have watch statements, I have... > simple very basic DEBUG > capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in > SSMS). I can have forms > with controls displaying data, buttons that cause things to happen. I can > FTP files, monitor > directories for activities. > > Believe me, if this exists in SSMS I will jump on that! Just don't tell me > to go spend thousands to > buy a tool that will do some of what C# can already do for me. > > Modern languages, and the .Net environment in particular have capabilities > that (AFAICT) TSQL can > not even dream of. TSQL dreams of manipulating data, C# dreams of > manipulating ... the world. > > Robert I am not "fighting learning it", I am light years beyond where I was > two years ago, and I > learn more TSQL and SQL Server every day. However from what I have seen so > far, TSQL will NEVER be > what I need for the supervisor. And answers like you just provided aren't > really much help. > > I am a consultant. I am a programmer, not a DBA, or a SQL Server > administrator, or a SQL Server > jock at a company somewhere. I need C# for automating my business, which is > NOT being a DBA (I > already said that I think). I will write C# applications for clients that > use SQL Server, and > others that do not use SQL Server. SQL Server is used by exactly ONE of my > clients, so I cannot > spend the time to do whatever it is that you envision SQL Server doing > because I cannot amortize the > learning over other clients. > > Now... if you want to answer my question, I am all ears. The first thing to > answer (If I am to do > all this in TSQL) is how do I get a modern tool with all of the debug > capabilities to write TSQL code. > > Let's take a concrete, real world example. I have a set of virtual machines > which run a third party > application. I place files into an "inbox" directory, and I wait for files > to pop out of the > running application into the "outbox" directory. Just an aside, the files > placed into the inbox are > created by a set of about 5 stored procedures, which can operate on any of > dozens of databases in my > server. > > It is the case that the third party application occasionally hangs. If it > does I need to rename a > file in a specific directory to cause that application to restart processing > that file. > > It is my intention to use C# to execute the SPs to create the files, move > the files into the VM, > then monitor files dropped into the inbox, and files popping out into the > outbox. Look for "no new > files in X minutes" and if that condition exists, rename the file. Log > files dropped into the > in-box, files popping out into the out-box, and the time it took for the > files to process through > the application. Log any "hangs" restarted. Supervisor stuff. > > Explain to me how (and more importantly WHY) I would use TSQL to do that? > Other than a "when TSQL > is all you have" (which is NOT true for me) mentality. > > A SMALL part of my business is to perform some moderately complex processes > using SQL Server and SPs > as a tool. I envision C# as a supervisor, not a "middle layer". I envision > an application (written > in C#) that runs on my desktop which starts, then monitors SQL Server > processes, displays errors, > restarts processes if necessary, creates databases, tables etc, backs them > up and detaches them when > done, logging everything that happens, showing it all to me me in real time > in a Windows > application. Possibly even automatically generating invoices for the work > done and emailing them to > the client. > > Explain to me one more time (with feeling) how (and more importantly WHY) I > would use TSQL to do all > of that stuff? > > John W. Colby > www.ColbyConsulting.com > > > Robert Stewart wrote: >> Sorry, but using C# to do what you would and could do simply using >> SQL Server is adding a middle layer you do not need. You can do >> exactly the same thing using SQL Server and stored procedures and >> writing to a log table about what the process is doing and keeping >> track of it that way. And, not have to use a middle layer language >> like C# to do it. >> >> I am not sure why writing T-SQL code is not like real programming to >> you. But, it is. And, if you are going to continue using SQL >> Server, you need to learn what it can do for you instead of always >> fighting learning it. >> >> This is coming from someone that does T-SQL, C#, VB.Net and Access >> VBA. So, I think I have a fair perspective of how they all work. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From marklbreen at gmail.com Thu Nov 5 01:43:49 2009 From: marklbreen at gmail.com (Mark Breen) Date: Thu, 5 Nov 2009 15:43:49 +0800 Subject: [dba-VB] Fwd: How do you run SQL Server processes In-Reply-To: References: <4AF207CD.2060307@colbyconsulting.com> Message-ID: Hello John, FWIW, I agree with you that you need a rich environment to do the work that you plan to do. However, I think that Robert is correct in suggestion to try, as much as possible to not cross the technology stack with volumes of data when you do not have to. IOW, do all the heavy lifting in the database, and by all means return as many parameters or log results back to the front end you are building in C#, but try to avoid returning your data when ever possible. I guess I was suggesting the same with with SSIS. Where you have to export data based on decisions, SSIS is an extremely fast option. Have you already used EXECUTE command to call an sproc within an sproc? This is a useful means of creating an sproc, that running a set of sprocs, and can insert / update a log table as it progresses. For data only actvity, this might be effective. For all the front end work however, I guess you will make C# sing, like only you can, so work away, and let TSQL do what it is good at, all the rest should not involve enormous volumes of data, so the technology bridge will be irrelevant. Good luck, you know you are lucky. Most Database people never get to read about a db as big as what you have :) On last thing, in SQL Server, Cursors are a dirty word, I never see them used. However, when you have low volumes of iteration (thousands or a few tens of thousands), cursors are a very powerful way of enumerating tasks. Eg, lets imagine you had a table of states, and you needed to perform ten seperate sprocs for each state. You could use a cursor to enumerate the states table and pass in the state name to the ten different sprocs. You could log between each iteration, and you could have the whole thing running in 30 minutes. In this sort of context, I agree with Robert, that TSQL can give you Raw power quickly and easily. C# could be watching your log table and your 'Status Update' table and simply returning 1 or 2 records from the db to C# per minute, whereas within your sproc that is enumerating it might be handling millions of records each two minutes. Those millions would never cross the stack divide of DB Server to .Net. Again this sharing of processing power is probably what Robert was referring to. BTW I recently started using FileZilla and find it super. Good Luck, Mark Mark From shamil at smsconsulting.spb.ru Thu Nov 5 07:01:18 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 16:01:18 +0300 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <4AF20981.3060804@colbyconsulting.com> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> <4AF1EE1E.9030406@colbyconsulting.com> <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> <4AF20981.3060804@colbyconsulting.com> Message-ID: <00da01ca5e18$119f5080$34ddf180$@spb.ru> Hi John, I just posted links as you asked about this feature but I must note I have used T-SQL debugging just once when I used T-SQL cursors (and that was my mistake I suppose) - IOW I doubt that cursors/T-SQL debugging will be ever needed for you as you plan to use C# to drive your T-SQL's SPs... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 2:09 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes Thanks for that. I guess I will need to install my professional edition license to go there. Worthwhile though I am guessing. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > <<< > The first thing to answer (If I am to do > all this in TSQL) is how do I get a modern tool with all > of the debug capabilities to write TSQL code. > > Overview of T-SQL and CLR debugging in SQL Server 2005 > http://blogs.msdn.com/sqlclr/archive/2006/06/29/651644.aspx > > How to: Enable SQL Server 2005 Debugging > http://msdn.microsoft.com/en-us/library/s0fk6z6e(VS.80).aspx > > HTH, > > --Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 12:12 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes > > Robert, > > I don't really want to get in an argument about this. I asked a serious > question and got a somewhat > brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY > answer I have received > so that is not particularly encouraging. > > 8( > > TSQL, at least as written from inside of SSMS is about like writing GBasic > code back in 1982. Well, > maybe a tiny step up but not much. > > You are right, I probably just don't know how, but in what I will call "a > real language" I can set > breakpoints, I can see a call stack, I have watch statements, I have... > simple very basic DEBUG > capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in > SSMS). I can have forms > with controls displaying data, buttons that cause things to happen. I can > FTP files, monitor > directories for activities. > > Believe me, if this exists in SSMS I will jump on that! Just don't tell me > to go spend thousands to > buy a tool that will do some of what C# can already do for me. > > Modern languages, and the .Net environment in particular have capabilities > that (AFAICT) TSQL can > not even dream of. TSQL dreams of manipulating data, C# dreams of > manipulating ... the world. > > Robert I am not "fighting learning it", I am light years beyond where I was > two years ago, and I > learn more TSQL and SQL Server every day. However from what I have seen so > far, TSQL will NEVER be > what I need for the supervisor. And answers like you just provided aren't > really much help. > > I am a consultant. I am a programmer, not a DBA, or a SQL Server > administrator, or a SQL Server > jock at a company somewhere. I need C# for automating my business, which is > NOT being a DBA (I > already said that I think). I will write C# applications for clients that > use SQL Server, and > others that do not use SQL Server. SQL Server is used by exactly ONE of my > clients, so I cannot > spend the time to do whatever it is that you envision SQL Server doing > because I cannot amortize the > learning over other clients. > > Now... if you want to answer my question, I am all ears. The first thing to > answer (If I am to do > all this in TSQL) is how do I get a modern tool with all of the debug > capabilities to write TSQL code. > > Let's take a concrete, real world example. I have a set of virtual machines > which run a third party > application. I place files into an "inbox" directory, and I wait for files > to pop out of the > running application into the "outbox" directory. Just an aside, the files > placed into the inbox are > created by a set of about 5 stored procedures, which can operate on any of > dozens of databases in my > server. > > It is the case that the third party application occasionally hangs. If it > does I need to rename a > file in a specific directory to cause that application to restart processing > that file. > > It is my intention to use C# to execute the SPs to create the files, move > the files into the VM, > then monitor files dropped into the inbox, and files popping out into the > outbox. Look for "no new > files in X minutes" and if that condition exists, rename the file. Log > files dropped into the > in-box, files popping out into the out-box, and the time it took for the > files to process through > the application. Log any "hangs" restarted. Supervisor stuff. > > Explain to me how (and more importantly WHY) I would use TSQL to do that? > Other than a "when TSQL > is all you have" (which is NOT true for me) mentality. > > A SMALL part of my business is to perform some moderately complex processes > using SQL Server and SPs > as a tool. I envision C# as a supervisor, not a "middle layer". I envision > an application (written > in C#) that runs on my desktop which starts, then monitors SQL Server > processes, displays errors, > restarts processes if necessary, creates databases, tables etc, backs them > up and detaches them when > done, logging everything that happens, showing it all to me me in real time > in a Windows > application. Possibly even automatically generating invoices for the work > done and emailing them to > the client. > > Explain to me one more time (with feeling) how (and more importantly WHY) I > would use TSQL to do all > of that stuff? > > John W. Colby > www.ColbyConsulting.com > > > Robert Stewart wrote: >> Sorry, but using C# to do what you would and could do simply using >> SQL Server is adding a middle layer you do not need. You can do >> exactly the same thing using SQL Server and stored procedures and >> writing to a log table about what the process is doing and keeping >> track of it that way. And, not have to use a middle layer language >> like C# to do it. >> >> I am not sure why writing T-SQL code is not like real programming to >> you. But, it is. And, if you are going to continue using SQL >> Server, you need to learn what it can do for you instead of always >> fighting learning it. >> >> This is coming from someone that does T-SQL, C#, VB.Net and Access >> VBA. So, I think I have a fair perspective of how they all work. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4575 (20091105) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Thu Nov 5 07:52:53 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 05 Nov 2009 08:52:53 -0500 Subject: [dba-VB] Fwd: How do you run SQL Server processes In-Reply-To: References: <4AF207CD.2060307@colbyconsulting.com> Message-ID: <4AF2D8B5.1030507@colbyconsulting.com> Mark, > However, I think that Robert is correct in suggestion to try, as much as possible to not cross the technology stack with volumes of data when you do not have to. I have never had a problem with letting each tool do what it does best. In my case (or for this specific client) the database is decidedly not "normal". I have many different databases, each with a few tables, and ZERO (yes ZERO) parent / child tables. I keep tables of names / addresses. That's it. So I am not doing transactions, I am not doing reports, I am not doing data entry, I am not doing any of the "normal" things that people do with databases. What I have to do is get the entire table of name / address out to an external process and then back in to the same database, and I have to do this at least monthly. We are talking 20, 50, 80 million names exported into 2 million record files, processed through this external program, and then imported back in again. This is a "huge" process, not in terms of sheer SQL complexity but rather in terms of the steps involved, the monitoring that has to occur, and the time it takes. And I have to do this on a half dozen large databases, monthly. The external program processes about 2 million records an hour start to finish. Do the math and 60 million records turns into 30 hours of external processing. I use three virtual machines to get three times the records / hour but you are still talking 10 hours and that doesn't count the time to get the records out and back in to SQL Server again. My thought was to start a discussion about that "periphery processing" so that I could get ideas from people who already do it. I am getting almost no response however with actual "I do that using..." responses, and I am wondering why. Perhaps because I deal with small clients, the cost of manual labor to do those kinds of things by hand is too high so they hire me to automate it. Perhaps in a big organization they simply throw people at the problem? Believe me, when you get down to a "one person company" (me!) you can't do this stuff manually for very long, you MUST automate it or you will spend you entire day doing stuff that should be done by the computer. My original question was aimed at discovering how other people handle this kind of processing. I have another client (in Access in this case) that routinely exports data using very specialized export functions (that I wrote) dictated by mainframe interface specs. The data has to be pulled, exported, formatted, sometimes encrypted, sometimes zipped, sometimes FTPd, sometimes emailed, (the "sometimes" part depending on the recipient's preferences), and emails sent announcing the data is on the way etc. I do all of this in Access currently but it is ugly in Access because VBA does not have the built-in tools to do ftp/zip/encrypt etc. C# has some of that built-in and you can usually find C# routines to do the rest. BTW, as I have mentioned, I am taking a C# class at the local community college. I met a Russian gentleman, 60ish, who has been in programming / IT all of his life, recently unemployed. He and I are the only two in the class that have actually worked in the industry (other than the teacher). We struck up a conversation and he asked me if I would like to bid with him on a couple of small C# jobs on a jobs board "just to get some real world experience". I counter-offered that he join me in automating some of my real-world tasks. He came over yesterday and we are going to be working together on the kinds of tasks I described above. He has a LOT of C# experience but almost no database experience beyond simple select queries. We should each learn a lot from the other, it should be fun, and hopefully I can get some of this automation done! John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > FWIW, I agree with you that you need a rich environment to do the work that > you plan to do. > > However, I think that Robert is correct in suggestion to try, as much as > possible to not cross the technology stack with volumes of data when you do > not have to. > > IOW, do all the heavy lifting in the database, and by all means return as > many parameters or log results back to the front end you are building in C#, > but try to avoid returning your data when ever possible. > > I guess I was suggesting the same with with SSIS. Where you have to export > data based on decisions, SSIS is an extremely fast option. > > Have you already used EXECUTE command to call an sproc within an sproc? > This is a useful means of creating an sproc, that running a set of sprocs, > and can insert / update a log table as it progresses. For data only > actvity, this might be effective. > > For all the front end work however, I guess you will make C# sing, like only > you can, so work away, and let TSQL do what it is good at, all the rest > should not involve enormous volumes of data, so the technology bridge will > be irrelevant. > > Good luck, you know you are lucky. Most Database people never get to read > about a db as big as what you have :) > > On last thing, in SQL Server, Cursors are a dirty word, I never see them > used. However, when you have low volumes of iteration (thousands or a few > tens of thousands), cursors are a very powerful way of enumerating tasks. > > Eg, lets imagine you had a table of states, and you needed to perform ten > seperate sprocs for each state. You could use a cursor to enumerate the > states table and pass in the state name to the ten different sprocs. > > You could log between each iteration, and you could have the whole thing > running in 30 minutes. In this sort of context, I agree with Robert, that > TSQL can give you Raw power quickly and easily. > > C# could be watching your log table and your 'Status Update' table and > simply returning 1 or 2 records from the db to C# per minute, whereas within > your sproc that is enumerating it might be handling millions of records each > two minutes. Those millions would never cross the stack divide of DB Server > to .Net. Again this sharing of processing power is probably what Robert was > referring to. > > BTW I recently started using FileZilla and find it super. > > Good Luck, > > Mark > > > > Mark > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From raibeart at gmail.com Thu Nov 5 09:16:01 2009 From: raibeart at gmail.com (Robert Stewart) Date: Thu, 05 Nov 2009 09:16:01 -0600 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: References: Message-ID: <4af2ec32.0305560a.75b1.0901@mx.google.com> John, Mark has it exactly correct. Use SQL Server to the max of what it can do for you before you go off creating something in c# to do essentially the same things. Data is what it does, C# is the GUI. Very similar to what we have all done with MS Access over the years. Now, lets get down to what you are doing... Running code out of a database table is not efficient. The optimizer has to work on it before it can be run. So, the processing will be slower. It would be better to have the code one or more stored procedures with all the parameters and run them against small data sets to get them optimized. Then they would be ready for the larger chunks of data. Each SP can write to a log table and your C# GUI can display the progress by reading data out of the log tables. By using the log tables, you can pass start times, records processed and the time at predetermined numbers of records and which stored proc is doing the work at the time, end times, and just about any other information you may want, Including things that cause errors. SSIS is extremely powerful and the real tool that should probably be used for what you want to do. It can do all of the logging, error trapping and handle passing errored records into a different table for analysis. The C# GUI could be used to read the output from the SSIS package to do the monitoring. It is also relatively simple to sopy SSIS packages and change the parameters around on them to allow them to be used for different extractions and builds. Robert At 01:43 AM 11/5/2009, you wrote: >Date: Thu, 5 Nov 2009 15:43:49 +0800 >From: Mark Breen >Subject: [dba-VB] Fwd: How do you run SQL Server processes >To: dba-vb at databaseadvisors.com >Message-ID: > >Content-Type: text/plain; charset=ISO-8859-1 > >Hello John, > >FWIW, I agree with you that you need a rich environment to do the work that >you plan to do. > >However, I think that Robert is correct in suggestion to try, as much as >possible to not cross the technology stack with volumes of data when you do >not have to. > >IOW, do all the heavy lifting in the database, and by all means return as >many parameters or log results back to the front end you are building in C#, >but try to avoid returning your data when ever possible. > >I guess I was suggesting the same with with SSIS. Where you have to export >data based on decisions, SSIS is an extremely fast option. > >Have you already used EXECUTE command to call an sproc within an sproc? > This is a useful means of creating an sproc, that running a set of sprocs, >and can insert / update a log table as it progresses. For data only >actvity, this might be effective. > >For all the front end work however, I guess you will make C# sing, like only >you can, so work away, and let TSQL do what it is good at, all the rest >should not involve enormous volumes of data, so the technology bridge will >be irrelevant. > >Good luck, you know you are lucky. Most Database people never get to read >about a db as big as what you have :) > >On last thing, in SQL Server, Cursors are a dirty word, I never see them >used. However, when you have low volumes of iteration (thousands or a few >tens of thousands), cursors are a very powerful way of enumerating tasks. > >Eg, lets imagine you had a table of states, and you needed to perform ten >seperate sprocs for each state. You could use a cursor to enumerate the >states table and pass in the state name to the ten different sprocs. > >You could log between each iteration, and you could have the whole thing >running in 30 minutes. In this sort of context, I agree with Robert, that >TSQL can give you Raw power quickly and easily. > >C# could be watching your log table and your 'Status Update' table and >simply returning 1 or 2 records from the db to C# per minute, whereas within >your sproc that is enumerating it might be handling millions of records each >two minutes. Those millions would never cross the stack divide of DB Server >to .Net. Again this sharing of processing power is probably what Robert was >referring to. > >BTW I recently started using FileZilla and find it super. > >Good Luck, > >Mark > > > >Mark From shamil at smsconsulting.spb.ru Thu Nov 5 09:22:02 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 18:22:02 +0300 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <4AF0F117.20209@colbyconsulting.com> References: <4AF0F117.20209@colbyconsulting.com> Message-ID: <00db01ca5e2b$bad004c0$30700e40$@spb.ru> Hi John, <<< I am hoping to eventually make the entire process "table driven" where I can store (in a table) the name of the sp, the parameters in a child table etc. >>> Using table-driven process could be not quite right in C#/.Net world, which proposes many other options to implement generic solutions - I'd just start with "hardcoded" solutions using calls to a set of small "generic" classes/methods executing custom (MS SQL) database processing, when that set of methods/functions stabilizes I'd try to generalize/"generic-ize" it if that will be needed at all... What is the purpose of your "going generic" for this sets of tasks/customer? Isn't it "just for fun" of generic programming? Would that "going generic" make your programming more effective? Developing/debugging/testing generic custom "table-driven"/"other-metadata media driven" solutions is always (much) more time consuming than "hardcoded" ones, and supporting "table-driven" generic custom solutions, especially in the context of constantly changing customer requirements also promise to be rather time consuming - I mean generic "table-"/metadata-driven solutions could only(?) pay back well in the case they are used on many deployment sites by many people. I can be wrong, just talking from my experience, which is not universal of course... I'd better propose to use metadata descriptions/specs (stored in db tables or (XML) files), if you're so bored to write repetitive code, to generate "hardcoded" solutions, then make quick fixes to this generated hardcode to satisfy your customers' current requests, then (in free time) adjust code generator, then when everything will stabilize (will that ever happen?) develop truly generic "table-driven" solution. You can also consider using F# and Windows Workflow Foundation (WF)... The following presentation seems to be useful to get true picture of "generic vs. specific" solutions trade-offs: http://www.infoq.com/presentations/Generic-Specific-Tradeoffs-Stefan-Tilkov Thank you. -- Shamil P.S. FYI: AFAIKR MS has got created Domain Specific Language (DSL) team to develop DSL tools somewhere in year 2004 and they planned to come with first release in one(?) year but they aren't yet there (?)... -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 04, 2009 6:12 AM To: VBA; Dba-Sqlserver Subject: [dba-VB] How do you run SQL Server processes I have processes that I run which use sets of stored procedures. For example two of the processes export / import records from large tables for address validation. The addresses need to be sorted in Zip5/Zip4/Address order in order to make the external validation process as fast as possible so I create a table that pulls millions of records, often tens of millions of records, into a table (created on-the-fly) in sorted order, then pulls out 2 million records at a time and BCPs them out for processing in an external program. The inverse occurs when the records finish processing, 2 million record chunks are imported back into a temp table, then each 2 million record chunk is appended to a main table. Once back in SQL Server I have processes that build clustered and non-clustered indexes, run sha-1 hash functions to generate hash keys, update address valid codes etc. I will run this entire project at least once a month (to keep the addresses valid as people move), and I do so for many tables - a table of about 65 million addresses, another of 21 million, another of 11 million etc. I store all of my stored procedures in a dedicated database that pretty much just contains the SPs and a few UDFs as well as a couple of logging type tables. At the moment I use an Access database to run the individual stored procedures in specific order. I am just embarking on a project to move that functioning but not very efficient Access database to C#. I want to be able to have better monitoring of SP completion, records processed (counts), time to complete each SP etc. I found a pretty nice set of C# classes that allow creation of "stored procedure" objects, with parameter objects, then stored into a collection as the SPs (class instances) are assembled and finally the whole collection of SPs executed. I created a test database of a million names / addresses and today I started testing my existing SPs on this small data set. In the next few days I expect to build out a preliminary "port" of the existing Access VBA code. I am hoping to eventually make the entire process "table driven" where I can store (in a table) the name of the sp, the parameters in a child table etc. and then have a supervisor pull out the records and execute the SPs based on what is in the table. This would allow me to create the SPs and then use a C# application (data entry forms) to organize the SPs, add / delete SPs from the process and so forth. That is down the road of course, for now I will have hard coded sequences - as I already do in supervisor SPs as well as in hard coded Access VBA modules. As a programmer, doing this supervisor level in C# feels more natural to me (than doing it inside of SQL Server), and gives me the flexibility to do what I want in code external to the SQL Server database being manipulated. I want to be able to do things external to the database processes, things like monitor the directories that the BCP processes export to / from, move files popping out of a bcp process off to a VM on a different server, monitor those VMs for files coming out of their processes, moving the files back over to the SQL Server directories, and trigger BCP processes that import the data back in to SQL server. I need to be able to FTP order and data files to clients, and get (FTP) files from process bureaus. So it FEELS to me like using a language such as C# to build a system outside of SQL Server is the way to go. BTW I stumbled across something called SMO (SQL Server Management Objects) which makes it really easy to get at the SQL Server object model from C#. I can apparently see the entire tree structure that is a server/database/tables/etc. and do I know not what (yet) with those objects. Looks pretty cool though. The database is a large part of what I do but it is not the whole picture and I have been handcuffed by my lack of ability to effectively do these external parts. But I am a programmer at heart. I want to know how you do this kind of stuff. I know that running such "systems" of stored procedures tied to external processes must be quite common and I am wondering what you guys do for supervising your processes. If anyone doing anything remotely similar would take the time to explain how you do it, I would be most appreciative. Thanks, -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4573 (20091104) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From shamil at smsconsulting.spb.ru Thu Nov 5 09:39:05 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 18:39:05 +0300 Subject: [dba-VB] Recommended watching: "Thoughts on the Generic vs. Specific Tradeoff" Message-ID: <00e801ca5e2e$1c7f93f0$557ebbd0$@spb.ru> Hi All, The following presentation is a one from a wise man talking from his real life experience - it's fun and so useful to watch, recommended: http://www.infoq.com/presentations/Generic-Specific-Tradeoffs-Stefan-Tilkov Thank you. -- Shamil From jwcolby at colbyconsulting.com Thu Nov 5 10:08:36 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 05 Nov 2009 11:08:36 -0500 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <4af2ec32.0305560a.75b1.0901@mx.google.com> References: <4af2ec32.0305560a.75b1.0901@mx.google.com> Message-ID: <4AF2F884.2040903@colbyconsulting.com> Robert, Thanks for the reply. The issue here is that I have to run the same code against a large set of databases / tables. Fir this discussion, the "process" is exporting millions of records to CSV and later importing the resulting CSVs back in again. For example I have about 6 different databases, each with it's own set of tables. Additionally I have to do this exact same process on "Orders" where I am selecting anywhere from a few thousand to a few million addresses to fill an order. The client wants those orders processed as well. Understand that I am not claiming that the following is the most efficient, just "what I know how to do". So my solution so far has been to build a set of stored procedures where I pass in the database and the source table (for the export process) and dynamically build up SQL statements in the SP. The generic process is to: 1) SP1: Build a big table to hold the entire set of records, sorted on Zip5/Zip4, with an autonumber for selection of 2 million record chunks. Move the records from TblSrc (or ViewSrc) to TblBig. Build an index on autonumber and the fields to export to make the subsequent processing faster. Params DB, TblNameSrc. 2) SP2: Build a chunk table. Move a chunk from TblBig to TblChunk Params: DBName, StartPKID, EndPKID 3) SP3: BCP TblChunk to CSV file Params: DBName, Destination Dir This is not an exact representation of what is going on but gets the point across, which is that I have to do this on a "random" database, probably a dozen a month. Initially I did this in stored procedures stored inside of each database but the maintenance of the stored procedures became a nightmare. As I would figure out a better way, a different way, fix a bug... I would be constantly trying to modify my code in each database. NOT FUN!!! Eventually I moved to the parameterized stored procedure with dynamically constructed SQL executed inside of try/catch blocks etc. I store all of these SPs out in a "management" database and execute them from there, passing in the database name, table name and any other parameters required to make each SP work. As I have said, I don't know enough to determine whether this is "the best way", I have enough problems just getting this much to work. I have in fact started using logging from inside of each SP. I designed a standard log table with fields for the SP name being processed, record counts, error codes if any, memo field so I can write little explanation of each step and so forth. I JUST started doing that. I created a SP (again parameterized, stored in the management db) that performs the write to the log, then I call that SP whenever I want to log anything. The thing everyone following this thread needs to understand is that I have no resource other than this list and Google to learn how to do this stuff. I work alone, I do what I have to, as quickly as I reasonably can, because my job (for this client) is to process data, NOT write code. I am a consultant providing a service, NOT a DBA. That said, I have no one but myself to assist me in automating this stuff so that I do not do every step manually, and believe me, when I started I did EVERY STEP manually. Everything that is now a SP I did manually, sometimes dozens of times over the period I have worked for this client. It was not only UGLY, it was DAMNED UGLY. I am a programmer by trade, a DBA/Programmer by necessity, a consultant/ business analyst/ DBA/ Programmer by job description. I view ALL of this from "How can I most quickly, with the least effort, make my job easier". I don't have the luxury of learning anything that I don't HAVE to know, right now, to do my job. I also don't have the luxury of spending weeks working on one specific problem. Do a task for client A, move on to the task for client B. At the end of the month, bill for my time. Thanks for the hint to use SSIS. That is one of those "haven't had the time to learn it" things but I have always known I probably should. When you are up to your ass in alligators, it is hard to STOP... and study SSIS. John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > John, > > Mark has it exactly correct. > > Use SQL Server to the max of what it can do for you before you go off > creating something in c# to do essentially the same things. Data is what > it does, C# is the GUI. Very similar to what we have all done with MS > Access over the years. > > Now, lets get down to what you are doing... > > Running code out of a database table is not efficient. The optimizer has to > work on it before it can be run. So, the processing will be slower. > > It would be better to have the code one or more stored procedures with all the > parameters and run them against small data sets to get them > optimized. Then they > would be ready for the larger chunks of data. > > Each SP can write to a log table and your C# GUI can display the progress by > reading data out of the log tables. > > By using the log tables, you can pass start times, records processed and the > time at predetermined numbers of records and which stored proc is doing the > work at the time, end times, and just about any other information you may want, > Including things that cause errors. > > SSIS is extremely powerful and the real tool that should probably be used for > what you want to do. It can do all of the logging, error trapping and handle > passing errored records into a different table for analysis. The C# GUI could > be used to read the output from the SSIS package to do the monitoring. It is > also relatively simple to sopy SSIS packages and change the parameters around > on them to allow them to be used for different extractions and builds. > > Robert From jwcolby at colbyconsulting.com Thu Nov 5 10:33:38 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 05 Nov 2009 11:33:38 -0500 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <00db01ca5e2b$bad004c0$30700e40$@spb.ru> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> Message-ID: <4AF2FE62.2050501@colbyconsulting.com> Shamil, > Using table-driven process could be not quite right in C#/.Net world, which proposes many other options to implement generic solutions - I'd just start with "hardcoded" solutions using calls to a set of small "generic" That is precisely what I do right now, and what I will do for the near future. The table driven thing is simply that I write "processes" as discussed in this thread. A process might be: Export To Accuzip: "Export dbX TblY to files in DirectoryZ". However that breaks down onto a set of a handful of stored procedures, with passed in parameters. Now, the process "Export To Accuzip" is applied against 6 or 8 different databases every month, but ALSO against random databases which are orders. Orders come in every month and they are a whole nother discussion, but PART of an order is the process Export to Accuzip. DatabaseA exports files to its own directory, DatabaseB to its own directory, OrderAA to its own directory and so forth. But the EXPORT itself is a very standard process. So I am examining the concept of storing the process in a table, so that I can simply select (from a form: Process X: Use SP1, params use SP4, Params use SP9, Params Process Y: Use SP1, Params Use SP2, Params Use SP5, Params Process Z: Use... Table driven (to the extent possible). Then a supervisor application could execute a process (set of SPs), read out of a table, passing in only the database name and possibly a directory on disk. Everything else is selected and set up in advance. To see what a process is, just open a table, to modify the process, select different stored procedures. This stuff is VERY standard, with just a few things that change from DB to db (the database name and where files are going) however I do occasionally shufffle around the interior processes (indexes created, fields used etc. That stuff is internal to the SP and if I change that it is in the SP itself. Then that change is applied to every database from then on. I am not saying that this will work but as my understand and knowledge changes I am spending a LOT of time rewriting stuff. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > <<< > I am hoping to eventually make the entire process "table driven" where I can > store (in a table) the > name of the sp, the parameters in a child table etc. > Using table-driven process could be not quite right in C#/.Net world, which > proposes many other options to implement generic solutions - I'd just start > with "hardcoded" solutions using calls to a set of small "generic" > classes/methods executing custom (MS SQL) database processing, when that set > of methods/functions stabilizes I'd try to generalize/"generic-ize" it if > that will be needed at all... > > What is the purpose of your "going generic" for this sets of tasks/customer? > > Isn't it "just for fun" of generic programming? > Would that "going generic" make your programming more effective? > > Developing/debugging/testing generic custom "table-driven"/"other-metadata > media driven" solutions is always (much) more time consuming than > "hardcoded" ones, and supporting "table-driven" generic custom solutions, > especially in the context of constantly changing customer requirements also > promise to be rather time consuming - I mean generic > "table-"/metadata-driven solutions could only(?) pay back well in the case > they are used on many deployment sites by many people. I can be wrong, just > talking from my experience, which is not universal of course... > > I'd better propose to use metadata descriptions/specs (stored in db tables > or (XML) files), if you're so bored to write repetitive code, to generate > "hardcoded" solutions, then make quick fixes to this generated hardcode to > satisfy your customers' current requests, then (in free time) adjust code > generator, then when everything will stabilize (will that ever happen?) > develop truly generic "table-driven" solution. > > You can also consider using F# and Windows Workflow Foundation (WF)... > > The following presentation seems to be useful to get true picture of > "generic vs. specific" solutions trade-offs: > > http://www.infoq.com/presentations/Generic-Specific-Tradeoffs-Stefan-Tilkov > > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Thu Nov 5 11:13:22 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 20:13:22 +0300 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <4AF2FE62.2050501@colbyconsulting.com> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> <4AF2FE62.2050501@colbyconsulting.com> Message-ID: <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> Hi John, If your "Process X", "Process Y",... calls to stored procedures do have constant parameters' values then I do not see (I can be missing them) clear reasons to use table-driven approach - I'd just have hardcoded ProcessX, ProcessY, ... calling C# wrapper classes' methods to execute SPs. And the latter wrapper classes can inherit from some generic base class(es) to keep executing context, collect statistics etc... As I noted previously WF (Windows Workflow Foundation) could be a good candidate for your subject area but WF has a steep learning curve so you will not be able to effectively apply it right now but with small dedicated classes, which you're writing right now (and delivering your custom solutions right now) you'll be able to relatively easy migrate to WF based solutions in the future. And if you will spend time working on generic custom table-driven solution you'll probably have to throw it away later when migrating to WF... Before you get comfortable with WF you can code you ProcessX, ProcessY, ... steps as "meta-process" text or xml files keeping fully qualified class names for your dedicated classes. And your "generic execution engine" will be just a simple code reading "meta-process" description, instantiating custom classes using .NET Reflection, calling their generic context/initialization methods, running SP by calling generic Run() method (parameters setting is inside your custom classes), then calling generic termination methods and iterating to the next step/finishing when "meta-process" file description ends. That could be it. Thank you. --Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 7:34 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] How do you run SQL Server processes Shamil, > Using table-driven process could be not quite right in C#/.Net world, which proposes many other options to implement generic solutions - I'd just start with "hardcoded" solutions using calls to a set of small "generic" That is precisely what I do right now, and what I will do for the near future. The table driven thing is simply that I write "processes" as discussed in this thread. A process might be: Export To Accuzip: "Export dbX TblY to files in DirectoryZ". However that breaks down onto a set of a handful of stored procedures, with passed in parameters. Now, the process "Export To Accuzip" is applied against 6 or 8 different databases every month, but ALSO against random databases which are orders. Orders come in every month and they are a whole nother discussion, but PART of an order is the process Export to Accuzip. DatabaseA exports files to its own directory, DatabaseB to its own directory, OrderAA to its own directory and so forth. But the EXPORT itself is a very standard process. So I am examining the concept of storing the process in a table, so that I can simply select (from a form: Process X: Use SP1, params use SP4, Params use SP9, Params Process Y: Use SP1, Params Use SP2, Params Use SP5, Params Process Z: Use... Table driven (to the extent possible). Then a supervisor application could execute a process (set of SPs), read out of a table, passing in only the database name and possibly a directory on disk. Everything else is selected and set up in advance. To see what a process is, just open a table, to modify the process, select different stored procedures. This stuff is VERY standard, with just a few things that change from DB to db (the database name and where files are going) however I do occasionally shufffle around the interior processes (indexes created, fields used etc. That stuff is internal to the SP and if I change that it is in the SP itself. Then that change is applied to every database from then on. I am not saying that this will work but as my understand and knowledge changes I am spending a LOT of time rewriting stuff. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > <<< > I am hoping to eventually make the entire process "table driven" where I can > store (in a table) the > name of the sp, the parameters in a child table etc. > Using table-driven process could be not quite right in C#/.Net world, which > proposes many other options to implement generic solutions - I'd just start > with "hardcoded" solutions using calls to a set of small "generic" > classes/methods executing custom (MS SQL) database processing, when that set > of methods/functions stabilizes I'd try to generalize/"generic-ize" it if > that will be needed at all... > > What is the purpose of your "going generic" for this sets of tasks/customer? > > Isn't it "just for fun" of generic programming? > Would that "going generic" make your programming more effective? > > Developing/debugging/testing generic custom "table-driven"/"other-metadata > media driven" solutions is always (much) more time consuming than > "hardcoded" ones, and supporting "table-driven" generic custom solutions, > especially in the context of constantly changing customer requirements also > promise to be rather time consuming - I mean generic > "table-"/metadata-driven solutions could only(?) pay back well in the case > they are used on many deployment sites by many people. I can be wrong, just > talking from my experience, which is not universal of course... > > I'd better propose to use metadata descriptions/specs (stored in db tables > or (XML) files), if you're so bored to write repetitive code, to generate > "hardcoded" solutions, then make quick fixes to this generated hardcode to > satisfy your customers' current requests, then (in free time) adjust code > generator, then when everything will stabilize (will that ever happen?) > develop truly generic "table-driven" solution. > > You can also consider using F# and Windows Workflow Foundation (WF)... > > The following presentation seems to be useful to get true picture of > "generic vs. specific" solutions trade-offs: > > http://www.infoq.com/presentations/Generic-Specific-Tradeoffs-Stefan-Tilkov > > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4576 (20091105) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Thu Nov 5 11:32:51 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 05 Nov 2009 12:32:51 -0500 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> <4AF2FE62.2050501@colbyconsulting.com> <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> Message-ID: <4AF30C43.8010508@colbyconsulting.com> Shamil, Listen to this... > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... steps as "meta-process" text or xml files keeping fully qualified class names for your dedicated classes. And your "generic execution engine" will be just a simple code reading "meta-process" description, instantiating custom classes using .NET Reflection, calling their generic context/initialization methods, running SP by calling generic Run() method (parameters setting is inside your custom classes), then calling generic termination methods and iterating to the next step/finishing when "meta-process" file description ends. Now read this. Open form Select stored procedure, select parameter, select parameter. click save. Select stored procedure, select parameter, select parameter. click save. Continue till done. ;) Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? Read your solution again. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > If your "Process X", "Process Y",... calls to stored procedures do have > constant parameters' values then I do not see (I can be missing them) clear > reasons to use table-driven approach - I'd just have hardcoded ProcessX, > ProcessY, ... calling C# wrapper classes' methods to execute SPs. And the > latter wrapper classes can inherit from some generic base class(es) to keep > executing context, collect statistics etc... > > As I noted previously WF (Windows Workflow Foundation) could be a good > candidate for your subject area but WF has a steep learning curve so you > will not be able to effectively apply it right now but with small dedicated > classes, which you're writing right now (and delivering your custom > solutions right now) you'll be able to relatively easy migrate to WF based > solutions in the future. And if you will spend time working on generic > custom table-driven solution you'll probably have to throw it away later > when migrating to WF... > > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... > steps as "meta-process" text or xml files keeping fully qualified class > names for your dedicated classes. And your "generic execution engine" will > be just a simple code reading "meta-process" description, instantiating > custom classes using .NET Reflection, calling their generic > context/initialization methods, running SP by calling generic Run() method > (parameters setting is inside your custom classes), then calling generic > termination methods and iterating to the next step/finishing when > "meta-process" file description ends. That could be it. > > Thank you. > > --Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 7:34 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] How do you run SQL Server processes > > Shamil, From shamil at smsconsulting.spb.ru Thu Nov 5 13:00:15 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 22:00:15 +0300 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <4AF30C43.8010508@colbyconsulting.com> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> <4AF2FE62.2050501@colbyconsulting.com> <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> <4AF30C43.8010508@colbyconsulting.com> Message-ID: <010c01ca5e4a$3735e430$a5a1ac90$@spb.ru> Hi John, <<< Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? >>> Yes, that is also what I mean: developing generic table-driven procedure for you currently could force you to learn more C#/.NET development stuff than when implementing "meta-process description" interpreter, which will consist of several C# code lines. But even the latter is not needed probably as you can hardcode your solutions for every custom task "shuffling" your parameterized wrappers classes/methods calling SPs, and instead of "Select stored procedure, select parameter,..." use "Create C# project, type C# code with calls to the wrapper classes/methods with Stored Procedures names and parameter names, types and values, then compile and run the custom solution". And it should be more suitable to debug/trace latter solution than "table-driven" or "meta-process description driven" ones... OK? (Am I still missing something here?) -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 8:33 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] How do you run SQL Server processes Shamil, Listen to this... > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... steps as "meta-process" text or xml files keeping fully qualified class names for your dedicated classes. And your "generic execution engine" will be just a simple code reading "meta-process" description, instantiating custom classes using .NET Reflection, calling their generic context/initialization methods, running SP by calling generic Run() method (parameters setting is inside your custom classes), then calling generic termination methods and iterating to the next step/finishing when "meta-process" file description ends. Now read this. Open form Select stored procedure, select parameter, select parameter. click save. Select stored procedure, select parameter, select parameter. click save. Continue till done. ;) Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? Read your solution again. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > If your "Process X", "Process Y",... calls to stored procedures do have > constant parameters' values then I do not see (I can be missing them) clear > reasons to use table-driven approach - I'd just have hardcoded ProcessX, > ProcessY, ... calling C# wrapper classes' methods to execute SPs. And the > latter wrapper classes can inherit from some generic base class(es) to keep > executing context, collect statistics etc... > > As I noted previously WF (Windows Workflow Foundation) could be a good > candidate for your subject area but WF has a steep learning curve so you > will not be able to effectively apply it right now but with small dedicated > classes, which you're writing right now (and delivering your custom > solutions right now) you'll be able to relatively easy migrate to WF based > solutions in the future. And if you will spend time working on generic > custom table-driven solution you'll probably have to throw it away later > when migrating to WF... > > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... > steps as "meta-process" text or xml files keeping fully qualified class > names for your dedicated classes. And your "generic execution engine" will > be just a simple code reading "meta-process" description, instantiating > custom classes using .NET Reflection, calling their generic > context/initialization methods, running SP by calling generic Run() method > (parameters setting is inside your custom classes), then calling generic > termination methods and iterating to the next step/finishing when > "meta-process" file description ends. That could be it. > > Thank you. > > --Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 7:34 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] How do you run SQL Server processes > > Shamil, _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4576 (20091105) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4576 (20091105) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From shamil at smsconsulting.spb.ru Sat Nov 7 05:28:32 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 7 Nov 2009 14:28:32 +0300 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <010c01ca5e4a$3735e430$a5a1ac90$@spb.ru> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> <4AF2FE62.2050501@colbyconsulting.com> <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> <4AF30C43.8010508@colbyconsulting.com> <010c01ca5e4a$3735e430$a5a1ac90$@spb.ru> Message-ID: <018c01ca5f9d$71389c00$53a9d400$@spb.ru> Hi John, Here is a .NET Reflection sample: using System; namespace NetReflectionSample { class Program { static void Main(string[] args) { string[] spRunnersNames = { "NetReflectionSample.SP1", "NetReflectionSample.SP2" }; foreach (string spRunnerName in spRunnersNames) { Type spRunnerType = Type.GetType(spRunnerName); StoredProcedureRunner runner = Activator.CreateInstance(spRunnerType) as StoredProcedureRunner; runner.Init(); runner.Run(); runner.CollectStats(); Console.WriteLine(); } } } public abstract class StoredProcedureRunner { public void Init() { Console.WriteLine("Initializing {0}...", this.GetType().ToString()); } public abstract void Run(); public void CollectStats() { Console.WriteLine("Collecting stats {0}...", this.GetType().ToString()); } } public class SP1 : StoredProcedureRunner { public override void Run() { Console.WriteLine("Running {0}...", this.GetType().ToString()); } } public class SP2 : StoredProcedureRunner { public override void Run() { Console.WriteLine("Running {0}...", this.GetType().ToString()); } } } Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: Thursday, November 05, 2009 10:00 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] How do you run SQL Server processes Hi John, <<< Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? >>> Yes, that is also what I mean: developing generic table-driven procedure for you currently could force you to learn more C#/.NET development stuff than when implementing "meta-process description" interpreter, which will consist of several C# code lines. But even the latter is not needed probably as you can hardcode your solutions for every custom task "shuffling" your parameterized wrappers classes/methods calling SPs, and instead of "Select stored procedure, select parameter,..." use "Create C# project, type C# code with calls to the wrapper classes/methods with Stored Procedures names and parameter names, types and values, then compile and run the custom solution". And it should be more suitable to debug/trace latter solution than "table-driven" or "meta-process description driven" ones... OK? (Am I still missing something here?) -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 8:33 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] How do you run SQL Server processes Shamil, Listen to this... > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... steps as "meta-process" text or xml files keeping fully qualified class names for your dedicated classes. And your "generic execution engine" will be just a simple code reading "meta-process" description, instantiating custom classes using .NET Reflection, calling their generic context/initialization methods, running SP by calling generic Run() method (parameters setting is inside your custom classes), then calling generic termination methods and iterating to the next step/finishing when "meta-process" file description ends. Now read this. Open form Select stored procedure, select parameter, select parameter. click save. Select stored procedure, select parameter, select parameter. click save. Continue till done. ;) Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? Read your solution again. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > If your "Process X", "Process Y",... calls to stored procedures do have > constant parameters' values then I do not see (I can be missing them) clear > reasons to use table-driven approach - I'd just have hardcoded ProcessX, > ProcessY, ... calling C# wrapper classes' methods to execute SPs. And the > latter wrapper classes can inherit from some generic base class(es) to keep > executing context, collect statistics etc... > > As I noted previously WF (Windows Workflow Foundation) could be a good > candidate for your subject area but WF has a steep learning curve so you > will not be able to effectively apply it right now but with small dedicated > classes, which you're writing right now (and delivering your custom > solutions right now) you'll be able to relatively easy migrate to WF based > solutions in the future. And if you will spend time working on generic > custom table-driven solution you'll probably have to throw it away later > when migrating to WF... > > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... > steps as "meta-process" text or xml files keeping fully qualified class > names for your dedicated classes. And your "generic execution engine" will > be just a simple code reading "meta-process" description, instantiating > custom classes using .NET Reflection, calling their generic > context/initialization methods, running SP by calling generic Run() method > (parameters setting is inside your custom classes), then calling generic > termination methods and iterating to the next step/finishing when > "meta-process" file description ends. That could be it. > > Thank you. > > --Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 7:34 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] How do you run SQL Server processes > > Shamil, From shamil at smsconsulting.spb.ru Sat Nov 7 08:19:11 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 7 Nov 2009 17:19:11 +0300 Subject: [dba-VB] NetReflectionSample - Part I of IV Message-ID: <000601ca5fb5$49b79dd0$dd26d970$@spb.ru> Hi John and all, Just wanted to post a bit more advanced sample on .NET reflection and how it can be used to instantiate custom classes and to execute their parameterized methods based on text definition: using System; using System.Reflection; #region Test Program namespace NetReflectionSample { /// /// Test class /// class Program { static void Main(string[] args) { // List of custom classes (with parameters) to instantiate and // call their Run(...) method string[] spRunnersCalls = { "NetReflectionSample.SP1", "NetReflectionSample.SP2|503|Test String|3.14|11/7/2009", "NetReflectionSample.SP3|123|Test String2" }; foreach (string spRunnerCall in spRunnersCalls) { // Parse current SP call string definition and // return StoredProcedureRunner instance StoredProcedureRunner runner = DynamicMethodCallParser.Parse(spRunnerCall); runner.Init(); // If there are calling parameters use CustomRunMethodInfo to // execute SP call wrapper method, else use default Run() method CustomRunMethodInfo customRunner = runner as CustomRunMethodInfo; if (customRunner.Parameters != null) { customRunner.ParameterizedRunMethodInfo.Invoke( customRunner, customRunner.Parameters); } else runner.Run(); // collect execution stats runner.CollectStats(); Console.WriteLine(); } } } } #endregion Thank you. --Shamil From shamil at smsconsulting.spb.ru Sat Nov 7 08:20:44 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 7 Nov 2009 17:20:44 +0300 Subject: [dba-VB] NetReflectionSample - Part II of IV Message-ID: <000b01ca5fb5$7f9438a0$7ebca9e0$@spb.ru> #region Custom business functionality namespace NetReflectionSample { /// /// SP1 wrapper class simulator /// public class SP1 : StoredProcedureRunner { public override void Run() { Console.WriteLine("Running {0}.Run...", this.GetType().ToString()); } } /// /// SP2 wrapper class simulator /// public class SP2 : StoredProcedureRunner { public void Run( int prm1, string prm2, decimal prm3, DateTime prm4) { Console.WriteLine("Running {0}.Run...", this.GetType().ToString()); Console.WriteLine("Parameters: prm1={0}, prm2={1}, prm3={2}, prm4={3}", prm1, prm2, prm3, prm4); } } /// /// SP3 wrapper class simulator /// public class SP3 : StoredProcedureRunner { public void Run( int prm1, string prm2) { Console.WriteLine("Running {0}.Run...", this.GetType().ToString()); Console.WriteLine("Parameters: prm1={0}, prm2={1}", prm1, prm2); } } } #endregion --Shamil From shamil at smsconsulting.spb.ru Sat Nov 7 08:37:01 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 7 Nov 2009 17:37:01 +0300 Subject: [dba-VB] NetReflectionSample - full code text Message-ID: <001a01ca5fb7$c5cab310$51601930$@spb.ru> Hi All, Here is a full text of the subject sample: http://smsconsulting.spb.ru/samples/NetReflectionSample.txt dba-VB robot currently blocks to post two parts of my four parts posting because they are longer than 20KB - dba-VB robot writes that those two postings are awaiting moderators approval but AFAIHF this approval never happens (at least that was for several my past postings with code) - therefore I'm posting the above link to get the whole code text available. Here are the results of the test program execution: === Initializing NetReflectionSample.SP1... Running NetReflectionSample.SP1.Run... Collecting stats NetReflectionSample.SP1... Initializing NetReflectionSample.SP2... Running NetReflectionSample.SP2.Run... Parameters: prm1=503, prm2=Test String, prm3=3.14, prm4=11/7/2009 12:00:00 AM Collecting stats NetReflectionSample.SP2... Initializing NetReflectionSample.SP3... Running NetReflectionSample.SP3.Run... Parameters: prm1=123, prm2=Test String2 Collecting stats NetReflectionSample.SP3... === Note: posted code is a "quick & dirty" first version, if you have time and desire to make a better version of this sample code you're welcome to do that. Any code review remarks and comments are very welcome. Thank you. -- Shamil From Gustav at cactus.dk Sat Nov 7 14:00:22 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Sat, 07 Nov 2009 21:00:22 +0100 Subject: [dba-VB] NetReflectionSample - full code text Message-ID: Hi Shamil Thanks! Good example. /gustav >>> shamil at smsconsulting.spb.ru 07-11-2009 15:37 >>> Any code review remarks and comments are very welcome. From shamil at smsconsulting.spb.ru Sun Nov 8 15:47:44 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 9 Nov 2009 00:47:44 +0300 Subject: [dba-VB] NetReflectionSample - full code text In-Reply-To: References: Message-ID: <000301ca60bd$1c0d9ca0$5428d5e0$@spb.ru> Thank you, Gustav. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Saturday, November 07, 2009 11:00 PM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] NetReflectionSample - full code text Hi Shamil Thanks! Good example. /gustav >>> shamil at smsconsulting.spb.ru 07-11-2009 15:37 >>> Any code review remarks and comments are very welcome. _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4585 (20091108) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sun Nov 8 20:20:20 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 08 Nov 2009 21:20:20 -0500 Subject: [dba-VB] NetReflectionSample - full code text In-Reply-To: References: Message-ID: <4AF77C64.5090707@colbyconsulting.com> But what does it DO? John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi Shamil > > Thanks! Good example. > > /gustav > >>>> shamil at smsconsulting.spb.ru 07-11-2009 15:37 >>> > > Any code review remarks and comments are very welcome. > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Mon Nov 9 00:11:28 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 9 Nov 2009 09:11:28 +0300 Subject: [dba-VB] NetReflectionSample - full code text In-Reply-To: <4AF77C64.5090707@colbyconsulting.com> References: <4AF77C64.5090707@colbyconsulting.com> Message-ID: <000901ca6103$7adfbd80$709f3880$@spb.ru> Hi John, Basically, this sample code demonstrates how to interpret a simple scripting "birds language" into a sequence of calls to the parameterized methods of custom classes. Substitute text script with meta-table(s) - and you'll get table-driven interpreter. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, November 09, 2009 5:20 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] NetReflectionSample - full code text But what does it DO? John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi Shamil > > Thanks! Good example. > > /gustav > >>>> shamil at smsconsulting.spb.ru 07-11-2009 15:37 >>> > > Any code review remarks and comments are very welcome. > > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4586 (20091108) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From shamil at smsconsulting.spb.ru Tue Nov 10 23:49:05 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Wed, 11 Nov 2009 08:49:05 +0300 Subject: [dba-VB] FYI: ORM, EDM, ESQL, Entity Framework, LINQ to SQL, LINQ to Entities - Confused? Message-ID: <003601ca6292$af1581d0$0d408570$@spb.ru> "ORM, EDM, ESQL, Entity Framework, LINQ to SQL, LINQ to Entities - Confused?" http://www.infoq.com/presentations/ORM-LINQ-Entity-Framework-Eric-Nelson --Shamil From Gustav at cactus.dk Wed Nov 11 12:52:48 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 11 Nov 2009 19:52:48 +0100 Subject: [dba-VB] Microsoft WebsiteSpark Program Message-ID: Hi all Have you heard of Microsoft? WebsiteSpark*? http://www.microsoft.com/web/websitespark/Support.aspx?tab=ProgramDetails It is designed to ignite the success of professional Web developers and designers. The program enables you to get software, support and business opportunities from Microsoft at no upfront costs for 3 years or exit from the program. WebSiteSpark provides access to a technology offering for development and production hosting of websites. Once enrolled, you can download the following software from Microsoft: For design, development, testing and demonstration of new websites * for a total of up to three users per Web design and development company: *Visual Studio Professional *Expression Studio 3 (1 user) and Expression Web 3 (up to 2 users) *Windows Web Server 2008 R2 *SQL Server 2008 Web Edition For production use * that is, to deploy and host new websites developed using Program software * using a total of up to four processors per Web design and development company, of the following (physical or virtual) dedicated servers: *Windows Web Server 2008 R2 *SQL Server 2008 Web Edition So if you consider "something else" than Access, this might help you. /gustav From jwcolby at colbyconsulting.com Wed Nov 11 13:08:24 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 11 Nov 2009 14:08:24 -0500 Subject: [dba-VB] SPAM-LOW: Microsoft WebsiteSpark Program In-Reply-To: References: Message-ID: <4AFB0BA8.8020105@colbyconsulting.com> This appears to be one part of this: http://www.microsoftstartupzone.com/pages/home.aspx which I posted about a month or so ago. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi all > > Have you heard of Microsoft? WebsiteSpark*? > > http://www.microsoft.com/web/websitespark/Support.aspx?tab=ProgramDetails > > > > It is designed to ignite the success of professional Web developers and > designers. The program enables you to get software, support and business > opportunities from Microsoft at no upfront costs for 3 years or exit > from the program. > > WebSiteSpark provides access to a technology offering for development > and production hosting of websites. Once enrolled, you can download the > following software from Microsoft: > > For design, development, testing and demonstration of new websites * > for a total of up to three users per Web design and development company: > > > *Visual Studio Professional > *Expression Studio 3 (1 user) and Expression Web 3 (up to 2 users) > *Windows Web Server 2008 R2 > *SQL Server 2008 Web Edition > > For production use * that is, to deploy and host new websites > developed using Program software * using a total of up to four > processors per Web design and development company, of the following > (physical or virtual) dedicated servers: > > *Windows Web Server 2008 R2 > *SQL Server 2008 Web Edition > > > > So if you consider "something else" than Access, this might help you. > > /gustav > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Thu Nov 12 11:22:55 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 12 Nov 2009 12:22:55 -0500 Subject: [dba-VB] What to do, what to do? Message-ID: <4AFC446F.6070207@colbyconsulting.com> I am writing an application in C# to merge records (previously discussed). It looks pretty simple, all things considered. Get data into a dataset object which will contain about 4 tables. Then use the records and fields collections of the recordset to iterate through the fields building up a record (or modifying the first record of a small set of records - the dupes). Copy a record from one table into another table inside of the dataset. Delete records in a table in the dataset. Etc. Etc all without going back out to the actual data tables on SQL Server until the very end when I will do an update back to the live data. Am I correct in assuming here that I can programmatically append records from one table in the dataset to another record in the dataset? Delete records in the tables in the dataset? Update records in tables in the dataset? All "in memory" without connecting back to the live data in the database? Is there anything I need to know before I launch into this? -- John W. Colby www.ColbyConsulting.com From cfoust at infostatsystems.com Thu Nov 12 11:29:43 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Thu, 12 Nov 2009 09:29:43 -0800 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFC446F.6070207@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> Message-ID: I think that depends on the objects you're using. .Net tends to complain that a record already belongs to another table when you try to import it. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 12, 2009 9:23 AM To: VBA Subject: [dba-VB] What to do, what to do? I am writing an application in C# to merge records (previously discussed). It looks pretty simple, all things considered. Get data into a dataset object which will contain about 4 tables. Then use the records and fields collections of the recordset to iterate through the fields building up a record (or modifying the first record of a small set of records - the dupes). Copy a record from one table into another table inside of the dataset. Delete records in a table in the dataset. Etc. Etc all without going back out to the actual data tables on SQL Server until the very end when I will do an update back to the live data. Am I correct in assuming here that I can programmatically append records from one table in the dataset to another record in the dataset? Delete records in the tables in the dataset? Update records in tables in the dataset? All "in memory" without connecting back to the live data in the database? Is there anything I need to know before I launch into this? -- John W. Colby www.ColbyConsulting.com _______________________________________________ From shamil at smsconsulting.spb.ru Fri Nov 13 00:04:33 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 09:04:33 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFC446F.6070207@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> Message-ID: <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> Hi John -- Here is a sample: using System; using System.Collections.Generic; using System.Text; using System.Data; using System.Data.SqlClient; namespace TestConsole.Samples { public class DataSetTests { public static void Run() { string connectionString = "Data Source=HAMBURG\\SQL2005;"+ "Initial Catalog=Northwind;User Id=sa;Password=N/A"; using (SqlConnection connection = new SqlConnection(connectionString)) { connection.Open(); string sql = "select * from [Shippers]"; string sql1 = "select * from [NewShippers] where(1=0)"; SqlCommand command = new SqlCommand(sql, connection); SqlDataAdapter adapter = new SqlDataAdapter(command); DataSet dataSet = new DataSet("Shippers"); adapter.Fill(dataSet); command = new SqlCommand(sql1, connection); adapter = new SqlDataAdapter(command); DataSet newDataSet = new DataSet("NewShippers"); adapter.Fill(newDataSet); Console.WriteLine("*** newDataSet ***"); foreach (DataRow dataRow in newDataSet.Tables[0].Rows) { Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); } Console.WriteLine("*** dataSet ***"); foreach (DataRow dataRow in dataSet.Tables[0].Rows) { // This row already belongs to another table. //newDataSet.Tables[0].Rows.Add(dataRow); object[] items = { dataRow[0], dataRow[1], dataRow[2] }; newDataSet.Tables[0].Rows.Add(items); Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); } Console.WriteLine("*** newDataSet ***"); foreach (DataRow dataRow in newDataSet.Tables[0].Rows) { Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); } newDataSet.Tables[0].TableName = "NewShippers"; SqlCommandBuilder builder = new SqlCommandBuilder(adapter); adapter.InsertCommand = builder.GetInsertCommand(true); adapter.Update(newDataSet.Tables[0]); } } } } Thank you. --Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 12, 2009 8:23 PM To: VBA Subject: [dba-VB] What to do, what to do? I am writing an application in C# to merge records (previously discussed). It looks pretty simple, all things considered. Get data into a dataset object which will contain about 4 tables. Then use the records and fields collections of the recordset to iterate through the fields building up a record (or modifying the first record of a small set of records - the dupes). Copy a record from one table into another table inside of the dataset. Delete records in a table in the dataset. Etc. Etc all without going back out to the actual data tables on SQL Server until the very end when I will do an update back to the live data. Am I correct in assuming here that I can programmatically append records from one table in the dataset to another record in the dataset? Delete records in the tables in the dataset? Update records in tables in the dataset? All "in memory" without connecting back to the live data in the database? Is there anything I need to know before I launch into this? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4600 (20091112) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From marklbreen at gmail.com Fri Nov 13 00:58:16 2009 From: marklbreen at gmail.com (Mark Breen) Date: Fri, 13 Nov 2009 14:58:16 +0800 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFC446F.6070207@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> Message-ID: Hello John, Just curious, how many records are you talking about - a few hundred or millions? Second question, other than for purely academic reasons, why are you bring the data into C#, to concatenate and insert into another table can so easily be done in one sproc, even if you need a temporary holding place, you would still use a temp table or even a local table. Of course if it is to learn datasets then thats another story, Mark 2009/11/13 jwcolby > I am writing an application in C# to merge records (previously discussed). > It looks pretty simple, > all things considered. Get data into a dataset object which will contain > about 4 tables. Then use > the records and fields collections of the recordset to iterate through the > fields building up a > record (or modifying the first record of a small set of records - the > dupes). Copy a record from > one table into another table inside of the dataset. Delete records in a > table in the dataset. Etc. > Etc all without going back out to the actual data tables on SQL Server > until the very end when I > will do an update back to the live data. > > Am I correct in assuming here that I can programmatically append records > from one table in the > dataset to another record in the dataset? Delete records in the tables in > the dataset? Update > records in tables in the dataset? All "in memory" without connecting back > to the live data in the > database? > > Is there anything I need to know before I launch into this? > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Fri Nov 13 03:27:08 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Fri, 13 Nov 2009 10:27:08 +0100 Subject: [dba-VB] What to do, what to do? Message-ID: Hi John Shamil knows the SqlClient stuff! As he knows, I prefer the datatable route - a matter of taste I think, but I admit working with SqlCommand may allow for some fine tuning not possible with the datatableadapters. Here's an example from an app that updates (syncs) one table from another. Not exactly what you wish to do but it should contain the building blocks. The tricky part here is that the unique id (autonumber) should not be compared. However, the id of a source row is stored in the target row as SyncId for later reference: private void ConvertOrder() { // DataTableAdapter to copy from. VrsCloneTableAdapters.OrderTableAdapter sourceDataTableAdapter = new VrsCloneTableAdapters.OrderTableAdapter(); VrsClone.OrderDataTable sourceDataTable; // DataTableAdapter to copy to. KarneliaTableAdapters.SyncOrderTableAdapter targetDataTableAdapter = new KarneliaTableAdapters.SyncOrderTableAdapter(); Karnelia.SyncOrderDataTable targetDataTable; sourceDataTableAdapter.Connection = ConnectionApplyPassword(sourceDataTableAdapter.Connection.ConnectionString); targetDataTableAdapter.Connection = ConnectionApplyPassword(targetDataTableAdapter.Connection.ConnectionString); // DataTable for source. sourceDataTable = sourceDataTableAdapter.GetDataSync(); // DataTable for target. targetDataTable = targetDataTableAdapter.GetData(); // Read each row from the source rows. // Update those found in the target table. // Append those not found in the target table. // Variable to hold primary key to locate. int key; // Row of target DataTable to be updated. Karnelia.SyncOrderRow targetRow = null; // Variables to hold hash codes of row contents. int sourceHash; int targetHash; int colCount = sourceDataTable.Columns.Count; int colItem; int colOrdinal; string colName; for (colItem = 0; colItem < colCount; colItem++) { colName = sourceDataTable.Columns[colItem].ColumnName; colOrdinal = sourceDataTable.Columns[colItem].Ordinal; targetDataTable.Columns[colName].SetOrdinal(colOrdinal + 1); } foreach (VrsClone.OrderRow sourceRow in sourceDataTable) { // Value of key to locate. key = sourceRow.SyncId; // Find a matching row in the target. targetRow = targetDataTable.FindBySyncId(key); if (targetRow == null) { // Row is not found. targetRow = targetDataTable.NewSyncOrderRow(); targetRow.BeginEdit(); targetRow.VrsOrderId = sourceRow.VrsOrderId; targetRow.ClientId = sourceRow.ClientId; targetRow.StylistId = sourceRow.StylistId; targetRow.DateCreated = sourceRow.DateCreated; targetRow.SyncId = sourceRow.SyncId; targetRow.EndEdit(); targetDataTable.AddSyncOrderRow(targetRow); Console.WriteLine("Insert " + targetDataTable.ToString() + ": " + sourceRow.VrsOrderId.ToString() + " - " + targetRow.RowState.ToString()); } else { // Row is found. // Calculate hash code to see if source row and target row match. // Id of targetrow (first column) is not present in sourcerow, thus // concatenate first column of targetrow with the columns of sourcerow. sourceHash = String.Concat(targetRow.ItemArray[0], String.Concat(sourceRow.ItemArray)).GetHashCode(); targetHash = String.Concat(targetRow.ItemArray).GetHashCode(); if (sourceHash != targetHash) { // The source row is updated. // Copy full content from source row to target row. // This sets RowState of the target row to Modified. targetRow.BeginEdit(); targetRow.VrsOrderId = sourceRow.VrsOrderId; targetRow.ClientId = sourceRow.ClientId; targetRow.StylistId = sourceRow.StylistId; targetRow.DateCreated = sourceRow.DateCreated; targetRow.EndEdit(); // Do NOT call AcceptChanges here as this would clear property RowState. Console.WriteLine("Update " + targetDataTable.ToString() + ": " + sourceRow.VrsOrderId.ToString() + " - " + targetRow.RowState.ToString()); } } } // Write back to the database table the updated target DataTable. targetDataTableAdapter.Update(targetDataTable); // Call AcceptChanges if needed for further processing. //targetDataTable.AcceptChanges(); } This database contains records by the thousands. How code like this will perform for millions of records has to be tested. /gustav >>> jwcolby at colbyconsulting.com 12-11-2009 18:22 >>> I am writing an application in C# to merge records (previously discussed). It looks pretty simple, all things considered. Get data into a dataset object which will contain about 4 tables. Then use the records and fields collections of the recordset to iterate through the fields building up a record (or modifying the first record of a small set of records - the dupes). Copy a record from one table into another table inside of the dataset. Delete records in a table in the dataset. Etc. Etc all without going back out to the actual data tables on SQL Server until the very end when I will do an update back to the live data. Am I correct in assuming here that I can programmatically append records from one table in the dataset to another record in the dataset? Delete records in the tables in the dataset? Update records in tables in the dataset? All "in memory" without connecting back to the live data in the database? Is there anything I need to know before I launch into this? -- John W. Colby www.ColbyConsulting.com From Gustav at cactus.dk Fri Nov 13 04:48:24 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Fri, 13 Nov 2009 11:48:24 +0100 Subject: [dba-VB] AWS (Amazon Web Services) SDK for .NET and VS Message-ID: Hi all This has just been announced for Visual Studio 2008: The AWS SDK for .NET makes it even easier for Windows developers to build .NET applications that tap into the cost-effective, scalable, and reliable AWS cloud. Using the SDK, developers will be able to build solutions for AWS infrastructure services, including Amazon Simple Storage Service (Amazon S3), Amazon Elastic Compute Cloud (Amazon EC2), and Amazon SimpleDB. With the AWS SDK for .NET, developers get started in minutes with a single, downloadable package complete with Visual Studio project templates, the AWS .NET library, C# code samples, and documentation. http://aws.amazon.com/sdkfornet/ /gustav From jwcolby at colbyconsulting.com Fri Nov 13 06:51:36 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 07:51:36 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> Message-ID: <4AFD5658.7000703@colbyconsulting.com> Shamil, That is precisely what I have in mind, with variations. Then at the end, write all updates back to SQL Server. Thanks for the demo code, always helpful. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Here is a sample: > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class DataSetTests > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;"+ > "Initial Catalog=Northwind;User Id=sa;Password=N/A"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { > connection.Open(); > string sql = "select * from [Shippers]"; > string sql1 = "select * from [NewShippers] where(1=0)"; > SqlCommand command = new SqlCommand(sql, connection); > SqlDataAdapter adapter = new SqlDataAdapter(command); > > DataSet dataSet = new DataSet("Shippers"); > adapter.Fill(dataSet); > > command = new SqlCommand(sql1, connection); > adapter = new SqlDataAdapter(command); > DataSet newDataSet = new DataSet("NewShippers"); > adapter.Fill(newDataSet); > > Console.WriteLine("*** newDataSet ***"); > foreach (DataRow dataRow in newDataSet.Tables[0].Rows) > { > Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); > } > > Console.WriteLine("*** dataSet ***"); > foreach (DataRow dataRow in dataSet.Tables[0].Rows) > { > // This row already belongs to another table. > //newDataSet.Tables[0].Rows.Add(dataRow); > > object[] items = { dataRow[0], dataRow[1], dataRow[2] }; > newDataSet.Tables[0].Rows.Add(items); > > Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); > } > > Console.WriteLine("*** newDataSet ***"); > foreach (DataRow dataRow in newDataSet.Tables[0].Rows) > { > Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); > } > > newDataSet.Tables[0].TableName = "NewShippers"; > SqlCommandBuilder builder = new SqlCommandBuilder(adapter); > adapter.InsertCommand = builder.GetInsertCommand(true); > adapter.Update(newDataSet.Tables[0]); > } > } > > } > } > > > Thank you. > > --Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 12, 2009 8:23 PM > To: VBA > Subject: [dba-VB] What to do, what to do? > > I am writing an application in C# to merge records (previously discussed). > It looks pretty simple, > all things considered. Get data into a dataset object which will contain > about 4 tables. Then use > the records and fields collections of the recordset to iterate through the > fields building up a > record (or modifying the first record of a small set of records - the > dupes). Copy a record from > one table into another table inside of the dataset. Delete records in a > table in the dataset. Etc. > Etc all without going back out to the actual data tables on SQL Server until > the very end when I > will do an update back to the live data. > > Am I correct in assuming here that I can programmatically append records > from one table in the > dataset to another record in the dataset? Delete records in the tables in > the dataset? Update > records in tables in the dataset? All "in memory" without connecting back > to the live data in the > database? > > Is there anything I need to know before I launch into this? > From shamil at smsconsulting.spb.ru Fri Nov 13 08:19:39 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 17:19:39 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD5658.7000703@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> Message-ID: <004801ca646c$5758e690$060ab3b0$@spb.ru> Hi John -- You can also consider using SqlBulkLoad - below is just a simple sample, and you can develop very flexible variations of this approach. SqlBulkLoad should be as quick as bcp maybe even quicker. In fact the source for SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this IDataReader to have all the data to bulkload then your programs will become as streamlined as they probably have never been, FYI: IDataReader can be built in code with data sources being everything as e.g. .csv files etc. (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) using System; using System.Collections.Generic; using System.Text; using System.Data; using System.Data.SqlClient; namespace TestConsole.Samples { public class SqlBulkLoadTest { public static void Run() { string connectionString = "Data Source=HAMBURG\\SQL2005;" + "Initial Catalog=Northwind;User Id=sa;Password=shms"; using (SqlConnection connection = new SqlConnection(connectionString)) { connection.Open(); string sql = "select * from [Shippers]"; SqlCommand command = new SqlCommand(sql, connection); SqlDataAdapter adapter = new SqlDataAdapter(command); DataSet dataSet = new DataSet("Shippers"); adapter.Fill(dataSet); Console.WriteLine("*** dataSet ***"); foreach (DataRow dataRow in dataSet.Tables[0].Rows) { object[] items = { dataRow[0], dataRow[1], dataRow[2] }; Console.WriteLine("{0} {1} {2}", dataRow[0], dataRow[1], dataRow[2]); } (new SqlCommand("delete from [NewShippers]",connection)) .ExecuteNonQuery(); SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, SqlBulkCopyOptions.KeepIdentity); bulkCopy.DestinationTableName = "NewShippers"; bulkCopy.WriteToServer(dataSet.Tables[0]); } } } } -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 13, 2009 3:52 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? Shamil, That is precisely what I have in mind, with variations. Then at the end, write all updates back to SQL Server. Thanks for the demo code, always helpful. John W. Colby www.ColbyConsulting.com <<< snip >> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4603 (20091113) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 13 08:30:34 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 09:30:34 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <004801ca646c$5758e690$060ab3b0$@spb.ru> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> <004801ca646c$5758e690$060ab3b0$@spb.ru> Message-ID: <4AFD6D8A.3060306@colbyconsulting.com> In fact this is a one shot deal. I need to do this one time, to one table. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > You can also consider using SqlBulkLoad - below is just a simple sample, and > you can develop very flexible variations of this approach. SqlBulkLoad > should be as quick as bcp maybe even quicker. In fact the source for > SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this > IDataReader to have all the data to bulkload then your programs will become > as streamlined as they probably have never been, FYI: IDataReader can be > built in code with data sources being everything as e.g. .csv files etc. > (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) > > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class SqlBulkLoadTest > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;" + > "Initial Catalog=Northwind;User Id=sa;Password=shms"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { > connection.Open(); > string sql = "select * from [Shippers]"; > SqlCommand command = new SqlCommand(sql, connection); > SqlDataAdapter adapter = new SqlDataAdapter(command); > > DataSet dataSet = new DataSet("Shippers"); > adapter.Fill(dataSet); > > Console.WriteLine("*** dataSet ***"); > foreach (DataRow dataRow in dataSet.Tables[0].Rows) > { > object[] items = { dataRow[0], dataRow[1], dataRow[2] }; > > Console.WriteLine("{0} {1} {2}", > dataRow[0], dataRow[1], dataRow[2]); > } > > > (new SqlCommand("delete from [NewShippers]",connection)) > .ExecuteNonQuery(); > > SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, > SqlBulkCopyOptions.KeepIdentity); > bulkCopy.DestinationTableName = "NewShippers"; > bulkCopy.WriteToServer(dataSet.Tables[0]); > } > } > > } > } > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 13, 2009 3:52 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > Shamil, > > That is precisely what I have in mind, with variations. Then at the end, > write all updates back to > SQL Server. > > Thanks for the demo code, always helpful. > > John W. Colby > www.ColbyConsulting.com > > > <<< snip >> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4603 (20091113) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 13 08:30:42 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 09:30:42 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: References: <4AFC446F.6070207@colbyconsulting.com> Message-ID: <4AFD6D92.50807@colbyconsulting.com> Mark, > Just curious, how many records are you talking about - a few hundred or millions? Millions, however I am talking about updating sets of records, from 2 to < 30 rows. Imagine a table where Mark Breen has 22 records in the table, John Colby has 17 records in the table, Shamil has 3 records etc. Each of Mark's records have data in a "random" handful of 525 fields. The objective is to get all of Mark's data into one record. Once Mark's records are processed, move on to John's records. Then process William's records. There are 6,514,623 records to be processed. These 6.5 million records represent perhaps about 3 million actual people. I am creating tables right now to store the hash field and a count of records that hash field represents. As an example I have 9 actual people with more than 10 records to be merged, 4,725 people with 5 records, 270,0839 people with 3 records to merge, and 2,771,885 people with two records to merge. The nice part is that the table I am discussing is not constantly changing, in fact this data is entirely static. And there is no time frame for getting it done. I can start the process running and if it takes all night or all week I don't care. I suspect that with efficient programming I can do this at hundreds of records per second. I intend to do it out in tables dedicated to this process in a database dedicated to this process. All of the updates between fields in different records will be taking place entirely in memory. I will probably pull in all of the records for a thousand people, do the merges, then update those records back to SQL Server and pull another chunk of a thousand records. > Second question, other than for purely academic reasons, why are you bring the data into C# Mostly because I know programming, and TSQL is far enough from my expertise as to be unusable in the immediate future. To be honest I am not sure it is even possible to do this easily in TSQL. That said I am sure some TSQL GURU will pop up with the answer. But I am not a TSQL Guru (nor a Guru of any type for that matter). But the thought of programming this kind of algorithm in TSQL sends shivers down my spine. I need to do the following: R1F1 F1F2 R1F3... R1F525 R2F1 R2F2 R2F3... R2F525 R3F1 R3F2 R3F3... R3F525 The lines above represent Rows R1, R2 and R3, Fields F1,F2,F3...F525 Start with R1 and R2 Start at column 1 If R1F1 is null AND R2F1 is NOT NULL THEN Search through sequential records, same field until data found endif Move to column 2, continue to all columns Move to R1 and R3 In English: There is a set of records for a given person. If R1Col(N) has data, do not update. If R1Col(N) has no data, get data from R2Col(N) If R2Col(N) has no data, get data from R3Col(N) Start at N = (column) 1. Search until data found, update R1Col(N), then abort search and move to next column C# has a wonderful representation of data inside of the data adapter. Each table is a collection of rows AND a collection of columns. So I can iterate through the collection of columns, and once I have a collection of fields in a single column, iterate through that looking at the data in row 1, row 2 row 3 until data is found. This data representation just immediately does 1/2 the work for me by presenting me with my data already loaded into these collections. > Of course if it is to learn datasets then thats another story It is to get work done that needs doing. Along the way I expect to learn datasets and get that much more comfortable with C#. Doesn't this sound like FUN programming? I am a programmer at heart, I love doing this kind of stuff, and C# is where I am going, so why not do it there? John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > Just curious, how many records are you talking about - a few hundred or > millions? > > Second question, other than for purely academic reasons, why are you bring > the data into C#, to concatenate and insert into another table can so easily > be done in one sproc, even if you need a temporary holding place, you would > still use a temp table or even a local table. > > Of course if it is to learn datasets then thats another story, > > Mark From shamil at smsconsulting.spb.ru Fri Nov 13 08:36:06 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 17:36:06 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD6D8A.3060306@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> <004801ca646c$5758e690$060ab3b0$@spb.ru> <4AFD6D8A.3060306@colbyconsulting.com> Message-ID: <004a01ca646e$a3b24570$eb16d050$@spb.ru> Hi John, OK, if you're adding several millions rows then SqlBulkLoad can make your "shot deal" in one shot :) -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 13, 2009 5:31 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? In fact this is a one shot deal. I need to do this one time, to one table. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > You can also consider using SqlBulkLoad - below is just a simple sample, and > you can develop very flexible variations of this approach. SqlBulkLoad > should be as quick as bcp maybe even quicker. In fact the source for > SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this > IDataReader to have all the data to bulkload then your programs will become > as streamlined as they probably have never been, FYI: IDataReader can be > built in code with data sources being everything as e.g. .csv files etc. > (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) > > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class SqlBulkLoadTest > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;" + > "Initial Catalog=Northwind;User Id=sa;Password=N/A"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { > connection.Open(); > string sql = "select * from [Shippers]"; > SqlCommand command = new SqlCommand(sql, connection); > SqlDataAdapter adapter = new SqlDataAdapter(command); > > DataSet dataSet = new DataSet("Shippers"); > adapter.Fill(dataSet); > > Console.WriteLine("*** dataSet ***"); > foreach (DataRow dataRow in dataSet.Tables[0].Rows) > { > object[] items = { dataRow[0], dataRow[1], dataRow[2] }; > > Console.WriteLine("{0} {1} {2}", > dataRow[0], dataRow[1], dataRow[2]); > } > > > (new SqlCommand("delete from [NewShippers]",connection)) > .ExecuteNonQuery(); > > SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, > SqlBulkCopyOptions.KeepIdentity); > bulkCopy.DestinationTableName = "NewShippers"; > bulkCopy.WriteToServer(dataSet.Tables[0]); > } > } > > } > } > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 13, 2009 3:52 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > Shamil, > > That is precisely what I have in mind, with variations. Then at the end, > write all updates back to > SQL Server. > > Thanks for the demo code, always helpful. > > John W. Colby > www.ColbyConsulting.com > > > <<< snip >> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4603 (20091113) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4603 (20091113) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 13 08:40:25 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 09:40:25 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <004a01ca646e$a3b24570$eb16d050$@spb.ru> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> <004801ca646c$5758e690$060ab3b0$@spb.ru> <4AFD6D8A.3060306@colbyconsulting.com> <004a01ca646e$a3b24570$eb16d050$@spb.ru> Message-ID: <4AFD6FD9.2040902@colbyconsulting.com> See my reply to Mark Breen coming up. This is not a bulk load thing. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > OK, if you're adding several millions rows then SqlBulkLoad can make your > "shot deal" in one shot :) > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 13, 2009 5:31 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > In fact this is a one shot deal. I need to do this one time, to one table. > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> You can also consider using SqlBulkLoad - below is just a simple sample, > and >> you can develop very flexible variations of this approach. SqlBulkLoad >> should be as quick as bcp maybe even quicker. In fact the source for >> SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this >> IDataReader to have all the data to bulkload then your programs will > become >> as streamlined as they probably have never been, FYI: IDataReader can be >> built in code with data sources being everything as e.g. .csv files etc. >> (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) >> >> >> using System; >> using System.Collections.Generic; >> using System.Text; >> >> using System.Data; >> using System.Data.SqlClient; >> >> namespace TestConsole.Samples >> { >> public class SqlBulkLoadTest >> { >> public static void Run() >> { >> string connectionString = "Data Source=HAMBURG\\SQL2005;" + >> "Initial Catalog=Northwind;User Id=sa;Password=N/A"; >> using (SqlConnection connection = >> new SqlConnection(connectionString)) >> { >> connection.Open(); >> string sql = "select * from [Shippers]"; >> SqlCommand command = new SqlCommand(sql, connection); >> SqlDataAdapter adapter = new SqlDataAdapter(command); >> >> DataSet dataSet = new DataSet("Shippers"); >> adapter.Fill(dataSet); >> >> Console.WriteLine("*** dataSet ***"); >> foreach (DataRow dataRow in dataSet.Tables[0].Rows) >> { >> object[] items = { dataRow[0], dataRow[1], dataRow[2] > }; >> Console.WriteLine("{0} {1} {2}", >> dataRow[0], dataRow[1], dataRow[2]); >> } >> >> >> (new SqlCommand("delete from [NewShippers]",connection)) >> .ExecuteNonQuery(); >> >> SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, >> SqlBulkCopyOptions.KeepIdentity); >> bulkCopy.DestinationTableName = "NewShippers"; >> bulkCopy.WriteToServer(dataSet.Tables[0]); >> } >> } >> >> } >> } >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: Friday, November 13, 2009 3:52 PM >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] What to do, what to do? >> >> Shamil, >> >> That is precisely what I have in mind, with variations. Then at the end, >> write all updates back to >> SQL Server. >> >> Thanks for the demo code, always helpful. >> >> John W. Colby >> www.ColbyConsulting.com >> >> >> <<< snip >> >> >> >> __________ Information from ESET NOD32 Antivirus, version of virus > signature >> database 4603 (20091113) __________ >> >> The message was checked by ESET NOD32 Antivirus. >> >> http://www.esetnod32.ru >> >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4603 (20091113) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Nov 13 08:54:16 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 17:54:16 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD6FD9.2040902@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> <004801ca646c$5758e690$060ab3b0$@spb.ru> <4AFD6D8A.3060306@colbyconsulting.com> <004a01ca646e$a3b24570$eb16d050$@spb.ru> <4AFD6FD9.2040902@colbyconsulting.com> Message-ID: <004b01ca6471$2d3f5dd0$87be1970$@spb.ru> OK, I see this is "update thing" not insert/bulkload. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 13, 2009 5:40 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? See my reply to Mark Breen coming up. This is not a bulk load thing. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > OK, if you're adding several millions rows then SqlBulkLoad can make your > "shot deal" in one shot :) > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 13, 2009 5:31 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > In fact this is a one shot deal. I need to do this one time, to one table. > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> You can also consider using SqlBulkLoad - below is just a simple sample, > and >> you can develop very flexible variations of this approach. SqlBulkLoad >> should be as quick as bcp maybe even quicker. In fact the source for >> SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this >> IDataReader to have all the data to bulkload then your programs will > become >> as streamlined as they probably have never been, FYI: IDataReader can be >> built in code with data sources being everything as e.g. .csv files etc. >> (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) >> >> >> using System; >> using System.Collections.Generic; >> using System.Text; >> >> using System.Data; >> using System.Data.SqlClient; >> >> namespace TestConsole.Samples >> { >> public class SqlBulkLoadTest >> { >> public static void Run() >> { >> string connectionString = "Data Source=HAMBURG\\SQL2005;" + >> "Initial Catalog=Northwind;User Id=sa;Password=N/A"; >> using (SqlConnection connection = >> new SqlConnection(connectionString)) >> { >> connection.Open(); >> string sql = "select * from [Shippers]"; >> SqlCommand command = new SqlCommand(sql, connection); >> SqlDataAdapter adapter = new SqlDataAdapter(command); >> >> DataSet dataSet = new DataSet("Shippers"); >> adapter.Fill(dataSet); >> >> Console.WriteLine("*** dataSet ***"); >> foreach (DataRow dataRow in dataSet.Tables[0].Rows) >> { >> object[] items = { dataRow[0], dataRow[1], dataRow[2] > }; >> Console.WriteLine("{0} {1} {2}", >> dataRow[0], dataRow[1], dataRow[2]); >> } >> >> >> (new SqlCommand("delete from [NewShippers]",connection)) >> .ExecuteNonQuery(); >> >> SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, >> SqlBulkCopyOptions.KeepIdentity); >> bulkCopy.DestinationTableName = "NewShippers"; >> bulkCopy.WriteToServer(dataSet.Tables[0]); >> } >> } >> >> } >> } >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: Friday, November 13, 2009 3:52 PM >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] What to do, what to do? >> >> Shamil, >> >> That is precisely what I have in mind, with variations. Then at the end, >> write all updates back to >> SQL Server. >> >> Thanks for the demo code, always helpful. >> >> John W. Colby >> www.ColbyConsulting.com >> >> >> <<< snip >> >> From jwcolby at colbyconsulting.com Fri Nov 13 10:39:14 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 11:39:14 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> Message-ID: <4AFD8BB2.5030203@colbyconsulting.com> Shamil, Given what you now know about what I want to do... Why does each table get created in it's own DataSet. I thought a data set could have multiple tables. I envisioned the following structure out in SQL Server: tblHSID - contains 6.4 million records. Source of records for tblMergeSrc. tblMergeSrc - Contains X thousand records for Y hundred people. Working table in memory. tblMergedRecs - the single record merged into will be created here (or copied here). Working table in memory. tblPurgedRecs - The multiple records merged FROM will be archived here. Working table in memory. tblHashPK - Hash for a person / address, PKs for person / address. There is one hash for each person / hash, there are multiple PKs for each person / address. One record for each record in tblHSID. tblHashCnt - Group by Hash, Cnt PK. Server two purposes, provides a "DISTINCT" hash for each person being processed as well as a count of how many records will be merged down. So... tblHSID is the source of the records to be processed. Contains ONLY people who actually have "duplicates", most people don't. The Top(N) records from tblHashCnt will select the Top(N) PEOPLE by number of records to be merged. Merge the ones with the most records first. That will be joined to tblHashPK to pull all of the PKs for the Top(N) people about to be processed. The PKs from that will be joined to tblHSID to pull a set of records to be merged. Copied into tblMergeSrc. This is the first time HSID records (records to be merged) will be moved into a table in a dataset. tblMergedRecs is empty but exists in the dataset. tblPurgedRecs is empty but exists in the dataset. The merge will most likely happen in tblMergeSrc although a new record could be created in tblMergedRecs to work in. Not decided yet. In the end, the merge happens in the dataset in memory. The actual ONE merged record for each person ends up in tblMergedRecs. The actual (multiple) records merged into other records are moved into tblPurgedRecs. This is an archive. The records in tblPurgedRecs are then deleted from tblMergedSrc, and eventually deleted out of tblHSID back in the database. The records for the next person are processed. Merged records moved to tblMergedRecs. "Purged" records moved to tblPurgedRecs. Deleted from tblMergeSrc. Wash, rinse repeat. Now... I don't care how many data sets I have but I just question why I would create a new dataset for each table above. It seems that having them all in a single dataset would minimize confusion and stuff. Logically it seems that working with multiple tables inside of a single DataSet object MIGHT be faster than moving data back and forth between tables in different DataSet objects, though maybe not. And of course... my book talks all about connections, then all about commands, and then all about parameters, then... How to work with a data reader. WTF over? What happened to my dataset and table objects? Not a WORD about those two objects. Sigh. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Here is a sample: > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class DataSetTests > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;"+ > "Initial Catalog=Northwind;User Id=sa;Password=N/A"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { From shamil at smsconsulting.spb.ru Fri Nov 13 11:22:31 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 20:22:31 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD8BB2.5030203@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD8BB2.5030203@colbyconsulting.com> Message-ID: <005b01ca6485$e42c31d0$ac849570$@spb.ru> John, Yes, of course you can have multiple DataTable(s) in a DataSet - I have just posted a very simple sample - one of a myriad possible combinations, the simplest one probably. And of course you can have your DataSet(s) structure created statically in VS designer. Yes, I think your plan is OK. You probably do not need a DataReader but if you'll find you need one - in P.S. is a simple sample. In fact Gustav knows DataSets better than I do - please address your questions to him also and to everybody else here - I will not have time to reply any more postings today... Thank you. -- Shamil P.S. DataReader sample: using System; using System.Collections.Generic; using System.Text; using System.Data; using System.Data.SqlClient; namespace TestConsole.Samples { public class SqlDataReaderTest { public static void Run() { string connectionString = "Data Source=HAMBURG\\SQL2005;" + "Initial Catalog=Northwind;User Id=sa;Password=N/A"; using (SqlConnection connection = new SqlConnection(connectionString)) { connection.Open(); // Create empty [NewShippers] dataset in memory string sql1 = "select * from [NewShippers] where(1=0)"; SqlCommand command = new SqlCommand(sql1, connection); SqlDataAdapter adapter = new SqlDataAdapter(command); DataSet newDataSet = new DataSet("NewShippers"); adapter.Fill(newDataSet); // Load source [Shippers] in memory string sql = "select * from [Shippers]"; command = new SqlCommand(sql, connection); SqlDataReader dataReader = command.ExecuteReader(); // Add loaded [Shippers] to [NewShippers] im memory Console.WriteLine("*** dataReader ***"); while (dataReader.Read()) { object[] items = { dataReader[0], dataReader[1], dataReader[2] }; newDataSet.Tables[0].Rows.Add(items); Console.WriteLine("{0} {1} {2}", items[0], items[1], items[2]); } dataReader.Close(); // ! always close DataReader // Insert [NewShippers] into db (new SqlCommand("delete from [NewShippers]",connection)) .ExecuteNonQuery(); SqlCommandBuilder builder = new SqlCommandBuilder(adapter); adapter.InsertCommand = builder.GetInsertCommand(true); adapter.Update(newDataSet.Tables[0]); // Check that [NewShippers] were inserted OK Console.WriteLine("*** newDataSet ***"); newDataSet.Clear(); adapter = new SqlDataAdapter( new SqlCommand("select * from [NewShippers]",connection)); adapter.Fill(newDataSet); foreach (DataRow dataRow in newDataSet.Tables[0].Rows) { Console.WriteLine("{0} {1} {2}", dataRow[0], dataRow[1], dataRow[2]); } } } } } <<< snip to get through dba-VB robot >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4604 (20091113) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From cfoust at infostatsystems.com Fri Nov 13 12:18:07 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Fri, 13 Nov 2009 10:18:07 -0800 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD8BB2.5030203@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com><002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD8BB2.5030203@colbyconsulting.com> Message-ID: You can have all sorts of tables in a dataset, John. But if you want to act on multiple non-cascading tables, then you probably need to validate multiple tables. If you have relationships set so you can cascade updates, it isn't too bad. Otherwise, you have to write code to individually handle the gotchas in each table before you update any of them. Without that, you'd wind up with a partial update. Some of my biggest headaches are the one rich typed dataset we use to allow the users to enter data into 3 or 4 different tables from a single form. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 13, 2009 8:39 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? Shamil, Given what you now know about what I want to do... Why does each table get created in it's own DataSet. I thought a data set could have multiple tables. I envisioned the following structure out in SQL Server: tblHSID - contains 6.4 million records. Source of records for tblMergeSrc. tblMergeSrc - Contains X thousand records for Y hundred people. Working table in memory. tblMergedRecs - the single record merged into will be created here (or copied here). Working table in memory. tblPurgedRecs - The multiple records merged FROM will be archived here. Working table in memory. tblHashPK - Hash for a person / address, PKs for person / address. There is one hash for each person / hash, there are multiple PKs for each person / address. One record for each record in tblHSID. tblHashCnt - Group by Hash, Cnt PK. Server two purposes, provides a "DISTINCT" hash for each person being processed as well as a count of how many records will be merged down. So... tblHSID is the source of the records to be processed. Contains ONLY people who actually have "duplicates", most people don't. The Top(N) records from tblHashCnt will select the Top(N) PEOPLE by number of records to be merged. Merge the ones with the most records first. That will be joined to tblHashPK to pull all of the PKs for the Top(N) people about to be processed. The PKs from that will be joined to tblHSID to pull a set of records to be merged. Copied into tblMergeSrc. This is the first time HSID records (records to be merged) will be moved into a table in a dataset. tblMergedRecs is empty but exists in the dataset. tblPurgedRecs is empty but exists in the dataset. The merge will most likely happen in tblMergeSrc although a new record could be created in tblMergedRecs to work in. Not decided yet. In the end, the merge happens in the dataset in memory. The actual ONE merged record for each person ends up in tblMergedRecs. The actual (multiple) records merged into other records are moved into tblPurgedRecs. This is an archive. The records in tblPurgedRecs are then deleted from tblMergedSrc, and eventually deleted out of tblHSID back in the database. The records for the next person are processed. Merged records moved to tblMergedRecs. "Purged" records moved to tblPurgedRecs. Deleted from tblMergeSrc. Wash, rinse repeat. Now... I don't care how many data sets I have but I just question why I would create a new dataset for each table above. It seems that having them all in a single dataset would minimize confusion and stuff. Logically it seems that working with multiple tables inside of a single DataSet object MIGHT be faster than moving data back and forth between tables in different DataSet objects, though maybe not. And of course... my book talks all about connections, then all about commands, and then all about parameters, then... How to work with a data reader. WTF over? What happened to my dataset and table objects? Not a WORD about those two objects. Sigh. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Here is a sample: > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class DataSetTests > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;"+ > "Initial Catalog=Northwind;User Id=sa;Password=N/A"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From raibeart at gmail.com Fri Nov 13 22:46:18 2009 From: raibeart at gmail.com (Robert Stewart) Date: Fri, 13 Nov 2009 22:46:18 -0600 Subject: [dba-VB] What to do, what to do? In-Reply-To: References: Message-ID: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert At 12:00 PM 11/13/2009, you wrote: >Date: Fri, 13 Nov 2009 11:39:14 -0500 >From: jwcolby >Subject: Re: [dba-VB] What to do, what to do? >To: "Discussion concerning Visual Basic and related programming > issues." >Message-ID: <4AFD8BB2.5030203 at colbyconsulting.com> >Content-Type: text/plain; charset=ISO-8859-1; format=flowed > >Shamil, > >Given what you now know about what I want to do... > >Why does each table get created in it's own DataSet. I thought a >data set could have multiple >tables. I envisioned the following structure out in SQL Server: > >tblHSID - contains 6.4 million records. Source of records for tblMergeSrc. >tblMergeSrc - Contains X thousand records for Y hundred >people. Working table in memory. >tblMergedRecs - the single record merged into will be created here >(or copied here). Working table >in memory. >tblPurgedRecs - The multiple records merged FROM will be archived >here. Working table in memory. > >tblHashPK - Hash for a person / address, PKs for person / >address. There is one hash for each >person / hash, there are multiple PKs for each person / >address. One record for each record in >tblHSID. > >tblHashCnt - Group by Hash, Cnt PK. Server two purposes, provides a >"DISTINCT" hash for each person >being processed as well as a count of how many records will be merged down. > >So... tblHSID is the source of the records to be >processed. Contains ONLY people who actually have >"duplicates", most people don't. > >The Top(N) records from tblHashCnt will select the Top(N) PEOPLE by >number of records to be merged. > Merge the ones with the most records first. > >That will be joined to tblHashPK to pull all of the PKs for the >Top(N) people about to be processed. > >The PKs from that will be joined to tblHSID to pull a set of records >to be merged. Copied into >tblMergeSrc. This is the first time HSID records (records to be >merged) will be moved into a table >in a dataset. > >tblMergedRecs is empty but exists in the dataset. >tblPurgedRecs is empty but exists in the dataset. > >The merge will most likely happen in tblMergeSrc although a new >record could be created in >tblMergedRecs to work in. Not decided yet. > >In the end, the merge happens in the dataset in memory. The actual >ONE merged record for each >person ends up in tblMergedRecs. > >The actual (multiple) records merged into other records are moved >into tblPurgedRecs. This is an >archive. > >The records in tblPurgedRecs are then deleted from tblMergedSrc, and >eventually deleted out of >tblHSID back in the database. > >The records for the next person are processed. Merged records moved >to tblMergedRecs. "Purged" >records moved to tblPurgedRecs. > >Deleted from tblMergeSrc. > >Wash, rinse repeat. > >Now... I don't care how many data sets I have but I just question >why I would create a new dataset >for each table above. It seems that having them all in a single >dataset would minimize confusion >and stuff. Logically it seems that working with multiple tables >inside of a single DataSet object >MIGHT be faster than moving data back and forth between tables in >different DataSet objects, though >maybe not. > >And of course... my book talks all about connections, then all about >commands, and then all about >parameters, then... > >How to work with a data reader. WTF over? What happened to my >dataset and table objects? Not a >WORD about those two objects. > >Sigh. > >John W. Colby From shamil at smsconsulting.spb.ru Sat Nov 14 04:40:43 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 14 Nov 2009 13:40:43 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> Message-ID: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Hi Robert, Yes, DataSets are mainly targeted for working with small number of records in memory - so the following is more a "theoretical" calculation: - JC has 64bit PC which allow to load in memory "practically unlimited" volume of data - 64bit logical (process) address space could be as large as 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 (http://en.wikipedia.org/wiki/Exbibyte). - the speed of data transfer for 64bit Intel's processor - "the 200 MHz McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); ----- So "theoretically" with modern speedy harddisks JC can load 50 millions of records into memory within minutes - within half a minute in the near future when large enough flash-memory disks will become available and relatively inexpensive? To load such large data volume it would be better to use SqlDataReader as it keeps loaded data in a very compact form. And 50 million records long SqlDataReader should be probably better split into several chunks... Again - this is just a "theoretical" consideration - in practice JC can process his 50 million records long data table in chunks as he has a hash field (tblHashPK), which can be used to load related records into memory: I mean he can split tblHashPK table keeping hash values into several groups, e.g.: 1 - 100,000 100,001 - 200,000 ... and process each group joining its records to source 50 million records... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart Sent: Saturday, November 14, 2009 7:46 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] What to do, what to do? John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From max.wanadoo at gmail.com Sat Nov 14 05:30:41 2009 From: max.wanadoo at gmail.com (Max Wanadoo) Date: Sat, 14 Nov 2009 11:30:41 -0000 Subject: [dba-VB] What to do, what to do? In-Reply-To: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Message-ID: <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> Very interesting Shamil What happens with all the other stuff that memory is taking up though, OS, Programs, etc and the physical memory he has placed in the machine? Your limits below presumably are saying that his physical memory is loaded to...what? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 10:41 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Robert, Yes, DataSets are mainly targeted for working with small number of records in memory - so the following is more a "theoretical" calculation: - JC has 64bit PC which allow to load in memory "practically unlimited" volume of data - 64bit logical (process) address space could be as large as 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 (http://en.wikipedia.org/wiki/Exbibyte). - the speed of data transfer for 64bit Intel's processor - "the 200 MHz McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); ----- So "theoretically" with modern speedy harddisks JC can load 50 millions of records into memory within minutes - within half a minute in the near future when large enough flash-memory disks will become available and relatively inexpensive? To load such large data volume it would be better to use SqlDataReader as it keeps loaded data in a very compact form. And 50 million records long SqlDataReader should be probably better split into several chunks... Again - this is just a "theoretical" consideration - in practice JC can process his 50 million records long data table in chunks as he has a hash field (tblHashPK), which can be used to load related records into memory: I mean he can split tblHashPK table keeping hash values into several groups, e.g.: 1 - 100,000 100,001 - 200,000 ... and process each group joining its records to source 50 million records... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart Sent: Saturday, November 14, 2009 7:46 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] What to do, what to do? John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Nov 14 06:30:35 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 14 Nov 2009 07:30:35 -0500 Subject: [dba-VB] SPAM-LOW: Re: What to do, what to do? In-Reply-To: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> Message-ID: <4AFEA2EB.5050507@colbyconsulting.com> Read the discussion carefully Robert. I am discussing doing chunks of thousands. John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > John, > > I don't think your machine, or any for that matter, have the memory > to load 50 million records, extract 40 million, transform them and > then load them back to the 50 million and update the database from > the recordset. Datasets are for working with a small number of > records in memory, not millions. > > Robert > > > At 12:00 PM 11/13/2009, you wrote: >> Date: Fri, 13 Nov 2009 11:39:14 -0500 >> From: jwcolby >> Subject: Re: [dba-VB] What to do, what to do? >> To: "Discussion concerning Visual Basic and related programming >> issues." >> Message-ID: <4AFD8BB2.5030203 at colbyconsulting.com> >> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >> >> Shamil, >> >> Given what you now know about what I want to do... >> >> Why does each table get created in it's own DataSet. I thought a >> data set could have multiple >> tables. I envisioned the following structure out in SQL Server: >> >> tblHSID - contains 6.4 million records. Source of records for tblMergeSrc. >> tblMergeSrc - Contains X thousand records for Y hundred >> people. Working table in memory. >> tblMergedRecs - the single record merged into will be created here >> (or copied here). Working table >> in memory. >> tblPurgedRecs - The multiple records merged FROM will be archived >> here. Working table in memory. >> >> tblHashPK - Hash for a person / address, PKs for person / >> address. There is one hash for each >> person / hash, there are multiple PKs for each person / >> address. One record for each record in >> tblHSID. >> >> tblHashCnt - Group by Hash, Cnt PK. Server two purposes, provides a >> "DISTINCT" hash for each person >> being processed as well as a count of how many records will be merged down. >> >> So... tblHSID is the source of the records to be >> processed. Contains ONLY people who actually have >> "duplicates", most people don't. >> >> The Top(N) records from tblHashCnt will select the Top(N) PEOPLE by >> number of records to be merged. >> Merge the ones with the most records first. >> >> That will be joined to tblHashPK to pull all of the PKs for the >> Top(N) people about to be processed. >> >> The PKs from that will be joined to tblHSID to pull a set of records >> to be merged. Copied into >> tblMergeSrc. This is the first time HSID records (records to be >> merged) will be moved into a table >> in a dataset. >> >> tblMergedRecs is empty but exists in the dataset. >> tblPurgedRecs is empty but exists in the dataset. >> >> The merge will most likely happen in tblMergeSrc although a new >> record could be created in >> tblMergedRecs to work in. Not decided yet. >> >> In the end, the merge happens in the dataset in memory. The actual >> ONE merged record for each >> person ends up in tblMergedRecs. >> >> The actual (multiple) records merged into other records are moved >> into tblPurgedRecs. This is an >> archive. >> >> The records in tblPurgedRecs are then deleted from tblMergedSrc, and >> eventually deleted out of >> tblHSID back in the database. >> >> The records for the next person are processed. Merged records moved >> to tblMergedRecs. "Purged" >> records moved to tblPurgedRecs. >> >> Deleted from tblMergeSrc. >> >> Wash, rinse repeat. >> >> Now... I don't care how many data sets I have but I just question >> why I would create a new dataset >> for each table above. It seems that having them all in a single >> dataset would minimize confusion >> and stuff. Logically it seems that working with multiple tables >> inside of a single DataSet object >> MIGHT be faster than moving data back and forth between tables in >> different DataSet objects, though >> maybe not. >> >> And of course... my book talks all about connections, then all about >> commands, and then all about >> parameters, then... >> >> How to work with a data reader. WTF over? What happened to my >> dataset and table objects? Not a >> WORD about those two objects. >> >> Sigh. >> >> John W. Colby > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Nov 14 07:46:21 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 14 Nov 2009 16:46:21 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> Message-ID: <008201ca6530$daf14140$90d3c3c0$@spb.ru> Hi Max, But OS, programs etc. - they all use their own logical address space, which is mapped to (shared (using OS directed swapping)) physical address space: with JC 64bit PC OS image/runtime will be kept in its own physical memory segments - and no swapping will occur, MS SQL image/runtime/data will be (mainly) kept in its own physical memory segments - and almost no swapping will occur, JC's program will have occupied its own physical memory segments - and no swapping will occur... 50 million records assuming every of them is equal to approx. 1KB will consume ~50GB - 64bit PC can have(?) that much RAM allocated for one process, and it all can be addresses directly within this process logical address space. And I have noted that it's currently a "theoretical consideration" but it may become practically achievable in not that far future(?) Correct/wrong? (I must say I didn't get checked how large can be logical address space for a Win7 process running on 64bit PC). Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Max Wanadoo Sent: Saturday, November 14, 2009 2:31 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Very interesting Shamil What happens with all the other stuff that memory is taking up though, OS, Programs, etc and the physical memory he has placed in the machine? Your limits below presumably are saying that his physical memory is loaded to...what? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 10:41 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Robert, Yes, DataSets are mainly targeted for working with small number of records in memory - so the following is more a "theoretical" calculation: - JC has 64bit PC which allow to load in memory "practically unlimited" volume of data - 64bit logical (process) address space could be as large as 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 (http://en.wikipedia.org/wiki/Exbibyte). - the speed of data transfer for 64bit Intel's processor - "the 200 MHz McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); ----- So "theoretically" with modern speedy harddisks JC can load 50 millions of records into memory within minutes - within half a minute in the near future when large enough flash-memory disks will become available and relatively inexpensive? To load such large data volume it would be better to use SqlDataReader as it keeps loaded data in a very compact form. And 50 million records long SqlDataReader should be probably better split into several chunks... Again - this is just a "theoretical" consideration - in practice JC can process his 50 million records long data table in chunks as he has a hash field (tblHashPK), which can be used to load related records into memory: I mean he can split tblHashPK table keeping hash values into several groups, e.g.: 1 - 100,000 100,001 - 200,000 ... and process each group joining its records to source 50 million records... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart Sent: Saturday, November 14, 2009 7:46 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] What to do, what to do? John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 14 08:23:42 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 14 Nov 2009 09:23:42 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Message-ID: <4AFEBD6E.1020709@colbyconsulting.com> All of which is beside the point. Divide and conquer. A principal discovered by military strategists thousands of years ago, and applied to problems of all sorts today. I can EASILY work with sets of thousands, and even hundreds of thousands of records. In real life, in my computer. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Robert, > > Yes, DataSets are mainly targeted for working with small number of records > in memory - so the following is more a "theoretical" calculation: > > - JC has 64bit PC which allow to load in memory "practically unlimited" > volume of data - 64bit logical (process) address space could be as large as > 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 > (http://en.wikipedia.org/wiki/Exbibyte). > > - the speed of data transfer for 64bit Intel's processor - "the 200 MHz > McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers > 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); > > ----- > So "theoretically" with modern speedy harddisks JC can load 50 millions of > records into memory within minutes - within half a minute in the near future > when large enough flash-memory disks will become available and relatively > inexpensive? > > To load such large data volume it would be better to use SqlDataReader as it > keeps loaded data in a very compact form. And 50 million records long > SqlDataReader should be probably better split into several chunks... > > Again - this is just a "theoretical" consideration - in practice JC can > process his 50 million records long data table in chunks as he has a hash > field (tblHashPK), which can be used to load related records into memory: I > mean he can split tblHashPK table keeping hash values into several groups, > e.g.: > > 1 - 100,000 > 100,001 - 200,000 > ... > > and process each group joining its records to source 50 million records... > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart > Sent: Saturday, November 14, 2009 7:46 AM > To: dba-vb at databaseadvisors.com > Subject: Re: [dba-VB] What to do, what to do? > > John, > > I don't think your machine, or any for that matter, have the memory > to load 50 million records, extract 40 million, transform them and > then load them back to the 50 million and update the database from > the recordset. Datasets are for working with a small number of > records in memory, not millions. > > Robert > > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4606 (20091114) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Nov 14 08:44:11 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 14 Nov 2009 17:44:11 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFEBD6E.1020709@colbyconsulting.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4AFEBD6E.1020709@colbyconsulting.com> Message-ID: <008301ca6538$eeef8910$ccce9b30$@spb.ru> Hi John, Yes, I have just "theoretically" noted that in the near future you'll be able to EVEN MORE EASILY work with sets of millions (and billions?) records by just loading them into memory, "crunching", and saving back updated... ... it will be just 1,2,3 instead of your today's (1,2,3) (1,2,3)... (1,2,3) (multi-threaded)... ... the time just to develop (1,2,3) (1,2,3)... (1,2,3)... will be more expensive than time to develop and run 1,2,3 on the near future hardware (hardware costs included)... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 14, 2009 5:24 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? All of which is beside the point. Divide and conquer. A principal discovered by military strategists thousands of years ago, and applied to problems of all sorts today. I can EASILY work with sets of thousands, and even hundreds of thousands of records. In real life, in my computer. John W. Colby www.ColbyConsulting.com <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4607 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From max.wanadoo at gmail.com Sat Nov 14 09:38:39 2009 From: max.wanadoo at gmail.com (Max Wanadoo) Date: Sat, 14 Nov 2009 15:38:39 -0000 Subject: [dba-VB] What to do, what to do? In-Reply-To: <008201ca6530$daf14140$90d3c3c0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> <008201ca6530$daf14140$90d3c3c0$@spb.ru> Message-ID: <4afecf2a.0702d00a.3bd1.4040@mx.google.com> Hi Shamil, I understand the logical size bit but for that to be mapped to physical memory requires physical memory to be present. Current there are limits to how much can be installed on the motherboard. 64Bit memory can address the sizes you mentioned but the physical memory doesn't exist, so (presumably) it maps it to the HD, in which case it is not in memory. Have I misunderstood something (it has been known to happen)? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 13:46 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Max, But OS, programs etc. - they all use their own logical address space, which is mapped to (shared (using OS directed swapping)) physical address space: with JC 64bit PC OS image/runtime will be kept in its own physical memory segments - and no swapping will occur, MS SQL image/runtime/data will be (mainly) kept in its own physical memory segments - and almost no swapping will occur, JC's program will have occupied its own physical memory segments - and no swapping will occur... 50 million records assuming every of them is equal to approx. 1KB will consume ~50GB - 64bit PC can have(?) that much RAM allocated for one process, and it all can be addresses directly within this process logical address space. And I have noted that it's currently a "theoretical consideration" but it may become practically achievable in not that far future(?) Correct/wrong? (I must say I didn't get checked how large can be logical address space for a Win7 process running on 64bit PC). Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Max Wanadoo Sent: Saturday, November 14, 2009 2:31 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Very interesting Shamil What happens with all the other stuff that memory is taking up though, OS, Programs, etc and the physical memory he has placed in the machine? Your limits below presumably are saying that his physical memory is loaded to...what? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 10:41 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Robert, Yes, DataSets are mainly targeted for working with small number of records in memory - so the following is more a "theoretical" calculation: - JC has 64bit PC which allow to load in memory "practically unlimited" volume of data - 64bit logical (process) address space could be as large as 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 (http://en.wikipedia.org/wiki/Exbibyte). - the speed of data transfer for 64bit Intel's processor - "the 200 MHz McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); ----- So "theoretically" with modern speedy harddisks JC can load 50 millions of records into memory within minutes - within half a minute in the near future when large enough flash-memory disks will become available and relatively inexpensive? To load such large data volume it would be better to use SqlDataReader as it keeps loaded data in a very compact form. And 50 million records long SqlDataReader should be probably better split into several chunks... Again - this is just a "theoretical" consideration - in practice JC can process his 50 million records long data table in chunks as he has a hash field (tblHashPK), which can be used to load related records into memory: I mean he can split tblHashPK table keeping hash values into several groups, e.g.: 1 - 100,000 100,001 - 200,000 ... and process each group joining its records to source 50 million records... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart Sent: Saturday, November 14, 2009 7:46 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] What to do, what to do? John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Sat Nov 14 10:08:26 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 14 Nov 2009 19:08:26 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4afecf2a.0702d00a.3bd1.4040@mx.google.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> <008201ca6530$daf14140$90d3c3c0$@spb.ru> <4afecf2a.0702d00a.3bd1.4040@mx.google.com> Message-ID: <008401ca6544$b4290890$1c7b19b0$@spb.ru> Hi Max, Here are theoretical limits I have found info about: Address space 64-bit Windows 32-bit Windows Virtual memory 16 Tb 4 Gb Swap file 512 Tb 16 Tb System cache 1 Tb 1 Gb In practice, mass market motherboards can already handle 64GB(?) - Not enough for JC tasks but this max RAM size limitations should be removed in the near future AFAICG, and Win7 is ready(?) to handle effectively that much RAM.... http://www.nextag.com/64-bit-pci-motherboard/search-html http://www.viva64.com/content/articles/64-bit-development/?f=Application_por t_to_64-bit_platforms_or_never_cackle_till_your_egg_is_laid.html&lang=en&con tent=64-bit-development http://blogs.zdnet.com/hardware/?p=4254 http://www.tomshardware.com/reviews/vista-workshop,1775-3.html Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Max Wanadoo Sent: Saturday, November 14, 2009 6:39 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Shamil, I understand the logical size bit but for that to be mapped to physical memory requires physical memory to be present. Current there are limits to how much can be installed on the motherboard. 64Bit memory can address the sizes you mentioned but the physical memory doesn't exist, so (presumably) it maps it to the HD, in which case it is not in memory. Have I misunderstood something (it has been known to happen)? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 13:46 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Max, But OS, programs etc. - they all use their own logical address space, which is mapped to (shared (using OS directed swapping)) physical address space: with JC 64bit PC OS image/runtime will be kept in its own physical memory segments - and no swapping will occur, MS SQL image/runtime/data will be (mainly) kept in its own physical memory segments - and almost no swapping will occur, JC's program will have occupied its own physical memory segments - and no swapping will occur... 50 million records assuming every of them is equal to approx. 1KB will consume ~50GB - 64bit PC can have(?) that much RAM allocated for one process, and it all can be addresses directly within this process logical address space. And I have noted that it's currently a "theoretical consideration" but it may become practically achievable in not that far future(?) Correct/wrong? (I must say I didn't get checked how large can be logical address space for a Win7 process running on 64bit PC). Thank you. -- Shamil <<< snipped >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4607 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From max.wanadoo at gmail.com Sat Nov 14 10:27:00 2009 From: max.wanadoo at gmail.com (Max Wanadoo) Date: Sat, 14 Nov 2009 16:27:00 -0000 Subject: [dba-VB] What to do, what to do? In-Reply-To: <008401ca6544$b4290890$1c7b19b0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> <008201ca6530$daf14140$90d3c3c0$@spb.ru> <4afecf2a.0702d00a.3bd1.4040@mx.google.com> <008401ca6544$b4290890$1c7b19b0$@spb.ru> Message-ID: <4afeda84.0508d00a.3cd0.ffffe9aa@mx.google.com> Ok, I thought you were referring to ram available now on common-or-garden PCs. So, we can look forward to physical ram being cheap enough to enable in-vitro processing for millions of records. Hurrah! And goodbye to latency... Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 16:08 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Max, Here are theoretical limits I have found info about: Address space 64-bit Windows 32-bit Windows Virtual memory 16 Tb 4 Gb Swap file 512 Tb 16 Tb System cache 1 Tb 1 Gb In practice, mass market motherboards can already handle 64GB(?) - Not enough for JC tasks but this max RAM size limitations should be removed in the near future AFAICG, and Win7 is ready(?) to handle effectively that much RAM.... http://www.nextag.com/64-bit-pci-motherboard/search-html http://www.viva64.com/content/articles/64-bit-development/?f=Application_por t_to_64-bit_platforms_or_never_cackle_till_your_egg_is_laid.html&lang=en&con tent=64-bit-development http://blogs.zdnet.com/hardware/?p=4254 http://www.tomshardware.com/reviews/vista-workshop,1775-3.html Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Max Wanadoo Sent: Saturday, November 14, 2009 6:39 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Shamil, I understand the logical size bit but for that to be mapped to physical memory requires physical memory to be present. Current there are limits to how much can be installed on the motherboard. 64Bit memory can address the sizes you mentioned but the physical memory doesn't exist, so (presumably) it maps it to the HD, in which case it is not in memory. Have I misunderstood something (it has been known to happen)? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 13:46 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Max, But OS, programs etc. - they all use their own logical address space, which is mapped to (shared (using OS directed swapping)) physical address space: with JC 64bit PC OS image/runtime will be kept in its own physical memory segments - and no swapping will occur, MS SQL image/runtime/data will be (mainly) kept in its own physical memory segments - and almost no swapping will occur, JC's program will have occupied its own physical memory segments - and no swapping will occur... 50 million records assuming every of them is equal to approx. 1KB will consume ~50GB - 64bit PC can have(?) that much RAM allocated for one process, and it all can be addresses directly within this process logical address space. And I have noted that it's currently a "theoretical consideration" but it may become practically achievable in not that far future(?) Correct/wrong? (I must say I didn't get checked how large can be logical address space for a Win7 process running on 64bit PC). Thank you. -- Shamil <<< snipped >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4607 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Nov 14 10:28:14 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 14 Nov 2009 11:28:14 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <008301ca6538$eeef8910$ccce9b30$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4AFEBD6E.1020709@colbyconsulting.com> <008301ca6538$eeef8910$ccce9b30$@spb.ru> Message-ID: <4AFEDA9E.9050101@colbyconsulting.com> And believe me I look forward to the day. And it will indeed be soon, of course the data sets get bigger as well. ;) I already have 16 gigs of RAM in my servers, and quad cores. I could easily have 64 gigs and 16 CPUs today if I had the bucks. And of course you are correct as well, the Flash drives are wicked fast for working with large data sets. They still have real issues when writing to them bot for reading they are an order of magnitude faster than rotating media. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > Yes, I have just "theoretically" noted that in the near future you'll be > able to EVEN MORE EASILY work with sets of millions (and billions?) records > by just loading them into memory, "crunching", and saving back updated... > > ... it will be just 1,2,3 instead of your today's (1,2,3) (1,2,3)... (1,2,3) > (multi-threaded)... > > ... the time just to develop (1,2,3) (1,2,3)... (1,2,3)... will be more > expensive than time to develop and run 1,2,3 on the near future hardware > (hardware costs included)... > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 14, 2009 5:24 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > All of which is beside the point. > > Divide and conquer. A principal discovered by military strategists > thousands of years ago, and > applied to problems of all sorts today. > > I can EASILY work with sets of thousands, and even hundreds of thousands of > records. In real life, > in my computer. > > John W. Colby > www.ColbyConsulting.com > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4607 (20091114) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Nov 14 16:36:57 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 14 Nov 2009 17:36:57 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Message-ID: <4AFF3109.40003@colbyconsulting.com> To get back to reality, I really need to process under 6 million records, and the vast majority will be merging two records into one. I will almost certainly pull about 10,000 records at a time, so I will be processing 5,000 people at a time. Merge two records, merge two more, do that ~ 5000 times, write back to SQL Server, load the next set of 10,000 records. Do that 600 times. I will see how many records can actually be loaded, perhaps 100,000 at a time would work and do that 60 times. Maybe break the process into threads. One thread watches the ADO recordsets in memory and keeps them filled, another thread performs the merge, another thread writes back to SQL Server. In any case there is simply no need to load all 6 million records at once. To get back to the original point however, while TSQL might be able to do what I need, I have serious doubts. Even if it could, trying to debug this kind of program written in TSQL would be a nightmare for me. Furthermore even TSQL could not do this on 6 million records at once. No sane person claims that any specific language is the best for everything, and for this kind of thing ADO and C# is probably a good choice. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Robert, > > Yes, DataSets are mainly targeted for working with small number of records > in memory - so the following is more a "theoretical" calculation: > From jwcolby at colbyconsulting.com Sat Nov 14 23:11:48 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 15 Nov 2009 00:11:48 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Message-ID: <4AFF8D94.8010409@colbyconsulting.com> http://arstechnica.com/hardware/news/2009/11/biography-solid-state-disk.ars John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Robert, > > Yes, DataSets are mainly targeted for working with small number of records > in memory - so the following is more a "theoretical" calculation: > > - JC has 64bit PC which allow to load in memory "practically unlimited" > volume of data - 64bit logical (process) address space could be as large as > 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 > (http://en.wikipedia.org/wiki/Exbibyte). > > - the speed of data transfer for 64bit Intel's processor - "the 200 MHz > McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers > 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); > > ----- > So "theoretically" with modern speedy harddisks JC can load 50 millions of > records into memory within minutes - within half a minute in the near future > when large enough flash-memory disks will become available and relatively > inexpensive? > > To load such large data volume it would be better to use SqlDataReader as it > keeps loaded data in a very compact form. And 50 million records long > SqlDataReader should be probably better split into several chunks... > > Again - this is just a "theoretical" consideration - in practice JC can > process his 50 million records long data table in chunks as he has a hash > field (tblHashPK), which can be used to load related records into memory: I > mean he can split tblHashPK table keeping hash values into several groups, > e.g.: > > 1 - 100,000 > 100,001 - 200,000 > ... > > and process each group joining its records to source 50 million records... > > -- > Shamil From jwcolby at colbyconsulting.com Mon Nov 16 07:42:52 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 16 Nov 2009 08:42:52 -0500 Subject: [dba-VB] Archives down Message-ID: <4B0156DC.5000407@colbyconsulting.com> I am unable to get at the archives for DBA-VB. Does anyone here use the ADO.Net Entity Framework? Why? Does anyone here specifically AVOID using the ADO.Net Entity Framework? Why? Thanks, -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Mon Nov 16 07:59:09 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 16 Nov 2009 16:59:09 +0300 Subject: [dba-VB] Archives down In-Reply-To: <4B0156DC.5000407@colbyconsulting.com> References: <4B0156DC.5000407@colbyconsulting.com> Message-ID: <00d901ca66c4$f94089e0$ebc19da0$@spb.ru> Hi John -- ADO.NET EF is a great thing. But maybe it will be wise to wait to use it in production GUI when VS2010 will be released then it will be possible to have natively bound WinForms and WPF Forms (and ASP.NET controls? And SilverLight controls), and where ADO.NET EF entities' mapping to db objects will be more flexible, and where it will be possible to have entity model driven(?) (database model) development... I have recently posted here a link to the following presentation: "ORM, EDM, ESQL, Entity Framework, LINQ to SQL, LINQ to Entities - Confused?" http://www.infoq.com/presentations/ORM-LINQ-Entity-Framework-Eric-Nelson I suppose it will answer most of your questions. BTW, our http://northwind.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=26600 project has a couple of ADO.NET EF samples - SPJWinFormsForEF_VS2008.zip source code, 838K, uploaded Jul 23 Application SPJWinFormsForEF_Executables.zip application, 203K, uploaded Jul 23 Thank you. -- Shamil P.S. BTW, I have mainly skipped "LINQ to SQL" - and I'm now happy (:)) I will not need to rewrite my applications as "LINQ to SQL" development is stopped there at MS AFAIU... -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, November 16, 2009 4:43 PM To: VBA Subject: [dba-VB] Archives down I am unable to get at the archives for DBA-VB. Does anyone here use the ADO.Net Entity Framework? Why? Does anyone here specifically AVOID using the ADO.Net Entity Framework? Why? Thanks, -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4611 (20091116) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From Gustav at cactus.dk Mon Nov 16 09:24:01 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 16 Nov 2009 16:24:01 +0100 Subject: [dba-VB] Archives down Message-ID: Hi John I like the concept and the design features, but have only used it for experiments and testing. It takes some time to get around it - I haven't concluded the tour yet ... /gustav >>> jwcolby at colbyconsulting.com 16-11-2009 14:42 >>> I am unable to get at the archives for DBA-VB. Does anyone here use the ADO.Net Entity Framework? Why? Does anyone here specifically AVOID using the ADO.Net Entity Framework? Why? Thanks, -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Tue Nov 17 17:04:12 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Tue, 17 Nov 2009 18:04:12 -0500 Subject: [dba-VB] First little applet Message-ID: <4B032BEC.9050508@colbyconsulting.com> Today I wrote my first C# applet. I have to import CSV files coming back from a processing house into SQL Server. These files had errors in one or more lines in the files. Given that there are tens of millions of lines in each file it isn't possible to just poke around and find the errors. So I wrote an applet that opens the CSV file, reads the header line and counts the " and , characters. Then line by line the applet reads the rest of the lines, comparing the counts to that header line counts. This found lines in the file to be imported where " characters had been dropped, which was causing the import wizard to error out at that point. So read, compare counts, write good lines to output file, write bad lines to an error file. Simple stuff really but it is my first real work done in C#. The speed of this kind of thing is pretty darned good too I thought. I was getting about 300K lines / second read / written. One file had 20.6 million lines, the other had 49 million lines. In the past I would have done this in VBA / Access. Of course it took me a tad longer to write in C# than it would have in VBA. -- John W. Colby www.ColbyConsulting.com From marklbreen at gmail.com Wed Nov 18 02:54:19 2009 From: marklbreen at gmail.com (Mark Breen) Date: Wed, 18 Nov 2009 08:54:19 +0000 Subject: [dba-VB] First little applet In-Reply-To: <4B032BEC.9050508@colbyconsulting.com> References: <4B032BEC.9050508@colbyconsulting.com> Message-ID: Congrats to you John, I suspect once you get a few more utils in your library, you will be flying from here on. good for you, Mark 2009/11/17 jwcolby > Today I wrote my first C# applet. I have to import CSV files coming back > from a processing house > into SQL Server. These files had errors in one or more lines in the files. > Given that there are > tens of millions of lines in each file it isn't possible to just poke > around and find the errors. > > So I wrote an applet that opens the CSV file, reads the header line and > counts the " and , > characters. Then line by line the applet reads the rest of the lines, > comparing the counts to that > header line counts. > > This found lines in the file to be imported where " characters had been > dropped, which was causing > the import wizard to error out at that point. > > So read, compare counts, write good lines to output file, write bad lines > to an error file. > > Simple stuff really but it is my first real work done in C#. > > The speed of this kind of thing is pretty darned good too I thought. I was > getting about 300K lines > / second read / written. One file had 20.6 million lines, the other had 49 > million lines. > > In the past I would have done this in VBA / Access. Of course it took me a > tad longer to write in > C# than it would have in VBA. > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Wed Nov 18 03:11:43 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 18 Nov 2009 10:11:43 +0100 Subject: [dba-VB] First little applet Message-ID: Hi John Great! The good thing is that code from this little applet can easily be incorporated in another project. /gustav >>> jwcolby at colbyconsulting.com 18-11-2009 00:04 >>> Today I wrote my first C# applet. I have to import CSV files coming back from a processing house into SQL Server. These files had errors in one or more lines in the files. Given that there are tens of millions of lines in each file it isn't possible to just poke around and find the errors. So I wrote an applet that opens the CSV file, reads the header line and counts the " and , characters. Then line by line the applet reads the rest of the lines, comparing the counts to that header line counts. This found lines in the file to be imported where " characters had been dropped, which was causing the import wizard to error out at that point. So read, compare counts, write good lines to output file, write bad lines to an error file. Simple stuff really but it is my first real work done in C#. The speed of this kind of thing is pretty darned good too I thought. I was getting about 300K lines / second read / written. One file had 20.6 million lines, the other had 49 million lines. In the past I would have done this in VBA / Access. Of course it took me a tad longer to write in C# than it would have in VBA. -- John W. Colby www.ColbyConsulting.com From andy at minstersystems.co.uk Wed Nov 18 05:23:43 2009 From: andy at minstersystems.co.uk (Andy Lacey) Date: Wed, 18 Nov 2009 12:23:43 +0100 Subject: [dba-VB] First little applet Message-ID: <200911181223.nAICNl4W018133@databaseadvisors.com> Congratulations John. Hope father and little applet are both doing well. Andy --------- Original Message -------- From: "Discussion concerning Visual Basic and related programming issues." To: "Discussion concerning Visual Basic and related programming issues." Subject: Re: [dba-VB] First little applet Date: 18/11/09 08:55 Congrats to you John, I suspect once you get a few more utils in your library, you will be flying from here on. good for you, Mark 2009/11/17 jwcolby > Today I wrote my first C# applet. I have to import CSV files coming back > from a processing house > into SQL Server. These files had errors in one or more lines in the files. > Given that there are > tens of millions of lines in each file it isn't possible to just poke > around and find the errors. > > So I wrote an applet that opens the CSV file, reads the header line and > counts the " and , > characters. Then line by line the applet reads the rest of the lines, > comparing the counts to that > header line counts. > > This found lines in the file to be imported where " characters had been > dropped, which was causing > the import wizard to error out at that point. > > So read, compare counts, write good lines to output file, write bad lines > to an error file. > > Simple stuff really but it is my first real work done in C#. > > The speed of this kind of thing is pretty darned good too I thought. I was > getting about 300K lines > / second read / written. One file had 20.6 million lines, the other had 49 > million lines. > > In the past I would have done this in VBA / Access. Of course it took me a > tad longer to write in > C# than it would have in VBA. > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com ________________________________________________ Message sent using UebiMiau 2.7.2 From jwcolby at colbyconsulting.com Wed Nov 18 06:45:16 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 07:45:16 -0500 Subject: [dba-VB] First little applet In-Reply-To: <200911181223.nAICNl4W018133@databaseadvisors.com> References: <200911181223.nAICNl4W018133@databaseadvisors.com> Message-ID: <4B03EC5C.4090101@colbyconsulting.com> ROTFL. Father is so proud of little applet. ;) John W. Colby www.ColbyConsulting.com Andy Lacey wrote: > Congratulations John. Hope father and little applet are both doing well. > > Andy > > > --------- Original Message -------- > From: "Discussion concerning Visual Basic and related programming issues." > > To: "Discussion concerning Visual Basic and related programming issues." > > Subject: Re: [dba-VB] First little applet > Date: 18/11/09 08:55 > > > Congrats to you John, > > I suspect once you get a few more utils in your library, you will be flying > from here on. > > good for you, > > Mark > > > > 2009/11/17 jwcolby > >> Today I wrote my first C# applet. I have to import CSV files coming back >> from a processing house >> into SQL Server. These files had errors in one or more lines in the files. >> Given that there are >> tens of millions of lines in each file it isn't possible to just poke >> around and find the errors. >> >> So I wrote an applet that opens the CSV file, reads the header line and >> counts the " and , >> characters. Then line by line the applet reads the rest of the lines, >> comparing the counts to that >> header line counts. >> >> This found lines in the file to be imported where " characters had been >> dropped, which was causing >> the import wizard to error out at that point. >> >> So read, compare counts, write good lines to output file, write bad lines >> to an error file. >> >> Simple stuff really but it is my first real work done in C#. >> >> The speed of this kind of thing is pretty darned good too I thought. I was >> getting about 300K lines >> / second read / written. One file had 20.6 million lines, the other had 49 >> million lines. >> >> In the past I would have done this in VBA / Access. Of course it took me a >> tad longer to write in >> C# than it would have in VBA. >> >> -- >> John W. Colby >> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > ________________________________________________ > Message sent using UebiMiau 2.7.2 > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Wed Nov 18 06:50:01 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 07:50:01 -0500 Subject: [dba-VB] SPAM-LOW: Re: First little applet In-Reply-To: References: Message-ID: <4B03ED79.5020704@colbyconsulting.com> I do imports from CSV for large files all of the time, and I have had similar failures in the past, so I will be keeping this applet for future use. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > Great! The good thing is that code from this little applet can easily be incorporated in another project. > > /gustav > > >>>> jwcolby at colbyconsulting.com 18-11-2009 00:04 >>> > Today I wrote my first C# applet. I have to import CSV files coming back from a processing house > into SQL Server. These files had errors in one or more lines in the files. Given that there are > tens of millions of lines in each file it isn't possible to just poke around and find the errors. > > So I wrote an applet that opens the CSV file, reads the header line and counts the " and , > characters. Then line by line the applet reads the rest of the lines, comparing the counts to that > header line counts. > > This found lines in the file to be imported where " characters had been dropped, which was causing > the import wizard to error out at that point. > > So read, compare counts, write good lines to output file, write bad lines to an error file. > > Simple stuff really but it is my first real work done in C#. > > The speed of this kind of thing is pretty darned good too I thought. I was getting about 300K lines > / second read / written. One file had 20.6 million lines, the other had 49 million lines. > > In the past I would have done this in VBA / Access. Of course it took me a tad longer to write in > C# than it would have in VBA. > From jwcolby at colbyconsulting.com Wed Nov 18 07:06:44 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 08:06:44 -0500 Subject: [dba-VB] Projects vs Solutions Message-ID: <4B03F164.6090601@colbyconsulting.com> I am hazy on how things are organized in .Net once you start building pieces of code. What is a project, what is a solution and how are they tied together. My role for the "DBFH" client involves lots of different tasks, for which there will be many different pieces. I would hope to tie the pieces together in one big application. For example I just wrote the little applet to cleanup CSV files, which will have to be used again. I do exports to Accuzip, imports from Accuzip, these two are closely related obviously. I build orders. I have to export data from tables for processing by external processing houses, which involves a very similar export / zip / FTP process. So as I build these processes in C#, how are they physically organized on the disk, and how are they tied back together into one program. From what I am reading, a Solution is the big picture, a Project is a small part of the Solution. But when I open VS2008 there is no "create a solution" or at least I don't see one. So how does one create a solution, and then look at just one project at a time inside of that solution? -- John W. Colby www.ColbyConsulting.com From Gustav at cactus.dk Wed Nov 18 07:17:08 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 18 Nov 2009 14:17:08 +0100 Subject: [dba-VB] Projects vs Solutions Message-ID: Hi John On the New Project pane at the bottom select the name for the (new) solution and mark Create directory for solution. That folder holds your projects of this solution. All projects of a solution are listed in the treeview (typically the right panel). Open only those projects you wish to. /gustav >>> jwcolby at colbyconsulting.com 18-11-2009 14:06 >>> I am hazy on how things are organized in .Net once you start building pieces of code. What is a project, what is a solution and how are they tied together. My role for the "DBFH" client involves lots of different tasks, for which there will be many different pieces. I would hope to tie the pieces together in one big application. For example I just wrote the little applet to cleanup CSV files, which will have to be used again. I do exports to Accuzip, imports from Accuzip, these two are closely related obviously. I build orders. I have to export data from tables for processing by external processing houses, which involves a very similar export / zip / FTP process. So as I build these processes in C#, how are they physically organized on the disk, and how are they tied back together into one program. From what I am reading, a Solution is the big picture, a Project is a small part of the Solution. But when I open VS2008 there is no "create a solution" or at least I don't see one. So how does one create a solution, and then look at just one project at a time inside of that solution? -- John W. Colby www.ColbyConsulting.com From cfoust at infostatsystems.com Wed Nov 18 10:12:41 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 10:12:41 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B03F164.6090601@colbyconsulting.com> References: <4B03F164.6090601@colbyconsulting.com> Message-ID: Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 5:07 AM To: VBA Subject: [dba-VB] Projects vs Solutions I am hazy on how things are organized in .Net once you start building pieces of code. What is a project, what is a solution and how are they tied together. My role for the "DBFH" client involves lots of different tasks, for which there will be many different pieces. I would hope to tie the pieces together in one big application. For example I just wrote the little applet to cleanup CSV files, which will have to be used again. I do exports to Accuzip, imports from Accuzip, these two are closely related obviously. I build orders. I have to export data from tables for processing by external processing houses, which involves a very similar export / zip / FTP process. So as I build these processes in C#, how are they physically organized on the disk, and how are they tied back together into one program. From what I am reading, a Solution is the big picture, a Project is a small part of the Solution. But when I open VS2008 there is no "create a solution" or at least I don't see one. So how does one create a solution, and then look at just one project at a time inside of that solution? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Wed Nov 18 11:57:56 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 12:57:56 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: <4B03F164.6090601@colbyconsulting.com> Message-ID: <4B0435A4.3050804@colbyconsulting.com> OK, this immediately causes issues referencing objects in another project. For example I have a "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I then need to be able to open a form in the other project where I gather information about the file to fix. The objects in that other project aren't visible in intellisense. "Normal" opening of a form is done with something like MyFormName Somename = new MyFormNname(); SomeName.Show(); Well... MyFormName is not valid in project main. While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. > > Charlotte Foust > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, November 18, 2009 5:07 AM > To: VBA > Subject: [dba-VB] Projects vs Solutions From cfoust at infostatsystems.com Wed Nov 18 12:16:55 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 12:16:55 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B0435A4.3050804@colbyconsulting.com> References: <4B03F164.6090601@colbyconsulting.com> <4B0435A4.3050804@colbyconsulting.com> Message-ID: That's where references and "Imports" come in. You need to be careful about project level references to avoid circular messes, so our UI and Data projects have references to the Configuration project, but not the other way around, and the UI has a reference to the Data project but not the other way around. You can always fully reference a class in another project in code starting with the Namespace, so if I need to use something in a project that isn't referenced in the current one, I can use Infostat.RIMDrill.Configuration.RIMConstants (for example) and have access to the methods and properties of RIMConstants even without a reference in the current project. Alternatively, I could use Imports Infostat.RIMDrill.Configuration in the declarations above my current class and get the same thing. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 9:58 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions OK, this immediately causes issues referencing objects in another project. For example I have a "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I then need to be able to open a form in the other project where I gather information about the file to fix. The objects in that other project aren't visible in intellisense. "Normal" opening of a form is done with something like MyFormName Somename = new MyFormNname(); SomeName.Show(); Well... MyFormName is not valid in project main. While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. > > Charlotte Foust > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, November 18, 2009 5:07 AM > To: VBA > Subject: [dba-VB] Projects vs Solutions _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From Gustav at cactus.dk Wed Nov 18 12:27:23 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 18 Nov 2009 19:27:23 +0100 Subject: [dba-VB] Projects vs Solutions Message-ID: Hi John In addition to the comments from Charlotte: Also, you can either add in your project Using ProjectOther; or state this explicitly ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); someName.Show(); /gustav >>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> OK, this immediately causes issues referencing objects in another project. For example I have a "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I then need to be able to open a form in the other project where I gather information about the file to fix. The objects in that other project aren't visible in intellisense. "Normal" opening of a form is done with something like MyFormName Somename = new MyFormNname(); SomeName.Show(); Well... MyFormName is not valid in project main. While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. > > Charlotte Foust From jwcolby at colbyconsulting.com Wed Nov 18 12:54:47 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 13:54:47 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: Message-ID: <4B0442F7.2070805@colbyconsulting.com> Using projOther doesn't work. When doing this is the project allowed to be a part of the solution? Is it even SUPPOSED TO BE a part of the solution. IOW are you supposed to create projects as separate entities and then reference them from other projects, or can you create SolutionMain / ProjMain / projOther1 / projOther2 etc. I have the latter situation (now). When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. This whole area seems poorly documented and rather messy. Lots of chatter on Google about how difficult it is to move a project and so forth. This is the kind of stuff I hate to even get involved in. I am trying to get work done but get sucked off into areas where it certainly looks like I could easily wreak havoc on working code. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > In addition to the comments from Charlotte: > > Also, you can either add in your project > > Using ProjectOther; > > or state this explicitly > > ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); > someName.Show(); > > /gustav > >>>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> > OK, this immediately causes issues referencing objects in another project. For example I have a > "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I > then need to be able to open a form in the other project where I gather information about the file > to fix. > > The objects in that other project aren't visible in intellisense. > > "Normal" opening of a form is done with something like > > MyFormName Somename = new MyFormNname(); > SomeName.Show(); > > Well... MyFormName is not valid in project main. > > While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. >> >> Charlotte Foust > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From cfoust at infostatsystems.com Wed Nov 18 13:03:55 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 13:03:55 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B0442F7.2070805@colbyconsulting.com> References: <4B0442F7.2070805@colbyconsulting.com> Message-ID: A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 10:55 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions Using projOther doesn't work. When doing this is the project allowed to be a part of the solution? Is it even SUPPOSED TO BE a part of the solution. IOW are you supposed to create projects as separate entities and then reference them from other projects, or can you create SolutionMain / ProjMain / projOther1 / projOther2 etc. I have the latter situation (now). When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. This whole area seems poorly documented and rather messy. Lots of chatter on Google about how difficult it is to move a project and so forth. This is the kind of stuff I hate to even get involved in. I am trying to get work done but get sucked off into areas where it certainly looks like I could easily wreak havoc on working code. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > In addition to the comments from Charlotte: > > Also, you can either add in your project > > Using ProjectOther; > > or state this explicitly > > ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); > someName.Show(); > > /gustav > >>>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> > OK, this immediately causes issues referencing objects in another > project. For example I have a "main" project, and a "CSVRepair" > project. The Main project form opens when I run the project but I > then need to be able to open a form in the other project where I gather information about the file to fix. > > The objects in that other project aren't visible in intellisense. > > "Normal" opening of a form is done with something like > > MyFormName Somename = new MyFormNname(); SomeName.Show(); > > Well... MyFormName is not valid in project main. > > While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. >> >> Charlotte Foust > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From Gustav at cactus.dk Wed Nov 18 13:30:04 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 18 Nov 2009 20:30:04 +0100 Subject: [dba-VB] Projects vs Solutions Message-ID: Hi John Sorry, I read it as if ProjectOther was in the same (current) solution. If not, as Charlotte explains, then in the current solution reference the project from the other solution where it is situated. You will be given the choice to either leave the files of the project or to import them. What to chose is a question of maintenance. /gustav >>> jwcolby at colbyconsulting.com 18-11-2009 19:54 >>> Using projOther doesn't work. When doing this is the project allowed to be a part of the solution? Is it even SUPPOSED TO BE a part of the solution. IOW are you supposed to create projects as separate entities and then reference them from other projects, or can you create SolutionMain / ProjMain / projOther1 / projOther2 etc. I have the latter situation (now). When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. This whole area seems poorly documented and rather messy. Lots of chatter on Google about how difficult it is to move a project and so forth. This is the kind of stuff I hate to even get involved in. I am trying to get work done but get sucked off into areas where it certainly looks like I could easily wreak havoc on working code. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > In addition to the comments from Charlotte: > > Also, you can either add in your project > > Using ProjectOther; > > or state this explicitly > > ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); > someName.Show(); > > /gustav > >>>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> > OK, this immediately causes issues referencing objects in another project. For example I have a > "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I > then need to be able to open a form in the other project where I gather information about the file > to fix. > > The objects in that other project aren't visible in intellisense. > > "Normal" opening of a form is done with something like > > MyFormName Somename = new MyFormNname(); > SomeName.Show(); > > Well... MyFormName is not valid in project main. > > While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. >> >> Charlotte Foust From shamil at smsconsulting.spb.ru Wed Nov 18 13:35:05 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Wed, 18 Nov 2009 22:35:05 +0300 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B0442F7.2070805@colbyconsulting.com> References: <4B0442F7.2070805@colbyconsulting.com> Message-ID: <000501ca6886$3c5fd420$b51f7c60$@spb.ru> Hi John -- In fact referencing other projects works very well in .NET, and it's clean and very useful: just note that 'using' directive is to refer to *namespaces* not to projects. If you right-click a project in Solution Explorer, and then select 'Properties' you can find that every project has 'Default Namespace', which is usually the same as project name but this is not necessary. When I'm developing my solution and I need "to get work done" I do sometimes postpone creating new projects within a solution - I just create subfolders in my main project's treeview and I put my code to be later moved to other projects into those subfolders, and I keep namespace for those subfolders different from the default project's namespace - check it out... You'll soon get fluent with that namespaces and projects stuff and then you'll be "cooking" new projects seamlessly - it's usual story to have 10+ projects within .NET solutions... <<< When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. >>> Do you have 'ProjOther' within your solution? Did you set reference to 'ProjOther'? Does 'ProjOther''s default namespace is 'ProjOther'? 'using' directive should be started with lowercase letter.... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 9:55 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions Using projOther doesn't work. When doing this is the project allowed to be a part of the solution? Is it even SUPPOSED TO BE a part of the solution. IOW are you supposed to create projects as separate entities and then reference them from other projects, or can you create SolutionMain / ProjMain / projOther1 / projOther2 etc. I have the latter situation (now). When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. This whole area seems poorly documented and rather messy. Lots of chatter on Google about how difficult it is to move a project and so forth. This is the kind of stuff I hate to even get involved in. I am trying to get work done but get sucked off into areas where it certainly looks like I could easily wreak havoc on working code. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > In addition to the comments from Charlotte: > > Also, you can either add in your project > > Using ProjectOther; > > or state this explicitly > > ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); > someName.Show(); > > /gustav > >>>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> > OK, this immediately causes issues referencing objects in another project. For example I have a > "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I > then need to be able to open a form in the other project where I gather information about the file > to fix. > > The objects in that other project aren't visible in intellisense. > > "Normal" opening of a form is done with something like > > MyFormName Somename = new MyFormNname(); > SomeName.Show(); > > Well... MyFormName is not valid in project main. > > While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. >> >> Charlotte Foust > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4618 (20091118) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4618 (20091118) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Wed Nov 18 13:40:52 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 14:40:52 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: <4B0442F7.2070805@colbyconsulting.com> Message-ID: <4B044DC4.2050304@colbyconsulting.com> OK, let's back up a minute. I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". Except that: a) I really want the Solution to be InfoEngine. b) I would PREFER that the other projects be physically stored under InfoEngine. c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. d) Each of these "projects" contains a solution file. So what is it, a solution or a project? Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. > > Charlotte Foust From shamil at smsconsulting.spb.ru Wed Nov 18 14:04:58 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Wed, 18 Nov 2009 23:04:58 +0300 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B044DC4.2050304@colbyconsulting.com> References: <4B0442F7.2070805@colbyconsulting.com> <4B044DC4.2050304@colbyconsulting.com> Message-ID: <000c01ca688a$68c16200$3a442600$@spb.ru> John -- Just make an experiment with a test solution from scratch: - start VS; - create C# Console project - TestConsole - that will create a project TestConsole and a solution - TestConsole; - File->New->Project... - create C# Class Library project - MyClassLib1; - File->New->Project... - create C# Class Library project - MyClassLib2; - ... - Right-Click TestConsole project in Solution Explorer -> Add Reference -> [Projects] (third tab) -> Select MyClassLib1; - ... - now in TestConsole Program.cs you can type: using MyClassLib1; and you'll have Class1 (from MyClassLib1 - MyclassLib1.Class1) available to Intellisense.... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 10:41 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions OK, let's back up a minute. I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". Except that: a) I really want the Solution to be InfoEngine. b) I would PREFER that the other projects be physically stored under InfoEngine. c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. d) Each of these "projects" contains a solution file. So what is it, a solution or a project? Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. > > Charlotte Foust __________ Information from ESET NOD32 Antivirus, version of virus signature database 4618 (20091118) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From cfoust at infostatsystems.com Wed Nov 18 14:49:22 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 14:49:22 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B044DC4.2050304@colbyconsulting.com> References: <4B0442F7.2070805@colbyconsulting.com> <4B044DC4.2050304@colbyconsulting.com> Message-ID: John, Susan and I ran into this when we wrote a beginner's article. In tools -> Options under Project and Solutions, make sure the Always Show Solution checkbox is checked. Otherwise, you only see a solution and you keep creating separate solutions for each project. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 11:41 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions OK, let's back up a minute. I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". Except that: a) I really want the Solution to be InfoEngine. b) I would PREFER that the other projects be physically stored under InfoEngine. c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. d) Each of these "projects" contains a solution file. So what is it, a solution or a project? Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. > > Charlotte Foust _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From cfoust at infostatsystems.com Wed Nov 18 18:21:50 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 18:21:50 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: <4B0442F7.2070805@colbyconsulting.com> <4B044DC4.2050304@colbyconsulting.com> Message-ID: Oops, I meant you only see a PROJECT unles the Always Show Solution checkbox is checked! Sorry!! Charlotte -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Charlotte Foust Sent: Wednesday, November 18, 2009 12:49 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions John, Susan and I ran into this when we wrote a beginner's article. In tools -> Options under Project and Solutions, make sure the Always Show Solution checkbox is checked. Otherwise, you only see a solution and you keep creating separate solutions for each project. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 11:41 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions OK, let's back up a minute. I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". Except that: a) I really want the Solution to be InfoEngine. b) I would PREFER that the other projects be physically stored under InfoEngine. c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. d) Each of these "projects" contains a solution file. So what is it, a solution or a project? Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. > > Charlotte Foust _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Wed Nov 18 20:14:37 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 21:14:37 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: <4B0442F7.2070805@colbyconsulting.com> <4B044DC4.2050304@colbyconsulting.com> Message-ID: <4B04AA0D.5030107@colbyconsulting.com> Creating a reference to the other project solved the immediate problem of referencing objects in the other project. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > Oops, I meant you only see a PROJECT unles the Always Show Solution checkbox is checked! Sorry!! > > Charlotte > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Charlotte Foust > Sent: Wednesday, November 18, 2009 12:49 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Projects vs Solutions > > John, > > Susan and I ran into this when we wrote a beginner's article. In tools -> Options under Project and Solutions, make sure the Always Show Solution checkbox is checked. Otherwise, you only see a solution and you keep creating separate solutions for each project. > > Charlotte Foust > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, November 18, 2009 11:41 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Projects vs Solutions > > OK, let's back up a minute. > > I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. > > Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". > > Except that: > > a) I really want the Solution to be InfoEngine. > b) I would PREFER that the other projects be physically stored under InfoEngine. > c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. > d) Each of these "projects" contains a solution file. So what is it, a solution or a project? > > Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? > Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? > > By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". > > And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. >> >> Charlotte Foust > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Thu Nov 19 03:41:50 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Thu, 19 Nov 2009 10:41:50 +0100 Subject: [dba-VB] Projects vs Solutions Message-ID: Hi John This raises the question if your coming applet will be the new little applet or will become a little bigger little applet from day one. Does a little applet grow up? /gustav >>> jwcolby at colbyconsulting.com 19-11-2009 03:14 >>> Creating a reference to the other project solved the immediate problem of referencing objects in the other project. John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Thu Nov 19 07:10:03 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 19 Nov 2009 08:10:03 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: Message-ID: <4B0543AB.5080700@colbyconsulting.com> Gustav, Well, the point is that I am doing my job and suddenly I have to do something for the client that I have no solution for. This latest was a good example. I tried to import three files coming back from a processing vendor and one imported, the other two failed. So I write an applet that scans through the file sorting lines of the file into two output files, good and bad. When I am done the remaining "good" file now imports. What do I do with this applet? One thing I can do is to just leave it where it is and hope I don't lose it, because sure as the sun rises I will need to do this again. Or do I create a bigger application, build a menu in it, and place this applet on the menu? Does this Fixit applet grow? Conceivably, if I find some file that isn't fixed (or problem found) as easily as these files were, then this applet could grow to include different tests. This applet has its own form with a set of text boxes to display paths / files, buttons to select folders and files and a button to perform the cleanup. I probably will add another button to open the resulting "bad" file in an editor so that I can see and fix the problems in the data. Perhaps a button to append the fixed lines back to the end of the "good" file before the import. Stuff like that. So applets can grow, and the main application grows as I write new applets to perform various processing tasks, and those new applets get added into the main application. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > This raises the question if your coming applet will be the new little applet or will become a little bigger little applet from day one. Does a little applet grow up? > > /gustav > > >>>> jwcolby at colbyconsulting.com 19-11-2009 03:14 >>> > Creating a reference to the other project solved the immediate problem of referencing objects in the > other project. > > John W. Colby > www.ColbyConsulting.com From marklbreen at gmail.com Fri Nov 20 03:05:11 2009 From: marklbreen at gmail.com (Mark Breen) Date: Fri, 20 Nov 2009 09:05:11 +0000 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B0543AB.5080700@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> Message-ID: Hello John, While I am not qualified to comment really on this subject, I am waiting / hoping that someone will let you now how you can move 90% of your applet to a DLL project, and then from your main program you will just reference the DLL and consume the functionality within that DLL. With that approach, you could start your library of the new Colby C# Framework, albeit a humble beginning. Am I correct in assuming that with this approach you would compile your applet to to a DLL, and then you main program, which could be a winform or console app, could call applet.InputFilePathSet, applet.OutPutFileSet, and then applet.ParseFile etc etc. Let I say, I am not qualified to advise on this, but perhaps some others would like to, thanks Mark 2009/11/19 jwcolby > Gustav, > > Well, the point is that I am doing my job and suddenly I have to do > something for the client that I > have no solution for. This latest was a good example. I tried to import > three files coming back > from a processing vendor and one imported, the other two failed. So I > write an applet that scans > through the file sorting lines of the file into two output files, good and > bad. When I am done the > remaining "good" file now imports. > > What do I do with this applet? One thing I can do is to just leave it > where it is and hope I don't > lose it, because sure as the sun rises I will need to do this again. Or do > I create a bigger > application, build a menu in it, and place this applet on the menu? > > Does this Fixit applet grow? Conceivably, if I find some file that isn't > fixed (or problem found) > as easily as these files were, then this applet could grow to include > different tests. This applet > has its own form with a set of text boxes to display paths / files, buttons > to select folders and > files and a button to perform the cleanup. I probably will add another > button to open the resulting > "bad" file in an editor so that I can see and fix the problems in the data. > Perhaps a button to > append the fixed lines back to the end of the "good" file before the > import. Stuff like that. > > So applets can grow, and the main application grows as I write new applets > to perform various > processing tasks, and those new applets get added into the main > application. > > John W. Colby > www.ColbyConsulting.com > > > Gustav Brock wrote: > > Hi John > > > > This raises the question if your coming applet will be the new little > applet or will become a little bigger little applet from day one. Does a > little applet grow up? > > > > /gustav > > > > > >>>> jwcolby at colbyconsulting.com 19-11-2009 03:14 >>> > > Creating a reference to the other project solved the immediate problem of > referencing objects in the > > other project. > > > > John W. Colby > > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 20 06:26:10 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 20 Nov 2009 07:26:10 -0500 Subject: [dba-VB] SPAM-LOW: Re: Projects vs Solutions In-Reply-To: References: <4B0543AB.5080700@colbyconsulting.com> Message-ID: <4B068AE2.9060301@colbyconsulting.com> The blind leading the blind here. 1) I built a main application 2) I referenced the existing file repair applet from the main application (project). 3) I set a using statement. It appears that you have to both reference it and then use the "using" statement. 4) I can now open forms out in the file repair applet from the main application. 5) I physically moved the file repair applet underneath the main application directory. 6) I changed the directory for the applet and it just worked. That was fairly easy. From this point on I "Add Project" to the main solution. I have added a class project to wrap the DMO. In case you haven't discovered it, the DMO is a real cool SQL Server Management Object API that allows you to see and manage database objects. I am just getting into it but it allows me to reference a server object, then see the database collection. The each database object has a table collection, the table object has a fields collection etc. Everything you can see and manage in the SQL Server management studio you can (apparently) see and manage from the SMO from C#. An example of what this does for me is allows me to see all of the databases in a server, and thus populate a combo with their names. Selecting a database from the combo I can see and fill a combo with the names of the tables. Selecting a database and a specific table I can then can then run my stored procedures that export that table in that database to CSV files. That kind of stuff is what I do a lot of and what the big application will manage. John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > While I am not qualified to comment really on this subject, I am waiting / > hoping that someone will let you now how you can move 90% of your applet to > a DLL project, and then from your main program you will just reference the > DLL and consume the functionality within that DLL. > > With that approach, you could start your library of the new Colby C# > Framework, albeit a humble beginning. > > Am I correct in assuming that with this approach you would compile your > applet to to a DLL, and then you main program, which could be a winform or > console app, could call applet.InputFilePathSet, applet.OutPutFileSet, and > then applet.ParseFile etc etc. > > Let I say, I am not qualified to advise on this, but perhaps some others > would like to, > > thanks > > Mark > > > 2009/11/19 jwcolby > >> Gustav, >> >> Well, the point is that I am doing my job and suddenly I have to do >> something for the client that I >> have no solution for. This latest was a good example. I tried to import >> three files coming back >> from a processing vendor and one imported, the other two failed. So I >> write an applet that scans >> through the file sorting lines of the file into two output files, good and >> bad. When I am done the >> remaining "good" file now imports. >> >> What do I do with this applet? One thing I can do is to just leave it >> where it is and hope I don't >> lose it, because sure as the sun rises I will need to do this again. Or do >> I create a bigger >> application, build a menu in it, and place this applet on the menu? >> >> Does this Fixit applet grow? Conceivably, if I find some file that isn't >> fixed (or problem found) >> as easily as these files were, then this applet could grow to include >> different tests. This applet >> has its own form with a set of text boxes to display paths / files, buttons >> to select folders and >> files and a button to perform the cleanup. I probably will add another >> button to open the resulting >> "bad" file in an editor so that I can see and fix the problems in the data. >> Perhaps a button to >> append the fixed lines back to the end of the "good" file before the >> import. Stuff like that. >> >> So applets can grow, and the main application grows as I write new applets >> to perform various >> processing tasks, and those new applets get added into the main >> application. >> >> John W. Colby >> www.ColbyConsulting.com >> >> >> Gustav Brock wrote: >>> Hi John >>> >>> This raises the question if your coming applet will be the new little >> applet or will become a little bigger little applet from day one. Does a >> little applet grow up? >>> /gustav >>> >>> >>>>>> jwcolby at colbyconsulting.com 19-11-2009 03:14 >>> >>> Creating a reference to the other project solved the immediate problem of >> referencing objects in the >>> other project. >>> >>> John W. Colby >>> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 20 09:35:42 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 20 Nov 2009 10:35:42 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B068AE2.9060301@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> Message-ID: <4B06B74E.9080909@colbyconsulting.com> The object I am referring to is the SMO or SQL Server Management object. In order to use it you have to add several references: Microsoft.SQLServer.SMO Microsoft.SQLServer.SMOEnum Microsoft.SQLServer.SQLEnum Microsoft.SQLServer.ConnectionInfo then in the classes using the SMO you have to do using Microsoft.SqlServer.Management.Smo; After that you can do things like: Server Svr; Svr = new Server("MyServerName") foreach (Database in Svr.Databases) { //Etc. // } This allows you to iterate collections of database objects, using them directly or just pulling the names out (as I did) to populate lists, combos, collections etc. As I mentioned, once you have a database object you can manipulate it. I am just starting to learn what I can do with this API but it looks pretty powerful. John W. Colby www.ColbyConsulting.com jwcolby wrote: > The blind leading the blind here. > > 1) I built a main application > 2) I referenced the existing file repair applet from the main application (project). > 3) I set a using statement. It appears that you have to both reference it and then use the "using" > statement. > 4) I can now open forms out in the file repair applet from the main application. > 5) I physically moved the file repair applet underneath the main application directory. > 6) I changed the directory for the applet and it just worked. That was fairly easy. > > From this point on I "Add Project" to the main solution. I have added a class project to wrap the > DMO. In case you haven't discovered it, the DMO is a real cool SQL Server Management Object API > that allows you to see and manage database objects. I am just getting into it but it allows me to > reference a server object, then see the database collection. The each database object has a table > collection, the table object has a fields collection etc. Everything you can see and manage in the > SQL Server management studio you can (apparently) see and manage from the SMO from C#. > > An example of what this does for me is allows me to see all of the databases in a server, and thus > populate a combo with their names. Selecting a database from the combo I can see and fill a combo > with the names of the tables. Selecting a database and a specific table I can then can then run my > stored procedures that export that table in that database to CSV files. > > That kind of stuff is what I do a lot of and what the big application will manage. > > John W. Colby > www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Fri Nov 20 14:18:40 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 20 Nov 2009 23:18:40 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B06B74E.9080909@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> Message-ID: <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> Hi John -- I'm wondering what's the use of that Microsoft.SQLServer.* when you have to have your customer tasks done first of all? Why not just use (static) custom settings to point to different SQL servers etc.? I suppose Microsoft.SQLServer.* is good for companies like http://www.red-gate.com/ for them to develop their tools used worldwide, and I wonder what customers' business tasks can be solved by using Microsoft.SQLServer.* ? Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 20, 2009 6:36 PM To: Discussion concerning Visual Basic and related programming issues. Subject: [dba-VB] SMO was Projects vs Solutions The object I am referring to is the SMO or SQL Server Management object. In order to use it you have to add several references: Microsoft.SQLServer.SMO Microsoft.SQLServer.SMOEnum Microsoft.SQLServer.SQLEnum Microsoft.SQLServer.ConnectionInfo then in the classes using the SMO you have to do using Microsoft.SqlServer.Management.Smo; After that you can do things like: Server Svr; Svr = new Server("MyServerName") foreach (Database in Svr.Databases) { //Etc. // } This allows you to iterate collections of database objects, using them directly or just pulling the names out (as I did) to populate lists, combos, collections etc. As I mentioned, once you have a database object you can manipulate it. I am just starting to learn what I can do with this API but it looks pretty powerful. John W. Colby www.ColbyConsulting.com jwcolby wrote: > The blind leading the blind here. > > 1) I built a main application > 2) I referenced the existing file repair applet from the main application (project). > 3) I set a using statement. It appears that you have to both reference it and then use the "using" > statement. > 4) I can now open forms out in the file repair applet from the main application. > 5) I physically moved the file repair applet underneath the main application directory. > 6) I changed the directory for the applet and it just worked. That was fairly easy. > > From this point on I "Add Project" to the main solution. I have added a class project to wrap the > DMO. In case you haven't discovered it, the DMO is a real cool SQL Server Management Object API > that allows you to see and manage database objects. I am just getting into it but it allows me to > reference a server object, then see the database collection. The each database object has a table > collection, the table object has a fields collection etc. Everything you can see and manage in the > SQL Server management studio you can (apparently) see and manage from the SMO from C#. > > An example of what this does for me is allows me to see all of the databases in a server, and thus > populate a combo with their names. Selecting a database from the combo I can see and fill a combo > with the names of the tables. Selecting a database and a specific table I can then can then run my > stored procedures that export that table in that database to CSV files. > > That kind of stuff is what I do a lot of and what the big application will manage. > > John W. Colby > www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4624 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4625 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 20 14:30:47 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 20 Nov 2009 15:30:47 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> Message-ID: <4B06FC77.6040008@colbyconsulting.com> I don't understand the question. What do you mean by Microsoft.SQLServer.* I do have multiple servers running SQL Server. The default is Azul but I may need to reference Stonehenge. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > I'm wondering what's the use of that Microsoft.SQLServer.* when you have to > have your customer tasks done first of all? > Why not just use (static) custom settings to point to different SQL servers > etc.? > > I suppose Microsoft.SQLServer.* is good for companies like > http://www.red-gate.com/ for them to develop their tools used worldwide, and > I wonder what customers' business tasks can be solved by using > Microsoft.SQLServer.* ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 20, 2009 6:36 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: [dba-VB] SMO was Projects vs Solutions > > The object I am referring to is the SMO or SQL Server Management object. In > order to use it you > have to add several references: > > Microsoft.SQLServer.SMO > Microsoft.SQLServer.SMOEnum > Microsoft.SQLServer.SQLEnum > Microsoft.SQLServer.ConnectionInfo > > then in the classes using the SMO you have to do > > using Microsoft.SqlServer.Management.Smo; > > After that you can do things like: > > Server Svr; > Svr = new Server("MyServerName") > > foreach (Database in Svr.Databases) > { > //Etc. > // > } > > This allows you to iterate collections of database objects, using them > directly or just pulling the > names out (as I did) to populate lists, combos, collections etc. > > As I mentioned, once you have a database object you can manipulate it. I am > just starting to learn > what I can do with this API but it looks pretty powerful. > > John W. Colby > www.ColbyConsulting.com > > > jwcolby wrote: >> The blind leading the blind here. >> >> 1) I built a main application >> 2) I referenced the existing file repair applet from the main application > (project). >> 3) I set a using statement. It appears that you have to both reference it > and then use the "using" >> statement. >> 4) I can now open forms out in the file repair applet from the main > application. >> 5) I physically moved the file repair applet underneath the main > application directory. >> 6) I changed the directory for the applet and it just worked. That was > fairly easy. >> From this point on I "Add Project" to the main solution. I have added a > class project to wrap the >> DMO. In case you haven't discovered it, the DMO is a real cool SQL Server > Management Object API >> that allows you to see and manage database objects. I am just getting > into it but it allows me to >> reference a server object, then see the database collection. The each > database object has a table >> collection, the table object has a fields collection etc. Everything you > can see and manage in the >> SQL Server management studio you can (apparently) see and manage from the > SMO from C#. >> An example of what this does for me is allows me to see all of the > databases in a server, and thus >> populate a combo with their names. Selecting a database from the combo I > can see and fill a combo >> with the names of the tables. Selecting a database and a specific table I > can then can then run my >> stored procedures that export that table in that database to CSV files. >> >> That kind of stuff is what I do a lot of and what the big application will > manage. >> John W. Colby >> www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4624 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4625 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Nov 20 14:50:35 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 20 Nov 2009 23:50:35 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B06FC77.6040008@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> Message-ID: <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> Hi John, By Microsoft.SQLServer.* I mean all the namespaces you mentioned > Microsoft.SQLServer.SMO > Microsoft.SQLServer.SMOEnum > Microsoft.SQLServer.SQLEnum > Microsoft.SQLServer.ConnectionInfo As well as their classes and those classes enumerations, methods, properties, events etc. <<< I do have multiple servers running SQL Server. The default is Azul but I may need to reference Stonehenge. >>> But do you need to change the reference(/connection string information) to SQL Server after you have your program started? Do you need to enumerate SQL Server objects to solve your customers' tasks? Or maybe you have developed a kind of code generator, which does need to enumerate MS SQL Server objects and some of their properties to generate some custom code? - I can understand the latter - if you have to develop a lot of repetitive custom code then it's often useful to generate it... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 20, 2009 11:31 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SMO was Projects vs Solutions I don't understand the question. What do you mean by Microsoft.SQLServer.* I do have multiple servers running SQL Server. The default is Azul but I may need to reference Stonehenge. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > I'm wondering what's the use of that Microsoft.SQLServer.* when you have to > have your customer tasks done first of all? > Why not just use (static) custom settings to point to different SQL servers > etc.? > > I suppose Microsoft.SQLServer.* is good for companies like > http://www.red-gate.com/ for them to develop their tools used worldwide, and > I wonder what customers' business tasks can be solved by using > Microsoft.SQLServer.* ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 20, 2009 6:36 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: [dba-VB] SMO was Projects vs Solutions > > The object I am referring to is the SMO or SQL Server Management object. In > order to use it you > have to add several references: > > Microsoft.SQLServer.SMO > Microsoft.SQLServer.SMOEnum > Microsoft.SQLServer.SQLEnum > Microsoft.SQLServer.ConnectionInfo > > then in the classes using the SMO you have to do > > using Microsoft.SqlServer.Management.Smo; > > After that you can do things like: > > Server Svr; > Svr = new Server("MyServerName") > > foreach (Database in Svr.Databases) > { > //Etc. > // > } > > This allows you to iterate collections of database objects, using them > directly or just pulling the > names out (as I did) to populate lists, combos, collections etc. > > As I mentioned, once you have a database object you can manipulate it. I am > just starting to learn > what I can do with this API but it looks pretty powerful. > > John W. Colby > www.ColbyConsulting.com > > > jwcolby wrote: >> The blind leading the blind here. >> >> 1) I built a main application >> 2) I referenced the existing file repair applet from the main application > (project). >> 3) I set a using statement. It appears that you have to both reference it > and then use the "using" >> statement. >> 4) I can now open forms out in the file repair applet from the main > application. >> 5) I physically moved the file repair applet underneath the main > application directory. >> 6) I changed the directory for the applet and it just worked. That was > fairly easy. >> From this point on I "Add Project" to the main solution. I have added a > class project to wrap the >> DMO. In case you haven't discovered it, the DMO is a real cool SQL Server > Management Object API >> that allows you to see and manage database objects. I am just getting > into it but it allows me to >> reference a server object, then see the database collection. The each > database object has a table >> collection, the table object has a fields collection etc. Everything you > can see and manage in the >> SQL Server management studio you can (apparently) see and manage from the > SMO from C#. >> An example of what this does for me is allows me to see all of the > databases in a server, and thus >> populate a combo with their names. Selecting a database from the combo I > can see and fill a combo >> with the names of the tables. Selecting a database and a specific table I > can then can then run my >> stored procedures that export that table in that database to CSV files. >> >> That kind of stuff is what I do a lot of and what the big application will > manage. >> John W. Colby >> www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4625 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 20 20:05:57 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 20 Nov 2009 21:05:57 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> Message-ID: <4B074B05.6000406@colbyconsulting.com> That is used for manipulating the SMO object and SQL Server. I am just learning this stuff and do what the demos tell me to do. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > By Microsoft.SQLServer.* I mean all the namespaces you mentioned > >> Microsoft.SQLServer.SMO >> Microsoft.SQLServer.SMOEnum >> Microsoft.SQLServer.SQLEnum >> Microsoft.SQLServer.ConnectionInfo > > As well as their classes and those classes enumerations, methods, > properties, events etc. > > <<< > I do have multiple servers running SQL Server. > The default is Azul but I may need to reference > Stonehenge. > But do you need to change the reference(/connection string information) to > SQL Server after you have your program started? > Do you need to enumerate SQL Server objects to solve your customers' tasks? > Or maybe you have developed a kind of code generator, which does need to > enumerate MS SQL Server objects and some of their properties to generate > some custom code? - I can understand the latter - if you have to develop a > lot of repetitive custom code then it's often useful to generate it... > > Thank you. > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 20, 2009 11:31 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > I don't understand the question. > > What do you mean by Microsoft.SQLServer.* > > I do have multiple servers running SQL Server. The default is Azul but I > may need to reference > Stonehenge. > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> I'm wondering what's the use of that Microsoft.SQLServer.* when you have > to >> have your customer tasks done first of all? >> Why not just use (static) custom settings to point to different SQL > servers >> etc.? >> >> I suppose Microsoft.SQLServer.* is good for companies like >> http://www.red-gate.com/ for them to develop their tools used worldwide, > and >> I wonder what customers' business tasks can be solved by using >> Microsoft.SQLServer.* ? >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: Friday, November 20, 2009 6:36 PM >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: [dba-VB] SMO was Projects vs Solutions >> >> The object I am referring to is the SMO or SQL Server Management object. > In >> order to use it you >> have to add several references: >> >> Microsoft.SQLServer.SMO >> Microsoft.SQLServer.SMOEnum >> Microsoft.SQLServer.SQLEnum >> Microsoft.SQLServer.ConnectionInfo >> >> then in the classes using the SMO you have to do >> >> using Microsoft.SqlServer.Management.Smo; >> >> After that you can do things like: >> >> Server Svr; >> Svr = new Server("MyServerName") >> >> foreach (Database in Svr.Databases) >> { >> //Etc. >> // >> } >> >> This allows you to iterate collections of database objects, using them >> directly or just pulling the >> names out (as I did) to populate lists, combos, collections etc. >> >> As I mentioned, once you have a database object you can manipulate it. I > am >> just starting to learn >> what I can do with this API but it looks pretty powerful. >> >> John W. Colby >> www.ColbyConsulting.com >> >> >> jwcolby wrote: >>> The blind leading the blind here. >>> >>> 1) I built a main application >>> 2) I referenced the existing file repair applet from the main application >> (project). >>> 3) I set a using statement. It appears that you have to both reference > it >> and then use the "using" >>> statement. >>> 4) I can now open forms out in the file repair applet from the main >> application. >>> 5) I physically moved the file repair applet underneath the main >> application directory. >>> 6) I changed the directory for the applet and it just worked. That was >> fairly easy. >>> From this point on I "Add Project" to the main solution. I have added a >> class project to wrap the >>> DMO. In case you haven't discovered it, the DMO is a real cool SQL > Server >> Management Object API >>> that allows you to see and manage database objects. I am just getting >> into it but it allows me to >>> reference a server object, then see the database collection. The each >> database object has a table >>> collection, the table object has a fields collection etc. Everything you >> can see and manage in the >>> SQL Server management studio you can (apparently) see and manage from the >> SMO from C#. >>> An example of what this does for me is allows me to see all of the >> databases in a server, and thus >>> populate a combo with their names. Selecting a database from the combo I >> can see and fill a combo >>> with the names of the tables. Selecting a database and a specific table > I >> can then can then run my >>> stored procedures that export that table in that database to CSV files. >>> >>> That kind of stuff is what I do a lot of and what the big application > will >> manage. >>> John W. Colby >>> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4625 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Nov 21 08:58:31 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 21 Nov 2009 17:58:31 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B074B05.6000406@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> Message-ID: <002001ca6abb$186807e0$493817a0$@spb.ru> Hi John -- Yes, I understand that. I'm wondering what for do you learning this stuff? - Just out of curiosity or do you plan to implement some custom applications using Microsoft.SQLServer.* ? Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 21, 2009 5:06 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SMO was Projects vs Solutions That is used for manipulating the SMO object and SQL Server. I am just learning this stuff and do what the demos tell me to do. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > By Microsoft.SQLServer.* I mean all the namespaces you mentioned > >> Microsoft.SQLServer.SMO >> Microsoft.SQLServer.SMOEnum >> Microsoft.SQLServer.SQLEnum >> Microsoft.SQLServer.ConnectionInfo > > As well as their classes and those classes enumerations, methods, > properties, events etc. > > <<< > I do have multiple servers running SQL Server. > The default is Azul but I may need to reference > Stonehenge. > But do you need to change the reference(/connection string information) to > SQL Server after you have your program started? > Do you need to enumerate SQL Server objects to solve your customers' tasks? > Or maybe you have developed a kind of code generator, which does need to > enumerate MS SQL Server objects and some of their properties to generate > some custom code? - I can understand the latter - if you have to develop a > lot of repetitive custom code then it's often useful to generate it... > > Thank you. > > -- > Shamil > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4626 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 21 09:42:12 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 10:42:12 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <002001ca6abb$186807e0$493817a0$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> Message-ID: <4B080A54.4030609@colbyconsulting.com> > or do you plan to implement some custom applications using Microsoft.SQLServer.* ? Exactly. My work for one of my clients is heavy manipulation of SQL Server. I routinely export files to an external application running on virtual machines on another server, then wait for the files to process, then import them back in to SQL server. These two processes together take about 10 stored procedures. The export / import has to occur on (currently) a half dozen large databases monthly. However as I obtain more lists that number goes up. Each process exports to it's own directory path, then imports back from a parallel directory path. While the stored procedures eliminate much of the manual labor, there is plenty left. Plus logging of everything etc. I also routinely process orders. These orders take about a dozen stored procedures and also use the exact same set of stored procedures as the above process, i.e. the export / import process is also used on each and every order. So there is just a ton of very repetitive work that has to be done, but which database / table varies from run to run. To this point I have used Access / VBA to automate these tasks. Now I am moving to C# for this automation work. SMO allows me to manipulate SQL Server from C# and so it is an API that I will use to assist me in getting this program working. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I understand that. > I'm wondering what for do you learning this stuff? - Just out of curiosity > or do you plan to implement some custom applications using > Microsoft.SQLServer.* ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 21, 2009 5:06 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > That is used for manipulating the SMO object and SQL Server. > > I am just learning this stuff and do what the demos tell me to do. > > John W. Colby > www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sat Nov 21 10:27:15 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 21 Nov 2009 19:27:15 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B080A54.4030609@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> Message-ID: <002601ca6ac7$7dcfb180$796f1480$@spb.ru> Hi John -- OK, but why not just use connections strings and System.Data.SqlClient classes, .... ? Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 21, 2009 6:42 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SMO was Projects vs Solutions > or do you plan to implement some custom applications using Microsoft.SQLServer.* ? Exactly. My work for one of my clients is heavy manipulation of SQL Server. I routinely export files to an external application running on virtual machines on another server, then wait for the files to process, then import them back in to SQL server. These two processes together take about 10 stored procedures. The export / import has to occur on (currently) a half dozen large databases monthly. However as I obtain more lists that number goes up. Each process exports to it's own directory path, then imports back from a parallel directory path. While the stored procedures eliminate much of the manual labor, there is plenty left. Plus logging of everything etc. I also routinely process orders. These orders take about a dozen stored procedures and also use the exact same set of stored procedures as the above process, i.e. the export / import process is also used on each and every order. So there is just a ton of very repetitive work that has to be done, but which database / table varies from run to run. To this point I have used Access / VBA to automate these tasks. Now I am moving to C# for this automation work. SMO allows me to manipulate SQL Server from C# and so it is an API that I will use to assist me in getting this program working. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I understand that. > I'm wondering what for do you learning this stuff? - Just out of curiosity > or do you plan to implement some custom applications using > Microsoft.SQLServer.* ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 21, 2009 5:06 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > That is used for manipulating the SMO object and SQL Server. > > I am just learning this stuff and do what the demos tell me to do. > > John W. Colby > www.ColbyConsulting.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4626 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 21 11:05:43 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 12:05:43 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <002601ca6ac7$7dcfb180$796f1480$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> <002601ca6ac7$7dcfb180$796f1480$@spb.ru> Message-ID: <4B081DE7.7060203@colbyconsulting.com> Shamil, As far as I can tell they are quite different things, and perform different jobs. I do use SQLClient. SQLClient appears to be about getting at data, and I do use that for the ADO side of things. http://msdn.microsoft.com/en-us/library/system.data.sqlclient.aspx SMO appears to be about getting at and manipulating the objects in the database. http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.aspx SMO appears to be similar to the DAO object where you can see and manipulate the table (not the data IN the table) the fields, indexes and so forth. It is a collection based object where you can drill down into a server and see the objects underneath the database, then (for example) drill down into a table and see the objects under the table. Just as an example, using SMO in a few lines of code I got a list of all of the databases in the database collection of my server. I used that to populate a database combo box. When I select the database that I want to export data from, I then use SMO (again a couple of more lines of code) to get a list of the tables in the selected database and use that to populate a combo of tables. I have no idea the relative merit of one way vs the other. To me it is just a tool. In this particular case I use it to quickly and easily get at the names of objects in the database. I understand that using SMO you can do other maintenance kinds of things as well. One of the things I have to do is copy an "order template" database to a new name in preparing to fill an order. SMO appears to have built-in methods for doing this programmatically from C# - as opposed to running TSQL or using some other method. Understand I am not an expert on any of this, in fact quite the opposite. I stumbled across SMO and saw an example of how easy it was to get at the database STRUCTURE information and decided to use it for that purpose, when I had that need. Is there another way to do this? Almost certainly, given that there is always a dozen ways to do anything in programming. Given your persistent questioning of my motives I guess now I have to ask whether I am causing myself problems doing this? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > OK, but why not just use connections strings and System.Data.SqlClient > classes, .... ? > > Thank you. From jwcolby at colbyconsulting.com Sat Nov 21 11:40:33 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 12:40:33 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <002601ca6ac7$7dcfb180$796f1480$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> <002601ca6ac7$7dcfb180$796f1480$@spb.ru> Message-ID: <4B082611.4040401@colbyconsulting.com> Shamil, By get an instance of the Server object you can see a collection of Database objects. The following is what you can do with the database object. So I can manipulate a given database as shown below. Several of those things look potentially useful, such as running TSQL statements, truncating log files and so forth. http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.database.aspx However the database object has a tables collection. I can get lists of the names of tables, select a specific table etc. http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.table.aspx http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.table_methods.aspx The point is that it is just a programming model that is available if you want to learn it, for manipulating a database from C#. I will almost certainly never use 99.9% of it, just as most people (myself included) rarely use 99.9% of the DAO object model. OTOH the .1% that I do use is pretty darned cool. You can find a lot of little articles about using SMO out on Google. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > OK, but why not just use connections strings and System.Data.SqlClient > classes, .... ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 21, 2009 6:42 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > > or do you plan to implement some custom applications using > Microsoft.SQLServer.* ? > > Exactly. My work for one of my clients is heavy manipulation of SQL Server. > I routinely export > files to an external application running on virtual machines on another > server, then wait for the > files to process, then import them back in to SQL server. These two > processes together take about > 10 stored procedures. The export / import has to occur on (currently) a > half dozen large databases > monthly. However as I obtain more lists that number goes up. Each process > exports to it's own > directory path, then imports back from a parallel directory path. While the > stored procedures > eliminate much of the manual labor, there is plenty left. Plus logging of > everything etc. > > I also routinely process orders. These orders take about a dozen stored > procedures and also use the > exact same set of stored procedures as the above process, i.e. the export / > import process is also > used on each and every order. > > So there is just a ton of very repetitive work that has to be done, but > which database / table > varies from run to run. To this point I have used Access / VBA to automate > these tasks. Now I am > moving to C# for this automation work. > > SMO allows me to manipulate SQL Server from C# and so it is an API that I > will use to assist me in > getting this program working. > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> Yes, I understand that. >> I'm wondering what for do you learning this stuff? - Just out of curiosity >> or do you plan to implement some custom applications using >> Microsoft.SQLServer.* ? >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: Saturday, November 21, 2009 5:06 AM >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] SMO was Projects vs Solutions >> >> That is used for manipulating the SMO object and SQL Server. >> >> I am just learning this stuff and do what the demos tell me to do. >> >> John W. Colby >> www.ColbyConsulting.com > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4626 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Nov 21 11:47:27 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 12:47:27 -0500 Subject: [dba-VB] Loss leader (or just plain loss) Message-ID: <4B0827AF.6040608@colbyconsulting.com> A friend has approached me about building an application for a small chain of stores to manage inventory. "Very simple". I haven't yet talked to her about the requirements. She did say they were quoted (and rejected) a bid of $4000 to do the job. If I am going to do this at all it will have to be very quick and dirty, and would be just to help a friend. Is Access 2007 runtime stable? Easy or difficult to use / install / implement on site? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sat Nov 21 11:55:29 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 21 Nov 2009 20:55:29 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B081DE7.7060203@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> <002601ca6ac7$7dcfb180$796f1480$@spb.ru> <4B081DE7.7060203@colbyconsulting.com> Message-ID: <002c01ca6ad3$d1e37930$75aa6b90$@spb.ru> Hi John -- <<< SMO appears to be similar to the DAO object where you can see and manipulate the table... >>> Yes, I understand/know that. <<< When I select the database that I want to export data from, I then use SMO (again a couple of more lines of code) to get a list of the tables in the selected database and use that to populate a combo of tables. >>> Yes, but how often would you need to do that? Why not just use (relatively static) program settings files, or utility program execution command string parameters - I mean you anyway have to do manual selection (in the case you'll use SMO) or manual editing and saving (in the case you'll use program setting files...) - what for to spend time on learning SMO if you very probably (am I wrong?) can solve your customer tasks without using SMO, and applying relatively the same efforts in both cases to implement custom logic (but in the case of using SMO you'll also have to spend time on learning SMO)... <<< Given your persistent questioning of my motives I guess now I have to ask whether I am causing myself problems doing this? >>> Just maybe by spending time on learning SMO when you don't need it to implement required custom logic? Not sure (I didn't check so I can be wrong) but SMO might need special installation procedures for customers' PCs without full MS SQL Version (standard, prof., enterprise) so if you'll develop some custom code to be reused on customers' PC then you might need to be prepared to develop custom setup - not a big issue just be prepared - if I'm not wrong in guessing that that custom setup would be needed for some customers system... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 21, 2009 8:06 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SMO was Projects vs Solutions Shamil, As far as I can tell they are quite different things, and perform different jobs. I do use SQLClient. SQLClient appears to be about getting at data, and I do use that for the ADO side of things. http://msdn.microsoft.com/en-us/library/system.data.sqlclient.aspx SMO appears to be about getting at and manipulating the objects in the database. http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.a spx SMO appears to be similar to the DAO object where you can see and manipulate the table (not the data IN the table) the fields, indexes and so forth. It is a collection based object where you can drill down into a server and see the objects underneath the database, then (for example) drill down into a table and see the objects under the table. Just as an example, using SMO in a few lines of code I got a list of all of the databases in the database collection of my server. I used that to populate a database combo box. When I select the database that I want to export data from, I then use SMO (again a couple of more lines of code) to get a list of the tables in the selected database and use that to populate a combo of tables. I have no idea the relative merit of one way vs the other. To me it is just a tool. In this particular case I use it to quickly and easily get at the names of objects in the database. I understand that using SMO you can do other maintenance kinds of things as well. One of the things I have to do is copy an "order template" database to a new name in preparing to fill an order. SMO appears to have built-in methods for doing this programmatically from C# - as opposed to running TSQL or using some other method. Understand I am not an expert on any of this, in fact quite the opposite. I stumbled across SMO and saw an example of how easy it was to get at the database STRUCTURE information and decided to use it for that purpose, when I had that need. Is there another way to do this? Almost certainly, given that there is always a dozen ways to do anything in programming. Given your persistent questioning of my motives I guess now I have to ask whether I am causing myself problems doing this? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > OK, but why not just use connections strings and System.Data.SqlClient > classes, .... ? > > Thank you. __________ Information from ESET NOD32 Antivirus, version of virus signature database 4626 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From dbdoug at gmail.com Sat Nov 21 12:26:06 2009 From: dbdoug at gmail.com (Doug Steele) Date: Sat, 21 Nov 2009 10:26:06 -0800 Subject: [dba-VB] Loss leader (or just plain loss) In-Reply-To: <4B0827AF.6040608@colbyconsulting.com> References: <4B0827AF.6040608@colbyconsulting.com> Message-ID: <4dd71a0c0911211026y79e4d583t99d6ca1dfb4a10f1@mail.gmail.com> Hi John: I've only ever used the Access 2003 runtime/installer, and that was for one job. Getting the runtime code installed, running, and generating/testing a proper installation package took the best part of a day. It was straightforward enough, just fiddly. After that, updates only consisted of recreating the .mde file and sending it to the client. From what I've read, the Access 2007 runtime is pretty stable now (it started off very badly) and it's the right price. My experience with projects that started out to be 'quick and dirty' has been consistently horrible. Not quick, and very dirty! Doug On Sat, Nov 21, 2009 at 9:47 AM, jwcolby wrote: > A friend has approached me about building an application for a small chain > of stores to manage > inventory. "Very simple". I haven't yet talked to her about the > requirements. She did say they > were quoted (and rejected) a bid of $4000 to do the job. > > If I am going to do this at all it will have to be very quick and dirty, > and would be just to help a > friend. > > Is Access 2007 runtime stable? Easy or difficult to use / install / > implement on site? > > > From shamil at smsconsulting.spb.ru Sat Nov 21 12:40:36 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 21 Nov 2009 21:40:36 +0300 Subject: [dba-VB] Loss leader (or just plain loss) In-Reply-To: <4dd71a0c0911211026y79e4d583t99d6ca1dfb4a10f1@mail.gmail.com> References: <4B0827AF.6040608@colbyconsulting.com> <4dd71a0c0911211026y79e4d583t99d6ca1dfb4a10f1@mail.gmail.com> Message-ID: <002d01ca6ada$1ecb5fa0$5c621ee0$@spb.ru> <<< Not quick, and very dirty! >>> Yes. And a high opportunity to lose a friend... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Doug Steele Sent: Saturday, November 21, 2009 9:26 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Loss leader (or just plain loss) Hi John: I've only ever used the Access 2003 runtime/installer, and that was for one job. Getting the runtime code installed, running, and generating/testing a proper installation package took the best part of a day. It was straightforward enough, just fiddly. After that, updates only consisted of recreating the .mde file and sending it to the client. From what I've read, the Access 2007 runtime is pretty stable now (it started off very badly) and it's the right price. My experience with projects that started out to be 'quick and dirty' has been consistently horrible. Not quick, and very dirty! Doug On Sat, Nov 21, 2009 at 9:47 AM, jwcolby wrote: > A friend has approached me about building an application for a small chain > of stores to manage > inventory. "Very simple". I haven't yet talked to her about the > requirements. She did say they > were quoted (and rejected) a bid of $4000 to do the job. > > If I am going to do this at all it will have to be very quick and dirty, > and would be just to help a > friend. > > Is Access 2007 runtime stable? Easy or difficult to use / install / > implement on site? > > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4626 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 21 13:30:39 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 14:30:39 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <002c01ca6ad3$d1e37930$75aa6b90$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> <002601ca6ac7$7dcfb180$796f1480$@spb.ru> <4B081DE7.7060203@colbyconsulting.com> <002c01ca6ad3$d1e37930$75aa6b90$@spb.ru> Message-ID: <4B083FDF.505@colbyconsulting.com> There is no "customer PC". This is my server in my office, running an application I will write, which I will operate. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > <<< > SMO appears to be similar to the DAO object > where you can see and manipulate the table... > Yes, I understand/know that. > > <<< > When I select the > database that I want to export data from, I then > use SMO (again a couple of more lines of code) to > get a list of the tables in the selected database > and use that to populate a combo of tables. > Yes, but how often would you need to do that? > Why not just use (relatively static) program settings files, or utility > program execution command string parameters - I mean you anyway have to do > manual selection (in the case you'll use SMO) or manual editing and saving > (in the case you'll use program setting files...) - what for to spend time > on learning SMO if you very probably (am I wrong?) can solve your customer > tasks without using SMO, and applying relatively the same efforts in both > cases to implement custom logic (but in the case of using SMO you'll also > have to spend time on learning SMO)... > > <<< > Given your persistent questioning of my motives > I guess now I have to ask whether I am causing > myself problems doing this? > Just maybe by spending time on learning SMO when you don't need it to > implement required custom logic? > > Not sure (I didn't check so I can be wrong) but SMO might need special > installation procedures for customers' PCs without full MS SQL Version > (standard, prof., enterprise) so if you'll develop some custom code to be > reused on customers' PC then you might need to be prepared to develop custom > setup - not a big issue just be prepared - if I'm not wrong in guessing that > that custom setup would be needed for some customers system... > > Thank you. > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 21, 2009 8:06 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > Shamil, > > As far as I can tell they are quite different things, and perform different > jobs. I do use > SQLClient. SQLClient appears to be about getting at data, and I do use that > for the ADO side of > things. > > http://msdn.microsoft.com/en-us/library/system.data.sqlclient.aspx > > SMO appears to be about getting at and manipulating the objects in the > database. > > http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.a > spx > > SMO appears to be similar to the DAO object where you can see and manipulate > the table (not the data > IN the table) the fields, indexes and so forth. It is a collection based > object where you can drill > down into a server and see the objects underneath the database, then (for > example) drill down into a > table and see the objects under the table. > > Just as an example, using SMO in a few lines of code I got a list of all of > the databases in the > database collection of my server. I used that to populate a database combo > box. When I select the > database that I want to export data from, I then use SMO (again a couple of > more lines of code) to > get a list of the tables in the selected database and use that to populate a > combo of tables. > > I have no idea the relative merit of one way vs the other. To me it is just > a tool. In this > particular case I use it to quickly and easily get at the names of objects > in the database. I > understand that using SMO you can do other maintenance kinds of things as > well. One of the things I > have to do is copy an "order template" database to a new name in preparing > to fill an order. SMO > appears to have built-in methods for doing this programmatically from C# - > as opposed to running > TSQL or using some other method. > > Understand I am not an expert on any of this, in fact quite the opposite. I > stumbled across SMO and > saw an example of how easy it was to get at the database STRUCTURE > information and decided to use it > for that purpose, when I had that need. Is there another way to do this? > Almost certainly, given > that there is always a dozen ways to do anything in programming. > > Given your persistent questioning of my motives I guess now I have to ask > whether I am causing > myself problems doing this? > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> OK, but why not just use connections strings and System.Data.SqlClient >> classes, .... ? >> >> Thank you. > > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4626 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sun Nov 22 22:46:38 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 22 Nov 2009 23:46:38 -0500 Subject: [dba-VB] Ya know you're in trouble when... Message-ID: <4B0A13AE.1030404@colbyconsulting.com> http://www.hanselman.com/blog/WhatGreatNETDevelopersOughtToKnowMoreNETInterviewQuestions.aspx -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Mon Nov 23 07:57:41 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 23 Nov 2009 08:57:41 -0500 Subject: [dba-VB] XNA game development Message-ID: <4B0A94D5.3010601@colbyconsulting.com> Have you ever wanted to write your own little windows game. Microsoft provides a free game development framework called XNA. It runs in C# express and everything is free free free. It is designed to get you writing games for the XBox BUT... it can write games for Windows as well, and in fact they are apparently interchangeable, once written they can be played on the XBox (but not for free). The best part is that they have a very basic cannon / flying saucer demo game with videos for how to develop this game, all of the source code, sprites etc. I have stepped through the first few chapters and it really does start at ground zero. They claim you don't even need to know C# to follow the tutorial. As we all know, writing a project that holds your interest is the best way to get involved and learn a language. If writing a game is your interest, this might be just the ticket to getting into C#. http://creators.xna.com/en-US/education/gettingstarted/bg2d/chapter1 -- John W. Colby www.ColbyConsulting.com From mmattys at rochester.rr.com Mon Nov 23 09:41:38 2009 From: mmattys at rochester.rr.com (Mike Mattys) Date: Mon, 23 Nov 2009 10:41:38 -0500 Subject: [dba-VB] Ya know you're in trouble when... References: <4B0A13AE.1030404@colbyconsulting.com> Message-ID: I've already come up against a great many of these Some of them I look forward to getting the answers for. If I went to an interview for all of it, I would hope it to be "open book" - Michael R Mattys MapPoint and Database Dev www.mattysconsulting.com - ----- Original Message ----- From: "jwcolby" To: "VBA" Sent: Sunday, November 22, 2009 11:46 PM Subject: [dba-VB] Ya know you're in trouble when... > > http://www.hanselman.com/blog/WhatGreatNETDevelopersOughtToKnowMoreNETInterviewQuestions.aspx > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > From dwaters at usinternet.com Mon Nov 23 12:22:19 2009 From: dwaters at usinternet.com (Dan Waters) Date: Mon, 23 Nov 2009 12:22:19 -0600 Subject: [dba-VB] Ya know you're in trouble when... In-Reply-To: References: <4B0A13AE.1030404@colbyconsulting.com> Message-ID: Hey! I knew one of these! What's the difference between late binding and early binding? But that's it. :-( Dan -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Mike Mattys Sent: Monday, November 23, 2009 9:42 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Ya know you're in trouble when... I've already come up against a great many of these Some of them I look forward to getting the answers for. If I went to an interview for all of it, I would hope it to be "open book" - Michael R Mattys MapPoint and Database Dev www.mattysconsulting.com - ----- Original Message ----- From: "jwcolby" To: "VBA" Sent: Sunday, November 22, 2009 11:46 PM Subject: [dba-VB] Ya know you're in trouble when... > > http://www.hanselman.com/blog/WhatGreatNETDevelopersOughtToKnowMoreNETInterv iewQuestions.aspx > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Mon Nov 23 15:57:12 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 23 Nov 2009 16:57:12 -0500 Subject: [dba-VB] XNA is darned fun Message-ID: <4B0B0538.1020904@colbyconsulting.com> I have to say that this is darned fun. I have worked through much of the tutorial I posted a link to. I have ships flying and cannonballs firing. Woohoo! Very very cool! I know what I will be doing over the Thanksgiving holiday. -- John W. Colby www.ColbyConsulting.com From michael at ddisolutions.com.au Mon Nov 23 23:38:16 2009 From: michael at ddisolutions.com.au (Michael Maddison) Date: Tue, 24 Nov 2009 16:38:16 +1100 Subject: [dba-VB] HTTPWebRequest blues Message-ID: <59A61174B1F5B54B97FD4ADDE71E7D01582726@ddi-01.DDI.local> Hi guys, I may be clutching at straws here but has anyone done much with HttpWebRequest? My process goes like this, GET /ExpressBet/Standard/Default.aspx?State=2 Store the cookie container for re use GET /ExpressBet/LoginFrame.aspx?State=2&ExpressBetMode=Standard Store viewState for re use in next POST POST /Login/LoginUser.aspx?State=2&ReturnURL=http://www.tab.com.au/ExpressBet/LoginFrame.aspx?State=2&ExpressBetMode=Standard Does login, this succeeds GET /ExpressBet/Aggregator.aspx?State=2&Controls=STD_MAIN&RacingCode=R&MeetingCode=M&FromDate=2009-11-24T00:00:00&RaceNumber=07&BetType=WinPlace&ExpressBetMode=Standard Store viewState for re use GET /Betting/BetTicket/DisplayClientSideBetTicket.aspx?State=2&ExpressBet=true&RacingCode=R&MeetingCode=M&FromDate=2009-11-24T00:00:00&BetType=WinPlace&amount=1.00&Selections=%3csc+id%3d%220%22+f%3d%22false%22+m%3d%220%22+s%3d%228%22+t%3d%22%22+k%3d%2207%22+%2f%3e+ The above redirects to GET /Betting/BetTicket/PlaceRaceBetConfirmation.aspx?State=2&CurrentBet=2901d195-c59a-4c1b-8b0a-c476fc274f06&ExpressBet=True Result looks ok in Fiddler, But response is unreadable in plain text. Anyone know what this is? I think I need to get the viewstate from it somehow. Snipped because of size constraints. _???2??-w?sF??'15??1??_i???i??_l?___???_?__?S??fY?d?R_adZ?Ri??__ 2?#??4+?='+??N!??_n^???????nQ^?`????_?_____?0???_?h??X????_???dvTH??`??[?k?Sh?SDp??[h?c?OH???k?K?{??3??@?'13OW7????0W??8,????r2????_=}A???{?@??? i???)?5__F_?[?S?'4N?__C]}?M?}?Af???s?H??K?-H?? ?u+N ?s?r,? The Last step I need to do is a final POST, This returns an error page, dammit. POST /Betting/BetTicket/PlaceRaceBetConfirmation.aspx?State=2&CurrentBet=2901d195-c59a-4c1b-8b0a-c476fc274f06&ExpressBet=True Redirects to an error.aspx page. The code for the last POST is the same for the successful calls as the failed call. Both are HTTPS, I am handling the certificates. Does this make sense to anyone? J Any ideas? Cheers Michael M From Gustav at cactus.dk Tue Nov 24 04:50:40 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Tue, 24 Nov 2009 11:50:40 +0100 Subject: [dba-VB] This is hot: WCF RIA Services Message-ID: Hi all RIA: Rich Internet Applications Microsoft WCF RIA Services simplifies the traditional n-tier application pattern by bringing together the ASP.NET and Silverlight platforms. RIA Services provides a pattern to write application logic that runs on the mid-tier and controls access to data for queries, changes and custom operations. It also provides end-to-end support for common tasks such as data validation, authentication and roles by integrating with Silverlight components on the client and ASP.NET on the mid-tier Though not explained at a place to find, what it does is automatically to set up for you a web service which your Silverlight frontend communicates with, and to establish the client side code that handles this communication. http://silverlight.net/getstarted/riaservices/ Watch the 1-hour video "Introduction to WCF RIA Services" and combine this with the racing speed at which Silverlight evolves, and it will erase any doubt you still may have about browser-based applications. Note too, that while this works in VS2008 and with Silverlight 3 it is really aimed at VS2010 and Silverlight 4 both currently available as betas. I attended a developer meeting a couple of days ago where we ran a similar walk-through, and it was very convincing. This is exactly what I for years have been looking for. /gustav From fuller.artful at gmail.com Tue Nov 24 10:26:24 2009 From: fuller.artful at gmail.com (Arthur Fuller) Date: Tue, 24 Nov 2009 11:26:24 -0500 Subject: [dba-VB] XNA is darned fun In-Reply-To: <4B0B0538.1020904@colbyconsulting.com> References: <4B0B0538.1020904@colbyconsulting.com> Message-ID: <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> When is American Thanksgiving? We already had ours last month. What is XNA, anyway? Arthur On Mon, Nov 23, 2009 at 4:57 PM, jwcolby wrote: > I have to say that this is darned fun. I have worked through much of the > tutorial I posted a link > to. I have ships flying and cannonballs firing. Woohoo! > > Very very cool! I know what I will be doing over the Thanksgiving holiday. > From cfoust at infostatsystems.com Tue Nov 24 10:48:12 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Tue, 24 Nov 2009 10:48:12 -0600 Subject: [dba-VB] XNA is darned fun In-Reply-To: <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> References: <4B0B0538.1020904@colbyconsulting.com> <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> Message-ID: Ours is Thursday. Charlotte -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Arthur Fuller Sent: Tuesday, November 24, 2009 8:26 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] XNA is darned fun When is American Thanksgiving? We already had ours last month. What is XNA, anyway? Arthur On Mon, Nov 23, 2009 at 4:57 PM, jwcolby wrote: > I have to say that this is darned fun. I have worked through much of > the tutorial I posted a link to. I have ships flying and cannonballs > firing. Woohoo! > > Very very cool! I know what I will be doing over the Thanksgiving holiday. > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Tue Nov 24 10:53:21 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Tue, 24 Nov 2009 11:53:21 -0500 Subject: [dba-VB] XNA is darned fun In-Reply-To: <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> References: <4B0B0538.1020904@colbyconsulting.com> <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> Message-ID: <4B0C0F81.9070806@colbyconsulting.com> > When is American Thanksgiving? We already had ours last month. Not sure exactly. IIRC it is the last Thursday in November so it is not a fixed date. > What is XNA, anyway? It is a Microsoft gaming framework. VERY COOL if you like to (or always wanted to) write a windows game. It runs under C# in Visual Studio, even the express version. The framework provides a game class that has a method for loading the "content" - video, sound, sprites etc. It has a method automatically called X times / second that is then used to perform updates and display the content. And it has a draw method that is called to draw the game. LOTS of game specific classes including math classes, 2d and 3d classes for doing game specific kinds of things, sprite classes, point classes, rectangle classes. You get the picture. REALLY really cool for the child in all of us. If you happen to have an XBox, you can write games that can run on the XBox as well as Windows (XP, Vista and Windows 7). I don't own an XBox so I am just running the game on Windows, but any game CAN be written to run on either, and apparently SENSE which it is running on. You can even use the XBox joystick / button controller on your Windows machine! There is a step by step video series that walks you through building a "space invaders" game, with cannon, cannon balls, UFOs. All graphics provided, all code stepped through in short videos. I did that and I have to say it is really easy. It was so cool as each step comes together, displaying the background, displaying the flying saucers moving across the screen, the cannon swiveling, then cannon balls firing, and finally cannon balls destroying the UFOs. Of course doing it all by myself would be not so easy but it gives a great feeling for what can be done and how it is done. And of course there are after market books on it. I have always wanted to write my own "Empire" replacement, and have even started doing so. I played Empire all of my life (still do) but it is long in the tooth and not updated any more. Plus the multi-player stuff never really worked very well. This will set me back to ground zero on that attempt but it will also provide a framework for doing it the right way. I just don't have enough to do with my time, as I am sure you can understand! John W. Colby www.ColbyConsulting.com Arthur Fuller wrote: > When is American Thanksgiving? We already had ours last month. > > What is XNA, anyway? > > Arthur > > On Mon, Nov 23, 2009 at 4:57 PM, jwcolby wrote: > >> I have to say that this is darned fun. I have worked through much of the >> tutorial I posted a link >> to. I have ships flying and cannonballs firing. Woohoo! >> >> Very very cool! I know what I will be doing over the Thanksgiving holiday. >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Wed Nov 25 08:24:39 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 09:24:39 -0500 Subject: [dba-VB] Part 1: An Introduction to Data Structures Message-ID: <4B0D3E27.9040601@colbyconsulting.com> A quick overview: http://creators.xna.com/en-US/article/datastructures A more detailed overview: http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Wed Nov 25 08:51:50 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Wed, 25 Nov 2009 17:51:50 +0300 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: <4B0D3E27.9040601@colbyconsulting.com> References: <4B0D3E27.9040601@colbyconsulting.com> Message-ID: <006e01ca6dde$d31de700$7959b500$@spb.ru> Hi John -- You might also find interesting: http://en.wikipedia.org/wiki/Microsoft_Robotics_Developer_Studio e.g., for programming: http://en.wikipedia.org/wiki/Lego_Mindstorms_NXT using C#... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 25, 2009 5:25 PM To: VBA; Nikolai Vitsyn; Judy Hawley Subject: [dba-VB] Part 1: An Introduction to Data Structures A quick overview: http://creators.xna.com/en-US/article/datastructures A more detailed overview: http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4636 (20091125) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From Gustav at cactus.dk Wed Nov 25 08:54:57 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 25 Nov 2009 15:54:57 +0100 Subject: [dba-VB] Part 1: An Introduction to Data Structures Message-ID: Hi John Thanks. "collision code" ... I love that. /gustav >>> jwcolby at colbyconsulting.com 25-11-2009 15:24 >>> A quick overview: http://creators.xna.com/en-US/article/datastructures A more detailed overview: http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Wed Nov 25 09:23:22 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 10:23:22 -0500 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: References: Message-ID: <4B0D4BEA.2050603@colbyconsulting.com> > Thanks. "collision code" ... I love that. LOL. Obviously this was aimed at gamers. A collision is when one rectangle (sprite) intersects another rectangle. BTW, check out this video of a game. http://www.youtube.com/watch?v=Lz-TQcdXDJU&feature=related LOTS of collisions going on here. ;) I am going to work on writing this game using the XNA gaming framework. The XNA framework is just awesome, and I am going to get my 8 year old son involved in helping me to write this thing. A fun, interactive way of teaching him C#. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > Thanks. "collision code" ... I love that. > > /gustav > > >>>> jwcolby at colbyconsulting.com 25-11-2009 15:24 >>> > A quick overview: > http://creators.xna.com/en-US/article/datastructures > > A more detailed overview: > http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx From jwcolby at colbyconsulting.com Wed Nov 25 09:24:15 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 10:24:15 -0500 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: <006e01ca6dde$d31de700$7959b500$@spb.ru> References: <4B0D3E27.9040601@colbyconsulting.com> <006e01ca6dde$d31de700$7959b500$@spb.ru> Message-ID: <4B0D4C1F.9050103@colbyconsulting.com> I have this thing. I haven't looked at it yet though. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > You might also find interesting: > > http://en.wikipedia.org/wiki/Microsoft_Robotics_Developer_Studio > > e.g., for programming: > > http://en.wikipedia.org/wiki/Lego_Mindstorms_NXT > > using C#... > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, November 25, 2009 5:25 PM > To: VBA; Nikolai Vitsyn; Judy Hawley > Subject: [dba-VB] Part 1: An Introduction to Data Structures > > A quick overview: > http://creators.xna.com/en-US/article/datastructures > > A more detailed overview: > http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx From stuart at lexacorp.com.pg Wed Nov 25 15:31:48 2009 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Thu, 26 Nov 2009 07:31:48 +1000 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: <4B0D4BEA.2050603@colbyconsulting.com> References: , <4B0D4BEA.2050603@colbyconsulting.com> Message-ID: <4B0DA244.1838.3178D240@stuart.lexacorp.com.pg> Gee you guys are taking me back - I used to write games for the Amiga twenty odd years ago - It all revolved around sprites and collisions. -- Stuart On 25 Nov 2009 at 10:23, jwcolby wrote: > > Thanks. "collision code" ... I love that. > > LOL. Obviously this was aimed at gamers. A collision is when one rectangle (sprite) intersects > another rectangle. > > BTW, check out this video of a game. > > http://www.youtube.com/watch?v=Lz-TQcdXDJU&feature=related > > LOTS of collisions going on here. ;) > > I am going to work on writing this game using the XNA gaming framework. The XNA framework is just > awesome, and I am going to get my 8 year old son involved in helping me to write this thing. A fun, > interactive way of teaching him C#. > > John W. Colby > www.ColbyConsulting.com > > > Gustav Brock wrote: > > Hi John > > > > Thanks. "collision code" ... I love that. > > > > /gustav > > > > > >>>> jwcolby at colbyconsulting.com 25-11-2009 15:24 >>> > > A quick overview: > > http://creators.xna.com/en-US/article/datastructures > > > > A more detailed overview: > > http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > From Gustav at cactus.dk Wed Nov 25 15:50:53 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 25 Nov 2009 22:50:53 +0100 Subject: [dba-VB] Part 1: An Introduction to Data Structures Message-ID: Hi John Oh my, will this never stop? I guess that is what this poor guy thinks, in real trouble as he is ... /gustav >>> jwcolby at colbyconsulting.com 25-11-2009 16:23 >>> > Thanks. "collision code" ... I love that. LOL. Obviously this was aimed at gamers. A collision is when one rectangle (sprite) intersects another rectangle. BTW, check out this video of a game. http://www.youtube.com/watch?v=Lz-TQcdXDJU&feature=related LOTS of collisions going on here. ;) I am going to work on writing this game using the XNA gaming framework. The XNA framework is just awesome, and I am going to get my 8 year old son involved in helping me to write this thing. A fun, interactive way of teaching him C#. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > Thanks. "collision code" ... I love that. > > /gustav > > >>>> jwcolby at colbyconsulting.com 25-11-2009 15:24 >>> > A quick overview: > http://creators.xna.com/en-US/article/datastructures > > A more detailed overview: > http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx From jwcolby at colbyconsulting.com Wed Nov 25 15:52:30 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 16:52:30 -0500 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: <4B0DA244.1838.3178D240@stuart.lexacorp.com.pg> References: , <4B0D4BEA.2050603@colbyconsulting.com> <4B0DA244.1838.3178D240@stuart.lexacorp.com.pg> Message-ID: <4B0DA71E.3030105@colbyconsulting.com> > Gee you guys are taking me back - I used to write games for the Amiga twenty odd years ago - It all revolved around sprites and collisions. LOL. Give it a whirl. I bet you find it much easier in this day and age. One of the reasons I am doing this is that I want to get really good at C#. I do have real work that I am starting to do in C#, but writing games is fun and will keep me in it on a daily / weekly basis whereas my work could be weeks without having to write any C# code. Plus I am hoping to slowly suck my son into the programming. He was up in my office changing the constants that determine how many space ships will be created in the shoot-em-up demo I was studying. He now knows how to edit that constant, how to run it, how to close the game, he knows now that he can't put any number he wants in an integer variable (he tried to run with 20000000000000 spaceships;). We'll see. John W. Colby www.ColbyConsulting.com Stuart McLachlan wrote: > Gee you guys are taking me back - I used to write games for the Amiga twenty odd years > ago - It all revolved around sprites and collisions. > From jwcolby at colbyconsulting.com Wed Nov 25 20:24:52 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 21:24:52 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. Message-ID: <4B0DE6F4.3070605@colbyconsulting.com> If a parameter in a stored procedure is declared as an output (returns a value) will the parameter object in C# / ADO contain that value when the stored procedure finishes? I haven't used output parameters yet in stored procedures. Can values be passed in via an output parameter or can it only be set from inside of the stored procedure? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Thu Nov 26 00:02:40 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 26 Nov 2009 09:02:40 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0DE6F4.3070605@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> Message-ID: <00a101ca6e5e$111e9ac0$335bd040$@spb.ru> Hi John, Yes. Set System.Data.ParameterDirection.Output for .Direction property of output parameters. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 26, 2009 5:25 AM To: VBA Subject: [dba-VB] Parameter objects in C# and ADO. If a parameter in a stored procedure is declared as an output (returns a value) will the parameter object in C# / ADO contain that value when the stored procedure finishes? I haven't used output parameters yet in stored procedures. Can values be passed in via an output parameter or can it only be set from inside of the stored procedure? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4637 (20091125) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4637 (20091125) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From marklbreen at gmail.com Thu Nov 26 03:04:54 2009 From: marklbreen at gmail.com (Mark Breen) Date: Thu, 26 Nov 2009 09:04:54 +0000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0DE6F4.3070605@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> Message-ID: Hello John, There are three ways to bring back values from an sproc, 1 the result set 2 the output parameters, of which there can be many 3 the return code afaik, a param can be input (the default direction) or output. I have not heard of an inputoutput param so unless anyone here says different, assume that you cannot have bi-directional. Now you have the ability to standardize on a return code which may relate to error codes, a complete recordset of data, and also a few params that passed back logging codes, temp counts etc etc etc. In fact, if the sproc is doing multiple stages, you will love the ability to store staged log codes and messages and then bring the whole lot back when it is finished, and still being 1M records as well ! Thanks Mark 2009/11/26 jwcolby > If a parameter in a stored procedure is declared as an output (returns a > value) will the parameter > object in C# / ADO contain that value when the stored procedure finishes? > > I haven't used output parameters yet in stored procedures. Can values be > passed in via an output > parameter or can it only be set from inside of the stored procedure? > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Thu Nov 26 04:26:58 2009 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Thu, 26 Nov 2009 20:26:58 +1000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: <4B0DE6F4.3070605@colbyconsulting.com>, Message-ID: <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> SP with a parameter that is both Input and Output In some situations, there is no need to store the output into an extra variable because the output must replace the input. CREATE PROCEDURE cap2 (@name NVARCHAR(20) OUT ) --Capitalizes first char, changes the rest to lower case --SP with one input/output parameter AS SET @name=UPPER(LEFT(@name,1))+LOWER(RIGHT(@name,LEN(@name)-1)) or for you .net types, see http://fabioscagliola.spaces.live.com/blog/cns!919F8FCDE3DC9AC4!121.entry -- Stuart On 26 Nov 2009 at 9:04, Mark Breen wrote: > > afaik, a param can be input (the default direction) or output. I have not > heard of an inputoutput param so unless anyone here says different, assume > that you cannot have bi-directional. > From shamil at smsconsulting.spb.ru Thu Nov 26 05:03:27 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 26 Nov 2009 14:03:27 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> References: <4B0DE6F4.3070605@colbyconsulting.com>, <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> Message-ID: <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> Hi Mark, > assume that you cannot have bi-directional. You can: System.Data.ParameterDirection.InputOutput -- Shamil On 26 Nov 2009 at 9:04, Mark Breen wrote: > > afaik, a param can be input (the default direction) or output. I have not > heard of an inputoutput param so unless anyone here says different, assume > that you cannot have bi-directional. > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4638 (20091126) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Thu Nov 26 06:55:59 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 26 Nov 2009 07:55:59 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: <4B0DE6F4.3070605@colbyconsulting.com> Message-ID: <4B0E7ADF.5010504@colbyconsulting.com> Thanks for the reply Mark. I am looking forward to having the flexibility of C# to handle errors appropriately. John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > There are three ways to bring back values from an sproc, > > 1 the result set > 2 the output parameters, of which there can be many > 3 the return code > > afaik, a param can be input (the default direction) or output. I have not > heard of an inputoutput param so unless anyone here says different, assume > that you cannot have bi-directional. > > Now you have the ability to standardize on a return code which may relate to > error codes, a complete recordset of data, and also a few params that passed > back logging codes, temp counts etc etc etc. > > In fact, if the sproc is doing multiple stages, you will love the ability to > store staged log codes and messages and then bring the whole lot back when > it is finished, and still being 1M records as well ! > > Thanks > > Mark > > > > > 2009/11/26 jwcolby > >> If a parameter in a stored procedure is declared as an output (returns a >> value) will the parameter >> object in C# / ADO contain that value when the stored procedure finishes? >> >> I haven't used output parameters yet in stored procedures. Can values be >> passed in via an output >> parameter or can it only be set from inside of the stored procedure? >> >> -- >> John W. Colby >> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From marklbreen at gmail.com Thu Nov 26 09:59:22 2009 From: marklbreen at gmail.com (Mark Breen) Date: Thu, 26 Nov 2009 15:59:22 +0000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> Message-ID: Hello Shamil and Stuart, I never knew params could be bi-directional, thanks a lot for that, Mark 2009/11/26 Shamil Salakhetdinov > Hi Mark, > > > assume that you cannot have bi-directional. > You can: > > System.Data.ParameterDirection.InputOutput > > -- > Shamil > > On 26 Nov 2009 at 9:04, Mark Breen wrote: > > > > afaik, a param can be input (the default direction) or output. I have > not > > heard of an inputoutput param so unless anyone here says different, > assume > > that you cannot have bi-directional. > > > > > > > > __________ Information from ESET NOD32 Antivirus, version of virus > signature > database 4638 (20091126) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Thu Nov 26 10:04:07 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 26 Nov 2009 11:04:07 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> References: <4B0DE6F4.3070605@colbyconsulting.com>, <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> Message-ID: <4B0EA6F7.4050600@colbyconsulting.com> Thanks Stuart. John W. Colby www.ColbyConsulting.com Stuart McLachlan wrote: > SP with a parameter that is both Input and Output > > In some situations, there is no need to store the output into an extra variable because the > output must replace the input. > > CREATE PROCEDURE cap2 (@name NVARCHAR(20) OUT ) > --Capitalizes first char, changes the rest to lower case > --SP with one input/output parameter > AS > SET @name=UPPER(LEFT(@name,1))+LOWER(RIGHT(@name,LEN(@name)-1)) > > or for you .net types, see > http://fabioscagliola.spaces.live.com/blog/cns!919F8FCDE3DC9AC4!121.entry > From jwcolby at colbyconsulting.com Thu Nov 26 10:05:27 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 26 Nov 2009 11:05:27 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> References: <4B0DE6F4.3070605@colbyconsulting.com>, <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> Message-ID: <4B0EA747.4020506@colbyconsulting.com> > System.Data.ParameterDirection.InputOutput Thanks Shamil, What do you do with this? Set it to true? Set something else equal to this (is this a constant)? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Mark, > >> assume that you cannot have bi-directional. > You can: > > System.Data.ParameterDirection.InputOutput > > -- > Shamil > > On 26 Nov 2009 at 9:04, Mark Breen wrote: >> afaik, a param can be input (the default direction) or output. I have > not >> heard of an inputoutput param so unless anyone here says different, assume >> that you cannot have bi-directional. >> > > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4638 (20091126) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Thu Nov 26 10:54:43 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 26 Nov 2009 19:54:43 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0EA747.4020506@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com>, <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> <4B0EA747.4020506@colbyconsulting.com> Message-ID: <00de01ca6eb9$287b8830$79729890$@spb.ru> Hi John -- This is enumeration value - have a look here how to use it (near to the bottom): http://stackoverflow.com/questions/1637646/need-a-return-value-from-a-stored -procedure-in-c -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 26, 2009 7:05 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. > System.Data.ParameterDirection.InputOutput Thanks Shamil, What do you do with this? Set it to true? Set something else equal to this (is this a constant)? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Mark, > >> assume that you cannot have bi-directional. > You can: > > System.Data.ParameterDirection.InputOutput > > -- > Shamil > > On 26 Nov 2009 at 9:04, Mark Breen wrote: >> afaik, a param can be input (the default direction) or output. I have > not >> heard of an inputoutput param so unless anyone here says different, assume >> that you cannot have bi-directional. >> > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4639 (20091126) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From marklbreen at gmail.com Fri Nov 27 04:00:01 2009 From: marklbreen at gmail.com (Mark Breen) Date: Fri, 27 Nov 2009 10:00:01 +0000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0EA747.4020506@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> <4B0EA747.4020506@colbyconsulting.com> Message-ID: Hello John, that Shamil has said is that you can use one param to pass in a value, and use the same param to accept back a value, which may be different to when it was passes in. You can pass in ProductId, and get back, ProductCode, or pass in username and get back fullname, or pass in product code and get back zero if the product code does not exist, else you get back the code you passed in or pass in productid and get back qty in stock, all with one param. My examples are not great, but there are plenty of times where you need to somehow inspect a value and it may not be ideal to have two params named paramOriginal paramNewValue with inputout you can have just one param named paramToBeVerified Thanks again to Shamil, Stuart and Mr Colby for asking the question. Mark 2009/11/26 jwcolby > > System.Data.ParameterDirection.InputOutput > > Thanks Shamil, > > What do you do with this? Set it to true? Set something else equal to > this (is this a constant)? > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: > > Hi Mark, > > > >> assume that you cannot have bi-directional. > > You can: > > > > System.Data.ParameterDirection.InputOutput > > > > -- > > Shamil > > > > On 26 Nov 2009 at 9:04, Mark Breen wrote: > >> afaik, a param can be input (the default direction) or output. I have > > not > >> heard of an inputoutput param so unless anyone here says different, > assume > >> that you cannot have bi-directional. > >> > > > > > > > > > > > > __________ Information from ESET NOD32 Antivirus, version of virus > signature > > database 4638 (20091126) __________ > > > > The message was checked by ESET NOD32 Antivirus. > > > > http://www.esetnod32.ru > > > > > > _______________________________________________ > > dba-VB mailing list > > dba-VB at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > http://www.databaseadvisors.com > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Fri Nov 27 04:44:46 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Fri, 27 Nov 2009 11:44:46 +0100 Subject: [dba-VB] Parameter objects in C# and ADO. Message-ID: Hi Mark et al What's the big deal? Are parameters costly? Hardly in money, but in resources? /gustav >>> marklbreen at gmail.com 27-11-2009 11:00 >>> Hello John, that Shamil has said is that you can use one param to pass in a value, and use the same param to accept back a value, which may be different to when it was passes in. You can pass in ProductId, and get back, ProductCode, or pass in username and get back fullname, or pass in product code and get back zero if the product code does not exist, else you get back the code you passed in or pass in productid and get back qty in stock, all with one param. My examples are not great, but there are plenty of times where you need to somehow inspect a value and it may not be ideal to have two params named paramOriginal paramNewValue with inputout you can have just one param named paramToBeVerified Thanks again to Shamil, Stuart and Mr Colby for asking the question. Mark From shamil at smsconsulting.spb.ru Fri Nov 27 06:43:24 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 27 Nov 2009 15:43:24 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: Message-ID: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> Hi Gustav at all, BTW, have you ever seen the following below error handling in CRUD SPs as I have found in one of my customers' databases? Isn't an overkill? Or do you use even more detailed/elaborated error handling in your CRUD SPs? How do you organize error handling in your CRUD SPs? (I usually just return error code in RETURN statement). CREATE procedure [dbo].[GettblUserByUsername] @Username varchar(20), @ErrorDesc varchar(100) output, @ErrorNo int output, @RowsAffected int output AS DECLARE @lErrorNo INTEGER DECLARE @lRowsAffected INTEGER Begin SELECT tblUser.tblUserId, tblUser.tblUserGroupId, tblUserGroup.groupCode as UserGroup, tblUser.Username, tblUser.tblUserLevelId FROM tblUser left outer join tblUserGroup on tblUser.tblUserGroupId = tblUserGroup.tblUserGroupId WHERE tblUser.Username = @username and tblUser.Authorised = 1 select @lErrorNo = @@ERROR, @lRowsAffected = @@ROWCOUNT IF @lErrorNo > 0 Begin set @ErrorDesc = 'error with get of tblUser' set @ErrorNo = @lErrorNo set @RowsAffected = 0 RETURN End Else Begin set @ErrorDesc = 'Get was Successful for tblUser' set @ErrorNo = 0 set @RowsAffected = @lRowsAffected RETURN End end Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Friday, November 27, 2009 1:45 PM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] Parameter objects in C# and ADO. Hi Mark et al What's the big deal? Are parameters costly? Hardly in money, but in resources? /gustav <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4641 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 27 08:11:29 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 09:11:29 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> References: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> Message-ID: <4B0FDE11.9070108@colbyconsulting.com> RowsAffected is not part of the error stuff and I definitely want that info in most cases. I haven't done this yet but I don't think that getting back the error code and description is excessive. If it is possible to easily and unambiguously look up the error code and get the exact description, each and every time, then passing back the description is not necessary. This is exactly the kind of thing I will be doing in the future. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Gustav at all, > > BTW, have you ever seen the following below error handling in CRUD SPs as I > have found in one of my customers' databases? > Isn't an overkill? > Or do you use even more detailed/elaborated error handling in your CRUD SPs? > How do you organize error handling in your CRUD SPs? (I usually just return > error code in RETURN statement). > > CREATE procedure [dbo].[GettblUserByUsername] > @Username varchar(20), > @ErrorDesc varchar(100) output, > @ErrorNo int output, > @RowsAffected int output > AS > DECLARE @lErrorNo INTEGER > DECLARE @lRowsAffected INTEGER > Begin > > SELECT tblUser.tblUserId, > tblUser.tblUserGroupId, > tblUserGroup.groupCode as UserGroup, > tblUser.Username, > tblUser.tblUserLevelId > FROM tblUser > left outer join tblUserGroup > on tblUser.tblUserGroupId = tblUserGroup.tblUserGroupId > WHERE tblUser.Username = @username > and tblUser.Authorised = 1 > > select @lErrorNo = @@ERROR, @lRowsAffected = @@ROWCOUNT > > IF @lErrorNo > 0 > Begin > set @ErrorDesc = 'error with get of tblUser' > set @ErrorNo = @lErrorNo > set @RowsAffected = 0 > RETURN > End > Else > Begin > set @ErrorDesc = 'Get was Successful for tblUser' > set @ErrorNo = 0 > set @RowsAffected = @lRowsAffected > RETURN > End > end > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, November 27, 2009 1:45 PM > To: dba-vb at databaseadvisors.com > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > Hi Mark et al > > What's the big deal? Are parameters costly? Hardly in money, but in > resources? > > /gustav > > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4641 (20091127) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From marklbreen at gmail.com Fri Nov 27 08:33:52 2009 From: marklbreen at gmail.com (Mark Breen) Date: Fri, 27 Nov 2009 14:33:52 +0000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: Message-ID: Hello Gustav, I presume they are not costly in terms of resources, and in fact, I would not dream of re-using a param for two different purposes. I think my delight was just that I never know you could share directions. I can imagine that Mr Colby will find a use for a standard piece of his arsenal to have a in-out param and once he does, he will wonder how we ever existing without utilising that aspect of params. Perhaps lastupdated as a field would be an appropriate use, where the last updated value is passed in to the sproc, and then passed back again with the new value after an update has been made, after checking for concurrency errors first. Thanks Mark 2009/11/27 Gustav Brock > Hi Mark et al > > What's the big deal? Are parameters costly? Hardly in money, but in > resources? > > /gustav > > > >>> marklbreen at gmail.com 27-11-2009 11:00 >>> > Hello John, > > that Shamil has said is that you can use one param to pass in a value, and > use the same param to accept back a value, which may be different to when > it > was passes in. > > You can pass in ProductId, and get back, ProductCode, > or pass in username and get back fullname, > > or pass in product code and get back zero if the product code does not > exist, else you get back the code you passed in > or pass in productid and get back qty in stock, > > all with one param. > > My examples are not great, but there are plenty of times where you need to > somehow inspect a value and it may not be ideal to have two params named > paramOriginal > paramNewValue > > with inputout you can have just one param named paramToBeVerified > > Thanks again to Shamil, Stuart and Mr Colby for asking the question. > > Mark > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 27 08:44:16 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 09:44:16 -0500 Subject: [dba-VB] SPAM-LOW: Re: Parameter objects in C# and ADO. In-Reply-To: References: Message-ID: <4B0FE5C0.5030409@colbyconsulting.com> > I can imagine that Mr Colby will find a use for a standard piece of his arsenal to have a in-out param and once he does, he will wonder how we ever existing without utilizing that aspect of params. LOL. Your confidence is encouraging. ;) While I use "by reference" values in normal programming, it is rare that I actually modify the passed in value. But it is something that can theoretically be useful and so it is nice to know it is available. > "Mr Colby" I assume you are either feeling particularly young this week or you discovered how old I actually am? ;) All of my kid's friends call me "Mr. Colby". Have you been hanging out with Robbie? John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello Gustav, > > I presume they are not costly in terms of resources, and in fact, I would > not dream of re-using a param for two different purposes. > > I think my delight was just that I never know you could share directions. > > I can imagine that Mr Colby will find a use for a standard piece of his > arsenal to have a in-out param and once he does, he will wonder how we ever > existing without utilising that aspect of params. > > Perhaps lastupdated as a field would be an appropriate use, where the last > updated value is passed in to the sproc, and then passed back again with the > new value after an update has been made, after checking for concurrency > errors first. > > Thanks > > Mark From shamil at smsconsulting.spb.ru Fri Nov 27 08:59:16 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 27 Nov 2009 17:59:16 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0FDE11.9070108@colbyconsulting.com> References: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> <4B0FDE11.9070108@colbyconsulting.com> Message-ID: <000d01ca6f72$31f97f70$95ec7e50$@spb.ru> Hi John -- This kind of "excessive" execution result/error information is useful in my opinion for non-CRUD SPs. But for CRUD ones one can just use SPs return value: - if it's zero or positive - all is OK and returned value is @@ROWCOUNT; - if it's negative - then this is @@ERROR (if actual @@ERROR value will be positive just return it as negative number). If some kinds of runtime errors happen in SPs then they can be trapped by calling C# code try/catch block and actual error message can be obtained from System.Data.SqlClient.SqlException. Well, above is what I'd call a "lightweight" error handling approach - and it works well in many real life systems AFAIK... But for long running (set of) SPs one can use "heavyweight" error handling approach using T-SQLs TRY...CATCH features as described in MS SQL books online (local link): ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/tsqlref9/html/248df62a-7334-4bca-8262- 235a28f4b07f.htm I could be missing something. Please correct me where I'm wrong, please write about your approach to T-SQL (and calling C# code) error handling. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 27, 2009 5:11 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. RowsAffected is not part of the error stuff and I definitely want that info in most cases. I haven't done this yet but I don't think that getting back the error code and description is excessive. If it is possible to easily and unambiguously look up the error code and get the exact description, each and every time, then passing back the description is not necessary. This is exactly the kind of thing I will be doing in the future. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Gustav at all, > > BTW, have you ever seen the following below error handling in CRUD SPs as I > have found in one of my customers' databases? > Isn't an overkill? > Or do you use even more detailed/elaborated error handling in your CRUD SPs? > How do you organize error handling in your CRUD SPs? (I usually just return > error code in RETURN statement). > > CREATE procedure [dbo].[GettblUserByUsername] > @Username varchar(20), > @ErrorDesc varchar(100) output, > @ErrorNo int output, > @RowsAffected int output > AS > DECLARE @lErrorNo INTEGER > DECLARE @lRowsAffected INTEGER > Begin > > SELECT tblUser.tblUserId, > tblUser.tblUserGroupId, > tblUserGroup.groupCode as UserGroup, > tblUser.Username, > tblUser.tblUserLevelId > FROM tblUser > left outer join tblUserGroup > on tblUser.tblUserGroupId = tblUserGroup.tblUserGroupId > WHERE tblUser.Username = @username > and tblUser.Authorised = 1 > > select @lErrorNo = @@ERROR, @lRowsAffected = @@ROWCOUNT > > IF @lErrorNo > 0 > Begin > set @ErrorDesc = 'error with get of tblUser' > set @ErrorNo = @lErrorNo > set @RowsAffected = 0 > RETURN > End > Else > Begin > set @ErrorDesc = 'Get was Successful for tblUser' > set @ErrorNo = 0 > set @RowsAffected = @lRowsAffected > RETURN > End > end > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, November 27, 2009 1:45 PM > To: dba-vb at databaseadvisors.com > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > Hi Mark et al > > What's the big deal? Are parameters costly? Hardly in money, but in > resources? > > /gustav > > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4641 (20091127) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4641 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4641 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 27 09:17:22 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 10:17:22 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: Message-ID: <4B0FED82.3080608@colbyconsulting.com> Over the last year or so I have built up an arsenal of stored procedures which form a system for performing big picture tasks. A set of SPs create a set of temp tables, move data into them, index them, the export chunks into large CSV files for address validation. Similar functionality on the way back in, create a chunk temp table, import the data from a CSV file, copy to a big temp table, create indexes on them, update hash codes and address valid flags, create more indexes on those fields etc. To this point I have executed these stored procedures from an Access database, simply because I could write Access VBA, however VBA is less than ideal for this functionality, not to mention that I had no ability to get data back from the SP. Thus the SP had to write to a log file which I could then read from Access. Again, less than ideal. Now that I am (slooooowwwwly) coming up to speed on C#, I have embarked on rewriting all of that control logic into C#. Once I figure out how to read back parameters from the SP so that I can see things like RecordsAffected, error numbers and messages etc. I will have made a giant step forward in fine grained control of my big picture processes. While I am at it I have to modify the stored procedures to pass back things like recordcount affected, error codes etc. Since I could not (did not know how to) get at them from VBA I hadn't bothered to add that kind of stuff to the SPs. John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello Gustav, > > I presume they are not costly in terms of resources, and in fact, I would > not dream of re-using a param for two different purposes. > > I think my delight was just that I never know you could share directions. > > I can imagine that Mr Colby will find a use for a standard piece of his > arsenal to have a in-out param and once he does, he will wonder how we ever > existing without utilising that aspect of params. > > Perhaps lastupdated as a field would be an appropriate use, where the last > updated value is passed in to the sproc, and then passed back again with the > new value after an update has been made, after checking for concurrency > errors first. > > Thanks > > Mark From jwcolby at colbyconsulting.com Fri Nov 27 09:24:15 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 10:24:15 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <000d01ca6f72$31f97f70$95ec7e50$@spb.ru> References: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> <4B0FDE11.9070108@colbyconsulting.com> <000d01ca6f72$31f97f70$95ec7e50$@spb.ru> Message-ID: <4B0FEF1F.8060905@colbyconsulting.com> Shamil, I am looking for guidance in this area myself. As you know by now, the particular application I am working on is not CRUD based objects but rather specific sets of SPs to perform specific operations in my business. To this point, I have used VBA and a function for executing a stored procedure provided by Charlotte. I could never figure out how to use that function to get at returned values so I have to this point not used that functionality at all. Now I expect to be able to do so, and as a result I will be expanding my processing to include such error handling. It will be a long time before I am correcting anyone on this stuff! John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > This kind of "excessive" execution result/error information is useful in my > opinion for non-CRUD SPs. > But for CRUD ones one can just use SPs return value: > > - if it's zero or positive - all is OK and returned value is @@ROWCOUNT; > - if it's negative - then this is @@ERROR (if actual @@ERROR value will be > positive just return it as negative number). > > If some kinds of runtime errors happen in SPs then they can be trapped by > calling C# code try/catch block and actual error message can be obtained > from System.Data.SqlClient.SqlException. > > Well, above is what I'd call a "lightweight" error handling approach - and > it works well in many real life systems AFAIK... > > But for long running (set of) SPs one can use "heavyweight" error handling > approach using T-SQLs > > TRY...CATCH > > features as described in MS SQL books online (local link): > > ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/tsqlref9/html/248df62a-7334-4bca-8262- > 235a28f4b07f.htm > > I could be missing something. > Please correct me where I'm wrong, please write about your approach to T-SQL > (and calling C# code) error handling. > > Thank you. > > -- > Shamil From accessd at shaw.ca Fri Nov 27 11:50:56 2009 From: accessd at shaw.ca (Jim Lawrence) Date: Fri, 27 Nov 2009 09:50:56 -0800 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0FDE11.9070108@colbyconsulting.com> References: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> <4B0FDE11.9070108@colbyconsulting.com> Message-ID: <805FE569738A4777BD83069AB8C812CE@creativesystemdesigns.com> Hi John: ADO always returns the number of rows affected in MS SQL SP call. It just has to be retrieved. Here is the code that I used as a sample/base before. http://msdn.microsoft.com/en-us/library/aa302325.aspx Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 27, 2009 6:11 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. RowsAffected is not part of the error stuff and I definitely want that info in most cases. I haven't done this yet but I don't think that getting back the error code and description is excessive. If it is possible to easily and unambiguously look up the error code and get the exact description, each and every time, then passing back the description is not necessary. This is exactly the kind of thing I will be doing in the future. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Gustav at all, > > BTW, have you ever seen the following below error handling in CRUD SPs as I > have found in one of my customers' databases? > Isn't an overkill? > Or do you use even more detailed/elaborated error handling in your CRUD SPs? > How do you organize error handling in your CRUD SPs? (I usually just return > error code in RETURN statement). > > CREATE procedure [dbo].[GettblUserByUsername] > @Username varchar(20), > @ErrorDesc varchar(100) output, > @ErrorNo int output, > @RowsAffected int output > AS > DECLARE @lErrorNo INTEGER > DECLARE @lRowsAffected INTEGER > Begin > > SELECT tblUser.tblUserId, > tblUser.tblUserGroupId, > tblUserGroup.groupCode as UserGroup, > tblUser.Username, > tblUser.tblUserLevelId > FROM tblUser > left outer join tblUserGroup > on tblUser.tblUserGroupId = tblUserGroup.tblUserGroupId > WHERE tblUser.Username = @username > and tblUser.Authorised = 1 > > select @lErrorNo = @@ERROR, @lRowsAffected = @@ROWCOUNT > > IF @lErrorNo > 0 > Begin > set @ErrorDesc = 'error with get of tblUser' > set @ErrorNo = @lErrorNo > set @RowsAffected = 0 > RETURN > End > Else > Begin > set @ErrorDesc = 'Get was Successful for tblUser' > set @ErrorNo = 0 > set @RowsAffected = @lRowsAffected > RETURN > End > end > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, November 27, 2009 1:45 PM > To: dba-vb at databaseadvisors.com > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > Hi Mark et al > > What's the big deal? Are parameters costly? Hardly in money, but in > resources? > > /gustav > > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4641 (20091127) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Nov 27 14:26:11 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 15:26:11 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0DE6F4.3070605@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> Message-ID: <4B1035E3.7000003@colbyconsulting.com> > If a parameter in a stored procedure is declared as an output (returns a value) will the parameter object in C# / ADO contain that value when the stored procedure finishes? This part of the question got lost, not directly answered AFAICT. I am directly creating parameter objects and adding them to the parameter collection of the command object. Code execution will halt until the stored procedure finishes executing and then the parameter objects in the command object's parameters collection can be looked at for the returned values? John W. Colby www.ColbyConsulting.com jwcolby wrote: > If a parameter in a stored procedure is declared as an output (returns a value) will the parameter > object in C# / ADO contain that value when the stored procedure finishes? > > I haven't used output parameters yet in stored procedures. Can values be passed in via an output > parameter or can it only be set from inside of the stored procedure? > From shamil at smsconsulting.spb.ru Fri Nov 27 15:14:53 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 00:14:53 +0300 Subject: [dba-VB] Calling SP with params from C# sample Message-ID: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> Hi John - Here is a sample for you: public static void TestSP() { string connectionString = "{{Your connection string here}}"; //CREATE Procedure [dbo].[TestParams] // @Username varchar(20), // @ErrorDesc varchar(100) output, // @ErrorNo int output, // @RowsAffected int output AS //DECLARE @lErrorNo INTEGER //DECLARE @lRowsAffected INTEGER //Begin // set @ErrorDesc = 'SUCCESS' + ', UserName = ' + IsNull(@USerName,'?') // set @ErrorNo = 1 -- OK // set @RowsAffected = 503 // RETURN 1234 //end try { using (SqlConnection connection = new SqlConnection(connectionString)) { connection.Open(); SqlCommand command = new SqlCommand("TestParams", connection); command.CommandType = System.Data.CommandType.StoredProcedure; // @Username varchar(20), command.Parameters.Add( new SqlParameter("@Username", System.Data.SqlDbType.VarChar, 20)); command.Parameters["@UserName"].Value = "SP_PARAMS_TEST"; // @ErrorDesc varchar(100) output, command.Parameters.Add( new SqlParameter("@ErrorDesc", System.Data.SqlDbType.VarChar, 100)); command.Parameters["@ErrorDesc"].Direction = System.Data.ParameterDirection.Output; // @ErrorNo int output, command.Parameters.Add( new SqlParameter("@ErrorNo", System.Data.SqlDbType.Int, 4)); command.Parameters["@ErrorNo"].Direction = System.Data.ParameterDirection.Output; // @RowsAffected int output AS command.Parameters.Add( new SqlParameter("@RowsAffected", System.Data.SqlDbType.Int, 4)); command.Parameters["@RowsAffected"].Direction = System.Data.ParameterDirection.Output; // RETURN command.Parameters.Add( new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int, 4)); command.Parameters["@RETURN_VALUE"].Direction = System.Data.ParameterDirection.ReturnValue; int retValue = command.ExecuteNonQuery(); Console.WriteLine( "retValue = {0}\n" + "@RETURN_VALUE = {1}\n" + "@ErrorDesc = '{2}'\n" + "@ErrorNo = {3}\n" + "@RowsAffected = {4}" , retValue, command.Parameters["@RETURN_VALUE"].Value.ToString(), command.Parameters["@ErrorDesc"].Value.ToString(), command.Parameters["@ErrorNo"].Value.ToString(), command.Parameters["@RowsAffected"].Value.ToString() ); } } catch (SqlException ex) { Console.WriteLine("T-SQL: {0}", ex.Message); } catch (Exception ex) { Console.WriteLine("{0}", ex.Message); } } Result: retValue = -1 @RETURN_VALUE = 1234 @ErrorDesc = 'SUCCESS, UserName = SP_PARAMS_TEST' @ErrorNo = 1 @RowsAffected = 503 -- Shamil From shamil at smsconsulting.spb.ru Fri Nov 27 15:31:02 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 00:31:02 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B1035E3.7000003@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B1035E3.7000003@colbyconsulting.com> Message-ID: <009501ca6fa8$ecaf22d0$c60d6870$@spb.ru> Hi John -- I have just posted here a sample for synchronous execution of SP with params. You can also run SPs from C# asynchronously - e.g. look MSDN for BeginExecuteNonQuery method of System.Data.SqlClient.SqlCommand class. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 27, 2009 11:26 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. > If a parameter in a stored procedure is declared as an output (returns a value) will the parameter object in C# / ADO contain that value when the stored procedure finishes? This part of the question got lost, not directly answered AFAICT. I am directly creating parameter objects and adding them to the parameter collection of the command object. Code execution will halt until the stored procedure finishes executing and then the parameter objects in the command object's parameters collection can be looked at for the returned values? John W. Colby www.ColbyConsulting.com jwcolby wrote: > If a parameter in a stored procedure is declared as an output (returns a value) will the parameter > object in C# / ADO contain that value when the stored procedure finishes? > > I haven't used output parameters yet in stored procedures. Can values be passed in via an output > parameter or can it only be set from inside of the stored procedure? > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4643 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 27 15:40:45 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 16:40:45 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <009501ca6fa8$ecaf22d0$c60d6870$@spb.ru> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B1035E3.7000003@colbyconsulting.com> <009501ca6fa8$ecaf22d0$c60d6870$@spb.ru> Message-ID: <4B10475D.8040102@colbyconsulting.com> Very cool! I am rewriting the code I found on the internet to allow me to pass in the direction as well as manipulate the param object when it returns. Thanks again for the example code. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > I have just posted here a sample for synchronous execution of SP with > params. > You can also run SPs from C# asynchronously - e.g. look MSDN for > BeginExecuteNonQuery method of System.Data.SqlClient.SqlCommand class. > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 27, 2009 11:26 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > > If a parameter in a stored procedure is declared as an output (returns a > value) will the > parameter object in C# / ADO contain that value when the stored procedure > finishes? > > This part of the question got lost, not directly answered AFAICT. > > I am directly creating parameter objects and adding them to the parameter > collection of the command > object. Code execution will halt until the stored procedure finishes > executing and then the > parameter objects in the command object's parameters collection can be > looked at for the returned > values? > > John W. Colby > www.ColbyConsulting.com > > > jwcolby wrote: >> If a parameter in a stored procedure is declared as an output (returns a > value) will the parameter >> object in C# / ADO contain that value when the stored procedure finishes? >> >> I haven't used output parameters yet in stored procedures. Can values be > passed in via an output >> parameter or can it only be set from inside of the stored procedure? >> > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4643 (20091127) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Nov 27 16:02:17 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 01:02:17 +0300 Subject: [dba-VB] Async calling SP with params from C# sample In-Reply-To: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> Message-ID: <009601ca6fad$4a2737f0$de75a7d0$@spb.ru> Another sample - async call (partially used MSDN sample): public static void runTest() { SPTester o = new SPTester(); o.TestSP(); while (!o.Completed) System.Threading.Thread.Sleep(1000); } public class SPTester { public bool Completed { get; set; } private SqlConnection _connection; public void TestSP() { Completed = false; string connectionString = "Data Source=HAMBURG\\SQL2005;"+ "Initial Catalog=X;"+ "User Id=sa;Password=Y;"+ "Asynchronous Processing=true"; //CREATE Procedure [dbo].[TestParams] // @Username varchar(20), // @ErrorDesc varchar(100) output, // @ErrorNo int output, // @RowsAffected int output AS //DECLARE @lErrorNo INTEGER //DECLARE @lRowsAffected INTEGER //Begin // set @ErrorDesc = 'SUCCESS' + ', UserName = ' + IsNull(@USerName,'?') // set @ErrorNo = 1 -- OK // set @RowsAffected = 503 // RETURN 1234 //end try { _connection = new SqlConnection(connectionString); { _connection.Open(); SqlCommand command = new SqlCommand("TestParams", _connection); command.CommandType = System.Data.CommandType.StoredProcedure; // @Username varchar(20), command.Parameters.Add( new SqlParameter("@Username", System.Data.SqlDbType.VarChar, 20)); command.Parameters["@UserName"].Value = "SP_PARAMS_TEST"; // @ErrorDesc varchar(100) output, command.Parameters.Add( new SqlParameter("@ErrorDesc", System.Data.SqlDbType.VarChar, 100)); command.Parameters["@ErrorDesc"].Direction = System.Data.ParameterDirection.Output; // @ErrorNo int output, command.Parameters.Add( new SqlParameter("@ErrorNo", System.Data.SqlDbType.Int, 4)); command.Parameters["@ErrorNo"].Direction = System.Data.ParameterDirection.Output; // @RowsAffected int output AS command.Parameters.Add( new SqlParameter("@RowsAffected", System.Data.SqlDbType.Int, 4)); command.Parameters["@RowsAffected"].Direction = System.Data.ParameterDirection.Output; // RETURN command.Parameters.Add( new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int, 4)); command.Parameters["@RETURN_VALUE"].Direction = System.Data.ParameterDirection.ReturnValue; AsyncCallback callback = new AsyncCallback(handleCallback); command.BeginExecuteNonQuery(callback, command); } } catch (Exception ex) { Console.WriteLine(ex.Message); if (_connection != null) { _connection.Close(); _connection = null; } } } private void handleCallback(IAsyncResult result) { try { SqlCommand command = (SqlCommand)result.AsyncState; int rowCount = command.EndExecuteNonQuery(result); Console.WriteLine( "rowCount = {0}\n" + "@RETURN_VALUE = {1}\n" + "@ErrorDesc = '{2}'\n" + "@ErrorNo = {3}\n" + "@RowsAffected = {4}" , rowCount, command.Parameters["@RETURN_VALUE"].Value.ToString(), command.Parameters["@ErrorDesc"].Value.ToString(), command.Parameters["@ErrorNo"].Value.ToString(), command.Parameters["@RowsAffected"].Value.ToString() ); } catch (SqlException ex) { Console.WriteLine("T-SQL: {0}", ex.Message); } catch (Exception ex) { Console.WriteLine("{0}", ex.Message); } finally { Completed = true; if (_connection != null) { _connection.Close(); _connection = null; } } } } -- Shamil From shamil at smsconsulting.spb.ru Fri Nov 27 16:09:52 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 01:09:52 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B10475D.8040102@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B1035E3.7000003@colbyconsulting.com> <009501ca6fa8$ecaf22d0$c60d6870$@spb.ru> <4B10475D.8040102@colbyconsulting.com> Message-ID: <009701ca6fae$591f3310$0b5d9930$@spb.ru> Hi John -- You're welcome! I have just posted another sample code where SP call from C# is processed asynchronously. -- Shamil P.S. There is an issue with the code I mentioned above - Completed flag should be also set to true in the catch() block of the calling method: public void TestSP() -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 28, 2009 12:41 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. Very cool! I am rewriting the code I found on the internet to allow me to pass in the direction as well as manipulate the param object when it returns. Thanks again for the example code. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > I have just posted here a sample for synchronous execution of SP with > params. > You can also run SPs from C# asynchronously - e.g. look MSDN for > BeginExecuteNonQuery method of System.Data.SqlClient.SqlCommand class. > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 27, 2009 11:26 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > > If a parameter in a stored procedure is declared as an output (returns a > value) will the > parameter object in C# / ADO contain that value when the stored procedure > finishes? > > This part of the question got lost, not directly answered AFAICT. > > I am directly creating parameter objects and adding them to the parameter > collection of the command > object. Code execution will halt until the stored procedure finishes > executing and then the > parameter objects in the command object's parameters collection can be > looked at for the returned > values? > > John W. Colby > www.ColbyConsulting.com > > > jwcolby wrote: >> If a parameter in a stored procedure is declared as an output (returns a > value) will the parameter >> object in C# / ADO contain that value when the stored procedure finishes? >> >> I haven't used output parameters yet in stored procedures. Can values be > passed in via an output >> parameter or can it only be set from inside of the stored procedure? >> > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4643 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 27 21:15:28 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 22:15:28 -0500 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> Message-ID: <4B1095D0.2050804@colbyconsulting.com> Hi Shamil, I assume that the parameters on the C# side have to be in the same order / name / datatype / size as the parameters in the stored procedure? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John - > > > > Here is a sample for you: > > > > public static void TestSP() > > { > > string connectionString = "{{Your connection string here}}"; > > > > //CREATE Procedure [dbo].[TestParams] > > // @Username varchar(20), > > // @ErrorDesc varchar(100) output, > > // @ErrorNo int output, > > // @RowsAffected int output AS > > //DECLARE @lErrorNo INTEGER > > //DECLARE @lRowsAffected INTEGER > > //Begin > > // set @ErrorDesc = 'SUCCESS' + ', UserName = ' + IsNull(@USerName,'?') > > // set @ErrorNo = 1 -- OK > > // set @RowsAffected = 503 > > // RETURN 1234 > > //end > > > > try > > { > > using (SqlConnection connection = new > SqlConnection(connectionString)) > > { > > connection.Open(); > > > > SqlCommand command = new SqlCommand("TestParams", connection); > > command.CommandType = System.Data.CommandType.StoredProcedure; > > // @Username varchar(20), > > command.Parameters.Add( > > new SqlParameter("@Username", System.Data.SqlDbType.VarChar, > 20)); > > command.Parameters["@UserName"].Value = "SP_PARAMS_TEST"; > > // @ErrorDesc varchar(100) output, > > command.Parameters.Add( > > new SqlParameter("@ErrorDesc", > System.Data.SqlDbType.VarChar, 100)); > > command.Parameters["@ErrorDesc"].Direction = > System.Data.ParameterDirection.Output; > > // @ErrorNo int output, > > command.Parameters.Add( > > new SqlParameter("@ErrorNo", System.Data.SqlDbType.Int, 4)); > > command.Parameters["@ErrorNo"].Direction = > System.Data.ParameterDirection.Output; > > // @RowsAffected int output AS > > command.Parameters.Add( > > new SqlParameter("@RowsAffected", System.Data.SqlDbType.Int, > 4)); > > command.Parameters["@RowsAffected"].Direction = > System.Data.ParameterDirection.Output; > > // RETURN > > command.Parameters.Add( > > new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int, > 4)); > > command.Parameters["@RETURN_VALUE"].Direction = > System.Data.ParameterDirection.ReturnValue; > > > > int retValue = command.ExecuteNonQuery(); > > > > Console.WriteLine( > > "retValue = {0}\n" + > > "@RETURN_VALUE = {1}\n" + > > "@ErrorDesc = '{2}'\n" + > > "@ErrorNo = {3}\n" + > > "@RowsAffected = {4}" > > , > > retValue, > > command.Parameters["@RETURN_VALUE"].Value.ToString(), > > command.Parameters["@ErrorDesc"].Value.ToString(), > > command.Parameters["@ErrorNo"].Value.ToString(), > > command.Parameters["@RowsAffected"].Value.ToString() > > ); > > } > > } > > catch (SqlException ex) > > { > > Console.WriteLine("T-SQL: {0}", ex.Message); > > } > > catch (Exception ex) > > { > > Console.WriteLine("{0}", ex.Message); > > } > > } > > > > Result: > > > > retValue = -1 > > @RETURN_VALUE = 1234 > > @ErrorDesc = 'SUCCESS, UserName = SP_PARAMS_TEST' > > @ErrorNo = 1 > > @RowsAffected = 503 > > > > -- > > Shamil > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 27 21:45:19 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 22:45:19 -0500 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> Message-ID: <4B109CCF.5020107@colbyconsulting.com> Shamil, I executed my first SP using your example code (first light as they say in the astronomy world). Man oh man is it ugly though. Tons of code to get one lonely little SP to execute (NOT your fault!!!). Thank you so much for the examples, I can definitely take it from here. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John - > > > > Here is a sample for you: > > > > public static void TestSP() > > { > > string connectionString = "{{Your connection string here}}"; > > > > //CREATE Procedure [dbo].[TestParams] > > // @Username varchar(20), > > // @ErrorDesc varchar(100) output, > > // @ErrorNo int output, > > // @RowsAffected int output AS > > //DECLARE @lErrorNo INTEGER > > //DECLARE @lRowsAffected INTEGER > > //Begin > > // set @ErrorDesc = 'SUCCESS' + ', UserName = ' + IsNull(@USerName,'?') > > // set @ErrorNo = 1 -- OK > > // set @RowsAffected = 503 > > // RETURN 1234 > > //end > > > > try > > { > > using (SqlConnection connection = new > SqlConnection(connectionString)) > > { > > connection.Open(); > > > > SqlCommand command = new SqlCommand("TestParams", connection); > > command.CommandType = System.Data.CommandType.StoredProcedure; > > // @Username varchar(20), > > command.Parameters.Add( > > new SqlParameter("@Username", System.Data.SqlDbType.VarChar, > 20)); > > command.Parameters["@UserName"].Value = "SP_PARAMS_TEST"; > > // @ErrorDesc varchar(100) output, > > command.Parameters.Add( > > new SqlParameter("@ErrorDesc", > System.Data.SqlDbType.VarChar, 100)); > > command.Parameters["@ErrorDesc"].Direction = > System.Data.ParameterDirection.Output; > > // @ErrorNo int output, > > command.Parameters.Add( > > new SqlParameter("@ErrorNo", System.Data.SqlDbType.Int, 4)); > > command.Parameters["@ErrorNo"].Direction = > System.Data.ParameterDirection.Output; > > // @RowsAffected int output AS > > command.Parameters.Add( > > new SqlParameter("@RowsAffected", System.Data.SqlDbType.Int, > 4)); > > command.Parameters["@RowsAffected"].Direction = > System.Data.ParameterDirection.Output; > > // RETURN > > command.Parameters.Add( > > new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int, > 4)); > > command.Parameters["@RETURN_VALUE"].Direction = > System.Data.ParameterDirection.ReturnValue; > > > > int retValue = command.ExecuteNonQuery(); > > > > Console.WriteLine( > > "retValue = {0}\n" + > > "@RETURN_VALUE = {1}\n" + > > "@ErrorDesc = '{2}'\n" + > > "@ErrorNo = {3}\n" + > > "@RowsAffected = {4}" > > , > > retValue, > > command.Parameters["@RETURN_VALUE"].Value.ToString(), > > command.Parameters["@ErrorDesc"].Value.ToString(), > > command.Parameters["@ErrorNo"].Value.ToString(), > > command.Parameters["@RowsAffected"].Value.ToString() > > ); > > } > > } > > catch (SqlException ex) > > { > > Console.WriteLine("T-SQL: {0}", ex.Message); > > } > > catch (Exception ex) > > { > > Console.WriteLine("{0}", ex.Message); > > } > > } > > > > Result: > > > > retValue = -1 > > @RETURN_VALUE = 1234 > > @ErrorDesc = 'SUCCESS, UserName = SP_PARAMS_TEST' > > @ErrorNo = 1 > > @RowsAffected = 503 > > > > -- > > Shamil > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Nov 28 03:19:35 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 12:19:35 +0300 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <4B1095D0.2050804@colbyconsulting.com> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B1095D0.2050804@colbyconsulting.com> Message-ID: <00a201ca700b$e98b6ba0$bca242e0$@spb.ru> Hi John, The order of parameters' creation on C# side doesn't matter, name and datatype and size do matter. There could be some variations for the latter two but better keep them in strict correspondence. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 28, 2009 6:15 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Calling SP with params from C# sample Hi Shamil, I assume that the parameters on the C# side have to be in the same order / name / datatype / size as the parameters in the stored procedure? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John - > > > > Here is a sample for you: > > > > public static void TestSP() > <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4643 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From shamil at smsconsulting.spb.ru Sat Nov 28 03:19:35 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 12:19:35 +0300 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <4B109CCF.5020107@colbyconsulting.com> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B109CCF.5020107@colbyconsulting.com> Message-ID: <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> John -- Tons of code for C# (or VB.NET) is not an issue at all. And for C#/VB.Net programmers also. You'll get adapted to that reality soon, I believe. I have projects with literally thousands of custom classes - they work in production for several years without any issues. And they recompile from cleaned solution with 10+ projects in 10+ seconds. And when you're working on such solution and change code/recompile/run then recompiling and restarting takes <1-2 seconds as VS recompiles/rebuilds only changed projects. Have a look e.g. on DotNetNuke(DNN) sources - and if you'll try to "dig them out" you'll find how "ugly" DNN sources are sometimes... For your case I'd use: - plain "dumb" manual coding if you have relatively static set of SPs and their parameters; - LINQ for SQL (now); - ADO.NET Entity Framework (VS2010); - statically generated custom classes to call SPs and process SPs' results. I'd not use dynamically generated SqlCommand and SqlParameters as this approach could result in too much and never ending "plumbing programming"... And in all the cases you can use the power of true OOP C# provides to minimize "copy and paste" coding approach... BTW, C# 4.0 provides some very powerful dynamic programming features - more powerful than late binding(?) - I must say I have seen some samples of that new C# 4.0 features usage but I didn't get how to use them :) - maybe they can help you to minimize "ugliness" of your C# coding but be careful to not spend too much time on "plumbing programming"... http://stackoverflow.com/questions/244302/what-do-you-think-of-the-new-c-4-0 -dynamic-keyword http://geekswithblogs.net/sdorman/archive/2008/11/16/c-4.0-dynamic-programmi ng.aspx -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 28, 2009 6:45 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Calling SP with params from C# sample Shamil, I executed my first SP using your example code (first light as they say in the astronomy world). Man oh man is it ugly though. Tons of code to get one lonely little SP to execute (NOT your fault!!!). Thank you so much for the examples, I can definitely take it from here. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John - > > > > Here is a sample for you: > > > > public static void TestSP() > > { <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4643 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From marklbreen at gmail.com Sat Nov 28 08:27:07 2009 From: marklbreen at gmail.com (Mark Breen) Date: Sat, 28 Nov 2009 14:27:07 +0000 Subject: [dba-VB] SPAM-LOW: Re: Parameter objects in C# and ADO. In-Reply-To: <4B0FE5C0.5030409@colbyconsulting.com> References: <4B0FE5C0.5030409@colbyconsulting.com> Message-ID: Haha, My kids call some people by their first name and I still call the same people Mr Dowling, sounds funny to hear it. Mark 2009/11/27 jwcolby > > I can imagine that Mr Colby will find a use for a standard piece of his > arsenal to have a in-out > param and once he does, he will wonder how we ever existing without > utilizing that aspect of params. > > LOL. Your confidence is encouraging. ;) > > While I use "by reference" values in normal programming, it is rare that I > actually modify the > passed in value. But it is something that can theoretically be useful and > so it is nice to know it > is available. > > > "Mr Colby" > > I assume you are either feeling particularly young this week or you > discovered how old I actually > am? ;) > > All of my kid's friends call me "Mr. Colby". Have you been hanging out > with Robbie? > > John W. Colby > www.ColbyConsulting.com > > > Mark Breen wrote: > > Hello Gustav, > > > > I presume they are not costly in terms of resources, and in fact, I would > > not dream of re-using a param for two different purposes. > > > > I think my delight was just that I never know you could share directions. > > > > I can imagine that Mr Colby will find a use for a standard piece of his > > arsenal to have a in-out param and once he does, he will wonder how we > ever > > existing without utilising that aspect of params. > > > > Perhaps lastupdated as a field would be an appropriate use, where the > last > > updated value is passed in to the sproc, and then passed back again with > the > > new value after an update has been made, after checking for concurrency > > errors first. > > > > Thanks > > > > Mark > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Nov 28 16:29:17 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 28 Nov 2009 17:29:17 -0500 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B109CCF.5020107@colbyconsulting.com> <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> Message-ID: <4B11A43D.70104@colbyconsulting.com> Shamil, I ended up creating a "stored procedure class". This class hosts a sCmd command object that can be programmed with the name of the SP, connection etc but then included wrapping the lines of code to create specific often used parameters into functions. /// /// The following code creates standard parameters used all of the time in my stored procedures /// Basically just wrapper code so that I can call a function to create specific parameters as needed /// /// It also allows me to completely standardize the names, data type and length of parameters in the SPs etc. /// /// public void paramDBName(string lstrDBName) { sCmd.Parameters.Add(new SqlParameter( "@DBName", SqlDbType.VarChar, 50)); sCmd.Parameters["@DBName"].Value = lstrDBName; } public void paramTblName(string lstrTblName) { sCmd.Parameters.Add(new SqlParameter("@TblName", SqlDbType.VarChar, 50)); sCmd.Parameters["@TblName"].Value = lstrTblName; } public void paramErrorDesc() { sCmd.Parameters.Add(new SqlParameter("@ErrorDesc", System.Data.SqlDbType.VarChar, 4000)); sCmd.Parameters["@ErrorDesc"].Direction = System.Data.ParameterDirection.Output; } Then I can do things like: // //sp_AZOut_AZExportToOrdered_Append // sp = new StoredProcedure(myConnection, "_aDataMaster.dbo.sp_AZOut_AZExportToOrdered_Append"); sp.paramDBName(lstrDBName); //@DBName varchar(50) input sp.paramTblName(lstrTblName); //@TblName varchar(50) input sp.paramErrorDesc(); //@ErrorDesc varchar(100) output sp.paramErrorNo(); //@ErrorNo int output, sp.paramReturnValue(); //RETURN retValue = sp.ExecuteSP(); I believe I am on my way now. Thanks again, John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > John -- > > Tons of code for C# (or VB.NET) is not an issue at all. And for C#/VB.Net > programmers also. You'll get adapted to that reality soon, I believe. From shamil at smsconsulting.spb.ru Sat Nov 28 17:12:11 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sun, 29 Nov 2009 02:12:11 +0300 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <4B11A43D.70104@colbyconsulting.com> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B109CCF.5020107@colbyconsulting.com> <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> <4B11A43D.70104@colbyconsulting.com> Message-ID: <00ba01ca7080$392fd000$ab8f7000$@spb.ru> John, Yes, that looks good with me. AFAIS you are getting what is usually declared as abstract class, and it can be inherited by custom classes, which would use their base abstract class's "services"/utilities methods. I suppose you can go even a bit further by combining sets of your standard parameters assignments into methods of your base abstract class. BTW, all the methods/properties of your abstract class, which are not planned to be called from outside of custom classes can be declared as 'protected', all the local class level common variables (of all kinds/types) can be also put with 'protected' declaration into your abstract class... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Sunday, November 29, 2009 1:29 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Calling SP with params from C# sample Shamil, I ended up creating a "stored procedure class". This class hosts a sCmd command object that can be programmed with the name of the SP, connection etc but then included wrapping the lines of code to create specific often used parameters into functions. /// /// The following code creates standard parameters used all of the time in my stored procedures /// Basically just wrapper code so that I can call a function to create specific parameters as needed /// /// It also allows me to completely standardize the names, data type and length of parameters in the SPs etc. /// /// public void paramDBName(string lstrDBName) { sCmd.Parameters.Add(new SqlParameter( "@DBName", SqlDbType.VarChar, 50)); sCmd.Parameters["@DBName"].Value = lstrDBName; } public void paramTblName(string lstrTblName) { sCmd.Parameters.Add(new SqlParameter("@TblName", SqlDbType.VarChar, 50)); sCmd.Parameters["@TblName"].Value = lstrTblName; } public void paramErrorDesc() { sCmd.Parameters.Add(new SqlParameter("@ErrorDesc", System.Data.SqlDbType.VarChar, 4000)); sCmd.Parameters["@ErrorDesc"].Direction = System.Data.ParameterDirection.Output; } Then I can do things like: // //sp_AZOut_AZExportToOrdered_Append // sp = new StoredProcedure(myConnection, "_aDataMaster.dbo.sp_AZOut_AZExportToOrdered_Append"); sp.paramDBName(lstrDBName); //@DBName varchar(50) input sp.paramTblName(lstrTblName); //@TblName varchar(50) input sp.paramErrorDesc(); //@ErrorDesc varchar(100) output sp.paramErrorNo(); //@ErrorNo int output, sp.paramReturnValue(); //RETURN retValue = sp.ExecuteSP(); I believe I am on my way now. Thanks again, John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > John -- > > Tons of code for C# (or VB.NET) is not an issue at all. And for C#/VB.Net > programmers also. You'll get adapted to that reality soon, I believe. __________ Information from ESET NOD32 Antivirus, version of virus signature database 4645 (20091128) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 28 17:19:19 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 28 Nov 2009 18:19:19 -0500 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <00ba01ca7080$392fd000$ab8f7000$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B109CCF.5020107@colbyconsulting.com> <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> <4B11A43D.70104@colbyconsulting.com> <00ba01ca7080$392fd000$ab8f7000$@spb.ru> Message-ID: <4B11AFF7.8010706@colbyconsulting.com> Perhaps I will do the inheritance thing once I get rolling. It certainly seems like a logical extension. For now I am just instantiating the SP class and manipulating methods of that class. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > John, > > Yes, that looks good with me. > AFAIS you are getting what is usually declared as abstract class, and it can > be inherited by custom classes, which would use their base abstract class's > "services"/utilities methods. I suppose you can go even a bit further by > combining sets of your standard parameters assignments into methods of your > base abstract class. BTW, all the methods/properties of your abstract class, > which are not planned to be called from outside of custom classes can be > declared as 'protected', all the local class level common variables (of all > kinds/types) can be also put with 'protected' declaration into your abstract > class... > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Sunday, November 29, 2009 1:29 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Calling SP with params from C# sample > > Shamil, > > I ended up creating a "stored procedure class". This class hosts a sCmd > command object that can be > programmed with the name of the SP, connection etc but then included > wrapping the lines of code to > create specific often used parameters into functions. > > /// > /// The following code creates standard parameters used all of the > time in my stored procedures > /// Basically just wrapper code so that I can call a function to > create specific parameters > as needed > /// > /// It also allows me to completely standardize the names, data > type and length of > parameters in the SPs etc. > /// > /// > public void paramDBName(string lstrDBName) > { > sCmd.Parameters.Add(new SqlParameter( "@DBName", > SqlDbType.VarChar, 50)); > sCmd.Parameters["@DBName"].Value = lstrDBName; > } > public void paramTblName(string lstrTblName) > { > sCmd.Parameters.Add(new SqlParameter("@TblName", > SqlDbType.VarChar, 50)); > sCmd.Parameters["@TblName"].Value = lstrTblName; > } > public void paramErrorDesc() > { > sCmd.Parameters.Add(new SqlParameter("@ErrorDesc", > System.Data.SqlDbType.VarChar, 4000)); > sCmd.Parameters["@ErrorDesc"].Direction = > System.Data.ParameterDirection.Output; > } > > Then I can do things like: > > // > //sp_AZOut_AZExportToOrdered_Append > // > sp = new StoredProcedure(myConnection, > "_aDataMaster.dbo.sp_AZOut_AZExportToOrdered_Append"); > sp.paramDBName(lstrDBName); //@DBName varchar(50) input > sp.paramTblName(lstrTblName); //@TblName varchar(50) input > sp.paramErrorDesc(); //@ErrorDesc varchar(100) > output > sp.paramErrorNo(); //@ErrorNo int output, > sp.paramReturnValue(); //RETURN > retValue = sp.ExecuteSP(); > > I believe I am on my way now. > > Thanks again, > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> John -- >> >> Tons of code for C# (or VB.NET) is not an issue at all. And for C#/VB.Net >> programmers also. You'll get adapted to that reality soon, I believe. > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4645 (20091128) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sun Nov 29 12:51:14 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 29 Nov 2009 13:51:14 -0500 Subject: [dba-VB] C# - Presenting process status Message-ID: <4B12C2A2.1010001@colbyconsulting.com> Just a general question about how you go about presenting process status back from classes. In Access I tended to use Withevents in my classes, then sink the class events in the form and use the event to pass in values to the form, and in the event sink display the status in a text box control on the form. For example the process that exports data from SQL Server to Accuzip (address validation) would display the name of the file just created and the PKFrom / PKTo pk range. I am also looking for tips on how to report status from a thread. I have vague recollections that a thread cannot directly update controls on a form (or something like that). So how do you folks display status information in a form? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sun Nov 29 15:33:05 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 30 Nov 2009 00:33:05 +0300 Subject: [dba-VB] C# - Presenting process status In-Reply-To: <4B12C2A2.1010001@colbyconsulting.com> References: <4B12C2A2.1010001@colbyconsulting.com> Message-ID: <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> Hi John, Here is one of the methods I use (there are zillion combinations) - I dunno where it comes from, did I borrowed it from somewhere or I got it elaborated myself, is it recommended way or not - but it just works, and I have "cooked" it for you here from scratch - this is codebehind for Windows Form named Form1: using System; using System.Windows.Forms; namespace WinFormsSample { public partial class Form1 : Form { // Create Windows Form Form1 with three controls: // // 1. Button: cmdTest1 -> cmdTest1_Click // 2. Button: cmdTest2 -> cmdTest2_Click // 3. TextBox: txtLog (Multiline = true) #region Test class public class Test { public delegate void ProgressStatus(string message); private static int _id; private System.Threading.Thread _thread; public void Run(ProgressStatus progressStatus) { _testId = ++_id; _progressStatus = progressStatus; System.Threading.ThreadStart ts = new System.Threading.ThreadStart(runThread); _thread = new System.Threading.Thread(ts); _thread.IsBackground = true; _thread.Start(); } private int _testId; private ProgressStatus _progressStatus; private void runThread() { for (int i = 1; i < 10; i++) { _progressStatus(string.Format("Test#{0}, cycle#{1}", _testId, i)); System.Threading.Thread.Sleep(500); } } } #endregion // Test class public Form1() { InitializeComponent(); } private Test _test1; private void cmdTest1_Click(object sender, EventArgs e) { _test1 = new Test(); _test1.Run(progressStatus); } private Test _test2; private void cmdTest2_Click(object sender, EventArgs e) { _test2 = new Test(); _test2.Run(progressStatus); } private void progressStatus(string message) { if (this.InvokeRequired) { object[] args = { message }; this.Invoke( new Test.ProgressStatus( progressStatusInThisFormThread), args); } else progressStatusInThisFormThread(message); } private static object _statusMessageOutputLocker = new object(); private void progressStatusInThisFormThread(string message) { lock (_statusMessageOutputLocker) { txtLog.Text = String.Format("{0:hh:mm:ss.fff}: {1}", DateTime.Now, message) + System.Environment.NewLine + txtLog.Text; } } } } I will try to post about another one I use in the coming days... Thank you. -- Shamil -----Original Message----- So how do you folks display status information in a form? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sun Nov 29 15:51:48 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 30 Nov 2009 00:51:48 +0300 Subject: [dba-VB] C# - Presenting process status In-Reply-To: <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> References: <4B12C2A2.1010001@colbyconsulting.com> <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> Message-ID: <00d301ca713e$28009bc0$7801d340$@spb.ru> OK, here is another sample more close to VB6/VBA WithEvents approach: using System; using System.Windows.Forms; namespace WinFormsSample { public partial class Form2 : Form { // Create Windows Form Form2 with three controls: // // 1. Button: cmdTest1 -> cmdTest1_Click // 2. Button: cmdTest2 -> cmdTest2_Click // 3. TextBox: txtLog (Multiline = true) #region Test class public class Test { public class ProgressStatusEventArgs : EventArgs { public string Message { get; set; } } public event EventHandler ProgressStatus; private static int _id; private System.Threading.Thread _thread; public void Run() { _testId = ++_id; System.Threading.ThreadStart ts = new System.Threading.ThreadStart(runThread); _thread = new System.Threading.Thread(ts); _thread.IsBackground = true; _thread.Start(); } private int _testId; private void runThread() { for (int i = 1; i < 10; i++) { string message = string.Format("Test#{0}, cycle#{1}", _testId, i); ProgressStatusEventArgs e = new ProgressStatusEventArgs(); e.Message = message; // raise event if (ProgressStatus != null) ProgressStatus(this, e); System.Threading.Thread.Sleep(500); } } } #endregion // Test class public Form2() { InitializeComponent(); } private Test _test1; private void cmdTest1_Click(object sender, EventArgs e) { _test1 = new Test(); _test1.ProgressStatus += new EventHandler(progressStatus); _test1.Run(); } private Test _test2; private void cmdTest2_Click(object sender, EventArgs e) { _test2 = new Test(); _test2.ProgressStatus += new EventHandler(progressStatus); _test2.Run(); } public delegate void ProgressStatusDelegate(string message); private void progressStatus(object sender, Test.ProgressStatusEventArgs e) { if (this.InvokeRequired) { object[] args = { e.Message }; this.Invoke( new ProgressStatusDelegate( progressStatusInThisFormThread), args); } else progressStatusInThisFormThread(e.Message); } private static object _statusMessageOutputLocker = new object(); private void progressStatusInThisFormThread(string message) { lock (_statusMessageOutputLocker) { txtLog.Text = String.Format("{0:hh:mm:ss.fff}: {1}", DateTime.Now, message) + System.Environment.NewLine + txtLog.Text; } } } } -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: Monday, November 30, 2009 12:33 AM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] C# - Presenting process status <<< I will try to post about another one I use in the coming days... >>> From jwcolby at colbyconsulting.com Sun Nov 29 22:09:43 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 29 Nov 2009 23:09:43 -0500 Subject: [dba-VB] C# - Presenting process status In-Reply-To: <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> References: <4B12C2A2.1010001@colbyconsulting.com> <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> Message-ID: <4B134587.9010005@colbyconsulting.com> Shamil, Thanks for this. I will need to study it. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > Here is one of the methods I use (there are zillion combinations) - I dunno > where it comes from, did I borrowed it from somewhere or I got it elaborated > myself, is it recommended way or not - but it just works, and I have > "cooked" it for you here from scratch - this is codebehind for Windows Form > named Form1: > > using System; > using System.Windows.Forms; > > namespace WinFormsSample > { > public partial class Form1 : Form > { > // Create Windows Form Form1 with three controls: > // > // 1. Button: cmdTest1 -> cmdTest1_Click > // 2. Button: cmdTest2 -> cmdTest2_Click > // 3. TextBox: txtLog (Multiline = true) > > #region Test class > public class Test > { > public delegate void ProgressStatus(string message); > > private static int _id; > private System.Threading.Thread _thread; > public void Run(ProgressStatus progressStatus) > { > _testId = ++_id; > _progressStatus = progressStatus; > > System.Threading.ThreadStart ts = > new System.Threading.ThreadStart(runThread); > _thread = new System.Threading.Thread(ts); > _thread.IsBackground = true; > _thread.Start(); > } > > private int _testId; > private ProgressStatus _progressStatus; > private void runThread() > { > for (int i = 1; i < 10; i++) > { > _progressStatus(string.Format("Test#{0}, cycle#{1}", > _testId, i)); > System.Threading.Thread.Sleep(500); > } > } > } > #endregion // Test class > > public Form1() > { > InitializeComponent(); > } > > private Test _test1; > private void cmdTest1_Click(object sender, EventArgs e) > { > _test1 = new Test(); > _test1.Run(progressStatus); > } > > private Test _test2; > private void cmdTest2_Click(object sender, EventArgs e) > { > _test2 = new Test(); > _test2.Run(progressStatus); > } > > private void progressStatus(string message) > { > if (this.InvokeRequired) > { > object[] args = { message }; > this.Invoke( > new Test.ProgressStatus( > progressStatusInThisFormThread), > args); > } > else > progressStatusInThisFormThread(message); > > } > > private static object _statusMessageOutputLocker = new object(); > private void progressStatusInThisFormThread(string message) > { > lock (_statusMessageOutputLocker) > { > txtLog.Text = > String.Format("{0:hh:mm:ss.fff}: {1}", > DateTime.Now, > message) + > System.Environment.NewLine + > txtLog.Text; > } > } > } > } > > I will try to post about another one I use in the coming days... > > Thank you. > > -- > Shamil > > -----Original Message----- > So how do you folks display status information in a form? From jwcolby at colbyconsulting.com Mon Nov 30 11:44:39 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 12:44:39 -0500 Subject: [dba-VB] C# - return value for stored procedures Message-ID: <4B140487.1030905@colbyconsulting.com> While I am getting back values in my "output" parameters for the SPs, I am getting a constant -1 for the return value. In the stored procedure I have tried just using RETURN 1234 and also creating an int variable, setting that to 1234 and returning that. declare @Ret int select @ret = 1234 RETURN @ret In all cases I get a -1 value which leads me to believe that the problem is on the C# side of things. My c# code looks like: try { if (sCmd.Connection.State != ConnectionState.Open) sCmd.Connection.Open(); int retValue = sCmd.ExecuteNonQuery(); return retValue; } catch (SqlException sqlEx) { throw sqlEx; } I am not throwing an error, the SP executes, but retValue is always -1. Any ideas? Do I need to add a SqlParameter to the Parameters collection for the return value? -- John W. Colby www.ColbyConsulting.com From james at fcidms.com Mon Nov 30 11:56:47 2009 From: james at fcidms.com (James Barash) Date: Mon, 30 Nov 2009 12:56:47 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <4B140487.1030905@colbyconsulting.com> Message-ID: <12D2F894654B53498DEF840AE12B7BB36B4B5797@fciexchange.fcidms.com> John: You do, indeed, need to define a Parameter for the return value. From the Microsoft documentation: -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, November 30, 2009 12:45 PM To: VBA Subject: [dba-VB] C# - return value for stored procedures While I am getting back values in my "output" parameters for the SPs, I am getting a constant -1 for the return value. In the stored procedure I have tried just using RETURN 1234 and also creating an int variable, setting that to 1234 and returning that. declare @Ret int select @ret = 1234 RETURN @ret In all cases I get a -1 value which leads me to believe that the problem is on the C# side of things. My c# code looks like: try { if (sCmd.Connection.State != ConnectionState.Open) sCmd.Connection.Open(); int retValue = sCmd.ExecuteNonQuery(); return retValue; } catch (SqlException sqlEx) { throw sqlEx; } I am not throwing an error, the SP executes, but retValue is always -1. Any ideas? Do I need to add a SqlParameter to the Parameters collection for the return value? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From james at fcidms.com Mon Nov 30 12:06:48 2009 From: james at fcidms.com (James Barash) Date: Mon, 30 Nov 2009 13:06:48 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <4B140487.1030905@colbyconsulting.com> Message-ID: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com> John: You do, indeed, need to define a Parameter for the return value. From the Microsoft documentation: SqlCommand.ExecuteNonQuery Method Executes a Transact-SQL statement against the connection and returns the number of rows affected. To add a Return Value parameter: SqlParameter ret = new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int); ret.Direction = ParameterDirection.ReturnValue; cmd.Parameters.Add(ret); Hope this helps. James Barash -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, November 30, 2009 12:45 PM To: VBA Subject: [dba-VB] C# - return value for stored procedures While I am getting back values in my "output" parameters for the SPs, I am getting a constant -1 for the return value. In the stored procedure I have tried just using RETURN 1234 and also creating an int variable, setting that to 1234 and returning that. declare @Ret int select @ret = 1234 RETURN @ret In all cases I get a -1 value which leads me to believe that the problem is on the C# side of things. My c# code looks like: try { if (sCmd.Connection.State != ConnectionState.Open) sCmd.Connection.Open(); int retValue = sCmd.ExecuteNonQuery(); return retValue; } catch (SqlException sqlEx) { throw sqlEx; } I am not throwing an error, the SP executes, but retValue is always -1. Any ideas? Do I need to add a SqlParameter to the Parameters collection for the return value? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Mon Nov 30 12:08:16 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 13:08:16 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <12D2F894654B53498DEF840AE12B7BB36B4B5797@fciexchange.fcidms.com> References: <12D2F894654B53498DEF840AE12B7BB36B4B5797@fciexchange.fcidms.com> Message-ID: <4B140A10.4020409@colbyconsulting.com> Thanks James. Your response did not provide a link or anything but knowing that I have to do this, I will go a Googling. Thanks, John W. Colby www.ColbyConsulting.com James Barash wrote: > John: > > You do, indeed, need to define a Parameter for the return value. From the Microsoft documentation: > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Monday, November 30, 2009 12:45 PM > To: VBA > Subject: [dba-VB] C# - return value for stored procedures > > While I am getting back values in my "output" parameters for the SPs, I am getting a constant -1 for the return value. > > In the stored procedure I have tried just using > > RETURN 1234 > > and also creating an int variable, setting that to 1234 and returning that. > > declare @Ret int > select @ret = 1234 > RETURN @ret > > In all cases I get a -1 value which leads me to believe that the problem is on the C# side of things. > > My c# code looks like: > > try > { > if (sCmd.Connection.State != ConnectionState.Open) > sCmd.Connection.Open(); > int retValue = sCmd.ExecuteNonQuery(); > return retValue; > } > catch (SqlException sqlEx) > { > throw sqlEx; > } > > I am not throwing an error, the SP executes, but retValue is always -1. > > Any ideas? Do I need to add a SqlParameter to the Parameters collection for the return value? > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Mon Nov 30 12:25:27 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 13:25:27 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com> References: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com> Message-ID: <4B140E17.3050400@colbyconsulting.com> James, So what does this do for me? int retValue = sCmd.ExecuteNonQuery(); From my Googling it seems like I have to do something like: retValue = Int32.Parse(sCmd.Parameters["@Return_Value"].Value.ToString()); which is working just fine. scmd.ExecuteNonQuery() returns a -1 (not sure why) and then the @Return_Value is parsed and returned. So that part is now working as well. Thanks again for the response James. John W. Colby www.ColbyConsulting.com James Barash wrote: > John: > > You do, indeed, need to define a Parameter for the return value. From the Microsoft documentation: > > SqlCommand.ExecuteNonQuery Method > > Executes a Transact-SQL statement against the connection and returns the number of rows affected. > > To add a Return Value parameter: > > SqlParameter ret = new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int); > ret.Direction = ParameterDirection.ReturnValue; > cmd.Parameters.Add(ret); > > Hope this helps. From jwcolby at colbyconsulting.com Mon Nov 30 13:55:11 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 14:55:11 -0500 Subject: [dba-VB] C# SQLCommand.Timeout Message-ID: <4B14231F.3020905@colbyconsulting.com> Any words of wisdom on setting the command.Timeout property? I have stored procedures that execute instantly, and others that can take an hour or more (large appends). How can I discover how long a stored procedure takes to run so that I can set this property dynamically? As an example I have cases where I am appending 65 million rows into a table of 6 fields. This can take a long time (at the very least perhaps 20 minutes) but if I could get a "RecordsAffected" count for such queries as well as the time it took to execute, then I could start to discover that it takes "X seconds / million" records, and set the timeout to a reasonable value based on the records to be appended. This doesn't have to be any exact time value. I assume a simple pair of time variables, then "stop time - start time"? Or should I just discover worst case and set it to that? -- John W. Colby www.ColbyConsulting.com From stuart at lexacorp.com.pg Mon Nov 30 15:07:43 2009 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Tue, 01 Dec 2009 07:07:43 +1000 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <4B140E17.3050400@colbyconsulting.com> References: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com>, <4B140E17.3050400@colbyconsulting.com> Message-ID: <4B14341F.21088.149E29F8@stuart.lexacorp.com.pg> -1 = True, meanimg that the sp executed with no errors? -- Stuart On 30 Nov 2009 at 13:25, jwcolby wrote: > which is working just fine. scmd.ExecuteNonQuery() returns a -1 (not sure why) and then the > @Return_Value is parsed and returned. > From jwcolby at colbyconsulting.com Mon Nov 30 15:35:18 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 16:35:18 -0500 Subject: [dba-VB] C# - Stored procedures working! Message-ID: <4B143A96.8080905@colbyconsulting.com> I would like to thank all of you for your assistance in getting my stored procedures working from C#. My strategy was to design a StoredProcedure class which contained all of the data and code required to run any of the stored procedures that I have written so far. We shall see whether I succeeded in that however I have a set of seven stored procedures that are required to export large tables in chunks to CSV files and I am successfully running all of those seven stored procedures and obtaining CSV files. This StoredProcedure class has a command object and a set of about eight "Parameter" methods which wrap the code for creating standard parameters used over and over in my various stored procedures, parameters like DBName, TblName, PKStart, PKend etc, as well as the return value and errorno and errormsg parameters. I then created an AccuzipExport class that "supervises" the Accuzip export process, i.e. holds all of the code to DEFINE these seven stored procedures, i.e. set up the required stored procedure class mentioned above for each of these seven SPs, and the specific parameters that each SP uses. This class then has a method that calls three of the stored procedures to create a standard named table, fill it with sorted data and index it in preparation for creating chunk files. After that the method sits in a while loop calling the remaining four stored procedures which create the chunk table, append a chunk of records to that, index it and then BCP it out. All of this stuff now works. This stuff was what the old Access database did, so I have essentially ported the Accuzip Export process to C#. However I have successfully returned a success/fail return value as well as an error code / message in the C# version, whereas I never had that ability in the Access version so I am now in some respects ahead of where I was in Access. There is more to do of course. Now that I have a robust dev platform at my fingertips I can log the process from beginning to end, errors, times for each SP to execute, stuff like that. And of course get the process status displaying on the form I use to start this process. Eventually I need to get this export process running in a thread so that I can use the big picture application for other things while this export process runs. But next comes the matching AccuzipInput class, which will pull the CSV files coming back from Accuzip back into SQL Server. With all of the grunt work done for executing Stored Procedures it should be relatively quick and easy to get the matching input class up. At any rate I wanted to say thanks to everyone for all the help and express how jazzed I am to be where I am today. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Mon Nov 30 15:37:43 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 16:37:43 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <4B14341F.21088.149E29F8@stuart.lexacorp.com.pg> References: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com>, <4B140E17.3050400@colbyconsulting.com> <4B14341F.21088.149E29F8@stuart.lexacorp.com.pg> Message-ID: <4B143B27.9050902@colbyconsulting.com> Stuart, I don't think so as I have had failures and I *think* I got a -1 regardless. I will have to pursue this one. John W. Colby www.ColbyConsulting.com Stuart McLachlan wrote: > -1 = True, meanimg that the sp executed with no errors? > From jwcolby at colbyconsulting.com Mon Nov 2 07:58:56 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 02 Nov 2009 08:58:56 -0500 Subject: [dba-VB] SQL Server 2005: Running stored procedures from C# / VBA Message-ID: <4AEEE5A0.4040901@colbyconsulting.com> I am embarking on the task of running the "DB from hell" from C#. I have several dozen stored procedures that perform various tasks and I currently run many of them from Access using a single procedure provided by Charlotte. I need a similar function written in C# (preferably) or VB.Net. ATM I have an Access database which has two specific forms, bound to two tables. These two forms each execute a set of SPs which build temp tables of data to be exported to an external process, build indexes to make the processes faster, BCP the data out to CSV files, and then the reverse on the way back in. I have another Access database that creates an order for the same client. Same kind of thing, temp tables created, indexes, fields added and so forth. My intention is to port each of these processes to C#. The reason is simply that C# has a couple of huge advantages over VBA, such as child threads and built-in capabilities in the framework that I have to really scrounge to make work in VBA. The first thing I need though is code that can execute a stored procedure, pass parameters to it, retrieve parameters back etc. With demo code preferably. Also I would like to open a conversation about how using .Net from inside of SQL Server (2005) works. Does anyone on the list use C# or VB.Net from inside of SQL Server? Where is the code stored? How to do you call it? What do you use it for? TIA, -- John W. Colby www.ColbyConsulting.com From marklbreen at gmail.com Mon Nov 2 10:18:13 2009 From: marklbreen at gmail.com (Mark Breen) Date: Tue, 3 Nov 2009 00:18:13 +0800 Subject: [dba-VB] SQL Server 2005: Running stored procedures from C# / VBA In-Reply-To: <4AEEE5A0.4040901@colbyconsulting.com> References: <4AEEE5A0.4040901@colbyconsulting.com> Message-ID: Hello John, First of all, good luck. Secondly, you mentioned a few days ago that you referred to yourself as an SQL Wannabe, well given the work you are currently doing, I think you can no longer refer to yourself as a wannabe, you are using SQL, and no doubt will be pushing it to it's limits. *SPROCs* There are loads of samples of code and I have nothing to hand here that will assist much, but I think you will be surprised how easy it is. You will get better exmaples, but as an overview you just need to create a connection - usually referred to as cnn. create a cmd to 'hold' the sproc you wish to call join the cnn to the sproc specify the type of the cmd (no results returned, or records returned or a few other types) then you add objects which are parameters, you need one each for each param in the sproc, you can even add a param that will be referred to as OUTPUT instead of INPUT - these output params can be used to pass back single values in additional or instead of, the actual result set. you can also receive back the result code of the sproc. then you just execute the cmd object. Once you get a few lines of code that handle the above, you can copy and paste for the next five years. There are a few other things that I am not as accustomed to, but when you know that you need to read in a result set, you can use a data reader. Someone else can advise on that. when you create the cnn object, usually people pull the actual string from a app.config file, so watch out for that snipit also. BTW, FWIW, other than for binding to grids or combo's, I never used a dataset in code. *BCP* You mentioned BCP, well I have no used that since SQL 6.5, but what I have used in the SSIS in SQL 2005. It is worth taking a look, you may find that you can do a lot of your work in SSIS. It can branch and call sprocs and import and export all as a package. It can be incredibly fast and would be worth take an hour or two and read up on what you can do with SSIS. *Embeding C#* Lastly, other than an article I read in MSDN magazine, where a guy wanted to do some hashing or encryption in side SQL Server I never heard of anyone using / executing code in side SQL Serer. For performance that you need, I would, with my limited skills, keep as much of the work in TSQL - IE Sprocs, and by doing so, you will keep the mass movement of data within the db. I would love to see if anyone here actually ever used the feature of including C# Code within the db. I know that this is not samples, but you can do this, and once you get a set of lines of code that 1) create the cnn 2) create the cmd 3) add the params 4) run the sproc you will not look back. Just keep as much work as possible inside sql server. *Scheduled Tasks* Finally, once last thing to mention, if you have a smtp server, it is so, so each to create scheduled tasks, that run at predefined times and then mail you when they are finished. Running the heavy jobs nightly, can take a lot of the hard work from your daily routines. Again you can manage all this with very little actually code and have working routines in minutes. HTH. Mark 2009/11/2 jwcolby > I am embarking on the task of running the "DB from hell" from C#. I have > several dozen stored > procedures that perform various tasks and I currently run many of them from > Access using a single > procedure provided by Charlotte. I need a similar function written in C# > (preferably) or VB.Net. > > ATM I have an Access database which has two specific forms, bound to two > tables. These two forms > each execute a set of SPs which build temp tables of data to be exported to > an external process, > build indexes to make the processes faster, BCP the data out to CSV files, > and then the reverse on > the way back in. > > I have another Access database that creates an order for the same client. > Same kind of thing, temp > tables created, indexes, fields added and so forth. > > My intention is to port each of these processes to C#. The reason is > simply that C# has a couple of > huge advantages over VBA, such as child threads and built-in capabilities > in the framework that I > have to really scrounge to make work in VBA. > > The first thing I need though is code that can execute a stored procedure, > pass parameters to it, > retrieve parameters back etc. With demo code preferably. > > Also I would like to open a conversation about how using .Net from inside > of SQL Server (2005) > works. Does anyone on the list use C# or VB.Net from inside of SQL Server? > Where is the code > stored? How to do you call it? What do you use it for? > > TIA, > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Mon Nov 2 11:51:47 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 02 Nov 2009 12:51:47 -0500 Subject: [dba-VB] SQL Server 2005: Running stored procedures from C# / VBA In-Reply-To: References: <4AEEE5A0.4040901@colbyconsulting.com> Message-ID: <4AEF1C33.1020904@colbyconsulting.com> Mark, > First of all, good luck. Uh-Ohhh... is that a warning? ;) I refer to myself as a wannabe simply because to this point SQL Server has been a small part of my day. As a sole proprietor my day contains so many different kinds of work that I find it hard to be an expert in something as complex as SQL Server. There are so many people on these lists that spend all day every day in SQL Server that I will truly never be anything other than a wannabe - from their perspective. I have always been more programmer than anything. I come at the world from the perspective of "how can I write code to make my job easier". Luckily I have enough programming background that I can generally accomplish that. Right now I am trying to switch my environment of choice from Access to C#, and BOY IS THAT TOUGH. Programming is always about spending enough time doing it that the syntax becomes second nature. C# is nowhere close to second nature yet. I can taste the power though, so tantalizing. ;) I am using the Idera free backup stuff. Very powerful, recommended. It took me a couple of hours to install on my two servers and write some stored procedures to make them work. Now I really want to be able to select one specific database to backup or restore. To do that I have to be able to: See/select a database on a server. Run a stored procedure that does the backup. How simple is that eh? I just spent the morning learning about the DMO object, the listview control and writing the code to read out the databases and display them in the listview. I still have to find and get working the code that executes the stored procedure. The problem is not that this is hard, it is that I am not fluent in C# yet. But I am getting there, sloooowly but surely. John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > First of all, good luck. > > Secondly, you mentioned a few days ago that you referred to yourself as an > SQL Wannabe, well given the work you are currently doing, I think you can no > longer refer to yourself as a wannabe, you are using SQL, and no doubt will > be pushing it to it's limits. > > *SPROCs* > > There are loads of samples of code and I have nothing to hand here that will > assist much, but I think you will be surprised how easy it is. > > You will get better exmaples, but as an overview you just need to > > create a connection - usually referred to as cnn. > create a cmd to 'hold' the sproc you wish to call > join the cnn to the sproc > specify the type of the cmd (no results returned, or records returned or a > few other types) > then you add objects which are parameters, you need one each for each param > in the sproc, > you can even add a param that will be referred to as OUTPUT instead of INPUT > - these output params can be used to pass back single values in additional > or instead of, the actual result set. > you can also receive back the result code of the sproc. > then you just execute the cmd object. > > Once you get a few lines of code that handle the above, you can copy and > paste for the next five years. > > There are a few other things that I am not as accustomed to, but when you > know that you need to read in a result set, you can use a data reader. > Someone else can advise on that. > > when you create the cnn object, usually people pull the actual string from a > app.config file, so watch out for that snipit also. > > BTW, FWIW, other than for binding to grids or combo's, I never used a > dataset in code. > > > *BCP* > You mentioned BCP, well I have no used that since SQL 6.5, but what I have > used in the SSIS in SQL 2005. It is worth taking a look, you may find that > you can do a lot of your work in SSIS. It can branch and call sprocs and > import and export all as a package. It can be incredibly fast and would be > worth take an hour or two and read up on what you can do with SSIS. > > *Embeding C#* > Lastly, other than an article I read in MSDN magazine, where a guy wanted to > do some hashing or encryption in side SQL Server I never heard of anyone > using / executing code in side SQL Serer. For performance that you need, I > would, with my limited skills, keep as much of the work in TSQL - IE Sprocs, > and by doing so, you will keep the mass movement of data within the db. I > would love to see if anyone here actually ever used the feature of including > C# Code within the db. > > > I know that this is not samples, but you can do this, and once you get a set > of lines of code that > 1) create the cnn > 2) create the cmd > 3) add the params > 4) run the sproc > > you will not look back. Just keep as much work as possible inside sql > server. > > *Scheduled Tasks* > Finally, once last thing to mention, if you have a smtp server, it is so, so > each to create scheduled tasks, that run at predefined times and then mail > you when they are finished. Running the heavy jobs nightly, can take a lot > of the hard work from your daily routines. Again you can manage all this > with very little actually code and have working routines in minutes. > > HTH. > > Mark > > > > > 2009/11/2 jwcolby > >> I am embarking on the task of running the "DB from hell" from C#. I have >> several dozen stored >> procedures that perform various tasks and I currently run many of them from >> Access using a single >> procedure provided by Charlotte. I need a similar function written in C# >> (preferably) or VB.Net. >> >> ATM I have an Access database which has two specific forms, bound to two >> tables. These two forms >> each execute a set of SPs which build temp tables of data to be exported to >> an external process, >> build indexes to make the processes faster, BCP the data out to CSV files, >> and then the reverse on >> the way back in. >> >> I have another Access database that creates an order for the same client. >> Same kind of thing, temp >> tables created, indexes, fields added and so forth. >> >> My intention is to port each of these processes to C#. The reason is >> simply that C# has a couple of >> huge advantages over VBA, such as child threads and built-in capabilities >> in the framework that I >> have to really scrounge to make work in VBA. >> >> The first thing I need though is code that can execute a stored procedure, >> pass parameters to it, >> retrieve parameters back etc. With demo code preferably. >> >> Also I would like to open a conversation about how using .Net from inside >> of SQL Server (2005) >> works. Does anyone on the list use C# or VB.Net from inside of SQL Server? >> Where is the code >> stored? How to do you call it? What do you use it for? >> >> TIA, >> >> -- >> John W. Colby >> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Mon Nov 2 12:41:24 2009 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Tue, 03 Nov 2009 04:41:24 +1000 Subject: [dba-VB] SQL Server 2005: Running stored procedures from C# / VBA In-Reply-To: References: <4AEEE5A0.4040901@colbyconsulting.com>, Message-ID: <4AEF27D4.29660.1C94540E@stuart.lexacorp.com.pg> If you don't havr an SMTP Server, grab the tiny one that Max put us on to a while ago: miniRelay. You can download it from here (site in Spanish): http://www.netvicious.com/miniRelay/miniRelay.zip English documentation and FAQ here (but download link is out of date) http://www.blat.net/miniRelay/ -- Stuart On 3 Nov 2009 at 0:18, Mark Breen wrote: > > *Scheduled Tasks* > Finally, once last thing to mention, if you have a smtp server, it is so, so > each to create scheduled tasks, that run at predefined times and then mail > you when they are finished. Running the heavy jobs nightly, can take a lot > of the hard work from your daily routines. Again you can manage all this > with very little actually code and have working routines in minutes. > From jwcolby at colbyconsulting.com Tue Nov 3 21:12:23 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Tue, 03 Nov 2009 22:12:23 -0500 Subject: [dba-VB] How do you run SQL Server processes Message-ID: <4AF0F117.20209@colbyconsulting.com> I have processes that I run which use sets of stored procedures. For example two of the processes export / import records from large tables for address validation. The addresses need to be sorted in Zip5/Zip4/Address order in order to make the external validation process as fast as possible so I create a table that pulls millions of records, often tens of millions of records, into a table (created on-the-fly) in sorted order, then pulls out 2 million records at a time and BCPs them out for processing in an external program. The inverse occurs when the records finish processing, 2 million record chunks are imported back into a temp table, then each 2 million record chunk is appended to a main table. Once back in SQL Server I have processes that build clustered and non-clustered indexes, run sha-1 hash functions to generate hash keys, update address valid codes etc. I will run this entire project at least once a month (to keep the addresses valid as people move), and I do so for many tables - a table of about 65 million addresses, another of 21 million, another of 11 million etc. I store all of my stored procedures in a dedicated database that pretty much just contains the SPs and a few UDFs as well as a couple of logging type tables. At the moment I use an Access database to run the individual stored procedures in specific order. I am just embarking on a project to move that functioning but not very efficient Access database to C#. I want to be able to have better monitoring of SP completion, records processed (counts), time to complete each SP etc. I found a pretty nice set of C# classes that allow creation of "stored procedure" objects, with parameter objects, then stored into a collection as the SPs (class instances) are assembled and finally the whole collection of SPs executed. I created a test database of a million names / addresses and today I started testing my existing SPs on this small data set. In the next few days I expect to build out a preliminary "port" of the existing Access VBA code. I am hoping to eventually make the entire process "table driven" where I can store (in a table) the name of the sp, the parameters in a child table etc. and then have a supervisor pull out the records and execute the SPs based on what is in the table. This would allow me to create the SPs and then use a C# application (data entry forms) to organize the SPs, add / delete SPs from the process and so forth. That is down the road of course, for now I will have hard coded sequences - as I already do in supervisor SPs as well as in hard coded Access VBA modules. As a programmer, doing this supervisor level in C# feels more natural to me (than doing it inside of SQL Server), and gives me the flexibility to do what I want in code external to the SQL Server database being manipulated. I want to be able to do things external to the database processes, things like monitor the directories that the BCP processes export to / from, move files popping out of a bcp process off to a VM on a different server, monitor those VMs for files coming out of their processes, moving the files back over to the SQL Server directories, and trigger BCP processes that import the data back in to SQL server. I need to be able to FTP order and data files to clients, and get (FTP) files from process bureaus. So it FEELS to me like using a language such as C# to build a system outside of SQL Server is the way to go. BTW I stumbled across something called SMO (SQL Server Management Objects) which makes it really easy to get at the SQL Server object model from C#. I can apparently see the entire tree structure that is a server/database/tables/etc. and do I know not what (yet) with those objects. Looks pretty cool though. The database is a large part of what I do but it is not the whole picture and I have been handcuffed by my lack of ability to effectively do these external parts. But I am a programmer at heart. I want to know how you do this kind of stuff. I know that running such "systems" of stored procedures tied to external processes must be quite common and I am wondering what you guys do for supervising your processes. If anyone doing anything remotely similar would take the time to explain how you do it, I would be most appreciative. Thanks, -- John W. Colby www.ColbyConsulting.com From raibeart at gmail.com Wed Nov 4 12:37:58 2009 From: raibeart at gmail.com (Robert Stewart) Date: Wed, 04 Nov 2009 12:37:58 -0600 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: References: Message-ID: <4af1ca07.0637560a.20ef.1362@mx.google.com> Sorry, but using C# to do what you would and could do simply using SQL Server is adding a middle layer you do not need. You can do exactly the same thing using SQL Server and stored procedures and writing to a log table about what the process is doing and keeping track of it that way. And, not have to use a middle layer language like C# to do it. I am not sure why writing T-SQL code is not like real programming to you. But, it is. And, if you are going to continue using SQL Server, you need to learn what it can do for you instead of always fighting learning it. This is coming from someone that does T-SQL, C#, VB.Net and Access VBA. So, I think I have a fair perspective of how they all work. At 12:00 PM 11/4/2009, you wrote: >Date: Tue, 03 Nov 2009 22:12:23 -0500 >From: jwcolby >Subject: [dba-VB] How do you run SQL Server processes >To: VBA , Dba-Sqlserver > >Message-ID: <4AF0F117.20209 at colbyconsulting.com> >Content-Type: text/plain; charset=ISO-8859-1; format=flowed > >I have processes that I run which use sets of stored procedures. > >For example two of the processes export / import records from large >tables for address validation. >The addresses need to be sorted in Zip5/Zip4/Address order in order >to make the external validation >process as fast as possible so I create a table that pulls millions >of records, often tens of >millions of records, into a table (created on-the-fly) in sorted >order, then pulls out 2 million >records at a time and BCPs them out for processing in an external program. > >The inverse occurs when the records finish processing, 2 million >record chunks are imported back >into a temp table, then each 2 million record chunk is appended to a >main table. Once back in SQL >Server I have processes that build clustered and non-clustered >indexes, run sha-1 hash functions to >generate hash keys, update address valid codes etc. I will run this >entire project at least once a >month (to keep the addresses valid as people move), and I do so for >many tables - a table of about >65 million addresses, another of 21 million, another of 11 million etc. > >I store all of my stored procedures in a dedicated database that >pretty much just contains the SPs >and a few UDFs as well as a couple of logging type tables. At the >moment I use an Access database >to run the individual stored procedures in specific order. > >I am just embarking on a project to move that functioning but not >very efficient Access database to >C#. I want to be able to have better monitoring of SP completion, >records processed (counts), time >to complete each SP etc. I found a pretty nice set of C# classes >that allow creation of "stored >procedure" objects, with parameter objects, then stored into a >collection as the SPs (class >instances) are assembled and finally the whole collection of SPs >executed. I created a test >database of a million names / addresses and today I started testing >my existing SPs on this small >data set. In the next few days I expect to build out a preliminary >"port" of the existing Access >VBA code. > >I am hoping to eventually make the entire process "table driven" >where I can store (in a table) the >name of the sp, the parameters in a child table etc. and then have a >supervisor pull out the records >and execute the SPs based on what is in the table. This would allow >me to create the SPs and then >use a C# application (data entry forms) to organize the SPs, add / >delete SPs from the process and >so forth. > >That is down the road of course, for now I will have hard coded >sequences - as I already do in >supervisor SPs as well as in hard coded Access VBA modules. > >As a programmer, doing this supervisor level in C# feels more >natural to me (than doing it inside of >SQL Server), and gives me the flexibility to do what I want in code >external to the SQL Server >database being manipulated. I want to be able to do things external >to the database processes, >things like monitor the directories that the BCP processes export to >/ from, move files popping out >of a bcp process off to a VM on a different server, monitor those >VMs for files coming out of their >processes, moving the files back over to the SQL Server directories, >and trigger BCP processes that >import the data back in to SQL server. I need to be able to FTP >order and data files to clients, >and get (FTP) files from process bureaus. So it FEELS to me like >using a language such as C# to >build a system outside of SQL Server is the way to go. > >BTW I stumbled across something called SMO (SQL Server Management >Objects) which makes it really >easy to get at the SQL Server object model from C#. I can >apparently see the entire tree structure >that is a server/database/tables/etc. and do I know not what (yet) >with those objects. Looks pretty >cool though. > >The database is a large part of what I do but it is not the whole >picture and I have been handcuffed >by my lack of ability to effectively do these external parts. But I >am a programmer at heart. I >want to know how you do this kind of stuff. > >I know that running such "systems" of stored procedures tied to >external processes must be quite >common and I am wondering what you guys do for supervising your >processes. If anyone doing anything >remotely similar would take the time to explain how you do it, I >would be most appreciative. > >Thanks, >-- >John W. Colby >www.ColbyConsulting.com > > >------------------------------ > >_______________________________________________ >dba-VB mailing list >dba-VB at databaseadvisors.com >http://databaseadvisors.com/mailman/listinfo/dba-vb > > >End of dba-VB Digest, Vol 73, Issue 3 >************************************* From jwcolby at colbyconsulting.com Wed Nov 4 15:11:58 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 04 Nov 2009 16:11:58 -0500 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <4af1ca07.0637560a.20ef.1362@mx.google.com> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> Message-ID: <4AF1EE1E.9030406@colbyconsulting.com> Robert, I don't really want to get in an argument about this. I asked a serious question and got a somewhat brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY answer I have received so that is not particularly encouraging. 8( TSQL, at least as written from inside of SSMS is about like writing GBasic code back in 1982. Well, maybe a tiny step up but not much. You are right, I probably just don't know how, but in what I will call "a real language" I can set breakpoints, I can see a call stack, I have watch statements, I have... simple very basic DEBUG capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in SSMS). I can have forms with controls displaying data, buttons that cause things to happen. I can FTP files, monitor directories for activities. Believe me, if this exists in SSMS I will jump on that! Just don't tell me to go spend thousands to buy a tool that will do some of what C# can already do for me. Modern languages, and the .Net environment in particular have capabilities that (AFAICT) TSQL can not even dream of. TSQL dreams of manipulating data, C# dreams of manipulating ... the world. Robert I am not "fighting learning it", I am light years beyond where I was two years ago, and I learn more TSQL and SQL Server every day. However from what I have seen so far, TSQL will NEVER be what I need for the supervisor. And answers like you just provided aren't really much help. I am a consultant. I am a programmer, not a DBA, or a SQL Server administrator, or a SQL Server jock at a company somewhere. I need C# for automating my business, which is NOT being a DBA (I already said that I think). I will write C# applications for clients that use SQL Server, and others that do not use SQL Server. SQL Server is used by exactly ONE of my clients, so I cannot spend the time to do whatever it is that you envision SQL Server doing because I cannot amortize the learning over other clients. Now... if you want to answer my question, I am all ears. The first thing to answer (If I am to do all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. Let's take a concrete, real world example. I have a set of virtual machines which run a third party application. I place files into an "inbox" directory, and I wait for files to pop out of the running application into the "outbox" directory. Just an aside, the files placed into the inbox are created by a set of about 5 stored procedures, which can operate on any of dozens of databases in my server. It is the case that the third party application occasionally hangs. If it does I need to rename a file in a specific directory to cause that application to restart processing that file. It is my intention to use C# to execute the SPs to create the files, move the files into the VM, then monitor files dropped into the inbox, and files popping out into the outbox. Look for "no new files in X minutes" and if that condition exists, rename the file. Log files dropped into the in-box, files popping out into the out-box, and the time it took for the files to process through the application. Log any "hangs" restarted. Supervisor stuff. Explain to me how (and more importantly WHY) I would use TSQL to do that? Other than a "when TSQL is all you have" (which is NOT true for me) mentality. A SMALL part of my business is to perform some moderately complex processes using SQL Server and SPs as a tool. I envision C# as a supervisor, not a "middle layer". I envision an application (written in C#) that runs on my desktop which starts, then monitors SQL Server processes, displays errors, restarts processes if necessary, creates databases, tables etc, backs them up and detaches them when done, logging everything that happens, showing it all to me me in real time in a Windows application. Possibly even automatically generating invoices for the work done and emailing them to the client. Explain to me one more time (with feeling) how (and more importantly WHY) I would use TSQL to do all of that stuff? John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > Sorry, but using C# to do what you would and could do simply using > SQL Server is adding a middle layer you do not need. You can do > exactly the same thing using SQL Server and stored procedures and > writing to a log table about what the process is doing and keeping > track of it that way. And, not have to use a middle layer language > like C# to do it. > > I am not sure why writing T-SQL code is not like real programming to > you. But, it is. And, if you are going to continue using SQL > Server, you need to learn what it can do for you instead of always > fighting learning it. > > This is coming from someone that does T-SQL, C#, VB.Net and Access > VBA. So, I think I have a fair perspective of how they all work. From Gustav at cactus.dk Wed Nov 4 16:21:22 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 04 Nov 2009 23:21:22 +0100 Subject: [dba-VB] How do you run SQL Server processes Message-ID: Hi John I have no answer to you but if it can bring you some relief, I feel much the same. The database is for storing data, a business layer and/or a front end is for the logic and interface. That said, nothing is perfect, and in the real world you will have to write some T-SQL or similar but I try to keep it at minimum, first because of the trouble of maintenance (for example, no version control I know of), second because I feel old when I write "GBasic style" code. And I'm not in a big corporation where I can get every expensive tool I can point at, and tools for SQL Server seem all to be enterprise-priced. So, writing views and T-SQL is because I have to, writing C# (and VBA for that matter) is because I like to. I guess Robert is a very skilled DBA and can write T-SQL with his left foot and that's very good - then we know where to ask for advice as I'll never be more than a decent low-level DBA. /gustav >>> jwcolby at colbyconsulting.com 04-11-2009 22:11 >>> Robert, I don't really want to get in an argument about this. I asked a serious question and got a somewhat brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY answer I have received so that is not particularly encouraging. 8( TSQL, at least as written from inside of SSMS is about like writing GBasic code back in 1982. Well, maybe a tiny step up but not much. You are right, I probably just don't know how, but in what I will call "a real language" I can set breakpoints, I can see a call stack, I have watch statements, I have... simple very basic DEBUG capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in SSMS). I can have forms with controls displaying data, buttons that cause things to happen. I can FTP files, monitor directories for activities. Believe me, if this exists in SSMS I will jump on that! Just don't tell me to go spend thousands to buy a tool that will do some of what C# can already do for me. Modern languages, and the .Net environment in particular have capabilities that (AFAICT) TSQL can not even dream of. TSQL dreams of manipulating data, C# dreams of manipulating ... the world. Robert I am not "fighting learning it", I am light years beyond where I was two years ago, and I learn more TSQL and SQL Server every day. However from what I have seen so far, TSQL will NEVER be what I need for the supervisor. And answers like you just provided aren't really much help. I am a consultant. I am a programmer, not a DBA, or a SQL Server administrator, or a SQL Server jock at a company somewhere. I need C# for automating my business, which is NOT being a DBA (I already said that I think). I will write C# applications for clients that use SQL Server, and others that do not use SQL Server. SQL Server is used by exactly ONE of my clients, so I cannot spend the time to do whatever it is that you envision SQL Server doing because I cannot amortize the learning over other clients. Now... if you want to answer my question, I am all ears. The first thing to answer (If I am to do all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. Let's take a concrete, real world example. I have a set of virtual machines which run a third party application. I place files into an "inbox" directory, and I wait for files to pop out of the running application into the "outbox" directory. Just an aside, the files placed into the inbox are created by a set of about 5 stored procedures, which can operate on any of dozens of databases in my server. It is the case that the third party application occasionally hangs. If it does I need to rename a file in a specific directory to cause that application to restart processing that file. It is my intention to use C# to execute the SPs to create the files, move the files into the VM, then monitor files dropped into the inbox, and files popping out into the outbox. Look for "no new files in X minutes" and if that condition exists, rename the file. Log files dropped into the in-box, files popping out into the out-box, and the time it took for the files to process through the application. Log any "hangs" restarted. Supervisor stuff. Explain to me how (and more importantly WHY) I would use TSQL to do that? Other than a "when TSQL is all you have" (which is NOT true for me) mentality. A SMALL part of my business is to perform some moderately complex processes using SQL Server and SPs as a tool. I envision C# as a supervisor, not a "middle layer". I envision an application (written in C#) that runs on my desktop which starts, then monitors SQL Server processes, displays errors, restarts processes if necessary, creates databases, tables etc, backs them up and detaches them when done, logging everything that happens, showing it all to me me in real time in a Windows application. Possibly even automatically generating invoices for the work done and emailing them to the client. Explain to me one more time (with feeling) how (and more importantly WHY) I would use TSQL to do all of that stuff? John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > Sorry, but using C# to do what you would and could do simply using > SQL Server is adding a middle layer you do not need. You can do > exactly the same thing using SQL Server and stored procedures and > writing to a log table about what the process is doing and keeping > track of it that way. And, not have to use a middle layer language > like C# to do it. > > I am not sure why writing T-SQL code is not like real programming to > you. But, it is. And, if you are going to continue using SQL > Server, you need to learn what it can do for you instead of always > fighting learning it. > > This is coming from someone that does T-SQL, C#, VB.Net and Access > VBA. So, I think I have a fair perspective of how they all work. _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Wed Nov 4 16:48:32 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 01:48:32 +0300 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <4AF1EE1E.9030406@colbyconsulting.com> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> <4AF1EE1E.9030406@colbyconsulting.com> Message-ID: <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> Hi John, <<< The first thing to answer (If I am to do all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. >>> Overview of T-SQL and CLR debugging in SQL Server 2005 http://blogs.msdn.com/sqlclr/archive/2006/06/29/651644.aspx How to: Enable SQL Server 2005 Debugging http://msdn.microsoft.com/en-us/library/s0fk6z6e(VS.80).aspx HTH, --Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 12:12 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes Robert, I don't really want to get in an argument about this. I asked a serious question and got a somewhat brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY answer I have received so that is not particularly encouraging. 8( TSQL, at least as written from inside of SSMS is about like writing GBasic code back in 1982. Well, maybe a tiny step up but not much. You are right, I probably just don't know how, but in what I will call "a real language" I can set breakpoints, I can see a call stack, I have watch statements, I have... simple very basic DEBUG capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in SSMS). I can have forms with controls displaying data, buttons that cause things to happen. I can FTP files, monitor directories for activities. Believe me, if this exists in SSMS I will jump on that! Just don't tell me to go spend thousands to buy a tool that will do some of what C# can already do for me. Modern languages, and the .Net environment in particular have capabilities that (AFAICT) TSQL can not even dream of. TSQL dreams of manipulating data, C# dreams of manipulating ... the world. Robert I am not "fighting learning it", I am light years beyond where I was two years ago, and I learn more TSQL and SQL Server every day. However from what I have seen so far, TSQL will NEVER be what I need for the supervisor. And answers like you just provided aren't really much help. I am a consultant. I am a programmer, not a DBA, or a SQL Server administrator, or a SQL Server jock at a company somewhere. I need C# for automating my business, which is NOT being a DBA (I already said that I think). I will write C# applications for clients that use SQL Server, and others that do not use SQL Server. SQL Server is used by exactly ONE of my clients, so I cannot spend the time to do whatever it is that you envision SQL Server doing because I cannot amortize the learning over other clients. Now... if you want to answer my question, I am all ears. The first thing to answer (If I am to do all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. Let's take a concrete, real world example. I have a set of virtual machines which run a third party application. I place files into an "inbox" directory, and I wait for files to pop out of the running application into the "outbox" directory. Just an aside, the files placed into the inbox are created by a set of about 5 stored procedures, which can operate on any of dozens of databases in my server. It is the case that the third party application occasionally hangs. If it does I need to rename a file in a specific directory to cause that application to restart processing that file. It is my intention to use C# to execute the SPs to create the files, move the files into the VM, then monitor files dropped into the inbox, and files popping out into the outbox. Look for "no new files in X minutes" and if that condition exists, rename the file. Log files dropped into the in-box, files popping out into the out-box, and the time it took for the files to process through the application. Log any "hangs" restarted. Supervisor stuff. Explain to me how (and more importantly WHY) I would use TSQL to do that? Other than a "when TSQL is all you have" (which is NOT true for me) mentality. A SMALL part of my business is to perform some moderately complex processes using SQL Server and SPs as a tool. I envision C# as a supervisor, not a "middle layer". I envision an application (written in C#) that runs on my desktop which starts, then monitors SQL Server processes, displays errors, restarts processes if necessary, creates databases, tables etc, backs them up and detaches them when done, logging everything that happens, showing it all to me me in real time in a Windows application. Possibly even automatically generating invoices for the work done and emailing them to the client. Explain to me one more time (with feeling) how (and more importantly WHY) I would use TSQL to do all of that stuff? John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > Sorry, but using C# to do what you would and could do simply using > SQL Server is adding a middle layer you do not need. You can do > exactly the same thing using SQL Server and stored procedures and > writing to a log table about what the process is doing and keeping > track of it that way. And, not have to use a middle layer language > like C# to do it. > > I am not sure why writing T-SQL code is not like real programming to > you. But, it is. And, if you are going to continue using SQL > Server, you need to learn what it can do for you instead of always > fighting learning it. > > This is coming from someone that does T-SQL, C#, VB.Net and Access > VBA. So, I think I have a fair perspective of how they all work. _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4574 (20091104) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4574 (20091104) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Wed Nov 4 17:01:33 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 04 Nov 2009 18:01:33 -0500 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: References: Message-ID: <4AF207CD.2060307@colbyconsulting.com> Gustav, >as I'll never be more than a decent low-level DBA. Likewise. And that is precisely my problem. I think when you spend all day in TSQL it is easy to lose sight of just how crude the environment is. I have finally gotten to the point where I can write a SP, debug it, get it functioning etc but it is just hard. Not that C# isn't hard, but C# is a general purpose language where the effort spent is useful for other things. And with the debugging tools in .Net, debugging things is really pretty easy, very similar to many other environments. I can go weeks without doing SQL development and when I go back it is a PITA to pick back up. What it does, it does well, what it doesn't do... it doesn't do quite spectacularly. > I guess Robert is a very skilled DBA and can write T-SQL with his left foot It certainly seems so. But with my current understanding of SQL Server and TSQL I cannot imagine doing the big picture of my work strictly with TSQL. With my current understanding, TSQL will only ever be a hand full of processes, 30% of the whole job. Don't get me wrong, there is no alternative to the power of SQL Server, but in the end it is just a place to store data and some tools for getting it out and back in to tables. Everything else is out in the real world. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > I have no answer to you but if it can bring you some relief, I feel much the same. The database is for storing data, a business layer and/or a front end is for the logic and interface. > That said, nothing is perfect, and in the real world you will have to write some T-SQL or similar but I try to keep it at minimum, first because of the trouble of maintenance (for example, no version control I know of), second because I feel old when I write "GBasic style" code. And I'm not in a big corporation where I can get every expensive tool I can point at, and tools for SQL Server seem all to be enterprise-priced. So, writing views and T-SQL is because I have to, writing C# (and VBA for that matter) is because I like to. > > I guess Robert is a very skilled DBA and can write T-SQL with his left foot and that's very good - then we know where to ask for advice as I'll never be more than a decent low-level DBA. > > /gustav > > >>>> jwcolby at colbyconsulting.com 04-11-2009 22:11 >>> > Robert, > > I don't really want to get in an argument about this. I asked a serious question and got a somewhat > brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY answer I have received > so that is not particularly encouraging. > > 8( > > TSQL, at least as written from inside of SSMS is about like writing GBasic code back in 1982. Well, > maybe a tiny step up but not much. > > You are right, I probably just don't know how, but in what I will call "a real language" I can set > breakpoints, I can see a call stack, I have watch statements, I have... simple very basic DEBUG > capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in SSMS). I can have forms > with controls displaying data, buttons that cause things to happen. I can FTP files, monitor > directories for activities. > > Believe me, if this exists in SSMS I will jump on that! Just don't tell me to go spend thousands to > buy a tool that will do some of what C# can already do for me. > > Modern languages, and the .Net environment in particular have capabilities that (AFAICT) TSQL can > not even dream of. TSQL dreams of manipulating data, C# dreams of manipulating ... the world. > > Robert I am not "fighting learning it", I am light years beyond where I was two years ago, and I > learn more TSQL and SQL Server every day. However from what I have seen so far, TSQL will NEVER be > what I need for the supervisor. And answers like you just provided aren't really much help. > > I am a consultant. I am a programmer, not a DBA, or a SQL Server administrator, or a SQL Server > jock at a company somewhere. I need C# for automating my business, which is NOT being a DBA (I > already said that I think). I will write C# applications for clients that use SQL Server, and > others that do not use SQL Server. SQL Server is used by exactly ONE of my clients, so I cannot > spend the time to do whatever it is that you envision SQL Server doing because I cannot amortize the > learning over other clients. > > Now... if you want to answer my question, I am all ears. The first thing to answer (If I am to do > all this in TSQL) is how do I get a modern tool with all of the debug capabilities to write TSQL code. > > Let's take a concrete, real world example. I have a set of virtual machines which run a third party > application. I place files into an "inbox" directory, and I wait for files to pop out of the > running application into the "outbox" directory. Just an aside, the files placed into the inbox are > created by a set of about 5 stored procedures, which can operate on any of dozens of databases in my > server. > > It is the case that the third party application occasionally hangs. If it does I need to rename a > file in a specific directory to cause that application to restart processing that file. > > It is my intention to use C# to execute the SPs to create the files, move the files into the VM, > then monitor files dropped into the inbox, and files popping out into the outbox. Look for "no new > files in X minutes" and if that condition exists, rename the file. Log files dropped into the > in-box, files popping out into the out-box, and the time it took for the files to process through > the application. Log any "hangs" restarted. Supervisor stuff. > > Explain to me how (and more importantly WHY) I would use TSQL to do that? Other than a "when TSQL > is all you have" (which is NOT true for me) mentality. > > A SMALL part of my business is to perform some moderately complex processes using SQL Server and SPs > as a tool. I envision C# as a supervisor, not a "middle layer". I envision an application (written > in C#) that runs on my desktop which starts, then monitors SQL Server processes, displays errors, > restarts processes if necessary, creates databases, tables etc, backs them up and detaches them when > done, logging everything that happens, showing it all to me me in real time in a Windows > application. Possibly even automatically generating invoices for the work done and emailing them to > the client. > > Explain to me one more time (with feeling) how (and more importantly WHY) I would use TSQL to do all > of that stuff? > > John W. Colby > www.ColbyConsulting.com > > > Robert Stewart wrote: >> Sorry, but using C# to do what you would and could do simply using >> SQL Server is adding a middle layer you do not need. You can do >> exactly the same thing using SQL Server and stored procedures and >> writing to a log table about what the process is doing and keeping >> track of it that way. And, not have to use a middle layer language >> like C# to do it. >> >> I am not sure why writing T-SQL code is not like real programming to >> you. But, it is. And, if you are going to continue using SQL >> Server, you need to learn what it can do for you instead of always >> fighting learning it. >> >> This is coming from someone that does T-SQL, C#, VB.Net and Access >> VBA. So, I think I have a fair perspective of how they all work. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From davidmcafee at gmail.com Wed Nov 4 17:06:37 2009 From: davidmcafee at gmail.com (David McAfee) Date: Wed, 4 Nov 2009 15:06:37 -0800 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> <4AF1EE1E.9030406@colbyconsulting.com> <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> Message-ID: <8786a4c00911041506o49c88352kee08cbbb64fd62b@mail.gmail.com> In SQL 2000 (QA) you would simply right click on the Sproc in the Object explorer and choose debug. Early beta versions of SSMS 2005 had the same ability (right click and choose "step into") but it was removed and moved to Visual Studio instead. :( On Wed, Nov 4, 2009 at 2:48 PM, Shamil Salakhetdinov wrote: > Hi John, > > <<< > The first thing to answer (If I am to do > all this in TSQL) is how do I get a modern tool with all > of the debug capabilities to write TSQL code. >>>> > > Overview of T-SQL and CLR debugging in SQL Server 2005 > http://blogs.msdn.com/sqlclr/archive/2006/06/29/651644.aspx > > How to: Enable SQL Server 2005 Debugging > http://msdn.microsoft.com/en-us/library/s0fk6z6e(VS.80).aspx > > HTH, > > --Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 12:12 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes > > Robert, > > I don't really want to get in an argument about this. ?I asked a serious > question and got a somewhat > brusque "answer" that isn't an answer at all. ?OTOH it is (so far) the ONLY > answer I have received > so that is not particularly encouraging. > > 8( > > TSQL, at least as written from inside of SSMS is about like writing GBasic > code back in 1982. ?Well, > maybe a tiny step up but not much. > > You are right, I probably just don't know how, but in what I will call "a > real language" I can set > breakpoints, I can see a call stack, I have watch statements, ?I have... > simple very basic DEBUG > capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in > SSMS). ?I can have forms > with controls displaying data, buttons that cause things to happen. ?I can > FTP files, monitor > directories for activities. > > Believe me, if this exists in SSMS I will jump on that! ?Just don't tell me > to go spend thousands to > buy a tool that will do some of what C# can already do for me. > > Modern languages, and the .Net environment in particular have capabilities > that (AFAICT) TSQL can > not even dream of. ?TSQL dreams of manipulating data, C# dreams of > manipulating ... the world. > > Robert I am not "fighting learning it", I am light years beyond where I was > two years ago, and I > learn more TSQL and SQL Server every day. ?However from what I have seen so > far, TSQL will NEVER be > what I need for the supervisor. ?And answers like you just provided aren't > really much help. > > I am a consultant. ?I am a programmer, not a DBA, or a SQL Server > administrator, or a SQL Server > jock at a company somewhere. ?I need C# for automating my business, which is > NOT being a DBA (I > already said that I think). ?I will write C# applications for clients that > use SQL Server, and > others that do not use SQL Server. ?SQL Server is used by exactly ONE of my > clients, so I cannot > spend the time to do whatever it is that you envision SQL Server doing > because I cannot amortize the > learning over other clients. > > Now... if you want to answer my question, I am all ears. ?The first thing to > answer (If I am to do > all this in TSQL) is how do I get a modern tool with all of the debug > capabilities to write TSQL code. > > Let's take a concrete, real world example. ?I have a set of virtual machines > which run a third party > application. ?I place files into an "inbox" directory, and I wait for files > to pop out of the > running application into the "outbox" directory. ?Just an aside, the files > placed into the inbox are > created by a set of about 5 stored procedures, which can operate on any of > dozens of databases in my > server. > > It is the case that the third party application occasionally hangs. ?If it > does I need to rename a > file in a specific directory to cause that application to restart processing > that file. > > It is my intention to use C# to execute the SPs to create the files, move > the files into the VM, > then monitor files dropped into the inbox, and files popping out into the > outbox. ?Look for "no new > files in X minutes" and if that condition exists, rename the file. ?Log > files dropped into the > in-box, files popping out into the out-box, and the time it took for the > files to process through > the application. ?Log any "hangs" restarted. ?Supervisor stuff. > > Explain to me how (and more importantly WHY) I would use TSQL to do that? > Other than a "when TSQL > is all you have" (which is NOT true for me) mentality. > > A SMALL part of my business is to perform some moderately complex processes > using SQL Server and SPs > as a tool. ?I envision C# as a supervisor, not a "middle layer". ?I envision > an application (written > in C#) that runs on my desktop which starts, then monitors SQL Server > processes, displays errors, > restarts processes if necessary, creates databases, tables etc, backs them > up and detaches them when > done, logging everything that happens, showing it all to me me in real time > in a Windows > application. ?Possibly even automatically generating invoices for the work > done and emailing them to > the client. > > Explain to me one more time (with feeling) how (and more importantly WHY) I > would use TSQL to do all > of that stuff? > > John W. Colby > www.ColbyConsulting.com > > > Robert Stewart wrote: >> Sorry, but using C# to do what you would and could do simply using >> SQL Server is adding a middle layer you do not need. You can do >> exactly the same thing using SQL Server and stored procedures and >> writing to a log table about what the process is doing and keeping >> track of it that way. And, not have to use a middle layer language >> like C# to do it. >> >> I am not sure why writing T-SQL code is not like real programming to >> you. ?But, it is. ?And, if you are going to continue using SQL >> Server, you need to learn what it can do for you instead of always >> fighting learning it. >> >> This is coming from someone that does T-SQL, C#, VB.Net and Access >> VBA. ?So, I think I have a fair perspective of how they all work. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Wed Nov 4 17:08:49 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 04 Nov 2009 18:08:49 -0500 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> <4AF1EE1E.9030406@colbyconsulting.com> <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> Message-ID: <4AF20981.3060804@colbyconsulting.com> Thanks for that. I guess I will need to install my professional edition license to go there. Worthwhile though I am guessing. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > <<< > The first thing to answer (If I am to do > all this in TSQL) is how do I get a modern tool with all > of the debug capabilities to write TSQL code. > > Overview of T-SQL and CLR debugging in SQL Server 2005 > http://blogs.msdn.com/sqlclr/archive/2006/06/29/651644.aspx > > How to: Enable SQL Server 2005 Debugging > http://msdn.microsoft.com/en-us/library/s0fk6z6e(VS.80).aspx > > HTH, > > --Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 12:12 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes > > Robert, > > I don't really want to get in an argument about this. I asked a serious > question and got a somewhat > brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY > answer I have received > so that is not particularly encouraging. > > 8( > > TSQL, at least as written from inside of SSMS is about like writing GBasic > code back in 1982. Well, > maybe a tiny step up but not much. > > You are right, I probably just don't know how, but in what I will call "a > real language" I can set > breakpoints, I can see a call stack, I have watch statements, I have... > simple very basic DEBUG > capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in > SSMS). I can have forms > with controls displaying data, buttons that cause things to happen. I can > FTP files, monitor > directories for activities. > > Believe me, if this exists in SSMS I will jump on that! Just don't tell me > to go spend thousands to > buy a tool that will do some of what C# can already do for me. > > Modern languages, and the .Net environment in particular have capabilities > that (AFAICT) TSQL can > not even dream of. TSQL dreams of manipulating data, C# dreams of > manipulating ... the world. > > Robert I am not "fighting learning it", I am light years beyond where I was > two years ago, and I > learn more TSQL and SQL Server every day. However from what I have seen so > far, TSQL will NEVER be > what I need for the supervisor. And answers like you just provided aren't > really much help. > > I am a consultant. I am a programmer, not a DBA, or a SQL Server > administrator, or a SQL Server > jock at a company somewhere. I need C# for automating my business, which is > NOT being a DBA (I > already said that I think). I will write C# applications for clients that > use SQL Server, and > others that do not use SQL Server. SQL Server is used by exactly ONE of my > clients, so I cannot > spend the time to do whatever it is that you envision SQL Server doing > because I cannot amortize the > learning over other clients. > > Now... if you want to answer my question, I am all ears. The first thing to > answer (If I am to do > all this in TSQL) is how do I get a modern tool with all of the debug > capabilities to write TSQL code. > > Let's take a concrete, real world example. I have a set of virtual machines > which run a third party > application. I place files into an "inbox" directory, and I wait for files > to pop out of the > running application into the "outbox" directory. Just an aside, the files > placed into the inbox are > created by a set of about 5 stored procedures, which can operate on any of > dozens of databases in my > server. > > It is the case that the third party application occasionally hangs. If it > does I need to rename a > file in a specific directory to cause that application to restart processing > that file. > > It is my intention to use C# to execute the SPs to create the files, move > the files into the VM, > then monitor files dropped into the inbox, and files popping out into the > outbox. Look for "no new > files in X minutes" and if that condition exists, rename the file. Log > files dropped into the > in-box, files popping out into the out-box, and the time it took for the > files to process through > the application. Log any "hangs" restarted. Supervisor stuff. > > Explain to me how (and more importantly WHY) I would use TSQL to do that? > Other than a "when TSQL > is all you have" (which is NOT true for me) mentality. > > A SMALL part of my business is to perform some moderately complex processes > using SQL Server and SPs > as a tool. I envision C# as a supervisor, not a "middle layer". I envision > an application (written > in C#) that runs on my desktop which starts, then monitors SQL Server > processes, displays errors, > restarts processes if necessary, creates databases, tables etc, backs them > up and detaches them when > done, logging everything that happens, showing it all to me me in real time > in a Windows > application. Possibly even automatically generating invoices for the work > done and emailing them to > the client. > > Explain to me one more time (with feeling) how (and more importantly WHY) I > would use TSQL to do all > of that stuff? > > John W. Colby > www.ColbyConsulting.com > > > Robert Stewart wrote: >> Sorry, but using C# to do what you would and could do simply using >> SQL Server is adding a middle layer you do not need. You can do >> exactly the same thing using SQL Server and stored procedures and >> writing to a log table about what the process is doing and keeping >> track of it that way. And, not have to use a middle layer language >> like C# to do it. >> >> I am not sure why writing T-SQL code is not like real programming to >> you. But, it is. And, if you are going to continue using SQL >> Server, you need to learn what it can do for you instead of always >> fighting learning it. >> >> This is coming from someone that does T-SQL, C#, VB.Net and Access >> VBA. So, I think I have a fair perspective of how they all work. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From marklbreen at gmail.com Thu Nov 5 01:43:49 2009 From: marklbreen at gmail.com (Mark Breen) Date: Thu, 5 Nov 2009 15:43:49 +0800 Subject: [dba-VB] Fwd: How do you run SQL Server processes In-Reply-To: References: <4AF207CD.2060307@colbyconsulting.com> Message-ID: Hello John, FWIW, I agree with you that you need a rich environment to do the work that you plan to do. However, I think that Robert is correct in suggestion to try, as much as possible to not cross the technology stack with volumes of data when you do not have to. IOW, do all the heavy lifting in the database, and by all means return as many parameters or log results back to the front end you are building in C#, but try to avoid returning your data when ever possible. I guess I was suggesting the same with with SSIS. Where you have to export data based on decisions, SSIS is an extremely fast option. Have you already used EXECUTE command to call an sproc within an sproc? This is a useful means of creating an sproc, that running a set of sprocs, and can insert / update a log table as it progresses. For data only actvity, this might be effective. For all the front end work however, I guess you will make C# sing, like only you can, so work away, and let TSQL do what it is good at, all the rest should not involve enormous volumes of data, so the technology bridge will be irrelevant. Good luck, you know you are lucky. Most Database people never get to read about a db as big as what you have :) On last thing, in SQL Server, Cursors are a dirty word, I never see them used. However, when you have low volumes of iteration (thousands or a few tens of thousands), cursors are a very powerful way of enumerating tasks. Eg, lets imagine you had a table of states, and you needed to perform ten seperate sprocs for each state. You could use a cursor to enumerate the states table and pass in the state name to the ten different sprocs. You could log between each iteration, and you could have the whole thing running in 30 minutes. In this sort of context, I agree with Robert, that TSQL can give you Raw power quickly and easily. C# could be watching your log table and your 'Status Update' table and simply returning 1 or 2 records from the db to C# per minute, whereas within your sproc that is enumerating it might be handling millions of records each two minutes. Those millions would never cross the stack divide of DB Server to .Net. Again this sharing of processing power is probably what Robert was referring to. BTW I recently started using FileZilla and find it super. Good Luck, Mark Mark From shamil at smsconsulting.spb.ru Thu Nov 5 07:01:18 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 16:01:18 +0300 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <4AF20981.3060804@colbyconsulting.com> References: <4af1ca07.0637560a.20ef.1362@mx.google.com> <4AF1EE1E.9030406@colbyconsulting.com> <00b401ca5da0$f10752e0$d315f8a0$@spb.ru> <4AF20981.3060804@colbyconsulting.com> Message-ID: <00da01ca5e18$119f5080$34ddf180$@spb.ru> Hi John, I just posted links as you asked about this feature but I must note I have used T-SQL debugging just once when I used T-SQL cursors (and that was my mistake I suppose) - IOW I doubt that cursors/T-SQL debugging will be ever needed for you as you plan to use C# to drive your T-SQL's SPs... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 2:09 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes Thanks for that. I guess I will need to install my professional edition license to go there. Worthwhile though I am guessing. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > <<< > The first thing to answer (If I am to do > all this in TSQL) is how do I get a modern tool with all > of the debug capabilities to write TSQL code. > > Overview of T-SQL and CLR debugging in SQL Server 2005 > http://blogs.msdn.com/sqlclr/archive/2006/06/29/651644.aspx > > How to: Enable SQL Server 2005 Debugging > http://msdn.microsoft.com/en-us/library/s0fk6z6e(VS.80).aspx > > HTH, > > --Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 12:12 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes > > Robert, > > I don't really want to get in an argument about this. I asked a serious > question and got a somewhat > brusque "answer" that isn't an answer at all. OTOH it is (so far) the ONLY > answer I have received > so that is not particularly encouraging. > > 8( > > TSQL, at least as written from inside of SSMS is about like writing GBasic > code back in 1982. Well, > maybe a tiny step up but not much. > > You are right, I probably just don't know how, but in what I will call "a > real language" I can set > breakpoints, I can see a call stack, I have watch statements, I have... > simple very basic DEBUG > capabilities that I have never ever ANYWHERE seen mentioned for TSQL (in > SSMS). I can have forms > with controls displaying data, buttons that cause things to happen. I can > FTP files, monitor > directories for activities. > > Believe me, if this exists in SSMS I will jump on that! Just don't tell me > to go spend thousands to > buy a tool that will do some of what C# can already do for me. > > Modern languages, and the .Net environment in particular have capabilities > that (AFAICT) TSQL can > not even dream of. TSQL dreams of manipulating data, C# dreams of > manipulating ... the world. > > Robert I am not "fighting learning it", I am light years beyond where I was > two years ago, and I > learn more TSQL and SQL Server every day. However from what I have seen so > far, TSQL will NEVER be > what I need for the supervisor. And answers like you just provided aren't > really much help. > > I am a consultant. I am a programmer, not a DBA, or a SQL Server > administrator, or a SQL Server > jock at a company somewhere. I need C# for automating my business, which is > NOT being a DBA (I > already said that I think). I will write C# applications for clients that > use SQL Server, and > others that do not use SQL Server. SQL Server is used by exactly ONE of my > clients, so I cannot > spend the time to do whatever it is that you envision SQL Server doing > because I cannot amortize the > learning over other clients. > > Now... if you want to answer my question, I am all ears. The first thing to > answer (If I am to do > all this in TSQL) is how do I get a modern tool with all of the debug > capabilities to write TSQL code. > > Let's take a concrete, real world example. I have a set of virtual machines > which run a third party > application. I place files into an "inbox" directory, and I wait for files > to pop out of the > running application into the "outbox" directory. Just an aside, the files > placed into the inbox are > created by a set of about 5 stored procedures, which can operate on any of > dozens of databases in my > server. > > It is the case that the third party application occasionally hangs. If it > does I need to rename a > file in a specific directory to cause that application to restart processing > that file. > > It is my intention to use C# to execute the SPs to create the files, move > the files into the VM, > then monitor files dropped into the inbox, and files popping out into the > outbox. Look for "no new > files in X minutes" and if that condition exists, rename the file. Log > files dropped into the > in-box, files popping out into the out-box, and the time it took for the > files to process through > the application. Log any "hangs" restarted. Supervisor stuff. > > Explain to me how (and more importantly WHY) I would use TSQL to do that? > Other than a "when TSQL > is all you have" (which is NOT true for me) mentality. > > A SMALL part of my business is to perform some moderately complex processes > using SQL Server and SPs > as a tool. I envision C# as a supervisor, not a "middle layer". I envision > an application (written > in C#) that runs on my desktop which starts, then monitors SQL Server > processes, displays errors, > restarts processes if necessary, creates databases, tables etc, backs them > up and detaches them when > done, logging everything that happens, showing it all to me me in real time > in a Windows > application. Possibly even automatically generating invoices for the work > done and emailing them to > the client. > > Explain to me one more time (with feeling) how (and more importantly WHY) I > would use TSQL to do all > of that stuff? > > John W. Colby > www.ColbyConsulting.com > > > Robert Stewart wrote: >> Sorry, but using C# to do what you would and could do simply using >> SQL Server is adding a middle layer you do not need. You can do >> exactly the same thing using SQL Server and stored procedures and >> writing to a log table about what the process is doing and keeping >> track of it that way. And, not have to use a middle layer language >> like C# to do it. >> >> I am not sure why writing T-SQL code is not like real programming to >> you. But, it is. And, if you are going to continue using SQL >> Server, you need to learn what it can do for you instead of always >> fighting learning it. >> >> This is coming from someone that does T-SQL, C#, VB.Net and Access >> VBA. So, I think I have a fair perspective of how they all work. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4574 (20091104) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4575 (20091105) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Thu Nov 5 07:52:53 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 05 Nov 2009 08:52:53 -0500 Subject: [dba-VB] Fwd: How do you run SQL Server processes In-Reply-To: References: <4AF207CD.2060307@colbyconsulting.com> Message-ID: <4AF2D8B5.1030507@colbyconsulting.com> Mark, > However, I think that Robert is correct in suggestion to try, as much as possible to not cross the technology stack with volumes of data when you do not have to. I have never had a problem with letting each tool do what it does best. In my case (or for this specific client) the database is decidedly not "normal". I have many different databases, each with a few tables, and ZERO (yes ZERO) parent / child tables. I keep tables of names / addresses. That's it. So I am not doing transactions, I am not doing reports, I am not doing data entry, I am not doing any of the "normal" things that people do with databases. What I have to do is get the entire table of name / address out to an external process and then back in to the same database, and I have to do this at least monthly. We are talking 20, 50, 80 million names exported into 2 million record files, processed through this external program, and then imported back in again. This is a "huge" process, not in terms of sheer SQL complexity but rather in terms of the steps involved, the monitoring that has to occur, and the time it takes. And I have to do this on a half dozen large databases, monthly. The external program processes about 2 million records an hour start to finish. Do the math and 60 million records turns into 30 hours of external processing. I use three virtual machines to get three times the records / hour but you are still talking 10 hours and that doesn't count the time to get the records out and back in to SQL Server again. My thought was to start a discussion about that "periphery processing" so that I could get ideas from people who already do it. I am getting almost no response however with actual "I do that using..." responses, and I am wondering why. Perhaps because I deal with small clients, the cost of manual labor to do those kinds of things by hand is too high so they hire me to automate it. Perhaps in a big organization they simply throw people at the problem? Believe me, when you get down to a "one person company" (me!) you can't do this stuff manually for very long, you MUST automate it or you will spend you entire day doing stuff that should be done by the computer. My original question was aimed at discovering how other people handle this kind of processing. I have another client (in Access in this case) that routinely exports data using very specialized export functions (that I wrote) dictated by mainframe interface specs. The data has to be pulled, exported, formatted, sometimes encrypted, sometimes zipped, sometimes FTPd, sometimes emailed, (the "sometimes" part depending on the recipient's preferences), and emails sent announcing the data is on the way etc. I do all of this in Access currently but it is ugly in Access because VBA does not have the built-in tools to do ftp/zip/encrypt etc. C# has some of that built-in and you can usually find C# routines to do the rest. BTW, as I have mentioned, I am taking a C# class at the local community college. I met a Russian gentleman, 60ish, who has been in programming / IT all of his life, recently unemployed. He and I are the only two in the class that have actually worked in the industry (other than the teacher). We struck up a conversation and he asked me if I would like to bid with him on a couple of small C# jobs on a jobs board "just to get some real world experience". I counter-offered that he join me in automating some of my real-world tasks. He came over yesterday and we are going to be working together on the kinds of tasks I described above. He has a LOT of C# experience but almost no database experience beyond simple select queries. We should each learn a lot from the other, it should be fun, and hopefully I can get some of this automation done! John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > FWIW, I agree with you that you need a rich environment to do the work that > you plan to do. > > However, I think that Robert is correct in suggestion to try, as much as > possible to not cross the technology stack with volumes of data when you do > not have to. > > IOW, do all the heavy lifting in the database, and by all means return as > many parameters or log results back to the front end you are building in C#, > but try to avoid returning your data when ever possible. > > I guess I was suggesting the same with with SSIS. Where you have to export > data based on decisions, SSIS is an extremely fast option. > > Have you already used EXECUTE command to call an sproc within an sproc? > This is a useful means of creating an sproc, that running a set of sprocs, > and can insert / update a log table as it progresses. For data only > actvity, this might be effective. > > For all the front end work however, I guess you will make C# sing, like only > you can, so work away, and let TSQL do what it is good at, all the rest > should not involve enormous volumes of data, so the technology bridge will > be irrelevant. > > Good luck, you know you are lucky. Most Database people never get to read > about a db as big as what you have :) > > On last thing, in SQL Server, Cursors are a dirty word, I never see them > used. However, when you have low volumes of iteration (thousands or a few > tens of thousands), cursors are a very powerful way of enumerating tasks. > > Eg, lets imagine you had a table of states, and you needed to perform ten > seperate sprocs for each state. You could use a cursor to enumerate the > states table and pass in the state name to the ten different sprocs. > > You could log between each iteration, and you could have the whole thing > running in 30 minutes. In this sort of context, I agree with Robert, that > TSQL can give you Raw power quickly and easily. > > C# could be watching your log table and your 'Status Update' table and > simply returning 1 or 2 records from the db to C# per minute, whereas within > your sproc that is enumerating it might be handling millions of records each > two minutes. Those millions would never cross the stack divide of DB Server > to .Net. Again this sharing of processing power is probably what Robert was > referring to. > > BTW I recently started using FileZilla and find it super. > > Good Luck, > > Mark > > > > Mark > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From raibeart at gmail.com Thu Nov 5 09:16:01 2009 From: raibeart at gmail.com (Robert Stewart) Date: Thu, 05 Nov 2009 09:16:01 -0600 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: References: Message-ID: <4af2ec32.0305560a.75b1.0901@mx.google.com> John, Mark has it exactly correct. Use SQL Server to the max of what it can do for you before you go off creating something in c# to do essentially the same things. Data is what it does, C# is the GUI. Very similar to what we have all done with MS Access over the years. Now, lets get down to what you are doing... Running code out of a database table is not efficient. The optimizer has to work on it before it can be run. So, the processing will be slower. It would be better to have the code one or more stored procedures with all the parameters and run them against small data sets to get them optimized. Then they would be ready for the larger chunks of data. Each SP can write to a log table and your C# GUI can display the progress by reading data out of the log tables. By using the log tables, you can pass start times, records processed and the time at predetermined numbers of records and which stored proc is doing the work at the time, end times, and just about any other information you may want, Including things that cause errors. SSIS is extremely powerful and the real tool that should probably be used for what you want to do. It can do all of the logging, error trapping and handle passing errored records into a different table for analysis. The C# GUI could be used to read the output from the SSIS package to do the monitoring. It is also relatively simple to sopy SSIS packages and change the parameters around on them to allow them to be used for different extractions and builds. Robert At 01:43 AM 11/5/2009, you wrote: >Date: Thu, 5 Nov 2009 15:43:49 +0800 >From: Mark Breen >Subject: [dba-VB] Fwd: How do you run SQL Server processes >To: dba-vb at databaseadvisors.com >Message-ID: > >Content-Type: text/plain; charset=ISO-8859-1 > >Hello John, > >FWIW, I agree with you that you need a rich environment to do the work that >you plan to do. > >However, I think that Robert is correct in suggestion to try, as much as >possible to not cross the technology stack with volumes of data when you do >not have to. > >IOW, do all the heavy lifting in the database, and by all means return as >many parameters or log results back to the front end you are building in C#, >but try to avoid returning your data when ever possible. > >I guess I was suggesting the same with with SSIS. Where you have to export >data based on decisions, SSIS is an extremely fast option. > >Have you already used EXECUTE command to call an sproc within an sproc? > This is a useful means of creating an sproc, that running a set of sprocs, >and can insert / update a log table as it progresses. For data only >actvity, this might be effective. > >For all the front end work however, I guess you will make C# sing, like only >you can, so work away, and let TSQL do what it is good at, all the rest >should not involve enormous volumes of data, so the technology bridge will >be irrelevant. > >Good luck, you know you are lucky. Most Database people never get to read >about a db as big as what you have :) > >On last thing, in SQL Server, Cursors are a dirty word, I never see them >used. However, when you have low volumes of iteration (thousands or a few >tens of thousands), cursors are a very powerful way of enumerating tasks. > >Eg, lets imagine you had a table of states, and you needed to perform ten >seperate sprocs for each state. You could use a cursor to enumerate the >states table and pass in the state name to the ten different sprocs. > >You could log between each iteration, and you could have the whole thing >running in 30 minutes. In this sort of context, I agree with Robert, that >TSQL can give you Raw power quickly and easily. > >C# could be watching your log table and your 'Status Update' table and >simply returning 1 or 2 records from the db to C# per minute, whereas within >your sproc that is enumerating it might be handling millions of records each >two minutes. Those millions would never cross the stack divide of DB Server >to .Net. Again this sharing of processing power is probably what Robert was >referring to. > >BTW I recently started using FileZilla and find it super. > >Good Luck, > >Mark > > > >Mark From shamil at smsconsulting.spb.ru Thu Nov 5 09:22:02 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 18:22:02 +0300 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <4AF0F117.20209@colbyconsulting.com> References: <4AF0F117.20209@colbyconsulting.com> Message-ID: <00db01ca5e2b$bad004c0$30700e40$@spb.ru> Hi John, <<< I am hoping to eventually make the entire process "table driven" where I can store (in a table) the name of the sp, the parameters in a child table etc. >>> Using table-driven process could be not quite right in C#/.Net world, which proposes many other options to implement generic solutions - I'd just start with "hardcoded" solutions using calls to a set of small "generic" classes/methods executing custom (MS SQL) database processing, when that set of methods/functions stabilizes I'd try to generalize/"generic-ize" it if that will be needed at all... What is the purpose of your "going generic" for this sets of tasks/customer? Isn't it "just for fun" of generic programming? Would that "going generic" make your programming more effective? Developing/debugging/testing generic custom "table-driven"/"other-metadata media driven" solutions is always (much) more time consuming than "hardcoded" ones, and supporting "table-driven" generic custom solutions, especially in the context of constantly changing customer requirements also promise to be rather time consuming - I mean generic "table-"/metadata-driven solutions could only(?) pay back well in the case they are used on many deployment sites by many people. I can be wrong, just talking from my experience, which is not universal of course... I'd better propose to use metadata descriptions/specs (stored in db tables or (XML) files), if you're so bored to write repetitive code, to generate "hardcoded" solutions, then make quick fixes to this generated hardcode to satisfy your customers' current requests, then (in free time) adjust code generator, then when everything will stabilize (will that ever happen?) develop truly generic "table-driven" solution. You can also consider using F# and Windows Workflow Foundation (WF)... The following presentation seems to be useful to get true picture of "generic vs. specific" solutions trade-offs: http://www.infoq.com/presentations/Generic-Specific-Tradeoffs-Stefan-Tilkov Thank you. -- Shamil P.S. FYI: AFAIKR MS has got created Domain Specific Language (DSL) team to develop DSL tools somewhere in year 2004 and they planned to come with first release in one(?) year but they aren't yet there (?)... -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 04, 2009 6:12 AM To: VBA; Dba-Sqlserver Subject: [dba-VB] How do you run SQL Server processes I have processes that I run which use sets of stored procedures. For example two of the processes export / import records from large tables for address validation. The addresses need to be sorted in Zip5/Zip4/Address order in order to make the external validation process as fast as possible so I create a table that pulls millions of records, often tens of millions of records, into a table (created on-the-fly) in sorted order, then pulls out 2 million records at a time and BCPs them out for processing in an external program. The inverse occurs when the records finish processing, 2 million record chunks are imported back into a temp table, then each 2 million record chunk is appended to a main table. Once back in SQL Server I have processes that build clustered and non-clustered indexes, run sha-1 hash functions to generate hash keys, update address valid codes etc. I will run this entire project at least once a month (to keep the addresses valid as people move), and I do so for many tables - a table of about 65 million addresses, another of 21 million, another of 11 million etc. I store all of my stored procedures in a dedicated database that pretty much just contains the SPs and a few UDFs as well as a couple of logging type tables. At the moment I use an Access database to run the individual stored procedures in specific order. I am just embarking on a project to move that functioning but not very efficient Access database to C#. I want to be able to have better monitoring of SP completion, records processed (counts), time to complete each SP etc. I found a pretty nice set of C# classes that allow creation of "stored procedure" objects, with parameter objects, then stored into a collection as the SPs (class instances) are assembled and finally the whole collection of SPs executed. I created a test database of a million names / addresses and today I started testing my existing SPs on this small data set. In the next few days I expect to build out a preliminary "port" of the existing Access VBA code. I am hoping to eventually make the entire process "table driven" where I can store (in a table) the name of the sp, the parameters in a child table etc. and then have a supervisor pull out the records and execute the SPs based on what is in the table. This would allow me to create the SPs and then use a C# application (data entry forms) to organize the SPs, add / delete SPs from the process and so forth. That is down the road of course, for now I will have hard coded sequences - as I already do in supervisor SPs as well as in hard coded Access VBA modules. As a programmer, doing this supervisor level in C# feels more natural to me (than doing it inside of SQL Server), and gives me the flexibility to do what I want in code external to the SQL Server database being manipulated. I want to be able to do things external to the database processes, things like monitor the directories that the BCP processes export to / from, move files popping out of a bcp process off to a VM on a different server, monitor those VMs for files coming out of their processes, moving the files back over to the SQL Server directories, and trigger BCP processes that import the data back in to SQL server. I need to be able to FTP order and data files to clients, and get (FTP) files from process bureaus. So it FEELS to me like using a language such as C# to build a system outside of SQL Server is the way to go. BTW I stumbled across something called SMO (SQL Server Management Objects) which makes it really easy to get at the SQL Server object model from C#. I can apparently see the entire tree structure that is a server/database/tables/etc. and do I know not what (yet) with those objects. Looks pretty cool though. The database is a large part of what I do but it is not the whole picture and I have been handcuffed by my lack of ability to effectively do these external parts. But I am a programmer at heart. I want to know how you do this kind of stuff. I know that running such "systems" of stored procedures tied to external processes must be quite common and I am wondering what you guys do for supervising your processes. If anyone doing anything remotely similar would take the time to explain how you do it, I would be most appreciative. Thanks, -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4573 (20091104) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From shamil at smsconsulting.spb.ru Thu Nov 5 09:39:05 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 18:39:05 +0300 Subject: [dba-VB] Recommended watching: "Thoughts on the Generic vs. Specific Tradeoff" Message-ID: <00e801ca5e2e$1c7f93f0$557ebbd0$@spb.ru> Hi All, The following presentation is a one from a wise man talking from his real life experience - it's fun and so useful to watch, recommended: http://www.infoq.com/presentations/Generic-Specific-Tradeoffs-Stefan-Tilkov Thank you. -- Shamil From jwcolby at colbyconsulting.com Thu Nov 5 10:08:36 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 05 Nov 2009 11:08:36 -0500 Subject: [dba-VB] SPAM-LOW: Re: How do you run SQL Server processes In-Reply-To: <4af2ec32.0305560a.75b1.0901@mx.google.com> References: <4af2ec32.0305560a.75b1.0901@mx.google.com> Message-ID: <4AF2F884.2040903@colbyconsulting.com> Robert, Thanks for the reply. The issue here is that I have to run the same code against a large set of databases / tables. Fir this discussion, the "process" is exporting millions of records to CSV and later importing the resulting CSVs back in again. For example I have about 6 different databases, each with it's own set of tables. Additionally I have to do this exact same process on "Orders" where I am selecting anywhere from a few thousand to a few million addresses to fill an order. The client wants those orders processed as well. Understand that I am not claiming that the following is the most efficient, just "what I know how to do". So my solution so far has been to build a set of stored procedures where I pass in the database and the source table (for the export process) and dynamically build up SQL statements in the SP. The generic process is to: 1) SP1: Build a big table to hold the entire set of records, sorted on Zip5/Zip4, with an autonumber for selection of 2 million record chunks. Move the records from TblSrc (or ViewSrc) to TblBig. Build an index on autonumber and the fields to export to make the subsequent processing faster. Params DB, TblNameSrc. 2) SP2: Build a chunk table. Move a chunk from TblBig to TblChunk Params: DBName, StartPKID, EndPKID 3) SP3: BCP TblChunk to CSV file Params: DBName, Destination Dir This is not an exact representation of what is going on but gets the point across, which is that I have to do this on a "random" database, probably a dozen a month. Initially I did this in stored procedures stored inside of each database but the maintenance of the stored procedures became a nightmare. As I would figure out a better way, a different way, fix a bug... I would be constantly trying to modify my code in each database. NOT FUN!!! Eventually I moved to the parameterized stored procedure with dynamically constructed SQL executed inside of try/catch blocks etc. I store all of these SPs out in a "management" database and execute them from there, passing in the database name, table name and any other parameters required to make each SP work. As I have said, I don't know enough to determine whether this is "the best way", I have enough problems just getting this much to work. I have in fact started using logging from inside of each SP. I designed a standard log table with fields for the SP name being processed, record counts, error codes if any, memo field so I can write little explanation of each step and so forth. I JUST started doing that. I created a SP (again parameterized, stored in the management db) that performs the write to the log, then I call that SP whenever I want to log anything. The thing everyone following this thread needs to understand is that I have no resource other than this list and Google to learn how to do this stuff. I work alone, I do what I have to, as quickly as I reasonably can, because my job (for this client) is to process data, NOT write code. I am a consultant providing a service, NOT a DBA. That said, I have no one but myself to assist me in automating this stuff so that I do not do every step manually, and believe me, when I started I did EVERY STEP manually. Everything that is now a SP I did manually, sometimes dozens of times over the period I have worked for this client. It was not only UGLY, it was DAMNED UGLY. I am a programmer by trade, a DBA/Programmer by necessity, a consultant/ business analyst/ DBA/ Programmer by job description. I view ALL of this from "How can I most quickly, with the least effort, make my job easier". I don't have the luxury of learning anything that I don't HAVE to know, right now, to do my job. I also don't have the luxury of spending weeks working on one specific problem. Do a task for client A, move on to the task for client B. At the end of the month, bill for my time. Thanks for the hint to use SSIS. That is one of those "haven't had the time to learn it" things but I have always known I probably should. When you are up to your ass in alligators, it is hard to STOP... and study SSIS. John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > John, > > Mark has it exactly correct. > > Use SQL Server to the max of what it can do for you before you go off > creating something in c# to do essentially the same things. Data is what > it does, C# is the GUI. Very similar to what we have all done with MS > Access over the years. > > Now, lets get down to what you are doing... > > Running code out of a database table is not efficient. The optimizer has to > work on it before it can be run. So, the processing will be slower. > > It would be better to have the code one or more stored procedures with all the > parameters and run them against small data sets to get them > optimized. Then they > would be ready for the larger chunks of data. > > Each SP can write to a log table and your C# GUI can display the progress by > reading data out of the log tables. > > By using the log tables, you can pass start times, records processed and the > time at predetermined numbers of records and which stored proc is doing the > work at the time, end times, and just about any other information you may want, > Including things that cause errors. > > SSIS is extremely powerful and the real tool that should probably be used for > what you want to do. It can do all of the logging, error trapping and handle > passing errored records into a different table for analysis. The C# GUI could > be used to read the output from the SSIS package to do the monitoring. It is > also relatively simple to sopy SSIS packages and change the parameters around > on them to allow them to be used for different extractions and builds. > > Robert From jwcolby at colbyconsulting.com Thu Nov 5 10:33:38 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 05 Nov 2009 11:33:38 -0500 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <00db01ca5e2b$bad004c0$30700e40$@spb.ru> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> Message-ID: <4AF2FE62.2050501@colbyconsulting.com> Shamil, > Using table-driven process could be not quite right in C#/.Net world, which proposes many other options to implement generic solutions - I'd just start with "hardcoded" solutions using calls to a set of small "generic" That is precisely what I do right now, and what I will do for the near future. The table driven thing is simply that I write "processes" as discussed in this thread. A process might be: Export To Accuzip: "Export dbX TblY to files in DirectoryZ". However that breaks down onto a set of a handful of stored procedures, with passed in parameters. Now, the process "Export To Accuzip" is applied against 6 or 8 different databases every month, but ALSO against random databases which are orders. Orders come in every month and they are a whole nother discussion, but PART of an order is the process Export to Accuzip. DatabaseA exports files to its own directory, DatabaseB to its own directory, OrderAA to its own directory and so forth. But the EXPORT itself is a very standard process. So I am examining the concept of storing the process in a table, so that I can simply select (from a form: Process X: Use SP1, params use SP4, Params use SP9, Params Process Y: Use SP1, Params Use SP2, Params Use SP5, Params Process Z: Use... Table driven (to the extent possible). Then a supervisor application could execute a process (set of SPs), read out of a table, passing in only the database name and possibly a directory on disk. Everything else is selected and set up in advance. To see what a process is, just open a table, to modify the process, select different stored procedures. This stuff is VERY standard, with just a few things that change from DB to db (the database name and where files are going) however I do occasionally shufffle around the interior processes (indexes created, fields used etc. That stuff is internal to the SP and if I change that it is in the SP itself. Then that change is applied to every database from then on. I am not saying that this will work but as my understand and knowledge changes I am spending a LOT of time rewriting stuff. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > <<< > I am hoping to eventually make the entire process "table driven" where I can > store (in a table) the > name of the sp, the parameters in a child table etc. > Using table-driven process could be not quite right in C#/.Net world, which > proposes many other options to implement generic solutions - I'd just start > with "hardcoded" solutions using calls to a set of small "generic" > classes/methods executing custom (MS SQL) database processing, when that set > of methods/functions stabilizes I'd try to generalize/"generic-ize" it if > that will be needed at all... > > What is the purpose of your "going generic" for this sets of tasks/customer? > > Isn't it "just for fun" of generic programming? > Would that "going generic" make your programming more effective? > > Developing/debugging/testing generic custom "table-driven"/"other-metadata > media driven" solutions is always (much) more time consuming than > "hardcoded" ones, and supporting "table-driven" generic custom solutions, > especially in the context of constantly changing customer requirements also > promise to be rather time consuming - I mean generic > "table-"/metadata-driven solutions could only(?) pay back well in the case > they are used on many deployment sites by many people. I can be wrong, just > talking from my experience, which is not universal of course... > > I'd better propose to use metadata descriptions/specs (stored in db tables > or (XML) files), if you're so bored to write repetitive code, to generate > "hardcoded" solutions, then make quick fixes to this generated hardcode to > satisfy your customers' current requests, then (in free time) adjust code > generator, then when everything will stabilize (will that ever happen?) > develop truly generic "table-driven" solution. > > You can also consider using F# and Windows Workflow Foundation (WF)... > > The following presentation seems to be useful to get true picture of > "generic vs. specific" solutions trade-offs: > > http://www.infoq.com/presentations/Generic-Specific-Tradeoffs-Stefan-Tilkov > > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Thu Nov 5 11:13:22 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 20:13:22 +0300 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <4AF2FE62.2050501@colbyconsulting.com> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> <4AF2FE62.2050501@colbyconsulting.com> Message-ID: <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> Hi John, If your "Process X", "Process Y",... calls to stored procedures do have constant parameters' values then I do not see (I can be missing them) clear reasons to use table-driven approach - I'd just have hardcoded ProcessX, ProcessY, ... calling C# wrapper classes' methods to execute SPs. And the latter wrapper classes can inherit from some generic base class(es) to keep executing context, collect statistics etc... As I noted previously WF (Windows Workflow Foundation) could be a good candidate for your subject area but WF has a steep learning curve so you will not be able to effectively apply it right now but with small dedicated classes, which you're writing right now (and delivering your custom solutions right now) you'll be able to relatively easy migrate to WF based solutions in the future. And if you will spend time working on generic custom table-driven solution you'll probably have to throw it away later when migrating to WF... Before you get comfortable with WF you can code you ProcessX, ProcessY, ... steps as "meta-process" text or xml files keeping fully qualified class names for your dedicated classes. And your "generic execution engine" will be just a simple code reading "meta-process" description, instantiating custom classes using .NET Reflection, calling their generic context/initialization methods, running SP by calling generic Run() method (parameters setting is inside your custom classes), then calling generic termination methods and iterating to the next step/finishing when "meta-process" file description ends. That could be it. Thank you. --Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 7:34 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] How do you run SQL Server processes Shamil, > Using table-driven process could be not quite right in C#/.Net world, which proposes many other options to implement generic solutions - I'd just start with "hardcoded" solutions using calls to a set of small "generic" That is precisely what I do right now, and what I will do for the near future. The table driven thing is simply that I write "processes" as discussed in this thread. A process might be: Export To Accuzip: "Export dbX TblY to files in DirectoryZ". However that breaks down onto a set of a handful of stored procedures, with passed in parameters. Now, the process "Export To Accuzip" is applied against 6 or 8 different databases every month, but ALSO against random databases which are orders. Orders come in every month and they are a whole nother discussion, but PART of an order is the process Export to Accuzip. DatabaseA exports files to its own directory, DatabaseB to its own directory, OrderAA to its own directory and so forth. But the EXPORT itself is a very standard process. So I am examining the concept of storing the process in a table, so that I can simply select (from a form: Process X: Use SP1, params use SP4, Params use SP9, Params Process Y: Use SP1, Params Use SP2, Params Use SP5, Params Process Z: Use... Table driven (to the extent possible). Then a supervisor application could execute a process (set of SPs), read out of a table, passing in only the database name and possibly a directory on disk. Everything else is selected and set up in advance. To see what a process is, just open a table, to modify the process, select different stored procedures. This stuff is VERY standard, with just a few things that change from DB to db (the database name and where files are going) however I do occasionally shufffle around the interior processes (indexes created, fields used etc. That stuff is internal to the SP and if I change that it is in the SP itself. Then that change is applied to every database from then on. I am not saying that this will work but as my understand and knowledge changes I am spending a LOT of time rewriting stuff. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > <<< > I am hoping to eventually make the entire process "table driven" where I can > store (in a table) the > name of the sp, the parameters in a child table etc. > Using table-driven process could be not quite right in C#/.Net world, which > proposes many other options to implement generic solutions - I'd just start > with "hardcoded" solutions using calls to a set of small "generic" > classes/methods executing custom (MS SQL) database processing, when that set > of methods/functions stabilizes I'd try to generalize/"generic-ize" it if > that will be needed at all... > > What is the purpose of your "going generic" for this sets of tasks/customer? > > Isn't it "just for fun" of generic programming? > Would that "going generic" make your programming more effective? > > Developing/debugging/testing generic custom "table-driven"/"other-metadata > media driven" solutions is always (much) more time consuming than > "hardcoded" ones, and supporting "table-driven" generic custom solutions, > especially in the context of constantly changing customer requirements also > promise to be rather time consuming - I mean generic > "table-"/metadata-driven solutions could only(?) pay back well in the case > they are used on many deployment sites by many people. I can be wrong, just > talking from my experience, which is not universal of course... > > I'd better propose to use metadata descriptions/specs (stored in db tables > or (XML) files), if you're so bored to write repetitive code, to generate > "hardcoded" solutions, then make quick fixes to this generated hardcode to > satisfy your customers' current requests, then (in free time) adjust code > generator, then when everything will stabilize (will that ever happen?) > develop truly generic "table-driven" solution. > > You can also consider using F# and Windows Workflow Foundation (WF)... > > The following presentation seems to be useful to get true picture of > "generic vs. specific" solutions trade-offs: > > http://www.infoq.com/presentations/Generic-Specific-Tradeoffs-Stefan-Tilkov > > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4576 (20091105) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Thu Nov 5 11:32:51 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 05 Nov 2009 12:32:51 -0500 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> <4AF2FE62.2050501@colbyconsulting.com> <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> Message-ID: <4AF30C43.8010508@colbyconsulting.com> Shamil, Listen to this... > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... steps as "meta-process" text or xml files keeping fully qualified class names for your dedicated classes. And your "generic execution engine" will be just a simple code reading "meta-process" description, instantiating custom classes using .NET Reflection, calling their generic context/initialization methods, running SP by calling generic Run() method (parameters setting is inside your custom classes), then calling generic termination methods and iterating to the next step/finishing when "meta-process" file description ends. Now read this. Open form Select stored procedure, select parameter, select parameter. click save. Select stored procedure, select parameter, select parameter. click save. Continue till done. ;) Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? Read your solution again. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > If your "Process X", "Process Y",... calls to stored procedures do have > constant parameters' values then I do not see (I can be missing them) clear > reasons to use table-driven approach - I'd just have hardcoded ProcessX, > ProcessY, ... calling C# wrapper classes' methods to execute SPs. And the > latter wrapper classes can inherit from some generic base class(es) to keep > executing context, collect statistics etc... > > As I noted previously WF (Windows Workflow Foundation) could be a good > candidate for your subject area but WF has a steep learning curve so you > will not be able to effectively apply it right now but with small dedicated > classes, which you're writing right now (and delivering your custom > solutions right now) you'll be able to relatively easy migrate to WF based > solutions in the future. And if you will spend time working on generic > custom table-driven solution you'll probably have to throw it away later > when migrating to WF... > > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... > steps as "meta-process" text or xml files keeping fully qualified class > names for your dedicated classes. And your "generic execution engine" will > be just a simple code reading "meta-process" description, instantiating > custom classes using .NET Reflection, calling their generic > context/initialization methods, running SP by calling generic Run() method > (parameters setting is inside your custom classes), then calling generic > termination methods and iterating to the next step/finishing when > "meta-process" file description ends. That could be it. > > Thank you. > > --Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 7:34 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] How do you run SQL Server processes > > Shamil, From shamil at smsconsulting.spb.ru Thu Nov 5 13:00:15 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 5 Nov 2009 22:00:15 +0300 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <4AF30C43.8010508@colbyconsulting.com> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> <4AF2FE62.2050501@colbyconsulting.com> <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> <4AF30C43.8010508@colbyconsulting.com> Message-ID: <010c01ca5e4a$3735e430$a5a1ac90$@spb.ru> Hi John, <<< Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? >>> Yes, that is also what I mean: developing generic table-driven procedure for you currently could force you to learn more C#/.NET development stuff than when implementing "meta-process description" interpreter, which will consist of several C# code lines. But even the latter is not needed probably as you can hardcode your solutions for every custom task "shuffling" your parameterized wrappers classes/methods calling SPs, and instead of "Select stored procedure, select parameter,..." use "Create C# project, type C# code with calls to the wrapper classes/methods with Stored Procedures names and parameter names, types and values, then compile and run the custom solution". And it should be more suitable to debug/trace latter solution than "table-driven" or "meta-process description driven" ones... OK? (Am I still missing something here?) -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 8:33 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] How do you run SQL Server processes Shamil, Listen to this... > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... steps as "meta-process" text or xml files keeping fully qualified class names for your dedicated classes. And your "generic execution engine" will be just a simple code reading "meta-process" description, instantiating custom classes using .NET Reflection, calling their generic context/initialization methods, running SP by calling generic Run() method (parameters setting is inside your custom classes), then calling generic termination methods and iterating to the next step/finishing when "meta-process" file description ends. Now read this. Open form Select stored procedure, select parameter, select parameter. click save. Select stored procedure, select parameter, select parameter. click save. Continue till done. ;) Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? Read your solution again. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > If your "Process X", "Process Y",... calls to stored procedures do have > constant parameters' values then I do not see (I can be missing them) clear > reasons to use table-driven approach - I'd just have hardcoded ProcessX, > ProcessY, ... calling C# wrapper classes' methods to execute SPs. And the > latter wrapper classes can inherit from some generic base class(es) to keep > executing context, collect statistics etc... > > As I noted previously WF (Windows Workflow Foundation) could be a good > candidate for your subject area but WF has a steep learning curve so you > will not be able to effectively apply it right now but with small dedicated > classes, which you're writing right now (and delivering your custom > solutions right now) you'll be able to relatively easy migrate to WF based > solutions in the future. And if you will spend time working on generic > custom table-driven solution you'll probably have to throw it away later > when migrating to WF... > > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... > steps as "meta-process" text or xml files keeping fully qualified class > names for your dedicated classes. And your "generic execution engine" will > be just a simple code reading "meta-process" description, instantiating > custom classes using .NET Reflection, calling their generic > context/initialization methods, running SP by calling generic Run() method > (parameters setting is inside your custom classes), then calling generic > termination methods and iterating to the next step/finishing when > "meta-process" file description ends. That could be it. > > Thank you. > > --Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 7:34 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] How do you run SQL Server processes > > Shamil, _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4576 (20091105) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4576 (20091105) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From shamil at smsconsulting.spb.ru Sat Nov 7 05:28:32 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 7 Nov 2009 14:28:32 +0300 Subject: [dba-VB] How do you run SQL Server processes In-Reply-To: <010c01ca5e4a$3735e430$a5a1ac90$@spb.ru> References: <4AF0F117.20209@colbyconsulting.com> <00db01ca5e2b$bad004c0$30700e40$@spb.ru> <4AF2FE62.2050501@colbyconsulting.com> <00fc01ca5e3b$48929d90$d9b7d8b0$@spb.ru> <4AF30C43.8010508@colbyconsulting.com> <010c01ca5e4a$3735e430$a5a1ac90$@spb.ru> Message-ID: <018c01ca5f9d$71389c00$53a9d400$@spb.ru> Hi John, Here is a .NET Reflection sample: using System; namespace NetReflectionSample { class Program { static void Main(string[] args) { string[] spRunnersNames = { "NetReflectionSample.SP1", "NetReflectionSample.SP2" }; foreach (string spRunnerName in spRunnersNames) { Type spRunnerType = Type.GetType(spRunnerName); StoredProcedureRunner runner = Activator.CreateInstance(spRunnerType) as StoredProcedureRunner; runner.Init(); runner.Run(); runner.CollectStats(); Console.WriteLine(); } } } public abstract class StoredProcedureRunner { public void Init() { Console.WriteLine("Initializing {0}...", this.GetType().ToString()); } public abstract void Run(); public void CollectStats() { Console.WriteLine("Collecting stats {0}...", this.GetType().ToString()); } } public class SP1 : StoredProcedureRunner { public override void Run() { Console.WriteLine("Running {0}...", this.GetType().ToString()); } } public class SP2 : StoredProcedureRunner { public override void Run() { Console.WriteLine("Running {0}...", this.GetType().ToString()); } } } Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: Thursday, November 05, 2009 10:00 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] How do you run SQL Server processes Hi John, <<< Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? >>> Yes, that is also what I mean: developing generic table-driven procedure for you currently could force you to learn more C#/.NET development stuff than when implementing "meta-process description" interpreter, which will consist of several C# code lines. But even the latter is not needed probably as you can hardcode your solutions for every custom task "shuffling" your parameterized wrappers classes/methods calling SPs, and instead of "Select stored procedure, select parameter,..." use "Create C# project, type C# code with calls to the wrapper classes/methods with Stored Procedures names and parameter names, types and values, then compile and run the custom solution". And it should be more suitable to debug/trace latter solution than "table-driven" or "meta-process description driven" ones... OK? (Am I still missing something here?) -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 05, 2009 8:33 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] How do you run SQL Server processes Shamil, Listen to this... > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... steps as "meta-process" text or xml files keeping fully qualified class names for your dedicated classes. And your "generic execution engine" will be just a simple code reading "meta-process" description, instantiating custom classes using .NET Reflection, calling their generic context/initialization methods, running SP by calling generic Run() method (parameters setting is inside your custom classes), then calling generic termination methods and iterating to the next step/finishing when "meta-process" file description ends. Now read this. Open form Select stored procedure, select parameter, select parameter. click save. Select stored procedure, select parameter, select parameter. click save. Continue till done. ;) Do you remember reading my statement about "learning the minimum required to do the task at hand, right now"? Read your solution again. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > If your "Process X", "Process Y",... calls to stored procedures do have > constant parameters' values then I do not see (I can be missing them) clear > reasons to use table-driven approach - I'd just have hardcoded ProcessX, > ProcessY, ... calling C# wrapper classes' methods to execute SPs. And the > latter wrapper classes can inherit from some generic base class(es) to keep > executing context, collect statistics etc... > > As I noted previously WF (Windows Workflow Foundation) could be a good > candidate for your subject area but WF has a steep learning curve so you > will not be able to effectively apply it right now but with small dedicated > classes, which you're writing right now (and delivering your custom > solutions right now) you'll be able to relatively easy migrate to WF based > solutions in the future. And if you will spend time working on generic > custom table-driven solution you'll probably have to throw it away later > when migrating to WF... > > Before you get comfortable with WF you can code you ProcessX, ProcessY, ... > steps as "meta-process" text or xml files keeping fully qualified class > names for your dedicated classes. And your "generic execution engine" will > be just a simple code reading "meta-process" description, instantiating > custom classes using .NET Reflection, calling their generic > context/initialization methods, running SP by calling generic Run() method > (parameters setting is inside your custom classes), then calling generic > termination methods and iterating to the next step/finishing when > "meta-process" file description ends. That could be it. > > Thank you. > > --Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 05, 2009 7:34 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] How do you run SQL Server processes > > Shamil, From shamil at smsconsulting.spb.ru Sat Nov 7 08:19:11 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 7 Nov 2009 17:19:11 +0300 Subject: [dba-VB] NetReflectionSample - Part I of IV Message-ID: <000601ca5fb5$49b79dd0$dd26d970$@spb.ru> Hi John and all, Just wanted to post a bit more advanced sample on .NET reflection and how it can be used to instantiate custom classes and to execute their parameterized methods based on text definition: using System; using System.Reflection; #region Test Program namespace NetReflectionSample { /// /// Test class /// class Program { static void Main(string[] args) { // List of custom classes (with parameters) to instantiate and // call their Run(...) method string[] spRunnersCalls = { "NetReflectionSample.SP1", "NetReflectionSample.SP2|503|Test String|3.14|11/7/2009", "NetReflectionSample.SP3|123|Test String2" }; foreach (string spRunnerCall in spRunnersCalls) { // Parse current SP call string definition and // return StoredProcedureRunner instance StoredProcedureRunner runner = DynamicMethodCallParser.Parse(spRunnerCall); runner.Init(); // If there are calling parameters use CustomRunMethodInfo to // execute SP call wrapper method, else use default Run() method CustomRunMethodInfo customRunner = runner as CustomRunMethodInfo; if (customRunner.Parameters != null) { customRunner.ParameterizedRunMethodInfo.Invoke( customRunner, customRunner.Parameters); } else runner.Run(); // collect execution stats runner.CollectStats(); Console.WriteLine(); } } } } #endregion Thank you. --Shamil From shamil at smsconsulting.spb.ru Sat Nov 7 08:20:44 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 7 Nov 2009 17:20:44 +0300 Subject: [dba-VB] NetReflectionSample - Part II of IV Message-ID: <000b01ca5fb5$7f9438a0$7ebca9e0$@spb.ru> #region Custom business functionality namespace NetReflectionSample { /// /// SP1 wrapper class simulator /// public class SP1 : StoredProcedureRunner { public override void Run() { Console.WriteLine("Running {0}.Run...", this.GetType().ToString()); } } /// /// SP2 wrapper class simulator /// public class SP2 : StoredProcedureRunner { public void Run( int prm1, string prm2, decimal prm3, DateTime prm4) { Console.WriteLine("Running {0}.Run...", this.GetType().ToString()); Console.WriteLine("Parameters: prm1={0}, prm2={1}, prm3={2}, prm4={3}", prm1, prm2, prm3, prm4); } } /// /// SP3 wrapper class simulator /// public class SP3 : StoredProcedureRunner { public void Run( int prm1, string prm2) { Console.WriteLine("Running {0}.Run...", this.GetType().ToString()); Console.WriteLine("Parameters: prm1={0}, prm2={1}", prm1, prm2); } } } #endregion --Shamil From shamil at smsconsulting.spb.ru Sat Nov 7 08:37:01 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 7 Nov 2009 17:37:01 +0300 Subject: [dba-VB] NetReflectionSample - full code text Message-ID: <001a01ca5fb7$c5cab310$51601930$@spb.ru> Hi All, Here is a full text of the subject sample: http://smsconsulting.spb.ru/samples/NetReflectionSample.txt dba-VB robot currently blocks to post two parts of my four parts posting because they are longer than 20KB - dba-VB robot writes that those two postings are awaiting moderators approval but AFAIHF this approval never happens (at least that was for several my past postings with code) - therefore I'm posting the above link to get the whole code text available. Here are the results of the test program execution: === Initializing NetReflectionSample.SP1... Running NetReflectionSample.SP1.Run... Collecting stats NetReflectionSample.SP1... Initializing NetReflectionSample.SP2... Running NetReflectionSample.SP2.Run... Parameters: prm1=503, prm2=Test String, prm3=3.14, prm4=11/7/2009 12:00:00 AM Collecting stats NetReflectionSample.SP2... Initializing NetReflectionSample.SP3... Running NetReflectionSample.SP3.Run... Parameters: prm1=123, prm2=Test String2 Collecting stats NetReflectionSample.SP3... === Note: posted code is a "quick & dirty" first version, if you have time and desire to make a better version of this sample code you're welcome to do that. Any code review remarks and comments are very welcome. Thank you. -- Shamil From Gustav at cactus.dk Sat Nov 7 14:00:22 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Sat, 07 Nov 2009 21:00:22 +0100 Subject: [dba-VB] NetReflectionSample - full code text Message-ID: Hi Shamil Thanks! Good example. /gustav >>> shamil at smsconsulting.spb.ru 07-11-2009 15:37 >>> Any code review remarks and comments are very welcome. From shamil at smsconsulting.spb.ru Sun Nov 8 15:47:44 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 9 Nov 2009 00:47:44 +0300 Subject: [dba-VB] NetReflectionSample - full code text In-Reply-To: References: Message-ID: <000301ca60bd$1c0d9ca0$5428d5e0$@spb.ru> Thank you, Gustav. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Saturday, November 07, 2009 11:00 PM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] NetReflectionSample - full code text Hi Shamil Thanks! Good example. /gustav >>> shamil at smsconsulting.spb.ru 07-11-2009 15:37 >>> Any code review remarks and comments are very welcome. _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4585 (20091108) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sun Nov 8 20:20:20 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 08 Nov 2009 21:20:20 -0500 Subject: [dba-VB] NetReflectionSample - full code text In-Reply-To: References: Message-ID: <4AF77C64.5090707@colbyconsulting.com> But what does it DO? John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi Shamil > > Thanks! Good example. > > /gustav > >>>> shamil at smsconsulting.spb.ru 07-11-2009 15:37 >>> > > Any code review remarks and comments are very welcome. > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Mon Nov 9 00:11:28 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 9 Nov 2009 09:11:28 +0300 Subject: [dba-VB] NetReflectionSample - full code text In-Reply-To: <4AF77C64.5090707@colbyconsulting.com> References: <4AF77C64.5090707@colbyconsulting.com> Message-ID: <000901ca6103$7adfbd80$709f3880$@spb.ru> Hi John, Basically, this sample code demonstrates how to interpret a simple scripting "birds language" into a sequence of calls to the parameterized methods of custom classes. Substitute text script with meta-table(s) - and you'll get table-driven interpreter. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, November 09, 2009 5:20 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] NetReflectionSample - full code text But what does it DO? John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi Shamil > > Thanks! Good example. > > /gustav > >>>> shamil at smsconsulting.spb.ru 07-11-2009 15:37 >>> > > Any code review remarks and comments are very welcome. > > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4586 (20091108) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From shamil at smsconsulting.spb.ru Tue Nov 10 23:49:05 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Wed, 11 Nov 2009 08:49:05 +0300 Subject: [dba-VB] FYI: ORM, EDM, ESQL, Entity Framework, LINQ to SQL, LINQ to Entities - Confused? Message-ID: <003601ca6292$af1581d0$0d408570$@spb.ru> "ORM, EDM, ESQL, Entity Framework, LINQ to SQL, LINQ to Entities - Confused?" http://www.infoq.com/presentations/ORM-LINQ-Entity-Framework-Eric-Nelson --Shamil From Gustav at cactus.dk Wed Nov 11 12:52:48 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 11 Nov 2009 19:52:48 +0100 Subject: [dba-VB] Microsoft WebsiteSpark Program Message-ID: Hi all Have you heard of Microsoft? WebsiteSpark*? http://www.microsoft.com/web/websitespark/Support.aspx?tab=ProgramDetails It is designed to ignite the success of professional Web developers and designers. The program enables you to get software, support and business opportunities from Microsoft at no upfront costs for 3 years or exit from the program. WebSiteSpark provides access to a technology offering for development and production hosting of websites. Once enrolled, you can download the following software from Microsoft: For design, development, testing and demonstration of new websites * for a total of up to three users per Web design and development company: *Visual Studio Professional *Expression Studio 3 (1 user) and Expression Web 3 (up to 2 users) *Windows Web Server 2008 R2 *SQL Server 2008 Web Edition For production use * that is, to deploy and host new websites developed using Program software * using a total of up to four processors per Web design and development company, of the following (physical or virtual) dedicated servers: *Windows Web Server 2008 R2 *SQL Server 2008 Web Edition So if you consider "something else" than Access, this might help you. /gustav From jwcolby at colbyconsulting.com Wed Nov 11 13:08:24 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 11 Nov 2009 14:08:24 -0500 Subject: [dba-VB] SPAM-LOW: Microsoft WebsiteSpark Program In-Reply-To: References: Message-ID: <4AFB0BA8.8020105@colbyconsulting.com> This appears to be one part of this: http://www.microsoftstartupzone.com/pages/home.aspx which I posted about a month or so ago. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi all > > Have you heard of Microsoft? WebsiteSpark*? > > http://www.microsoft.com/web/websitespark/Support.aspx?tab=ProgramDetails > > > > It is designed to ignite the success of professional Web developers and > designers. The program enables you to get software, support and business > opportunities from Microsoft at no upfront costs for 3 years or exit > from the program. > > WebSiteSpark provides access to a technology offering for development > and production hosting of websites. Once enrolled, you can download the > following software from Microsoft: > > For design, development, testing and demonstration of new websites * > for a total of up to three users per Web design and development company: > > > *Visual Studio Professional > *Expression Studio 3 (1 user) and Expression Web 3 (up to 2 users) > *Windows Web Server 2008 R2 > *SQL Server 2008 Web Edition > > For production use * that is, to deploy and host new websites > developed using Program software * using a total of up to four > processors per Web design and development company, of the following > (physical or virtual) dedicated servers: > > *Windows Web Server 2008 R2 > *SQL Server 2008 Web Edition > > > > So if you consider "something else" than Access, this might help you. > > /gustav > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Thu Nov 12 11:22:55 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 12 Nov 2009 12:22:55 -0500 Subject: [dba-VB] What to do, what to do? Message-ID: <4AFC446F.6070207@colbyconsulting.com> I am writing an application in C# to merge records (previously discussed). It looks pretty simple, all things considered. Get data into a dataset object which will contain about 4 tables. Then use the records and fields collections of the recordset to iterate through the fields building up a record (or modifying the first record of a small set of records - the dupes). Copy a record from one table into another table inside of the dataset. Delete records in a table in the dataset. Etc. Etc all without going back out to the actual data tables on SQL Server until the very end when I will do an update back to the live data. Am I correct in assuming here that I can programmatically append records from one table in the dataset to another record in the dataset? Delete records in the tables in the dataset? Update records in tables in the dataset? All "in memory" without connecting back to the live data in the database? Is there anything I need to know before I launch into this? -- John W. Colby www.ColbyConsulting.com From cfoust at infostatsystems.com Thu Nov 12 11:29:43 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Thu, 12 Nov 2009 09:29:43 -0800 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFC446F.6070207@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> Message-ID: I think that depends on the objects you're using. .Net tends to complain that a record already belongs to another table when you try to import it. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 12, 2009 9:23 AM To: VBA Subject: [dba-VB] What to do, what to do? I am writing an application in C# to merge records (previously discussed). It looks pretty simple, all things considered. Get data into a dataset object which will contain about 4 tables. Then use the records and fields collections of the recordset to iterate through the fields building up a record (or modifying the first record of a small set of records - the dupes). Copy a record from one table into another table inside of the dataset. Delete records in a table in the dataset. Etc. Etc all without going back out to the actual data tables on SQL Server until the very end when I will do an update back to the live data. Am I correct in assuming here that I can programmatically append records from one table in the dataset to another record in the dataset? Delete records in the tables in the dataset? Update records in tables in the dataset? All "in memory" without connecting back to the live data in the database? Is there anything I need to know before I launch into this? -- John W. Colby www.ColbyConsulting.com _______________________________________________ From shamil at smsconsulting.spb.ru Fri Nov 13 00:04:33 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 09:04:33 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFC446F.6070207@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> Message-ID: <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> Hi John -- Here is a sample: using System; using System.Collections.Generic; using System.Text; using System.Data; using System.Data.SqlClient; namespace TestConsole.Samples { public class DataSetTests { public static void Run() { string connectionString = "Data Source=HAMBURG\\SQL2005;"+ "Initial Catalog=Northwind;User Id=sa;Password=N/A"; using (SqlConnection connection = new SqlConnection(connectionString)) { connection.Open(); string sql = "select * from [Shippers]"; string sql1 = "select * from [NewShippers] where(1=0)"; SqlCommand command = new SqlCommand(sql, connection); SqlDataAdapter adapter = new SqlDataAdapter(command); DataSet dataSet = new DataSet("Shippers"); adapter.Fill(dataSet); command = new SqlCommand(sql1, connection); adapter = new SqlDataAdapter(command); DataSet newDataSet = new DataSet("NewShippers"); adapter.Fill(newDataSet); Console.WriteLine("*** newDataSet ***"); foreach (DataRow dataRow in newDataSet.Tables[0].Rows) { Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); } Console.WriteLine("*** dataSet ***"); foreach (DataRow dataRow in dataSet.Tables[0].Rows) { // This row already belongs to another table. //newDataSet.Tables[0].Rows.Add(dataRow); object[] items = { dataRow[0], dataRow[1], dataRow[2] }; newDataSet.Tables[0].Rows.Add(items); Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); } Console.WriteLine("*** newDataSet ***"); foreach (DataRow dataRow in newDataSet.Tables[0].Rows) { Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); } newDataSet.Tables[0].TableName = "NewShippers"; SqlCommandBuilder builder = new SqlCommandBuilder(adapter); adapter.InsertCommand = builder.GetInsertCommand(true); adapter.Update(newDataSet.Tables[0]); } } } } Thank you. --Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 12, 2009 8:23 PM To: VBA Subject: [dba-VB] What to do, what to do? I am writing an application in C# to merge records (previously discussed). It looks pretty simple, all things considered. Get data into a dataset object which will contain about 4 tables. Then use the records and fields collections of the recordset to iterate through the fields building up a record (or modifying the first record of a small set of records - the dupes). Copy a record from one table into another table inside of the dataset. Delete records in a table in the dataset. Etc. Etc all without going back out to the actual data tables on SQL Server until the very end when I will do an update back to the live data. Am I correct in assuming here that I can programmatically append records from one table in the dataset to another record in the dataset? Delete records in the tables in the dataset? Update records in tables in the dataset? All "in memory" without connecting back to the live data in the database? Is there anything I need to know before I launch into this? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4600 (20091112) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From marklbreen at gmail.com Fri Nov 13 00:58:16 2009 From: marklbreen at gmail.com (Mark Breen) Date: Fri, 13 Nov 2009 14:58:16 +0800 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFC446F.6070207@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> Message-ID: Hello John, Just curious, how many records are you talking about - a few hundred or millions? Second question, other than for purely academic reasons, why are you bring the data into C#, to concatenate and insert into another table can so easily be done in one sproc, even if you need a temporary holding place, you would still use a temp table or even a local table. Of course if it is to learn datasets then thats another story, Mark 2009/11/13 jwcolby > I am writing an application in C# to merge records (previously discussed). > It looks pretty simple, > all things considered. Get data into a dataset object which will contain > about 4 tables. Then use > the records and fields collections of the recordset to iterate through the > fields building up a > record (or modifying the first record of a small set of records - the > dupes). Copy a record from > one table into another table inside of the dataset. Delete records in a > table in the dataset. Etc. > Etc all without going back out to the actual data tables on SQL Server > until the very end when I > will do an update back to the live data. > > Am I correct in assuming here that I can programmatically append records > from one table in the > dataset to another record in the dataset? Delete records in the tables in > the dataset? Update > records in tables in the dataset? All "in memory" without connecting back > to the live data in the > database? > > Is there anything I need to know before I launch into this? > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Fri Nov 13 03:27:08 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Fri, 13 Nov 2009 10:27:08 +0100 Subject: [dba-VB] What to do, what to do? Message-ID: Hi John Shamil knows the SqlClient stuff! As he knows, I prefer the datatable route - a matter of taste I think, but I admit working with SqlCommand may allow for some fine tuning not possible with the datatableadapters. Here's an example from an app that updates (syncs) one table from another. Not exactly what you wish to do but it should contain the building blocks. The tricky part here is that the unique id (autonumber) should not be compared. However, the id of a source row is stored in the target row as SyncId for later reference: private void ConvertOrder() { // DataTableAdapter to copy from. VrsCloneTableAdapters.OrderTableAdapter sourceDataTableAdapter = new VrsCloneTableAdapters.OrderTableAdapter(); VrsClone.OrderDataTable sourceDataTable; // DataTableAdapter to copy to. KarneliaTableAdapters.SyncOrderTableAdapter targetDataTableAdapter = new KarneliaTableAdapters.SyncOrderTableAdapter(); Karnelia.SyncOrderDataTable targetDataTable; sourceDataTableAdapter.Connection = ConnectionApplyPassword(sourceDataTableAdapter.Connection.ConnectionString); targetDataTableAdapter.Connection = ConnectionApplyPassword(targetDataTableAdapter.Connection.ConnectionString); // DataTable for source. sourceDataTable = sourceDataTableAdapter.GetDataSync(); // DataTable for target. targetDataTable = targetDataTableAdapter.GetData(); // Read each row from the source rows. // Update those found in the target table. // Append those not found in the target table. // Variable to hold primary key to locate. int key; // Row of target DataTable to be updated. Karnelia.SyncOrderRow targetRow = null; // Variables to hold hash codes of row contents. int sourceHash; int targetHash; int colCount = sourceDataTable.Columns.Count; int colItem; int colOrdinal; string colName; for (colItem = 0; colItem < colCount; colItem++) { colName = sourceDataTable.Columns[colItem].ColumnName; colOrdinal = sourceDataTable.Columns[colItem].Ordinal; targetDataTable.Columns[colName].SetOrdinal(colOrdinal + 1); } foreach (VrsClone.OrderRow sourceRow in sourceDataTable) { // Value of key to locate. key = sourceRow.SyncId; // Find a matching row in the target. targetRow = targetDataTable.FindBySyncId(key); if (targetRow == null) { // Row is not found. targetRow = targetDataTable.NewSyncOrderRow(); targetRow.BeginEdit(); targetRow.VrsOrderId = sourceRow.VrsOrderId; targetRow.ClientId = sourceRow.ClientId; targetRow.StylistId = sourceRow.StylistId; targetRow.DateCreated = sourceRow.DateCreated; targetRow.SyncId = sourceRow.SyncId; targetRow.EndEdit(); targetDataTable.AddSyncOrderRow(targetRow); Console.WriteLine("Insert " + targetDataTable.ToString() + ": " + sourceRow.VrsOrderId.ToString() + " - " + targetRow.RowState.ToString()); } else { // Row is found. // Calculate hash code to see if source row and target row match. // Id of targetrow (first column) is not present in sourcerow, thus // concatenate first column of targetrow with the columns of sourcerow. sourceHash = String.Concat(targetRow.ItemArray[0], String.Concat(sourceRow.ItemArray)).GetHashCode(); targetHash = String.Concat(targetRow.ItemArray).GetHashCode(); if (sourceHash != targetHash) { // The source row is updated. // Copy full content from source row to target row. // This sets RowState of the target row to Modified. targetRow.BeginEdit(); targetRow.VrsOrderId = sourceRow.VrsOrderId; targetRow.ClientId = sourceRow.ClientId; targetRow.StylistId = sourceRow.StylistId; targetRow.DateCreated = sourceRow.DateCreated; targetRow.EndEdit(); // Do NOT call AcceptChanges here as this would clear property RowState. Console.WriteLine("Update " + targetDataTable.ToString() + ": " + sourceRow.VrsOrderId.ToString() + " - " + targetRow.RowState.ToString()); } } } // Write back to the database table the updated target DataTable. targetDataTableAdapter.Update(targetDataTable); // Call AcceptChanges if needed for further processing. //targetDataTable.AcceptChanges(); } This database contains records by the thousands. How code like this will perform for millions of records has to be tested. /gustav >>> jwcolby at colbyconsulting.com 12-11-2009 18:22 >>> I am writing an application in C# to merge records (previously discussed). It looks pretty simple, all things considered. Get data into a dataset object which will contain about 4 tables. Then use the records and fields collections of the recordset to iterate through the fields building up a record (or modifying the first record of a small set of records - the dupes). Copy a record from one table into another table inside of the dataset. Delete records in a table in the dataset. Etc. Etc all without going back out to the actual data tables on SQL Server until the very end when I will do an update back to the live data. Am I correct in assuming here that I can programmatically append records from one table in the dataset to another record in the dataset? Delete records in the tables in the dataset? Update records in tables in the dataset? All "in memory" without connecting back to the live data in the database? Is there anything I need to know before I launch into this? -- John W. Colby www.ColbyConsulting.com From Gustav at cactus.dk Fri Nov 13 04:48:24 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Fri, 13 Nov 2009 11:48:24 +0100 Subject: [dba-VB] AWS (Amazon Web Services) SDK for .NET and VS Message-ID: Hi all This has just been announced for Visual Studio 2008: The AWS SDK for .NET makes it even easier for Windows developers to build .NET applications that tap into the cost-effective, scalable, and reliable AWS cloud. Using the SDK, developers will be able to build solutions for AWS infrastructure services, including Amazon Simple Storage Service (Amazon S3), Amazon Elastic Compute Cloud (Amazon EC2), and Amazon SimpleDB. With the AWS SDK for .NET, developers get started in minutes with a single, downloadable package complete with Visual Studio project templates, the AWS .NET library, C# code samples, and documentation. http://aws.amazon.com/sdkfornet/ /gustav From jwcolby at colbyconsulting.com Fri Nov 13 06:51:36 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 07:51:36 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> Message-ID: <4AFD5658.7000703@colbyconsulting.com> Shamil, That is precisely what I have in mind, with variations. Then at the end, write all updates back to SQL Server. Thanks for the demo code, always helpful. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Here is a sample: > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class DataSetTests > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;"+ > "Initial Catalog=Northwind;User Id=sa;Password=N/A"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { > connection.Open(); > string sql = "select * from [Shippers]"; > string sql1 = "select * from [NewShippers] where(1=0)"; > SqlCommand command = new SqlCommand(sql, connection); > SqlDataAdapter adapter = new SqlDataAdapter(command); > > DataSet dataSet = new DataSet("Shippers"); > adapter.Fill(dataSet); > > command = new SqlCommand(sql1, connection); > adapter = new SqlDataAdapter(command); > DataSet newDataSet = new DataSet("NewShippers"); > adapter.Fill(newDataSet); > > Console.WriteLine("*** newDataSet ***"); > foreach (DataRow dataRow in newDataSet.Tables[0].Rows) > { > Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); > } > > Console.WriteLine("*** dataSet ***"); > foreach (DataRow dataRow in dataSet.Tables[0].Rows) > { > // This row already belongs to another table. > //newDataSet.Tables[0].Rows.Add(dataRow); > > object[] items = { dataRow[0], dataRow[1], dataRow[2] }; > newDataSet.Tables[0].Rows.Add(items); > > Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); > } > > Console.WriteLine("*** newDataSet ***"); > foreach (DataRow dataRow in newDataSet.Tables[0].Rows) > { > Console.WriteLine("{0} {1}", dataRow[0], dataRow[1]); > } > > newDataSet.Tables[0].TableName = "NewShippers"; > SqlCommandBuilder builder = new SqlCommandBuilder(adapter); > adapter.InsertCommand = builder.GetInsertCommand(true); > adapter.Update(newDataSet.Tables[0]); > } > } > > } > } > > > Thank you. > > --Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Thursday, November 12, 2009 8:23 PM > To: VBA > Subject: [dba-VB] What to do, what to do? > > I am writing an application in C# to merge records (previously discussed). > It looks pretty simple, > all things considered. Get data into a dataset object which will contain > about 4 tables. Then use > the records and fields collections of the recordset to iterate through the > fields building up a > record (or modifying the first record of a small set of records - the > dupes). Copy a record from > one table into another table inside of the dataset. Delete records in a > table in the dataset. Etc. > Etc all without going back out to the actual data tables on SQL Server until > the very end when I > will do an update back to the live data. > > Am I correct in assuming here that I can programmatically append records > from one table in the > dataset to another record in the dataset? Delete records in the tables in > the dataset? Update > records in tables in the dataset? All "in memory" without connecting back > to the live data in the > database? > > Is there anything I need to know before I launch into this? > From shamil at smsconsulting.spb.ru Fri Nov 13 08:19:39 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 17:19:39 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD5658.7000703@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> Message-ID: <004801ca646c$5758e690$060ab3b0$@spb.ru> Hi John -- You can also consider using SqlBulkLoad - below is just a simple sample, and you can develop very flexible variations of this approach. SqlBulkLoad should be as quick as bcp maybe even quicker. In fact the source for SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this IDataReader to have all the data to bulkload then your programs will become as streamlined as they probably have never been, FYI: IDataReader can be built in code with data sources being everything as e.g. .csv files etc. (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) using System; using System.Collections.Generic; using System.Text; using System.Data; using System.Data.SqlClient; namespace TestConsole.Samples { public class SqlBulkLoadTest { public static void Run() { string connectionString = "Data Source=HAMBURG\\SQL2005;" + "Initial Catalog=Northwind;User Id=sa;Password=shms"; using (SqlConnection connection = new SqlConnection(connectionString)) { connection.Open(); string sql = "select * from [Shippers]"; SqlCommand command = new SqlCommand(sql, connection); SqlDataAdapter adapter = new SqlDataAdapter(command); DataSet dataSet = new DataSet("Shippers"); adapter.Fill(dataSet); Console.WriteLine("*** dataSet ***"); foreach (DataRow dataRow in dataSet.Tables[0].Rows) { object[] items = { dataRow[0], dataRow[1], dataRow[2] }; Console.WriteLine("{0} {1} {2}", dataRow[0], dataRow[1], dataRow[2]); } (new SqlCommand("delete from [NewShippers]",connection)) .ExecuteNonQuery(); SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, SqlBulkCopyOptions.KeepIdentity); bulkCopy.DestinationTableName = "NewShippers"; bulkCopy.WriteToServer(dataSet.Tables[0]); } } } } -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 13, 2009 3:52 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? Shamil, That is precisely what I have in mind, with variations. Then at the end, write all updates back to SQL Server. Thanks for the demo code, always helpful. John W. Colby www.ColbyConsulting.com <<< snip >> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4603 (20091113) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 13 08:30:34 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 09:30:34 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <004801ca646c$5758e690$060ab3b0$@spb.ru> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> <004801ca646c$5758e690$060ab3b0$@spb.ru> Message-ID: <4AFD6D8A.3060306@colbyconsulting.com> In fact this is a one shot deal. I need to do this one time, to one table. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > You can also consider using SqlBulkLoad - below is just a simple sample, and > you can develop very flexible variations of this approach. SqlBulkLoad > should be as quick as bcp maybe even quicker. In fact the source for > SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this > IDataReader to have all the data to bulkload then your programs will become > as streamlined as they probably have never been, FYI: IDataReader can be > built in code with data sources being everything as e.g. .csv files etc. > (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) > > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class SqlBulkLoadTest > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;" + > "Initial Catalog=Northwind;User Id=sa;Password=shms"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { > connection.Open(); > string sql = "select * from [Shippers]"; > SqlCommand command = new SqlCommand(sql, connection); > SqlDataAdapter adapter = new SqlDataAdapter(command); > > DataSet dataSet = new DataSet("Shippers"); > adapter.Fill(dataSet); > > Console.WriteLine("*** dataSet ***"); > foreach (DataRow dataRow in dataSet.Tables[0].Rows) > { > object[] items = { dataRow[0], dataRow[1], dataRow[2] }; > > Console.WriteLine("{0} {1} {2}", > dataRow[0], dataRow[1], dataRow[2]); > } > > > (new SqlCommand("delete from [NewShippers]",connection)) > .ExecuteNonQuery(); > > SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, > SqlBulkCopyOptions.KeepIdentity); > bulkCopy.DestinationTableName = "NewShippers"; > bulkCopy.WriteToServer(dataSet.Tables[0]); > } > } > > } > } > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 13, 2009 3:52 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > Shamil, > > That is precisely what I have in mind, with variations. Then at the end, > write all updates back to > SQL Server. > > Thanks for the demo code, always helpful. > > John W. Colby > www.ColbyConsulting.com > > > <<< snip >> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4603 (20091113) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 13 08:30:42 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 09:30:42 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: References: <4AFC446F.6070207@colbyconsulting.com> Message-ID: <4AFD6D92.50807@colbyconsulting.com> Mark, > Just curious, how many records are you talking about - a few hundred or millions? Millions, however I am talking about updating sets of records, from 2 to < 30 rows. Imagine a table where Mark Breen has 22 records in the table, John Colby has 17 records in the table, Shamil has 3 records etc. Each of Mark's records have data in a "random" handful of 525 fields. The objective is to get all of Mark's data into one record. Once Mark's records are processed, move on to John's records. Then process William's records. There are 6,514,623 records to be processed. These 6.5 million records represent perhaps about 3 million actual people. I am creating tables right now to store the hash field and a count of records that hash field represents. As an example I have 9 actual people with more than 10 records to be merged, 4,725 people with 5 records, 270,0839 people with 3 records to merge, and 2,771,885 people with two records to merge. The nice part is that the table I am discussing is not constantly changing, in fact this data is entirely static. And there is no time frame for getting it done. I can start the process running and if it takes all night or all week I don't care. I suspect that with efficient programming I can do this at hundreds of records per second. I intend to do it out in tables dedicated to this process in a database dedicated to this process. All of the updates between fields in different records will be taking place entirely in memory. I will probably pull in all of the records for a thousand people, do the merges, then update those records back to SQL Server and pull another chunk of a thousand records. > Second question, other than for purely academic reasons, why are you bring the data into C# Mostly because I know programming, and TSQL is far enough from my expertise as to be unusable in the immediate future. To be honest I am not sure it is even possible to do this easily in TSQL. That said I am sure some TSQL GURU will pop up with the answer. But I am not a TSQL Guru (nor a Guru of any type for that matter). But the thought of programming this kind of algorithm in TSQL sends shivers down my spine. I need to do the following: R1F1 F1F2 R1F3... R1F525 R2F1 R2F2 R2F3... R2F525 R3F1 R3F2 R3F3... R3F525 The lines above represent Rows R1, R2 and R3, Fields F1,F2,F3...F525 Start with R1 and R2 Start at column 1 If R1F1 is null AND R2F1 is NOT NULL THEN Search through sequential records, same field until data found endif Move to column 2, continue to all columns Move to R1 and R3 In English: There is a set of records for a given person. If R1Col(N) has data, do not update. If R1Col(N) has no data, get data from R2Col(N) If R2Col(N) has no data, get data from R3Col(N) Start at N = (column) 1. Search until data found, update R1Col(N), then abort search and move to next column C# has a wonderful representation of data inside of the data adapter. Each table is a collection of rows AND a collection of columns. So I can iterate through the collection of columns, and once I have a collection of fields in a single column, iterate through that looking at the data in row 1, row 2 row 3 until data is found. This data representation just immediately does 1/2 the work for me by presenting me with my data already loaded into these collections. > Of course if it is to learn datasets then thats another story It is to get work done that needs doing. Along the way I expect to learn datasets and get that much more comfortable with C#. Doesn't this sound like FUN programming? I am a programmer at heart, I love doing this kind of stuff, and C# is where I am going, so why not do it there? John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > Just curious, how many records are you talking about - a few hundred or > millions? > > Second question, other than for purely academic reasons, why are you bring > the data into C#, to concatenate and insert into another table can so easily > be done in one sproc, even if you need a temporary holding place, you would > still use a temp table or even a local table. > > Of course if it is to learn datasets then thats another story, > > Mark From shamil at smsconsulting.spb.ru Fri Nov 13 08:36:06 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 17:36:06 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD6D8A.3060306@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> <004801ca646c$5758e690$060ab3b0$@spb.ru> <4AFD6D8A.3060306@colbyconsulting.com> Message-ID: <004a01ca646e$a3b24570$eb16d050$@spb.ru> Hi John, OK, if you're adding several millions rows then SqlBulkLoad can make your "shot deal" in one shot :) -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 13, 2009 5:31 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? In fact this is a one shot deal. I need to do this one time, to one table. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > You can also consider using SqlBulkLoad - below is just a simple sample, and > you can develop very flexible variations of this approach. SqlBulkLoad > should be as quick as bcp maybe even quicker. In fact the source for > SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this > IDataReader to have all the data to bulkload then your programs will become > as streamlined as they probably have never been, FYI: IDataReader can be > built in code with data sources being everything as e.g. .csv files etc. > (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) > > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class SqlBulkLoadTest > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;" + > "Initial Catalog=Northwind;User Id=sa;Password=N/A"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { > connection.Open(); > string sql = "select * from [Shippers]"; > SqlCommand command = new SqlCommand(sql, connection); > SqlDataAdapter adapter = new SqlDataAdapter(command); > > DataSet dataSet = new DataSet("Shippers"); > adapter.Fill(dataSet); > > Console.WriteLine("*** dataSet ***"); > foreach (DataRow dataRow in dataSet.Tables[0].Rows) > { > object[] items = { dataRow[0], dataRow[1], dataRow[2] }; > > Console.WriteLine("{0} {1} {2}", > dataRow[0], dataRow[1], dataRow[2]); > } > > > (new SqlCommand("delete from [NewShippers]",connection)) > .ExecuteNonQuery(); > > SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, > SqlBulkCopyOptions.KeepIdentity); > bulkCopy.DestinationTableName = "NewShippers"; > bulkCopy.WriteToServer(dataSet.Tables[0]); > } > } > > } > } > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 13, 2009 3:52 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > Shamil, > > That is precisely what I have in mind, with variations. Then at the end, > write all updates back to > SQL Server. > > Thanks for the demo code, always helpful. > > John W. Colby > www.ColbyConsulting.com > > > <<< snip >> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4603 (20091113) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4603 (20091113) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 13 08:40:25 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 09:40:25 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <004a01ca646e$a3b24570$eb16d050$@spb.ru> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> <004801ca646c$5758e690$060ab3b0$@spb.ru> <4AFD6D8A.3060306@colbyconsulting.com> <004a01ca646e$a3b24570$eb16d050$@spb.ru> Message-ID: <4AFD6FD9.2040902@colbyconsulting.com> See my reply to Mark Breen coming up. This is not a bulk load thing. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > OK, if you're adding several millions rows then SqlBulkLoad can make your > "shot deal" in one shot :) > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 13, 2009 5:31 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > In fact this is a one shot deal. I need to do this one time, to one table. > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> You can also consider using SqlBulkLoad - below is just a simple sample, > and >> you can develop very flexible variations of this approach. SqlBulkLoad >> should be as quick as bcp maybe even quicker. In fact the source for >> SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this >> IDataReader to have all the data to bulkload then your programs will > become >> as streamlined as they probably have never been, FYI: IDataReader can be >> built in code with data sources being everything as e.g. .csv files etc. >> (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) >> >> >> using System; >> using System.Collections.Generic; >> using System.Text; >> >> using System.Data; >> using System.Data.SqlClient; >> >> namespace TestConsole.Samples >> { >> public class SqlBulkLoadTest >> { >> public static void Run() >> { >> string connectionString = "Data Source=HAMBURG\\SQL2005;" + >> "Initial Catalog=Northwind;User Id=sa;Password=N/A"; >> using (SqlConnection connection = >> new SqlConnection(connectionString)) >> { >> connection.Open(); >> string sql = "select * from [Shippers]"; >> SqlCommand command = new SqlCommand(sql, connection); >> SqlDataAdapter adapter = new SqlDataAdapter(command); >> >> DataSet dataSet = new DataSet("Shippers"); >> adapter.Fill(dataSet); >> >> Console.WriteLine("*** dataSet ***"); >> foreach (DataRow dataRow in dataSet.Tables[0].Rows) >> { >> object[] items = { dataRow[0], dataRow[1], dataRow[2] > }; >> Console.WriteLine("{0} {1} {2}", >> dataRow[0], dataRow[1], dataRow[2]); >> } >> >> >> (new SqlCommand("delete from [NewShippers]",connection)) >> .ExecuteNonQuery(); >> >> SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, >> SqlBulkCopyOptions.KeepIdentity); >> bulkCopy.DestinationTableName = "NewShippers"; >> bulkCopy.WriteToServer(dataSet.Tables[0]); >> } >> } >> >> } >> } >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: Friday, November 13, 2009 3:52 PM >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] What to do, what to do? >> >> Shamil, >> >> That is precisely what I have in mind, with variations. Then at the end, >> write all updates back to >> SQL Server. >> >> Thanks for the demo code, always helpful. >> >> John W. Colby >> www.ColbyConsulting.com >> >> >> <<< snip >> >> >> >> __________ Information from ESET NOD32 Antivirus, version of virus > signature >> database 4603 (20091113) __________ >> >> The message was checked by ESET NOD32 Antivirus. >> >> http://www.esetnod32.ru >> >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4603 (20091113) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Nov 13 08:54:16 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 17:54:16 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD6FD9.2040902@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD5658.7000703@colbyconsulting.com> <004801ca646c$5758e690$060ab3b0$@spb.ru> <4AFD6D8A.3060306@colbyconsulting.com> <004a01ca646e$a3b24570$eb16d050$@spb.ru> <4AFD6FD9.2040902@colbyconsulting.com> Message-ID: <004b01ca6471$2d3f5dd0$87be1970$@spb.ru> OK, I see this is "update thing" not insert/bulkload. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 13, 2009 5:40 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? See my reply to Mark Breen coming up. This is not a bulk load thing. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > OK, if you're adding several millions rows then SqlBulkLoad can make your > "shot deal" in one shot :) > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 13, 2009 5:31 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > In fact this is a one shot deal. I need to do this one time, to one table. > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> You can also consider using SqlBulkLoad - below is just a simple sample, > and >> you can develop very flexible variations of this approach. SqlBulkLoad >> should be as quick as bcp maybe even quicker. In fact the source for >> SqlBulkLoad could be a SqlDataReader/IDataReader - if you manage this >> IDataReader to have all the data to bulkload then your programs will > become >> as streamlined as they probably have never been, FYI: IDataReader can be >> built in code with data sources being everything as e.g. .csv files etc. >> (and of course .mdb tables or .xls (ISAM) "tables" etc.etc.) >> >> >> using System; >> using System.Collections.Generic; >> using System.Text; >> >> using System.Data; >> using System.Data.SqlClient; >> >> namespace TestConsole.Samples >> { >> public class SqlBulkLoadTest >> { >> public static void Run() >> { >> string connectionString = "Data Source=HAMBURG\\SQL2005;" + >> "Initial Catalog=Northwind;User Id=sa;Password=N/A"; >> using (SqlConnection connection = >> new SqlConnection(connectionString)) >> { >> connection.Open(); >> string sql = "select * from [Shippers]"; >> SqlCommand command = new SqlCommand(sql, connection); >> SqlDataAdapter adapter = new SqlDataAdapter(command); >> >> DataSet dataSet = new DataSet("Shippers"); >> adapter.Fill(dataSet); >> >> Console.WriteLine("*** dataSet ***"); >> foreach (DataRow dataRow in dataSet.Tables[0].Rows) >> { >> object[] items = { dataRow[0], dataRow[1], dataRow[2] > }; >> Console.WriteLine("{0} {1} {2}", >> dataRow[0], dataRow[1], dataRow[2]); >> } >> >> >> (new SqlCommand("delete from [NewShippers]",connection)) >> .ExecuteNonQuery(); >> >> SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString, >> SqlBulkCopyOptions.KeepIdentity); >> bulkCopy.DestinationTableName = "NewShippers"; >> bulkCopy.WriteToServer(dataSet.Tables[0]); >> } >> } >> >> } >> } >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: Friday, November 13, 2009 3:52 PM >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] What to do, what to do? >> >> Shamil, >> >> That is precisely what I have in mind, with variations. Then at the end, >> write all updates back to >> SQL Server. >> >> Thanks for the demo code, always helpful. >> >> John W. Colby >> www.ColbyConsulting.com >> >> >> <<< snip >> >> From jwcolby at colbyconsulting.com Fri Nov 13 10:39:14 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 13 Nov 2009 11:39:14 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> Message-ID: <4AFD8BB2.5030203@colbyconsulting.com> Shamil, Given what you now know about what I want to do... Why does each table get created in it's own DataSet. I thought a data set could have multiple tables. I envisioned the following structure out in SQL Server: tblHSID - contains 6.4 million records. Source of records for tblMergeSrc. tblMergeSrc - Contains X thousand records for Y hundred people. Working table in memory. tblMergedRecs - the single record merged into will be created here (or copied here). Working table in memory. tblPurgedRecs - The multiple records merged FROM will be archived here. Working table in memory. tblHashPK - Hash for a person / address, PKs for person / address. There is one hash for each person / hash, there are multiple PKs for each person / address. One record for each record in tblHSID. tblHashCnt - Group by Hash, Cnt PK. Server two purposes, provides a "DISTINCT" hash for each person being processed as well as a count of how many records will be merged down. So... tblHSID is the source of the records to be processed. Contains ONLY people who actually have "duplicates", most people don't. The Top(N) records from tblHashCnt will select the Top(N) PEOPLE by number of records to be merged. Merge the ones with the most records first. That will be joined to tblHashPK to pull all of the PKs for the Top(N) people about to be processed. The PKs from that will be joined to tblHSID to pull a set of records to be merged. Copied into tblMergeSrc. This is the first time HSID records (records to be merged) will be moved into a table in a dataset. tblMergedRecs is empty but exists in the dataset. tblPurgedRecs is empty but exists in the dataset. The merge will most likely happen in tblMergeSrc although a new record could be created in tblMergedRecs to work in. Not decided yet. In the end, the merge happens in the dataset in memory. The actual ONE merged record for each person ends up in tblMergedRecs. The actual (multiple) records merged into other records are moved into tblPurgedRecs. This is an archive. The records in tblPurgedRecs are then deleted from tblMergedSrc, and eventually deleted out of tblHSID back in the database. The records for the next person are processed. Merged records moved to tblMergedRecs. "Purged" records moved to tblPurgedRecs. Deleted from tblMergeSrc. Wash, rinse repeat. Now... I don't care how many data sets I have but I just question why I would create a new dataset for each table above. It seems that having them all in a single dataset would minimize confusion and stuff. Logically it seems that working with multiple tables inside of a single DataSet object MIGHT be faster than moving data back and forth between tables in different DataSet objects, though maybe not. And of course... my book talks all about connections, then all about commands, and then all about parameters, then... How to work with a data reader. WTF over? What happened to my dataset and table objects? Not a WORD about those two objects. Sigh. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Here is a sample: > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class DataSetTests > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;"+ > "Initial Catalog=Northwind;User Id=sa;Password=N/A"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { From shamil at smsconsulting.spb.ru Fri Nov 13 11:22:31 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 13 Nov 2009 20:22:31 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD8BB2.5030203@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com> <002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD8BB2.5030203@colbyconsulting.com> Message-ID: <005b01ca6485$e42c31d0$ac849570$@spb.ru> John, Yes, of course you can have multiple DataTable(s) in a DataSet - I have just posted a very simple sample - one of a myriad possible combinations, the simplest one probably. And of course you can have your DataSet(s) structure created statically in VS designer. Yes, I think your plan is OK. You probably do not need a DataReader but if you'll find you need one - in P.S. is a simple sample. In fact Gustav knows DataSets better than I do - please address your questions to him also and to everybody else here - I will not have time to reply any more postings today... Thank you. -- Shamil P.S. DataReader sample: using System; using System.Collections.Generic; using System.Text; using System.Data; using System.Data.SqlClient; namespace TestConsole.Samples { public class SqlDataReaderTest { public static void Run() { string connectionString = "Data Source=HAMBURG\\SQL2005;" + "Initial Catalog=Northwind;User Id=sa;Password=N/A"; using (SqlConnection connection = new SqlConnection(connectionString)) { connection.Open(); // Create empty [NewShippers] dataset in memory string sql1 = "select * from [NewShippers] where(1=0)"; SqlCommand command = new SqlCommand(sql1, connection); SqlDataAdapter adapter = new SqlDataAdapter(command); DataSet newDataSet = new DataSet("NewShippers"); adapter.Fill(newDataSet); // Load source [Shippers] in memory string sql = "select * from [Shippers]"; command = new SqlCommand(sql, connection); SqlDataReader dataReader = command.ExecuteReader(); // Add loaded [Shippers] to [NewShippers] im memory Console.WriteLine("*** dataReader ***"); while (dataReader.Read()) { object[] items = { dataReader[0], dataReader[1], dataReader[2] }; newDataSet.Tables[0].Rows.Add(items); Console.WriteLine("{0} {1} {2}", items[0], items[1], items[2]); } dataReader.Close(); // ! always close DataReader // Insert [NewShippers] into db (new SqlCommand("delete from [NewShippers]",connection)) .ExecuteNonQuery(); SqlCommandBuilder builder = new SqlCommandBuilder(adapter); adapter.InsertCommand = builder.GetInsertCommand(true); adapter.Update(newDataSet.Tables[0]); // Check that [NewShippers] were inserted OK Console.WriteLine("*** newDataSet ***"); newDataSet.Clear(); adapter = new SqlDataAdapter( new SqlCommand("select * from [NewShippers]",connection)); adapter.Fill(newDataSet); foreach (DataRow dataRow in newDataSet.Tables[0].Rows) { Console.WriteLine("{0} {1} {2}", dataRow[0], dataRow[1], dataRow[2]); } } } } } <<< snip to get through dba-VB robot >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4604 (20091113) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From cfoust at infostatsystems.com Fri Nov 13 12:18:07 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Fri, 13 Nov 2009 10:18:07 -0800 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFD8BB2.5030203@colbyconsulting.com> References: <4AFC446F.6070207@colbyconsulting.com><002301ca6427$2ee6f2f0$8cb4d8d0$@spb.ru> <4AFD8BB2.5030203@colbyconsulting.com> Message-ID: You can have all sorts of tables in a dataset, John. But if you want to act on multiple non-cascading tables, then you probably need to validate multiple tables. If you have relationships set so you can cascade updates, it isn't too bad. Otherwise, you have to write code to individually handle the gotchas in each table before you update any of them. Without that, you'd wind up with a partial update. Some of my biggest headaches are the one rich typed dataset we use to allow the users to enter data into 3 or 4 different tables from a single form. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 13, 2009 8:39 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? Shamil, Given what you now know about what I want to do... Why does each table get created in it's own DataSet. I thought a data set could have multiple tables. I envisioned the following structure out in SQL Server: tblHSID - contains 6.4 million records. Source of records for tblMergeSrc. tblMergeSrc - Contains X thousand records for Y hundred people. Working table in memory. tblMergedRecs - the single record merged into will be created here (or copied here). Working table in memory. tblPurgedRecs - The multiple records merged FROM will be archived here. Working table in memory. tblHashPK - Hash for a person / address, PKs for person / address. There is one hash for each person / hash, there are multiple PKs for each person / address. One record for each record in tblHSID. tblHashCnt - Group by Hash, Cnt PK. Server two purposes, provides a "DISTINCT" hash for each person being processed as well as a count of how many records will be merged down. So... tblHSID is the source of the records to be processed. Contains ONLY people who actually have "duplicates", most people don't. The Top(N) records from tblHashCnt will select the Top(N) PEOPLE by number of records to be merged. Merge the ones with the most records first. That will be joined to tblHashPK to pull all of the PKs for the Top(N) people about to be processed. The PKs from that will be joined to tblHSID to pull a set of records to be merged. Copied into tblMergeSrc. This is the first time HSID records (records to be merged) will be moved into a table in a dataset. tblMergedRecs is empty but exists in the dataset. tblPurgedRecs is empty but exists in the dataset. The merge will most likely happen in tblMergeSrc although a new record could be created in tblMergedRecs to work in. Not decided yet. In the end, the merge happens in the dataset in memory. The actual ONE merged record for each person ends up in tblMergedRecs. The actual (multiple) records merged into other records are moved into tblPurgedRecs. This is an archive. The records in tblPurgedRecs are then deleted from tblMergedSrc, and eventually deleted out of tblHSID back in the database. The records for the next person are processed. Merged records moved to tblMergedRecs. "Purged" records moved to tblPurgedRecs. Deleted from tblMergeSrc. Wash, rinse repeat. Now... I don't care how many data sets I have but I just question why I would create a new dataset for each table above. It seems that having them all in a single dataset would minimize confusion and stuff. Logically it seems that working with multiple tables inside of a single DataSet object MIGHT be faster than moving data back and forth between tables in different DataSet objects, though maybe not. And of course... my book talks all about connections, then all about commands, and then all about parameters, then... How to work with a data reader. WTF over? What happened to my dataset and table objects? Not a WORD about those two objects. Sigh. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Here is a sample: > > using System; > using System.Collections.Generic; > using System.Text; > > using System.Data; > using System.Data.SqlClient; > > namespace TestConsole.Samples > { > public class DataSetTests > { > public static void Run() > { > string connectionString = "Data Source=HAMBURG\\SQL2005;"+ > "Initial Catalog=Northwind;User Id=sa;Password=N/A"; > using (SqlConnection connection = > new SqlConnection(connectionString)) > { _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From raibeart at gmail.com Fri Nov 13 22:46:18 2009 From: raibeart at gmail.com (Robert Stewart) Date: Fri, 13 Nov 2009 22:46:18 -0600 Subject: [dba-VB] What to do, what to do? In-Reply-To: References: Message-ID: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert At 12:00 PM 11/13/2009, you wrote: >Date: Fri, 13 Nov 2009 11:39:14 -0500 >From: jwcolby >Subject: Re: [dba-VB] What to do, what to do? >To: "Discussion concerning Visual Basic and related programming > issues." >Message-ID: <4AFD8BB2.5030203 at colbyconsulting.com> >Content-Type: text/plain; charset=ISO-8859-1; format=flowed > >Shamil, > >Given what you now know about what I want to do... > >Why does each table get created in it's own DataSet. I thought a >data set could have multiple >tables. I envisioned the following structure out in SQL Server: > >tblHSID - contains 6.4 million records. Source of records for tblMergeSrc. >tblMergeSrc - Contains X thousand records for Y hundred >people. Working table in memory. >tblMergedRecs - the single record merged into will be created here >(or copied here). Working table >in memory. >tblPurgedRecs - The multiple records merged FROM will be archived >here. Working table in memory. > >tblHashPK - Hash for a person / address, PKs for person / >address. There is one hash for each >person / hash, there are multiple PKs for each person / >address. One record for each record in >tblHSID. > >tblHashCnt - Group by Hash, Cnt PK. Server two purposes, provides a >"DISTINCT" hash for each person >being processed as well as a count of how many records will be merged down. > >So... tblHSID is the source of the records to be >processed. Contains ONLY people who actually have >"duplicates", most people don't. > >The Top(N) records from tblHashCnt will select the Top(N) PEOPLE by >number of records to be merged. > Merge the ones with the most records first. > >That will be joined to tblHashPK to pull all of the PKs for the >Top(N) people about to be processed. > >The PKs from that will be joined to tblHSID to pull a set of records >to be merged. Copied into >tblMergeSrc. This is the first time HSID records (records to be >merged) will be moved into a table >in a dataset. > >tblMergedRecs is empty but exists in the dataset. >tblPurgedRecs is empty but exists in the dataset. > >The merge will most likely happen in tblMergeSrc although a new >record could be created in >tblMergedRecs to work in. Not decided yet. > >In the end, the merge happens in the dataset in memory. The actual >ONE merged record for each >person ends up in tblMergedRecs. > >The actual (multiple) records merged into other records are moved >into tblPurgedRecs. This is an >archive. > >The records in tblPurgedRecs are then deleted from tblMergedSrc, and >eventually deleted out of >tblHSID back in the database. > >The records for the next person are processed. Merged records moved >to tblMergedRecs. "Purged" >records moved to tblPurgedRecs. > >Deleted from tblMergeSrc. > >Wash, rinse repeat. > >Now... I don't care how many data sets I have but I just question >why I would create a new dataset >for each table above. It seems that having them all in a single >dataset would minimize confusion >and stuff. Logically it seems that working with multiple tables >inside of a single DataSet object >MIGHT be faster than moving data back and forth between tables in >different DataSet objects, though >maybe not. > >And of course... my book talks all about connections, then all about >commands, and then all about >parameters, then... > >How to work with a data reader. WTF over? What happened to my >dataset and table objects? Not a >WORD about those two objects. > >Sigh. > >John W. Colby From shamil at smsconsulting.spb.ru Sat Nov 14 04:40:43 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 14 Nov 2009 13:40:43 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> Message-ID: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Hi Robert, Yes, DataSets are mainly targeted for working with small number of records in memory - so the following is more a "theoretical" calculation: - JC has 64bit PC which allow to load in memory "practically unlimited" volume of data - 64bit logical (process) address space could be as large as 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 (http://en.wikipedia.org/wiki/Exbibyte). - the speed of data transfer for 64bit Intel's processor - "the 200 MHz McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); ----- So "theoretically" with modern speedy harddisks JC can load 50 millions of records into memory within minutes - within half a minute in the near future when large enough flash-memory disks will become available and relatively inexpensive? To load such large data volume it would be better to use SqlDataReader as it keeps loaded data in a very compact form. And 50 million records long SqlDataReader should be probably better split into several chunks... Again - this is just a "theoretical" consideration - in practice JC can process his 50 million records long data table in chunks as he has a hash field (tblHashPK), which can be used to load related records into memory: I mean he can split tblHashPK table keeping hash values into several groups, e.g.: 1 - 100,000 100,001 - 200,000 ... and process each group joining its records to source 50 million records... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart Sent: Saturday, November 14, 2009 7:46 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] What to do, what to do? John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From max.wanadoo at gmail.com Sat Nov 14 05:30:41 2009 From: max.wanadoo at gmail.com (Max Wanadoo) Date: Sat, 14 Nov 2009 11:30:41 -0000 Subject: [dba-VB] What to do, what to do? In-Reply-To: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Message-ID: <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> Very interesting Shamil What happens with all the other stuff that memory is taking up though, OS, Programs, etc and the physical memory he has placed in the machine? Your limits below presumably are saying that his physical memory is loaded to...what? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 10:41 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Robert, Yes, DataSets are mainly targeted for working with small number of records in memory - so the following is more a "theoretical" calculation: - JC has 64bit PC which allow to load in memory "practically unlimited" volume of data - 64bit logical (process) address space could be as large as 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 (http://en.wikipedia.org/wiki/Exbibyte). - the speed of data transfer for 64bit Intel's processor - "the 200 MHz McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); ----- So "theoretically" with modern speedy harddisks JC can load 50 millions of records into memory within minutes - within half a minute in the near future when large enough flash-memory disks will become available and relatively inexpensive? To load such large data volume it would be better to use SqlDataReader as it keeps loaded data in a very compact form. And 50 million records long SqlDataReader should be probably better split into several chunks... Again - this is just a "theoretical" consideration - in practice JC can process his 50 million records long data table in chunks as he has a hash field (tblHashPK), which can be used to load related records into memory: I mean he can split tblHashPK table keeping hash values into several groups, e.g.: 1 - 100,000 100,001 - 200,000 ... and process each group joining its records to source 50 million records... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart Sent: Saturday, November 14, 2009 7:46 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] What to do, what to do? John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Nov 14 06:30:35 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 14 Nov 2009 07:30:35 -0500 Subject: [dba-VB] SPAM-LOW: Re: What to do, what to do? In-Reply-To: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> Message-ID: <4AFEA2EB.5050507@colbyconsulting.com> Read the discussion carefully Robert. I am discussing doing chunks of thousands. John W. Colby www.ColbyConsulting.com Robert Stewart wrote: > John, > > I don't think your machine, or any for that matter, have the memory > to load 50 million records, extract 40 million, transform them and > then load them back to the 50 million and update the database from > the recordset. Datasets are for working with a small number of > records in memory, not millions. > > Robert > > > At 12:00 PM 11/13/2009, you wrote: >> Date: Fri, 13 Nov 2009 11:39:14 -0500 >> From: jwcolby >> Subject: Re: [dba-VB] What to do, what to do? >> To: "Discussion concerning Visual Basic and related programming >> issues." >> Message-ID: <4AFD8BB2.5030203 at colbyconsulting.com> >> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >> >> Shamil, >> >> Given what you now know about what I want to do... >> >> Why does each table get created in it's own DataSet. I thought a >> data set could have multiple >> tables. I envisioned the following structure out in SQL Server: >> >> tblHSID - contains 6.4 million records. Source of records for tblMergeSrc. >> tblMergeSrc - Contains X thousand records for Y hundred >> people. Working table in memory. >> tblMergedRecs - the single record merged into will be created here >> (or copied here). Working table >> in memory. >> tblPurgedRecs - The multiple records merged FROM will be archived >> here. Working table in memory. >> >> tblHashPK - Hash for a person / address, PKs for person / >> address. There is one hash for each >> person / hash, there are multiple PKs for each person / >> address. One record for each record in >> tblHSID. >> >> tblHashCnt - Group by Hash, Cnt PK. Server two purposes, provides a >> "DISTINCT" hash for each person >> being processed as well as a count of how many records will be merged down. >> >> So... tblHSID is the source of the records to be >> processed. Contains ONLY people who actually have >> "duplicates", most people don't. >> >> The Top(N) records from tblHashCnt will select the Top(N) PEOPLE by >> number of records to be merged. >> Merge the ones with the most records first. >> >> That will be joined to tblHashPK to pull all of the PKs for the >> Top(N) people about to be processed. >> >> The PKs from that will be joined to tblHSID to pull a set of records >> to be merged. Copied into >> tblMergeSrc. This is the first time HSID records (records to be >> merged) will be moved into a table >> in a dataset. >> >> tblMergedRecs is empty but exists in the dataset. >> tblPurgedRecs is empty but exists in the dataset. >> >> The merge will most likely happen in tblMergeSrc although a new >> record could be created in >> tblMergedRecs to work in. Not decided yet. >> >> In the end, the merge happens in the dataset in memory. The actual >> ONE merged record for each >> person ends up in tblMergedRecs. >> >> The actual (multiple) records merged into other records are moved >> into tblPurgedRecs. This is an >> archive. >> >> The records in tblPurgedRecs are then deleted from tblMergedSrc, and >> eventually deleted out of >> tblHSID back in the database. >> >> The records for the next person are processed. Merged records moved >> to tblMergedRecs. "Purged" >> records moved to tblPurgedRecs. >> >> Deleted from tblMergeSrc. >> >> Wash, rinse repeat. >> >> Now... I don't care how many data sets I have but I just question >> why I would create a new dataset >> for each table above. It seems that having them all in a single >> dataset would minimize confusion >> and stuff. Logically it seems that working with multiple tables >> inside of a single DataSet object >> MIGHT be faster than moving data back and forth between tables in >> different DataSet objects, though >> maybe not. >> >> And of course... my book talks all about connections, then all about >> commands, and then all about >> parameters, then... >> >> How to work with a data reader. WTF over? What happened to my >> dataset and table objects? Not a >> WORD about those two objects. >> >> Sigh. >> >> John W. Colby > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Nov 14 07:46:21 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 14 Nov 2009 16:46:21 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> Message-ID: <008201ca6530$daf14140$90d3c3c0$@spb.ru> Hi Max, But OS, programs etc. - they all use their own logical address space, which is mapped to (shared (using OS directed swapping)) physical address space: with JC 64bit PC OS image/runtime will be kept in its own physical memory segments - and no swapping will occur, MS SQL image/runtime/data will be (mainly) kept in its own physical memory segments - and almost no swapping will occur, JC's program will have occupied its own physical memory segments - and no swapping will occur... 50 million records assuming every of them is equal to approx. 1KB will consume ~50GB - 64bit PC can have(?) that much RAM allocated for one process, and it all can be addresses directly within this process logical address space. And I have noted that it's currently a "theoretical consideration" but it may become practically achievable in not that far future(?) Correct/wrong? (I must say I didn't get checked how large can be logical address space for a Win7 process running on 64bit PC). Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Max Wanadoo Sent: Saturday, November 14, 2009 2:31 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Very interesting Shamil What happens with all the other stuff that memory is taking up though, OS, Programs, etc and the physical memory he has placed in the machine? Your limits below presumably are saying that his physical memory is loaded to...what? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 10:41 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Robert, Yes, DataSets are mainly targeted for working with small number of records in memory - so the following is more a "theoretical" calculation: - JC has 64bit PC which allow to load in memory "practically unlimited" volume of data - 64bit logical (process) address space could be as large as 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 (http://en.wikipedia.org/wiki/Exbibyte). - the speed of data transfer for 64bit Intel's processor - "the 200 MHz McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); ----- So "theoretically" with modern speedy harddisks JC can load 50 millions of records into memory within minutes - within half a minute in the near future when large enough flash-memory disks will become available and relatively inexpensive? To load such large data volume it would be better to use SqlDataReader as it keeps loaded data in a very compact form. And 50 million records long SqlDataReader should be probably better split into several chunks... Again - this is just a "theoretical" consideration - in practice JC can process his 50 million records long data table in chunks as he has a hash field (tblHashPK), which can be used to load related records into memory: I mean he can split tblHashPK table keeping hash values into several groups, e.g.: 1 - 100,000 100,001 - 200,000 ... and process each group joining its records to source 50 million records... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart Sent: Saturday, November 14, 2009 7:46 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] What to do, what to do? John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 14 08:23:42 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 14 Nov 2009 09:23:42 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Message-ID: <4AFEBD6E.1020709@colbyconsulting.com> All of which is beside the point. Divide and conquer. A principal discovered by military strategists thousands of years ago, and applied to problems of all sorts today. I can EASILY work with sets of thousands, and even hundreds of thousands of records. In real life, in my computer. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Robert, > > Yes, DataSets are mainly targeted for working with small number of records > in memory - so the following is more a "theoretical" calculation: > > - JC has 64bit PC which allow to load in memory "practically unlimited" > volume of data - 64bit logical (process) address space could be as large as > 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 > (http://en.wikipedia.org/wiki/Exbibyte). > > - the speed of data transfer for 64bit Intel's processor - "the 200 MHz > McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers > 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); > > ----- > So "theoretically" with modern speedy harddisks JC can load 50 millions of > records into memory within minutes - within half a minute in the near future > when large enough flash-memory disks will become available and relatively > inexpensive? > > To load such large data volume it would be better to use SqlDataReader as it > keeps loaded data in a very compact form. And 50 million records long > SqlDataReader should be probably better split into several chunks... > > Again - this is just a "theoretical" consideration - in practice JC can > process his 50 million records long data table in chunks as he has a hash > field (tblHashPK), which can be used to load related records into memory: I > mean he can split tblHashPK table keeping hash values into several groups, > e.g.: > > 1 - 100,000 > 100,001 - 200,000 > ... > > and process each group joining its records to source 50 million records... > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart > Sent: Saturday, November 14, 2009 7:46 AM > To: dba-vb at databaseadvisors.com > Subject: Re: [dba-VB] What to do, what to do? > > John, > > I don't think your machine, or any for that matter, have the memory > to load 50 million records, extract 40 million, transform them and > then load them back to the 50 million and update the database from > the recordset. Datasets are for working with a small number of > records in memory, not millions. > > Robert > > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4606 (20091114) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Nov 14 08:44:11 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 14 Nov 2009 17:44:11 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4AFEBD6E.1020709@colbyconsulting.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4AFEBD6E.1020709@colbyconsulting.com> Message-ID: <008301ca6538$eeef8910$ccce9b30$@spb.ru> Hi John, Yes, I have just "theoretically" noted that in the near future you'll be able to EVEN MORE EASILY work with sets of millions (and billions?) records by just loading them into memory, "crunching", and saving back updated... ... it will be just 1,2,3 instead of your today's (1,2,3) (1,2,3)... (1,2,3) (multi-threaded)... ... the time just to develop (1,2,3) (1,2,3)... (1,2,3)... will be more expensive than time to develop and run 1,2,3 on the near future hardware (hardware costs included)... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 14, 2009 5:24 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] What to do, what to do? All of which is beside the point. Divide and conquer. A principal discovered by military strategists thousands of years ago, and applied to problems of all sorts today. I can EASILY work with sets of thousands, and even hundreds of thousands of records. In real life, in my computer. John W. Colby www.ColbyConsulting.com <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4607 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From max.wanadoo at gmail.com Sat Nov 14 09:38:39 2009 From: max.wanadoo at gmail.com (Max Wanadoo) Date: Sat, 14 Nov 2009 15:38:39 -0000 Subject: [dba-VB] What to do, what to do? In-Reply-To: <008201ca6530$daf14140$90d3c3c0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> <008201ca6530$daf14140$90d3c3c0$@spb.ru> Message-ID: <4afecf2a.0702d00a.3bd1.4040@mx.google.com> Hi Shamil, I understand the logical size bit but for that to be mapped to physical memory requires physical memory to be present. Current there are limits to how much can be installed on the motherboard. 64Bit memory can address the sizes you mentioned but the physical memory doesn't exist, so (presumably) it maps it to the HD, in which case it is not in memory. Have I misunderstood something (it has been known to happen)? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 13:46 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Max, But OS, programs etc. - they all use their own logical address space, which is mapped to (shared (using OS directed swapping)) physical address space: with JC 64bit PC OS image/runtime will be kept in its own physical memory segments - and no swapping will occur, MS SQL image/runtime/data will be (mainly) kept in its own physical memory segments - and almost no swapping will occur, JC's program will have occupied its own physical memory segments - and no swapping will occur... 50 million records assuming every of them is equal to approx. 1KB will consume ~50GB - 64bit PC can have(?) that much RAM allocated for one process, and it all can be addresses directly within this process logical address space. And I have noted that it's currently a "theoretical consideration" but it may become practically achievable in not that far future(?) Correct/wrong? (I must say I didn't get checked how large can be logical address space for a Win7 process running on 64bit PC). Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Max Wanadoo Sent: Saturday, November 14, 2009 2:31 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Very interesting Shamil What happens with all the other stuff that memory is taking up though, OS, Programs, etc and the physical memory he has placed in the machine? Your limits below presumably are saying that his physical memory is loaded to...what? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 10:41 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Robert, Yes, DataSets are mainly targeted for working with small number of records in memory - so the following is more a "theoretical" calculation: - JC has 64bit PC which allow to load in memory "practically unlimited" volume of data - 64bit logical (process) address space could be as large as 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 (http://en.wikipedia.org/wiki/Exbibyte). - the speed of data transfer for 64bit Intel's processor - "the 200 MHz McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); ----- So "theoretically" with modern speedy harddisks JC can load 50 millions of records into memory within minutes - within half a minute in the near future when large enough flash-memory disks will become available and relatively inexpensive? To load such large data volume it would be better to use SqlDataReader as it keeps loaded data in a very compact form. And 50 million records long SqlDataReader should be probably better split into several chunks... Again - this is just a "theoretical" consideration - in practice JC can process his 50 million records long data table in chunks as he has a hash field (tblHashPK), which can be used to load related records into memory: I mean he can split tblHashPK table keeping hash values into several groups, e.g.: 1 - 100,000 100,001 - 200,000 ... and process each group joining its records to source 50 million records... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Robert Stewart Sent: Saturday, November 14, 2009 7:46 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] What to do, what to do? John, I don't think your machine, or any for that matter, have the memory to load 50 million records, extract 40 million, transform them and then load them back to the 50 million and update the database from the recordset. Datasets are for working with a small number of records in memory, not millions. Robert <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4606 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Sat Nov 14 10:08:26 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 14 Nov 2009 19:08:26 +0300 Subject: [dba-VB] What to do, what to do? In-Reply-To: <4afecf2a.0702d00a.3bd1.4040@mx.google.com> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> <008201ca6530$daf14140$90d3c3c0$@spb.ru> <4afecf2a.0702d00a.3bd1.4040@mx.google.com> Message-ID: <008401ca6544$b4290890$1c7b19b0$@spb.ru> Hi Max, Here are theoretical limits I have found info about: Address space 64-bit Windows 32-bit Windows Virtual memory 16 Tb 4 Gb Swap file 512 Tb 16 Tb System cache 1 Tb 1 Gb In practice, mass market motherboards can already handle 64GB(?) - Not enough for JC tasks but this max RAM size limitations should be removed in the near future AFAICG, and Win7 is ready(?) to handle effectively that much RAM.... http://www.nextag.com/64-bit-pci-motherboard/search-html http://www.viva64.com/content/articles/64-bit-development/?f=Application_por t_to_64-bit_platforms_or_never_cackle_till_your_egg_is_laid.html&lang=en&con tent=64-bit-development http://blogs.zdnet.com/hardware/?p=4254 http://www.tomshardware.com/reviews/vista-workshop,1775-3.html Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Max Wanadoo Sent: Saturday, November 14, 2009 6:39 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Shamil, I understand the logical size bit but for that to be mapped to physical memory requires physical memory to be present. Current there are limits to how much can be installed on the motherboard. 64Bit memory can address the sizes you mentioned but the physical memory doesn't exist, so (presumably) it maps it to the HD, in which case it is not in memory. Have I misunderstood something (it has been known to happen)? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 13:46 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Max, But OS, programs etc. - they all use their own logical address space, which is mapped to (shared (using OS directed swapping)) physical address space: with JC 64bit PC OS image/runtime will be kept in its own physical memory segments - and no swapping will occur, MS SQL image/runtime/data will be (mainly) kept in its own physical memory segments - and almost no swapping will occur, JC's program will have occupied its own physical memory segments - and no swapping will occur... 50 million records assuming every of them is equal to approx. 1KB will consume ~50GB - 64bit PC can have(?) that much RAM allocated for one process, and it all can be addresses directly within this process logical address space. And I have noted that it's currently a "theoretical consideration" but it may become practically achievable in not that far future(?) Correct/wrong? (I must say I didn't get checked how large can be logical address space for a Win7 process running on 64bit PC). Thank you. -- Shamil <<< snipped >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4607 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From max.wanadoo at gmail.com Sat Nov 14 10:27:00 2009 From: max.wanadoo at gmail.com (Max Wanadoo) Date: Sat, 14 Nov 2009 16:27:00 -0000 Subject: [dba-VB] What to do, what to do? In-Reply-To: <008401ca6544$b4290890$1c7b19b0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4afe94f5.1701d00a.7b63.ffffca5a@mx.google.com> <008201ca6530$daf14140$90d3c3c0$@spb.ru> <4afecf2a.0702d00a.3bd1.4040@mx.google.com> <008401ca6544$b4290890$1c7b19b0$@spb.ru> Message-ID: <4afeda84.0508d00a.3cd0.ffffe9aa@mx.google.com> Ok, I thought you were referring to ram available now on common-or-garden PCs. So, we can look forward to physical ram being cheap enough to enable in-vitro processing for millions of records. Hurrah! And goodbye to latency... Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 16:08 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Max, Here are theoretical limits I have found info about: Address space 64-bit Windows 32-bit Windows Virtual memory 16 Tb 4 Gb Swap file 512 Tb 16 Tb System cache 1 Tb 1 Gb In practice, mass market motherboards can already handle 64GB(?) - Not enough for JC tasks but this max RAM size limitations should be removed in the near future AFAICG, and Win7 is ready(?) to handle effectively that much RAM.... http://www.nextag.com/64-bit-pci-motherboard/search-html http://www.viva64.com/content/articles/64-bit-development/?f=Application_por t_to_64-bit_platforms_or_never_cackle_till_your_egg_is_laid.html&lang=en&con tent=64-bit-development http://blogs.zdnet.com/hardware/?p=4254 http://www.tomshardware.com/reviews/vista-workshop,1775-3.html Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Max Wanadoo Sent: Saturday, November 14, 2009 6:39 PM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Shamil, I understand the logical size bit but for that to be mapped to physical memory requires physical memory to be present. Current there are limits to how much can be installed on the motherboard. 64Bit memory can address the sizes you mentioned but the physical memory doesn't exist, so (presumably) it maps it to the HD, in which case it is not in memory. Have I misunderstood something (it has been known to happen)? Max -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 14 November 2009 13:46 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] What to do, what to do? Hi Max, But OS, programs etc. - they all use their own logical address space, which is mapped to (shared (using OS directed swapping)) physical address space: with JC 64bit PC OS image/runtime will be kept in its own physical memory segments - and no swapping will occur, MS SQL image/runtime/data will be (mainly) kept in its own physical memory segments - and almost no swapping will occur, JC's program will have occupied its own physical memory segments - and no swapping will occur... 50 million records assuming every of them is equal to approx. 1KB will consume ~50GB - 64bit PC can have(?) that much RAM allocated for one process, and it all can be addresses directly within this process logical address space. And I have noted that it's currently a "theoretical consideration" but it may become practically achievable in not that far future(?) Correct/wrong? (I must say I didn't get checked how large can be logical address space for a Win7 process running on 64bit PC). Thank you. -- Shamil <<< snipped >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4607 (20091114) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Nov 14 10:28:14 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 14 Nov 2009 11:28:14 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <008301ca6538$eeef8910$ccce9b30$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> <4AFEBD6E.1020709@colbyconsulting.com> <008301ca6538$eeef8910$ccce9b30$@spb.ru> Message-ID: <4AFEDA9E.9050101@colbyconsulting.com> And believe me I look forward to the day. And it will indeed be soon, of course the data sets get bigger as well. ;) I already have 16 gigs of RAM in my servers, and quad cores. I could easily have 64 gigs and 16 CPUs today if I had the bucks. And of course you are correct as well, the Flash drives are wicked fast for working with large data sets. They still have real issues when writing to them bot for reading they are an order of magnitude faster than rotating media. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > Yes, I have just "theoretically" noted that in the near future you'll be > able to EVEN MORE EASILY work with sets of millions (and billions?) records > by just loading them into memory, "crunching", and saving back updated... > > ... it will be just 1,2,3 instead of your today's (1,2,3) (1,2,3)... (1,2,3) > (multi-threaded)... > > ... the time just to develop (1,2,3) (1,2,3)... (1,2,3)... will be more > expensive than time to develop and run 1,2,3 on the near future hardware > (hardware costs included)... > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 14, 2009 5:24 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] What to do, what to do? > > All of which is beside the point. > > Divide and conquer. A principal discovered by military strategists > thousands of years ago, and > applied to problems of all sorts today. > > I can EASILY work with sets of thousands, and even hundreds of thousands of > records. In real life, > in my computer. > > John W. Colby > www.ColbyConsulting.com > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4607 (20091114) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Nov 14 16:36:57 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 14 Nov 2009 17:36:57 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Message-ID: <4AFF3109.40003@colbyconsulting.com> To get back to reality, I really need to process under 6 million records, and the vast majority will be merging two records into one. I will almost certainly pull about 10,000 records at a time, so I will be processing 5,000 people at a time. Merge two records, merge two more, do that ~ 5000 times, write back to SQL Server, load the next set of 10,000 records. Do that 600 times. I will see how many records can actually be loaded, perhaps 100,000 at a time would work and do that 60 times. Maybe break the process into threads. One thread watches the ADO recordsets in memory and keeps them filled, another thread performs the merge, another thread writes back to SQL Server. In any case there is simply no need to load all 6 million records at once. To get back to the original point however, while TSQL might be able to do what I need, I have serious doubts. Even if it could, trying to debug this kind of program written in TSQL would be a nightmare for me. Furthermore even TSQL could not do this on 6 million records at once. No sane person claims that any specific language is the best for everything, and for this kind of thing ADO and C# is probably a good choice. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Robert, > > Yes, DataSets are mainly targeted for working with small number of records > in memory - so the following is more a "theoretical" calculation: > From jwcolby at colbyconsulting.com Sat Nov 14 23:11:48 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 15 Nov 2009 00:11:48 -0500 Subject: [dba-VB] What to do, what to do? In-Reply-To: <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> References: <4afe3613.0f0bca0a.2938.ffffb791@mx.google.com> <007501ca6516$ebef87f0$c3ce97d0$@spb.ru> Message-ID: <4AFF8D94.8010409@colbyconsulting.com> http://arstechnica.com/hardware/news/2009/11/biography-solid-state-disk.ars John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Robert, > > Yes, DataSets are mainly targeted for working with small number of records > in memory - so the following is more a "theoretical" calculation: > > - JC has 64bit PC which allow to load in memory "practically unlimited" > volume of data - 64bit logical (process) address space could be as large as > 16 exbibytes = 1,152,921,504,606,846,976 bytes * 16 > (http://en.wikipedia.org/wiki/Exbibyte). > > - the speed of data transfer for 64bit Intel's processor - "the 200 MHz > McKinley bus transferred 6.4 GiB/s), and the 533 MHz Montecito bus transfers > 17.056 GiB/s (http://en.wikipedia.org/wiki/IA-64#Memory_architecture); > > ----- > So "theoretically" with modern speedy harddisks JC can load 50 millions of > records into memory within minutes - within half a minute in the near future > when large enough flash-memory disks will become available and relatively > inexpensive? > > To load such large data volume it would be better to use SqlDataReader as it > keeps loaded data in a very compact form. And 50 million records long > SqlDataReader should be probably better split into several chunks... > > Again - this is just a "theoretical" consideration - in practice JC can > process his 50 million records long data table in chunks as he has a hash > field (tblHashPK), which can be used to load related records into memory: I > mean he can split tblHashPK table keeping hash values into several groups, > e.g.: > > 1 - 100,000 > 100,001 - 200,000 > ... > > and process each group joining its records to source 50 million records... > > -- > Shamil From jwcolby at colbyconsulting.com Mon Nov 16 07:42:52 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 16 Nov 2009 08:42:52 -0500 Subject: [dba-VB] Archives down Message-ID: <4B0156DC.5000407@colbyconsulting.com> I am unable to get at the archives for DBA-VB. Does anyone here use the ADO.Net Entity Framework? Why? Does anyone here specifically AVOID using the ADO.Net Entity Framework? Why? Thanks, -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Mon Nov 16 07:59:09 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 16 Nov 2009 16:59:09 +0300 Subject: [dba-VB] Archives down In-Reply-To: <4B0156DC.5000407@colbyconsulting.com> References: <4B0156DC.5000407@colbyconsulting.com> Message-ID: <00d901ca66c4$f94089e0$ebc19da0$@spb.ru> Hi John -- ADO.NET EF is a great thing. But maybe it will be wise to wait to use it in production GUI when VS2010 will be released then it will be possible to have natively bound WinForms and WPF Forms (and ASP.NET controls? And SilverLight controls), and where ADO.NET EF entities' mapping to db objects will be more flexible, and where it will be possible to have entity model driven(?) (database model) development... I have recently posted here a link to the following presentation: "ORM, EDM, ESQL, Entity Framework, LINQ to SQL, LINQ to Entities - Confused?" http://www.infoq.com/presentations/ORM-LINQ-Entity-Framework-Eric-Nelson I suppose it will answer most of your questions. BTW, our http://northwind.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=26600 project has a couple of ADO.NET EF samples - SPJWinFormsForEF_VS2008.zip source code, 838K, uploaded Jul 23 Application SPJWinFormsForEF_Executables.zip application, 203K, uploaded Jul 23 Thank you. -- Shamil P.S. BTW, I have mainly skipped "LINQ to SQL" - and I'm now happy (:)) I will not need to rewrite my applications as "LINQ to SQL" development is stopped there at MS AFAIU... -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, November 16, 2009 4:43 PM To: VBA Subject: [dba-VB] Archives down I am unable to get at the archives for DBA-VB. Does anyone here use the ADO.Net Entity Framework? Why? Does anyone here specifically AVOID using the ADO.Net Entity Framework? Why? Thanks, -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4611 (20091116) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From Gustav at cactus.dk Mon Nov 16 09:24:01 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 16 Nov 2009 16:24:01 +0100 Subject: [dba-VB] Archives down Message-ID: Hi John I like the concept and the design features, but have only used it for experiments and testing. It takes some time to get around it - I haven't concluded the tour yet ... /gustav >>> jwcolby at colbyconsulting.com 16-11-2009 14:42 >>> I am unable to get at the archives for DBA-VB. Does anyone here use the ADO.Net Entity Framework? Why? Does anyone here specifically AVOID using the ADO.Net Entity Framework? Why? Thanks, -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Tue Nov 17 17:04:12 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Tue, 17 Nov 2009 18:04:12 -0500 Subject: [dba-VB] First little applet Message-ID: <4B032BEC.9050508@colbyconsulting.com> Today I wrote my first C# applet. I have to import CSV files coming back from a processing house into SQL Server. These files had errors in one or more lines in the files. Given that there are tens of millions of lines in each file it isn't possible to just poke around and find the errors. So I wrote an applet that opens the CSV file, reads the header line and counts the " and , characters. Then line by line the applet reads the rest of the lines, comparing the counts to that header line counts. This found lines in the file to be imported where " characters had been dropped, which was causing the import wizard to error out at that point. So read, compare counts, write good lines to output file, write bad lines to an error file. Simple stuff really but it is my first real work done in C#. The speed of this kind of thing is pretty darned good too I thought. I was getting about 300K lines / second read / written. One file had 20.6 million lines, the other had 49 million lines. In the past I would have done this in VBA / Access. Of course it took me a tad longer to write in C# than it would have in VBA. -- John W. Colby www.ColbyConsulting.com From marklbreen at gmail.com Wed Nov 18 02:54:19 2009 From: marklbreen at gmail.com (Mark Breen) Date: Wed, 18 Nov 2009 08:54:19 +0000 Subject: [dba-VB] First little applet In-Reply-To: <4B032BEC.9050508@colbyconsulting.com> References: <4B032BEC.9050508@colbyconsulting.com> Message-ID: Congrats to you John, I suspect once you get a few more utils in your library, you will be flying from here on. good for you, Mark 2009/11/17 jwcolby > Today I wrote my first C# applet. I have to import CSV files coming back > from a processing house > into SQL Server. These files had errors in one or more lines in the files. > Given that there are > tens of millions of lines in each file it isn't possible to just poke > around and find the errors. > > So I wrote an applet that opens the CSV file, reads the header line and > counts the " and , > characters. Then line by line the applet reads the rest of the lines, > comparing the counts to that > header line counts. > > This found lines in the file to be imported where " characters had been > dropped, which was causing > the import wizard to error out at that point. > > So read, compare counts, write good lines to output file, write bad lines > to an error file. > > Simple stuff really but it is my first real work done in C#. > > The speed of this kind of thing is pretty darned good too I thought. I was > getting about 300K lines > / second read / written. One file had 20.6 million lines, the other had 49 > million lines. > > In the past I would have done this in VBA / Access. Of course it took me a > tad longer to write in > C# than it would have in VBA. > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Wed Nov 18 03:11:43 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 18 Nov 2009 10:11:43 +0100 Subject: [dba-VB] First little applet Message-ID: Hi John Great! The good thing is that code from this little applet can easily be incorporated in another project. /gustav >>> jwcolby at colbyconsulting.com 18-11-2009 00:04 >>> Today I wrote my first C# applet. I have to import CSV files coming back from a processing house into SQL Server. These files had errors in one or more lines in the files. Given that there are tens of millions of lines in each file it isn't possible to just poke around and find the errors. So I wrote an applet that opens the CSV file, reads the header line and counts the " and , characters. Then line by line the applet reads the rest of the lines, comparing the counts to that header line counts. This found lines in the file to be imported where " characters had been dropped, which was causing the import wizard to error out at that point. So read, compare counts, write good lines to output file, write bad lines to an error file. Simple stuff really but it is my first real work done in C#. The speed of this kind of thing is pretty darned good too I thought. I was getting about 300K lines / second read / written. One file had 20.6 million lines, the other had 49 million lines. In the past I would have done this in VBA / Access. Of course it took me a tad longer to write in C# than it would have in VBA. -- John W. Colby www.ColbyConsulting.com From andy at minstersystems.co.uk Wed Nov 18 05:23:43 2009 From: andy at minstersystems.co.uk (Andy Lacey) Date: Wed, 18 Nov 2009 12:23:43 +0100 Subject: [dba-VB] First little applet Message-ID: <200911181223.nAICNl4W018133@databaseadvisors.com> Congratulations John. Hope father and little applet are both doing well. Andy --------- Original Message -------- From: "Discussion concerning Visual Basic and related programming issues." To: "Discussion concerning Visual Basic and related programming issues." Subject: Re: [dba-VB] First little applet Date: 18/11/09 08:55 Congrats to you John, I suspect once you get a few more utils in your library, you will be flying from here on. good for you, Mark 2009/11/17 jwcolby > Today I wrote my first C# applet. I have to import CSV files coming back > from a processing house > into SQL Server. These files had errors in one or more lines in the files. > Given that there are > tens of millions of lines in each file it isn't possible to just poke > around and find the errors. > > So I wrote an applet that opens the CSV file, reads the header line and > counts the " and , > characters. Then line by line the applet reads the rest of the lines, > comparing the counts to that > header line counts. > > This found lines in the file to be imported where " characters had been > dropped, which was causing > the import wizard to error out at that point. > > So read, compare counts, write good lines to output file, write bad lines > to an error file. > > Simple stuff really but it is my first real work done in C#. > > The speed of this kind of thing is pretty darned good too I thought. I was > getting about 300K lines > / second read / written. One file had 20.6 million lines, the other had 49 > million lines. > > In the past I would have done this in VBA / Access. Of course it took me a > tad longer to write in > C# than it would have in VBA. > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com ________________________________________________ Message sent using UebiMiau 2.7.2 From jwcolby at colbyconsulting.com Wed Nov 18 06:45:16 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 07:45:16 -0500 Subject: [dba-VB] First little applet In-Reply-To: <200911181223.nAICNl4W018133@databaseadvisors.com> References: <200911181223.nAICNl4W018133@databaseadvisors.com> Message-ID: <4B03EC5C.4090101@colbyconsulting.com> ROTFL. Father is so proud of little applet. ;) John W. Colby www.ColbyConsulting.com Andy Lacey wrote: > Congratulations John. Hope father and little applet are both doing well. > > Andy > > > --------- Original Message -------- > From: "Discussion concerning Visual Basic and related programming issues." > > To: "Discussion concerning Visual Basic and related programming issues." > > Subject: Re: [dba-VB] First little applet > Date: 18/11/09 08:55 > > > Congrats to you John, > > I suspect once you get a few more utils in your library, you will be flying > from here on. > > good for you, > > Mark > > > > 2009/11/17 jwcolby > >> Today I wrote my first C# applet. I have to import CSV files coming back >> from a processing house >> into SQL Server. These files had errors in one or more lines in the files. >> Given that there are >> tens of millions of lines in each file it isn't possible to just poke >> around and find the errors. >> >> So I wrote an applet that opens the CSV file, reads the header line and >> counts the " and , >> characters. Then line by line the applet reads the rest of the lines, >> comparing the counts to that >> header line counts. >> >> This found lines in the file to be imported where " characters had been >> dropped, which was causing >> the import wizard to error out at that point. >> >> So read, compare counts, write good lines to output file, write bad lines >> to an error file. >> >> Simple stuff really but it is my first real work done in C#. >> >> The speed of this kind of thing is pretty darned good too I thought. I was >> getting about 300K lines >> / second read / written. One file had 20.6 million lines, the other had 49 >> million lines. >> >> In the past I would have done this in VBA / Access. Of course it took me a >> tad longer to write in >> C# than it would have in VBA. >> >> -- >> John W. Colby >> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > ________________________________________________ > Message sent using UebiMiau 2.7.2 > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Wed Nov 18 06:50:01 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 07:50:01 -0500 Subject: [dba-VB] SPAM-LOW: Re: First little applet In-Reply-To: References: Message-ID: <4B03ED79.5020704@colbyconsulting.com> I do imports from CSV for large files all of the time, and I have had similar failures in the past, so I will be keeping this applet for future use. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > Great! The good thing is that code from this little applet can easily be incorporated in another project. > > /gustav > > >>>> jwcolby at colbyconsulting.com 18-11-2009 00:04 >>> > Today I wrote my first C# applet. I have to import CSV files coming back from a processing house > into SQL Server. These files had errors in one or more lines in the files. Given that there are > tens of millions of lines in each file it isn't possible to just poke around and find the errors. > > So I wrote an applet that opens the CSV file, reads the header line and counts the " and , > characters. Then line by line the applet reads the rest of the lines, comparing the counts to that > header line counts. > > This found lines in the file to be imported where " characters had been dropped, which was causing > the import wizard to error out at that point. > > So read, compare counts, write good lines to output file, write bad lines to an error file. > > Simple stuff really but it is my first real work done in C#. > > The speed of this kind of thing is pretty darned good too I thought. I was getting about 300K lines > / second read / written. One file had 20.6 million lines, the other had 49 million lines. > > In the past I would have done this in VBA / Access. Of course it took me a tad longer to write in > C# than it would have in VBA. > From jwcolby at colbyconsulting.com Wed Nov 18 07:06:44 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 08:06:44 -0500 Subject: [dba-VB] Projects vs Solutions Message-ID: <4B03F164.6090601@colbyconsulting.com> I am hazy on how things are organized in .Net once you start building pieces of code. What is a project, what is a solution and how are they tied together. My role for the "DBFH" client involves lots of different tasks, for which there will be many different pieces. I would hope to tie the pieces together in one big application. For example I just wrote the little applet to cleanup CSV files, which will have to be used again. I do exports to Accuzip, imports from Accuzip, these two are closely related obviously. I build orders. I have to export data from tables for processing by external processing houses, which involves a very similar export / zip / FTP process. So as I build these processes in C#, how are they physically organized on the disk, and how are they tied back together into one program. From what I am reading, a Solution is the big picture, a Project is a small part of the Solution. But when I open VS2008 there is no "create a solution" or at least I don't see one. So how does one create a solution, and then look at just one project at a time inside of that solution? -- John W. Colby www.ColbyConsulting.com From Gustav at cactus.dk Wed Nov 18 07:17:08 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 18 Nov 2009 14:17:08 +0100 Subject: [dba-VB] Projects vs Solutions Message-ID: Hi John On the New Project pane at the bottom select the name for the (new) solution and mark Create directory for solution. That folder holds your projects of this solution. All projects of a solution are listed in the treeview (typically the right panel). Open only those projects you wish to. /gustav >>> jwcolby at colbyconsulting.com 18-11-2009 14:06 >>> I am hazy on how things are organized in .Net once you start building pieces of code. What is a project, what is a solution and how are they tied together. My role for the "DBFH" client involves lots of different tasks, for which there will be many different pieces. I would hope to tie the pieces together in one big application. For example I just wrote the little applet to cleanup CSV files, which will have to be used again. I do exports to Accuzip, imports from Accuzip, these two are closely related obviously. I build orders. I have to export data from tables for processing by external processing houses, which involves a very similar export / zip / FTP process. So as I build these processes in C#, how are they physically organized on the disk, and how are they tied back together into one program. From what I am reading, a Solution is the big picture, a Project is a small part of the Solution. But when I open VS2008 there is no "create a solution" or at least I don't see one. So how does one create a solution, and then look at just one project at a time inside of that solution? -- John W. Colby www.ColbyConsulting.com From cfoust at infostatsystems.com Wed Nov 18 10:12:41 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 10:12:41 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B03F164.6090601@colbyconsulting.com> References: <4B03F164.6090601@colbyconsulting.com> Message-ID: Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 5:07 AM To: VBA Subject: [dba-VB] Projects vs Solutions I am hazy on how things are organized in .Net once you start building pieces of code. What is a project, what is a solution and how are they tied together. My role for the "DBFH" client involves lots of different tasks, for which there will be many different pieces. I would hope to tie the pieces together in one big application. For example I just wrote the little applet to cleanup CSV files, which will have to be used again. I do exports to Accuzip, imports from Accuzip, these two are closely related obviously. I build orders. I have to export data from tables for processing by external processing houses, which involves a very similar export / zip / FTP process. So as I build these processes in C#, how are they physically organized on the disk, and how are they tied back together into one program. From what I am reading, a Solution is the big picture, a Project is a small part of the Solution. But when I open VS2008 there is no "create a solution" or at least I don't see one. So how does one create a solution, and then look at just one project at a time inside of that solution? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Wed Nov 18 11:57:56 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 12:57:56 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: <4B03F164.6090601@colbyconsulting.com> Message-ID: <4B0435A4.3050804@colbyconsulting.com> OK, this immediately causes issues referencing objects in another project. For example I have a "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I then need to be able to open a form in the other project where I gather information about the file to fix. The objects in that other project aren't visible in intellisense. "Normal" opening of a form is done with something like MyFormName Somename = new MyFormNname(); SomeName.Show(); Well... MyFormName is not valid in project main. While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. > > Charlotte Foust > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, November 18, 2009 5:07 AM > To: VBA > Subject: [dba-VB] Projects vs Solutions From cfoust at infostatsystems.com Wed Nov 18 12:16:55 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 12:16:55 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B0435A4.3050804@colbyconsulting.com> References: <4B03F164.6090601@colbyconsulting.com> <4B0435A4.3050804@colbyconsulting.com> Message-ID: That's where references and "Imports" come in. You need to be careful about project level references to avoid circular messes, so our UI and Data projects have references to the Configuration project, but not the other way around, and the UI has a reference to the Data project but not the other way around. You can always fully reference a class in another project in code starting with the Namespace, so if I need to use something in a project that isn't referenced in the current one, I can use Infostat.RIMDrill.Configuration.RIMConstants (for example) and have access to the methods and properties of RIMConstants even without a reference in the current project. Alternatively, I could use Imports Infostat.RIMDrill.Configuration in the declarations above my current class and get the same thing. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 9:58 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions OK, this immediately causes issues referencing objects in another project. For example I have a "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I then need to be able to open a form in the other project where I gather information about the file to fix. The objects in that other project aren't visible in intellisense. "Normal" opening of a form is done with something like MyFormName Somename = new MyFormNname(); SomeName.Show(); Well... MyFormName is not valid in project main. While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. > > Charlotte Foust > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, November 18, 2009 5:07 AM > To: VBA > Subject: [dba-VB] Projects vs Solutions _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From Gustav at cactus.dk Wed Nov 18 12:27:23 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 18 Nov 2009 19:27:23 +0100 Subject: [dba-VB] Projects vs Solutions Message-ID: Hi John In addition to the comments from Charlotte: Also, you can either add in your project Using ProjectOther; or state this explicitly ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); someName.Show(); /gustav >>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> OK, this immediately causes issues referencing objects in another project. For example I have a "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I then need to be able to open a form in the other project where I gather information about the file to fix. The objects in that other project aren't visible in intellisense. "Normal" opening of a form is done with something like MyFormName Somename = new MyFormNname(); SomeName.Show(); Well... MyFormName is not valid in project main. While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. > > Charlotte Foust From jwcolby at colbyconsulting.com Wed Nov 18 12:54:47 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 13:54:47 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: Message-ID: <4B0442F7.2070805@colbyconsulting.com> Using projOther doesn't work. When doing this is the project allowed to be a part of the solution? Is it even SUPPOSED TO BE a part of the solution. IOW are you supposed to create projects as separate entities and then reference them from other projects, or can you create SolutionMain / ProjMain / projOther1 / projOther2 etc. I have the latter situation (now). When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. This whole area seems poorly documented and rather messy. Lots of chatter on Google about how difficult it is to move a project and so forth. This is the kind of stuff I hate to even get involved in. I am trying to get work done but get sucked off into areas where it certainly looks like I could easily wreak havoc on working code. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > In addition to the comments from Charlotte: > > Also, you can either add in your project > > Using ProjectOther; > > or state this explicitly > > ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); > someName.Show(); > > /gustav > >>>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> > OK, this immediately causes issues referencing objects in another project. For example I have a > "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I > then need to be able to open a form in the other project where I gather information about the file > to fix. > > The objects in that other project aren't visible in intellisense. > > "Normal" opening of a form is done with something like > > MyFormName Somename = new MyFormNname(); > SomeName.Show(); > > Well... MyFormName is not valid in project main. > > While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. >> >> Charlotte Foust > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From cfoust at infostatsystems.com Wed Nov 18 13:03:55 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 13:03:55 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B0442F7.2070805@colbyconsulting.com> References: <4B0442F7.2070805@colbyconsulting.com> Message-ID: A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 10:55 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions Using projOther doesn't work. When doing this is the project allowed to be a part of the solution? Is it even SUPPOSED TO BE a part of the solution. IOW are you supposed to create projects as separate entities and then reference them from other projects, or can you create SolutionMain / ProjMain / projOther1 / projOther2 etc. I have the latter situation (now). When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. This whole area seems poorly documented and rather messy. Lots of chatter on Google about how difficult it is to move a project and so forth. This is the kind of stuff I hate to even get involved in. I am trying to get work done but get sucked off into areas where it certainly looks like I could easily wreak havoc on working code. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > In addition to the comments from Charlotte: > > Also, you can either add in your project > > Using ProjectOther; > > or state this explicitly > > ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); > someName.Show(); > > /gustav > >>>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> > OK, this immediately causes issues referencing objects in another > project. For example I have a "main" project, and a "CSVRepair" > project. The Main project form opens when I run the project but I > then need to be able to open a form in the other project where I gather information about the file to fix. > > The objects in that other project aren't visible in intellisense. > > "Normal" opening of a form is done with something like > > MyFormName Somename = new MyFormNname(); SomeName.Show(); > > Well... MyFormName is not valid in project main. > > While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. >> >> Charlotte Foust > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From Gustav at cactus.dk Wed Nov 18 13:30:04 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 18 Nov 2009 20:30:04 +0100 Subject: [dba-VB] Projects vs Solutions Message-ID: Hi John Sorry, I read it as if ProjectOther was in the same (current) solution. If not, as Charlotte explains, then in the current solution reference the project from the other solution where it is situated. You will be given the choice to either leave the files of the project or to import them. What to chose is a question of maintenance. /gustav >>> jwcolby at colbyconsulting.com 18-11-2009 19:54 >>> Using projOther doesn't work. When doing this is the project allowed to be a part of the solution? Is it even SUPPOSED TO BE a part of the solution. IOW are you supposed to create projects as separate entities and then reference them from other projects, or can you create SolutionMain / ProjMain / projOther1 / projOther2 etc. I have the latter situation (now). When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. This whole area seems poorly documented and rather messy. Lots of chatter on Google about how difficult it is to move a project and so forth. This is the kind of stuff I hate to even get involved in. I am trying to get work done but get sucked off into areas where it certainly looks like I could easily wreak havoc on working code. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > In addition to the comments from Charlotte: > > Also, you can either add in your project > > Using ProjectOther; > > or state this explicitly > > ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); > someName.Show(); > > /gustav > >>>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> > OK, this immediately causes issues referencing objects in another project. For example I have a > "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I > then need to be able to open a form in the other project where I gather information about the file > to fix. > > The objects in that other project aren't visible in intellisense. > > "Normal" opening of a form is done with something like > > MyFormName Somename = new MyFormNname(); > SomeName.Show(); > > Well... MyFormName is not valid in project main. > > While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. >> >> Charlotte Foust From shamil at smsconsulting.spb.ru Wed Nov 18 13:35:05 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Wed, 18 Nov 2009 22:35:05 +0300 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B0442F7.2070805@colbyconsulting.com> References: <4B0442F7.2070805@colbyconsulting.com> Message-ID: <000501ca6886$3c5fd420$b51f7c60$@spb.ru> Hi John -- In fact referencing other projects works very well in .NET, and it's clean and very useful: just note that 'using' directive is to refer to *namespaces* not to projects. If you right-click a project in Solution Explorer, and then select 'Properties' you can find that every project has 'Default Namespace', which is usually the same as project name but this is not necessary. When I'm developing my solution and I need "to get work done" I do sometimes postpone creating new projects within a solution - I just create subfolders in my main project's treeview and I put my code to be later moved to other projects into those subfolders, and I keep namespace for those subfolders different from the default project's namespace - check it out... You'll soon get fluent with that namespaces and projects stuff and then you'll be "cooking" new projects seamlessly - it's usual story to have 10+ projects within .NET solutions... <<< When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. >>> Do you have 'ProjOther' within your solution? Did you set reference to 'ProjOther'? Does 'ProjOther''s default namespace is 'ProjOther'? 'using' directive should be started with lowercase letter.... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 9:55 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions Using projOther doesn't work. When doing this is the project allowed to be a part of the solution? Is it even SUPPOSED TO BE a part of the solution. IOW are you supposed to create projects as separate entities and then reference them from other projects, or can you create SolutionMain / ProjMain / projOther1 / projOther2 etc. I have the latter situation (now). When I try to do "Using projOther1;" from inside of ProjMain it says that ProjOther doesn't exist. This whole area seems poorly documented and rather messy. Lots of chatter on Google about how difficult it is to move a project and so forth. This is the kind of stuff I hate to even get involved in. I am trying to get work done but get sucked off into areas where it certainly looks like I could easily wreak havoc on working code. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > In addition to the comments from Charlotte: > > Also, you can either add in your project > > Using ProjectOther; > > or state this explicitly > > ProjectOther.MyFormName someName = new ProjectOther.MyFormName(); > someName.Show(); > > /gustav > >>>> jwcolby at colbyconsulting.com 18-11-2009 18:57 >>> > OK, this immediately causes issues referencing objects in another project. For example I have a > "main" project, and a "CSVRepair" project. The Main project form opens when I run the project but I > then need to be able to open a form in the other project where I gather information about the file > to fix. > > The objects in that other project aren't visible in intellisense. > > "Normal" opening of a form is done with something like > > MyFormName Somename = new MyFormNname(); > SomeName.Show(); > > Well... MyFormName is not valid in project main. > > While we are on the subject, what happens if I just need to dimension a class used in project CSVRepair? > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> Think of the solution as the application. The projects within the solution can be shared with other applications. We use the projects primarily to group parts of our apps, like UI, Reports, Data, Configuration, Security, etc., but it really is up to the architect of the app. >> >> Charlotte Foust > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4618 (20091118) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4618 (20091118) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Wed Nov 18 13:40:52 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 14:40:52 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: <4B0442F7.2070805@colbyconsulting.com> Message-ID: <4B044DC4.2050304@colbyconsulting.com> OK, let's back up a minute. I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". Except that: a) I really want the Solution to be InfoEngine. b) I would PREFER that the other projects be physically stored under InfoEngine. c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. d) Each of these "projects" contains a solution file. So what is it, a solution or a project? Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. > > Charlotte Foust From shamil at smsconsulting.spb.ru Wed Nov 18 14:04:58 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Wed, 18 Nov 2009 23:04:58 +0300 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B044DC4.2050304@colbyconsulting.com> References: <4B0442F7.2070805@colbyconsulting.com> <4B044DC4.2050304@colbyconsulting.com> Message-ID: <000c01ca688a$68c16200$3a442600$@spb.ru> John -- Just make an experiment with a test solution from scratch: - start VS; - create C# Console project - TestConsole - that will create a project TestConsole and a solution - TestConsole; - File->New->Project... - create C# Class Library project - MyClassLib1; - File->New->Project... - create C# Class Library project - MyClassLib2; - ... - Right-Click TestConsole project in Solution Explorer -> Add Reference -> [Projects] (third tab) -> Select MyClassLib1; - ... - now in TestConsole Program.cs you can type: using MyClassLib1; and you'll have Class1 (from MyClassLib1 - MyclassLib1.Class1) available to Intellisense.... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 10:41 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions OK, let's back up a minute. I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". Except that: a) I really want the Solution to be InfoEngine. b) I would PREFER that the other projects be physically stored under InfoEngine. c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. d) Each of these "projects" contains a solution file. So what is it, a solution or a project? Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. > > Charlotte Foust __________ Information from ESET NOD32 Antivirus, version of virus signature database 4618 (20091118) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From cfoust at infostatsystems.com Wed Nov 18 14:49:22 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 14:49:22 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B044DC4.2050304@colbyconsulting.com> References: <4B0442F7.2070805@colbyconsulting.com> <4B044DC4.2050304@colbyconsulting.com> Message-ID: John, Susan and I ran into this when we wrote a beginner's article. In tools -> Options under Project and Solutions, make sure the Always Show Solution checkbox is checked. Otherwise, you only see a solution and you keep creating separate solutions for each project. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 11:41 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions OK, let's back up a minute. I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". Except that: a) I really want the Solution to be InfoEngine. b) I would PREFER that the other projects be physically stored under InfoEngine. c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. d) Each of these "projects" contains a solution file. So what is it, a solution or a project? Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. > > Charlotte Foust _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From cfoust at infostatsystems.com Wed Nov 18 18:21:50 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Wed, 18 Nov 2009 18:21:50 -0600 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: <4B0442F7.2070805@colbyconsulting.com> <4B044DC4.2050304@colbyconsulting.com> Message-ID: Oops, I meant you only see a PROJECT unles the Always Show Solution checkbox is checked! Sorry!! Charlotte -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Charlotte Foust Sent: Wednesday, November 18, 2009 12:49 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions John, Susan and I ran into this when we wrote a beginner's article. In tools -> Options under Project and Solutions, make sure the Always Show Solution checkbox is checked. Otherwise, you only see a solution and you keep creating separate solutions for each project. Charlotte Foust -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 18, 2009 11:41 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Projects vs Solutions OK, let's back up a minute. I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". Except that: a) I really want the Solution to be InfoEngine. b) I would PREFER that the other projects be physically stored under InfoEngine. c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. d) Each of these "projects" contains a solution file. So what is it, a solution or a project? Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. > > Charlotte Foust _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Wed Nov 18 20:14:37 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 18 Nov 2009 21:14:37 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: <4B0442F7.2070805@colbyconsulting.com> <4B044DC4.2050304@colbyconsulting.com> Message-ID: <4B04AA0D.5030107@colbyconsulting.com> Creating a reference to the other project solved the immediate problem of referencing objects in the other project. John W. Colby www.ColbyConsulting.com Charlotte Foust wrote: > Oops, I meant you only see a PROJECT unles the Always Show Solution checkbox is checked! Sorry!! > > Charlotte > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Charlotte Foust > Sent: Wednesday, November 18, 2009 12:49 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Projects vs Solutions > > John, > > Susan and I ran into this when we wrote a beginner's article. In tools -> Options under Project and Solutions, make sure the Always Show Solution checkbox is checked. Otherwise, you only see a solution and you keep creating separate solutions for each project. > > Charlotte Foust > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, November 18, 2009 11:41 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Projects vs Solutions > > OK, let's back up a minute. > > I created a project InfoEngine. I created another project ScanCSVForErrors. I created another project MergePurge. Etc. > > Each one of these is both a solution and a project. What I mean by that is that when you open Visual Studio you "Create" projects, at least unless you do some unknown thing. There is no "New Solution", there is only "new project". > > Except that: > > a) I really want the Solution to be InfoEngine. > b) I would PREFER that the other projects be physically stored under InfoEngine. > c) Of course I want these projects to be considered by the SOLUTION InfoEngine to belong to the solution InfoEngine. > d) Each of these "projects" contains a solution file. So what is it, a solution or a project? > > Is this a "start over stupid" kind of moment? Is it just me or is this whole Solution / Project thing extremely poorly defined? Why do we discuss Solutions and can't even purposely create one? > Why does a project get a solution file if it is a project? What the hell IS a solution? Where are the tools for migrating existing projects into a solution? > > By the by, I have a solution explorer up and it contains two projects now, but I can't reference, EVEN PURPOSELY directly in code, objects in one project from the other project. It just tells me "doesn't exist". > > And so... instead of writing a solution (or is that a project? Or perhaps a project in a solution???), I walk around the forest in circles. > > John W. Colby > www.ColbyConsulting.com > > > Charlotte Foust wrote: >> A project IS part of the solution. The solution is the outer namespace for the application. It contains projects. Your current situation is the way we do it (I think). The several projects don't contain references to one another unless you create them by right clicking the project and select Add Reference. >> >> Charlotte Foust > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Thu Nov 19 03:41:50 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Thu, 19 Nov 2009 10:41:50 +0100 Subject: [dba-VB] Projects vs Solutions Message-ID: Hi John This raises the question if your coming applet will be the new little applet or will become a little bigger little applet from day one. Does a little applet grow up? /gustav >>> jwcolby at colbyconsulting.com 19-11-2009 03:14 >>> Creating a reference to the other project solved the immediate problem of referencing objects in the other project. John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Thu Nov 19 07:10:03 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 19 Nov 2009 08:10:03 -0500 Subject: [dba-VB] Projects vs Solutions In-Reply-To: References: Message-ID: <4B0543AB.5080700@colbyconsulting.com> Gustav, Well, the point is that I am doing my job and suddenly I have to do something for the client that I have no solution for. This latest was a good example. I tried to import three files coming back from a processing vendor and one imported, the other two failed. So I write an applet that scans through the file sorting lines of the file into two output files, good and bad. When I am done the remaining "good" file now imports. What do I do with this applet? One thing I can do is to just leave it where it is and hope I don't lose it, because sure as the sun rises I will need to do this again. Or do I create a bigger application, build a menu in it, and place this applet on the menu? Does this Fixit applet grow? Conceivably, if I find some file that isn't fixed (or problem found) as easily as these files were, then this applet could grow to include different tests. This applet has its own form with a set of text boxes to display paths / files, buttons to select folders and files and a button to perform the cleanup. I probably will add another button to open the resulting "bad" file in an editor so that I can see and fix the problems in the data. Perhaps a button to append the fixed lines back to the end of the "good" file before the import. Stuff like that. So applets can grow, and the main application grows as I write new applets to perform various processing tasks, and those new applets get added into the main application. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > This raises the question if your coming applet will be the new little applet or will become a little bigger little applet from day one. Does a little applet grow up? > > /gustav > > >>>> jwcolby at colbyconsulting.com 19-11-2009 03:14 >>> > Creating a reference to the other project solved the immediate problem of referencing objects in the > other project. > > John W. Colby > www.ColbyConsulting.com From marklbreen at gmail.com Fri Nov 20 03:05:11 2009 From: marklbreen at gmail.com (Mark Breen) Date: Fri, 20 Nov 2009 09:05:11 +0000 Subject: [dba-VB] Projects vs Solutions In-Reply-To: <4B0543AB.5080700@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> Message-ID: Hello John, While I am not qualified to comment really on this subject, I am waiting / hoping that someone will let you now how you can move 90% of your applet to a DLL project, and then from your main program you will just reference the DLL and consume the functionality within that DLL. With that approach, you could start your library of the new Colby C# Framework, albeit a humble beginning. Am I correct in assuming that with this approach you would compile your applet to to a DLL, and then you main program, which could be a winform or console app, could call applet.InputFilePathSet, applet.OutPutFileSet, and then applet.ParseFile etc etc. Let I say, I am not qualified to advise on this, but perhaps some others would like to, thanks Mark 2009/11/19 jwcolby > Gustav, > > Well, the point is that I am doing my job and suddenly I have to do > something for the client that I > have no solution for. This latest was a good example. I tried to import > three files coming back > from a processing vendor and one imported, the other two failed. So I > write an applet that scans > through the file sorting lines of the file into two output files, good and > bad. When I am done the > remaining "good" file now imports. > > What do I do with this applet? One thing I can do is to just leave it > where it is and hope I don't > lose it, because sure as the sun rises I will need to do this again. Or do > I create a bigger > application, build a menu in it, and place this applet on the menu? > > Does this Fixit applet grow? Conceivably, if I find some file that isn't > fixed (or problem found) > as easily as these files were, then this applet could grow to include > different tests. This applet > has its own form with a set of text boxes to display paths / files, buttons > to select folders and > files and a button to perform the cleanup. I probably will add another > button to open the resulting > "bad" file in an editor so that I can see and fix the problems in the data. > Perhaps a button to > append the fixed lines back to the end of the "good" file before the > import. Stuff like that. > > So applets can grow, and the main application grows as I write new applets > to perform various > processing tasks, and those new applets get added into the main > application. > > John W. Colby > www.ColbyConsulting.com > > > Gustav Brock wrote: > > Hi John > > > > This raises the question if your coming applet will be the new little > applet or will become a little bigger little applet from day one. Does a > little applet grow up? > > > > /gustav > > > > > >>>> jwcolby at colbyconsulting.com 19-11-2009 03:14 >>> > > Creating a reference to the other project solved the immediate problem of > referencing objects in the > > other project. > > > > John W. Colby > > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 20 06:26:10 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 20 Nov 2009 07:26:10 -0500 Subject: [dba-VB] SPAM-LOW: Re: Projects vs Solutions In-Reply-To: References: <4B0543AB.5080700@colbyconsulting.com> Message-ID: <4B068AE2.9060301@colbyconsulting.com> The blind leading the blind here. 1) I built a main application 2) I referenced the existing file repair applet from the main application (project). 3) I set a using statement. It appears that you have to both reference it and then use the "using" statement. 4) I can now open forms out in the file repair applet from the main application. 5) I physically moved the file repair applet underneath the main application directory. 6) I changed the directory for the applet and it just worked. That was fairly easy. From this point on I "Add Project" to the main solution. I have added a class project to wrap the DMO. In case you haven't discovered it, the DMO is a real cool SQL Server Management Object API that allows you to see and manage database objects. I am just getting into it but it allows me to reference a server object, then see the database collection. The each database object has a table collection, the table object has a fields collection etc. Everything you can see and manage in the SQL Server management studio you can (apparently) see and manage from the SMO from C#. An example of what this does for me is allows me to see all of the databases in a server, and thus populate a combo with their names. Selecting a database from the combo I can see and fill a combo with the names of the tables. Selecting a database and a specific table I can then can then run my stored procedures that export that table in that database to CSV files. That kind of stuff is what I do a lot of and what the big application will manage. John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > While I am not qualified to comment really on this subject, I am waiting / > hoping that someone will let you now how you can move 90% of your applet to > a DLL project, and then from your main program you will just reference the > DLL and consume the functionality within that DLL. > > With that approach, you could start your library of the new Colby C# > Framework, albeit a humble beginning. > > Am I correct in assuming that with this approach you would compile your > applet to to a DLL, and then you main program, which could be a winform or > console app, could call applet.InputFilePathSet, applet.OutPutFileSet, and > then applet.ParseFile etc etc. > > Let I say, I am not qualified to advise on this, but perhaps some others > would like to, > > thanks > > Mark > > > 2009/11/19 jwcolby > >> Gustav, >> >> Well, the point is that I am doing my job and suddenly I have to do >> something for the client that I >> have no solution for. This latest was a good example. I tried to import >> three files coming back >> from a processing vendor and one imported, the other two failed. So I >> write an applet that scans >> through the file sorting lines of the file into two output files, good and >> bad. When I am done the >> remaining "good" file now imports. >> >> What do I do with this applet? One thing I can do is to just leave it >> where it is and hope I don't >> lose it, because sure as the sun rises I will need to do this again. Or do >> I create a bigger >> application, build a menu in it, and place this applet on the menu? >> >> Does this Fixit applet grow? Conceivably, if I find some file that isn't >> fixed (or problem found) >> as easily as these files were, then this applet could grow to include >> different tests. This applet >> has its own form with a set of text boxes to display paths / files, buttons >> to select folders and >> files and a button to perform the cleanup. I probably will add another >> button to open the resulting >> "bad" file in an editor so that I can see and fix the problems in the data. >> Perhaps a button to >> append the fixed lines back to the end of the "good" file before the >> import. Stuff like that. >> >> So applets can grow, and the main application grows as I write new applets >> to perform various >> processing tasks, and those new applets get added into the main >> application. >> >> John W. Colby >> www.ColbyConsulting.com >> >> >> Gustav Brock wrote: >>> Hi John >>> >>> This raises the question if your coming applet will be the new little >> applet or will become a little bigger little applet from day one. Does a >> little applet grow up? >>> /gustav >>> >>> >>>>>> jwcolby at colbyconsulting.com 19-11-2009 03:14 >>> >>> Creating a reference to the other project solved the immediate problem of >> referencing objects in the >>> other project. >>> >>> John W. Colby >>> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 20 09:35:42 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 20 Nov 2009 10:35:42 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B068AE2.9060301@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> Message-ID: <4B06B74E.9080909@colbyconsulting.com> The object I am referring to is the SMO or SQL Server Management object. In order to use it you have to add several references: Microsoft.SQLServer.SMO Microsoft.SQLServer.SMOEnum Microsoft.SQLServer.SQLEnum Microsoft.SQLServer.ConnectionInfo then in the classes using the SMO you have to do using Microsoft.SqlServer.Management.Smo; After that you can do things like: Server Svr; Svr = new Server("MyServerName") foreach (Database in Svr.Databases) { //Etc. // } This allows you to iterate collections of database objects, using them directly or just pulling the names out (as I did) to populate lists, combos, collections etc. As I mentioned, once you have a database object you can manipulate it. I am just starting to learn what I can do with this API but it looks pretty powerful. John W. Colby www.ColbyConsulting.com jwcolby wrote: > The blind leading the blind here. > > 1) I built a main application > 2) I referenced the existing file repair applet from the main application (project). > 3) I set a using statement. It appears that you have to both reference it and then use the "using" > statement. > 4) I can now open forms out in the file repair applet from the main application. > 5) I physically moved the file repair applet underneath the main application directory. > 6) I changed the directory for the applet and it just worked. That was fairly easy. > > From this point on I "Add Project" to the main solution. I have added a class project to wrap the > DMO. In case you haven't discovered it, the DMO is a real cool SQL Server Management Object API > that allows you to see and manage database objects. I am just getting into it but it allows me to > reference a server object, then see the database collection. The each database object has a table > collection, the table object has a fields collection etc. Everything you can see and manage in the > SQL Server management studio you can (apparently) see and manage from the SMO from C#. > > An example of what this does for me is allows me to see all of the databases in a server, and thus > populate a combo with their names. Selecting a database from the combo I can see and fill a combo > with the names of the tables. Selecting a database and a specific table I can then can then run my > stored procedures that export that table in that database to CSV files. > > That kind of stuff is what I do a lot of and what the big application will manage. > > John W. Colby > www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Fri Nov 20 14:18:40 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 20 Nov 2009 23:18:40 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B06B74E.9080909@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> Message-ID: <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> Hi John -- I'm wondering what's the use of that Microsoft.SQLServer.* when you have to have your customer tasks done first of all? Why not just use (static) custom settings to point to different SQL servers etc.? I suppose Microsoft.SQLServer.* is good for companies like http://www.red-gate.com/ for them to develop their tools used worldwide, and I wonder what customers' business tasks can be solved by using Microsoft.SQLServer.* ? Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 20, 2009 6:36 PM To: Discussion concerning Visual Basic and related programming issues. Subject: [dba-VB] SMO was Projects vs Solutions The object I am referring to is the SMO or SQL Server Management object. In order to use it you have to add several references: Microsoft.SQLServer.SMO Microsoft.SQLServer.SMOEnum Microsoft.SQLServer.SQLEnum Microsoft.SQLServer.ConnectionInfo then in the classes using the SMO you have to do using Microsoft.SqlServer.Management.Smo; After that you can do things like: Server Svr; Svr = new Server("MyServerName") foreach (Database in Svr.Databases) { //Etc. // } This allows you to iterate collections of database objects, using them directly or just pulling the names out (as I did) to populate lists, combos, collections etc. As I mentioned, once you have a database object you can manipulate it. I am just starting to learn what I can do with this API but it looks pretty powerful. John W. Colby www.ColbyConsulting.com jwcolby wrote: > The blind leading the blind here. > > 1) I built a main application > 2) I referenced the existing file repair applet from the main application (project). > 3) I set a using statement. It appears that you have to both reference it and then use the "using" > statement. > 4) I can now open forms out in the file repair applet from the main application. > 5) I physically moved the file repair applet underneath the main application directory. > 6) I changed the directory for the applet and it just worked. That was fairly easy. > > From this point on I "Add Project" to the main solution. I have added a class project to wrap the > DMO. In case you haven't discovered it, the DMO is a real cool SQL Server Management Object API > that allows you to see and manage database objects. I am just getting into it but it allows me to > reference a server object, then see the database collection. The each database object has a table > collection, the table object has a fields collection etc. Everything you can see and manage in the > SQL Server management studio you can (apparently) see and manage from the SMO from C#. > > An example of what this does for me is allows me to see all of the databases in a server, and thus > populate a combo with their names. Selecting a database from the combo I can see and fill a combo > with the names of the tables. Selecting a database and a specific table I can then can then run my > stored procedures that export that table in that database to CSV files. > > That kind of stuff is what I do a lot of and what the big application will manage. > > John W. Colby > www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4624 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4625 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 20 14:30:47 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 20 Nov 2009 15:30:47 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> Message-ID: <4B06FC77.6040008@colbyconsulting.com> I don't understand the question. What do you mean by Microsoft.SQLServer.* I do have multiple servers running SQL Server. The default is Azul but I may need to reference Stonehenge. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > I'm wondering what's the use of that Microsoft.SQLServer.* when you have to > have your customer tasks done first of all? > Why not just use (static) custom settings to point to different SQL servers > etc.? > > I suppose Microsoft.SQLServer.* is good for companies like > http://www.red-gate.com/ for them to develop their tools used worldwide, and > I wonder what customers' business tasks can be solved by using > Microsoft.SQLServer.* ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 20, 2009 6:36 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: [dba-VB] SMO was Projects vs Solutions > > The object I am referring to is the SMO or SQL Server Management object. In > order to use it you > have to add several references: > > Microsoft.SQLServer.SMO > Microsoft.SQLServer.SMOEnum > Microsoft.SQLServer.SQLEnum > Microsoft.SQLServer.ConnectionInfo > > then in the classes using the SMO you have to do > > using Microsoft.SqlServer.Management.Smo; > > After that you can do things like: > > Server Svr; > Svr = new Server("MyServerName") > > foreach (Database in Svr.Databases) > { > //Etc. > // > } > > This allows you to iterate collections of database objects, using them > directly or just pulling the > names out (as I did) to populate lists, combos, collections etc. > > As I mentioned, once you have a database object you can manipulate it. I am > just starting to learn > what I can do with this API but it looks pretty powerful. > > John W. Colby > www.ColbyConsulting.com > > > jwcolby wrote: >> The blind leading the blind here. >> >> 1) I built a main application >> 2) I referenced the existing file repair applet from the main application > (project). >> 3) I set a using statement. It appears that you have to both reference it > and then use the "using" >> statement. >> 4) I can now open forms out in the file repair applet from the main > application. >> 5) I physically moved the file repair applet underneath the main > application directory. >> 6) I changed the directory for the applet and it just worked. That was > fairly easy. >> From this point on I "Add Project" to the main solution. I have added a > class project to wrap the >> DMO. In case you haven't discovered it, the DMO is a real cool SQL Server > Management Object API >> that allows you to see and manage database objects. I am just getting > into it but it allows me to >> reference a server object, then see the database collection. The each > database object has a table >> collection, the table object has a fields collection etc. Everything you > can see and manage in the >> SQL Server management studio you can (apparently) see and manage from the > SMO from C#. >> An example of what this does for me is allows me to see all of the > databases in a server, and thus >> populate a combo with their names. Selecting a database from the combo I > can see and fill a combo >> with the names of the tables. Selecting a database and a specific table I > can then can then run my >> stored procedures that export that table in that database to CSV files. >> >> That kind of stuff is what I do a lot of and what the big application will > manage. >> John W. Colby >> www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4624 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4625 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Nov 20 14:50:35 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 20 Nov 2009 23:50:35 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B06FC77.6040008@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> Message-ID: <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> Hi John, By Microsoft.SQLServer.* I mean all the namespaces you mentioned > Microsoft.SQLServer.SMO > Microsoft.SQLServer.SMOEnum > Microsoft.SQLServer.SQLEnum > Microsoft.SQLServer.ConnectionInfo As well as their classes and those classes enumerations, methods, properties, events etc. <<< I do have multiple servers running SQL Server. The default is Azul but I may need to reference Stonehenge. >>> But do you need to change the reference(/connection string information) to SQL Server after you have your program started? Do you need to enumerate SQL Server objects to solve your customers' tasks? Or maybe you have developed a kind of code generator, which does need to enumerate MS SQL Server objects and some of their properties to generate some custom code? - I can understand the latter - if you have to develop a lot of repetitive custom code then it's often useful to generate it... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 20, 2009 11:31 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SMO was Projects vs Solutions I don't understand the question. What do you mean by Microsoft.SQLServer.* I do have multiple servers running SQL Server. The default is Azul but I may need to reference Stonehenge. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > I'm wondering what's the use of that Microsoft.SQLServer.* when you have to > have your customer tasks done first of all? > Why not just use (static) custom settings to point to different SQL servers > etc.? > > I suppose Microsoft.SQLServer.* is good for companies like > http://www.red-gate.com/ for them to develop their tools used worldwide, and > I wonder what customers' business tasks can be solved by using > Microsoft.SQLServer.* ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 20, 2009 6:36 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: [dba-VB] SMO was Projects vs Solutions > > The object I am referring to is the SMO or SQL Server Management object. In > order to use it you > have to add several references: > > Microsoft.SQLServer.SMO > Microsoft.SQLServer.SMOEnum > Microsoft.SQLServer.SQLEnum > Microsoft.SQLServer.ConnectionInfo > > then in the classes using the SMO you have to do > > using Microsoft.SqlServer.Management.Smo; > > After that you can do things like: > > Server Svr; > Svr = new Server("MyServerName") > > foreach (Database in Svr.Databases) > { > //Etc. > // > } > > This allows you to iterate collections of database objects, using them > directly or just pulling the > names out (as I did) to populate lists, combos, collections etc. > > As I mentioned, once you have a database object you can manipulate it. I am > just starting to learn > what I can do with this API but it looks pretty powerful. > > John W. Colby > www.ColbyConsulting.com > > > jwcolby wrote: >> The blind leading the blind here. >> >> 1) I built a main application >> 2) I referenced the existing file repair applet from the main application > (project). >> 3) I set a using statement. It appears that you have to both reference it > and then use the "using" >> statement. >> 4) I can now open forms out in the file repair applet from the main > application. >> 5) I physically moved the file repair applet underneath the main > application directory. >> 6) I changed the directory for the applet and it just worked. That was > fairly easy. >> From this point on I "Add Project" to the main solution. I have added a > class project to wrap the >> DMO. In case you haven't discovered it, the DMO is a real cool SQL Server > Management Object API >> that allows you to see and manage database objects. I am just getting > into it but it allows me to >> reference a server object, then see the database collection. The each > database object has a table >> collection, the table object has a fields collection etc. Everything you > can see and manage in the >> SQL Server management studio you can (apparently) see and manage from the > SMO from C#. >> An example of what this does for me is allows me to see all of the > databases in a server, and thus >> populate a combo with their names. Selecting a database from the combo I > can see and fill a combo >> with the names of the tables. Selecting a database and a specific table I > can then can then run my >> stored procedures that export that table in that database to CSV files. >> >> That kind of stuff is what I do a lot of and what the big application will > manage. >> John W. Colby >> www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4625 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 20 20:05:57 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 20 Nov 2009 21:05:57 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> Message-ID: <4B074B05.6000406@colbyconsulting.com> That is used for manipulating the SMO object and SQL Server. I am just learning this stuff and do what the demos tell me to do. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > By Microsoft.SQLServer.* I mean all the namespaces you mentioned > >> Microsoft.SQLServer.SMO >> Microsoft.SQLServer.SMOEnum >> Microsoft.SQLServer.SQLEnum >> Microsoft.SQLServer.ConnectionInfo > > As well as their classes and those classes enumerations, methods, > properties, events etc. > > <<< > I do have multiple servers running SQL Server. > The default is Azul but I may need to reference > Stonehenge. > But do you need to change the reference(/connection string information) to > SQL Server after you have your program started? > Do you need to enumerate SQL Server objects to solve your customers' tasks? > Or maybe you have developed a kind of code generator, which does need to > enumerate MS SQL Server objects and some of their properties to generate > some custom code? - I can understand the latter - if you have to develop a > lot of repetitive custom code then it's often useful to generate it... > > Thank you. > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 20, 2009 11:31 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > I don't understand the question. > > What do you mean by Microsoft.SQLServer.* > > I do have multiple servers running SQL Server. The default is Azul but I > may need to reference > Stonehenge. > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> I'm wondering what's the use of that Microsoft.SQLServer.* when you have > to >> have your customer tasks done first of all? >> Why not just use (static) custom settings to point to different SQL > servers >> etc.? >> >> I suppose Microsoft.SQLServer.* is good for companies like >> http://www.red-gate.com/ for them to develop their tools used worldwide, > and >> I wonder what customers' business tasks can be solved by using >> Microsoft.SQLServer.* ? >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: Friday, November 20, 2009 6:36 PM >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: [dba-VB] SMO was Projects vs Solutions >> >> The object I am referring to is the SMO or SQL Server Management object. > In >> order to use it you >> have to add several references: >> >> Microsoft.SQLServer.SMO >> Microsoft.SQLServer.SMOEnum >> Microsoft.SQLServer.SQLEnum >> Microsoft.SQLServer.ConnectionInfo >> >> then in the classes using the SMO you have to do >> >> using Microsoft.SqlServer.Management.Smo; >> >> After that you can do things like: >> >> Server Svr; >> Svr = new Server("MyServerName") >> >> foreach (Database in Svr.Databases) >> { >> //Etc. >> // >> } >> >> This allows you to iterate collections of database objects, using them >> directly or just pulling the >> names out (as I did) to populate lists, combos, collections etc. >> >> As I mentioned, once you have a database object you can manipulate it. I > am >> just starting to learn >> what I can do with this API but it looks pretty powerful. >> >> John W. Colby >> www.ColbyConsulting.com >> >> >> jwcolby wrote: >>> The blind leading the blind here. >>> >>> 1) I built a main application >>> 2) I referenced the existing file repair applet from the main application >> (project). >>> 3) I set a using statement. It appears that you have to both reference > it >> and then use the "using" >>> statement. >>> 4) I can now open forms out in the file repair applet from the main >> application. >>> 5) I physically moved the file repair applet underneath the main >> application directory. >>> 6) I changed the directory for the applet and it just worked. That was >> fairly easy. >>> From this point on I "Add Project" to the main solution. I have added a >> class project to wrap the >>> DMO. In case you haven't discovered it, the DMO is a real cool SQL > Server >> Management Object API >>> that allows you to see and manage database objects. I am just getting >> into it but it allows me to >>> reference a server object, then see the database collection. The each >> database object has a table >>> collection, the table object has a fields collection etc. Everything you >> can see and manage in the >>> SQL Server management studio you can (apparently) see and manage from the >> SMO from C#. >>> An example of what this does for me is allows me to see all of the >> databases in a server, and thus >>> populate a combo with their names. Selecting a database from the combo I >> can see and fill a combo >>> with the names of the tables. Selecting a database and a specific table > I >> can then can then run my >>> stored procedures that export that table in that database to CSV files. >>> >>> That kind of stuff is what I do a lot of and what the big application > will >> manage. >>> John W. Colby >>> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4625 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Nov 21 08:58:31 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 21 Nov 2009 17:58:31 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B074B05.6000406@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> Message-ID: <002001ca6abb$186807e0$493817a0$@spb.ru> Hi John -- Yes, I understand that. I'm wondering what for do you learning this stuff? - Just out of curiosity or do you plan to implement some custom applications using Microsoft.SQLServer.* ? Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 21, 2009 5:06 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SMO was Projects vs Solutions That is used for manipulating the SMO object and SQL Server. I am just learning this stuff and do what the demos tell me to do. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > By Microsoft.SQLServer.* I mean all the namespaces you mentioned > >> Microsoft.SQLServer.SMO >> Microsoft.SQLServer.SMOEnum >> Microsoft.SQLServer.SQLEnum >> Microsoft.SQLServer.ConnectionInfo > > As well as their classes and those classes enumerations, methods, > properties, events etc. > > <<< > I do have multiple servers running SQL Server. > The default is Azul but I may need to reference > Stonehenge. > But do you need to change the reference(/connection string information) to > SQL Server after you have your program started? > Do you need to enumerate SQL Server objects to solve your customers' tasks? > Or maybe you have developed a kind of code generator, which does need to > enumerate MS SQL Server objects and some of their properties to generate > some custom code? - I can understand the latter - if you have to develop a > lot of repetitive custom code then it's often useful to generate it... > > Thank you. > > -- > Shamil > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4626 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 21 09:42:12 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 10:42:12 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <002001ca6abb$186807e0$493817a0$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> Message-ID: <4B080A54.4030609@colbyconsulting.com> > or do you plan to implement some custom applications using Microsoft.SQLServer.* ? Exactly. My work for one of my clients is heavy manipulation of SQL Server. I routinely export files to an external application running on virtual machines on another server, then wait for the files to process, then import them back in to SQL server. These two processes together take about 10 stored procedures. The export / import has to occur on (currently) a half dozen large databases monthly. However as I obtain more lists that number goes up. Each process exports to it's own directory path, then imports back from a parallel directory path. While the stored procedures eliminate much of the manual labor, there is plenty left. Plus logging of everything etc. I also routinely process orders. These orders take about a dozen stored procedures and also use the exact same set of stored procedures as the above process, i.e. the export / import process is also used on each and every order. So there is just a ton of very repetitive work that has to be done, but which database / table varies from run to run. To this point I have used Access / VBA to automate these tasks. Now I am moving to C# for this automation work. SMO allows me to manipulate SQL Server from C# and so it is an API that I will use to assist me in getting this program working. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I understand that. > I'm wondering what for do you learning this stuff? - Just out of curiosity > or do you plan to implement some custom applications using > Microsoft.SQLServer.* ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 21, 2009 5:06 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > That is used for manipulating the SMO object and SQL Server. > > I am just learning this stuff and do what the demos tell me to do. > > John W. Colby > www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sat Nov 21 10:27:15 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 21 Nov 2009 19:27:15 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B080A54.4030609@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> Message-ID: <002601ca6ac7$7dcfb180$796f1480$@spb.ru> Hi John -- OK, but why not just use connections strings and System.Data.SqlClient classes, .... ? Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 21, 2009 6:42 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SMO was Projects vs Solutions > or do you plan to implement some custom applications using Microsoft.SQLServer.* ? Exactly. My work for one of my clients is heavy manipulation of SQL Server. I routinely export files to an external application running on virtual machines on another server, then wait for the files to process, then import them back in to SQL server. These two processes together take about 10 stored procedures. The export / import has to occur on (currently) a half dozen large databases monthly. However as I obtain more lists that number goes up. Each process exports to it's own directory path, then imports back from a parallel directory path. While the stored procedures eliminate much of the manual labor, there is plenty left. Plus logging of everything etc. I also routinely process orders. These orders take about a dozen stored procedures and also use the exact same set of stored procedures as the above process, i.e. the export / import process is also used on each and every order. So there is just a ton of very repetitive work that has to be done, but which database / table varies from run to run. To this point I have used Access / VBA to automate these tasks. Now I am moving to C# for this automation work. SMO allows me to manipulate SQL Server from C# and so it is an API that I will use to assist me in getting this program working. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I understand that. > I'm wondering what for do you learning this stuff? - Just out of curiosity > or do you plan to implement some custom applications using > Microsoft.SQLServer.* ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 21, 2009 5:06 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > That is used for manipulating the SMO object and SQL Server. > > I am just learning this stuff and do what the demos tell me to do. > > John W. Colby > www.ColbyConsulting.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4626 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 21 11:05:43 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 12:05:43 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <002601ca6ac7$7dcfb180$796f1480$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> <002601ca6ac7$7dcfb180$796f1480$@spb.ru> Message-ID: <4B081DE7.7060203@colbyconsulting.com> Shamil, As far as I can tell they are quite different things, and perform different jobs. I do use SQLClient. SQLClient appears to be about getting at data, and I do use that for the ADO side of things. http://msdn.microsoft.com/en-us/library/system.data.sqlclient.aspx SMO appears to be about getting at and manipulating the objects in the database. http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.aspx SMO appears to be similar to the DAO object where you can see and manipulate the table (not the data IN the table) the fields, indexes and so forth. It is a collection based object where you can drill down into a server and see the objects underneath the database, then (for example) drill down into a table and see the objects under the table. Just as an example, using SMO in a few lines of code I got a list of all of the databases in the database collection of my server. I used that to populate a database combo box. When I select the database that I want to export data from, I then use SMO (again a couple of more lines of code) to get a list of the tables in the selected database and use that to populate a combo of tables. I have no idea the relative merit of one way vs the other. To me it is just a tool. In this particular case I use it to quickly and easily get at the names of objects in the database. I understand that using SMO you can do other maintenance kinds of things as well. One of the things I have to do is copy an "order template" database to a new name in preparing to fill an order. SMO appears to have built-in methods for doing this programmatically from C# - as opposed to running TSQL or using some other method. Understand I am not an expert on any of this, in fact quite the opposite. I stumbled across SMO and saw an example of how easy it was to get at the database STRUCTURE information and decided to use it for that purpose, when I had that need. Is there another way to do this? Almost certainly, given that there is always a dozen ways to do anything in programming. Given your persistent questioning of my motives I guess now I have to ask whether I am causing myself problems doing this? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > OK, but why not just use connections strings and System.Data.SqlClient > classes, .... ? > > Thank you. From jwcolby at colbyconsulting.com Sat Nov 21 11:40:33 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 12:40:33 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <002601ca6ac7$7dcfb180$796f1480$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> <002601ca6ac7$7dcfb180$796f1480$@spb.ru> Message-ID: <4B082611.4040401@colbyconsulting.com> Shamil, By get an instance of the Server object you can see a collection of Database objects. The following is what you can do with the database object. So I can manipulate a given database as shown below. Several of those things look potentially useful, such as running TSQL statements, truncating log files and so forth. http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.database.aspx However the database object has a tables collection. I can get lists of the names of tables, select a specific table etc. http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.table.aspx http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.table_methods.aspx The point is that it is just a programming model that is available if you want to learn it, for manipulating a database from C#. I will almost certainly never use 99.9% of it, just as most people (myself included) rarely use 99.9% of the DAO object model. OTOH the .1% that I do use is pretty darned cool. You can find a lot of little articles about using SMO out on Google. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > OK, but why not just use connections strings and System.Data.SqlClient > classes, .... ? > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 21, 2009 6:42 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > > or do you plan to implement some custom applications using > Microsoft.SQLServer.* ? > > Exactly. My work for one of my clients is heavy manipulation of SQL Server. > I routinely export > files to an external application running on virtual machines on another > server, then wait for the > files to process, then import them back in to SQL server. These two > processes together take about > 10 stored procedures. The export / import has to occur on (currently) a > half dozen large databases > monthly. However as I obtain more lists that number goes up. Each process > exports to it's own > directory path, then imports back from a parallel directory path. While the > stored procedures > eliminate much of the manual labor, there is plenty left. Plus logging of > everything etc. > > I also routinely process orders. These orders take about a dozen stored > procedures and also use the > exact same set of stored procedures as the above process, i.e. the export / > import process is also > used on each and every order. > > So there is just a ton of very repetitive work that has to be done, but > which database / table > varies from run to run. To this point I have used Access / VBA to automate > these tasks. Now I am > moving to C# for this automation work. > > SMO allows me to manipulate SQL Server from C# and so it is an API that I > will use to assist me in > getting this program working. > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> Yes, I understand that. >> I'm wondering what for do you learning this stuff? - Just out of curiosity >> or do you plan to implement some custom applications using >> Microsoft.SQLServer.* ? >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: Saturday, November 21, 2009 5:06 AM >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] SMO was Projects vs Solutions >> >> That is used for manipulating the SMO object and SQL Server. >> >> I am just learning this stuff and do what the demos tell me to do. >> >> John W. Colby >> www.ColbyConsulting.com > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4626 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Nov 21 11:47:27 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 12:47:27 -0500 Subject: [dba-VB] Loss leader (or just plain loss) Message-ID: <4B0827AF.6040608@colbyconsulting.com> A friend has approached me about building an application for a small chain of stores to manage inventory. "Very simple". I haven't yet talked to her about the requirements. She did say they were quoted (and rejected) a bid of $4000 to do the job. If I am going to do this at all it will have to be very quick and dirty, and would be just to help a friend. Is Access 2007 runtime stable? Easy or difficult to use / install / implement on site? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sat Nov 21 11:55:29 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 21 Nov 2009 20:55:29 +0300 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <4B081DE7.7060203@colbyconsulting.com> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> <002601ca6ac7$7dcfb180$796f1480$@spb.ru> <4B081DE7.7060203@colbyconsulting.com> Message-ID: <002c01ca6ad3$d1e37930$75aa6b90$@spb.ru> Hi John -- <<< SMO appears to be similar to the DAO object where you can see and manipulate the table... >>> Yes, I understand/know that. <<< When I select the database that I want to export data from, I then use SMO (again a couple of more lines of code) to get a list of the tables in the selected database and use that to populate a combo of tables. >>> Yes, but how often would you need to do that? Why not just use (relatively static) program settings files, or utility program execution command string parameters - I mean you anyway have to do manual selection (in the case you'll use SMO) or manual editing and saving (in the case you'll use program setting files...) - what for to spend time on learning SMO if you very probably (am I wrong?) can solve your customer tasks without using SMO, and applying relatively the same efforts in both cases to implement custom logic (but in the case of using SMO you'll also have to spend time on learning SMO)... <<< Given your persistent questioning of my motives I guess now I have to ask whether I am causing myself problems doing this? >>> Just maybe by spending time on learning SMO when you don't need it to implement required custom logic? Not sure (I didn't check so I can be wrong) but SMO might need special installation procedures for customers' PCs without full MS SQL Version (standard, prof., enterprise) so if you'll develop some custom code to be reused on customers' PC then you might need to be prepared to develop custom setup - not a big issue just be prepared - if I'm not wrong in guessing that that custom setup would be needed for some customers system... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 21, 2009 8:06 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SMO was Projects vs Solutions Shamil, As far as I can tell they are quite different things, and perform different jobs. I do use SQLClient. SQLClient appears to be about getting at data, and I do use that for the ADO side of things. http://msdn.microsoft.com/en-us/library/system.data.sqlclient.aspx SMO appears to be about getting at and manipulating the objects in the database. http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.a spx SMO appears to be similar to the DAO object where you can see and manipulate the table (not the data IN the table) the fields, indexes and so forth. It is a collection based object where you can drill down into a server and see the objects underneath the database, then (for example) drill down into a table and see the objects under the table. Just as an example, using SMO in a few lines of code I got a list of all of the databases in the database collection of my server. I used that to populate a database combo box. When I select the database that I want to export data from, I then use SMO (again a couple of more lines of code) to get a list of the tables in the selected database and use that to populate a combo of tables. I have no idea the relative merit of one way vs the other. To me it is just a tool. In this particular case I use it to quickly and easily get at the names of objects in the database. I understand that using SMO you can do other maintenance kinds of things as well. One of the things I have to do is copy an "order template" database to a new name in preparing to fill an order. SMO appears to have built-in methods for doing this programmatically from C# - as opposed to running TSQL or using some other method. Understand I am not an expert on any of this, in fact quite the opposite. I stumbled across SMO and saw an example of how easy it was to get at the database STRUCTURE information and decided to use it for that purpose, when I had that need. Is there another way to do this? Almost certainly, given that there is always a dozen ways to do anything in programming. Given your persistent questioning of my motives I guess now I have to ask whether I am causing myself problems doing this? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > OK, but why not just use connections strings and System.Data.SqlClient > classes, .... ? > > Thank you. __________ Information from ESET NOD32 Antivirus, version of virus signature database 4626 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From dbdoug at gmail.com Sat Nov 21 12:26:06 2009 From: dbdoug at gmail.com (Doug Steele) Date: Sat, 21 Nov 2009 10:26:06 -0800 Subject: [dba-VB] Loss leader (or just plain loss) In-Reply-To: <4B0827AF.6040608@colbyconsulting.com> References: <4B0827AF.6040608@colbyconsulting.com> Message-ID: <4dd71a0c0911211026y79e4d583t99d6ca1dfb4a10f1@mail.gmail.com> Hi John: I've only ever used the Access 2003 runtime/installer, and that was for one job. Getting the runtime code installed, running, and generating/testing a proper installation package took the best part of a day. It was straightforward enough, just fiddly. After that, updates only consisted of recreating the .mde file and sending it to the client. From what I've read, the Access 2007 runtime is pretty stable now (it started off very badly) and it's the right price. My experience with projects that started out to be 'quick and dirty' has been consistently horrible. Not quick, and very dirty! Doug On Sat, Nov 21, 2009 at 9:47 AM, jwcolby wrote: > A friend has approached me about building an application for a small chain > of stores to manage > inventory. "Very simple". I haven't yet talked to her about the > requirements. She did say they > were quoted (and rejected) a bid of $4000 to do the job. > > If I am going to do this at all it will have to be very quick and dirty, > and would be just to help a > friend. > > Is Access 2007 runtime stable? Easy or difficult to use / install / > implement on site? > > > From shamil at smsconsulting.spb.ru Sat Nov 21 12:40:36 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 21 Nov 2009 21:40:36 +0300 Subject: [dba-VB] Loss leader (or just plain loss) In-Reply-To: <4dd71a0c0911211026y79e4d583t99d6ca1dfb4a10f1@mail.gmail.com> References: <4B0827AF.6040608@colbyconsulting.com> <4dd71a0c0911211026y79e4d583t99d6ca1dfb4a10f1@mail.gmail.com> Message-ID: <002d01ca6ada$1ecb5fa0$5c621ee0$@spb.ru> <<< Not quick, and very dirty! >>> Yes. And a high opportunity to lose a friend... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Doug Steele Sent: Saturday, November 21, 2009 9:26 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Loss leader (or just plain loss) Hi John: I've only ever used the Access 2003 runtime/installer, and that was for one job. Getting the runtime code installed, running, and generating/testing a proper installation package took the best part of a day. It was straightforward enough, just fiddly. After that, updates only consisted of recreating the .mde file and sending it to the client. From what I've read, the Access 2007 runtime is pretty stable now (it started off very badly) and it's the right price. My experience with projects that started out to be 'quick and dirty' has been consistently horrible. Not quick, and very dirty! Doug On Sat, Nov 21, 2009 at 9:47 AM, jwcolby wrote: > A friend has approached me about building an application for a small chain > of stores to manage > inventory. "Very simple". I haven't yet talked to her about the > requirements. She did say they > were quoted (and rejected) a bid of $4000 to do the job. > > If I am going to do this at all it will have to be very quick and dirty, > and would be just to help a > friend. > > Is Access 2007 runtime stable? Easy or difficult to use / install / > implement on site? > > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4626 (20091120) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 21 13:30:39 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 21 Nov 2009 14:30:39 -0500 Subject: [dba-VB] SMO was Projects vs Solutions In-Reply-To: <002c01ca6ad3$d1e37930$75aa6b90$@spb.ru> References: <4B0543AB.5080700@colbyconsulting.com> <4B068AE2.9060301@colbyconsulting.com> <4B06B74E.9080909@colbyconsulting.com> <005d01ca6a1e$a7e709e0$f7b51da0$@spb.ru> <4B06FC77.6040008@colbyconsulting.com> <005f01ca6a23$1d0ecfb0$572c6f10$@spb.ru> <4B074B05.6000406@colbyconsulting.com> <002001ca6abb$186807e0$493817a0$@spb.ru> <4B080A54.4030609@colbyconsulting.com> <002601ca6ac7$7dcfb180$796f1480$@spb.ru> <4B081DE7.7060203@colbyconsulting.com> <002c01ca6ad3$d1e37930$75aa6b90$@spb.ru> Message-ID: <4B083FDF.505@colbyconsulting.com> There is no "customer PC". This is my server in my office, running an application I will write, which I will operate. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > <<< > SMO appears to be similar to the DAO object > where you can see and manipulate the table... > Yes, I understand/know that. > > <<< > When I select the > database that I want to export data from, I then > use SMO (again a couple of more lines of code) to > get a list of the tables in the selected database > and use that to populate a combo of tables. > Yes, but how often would you need to do that? > Why not just use (relatively static) program settings files, or utility > program execution command string parameters - I mean you anyway have to do > manual selection (in the case you'll use SMO) or manual editing and saving > (in the case you'll use program setting files...) - what for to spend time > on learning SMO if you very probably (am I wrong?) can solve your customer > tasks without using SMO, and applying relatively the same efforts in both > cases to implement custom logic (but in the case of using SMO you'll also > have to spend time on learning SMO)... > > <<< > Given your persistent questioning of my motives > I guess now I have to ask whether I am causing > myself problems doing this? > Just maybe by spending time on learning SMO when you don't need it to > implement required custom logic? > > Not sure (I didn't check so I can be wrong) but SMO might need special > installation procedures for customers' PCs without full MS SQL Version > (standard, prof., enterprise) so if you'll develop some custom code to be > reused on customers' PC then you might need to be prepared to develop custom > setup - not a big issue just be prepared - if I'm not wrong in guessing that > that custom setup would be needed for some customers system... > > Thank you. > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Saturday, November 21, 2009 8:06 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] SMO was Projects vs Solutions > > Shamil, > > As far as I can tell they are quite different things, and perform different > jobs. I do use > SQLClient. SQLClient appears to be about getting at data, and I do use that > for the ADO side of > things. > > http://msdn.microsoft.com/en-us/library/system.data.sqlclient.aspx > > SMO appears to be about getting at and manipulating the objects in the > database. > > http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.a > spx > > SMO appears to be similar to the DAO object where you can see and manipulate > the table (not the data > IN the table) the fields, indexes and so forth. It is a collection based > object where you can drill > down into a server and see the objects underneath the database, then (for > example) drill down into a > table and see the objects under the table. > > Just as an example, using SMO in a few lines of code I got a list of all of > the databases in the > database collection of my server. I used that to populate a database combo > box. When I select the > database that I want to export data from, I then use SMO (again a couple of > more lines of code) to > get a list of the tables in the selected database and use that to populate a > combo of tables. > > I have no idea the relative merit of one way vs the other. To me it is just > a tool. In this > particular case I use it to quickly and easily get at the names of objects > in the database. I > understand that using SMO you can do other maintenance kinds of things as > well. One of the things I > have to do is copy an "order template" database to a new name in preparing > to fill an order. SMO > appears to have built-in methods for doing this programmatically from C# - > as opposed to running > TSQL or using some other method. > > Understand I am not an expert on any of this, in fact quite the opposite. I > stumbled across SMO and > saw an example of how easy it was to get at the database STRUCTURE > information and decided to use it > for that purpose, when I had that need. Is there another way to do this? > Almost certainly, given > that there is always a dozen ways to do anything in programming. > > Given your persistent questioning of my motives I guess now I have to ask > whether I am causing > myself problems doing this? > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> Hi John -- >> >> OK, but why not just use connections strings and System.Data.SqlClient >> classes, .... ? >> >> Thank you. > > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4626 (20091120) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sun Nov 22 22:46:38 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 22 Nov 2009 23:46:38 -0500 Subject: [dba-VB] Ya know you're in trouble when... Message-ID: <4B0A13AE.1030404@colbyconsulting.com> http://www.hanselman.com/blog/WhatGreatNETDevelopersOughtToKnowMoreNETInterviewQuestions.aspx -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Mon Nov 23 07:57:41 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 23 Nov 2009 08:57:41 -0500 Subject: [dba-VB] XNA game development Message-ID: <4B0A94D5.3010601@colbyconsulting.com> Have you ever wanted to write your own little windows game. Microsoft provides a free game development framework called XNA. It runs in C# express and everything is free free free. It is designed to get you writing games for the XBox BUT... it can write games for Windows as well, and in fact they are apparently interchangeable, once written they can be played on the XBox (but not for free). The best part is that they have a very basic cannon / flying saucer demo game with videos for how to develop this game, all of the source code, sprites etc. I have stepped through the first few chapters and it really does start at ground zero. They claim you don't even need to know C# to follow the tutorial. As we all know, writing a project that holds your interest is the best way to get involved and learn a language. If writing a game is your interest, this might be just the ticket to getting into C#. http://creators.xna.com/en-US/education/gettingstarted/bg2d/chapter1 -- John W. Colby www.ColbyConsulting.com From mmattys at rochester.rr.com Mon Nov 23 09:41:38 2009 From: mmattys at rochester.rr.com (Mike Mattys) Date: Mon, 23 Nov 2009 10:41:38 -0500 Subject: [dba-VB] Ya know you're in trouble when... References: <4B0A13AE.1030404@colbyconsulting.com> Message-ID: I've already come up against a great many of these Some of them I look forward to getting the answers for. If I went to an interview for all of it, I would hope it to be "open book" - Michael R Mattys MapPoint and Database Dev www.mattysconsulting.com - ----- Original Message ----- From: "jwcolby" To: "VBA" Sent: Sunday, November 22, 2009 11:46 PM Subject: [dba-VB] Ya know you're in trouble when... > > http://www.hanselman.com/blog/WhatGreatNETDevelopersOughtToKnowMoreNETInterviewQuestions.aspx > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > From dwaters at usinternet.com Mon Nov 23 12:22:19 2009 From: dwaters at usinternet.com (Dan Waters) Date: Mon, 23 Nov 2009 12:22:19 -0600 Subject: [dba-VB] Ya know you're in trouble when... In-Reply-To: References: <4B0A13AE.1030404@colbyconsulting.com> Message-ID: Hey! I knew one of these! What's the difference between late binding and early binding? But that's it. :-( Dan -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Mike Mattys Sent: Monday, November 23, 2009 9:42 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Ya know you're in trouble when... I've already come up against a great many of these Some of them I look forward to getting the answers for. If I went to an interview for all of it, I would hope it to be "open book" - Michael R Mattys MapPoint and Database Dev www.mattysconsulting.com - ----- Original Message ----- From: "jwcolby" To: "VBA" Sent: Sunday, November 22, 2009 11:46 PM Subject: [dba-VB] Ya know you're in trouble when... > > http://www.hanselman.com/blog/WhatGreatNETDevelopersOughtToKnowMoreNETInterv iewQuestions.aspx > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Mon Nov 23 15:57:12 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 23 Nov 2009 16:57:12 -0500 Subject: [dba-VB] XNA is darned fun Message-ID: <4B0B0538.1020904@colbyconsulting.com> I have to say that this is darned fun. I have worked through much of the tutorial I posted a link to. I have ships flying and cannonballs firing. Woohoo! Very very cool! I know what I will be doing over the Thanksgiving holiday. -- John W. Colby www.ColbyConsulting.com From michael at ddisolutions.com.au Mon Nov 23 23:38:16 2009 From: michael at ddisolutions.com.au (Michael Maddison) Date: Tue, 24 Nov 2009 16:38:16 +1100 Subject: [dba-VB] HTTPWebRequest blues Message-ID: <59A61174B1F5B54B97FD4ADDE71E7D01582726@ddi-01.DDI.local> Hi guys, I may be clutching at straws here but has anyone done much with HttpWebRequest? My process goes like this, GET /ExpressBet/Standard/Default.aspx?State=2 Store the cookie container for re use GET /ExpressBet/LoginFrame.aspx?State=2&ExpressBetMode=Standard Store viewState for re use in next POST POST /Login/LoginUser.aspx?State=2&ReturnURL=http://www.tab.com.au/ExpressBet/LoginFrame.aspx?State=2&ExpressBetMode=Standard Does login, this succeeds GET /ExpressBet/Aggregator.aspx?State=2&Controls=STD_MAIN&RacingCode=R&MeetingCode=M&FromDate=2009-11-24T00:00:00&RaceNumber=07&BetType=WinPlace&ExpressBetMode=Standard Store viewState for re use GET /Betting/BetTicket/DisplayClientSideBetTicket.aspx?State=2&ExpressBet=true&RacingCode=R&MeetingCode=M&FromDate=2009-11-24T00:00:00&BetType=WinPlace&amount=1.00&Selections=%3csc+id%3d%220%22+f%3d%22false%22+m%3d%220%22+s%3d%228%22+t%3d%22%22+k%3d%2207%22+%2f%3e+ The above redirects to GET /Betting/BetTicket/PlaceRaceBetConfirmation.aspx?State=2&CurrentBet=2901d195-c59a-4c1b-8b0a-c476fc274f06&ExpressBet=True Result looks ok in Fiddler, But response is unreadable in plain text. Anyone know what this is? I think I need to get the viewstate from it somehow. Snipped because of size constraints. _???2??-w?sF??'15??1??_i???i??_l?___???_?__?S??fY?d?R_adZ?Ri??__ 2?#??4+?='+??N!??_n^???????nQ^?`????_?_____?0???_?h??X????_???dvTH??`??[?k?Sh?SDp??[h?c?OH???k?K?{??3??@?'13OW7????0W??8,????r2????_=}A???{?@??? i???)?5__F_?[?S?'4N?__C]}?M?}?Af???s?H??K?-H?? ?u+N ?s?r,? The Last step I need to do is a final POST, This returns an error page, dammit. POST /Betting/BetTicket/PlaceRaceBetConfirmation.aspx?State=2&CurrentBet=2901d195-c59a-4c1b-8b0a-c476fc274f06&ExpressBet=True Redirects to an error.aspx page. The code for the last POST is the same for the successful calls as the failed call. Both are HTTPS, I am handling the certificates. Does this make sense to anyone? J Any ideas? Cheers Michael M From Gustav at cactus.dk Tue Nov 24 04:50:40 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Tue, 24 Nov 2009 11:50:40 +0100 Subject: [dba-VB] This is hot: WCF RIA Services Message-ID: Hi all RIA: Rich Internet Applications Microsoft WCF RIA Services simplifies the traditional n-tier application pattern by bringing together the ASP.NET and Silverlight platforms. RIA Services provides a pattern to write application logic that runs on the mid-tier and controls access to data for queries, changes and custom operations. It also provides end-to-end support for common tasks such as data validation, authentication and roles by integrating with Silverlight components on the client and ASP.NET on the mid-tier Though not explained at a place to find, what it does is automatically to set up for you a web service which your Silverlight frontend communicates with, and to establish the client side code that handles this communication. http://silverlight.net/getstarted/riaservices/ Watch the 1-hour video "Introduction to WCF RIA Services" and combine this with the racing speed at which Silverlight evolves, and it will erase any doubt you still may have about browser-based applications. Note too, that while this works in VS2008 and with Silverlight 3 it is really aimed at VS2010 and Silverlight 4 both currently available as betas. I attended a developer meeting a couple of days ago where we ran a similar walk-through, and it was very convincing. This is exactly what I for years have been looking for. /gustav From fuller.artful at gmail.com Tue Nov 24 10:26:24 2009 From: fuller.artful at gmail.com (Arthur Fuller) Date: Tue, 24 Nov 2009 11:26:24 -0500 Subject: [dba-VB] XNA is darned fun In-Reply-To: <4B0B0538.1020904@colbyconsulting.com> References: <4B0B0538.1020904@colbyconsulting.com> Message-ID: <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> When is American Thanksgiving? We already had ours last month. What is XNA, anyway? Arthur On Mon, Nov 23, 2009 at 4:57 PM, jwcolby wrote: > I have to say that this is darned fun. I have worked through much of the > tutorial I posted a link > to. I have ships flying and cannonballs firing. Woohoo! > > Very very cool! I know what I will be doing over the Thanksgiving holiday. > From cfoust at infostatsystems.com Tue Nov 24 10:48:12 2009 From: cfoust at infostatsystems.com (Charlotte Foust) Date: Tue, 24 Nov 2009 10:48:12 -0600 Subject: [dba-VB] XNA is darned fun In-Reply-To: <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> References: <4B0B0538.1020904@colbyconsulting.com> <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> Message-ID: Ours is Thursday. Charlotte -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Arthur Fuller Sent: Tuesday, November 24, 2009 8:26 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] XNA is darned fun When is American Thanksgiving? We already had ours last month. What is XNA, anyway? Arthur On Mon, Nov 23, 2009 at 4:57 PM, jwcolby wrote: > I have to say that this is darned fun. I have worked through much of > the tutorial I posted a link to. I have ships flying and cannonballs > firing. Woohoo! > > Very very cool! I know what I will be doing over the Thanksgiving holiday. > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Tue Nov 24 10:53:21 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Tue, 24 Nov 2009 11:53:21 -0500 Subject: [dba-VB] XNA is darned fun In-Reply-To: <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> References: <4B0B0538.1020904@colbyconsulting.com> <29f585dd0911240826k125166ddu413e2e04f409096@mail.gmail.com> Message-ID: <4B0C0F81.9070806@colbyconsulting.com> > When is American Thanksgiving? We already had ours last month. Not sure exactly. IIRC it is the last Thursday in November so it is not a fixed date. > What is XNA, anyway? It is a Microsoft gaming framework. VERY COOL if you like to (or always wanted to) write a windows game. It runs under C# in Visual Studio, even the express version. The framework provides a game class that has a method for loading the "content" - video, sound, sprites etc. It has a method automatically called X times / second that is then used to perform updates and display the content. And it has a draw method that is called to draw the game. LOTS of game specific classes including math classes, 2d and 3d classes for doing game specific kinds of things, sprite classes, point classes, rectangle classes. You get the picture. REALLY really cool for the child in all of us. If you happen to have an XBox, you can write games that can run on the XBox as well as Windows (XP, Vista and Windows 7). I don't own an XBox so I am just running the game on Windows, but any game CAN be written to run on either, and apparently SENSE which it is running on. You can even use the XBox joystick / button controller on your Windows machine! There is a step by step video series that walks you through building a "space invaders" game, with cannon, cannon balls, UFOs. All graphics provided, all code stepped through in short videos. I did that and I have to say it is really easy. It was so cool as each step comes together, displaying the background, displaying the flying saucers moving across the screen, the cannon swiveling, then cannon balls firing, and finally cannon balls destroying the UFOs. Of course doing it all by myself would be not so easy but it gives a great feeling for what can be done and how it is done. And of course there are after market books on it. I have always wanted to write my own "Empire" replacement, and have even started doing so. I played Empire all of my life (still do) but it is long in the tooth and not updated any more. Plus the multi-player stuff never really worked very well. This will set me back to ground zero on that attempt but it will also provide a framework for doing it the right way. I just don't have enough to do with my time, as I am sure you can understand! John W. Colby www.ColbyConsulting.com Arthur Fuller wrote: > When is American Thanksgiving? We already had ours last month. > > What is XNA, anyway? > > Arthur > > On Mon, Nov 23, 2009 at 4:57 PM, jwcolby wrote: > >> I have to say that this is darned fun. I have worked through much of the >> tutorial I posted a link >> to. I have ships flying and cannonballs firing. Woohoo! >> >> Very very cool! I know what I will be doing over the Thanksgiving holiday. >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Wed Nov 25 08:24:39 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 09:24:39 -0500 Subject: [dba-VB] Part 1: An Introduction to Data Structures Message-ID: <4B0D3E27.9040601@colbyconsulting.com> A quick overview: http://creators.xna.com/en-US/article/datastructures A more detailed overview: http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Wed Nov 25 08:51:50 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Wed, 25 Nov 2009 17:51:50 +0300 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: <4B0D3E27.9040601@colbyconsulting.com> References: <4B0D3E27.9040601@colbyconsulting.com> Message-ID: <006e01ca6dde$d31de700$7959b500$@spb.ru> Hi John -- You might also find interesting: http://en.wikipedia.org/wiki/Microsoft_Robotics_Developer_Studio e.g., for programming: http://en.wikipedia.org/wiki/Lego_Mindstorms_NXT using C#... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, November 25, 2009 5:25 PM To: VBA; Nikolai Vitsyn; Judy Hawley Subject: [dba-VB] Part 1: An Introduction to Data Structures A quick overview: http://creators.xna.com/en-US/article/datastructures A more detailed overview: http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4636 (20091125) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From Gustav at cactus.dk Wed Nov 25 08:54:57 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 25 Nov 2009 15:54:57 +0100 Subject: [dba-VB] Part 1: An Introduction to Data Structures Message-ID: Hi John Thanks. "collision code" ... I love that. /gustav >>> jwcolby at colbyconsulting.com 25-11-2009 15:24 >>> A quick overview: http://creators.xna.com/en-US/article/datastructures A more detailed overview: http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Wed Nov 25 09:23:22 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 10:23:22 -0500 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: References: Message-ID: <4B0D4BEA.2050603@colbyconsulting.com> > Thanks. "collision code" ... I love that. LOL. Obviously this was aimed at gamers. A collision is when one rectangle (sprite) intersects another rectangle. BTW, check out this video of a game. http://www.youtube.com/watch?v=Lz-TQcdXDJU&feature=related LOTS of collisions going on here. ;) I am going to work on writing this game using the XNA gaming framework. The XNA framework is just awesome, and I am going to get my 8 year old son involved in helping me to write this thing. A fun, interactive way of teaching him C#. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > Thanks. "collision code" ... I love that. > > /gustav > > >>>> jwcolby at colbyconsulting.com 25-11-2009 15:24 >>> > A quick overview: > http://creators.xna.com/en-US/article/datastructures > > A more detailed overview: > http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx From jwcolby at colbyconsulting.com Wed Nov 25 09:24:15 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 10:24:15 -0500 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: <006e01ca6dde$d31de700$7959b500$@spb.ru> References: <4B0D3E27.9040601@colbyconsulting.com> <006e01ca6dde$d31de700$7959b500$@spb.ru> Message-ID: <4B0D4C1F.9050103@colbyconsulting.com> I have this thing. I haven't looked at it yet though. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > You might also find interesting: > > http://en.wikipedia.org/wiki/Microsoft_Robotics_Developer_Studio > > e.g., for programming: > > http://en.wikipedia.org/wiki/Lego_Mindstorms_NXT > > using C#... > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, November 25, 2009 5:25 PM > To: VBA; Nikolai Vitsyn; Judy Hawley > Subject: [dba-VB] Part 1: An Introduction to Data Structures > > A quick overview: > http://creators.xna.com/en-US/article/datastructures > > A more detailed overview: > http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx From stuart at lexacorp.com.pg Wed Nov 25 15:31:48 2009 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Thu, 26 Nov 2009 07:31:48 +1000 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: <4B0D4BEA.2050603@colbyconsulting.com> References: , <4B0D4BEA.2050603@colbyconsulting.com> Message-ID: <4B0DA244.1838.3178D240@stuart.lexacorp.com.pg> Gee you guys are taking me back - I used to write games for the Amiga twenty odd years ago - It all revolved around sprites and collisions. -- Stuart On 25 Nov 2009 at 10:23, jwcolby wrote: > > Thanks. "collision code" ... I love that. > > LOL. Obviously this was aimed at gamers. A collision is when one rectangle (sprite) intersects > another rectangle. > > BTW, check out this video of a game. > > http://www.youtube.com/watch?v=Lz-TQcdXDJU&feature=related > > LOTS of collisions going on here. ;) > > I am going to work on writing this game using the XNA gaming framework. The XNA framework is just > awesome, and I am going to get my 8 year old son involved in helping me to write this thing. A fun, > interactive way of teaching him C#. > > John W. Colby > www.ColbyConsulting.com > > > Gustav Brock wrote: > > Hi John > > > > Thanks. "collision code" ... I love that. > > > > /gustav > > > > > >>>> jwcolby at colbyconsulting.com 25-11-2009 15:24 >>> > > A quick overview: > > http://creators.xna.com/en-US/article/datastructures > > > > A more detailed overview: > > http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > From Gustav at cactus.dk Wed Nov 25 15:50:53 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Wed, 25 Nov 2009 22:50:53 +0100 Subject: [dba-VB] Part 1: An Introduction to Data Structures Message-ID: Hi John Oh my, will this never stop? I guess that is what this poor guy thinks, in real trouble as he is ... /gustav >>> jwcolby at colbyconsulting.com 25-11-2009 16:23 >>> > Thanks. "collision code" ... I love that. LOL. Obviously this was aimed at gamers. A collision is when one rectangle (sprite) intersects another rectangle. BTW, check out this video of a game. http://www.youtube.com/watch?v=Lz-TQcdXDJU&feature=related LOTS of collisions going on here. ;) I am going to work on writing this game using the XNA gaming framework. The XNA framework is just awesome, and I am going to get my 8 year old son involved in helping me to write this thing. A fun, interactive way of teaching him C#. John W. Colby www.ColbyConsulting.com Gustav Brock wrote: > Hi John > > Thanks. "collision code" ... I love that. > > /gustav > > >>>> jwcolby at colbyconsulting.com 25-11-2009 15:24 >>> > A quick overview: > http://creators.xna.com/en-US/article/datastructures > > A more detailed overview: > http://msdn.microsoft.com/en-us/library/ms379570(VS.80).aspx From jwcolby at colbyconsulting.com Wed Nov 25 15:52:30 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 16:52:30 -0500 Subject: [dba-VB] Part 1: An Introduction to Data Structures In-Reply-To: <4B0DA244.1838.3178D240@stuart.lexacorp.com.pg> References: , <4B0D4BEA.2050603@colbyconsulting.com> <4B0DA244.1838.3178D240@stuart.lexacorp.com.pg> Message-ID: <4B0DA71E.3030105@colbyconsulting.com> > Gee you guys are taking me back - I used to write games for the Amiga twenty odd years ago - It all revolved around sprites and collisions. LOL. Give it a whirl. I bet you find it much easier in this day and age. One of the reasons I am doing this is that I want to get really good at C#. I do have real work that I am starting to do in C#, but writing games is fun and will keep me in it on a daily / weekly basis whereas my work could be weeks without having to write any C# code. Plus I am hoping to slowly suck my son into the programming. He was up in my office changing the constants that determine how many space ships will be created in the shoot-em-up demo I was studying. He now knows how to edit that constant, how to run it, how to close the game, he knows now that he can't put any number he wants in an integer variable (he tried to run with 20000000000000 spaceships;). We'll see. John W. Colby www.ColbyConsulting.com Stuart McLachlan wrote: > Gee you guys are taking me back - I used to write games for the Amiga twenty odd years > ago - It all revolved around sprites and collisions. > From jwcolby at colbyconsulting.com Wed Nov 25 20:24:52 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 25 Nov 2009 21:24:52 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. Message-ID: <4B0DE6F4.3070605@colbyconsulting.com> If a parameter in a stored procedure is declared as an output (returns a value) will the parameter object in C# / ADO contain that value when the stored procedure finishes? I haven't used output parameters yet in stored procedures. Can values be passed in via an output parameter or can it only be set from inside of the stored procedure? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Thu Nov 26 00:02:40 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 26 Nov 2009 09:02:40 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0DE6F4.3070605@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> Message-ID: <00a101ca6e5e$111e9ac0$335bd040$@spb.ru> Hi John, Yes. Set System.Data.ParameterDirection.Output for .Direction property of output parameters. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 26, 2009 5:25 AM To: VBA Subject: [dba-VB] Parameter objects in C# and ADO. If a parameter in a stored procedure is declared as an output (returns a value) will the parameter object in C# / ADO contain that value when the stored procedure finishes? I haven't used output parameters yet in stored procedures. Can values be passed in via an output parameter or can it only be set from inside of the stored procedure? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4637 (20091125) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4637 (20091125) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From marklbreen at gmail.com Thu Nov 26 03:04:54 2009 From: marklbreen at gmail.com (Mark Breen) Date: Thu, 26 Nov 2009 09:04:54 +0000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0DE6F4.3070605@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> Message-ID: Hello John, There are three ways to bring back values from an sproc, 1 the result set 2 the output parameters, of which there can be many 3 the return code afaik, a param can be input (the default direction) or output. I have not heard of an inputoutput param so unless anyone here says different, assume that you cannot have bi-directional. Now you have the ability to standardize on a return code which may relate to error codes, a complete recordset of data, and also a few params that passed back logging codes, temp counts etc etc etc. In fact, if the sproc is doing multiple stages, you will love the ability to store staged log codes and messages and then bring the whole lot back when it is finished, and still being 1M records as well ! Thanks Mark 2009/11/26 jwcolby > If a parameter in a stored procedure is declared as an output (returns a > value) will the parameter > object in C# / ADO contain that value when the stored procedure finishes? > > I haven't used output parameters yet in stored procedures. Can values be > passed in via an output > parameter or can it only be set from inside of the stored procedure? > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Thu Nov 26 04:26:58 2009 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Thu, 26 Nov 2009 20:26:58 +1000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: <4B0DE6F4.3070605@colbyconsulting.com>, Message-ID: <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> SP with a parameter that is both Input and Output In some situations, there is no need to store the output into an extra variable because the output must replace the input. CREATE PROCEDURE cap2 (@name NVARCHAR(20) OUT ) --Capitalizes first char, changes the rest to lower case --SP with one input/output parameter AS SET @name=UPPER(LEFT(@name,1))+LOWER(RIGHT(@name,LEN(@name)-1)) or for you .net types, see http://fabioscagliola.spaces.live.com/blog/cns!919F8FCDE3DC9AC4!121.entry -- Stuart On 26 Nov 2009 at 9:04, Mark Breen wrote: > > afaik, a param can be input (the default direction) or output. I have not > heard of an inputoutput param so unless anyone here says different, assume > that you cannot have bi-directional. > From shamil at smsconsulting.spb.ru Thu Nov 26 05:03:27 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 26 Nov 2009 14:03:27 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> References: <4B0DE6F4.3070605@colbyconsulting.com>, <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> Message-ID: <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> Hi Mark, > assume that you cannot have bi-directional. You can: System.Data.ParameterDirection.InputOutput -- Shamil On 26 Nov 2009 at 9:04, Mark Breen wrote: > > afaik, a param can be input (the default direction) or output. I have not > heard of an inputoutput param so unless anyone here says different, assume > that you cannot have bi-directional. > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4638 (20091126) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Thu Nov 26 06:55:59 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 26 Nov 2009 07:55:59 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: <4B0DE6F4.3070605@colbyconsulting.com> Message-ID: <4B0E7ADF.5010504@colbyconsulting.com> Thanks for the reply Mark. I am looking forward to having the flexibility of C# to handle errors appropriately. John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello John, > > There are three ways to bring back values from an sproc, > > 1 the result set > 2 the output parameters, of which there can be many > 3 the return code > > afaik, a param can be input (the default direction) or output. I have not > heard of an inputoutput param so unless anyone here says different, assume > that you cannot have bi-directional. > > Now you have the ability to standardize on a return code which may relate to > error codes, a complete recordset of data, and also a few params that passed > back logging codes, temp counts etc etc etc. > > In fact, if the sproc is doing multiple stages, you will love the ability to > store staged log codes and messages and then bring the whole lot back when > it is finished, and still being 1M records as well ! > > Thanks > > Mark > > > > > 2009/11/26 jwcolby > >> If a parameter in a stored procedure is declared as an output (returns a >> value) will the parameter >> object in C# / ADO contain that value when the stored procedure finishes? >> >> I haven't used output parameters yet in stored procedures. Can values be >> passed in via an output >> parameter or can it only be set from inside of the stored procedure? >> >> -- >> John W. Colby >> www.ColbyConsulting.com >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From marklbreen at gmail.com Thu Nov 26 09:59:22 2009 From: marklbreen at gmail.com (Mark Breen) Date: Thu, 26 Nov 2009 15:59:22 +0000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> Message-ID: Hello Shamil and Stuart, I never knew params could be bi-directional, thanks a lot for that, Mark 2009/11/26 Shamil Salakhetdinov > Hi Mark, > > > assume that you cannot have bi-directional. > You can: > > System.Data.ParameterDirection.InputOutput > > -- > Shamil > > On 26 Nov 2009 at 9:04, Mark Breen wrote: > > > > afaik, a param can be input (the default direction) or output. I have > not > > heard of an inputoutput param so unless anyone here says different, > assume > > that you cannot have bi-directional. > > > > > > > > __________ Information from ESET NOD32 Antivirus, version of virus > signature > database 4638 (20091126) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Thu Nov 26 10:04:07 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 26 Nov 2009 11:04:07 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> References: <4B0DE6F4.3070605@colbyconsulting.com>, <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> Message-ID: <4B0EA6F7.4050600@colbyconsulting.com> Thanks Stuart. John W. Colby www.ColbyConsulting.com Stuart McLachlan wrote: > SP with a parameter that is both Input and Output > > In some situations, there is no need to store the output into an extra variable because the > output must replace the input. > > CREATE PROCEDURE cap2 (@name NVARCHAR(20) OUT ) > --Capitalizes first char, changes the rest to lower case > --SP with one input/output parameter > AS > SET @name=UPPER(LEFT(@name,1))+LOWER(RIGHT(@name,LEN(@name)-1)) > > or for you .net types, see > http://fabioscagliola.spaces.live.com/blog/cns!919F8FCDE3DC9AC4!121.entry > From jwcolby at colbyconsulting.com Thu Nov 26 10:05:27 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 26 Nov 2009 11:05:27 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> References: <4B0DE6F4.3070605@colbyconsulting.com>, <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> Message-ID: <4B0EA747.4020506@colbyconsulting.com> > System.Data.ParameterDirection.InputOutput Thanks Shamil, What do you do with this? Set it to true? Set something else equal to this (is this a constant)? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Mark, > >> assume that you cannot have bi-directional. > You can: > > System.Data.ParameterDirection.InputOutput > > -- > Shamil > > On 26 Nov 2009 at 9:04, Mark Breen wrote: >> afaik, a param can be input (the default direction) or output. I have > not >> heard of an inputoutput param so unless anyone here says different, assume >> that you cannot have bi-directional. >> > > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4638 (20091126) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Thu Nov 26 10:54:43 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 26 Nov 2009 19:54:43 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0EA747.4020506@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com>, <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> <4B0EA747.4020506@colbyconsulting.com> Message-ID: <00de01ca6eb9$287b8830$79729890$@spb.ru> Hi John -- This is enumeration value - have a look here how to use it (near to the bottom): http://stackoverflow.com/questions/1637646/need-a-return-value-from-a-stored -procedure-in-c -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Thursday, November 26, 2009 7:05 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. > System.Data.ParameterDirection.InputOutput Thanks Shamil, What do you do with this? Set it to true? Set something else equal to this (is this a constant)? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Mark, > >> assume that you cannot have bi-directional. > You can: > > System.Data.ParameterDirection.InputOutput > > -- > Shamil > > On 26 Nov 2009 at 9:04, Mark Breen wrote: >> afaik, a param can be input (the default direction) or output. I have > not >> heard of an inputoutput param so unless anyone here says different, assume >> that you cannot have bi-directional. >> > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4639 (20091126) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From marklbreen at gmail.com Fri Nov 27 04:00:01 2009 From: marklbreen at gmail.com (Mark Breen) Date: Fri, 27 Nov 2009 10:00:01 +0000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0EA747.4020506@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B0E57F2.28145.343E8C1B@stuart.lexacorp.com.pg> <00be01ca6e88$15cc5240$4164f6c0$@spb.ru> <4B0EA747.4020506@colbyconsulting.com> Message-ID: Hello John, that Shamil has said is that you can use one param to pass in a value, and use the same param to accept back a value, which may be different to when it was passes in. You can pass in ProductId, and get back, ProductCode, or pass in username and get back fullname, or pass in product code and get back zero if the product code does not exist, else you get back the code you passed in or pass in productid and get back qty in stock, all with one param. My examples are not great, but there are plenty of times where you need to somehow inspect a value and it may not be ideal to have two params named paramOriginal paramNewValue with inputout you can have just one param named paramToBeVerified Thanks again to Shamil, Stuart and Mr Colby for asking the question. Mark 2009/11/26 jwcolby > > System.Data.ParameterDirection.InputOutput > > Thanks Shamil, > > What do you do with this? Set it to true? Set something else equal to > this (is this a constant)? > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: > > Hi Mark, > > > >> assume that you cannot have bi-directional. > > You can: > > > > System.Data.ParameterDirection.InputOutput > > > > -- > > Shamil > > > > On 26 Nov 2009 at 9:04, Mark Breen wrote: > >> afaik, a param can be input (the default direction) or output. I have > > not > >> heard of an inputoutput param so unless anyone here says different, > assume > >> that you cannot have bi-directional. > >> > > > > > > > > > > > > __________ Information from ESET NOD32 Antivirus, version of virus > signature > > database 4638 (20091126) __________ > > > > The message was checked by ESET NOD32 Antivirus. > > > > http://www.esetnod32.ru > > > > > > _______________________________________________ > > dba-VB mailing list > > dba-VB at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > http://www.databaseadvisors.com > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Fri Nov 27 04:44:46 2009 From: Gustav at cactus.dk (Gustav Brock) Date: Fri, 27 Nov 2009 11:44:46 +0100 Subject: [dba-VB] Parameter objects in C# and ADO. Message-ID: Hi Mark et al What's the big deal? Are parameters costly? Hardly in money, but in resources? /gustav >>> marklbreen at gmail.com 27-11-2009 11:00 >>> Hello John, that Shamil has said is that you can use one param to pass in a value, and use the same param to accept back a value, which may be different to when it was passes in. You can pass in ProductId, and get back, ProductCode, or pass in username and get back fullname, or pass in product code and get back zero if the product code does not exist, else you get back the code you passed in or pass in productid and get back qty in stock, all with one param. My examples are not great, but there are plenty of times where you need to somehow inspect a value and it may not be ideal to have two params named paramOriginal paramNewValue with inputout you can have just one param named paramToBeVerified Thanks again to Shamil, Stuart and Mr Colby for asking the question. Mark From shamil at smsconsulting.spb.ru Fri Nov 27 06:43:24 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 27 Nov 2009 15:43:24 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: Message-ID: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> Hi Gustav at all, BTW, have you ever seen the following below error handling in CRUD SPs as I have found in one of my customers' databases? Isn't an overkill? Or do you use even more detailed/elaborated error handling in your CRUD SPs? How do you organize error handling in your CRUD SPs? (I usually just return error code in RETURN statement). CREATE procedure [dbo].[GettblUserByUsername] @Username varchar(20), @ErrorDesc varchar(100) output, @ErrorNo int output, @RowsAffected int output AS DECLARE @lErrorNo INTEGER DECLARE @lRowsAffected INTEGER Begin SELECT tblUser.tblUserId, tblUser.tblUserGroupId, tblUserGroup.groupCode as UserGroup, tblUser.Username, tblUser.tblUserLevelId FROM tblUser left outer join tblUserGroup on tblUser.tblUserGroupId = tblUserGroup.tblUserGroupId WHERE tblUser.Username = @username and tblUser.Authorised = 1 select @lErrorNo = @@ERROR, @lRowsAffected = @@ROWCOUNT IF @lErrorNo > 0 Begin set @ErrorDesc = 'error with get of tblUser' set @ErrorNo = @lErrorNo set @RowsAffected = 0 RETURN End Else Begin set @ErrorDesc = 'Get was Successful for tblUser' set @ErrorNo = 0 set @RowsAffected = @lRowsAffected RETURN End end Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Friday, November 27, 2009 1:45 PM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] Parameter objects in C# and ADO. Hi Mark et al What's the big deal? Are parameters costly? Hardly in money, but in resources? /gustav <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4641 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 27 08:11:29 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 09:11:29 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> References: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> Message-ID: <4B0FDE11.9070108@colbyconsulting.com> RowsAffected is not part of the error stuff and I definitely want that info in most cases. I haven't done this yet but I don't think that getting back the error code and description is excessive. If it is possible to easily and unambiguously look up the error code and get the exact description, each and every time, then passing back the description is not necessary. This is exactly the kind of thing I will be doing in the future. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Gustav at all, > > BTW, have you ever seen the following below error handling in CRUD SPs as I > have found in one of my customers' databases? > Isn't an overkill? > Or do you use even more detailed/elaborated error handling in your CRUD SPs? > How do you organize error handling in your CRUD SPs? (I usually just return > error code in RETURN statement). > > CREATE procedure [dbo].[GettblUserByUsername] > @Username varchar(20), > @ErrorDesc varchar(100) output, > @ErrorNo int output, > @RowsAffected int output > AS > DECLARE @lErrorNo INTEGER > DECLARE @lRowsAffected INTEGER > Begin > > SELECT tblUser.tblUserId, > tblUser.tblUserGroupId, > tblUserGroup.groupCode as UserGroup, > tblUser.Username, > tblUser.tblUserLevelId > FROM tblUser > left outer join tblUserGroup > on tblUser.tblUserGroupId = tblUserGroup.tblUserGroupId > WHERE tblUser.Username = @username > and tblUser.Authorised = 1 > > select @lErrorNo = @@ERROR, @lRowsAffected = @@ROWCOUNT > > IF @lErrorNo > 0 > Begin > set @ErrorDesc = 'error with get of tblUser' > set @ErrorNo = @lErrorNo > set @RowsAffected = 0 > RETURN > End > Else > Begin > set @ErrorDesc = 'Get was Successful for tblUser' > set @ErrorNo = 0 > set @RowsAffected = @lRowsAffected > RETURN > End > end > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, November 27, 2009 1:45 PM > To: dba-vb at databaseadvisors.com > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > Hi Mark et al > > What's the big deal? Are parameters costly? Hardly in money, but in > resources? > > /gustav > > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4641 (20091127) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From marklbreen at gmail.com Fri Nov 27 08:33:52 2009 From: marklbreen at gmail.com (Mark Breen) Date: Fri, 27 Nov 2009 14:33:52 +0000 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: Message-ID: Hello Gustav, I presume they are not costly in terms of resources, and in fact, I would not dream of re-using a param for two different purposes. I think my delight was just that I never know you could share directions. I can imagine that Mr Colby will find a use for a standard piece of his arsenal to have a in-out param and once he does, he will wonder how we ever existing without utilising that aspect of params. Perhaps lastupdated as a field would be an appropriate use, where the last updated value is passed in to the sproc, and then passed back again with the new value after an update has been made, after checking for concurrency errors first. Thanks Mark 2009/11/27 Gustav Brock > Hi Mark et al > > What's the big deal? Are parameters costly? Hardly in money, but in > resources? > > /gustav > > > >>> marklbreen at gmail.com 27-11-2009 11:00 >>> > Hello John, > > that Shamil has said is that you can use one param to pass in a value, and > use the same param to accept back a value, which may be different to when > it > was passes in. > > You can pass in ProductId, and get back, ProductCode, > or pass in username and get back fullname, > > or pass in product code and get back zero if the product code does not > exist, else you get back the code you passed in > or pass in productid and get back qty in stock, > > all with one param. > > My examples are not great, but there are plenty of times where you need to > somehow inspect a value and it may not be ideal to have two params named > paramOriginal > paramNewValue > > with inputout you can have just one param named paramToBeVerified > > Thanks again to Shamil, Stuart and Mr Colby for asking the question. > > Mark > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 27 08:44:16 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 09:44:16 -0500 Subject: [dba-VB] SPAM-LOW: Re: Parameter objects in C# and ADO. In-Reply-To: References: Message-ID: <4B0FE5C0.5030409@colbyconsulting.com> > I can imagine that Mr Colby will find a use for a standard piece of his arsenal to have a in-out param and once he does, he will wonder how we ever existing without utilizing that aspect of params. LOL. Your confidence is encouraging. ;) While I use "by reference" values in normal programming, it is rare that I actually modify the passed in value. But it is something that can theoretically be useful and so it is nice to know it is available. > "Mr Colby" I assume you are either feeling particularly young this week or you discovered how old I actually am? ;) All of my kid's friends call me "Mr. Colby". Have you been hanging out with Robbie? John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello Gustav, > > I presume they are not costly in terms of resources, and in fact, I would > not dream of re-using a param for two different purposes. > > I think my delight was just that I never know you could share directions. > > I can imagine that Mr Colby will find a use for a standard piece of his > arsenal to have a in-out param and once he does, he will wonder how we ever > existing without utilising that aspect of params. > > Perhaps lastupdated as a field would be an appropriate use, where the last > updated value is passed in to the sproc, and then passed back again with the > new value after an update has been made, after checking for concurrency > errors first. > > Thanks > > Mark From shamil at smsconsulting.spb.ru Fri Nov 27 08:59:16 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 27 Nov 2009 17:59:16 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0FDE11.9070108@colbyconsulting.com> References: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> <4B0FDE11.9070108@colbyconsulting.com> Message-ID: <000d01ca6f72$31f97f70$95ec7e50$@spb.ru> Hi John -- This kind of "excessive" execution result/error information is useful in my opinion for non-CRUD SPs. But for CRUD ones one can just use SPs return value: - if it's zero or positive - all is OK and returned value is @@ROWCOUNT; - if it's negative - then this is @@ERROR (if actual @@ERROR value will be positive just return it as negative number). If some kinds of runtime errors happen in SPs then they can be trapped by calling C# code try/catch block and actual error message can be obtained from System.Data.SqlClient.SqlException. Well, above is what I'd call a "lightweight" error handling approach - and it works well in many real life systems AFAIK... But for long running (set of) SPs one can use "heavyweight" error handling approach using T-SQLs TRY...CATCH features as described in MS SQL books online (local link): ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/tsqlref9/html/248df62a-7334-4bca-8262- 235a28f4b07f.htm I could be missing something. Please correct me where I'm wrong, please write about your approach to T-SQL (and calling C# code) error handling. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 27, 2009 5:11 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. RowsAffected is not part of the error stuff and I definitely want that info in most cases. I haven't done this yet but I don't think that getting back the error code and description is excessive. If it is possible to easily and unambiguously look up the error code and get the exact description, each and every time, then passing back the description is not necessary. This is exactly the kind of thing I will be doing in the future. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Gustav at all, > > BTW, have you ever seen the following below error handling in CRUD SPs as I > have found in one of my customers' databases? > Isn't an overkill? > Or do you use even more detailed/elaborated error handling in your CRUD SPs? > How do you organize error handling in your CRUD SPs? (I usually just return > error code in RETURN statement). > > CREATE procedure [dbo].[GettblUserByUsername] > @Username varchar(20), > @ErrorDesc varchar(100) output, > @ErrorNo int output, > @RowsAffected int output > AS > DECLARE @lErrorNo INTEGER > DECLARE @lRowsAffected INTEGER > Begin > > SELECT tblUser.tblUserId, > tblUser.tblUserGroupId, > tblUserGroup.groupCode as UserGroup, > tblUser.Username, > tblUser.tblUserLevelId > FROM tblUser > left outer join tblUserGroup > on tblUser.tblUserGroupId = tblUserGroup.tblUserGroupId > WHERE tblUser.Username = @username > and tblUser.Authorised = 1 > > select @lErrorNo = @@ERROR, @lRowsAffected = @@ROWCOUNT > > IF @lErrorNo > 0 > Begin > set @ErrorDesc = 'error with get of tblUser' > set @ErrorNo = @lErrorNo > set @RowsAffected = 0 > RETURN > End > Else > Begin > set @ErrorDesc = 'Get was Successful for tblUser' > set @ErrorNo = 0 > set @RowsAffected = @lRowsAffected > RETURN > End > end > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, November 27, 2009 1:45 PM > To: dba-vb at databaseadvisors.com > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > Hi Mark et al > > What's the big deal? Are parameters costly? Hardly in money, but in > resources? > > /gustav > > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4641 (20091127) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4641 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru __________ Information from ESET NOD32 Antivirus, version of virus signature database 4641 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 27 09:17:22 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 10:17:22 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: References: Message-ID: <4B0FED82.3080608@colbyconsulting.com> Over the last year or so I have built up an arsenal of stored procedures which form a system for performing big picture tasks. A set of SPs create a set of temp tables, move data into them, index them, the export chunks into large CSV files for address validation. Similar functionality on the way back in, create a chunk temp table, import the data from a CSV file, copy to a big temp table, create indexes on them, update hash codes and address valid flags, create more indexes on those fields etc. To this point I have executed these stored procedures from an Access database, simply because I could write Access VBA, however VBA is less than ideal for this functionality, not to mention that I had no ability to get data back from the SP. Thus the SP had to write to a log file which I could then read from Access. Again, less than ideal. Now that I am (slooooowwwwly) coming up to speed on C#, I have embarked on rewriting all of that control logic into C#. Once I figure out how to read back parameters from the SP so that I can see things like RecordsAffected, error numbers and messages etc. I will have made a giant step forward in fine grained control of my big picture processes. While I am at it I have to modify the stored procedures to pass back things like recordcount affected, error codes etc. Since I could not (did not know how to) get at them from VBA I hadn't bothered to add that kind of stuff to the SPs. John W. Colby www.ColbyConsulting.com Mark Breen wrote: > Hello Gustav, > > I presume they are not costly in terms of resources, and in fact, I would > not dream of re-using a param for two different purposes. > > I think my delight was just that I never know you could share directions. > > I can imagine that Mr Colby will find a use for a standard piece of his > arsenal to have a in-out param and once he does, he will wonder how we ever > existing without utilising that aspect of params. > > Perhaps lastupdated as a field would be an appropriate use, where the last > updated value is passed in to the sproc, and then passed back again with the > new value after an update has been made, after checking for concurrency > errors first. > > Thanks > > Mark From jwcolby at colbyconsulting.com Fri Nov 27 09:24:15 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 10:24:15 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <000d01ca6f72$31f97f70$95ec7e50$@spb.ru> References: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> <4B0FDE11.9070108@colbyconsulting.com> <000d01ca6f72$31f97f70$95ec7e50$@spb.ru> Message-ID: <4B0FEF1F.8060905@colbyconsulting.com> Shamil, I am looking for guidance in this area myself. As you know by now, the particular application I am working on is not CRUD based objects but rather specific sets of SPs to perform specific operations in my business. To this point, I have used VBA and a function for executing a stored procedure provided by Charlotte. I could never figure out how to use that function to get at returned values so I have to this point not used that functionality at all. Now I expect to be able to do so, and as a result I will be expanding my processing to include such error handling. It will be a long time before I am correcting anyone on this stuff! John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > This kind of "excessive" execution result/error information is useful in my > opinion for non-CRUD SPs. > But for CRUD ones one can just use SPs return value: > > - if it's zero or positive - all is OK and returned value is @@ROWCOUNT; > - if it's negative - then this is @@ERROR (if actual @@ERROR value will be > positive just return it as negative number). > > If some kinds of runtime errors happen in SPs then they can be trapped by > calling C# code try/catch block and actual error message can be obtained > from System.Data.SqlClient.SqlException. > > Well, above is what I'd call a "lightweight" error handling approach - and > it works well in many real life systems AFAIK... > > But for long running (set of) SPs one can use "heavyweight" error handling > approach using T-SQLs > > TRY...CATCH > > features as described in MS SQL books online (local link): > > ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/tsqlref9/html/248df62a-7334-4bca-8262- > 235a28f4b07f.htm > > I could be missing something. > Please correct me where I'm wrong, please write about your approach to T-SQL > (and calling C# code) error handling. > > Thank you. > > -- > Shamil From accessd at shaw.ca Fri Nov 27 11:50:56 2009 From: accessd at shaw.ca (Jim Lawrence) Date: Fri, 27 Nov 2009 09:50:56 -0800 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0FDE11.9070108@colbyconsulting.com> References: <000b01ca6f5f$37082f60$a5188e20$@spb.ru> <4B0FDE11.9070108@colbyconsulting.com> Message-ID: <805FE569738A4777BD83069AB8C812CE@creativesystemdesigns.com> Hi John: ADO always returns the number of rows affected in MS SQL SP call. It just has to be retrieved. Here is the code that I used as a sample/base before. http://msdn.microsoft.com/en-us/library/aa302325.aspx Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 27, 2009 6:11 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. RowsAffected is not part of the error stuff and I definitely want that info in most cases. I haven't done this yet but I don't think that getting back the error code and description is excessive. If it is possible to easily and unambiguously look up the error code and get the exact description, each and every time, then passing back the description is not necessary. This is exactly the kind of thing I will be doing in the future. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi Gustav at all, > > BTW, have you ever seen the following below error handling in CRUD SPs as I > have found in one of my customers' databases? > Isn't an overkill? > Or do you use even more detailed/elaborated error handling in your CRUD SPs? > How do you organize error handling in your CRUD SPs? (I usually just return > error code in RETURN statement). > > CREATE procedure [dbo].[GettblUserByUsername] > @Username varchar(20), > @ErrorDesc varchar(100) output, > @ErrorNo int output, > @RowsAffected int output > AS > DECLARE @lErrorNo INTEGER > DECLARE @lRowsAffected INTEGER > Begin > > SELECT tblUser.tblUserId, > tblUser.tblUserGroupId, > tblUserGroup.groupCode as UserGroup, > tblUser.Username, > tblUser.tblUserLevelId > FROM tblUser > left outer join tblUserGroup > on tblUser.tblUserGroupId = tblUserGroup.tblUserGroupId > WHERE tblUser.Username = @username > and tblUser.Authorised = 1 > > select @lErrorNo = @@ERROR, @lRowsAffected = @@ROWCOUNT > > IF @lErrorNo > 0 > Begin > set @ErrorDesc = 'error with get of tblUser' > set @ErrorNo = @lErrorNo > set @RowsAffected = 0 > RETURN > End > Else > Begin > set @ErrorDesc = 'Get was Successful for tblUser' > set @ErrorNo = 0 > set @RowsAffected = @lRowsAffected > RETURN > End > end > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, November 27, 2009 1:45 PM > To: dba-vb at databaseadvisors.com > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > Hi Mark et al > > What's the big deal? Are parameters costly? Hardly in money, but in > resources? > > /gustav > > > <<< snip >>> > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4641 (20091127) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Nov 27 14:26:11 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 15:26:11 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B0DE6F4.3070605@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> Message-ID: <4B1035E3.7000003@colbyconsulting.com> > If a parameter in a stored procedure is declared as an output (returns a value) will the parameter object in C# / ADO contain that value when the stored procedure finishes? This part of the question got lost, not directly answered AFAICT. I am directly creating parameter objects and adding them to the parameter collection of the command object. Code execution will halt until the stored procedure finishes executing and then the parameter objects in the command object's parameters collection can be looked at for the returned values? John W. Colby www.ColbyConsulting.com jwcolby wrote: > If a parameter in a stored procedure is declared as an output (returns a value) will the parameter > object in C# / ADO contain that value when the stored procedure finishes? > > I haven't used output parameters yet in stored procedures. Can values be passed in via an output > parameter or can it only be set from inside of the stored procedure? > From shamil at smsconsulting.spb.ru Fri Nov 27 15:14:53 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 00:14:53 +0300 Subject: [dba-VB] Calling SP with params from C# sample Message-ID: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> Hi John - Here is a sample for you: public static void TestSP() { string connectionString = "{{Your connection string here}}"; //CREATE Procedure [dbo].[TestParams] // @Username varchar(20), // @ErrorDesc varchar(100) output, // @ErrorNo int output, // @RowsAffected int output AS //DECLARE @lErrorNo INTEGER //DECLARE @lRowsAffected INTEGER //Begin // set @ErrorDesc = 'SUCCESS' + ', UserName = ' + IsNull(@USerName,'?') // set @ErrorNo = 1 -- OK // set @RowsAffected = 503 // RETURN 1234 //end try { using (SqlConnection connection = new SqlConnection(connectionString)) { connection.Open(); SqlCommand command = new SqlCommand("TestParams", connection); command.CommandType = System.Data.CommandType.StoredProcedure; // @Username varchar(20), command.Parameters.Add( new SqlParameter("@Username", System.Data.SqlDbType.VarChar, 20)); command.Parameters["@UserName"].Value = "SP_PARAMS_TEST"; // @ErrorDesc varchar(100) output, command.Parameters.Add( new SqlParameter("@ErrorDesc", System.Data.SqlDbType.VarChar, 100)); command.Parameters["@ErrorDesc"].Direction = System.Data.ParameterDirection.Output; // @ErrorNo int output, command.Parameters.Add( new SqlParameter("@ErrorNo", System.Data.SqlDbType.Int, 4)); command.Parameters["@ErrorNo"].Direction = System.Data.ParameterDirection.Output; // @RowsAffected int output AS command.Parameters.Add( new SqlParameter("@RowsAffected", System.Data.SqlDbType.Int, 4)); command.Parameters["@RowsAffected"].Direction = System.Data.ParameterDirection.Output; // RETURN command.Parameters.Add( new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int, 4)); command.Parameters["@RETURN_VALUE"].Direction = System.Data.ParameterDirection.ReturnValue; int retValue = command.ExecuteNonQuery(); Console.WriteLine( "retValue = {0}\n" + "@RETURN_VALUE = {1}\n" + "@ErrorDesc = '{2}'\n" + "@ErrorNo = {3}\n" + "@RowsAffected = {4}" , retValue, command.Parameters["@RETURN_VALUE"].Value.ToString(), command.Parameters["@ErrorDesc"].Value.ToString(), command.Parameters["@ErrorNo"].Value.ToString(), command.Parameters["@RowsAffected"].Value.ToString() ); } } catch (SqlException ex) { Console.WriteLine("T-SQL: {0}", ex.Message); } catch (Exception ex) { Console.WriteLine("{0}", ex.Message); } } Result: retValue = -1 @RETURN_VALUE = 1234 @ErrorDesc = 'SUCCESS, UserName = SP_PARAMS_TEST' @ErrorNo = 1 @RowsAffected = 503 -- Shamil From shamil at smsconsulting.spb.ru Fri Nov 27 15:31:02 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 00:31:02 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B1035E3.7000003@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B1035E3.7000003@colbyconsulting.com> Message-ID: <009501ca6fa8$ecaf22d0$c60d6870$@spb.ru> Hi John -- I have just posted here a sample for synchronous execution of SP with params. You can also run SPs from C# asynchronously - e.g. look MSDN for BeginExecuteNonQuery method of System.Data.SqlClient.SqlCommand class. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Friday, November 27, 2009 11:26 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. > If a parameter in a stored procedure is declared as an output (returns a value) will the parameter object in C# / ADO contain that value when the stored procedure finishes? This part of the question got lost, not directly answered AFAICT. I am directly creating parameter objects and adding them to the parameter collection of the command object. Code execution will halt until the stored procedure finishes executing and then the parameter objects in the command object's parameters collection can be looked at for the returned values? John W. Colby www.ColbyConsulting.com jwcolby wrote: > If a parameter in a stored procedure is declared as an output (returns a value) will the parameter > object in C# / ADO contain that value when the stored procedure finishes? > > I haven't used output parameters yet in stored procedures. Can values be passed in via an output > parameter or can it only be set from inside of the stored procedure? > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4643 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 27 15:40:45 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 16:40:45 -0500 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <009501ca6fa8$ecaf22d0$c60d6870$@spb.ru> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B1035E3.7000003@colbyconsulting.com> <009501ca6fa8$ecaf22d0$c60d6870$@spb.ru> Message-ID: <4B10475D.8040102@colbyconsulting.com> Very cool! I am rewriting the code I found on the internet to allow me to pass in the direction as well as manipulate the param object when it returns. Thanks again for the example code. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > I have just posted here a sample for synchronous execution of SP with > params. > You can also run SPs from C# asynchronously - e.g. look MSDN for > BeginExecuteNonQuery method of System.Data.SqlClient.SqlCommand class. > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 27, 2009 11:26 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > > If a parameter in a stored procedure is declared as an output (returns a > value) will the > parameter object in C# / ADO contain that value when the stored procedure > finishes? > > This part of the question got lost, not directly answered AFAICT. > > I am directly creating parameter objects and adding them to the parameter > collection of the command > object. Code execution will halt until the stored procedure finishes > executing and then the > parameter objects in the command object's parameters collection can be > looked at for the returned > values? > > John W. Colby > www.ColbyConsulting.com > > > jwcolby wrote: >> If a parameter in a stored procedure is declared as an output (returns a > value) will the parameter >> object in C# / ADO contain that value when the stored procedure finishes? >> >> I haven't used output parameters yet in stored procedures. Can values be > passed in via an output >> parameter or can it only be set from inside of the stored procedure? >> > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4643 (20091127) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Nov 27 16:02:17 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 01:02:17 +0300 Subject: [dba-VB] Async calling SP with params from C# sample In-Reply-To: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> Message-ID: <009601ca6fad$4a2737f0$de75a7d0$@spb.ru> Another sample - async call (partially used MSDN sample): public static void runTest() { SPTester o = new SPTester(); o.TestSP(); while (!o.Completed) System.Threading.Thread.Sleep(1000); } public class SPTester { public bool Completed { get; set; } private SqlConnection _connection; public void TestSP() { Completed = false; string connectionString = "Data Source=HAMBURG\\SQL2005;"+ "Initial Catalog=X;"+ "User Id=sa;Password=Y;"+ "Asynchronous Processing=true"; //CREATE Procedure [dbo].[TestParams] // @Username varchar(20), // @ErrorDesc varchar(100) output, // @ErrorNo int output, // @RowsAffected int output AS //DECLARE @lErrorNo INTEGER //DECLARE @lRowsAffected INTEGER //Begin // set @ErrorDesc = 'SUCCESS' + ', UserName = ' + IsNull(@USerName,'?') // set @ErrorNo = 1 -- OK // set @RowsAffected = 503 // RETURN 1234 //end try { _connection = new SqlConnection(connectionString); { _connection.Open(); SqlCommand command = new SqlCommand("TestParams", _connection); command.CommandType = System.Data.CommandType.StoredProcedure; // @Username varchar(20), command.Parameters.Add( new SqlParameter("@Username", System.Data.SqlDbType.VarChar, 20)); command.Parameters["@UserName"].Value = "SP_PARAMS_TEST"; // @ErrorDesc varchar(100) output, command.Parameters.Add( new SqlParameter("@ErrorDesc", System.Data.SqlDbType.VarChar, 100)); command.Parameters["@ErrorDesc"].Direction = System.Data.ParameterDirection.Output; // @ErrorNo int output, command.Parameters.Add( new SqlParameter("@ErrorNo", System.Data.SqlDbType.Int, 4)); command.Parameters["@ErrorNo"].Direction = System.Data.ParameterDirection.Output; // @RowsAffected int output AS command.Parameters.Add( new SqlParameter("@RowsAffected", System.Data.SqlDbType.Int, 4)); command.Parameters["@RowsAffected"].Direction = System.Data.ParameterDirection.Output; // RETURN command.Parameters.Add( new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int, 4)); command.Parameters["@RETURN_VALUE"].Direction = System.Data.ParameterDirection.ReturnValue; AsyncCallback callback = new AsyncCallback(handleCallback); command.BeginExecuteNonQuery(callback, command); } } catch (Exception ex) { Console.WriteLine(ex.Message); if (_connection != null) { _connection.Close(); _connection = null; } } } private void handleCallback(IAsyncResult result) { try { SqlCommand command = (SqlCommand)result.AsyncState; int rowCount = command.EndExecuteNonQuery(result); Console.WriteLine( "rowCount = {0}\n" + "@RETURN_VALUE = {1}\n" + "@ErrorDesc = '{2}'\n" + "@ErrorNo = {3}\n" + "@RowsAffected = {4}" , rowCount, command.Parameters["@RETURN_VALUE"].Value.ToString(), command.Parameters["@ErrorDesc"].Value.ToString(), command.Parameters["@ErrorNo"].Value.ToString(), command.Parameters["@RowsAffected"].Value.ToString() ); } catch (SqlException ex) { Console.WriteLine("T-SQL: {0}", ex.Message); } catch (Exception ex) { Console.WriteLine("{0}", ex.Message); } finally { Completed = true; if (_connection != null) { _connection.Close(); _connection = null; } } } } -- Shamil From shamil at smsconsulting.spb.ru Fri Nov 27 16:09:52 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 01:09:52 +0300 Subject: [dba-VB] Parameter objects in C# and ADO. In-Reply-To: <4B10475D.8040102@colbyconsulting.com> References: <4B0DE6F4.3070605@colbyconsulting.com> <4B1035E3.7000003@colbyconsulting.com> <009501ca6fa8$ecaf22d0$c60d6870$@spb.ru> <4B10475D.8040102@colbyconsulting.com> Message-ID: <009701ca6fae$591f3310$0b5d9930$@spb.ru> Hi John -- You're welcome! I have just posted another sample code where SP call from C# is processed asynchronously. -- Shamil P.S. There is an issue with the code I mentioned above - Completed flag should be also set to true in the catch() block of the calling method: public void TestSP() -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 28, 2009 12:41 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Parameter objects in C# and ADO. Very cool! I am rewriting the code I found on the internet to allow me to pass in the direction as well as manipulate the param object when it returns. Thanks again for the example code. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John -- > > I have just posted here a sample for synchronous execution of SP with > params. > You can also run SPs from C# asynchronously - e.g. look MSDN for > BeginExecuteNonQuery method of System.Data.SqlClient.SqlCommand class. > > -- > Shamil > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Friday, November 27, 2009 11:26 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Parameter objects in C# and ADO. > > > If a parameter in a stored procedure is declared as an output (returns a > value) will the > parameter object in C# / ADO contain that value when the stored procedure > finishes? > > This part of the question got lost, not directly answered AFAICT. > > I am directly creating parameter objects and adding them to the parameter > collection of the command > object. Code execution will halt until the stored procedure finishes > executing and then the > parameter objects in the command object's parameters collection can be > looked at for the returned > values? > > John W. Colby > www.ColbyConsulting.com > > > jwcolby wrote: >> If a parameter in a stored procedure is declared as an output (returns a > value) will the parameter >> object in C# / ADO contain that value when the stored procedure finishes? >> >> I haven't used output parameters yet in stored procedures. Can values be > passed in via an output >> parameter or can it only be set from inside of the stored procedure? >> > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 4643 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Fri Nov 27 21:15:28 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 22:15:28 -0500 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> Message-ID: <4B1095D0.2050804@colbyconsulting.com> Hi Shamil, I assume that the parameters on the C# side have to be in the same order / name / datatype / size as the parameters in the stored procedure? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John - > > > > Here is a sample for you: > > > > public static void TestSP() > > { > > string connectionString = "{{Your connection string here}}"; > > > > //CREATE Procedure [dbo].[TestParams] > > // @Username varchar(20), > > // @ErrorDesc varchar(100) output, > > // @ErrorNo int output, > > // @RowsAffected int output AS > > //DECLARE @lErrorNo INTEGER > > //DECLARE @lRowsAffected INTEGER > > //Begin > > // set @ErrorDesc = 'SUCCESS' + ', UserName = ' + IsNull(@USerName,'?') > > // set @ErrorNo = 1 -- OK > > // set @RowsAffected = 503 > > // RETURN 1234 > > //end > > > > try > > { > > using (SqlConnection connection = new > SqlConnection(connectionString)) > > { > > connection.Open(); > > > > SqlCommand command = new SqlCommand("TestParams", connection); > > command.CommandType = System.Data.CommandType.StoredProcedure; > > // @Username varchar(20), > > command.Parameters.Add( > > new SqlParameter("@Username", System.Data.SqlDbType.VarChar, > 20)); > > command.Parameters["@UserName"].Value = "SP_PARAMS_TEST"; > > // @ErrorDesc varchar(100) output, > > command.Parameters.Add( > > new SqlParameter("@ErrorDesc", > System.Data.SqlDbType.VarChar, 100)); > > command.Parameters["@ErrorDesc"].Direction = > System.Data.ParameterDirection.Output; > > // @ErrorNo int output, > > command.Parameters.Add( > > new SqlParameter("@ErrorNo", System.Data.SqlDbType.Int, 4)); > > command.Parameters["@ErrorNo"].Direction = > System.Data.ParameterDirection.Output; > > // @RowsAffected int output AS > > command.Parameters.Add( > > new SqlParameter("@RowsAffected", System.Data.SqlDbType.Int, > 4)); > > command.Parameters["@RowsAffected"].Direction = > System.Data.ParameterDirection.Output; > > // RETURN > > command.Parameters.Add( > > new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int, > 4)); > > command.Parameters["@RETURN_VALUE"].Direction = > System.Data.ParameterDirection.ReturnValue; > > > > int retValue = command.ExecuteNonQuery(); > > > > Console.WriteLine( > > "retValue = {0}\n" + > > "@RETURN_VALUE = {1}\n" + > > "@ErrorDesc = '{2}'\n" + > > "@ErrorNo = {3}\n" + > > "@RowsAffected = {4}" > > , > > retValue, > > command.Parameters["@RETURN_VALUE"].Value.ToString(), > > command.Parameters["@ErrorDesc"].Value.ToString(), > > command.Parameters["@ErrorNo"].Value.ToString(), > > command.Parameters["@RowsAffected"].Value.ToString() > > ); > > } > > } > > catch (SqlException ex) > > { > > Console.WriteLine("T-SQL: {0}", ex.Message); > > } > > catch (Exception ex) > > { > > Console.WriteLine("{0}", ex.Message); > > } > > } > > > > Result: > > > > retValue = -1 > > @RETURN_VALUE = 1234 > > @ErrorDesc = 'SUCCESS, UserName = SP_PARAMS_TEST' > > @ErrorNo = 1 > > @RowsAffected = 503 > > > > -- > > Shamil > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Nov 27 21:45:19 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 27 Nov 2009 22:45:19 -0500 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> Message-ID: <4B109CCF.5020107@colbyconsulting.com> Shamil, I executed my first SP using your example code (first light as they say in the astronomy world). Man oh man is it ugly though. Tons of code to get one lonely little SP to execute (NOT your fault!!!). Thank you so much for the examples, I can definitely take it from here. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John - > > > > Here is a sample for you: > > > > public static void TestSP() > > { > > string connectionString = "{{Your connection string here}}"; > > > > //CREATE Procedure [dbo].[TestParams] > > // @Username varchar(20), > > // @ErrorDesc varchar(100) output, > > // @ErrorNo int output, > > // @RowsAffected int output AS > > //DECLARE @lErrorNo INTEGER > > //DECLARE @lRowsAffected INTEGER > > //Begin > > // set @ErrorDesc = 'SUCCESS' + ', UserName = ' + IsNull(@USerName,'?') > > // set @ErrorNo = 1 -- OK > > // set @RowsAffected = 503 > > // RETURN 1234 > > //end > > > > try > > { > > using (SqlConnection connection = new > SqlConnection(connectionString)) > > { > > connection.Open(); > > > > SqlCommand command = new SqlCommand("TestParams", connection); > > command.CommandType = System.Data.CommandType.StoredProcedure; > > // @Username varchar(20), > > command.Parameters.Add( > > new SqlParameter("@Username", System.Data.SqlDbType.VarChar, > 20)); > > command.Parameters["@UserName"].Value = "SP_PARAMS_TEST"; > > // @ErrorDesc varchar(100) output, > > command.Parameters.Add( > > new SqlParameter("@ErrorDesc", > System.Data.SqlDbType.VarChar, 100)); > > command.Parameters["@ErrorDesc"].Direction = > System.Data.ParameterDirection.Output; > > // @ErrorNo int output, > > command.Parameters.Add( > > new SqlParameter("@ErrorNo", System.Data.SqlDbType.Int, 4)); > > command.Parameters["@ErrorNo"].Direction = > System.Data.ParameterDirection.Output; > > // @RowsAffected int output AS > > command.Parameters.Add( > > new SqlParameter("@RowsAffected", System.Data.SqlDbType.Int, > 4)); > > command.Parameters["@RowsAffected"].Direction = > System.Data.ParameterDirection.Output; > > // RETURN > > command.Parameters.Add( > > new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int, > 4)); > > command.Parameters["@RETURN_VALUE"].Direction = > System.Data.ParameterDirection.ReturnValue; > > > > int retValue = command.ExecuteNonQuery(); > > > > Console.WriteLine( > > "retValue = {0}\n" + > > "@RETURN_VALUE = {1}\n" + > > "@ErrorDesc = '{2}'\n" + > > "@ErrorNo = {3}\n" + > > "@RowsAffected = {4}" > > , > > retValue, > > command.Parameters["@RETURN_VALUE"].Value.ToString(), > > command.Parameters["@ErrorDesc"].Value.ToString(), > > command.Parameters["@ErrorNo"].Value.ToString(), > > command.Parameters["@RowsAffected"].Value.ToString() > > ); > > } > > } > > catch (SqlException ex) > > { > > Console.WriteLine("T-SQL: {0}", ex.Message); > > } > > catch (Exception ex) > > { > > Console.WriteLine("{0}", ex.Message); > > } > > } > > > > Result: > > > > retValue = -1 > > @RETURN_VALUE = 1234 > > @ErrorDesc = 'SUCCESS, UserName = SP_PARAMS_TEST' > > @ErrorNo = 1 > > @RowsAffected = 503 > > > > -- > > Shamil > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Nov 28 03:19:35 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 12:19:35 +0300 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <4B1095D0.2050804@colbyconsulting.com> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B1095D0.2050804@colbyconsulting.com> Message-ID: <00a201ca700b$e98b6ba0$bca242e0$@spb.ru> Hi John, The order of parameters' creation on C# side doesn't matter, name and datatype and size do matter. There could be some variations for the latter two but better keep them in strict correspondence. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 28, 2009 6:15 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Calling SP with params from C# sample Hi Shamil, I assume that the parameters on the C# side have to be in the same order / name / datatype / size as the parameters in the stored procedure? John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John - > > > > Here is a sample for you: > > > > public static void TestSP() > <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4643 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From shamil at smsconsulting.spb.ru Sat Nov 28 03:19:35 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 28 Nov 2009 12:19:35 +0300 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <4B109CCF.5020107@colbyconsulting.com> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B109CCF.5020107@colbyconsulting.com> Message-ID: <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> John -- Tons of code for C# (or VB.NET) is not an issue at all. And for C#/VB.Net programmers also. You'll get adapted to that reality soon, I believe. I have projects with literally thousands of custom classes - they work in production for several years without any issues. And they recompile from cleaned solution with 10+ projects in 10+ seconds. And when you're working on such solution and change code/recompile/run then recompiling and restarting takes <1-2 seconds as VS recompiles/rebuilds only changed projects. Have a look e.g. on DotNetNuke(DNN) sources - and if you'll try to "dig them out" you'll find how "ugly" DNN sources are sometimes... For your case I'd use: - plain "dumb" manual coding if you have relatively static set of SPs and their parameters; - LINQ for SQL (now); - ADO.NET Entity Framework (VS2010); - statically generated custom classes to call SPs and process SPs' results. I'd not use dynamically generated SqlCommand and SqlParameters as this approach could result in too much and never ending "plumbing programming"... And in all the cases you can use the power of true OOP C# provides to minimize "copy and paste" coding approach... BTW, C# 4.0 provides some very powerful dynamic programming features - more powerful than late binding(?) - I must say I have seen some samples of that new C# 4.0 features usage but I didn't get how to use them :) - maybe they can help you to minimize "ugliness" of your C# coding but be careful to not spend too much time on "plumbing programming"... http://stackoverflow.com/questions/244302/what-do-you-think-of-the-new-c-4-0 -dynamic-keyword http://geekswithblogs.net/sdorman/archive/2008/11/16/c-4.0-dynamic-programmi ng.aspx -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Saturday, November 28, 2009 6:45 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Calling SP with params from C# sample Shamil, I executed my first SP using your example code (first light as they say in the astronomy world). Man oh man is it ugly though. Tons of code to get one lonely little SP to execute (NOT your fault!!!). Thank you so much for the examples, I can definitely take it from here. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John - > > > > Here is a sample for you: > > > > public static void TestSP() > > { <<< snip >>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4643 (20091127) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From marklbreen at gmail.com Sat Nov 28 08:27:07 2009 From: marklbreen at gmail.com (Mark Breen) Date: Sat, 28 Nov 2009 14:27:07 +0000 Subject: [dba-VB] SPAM-LOW: Re: Parameter objects in C# and ADO. In-Reply-To: <4B0FE5C0.5030409@colbyconsulting.com> References: <4B0FE5C0.5030409@colbyconsulting.com> Message-ID: Haha, My kids call some people by their first name and I still call the same people Mr Dowling, sounds funny to hear it. Mark 2009/11/27 jwcolby > > I can imagine that Mr Colby will find a use for a standard piece of his > arsenal to have a in-out > param and once he does, he will wonder how we ever existing without > utilizing that aspect of params. > > LOL. Your confidence is encouraging. ;) > > While I use "by reference" values in normal programming, it is rare that I > actually modify the > passed in value. But it is something that can theoretically be useful and > so it is nice to know it > is available. > > > "Mr Colby" > > I assume you are either feeling particularly young this week or you > discovered how old I actually > am? ;) > > All of my kid's friends call me "Mr. Colby". Have you been hanging out > with Robbie? > > John W. Colby > www.ColbyConsulting.com > > > Mark Breen wrote: > > Hello Gustav, > > > > I presume they are not costly in terms of resources, and in fact, I would > > not dream of re-using a param for two different purposes. > > > > I think my delight was just that I never know you could share directions. > > > > I can imagine that Mr Colby will find a use for a standard piece of his > > arsenal to have a in-out param and once he does, he will wonder how we > ever > > existing without utilising that aspect of params. > > > > Perhaps lastupdated as a field would be an appropriate use, where the > last > > updated value is passed in to the sproc, and then passed back again with > the > > new value after an update has been made, after checking for concurrency > > errors first. > > > > Thanks > > > > Mark > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Nov 28 16:29:17 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 28 Nov 2009 17:29:17 -0500 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B109CCF.5020107@colbyconsulting.com> <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> Message-ID: <4B11A43D.70104@colbyconsulting.com> Shamil, I ended up creating a "stored procedure class". This class hosts a sCmd command object that can be programmed with the name of the SP, connection etc but then included wrapping the lines of code to create specific often used parameters into functions. /// /// The following code creates standard parameters used all of the time in my stored procedures /// Basically just wrapper code so that I can call a function to create specific parameters as needed /// /// It also allows me to completely standardize the names, data type and length of parameters in the SPs etc. /// /// public void paramDBName(string lstrDBName) { sCmd.Parameters.Add(new SqlParameter( "@DBName", SqlDbType.VarChar, 50)); sCmd.Parameters["@DBName"].Value = lstrDBName; } public void paramTblName(string lstrTblName) { sCmd.Parameters.Add(new SqlParameter("@TblName", SqlDbType.VarChar, 50)); sCmd.Parameters["@TblName"].Value = lstrTblName; } public void paramErrorDesc() { sCmd.Parameters.Add(new SqlParameter("@ErrorDesc", System.Data.SqlDbType.VarChar, 4000)); sCmd.Parameters["@ErrorDesc"].Direction = System.Data.ParameterDirection.Output; } Then I can do things like: // //sp_AZOut_AZExportToOrdered_Append // sp = new StoredProcedure(myConnection, "_aDataMaster.dbo.sp_AZOut_AZExportToOrdered_Append"); sp.paramDBName(lstrDBName); //@DBName varchar(50) input sp.paramTblName(lstrTblName); //@TblName varchar(50) input sp.paramErrorDesc(); //@ErrorDesc varchar(100) output sp.paramErrorNo(); //@ErrorNo int output, sp.paramReturnValue(); //RETURN retValue = sp.ExecuteSP(); I believe I am on my way now. Thanks again, John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > John -- > > Tons of code for C# (or VB.NET) is not an issue at all. And for C#/VB.Net > programmers also. You'll get adapted to that reality soon, I believe. From shamil at smsconsulting.spb.ru Sat Nov 28 17:12:11 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sun, 29 Nov 2009 02:12:11 +0300 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <4B11A43D.70104@colbyconsulting.com> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B109CCF.5020107@colbyconsulting.com> <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> <4B11A43D.70104@colbyconsulting.com> Message-ID: <00ba01ca7080$392fd000$ab8f7000$@spb.ru> John, Yes, that looks good with me. AFAIS you are getting what is usually declared as abstract class, and it can be inherited by custom classes, which would use their base abstract class's "services"/utilities methods. I suppose you can go even a bit further by combining sets of your standard parameters assignments into methods of your base abstract class. BTW, all the methods/properties of your abstract class, which are not planned to be called from outside of custom classes can be declared as 'protected', all the local class level common variables (of all kinds/types) can be also put with 'protected' declaration into your abstract class... -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Sunday, November 29, 2009 1:29 AM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Calling SP with params from C# sample Shamil, I ended up creating a "stored procedure class". This class hosts a sCmd command object that can be programmed with the name of the SP, connection etc but then included wrapping the lines of code to create specific often used parameters into functions. /// /// The following code creates standard parameters used all of the time in my stored procedures /// Basically just wrapper code so that I can call a function to create specific parameters as needed /// /// It also allows me to completely standardize the names, data type and length of parameters in the SPs etc. /// /// public void paramDBName(string lstrDBName) { sCmd.Parameters.Add(new SqlParameter( "@DBName", SqlDbType.VarChar, 50)); sCmd.Parameters["@DBName"].Value = lstrDBName; } public void paramTblName(string lstrTblName) { sCmd.Parameters.Add(new SqlParameter("@TblName", SqlDbType.VarChar, 50)); sCmd.Parameters["@TblName"].Value = lstrTblName; } public void paramErrorDesc() { sCmd.Parameters.Add(new SqlParameter("@ErrorDesc", System.Data.SqlDbType.VarChar, 4000)); sCmd.Parameters["@ErrorDesc"].Direction = System.Data.ParameterDirection.Output; } Then I can do things like: // //sp_AZOut_AZExportToOrdered_Append // sp = new StoredProcedure(myConnection, "_aDataMaster.dbo.sp_AZOut_AZExportToOrdered_Append"); sp.paramDBName(lstrDBName); //@DBName varchar(50) input sp.paramTblName(lstrTblName); //@TblName varchar(50) input sp.paramErrorDesc(); //@ErrorDesc varchar(100) output sp.paramErrorNo(); //@ErrorNo int output, sp.paramReturnValue(); //RETURN retValue = sp.ExecuteSP(); I believe I am on my way now. Thanks again, John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > John -- > > Tons of code for C# (or VB.NET) is not an issue at all. And for C#/VB.Net > programmers also. You'll get adapted to that reality soon, I believe. __________ Information from ESET NOD32 Antivirus, version of virus signature database 4645 (20091128) __________ The message was checked by ESET NOD32 Antivirus. http://www.esetnod32.ru From jwcolby at colbyconsulting.com Sat Nov 28 17:19:19 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 28 Nov 2009 18:19:19 -0500 Subject: [dba-VB] Calling SP with params from C# sample In-Reply-To: <00ba01ca7080$392fd000$ab8f7000$@spb.ru> References: <009001ca6fa6$ab11df90$01359eb0$@spb.ru> <4B109CCF.5020107@colbyconsulting.com> <00a101ca700b$e84caa10$b8e5fe30$@spb.ru> <4B11A43D.70104@colbyconsulting.com> <00ba01ca7080$392fd000$ab8f7000$@spb.ru> Message-ID: <4B11AFF7.8010706@colbyconsulting.com> Perhaps I will do the inheritance thing once I get rolling. It certainly seems like a logical extension. For now I am just instantiating the SP class and manipulating methods of that class. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > John, > > Yes, that looks good with me. > AFAIS you are getting what is usually declared as abstract class, and it can > be inherited by custom classes, which would use their base abstract class's > "services"/utilities methods. I suppose you can go even a bit further by > combining sets of your standard parameters assignments into methods of your > base abstract class. BTW, all the methods/properties of your abstract class, > which are not planned to be called from outside of custom classes can be > declared as 'protected', all the local class level common variables (of all > kinds/types) can be also put with 'protected' declaration into your abstract > class... > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Sunday, November 29, 2009 1:29 AM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] Calling SP with params from C# sample > > Shamil, > > I ended up creating a "stored procedure class". This class hosts a sCmd > command object that can be > programmed with the name of the SP, connection etc but then included > wrapping the lines of code to > create specific often used parameters into functions. > > /// > /// The following code creates standard parameters used all of the > time in my stored procedures > /// Basically just wrapper code so that I can call a function to > create specific parameters > as needed > /// > /// It also allows me to completely standardize the names, data > type and length of > parameters in the SPs etc. > /// > /// > public void paramDBName(string lstrDBName) > { > sCmd.Parameters.Add(new SqlParameter( "@DBName", > SqlDbType.VarChar, 50)); > sCmd.Parameters["@DBName"].Value = lstrDBName; > } > public void paramTblName(string lstrTblName) > { > sCmd.Parameters.Add(new SqlParameter("@TblName", > SqlDbType.VarChar, 50)); > sCmd.Parameters["@TblName"].Value = lstrTblName; > } > public void paramErrorDesc() > { > sCmd.Parameters.Add(new SqlParameter("@ErrorDesc", > System.Data.SqlDbType.VarChar, 4000)); > sCmd.Parameters["@ErrorDesc"].Direction = > System.Data.ParameterDirection.Output; > } > > Then I can do things like: > > // > //sp_AZOut_AZExportToOrdered_Append > // > sp = new StoredProcedure(myConnection, > "_aDataMaster.dbo.sp_AZOut_AZExportToOrdered_Append"); > sp.paramDBName(lstrDBName); //@DBName varchar(50) input > sp.paramTblName(lstrTblName); //@TblName varchar(50) input > sp.paramErrorDesc(); //@ErrorDesc varchar(100) > output > sp.paramErrorNo(); //@ErrorNo int output, > sp.paramReturnValue(); //RETURN > retValue = sp.ExecuteSP(); > > I believe I am on my way now. > > Thanks again, > > John W. Colby > www.ColbyConsulting.com > > > Shamil Salakhetdinov wrote: >> John -- >> >> Tons of code for C# (or VB.NET) is not an issue at all. And for C#/VB.Net >> programmers also. You'll get adapted to that reality soon, I believe. > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature > database 4645 (20091128) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.esetnod32.ru > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sun Nov 29 12:51:14 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 29 Nov 2009 13:51:14 -0500 Subject: [dba-VB] C# - Presenting process status Message-ID: <4B12C2A2.1010001@colbyconsulting.com> Just a general question about how you go about presenting process status back from classes. In Access I tended to use Withevents in my classes, then sink the class events in the form and use the event to pass in values to the form, and in the event sink display the status in a text box control on the form. For example the process that exports data from SQL Server to Accuzip (address validation) would display the name of the file just created and the PKFrom / PKTo pk range. I am also looking for tips on how to report status from a thread. I have vague recollections that a thread cannot directly update controls on a form (or something like that). So how do you folks display status information in a form? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sun Nov 29 15:33:05 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 30 Nov 2009 00:33:05 +0300 Subject: [dba-VB] C# - Presenting process status In-Reply-To: <4B12C2A2.1010001@colbyconsulting.com> References: <4B12C2A2.1010001@colbyconsulting.com> Message-ID: <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> Hi John, Here is one of the methods I use (there are zillion combinations) - I dunno where it comes from, did I borrowed it from somewhere or I got it elaborated myself, is it recommended way or not - but it just works, and I have "cooked" it for you here from scratch - this is codebehind for Windows Form named Form1: using System; using System.Windows.Forms; namespace WinFormsSample { public partial class Form1 : Form { // Create Windows Form Form1 with three controls: // // 1. Button: cmdTest1 -> cmdTest1_Click // 2. Button: cmdTest2 -> cmdTest2_Click // 3. TextBox: txtLog (Multiline = true) #region Test class public class Test { public delegate void ProgressStatus(string message); private static int _id; private System.Threading.Thread _thread; public void Run(ProgressStatus progressStatus) { _testId = ++_id; _progressStatus = progressStatus; System.Threading.ThreadStart ts = new System.Threading.ThreadStart(runThread); _thread = new System.Threading.Thread(ts); _thread.IsBackground = true; _thread.Start(); } private int _testId; private ProgressStatus _progressStatus; private void runThread() { for (int i = 1; i < 10; i++) { _progressStatus(string.Format("Test#{0}, cycle#{1}", _testId, i)); System.Threading.Thread.Sleep(500); } } } #endregion // Test class public Form1() { InitializeComponent(); } private Test _test1; private void cmdTest1_Click(object sender, EventArgs e) { _test1 = new Test(); _test1.Run(progressStatus); } private Test _test2; private void cmdTest2_Click(object sender, EventArgs e) { _test2 = new Test(); _test2.Run(progressStatus); } private void progressStatus(string message) { if (this.InvokeRequired) { object[] args = { message }; this.Invoke( new Test.ProgressStatus( progressStatusInThisFormThread), args); } else progressStatusInThisFormThread(message); } private static object _statusMessageOutputLocker = new object(); private void progressStatusInThisFormThread(string message) { lock (_statusMessageOutputLocker) { txtLog.Text = String.Format("{0:hh:mm:ss.fff}: {1}", DateTime.Now, message) + System.Environment.NewLine + txtLog.Text; } } } } I will try to post about another one I use in the coming days... Thank you. -- Shamil -----Original Message----- So how do you folks display status information in a form? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sun Nov 29 15:51:48 2009 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 30 Nov 2009 00:51:48 +0300 Subject: [dba-VB] C# - Presenting process status In-Reply-To: <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> References: <4B12C2A2.1010001@colbyconsulting.com> <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> Message-ID: <00d301ca713e$28009bc0$7801d340$@spb.ru> OK, here is another sample more close to VB6/VBA WithEvents approach: using System; using System.Windows.Forms; namespace WinFormsSample { public partial class Form2 : Form { // Create Windows Form Form2 with three controls: // // 1. Button: cmdTest1 -> cmdTest1_Click // 2. Button: cmdTest2 -> cmdTest2_Click // 3. TextBox: txtLog (Multiline = true) #region Test class public class Test { public class ProgressStatusEventArgs : EventArgs { public string Message { get; set; } } public event EventHandler ProgressStatus; private static int _id; private System.Threading.Thread _thread; public void Run() { _testId = ++_id; System.Threading.ThreadStart ts = new System.Threading.ThreadStart(runThread); _thread = new System.Threading.Thread(ts); _thread.IsBackground = true; _thread.Start(); } private int _testId; private void runThread() { for (int i = 1; i < 10; i++) { string message = string.Format("Test#{0}, cycle#{1}", _testId, i); ProgressStatusEventArgs e = new ProgressStatusEventArgs(); e.Message = message; // raise event if (ProgressStatus != null) ProgressStatus(this, e); System.Threading.Thread.Sleep(500); } } } #endregion // Test class public Form2() { InitializeComponent(); } private Test _test1; private void cmdTest1_Click(object sender, EventArgs e) { _test1 = new Test(); _test1.ProgressStatus += new EventHandler(progressStatus); _test1.Run(); } private Test _test2; private void cmdTest2_Click(object sender, EventArgs e) { _test2 = new Test(); _test2.ProgressStatus += new EventHandler(progressStatus); _test2.Run(); } public delegate void ProgressStatusDelegate(string message); private void progressStatus(object sender, Test.ProgressStatusEventArgs e) { if (this.InvokeRequired) { object[] args = { e.Message }; this.Invoke( new ProgressStatusDelegate( progressStatusInThisFormThread), args); } else progressStatusInThisFormThread(e.Message); } private static object _statusMessageOutputLocker = new object(); private void progressStatusInThisFormThread(string message) { lock (_statusMessageOutputLocker) { txtLog.Text = String.Format("{0:hh:mm:ss.fff}: {1}", DateTime.Now, message) + System.Environment.NewLine + txtLog.Text; } } } } -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: Monday, November 30, 2009 12:33 AM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] C# - Presenting process status <<< I will try to post about another one I use in the coming days... >>> From jwcolby at colbyconsulting.com Sun Nov 29 22:09:43 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 29 Nov 2009 23:09:43 -0500 Subject: [dba-VB] C# - Presenting process status In-Reply-To: <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> References: <4B12C2A2.1010001@colbyconsulting.com> <00d201ca713b$8aeb5c50$a0c214f0$@spb.ru> Message-ID: <4B134587.9010005@colbyconsulting.com> Shamil, Thanks for this. I will need to study it. John W. Colby www.ColbyConsulting.com Shamil Salakhetdinov wrote: > Hi John, > > Here is one of the methods I use (there are zillion combinations) - I dunno > where it comes from, did I borrowed it from somewhere or I got it elaborated > myself, is it recommended way or not - but it just works, and I have > "cooked" it for you here from scratch - this is codebehind for Windows Form > named Form1: > > using System; > using System.Windows.Forms; > > namespace WinFormsSample > { > public partial class Form1 : Form > { > // Create Windows Form Form1 with three controls: > // > // 1. Button: cmdTest1 -> cmdTest1_Click > // 2. Button: cmdTest2 -> cmdTest2_Click > // 3. TextBox: txtLog (Multiline = true) > > #region Test class > public class Test > { > public delegate void ProgressStatus(string message); > > private static int _id; > private System.Threading.Thread _thread; > public void Run(ProgressStatus progressStatus) > { > _testId = ++_id; > _progressStatus = progressStatus; > > System.Threading.ThreadStart ts = > new System.Threading.ThreadStart(runThread); > _thread = new System.Threading.Thread(ts); > _thread.IsBackground = true; > _thread.Start(); > } > > private int _testId; > private ProgressStatus _progressStatus; > private void runThread() > { > for (int i = 1; i < 10; i++) > { > _progressStatus(string.Format("Test#{0}, cycle#{1}", > _testId, i)); > System.Threading.Thread.Sleep(500); > } > } > } > #endregion // Test class > > public Form1() > { > InitializeComponent(); > } > > private Test _test1; > private void cmdTest1_Click(object sender, EventArgs e) > { > _test1 = new Test(); > _test1.Run(progressStatus); > } > > private Test _test2; > private void cmdTest2_Click(object sender, EventArgs e) > { > _test2 = new Test(); > _test2.Run(progressStatus); > } > > private void progressStatus(string message) > { > if (this.InvokeRequired) > { > object[] args = { message }; > this.Invoke( > new Test.ProgressStatus( > progressStatusInThisFormThread), > args); > } > else > progressStatusInThisFormThread(message); > > } > > private static object _statusMessageOutputLocker = new object(); > private void progressStatusInThisFormThread(string message) > { > lock (_statusMessageOutputLocker) > { > txtLog.Text = > String.Format("{0:hh:mm:ss.fff}: {1}", > DateTime.Now, > message) + > System.Environment.NewLine + > txtLog.Text; > } > } > } > } > > I will try to post about another one I use in the coming days... > > Thank you. > > -- > Shamil > > -----Original Message----- > So how do you folks display status information in a form? From jwcolby at colbyconsulting.com Mon Nov 30 11:44:39 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 12:44:39 -0500 Subject: [dba-VB] C# - return value for stored procedures Message-ID: <4B140487.1030905@colbyconsulting.com> While I am getting back values in my "output" parameters for the SPs, I am getting a constant -1 for the return value. In the stored procedure I have tried just using RETURN 1234 and also creating an int variable, setting that to 1234 and returning that. declare @Ret int select @ret = 1234 RETURN @ret In all cases I get a -1 value which leads me to believe that the problem is on the C# side of things. My c# code looks like: try { if (sCmd.Connection.State != ConnectionState.Open) sCmd.Connection.Open(); int retValue = sCmd.ExecuteNonQuery(); return retValue; } catch (SqlException sqlEx) { throw sqlEx; } I am not throwing an error, the SP executes, but retValue is always -1. Any ideas? Do I need to add a SqlParameter to the Parameters collection for the return value? -- John W. Colby www.ColbyConsulting.com From james at fcidms.com Mon Nov 30 11:56:47 2009 From: james at fcidms.com (James Barash) Date: Mon, 30 Nov 2009 12:56:47 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <4B140487.1030905@colbyconsulting.com> Message-ID: <12D2F894654B53498DEF840AE12B7BB36B4B5797@fciexchange.fcidms.com> John: You do, indeed, need to define a Parameter for the return value. From the Microsoft documentation: -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, November 30, 2009 12:45 PM To: VBA Subject: [dba-VB] C# - return value for stored procedures While I am getting back values in my "output" parameters for the SPs, I am getting a constant -1 for the return value. In the stored procedure I have tried just using RETURN 1234 and also creating an int variable, setting that to 1234 and returning that. declare @Ret int select @ret = 1234 RETURN @ret In all cases I get a -1 value which leads me to believe that the problem is on the C# side of things. My c# code looks like: try { if (sCmd.Connection.State != ConnectionState.Open) sCmd.Connection.Open(); int retValue = sCmd.ExecuteNonQuery(); return retValue; } catch (SqlException sqlEx) { throw sqlEx; } I am not throwing an error, the SP executes, but retValue is always -1. Any ideas? Do I need to add a SqlParameter to the Parameters collection for the return value? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From james at fcidms.com Mon Nov 30 12:06:48 2009 From: james at fcidms.com (James Barash) Date: Mon, 30 Nov 2009 13:06:48 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <4B140487.1030905@colbyconsulting.com> Message-ID: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com> John: You do, indeed, need to define a Parameter for the return value. From the Microsoft documentation: SqlCommand.ExecuteNonQuery Method Executes a Transact-SQL statement against the connection and returns the number of rows affected. To add a Return Value parameter: SqlParameter ret = new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int); ret.Direction = ParameterDirection.ReturnValue; cmd.Parameters.Add(ret); Hope this helps. James Barash -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, November 30, 2009 12:45 PM To: VBA Subject: [dba-VB] C# - return value for stored procedures While I am getting back values in my "output" parameters for the SPs, I am getting a constant -1 for the return value. In the stored procedure I have tried just using RETURN 1234 and also creating an int variable, setting that to 1234 and returning that. declare @Ret int select @ret = 1234 RETURN @ret In all cases I get a -1 value which leads me to believe that the problem is on the C# side of things. My c# code looks like: try { if (sCmd.Connection.State != ConnectionState.Open) sCmd.Connection.Open(); int retValue = sCmd.ExecuteNonQuery(); return retValue; } catch (SqlException sqlEx) { throw sqlEx; } I am not throwing an error, the SP executes, but retValue is always -1. Any ideas? Do I need to add a SqlParameter to the Parameters collection for the return value? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Mon Nov 30 12:08:16 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 13:08:16 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <12D2F894654B53498DEF840AE12B7BB36B4B5797@fciexchange.fcidms.com> References: <12D2F894654B53498DEF840AE12B7BB36B4B5797@fciexchange.fcidms.com> Message-ID: <4B140A10.4020409@colbyconsulting.com> Thanks James. Your response did not provide a link or anything but knowing that I have to do this, I will go a Googling. Thanks, John W. Colby www.ColbyConsulting.com James Barash wrote: > John: > > You do, indeed, need to define a Parameter for the return value. From the Microsoft documentation: > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Monday, November 30, 2009 12:45 PM > To: VBA > Subject: [dba-VB] C# - return value for stored procedures > > While I am getting back values in my "output" parameters for the SPs, I am getting a constant -1 for the return value. > > In the stored procedure I have tried just using > > RETURN 1234 > > and also creating an int variable, setting that to 1234 and returning that. > > declare @Ret int > select @ret = 1234 > RETURN @ret > > In all cases I get a -1 value which leads me to believe that the problem is on the C# side of things. > > My c# code looks like: > > try > { > if (sCmd.Connection.State != ConnectionState.Open) > sCmd.Connection.Open(); > int retValue = sCmd.ExecuteNonQuery(); > return retValue; > } > catch (SqlException sqlEx) > { > throw sqlEx; > } > > I am not throwing an error, the SP executes, but retValue is always -1. > > Any ideas? Do I need to add a SqlParameter to the Parameters collection for the return value? > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Mon Nov 30 12:25:27 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 13:25:27 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com> References: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com> Message-ID: <4B140E17.3050400@colbyconsulting.com> James, So what does this do for me? int retValue = sCmd.ExecuteNonQuery(); From my Googling it seems like I have to do something like: retValue = Int32.Parse(sCmd.Parameters["@Return_Value"].Value.ToString()); which is working just fine. scmd.ExecuteNonQuery() returns a -1 (not sure why) and then the @Return_Value is parsed and returned. So that part is now working as well. Thanks again for the response James. John W. Colby www.ColbyConsulting.com James Barash wrote: > John: > > You do, indeed, need to define a Parameter for the return value. From the Microsoft documentation: > > SqlCommand.ExecuteNonQuery Method > > Executes a Transact-SQL statement against the connection and returns the number of rows affected. > > To add a Return Value parameter: > > SqlParameter ret = new SqlParameter("@RETURN_VALUE", System.Data.SqlDbType.Int); > ret.Direction = ParameterDirection.ReturnValue; > cmd.Parameters.Add(ret); > > Hope this helps. From jwcolby at colbyconsulting.com Mon Nov 30 13:55:11 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 14:55:11 -0500 Subject: [dba-VB] C# SQLCommand.Timeout Message-ID: <4B14231F.3020905@colbyconsulting.com> Any words of wisdom on setting the command.Timeout property? I have stored procedures that execute instantly, and others that can take an hour or more (large appends). How can I discover how long a stored procedure takes to run so that I can set this property dynamically? As an example I have cases where I am appending 65 million rows into a table of 6 fields. This can take a long time (at the very least perhaps 20 minutes) but if I could get a "RecordsAffected" count for such queries as well as the time it took to execute, then I could start to discover that it takes "X seconds / million" records, and set the timeout to a reasonable value based on the records to be appended. This doesn't have to be any exact time value. I assume a simple pair of time variables, then "stop time - start time"? Or should I just discover worst case and set it to that? -- John W. Colby www.ColbyConsulting.com From stuart at lexacorp.com.pg Mon Nov 30 15:07:43 2009 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Tue, 01 Dec 2009 07:07:43 +1000 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <4B140E17.3050400@colbyconsulting.com> References: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com>, <4B140E17.3050400@colbyconsulting.com> Message-ID: <4B14341F.21088.149E29F8@stuart.lexacorp.com.pg> -1 = True, meanimg that the sp executed with no errors? -- Stuart On 30 Nov 2009 at 13:25, jwcolby wrote: > which is working just fine. scmd.ExecuteNonQuery() returns a -1 (not sure why) and then the > @Return_Value is parsed and returned. > From jwcolby at colbyconsulting.com Mon Nov 30 15:35:18 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 16:35:18 -0500 Subject: [dba-VB] C# - Stored procedures working! Message-ID: <4B143A96.8080905@colbyconsulting.com> I would like to thank all of you for your assistance in getting my stored procedures working from C#. My strategy was to design a StoredProcedure class which contained all of the data and code required to run any of the stored procedures that I have written so far. We shall see whether I succeeded in that however I have a set of seven stored procedures that are required to export large tables in chunks to CSV files and I am successfully running all of those seven stored procedures and obtaining CSV files. This StoredProcedure class has a command object and a set of about eight "Parameter" methods which wrap the code for creating standard parameters used over and over in my various stored procedures, parameters like DBName, TblName, PKStart, PKend etc, as well as the return value and errorno and errormsg parameters. I then created an AccuzipExport class that "supervises" the Accuzip export process, i.e. holds all of the code to DEFINE these seven stored procedures, i.e. set up the required stored procedure class mentioned above for each of these seven SPs, and the specific parameters that each SP uses. This class then has a method that calls three of the stored procedures to create a standard named table, fill it with sorted data and index it in preparation for creating chunk files. After that the method sits in a while loop calling the remaining four stored procedures which create the chunk table, append a chunk of records to that, index it and then BCP it out. All of this stuff now works. This stuff was what the old Access database did, so I have essentially ported the Accuzip Export process to C#. However I have successfully returned a success/fail return value as well as an error code / message in the C# version, whereas I never had that ability in the Access version so I am now in some respects ahead of where I was in Access. There is more to do of course. Now that I have a robust dev platform at my fingertips I can log the process from beginning to end, errors, times for each SP to execute, stuff like that. And of course get the process status displaying on the form I use to start this process. Eventually I need to get this export process running in a thread so that I can use the big picture application for other things while this export process runs. But next comes the matching AccuzipInput class, which will pull the CSV files coming back from Accuzip back into SQL Server. With all of the grunt work done for executing Stored Procedures it should be relatively quick and easy to get the matching input class up. At any rate I wanted to say thanks to everyone for all the help and express how jazzed I am to be where I am today. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Mon Nov 30 15:37:43 2009 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 30 Nov 2009 16:37:43 -0500 Subject: [dba-VB] C# - return value for stored procedures In-Reply-To: <4B14341F.21088.149E29F8@stuart.lexacorp.com.pg> References: <12D2F894654B53498DEF840AE12B7BB36B4B5799@fciexchange.fcidms.com>, <4B140E17.3050400@colbyconsulting.com> <4B14341F.21088.149E29F8@stuart.lexacorp.com.pg> Message-ID: <4B143B27.9050902@colbyconsulting.com> Stuart, I don't think so as I have had failures and I *think* I got a -1 regardless. I will have to pursue this one. John W. Colby www.ColbyConsulting.com Stuart McLachlan wrote: > -1 = True, meanimg that the sp executed with no errors? >