From jwcolby at colbyconsulting.com Wed Mar 2 20:11:33 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 02 Mar 2011 21:11:33 -0500 Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews Message-ID: <4D6EF8D5.7060804@colbyconsulting.com> -- John W. Colby www.ColbyConsulting.com http://www.legitreviews.com/news/10164/ From accessd at shaw.ca Thu Mar 3 14:55:06 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Thu, 3 Mar 2011 12:55:06 -0800 Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews In-Reply-To: <4D6EF8D5.7060804@colbyconsulting.com> References: <4D6EF8D5.7060804@colbyconsulting.com> Message-ID: <7AF8F6950EC14685B928DE79F836CB0D@creativesystemdesigns.com> Soon to be into John's office. ;-) Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, March 02, 2011 6:12 PM To: Access Developers discussion and problem solving; Sqlserver-Dba; VBA Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews -- John W. Colby www.ColbyConsulting.com http://www.legitreviews.com/news/10164/ _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Thu Mar 3 15:05:20 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 03 Mar 2011 16:05:20 -0500 Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews In-Reply-To: <7AF8F6950EC14685B928DE79F836CB0D@creativesystemdesigns.com> References: <4D6EF8D5.7060804@colbyconsulting.com> <7AF8F6950EC14685B928DE79F836CB0D@creativesystemdesigns.com> Message-ID: <4D700290.4010308@colbyconsulting.com> I wish. It will most likely be beyond my means for some time. John W. Colby www.ColbyConsulting.com On 3/3/2011 3:55 PM, Jim Lawrence wrote: > Soon to be into John's office. ;-) > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, March 02, 2011 6:12 PM > To: Access Developers discussion and problem solving; Sqlserver-Dba; VBA > Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - > Legit Reviews > > From michael at ddisolutions.com.au Sun Mar 6 19:26:08 2011 From: michael at ddisolutions.com.au (Michael Maddison) Date: Mon, 7 Mar 2011 12:26:08 +1100 Subject: [dba-VB] Java Help? Message-ID: <99266C61B516644D9727F983FAFAB465046ABC@remote.ddisolutions.com.au> Hi Guys, A while ago I asked about creating a COM dll to be called from Java. Does anyone have the Java skills to see if you can call the dll to test for me? Cheers Michael M From dbdoug at gmail.com Sun Mar 6 21:04:47 2011 From: dbdoug at gmail.com (Doug Steele) Date: Sun, 6 Mar 2011 19:04:47 -0800 Subject: [dba-VB] ASP.Net development Message-ID: Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele From Gustav at cactus.dk Mon Mar 7 04:08:58 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 07 Mar 2011 11:08:58 +0100 Subject: [dba-VB] ASP.Net development Message-ID: Hi Doug I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. /gustav >>> dbdoug at gmail.com 07-03-2011 04:04 >>> Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele From accessd at shaw.ca Mon Mar 7 09:22:24 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Mon, 7 Mar 2011 07:22:24 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: Hi Gustav: Showing my ignorance about Winforms; does Winforms create browser based application or create a Windows based applications that uses a browser? Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Monday, March 07, 2011 2:09 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] ASP.Net development Hi Doug I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. /gustav >>> dbdoug at gmail.com 07-03-2011 04:04 >>> Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From dbdoug at gmail.com Mon Mar 7 09:55:51 2011 From: dbdoug at gmail.com (Doug Steele) Date: Mon, 7 Mar 2011 07:55:51 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: Hi Gustav: Thanks for the reply. If you build a table adapter which consists of data from more than one table (like a one to many relationship) in the design surface, VS2008 will refuse to create any CRUD query code - you have to build it yourself. Maybe you only used one-table adapters? Doug On Mon, Mar 7, 2011 at 2:08 AM, Gustav Brock wrote: > Hi Doug > > I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. > > I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. > You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. > > As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. > > /gustav > > >>>> dbdoug at gmail.com 07-03-2011 04:04 >>> > Hello All: > > A lot of the Microsoft sample code for asp.net data access is built > using strongly typed tableadapters with the .xsd design surface. ?I've > built two apps using these in the last year, but they seem to be kind > of fiddly - they don't work well with multi table update queries, for > instance, and they generate a HUGE amount of background source code. > It is nice to have the strong typing, however. > > I have never found any non-Microsoft sample code which uses this > technique. ?Admittedly, most of the sample code in the wild consists > of short snippets built to illustrate a single technique, not a > complete application. > > Two questions: > 1. Are there developers out there using this approach for real life apps? > 2. If not, what do you recommend? > > > Thanks, > > Doug Steele > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Mon Mar 7 11:04:59 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 07 Mar 2011 18:04:59 +0100 Subject: [dba-VB] ASP.Net development Message-ID: Hi Jim WinForms are true Windows desktop apps (I realise I should have changed the subject line, sorry). If you wish a browser based app, that's ASP.NET or Silverlight if we talk about Visual Studio and dotNet. /gustav >>> accessd at shaw.ca 07-03-2011 16:22 >>> Hi Gustav: Showing my ignorance about Winforms; does Winforms create browser based application or create a Windows based applications that uses a browser? Jim From Gustav at cactus.dk Mon Mar 7 11:17:36 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 07 Mar 2011 18:17:36 +0100 Subject: [dba-VB] ASP.Net development Message-ID: Hi Doug Now I see. Yes, for updates I use one-table tableadapters. Multi-table tableadapters are mainly used as source for reports and list-type forms where nothing is updated - and as source for combo- and listboxes. /gustav >>> dbdoug at gmail.com 07-03-2011 16:55 >>> Hi Gustav: Thanks for the reply. If you build a table adapter which consists of data from more than one table (like a one to many relationship) in the design surface, VS2008 will refuse to create any CRUD query code - you have to build it yourself. Maybe you only used one-table adapters? Doug On Mon, Mar 7, 2011 at 2:08 AM, Gustav Brock wrote: > Hi Doug > > I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. > > I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. > You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. > > As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. > > /gustav > > >>>> dbdoug at gmail.com 07-03-2011 04:04 >>> > Hello All: > > A lot of the Microsoft sample code for asp.net data access is built > using strongly typed tableadapters with the .xsd design surface. I've > built two apps using these in the last year, but they seem to be kind > of fiddly - they don't work well with multi table update queries, for > instance, and they generate a HUGE amount of background source code. > It is nice to have the strong typing, however. > > I have never found any non-Microsoft sample code which uses this > technique. Admittedly, most of the sample code in the wild consists > of short snippets built to illustrate a single technique, not a > complete application. > > Two questions: > 1. Are there developers out there using this approach for real life apps? > 2. If not, what do you recommend? > > > Thanks, > > Doug Steele From accessd at shaw.ca Mon Mar 7 11:52:58 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Mon, 7 Mar 2011 09:52:58 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: <9F8E42F58F6F45B18FBDE74842047BF9@creativesystemdesigns.com> Hi Gustav: Thanks for the information. Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Monday, March 07, 2011 9:05 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] ASP.Net development Hi Jim WinForms are true Windows desktop apps (I realise I should have changed the subject line, sorry). If you wish a browser based app, that's ASP.NET or Silverlight if we talk about Visual Studio and dotNet. /gustav >>> accessd at shaw.ca 07-03-2011 16:22 >>> Hi Gustav: Showing my ignorance about Winforms; does Winforms create browser based application or create a Windows based applications that uses a browser? Jim _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Mon Mar 7 12:45:14 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 7 Mar 2011 21:45:14 +0300 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: <5C52BAB72E1246CFBBFB629233371E72@nant> Hi Doug -- I think I have seen an ASP.NET book with sources where strongly typed datasets were used but I can't find it now. But maybe that was this book: Pro ASP.NET 2.0 E-Commerce in C# 2005 (Expert's Voice in .NET) http://www.amazon.com/ASP-NET-Commerce-2005-Experts-Voice/dp/1590597249/ref= pd_sim_b_8 It does have complete source code and the DAL code does use ADO.NET DataSets but not strongly typed. I have got programmed rather large ASP.NET application - more than 150 webforms with I suppose more than 70-80 heavily related MS SQL tables with millions rows in some of tables, hundreds SPs, ... several thousands C# custom classes, ORM layer etc. and this web application does work well forth year in row with a few support work - the latter mainly being adding new functionality. I have used asp.net development approach described in this book: ASP.NET 2.0 Website Programming: Problem - Design - Solution (Programmer to Programmer) http://www.amazon.com/ASP-NET-2-0-Website-Programming-Programmer/dp/07645846 42/ref=sr_1_2?s=books&ie=UTF8&qid=1299518125&sr=1-2 BTW, the same approach is used in DotNetNuke (but I didn't know that when I have got started to develop my customer web application I mentioned above.)) I have had to develop custom ORM mapping + SQL code generator to implement this approach. But it should be possible to use some existing code generators. This approach looks "low level" and it needs quite some coding (code generators help here) but good news is that it will never get outdated. And here is a book on how to use LINQ 2 SQL for ASP.NET applications (I haven't read and I haven't tried this approach) for the same sample ASP.NET application, which is used in the previously referred book: LINQ-ifying TheBeerHouse: An N-Tier LINQ Web Application with ASP.NET 2.0 Website Programming Problem Design Solution (Wrox Briefs) http://www.amazon.com/LINQ-ifying-TheBeerHouse-Application-Programming-Solut ion/dp/B0026A6AQU/ref=sr_1_106?s=books&ie=UTF8&qid=1299518458&sr=1-106 The most modern approach is to use ADO.NET Entity Framework for DAL as Gustav noted, and when ADO.NET EF is used then it should be possible to map logical/business layer objects to the custom stored procedures, which can insert, update, delete, select data from one or database tables... I haven't yet seen any books where ADO.NET Entity Framework is used with classical WebForms based ASP.NET applications - anybody? Some more information on ADO.NET EF and ASP.NET: - here seems to be a good book on ADO.NET EF: Entity Framework 4.0 Recipes: A Problem-Solution Approach (Expert's Voice in .NET) http://www.amazon.com/Entity-Framework-4-0-Recipes-Problem-Solution/dp/14302 27036/ref=sr_1_168?s=books&ie=UTF8&qid=1299519467&sr=1-168 - and this one "opens secretes" on how to build ultra-fast and robust asp.net applications: Ultra-Fast ASP.NET: Build Ultra-Fast and Ultra-Scalable web sites using ASP.NET and SQL Server [Paperback] http://www.amazon.com/Ultra-Fast-ASP-NET-Build-Ultra-Scalable-Server/dp/1430 223839/ref=sr_1_121?s=books&ie=UTF8&qid=1299518863&sr=1-121 Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Doug Steele Sent: 7 ????? 2011 ?. 6:05 To: dba-VB at databaseadvisors.com Subject: [dba-VB] ASP.Net development Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From dbdoug at gmail.com Mon Mar 7 23:34:51 2011 From: dbdoug at gmail.com (Doug Steele) Date: Mon, 7 Mar 2011 21:34:51 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: <5C52BAB72E1246CFBBFB629233371E72@nant> References: <5C52BAB72E1246CFBBFB629233371E72@nant> Message-ID: Hi Shamil: Thanks very much for your reply. I'll be checking your references out.... Doug On Mon, Mar 7, 2011 at 10:45 AM, Shamil Salakhetdinov wrote: > Hi Doug -- > > I think I have seen an ASP.NET book with sources where strongly typed > datasets were used but I can't find it now. > But maybe that was this book: > > Pro ASP.NET 2.0 E-Commerce in C# 2005 (Expert's Voice in .NET) > http://www.amazon.com/ASP-NET-Commerce-2005-Experts-Voice/dp/1590597249/ref= > pd_sim_b_8 > > It does have complete source code and the DAL code does use ADO.NET DataSets > but not strongly typed. > > I have got programmed rather large ASP.NET application - more than 150 > webforms with I suppose more than 70-80 heavily related MS SQL tables with > millions rows in some of tables, hundreds SPs, ... several thousands C# > custom classes, ORM layer etc. and this web application does work well forth > year in row with a few support work - the latter mainly being adding new > functionality. I have used asp.net development approach described in this > book: > > ASP.NET 2.0 Website Programming: Problem - Design - Solution (Programmer to > Programmer) > http://www.amazon.com/ASP-NET-2-0-Website-Programming-Programmer/dp/07645846 > 42/ref=sr_1_2?s=books&ie=UTF8&qid=1299518125&sr=1-2 > > BTW, the same approach is used in DotNetNuke (but I didn't know that when I > have got started to develop my customer web application I mentioned above.)) > > I have had to develop custom ORM mapping + SQL code generator to implement > this approach. > But it should be possible to use some existing code generators. > > This approach looks "low level" and it needs quite some coding (code > generators help here) but good news is that it will never get outdated. > > And here is a book on how to use LINQ 2 SQL for ASP.NET applications (I > haven't read and I haven't tried this approach) for the same sample ASP.NET > application, which is used in the previously referred book: > > LINQ-ifying TheBeerHouse: An N-Tier LINQ Web Application with ASP.NET 2.0 > Website Programming Problem Design Solution (Wrox Briefs) > http://www.amazon.com/LINQ-ifying-TheBeerHouse-Application-Programming-Solut > ion/dp/B0026A6AQU/ref=sr_1_106?s=books&ie=UTF8&qid=1299518458&sr=1-106 > > The most modern approach is to use ADO.NET Entity Framework for DAL as > Gustav noted, and when ADO.NET EF is used then it should be possible to map > logical/business layer objects to the custom stored procedures, which can > insert, update, delete, select data from one or database tables... > > I haven't yet seen any books where ADO.NET Entity Framework is used with > classical WebForms based ASP.NET applications - anybody? > > > Some more information on ADO.NET EF and ASP.NET: > > - here seems to be a good book on ADO.NET EF: > > Entity Framework 4.0 Recipes: A Problem-Solution Approach (Expert's Voice in > .NET) > http://www.amazon.com/Entity-Framework-4-0-Recipes-Problem-Solution/dp/14302 > 27036/ref=sr_1_168?s=books&ie=UTF8&qid=1299519467&sr=1-168 > > - and this one "opens secretes" on how to build ultra-fast and robust > asp.net applications: > > Ultra-Fast ASP.NET: Build Ultra-Fast and Ultra-Scalable web sites using > ASP.NET and SQL Server [Paperback] > http://www.amazon.com/Ultra-Fast-ASP-NET-Build-Ultra-Scalable-Server/dp/1430 > 223839/ref=sr_1_121?s=books&ie=UTF8&qid=1299518863&sr=1-121 > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Doug Steele > Sent: 7 ????? 2011 ?. 6:05 > To: dba-VB at databaseadvisors.com > Subject: [dba-VB] ASP.Net development > > Hello All: > > A lot of the Microsoft sample code for asp.net data access is built using > strongly typed tableadapters with the .xsd design surface. ?I've built two > apps using these in the last year, but they seem to be kind of fiddly - they > don't work well with multi table update queries, for instance, and they > generate a HUGE amount of background source code. > It is nice to have the strong typing, however. > > I have never found any non-Microsoft sample code which uses this technique. > Admittedly, most of the sample code in the wild consists of short snippets > built to illustrate a single technique, not a complete application. > > Two questions: > 1. Are there developers out there using this approach for real life apps? > 2. If not, what do you recommend? > > > Thanks, > > Doug Steele > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From raibeart at gmail.com Tue Mar 8 11:44:25 2011 From: raibeart at gmail.com (Robert Stewart) Date: Tue, 08 Mar 2011 11:44:25 -0600 Subject: [dba-VB] I'm getting nowhere In-Reply-To: References: Message-ID: <4d766a81.cac4e60a.3ab2.255f@mx.google.com> John, Logins: If you are in mixed mode, which for your application you should be, every "group" will have it's own login with a password. User: References a login. A user is given rights to a database and specific objects in a database. Roles: Generally, you can assign DataReader and DataWriter roles to a User in a database to give them CRUD rights to tables. You will need to GRANT EXECUTE rights to stored procedures. Does that help? Robert At 09:22 AM 3/7/2011, you wrote: >From: dba-vb-bounces at databaseadvisors.com >[mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >Sent: Saturday, February 19, 2011 2:54 PM >To: Sqlserver-Dba; VBA; Access Developers discussion and problem solving >Subject: [dba-VB] I'm getting nowhere > >I am getting nowhere on understanding SQL Server security. Microsoft >provides us with SQL Server Express which implies that joe blow (me) is >going to install / maintain it. > >I am not a SQL Server Admin and I cannot afford to spend the time to be one. > >Google is my friend. BOL is not. > >Except that Google is taking me to these places where I am expected to >already know how this stuff works, and then wants to make me a *better* >administrator. Which of course is useless because I am not an administrator >at all. > >OTOH I am not stupid. If I could find something that started at the "This >is SQL Server security" >basics I could learn this stuff. Before anyone says "RTFM (BOL)" let me >simply say, "not happening". I have tried BOL and it simply sucks for my >level of expertise (my opinion of course). > If that is your advice, simply stay out of this thread. Thanks! > >So... my needs: > >I need to set up several SQL Server databases for use by different, very >small groups (5-20 people) of entirely unrelated people. What I mean by >that is that each DB is for a different "company" if you will. I need to >access these databases from C#. I understand the group / user paradigm. I >would like to create groups and users. Specific groups can do specific >things in the database, some can see data but not modify it. Some can add >records in specific tables but not others. Some can run reports (view). > >I do *NOT* want to create windows level groups and users if I can avoid it. >These are people that I do not necessarily know and I do not want to give >them any rights at the machine level, and I prefer to not maintain such >lists at the machine level. > >Unfortunately SQL Server does not seem to model Groups / users. I go into >SQL Server and see a security tab. It has "logins". Is that a user? A >specific ability to log in with a password? To what? The server itself? A >specific database? Groups of databases? > >I see "roles" but these appear to be aimed at the server and none of these >people are going to be doing anything at the server level. > >Can I safely ignore everything under the server security tab? > >I go to a database and I see a security tab. It has users and roles. >Hmm... better (I would think). I would like to add users "under" the >specific database that the user will access. > >So I try to add a new user but I do not see anywhere to require a password. >Hmmm... > >I go into roles and I do not see any predefined role that looks like it >would be useful to me in meeting my needs described above. If I look at >"add new role" it asks for a password. The User / group model does nto >assign passwords at the group level which implies that a role is not a group >at the user / group paradigm. > >Is it just me, or is SQL Server security just... different? Am I correct in >assuming that it doesn't implement a user / group paradigm? > >And more importantly, where can I go to get a plain, simple, English >description of how this mess works? > >And please excuse the tone that results from my frustration. The only help >documents that I have found (and I have extensive lists of bookmarked web >pages) so far assume that I am an administrator. > I am not, and cannot afford to become one. And yet MS pushes SQL Express >as if I (non-admin) should be able to use this as a data store pool. > >Help! > >-- >John W. Colby >www.ColbyConsulting.com >_______________________________________________ From jwcolby at colbyconsulting.com Thu Mar 10 07:12:44 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 08:12:44 -0500 Subject: [dba-VB] c# lock() Message-ID: <4D78CE4C.5000308@colbyconsulting.com> I am using multi-threading and need to lock list objects so that two different threads can access them. Do I even need to lock the list inside of the property as follows: public Dictionary pLstFiles { get { lock (lstFiles) { return lstFiles; } } } or only when I actually access the list? If I create a property to return the list (without locking it in the property) public Dictionary pLstFiles { get { return lstFiles; } } and I then get a pointer to the list using that property Dictionary LstFiles; LstFiles = SomeObj.pLstFiles; and I then lock the pointer while using it lock(LstFiles) { do stuff to list here... } Does that in fact lock the list to other objects doing the same thing (getting the list through that property)? IOW is the lock attribute affixed to the list at the list itself so that anything trying to use the list anywhere in the program can see that the list is locked? -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Thu Mar 10 10:29:39 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 11:29:39 -0500 Subject: [dba-VB] split an SVN database Message-ID: <4D78FC73.6000202@colbyconsulting.com> I use SVN. I am looking at doing a major split of a project into two projects. I want to leave the existing solution until I have one half of the current project carved out, debugged and running and in it's own SVN database. However I don't think I want to carry along the baggage of the old revisions into the new database. How do I go about this? -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Thu Mar 10 14:37:01 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 15:37:01 -0500 Subject: [dba-VB] c# - controlling the run state / status of your threaded classes Message-ID: <4D79366D.8060905@colbyconsulting.com> As I write classes that consume threads, I need to be able to start and stop these threads and the class that consumes the threads. For example, I open a form and load a class instance that starts performing some process - exporting sets of records to csv files for example. Once that class is loaded and processing, I need to be able to ask that class to stop exporting files. However there may be a time lag between asking and the currently processing file finishing it's export. That is precisely the purpose of a thread - tell it to go and do something else. So (for example) I have a check box that I check that tells the class to start exporting. When it starts it sets a flag saying that it is running and places text into the check box label saying that it is exporting (busy). If I then uncheck the check box I am telling the class to stop exporting. The class is monitoring the check box and will not process the next file but it can't just exit. So it needs to set a flag saying that it is stopping and display "stopping" on the check box label. When it finishes exporting the file, it sets a stopped flag and says displays "terminated" in the check box label. So we have a couple of "commands" (start / stop) and a few statuses (stopped, stopping, running). I would like to "standardize" this into a class so that any class which needs this interface does it in the same way. Some controlling object can tell the object to start and stop, and the class can tell the controlling object that is is stopped, stopping or running. this class does not6 need to do any updating of labels or anything like that, just hold the variables, properties, methods and code to perform this in a standard way. Is anyone doing this? Have existing code you would like to share? Have existing experience trying to do this you would like to share? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Thu Mar 10 15:35:24 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 00:35:24 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D78CE4C.5000308@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com> Message-ID: <3693FCAC03E94BCA9C150F6E7AFE4A71@nant> Hi John -- <<< Does that in fact lock the list to other objects doing the same thing (getting the list through that property)? >>> You have to lock/synchronize/serialize not the list itself but simultaneous access to the list from different threads. You'd better use special static object variable to implement locking. I will post sample code in my following posting here... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 16:13 To: VBA Subject: [dba-VB] c# lock() I am using multi-threading and need to lock list objects so that two different threads can access them. Do I even need to lock the list inside of the property as follows: public Dictionary pLstFiles { get { lock (lstFiles) { return lstFiles; } } } or only when I actually access the list? If I create a property to return the list (without locking it in the property) public Dictionary pLstFiles { get { return lstFiles; } } and I then get a pointer to the list using that property Dictionary LstFiles; LstFiles = SomeObj.pLstFiles; and I then lock the pointer while using it lock(LstFiles) { do stuff to list here... } Does that in fact lock the list to other objects doing the same thing (getting the list through that property)? IOW is the lock attribute affixed to the list at the list itself so that anything trying to use the list anywhere in the program can see that the list is locked? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Thu Mar 10 15:36:58 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 00:36:58 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D78CE4C.5000308@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com> Message-ID: <4889C56E6FF246D6AB908FEFD1069A68@nant> Hi John -- Here is sample code I promised to post. using System; using System.Collections.Generic; namespace ConsoleApplication1 { class Log { public static bool On { get; set; } public static void WriteLine(string message) { WriteLine(message, false); } public static void WriteLine(string message, bool forceLogging) { if (Log.On || forceLogging) System.Console.WriteLine(message); } } class MyObject { static int _newId; public MyObject() { _id = System.Threading.Interlocked.Increment(ref _newId); _timeStamp = DateTime.Now; } int _id; public int ID { get { return _id; } set { _id = value; } } DateTime _timeStamp; public DateTime TimeStamp { get { return _timeStamp; } set { _timeStamp = value; } } } class MyObjectsStore { private List _keys = new List(); private Dictionary _myObjects = new Dictionary(); public MyObject this[int key] { get { MyObject myObject = null; lock (_myObjectsLocker) { _myObjects.TryGetValue(key, out myObject); } return myObject; } } public int Count { get { lock (_myObjectsLocker) { return _myObjects.Count; } } } private static object _myObjectsLocker = new object(); public void Add(MyObject myObject) { lock (_myObjectsLocker) { _myObjects.Add(myObject.ID, myObject); _keys.Add(myObject.ID); } } public MyObject Remove(int id) { MyObject myobject = null; lock (_myObjectsLocker) { myobject = _myObjects[id]; _myObjects.Remove(id); } return myobject; } static Random _rnd = new Random(); public MyObject RemoveRandomObject() { MyObject myobject = null; int index = -1; int key = -1; try { lock (_myObjectsLocker) { index = _rnd.Next(_keys.Count - 1); key = _keys[index]; myobject = Remove(key); _keys.Remove(key); } } catch (Exception ex) { Log.WriteLine(string.Format("[{0}]: RemoveRandomObject: {1}, index = {2}, key = {3}", System.Threading.Thread.CurrentThread.ManagedThreadId, ex.Message, index , key)); } return myobject; } } class Program { public const int OBJECTS_PER_THREAD_COUNT = 7; public const int REMOVAL_MAX_RETRIES_COUNT = 5; public static void AddObjectsTest() { for (int i = 1; i <= OBJECTS_PER_THREAD_COUNT; i++) { MyObject myObject = new MyObject(); _store.Add(myObject); Log.WriteLine(string.Format("[{0}]: Object Added - ID = {1}, TS = {2}", System.Threading.Thread.CurrentThread.ManagedThreadId, myObject.ID, myObject.TimeStamp.Ticks)); System.Threading.Thread.Sleep(2); System.Threading.Interlocked.Increment(ref _createdObjectsCount); } System.Threading.Interlocked.Decrement(ref _threadsCount); } public static void RemoveObjectsTest() { for (int i = 1; i <= OBJECTS_PER_THREAD_COUNT; i++) { int retryCount = REMOVAL_MAX_RETRIES_COUNT; retry: MyObject myObject = _store.RemoveRandomObject(); if (myObject != null) { Log.WriteLine(string.Format("[{0}]: Object Removed - ID = {1}, TS = {2}", System.Threading.Thread.CurrentThread.ManagedThreadId, myObject.ID, myObject.TimeStamp.Ticks)); System.Threading.Thread.Sleep(2); } System.Threading.Thread.Sleep(2); if (myObject == null && --retryCount >= 0) { Log.WriteLine(string.Format("[{0}]: Object Removal failed, retry attempt# {1}", System.Threading.Thread.CurrentThread.ManagedThreadId, REMOVAL_MAX_RETRIES_COUNT-retryCount)); System.Threading.Interlocked.Increment(ref _totalRetriesCount); goto retry; } else if (myObject == null) { Log.WriteLine(string.Format("[{0}]: Object Removal failed", System.Threading.Thread.CurrentThread.ManagedThreadId)); System.Threading.Interlocked.Increment(ref _failedObjectRemovalsCount); } else System.Threading.Interlocked.Increment(ref _removedObjectsCount); } System.Threading.Interlocked.Decrement(ref _threadsCount); } static int _createdObjectsCount; static int _removedObjectsCount; static int _totalRetriesCount; static int _failedObjectRemovalsCount; static int _threadsCount; static MyObjectsStore _store; const int THREADS_COUNT = 20; //000; static void Main(string[] args) { Log.On = true; Log.WriteLine(string.Format("{0}: Test Started.", DateTime.Now)); try { Log.On = false; _store = new MyObjectsStore(); for (int i = 1; i <= THREADS_COUNT; i++) { if (i % 2 == 1) new System.Threading.Thread( new System.Threading.ThreadStart(AddObjectsTest)) .Start(); else new System.Threading.Thread( new System.Threading.ThreadStart(RemoveObjectsTest)) .Start(); _threadsCount++; } while (_threadsCount > 0) System.Threading.Thread.Sleep(100); System.Threading.Thread.Sleep(100); Log.On = true; Log.WriteLine(string.Format("Total Threads Count = {0}", THREADS_COUNT)); Log.WriteLine(string.Format("Store has {0} objects, Created = {1}, Removed = {2}", _store.Count, _createdObjectsCount, _removedObjectsCount)); Log.WriteLine(string.Format("RemovalRetriesCount = {0}, FailedRemovalsCount = {1}", _totalRetriesCount, _failedObjectRemovalsCount)); } catch (Exception ex) { Log.On = true; Log.WriteLine(string.Format("Program.Main: " + ex.Message)); } finally { Log.On = true; } Log.WriteLine(string.Format("{0}: Test Finished.", DateTime.Now)); } } } Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 16:13 To: VBA Subject: [dba-VB] c# lock() <<< snip >>> From shamil at smsconsulting.spb.ru Thu Mar 10 15:38:46 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 00:38:46 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D78CE4C.5000308@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com> Message-ID: <50A8FE7A468B41D48B9CC4F729CB02E3@nant> Hi John -- Here are my test results for sample code I've just posted (hopefully it get through AccessD - it was a bit lengthy) //11/03/2011 0:32:50: Test Started. //Total Threads Count = 20 //Store has 0 objects, Created = 70, Removed = 70 //RemovalRetriesCount = 0, FailedRemovalsCount = 0 //11/03/2011 0:32:50: Test Finished. //11/03/2011 0:32:25: Test Started. //Total Threads Count = 200 //Store has 0 objects, Created = 700, Removed = 700 //RemovalRetriesCount = 0, FailedRemovalsCount = 0 //11/03/2011 0:32:25: Test Finished. //11/03/2011 0:31:32: Test Started. //Total Threads Count = 20000 //Store has 19 objects, Created = 70000, Removed = 69981 //RemovalRetriesCount = 295, FailedRemovalsCount = 19 //11/03/2011 0:31:43: Test Finished. //11/03/2011 0:33:32: Test Started. //[3]: Object Added - ID = 1, TS = 634354004122968750 //[5]: Object Added - ID = 2, TS = 634354004122968750 //[3]: Object Added - ID = 3, TS = 634354004122968750 //[4]: Object Removed - ID = 2, TS = 634354004122968750 //[5]: Object Added - ID = 4, TS = 634354004122968750 //[6]: Object Removed - ID = 3, TS = 634354004122968750 //[7]: Object Added - ID = 5, TS = 634354004122968750 //[8]: Object Removed - ID = 4, TS = 634354004122968750 //[9]: Object Added - ID = 6, TS = 634354004122968750 //[10]: Object Removed - ID = 1, TS = 634354004122968750 //[11]: Object Added - ID = 7, TS = 634354004122968750 //[3]: Object Added - ID = 8, TS = 634354004122968750 //[12]: Object Removed - ID = 7, TS = 634354004122968750 //[5]: Object Added - ID = 9, TS = 634354004122968750 //[7]: Object Added - ID = 11, TS = 634354004122968750 //[9]: Object Added - ID = 10, TS = 634354004122968750 //[4]: Object Removed - ID = 6, TS = 634354004122968750 //[11]: Object Added - ID = 12, TS = 634354004122968750 //[3]: Object Added - ID = 13, TS = 634354004122968750 //[7]: Object Added - ID = 16, TS = 634354004123125000 //[6]: Object Removed - ID = 13, TS = 634354004122968750 //[11]: Object Added - ID = 17, TS = 634354004123125000 //[9]: Object Added - ID = 15, TS = 634354004123125000 //[4]: Object Removed - ID = 16, TS = 634354004123125000 //[8]: Object Removed - ID = 11, TS = 634354004122968750 //[10]: Object Removed - ID = 8, TS = 634354004122968750 //[5]: Object Added - ID = 14, TS = 634354004122968750 //[11]: Object Added - ID = 21, TS = 634354004123125000 //[3]: Object Added - ID = 18, TS = 634354004123125000 //[7]: Object Added - ID = 19, TS = 634354004123125000 //[10]: Object Removed - ID = 15, TS = 634354004123125000 //[6]: Object Removed - ID = 5, TS = 634354004122968750 //[4]: Object Removed - ID = 19, TS = 634354004123125000 //[8]: Object Removed - ID = 9, TS = 634354004122968750 //[10]: Object Removed - ID = 21, TS = 634354004123125000 //[6]: Object Removed - ID = 17, TS = 634354004123125000 //[8]: Object Removed - ID = 12, TS = 634354004122968750 //[4]: Object Removed - ID = 22, TS = 634354004123281250 //[12]: Object Removed - ID = 10, TS = 634354004122968750 //[6]: Object Removed - ID = 18, TS = 634354004123125000 //[4]: Object Removed - ID = 24, TS = 634354004123437500 //[8]: Object Removed - ID = 20, TS = 634354004123125000 //[6]: Object Removed - ID = 23, TS = 634354004123437500 //[12]: Object Removed - ID = 25, TS = 634354004123437500 //[10]: Object Removed - ID = 14, TS = 634354004122968750 //[9]: Object Added - ID = 20, TS = 634354004123125000 //[5]: Object Added - ID = 22, TS = 634354004123281250 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[10]: Object Removed - ID = 26, TS = 634354004124062500 //[6]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 1 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[10]: Object Removed - ID = 27, TS = 634354004124062500 //[3]: Object Added - ID = 24, TS = 634354004123437500 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[6]: Object Removal failed, retry attempt# 1 //[6]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 2 //[4]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[8]: Object Removal failed, retry attempt# 1 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[11]: Object Added - ID = 23, TS = 634354004123437500 //[7]: Object Added - ID = 25, TS = 634354004123437500 //[6]: Object Removal failed, retry attempt# 2 //[12]: Object Removed - ID = 28, TS = 634354004124531250 //[4]: Object Removal failed, retry attempt# 1 //[8]: Object Removal failed, retry attempt# 2 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[6]: Object Removed - ID = 29, TS = 634354004125000000 //[4]: Object Removed - ID = 30, TS = 634354004125000000 //[8]: Object Removal failed, retry attempt# 3 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 1 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[8]: Object Removal failed, retry attempt# 4 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 2 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[3]: Object Added - ID = 28, TS = 634354004124531250 //[12]: Object Removal failed, retry attempt# 3 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[8]: Object Removal failed, retry attempt# 5 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 4 //[7]: Object Added - ID = 30, TS = 634354004125000000 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[5]: Object Added - ID = 27, TS = 634354004124062500 //[9]: Object Added - ID = 26, TS = 634354004124062500 //[12]: Object Removal failed, retry attempt# 5 //[12]: Object Removed - ID = 31, TS = 634354004126093750 //[8]: Object Removal failed //[8]: Object Removed - ID = 32, TS = 634354004126250000 //[9]: Object Added - ID = 33, TS = 634354004126250000 //[12]: Object Removed - ID = 33, TS = 634354004126250000 //[11]: Object Added - ID = 29, TS = 634354004125000000 //[9]: Object Added - ID = 34, TS = 634354004126406250 //[12]: Object Removed - ID = 34, TS = 634354004126406250 //[7]: Object Added - ID = 31, TS = 634354004126093750 //[5]: Object Added - ID = 32, TS = 634354004126250000 //[11]: Object Added - ID = 35, TS = 634354004126406250 //Total Threads Count = 10 //Store has 1 objects, Created = 35, Removed = 34 //RemovalRetriesCount = 15, FailedRemovalsCount = 1 //11/03/2011 0:33:32: Test Finished. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 16:13 To: VBA Subject: [dba-VB] c# lock() <<< snip >>> From jwcolby at colbyconsulting.com Thu Mar 10 16:27:16 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 17:27:16 -0500 Subject: [dba-VB] c# lock() In-Reply-To: <3693FCAC03E94BCA9C150F6E7AFE4A71@nant> References: <4D78CE4C.5000308@colbyconsulting.com> <3693FCAC03E94BCA9C150F6E7AFE4A71@nant> Message-ID: <4D795044.6080808@colbyconsulting.com> Shamil, Thanks for the code but I think it is overkill for my needs so far, though I will put it in the archives for when I need it. In my system so far I have (or can arrange to have ) only two threads trying to access a list. Basically what I am doing is reading records out of sql server into a class factory, and placing the class instances into a list. Another object with its own thread is coming by and looking at the objects in the list to see if there is any work for it to do. If it finds an object to work on, it grabs a pointer to that item in the list and stores it in a pointer and releases the list. One of the ways I saw this handled was to create factory which creates a brand new list object, grabs a pointer to the list, locks that and then copies the objects in the list pointers to classes) into that new list, unlocks the list and hands back a pointer to this brand new list. thus the new list could be iterated at will. I am just trying to lock the list while one thread adds objects to the list or another thread gets objects out of the list. So if I have object A which contains a list and is adding / deleting objects to that list... and object B which needs to iterate through the list looking for an object of interest in the list, and grabbing a pointer to that object, there is no way to do this without all of the code you wrote? John W. Colby www.ColbyConsulting.com On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: > Hi John -- > > <<< > Does that in fact lock the list to other objects doing the same thing > (getting the list through that property)? >>>> > You have to lock/synchronize/serialize not the list itself but simultaneous > access to the list from different threads. > You'd better use special static object variable to implement locking. > > I will post sample code in my following posting here... > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Thu Mar 10 16:51:26 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 01:51:26 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D795044.6080808@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com> Message-ID: <3B9DE744CEA244829179A81CA818F92F@nant> Hi John -- Do you you need add, remove and iterate a list to get an object or you need to add, remove and get an object by its key? - if the latter - here is the code you can use: private Dictionary _myObjects = new Dictionary(); private static object _myObjectsLocker = new object(); public void Add(MyObject myObject) { lock (_myObjectsLocker) { _myObjects.Add(myObject.ID, myObject); _keys.Add(myObject.ID); } } public MyObject Remove(int id) { MyObject myobject = null; lock (_myObjectsLocker) { myobject = _myObjects[id]; _myObjects.Remove(id); } return myobject; } public MyObject this[int key] { get { MyObject myObject = null; lock (_myObjectsLocker) { _myObjects.TryGetValue(key, out myObject); } return myObject; } } for the case of iteration just replace: _myObjects.TryGetValue(key, out myObject); with your iteration code. As for the sample code - changing qty of threads and changing total computer workload (outer context for the test code) you can see how different could be final results when heavy multi-threading is used. And even if it's not heavy - it still can result in weird output if special multi-threading coding technique aren't used... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 1:27 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, Thanks for the code but I think it is overkill for my needs so far, though I will put it in the archives for when I need it. In my system so far I have (or can arrange to have ) only two threads trying to access a list. Basically what I am doing is reading records out of sql server into a class factory, and placing the class instances into a list. Another object with its own thread is coming by and looking at the objects in the list to see if there is any work for it to do. If it finds an object to work on, it grabs a pointer to that item in the list and stores it in a pointer and releases the list. One of the ways I saw this handled was to create factory which creates a brand new list object, grabs a pointer to the list, locks that and then copies the objects in the list pointers to classes) into that new list, unlocks the list and hands back a pointer to this brand new list. thus the new list could be iterated at will. I am just trying to lock the list while one thread adds objects to the list or another thread gets objects out of the list. So if I have object A which contains a list and is adding / deleting objects to that list... and object B which needs to iterate through the list looking for an object of interest in the list, and grabbing a pointer to that object, there is no way to do this without all of the code you wrote? John W. Colby www.ColbyConsulting.com On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: > Hi John -- > > <<< > Does that in fact lock the list to other objects doing the same thing > (getting the list through that property)? >>>> > You have to lock/synchronize/serialize not the list itself but > simultaneous access to the list from different threads. > You'd better use special static object variable to implement locking. > > I will post sample code in my following posting here... > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Thu Mar 10 17:11:33 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 02:11:33 +0300 Subject: [dba-VB] c# - controlling the run state / status of your threadedclasses In-Reply-To: <4D79366D.8060905@colbyconsulting.com> References: <4D79366D.8060905@colbyconsulting.com> Message-ID: <3B44AE877A14441CA13A51D1BA80FF65@nant> Hi John -- <<< So we have a couple of "commands" (start / stop) and a few statuses (stopped, stopping, running). >>> You'd better also add 'paused' (and maybe 'pausing' and 'resuming') for your list of states. I must note it's not easy to "standardize" such a coding - as with every framework/standard there immediately come non-standard situations to code into framework/standard - that's endless story... I have such a coding but it's rather complicated, several nesting levels etc.etc. - and it was programmed for a customer, I have reused it already in three projects of that customer but every time I do reuse it I have to do some additions/changes. It tested with up to hundred heavy duty threads talking to web services, MS Access(!) be databases, informing FE UI on threads' status changes etc. - and FE is able to work for days without restarting but that is a result of a very big work - it took two years+ to get from first version to the current one. Of course I didn't program it for two years, but getting all that experience to make it as it's now - that was rather time consuming - at least for me... If I ever will make that code less complicated I will share it but that will not happen soon, sorry... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 23:37 To: VBA Subject: [dba-VB] c# - controlling the run state / status of your threadedclasses As I write classes that consume threads, I need to be able to start and stop these threads and the class that consumes the threads. For example, I open a form and load a class instance that starts performing some process - exporting sets of records to csv files for example. Once that class is loaded and processing, I need to be able to ask that class to stop exporting files. However there may be a time lag between asking and the currently processing file finishing it's export. That is precisely the purpose of a thread - tell it to go and do something else. So (for example) I have a check box that I check that tells the class to start exporting. When it starts it sets a flag saying that it is running and places text into the check box label saying that it is exporting (busy). If I then uncheck the check box I am telling the class to stop exporting. The class is monitoring the check box and will not process the next file but it can't just exit. So it needs to set a flag saying that it is stopping and display "stopping" on the check box label. When it finishes exporting the file, it sets a stopped flag and says displays "terminated" in the check box label. So we have a couple of "commands" (start / stop) and a few statuses (stopped, stopping, running). I would like to "standardize" this into a class so that any class which needs this interface does it in the same way. Some controlling object can tell the object to start and stop, and the class can tell the controlling object that is is stopped, stopping or running. this class does not6 need to do any updating of labels or anything like that, just hold the variables, properties, methods and code to perform this in a standard way. Is anyone doing this? Have existing code you would like to share? Have existing experience trying to do this you would like to share? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 11 08:26:06 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 09:26:06 -0500 Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath Message-ID: <4D7A30FE.8050500@colbyconsulting.com> I am trying to copy a solution to a new location, strip all version control from it, delete a ton of projects within that solution and then version control the new solution. I copied the entire solution from my laptop to a VM and under my user / projects there. The copy still appears to be under version control. I looked in help and it says to copy the directory to itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the top level directory (the solution) but left the check mark on all of the subdirectories (projects and stuff). Does anyone have a clue how to completely and entirely remove a directory from version control - remove all green check marks from every subdirectory etc.? -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Fri Mar 11 08:59:23 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 09:59:23 -0500 Subject: [dba-VB] [dba-SQLServer] SVN - removing all version control for a directory and all files underneath In-Reply-To: <4D7A30FE.8050500@colbyconsulting.com> References: <4D7A30FE.8050500@colbyconsulting.com> Message-ID: <4D7A38CB.7090807@colbyconsulting.com> As it turns out, doing the "export to same directory" seems to actually "unregister" the directory from source control. It leaves the green check marks in explorer - not sure what that means. However if I go into the solution itself, it no longer has the version control dots next to the items in the solution explorer. So it appears to have removed that local copy of the solution from source control. John W. Colby www.ColbyConsulting.com On 3/11/2011 9:26 AM, jwcolby wrote: > I am trying to copy a solution to a new location, strip all version control from it, delete a ton of > projects within that solution and then version control the new solution. > > I copied the entire solution from my laptop to a VM and under my user / projects there. The copy > still appears to be under version control. I looked in help and it says to copy the directory to > itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the > top level directory (the solution) but left the check mark on all of the subdirectories (projects > and stuff). > > Does anyone have a clue how to completely and entirely remove a directory from version control - > remove all green check marks from every subdirectory etc.? From shamil at smsconsulting.spb.ru Fri Mar 11 09:00:05 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 18:00:05 +0300 Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath In-Reply-To: <4D7A30FE.8050500@colbyconsulting.com> References: <4D7A30FE.8050500@colbyconsulting.com> Message-ID: <9192624AA1BA4A6BB323E053411525B8@nant> Hi John -- 1. Just delete hidden .svn subfolder from root folder and all subfolders. 2. Right-click folder in MS Windows Explorer window and use Tortoise SVN Export... command from popup-menu 3. Use command line SVN utility with export command http://svnbook.red-bean.com/nightly/en/svn.ref.svn.c.export.html Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 17:26 To: Access Developers discussion and problem solving; VBA; Sqlserver-Dba Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath I am trying to copy a solution to a new location, strip all version control from it, delete a ton of projects within that solution and then version control the new solution. I copied the entire solution from my laptop to a VM and under my user / projects there. The copy still appears to be under version control. I looked in help and it says to copy the directory to itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the top level directory (the solution) but left the check mark on all of the subdirectories (projects and stuff). Does anyone have a clue how to completely and entirely remove a directory from version control - remove all green check marks from every subdirectory etc.? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 11 10:56:44 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 11:56:44 -0500 Subject: [dba-VB] c# - Creating and using a shared module Message-ID: <4D7A544C.7090904@colbyconsulting.com> I have split my solution into two solutions. However each of those two solutions uses a shared project which I call "base objects". Obviously I want to split that off and then reference that solution from both of the main solutions. I have never done this before. Any suggestions, tips and tricks, warnings, how to etc.? -- John W. Colby www.ColbyConsulting.com From gustav at cactus.dk Fri Mar 11 11:59:11 2011 From: gustav at cactus.dk (Gustav Brock) Date: Fri, 11 Mar 2011 18:59:11 +0100 Subject: [dba-VB] Visual Studio 2010 SP1 Message-ID: Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-475d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav From shamil at smsconsulting.spb.ru Fri Mar 11 11:52:31 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 20:52:31 +0300 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7A544C.7090904@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> Message-ID: <1573E00753994C19BE24FA9E50093615@nant> Hi John -- <<< Any suggestions, tips and tricks, warnings, how to etc.? >>> Just do it! :) I mean if you managed to handle MS Access library databases then handling .NET class libraries would be a "breeze" work for you. Create class library project - you can keep it within its own solution having main (console) project of that solution as a (unit) test one. Put all three solutions - two application ones and class library one in subfolders of the same root folder. (You can do it differently but this is how it's done usually AFAIK) Add existing class library project to application solutions. Reference class library projects from application projects. if you use VS2008 and you have a good PC with plenty of memory (and you have one don't you?) then you can keep all three solutions open in three instances of VS2008. Just keep tack from which solution you have made class library code changes last time - when you'll switch to the other solution and if you have the same source code file opened in it you'll get a message like that "Source file changed - would you like to reload it?" - reply yes... ... Use SVN or Mercurial for source code control. Do commit source code changes to the source code repository from time to time... ... I have 58 projects in one of the solutions here - that results in more than 5 apps and quite some class libs compiled and built - AFAIK folks do have more than hundred projects within solutions sometimes... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 19:57 To: VBA Subject: [dba-VB] c# - Creating and using a shared module I have split my solution into two solutions. However each of those two solutions uses a shared project which I call "base objects". Obviously I want to split that off and then reference that solution from both of the main solutions. I have never done this before. Any suggestions, tips and tricks, warnings, how to etc.? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From df.waters at comcast.net Fri Mar 11 13:46:10 2011 From: df.waters at comcast.net (Dan Waters) Date: Fri, 11 Mar 2011 13:46:10 -0600 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: Message-ID: <003001cbe024$f9c71750$ed5545f0$@comcast.net> I did this yesterday. The download iso image was actually about 1.5 Gb, but I downloaded, burned to disc, and installed in about an hour. Your mileage may vary! Dan -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-47 5d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 11 20:37:10 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 21:37:10 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <1573E00753994C19BE24FA9E50093615@nant> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant> Message-ID: <4D7ADC56.1040806@colbyconsulting.com> You make it sound so easy, but it is an existing project already which was not created as a class library project. I need to create the class library project then somehow get the directories and files into that project and get it to compile. At that point I can reference it by other solutions. John W. Colby www.ColbyConsulting.com On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: > Hi John -- > > <<< > Any suggestions, tips and tricks, warnings, how to etc.? >>>> > Just do it! :) > > I mean if you managed to handle MS Access library databases then handling > .NET class libraries would be a "breeze" work for you. > > Create class library project - you can keep it within its own solution > having main (console) project of that solution as a (unit) test one. > Put all three solutions - two application ones and class library one in > subfolders of the same root folder. (You can do it differently but this is > how it's done usually AFAIK) > Add existing class library project to application solutions. > Reference class library projects from application projects. > if you use VS2008 and you have a good PC with plenty of memory (and you have > one don't you?) then you can keep all three solutions open in three > instances of VS2008. > Just keep tack from which solution you have made class library code changes > last time - when you'll switch to the other solution and if you have the > same source code file opened in it you'll get a message like that "Source > file changed - would you like to reload it?" - reply yes... > > ... > > Use SVN or Mercurial for source code control. Do commit source code changes > to the source code repository from time to time... > ... > > I have 58 projects in one of the solutions here - that results in more than > 5 apps and quite some class libs compiled and built - AFAIK folks do have > more than hundred projects within solutions sometimes... > > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 11 ????? 2011 ?. 19:57 > To: VBA > Subject: [dba-VB] c# - Creating and using a shared module > > I have split my solution into two solutions. However each of those two > solutions uses a shared project which I call "base objects". Obviously I > want to split that off and then reference that solution from both of the > main solutions. > > I have never done this before. Any suggestions, tips and tricks, warnings, > how to etc.? > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From michael at ddisolutions.com.au Fri Mar 11 20:39:50 2011 From: michael at ddisolutions.com.au (Michael Maddison) Date: Sat, 12 Mar 2011 13:39:50 +1100 Subject: [dba-VB] SVN - removing all version control for a directory andall files underneath References: <4D7A30FE.8050500@colbyconsulting.com> <9192624AA1BA4A6BB323E053411525B8@nant> Message-ID: <99266C61B516644D9727F983FAFAB46508623F@remote.ddisolutions.com.au> Hi Guys, I found a reg file that adds a right click menu item to remove all hidden svn dirs. And files from the selected dir. Create a .reg file, Paste this code in, Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Folder\shell\DeleteSVN] @="Delete SVN Folders" [HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Folder\shell\DeleteSVN\command] @="cmd.exe /c \"TITLE Removing SVN Folders in %1 && COLOR 9A && FOR /r \"%1\" %%f IN (.svn) DO RD /s /q \"%%f\" \"" Save it, Merge it to your reg. Works for me. Cheers Michael From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: Saturday, 12 March 2011 2:00 AM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] SVN - removing all version control for a directory andall files underneath Hi John -- 1. Just delete hidden .svn subfolder from root folder and all subfolders. 2. Right-click folder in MS Windows Explorer window and use Tortoise SVN Export... command from popup-menu 3. Use command line SVN utility with export command http://svnbook.red-bean.com/nightly/en/svn.ref.svn.c.export.html Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 17:26 To: Access Developers discussion and problem solving; VBA; Sqlserver-Dba Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath I am trying to copy a solution to a new location, strip all version control from it, delete a ton of projects within that solution and then version control the new solution. I copied the entire solution from my laptop to a VM and under my user / projects there. The copy still appears to be under version control. I looked in help and it says to copy the directory to itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the top level directory (the solution) but left the check mark on all of the subdirectories (projects and stuff). Does anyone have a clue how to completely and entirely remove a directory from version control - remove all green check marks from every subdirectory etc.? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com ________________________________ No virus found in this message. Checked by AVG - www.avg.com Version: 10.0.1204 / Virus Database: 1498/3501 - Release Date: 03/11/11 From jwcolby at colbyconsulting.com Fri Mar 11 21:09:49 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 22:09:49 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7ADC56.1040806@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant> <4D7ADC56.1040806@colbyconsulting.com> Message-ID: <4D7AE3FD.1040206@colbyconsulting.com> I found a "make project from existing files. When I did that it offered to use a selected directory so I dragged the project directory into a new directory and told it to use that. I selected a class library type of project and it went to work "converting" my existing project into a class library. 112 compile errors, most having to do with the fact that it no longer understands using System.Windows.XYZ Many of the common classes did things like accept a combo loaded data into it in a standard way. Stuff like that. So there we are. I "just did it"! ;) John W. Colby www.ColbyConsulting.com On 3/11/2011 9:37 PM, jwcolby wrote: > You make it sound so easy, but it is an existing project already which was not created as a class > library project. I need to create the class library project then somehow get the directories and > files into that project and get it to compile. > > At that point I can reference it by other solutions. > > John W. Colby > www.ColbyConsulting.com > > On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Any suggestions, tips and tricks, warnings, how to etc.? >>>>> >> Just do it! :) >> >> I mean if you managed to handle MS Access library databases then handling >> .NET class libraries would be a "breeze" work for you. >> >> Create class library project - you can keep it within its own solution >> having main (console) project of that solution as a (unit) test one. >> Put all three solutions - two application ones and class library one in >> subfolders of the same root folder. (You can do it differently but this is >> how it's done usually AFAIK) >> Add existing class library project to application solutions. >> Reference class library projects from application projects. >> if you use VS2008 and you have a good PC with plenty of memory (and you have >> one don't you?) then you can keep all three solutions open in three >> instances of VS2008. >> Just keep tack from which solution you have made class library code changes >> last time - when you'll switch to the other solution and if you have the >> same source code file opened in it you'll get a message like that "Source >> file changed - would you like to reload it?" - reply yes... >> >> ... >> >> Use SVN or Mercurial for source code control. Do commit source code changes >> to the source code repository from time to time... >> ... >> >> I have 58 projects in one of the solutions here - that results in more than >> 5 apps and quite some class libs compiled and built - AFAIK folks do have >> more than hundred projects within solutions sometimes... >> >> >> Thank you. >> >> -- >> Shamil From shamil at smsconsulting.spb.ru Sat Mar 12 03:34:15 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 12 Mar 2011 12:34:15 +0300 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7AE3FD.1040206@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant><4D7ADC56.1040806@colbyconsulting.com> <4D7AE3FD.1040206@colbyconsulting.com> Message-ID: <8458BA03716A4820980078DE24EC3D06@nant> Hi John -- You have to add references to some of System.Windows.XYZ: - System.Windows.Forms - ... Just compare references of your source project with a classlib project created from it - and add missing references... <<< I "just did it"! ;) >>> But you did it, didn't you? ;) Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 12 ????? 2011 ?. 6:10 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# - Creating and using a shared module I found a "make project from existing files. When I did that it offered to use a selected directory so I dragged the project directory into a new directory and told it to use that. I selected a class library type of project and it went to work "converting" my existing project into a class library. 112 compile errors, most having to do with the fact that it no longer understands using System.Windows.XYZ Many of the common classes did things like accept a combo loaded data into it in a standard way. Stuff like that. So there we are. I "just did it"! ;) John W. Colby www.ColbyConsulting.com On 3/11/2011 9:37 PM, jwcolby wrote: > You make it sound so easy, but it is an existing project already which > was not created as a class library project. I need to create the class > library project then somehow get the directories and files into that project and get it to compile. > > At that point I can reference it by other solutions. > > John W. Colby > www.ColbyConsulting.com > > On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Any suggestions, tips and tricks, warnings, how to etc.? >>>>> >> Just do it! :) >> >> I mean if you managed to handle MS Access library databases then >> handling .NET class libraries would be a "breeze" work for you. >> >> Create class library project - you can keep it within its own >> solution having main (console) project of that solution as a (unit) test one. >> Put all three solutions - two application ones and class library one >> in subfolders of the same root folder. (You can do it differently but >> this is how it's done usually AFAIK) Add existing class library >> project to application solutions. >> Reference class library projects from application projects. >> if you use VS2008 and you have a good PC with plenty of memory (and >> you have one don't you?) then you can keep all three solutions open >> in three instances of VS2008. >> Just keep tack from which solution you have made class library code >> changes last time - when you'll switch to the other solution and if >> you have the same source code file opened in it you'll get a message >> like that "Source file changed - would you like to reload it?" - reply yes... >> >> ... >> >> Use SVN or Mercurial for source code control. Do commit source code >> changes to the source code repository from time to time... >> ... >> >> I have 58 projects in one of the solutions here - that results in >> more than >> 5 apps and quite some class libs compiled and built - AFAIK folks do >> have more than hundred projects within solutions sometimes... >> >> >> Thank you. >> >> -- >> Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Mar 12 06:27:48 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 12 Mar 2011 07:27:48 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <8458BA03716A4820980078DE24EC3D06@nant> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant><4D7ADC56.1040806@colbyconsulting.com> <4D7AE3FD.1040206@colbyconsulting.com> <8458BA03716A4820980078DE24EC3D06@nant> Message-ID: <4D7B66C4.6040404@colbyconsulting.com> Well... the whole thing compiled (the project) when it was part of the bigger solution. I'll give that a try. John W. Colby www.ColbyConsulting.com On 3/12/2011 4:34 AM, Shamil Salakhetdinov wrote: > Hi John -- > > You have to add references to some of System.Windows.XYZ: > > - System.Windows.Forms > - ... > > Just compare references of your source project with a classlib project > created from it - and add missing references... > > <<< > I "just did it"! ;) >>>> > But you did it, didn't you? ;) > > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 12 ????? 2011 ?. 6:10 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# - Creating and using a shared module > > I found a "make project from existing files. When I did that it offered to > use a selected directory so I dragged the project directory into a new > directory and told it to use that. I selected a class library type of > project and it went to work "converting" my existing project into a class > library. > > 112 compile errors, most having to do with the fact that it no longer > understands > > using System.Windows.XYZ > > Many of the common classes did things like accept a combo loaded data into > it in a standard way. > Stuff like that. > > So there we are. > > I "just did it"! ;) > > John W. Colby > www.ColbyConsulting.com > > On 3/11/2011 9:37 PM, jwcolby wrote: >> You make it sound so easy, but it is an existing project already which >> was not created as a class library project. I need to create the class >> library project then somehow get the directories and files into that > project and get it to compile. >> >> At that point I can reference it by other solutions. >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >>> Hi John -- >>> >>> <<< >>> Any suggestions, tips and tricks, warnings, how to etc.? >>>>>> >>> Just do it! :) >>> >>> I mean if you managed to handle MS Access library databases then >>> handling .NET class libraries would be a "breeze" work for you. >>> >>> Create class library project - you can keep it within its own >>> solution having main (console) project of that solution as a (unit) test > one. >>> Put all three solutions - two application ones and class library one >>> in subfolders of the same root folder. (You can do it differently but >>> this is how it's done usually AFAIK) Add existing class library >>> project to application solutions. >>> Reference class library projects from application projects. >>> if you use VS2008 and you have a good PC with plenty of memory (and >>> you have one don't you?) then you can keep all three solutions open >>> in three instances of VS2008. >>> Just keep tack from which solution you have made class library code >>> changes last time - when you'll switch to the other solution and if >>> you have the same source code file opened in it you'll get a message >>> like that "Source file changed - would you like to reload it?" - reply > yes... >>> >>> ... >>> >>> Use SVN or Mercurial for source code control. Do commit source code >>> changes to the source code repository from time to time... >>> ... >>> >>> I have 58 projects in one of the solutions here - that results in >>> more than >>> 5 apps and quite some class libs compiled and built - AFAIK folks do >>> have more than hundred projects within solutions sometimes... >>> >>> >>> Thank you. >>> >>> -- >>> Shamil > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 12 15:00:17 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 12 Mar 2011 16:00:17 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7B66C4.6040404@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant><4D7ADC56.1040806@colbyconsulting.com> <4D7AE3FD.1040206@colbyconsulting.com> <8458BA03716A4820980078DE24EC3D06@nant> <4D7B66C4.6040404@colbyconsulting.com> Message-ID: <4D7BDEE1.40400@colbyconsulting.com> OK, I got all of the references fixed up and got it checked into VSN. John W. Colby www.ColbyConsulting.com On 3/12/2011 7:27 AM, jwcolby wrote: > Well... the whole thing compiled (the project) when it was part of the bigger solution. > > I'll give that a try. > > John W. Colby > www.ColbyConsulting.com > > On 3/12/2011 4:34 AM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> You have to add references to some of System.Windows.XYZ: >> >> - System.Windows.Forms >> - ... >> >> Just compare references of your source project with a classlib project >> created from it - and add missing references... >> >> <<< >> I "just did it"! ;) >>>>> >> But you did it, didn't you? ;) >> >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 12 ????? 2011 ?. 6:10 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# - Creating and using a shared module >> >> I found a "make project from existing files. When I did that it offered to >> use a selected directory so I dragged the project directory into a new >> directory and told it to use that. I selected a class library type of >> project and it went to work "converting" my existing project into a class >> library. >> >> 112 compile errors, most having to do with the fact that it no longer >> understands >> >> using System.Windows.XYZ >> >> Many of the common classes did things like accept a combo loaded data into >> it in a standard way. >> Stuff like that. >> >> So there we are. >> >> I "just did it"! ;) >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/11/2011 9:37 PM, jwcolby wrote: >>> You make it sound so easy, but it is an existing project already which >>> was not created as a class library project. I need to create the class >>> library project then somehow get the directories and files into that >> project and get it to compile. >>> >>> At that point I can reference it by other solutions. >>> >>> John W. Colby >>> www.ColbyConsulting.com >>> >>> On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >>>> Hi John -- >>>> >>>> <<< >>>> Any suggestions, tips and tricks, warnings, how to etc.? >>>>>>> >>>> Just do it! :) >>>> >>>> I mean if you managed to handle MS Access library databases then >>>> handling .NET class libraries would be a "breeze" work for you. >>>> >>>> Create class library project - you can keep it within its own >>>> solution having main (console) project of that solution as a (unit) test >> one. >>>> Put all three solutions - two application ones and class library one >>>> in subfolders of the same root folder. (You can do it differently but >>>> this is how it's done usually AFAIK) Add existing class library >>>> project to application solutions. >>>> Reference class library projects from application projects. >>>> if you use VS2008 and you have a good PC with plenty of memory (and >>>> you have one don't you?) then you can keep all three solutions open >>>> in three instances of VS2008. >>>> Just keep tack from which solution you have made class library code >>>> changes last time - when you'll switch to the other solution and if >>>> you have the same source code file opened in it you'll get a message >>>> like that "Source file changed - would you like to reload it?" - reply >> yes... >>>> >>>> ... >>>> >>>> Use SVN or Mercurial for source code control. Do commit source code >>>> changes to the source code repository from time to time... >>>> ... >>>> >>>> I have 58 projects in one of the solutions here - that results in >>>> more than >>>> 5 apps and quite some class libs compiled and built - AFAIK folks do >>>> have more than hundred projects within solutions sometimes... >>>> >>>> >>>> Thank you. >>>> >>>> -- >>>> Shamil >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From bheid at sc.rr.com Sun Mar 13 09:26:07 2011 From: bheid at sc.rr.com (Bobby Heid) Date: Sun, 13 Mar 2011 10:26:07 -0400 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: Message-ID: <005d01cbe18a$977c6700$c6753500$@rr.com> Only took me about 40 minutes on my machine (Win 7 Ultimate, VS22010 Ultimate, and SQL 2008 Standard). Bobby -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Friday, March 11, 2011 12:59 PM To: dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-47 5d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sun Mar 13 21:58:15 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 13 Mar 2011 22:58:15 -0400 Subject: [dba-VB] Access runtime (an maybe full install?) Message-ID: <4D7D8447.4000003@colbyconsulting.com> I am trying to get an Access 2007 runtime to run reliably anywhere I install it. I had a Windows 2003 virtual machine hanging around so I cleaned Office off of it, and I installed the runtime. I got the directory path declared a safe location and voila, the application ran. Full of confidence I installed Hamachi, the runtime package and the app on a friend's (rather old) XP and it doesn't run. I dug out my old dev machine which it appears that I wiped and installed XP on some time ago. I installed Hamachi, got it on the VPN for this database etc. I then installed the runtime there and tried to run. No go. Of course the runtime doesn't report errors or at least reliably. So I built a little test version with just a single table and an autoform which automatically opened when the app opened. It actually reported that it couldn't talk to the odbc native sql driver (not the exact words of course). So I started Googling and soon discovered that this driver is installed by SQL Server as it installs. However an installer called sqlncli.msi is out there for download. I downloaded that and tried to run. No joy. I turned the machine off, and ran a network cable so it is directly on my network and voila, it runs. I then disabled the cable so that it is connecting to my network via my wireless and it runs. So it appears that the SQL driver is not installed as part of XP, which makes sense since this is SQL Server 2008 Express version I am connecting to. However I would have expected this odbc SQL driver to be part of the Access 2007 runtime, but it seems not. It does appear that a download / install / reboot of the machine will allow me to hook up to the sql server instance. So I am now running Hamachi, joined to this specific network, with the sqlncli.msi installer installing the ODBC SQL Server native driver, plus the Access 2007 runtime, running the full on app, hitting the sql server over the vpn on 5.203.167.79 (the Hamachi IP for the server). And it works on this machine of mine. Now I have to install the SQL Server drivers on the friend's old machine (also running XP but on 512 megs of ram) and ensure that will run. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sun Mar 13 22:27:45 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 13 Mar 2011 23:27:45 -0400 Subject: [dba-VB] [AccessD] Access runtime (an maybe full install?) In-Reply-To: <4D7D8447.4000003@colbyconsulting.com> References: <4D7D8447.4000003@colbyconsulting.com> Message-ID: <4D7D8B31.4080405@colbyconsulting.com> And it runs on the friend's old machine as well. It feels like I might be working now. I have to say this has been one of the most complex jobs I have taken on, with Hamachi VPN networks, a virtual machine running the sql server database, and then the client machine running Hamachi and an Access 2007 runtime. Safe zones, sql server drivers not installed. It has been a challenge getting this running. John W. Colby www.ColbyConsulting.com On 3/13/2011 10:58 PM, jwcolby wrote: > I am trying to get an Access 2007 runtime to run reliably anywhere I install it. I had a Windows > 2003 virtual machine hanging around so I cleaned Office off of it, and I installed the runtime. I > got the directory path declared a safe location and voila, the application ran. > > Full of confidence I installed Hamachi, the runtime package and the app on a friend's (rather old) > XP and it doesn't run. I dug out my old dev machine which it appears that I wiped and installed XP > on some time ago. I installed Hamachi, got it on the VPN for this database etc. I then installed the > runtime there and tried to run. No go. > > Of course the runtime doesn't report errors or at least reliably. So I built a little test version > with just a single table and an autoform which automatically opened when the app opened. It actually > reported that it couldn't talk to the odbc native sql driver (not the exact words of course). So I > started Googling and soon discovered that this driver is installed by SQL Server as it installs. > However an installer called sqlncli.msi is out there for download. > > I downloaded that and tried to run. No joy. I turned the machine off, and ran a network cable so it > is directly on my network and voila, it runs. I then disabled the cable so that it is connecting to > my network via my wireless and it runs. > > So it appears that the SQL driver is not installed as part of XP, which makes sense since this is > SQL Server 2008 Express version I am connecting to. However I would have expected this odbc SQL > driver to be part of the Access 2007 runtime, but it seems not. > > It does appear that a download / install / reboot of the machine will allow me to hook up to the sql > server instance. > > So I am now running Hamachi, joined to this specific network, with the sqlncli.msi installer > installing the ODBC SQL Server native driver, plus the Access 2007 runtime, running the full on app, > hitting the sql server over the vpn on 5.203.167.79 (the Hamachi IP for the server). > > And it works on this machine of mine. > > Now I have to install the SQL Server drivers on the friend's old machine (also running XP but on 512 > megs of ram) and ensure that will run. > > From Gustav at cactus.dk Mon Mar 14 05:38:03 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 14 Mar 2011 11:38:03 +0100 Subject: [dba-VB] Visual Studio 2010 SP1 Message-ID: Hi all OK, don't blame the child. At one time it even asked for the VS2010 dvd - to roll back the installation. Another install was in progress, or at least I hadn't rebooted the machine. So I did and ran the SP1 again. This time - as everything was downloaded - it took about an hour to finish. /gustav >>> gustav at cactus.dk 11-03-2011 18:59 >>> Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-475d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav From jwcolby at colbyconsulting.com Mon Mar 14 13:16:52 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 14:16:52 -0400 Subject: [dba-VB] C# combo boxes Message-ID: <4D7E5B94.2020805@colbyconsulting.com> I am trying to set up a combo box bound to a field in the form such that the combo populates with the PKID and a text field, and when the form opens, selecting something from the combo drops the PKID into the field in the form. I am not getting what to set the properties of the combo box to. I am pulling the ID and the string to display into the combo but I cannot cause the combo to store the id and release control to the next control on the form. -- John W. Colby www.ColbyConsulting.com From Gustav at cactus.dk Mon Mar 14 14:02:27 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 14 Mar 2011 20:02:27 +0100 Subject: [dba-VB] C# combo boxes Message-ID: Hi John If you (typically) have this in InitializeComponent() of the .Designer.cs file: // comboBoxCountry // this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); this.comboBoxCountry.DisplayMember = "Id"; this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; this.comboBoxCountry.FormattingEnabled = true; this.comboBoxCountry.ItemHeight = 13; this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); this.comboBoxCountry.Name = "comboBoxCountry"; this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); this.comboBoxCountry.Sorted = true; this.comboBoxCountry.TabIndex = 7; this.comboBoxCountry.ValueMember = "Id"; You can do something like this in the Load event: this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); Then: private void FillComboBoxCountryCode() { // Order the country names of countryDataTable. string filterExpression = ""; string sortExpression = "CountryNameUK asc"; DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); this.comboBoxCountry.DataSource = countryList; this.comboBoxCountry.ValueMember = "Id"; this.comboBoxCountry.DisplayMember = "CountryNameUK"; } And: private void ChangeCountrySelection() { if (this.comboBoxCountry.SelectedValue != null) { _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); this.comboBoxPostalCode.Text = String.Empty; // Read postal codes of selected country. FillComboBoxPostalCode(); } } This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. /gustav >>> jwcolby at colbyconsulting.com 14-03-2011 19:16 >>> I am trying to set up a combo box bound to a field in the form such that the combo populates with the PKID and a text field, and when the form opens, selecting something from the combo drops the PKID into the field in the form. I am not getting what to set the properties of the combo box to. I am pulling the ID and the string to display into the combo but I cannot cause the combo to store the id and release control to the next control on the form. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Mon Mar 14 16:36:55 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 17:36:55 -0400 Subject: [dba-VB] C# combo boxes In-Reply-To: References: Message-ID: <4D7E8A77.7070009@colbyconsulting.com> Thanks Gustav, I don't know why you are taking me into code in code. I have data connections. This particular data connection has a single view that allows me to write into a table. It has a view that filters a combo. I drag and drop the view for the table onto the form. I have told that table object that specific fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am trying to set up that combo to 1) Display a text value 2) Contain the PKID from the table the combo draws from 3) When I select an item, place the selected ID into the field the combo is bound to. It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or going into code). Actually I have a kinda sorta related question. When I look at server explorer I see a data connection object, however there are two objects underneath, neither of which are for this solution. Furthermore the objects under the data connection are things found on an entirely unrelated server, for other projects that I did work on in the past. The question then is why is the server explorer choosing to display objects out on a virtual machine instead of the database that was originally selected for this solution? We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound form stuff. I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values in most cases) the developer, the product, the type of work, then a couple of fields for date time values for start / stop work. The table the data is going into has a few thousand records already in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound forms is a natural for this. A long time ago I made this (kind of thing) work. I remember that getting the properties of the combo set correctly was non-intuitive but once I figured it out it just worked. John W. Colby www.ColbyConsulting.com On 3/14/2011 3:02 PM, Gustav Brock wrote: > Hi John > > If you (typically) have this in InitializeComponent() of the .Designer.cs file: > > // comboBoxCountry > // > this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; > this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; > this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; > this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); > this.comboBoxCountry.DisplayMember = "Id"; > this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; > this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); > this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; > this.comboBoxCountry.FormattingEnabled = true; > this.comboBoxCountry.ItemHeight = 13; > this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); > this.comboBoxCountry.Name = "comboBoxCountry"; > this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); > this.comboBoxCountry.Sorted = true; > this.comboBoxCountry.TabIndex = 7; > this.comboBoxCountry.ValueMember = "Id"; > > You can do something like this in the Load event: > > this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); > > Then: > private void FillComboBoxCountryCode() > { > // Order the country names of countryDataTable. > string filterExpression = ""; > string sortExpression = "CountryNameUK asc"; > DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); > > this.comboBoxCountry.DataSource = countryList; > this.comboBoxCountry.ValueMember = "Id"; > this.comboBoxCountry.DisplayMember = "CountryNameUK"; > } > > And: > private void ChangeCountrySelection() > { > if (this.comboBoxCountry.SelectedValue != null) > { > _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); > this.comboBoxPostalCode.Text = String.Empty; > // Read postal codes of selected country. > FillComboBoxPostalCode(); > } > } > > This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> > I am trying to set up a combo box bound to a field in the form such that the combo populates with > the PKID and a text field, and when the form opens, selecting something from the combo drops the > PKID into the field in the form. > > I am not getting what to set the properties of the combo box to. I am pulling the ID and the string > to display into the combo but I cannot cause the combo to store the id and release control to the > next control on the form. From gustav at cactus.dk Mon Mar 14 17:15:35 2011 From: gustav at cactus.dk (Gustav Brock) Date: Mon, 14 Mar 2011 23:15:35 +0100 Subject: [dba-VB] C# combo boxes Message-ID: Hi John Eh, are you commenting or? Not quite clear to me. I thought you were looking for some code samples ... I have connections too but I use them only to feed the datatableadapters. I refrain from working with SqlCommand and the like; waste of time in my opinion. /gustav >>> jwcolby at colbyconsulting.com 14-03-2011 22:36 >>> Thanks Gustav, I don't know why you are taking me into code in code. I have data connections. This particular data connection has a single view that allows me to write into a table. It has a view that filters a combo. I drag and drop the view for the table onto the form. I have told that table object that specific fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am trying to set up that combo to 1) Display a text value 2) Contain the PKID from the table the combo draws from 3) When I select an item, place the selected ID into the field the combo is bound to. It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or going into code). Actually I have a kinda sorta related question. When I look at server explorer I see a data connection object, however there are two objects underneath, neither of which are for this solution. Furthermore the objects under the data connection are things found on an entirely unrelated server, for other projects that I did work on in the past. The question then is why is the server explorer choosing to display objects out on a virtual machine instead of the database that was originally selected for this solution? We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound form stuff. I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values in most cases) the developer, the product, the type of work, then a couple of fields for date time values for start / stop work. The table the data is going into has a few thousand records already in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound forms is a natural for this. A long time ago I made this (kind of thing) work. I remember that getting the properties of the combo set correctly was non-intuitive but once I figured it out it just worked. John W. Colby www.ColbyConsulting.com On 3/14/2011 3:02 PM, Gustav Brock wrote: > Hi John > > If you (typically) have this in InitializeComponent() of the .Designer.cs file: > > // comboBoxCountry > // > this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; > this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; > this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; > this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); > this.comboBoxCountry.DisplayMember = "Id"; > this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; > this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); > this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; > this.comboBoxCountry.FormattingEnabled = true; > this.comboBoxCountry.ItemHeight = 13; > this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); > this.comboBoxCountry.Name = "comboBoxCountry"; > this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); > this.comboBoxCountry.Sorted = true; > this.comboBoxCountry.TabIndex = 7; > this.comboBoxCountry.ValueMember = "Id"; > > You can do something like this in the Load event: > > this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); > > Then: > private void FillComboBoxCountryCode() > { > // Order the country names of countryDataTable. > string filterExpression = ""; > string sortExpression = "CountryNameUK asc"; > DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); > > this.comboBoxCountry.DataSource = countryList; > this.comboBoxCountry.ValueMember = "Id"; > this.comboBoxCountry.DisplayMember = "CountryNameUK"; > } > > And: > private void ChangeCountrySelection() > { > if (this.comboBoxCountry.SelectedValue != null) > { > _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); > this.comboBoxPostalCode.Text = String.Empty; > // Read postal codes of selected country. > FillComboBoxPostalCode(); > } > } > > This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> > I am trying to set up a combo box bound to a field in the form such that the combo populates with > the PKID and a text field, and when the form opens, selecting something from the combo drops the > PKID into the field in the form. > > I am not getting what to set the properties of the combo box to. I am pulling the ID and the string > to display into the combo but I cannot cause the combo to store the id and release control to the > next control on the form. From jwcolby at colbyconsulting.com Mon Mar 14 17:25:08 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 18:25:08 -0400 Subject: [dba-VB] C# combo boxes In-Reply-To: References: Message-ID: <4D7E95C4.9020203@colbyconsulting.com> I am trying (unsuccessfully) to do an "Access like bound form". And I always comment. ;) ATM the form displays the values if dropped down, but if not dropped down it just displays an integer (the PKID of the data in the combo is my guess, or perhaps the PKID in the field in the form. So... How do I cause the combo to display Paul's name instead of 6. John W. Colby www.ColbyConsulting.com On 3/14/2011 6:15 PM, Gustav Brock wrote: > Hi John > > Eh, are you commenting or? Not quite clear to me. I thought you were looking for some code samples ... > > I have connections too but I use them only to feed the datatableadapters. I refrain from working with SqlCommand and the like; waste of time in my opinion. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> > Thanks Gustav, > I don't know why you are taking me into code in code. > > I have data connections. This particular data connection has a single view that allows me to write > into a table. It has a view that filters a combo. > > I drag and drop the view for the table onto the form. I have told that table object that specific > fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am > trying to set up that combo to > > 1) Display a text value > 2) Contain the PKID from the table the combo draws from > 3) When I select an item, place the selected ID into the field the combo is bound to. > > It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or > going into code). > > Actually I have a kinda sorta related question. > > When I look at server explorer I see a data connection object, however there are two objects > underneath, neither of which are for this solution. Furthermore the objects under the data > connection are things found on an entirely unrelated server, for other projects that I did work on > in the past. > > The question then is why is the server explorer choosing to display objects out on a virtual machine > instead of the database that was originally selected for this solution? > > We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound > form stuff. > > I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values > in most cases) the developer, the product, the type of work, then a couple of fields for date time > values for start / stop work. The table the data is going into has a few thousand records already > in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound > forms is a natural for this. > > A long time ago I made this (kind of thing) work. I remember that getting the properties of the > combo set correctly was non-intuitive but once I figured it out it just worked. > > > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 3:02 PM, Gustav Brock wrote: >> Hi John >> >> If you (typically) have this in InitializeComponent() of the .Designer.cs file: >> >> // comboBoxCountry >> // >> this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; >> this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; >> this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; >> this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); >> this.comboBoxCountry.DisplayMember = "Id"; >> this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; >> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); >> this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; >> this.comboBoxCountry.FormattingEnabled = true; >> this.comboBoxCountry.ItemHeight = 13; >> this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); >> this.comboBoxCountry.Name = "comboBoxCountry"; >> this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); >> this.comboBoxCountry.Sorted = true; >> this.comboBoxCountry.TabIndex = 7; >> this.comboBoxCountry.ValueMember = "Id"; >> >> You can do something like this in the Load event: >> >> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); >> >> Then: >> private void FillComboBoxCountryCode() >> { >> // Order the country names of countryDataTable. >> string filterExpression = ""; >> string sortExpression = "CountryNameUK asc"; >> DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); >> >> this.comboBoxCountry.DataSource = countryList; >> this.comboBoxCountry.ValueMember = "Id"; >> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >> } >> >> And: >> private void ChangeCountrySelection() >> { >> if (this.comboBoxCountry.SelectedValue != null) >> { >> _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); >> this.comboBoxPostalCode.Text = String.Empty; >> // Read postal codes of selected country. >> FillComboBoxPostalCode(); >> } >> } >> >> This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >> I am trying to set up a combo box bound to a field in the form such that the combo populates with >> the PKID and a text field, and when the form opens, selecting something from the combo drops the >> PKID into the field in the form. >> >> I am not getting what to set the properties of the combo box to. I am pulling the ID and the string >> to display into the combo but I cannot cause the combo to store the id and release control to the >> next control on the form. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From accessd at shaw.ca Mon Mar 14 18:59:26 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Mon, 14 Mar 2011 16:59:26 -0700 Subject: [dba-VB] C# combo boxes In-Reply-To: <4D7E95C4.9020203@colbyconsulting.com> References: <4D7E95C4.9020203@colbyconsulting.com> Message-ID: Hi John: Try and use ADO. It is on every Windows based computer already and requires no additional ODBC to be setup. In other words, you do not even have to see the computers on which the application is going to have a sable connection. Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, March 14, 2011 3:25 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# combo boxes I am trying (unsuccessfully) to do an "Access like bound form". And I always comment. ;) ATM the form displays the values if dropped down, but if not dropped down it just displays an integer (the PKID of the data in the combo is my guess, or perhaps the PKID in the field in the form. So... How do I cause the combo to display Paul's name instead of 6. John W. Colby www.ColbyConsulting.com On 3/14/2011 6:15 PM, Gustav Brock wrote: > Hi John > > Eh, are you commenting or? Not quite clear to me. I thought you were looking for some code samples ... > > I have connections too but I use them only to feed the datatableadapters. I refrain from working with SqlCommand and the like; waste of time in my opinion. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> > Thanks Gustav, > I don't know why you are taking me into code in code. > > I have data connections. This particular data connection has a single view that allows me to write > into a table. It has a view that filters a combo. > > I drag and drop the view for the table onto the form. I have told that table object that specific > fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am > trying to set up that combo to > > 1) Display a text value > 2) Contain the PKID from the table the combo draws from > 3) When I select an item, place the selected ID into the field the combo is bound to. > > It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or > going into code). > > Actually I have a kinda sorta related question. > > When I look at server explorer I see a data connection object, however there are two objects > underneath, neither of which are for this solution. Furthermore the objects under the data > connection are things found on an entirely unrelated server, for other projects that I did work on > in the past. > > The question then is why is the server explorer choosing to display objects out on a virtual machine > instead of the database that was originally selected for this solution? > > We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound > form stuff. > > I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values > in most cases) the developer, the product, the type of work, then a couple of fields for date time > values for start / stop work. The table the data is going into has a few thousand records already > in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound > forms is a natural for this. > > A long time ago I made this (kind of thing) work. I remember that getting the properties of the > combo set correctly was non-intuitive but once I figured it out it just worked. > > > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 3:02 PM, Gustav Brock wrote: >> Hi John >> >> If you (typically) have this in InitializeComponent() of the .Designer.cs file: >> >> // comboBoxCountry >> // >> this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; >> this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; >> this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; >> this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); >> this.comboBoxCountry.DisplayMember = "Id"; >> this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; >> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); >> this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; >> this.comboBoxCountry.FormattingEnabled = true; >> this.comboBoxCountry.ItemHeight = 13; >> this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); >> this.comboBoxCountry.Name = "comboBoxCountry"; >> this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); >> this.comboBoxCountry.Sorted = true; >> this.comboBoxCountry.TabIndex = 7; >> this.comboBoxCountry.ValueMember = "Id"; >> >> You can do something like this in the Load event: >> >> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); >> >> Then: >> private void FillComboBoxCountryCode() >> { >> // Order the country names of countryDataTable. >> string filterExpression = ""; >> string sortExpression = "CountryNameUK asc"; >> DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); >> >> this.comboBoxCountry.DataSource = countryList; >> this.comboBoxCountry.ValueMember = "Id"; >> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >> } >> >> And: >> private void ChangeCountrySelection() >> { >> if (this.comboBoxCountry.SelectedValue != null) >> { >> _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); >> this.comboBoxPostalCode.Text = String.Empty; >> // Read postal codes of selected country. >> FillComboBoxPostalCode(); >> } >> } >> >> This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >> I am trying to set up a combo box bound to a field in the form such that the combo populates with >> the PKID and a text field, and when the form opens, selecting something from the combo drops the >> PKID into the field in the form. >> >> I am not getting what to set the properties of the combo box to. I am pulling the ID and the string >> to display into the combo but I cannot cause the combo to store the id and release control to the >> next control on the form. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Mon Mar 14 21:13:25 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 22:13:25 -0400 Subject: [dba-VB] C# combo boxes In-Reply-To: References: <4D7E95C4.9020203@colbyconsulting.com> Message-ID: <4D7ECB45.8060609@colbyconsulting.com> I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com On 3/14/2011 7:59 PM, Jim Lawrence wrote: > Hi John: > > Try and use ADO. It is on every Windows based computer already and requires > no additional ODBC to be setup. In other words, you do not even have to see > the computers on which the application is going to have a sable connection. > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Monday, March 14, 2011 3:25 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] C# combo boxes > > I am trying (unsuccessfully) to do an "Access like bound form". > > And I always comment. ;) > > ATM the form displays the values if dropped down, but if not dropped down it > just displays an > integer (the PKID of the data in the combo is my guess, or perhaps the PKID > in the field in the form. > > So... > > How do I cause the combo to display Paul's name instead of 6. > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 6:15 PM, Gustav Brock wrote: >> Hi John >> >> Eh, are you commenting or? Not quite clear to me. I thought you were > looking for some code samples ... >> >> I have connections too but I use them only to feed the datatableadapters. > I refrain from working with SqlCommand and the like; waste of time in my > opinion. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> >> Thanks Gustav, >> I don't know why you are taking me into code in code. >> >> I have data connections. This particular data connection has a single > view that allows me to write >> into a table. It has a view that filters a combo. >> >> I drag and drop the view for the table onto the form. I have told that > table object that specific >> fields are combos so when I drag it out, I have combos (for the FKs - PKs > from other tables). I am >> trying to set up that combo to >> >> 1) Display a text value >> 2) Contain the PKID from the table the combo draws from >> 3) When I select an item, place the selected ID into the field the combo > is bound to. >> >> It seems this is a normal daily requirement in building forms, and > shouldn't be rocket science (or >> going into code). >> >> Actually I have a kinda sorta related question. >> >> When I look at server explorer I see a data connection object, however > there are two objects >> underneath, neither of which are for this solution. Furthermore the > objects under the data >> connection are things found on an entirely unrelated server, for other > projects that I did work on >> in the past. >> >> The question then is why is the server explorer choosing to display > objects out on a virtual machine >> instead of the database that was originally selected for this solution? >> >> We have done a ton of coding sql server stuff here but we have not yet > learned how to do this bound >> form stuff. >> >> I am trying to build a timeclock kind of thing. Very simple, a few combos > to select (default values >> in most cases) the developer, the product, the type of work, then a couple > of fields for date time >> values for start / stop work. The table the data is going into has a few > thousand records already >> in it, the combos pull from tables with 20-30 records etc. It seems (to > me) that just using bound >> forms is a natural for this. >> >> A long time ago I made this (kind of thing) work. I remember that getting > the properties of the >> combo set correctly was non-intuitive but once I figured it out it just > worked. >> >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/14/2011 3:02 PM, Gustav Brock wrote: >>> Hi John >>> >>> If you (typically) have this in InitializeComponent() of the .Designer.cs > file: >>> >>> // comboBoxCountry >>> // >>> this.comboBoxCountry.AutoCompleteMode = > System.Windows.Forms.AutoCompleteMode.Suggest; >>> this.comboBoxCountry.AutoCompleteSource = > System.Windows.Forms.AutoCompleteSource.ListItems; >>> this.comboBoxCountry.BackColor = > System.Drawing.Color.LavenderBlush; >>> this.comboBoxCountry.DataBindings.Add(new > System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, > "CountryId", true)); >>> this.comboBoxCountry.DisplayMember = "Id"; >>> this.comboBoxCountry.FlatStyle = > System.Windows.Forms.FlatStyle.Flat; >>> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe > UI", 8F, System.Drawing.FontStyle.Regular, > System.Drawing.GraphicsUnit.Point, ((byte)(0))); >>> this.comboBoxCountry.ForeColor = > System.Drawing.Color.DarkMagenta; >>> this.comboBoxCountry.FormattingEnabled = true; >>> this.comboBoxCountry.ItemHeight = 13; >>> this.comboBoxCountry.Location = new > System.Drawing.Point(21, 136); >>> this.comboBoxCountry.Name = "comboBoxCountry"; >>> this.comboBoxCountry.Size = new System.Drawing.Size(117, > 21); >>> this.comboBoxCountry.Sorted = true; >>> this.comboBoxCountry.TabIndex = 7; >>> this.comboBoxCountry.ValueMember = "Id"; >>> >>> You can do something like this in the Load event: >>> >>> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => > ChangeCountrySelection(); >>> >>> Then: >>> private void FillComboBoxCountryCode() >>> { >>> // Order the country names of countryDataTable. >>> string filterExpression = ""; >>> string sortExpression = "CountryNameUK asc"; >>> DataRow[] countryList = > _countryDataTable.Select(filterExpression, sortExpression); >>> >>> this.comboBoxCountry.DataSource = countryList; >>> this.comboBoxCountry.ValueMember = "Id"; >>> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >>> } >>> >>> And: >>> private void ChangeCountrySelection() >>> { >>> if (this.comboBoxCountry.SelectedValue != null) >>> { >>> _countryId = > Convert.ToInt32(this.comboBoxCountry.SelectedValue); >>> this.comboBoxPostalCode.Text = String.Empty; >>> // Read postal codes of selected country. >>> FillComboBoxPostalCode(); >>> } >>> } >>> >>> This is pretty standard using tableadapters and bindingnavigator. Nothing > fancy. >>> >>> /gustav >>> >>> >>>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >>> I am trying to set up a combo box bound to a field in the form such that > the combo populates with >>> the PKID and a text field, and when the form opens, selecting something > from the combo drops the >>> PKID into the field in the form. >>> >>> I am not getting what to set the properties of the combo box to. I am > pulling the ID and the string >>> to display into the combo but I cannot cause the combo to store the id > and release control to the >>> next control on the form. >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From accessd at shaw.ca Tue Mar 15 02:19:11 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Tue, 15 Mar 2011 00:19:11 -0700 Subject: [dba-VB] C# combo boxes In-Reply-To: <4D7ECB45.8060609@colbyconsulting.com> References: <4D7E95C4.9020203@colbyconsulting.com> <4D7ECB45.8060609@colbyconsulting.com> Message-ID: Congratulation John. Keep us posted on it stability. Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, March 14, 2011 7:13 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# combo boxes I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com On 3/14/2011 7:59 PM, Jim Lawrence wrote: > Hi John: > > Try and use ADO. It is on every Windows based computer already and requires > no additional ODBC to be setup. In other words, you do not even have to see > the computers on which the application is going to have a sable connection. > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Monday, March 14, 2011 3:25 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] C# combo boxes > > I am trying (unsuccessfully) to do an "Access like bound form". > > And I always comment. ;) > > ATM the form displays the values if dropped down, but if not dropped down it > just displays an > integer (the PKID of the data in the combo is my guess, or perhaps the PKID > in the field in the form. > > So... > > How do I cause the combo to display Paul's name instead of 6. > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 6:15 PM, Gustav Brock wrote: >> Hi John >> >> Eh, are you commenting or? Not quite clear to me. I thought you were > looking for some code samples ... >> >> I have connections too but I use them only to feed the datatableadapters. > I refrain from working with SqlCommand and the like; waste of time in my > opinion. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> >> Thanks Gustav, >> I don't know why you are taking me into code in code. >> >> I have data connections. This particular data connection has a single > view that allows me to write >> into a table. It has a view that filters a combo. >> >> I drag and drop the view for the table onto the form. I have told that > table object that specific >> fields are combos so when I drag it out, I have combos (for the FKs - PKs > from other tables). I am >> trying to set up that combo to >> >> 1) Display a text value >> 2) Contain the PKID from the table the combo draws from >> 3) When I select an item, place the selected ID into the field the combo > is bound to. >> >> It seems this is a normal daily requirement in building forms, and > shouldn't be rocket science (or >> going into code). >> >> Actually I have a kinda sorta related question. >> >> When I look at server explorer I see a data connection object, however > there are two objects >> underneath, neither of which are for this solution. Furthermore the > objects under the data >> connection are things found on an entirely unrelated server, for other > projects that I did work on >> in the past. >> >> The question then is why is the server explorer choosing to display > objects out on a virtual machine >> instead of the database that was originally selected for this solution? >> >> We have done a ton of coding sql server stuff here but we have not yet > learned how to do this bound >> form stuff. >> >> I am trying to build a timeclock kind of thing. Very simple, a few combos > to select (default values >> in most cases) the developer, the product, the type of work, then a couple > of fields for date time >> values for start / stop work. The table the data is going into has a few > thousand records already >> in it, the combos pull from tables with 20-30 records etc. It seems (to > me) that just using bound >> forms is a natural for this. >> >> A long time ago I made this (kind of thing) work. I remember that getting > the properties of the >> combo set correctly was non-intuitive but once I figured it out it just > worked. >> >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/14/2011 3:02 PM, Gustav Brock wrote: >>> Hi John >>> >>> If you (typically) have this in InitializeComponent() of the .Designer.cs > file: >>> >>> // comboBoxCountry >>> // >>> this.comboBoxCountry.AutoCompleteMode = > System.Windows.Forms.AutoCompleteMode.Suggest; >>> this.comboBoxCountry.AutoCompleteSource = > System.Windows.Forms.AutoCompleteSource.ListItems; >>> this.comboBoxCountry.BackColor = > System.Drawing.Color.LavenderBlush; >>> this.comboBoxCountry.DataBindings.Add(new > System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, > "CountryId", true)); >>> this.comboBoxCountry.DisplayMember = "Id"; >>> this.comboBoxCountry.FlatStyle = > System.Windows.Forms.FlatStyle.Flat; >>> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe > UI", 8F, System.Drawing.FontStyle.Regular, > System.Drawing.GraphicsUnit.Point, ((byte)(0))); >>> this.comboBoxCountry.ForeColor = > System.Drawing.Color.DarkMagenta; >>> this.comboBoxCountry.FormattingEnabled = true; >>> this.comboBoxCountry.ItemHeight = 13; >>> this.comboBoxCountry.Location = new > System.Drawing.Point(21, 136); >>> this.comboBoxCountry.Name = "comboBoxCountry"; >>> this.comboBoxCountry.Size = new System.Drawing.Size(117, > 21); >>> this.comboBoxCountry.Sorted = true; >>> this.comboBoxCountry.TabIndex = 7; >>> this.comboBoxCountry.ValueMember = "Id"; >>> >>> You can do something like this in the Load event: >>> >>> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => > ChangeCountrySelection(); >>> >>> Then: >>> private void FillComboBoxCountryCode() >>> { >>> // Order the country names of countryDataTable. >>> string filterExpression = ""; >>> string sortExpression = "CountryNameUK asc"; >>> DataRow[] countryList = > _countryDataTable.Select(filterExpression, sortExpression); >>> >>> this.comboBoxCountry.DataSource = countryList; >>> this.comboBoxCountry.ValueMember = "Id"; >>> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >>> } >>> >>> And: >>> private void ChangeCountrySelection() >>> { >>> if (this.comboBoxCountry.SelectedValue != null) >>> { >>> _countryId = > Convert.ToInt32(this.comboBoxCountry.SelectedValue); >>> this.comboBoxPostalCode.Text = String.Empty; >>> // Read postal codes of selected country. >>> FillComboBoxPostalCode(); >>> } >>> } >>> >>> This is pretty standard using tableadapters and bindingnavigator. Nothing > fancy. >>> >>> /gustav >>> >>> >>>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >>> I am trying to set up a combo box bound to a field in the form such that > the combo populates with >>> the PKID and a text field, and when the form opens, selecting something > from the combo drops the >>> PKID into the field in the form. >>> >>> I am not getting what to set the properties of the combo box to. I am > pulling the ID and the string >>> to display into the combo but I cannot cause the combo to store the id > and release control to the >>> next control on the form. >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From Gustav at cactus.dk Tue Mar 15 04:55:02 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Tue, 15 Mar 2011 10:55:02 +0100 Subject: [dba-VB] C# combo boxes Message-ID: Hi John Good! Yes, the content of Text is handled by DisplayMember: this.comboBoxCountry.DataSource = countryList; this.comboBoxCountry.ValueMember = "Id"; this.comboBoxCountry.DisplayMember = "CountryNameUK"; /gustav >>> jwcolby at colbyconsulting.com 15-03-2011 03:13 >>> I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com From df.waters at comcast.net Tue Mar 15 08:54:00 2011 From: df.waters at comcast.net (Dan Waters) Date: Tue, 15 Mar 2011 08:54:00 -0500 Subject: [dba-VB] C# combo boxes In-Reply-To: <4D7ECB45.8060609@colbyconsulting.com> References: <4D7E95C4.9020203@colbyconsulting.com> <4D7ECB45.8060609@colbyconsulting.com> Message-ID: <002101cbe318$713acbb0$53b06310$@comcast.net> Hi John, Take a look at www.updatecontrols.net. This has a free download which integrates into visual studio and provides a way to simply databinding. I haven't used it yet but it looks promising. Note: At the top of the screens is a combobox which selects either VB or C# for the code examples. Dan -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, March 14, 2011 9:13 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# combo boxes I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com On 3/14/2011 7:59 PM, Jim Lawrence wrote: > Hi John: > > Try and use ADO. It is on every Windows based computer already and > requires no additional ODBC to be setup. In other words, you do not > even have to see the computers on which the application is going to have a sable connection. > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Monday, March 14, 2011 3:25 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] C# combo boxes > > I am trying (unsuccessfully) to do an "Access like bound form". > > And I always comment. ;) > > ATM the form displays the values if dropped down, but if not dropped > down it just displays an integer (the PKID of the data in the combo is > my guess, or perhaps the PKID in the field in the form. > > So... > > How do I cause the combo to display Paul's name instead of 6. > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 6:15 PM, Gustav Brock wrote: >> Hi John >> >> Eh, are you commenting or? Not quite clear to me. I thought you were > looking for some code samples ... >> >> I have connections too but I use them only to feed the datatableadapters. > I refrain from working with SqlCommand and the like; waste of time in > my opinion. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> >> Thanks Gustav, >> I don't know why you are taking me into code in code. >> >> I have data connections. This particular data connection has a >> single > view that allows me to write >> into a table. It has a view that filters a combo. >> >> I drag and drop the view for the table onto the form. I have told >> that > table object that specific >> fields are combos so when I drag it out, I have combos (for the FKs - >> PKs > from other tables). I am >> trying to set up that combo to >> >> 1) Display a text value >> 2) Contain the PKID from the table the combo draws from >> 3) When I select an item, place the selected ID into the field the >> combo > is bound to. >> >> It seems this is a normal daily requirement in building forms, and > shouldn't be rocket science (or >> going into code). >> >> Actually I have a kinda sorta related question. >> >> When I look at server explorer I see a data connection object, >> however > there are two objects >> underneath, neither of which are for this solution. Furthermore the > objects under the data >> connection are things found on an entirely unrelated server, for >> other > projects that I did work on >> in the past. >> >> The question then is why is the server explorer choosing to display > objects out on a virtual machine >> instead of the database that was originally selected for this solution? >> >> We have done a ton of coding sql server stuff here but we have not >> yet > learned how to do this bound >> form stuff. >> >> I am trying to build a timeclock kind of thing. Very simple, a few >> combos > to select (default values >> in most cases) the developer, the product, the type of work, then a >> couple > of fields for date time >> values for start / stop work. The table the data is going into has a >> few > thousand records already >> in it, the combos pull from tables with 20-30 records etc. It seems >> (to > me) that just using bound >> forms is a natural for this. >> >> A long time ago I made this (kind of thing) work. I remember that >> getting > the properties of the >> combo set correctly was non-intuitive but once I figured it out it >> just > worked. >> >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/14/2011 3:02 PM, Gustav Brock wrote: >>> Hi John >>> >>> If you (typically) have this in InitializeComponent() of the >>> .Designer.cs > file: >>> >>> // comboBoxCountry >>> // >>> this.comboBoxCountry.AutoCompleteMode = > System.Windows.Forms.AutoCompleteMode.Suggest; >>> this.comboBoxCountry.AutoCompleteSource = > System.Windows.Forms.AutoCompleteSource.ListItems; >>> this.comboBoxCountry.BackColor = > System.Drawing.Color.LavenderBlush; >>> this.comboBoxCountry.DataBindings.Add(new > System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, > "CountryId", true)); >>> this.comboBoxCountry.DisplayMember = "Id"; >>> this.comboBoxCountry.FlatStyle = > System.Windows.Forms.FlatStyle.Flat; >>> this.comboBoxCountry.Font = new >>> System.Drawing.Font("Segoe > UI", 8F, System.Drawing.FontStyle.Regular, > System.Drawing.GraphicsUnit.Point, ((byte)(0))); >>> this.comboBoxCountry.ForeColor = > System.Drawing.Color.DarkMagenta; >>> this.comboBoxCountry.FormattingEnabled = true; >>> this.comboBoxCountry.ItemHeight = 13; >>> this.comboBoxCountry.Location = new > System.Drawing.Point(21, 136); >>> this.comboBoxCountry.Name = "comboBoxCountry"; >>> this.comboBoxCountry.Size = new >>> System.Drawing.Size(117, > 21); >>> this.comboBoxCountry.Sorted = true; >>> this.comboBoxCountry.TabIndex = 7; >>> this.comboBoxCountry.ValueMember = "Id"; >>> >>> You can do something like this in the Load event: >>> >>> this.comboBoxCountry.SelectedValueChanged += (oo, ee) >>> => > ChangeCountrySelection(); >>> >>> Then: >>> private void FillComboBoxCountryCode() >>> { >>> // Order the country names of countryDataTable. >>> string filterExpression = ""; >>> string sortExpression = "CountryNameUK asc"; >>> DataRow[] countryList = > _countryDataTable.Select(filterExpression, sortExpression); >>> >>> this.comboBoxCountry.DataSource = countryList; >>> this.comboBoxCountry.ValueMember = "Id"; >>> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >>> } >>> >>> And: >>> private void ChangeCountrySelection() >>> { >>> if (this.comboBoxCountry.SelectedValue != null) >>> { >>> _countryId = > Convert.ToInt32(this.comboBoxCountry.SelectedValue); >>> this.comboBoxPostalCode.Text = String.Empty; >>> // Read postal codes of selected country. >>> FillComboBoxPostalCode(); >>> } >>> } >>> >>> This is pretty standard using tableadapters and bindingnavigator. >>> Nothing > fancy. >>> >>> /gustav >>> >>> >>>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >>> I am trying to set up a combo box bound to a field in the form such >>> that > the combo populates with >>> the PKID and a text field, and when the form opens, selecting >>> something > from the combo drops the >>> PKID into the field in the form. >>> >>> I am not getting what to set the properties of the combo box to. I >>> am > pulling the ID and the string >>> to display into the combo but I cannot cause the combo to store the >>> id > and release control to the >>> next control on the form. >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From dbdoug at gmail.com Wed Mar 16 10:28:18 2011 From: dbdoug at gmail.com (Doug Steele) Date: Wed, 16 Mar 2011 08:28:18 -0700 Subject: [dba-VB] VS2010 install Message-ID: Hello all: I just finished doing a clean install of Windows 7 then VS2010. To my surprise, I also appear to have a functioning install of VS2008, loading with the name 'Shell' where my old VS2008 says 'Professional Edition'. I couldn't see anything on the net about this; does anyone have any further information? Thanks, Doug From DWUTKA at Marlow.com Fri Mar 18 14:12:52 2011 From: DWUTKA at Marlow.com (Drew Wutka) Date: Fri, 18 Mar 2011 14:12:52 -0500 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: Message-ID: 500 megs... that's about 10 minutes... wouldn't have time for a full dinner.... Drew -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-810 7-475d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com The information contained in this transmission is intended only for the person or entity to which it is addressed and may contain II-VI Proprietary and/or II-VI Business Sensitive material. If you are not the intended recipient, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. You are notified that any review, retransmission, copying, disclosure, dissemination, or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. From stuart at lexacorp.com.pg Fri Mar 18 18:50:01 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 09:50:01 +1000 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: , Message-ID: <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg> That's about 10 hours and US$35 for me :-( -- Stuart On 18 Mar 2011 at 14:12, Drew Wutka wrote: > 500 megs... that's about 10 minutes... wouldn't have time for a full > dinner.... > > Drew > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com > Subject: [dba-VB] Visual Studio 2010 SP1 > > Hi all > > If you go for this: > http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8 > 10 7-475d-948a-ef22627e57a5&displaylang=en > > it's typically a 500MB+ download with a streaming install. > > Don't go for coffee during this, don't go for lunch, not even a > single-course dinner, don't settle for less than a full menu dinner > followed by coffee and brandy. Or book an evening in town! > > /gustav > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > The information contained in this transmission is intended only for > the person or entity to which it is addressed and may contain II-VI > Proprietary and/or II-VI Business Sensitive material. If you are not > the intended recipient, please contact the sender immediately and > destroy the material in its entirety, whether electronic or hard copy. > You are notified that any review, retransmission, copying, disclosure, > dissemination, or other use of, or taking of any action in reliance > upon this information by persons or entities other than the intended > recipient is prohibited. > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Mar 18 19:02:30 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 03:02:30 +0300 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg> References: , <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg> Message-ID: Hi Stuart -- <<< That's about 10 hours and US$35 for me :-( >>> But you never have a snow there, and your local police doesn't mind if you drive home your car after a couple of beers? :) I mean 500megs download takes just a couple of minutes here and near to zero expences but we still have a snow here and Finnish Gulf is covered by 1 meter+ wide ice and I'm skiing here almost everyday, and if stopped by local road police after a couple beers I would definitely loose my car driving license for 1.5 years, at least... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart McLachlan Sent: 19 ????? 2011 ?. 2:50 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Visual Studio 2010 SP1 That's about 10 hours and US$35 for me :-( -- Stuart On 18 Mar 2011 at 14:12, Drew Wutka wrote: > 500 megs... that's about 10 minutes... wouldn't have time for a full > dinner.... > > Drew > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com > Subject: [dba-VB] Visual Studio 2010 SP1 > > Hi all > > If you go for this: > http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8 > 10 7-475d-948a-ef22627e57a5&displaylang=en > > it's typically a 500MB+ download with a streaming install. > > Don't go for coffee during this, don't go for lunch, not even a > single-course dinner, don't settle for less than a full menu dinner > followed by coffee and brandy. Or book an evening in town! > > /gustav > > From stuart at lexacorp.com.pg Fri Mar 18 19:09:18 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 10:09:18 +1000 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: , <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg>, Message-ID: <4D83F42E.13556.8558BC@stuart.lexacorp.com.pg> Yep, The good points still outweigh the bad. :-) -- Stuart On 19 Mar 2011 at 3:02, Shamil Salakhetdinov wrote: > Hi Stuart -- > > <<< > That's about 10 hours and US$35 for me :-( > >>> > But you never have a snow there, and your local police doesn't mind if > you drive home your car after a couple of beers? :) > > I mean 500megs download takes just a couple of minutes here and near > to zero expences but we still have a snow here and Finnish Gulf is > covered by 1 meter+ wide ice and I'm skiing here almost everyday, and > if stopped by local road police after a couple beers I would > definitely loose my car driving license for 1.5 years, at least... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart > McLachlan Sent: 19 ????? 2011 ?. 2:50 To: Discussion concerning Visual > Basic and related programming issues. Subject: Re: [dba-VB] Visual > Studio 2010 SP1 > > That's about 10 hours and US$35 for me :-( > > -- > Stuart > > On 18 Mar 2011 at 14:12, Drew Wutka wrote: > > > 500 megs... that's about 10 minutes... wouldn't have time for a full > > dinner.... > > > > Drew > > > > -----Original Message----- > > From: dba-vb-bounces at databaseadvisors.com > > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav > > Brock Sent: Friday, March 11, 2011 11:59 AM To: > > dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 > > > > Hi all > > > > If you go for this: > > http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6 > > -8 10 7-475d-948a-ef22627e57a5&displaylang=en > > > > it's typically a 500MB+ download with a streaming install. > > > > Don't go for coffee during this, don't go for lunch, not even a > > single-course dinner, don't settle for less than a full menu dinner > > followed by coffee and brandy. Or book an evening in town! > > > > /gustav > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Mar 18 21:08:01 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 18 Mar 2011 22:08:01 -0400 Subject: [dba-VB] SHA1 to compute a hash Message-ID: <4D841001.2060200@colbyconsulting.com> In my databases I create SHA1 hashes to enable joining between tables and pull identical records (identical for the fields hashed). I create: 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the three values and feed them into SHa1 and out pops a number which I store in a field in my table. 2) A HashFamily of the Zip5, Zip4, Addr ad LName. 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. I am getting known collisions between different addresses (I have discovered and investigated collisions) in my HashAddr when I have many millions of addresses. I need to address this. Back when I made my design decisions (2004) my hardware consisted of single core processors, 4 gigs ram, Windows x32 etc. Now I have 8 cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great extent constrained by my hardware "back in the day" whereas I am much less so now. I am about to redesign my process. I am considering simply appending in the city and state strings to all of the inputs: Addr, City, St, Zip5, Zip4 as the address base and then the same with LName and FName for the other two respective hashes. The objective is to minimize hash collisions, not prevent some crypto attack. I use these hash fields to join between multi million record tables so If I need to discover info in TableA where the HashAddr is the same as in TableB, I need the probability of a collision between different addresses (family/Person) to be as close to zero as I can get it. My questions are: 1) Whether anyone out there is using a hash in this manner? 2) Has anyone seen a table of collision probability between messages of a given (short) message length. My message is 9 digits for the zip5/4 and the address could be something as short as PO Box 1, or Apt 1. IOW the total message length of 14 is pretty common. Adding the state would give me minimum message lengths of only 16 and City would only add a few more characters. 3) Does anyone know if just adding the same data back in again would decrease the collision probability. IOW Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. Any experience out there? -- John W. Colby www.ColbyConsulting.com From stuart at lexacorp.com.pg Sat Mar 19 05:20:16 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 20:20:16 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D841001.2060200@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com> Message-ID: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> How are you creating your hash? Can you post a few examples of different data strings and colliding SHA1 hashes. I can probably make a lot of money out of them. AFAIK, no one other than you has found any. -- Stuart On 18 Mar 2011 at 22:08, jwcolby wrote: > In my databases I create SHA1 hashes to enable joining between tables > and pull identical records (identical for the fields hashed). I > create: > > 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the > three values and feed them into SHa1 and out pops a number which I > store in a field in my table. > > 2) A HashFamily of the Zip5, Zip4, Addr ad LName. > > 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. > > I am getting known collisions between different addresses (I have > discovered and investigated collisions) in my HashAddr when I have > many millions of addresses. I need to address this. > > Back when I made my design decisions (2004) my hardware consisted of > single core processors, 4 gigs ram, Windows x32 etc. Now I have 8 > cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great extent > constrained by my hardware "back in the day" whereas I am much less so > now. > > I am about to redesign my process. > > I am considering simply appending in the city and state strings to all > of the inputs: Addr, City, St, Zip5, Zip4 as the address base and then > the same with LName and FName for the other two respective hashes. > > The objective is to minimize hash collisions, not prevent some crypto > attack. I use these hash fields to join between multi million record > tables so If I need to discover info in TableA where the HashAddr is > the same as in TableB, I need the probability of a collision between > different addresses (family/Person) to be as close to zero as I can > get it. > > My questions are: > > 1) Whether anyone out there is using a hash in this manner? > 2) Has anyone seen a table of collision probability between messages > of a given (short) message length. My message is 9 digits for the > zip5/4 and the address could be something as short as PO Box 1, or Apt > 1. IOW the total message length of 14 is pretty common. Adding the > state would give me minimum message lengths of only 16 and City would > only add a few more characters. 3) Does anyone know if just adding the > same data back in again would decrease the collision probability. IOW > Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. > > Any experience out there? > > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 06:20:26 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 21:20:26 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> Using your SHA1 function, what message digests do you get for the standard test cases: 1. abc 2. abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq These should return: 1. A9993E36 4706816A BA3E2571 7850C26C 9CD0D89D 2. 84983E44 1C3BD26E BAAE4AA1 F95129E5 E54670F1 See Examples 1 and 2 at http://www.itl.nist.gov/fipspubs/fip180-1.htm -- Stuart On 19 Mar 2011 at 20:20, Stuart McLachlan wrote: > How are you creating your hash? > > Can you post a few examples of different data strings and colliding > SHA1 hashes. I can probably make a lot of money out of them. > AFAIK, no one other than you has found any. > > -- > Stuart > > > On 18 Mar 2011 at 22:08, jwcolby wrote: > > > In my databases I create SHA1 hashes to enable joining between > > tables and pull identical records (identical for the fields hashed). > > I create: > > > > 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the > > three values and feed them into SHa1 and out pops a number which I > > store in a field in my table. > > > > 2) A HashFamily of the Zip5, Zip4, Addr ad LName. > > > > 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. > > > > I am getting known collisions between different addresses (I have > > discovered and investigated collisions) in my HashAddr when I have > > many millions of addresses. I need to address this. > > > > Back when I made my design decisions (2004) my hardware consisted of > > single core processors, 4 gigs ram, Windows x32 etc. Now I have 8 > > cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great extent > > constrained by my hardware "back in the day" whereas I am much less > > so now. > > > > I am about to redesign my process. > > > > I am considering simply appending in the city and state strings to > > all of the inputs: Addr, City, St, Zip5, Zip4 as the address base > > and then the same with LName and FName for the other two respective > > hashes. > > > > The objective is to minimize hash collisions, not prevent some > > crypto attack. I use these hash fields to join between multi > > million record tables so If I need to discover info in TableA where > > the HashAddr is the same as in TableB, I need the probability of a > > collision between different addresses (family/Person) to be as close > > to zero as I can get it. > > > > My questions are: > > > > 1) Whether anyone out there is using a hash in this manner? > > 2) Has anyone seen a table of collision probability between messages > > of a given (short) message length. My message is 9 digits for the > > zip5/4 and the address could be something as short as PO Box 1, or > > Apt 1. IOW the total message length of 14 is pretty common. Adding > > the state would give me minimum message lengths of only 16 and City > > would only add a few more characters. 3) Does anyone know if just > > adding the same data back in again would decrease the collision > > probability. IOW Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. > > > > Any experience out there? > > > > > > -- > > John W. Colby > > www.ColbyConsulting.com > > _______________________________________________ > > dba-VB mailing list > > dba-VB at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > http://www.databaseadvisors.com > > > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 07:06:47 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 22:06:47 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> Message-ID: <4D849C57.3867.31639E4@stuart.lexacorp.com.pg> That got my interest piqued. So I've just built a small DLL (13KB) to calculate SHA1 digests for either strings or files. All you need to use it is to put the DLL somewhere in your PATH such as %SYSTEMROOT\System32 and include a couple of declarations in your application. Declare Function SHA1StringDigest Lib "LXSHA1.DLL" Alias "SHA1STRINGDIGEST" (Message As String) As String Declare Function SHA1FileDigest Lib "LXSHA1.DLL" Alias "SHA1FILEDIGEST" (FileName As String) As String It correctly returns the test results below for either the string or a file containing the same data. If anyone wants a copy, let me know. -- Stuart On 19 Mar 2011 at 21:20, Stuart McLachlan wrote: > Using your SHA1 function, what message digests do you get for the > standard test cases: > > 1. abc > 2. abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq > > These should return: > > 1. A9993E36 4706816A BA3E2571 7850C26C 9CD0D89D > 2. 84983E44 1C3BD26E BAAE4AA1 F95129E5 E54670F1 > > See Examples 1 and 2 at > http://www.itl.nist.gov/fipspubs/fip180-1.htm > > > > -- > Stuart > > > On 19 Mar 2011 at 20:20, Stuart McLachlan wrote: > > > How are you creating your hash? > > > > Can you post a few examples of different data strings and colliding > > SHA1 hashes. I can probably make a lot of money out of them. > > AFAIK, no one other than you has found any. > > > > -- > > Stuart > > > > > > On 18 Mar 2011 at 22:08, jwcolby wrote: > > > > > In my databases I create SHA1 hashes to enable joining between > > > tables and pull identical records (identical for the fields > > > hashed). > > > I create: > > > > > > 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the > > > three values and feed them into SHa1 and out pops a number which I > > > store in a field in my table. > > > > > > 2) A HashFamily of the Zip5, Zip4, Addr ad LName. > > > > > > 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. > > > > > > I am getting known collisions between different addresses (I have > > > discovered and investigated collisions) in my HashAddr when I have > > > many millions of addresses. I need to address this. > > > > > > Back when I made my design decisions (2004) my hardware consisted > > > of single core processors, 4 gigs ram, Windows x32 etc. Now I > > > have 8 cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great > > > extent constrained by my hardware "back in the day" whereas I am > > > much less so now. > > > > > > I am about to redesign my process. > > > > > > I am considering simply appending in the city and state strings to > > > all of the inputs: Addr, City, St, Zip5, Zip4 as the address base > > > and then the same with LName and FName for the other two > > > respective hashes. > > > > > > The objective is to minimize hash collisions, not prevent some > > > crypto attack. I use these hash fields to join between multi > > > million record tables so If I need to discover info in TableA > > > where the HashAddr is the same as in TableB, I need the > > > probability of a collision between different addresses > > > (family/Person) to be as close to zero as I can get it. > > > > > > My questions are: > > > > > > 1) Whether anyone out there is using a hash in this manner? > > > 2) Has anyone seen a table of collision probability between > > > messages of a given (short) message length. My message is 9 > > > digits for the zip5/4 and the address could be something as short > > > as PO Box 1, or Apt 1. IOW the total message length of 14 is > > > pretty common. Adding the state would give me minimum message > > > lengths of only 16 and City would only add a few more characters. > > > 3) Does anyone know if just adding the same data back in again > > > would decrease the collision probability. IOW > > > Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. > > > > > > Any experience out there? > > > > > > > > > -- > > > John W. Colby > > > www.ColbyConsulting.com > > > _______________________________________________ > > > dba-VB mailing list > > > dba-VB at databaseadvisors.com > > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > > http://www.databaseadvisors.com > > > > > > > > > > > > > > _______________________________________________ > > dba-VB mailing list > > dba-VB at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > http://www.databaseadvisors.com > > > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 07:09:56 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 08:09:56 -0400 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com> <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: <4D849D14.70906@colbyconsulting.com> >AFAIK, no one other than you has found any. LOL. You are thinking of hashing messages with thousands of characters. here's the deal. Let's assume that I hash X characters in a single field. The message length is the key, not the number of fields. Assume also that I am really only hashing letters, numbers and special characters - the characters that are in a name or address. If I hash 100 1 character messages, my probability of a collision if extremely high since there are only about 128 such alphanumeric characters (even less). Now hash a thousand 1 character messages. The probably just climbed immensely. Now hash a million such 1 character messages. What I am really calculating is the probability that I will repeat the same character string. However if I up the number of characters, let's say 10 character messages. *Now* the hash is really trying to prevent the same output for two *different* inputs. Well that is always what a hash is trying to do. But the point is that the probability of a collision decreases with message length increase. The probability of a collision between any two messages of 10 random characters is much higher than the probability of a collision between any two 100 character messages, and the probability of a collision between any two 1000 character messages is much higher still. That is just the way hash functions work. Nobody claims that hashes don't create collisions, and everyone agrees that the longer the message, the lower the probability of a collision between two messages (to a point). So, my address "messages" currently look like 89364 4456 PO Box 1 76543 9876 Apt 2 97867 3546 1723 Twin Pines Dr The point here is that all address "messages" are short. Now turn the address message into 89364 4456 PO Box 1 CA San Diego 76543 9876 Apt 2 TX Dallas 97867 3546 1723 Twin Pines Dr NC Hudson And the length of the message increases fairly dramatically, decreasing the probability of a collision. Remember too that I am hashing hundreds of millions of records - about 350 million addresses in various tables so far. The other thing that affects the probability of a hash collision is the number of messages hashed. Hash enough records and you *will* 100% probability create a hash that is the same for two different input strings - even for long messages. That is just the nature of the business. The probability of a collision is determined by three things. 1) The (strength of the) hash algorithm. 2) The message length 3) The quantity of messages. 1) I am using a fairly strong hash algorithm - SHA1 2) I am using a fairly short hash message - Zip5 / zip4 / Addr 3) I am hashing a fairly high number of messages - 350 million (and counting) messages (addresses) Believe me, hashes have already occurred, I have seen them. It is an easy thing to test. Join two tables on the same hash and in the query compare that each address field is the same between the two tables. Zip5 <> zip5 *or* zip4 <> zip4 *or* addr <> addr. If the hash is the same (inner join) AND one of the three fields is different then you have a collision. I have done this test and I have found collisions between my address hash. I have never tested my last name or first name hash fields. The problem with finding hashes is that they are rare, and in order to do an exhaustive test I would need to pull all the hash and data fields into a single table and compare that table against itself. My data is contained in a bunch of databases. 11 million "dogs and cats", 21 million "kids", 7 million "smokers", 23 million (and about to double) emails. 65 million "all adults", 100 million "mortgage" etc. So it is difficult to find hash collisions because (at least in the small tables) the probability of a collision is low entirely within a table. Believe me, it *does* happen though (at least for the address hash). John W. Colby www.ColbyConsulting.com On 3/19/2011 6:20 AM, Stuart McLachlan wrote: > How are you creating your hash? > > Can you post a few examples of different data strings and colliding SHA1 hashes. I can > probably make a lot of money out of them. AFAIK, no one other than you has found any. > From jwcolby at colbyconsulting.com Sat Mar 19 07:11:32 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 08:11:32 -0400 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D849C57.3867.31639E4@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> <4D849C57.3867.31639E4@stuart.lexacorp.com.pg> Message-ID: <4D849D74.2040104@colbyconsulting.com> I will do a collision test in my email file. AFAICT I have some collisions in there. If I find any I will send you the message and hash so that you can see what collisions look like. John W. Colby www.ColbyConsulting.com On 3/19/2011 8:06 AM, Stuart McLachlan wrote: > That got my interest piqued. > > So I've just built a small DLL (13KB) to calculate SHA1 digests for either strings or files. > All you need to use it is to put the DLL somewhere in your PATH such as > %SYSTEMROOT\System32 and include a couple of declarations in your application. > > Declare Function SHA1StringDigest Lib "LXSHA1.DLL" Alias "SHA1STRINGDIGEST" > (Message As String) As String > > Declare Function SHA1FileDigest Lib "LXSHA1.DLL" Alias "SHA1FILEDIGEST" (FileName > As String) As String > > It correctly returns the test results below for either the string or a file containing the same > data. > > If anyone wants a copy, let me know. > From harlan at powerzone.com Sat Mar 19 07:17:25 2011 From: harlan at powerzone.com (Harlan Koehn) Date: Sat, 19 Mar 2011 06:17:25 -0600 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D849D14.70906@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com> <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> <4D849D14.70906@colbyconsulting.com> Message-ID: <76EEB091F7CED543A80B5C2BC153975E0165A7C108@PZSERVER01.powerzone.local> John, I'm fairly ignorant when it comes to hashing algorithms but I'm curious if you have considered using a hashing algorithm that would generate a longer hash, which I'm assuming could reduce the likelihood of a collision. Harlan From stuart at lexacorp.com.pg Sat Mar 19 07:51:13 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 22:51:13 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D849D14.70906@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D849D14.70906@colbyconsulting.com> Message-ID: <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> In the case of 1000 single character messages, you are bound to get collisions since there are only 256 possible original messages. You will be hashing the same value multiple times Apart from that, your understanding is incorrect. It doesn't matter how long the string is. The chances of a collision with two different messages remains the same. The message is hashed using padded blocks of a fixed length. There is no more chance of a collision between "a" and "b" than there is between "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". Specifically, the chance of a collision within n different messages, using b bits of encryption is (n*(n-1)/2) * (1/2^b). Note that the length of the message doesn't come into that equation. The probability of a collision is determined by only TWO things. 1) The length of the digest. 2) The quantity of messages. As for "everyone agrees". Noone who understands how it works, agrees. -- Stuart On 19 Mar 2011 at 8:09, jwcolby wrote: > >AFAIK, no one other than you has found any. > > LOL. > > You are thinking of hashing messages with thousands of characters. > here's the deal. > > Let's assume that I hash X characters in a single field. The message > length is the key, not the number of fields. Assume also that I am > really only hashing letters, numbers and special characters - the > characters that are in a name or address. > > > If I hash 100 1 character messages, my probability of a collision if > extremely high since there are only about 128 such alphanumeric > characters (even less). > > Now hash a thousand 1 character messages. The probably just climbed > immensely. Now hash a million such 1 character messages. > What I am really calculating is the probability that I will repeat the > same character string. > > However if I up the number of characters, let's say 10 character > messages. *Now* the hash is really trying to prevent the same output > for two *different* inputs. Well that is always what a hash is trying > to do. But the point is that the probability of a collision decreases > with message length increase. > > The probability of a collision between any two messages of 10 random > characters is much higher than the probability of a collision between > any two 100 character messages, and the probability of a collision > between any two 1000 character messages is much higher still. > > That is just the way hash functions work. Nobody claims that hashes > don't create collisions, and everyone agrees that the longer the > message, the lower the probability of a collision between two messages > (to a point). > > So, my address "messages" currently look like > > 89364 4456 PO Box 1 > 76543 9876 Apt 2 > 97867 3546 1723 Twin Pines Dr > > The point here is that all address "messages" are short. Now turn the > address message into > > 89364 4456 PO Box 1 CA San Diego > 76543 9876 Apt 2 TX Dallas > 97867 3546 1723 Twin Pines Dr NC Hudson > > And the length of the message increases fairly dramatically, > decreasing the probability of a collision. > > Remember too that I am hashing hundreds of millions of records - about > 350 million addresses in various tables so far. The other thing that > affects the probability of a hash collision is the number of messages > hashed. Hash enough records and you *will* 100% probability create a > hash that is the same for two different input strings - even for long > messages. That is just the nature of the business. > > The probability of a collision is determined by three things. > > 1) The (strength of the) hash algorithm. > 2) The message length > 3) The quantity of messages. ... From shamil at smsconsulting.spb.ru Sat Mar 19 08:03:36 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 16:03:36 +0300 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com> <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: Hi Stuart and John, <<< AFAIK, no one other than you has found any >>> Yes, I was also confused how it comes to get SHA1 hashes collisions but then I have thought that John probably "compacts" SHA1 hashes into 8 bytes surrogate keys? Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions for 6 millions cycles for 8 bytes keys. Please correct me if you find some errors in my sample code Thank you. -- Shamil P.S. Code: public class GenerateHashes { private static Dictionary _generatedHashes; private static Dictionary _generatedLongHashes; public static void Run(int minHashedStringLength, int maxHashedStringLength, long testCasesQty) { System.Console.WriteLine("Started at: " + System.DateTime.Now); _generatedHashes = new Dictionary(); _generatedLongHashes = new Dictionary(); int collisionsQty = 0; int longHashCollisionsQty = 0; decimal totalLengthOfProcessedStrings = 0; decimal averageStringLength = 0; Random random = new Random((int)System.DateTime.Now.Ticks); for (long i = 1; i <= testCasesQty; i++) { int stringLength = random.Next(minHashedStringLength, maxHashedStringLength); byte[] plainTextBytes = (byte[])Array.CreateInstance(typeof(byte), stringLength); random.NextBytes(plainTextBytes); System.Security.Cryptography.HashAlgorithm hash = new SHA1Managed(); byte[] hashBytes = hash.ComputeHash(plainTextBytes); string hashValue = Convert.ToBase64String(hashBytes); string testValue; if (_generatedHashes.TryGetValue(hashValue, out testValue)) collisionsQty++; else _generatedHashes.Add(hashValue, hashValue); if (i % 1000000 == 0) System.Console.WriteLine( "{0}: i={1}, cs={2}, cu={3}", DateTime.Now, i, collisionsQty, longHashCollisionsQty); ulong longHash = 0xFFFFFFFF; foreach (byte b in hashValue) { longHash ^= b; longHash = longHash << 8 + 0xFF; } ulong longHashTestValue; if (_generatedLongHashes.TryGetValue(longHash, out longHashTestValue)) { //Long Hash Collision longHashCollisionsQty++; System.Console.WriteLine(" * step {0}, hash '{1}', longHash: {2:X}", i, hashValue, longHash); } else _generatedLongHashes.Add(longHash, longHash); totalLengthOfProcessedStrings += stringLength; averageStringLength = totalLengthOfProcessedStrings / i; } System.Console.WriteLine( "\nTOTALS:\n" + "Min String Length = {0}\n" + "Max String Length = {1}\n" + "Avg String Length = {2}\n" + "Test cases Qty = {3}\n" + "Collisions Qty = {4}\n" + "ulong Collisions Qty = {5}", minHashedStringLength, maxHashedStringLength, averageStringLength, testCasesQty, collisionsQty, longHashCollisionsQty); System.Console.WriteLine("Finished at: " + System.DateTime.Now); } } P.P.S. Stats Started at: 19/03/2011 15:51:04 19/03/2011 15:51:16: i=1000000, cs=0, cu=0 * step 1712197, hash 'Rt4NY5FHg0uMDszVql+l6ksx6Tg=', longHash: 36D7CFC36A99DE80 19/03/2011 15:51:29: i=2000000, cs=0, cu=1 * step 2040324, hash 'laoPHdqlfeZOx9DI8RDaQZjhKZs=', longHash: D1B5AB44BB5CDE80 * step 2043449, hash '1m+tgKu7/l9fl8nUNuz5I/OWF5E=', longHash: C95F3EBC66B15E80 19/03/2011 15:51:44: i=3000000, cs=0, cu=3 19/03/2011 15:52:00: i=4000000, cs=0, cu=3 * step 4369446, hash '2sZrglXDp3NnNPuqdILqwmUKq30=', longHash: F7DB565F166C1E80 19/03/2011 15:52:17: i=5000000, cs=0, cu=4 TOTALS: Min String Length = 7 Max String Length = 25 Avg String Length = 15.4981354 Test cases Qty = 5,000,000 Collisions Qty = 0 ulong Collisions Qty = 4 Finished at: 19/03/2011 15:52:17 -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart McLachlan Sent: 19 ????? 2011 ?. 13:20 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SHA1 to compute a hash How are you creating your hash? Can you post a few examples of different data strings and colliding SHA1 hashes. I can probably make a lot of money out of them. AFAIK, no one other than you has found any. -- Stuart <<< snip >>> From stuart at lexacorp.com.pg Sat Mar 19 08:10:47 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 23:10:47 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D849D14.70906@colbyconsulting.com>, <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> Message-ID: <4D84AB57.31948.350CF8F@stuart.lexacorp.com.pg> I've just realised where the confusion comes in. When you talk about the chance of collisions increasing decreasing message length, you are talking about general hashing algorithms such as CRC etc. You need to undersand the difference between simple and cryptographic hashing. Cryptographic hashing algorithms are specifically designed to be "collision resistant". I repeat, to date no one has found a collision in SHA1 (except you apparently ) -- Stuart On 19 Mar 2011 at 22:51, Stuart McLachlan wrote: > In the case of 1000 single character messages, you are bound to get > collisions since there are only 256 possible original messages. You > will be hashing the same value multiple times > > Apart from that, your understanding is incorrect. It doesn't matter > how long the string is. The chances of a collision with two different > messages remains the same. > > The message is hashed using padded blocks of a fixed length. > There is no more chance of a collision between "a" and "b" than there > is between "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". > > Specifically, the chance of a collision within n different messages, > using b bits of encryption is (n*(n-1)/2) * (1/2^b). > > Note that the length of the message doesn't come into that equation. > > The probability of a collision is determined by only TWO things. > > 1) The length of the digest. > 2) The quantity of messages. > > As for "everyone agrees". Noone who understands how it works, agrees. > > -- > Stuart > > On 19 Mar 2011 at 8:09, jwcolby wrote: > > > >AFAIK, no one other than you has found any. > > > > LOL. > > > > You are thinking of hashing messages with thousands of characters. > > here's the deal. > > > > Let's assume that I hash X characters in a single field. The > > message length is the key, not the number of fields. Assume also > > that I am really only hashing letters, numbers and special > > characters - the characters that are in a name or address. > > > > > > If I hash 100 1 character messages, my probability of a collision if > > extremely high since there are only about 128 such alphanumeric > > characters (even less). > > > > Now hash a thousand 1 character messages. The probably just climbed > > immensely. Now hash a million such 1 character messages. What I am > > really calculating is the probability that I will repeat the same > > character string. > > > > However if I up the number of characters, let's say 10 character > > messages. *Now* the hash is really trying to prevent the same > > output for two *different* inputs. Well that is always what a hash > > is trying to do. But the point is that the probability of a > > collision decreases with message length increase. > > > > The probability of a collision between any two messages of 10 random > > characters is much higher than the probability of a collision > > between any two 100 character messages, and the probability of a > > collision between any two 1000 character messages is much higher > > still. > > > > That is just the way hash functions work. Nobody claims that hashes > > don't create collisions, and everyone agrees that the longer the > > message, the lower the probability of a collision between two > > messages (to a point). > > > > So, my address "messages" currently look like > > > > 89364 4456 PO Box 1 > > 76543 9876 Apt 2 > > 97867 3546 1723 Twin Pines Dr > > > > The point here is that all address "messages" are short. Now turn > > the address message into > > > > 89364 4456 PO Box 1 CA San Diego > > 76543 9876 Apt 2 TX Dallas > > 97867 3546 1723 Twin Pines Dr NC Hudson > > > > And the length of the message increases fairly dramatically, > > decreasing the probability of a collision. > > > > Remember too that I am hashing hundreds of millions of records - > > about 350 million addresses in various tables so far. The other > > thing that affects the probability of a hash collision is the number > > of messages hashed. Hash enough records and you *will* 100% > > probability create a hash that is the same for two different input > > strings - even for long messages. That is just the nature of the > > business. > > > > The probability of a collision is determined by three things. > > > > 1) The (strength of the) hash algorithm. > > 2) The message length > > 3) The quantity of messages. > ... > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 08:20:11 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 23:20:11 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, Message-ID: <4D84AD8B.14064.3596D15@stuart.lexacorp.com.pg> If he is compacting a 160 bit hash to a 64 bit hash, he is certainly increasing the odds of a collision - but I suspect that even at 64 bits, the chances of a collision would be very low. But that does not appear to be what you are testing. It looks as though you created 6 million random strings and four of them turned out to be identical. Therefore also had the same SHA1 digest. What happens if you try it again, making sure that all of your strings are unique. Alternatively, what do you get if you add some code to show the pairs of strings. Are they different? -- Stuart On 19 Mar 2011 at 16:03, Shamil Salakhetdinov wrote: > Hi Stuart and John, > > <<< > AFAIK, no one other than you has found any > >>> > Yes, I was also confused how it comes to get SHA1 hashes collisions > but then I have thought that John probably "compacts" SHA1 hashes into > 8 bytes surrogate keys? > > Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions > for 6 millions cycles for 8 bytes keys. > > Please correct me if you find some errors in my sample code > > Thank you. > > -- > Shamil > > P.S. Code: > > public class GenerateHashes > { > private static Dictionary _generatedHashes; > private static Dictionary _generatedLongHashes; > > public static void Run(int minHashedStringLength, int > maxHashedStringLength, long testCasesQty) > { > System.Console.WriteLine("Started at: " + > System.DateTime.Now); > > _generatedHashes = new Dictionary(); > _generatedLongHashes = new Dictionary(); > > int collisionsQty = 0; > int longHashCollisionsQty = 0; > > decimal totalLengthOfProcessedStrings = 0; > decimal averageStringLength = 0; > > > Random random = new > Random((int)System.DateTime.Now.Ticks); for (long i = 1; i > <= testCasesQty; i++) { > int stringLength = random.Next(minHashedStringLength, > maxHashedStringLength); > byte[] plainTextBytes = > (byte[])Array.CreateInstance(typeof(byte), stringLength); > random.NextBytes(plainTextBytes); > System.Security.Cryptography.HashAlgorithm hash = new > SHA1Managed(); > byte[] hashBytes = hash.ComputeHash(plainTextBytes); > string hashValue = Convert.ToBase64String(hashBytes); > string testValue; if > (_generatedHashes.TryGetValue(hashValue, out > testValue)) > collisionsQty++; > else _generatedHashes.Add(hashValue, hashValue); > if (i % 1000000 == 0) > System.Console.WriteLine( > "{0}: i={1}, cs={2}, cu={3}", > DateTime.Now, i, collisionsQty, > longHashCollisionsQty); > > ulong longHash = 0xFFFFFFFF; > foreach (byte b in hashValue) > { > longHash ^= b; > longHash = longHash << 8 + 0xFF; > } > ulong longHashTestValue; > if (_generatedLongHashes.TryGetValue(longHash, out > longHashTestValue)) > { > //Long Hash Collision > longHashCollisionsQty++; > System.Console.WriteLine(" * step {0}, hash '{1}', > longHash: {2:X}", > i, hashValue, longHash); > } > else _generatedLongHashes.Add(longHash, longHash); > > totalLengthOfProcessedStrings += stringLength; > averageStringLength = totalLengthOfProcessedStrings / > i; > } > > System.Console.WriteLine( > "\nTOTALS:\n" + > "Min String Length = {0}\n" + > "Max String Length = {1}\n" + > "Avg String Length = {2}\n" + > "Test cases Qty = {3}\n" + > "Collisions Qty = {4}\n" + > "ulong Collisions Qty = {5}", > minHashedStringLength, maxHashedStringLength, > averageStringLength, testCasesQty, > collisionsQty, > longHashCollisionsQty); > > System.Console.WriteLine("Finished at: " + > System.DateTime.Now); > > } > } > > > P.P.S. Stats > > Started at: 19/03/2011 15:51:04 > 19/03/2011 15:51:16: i=1000000, cs=0, cu=0 > * step 1712197, hash 'Rt4NY5FHg0uMDszVql+l6ksx6Tg=', longHash: > 36D7CFC36A99DE80 > 19/03/2011 15:51:29: i=2000000, cs=0, cu=1 > * step 2040324, hash 'laoPHdqlfeZOx9DI8RDaQZjhKZs=', longHash: > D1B5AB44BB5CDE80 > * step 2043449, hash '1m+tgKu7/l9fl8nUNuz5I/OWF5E=', longHash: > C95F3EBC66B15E80 > 19/03/2011 15:51:44: i=3000000, cs=0, cu=3 > 19/03/2011 15:52:00: i=4000000, cs=0, cu=3 > * step 4369446, hash '2sZrglXDp3NnNPuqdILqwmUKq30=', longHash: > F7DB565F166C1E80 > 19/03/2011 15:52:17: i=5000000, cs=0, cu=4 > > TOTALS: > Min String Length = 7 > Max String Length = 25 > Avg String Length = 15.4981354 > Test cases Qty = 5,000,000 > Collisions Qty = 0 > ulong Collisions Qty = 4 > Finished at: 19/03/2011 15:52:17 > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart > McLachlan Sent: 19 ????? 2011 ?. 13:20 To: Discussion concerning > Visual Basic and related programming issues. Subject: Re: [dba-VB] > SHA1 to compute a hash > > How are you creating your hash? > > Can you post a few examples of different data strings and colliding > SHA1 hashes. I can probably make a lot of money out of them. > AFAIK, no one other than you has found any. > > -- > Stuart > > <<< snip >>> > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 08:21:58 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 09:21:58 -0400 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D849D14.70906@colbyconsulting.com> <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> Message-ID: <4D84ADF6.4000509@colbyconsulting.com> Stuart, Thanks for that. I am not a cryptologist and all I can do is try to use a tool I do not understand to solve a problem I do understand. If my hashes really are that "unique / secure" then I will stop worrying about it. I still need to do a check of actual data though because I went looking one day and discovered "collisions". Given that we believe the collision is not in the key itself, it must have been in my method of finding the collision. In any case, I am not going to worry any more so thanks for the peace of mind! ;) John W. Colby www.ColbyConsulting.com On 3/19/2011 8:51 AM, Stuart McLachlan wrote: > In the case of 1000 single character messages, you are bound to get collisions since there > are only 256 possible original messages. You will be hashing the same value multiple times > > Apart from that, your understanding is incorrect. It doesn't matter how long the string is. The > chances of a collision with two different messages remains the same. > > The message is hashed using padded blocks of a fixed length. > There is no more chance of a collision between "a" and "b" than there is between > "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". > > Specifically, the chance of a collision within n different messages, using b bits of encryption is > (n*(n-1)/2) * (1/2^b). > > Note that the length of the message doesn't come into that equation. > > The probability of a collision is determined by only TWO things. > > 1) The length of the digest. > 2) The quantity of messages. > > As for "everyone agrees". Noone who understands how it works, agrees. > From shamil at smsconsulting.spb.ru Sat Mar 19 08:21:26 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 16:21:26 +0300 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: References: <4D841001.2060200@colbyconsulting.com><4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: <49B2F00A3FD3442685D8283AE7678544@nant> Hi All -- Sorry I have mentioned 6 millions cycles but my sample was for just 5 millions, and I have got some collisions for 8 bytes long surrogate keys. Here I'm posting a patch for hashing part of the code - and no collisions now for 8 bytes long hashes for 5 millions cycles. .... UInt64 longHashResult = 0xFFFFFFFFFFFFFFFF; UInt64 longHash = 0xFFFFFFFFFFFFFFFF; int shiftCounter = 0; foreach (byte b in hashValue) { longHash ^= b; longHash = longHash << 8 + 0xFF; if (++shiftCounter == 8) { longHashResult ^= longHash; shiftCounter = 0; } } longHash = longHashResult; ... Started at: 19/03/2011 16:08:53 19/03/2011 16:09:04: i=1000000, cs=0, cu=0 19/03/2011 16:09:17: i=2000000, cs=0, cu=0 19/03/2011 16:09:33: i=3000000, cs=0, cu=0 19/03/2011 16:09:49: i=4000000, cs=0, cu=0 19/03/2011 16:10:06: i=5000000, cs=0, cu=0 TOTALS: Min String Length = 7 Max String Length = 25 Avg String Length = 15.4966972 Test cases Qty = 5000000 Collisions Qty = 0 ulong Collisions Qty = 0 Finished at: 19/03/2011 16:10:06 Please correct me if I've got wrong somewhere. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 19 ????? 2011 ?. 16:04 To: stuart at lexacorp.com.pg; 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] SHA1 to compute a hash Hi Stuart and John, <<< AFAIK, no one other than you has found any >>> Yes, I was also confused how it comes to get SHA1 hashes collisions but then I have thought that John probably "compacts" SHA1 hashes into 8 bytes surrogate keys? Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions for 6 millions cycles for 8 bytes keys. Please correct me if you find some errors in my sample code Thank you. -- Shamil <<< snip >>> From jwcolby at colbyconsulting.com Sat Mar 19 08:24:54 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 09:24:54 -0400 Subject: [dba-VB] On a lighter note Message-ID: <4D84AEA6.5060107@colbyconsulting.com> I rooted my droid X last night, bought a "freezer" program, froze all the bloatware, and downloaded and installed a tether program. Here at my house I get about 1.25 mbit down and .5 mbit up via the tether. Not bad considering it is over cell. -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sat Mar 19 08:24:33 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 16:24:33 +0300 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84AD8B.14064.3596D15@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D84AD8B.14064.3596D15@stuart.lexacorp.com.pg> Message-ID: <14BB8E00F24E4EBAB5842E5D75537D2F@nant> Hi Stuart -- <<< It looks as though you created 6 million random strings and four of them turned out to be identical. >>> No, all SHA1 strings are different - these are 64 bit hashes of that strings, which have got some collisions. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart McLachlan Sent: 19 ????? 2011 ?. 16:20 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SHA1 to compute a hash If he is compacting a 160 bit hash to a 64 bit hash, he is certainly increasing the odds of a collision - but I suspect that even at 64 bits, the chances of a collision would be very low. But that does not appear to be what you are testing. It looks as though you created 6 million random strings and four of them turned out to be identical. Therefore also had the same SHA1 digest. What happens if you try it again, making sure that all of your strings are unique. Alternatively, what do you get if you add some code to show the pairs of strings. Are they different? -- Stuart On 19 Mar 2011 at 16:03, Shamil Salakhetdinov wrote: > Hi Stuart and John, > > <<< > AFAIK, no one other than you has found any > >>> > Yes, I was also confused how it comes to get SHA1 hashes collisions > but then I have thought that John probably "compacts" SHA1 hashes into > 8 bytes surrogate keys? > > Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions > for 6 millions cycles for 8 bytes keys. > > Please correct me if you find some errors in my sample code > > Thank you. > > -- > Shamil > <<< snip >>> From stuart at lexacorp.com.pg Sat Mar 19 08:29:55 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 23:29:55 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84ADF6.4000509@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com>, <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg>, <4D84ADF6.4000509@colbyconsulting.com> Message-ID: <4D84AFD3.25991.3625487@stuart.lexacorp.com.pg> Using a 160 bit SHA1, the odds of getting a collision are 50% when you have approximately 2^80 messages. If you ever get near that number of records, I suggest you switch to SHA2 which is 256 bit :-) -- Stuart On 19 Mar 2011 at 9:21, jwcolby wrote: > Stuart, > > Thanks for that. I am not a cryptologist and all I can do is try to > use a tool I do not understand to solve a problem I do understand. If > my hashes really are that "unique / secure" then I will stop worrying > about it. I still need to do a check of actual data though because I > went looking one day and discovered "collisions". Given that we > believe the collision is not in the key itself, it must have been in > my method of finding the collision. > > In any case, I am not going to worry any more so thanks for the peace > of mind! ;) > > John W. Colby > www.ColbyConsulting.com > > On 3/19/2011 8:51 AM, Stuart McLachlan wrote: > > In the case of 1000 single character messages, you are bound to get > > collisions since there are only 256 possible original messages. You > > will be hashing the same value multiple times > > > > Apart from that, your understanding is incorrect. It doesn't matter > > how long the string is. The chances of a collision with two > > different messages remains the same. > > > > The message is hashed using padded blocks of a fixed length. > > There is no more chance of a collision between "a" and "b" than > > there is between "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". > > > > Specifically, the chance of a collision within n different messages, > > using b bits of encryption is (n*(n-1)/2) * (1/2^b). > > > > Note that the length of the message doesn't come into that equation. > > > > The probability of a collision is determined by only TWO things. > > > > 1) The length of the digest. > > 2) The quantity of messages. > > > > As for "everyone agrees". Noone who understands how it works, > > agrees. > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 08:55:51 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 09:55:51 -0400 Subject: [dba-VB] Using the hash for real world problems Message-ID: <4D84B5E7.3010204@colbyconsulting.com> I got into the hashing business in order to create a surrogate key of sorts, in fact a trio of them. I have an address hash, a family hash and a person hash. I compute these three hashes for every table of persons. Now that I have these surrogate keys I have "identical" values in a single field where the input is identical in multiple fields. Thus I do not have to do multi-field joins where I have a hash for all of those fields. I am happy to hear that I am secure in my surrogate hash algorithm choice. OK, so I have these hashes. Every record has these three hashes. Now some definitions: HashAddr: A unique address is addr/zip5/zip4 different HashFamily: A unique family is LName/Addr/Zip5/Zip4 different HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. Using this information, I need to calculate the count of addresses with a single person, with two persons etc for as many combinations as I have. Addresses People 22,538,240 1 780,462,346 2 52,234 3 people etc etc till every quantity of people is found. What is the simplest way to accomplish this with SQL. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 19 09:13:51 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 10:13:51 -0400 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84B5E7.3010204@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com> Message-ID: <4D84BA1F.2060101@colbyconsulting.com> Ooops. My examples were hosed. Let's try again Addresses People 22,538,240 1 00,708,462 2 00,052,234 3 and so forth. What the client really wants to know is "how many people with emails are at a specific address" "how many people with emails are in a specific family" etc. I tie my address table to my email table using an auto increment long increment "I must be careful not to call this a surrogate" key and then perform the counts using the hash fields in a Group By / count query. John W. Colby www.ColbyConsulting.com On 3/19/2011 9:55 AM, jwcolby wrote: > I got into the hashing business in order to create a surrogate key of sorts, in fact a trio of them. > > I have an address hash, a family hash and a person hash. I compute these three hashes for every > table of persons. Now that I have these surrogate keys I have "identical" values in a single field > where the input is identical in multiple fields. Thus I do not have to do multi-field joins where I > have a hash for all of those fields. > > I am happy to hear that I am secure in my surrogate hash algorithm choice. > > OK, so I have these hashes. Every record has these three hashes. > > Now some definitions: > > HashAddr: A unique address is addr/zip5/zip4 different > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > Using this information, I need to calculate the count of addresses with a single person, with two > persons etc for as many combinations as I have. > > Addresses People > 22,538,240 1 > 780,462,346 2 > 52,234 3 people > etc > etc till every quantity of people is found. > > What is the simplest way to accomplish this with SQL. > > > > From shamil at smsconsulting.spb.ru Sat Mar 19 10:07:08 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 18:07:08 +0300 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84BA1F.2060101@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com> <4D84BA1F.2060101@colbyconsulting.com> Message-ID: Hi John -- I suppose you can reuse the following SQL expression for your case: select count(Occurrence) QtyTheSameOccurrences, Occurrence from ( select Country, Count(Country) Occurrence from Customer Group by Country ) o group by o.Occurrence order by o.Occurrence Just replace Country name columns with hashAddress column and Country table name with your table name ... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 19 ????? 2011 ?. 17:14 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Using the hash for real world problems Ooops. My examples were hosed. Let's try again Addresses People 22,538,240 1 00,708,462 2 00,052,234 3 and so forth. What the client really wants to know is "how many people with emails are at a specific address" "how many people with emails are in a specific family" etc. I tie my address table to my email table using an auto increment long increment "I must be careful not to call this a surrogate" key and then perform the counts using the hash fields in a Group By / count query. John W. Colby www.ColbyConsulting.com On 3/19/2011 9:55 AM, jwcolby wrote: > I got into the hashing business in order to create a surrogate key of sorts, in fact a trio of them. > > I have an address hash, a family hash and a person hash. I compute > these three hashes for every table of persons. Now that I have these > surrogate keys I have "identical" values in a single field where the > input is identical in multiple fields. Thus I do not have to do multi-field joins where I have a hash for all of those fields. > > I am happy to hear that I am secure in my surrogate hash algorithm choice. > > OK, so I have these hashes. Every record has these three hashes. > > Now some definitions: > > HashAddr: A unique address is addr/zip5/zip4 different > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > Using this information, I need to calculate the count of addresses > with a single person, with two persons etc for as many combinations as I have. > > Addresses People > 22,538,240 1 > 780,462,346 2 > 52,234 3 people > etc > etc till every quantity of people is found. > > What is the simplest way to accomplish this with SQL. > > > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Mar 19 14:50:55 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 15:50:55 -0400 Subject: [dba-VB] Most useful droid apps Message-ID: <4D85091F.5060807@colbyconsulting.com> What are the most useful apps you run on your droid? -- John W. Colby www.ColbyConsulting.com From davidmcafee at gmail.com Sat Mar 19 15:53:32 2011 From: davidmcafee at gmail.com (David McAfee) Date: Sat, 19 Mar 2011 13:53:32 -0700 Subject: [dba-VB] Most useful droid apps Message-ID: Advanced task killer Weather channel Lookout antivirus AK Notepad Bank of America Acar auto maintenance Barcode scanner Bubble level Bump Droid light Google goggles Google translate IMDB Key ring Print share Trapster Sent from my Droid phone. On Mar 19, 2011 12:52 PM, "jwcolby" wrote: From stuart at lexacorp.com.pg Sat Mar 19 18:11:26 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sun, 20 Mar 2011 09:11:26 +1000 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84BA1F.2060101@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com>, <4D84BA1F.2060101@colbyconsulting.com> Message-ID: <4D85381E.9973.576BE3C@stuart.lexacorp.com.pg> Caaution. All aircode below: To get the list output below, I would use something like Select Distinct Count(HashAddr) as Addresses, People >From (Select distinct HashAddr,Count(HashAddr) as People From tblPeople Group By HashAddr) Group By People Order by People To find out details for a *specific* Address Find the Hash for the desired Address and then pass it to something like this: Select Distinct StreetNo,Street,City,Zip,State,Count(HashAddr) as Address >From tblPeople P Inner Join tblEmails E on P.PK = E.PeopleFK Group By HashAddr StreetNo,Street,City,Zip,State Where HashAddr = @SearchHash On 19 Mar 2011 at 10:13, jwcolby wrote: > Ooops. My examples were hosed. > > Let's try again > > Addresses People > 22,538,240 1 > 00,708,462 2 > 00,052,234 3 > > and so forth. > > What the client really wants to know is > > "how many people with emails are at a specific address" > "how many people with emails are in a specific family" > > etc. > > I tie my address table to my email table using an auto increment long > increment "I must be careful not to call this a surrogate" key and > then perform the counts using the hash fields in a Group By / count > query. > > John W. Colby > www.ColbyConsulting.com > > On 3/19/2011 9:55 AM, jwcolby wrote: > > I got into the hashing business in order to create a surrogate key > > of sorts, in fact a trio of them. > > > > I have an address hash, a family hash and a person hash. I compute > > these three hashes for every table of persons. Now that I have these > > surrogate keys I have "identical" values in a single field where the > > input is identical in multiple fields. Thus I do not have to do > > multi-field joins where I have a hash for all of those fields. > > > > I am happy to hear that I am secure in my surrogate hash algorithm > > choice. > > > > OK, so I have these hashes. Every record has these three hashes. > > > > Now some definitions: > > > > HashAddr: A unique address is addr/zip5/zip4 different > > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > > > Using this information, I need to calculate the count of addresses > > with a single person, with two persons etc for as many combinations > > as I have. > > > > Addresses People > > 22,538,240 1 > > 780,462,346 2 > > 52,234 3 people > > etc > > etc till every quantity of people is found. > > > > What is the simplest way to accomplish this with SQL. > > > > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 18:21:18 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sun, 20 Mar 2011 09:21:18 +1000 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84B5E7.3010204@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com> Message-ID: <4D853A6E.20716.57FC618@stuart.lexacorp.com.pg> Answered on VB list. On 19 Mar 2011 at 9:55, jwcolby wrote: > I got into the hashing business in order to create a surrogate key of > sorts, in fact a trio of them. > > I have an address hash, a family hash and a person hash. I compute > these three hashes for every table of persons. Now that I have these > surrogate keys I have "identical" values in a single field where the > input is identical in multiple fields. Thus I do not have to do > multi-field joins where I have a hash for all of those fields. > > I am happy to hear that I am secure in my surrogate hash algorithm > choice. > > OK, so I have these hashes. Every record has these three hashes. > > Now some definitions: > > HashAddr: A unique address is addr/zip5/zip4 different > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > Using this information, I need to calculate the count of addresses > with a single person, with two persons etc for as many combinations as > I have. > > Addresses People > 22,538,240 1 > 780,462,346 2 > 52,234 3 people > etc > etc till every quantity of people is found. > > What is the simplest way to accomplish this with SQL. > > > > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 21:01:04 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 22:01:04 -0400 Subject: [dba-VB] Most useful droid apps In-Reply-To: References: Message-ID: <4D855FE0.9070004@colbyconsulting.com> I am trying various tethering solutions. My laptop is vista, my ip range is 192.168.122.x. I got WiFi Tether to work, but only without encryption. I was getting about 1.5 mbit down and .5 mbit up. As soon as I turned on encryption I was unable to set up the laptop password to match (that was the only thing that changed). No encryption is unacceptable to me as I would only really use this in public places, exactly where you would most want encryption. I tried barnacle. That one just didn't work at all. I am now trying PDANet which is a USB tether. Since the PC is tied to the phone by a usb, encryption seems to be unnecessary? Unfortunately this one is not working either. The PDANet desktop app says it is connected and the droid pdanet app says it is connected but the PC just can't see the internet. I can't ping anything etc. Any suggestions for this beast? John W. Colby www.ColbyConsulting.com On 3/19/2011 4:53 PM, David McAfee wrote: > Advanced task killer > Weather channel > Lookout antivirus > AK Notepad > Bank of America > Acar auto maintenance > Barcode scanner > Bubble level > Bump > Droid light > Google goggles > Google translate > IMDB > Key ring > Print share > Trapster > > Sent from my Droid phone. > On Mar 19, 2011 12:52 PM, "jwcolby" wrote: > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 21:34:03 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sun, 20 Mar 2011 12:34:03 +1000 Subject: [dba-VB] Most useful droid apps In-Reply-To: <4D855FE0.9070004@colbyconsulting.com> References: , <4D855FE0.9070004@colbyconsulting.com> Message-ID: <4D85679B.30595.6303F8F@stuart.lexacorp.com.pg> Are you on AT&T? If so, you may be interested in this: http://www.theregister.co.uk/2011/03/18/new_att_tethering_policy/ On 19 Mar 2011 at 22:01, jwcolby wrote: > I am trying various tethering solutions. My laptop is vista, my ip > range is 192.168.122.x. > > I got WiFi Tether to work, but only without encryption. I was getting > about 1.5 mbit down and .5 mbit up. > > As soon as I turned on encryption I was unable to set up the laptop > password to match (that was the only thing that changed). No > encryption is unacceptable to me as I would only really use this in > public places, exactly where you would most want encryption. > > I tried barnacle. That one just didn't work at all. > > I am now trying PDANet which is a USB tether. Since the PC is tied to > the phone by a usb, encryption seems to be unnecessary? Unfortunately > this one is not working either. The PDANet desktop app says it is > connected and the droid pdanet app says it is connected but the PC > just can't see the internet. I can't ping anything etc. > > Any suggestions for this beast? > > > John W. Colby > www.ColbyConsulting.com > > On 3/19/2011 4:53 PM, David McAfee wrote: > > Advanced task killer > > Weather channel > > Lookout antivirus > > AK Notepad > > Bank of America > > Acar auto maintenance > > Barcode scanner > > Bubble level > > Bump > > Droid light > > Google goggles > > Google translate > > IMDB > > Key ring > > Print share > > Trapster > > > > Sent from my Droid phone. > > On Mar 19, 2011 12:52 PM, "jwcolby" > > wrote: _______________________________________________ dba-VB > > mailing list dba-VB at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > http://www.databaseadvisors.com > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 21:45:39 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 22:45:39 -0400 Subject: [dba-VB] Most useful droid apps In-Reply-To: <4D85679B.30595.6303F8F@stuart.lexacorp.com.pg> References: , <4D855FE0.9070004@colbyconsulting.com> <4D85679B.30595.6303F8F@stuart.lexacorp.com.pg> Message-ID: <4D856A53.7010507@colbyconsulting.com> Nope, Verizon John W. Colby www.ColbyConsulting.com On 3/19/2011 10:34 PM, Stuart McLachlan wrote: > Are you on AT&T? > > If so, you may be interested in this: > > http://www.theregister.co.uk/2011/03/18/new_att_tethering_policy/ > > > On 19 Mar 2011 at 22:01, jwcolby wrote: > >> I am trying various tethering solutions. My laptop is vista, my ip >> range is 192.168.122.x. >> >> I got WiFi Tether to work, but only without encryption. I was getting >> about 1.5 mbit down and .5 mbit up. >> >> As soon as I turned on encryption I was unable to set up the laptop >> password to match (that was the only thing that changed). No >> encryption is unacceptable to me as I would only really use this in >> public places, exactly where you would most want encryption. >> >> I tried barnacle. That one just didn't work at all. >> >> I am now trying PDANet which is a USB tether. Since the PC is tied to >> the phone by a usb, encryption seems to be unnecessary? Unfortunately >> this one is not working either. The PDANet desktop app says it is >> connected and the droid pdanet app says it is connected but the PC >> just can't see the internet. I can't ping anything etc. >> >> Any suggestions for this beast? >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/19/2011 4:53 PM, David McAfee wrote: >>> Advanced task killer >>> Weather channel >>> Lookout antivirus >>> AK Notepad >>> Bank of America >>> Acar auto maintenance >>> Barcode scanner >>> Bubble level >>> Bump >>> Droid light >>> Google goggles >>> Google translate >>> IMDB >>> Key ring >>> Print share >>> Trapster >>> >>> Sent from my Droid phone. >>> On Mar 19, 2011 12:52 PM, "jwcolby" >>> wrote: _______________________________________________ dba-VB >>> mailing list dba-VB at databaseadvisors.com >>> http://databaseadvisors.com/mailman/listinfo/dba-vb >>> http://www.databaseadvisors.com >>> >>> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From listmaster at databaseadvisors.com Wed Mar 23 14:34:46 2011 From: listmaster at databaseadvisors.com (Bryan Carbonnell) Date: Wed, 23 Mar 2011 15:34:46 -0400 Subject: [dba-VB] Administrivia - Spam Filtering on DBA Servers In-Reply-To: References: Message-ID: We have made a change to the way our mailservers deal with spam filtering. We were getting too many false positive reports from one of the spam black list services we use, so we have discontinued using them. Hopefully your bounce troubles should be over, or at the very least greatly minimized. If you continue to have issues surrounding bounces, PLEASE, PLEASE, PLEASE get in touch with me (listmaster at databaseadvisors OR carbonnb at gmail.com if the listmaster address bounces). -- Bryan Carbonnell - listmaster at databaseadvisors.com Life's journey is not to arrive at the grave safely in a well preserved body, but rather to skid in sideways, totally worn out, shouting "What a great ride!" From jwcolby at colbyconsulting.com Thu Mar 24 13:07:19 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 24 Mar 2011 14:07:19 -0400 Subject: [dba-VB] C# protected Message-ID: <4D8B8857.60604@colbyconsulting.com> I am trying to create a method of a ClassA which can be seen from the object ClassB that instantiates ClassA but not by objectClassC which instantiates ClassB. IOW ClassA "belongs to" ClassB so specific methods / properties of ClassB should be manipulateable by ClassB. However those properties / methods should not be visible to anything outside of ClassA and ClassB. Possible? Oh, and BTW I want ClassA to be out in a class library, external to the project using it. -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Thu Mar 24 15:14:59 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 24 Mar 2011 23:14:59 +0300 Subject: [dba-VB] C# protected In-Reply-To: <4D8B8857.60604@colbyconsulting.com> References: <4D8B8857.60604@colbyconsulting.com> Message-ID: Hi John -- Put ClassA and ClassB into a class library. Use internal access modifier with their methods and properties to be seen within their class library but not from the "outer world". Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 24 ????? 2011 ?. 21:07 To: VBA Subject: [dba-VB] C# protected I am trying to create a method of a ClassA which can be seen from the object ClassB that instantiates ClassA but not by objectClassC which instantiates ClassB. IOW ClassA "belongs to" ClassB so specific methods / properties of ClassB should be manipulateable by ClassB. However those properties / methods should not be visible to anything outside of ClassA and ClassB. Possible? Oh, and BTW I want ClassA to be out in a class library, external to the project using it. -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Thu Mar 24 15:28:03 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 24 Mar 2011 16:28:03 -0400 Subject: [dba-VB] C# protected In-Reply-To: References: <4D8B8857.60604@colbyconsulting.com> Message-ID: <4D8BA953.4030602@colbyconsulting.com> ClassA can go in the class library, but classB is part of the application. John W. Colby www.ColbyConsulting.com On 3/24/2011 4:14 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Put ClassA and ClassB into a class library. > Use > > internal > > access modifier with their methods and properties to be seen within their > class library but not from the "outer world". > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 24 ????? 2011 ?. 21:07 > To: VBA > Subject: [dba-VB] C# protected > > I am trying to create a method of a ClassA which can be seen from the object > ClassB that instantiates ClassA but not by objectClassC which instantiates > ClassB. > > IOW ClassA "belongs to" ClassB so specific methods / properties of ClassB > should be manipulateable by ClassB. However those properties / methods > should not be visible to anything outside of ClassA and ClassB. > > Possible? > > Oh, and BTW I want ClassA to be out in a class library, external to the > project using it. > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Thu Mar 24 16:37:22 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 25 Mar 2011 00:37:22 +0300 Subject: [dba-VB] C# protected In-Reply-To: <4D8BA953.4030602@colbyconsulting.com> References: <4D8B8857.60604@colbyconsulting.com> <4D8BA953.4030602@colbyconsulting.com> Message-ID: Hi John -- Then use protected access modifier for methods and properties of Class A which is inherited by class B (IOW no need to instantiate class A explicitly)... Class C which will instantiate Class B will no see/have access to protected properties and methods of class A inherithed by class B. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 24 ????? 2011 ?. 23:28 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# protected ClassA can go in the class library, but classB is part of the application. John W. Colby www.ColbyConsulting.com On 3/24/2011 4:14 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Put ClassA and ClassB into a class library. > Use > > internal > > access modifier with their methods and properties to be seen within > their class library but not from the "outer world". > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 24 ????? 2011 ?. 21:07 > To: VBA > Subject: [dba-VB] C# protected > > I am trying to create a method of a ClassA which can be seen from the > object ClassB that instantiates ClassA but not by objectClassC which > instantiates ClassB. > > IOW ClassA "belongs to" ClassB so specific methods / properties of > ClassB should be manipulateable by ClassB. However those properties / > methods should not be visible to anything outside of ClassA and ClassB. > > Possible? > > Oh, and BTW I want ClassA to be out in a class library, external to > the project using it. > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 25 14:58:27 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 25 Mar 2011 15:58:27 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <3B9DE744CEA244829179A81CA818F92F@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com> <3B9DE744CEA244829179A81CA818F92F@nant> Message-ID: <4D8CF3E3.3050608@colbyconsulting.com> Shamil, Does this allow using a single "lock object" to lock various operations in a class. For example I want to lock a date flag variable while using a property to either get or set the variable. private Object thisLock = new Object(); public DateTime pFlagDte { get { lock (thisLock) { return dteFlag; } } set { lock (thisLock) { dteFlag = value; } } } The issue is not multiple threads trying to write to the date variable but rather one thread trying to read it while another is writing it. John W. Colby www.ColbyConsulting.com On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Do you you need add, remove and iterate a list to get an object or you need > to add, remove and get an object by its key? - if the latter - here is the > code you can use: > > private Dictionary _myObjects = new Dictionary MyObject>(); > private static object _myObjectsLocker = new object(); > public void Add(MyObject myObject) > { > lock (_myObjectsLocker) > { > _myObjects.Add(myObject.ID, myObject); > _keys.Add(myObject.ID); > } > } > public MyObject Remove(int id) > { > MyObject myobject = null; > lock (_myObjectsLocker) > { > myobject = _myObjects[id]; > _myObjects.Remove(id); > } > return myobject; > } > public MyObject this[int key] > { > get > { > MyObject myObject = null; > lock (_myObjectsLocker) > { > _myObjects.TryGetValue(key, out myObject); > } > return myObject; > } > } > > for the case of iteration just replace: > > _myObjects.TryGetValue(key, out myObject); > > with your iteration code. > > As for the sample code - changing qty of threads and changing total computer > workload (outer context for the test code) you can see how different could > be final results when heavy multi-threading is used. And even if it's not > heavy - it still can result in weird output if special multi-threading > coding technique aren't used... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 11 ????? 2011 ?. 1:27 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Thanks for the code but I think it is overkill for my needs so far, though I > will put it in the archives for when I need it. > > In my system so far I have (or can arrange to have ) only two threads > trying to access a list. > > Basically what I am doing is reading records out of sql server into a class > factory, and placing the class instances into a list. > > Another object with its own thread is coming by and looking at the objects > in the list to see if there is any work for it to do. If it finds an object > to work on, it grabs a pointer to that item in the list and stores it in a > pointer and releases the list. > > One of the ways I saw this handled was to create factory which creates a > brand new list object, grabs a pointer to the list, locks that and then > copies the objects in the list pointers to classes) into that new list, > unlocks the list and hands back a pointer to this brand new list. thus the > new list could be iterated at will. > > I am just trying to lock the list while one thread adds objects to the list > or another thread gets objects out of the list. > > So if I have object A which contains a list and is adding / deleting objects > to that list... > and object B which needs to iterate through the list looking for an object > of interest in the list, and grabbing a pointer to that object, there is no > way to do this without all of the code you wrote? > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Does that in fact lock the list to other objects doing the same thing >> (getting the list through that property)? >>>>> >> You have to lock/synchronize/serialize not the list itself but >> simultaneous access to the list from different threads. >> You'd better use special static object variable to implement locking. >> >> I will post sample code in my following posting here... >> >> Thank you. >> >> -- >> Shamil > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Mar 25 16:05:57 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 26 Mar 2011 00:05:57 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8CF3E3.3050608@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com> Message-ID: <3F0DA84A905C4947819126F43CFC39DB@nant> Hi John -- Your sample is a correct usage of locking - just use static thisLock variable private static Object thisLock = new Object() as non-static variable *is not* thread safe AFAIU. Yes, such locking can be done for various operations of that class - just make sure you'll not get dead-locked... Also use *one lock object* to lock *one resource* in a class - otherwise "deadlock" will become your "everyday guest"... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 25 ????? 2011 ?. 22:58 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, Does this allow using a single "lock object" to lock various operations in a class. For example I want to lock a date flag variable while using a property to either get or set the variable. private Object thisLock = new Object(); public DateTime pFlagDte { get { lock (thisLock) { return dteFlag; } } set { lock (thisLock) { dteFlag = value; } } } The issue is not multiple threads trying to write to the date variable but rather one thread trying to read it while another is writing it. John W. Colby www.ColbyConsulting.com On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Do you you need add, remove and iterate a list to get an object or you > need to add, remove and get an object by its key? - if the latter - > here is the code you can use: > > private Dictionary _myObjects = new Dictionary MyObject>(); > private static object _myObjectsLocker = new object(); public void > Add(MyObject myObject) { > lock (_myObjectsLocker) > { > _myObjects.Add(myObject.ID, myObject); > _keys.Add(myObject.ID); > } > } > public MyObject Remove(int id) > { > MyObject myobject = null; > lock (_myObjectsLocker) > { > myobject = _myObjects[id]; > _myObjects.Remove(id); > } > return myobject; > } > public MyObject this[int key] > { > get > { > MyObject myObject = null; > lock (_myObjectsLocker) > { > _myObjects.TryGetValue(key, out myObject); > } > return myObject; > } > } > > for the case of iteration just replace: > > _myObjects.TryGetValue(key, out myObject); > > with your iteration code. > > As for the sample code - changing qty of threads and changing total > computer workload (outer context for the test code) you can see how > different could be final results when heavy multi-threading is used. > And even if it's not heavy - it still can result in weird output if > special multi-threading coding technique aren't used... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 11 ????? 2011 ?. 1:27 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Thanks for the code but I think it is overkill for my needs so far, > though I will put it in the archives for when I need it. > > In my system so far I have (or can arrange to have ) only two threads > trying to access a list. > > Basically what I am doing is reading records out of sql server into a > class factory, and placing the class instances into a list. > > Another object with its own thread is coming by and looking at the > objects in the list to see if there is any work for it to do. If it > finds an object to work on, it grabs a pointer to that item in the > list and stores it in a pointer and releases the list. > > One of the ways I saw this handled was to create factory which creates > a brand new list object, grabs a pointer to the list, locks that and > then copies the objects in the list pointers to classes) into that new > list, unlocks the list and hands back a pointer to this brand new > list. thus the new list could be iterated at will. > > I am just trying to lock the list while one thread adds objects to the > list or another thread gets objects out of the list. > > So if I have object A which contains a list and is adding / deleting > objects to that list... > and object B which needs to iterate through the list looking for an > object of interest in the list, and grabbing a pointer to that object, > there is no way to do this without all of the code you wrote? > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Does that in fact lock the list to other objects doing the same thing >> (getting the list through that property)? >>>>> >> You have to lock/synchronize/serialize not the list itself but >> simultaneous access to the list from different threads. >> You'd better use special static object variable to implement locking. >> >> I will post sample code in my following posting here... >> >> Thank you. >> >> -- >> Shamil > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 25 16:38:19 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 25 Mar 2011 17:38:19 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <3F0DA84A905C4947819126F43CFC39DB@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com> <3F0DA84A905C4947819126F43CFC39DB@nant> Message-ID: <4D8D0B4B.9070004@colbyconsulting.com> When you say resource, you mean variable? IOW use private static Object LockBln = new Object(); private static Object LockDte = new Object(); and use the LockBln to lock the boolean object while using LockDte to lock the date object? John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - just > make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - otherwise > "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various operations in a > class. > > For example I want to lock a date flag variable while using a property to > either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable but > rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Do you you need add, remove and iterate a list to get an object or you >> need to add, remove and get an object by its key? - if the latter - >> here is the code you can use: >> >> private Dictionary _myObjects = new Dictionary> MyObject>(); >> private static object _myObjectsLocker = new object(); public void >> Add(MyObject myObject) { >> lock (_myObjectsLocker) >> { >> _myObjects.Add(myObject.ID, myObject); >> _keys.Add(myObject.ID); >> } >> } >> public MyObject Remove(int id) >> { >> MyObject myobject = null; >> lock (_myObjectsLocker) >> { >> myobject = _myObjects[id]; >> _myObjects.Remove(id); >> } >> return myobject; >> } >> public MyObject this[int key] >> { >> get >> { >> MyObject myObject = null; >> lock (_myObjectsLocker) >> { >> _myObjects.TryGetValue(key, out myObject); >> } >> return myObject; >> } >> } >> >> for the case of iteration just replace: >> >> _myObjects.TryGetValue(key, out myObject); >> >> with your iteration code. >> >> As for the sample code - changing qty of threads and changing total >> computer workload (outer context for the test code) you can see how >> different could be final results when heavy multi-threading is used. >> And even if it's not heavy - it still can result in weird output if >> special multi-threading coding technique aren't used... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 11 ????? 2011 ?. 1:27 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Thanks for the code but I think it is overkill for my needs so far, >> though I will put it in the archives for when I need it. >> >> In my system so far I have (or can arrange to have ) only two threads >> trying to access a list. >> >> Basically what I am doing is reading records out of sql server into a >> class factory, and placing the class instances into a list. >> >> Another object with its own thread is coming by and looking at the >> objects in the list to see if there is any work for it to do. If it >> finds an object to work on, it grabs a pointer to that item in the >> list and stores it in a pointer and releases the list. >> >> One of the ways I saw this handled was to create factory which creates >> a brand new list object, grabs a pointer to the list, locks that and >> then copies the objects in the list pointers to classes) into that new >> list, unlocks the list and hands back a pointer to this brand new >> list. thus the new list could be iterated at will. >> >> I am just trying to lock the list while one thread adds objects to the >> list or another thread gets objects out of the list. >> >> So if I have object A which contains a list and is adding / deleting >> objects to that list... >> and object B which needs to iterate through the list looking for an >> object of interest in the list, and grabbing a pointer to that object, >> there is no way to do this without all of the code you wrote? >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >>> Hi John -- >>> >>> <<< >>> Does that in fact lock the list to other objects doing the same thing >>> (getting the list through that property)? >>>>>> >>> You have to lock/synchronize/serialize not the list itself but >>> simultaneous access to the list from different threads. >>> You'd better use special static object variable to implement locking. >>> >>> I will post sample code in my following posting here... >>> >>> Thank you. >>> >>> -- >>> Shamil >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Mar 25 17:46:44 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 26 Mar 2011 01:46:44 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8D0B4B.9070004@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8D0B4B.9070004@colbyconsulting.com> Message-ID: Hi John -- By resource I mean access to a read/write resource - it could be not only a variable... If you need two or more class level variables (fields, properties) within one class instance to be locked then keep them in a class/structure and lock that class/structure instance... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 26 ????? 2011 ?. 0:38 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() When you say resource, you mean variable? IOW use private static Object LockBln = new Object(); private static Object LockDte = new Object(); and use the LockBln to lock the boolean object while using LockDte to lock the date object? John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - > just make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - > otherwise "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various > operations in a class. > > For example I want to lock a date flag variable while using a property > to either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable > but rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Do you you need add, remove and iterate a list to get an object or >> you need to add, remove and get an object by its key? - if the latter >> - here is the code you can use: >> >> private Dictionary _myObjects = new Dictionary> MyObject>(); >> private static object _myObjectsLocker = new object(); public void >> Add(MyObject myObject) { >> lock (_myObjectsLocker) >> { >> _myObjects.Add(myObject.ID, myObject); >> _keys.Add(myObject.ID); >> } >> } >> public MyObject Remove(int id) >> { >> MyObject myobject = null; >> lock (_myObjectsLocker) >> { >> myobject = _myObjects[id]; >> _myObjects.Remove(id); >> } >> return myobject; >> } >> public MyObject this[int key] >> { >> get >> { >> MyObject myObject = null; >> lock (_myObjectsLocker) >> { >> _myObjects.TryGetValue(key, out myObject); >> } >> return myObject; >> } >> } >> >> for the case of iteration just replace: >> >> _myObjects.TryGetValue(key, out myObject); >> >> with your iteration code. >> >> As for the sample code - changing qty of threads and changing total >> computer workload (outer context for the test code) you can see how >> different could be final results when heavy multi-threading is used. >> And even if it's not heavy - it still can result in weird output if >> special multi-threading coding technique aren't used... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 11 ????? 2011 ?. 1:27 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Thanks for the code but I think it is overkill for my needs so far, >> though I will put it in the archives for when I need it. >> >> In my system so far I have (or can arrange to have ) only two >> threads trying to access a list. >> >> Basically what I am doing is reading records out of sql server into a >> class factory, and placing the class instances into a list. >> >> Another object with its own thread is coming by and looking at the >> objects in the list to see if there is any work for it to do. If it >> finds an object to work on, it grabs a pointer to that item in the >> list and stores it in a pointer and releases the list. >> >> One of the ways I saw this handled was to create factory which >> creates a brand new list object, grabs a pointer to the list, locks >> that and then copies the objects in the list pointers to classes) >> into that new list, unlocks the list and hands back a pointer to this >> brand new list. thus the new list could be iterated at will. >> >> I am just trying to lock the list while one thread adds objects to >> the list or another thread gets objects out of the list. >> >> So if I have object A which contains a list and is adding / deleting >> objects to that list... >> and object B which needs to iterate through the list looking for an >> object of interest in the list, and grabbing a pointer to that >> object, there is no way to do this without all of the code you wrote? >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >>> Hi John -- >>> >>> <<< >>> Does that in fact lock the list to other objects doing the same >>> thing (getting the list through that property)? >>>>>> >>> You have to lock/synchronize/serialize not the list itself but >>> simultaneous access to the list from different threads. >>> You'd better use special static object variable to implement locking. >>> >>> I will post sample code in my following posting here... >>> >>> Thank you. >>> >>> -- >>> Shamil >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 25 21:57:22 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 25 Mar 2011 22:57:22 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8D0B4B.9070004@colbyconsulting.com> Message-ID: <4D8D5612.6050701@colbyconsulting.com> Shamil, I have processes that log results to flags. For example, make a database (log that it was made), build a table (log that it was built), pull umpteen million records in sorted order (log that it was filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build another chunk table (log that it was filled), BCP out (log that it was exported). The objective is to be able to sustain interruptions and pick up where we left off. These processes can take minutes (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). So each thing I do represents a step in the process and each step is logged in a field in a record in SQL server using a datetime. There are so many of these flags that I am trying to standardize the process by building a class that can be instantiated, filled with data and log itself to SQL Server by one thread and be checked by another thread. These flag class instances will be checked by multiple threads, each thread trying to decide whether it should be doing the next step because another thread has finished it's part. IOW if a file has been written to disk, then the next thread will write it to a VM for processing. If it moved to the VM the next thread will watch the VM's output directory for a file to pop out and move it back to a directory on the server. If the file (a couple of files actually) successfully copied back to the server staging then another thread will import it back into a chunk table in an input database. If the file successfully imported then another thread will... In general one thread will "own" the flag and use it to log its status and one other thread will be checking the status of the flag to determine that it can go to work on that work chunk. You get the picture. I am trying to build an entirely asynchronous highly threaded process which exports a huge table into multiple files, processes every file through a third party app and gets the results back into SQL Server. All while logging each and every step so that no piece can possibly be dropped at any stage, even if the server goes down (or the VM goes down). Eventually this process will run on my server 24/7. It has been working for some time but I am getting threading issues, and I need to work on the high level control so that all of the processes can cleanly start up and shut down and every stage can pick back up when the program restarts should a shutdown occur. A single database can be up to a hundred million records (the biggest so far), and the external program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour depending on many different things so that example will take 50 chunks and could take 40 to 50 hours to complete. It takes about 20 processing steps to handle each file from end to end. It needs to just work, and I need to be able to view status in a meaningful way. And I need to process that and a dozen other files every single month, automatically, with no manual intervention required. John W. Colby www.ColbyConsulting.com On 3/25/2011 6:46 PM, Shamil Salakhetdinov wrote: > Hi John -- > > By resource I mean access to a read/write resource - it could be not only a > variable... > > If you need two or more class level variables (fields, properties) within > one class instance to be locked then keep them in a class/structure and lock > that class/structure instance... > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Sat Mar 26 00:37:25 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 26 Mar 2011 08:37:25 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8D5612.6050701@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8D0B4B.9070004@colbyconsulting.com> <4D8D5612.6050701@colbyconsulting.com> Message-ID: <4D87466503764D98871B609159033CB6@nant> John -- But you can try to use several instances of .NET Framework Queue class http://msdn.microsoft.com/en-us/library/system.collections.queue.aspx ? As far as I understand you have a set of sequential tasks to be performed on every (input/start) chunk so you can make a dedicated Queue instance for every task, and that Queue instance will be handled by dedicated "processor" classes: - TaskA Queue items will be processed by TaskAQueueProcessor, - TaksB .... by TaskBQueueProcessor, -- - TaskZ... by TaskZQueueProcessor There could be many such TaskA(B, ....Z)QueueProcessor instances "floating around" looking for an item from their dedicated tasks' queue to process: when they find it, they pick it up, process and put into another Queue based on "workflow plan"... You can also try to consider using Windows Workflow Foundation(WF) if that wouldn't be an "overkill" for your case... http://msdn.microsoft.com/en-us/netframework/wf-screencasts.aspx#introwf4 Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 26 ????? 2011 ?. 5:57 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, I have processes that log results to flags. For example, make a database (log that it was made), build a table (log that it was built), pull umpteen million records in sorted order (log that it was filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build another chunk table (log that it was filled), BCP out (log that it was exported). The objective is to be able to sustain interruptions and pick up where we left off. These processes can take minutes (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). So each thing I do represents a step in the process and each step is logged in a field in a record in SQL server using a datetime. There are so many of these flags that I am trying to standardize the process by building a class that can be instantiated, filled with data and log itself to SQL Server by one thread and be checked by another thread. These flag class instances will be checked by multiple threads, each thread trying to decide whether it should be doing the next step because another thread has finished it's part. IOW if a file has been written to disk, then the next thread will write it to a VM for processing. If it moved to the VM the next thread will watch the VM's output directory for a file to pop out and move it back to a directory on the server. If the file (a couple of files actually) successfully copied back to the server staging then another thread will import it back into a chunk table in an input database. If the file successfully imported then another thread will... In general one thread will "own" the flag and use it to log its status and one other thread will be checking the status of the flag to determine that it can go to work on that work chunk. You get the picture. I am trying to build an entirely asynchronous highly threaded process which exports a huge table into multiple files, processes every file through a third party app and gets the results back into SQL Server. All while logging each and every step so that no piece can possibly be dropped at any stage, even if the server goes down (or the VM goes down). Eventually this process will run on my server 24/7. It has been working for some time but I am getting threading issues, and I need to work on the high level control so that all of the processes can cleanly start up and shut down and every stage can pick back up when the program restarts should a shutdown occur. A single database can be up to a hundred million records (the biggest so far), and the external program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour depending on many different things so that example will take 50 chunks and could take 40 to 50 hours to complete. It takes about 20 processing steps to handle each file from end to end. It needs to just work, and I need to be able to view status in a meaningful way. And I need to process that and a dozen other files every single month, automatically, with no manual intervention required. John W. Colby www.ColbyConsulting.com On 3/25/2011 6:46 PM, Shamil Salakhetdinov wrote: > Hi John -- > > By resource I mean access to a read/write resource - it could be not > only a variable... > > If you need two or more class level variables (fields, properties) > within one class instance to be locked then keep them in a > class/structure and lock that class/structure instance... > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From gustav at cactus.dk Sat Mar 26 07:41:44 2011 From: gustav at cactus.dk (Gustav Brock) Date: Sat, 26 Mar 2011 13:41:44 +0100 Subject: [dba-VB] c# lock() Message-ID: Hi John My first thought was Windows Workflow Foundation(WF) but, as Shamil mentions, it may be overkill. On the other hand it is exactly for controlling scenarios where "if task 1 is done, start task 2 and task 3, wait for task 2 and task 3 to finish, check something, then start task 4, etc.". However, that reminded me about a series of articles about the Task Parallel Library of .Net: http://www.codeproject.com/KB/cs/TPL1.aspx which I found very interesting, though I haven't had any use for it yet. Among other topics it discusses carefully canceling and error handling which I guess is quite important for your purpose. /gustav >>> jwcolby at colbyconsulting.com 26-03-2011 03:57 >>> Shamil, I have processes that log results to flags. For example, make a database (log that it was made), build a table (log that it was built), pull umpteen million records in sorted order (log that it was filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build another chunk table (log that it was filled), BCP out (log that it was exported). The objective is to be able to sustain interruptions and pick up where we left off. These processes can take minutes (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). So each thing I do represents a step in the process and each step is logged in a field in a record in SQL server using a datetime. There are so many of these flags that I am trying to standardize the process by building a class that can be instantiated, filled with data and log itself to SQL Server by one thread and be checked by another thread. These flag class instances will be checked by multiple threads, each thread trying to decide whether it should be doing the next step because another thread has finished it's part. IOW if a file has been written to disk, then the next thread will write it to a VM for processing. If it moved to the VM the next thread will watch the VM's output directory for a file to pop out and move it back to a directory on the server. If the file (a couple of files actually) successfully copied back to the server staging then another thread will import it back into a chunk table in an input database. If the file successfully imported then another thread will... In general one thread will "own" the flag and use it to log its status and one other thread will be checking the status of the flag to determine that it can go to work on that work chunk. You get the picture. I am trying to build an entirely asynchronous highly threaded process which exports a huge table into multiple files, processes every file through a third party app and gets the results back into SQL Server. All while logging each and every step so that no piece can possibly be dropped at any stage, even if the server goes down (or the VM goes down). Eventually this process will run on my server 24/7. It has been working for some time but I am getting threading issues, and I need to work on the high level control so that all of the processes can cleanly start up and shut down and every stage can pick back up when the program restarts should a shutdown occur. A single database can be up to a hundred million records (the biggest so far), and the external program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour depending on many different things so that example will take 50 chunks and could take 40 to 50 hours to complete. It takes about 20 processing steps to handle each file from end to end. It needs to just work, and I need to be able to view status in a meaningful way. And I need to process that and a dozen other files every single month, automatically, with no manual intervention required. John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 26 07:44:54 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 08:44:54 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: Message-ID: <4D8DDFC6.4040004@colbyconsulting.com> Well... I haven't migrated to VS2010 yet. I tried it about 6 months ago and it was universally a pig on my systems. I thought I would wait until MS polished it a bit. John W. Colby www.ColbyConsulting.com On 3/26/2011 8:41 AM, Gustav Brock wrote: > Hi John > > My first thought was Windows Workflow Foundation(WF) but, as Shamil mentions, it may be overkill. On the other hand it is exactly for controlling scenarios where "if task 1 is done, start task 2 and task 3, wait for task 2 and task 3 to finish, check something, then start task 4, etc.". > > However, that reminded me about a series of articles about the Task Parallel Library of .Net: > > http://www.codeproject.com/KB/cs/TPL1.aspx > > which I found very interesting, though I haven't had any use for it yet. Among other topics it discusses carefully canceling and error handling which I guess is quite important for your purpose. > > /gustav > > >>>> jwcolby at colbyconsulting.com 26-03-2011 03:57>>> > Shamil, > > I have processes that log results to flags. For example, make a database (log that it was made), > build a table (log that it was built), pull umpteen million records in sorted order (log that it was > filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build > another chunk table (log that it was filled), BCP out (log that it was exported). The objective is > to be able to sustain interruptions and pick up where we left off. These processes can take minutes > (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). > > So each thing I do represents a step in the process and each step is logged in a field in a record > in SQL server using a datetime. There are so many of these flags that I am trying to standardize > the process by building a class that can be instantiated, filled with data and log itself to SQL > Server by one thread and be checked by another thread. > > These flag class instances will be checked by multiple threads, each thread trying to decide whether > it should be doing the next step because another thread has finished it's part. IOW if a file has > been written to disk, then the next thread will write it to a VM for processing. If it moved to the > VM the next thread will watch the VM's output directory for a file to pop out and move it back to a > directory on the server. If the file (a couple of files actually) successfully copied back to the > server staging then another thread will import it back into a chunk table in an input database. If > the file successfully imported then another thread will... In general one thread will "own" the > flag and use it to log its status and one other thread will be checking the status of the flag to > determine that it can go to work on that work chunk. > > You get the picture. > > I am trying to build an entirely asynchronous highly threaded process which exports a huge table > into multiple files, processes every file through a third party app and gets the results back into > SQL Server. All while logging each and every step so that no piece can possibly be dropped at any > stage, even if the server goes down (or the VM goes down). Eventually this process will run on my > server 24/7. > > It has been working for some time but I am getting threading issues, and I need to work on the high > level control so that all of the processes can cleanly start up and shut down and every stage can > pick back up when the program restarts should a shutdown occur. > > A single database can be up to a hundred million records (the biggest so far), and the external > program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour > depending on many different things so that example will take 50 chunks and could take 40 to 50 hours > to complete. It takes about 20 processing steps to handle each file from end to end. It needs to > just work, and I need to be able to view status in a meaningful way. And I need to process that and > a dozen other files every single month, automatically, with no manual intervention required. > > > > John W. Colby > www.ColbyConsulting.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From gustav at cactus.dk Sat Mar 26 08:08:10 2011 From: gustav at cactus.dk (Gustav Brock) Date: Sat, 26 Mar 2011 14:08:10 +0100 Subject: [dba-VB] c# lock() Message-ID: Hi John Oh, I recall your trouble now that you mention it. Strange, as it runs fine on my antique Pentium 4 IBM IntelliStation with 2 GB ram. However, VS2010 _has_ been polished: The Service Pack 1 was released a few weeks ago. /gustav >>> jwcolby at colbyconsulting.com 26-03-2011 13:44 >>> Well... I haven't migrated to VS2010 yet. I tried it about 6 months ago and it was universally a pig on my systems. I thought I would wait until MS polished it a bit. John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 26 08:26:26 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 09:26:26 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: Message-ID: <4D8DE982.7060707@colbyconsulting.com> When I had my speed issues I Googled around and found tons with speed issues, and tons without. There are entire threads bitterly complaining about what a dog it is. There was no explanations of when or why. Mine would take forever to do anything. I will definitely try again now that SP1 is out. John W. Colby www.ColbyConsulting.com On 3/26/2011 9:08 AM, Gustav Brock wrote: > Hi John > > Oh, I recall your trouble now that you mention it. Strange, as it runs fine on my antique Pentium 4 IBM IntelliStation with 2 GB ram. > However, VS2010 _has_ been polished: The Service Pack 1 was released a few weeks ago. > > /gustav > > >>>> jwcolby at colbyconsulting.com 26-03-2011 13:44>>> > Well... I haven't migrated to VS2010 yet. > > I tried it about 6 months ago and it was universally a pig on my systems. I thought I would wait > until MS polished it a bit. > > John W. Colby > www.ColbyConsulting.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 26 16:53:15 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 17:53:15 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <3F0DA84A905C4947819126F43CFC39DB@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com> <3F0DA84A905C4947819126F43CFC39DB@nant> Message-ID: <4D8E604B.5040001@colbyconsulting.com> Shamil, I am trying to build a base class which contains a set of lock objects and some standard variables. Having defined the lock objects static per your code below, the derived class cannot access the lock objects even though I set them protected: protected static Object LockBln = new Object(); In the derived class when I try to access it: lock(this.LockBln) { } I get a compile error: "Member cannot be accesses with an instance reference; Qualify it with a type name instead." What does this mean? Reading on the internet it says that static properties are shared between all instances of the class. I don't see how I can share a property between 50 instances and then use a lock(MyStaticProperty) {}. I'm so confused... John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - just > make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - otherwise > "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various operations in a > class. > > For example I want to lock a date flag variable while using a property to > either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable but > rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sat Mar 26 17:27:17 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sun, 27 Mar 2011 01:27:17 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8E604B.5040001@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8E604B.5040001@colbyconsulting.com> Message-ID: <00D0A03B172743B6868A8BE736F7E560@nant> John -- Let's define your target (application) system architecture on conceptual level - I guess that using "micro-level locking" you're trying (?) to achieve isn't needed... 1) you have a huge input data set; 2) you split input data set into X chunks; 3) every chunk gets processed using the same several - Y - steps; 4) on completion of step Yn input/semi-processed chunk Xn is submitted to the Y(n+1) processor class; 5) when all X chunks are processed in Y steps they get collected into a final "huge" output set to produce Z reports. That's it from conceptual high level point of view? Something important missing? Too simplified to be true? John, please take into account that I can't reply promptly here - it's weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to sleep, and then I have quite some work to do starting Monday - so I expect this thread will be supported by other AccessD/dba-VB members... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 0:53 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, I am trying to build a base class which contains a set of lock objects and some standard variables. Having defined the lock objects static per your code below, the derived class cannot access the lock objects even though I set them protected: protected static Object LockBln = new Object(); In the derived class when I try to access it: lock(this.LockBln) { } I get a compile error: "Member cannot be accesses with an instance reference; Qualify it with a type name instead." What does this mean? Reading on the internet it says that static properties are shared between all instances of the class. I don't see how I can share a property between 50 instances and then use a lock(MyStaticProperty) {}. I'm so confused... John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - > just make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - > otherwise "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various > operations in a class. > > For example I want to lock a date flag variable while using a property > to either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable > but rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Mar 26 18:31:31 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 19:31:31 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <00D0A03B172743B6868A8BE736F7E560@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant> Message-ID: <4D8E7753.8090006@colbyconsulting.com> I'm just trying to get a handle on locking things (in general). I am seeing things that say "just lock the object itself", and then other things that say "use a static object (the way you were showing me). If locking the object itself works, why do the static thing? John W. Colby www.ColbyConsulting.com On 3/26/2011 6:27 PM, Shamil Salakhetdinov wrote: > John -- > > Let's define your target (application) system architecture on conceptual > level - I guess that using "micro-level locking" you're trying (?) to > achieve isn't needed... > > 1) you have a huge input data set; > 2) you split input data set into X chunks; > 3) every chunk gets processed using the same several - Y - steps; > 4) on completion of step Yn input/semi-processed chunk Xn is submitted to > the Y(n+1) processor class; > 5) when all X chunks are processed in Y steps they get collected into a > final "huge" output set to produce Z reports. > > That's it from conceptual high level point of view? > Something important missing? > Too simplified to be true? > > John, please take into account that I can't reply promptly here - it's > weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to sleep, > and then I have quite some work to do starting Monday - so I expect this > thread will be supported by other AccessD/dba-VB members... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 27 ????? 2011 ?. 0:53 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > I am trying to build a base class which contains a set of lock objects and > some standard variables. > Having defined the lock objects static per your code below, the derived > class cannot access the lock objects even though I set them protected: > > protected static Object LockBln = new Object(); > > In the derived class when I try to access it: > > lock(this.LockBln) > { > } > > I get a compile error: > > "Member cannot be accesses with an instance reference; Qualify it with a > type name instead." > > What does this mean? > > Reading on the internet it says that static properties are shared between > all instances of the class. I don't see how I can share a property between > 50 instances and then use a > lock(MyStaticProperty) {}. > > I'm so confused... > > John W. Colby > www.ColbyConsulting.com > > On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Your sample is a correct usage of locking - just use static thisLock >> variable >> >> private static Object thisLock = new Object() >> >> as non-static variable *is not* thread safe AFAIU. >> >> Yes, such locking can be done for various operations of that class - >> just make sure you'll not get dead-locked... >> Also use *one lock object* to lock *one resource* in a class - >> otherwise "deadlock" will become your "everyday guest"... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 25 ????? 2011 ?. 22:58 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Does this allow using a single "lock object" to lock various >> operations in a class. >> >> For example I want to lock a date flag variable while using a property >> to either get or set the variable. >> >> private Object thisLock = new Object(); >> >> public DateTime pFlagDte { >> get { >> lock (thisLock) >> { >> return dteFlag; >> } >> } >> set >> { >> lock (thisLock) >> { >> dteFlag = value; >> } >> } >> } >> >> The issue is not multiple threads trying to write to the date variable >> but rather one thread trying to read it while another is writing it. >> >> John W. Colby >> www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Mar 26 19:08:10 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sun, 27 Mar 2011 04:08:10 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <00D0A03B172743B6868A8BE736F7E560@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant> Message-ID: <0CD7EF3475CB43C69334942027697B63@nant> John -- I'm falling asleep here now - would that be correct assumption that you can generalize your global task like that - you have, say: - split input data into 50 Chunks; - every chunk should be processed in 10 steps; - every step's average processing time is 100 minutes. Then in total it is needed 50*10*100 = 50,000 minutes or ~833.(3) hours to process all your input data sequentially, chunk after chunk step after step, say, you don't have enough computer power to process some chunks in parallel, and you can't process the whole not split input data as it's too large for your system... But you have "50 small computers" and you can process every of 50 input chunks in parallel - then you'll need just 10*100 = 1,000 minutes (~16.67 hours) to complete the job. I assume that all the chunks processing is independent - then you can: 1) Create Scheduler class instance; 2) Schedule class creates 10 TaskQueue class instances - 1 queue for every processing step type; 3) Scheduler class defines splitting criteria to split input data into 50 chunks; 4) Schedule defines completion criteria - when all 50 chunks get collected in 11th FinalQueue; 5) Schedule seeds the first InputChunkSplitTaskQueue with 50 SplitTasksDescriptors; 4) From time to time - say every half a minute Scheduler class instance creates 500 worker classes in 500 threads (500 worker classes => 50 chunks * 10 steps.), which "hunt for the job" in their dedicate Queues: if they find Job they work on it, if not - they "die"... 5) When a worker class completes processing its job it queues its "task results" into the next step Queue, if a worker class fails then it puts its incomplete task descriptor back into its dedicated queue and "dies", or that could be Scheduler's (or SchedulerAssitant's) task job to find failed workers' "trap" and to resubmit their failed work to the proper queue according to the custom workflow descriptors attached to every chunk... 500 worker classes is an overkill as only 50 worker classes can have job every time but that seems to be an easy "brute force" and "lazy parallel programming" approach - and it should work... Or you can make worker class instances production processes smarter: Scheduler class can start special thread for WorkerClassGenerator instance, which will monitor all the 10 TaskQueues, and if it finds an item in a Queque, it will pick it up, it will create corresponding Worker class in parallel thread and it will pass WorkItem to the Worker class fro processing... When that described above approach will work there then you can easily(?) scale it splitting your (constant size) input data into 100 chunks, and then if every chunk can be processes in half time - in average (50 minutes) - then all job can be completed in 500 minutes = ~8.33(3) hours etc... Please correct me if I oversimplified your application business area... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 27 ????? 2011 ?. 1:27 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] c# lock() John -- Let's define your target (application) system architecture on conceptual level - I guess that using "micro-level locking" you're trying (?) to achieve isn't needed... 1) you have a huge input data set; 2) you split input data set into X chunks; 3) every chunk gets processed using the same several - Y - steps; 4) on completion of step Yn input/semi-processed chunk Xn is submitted to the Y(n+1) processor class; 5) when all X chunks are processed in Y steps they get collected into a final "huge" output set to produce Z reports. That's it from conceptual high level point of view? Something important missing? Too simplified to be true? John, please take into account that I can't reply promptly here - it's weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to sleep, and then I have quite some work to do starting Monday - so I expect this thread will be supported by other AccessD/dba-VB members... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 0:53 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, I am trying to build a base class which contains a set of lock objects and some standard variables. Having defined the lock objects static per your code below, the derived class cannot access the lock objects even though I set them protected: protected static Object LockBln = new Object(); In the derived class when I try to access it: lock(this.LockBln) { } I get a compile error: "Member cannot be accesses with an instance reference; Qualify it with a type name instead." What does this mean? Reading on the internet it says that static properties are shared between all instances of the class. I don't see how I can share a property between 50 instances and then use a lock(MyStaticProperty) {}. I'm so confused... John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - > just make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - > otherwise "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various > operations in a class. > > For example I want to lock a date flag variable while using a property > to either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable > but rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Sat Mar 26 19:38:28 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sun, 27 Mar 2011 04:38:28 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <4D8E7753.8090006@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8E604B.5040001@colbyconsulting.com><00D0A03B172743B6868A8BE736F7E560@nant> <4D8E7753.8090006@colbyconsulting.com> Message-ID: Hi John -- <<< If locking the object itself works, why do the static thing? >>> Static is used to lock shared resources - in your case there are no such resources at all? Log writer is usually a static shared resource... And yes, you can use your locking code to lock parallel access to your class instances' local variables: >> private Object thisLock = new Object(); >> >> public DateTime pFlagDte { >> get { >> lock (thisLock) >> { >> return dteFlag; >> } >> } >> set >> { >> lock (thisLock) >> { >> dteFlag = value; >> } >> } >> } But as I noted I'm not sure if that "micro-level locking" is needed at all (and you mentioned you have several such shared variables within a class instance) - using Queues seems to be more simple and (thread-)safe approach without almost any custom locking schemes.... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 3:32 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() I'm just trying to get a handle on locking things (in general). I am seeing things that say "just lock the object itself", and then other things that say "use a static object (the way you were showing me). If locking the object itself works, why do the static thing? John W. Colby www.ColbyConsulting.com On 3/26/2011 6:27 PM, Shamil Salakhetdinov wrote: > John -- > > Let's define your target (application) system architecture on > conceptual level - I guess that using "micro-level locking" you're > trying (?) to achieve isn't needed... > > 1) you have a huge input data set; > 2) you split input data set into X chunks; > 3) every chunk gets processed using the same several - Y - steps; > 4) on completion of step Yn input/semi-processed chunk Xn is submitted > to the Y(n+1) processor class; > 5) when all X chunks are processed in Y steps they get collected into > a final "huge" output set to produce Z reports. > > That's it from conceptual high level point of view? > Something important missing? > Too simplified to be true? > > John, please take into account that I can't reply promptly here - it's > weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to > sleep, and then I have quite some work to do starting Monday - so I > expect this thread will be supported by other AccessD/dba-VB members... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 27 ????? 2011 ?. 0:53 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > I am trying to build a base class which contains a set of lock objects > and some standard variables. > Having defined the lock objects static per your code below, the > derived class cannot access the lock objects even though I set them protected: > > protected static Object LockBln = new Object(); > > In the derived class when I try to access it: > > lock(this.LockBln) > { > } > > I get a compile error: > > "Member cannot be accesses with an instance reference; Qualify it with > a type name instead." > > What does this mean? > > Reading on the internet it says that static properties are shared > between all instances of the class. I don't see how I can share a > property between 50 instances and then use a > lock(MyStaticProperty) {}. > > I'm so confused... > > John W. Colby > www.ColbyConsulting.com > > On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Your sample is a correct usage of locking - just use static thisLock >> variable >> >> private static Object thisLock = new Object() >> >> as non-static variable *is not* thread safe AFAIU. >> >> Yes, such locking can be done for various operations of that class - >> just make sure you'll not get dead-locked... >> Also use *one lock object* to lock *one resource* in a class - >> otherwise "deadlock" will become your "everyday guest"... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 25 ????? 2011 ?. 22:58 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Does this allow using a single "lock object" to lock various >> operations in a class. >> >> For example I want to lock a date flag variable while using a >> property to either get or set the variable. >> >> private Object thisLock = new Object(); >> >> public DateTime pFlagDte { >> get { >> lock (thisLock) >> { >> return dteFlag; >> } >> } >> set >> { >> lock (thisLock) >> { >> dteFlag = value; >> } >> } >> } >> >> The issue is not multiple threads trying to write to the date >> variable but rather one thread trying to read it while another is writing it. >> >> John W. Colby >> www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 26 23:12:20 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 27 Mar 2011 00:12:20 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <0CD7EF3475CB43C69334942027697B63@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant> <0CD7EF3475CB43C69334942027697B63@nant> Message-ID: <4D8EB924.5080708@colbyconsulting.com> Well... In programming you can do any task 50 different ways. The way I did it is to create a supervisor and ProcessFile table in SQL Server. The supervisor represents a table in SQL Server that contains an address table which needs to be processed. The ProcessFile records are child to tblSupervisor and represent chunks (files). The supervisor record stores all of the high level flags about the database (address table) being processed. The ProcessFile contains all of the flags about each chunk being processed. I have four different high level "stages" as I call them that a supervisor goes through. AccuzipOut - table to csv files in staging out (stage 1), Staging to VM (stage 2), VM to Staging in (stage 3, Staging in to SQL Server (stage 4) 1) Export all chunks (files) to a SQL Out staging directory. Each chunk ends up being a CSV file with PKID, FName, LName, Addr, City, St, Zip5, Zip4. The ProcessFile needs to be sorted by Zip5 / Zip4 order because the third party application that processes these files is immensely faster if the file is presorted. 2) One by one, copy each chunk file to a virtual machine - AccuzipIn directory. The Accuzip virtual machine runs on a Windows 2008 server running Hyper-V manager. I currently run 3 different VMs, only one of which runs Accuzip. For awhile I actually ran 3 VMs running Accuzip, and I am designing such that I can add additional Accuzip VMs in the future. What this means is that each file could go to any valid Accuzip VM, and when the chunk file is placed on a VM it has to log which VM it went to because... 3) The Accuzip software automatically senses and processes the file in the InputTo directory. A file of 2 million records takes about 25 minutes to CASS process locally to the VM, then is uploaded to a server in California for further NCOA processing. The server in California belongs to Accuzip and they stage big files so it can take awhile before it even begins to process. Once it processes, the NCOA server sends a small file back to my VM where Address change information is integrated back in to the local Accuzip database file (foxpro). After that it is exported back to CSV with a bunch of additional fields and is placed in the Accuzip OutputFrom directory along with a PDF which contains the CASS / NCOA statistics. This PDF is a "proof of processing" document that I have to keep forever, and I have to send to the client. I have a Directory Watcher which is watching the OutputFrom directory. When *all* of the resulting Accuzip output files are in the OutputFrom directory, my Directory Watcher event triggers my program to copy all of these files back to the SQL Server In staging directory. 4) Once all of the Accuzip files for a single chunk are back on the SQL Server In directory, my application BCPs the CSV into a input chunk table. From there decisions are made whether I need to import every address back in or only the moves. In any event some or all of the address records (with the additional fields from Accuzip) are pulled into an AZData table back in the database that the address data originally came from. So... 1) A supervisor object creates a clsDbAZOut instance. 2) clsDBAZOut creates an actual database dbXyzAZOut temporary database. 3) clsDBAZOut pulls every record needing to be accuzipped into a table with a super PK, sorted on Zip5/4 4) clsDBAZOut counts the records in tblAZOut, and starts to build clsProcessFile instances which holds the information about the chunk processing. clsProcessFile can write its info back to the ProcessFile records on SQL Server. 5) clsDbAZOut has a thread which builds tblChunk, gets ChunkSize records into the chunk table and BCPs tblChunk to CSV files on a staging out directory. 6) clsDbAZOut BCPs the chunk to file 7) At this point the clsProcessFile is handed off to clsVM. clsSupervisor has a strongly typed dictionary which stores the clsProcessFile instances created by clsDbAZOut. clsSupervisor can read ProcessFile instances from SQL Server into a clsProcessFile factory or clsDbAZOut can use a class factory to create new ones. In all cases they are stored in the dictionary in clsSupervisor. clsProcessfile has flags which document that: a) The chunk table was created b) The chunk table exported to file c) The chunk went to a VM (and which one) d) The chunk file was (eventually) found in the Accuzip OutputFrom directory e) The resulting AZData file (and PDF) were moved from the VM back to SQL ServerIn staging f) The file in SQL Server In staging was pulled back in to a chunk table in a second database dbXyzAZIn temporary database g) That the chunk table was imported back into the original database. As you can see, clsProcessFile is the center of the universe for a single chunk. clsSupervisor is the center of the universe for the entire process. It is a bit difficult to discuss how the entire program works in a single email. A clsManager loads one or a dozen clsSupervisor instances. clsManager loads clsVM which supervises the (a) virtual machine which runs a third party process. clsVM moves files from SQL Staging Out into the VM one chunk at a time, updating clsProcessTable flags. clsVM moves files from the the VM back to the SQL Staging IN, updating flags. The VM may not be available, or the Accuzip program may be down etc. clsSupervisor loads one to N clsProcessTable instances. clsSupervisor loads a clsDBExp and clsDBImp. clsDBExp creates new clsProcessTable instances, one for each chunk it exports - storing them in clsSupervisor. clsDBExp supervises the export process. clsDBImp supervises the import process getting the chunks back into SQL Server into a temp database, and from there into the live database. I am trying to design the system such that it is asynchronous. Stage1, stage 2/3 and stage 4 can all do their thing independently of each other. Each step of each stage is logged (dateTime flags written) so that any piece can just pick up where it left off. That means any Supervisor and any ProcessFile (chunk) can pick up from any point. In the meantime, the clsManager is watching the tblSupervisor for supervisor records ready to process - a thread. clsSupervisor is watching for ProcessTable records and dispatching them as required to the clsDBExp, clsDBImp or clsVM - a thread in each supervisor. clsDBExp is processing chunks, creating CSV files - a thread. clsVM is watching for clsProcessTable instances ready to go to the VM. It is also watching the output directory and sending files back to the SQL Server - a thread and two file watchers (and two events). clsDBImp is watching for clsProcessTable instances ready to import back into SQL Server - a thread. Most of these threads are writing flags (as they finish their specific process) and reading flags in clsProcessTable to decide what chunk has to be processed in which class. Oh, and I have three list controls on a form which the various classes write to to display the status for Stage1, Stage 2/3 and Stage 4. So pretty much a single supervisor is processing at a time, exporting files, sending them to the vm, importing them back in etc. It is possible for an order supervisor to cut in at the head of the line however. Not a simple task co-ordinating all of this stuff. I used to co-ordinate it all manually! ;) John W. Colby www.ColbyConsulting.com On 3/26/2011 8:08 PM, Shamil Salakhetdinov wrote: > John -- > > I'm falling asleep here now - would that be correct assumption that you can > generalize your global task like that - you have, say: > > - split input data into 50 Chunks; > - every chunk should be processed in 10 steps; > - every step's average processing time is 100 minutes. > > Then in total it is needed 50*10*100 = 50,000 minutes or ~833.(3) hours to > process all your input data sequentially, chunk after chunk step after step, > say, you don't have enough computer power to process some chunks in > parallel, and you can't process the whole not split input data as it's too > large for your system... > > But you have "50 small computers" and you can process every of 50 input > chunks in parallel - then you'll need just 10*100 = 1,000 minutes (~16.67 > hours) to complete the job. > > I assume that all the chunks processing is independent - then you can: > > 1) Create Scheduler class instance; > 2) Schedule class creates 10 TaskQueue class instances - 1 queue for every > processing step type; > 3) Scheduler class defines splitting criteria to split input data into 50 > chunks; > 4) Schedule defines completion criteria - when all 50 chunks get collected > in 11th FinalQueue; > 5) Schedule seeds the first InputChunkSplitTaskQueue with 50 > SplitTasksDescriptors; > 4) From time to time - say every half a minute Scheduler class instance > creates 500 worker classes in 500 threads (500 worker classes => 50 chunks * > 10 steps.), which "hunt for the job" in their dedicate Queues: if they find > Job they work on it, if not - they "die"... > 5) When a worker class completes processing its job it queues its "task > results" into the next step Queue, if a worker class fails then it puts its > incomplete task descriptor back into its dedicated queue and "dies", or that > could be Scheduler's (or SchedulerAssitant's) task job to find failed > workers' "trap" and to resubmit their failed work to the proper queue > according to the custom workflow descriptors attached to every chunk... > > 500 worker classes is an overkill as only 50 worker classes can have job > every time but that seems to be an easy "brute force" and "lazy parallel > programming" approach - and it should work... > Or you can make worker class instances production processes smarter: > Scheduler class can start special thread for WorkerClassGenerator instance, > which will monitor all the 10 TaskQueues, and if it finds an item in a > Queque, it will pick it up, it will create corresponding Worker class in > parallel thread and it will pass WorkItem to the Worker class fro > processing... > > When that described above approach will work there then you can easily(?) > scale it splitting your (constant size) input data into 100 chunks, and then > if every chunk can be processes in half time - in average (50 minutes) - > then all job can be completed in 500 minutes = ~8.33(3) hours etc... > > Please correct me if I oversimplified your application business area... > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Sun Mar 27 19:08:25 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 28 Mar 2011 04:08:25 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <4D8EB924.5080708@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> Message-ID: Hi John -- Yes, I see: you tend to build a multi-threaded application/utility system with centralized control - and I tend to build an anarchical free-market model based application/utility system :) Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 8:12 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Well... In programming you can do any task 50 different ways. The way I did it is to create a supervisor and ProcessFile table in SQL Server. The supervisor represents a table in SQL Server that contains an address table which needs to be processed. The ProcessFile records are child to tblSupervisor and represent chunks (files). The supervisor record stores all of the high level flags about the database (address table) being processed. The ProcessFile contains all of the flags about each chunk being processed. I have four different high level "stages" as I call them that a supervisor goes through. AccuzipOut - table to csv files in staging out (stage 1), Staging to VM (stage 2), VM to Staging in (stage 3, Staging in to SQL Server (stage 4) 1) Export all chunks (files) to a SQL Out staging directory. Each chunk ends up being a CSV file with PKID, FName, LName, Addr, City, St, Zip5, Zip4. The ProcessFile needs to be sorted by Zip5 / Zip4 order because the third party application that processes these files is immensely faster if the file is presorted. 2) One by one, copy each chunk file to a virtual machine - AccuzipIn directory. The Accuzip virtual machine runs on a Windows 2008 server running Hyper-V manager. I currently run 3 different VMs, only one of which runs Accuzip. For awhile I actually ran 3 VMs running Accuzip, and I am designing such that I can add additional Accuzip VMs in the future. What this means is that each file could go to any valid Accuzip VM, and when the chunk file is placed on a VM it has to log which VM it went to because... 3) The Accuzip software automatically senses and processes the file in the InputTo directory. A file of 2 million records takes about 25 minutes to CASS process locally to the VM, then is uploaded to a server in California for further NCOA processing. The server in California belongs to Accuzip and they stage big files so it can take awhile before it even begins to process. Once it processes, the NCOA server sends a small file back to my VM where Address change information is integrated back in to the local Accuzip database file (foxpro). After that it is exported back to CSV with a bunch of additional fields and is placed in the Accuzip OutputFrom directory along with a PDF which contains the CASS / NCOA statistics. This PDF is a "proof of processing" document that I have to keep forever, and I have to send to the client. I have a Directory Watcher which is watching the OutputFrom directory. When *all* of the resulting Accuzip output files are in the OutputFrom directory, my Directory Watcher event triggers my program to copy all of these files back to the SQL Server In staging directory. 4) Once all of the Accuzip files for a single chunk are back on the SQL Server In directory, my application BCPs the CSV into a input chunk table. >From there decisions are made whether I need to import every address back in or only the moves. In any event some or all of the address records (with the additional fields from Accuzip) are pulled into an AZData table back in the database that the address data originally came from. So... 1) A supervisor object creates a clsDbAZOut instance. 2) clsDBAZOut creates an actual database dbXyzAZOut temporary database. 3) clsDBAZOut pulls every record needing to be accuzipped into a table with a super PK, sorted on Zip5/4 4) clsDBAZOut counts the records in tblAZOut, and starts to build clsProcessFile instances which holds the information about the chunk processing. clsProcessFile can write its info back to the ProcessFile records on SQL Server. 5) clsDbAZOut has a thread which builds tblChunk, gets ChunkSize records into the chunk table and BCPs tblChunk to CSV files on a staging out directory. 6) clsDbAZOut BCPs the chunk to file 7) At this point the clsProcessFile is handed off to clsVM. clsSupervisor has a strongly typed dictionary which stores the clsProcessFile instances created by clsDbAZOut. clsSupervisor can read ProcessFile instances from SQL Server into a clsProcessFile factory or clsDbAZOut can use a class factory to create new ones. In all cases they are stored in the dictionary in clsSupervisor. clsProcessfile has flags which document that: a) The chunk table was created b) The chunk table exported to file c) The chunk went to a VM (and which one) d) The chunk file was (eventually) found in the Accuzip OutputFrom directory e) The resulting AZData file (and PDF) were moved from the VM back to SQL ServerIn staging f) The file in SQL Server In staging was pulled back in to a chunk table in a second database dbXyzAZIn temporary database g) That the chunk table was imported back into the original database. As you can see, clsProcessFile is the center of the universe for a single chunk. clsSupervisor is the center of the universe for the entire process. It is a bit difficult to discuss how the entire program works in a single email. A clsManager loads one or a dozen clsSupervisor instances. clsManager loads clsVM which supervises the (a) virtual machine which runs a third party process. clsVM moves files from SQL Staging Out into the VM one chunk at a time, updating clsProcessTable flags. clsVM moves files from the the VM back to the SQL Staging IN, updating flags. The VM may not be available, or the Accuzip program may be down etc. clsSupervisor loads one to N clsProcessTable instances. clsSupervisor loads a clsDBExp and clsDBImp. clsDBExp creates new clsProcessTable instances, one for each chunk it exports - storing them in clsSupervisor. clsDBExp supervises the export process. clsDBImp supervises the import process getting the chunks back into SQL Server into a temp database, and from there into the live database. I am trying to design the system such that it is asynchronous. Stage1, stage 2/3 and stage 4 can all do their thing independently of each other. Each step of each stage is logged (dateTime flags written) so that any piece can just pick up where it left off. That means any Supervisor and any ProcessFile (chunk) can pick up from any point. In the meantime, the clsManager is watching the tblSupervisor for supervisor records ready to process - a thread. clsSupervisor is watching for ProcessTable records and dispatching them as required to the clsDBExp, clsDBImp or clsVM - a thread in each supervisor. clsDBExp is processing chunks, creating CSV files - a thread. clsVM is watching for clsProcessTable instances ready to go to the VM. It is also watching the output directory and sending files back to the SQL Server - a thread and two file watchers (and two events). clsDBImp is watching for clsProcessTable instances ready to import back into SQL Server - a thread. Most of these threads are writing flags (as they finish their specific process) and reading flags in clsProcessTable to decide what chunk has to be processed in which class. Oh, and I have three list controls on a form which the various classes write to to display the status for Stage1, Stage 2/3 and Stage 4. So pretty much a single supervisor is processing at a time, exporting files, sending them to the vm, importing them back in etc. It is possible for an order supervisor to cut in at the head of the line however. Not a simple task co-ordinating all of this stuff. I used to co-ordinate it all manually! ;) John W. Colby www.ColbyConsulting.com On 3/26/2011 8:08 PM, Shamil Salakhetdinov wrote: > John -- > > I'm falling asleep here now - would that be correct assumption that > you can generalize your global task like that - you have, say: > > - split input data into 50 Chunks; > - every chunk should be processed in 10 steps; > - every step's average processing time is 100 minutes. > > Then in total it is needed 50*10*100 = 50,000 minutes or ~833.(3) > hours to process all your input data sequentially, chunk after chunk > step after step, say, you don't have enough computer power to process > some chunks in parallel, and you can't process the whole not split > input data as it's too large for your system... > > But you have "50 small computers" and you can process every of 50 > input chunks in parallel - then you'll need just 10*100 = 1,000 > minutes (~16.67 > hours) to complete the job. > > I assume that all the chunks processing is independent - then you can: > > 1) Create Scheduler class instance; > 2) Schedule class creates 10 TaskQueue class instances - 1 queue for > every processing step type; > 3) Scheduler class defines splitting criteria to split input data into > 50 chunks; > 4) Schedule defines completion criteria - when all 50 chunks get > collected in 11th FinalQueue; > 5) Schedule seeds the first InputChunkSplitTaskQueue with 50 > SplitTasksDescriptors; > 4) From time to time - say every half a minute Scheduler class > instance creates 500 worker classes in 500 threads (500 worker classes > => 50 chunks * 10 steps.), which "hunt for the job" in their dedicate > Queues: if they find Job they work on it, if not - they "die"... > 5) When a worker class completes processing its job it queues its > "task results" into the next step Queue, if a worker class fails then > it puts its incomplete task descriptor back into its dedicated queue > and "dies", or that could be Scheduler's (or SchedulerAssitant's) task > job to find failed workers' "trap" and to resubmit their failed work > to the proper queue according to the custom workflow descriptors attached to every chunk... > > 500 worker classes is an overkill as only 50 worker classes can have > job every time but that seems to be an easy "brute force" and "lazy > parallel programming" approach - and it should work... > Or you can make worker class instances production processes smarter: > Scheduler class can start special thread for WorkerClassGenerator > instance, which will monitor all the 10 TaskQueues, and if it finds an > item in a Queque, it will pick it up, it will create corresponding > Worker class in parallel thread and it will pass WorkItem to the > Worker class fro processing... > > When that described above approach will work there then you can > easily(?) scale it splitting your (constant size) input data into 100 > chunks, and then if every chunk can be processes in half time - in > average (50 minutes) - then all job can be completed in 500 minutes = ~8.33(3) hours etc... > > Please correct me if I oversimplified your application business area... > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sun Mar 27 22:01:44 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 27 Mar 2011 23:01:44 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> Message-ID: <4D8FFA18.9020902@colbyconsulting.com> Shamil, > Yes, I see: you tend to build a multi-threaded application/utility system > with centralized control - and I tend to build an anarchical free-market > model based application/utility system :) LOL, I guess. To be quite honest I didn't understand exactly what you were saying. Remember that my C# skills are still quite rudimentary. Accuzip Processing system basically has developed over time. I didn't sit down to develop an app from scratch to do this. I started in late 2004 to just get one single list into SQL server and serve up counts to the client. I did the whole thing manually, building queries to import the data and clean it up using only SQL Server Management Server. When I started I had never touched SQL Server. Then I was asked to research an application for doing CASS http://en.wikipedia.org/wiki/Coding_Accuracy_Support_System and NCOA http://en.wikipedia.org/wiki/National_Change_Of_Address on the addresses in the list. There was only one list in the beginning which I originally labeled "the database from hell". So I researched and discovered a vendor (Accuzip) that would allow us a fixed price license against which I could process as many records as I wanted. All other vendors (that I found) charge on a sliding scale Cents / record kind of thing, and we had a LOT of records - 65 million in that first list. So I started building a system of stored procedures which performed the export out to Accuzip and then import the Accuzipped records back in. It took roughly 25 stored procedures to perform this process. I had a paper list of instructions on how to run the stored procedures, parameters to feed in etc, plus manually copying the files out and back in. I would manually type EXEC commands into a query window reading from my paper instructions. It was *sloooow* and *error prone*. Next came a stored procedure that ran the stored procedures. Etc. Etc. It just morphed over time. Plus I got more lists. The client was so impressed that I managed to do all this that I got actual orders (export the data and send to the clients) rather than just counts, and the business grew. I started using Access ( the tool I knew ) to automate some of it but it was a very poor fit. Since every operation ran in a single thread the entire application would lock up when SQL Server went out for 30 minutes to do some operation. And of course my hardware was underpowered. In the summer of 2009 I decided I really needed to automate this. The business had grown and the client was suddenly 50% of my income and it was a huge struggle to process these lists manually, even using stored procedures of lists of stored procedures. TSQL as a programming environment is crude to say the least. So I went to the local community college in Sept. 2009 and took the first of two available C# courses. By Christmas I was launching in to my automation attempt. Understand my C# skills were rudimentary at best but the language was robust and I could see the power. I really just started building a method of moving the paper list of stored procedures into C#, IOW C# would execute the stored procedures and monitor their completion. I was not focused on a top down "build a system" but rather let's get my manual process automated ASAP. I was overworked, and wasn't able to handle the load. It turns out that C# is very capable in this regard and the program (such as it was) started working and working well. At the end of the second semester (May 2010) I hired Paul, a young college student who was graduating from the community college to come to work for me part time. I started teaching him what I know about general programming and together we worked on this program. And it continued to morph. We started using NLog for logging, Visual SVN for source control etc. At some point I decided to just move the Stored Procedures out of SQL Server and into dynamic code generated directly in C#. Each step had to keep the program running because I actually used the program to fill orders and do counts. And remember our C# skills are still rudimentary. Then we "discovered" threads and soon thereafter events. Last fall I decided to remodel the program into a manager / supervisor / worker model where the manager tells the supervisor to do something, the supervisor tells other workers to do something and basically just check results. In this model, each level uses threads to scan it's tasks (watch flags) and keep its workers busy. So the dbExport class has a thread which build a database, pulls records, builds chunks, exports to disk. That is it's job. When the file is on disk it is done and control for that chunk is passed to another class (the Virtual Machine). The dbExport class uses flags to store each step of the process. The Virtual Machine class picks up the files and pulls them over to the VMs directory, monitors that Accuzip processes them, tracks errors (if any) from Accuzip, and places the resulting files back on a staging directory (as I call it) for another class to take over. The dbImport class is tasked with creating a temp database, pulling each chunk file back in from staging and getting the data back into the original list database. So the manager class (a single instance) loads supervisor classes, one per list database that needs processing. The Manager class also loads the VM class (only one of these at the moment but there could be more, with more licenses from Accuzip). Each supervisor loads a dbExport and dbImport class to process the records in its list. dbExport creates Process records and stores them in SQL Server and creates a process class for each process record. Each process record represents a chunk of data from a specific list database. the process class doesn't really do anything it is just a place for the flags to be loaded / manipulated, and it knows how to persist itself to SQL Server. dbImport basically gets Process class instances which are finished with stage 2/3 (out of Accuzip) and starts importing them back in to SQL Server. My concept was a factory kind of application. There can be none or dozens of databases needing to be processed. SQL Server and my disk system can only efficiently handle one at a time (at any stage) and Accuzip is the biggest bottleneck so the VM class sits in the middle of an export and an import class. Every class has its own job, every class has at least one thread for its use. dbExport has a thread to use in exporting data to a table and another thread to bcp the data out while the next chunk is created in the export database. dbImport has a thread to BCP data back into chunk tables, and another thread to take the chunk tables and merge them back into live data. The VM has a thread to monitor the input directory and another thread to monitor the output directory, moving files into and out of the VM. Every step has to be logged back to SQL Server as it completes, but the clsSupervisor keeps the progress flags in memory as long as a specific list database is being processed, and the supervisor holds a list of process classes which hold all of the flags for the steps for a single chunk. It is a busy system. I did not originally use threads, and everything ran on the main program thread. And I was back to my user interface locking up as long running processes happened. Now with threading, each stage (export / vm / import) has its own list control which is used to display the status of the list database / chunks as they progress through the system. Everything is logged to flags, but also to NLog tables. Each class has threads to keep its own processes fed and running, and each class does its own piece of the whole to process a list table through Accuzip. BTW, there is another entire C# program which processes orders for the client. It is also threaded, and one of its steps is to create a supervisor record to tell the Accuzip Monitor to Accuzip the addresses being processed for the order. Orders are the Accuzip processing which I mentioned have to cut in at the head of the line in Accuzip Monitoring. As you have seen, I could be processing a 65 million record list database when an order comes in. 65 million records is going to take at least 50 hours to process, and the order has to go out ASAP so the Accuzip Monitor (the Manager class actually) has to watch for orders and interrupt the data supervisor to allow the Order supervisor to cut in. Once Accuzip starts a process (chunk) I don't interrupt it, but when that process (chunk) finishes Accuzip, only Order chunks will process until the order is finished. As I said, it is a busy system. And remember that our C# skills are still rudimentary. I started coding in C# in Sept 2009 and started migrating this process to c# about December 2009. By May 2010 when Paul started work I was actually running an Accuzip process and doing orders using C#. Paul is a youngster right out of school. So much of the stuff you talk about goes right over my head Shamil. John W. Colby www.ColbyConsulting.com On 3/27/2011 8:08 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I see: you tend to build a multi-threaded application/utility system > with centralized control - and I tend to build an anarchical free-market > model based application/utility system :) > > Thank you. > > -- > Shamil From jwcolby at colbyconsulting.com Sun Mar 27 22:20:36 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 27 Mar 2011 23:20:36 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> Message-ID: <4D8FFE84.5090609@colbyconsulting.com> OK, so I have explained the process. As you can see I am using threads pretty heavily. In general only one thread writes to any given flag field in a class and in general only one (other) thread reads from that flag. However threads are peeking at flags all the time "checking" whether the chunk (in particular) is ready to proceed to the next stage. This weekend I decided to try a dedicated flag class. My idea is simply that each flag represents a field in a table. Each flag represents a specific data element from a specific record. It is only written once (generally) but potentially read often. If I build a dedicated flag class it can be initialized with the table / field / pkid etc for its specific field / record and then when the thread writes the flag, the flag class can write its value out to the right field / record table. Again it is normally only ever written once. Most of my flags are a date / time that the process step occurred. So I am trying to wrap the whole flag concept into a class to allow me to process (read / write / test) flags in an identical way in all of my process classes. It was this flag class which I was discussing the end of last week. It will be accessed by at least two threads so it needs to be thread safe. John W. Colby www.ColbyConsulting.com On 3/27/2011 8:08 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I see: you tend to build a multi-threaded application/utility system > with centralized control - and I tend to build an anarchical free-market > model based application/utility system :) > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Mon Mar 28 17:39:57 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Tue, 29 Mar 2011 02:39:57 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <4D8FFE84.5090609@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> <4D8FFE84.5090609@colbyconsulting.com> Message-ID: <5FD8C68EEEF142DA8632253753C51323@nant> Hi John -- <<< It was this flag class which I was discussing the end of last week. It will be accessed by at least two threads so it needs to be thread safe. >>> Yes, I see - I do wish it worked efficiently there real soon... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 28 ????? 2011 ?. 7:21 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() OK, so I have explained the process. As you can see I am using threads pretty heavily. In general only one thread writes to any given flag field in a class and in general only one (other) thread reads from that flag. However threads are peeking at flags all the time "checking" whether the chunk (in particular) is ready to proceed to the next stage. This weekend I decided to try a dedicated flag class. My idea is simply that each flag represents a field in a table. Each flag represents a specific data element from a specific record. It is only written once (generally) but potentially read often. If I build a dedicated flag class it can be initialized with the table / field / pkid etc for its specific field / record and then when the thread writes the flag, the flag class can write its value out to the right field / record table. Again it is normally only ever written once. Most of my flags are a date / time that the process step occurred. So I am trying to wrap the whole flag concept into a class to allow me to process (read / write / test) flags in an identical way in all of my process classes. It was this flag class which I was discussing the end of last week. It will be accessed by at least two threads so it needs to be thread safe. John W. Colby www.ColbyConsulting.com On 3/27/2011 8:08 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I see: you tend to build a multi-threaded application/utility > system with centralized control - and I tend to build an anarchical > free-market model based application/utility system :) > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From DWUTKA at Marlow.com Thu Mar 31 10:34:13 2011 From: DWUTKA at Marlow.com (Drew Wutka) Date: Thu, 31 Mar 2011 10:34:13 -0500 Subject: [dba-VB] Most useful droid apps In-Reply-To: References: Message-ID: AndroXplorer Very handy for transferring files around between Droid, Network and SD Card. Drew -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of David McAfee Sent: Saturday, March 19, 2011 3:54 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Most useful droid apps Advanced task killer Weather channel Lookout antivirus AK Notepad Bank of America Acar auto maintenance Barcode scanner Bubble level Bump Droid light Google goggles Google translate IMDB Key ring Print share Trapster Sent from my Droid phone. On Mar 19, 2011 12:52 PM, "jwcolby" wrote: _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com The information contained in this transmission is intended only for the person or entity to which it is addressed and may contain II-VI Proprietary and/or II-VI Business Sensitive material. If you are not the intended recipient, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. You are notified that any review, retransmission, copying, disclosure, dissemination, or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. From jwcolby at colbyconsulting.com Wed Mar 2 20:11:33 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 02 Mar 2011 21:11:33 -0500 Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews Message-ID: <4D6EF8D5.7060804@colbyconsulting.com> -- John W. Colby www.ColbyConsulting.com http://www.legitreviews.com/news/10164/ From accessd at shaw.ca Thu Mar 3 14:55:06 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Thu, 3 Mar 2011 12:55:06 -0800 Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews In-Reply-To: <4D6EF8D5.7060804@colbyconsulting.com> References: <4D6EF8D5.7060804@colbyconsulting.com> Message-ID: <7AF8F6950EC14685B928DE79F836CB0D@creativesystemdesigns.com> Soon to be into John's office. ;-) Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, March 02, 2011 6:12 PM To: Access Developers discussion and problem solving; Sqlserver-Dba; VBA Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews -- John W. Colby www.ColbyConsulting.com http://www.legitreviews.com/news/10164/ _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Thu Mar 3 15:05:20 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 03 Mar 2011 16:05:20 -0500 Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews In-Reply-To: <7AF8F6950EC14685B928DE79F836CB0D@creativesystemdesigns.com> References: <4D6EF8D5.7060804@colbyconsulting.com> <7AF8F6950EC14685B928DE79F836CB0D@creativesystemdesigns.com> Message-ID: <4D700290.4010308@colbyconsulting.com> I wish. It will most likely be beyond my means for some time. John W. Colby www.ColbyConsulting.com On 3/3/2011 3:55 PM, Jim Lawrence wrote: > Soon to be into John's office. ;-) > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, March 02, 2011 6:12 PM > To: Access Developers discussion and problem solving; Sqlserver-Dba; VBA > Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - > Legit Reviews > > From michael at ddisolutions.com.au Sun Mar 6 19:26:08 2011 From: michael at ddisolutions.com.au (Michael Maddison) Date: Mon, 7 Mar 2011 12:26:08 +1100 Subject: [dba-VB] Java Help? Message-ID: <99266C61B516644D9727F983FAFAB465046ABC@remote.ddisolutions.com.au> Hi Guys, A while ago I asked about creating a COM dll to be called from Java. Does anyone have the Java skills to see if you can call the dll to test for me? Cheers Michael M From dbdoug at gmail.com Sun Mar 6 21:04:47 2011 From: dbdoug at gmail.com (Doug Steele) Date: Sun, 6 Mar 2011 19:04:47 -0800 Subject: [dba-VB] ASP.Net development Message-ID: Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele From Gustav at cactus.dk Mon Mar 7 04:08:58 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 07 Mar 2011 11:08:58 +0100 Subject: [dba-VB] ASP.Net development Message-ID: Hi Doug I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. /gustav >>> dbdoug at gmail.com 07-03-2011 04:04 >>> Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele From accessd at shaw.ca Mon Mar 7 09:22:24 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Mon, 7 Mar 2011 07:22:24 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: Hi Gustav: Showing my ignorance about Winforms; does Winforms create browser based application or create a Windows based applications that uses a browser? Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Monday, March 07, 2011 2:09 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] ASP.Net development Hi Doug I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. /gustav >>> dbdoug at gmail.com 07-03-2011 04:04 >>> Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From dbdoug at gmail.com Mon Mar 7 09:55:51 2011 From: dbdoug at gmail.com (Doug Steele) Date: Mon, 7 Mar 2011 07:55:51 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: Hi Gustav: Thanks for the reply. If you build a table adapter which consists of data from more than one table (like a one to many relationship) in the design surface, VS2008 will refuse to create any CRUD query code - you have to build it yourself. Maybe you only used one-table adapters? Doug On Mon, Mar 7, 2011 at 2:08 AM, Gustav Brock wrote: > Hi Doug > > I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. > > I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. > You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. > > As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. > > /gustav > > >>>> dbdoug at gmail.com 07-03-2011 04:04 >>> > Hello All: > > A lot of the Microsoft sample code for asp.net data access is built > using strongly typed tableadapters with the .xsd design surface. ?I've > built two apps using these in the last year, but they seem to be kind > of fiddly - they don't work well with multi table update queries, for > instance, and they generate a HUGE amount of background source code. > It is nice to have the strong typing, however. > > I have never found any non-Microsoft sample code which uses this > technique. ?Admittedly, most of the sample code in the wild consists > of short snippets built to illustrate a single technique, not a > complete application. > > Two questions: > 1. Are there developers out there using this approach for real life apps? > 2. If not, what do you recommend? > > > Thanks, > > Doug Steele > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Mon Mar 7 11:04:59 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 07 Mar 2011 18:04:59 +0100 Subject: [dba-VB] ASP.Net development Message-ID: Hi Jim WinForms are true Windows desktop apps (I realise I should have changed the subject line, sorry). If you wish a browser based app, that's ASP.NET or Silverlight if we talk about Visual Studio and dotNet. /gustav >>> accessd at shaw.ca 07-03-2011 16:22 >>> Hi Gustav: Showing my ignorance about Winforms; does Winforms create browser based application or create a Windows based applications that uses a browser? Jim From Gustav at cactus.dk Mon Mar 7 11:17:36 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 07 Mar 2011 18:17:36 +0100 Subject: [dba-VB] ASP.Net development Message-ID: Hi Doug Now I see. Yes, for updates I use one-table tableadapters. Multi-table tableadapters are mainly used as source for reports and list-type forms where nothing is updated - and as source for combo- and listboxes. /gustav >>> dbdoug at gmail.com 07-03-2011 16:55 >>> Hi Gustav: Thanks for the reply. If you build a table adapter which consists of data from more than one table (like a one to many relationship) in the design surface, VS2008 will refuse to create any CRUD query code - you have to build it yourself. Maybe you only used one-table adapters? Doug On Mon, Mar 7, 2011 at 2:08 AM, Gustav Brock wrote: > Hi Doug > > I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. > > I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. > You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. > > As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. > > /gustav > > >>>> dbdoug at gmail.com 07-03-2011 04:04 >>> > Hello All: > > A lot of the Microsoft sample code for asp.net data access is built > using strongly typed tableadapters with the .xsd design surface. I've > built two apps using these in the last year, but they seem to be kind > of fiddly - they don't work well with multi table update queries, for > instance, and they generate a HUGE amount of background source code. > It is nice to have the strong typing, however. > > I have never found any non-Microsoft sample code which uses this > technique. Admittedly, most of the sample code in the wild consists > of short snippets built to illustrate a single technique, not a > complete application. > > Two questions: > 1. Are there developers out there using this approach for real life apps? > 2. If not, what do you recommend? > > > Thanks, > > Doug Steele From accessd at shaw.ca Mon Mar 7 11:52:58 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Mon, 7 Mar 2011 09:52:58 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: <9F8E42F58F6F45B18FBDE74842047BF9@creativesystemdesigns.com> Hi Gustav: Thanks for the information. Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Monday, March 07, 2011 9:05 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] ASP.Net development Hi Jim WinForms are true Windows desktop apps (I realise I should have changed the subject line, sorry). If you wish a browser based app, that's ASP.NET or Silverlight if we talk about Visual Studio and dotNet. /gustav >>> accessd at shaw.ca 07-03-2011 16:22 >>> Hi Gustav: Showing my ignorance about Winforms; does Winforms create browser based application or create a Windows based applications that uses a browser? Jim _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Mon Mar 7 12:45:14 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 7 Mar 2011 21:45:14 +0300 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: <5C52BAB72E1246CFBBFB629233371E72@nant> Hi Doug -- I think I have seen an ASP.NET book with sources where strongly typed datasets were used but I can't find it now. But maybe that was this book: Pro ASP.NET 2.0 E-Commerce in C# 2005 (Expert's Voice in .NET) http://www.amazon.com/ASP-NET-Commerce-2005-Experts-Voice/dp/1590597249/ref= pd_sim_b_8 It does have complete source code and the DAL code does use ADO.NET DataSets but not strongly typed. I have got programmed rather large ASP.NET application - more than 150 webforms with I suppose more than 70-80 heavily related MS SQL tables with millions rows in some of tables, hundreds SPs, ... several thousands C# custom classes, ORM layer etc. and this web application does work well forth year in row with a few support work - the latter mainly being adding new functionality. I have used asp.net development approach described in this book: ASP.NET 2.0 Website Programming: Problem - Design - Solution (Programmer to Programmer) http://www.amazon.com/ASP-NET-2-0-Website-Programming-Programmer/dp/07645846 42/ref=sr_1_2?s=books&ie=UTF8&qid=1299518125&sr=1-2 BTW, the same approach is used in DotNetNuke (but I didn't know that when I have got started to develop my customer web application I mentioned above.)) I have had to develop custom ORM mapping + SQL code generator to implement this approach. But it should be possible to use some existing code generators. This approach looks "low level" and it needs quite some coding (code generators help here) but good news is that it will never get outdated. And here is a book on how to use LINQ 2 SQL for ASP.NET applications (I haven't read and I haven't tried this approach) for the same sample ASP.NET application, which is used in the previously referred book: LINQ-ifying TheBeerHouse: An N-Tier LINQ Web Application with ASP.NET 2.0 Website Programming Problem Design Solution (Wrox Briefs) http://www.amazon.com/LINQ-ifying-TheBeerHouse-Application-Programming-Solut ion/dp/B0026A6AQU/ref=sr_1_106?s=books&ie=UTF8&qid=1299518458&sr=1-106 The most modern approach is to use ADO.NET Entity Framework for DAL as Gustav noted, and when ADO.NET EF is used then it should be possible to map logical/business layer objects to the custom stored procedures, which can insert, update, delete, select data from one or database tables... I haven't yet seen any books where ADO.NET Entity Framework is used with classical WebForms based ASP.NET applications - anybody? Some more information on ADO.NET EF and ASP.NET: - here seems to be a good book on ADO.NET EF: Entity Framework 4.0 Recipes: A Problem-Solution Approach (Expert's Voice in .NET) http://www.amazon.com/Entity-Framework-4-0-Recipes-Problem-Solution/dp/14302 27036/ref=sr_1_168?s=books&ie=UTF8&qid=1299519467&sr=1-168 - and this one "opens secretes" on how to build ultra-fast and robust asp.net applications: Ultra-Fast ASP.NET: Build Ultra-Fast and Ultra-Scalable web sites using ASP.NET and SQL Server [Paperback] http://www.amazon.com/Ultra-Fast-ASP-NET-Build-Ultra-Scalable-Server/dp/1430 223839/ref=sr_1_121?s=books&ie=UTF8&qid=1299518863&sr=1-121 Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Doug Steele Sent: 7 ????? 2011 ?. 6:05 To: dba-VB at databaseadvisors.com Subject: [dba-VB] ASP.Net development Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From dbdoug at gmail.com Mon Mar 7 23:34:51 2011 From: dbdoug at gmail.com (Doug Steele) Date: Mon, 7 Mar 2011 21:34:51 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: <5C52BAB72E1246CFBBFB629233371E72@nant> References: <5C52BAB72E1246CFBBFB629233371E72@nant> Message-ID: Hi Shamil: Thanks very much for your reply. I'll be checking your references out.... Doug On Mon, Mar 7, 2011 at 10:45 AM, Shamil Salakhetdinov wrote: > Hi Doug -- > > I think I have seen an ASP.NET book with sources where strongly typed > datasets were used but I can't find it now. > But maybe that was this book: > > Pro ASP.NET 2.0 E-Commerce in C# 2005 (Expert's Voice in .NET) > http://www.amazon.com/ASP-NET-Commerce-2005-Experts-Voice/dp/1590597249/ref= > pd_sim_b_8 > > It does have complete source code and the DAL code does use ADO.NET DataSets > but not strongly typed. > > I have got programmed rather large ASP.NET application - more than 150 > webforms with I suppose more than 70-80 heavily related MS SQL tables with > millions rows in some of tables, hundreds SPs, ... several thousands C# > custom classes, ORM layer etc. and this web application does work well forth > year in row with a few support work - the latter mainly being adding new > functionality. I have used asp.net development approach described in this > book: > > ASP.NET 2.0 Website Programming: Problem - Design - Solution (Programmer to > Programmer) > http://www.amazon.com/ASP-NET-2-0-Website-Programming-Programmer/dp/07645846 > 42/ref=sr_1_2?s=books&ie=UTF8&qid=1299518125&sr=1-2 > > BTW, the same approach is used in DotNetNuke (but I didn't know that when I > have got started to develop my customer web application I mentioned above.)) > > I have had to develop custom ORM mapping + SQL code generator to implement > this approach. > But it should be possible to use some existing code generators. > > This approach looks "low level" and it needs quite some coding (code > generators help here) but good news is that it will never get outdated. > > And here is a book on how to use LINQ 2 SQL for ASP.NET applications (I > haven't read and I haven't tried this approach) for the same sample ASP.NET > application, which is used in the previously referred book: > > LINQ-ifying TheBeerHouse: An N-Tier LINQ Web Application with ASP.NET 2.0 > Website Programming Problem Design Solution (Wrox Briefs) > http://www.amazon.com/LINQ-ifying-TheBeerHouse-Application-Programming-Solut > ion/dp/B0026A6AQU/ref=sr_1_106?s=books&ie=UTF8&qid=1299518458&sr=1-106 > > The most modern approach is to use ADO.NET Entity Framework for DAL as > Gustav noted, and when ADO.NET EF is used then it should be possible to map > logical/business layer objects to the custom stored procedures, which can > insert, update, delete, select data from one or database tables... > > I haven't yet seen any books where ADO.NET Entity Framework is used with > classical WebForms based ASP.NET applications - anybody? > > > Some more information on ADO.NET EF and ASP.NET: > > - here seems to be a good book on ADO.NET EF: > > Entity Framework 4.0 Recipes: A Problem-Solution Approach (Expert's Voice in > .NET) > http://www.amazon.com/Entity-Framework-4-0-Recipes-Problem-Solution/dp/14302 > 27036/ref=sr_1_168?s=books&ie=UTF8&qid=1299519467&sr=1-168 > > - and this one "opens secretes" on how to build ultra-fast and robust > asp.net applications: > > Ultra-Fast ASP.NET: Build Ultra-Fast and Ultra-Scalable web sites using > ASP.NET and SQL Server [Paperback] > http://www.amazon.com/Ultra-Fast-ASP-NET-Build-Ultra-Scalable-Server/dp/1430 > 223839/ref=sr_1_121?s=books&ie=UTF8&qid=1299518863&sr=1-121 > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Doug Steele > Sent: 7 ????? 2011 ?. 6:05 > To: dba-VB at databaseadvisors.com > Subject: [dba-VB] ASP.Net development > > Hello All: > > A lot of the Microsoft sample code for asp.net data access is built using > strongly typed tableadapters with the .xsd design surface. ?I've built two > apps using these in the last year, but they seem to be kind of fiddly - they > don't work well with multi table update queries, for instance, and they > generate a HUGE amount of background source code. > It is nice to have the strong typing, however. > > I have never found any non-Microsoft sample code which uses this technique. > Admittedly, most of the sample code in the wild consists of short snippets > built to illustrate a single technique, not a complete application. > > Two questions: > 1. Are there developers out there using this approach for real life apps? > 2. If not, what do you recommend? > > > Thanks, > > Doug Steele > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From raibeart at gmail.com Tue Mar 8 11:44:25 2011 From: raibeart at gmail.com (Robert Stewart) Date: Tue, 08 Mar 2011 11:44:25 -0600 Subject: [dba-VB] I'm getting nowhere In-Reply-To: References: Message-ID: <4d766a81.cac4e60a.3ab2.255f@mx.google.com> John, Logins: If you are in mixed mode, which for your application you should be, every "group" will have it's own login with a password. User: References a login. A user is given rights to a database and specific objects in a database. Roles: Generally, you can assign DataReader and DataWriter roles to a User in a database to give them CRUD rights to tables. You will need to GRANT EXECUTE rights to stored procedures. Does that help? Robert At 09:22 AM 3/7/2011, you wrote: >From: dba-vb-bounces at databaseadvisors.com >[mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >Sent: Saturday, February 19, 2011 2:54 PM >To: Sqlserver-Dba; VBA; Access Developers discussion and problem solving >Subject: [dba-VB] I'm getting nowhere > >I am getting nowhere on understanding SQL Server security. Microsoft >provides us with SQL Server Express which implies that joe blow (me) is >going to install / maintain it. > >I am not a SQL Server Admin and I cannot afford to spend the time to be one. > >Google is my friend. BOL is not. > >Except that Google is taking me to these places where I am expected to >already know how this stuff works, and then wants to make me a *better* >administrator. Which of course is useless because I am not an administrator >at all. > >OTOH I am not stupid. If I could find something that started at the "This >is SQL Server security" >basics I could learn this stuff. Before anyone says "RTFM (BOL)" let me >simply say, "not happening". I have tried BOL and it simply sucks for my >level of expertise (my opinion of course). > If that is your advice, simply stay out of this thread. Thanks! > >So... my needs: > >I need to set up several SQL Server databases for use by different, very >small groups (5-20 people) of entirely unrelated people. What I mean by >that is that each DB is for a different "company" if you will. I need to >access these databases from C#. I understand the group / user paradigm. I >would like to create groups and users. Specific groups can do specific >things in the database, some can see data but not modify it. Some can add >records in specific tables but not others. Some can run reports (view). > >I do *NOT* want to create windows level groups and users if I can avoid it. >These are people that I do not necessarily know and I do not want to give >them any rights at the machine level, and I prefer to not maintain such >lists at the machine level. > >Unfortunately SQL Server does not seem to model Groups / users. I go into >SQL Server and see a security tab. It has "logins". Is that a user? A >specific ability to log in with a password? To what? The server itself? A >specific database? Groups of databases? > >I see "roles" but these appear to be aimed at the server and none of these >people are going to be doing anything at the server level. > >Can I safely ignore everything under the server security tab? > >I go to a database and I see a security tab. It has users and roles. >Hmm... better (I would think). I would like to add users "under" the >specific database that the user will access. > >So I try to add a new user but I do not see anywhere to require a password. >Hmmm... > >I go into roles and I do not see any predefined role that looks like it >would be useful to me in meeting my needs described above. If I look at >"add new role" it asks for a password. The User / group model does nto >assign passwords at the group level which implies that a role is not a group >at the user / group paradigm. > >Is it just me, or is SQL Server security just... different? Am I correct in >assuming that it doesn't implement a user / group paradigm? > >And more importantly, where can I go to get a plain, simple, English >description of how this mess works? > >And please excuse the tone that results from my frustration. The only help >documents that I have found (and I have extensive lists of bookmarked web >pages) so far assume that I am an administrator. > I am not, and cannot afford to become one. And yet MS pushes SQL Express >as if I (non-admin) should be able to use this as a data store pool. > >Help! > >-- >John W. Colby >www.ColbyConsulting.com >_______________________________________________ From jwcolby at colbyconsulting.com Thu Mar 10 07:12:44 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 08:12:44 -0500 Subject: [dba-VB] c# lock() Message-ID: <4D78CE4C.5000308@colbyconsulting.com> I am using multi-threading and need to lock list objects so that two different threads can access them. Do I even need to lock the list inside of the property as follows: public Dictionary pLstFiles { get { lock (lstFiles) { return lstFiles; } } } or only when I actually access the list? If I create a property to return the list (without locking it in the property) public Dictionary pLstFiles { get { return lstFiles; } } and I then get a pointer to the list using that property Dictionary LstFiles; LstFiles = SomeObj.pLstFiles; and I then lock the pointer while using it lock(LstFiles) { do stuff to list here... } Does that in fact lock the list to other objects doing the same thing (getting the list through that property)? IOW is the lock attribute affixed to the list at the list itself so that anything trying to use the list anywhere in the program can see that the list is locked? -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Thu Mar 10 10:29:39 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 11:29:39 -0500 Subject: [dba-VB] split an SVN database Message-ID: <4D78FC73.6000202@colbyconsulting.com> I use SVN. I am looking at doing a major split of a project into two projects. I want to leave the existing solution until I have one half of the current project carved out, debugged and running and in it's own SVN database. However I don't think I want to carry along the baggage of the old revisions into the new database. How do I go about this? -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Thu Mar 10 14:37:01 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 15:37:01 -0500 Subject: [dba-VB] c# - controlling the run state / status of your threaded classes Message-ID: <4D79366D.8060905@colbyconsulting.com> As I write classes that consume threads, I need to be able to start and stop these threads and the class that consumes the threads. For example, I open a form and load a class instance that starts performing some process - exporting sets of records to csv files for example. Once that class is loaded and processing, I need to be able to ask that class to stop exporting files. However there may be a time lag between asking and the currently processing file finishing it's export. That is precisely the purpose of a thread - tell it to go and do something else. So (for example) I have a check box that I check that tells the class to start exporting. When it starts it sets a flag saying that it is running and places text into the check box label saying that it is exporting (busy). If I then uncheck the check box I am telling the class to stop exporting. The class is monitoring the check box and will not process the next file but it can't just exit. So it needs to set a flag saying that it is stopping and display "stopping" on the check box label. When it finishes exporting the file, it sets a stopped flag and says displays "terminated" in the check box label. So we have a couple of "commands" (start / stop) and a few statuses (stopped, stopping, running). I would like to "standardize" this into a class so that any class which needs this interface does it in the same way. Some controlling object can tell the object to start and stop, and the class can tell the controlling object that is is stopped, stopping or running. this class does not6 need to do any updating of labels or anything like that, just hold the variables, properties, methods and code to perform this in a standard way. Is anyone doing this? Have existing code you would like to share? Have existing experience trying to do this you would like to share? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Thu Mar 10 15:35:24 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 00:35:24 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D78CE4C.5000308@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com> Message-ID: <3693FCAC03E94BCA9C150F6E7AFE4A71@nant> Hi John -- <<< Does that in fact lock the list to other objects doing the same thing (getting the list through that property)? >>> You have to lock/synchronize/serialize not the list itself but simultaneous access to the list from different threads. You'd better use special static object variable to implement locking. I will post sample code in my following posting here... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 16:13 To: VBA Subject: [dba-VB] c# lock() I am using multi-threading and need to lock list objects so that two different threads can access them. Do I even need to lock the list inside of the property as follows: public Dictionary pLstFiles { get { lock (lstFiles) { return lstFiles; } } } or only when I actually access the list? If I create a property to return the list (without locking it in the property) public Dictionary pLstFiles { get { return lstFiles; } } and I then get a pointer to the list using that property Dictionary LstFiles; LstFiles = SomeObj.pLstFiles; and I then lock the pointer while using it lock(LstFiles) { do stuff to list here... } Does that in fact lock the list to other objects doing the same thing (getting the list through that property)? IOW is the lock attribute affixed to the list at the list itself so that anything trying to use the list anywhere in the program can see that the list is locked? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Thu Mar 10 15:36:58 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 00:36:58 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D78CE4C.5000308@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com> Message-ID: <4889C56E6FF246D6AB908FEFD1069A68@nant> Hi John -- Here is sample code I promised to post. using System; using System.Collections.Generic; namespace ConsoleApplication1 { class Log { public static bool On { get; set; } public static void WriteLine(string message) { WriteLine(message, false); } public static void WriteLine(string message, bool forceLogging) { if (Log.On || forceLogging) System.Console.WriteLine(message); } } class MyObject { static int _newId; public MyObject() { _id = System.Threading.Interlocked.Increment(ref _newId); _timeStamp = DateTime.Now; } int _id; public int ID { get { return _id; } set { _id = value; } } DateTime _timeStamp; public DateTime TimeStamp { get { return _timeStamp; } set { _timeStamp = value; } } } class MyObjectsStore { private List _keys = new List(); private Dictionary _myObjects = new Dictionary(); public MyObject this[int key] { get { MyObject myObject = null; lock (_myObjectsLocker) { _myObjects.TryGetValue(key, out myObject); } return myObject; } } public int Count { get { lock (_myObjectsLocker) { return _myObjects.Count; } } } private static object _myObjectsLocker = new object(); public void Add(MyObject myObject) { lock (_myObjectsLocker) { _myObjects.Add(myObject.ID, myObject); _keys.Add(myObject.ID); } } public MyObject Remove(int id) { MyObject myobject = null; lock (_myObjectsLocker) { myobject = _myObjects[id]; _myObjects.Remove(id); } return myobject; } static Random _rnd = new Random(); public MyObject RemoveRandomObject() { MyObject myobject = null; int index = -1; int key = -1; try { lock (_myObjectsLocker) { index = _rnd.Next(_keys.Count - 1); key = _keys[index]; myobject = Remove(key); _keys.Remove(key); } } catch (Exception ex) { Log.WriteLine(string.Format("[{0}]: RemoveRandomObject: {1}, index = {2}, key = {3}", System.Threading.Thread.CurrentThread.ManagedThreadId, ex.Message, index , key)); } return myobject; } } class Program { public const int OBJECTS_PER_THREAD_COUNT = 7; public const int REMOVAL_MAX_RETRIES_COUNT = 5; public static void AddObjectsTest() { for (int i = 1; i <= OBJECTS_PER_THREAD_COUNT; i++) { MyObject myObject = new MyObject(); _store.Add(myObject); Log.WriteLine(string.Format("[{0}]: Object Added - ID = {1}, TS = {2}", System.Threading.Thread.CurrentThread.ManagedThreadId, myObject.ID, myObject.TimeStamp.Ticks)); System.Threading.Thread.Sleep(2); System.Threading.Interlocked.Increment(ref _createdObjectsCount); } System.Threading.Interlocked.Decrement(ref _threadsCount); } public static void RemoveObjectsTest() { for (int i = 1; i <= OBJECTS_PER_THREAD_COUNT; i++) { int retryCount = REMOVAL_MAX_RETRIES_COUNT; retry: MyObject myObject = _store.RemoveRandomObject(); if (myObject != null) { Log.WriteLine(string.Format("[{0}]: Object Removed - ID = {1}, TS = {2}", System.Threading.Thread.CurrentThread.ManagedThreadId, myObject.ID, myObject.TimeStamp.Ticks)); System.Threading.Thread.Sleep(2); } System.Threading.Thread.Sleep(2); if (myObject == null && --retryCount >= 0) { Log.WriteLine(string.Format("[{0}]: Object Removal failed, retry attempt# {1}", System.Threading.Thread.CurrentThread.ManagedThreadId, REMOVAL_MAX_RETRIES_COUNT-retryCount)); System.Threading.Interlocked.Increment(ref _totalRetriesCount); goto retry; } else if (myObject == null) { Log.WriteLine(string.Format("[{0}]: Object Removal failed", System.Threading.Thread.CurrentThread.ManagedThreadId)); System.Threading.Interlocked.Increment(ref _failedObjectRemovalsCount); } else System.Threading.Interlocked.Increment(ref _removedObjectsCount); } System.Threading.Interlocked.Decrement(ref _threadsCount); } static int _createdObjectsCount; static int _removedObjectsCount; static int _totalRetriesCount; static int _failedObjectRemovalsCount; static int _threadsCount; static MyObjectsStore _store; const int THREADS_COUNT = 20; //000; static void Main(string[] args) { Log.On = true; Log.WriteLine(string.Format("{0}: Test Started.", DateTime.Now)); try { Log.On = false; _store = new MyObjectsStore(); for (int i = 1; i <= THREADS_COUNT; i++) { if (i % 2 == 1) new System.Threading.Thread( new System.Threading.ThreadStart(AddObjectsTest)) .Start(); else new System.Threading.Thread( new System.Threading.ThreadStart(RemoveObjectsTest)) .Start(); _threadsCount++; } while (_threadsCount > 0) System.Threading.Thread.Sleep(100); System.Threading.Thread.Sleep(100); Log.On = true; Log.WriteLine(string.Format("Total Threads Count = {0}", THREADS_COUNT)); Log.WriteLine(string.Format("Store has {0} objects, Created = {1}, Removed = {2}", _store.Count, _createdObjectsCount, _removedObjectsCount)); Log.WriteLine(string.Format("RemovalRetriesCount = {0}, FailedRemovalsCount = {1}", _totalRetriesCount, _failedObjectRemovalsCount)); } catch (Exception ex) { Log.On = true; Log.WriteLine(string.Format("Program.Main: " + ex.Message)); } finally { Log.On = true; } Log.WriteLine(string.Format("{0}: Test Finished.", DateTime.Now)); } } } Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 16:13 To: VBA Subject: [dba-VB] c# lock() <<< snip >>> From shamil at smsconsulting.spb.ru Thu Mar 10 15:38:46 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 00:38:46 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D78CE4C.5000308@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com> Message-ID: <50A8FE7A468B41D48B9CC4F729CB02E3@nant> Hi John -- Here are my test results for sample code I've just posted (hopefully it get through AccessD - it was a bit lengthy) //11/03/2011 0:32:50: Test Started. //Total Threads Count = 20 //Store has 0 objects, Created = 70, Removed = 70 //RemovalRetriesCount = 0, FailedRemovalsCount = 0 //11/03/2011 0:32:50: Test Finished. //11/03/2011 0:32:25: Test Started. //Total Threads Count = 200 //Store has 0 objects, Created = 700, Removed = 700 //RemovalRetriesCount = 0, FailedRemovalsCount = 0 //11/03/2011 0:32:25: Test Finished. //11/03/2011 0:31:32: Test Started. //Total Threads Count = 20000 //Store has 19 objects, Created = 70000, Removed = 69981 //RemovalRetriesCount = 295, FailedRemovalsCount = 19 //11/03/2011 0:31:43: Test Finished. //11/03/2011 0:33:32: Test Started. //[3]: Object Added - ID = 1, TS = 634354004122968750 //[5]: Object Added - ID = 2, TS = 634354004122968750 //[3]: Object Added - ID = 3, TS = 634354004122968750 //[4]: Object Removed - ID = 2, TS = 634354004122968750 //[5]: Object Added - ID = 4, TS = 634354004122968750 //[6]: Object Removed - ID = 3, TS = 634354004122968750 //[7]: Object Added - ID = 5, TS = 634354004122968750 //[8]: Object Removed - ID = 4, TS = 634354004122968750 //[9]: Object Added - ID = 6, TS = 634354004122968750 //[10]: Object Removed - ID = 1, TS = 634354004122968750 //[11]: Object Added - ID = 7, TS = 634354004122968750 //[3]: Object Added - ID = 8, TS = 634354004122968750 //[12]: Object Removed - ID = 7, TS = 634354004122968750 //[5]: Object Added - ID = 9, TS = 634354004122968750 //[7]: Object Added - ID = 11, TS = 634354004122968750 //[9]: Object Added - ID = 10, TS = 634354004122968750 //[4]: Object Removed - ID = 6, TS = 634354004122968750 //[11]: Object Added - ID = 12, TS = 634354004122968750 //[3]: Object Added - ID = 13, TS = 634354004122968750 //[7]: Object Added - ID = 16, TS = 634354004123125000 //[6]: Object Removed - ID = 13, TS = 634354004122968750 //[11]: Object Added - ID = 17, TS = 634354004123125000 //[9]: Object Added - ID = 15, TS = 634354004123125000 //[4]: Object Removed - ID = 16, TS = 634354004123125000 //[8]: Object Removed - ID = 11, TS = 634354004122968750 //[10]: Object Removed - ID = 8, TS = 634354004122968750 //[5]: Object Added - ID = 14, TS = 634354004122968750 //[11]: Object Added - ID = 21, TS = 634354004123125000 //[3]: Object Added - ID = 18, TS = 634354004123125000 //[7]: Object Added - ID = 19, TS = 634354004123125000 //[10]: Object Removed - ID = 15, TS = 634354004123125000 //[6]: Object Removed - ID = 5, TS = 634354004122968750 //[4]: Object Removed - ID = 19, TS = 634354004123125000 //[8]: Object Removed - ID = 9, TS = 634354004122968750 //[10]: Object Removed - ID = 21, TS = 634354004123125000 //[6]: Object Removed - ID = 17, TS = 634354004123125000 //[8]: Object Removed - ID = 12, TS = 634354004122968750 //[4]: Object Removed - ID = 22, TS = 634354004123281250 //[12]: Object Removed - ID = 10, TS = 634354004122968750 //[6]: Object Removed - ID = 18, TS = 634354004123125000 //[4]: Object Removed - ID = 24, TS = 634354004123437500 //[8]: Object Removed - ID = 20, TS = 634354004123125000 //[6]: Object Removed - ID = 23, TS = 634354004123437500 //[12]: Object Removed - ID = 25, TS = 634354004123437500 //[10]: Object Removed - ID = 14, TS = 634354004122968750 //[9]: Object Added - ID = 20, TS = 634354004123125000 //[5]: Object Added - ID = 22, TS = 634354004123281250 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[10]: Object Removed - ID = 26, TS = 634354004124062500 //[6]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 1 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[10]: Object Removed - ID = 27, TS = 634354004124062500 //[3]: Object Added - ID = 24, TS = 634354004123437500 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[6]: Object Removal failed, retry attempt# 1 //[6]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 2 //[4]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[8]: Object Removal failed, retry attempt# 1 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[11]: Object Added - ID = 23, TS = 634354004123437500 //[7]: Object Added - ID = 25, TS = 634354004123437500 //[6]: Object Removal failed, retry attempt# 2 //[12]: Object Removed - ID = 28, TS = 634354004124531250 //[4]: Object Removal failed, retry attempt# 1 //[8]: Object Removal failed, retry attempt# 2 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[6]: Object Removed - ID = 29, TS = 634354004125000000 //[4]: Object Removed - ID = 30, TS = 634354004125000000 //[8]: Object Removal failed, retry attempt# 3 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 1 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[8]: Object Removal failed, retry attempt# 4 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 2 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[3]: Object Added - ID = 28, TS = 634354004124531250 //[12]: Object Removal failed, retry attempt# 3 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[8]: Object Removal failed, retry attempt# 5 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 4 //[7]: Object Added - ID = 30, TS = 634354004125000000 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[5]: Object Added - ID = 27, TS = 634354004124062500 //[9]: Object Added - ID = 26, TS = 634354004124062500 //[12]: Object Removal failed, retry attempt# 5 //[12]: Object Removed - ID = 31, TS = 634354004126093750 //[8]: Object Removal failed //[8]: Object Removed - ID = 32, TS = 634354004126250000 //[9]: Object Added - ID = 33, TS = 634354004126250000 //[12]: Object Removed - ID = 33, TS = 634354004126250000 //[11]: Object Added - ID = 29, TS = 634354004125000000 //[9]: Object Added - ID = 34, TS = 634354004126406250 //[12]: Object Removed - ID = 34, TS = 634354004126406250 //[7]: Object Added - ID = 31, TS = 634354004126093750 //[5]: Object Added - ID = 32, TS = 634354004126250000 //[11]: Object Added - ID = 35, TS = 634354004126406250 //Total Threads Count = 10 //Store has 1 objects, Created = 35, Removed = 34 //RemovalRetriesCount = 15, FailedRemovalsCount = 1 //11/03/2011 0:33:32: Test Finished. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 16:13 To: VBA Subject: [dba-VB] c# lock() <<< snip >>> From jwcolby at colbyconsulting.com Thu Mar 10 16:27:16 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 17:27:16 -0500 Subject: [dba-VB] c# lock() In-Reply-To: <3693FCAC03E94BCA9C150F6E7AFE4A71@nant> References: <4D78CE4C.5000308@colbyconsulting.com> <3693FCAC03E94BCA9C150F6E7AFE4A71@nant> Message-ID: <4D795044.6080808@colbyconsulting.com> Shamil, Thanks for the code but I think it is overkill for my needs so far, though I will put it in the archives for when I need it. In my system so far I have (or can arrange to have ) only two threads trying to access a list. Basically what I am doing is reading records out of sql server into a class factory, and placing the class instances into a list. Another object with its own thread is coming by and looking at the objects in the list to see if there is any work for it to do. If it finds an object to work on, it grabs a pointer to that item in the list and stores it in a pointer and releases the list. One of the ways I saw this handled was to create factory which creates a brand new list object, grabs a pointer to the list, locks that and then copies the objects in the list pointers to classes) into that new list, unlocks the list and hands back a pointer to this brand new list. thus the new list could be iterated at will. I am just trying to lock the list while one thread adds objects to the list or another thread gets objects out of the list. So if I have object A which contains a list and is adding / deleting objects to that list... and object B which needs to iterate through the list looking for an object of interest in the list, and grabbing a pointer to that object, there is no way to do this without all of the code you wrote? John W. Colby www.ColbyConsulting.com On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: > Hi John -- > > <<< > Does that in fact lock the list to other objects doing the same thing > (getting the list through that property)? >>>> > You have to lock/synchronize/serialize not the list itself but simultaneous > access to the list from different threads. > You'd better use special static object variable to implement locking. > > I will post sample code in my following posting here... > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Thu Mar 10 16:51:26 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 01:51:26 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D795044.6080808@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com> Message-ID: <3B9DE744CEA244829179A81CA818F92F@nant> Hi John -- Do you you need add, remove and iterate a list to get an object or you need to add, remove and get an object by its key? - if the latter - here is the code you can use: private Dictionary _myObjects = new Dictionary(); private static object _myObjectsLocker = new object(); public void Add(MyObject myObject) { lock (_myObjectsLocker) { _myObjects.Add(myObject.ID, myObject); _keys.Add(myObject.ID); } } public MyObject Remove(int id) { MyObject myobject = null; lock (_myObjectsLocker) { myobject = _myObjects[id]; _myObjects.Remove(id); } return myobject; } public MyObject this[int key] { get { MyObject myObject = null; lock (_myObjectsLocker) { _myObjects.TryGetValue(key, out myObject); } return myObject; } } for the case of iteration just replace: _myObjects.TryGetValue(key, out myObject); with your iteration code. As for the sample code - changing qty of threads and changing total computer workload (outer context for the test code) you can see how different could be final results when heavy multi-threading is used. And even if it's not heavy - it still can result in weird output if special multi-threading coding technique aren't used... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 1:27 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, Thanks for the code but I think it is overkill for my needs so far, though I will put it in the archives for when I need it. In my system so far I have (or can arrange to have ) only two threads trying to access a list. Basically what I am doing is reading records out of sql server into a class factory, and placing the class instances into a list. Another object with its own thread is coming by and looking at the objects in the list to see if there is any work for it to do. If it finds an object to work on, it grabs a pointer to that item in the list and stores it in a pointer and releases the list. One of the ways I saw this handled was to create factory which creates a brand new list object, grabs a pointer to the list, locks that and then copies the objects in the list pointers to classes) into that new list, unlocks the list and hands back a pointer to this brand new list. thus the new list could be iterated at will. I am just trying to lock the list while one thread adds objects to the list or another thread gets objects out of the list. So if I have object A which contains a list and is adding / deleting objects to that list... and object B which needs to iterate through the list looking for an object of interest in the list, and grabbing a pointer to that object, there is no way to do this without all of the code you wrote? John W. Colby www.ColbyConsulting.com On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: > Hi John -- > > <<< > Does that in fact lock the list to other objects doing the same thing > (getting the list through that property)? >>>> > You have to lock/synchronize/serialize not the list itself but > simultaneous access to the list from different threads. > You'd better use special static object variable to implement locking. > > I will post sample code in my following posting here... > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Thu Mar 10 17:11:33 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 02:11:33 +0300 Subject: [dba-VB] c# - controlling the run state / status of your threadedclasses In-Reply-To: <4D79366D.8060905@colbyconsulting.com> References: <4D79366D.8060905@colbyconsulting.com> Message-ID: <3B44AE877A14441CA13A51D1BA80FF65@nant> Hi John -- <<< So we have a couple of "commands" (start / stop) and a few statuses (stopped, stopping, running). >>> You'd better also add 'paused' (and maybe 'pausing' and 'resuming') for your list of states. I must note it's not easy to "standardize" such a coding - as with every framework/standard there immediately come non-standard situations to code into framework/standard - that's endless story... I have such a coding but it's rather complicated, several nesting levels etc.etc. - and it was programmed for a customer, I have reused it already in three projects of that customer but every time I do reuse it I have to do some additions/changes. It tested with up to hundred heavy duty threads talking to web services, MS Access(!) be databases, informing FE UI on threads' status changes etc. - and FE is able to work for days without restarting but that is a result of a very big work - it took two years+ to get from first version to the current one. Of course I didn't program it for two years, but getting all that experience to make it as it's now - that was rather time consuming - at least for me... If I ever will make that code less complicated I will share it but that will not happen soon, sorry... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 23:37 To: VBA Subject: [dba-VB] c# - controlling the run state / status of your threadedclasses As I write classes that consume threads, I need to be able to start and stop these threads and the class that consumes the threads. For example, I open a form and load a class instance that starts performing some process - exporting sets of records to csv files for example. Once that class is loaded and processing, I need to be able to ask that class to stop exporting files. However there may be a time lag between asking and the currently processing file finishing it's export. That is precisely the purpose of a thread - tell it to go and do something else. So (for example) I have a check box that I check that tells the class to start exporting. When it starts it sets a flag saying that it is running and places text into the check box label saying that it is exporting (busy). If I then uncheck the check box I am telling the class to stop exporting. The class is monitoring the check box and will not process the next file but it can't just exit. So it needs to set a flag saying that it is stopping and display "stopping" on the check box label. When it finishes exporting the file, it sets a stopped flag and says displays "terminated" in the check box label. So we have a couple of "commands" (start / stop) and a few statuses (stopped, stopping, running). I would like to "standardize" this into a class so that any class which needs this interface does it in the same way. Some controlling object can tell the object to start and stop, and the class can tell the controlling object that is is stopped, stopping or running. this class does not6 need to do any updating of labels or anything like that, just hold the variables, properties, methods and code to perform this in a standard way. Is anyone doing this? Have existing code you would like to share? Have existing experience trying to do this you would like to share? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 11 08:26:06 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 09:26:06 -0500 Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath Message-ID: <4D7A30FE.8050500@colbyconsulting.com> I am trying to copy a solution to a new location, strip all version control from it, delete a ton of projects within that solution and then version control the new solution. I copied the entire solution from my laptop to a VM and under my user / projects there. The copy still appears to be under version control. I looked in help and it says to copy the directory to itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the top level directory (the solution) but left the check mark on all of the subdirectories (projects and stuff). Does anyone have a clue how to completely and entirely remove a directory from version control - remove all green check marks from every subdirectory etc.? -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Fri Mar 11 08:59:23 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 09:59:23 -0500 Subject: [dba-VB] [dba-SQLServer] SVN - removing all version control for a directory and all files underneath In-Reply-To: <4D7A30FE.8050500@colbyconsulting.com> References: <4D7A30FE.8050500@colbyconsulting.com> Message-ID: <4D7A38CB.7090807@colbyconsulting.com> As it turns out, doing the "export to same directory" seems to actually "unregister" the directory from source control. It leaves the green check marks in explorer - not sure what that means. However if I go into the solution itself, it no longer has the version control dots next to the items in the solution explorer. So it appears to have removed that local copy of the solution from source control. John W. Colby www.ColbyConsulting.com On 3/11/2011 9:26 AM, jwcolby wrote: > I am trying to copy a solution to a new location, strip all version control from it, delete a ton of > projects within that solution and then version control the new solution. > > I copied the entire solution from my laptop to a VM and under my user / projects there. The copy > still appears to be under version control. I looked in help and it says to copy the directory to > itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the > top level directory (the solution) but left the check mark on all of the subdirectories (projects > and stuff). > > Does anyone have a clue how to completely and entirely remove a directory from version control - > remove all green check marks from every subdirectory etc.? From shamil at smsconsulting.spb.ru Fri Mar 11 09:00:05 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 18:00:05 +0300 Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath In-Reply-To: <4D7A30FE.8050500@colbyconsulting.com> References: <4D7A30FE.8050500@colbyconsulting.com> Message-ID: <9192624AA1BA4A6BB323E053411525B8@nant> Hi John -- 1. Just delete hidden .svn subfolder from root folder and all subfolders. 2. Right-click folder in MS Windows Explorer window and use Tortoise SVN Export... command from popup-menu 3. Use command line SVN utility with export command http://svnbook.red-bean.com/nightly/en/svn.ref.svn.c.export.html Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 17:26 To: Access Developers discussion and problem solving; VBA; Sqlserver-Dba Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath I am trying to copy a solution to a new location, strip all version control from it, delete a ton of projects within that solution and then version control the new solution. I copied the entire solution from my laptop to a VM and under my user / projects there. The copy still appears to be under version control. I looked in help and it says to copy the directory to itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the top level directory (the solution) but left the check mark on all of the subdirectories (projects and stuff). Does anyone have a clue how to completely and entirely remove a directory from version control - remove all green check marks from every subdirectory etc.? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 11 10:56:44 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 11:56:44 -0500 Subject: [dba-VB] c# - Creating and using a shared module Message-ID: <4D7A544C.7090904@colbyconsulting.com> I have split my solution into two solutions. However each of those two solutions uses a shared project which I call "base objects". Obviously I want to split that off and then reference that solution from both of the main solutions. I have never done this before. Any suggestions, tips and tricks, warnings, how to etc.? -- John W. Colby www.ColbyConsulting.com From gustav at cactus.dk Fri Mar 11 11:59:11 2011 From: gustav at cactus.dk (Gustav Brock) Date: Fri, 11 Mar 2011 18:59:11 +0100 Subject: [dba-VB] Visual Studio 2010 SP1 Message-ID: Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-475d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav From shamil at smsconsulting.spb.ru Fri Mar 11 11:52:31 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 20:52:31 +0300 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7A544C.7090904@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> Message-ID: <1573E00753994C19BE24FA9E50093615@nant> Hi John -- <<< Any suggestions, tips and tricks, warnings, how to etc.? >>> Just do it! :) I mean if you managed to handle MS Access library databases then handling .NET class libraries would be a "breeze" work for you. Create class library project - you can keep it within its own solution having main (console) project of that solution as a (unit) test one. Put all three solutions - two application ones and class library one in subfolders of the same root folder. (You can do it differently but this is how it's done usually AFAIK) Add existing class library project to application solutions. Reference class library projects from application projects. if you use VS2008 and you have a good PC with plenty of memory (and you have one don't you?) then you can keep all three solutions open in three instances of VS2008. Just keep tack from which solution you have made class library code changes last time - when you'll switch to the other solution and if you have the same source code file opened in it you'll get a message like that "Source file changed - would you like to reload it?" - reply yes... ... Use SVN or Mercurial for source code control. Do commit source code changes to the source code repository from time to time... ... I have 58 projects in one of the solutions here - that results in more than 5 apps and quite some class libs compiled and built - AFAIK folks do have more than hundred projects within solutions sometimes... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 19:57 To: VBA Subject: [dba-VB] c# - Creating and using a shared module I have split my solution into two solutions. However each of those two solutions uses a shared project which I call "base objects". Obviously I want to split that off and then reference that solution from both of the main solutions. I have never done this before. Any suggestions, tips and tricks, warnings, how to etc.? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From df.waters at comcast.net Fri Mar 11 13:46:10 2011 From: df.waters at comcast.net (Dan Waters) Date: Fri, 11 Mar 2011 13:46:10 -0600 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: Message-ID: <003001cbe024$f9c71750$ed5545f0$@comcast.net> I did this yesterday. The download iso image was actually about 1.5 Gb, but I downloaded, burned to disc, and installed in about an hour. Your mileage may vary! Dan -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-47 5d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 11 20:37:10 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 21:37:10 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <1573E00753994C19BE24FA9E50093615@nant> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant> Message-ID: <4D7ADC56.1040806@colbyconsulting.com> You make it sound so easy, but it is an existing project already which was not created as a class library project. I need to create the class library project then somehow get the directories and files into that project and get it to compile. At that point I can reference it by other solutions. John W. Colby www.ColbyConsulting.com On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: > Hi John -- > > <<< > Any suggestions, tips and tricks, warnings, how to etc.? >>>> > Just do it! :) > > I mean if you managed to handle MS Access library databases then handling > .NET class libraries would be a "breeze" work for you. > > Create class library project - you can keep it within its own solution > having main (console) project of that solution as a (unit) test one. > Put all three solutions - two application ones and class library one in > subfolders of the same root folder. (You can do it differently but this is > how it's done usually AFAIK) > Add existing class library project to application solutions. > Reference class library projects from application projects. > if you use VS2008 and you have a good PC with plenty of memory (and you have > one don't you?) then you can keep all three solutions open in three > instances of VS2008. > Just keep tack from which solution you have made class library code changes > last time - when you'll switch to the other solution and if you have the > same source code file opened in it you'll get a message like that "Source > file changed - would you like to reload it?" - reply yes... > > ... > > Use SVN or Mercurial for source code control. Do commit source code changes > to the source code repository from time to time... > ... > > I have 58 projects in one of the solutions here - that results in more than > 5 apps and quite some class libs compiled and built - AFAIK folks do have > more than hundred projects within solutions sometimes... > > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 11 ????? 2011 ?. 19:57 > To: VBA > Subject: [dba-VB] c# - Creating and using a shared module > > I have split my solution into two solutions. However each of those two > solutions uses a shared project which I call "base objects". Obviously I > want to split that off and then reference that solution from both of the > main solutions. > > I have never done this before. Any suggestions, tips and tricks, warnings, > how to etc.? > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From michael at ddisolutions.com.au Fri Mar 11 20:39:50 2011 From: michael at ddisolutions.com.au (Michael Maddison) Date: Sat, 12 Mar 2011 13:39:50 +1100 Subject: [dba-VB] SVN - removing all version control for a directory andall files underneath References: <4D7A30FE.8050500@colbyconsulting.com> <9192624AA1BA4A6BB323E053411525B8@nant> Message-ID: <99266C61B516644D9727F983FAFAB46508623F@remote.ddisolutions.com.au> Hi Guys, I found a reg file that adds a right click menu item to remove all hidden svn dirs. And files from the selected dir. Create a .reg file, Paste this code in, Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Folder\shell\DeleteSVN] @="Delete SVN Folders" [HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Folder\shell\DeleteSVN\command] @="cmd.exe /c \"TITLE Removing SVN Folders in %1 && COLOR 9A && FOR /r \"%1\" %%f IN (.svn) DO RD /s /q \"%%f\" \"" Save it, Merge it to your reg. Works for me. Cheers Michael From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: Saturday, 12 March 2011 2:00 AM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] SVN - removing all version control for a directory andall files underneath Hi John -- 1. Just delete hidden .svn subfolder from root folder and all subfolders. 2. Right-click folder in MS Windows Explorer window and use Tortoise SVN Export... command from popup-menu 3. Use command line SVN utility with export command http://svnbook.red-bean.com/nightly/en/svn.ref.svn.c.export.html Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 17:26 To: Access Developers discussion and problem solving; VBA; Sqlserver-Dba Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath I am trying to copy a solution to a new location, strip all version control from it, delete a ton of projects within that solution and then version control the new solution. I copied the entire solution from my laptop to a VM and under my user / projects there. The copy still appears to be under version control. I looked in help and it says to copy the directory to itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the top level directory (the solution) but left the check mark on all of the subdirectories (projects and stuff). Does anyone have a clue how to completely and entirely remove a directory from version control - remove all green check marks from every subdirectory etc.? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com ________________________________ No virus found in this message. Checked by AVG - www.avg.com Version: 10.0.1204 / Virus Database: 1498/3501 - Release Date: 03/11/11 From jwcolby at colbyconsulting.com Fri Mar 11 21:09:49 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 22:09:49 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7ADC56.1040806@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant> <4D7ADC56.1040806@colbyconsulting.com> Message-ID: <4D7AE3FD.1040206@colbyconsulting.com> I found a "make project from existing files. When I did that it offered to use a selected directory so I dragged the project directory into a new directory and told it to use that. I selected a class library type of project and it went to work "converting" my existing project into a class library. 112 compile errors, most having to do with the fact that it no longer understands using System.Windows.XYZ Many of the common classes did things like accept a combo loaded data into it in a standard way. Stuff like that. So there we are. I "just did it"! ;) John W. Colby www.ColbyConsulting.com On 3/11/2011 9:37 PM, jwcolby wrote: > You make it sound so easy, but it is an existing project already which was not created as a class > library project. I need to create the class library project then somehow get the directories and > files into that project and get it to compile. > > At that point I can reference it by other solutions. > > John W. Colby > www.ColbyConsulting.com > > On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Any suggestions, tips and tricks, warnings, how to etc.? >>>>> >> Just do it! :) >> >> I mean if you managed to handle MS Access library databases then handling >> .NET class libraries would be a "breeze" work for you. >> >> Create class library project - you can keep it within its own solution >> having main (console) project of that solution as a (unit) test one. >> Put all three solutions - two application ones and class library one in >> subfolders of the same root folder. (You can do it differently but this is >> how it's done usually AFAIK) >> Add existing class library project to application solutions. >> Reference class library projects from application projects. >> if you use VS2008 and you have a good PC with plenty of memory (and you have >> one don't you?) then you can keep all three solutions open in three >> instances of VS2008. >> Just keep tack from which solution you have made class library code changes >> last time - when you'll switch to the other solution and if you have the >> same source code file opened in it you'll get a message like that "Source >> file changed - would you like to reload it?" - reply yes... >> >> ... >> >> Use SVN or Mercurial for source code control. Do commit source code changes >> to the source code repository from time to time... >> ... >> >> I have 58 projects in one of the solutions here - that results in more than >> 5 apps and quite some class libs compiled and built - AFAIK folks do have >> more than hundred projects within solutions sometimes... >> >> >> Thank you. >> >> -- >> Shamil From shamil at smsconsulting.spb.ru Sat Mar 12 03:34:15 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 12 Mar 2011 12:34:15 +0300 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7AE3FD.1040206@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant><4D7ADC56.1040806@colbyconsulting.com> <4D7AE3FD.1040206@colbyconsulting.com> Message-ID: <8458BA03716A4820980078DE24EC3D06@nant> Hi John -- You have to add references to some of System.Windows.XYZ: - System.Windows.Forms - ... Just compare references of your source project with a classlib project created from it - and add missing references... <<< I "just did it"! ;) >>> But you did it, didn't you? ;) Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 12 ????? 2011 ?. 6:10 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# - Creating and using a shared module I found a "make project from existing files. When I did that it offered to use a selected directory so I dragged the project directory into a new directory and told it to use that. I selected a class library type of project and it went to work "converting" my existing project into a class library. 112 compile errors, most having to do with the fact that it no longer understands using System.Windows.XYZ Many of the common classes did things like accept a combo loaded data into it in a standard way. Stuff like that. So there we are. I "just did it"! ;) John W. Colby www.ColbyConsulting.com On 3/11/2011 9:37 PM, jwcolby wrote: > You make it sound so easy, but it is an existing project already which > was not created as a class library project. I need to create the class > library project then somehow get the directories and files into that project and get it to compile. > > At that point I can reference it by other solutions. > > John W. Colby > www.ColbyConsulting.com > > On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Any suggestions, tips and tricks, warnings, how to etc.? >>>>> >> Just do it! :) >> >> I mean if you managed to handle MS Access library databases then >> handling .NET class libraries would be a "breeze" work for you. >> >> Create class library project - you can keep it within its own >> solution having main (console) project of that solution as a (unit) test one. >> Put all three solutions - two application ones and class library one >> in subfolders of the same root folder. (You can do it differently but >> this is how it's done usually AFAIK) Add existing class library >> project to application solutions. >> Reference class library projects from application projects. >> if you use VS2008 and you have a good PC with plenty of memory (and >> you have one don't you?) then you can keep all three solutions open >> in three instances of VS2008. >> Just keep tack from which solution you have made class library code >> changes last time - when you'll switch to the other solution and if >> you have the same source code file opened in it you'll get a message >> like that "Source file changed - would you like to reload it?" - reply yes... >> >> ... >> >> Use SVN or Mercurial for source code control. Do commit source code >> changes to the source code repository from time to time... >> ... >> >> I have 58 projects in one of the solutions here - that results in >> more than >> 5 apps and quite some class libs compiled and built - AFAIK folks do >> have more than hundred projects within solutions sometimes... >> >> >> Thank you. >> >> -- >> Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Mar 12 06:27:48 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 12 Mar 2011 07:27:48 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <8458BA03716A4820980078DE24EC3D06@nant> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant><4D7ADC56.1040806@colbyconsulting.com> <4D7AE3FD.1040206@colbyconsulting.com> <8458BA03716A4820980078DE24EC3D06@nant> Message-ID: <4D7B66C4.6040404@colbyconsulting.com> Well... the whole thing compiled (the project) when it was part of the bigger solution. I'll give that a try. John W. Colby www.ColbyConsulting.com On 3/12/2011 4:34 AM, Shamil Salakhetdinov wrote: > Hi John -- > > You have to add references to some of System.Windows.XYZ: > > - System.Windows.Forms > - ... > > Just compare references of your source project with a classlib project > created from it - and add missing references... > > <<< > I "just did it"! ;) >>>> > But you did it, didn't you? ;) > > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 12 ????? 2011 ?. 6:10 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# - Creating and using a shared module > > I found a "make project from existing files. When I did that it offered to > use a selected directory so I dragged the project directory into a new > directory and told it to use that. I selected a class library type of > project and it went to work "converting" my existing project into a class > library. > > 112 compile errors, most having to do with the fact that it no longer > understands > > using System.Windows.XYZ > > Many of the common classes did things like accept a combo loaded data into > it in a standard way. > Stuff like that. > > So there we are. > > I "just did it"! ;) > > John W. Colby > www.ColbyConsulting.com > > On 3/11/2011 9:37 PM, jwcolby wrote: >> You make it sound so easy, but it is an existing project already which >> was not created as a class library project. I need to create the class >> library project then somehow get the directories and files into that > project and get it to compile. >> >> At that point I can reference it by other solutions. >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >>> Hi John -- >>> >>> <<< >>> Any suggestions, tips and tricks, warnings, how to etc.? >>>>>> >>> Just do it! :) >>> >>> I mean if you managed to handle MS Access library databases then >>> handling .NET class libraries would be a "breeze" work for you. >>> >>> Create class library project - you can keep it within its own >>> solution having main (console) project of that solution as a (unit) test > one. >>> Put all three solutions - two application ones and class library one >>> in subfolders of the same root folder. (You can do it differently but >>> this is how it's done usually AFAIK) Add existing class library >>> project to application solutions. >>> Reference class library projects from application projects. >>> if you use VS2008 and you have a good PC with plenty of memory (and >>> you have one don't you?) then you can keep all three solutions open >>> in three instances of VS2008. >>> Just keep tack from which solution you have made class library code >>> changes last time - when you'll switch to the other solution and if >>> you have the same source code file opened in it you'll get a message >>> like that "Source file changed - would you like to reload it?" - reply > yes... >>> >>> ... >>> >>> Use SVN or Mercurial for source code control. Do commit source code >>> changes to the source code repository from time to time... >>> ... >>> >>> I have 58 projects in one of the solutions here - that results in >>> more than >>> 5 apps and quite some class libs compiled and built - AFAIK folks do >>> have more than hundred projects within solutions sometimes... >>> >>> >>> Thank you. >>> >>> -- >>> Shamil > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 12 15:00:17 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 12 Mar 2011 16:00:17 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7B66C4.6040404@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant><4D7ADC56.1040806@colbyconsulting.com> <4D7AE3FD.1040206@colbyconsulting.com> <8458BA03716A4820980078DE24EC3D06@nant> <4D7B66C4.6040404@colbyconsulting.com> Message-ID: <4D7BDEE1.40400@colbyconsulting.com> OK, I got all of the references fixed up and got it checked into VSN. John W. Colby www.ColbyConsulting.com On 3/12/2011 7:27 AM, jwcolby wrote: > Well... the whole thing compiled (the project) when it was part of the bigger solution. > > I'll give that a try. > > John W. Colby > www.ColbyConsulting.com > > On 3/12/2011 4:34 AM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> You have to add references to some of System.Windows.XYZ: >> >> - System.Windows.Forms >> - ... >> >> Just compare references of your source project with a classlib project >> created from it - and add missing references... >> >> <<< >> I "just did it"! ;) >>>>> >> But you did it, didn't you? ;) >> >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 12 ????? 2011 ?. 6:10 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# - Creating and using a shared module >> >> I found a "make project from existing files. When I did that it offered to >> use a selected directory so I dragged the project directory into a new >> directory and told it to use that. I selected a class library type of >> project and it went to work "converting" my existing project into a class >> library. >> >> 112 compile errors, most having to do with the fact that it no longer >> understands >> >> using System.Windows.XYZ >> >> Many of the common classes did things like accept a combo loaded data into >> it in a standard way. >> Stuff like that. >> >> So there we are. >> >> I "just did it"! ;) >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/11/2011 9:37 PM, jwcolby wrote: >>> You make it sound so easy, but it is an existing project already which >>> was not created as a class library project. I need to create the class >>> library project then somehow get the directories and files into that >> project and get it to compile. >>> >>> At that point I can reference it by other solutions. >>> >>> John W. Colby >>> www.ColbyConsulting.com >>> >>> On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >>>> Hi John -- >>>> >>>> <<< >>>> Any suggestions, tips and tricks, warnings, how to etc.? >>>>>>> >>>> Just do it! :) >>>> >>>> I mean if you managed to handle MS Access library databases then >>>> handling .NET class libraries would be a "breeze" work for you. >>>> >>>> Create class library project - you can keep it within its own >>>> solution having main (console) project of that solution as a (unit) test >> one. >>>> Put all three solutions - two application ones and class library one >>>> in subfolders of the same root folder. (You can do it differently but >>>> this is how it's done usually AFAIK) Add existing class library >>>> project to application solutions. >>>> Reference class library projects from application projects. >>>> if you use VS2008 and you have a good PC with plenty of memory (and >>>> you have one don't you?) then you can keep all three solutions open >>>> in three instances of VS2008. >>>> Just keep tack from which solution you have made class library code >>>> changes last time - when you'll switch to the other solution and if >>>> you have the same source code file opened in it you'll get a message >>>> like that "Source file changed - would you like to reload it?" - reply >> yes... >>>> >>>> ... >>>> >>>> Use SVN or Mercurial for source code control. Do commit source code >>>> changes to the source code repository from time to time... >>>> ... >>>> >>>> I have 58 projects in one of the solutions here - that results in >>>> more than >>>> 5 apps and quite some class libs compiled and built - AFAIK folks do >>>> have more than hundred projects within solutions sometimes... >>>> >>>> >>>> Thank you. >>>> >>>> -- >>>> Shamil >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From bheid at sc.rr.com Sun Mar 13 09:26:07 2011 From: bheid at sc.rr.com (Bobby Heid) Date: Sun, 13 Mar 2011 10:26:07 -0400 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: Message-ID: <005d01cbe18a$977c6700$c6753500$@rr.com> Only took me about 40 minutes on my machine (Win 7 Ultimate, VS22010 Ultimate, and SQL 2008 Standard). Bobby -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Friday, March 11, 2011 12:59 PM To: dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-47 5d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sun Mar 13 21:58:15 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 13 Mar 2011 22:58:15 -0400 Subject: [dba-VB] Access runtime (an maybe full install?) Message-ID: <4D7D8447.4000003@colbyconsulting.com> I am trying to get an Access 2007 runtime to run reliably anywhere I install it. I had a Windows 2003 virtual machine hanging around so I cleaned Office off of it, and I installed the runtime. I got the directory path declared a safe location and voila, the application ran. Full of confidence I installed Hamachi, the runtime package and the app on a friend's (rather old) XP and it doesn't run. I dug out my old dev machine which it appears that I wiped and installed XP on some time ago. I installed Hamachi, got it on the VPN for this database etc. I then installed the runtime there and tried to run. No go. Of course the runtime doesn't report errors or at least reliably. So I built a little test version with just a single table and an autoform which automatically opened when the app opened. It actually reported that it couldn't talk to the odbc native sql driver (not the exact words of course). So I started Googling and soon discovered that this driver is installed by SQL Server as it installs. However an installer called sqlncli.msi is out there for download. I downloaded that and tried to run. No joy. I turned the machine off, and ran a network cable so it is directly on my network and voila, it runs. I then disabled the cable so that it is connecting to my network via my wireless and it runs. So it appears that the SQL driver is not installed as part of XP, which makes sense since this is SQL Server 2008 Express version I am connecting to. However I would have expected this odbc SQL driver to be part of the Access 2007 runtime, but it seems not. It does appear that a download / install / reboot of the machine will allow me to hook up to the sql server instance. So I am now running Hamachi, joined to this specific network, with the sqlncli.msi installer installing the ODBC SQL Server native driver, plus the Access 2007 runtime, running the full on app, hitting the sql server over the vpn on 5.203.167.79 (the Hamachi IP for the server). And it works on this machine of mine. Now I have to install the SQL Server drivers on the friend's old machine (also running XP but on 512 megs of ram) and ensure that will run. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sun Mar 13 22:27:45 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 13 Mar 2011 23:27:45 -0400 Subject: [dba-VB] [AccessD] Access runtime (an maybe full install?) In-Reply-To: <4D7D8447.4000003@colbyconsulting.com> References: <4D7D8447.4000003@colbyconsulting.com> Message-ID: <4D7D8B31.4080405@colbyconsulting.com> And it runs on the friend's old machine as well. It feels like I might be working now. I have to say this has been one of the most complex jobs I have taken on, with Hamachi VPN networks, a virtual machine running the sql server database, and then the client machine running Hamachi and an Access 2007 runtime. Safe zones, sql server drivers not installed. It has been a challenge getting this running. John W. Colby www.ColbyConsulting.com On 3/13/2011 10:58 PM, jwcolby wrote: > I am trying to get an Access 2007 runtime to run reliably anywhere I install it. I had a Windows > 2003 virtual machine hanging around so I cleaned Office off of it, and I installed the runtime. I > got the directory path declared a safe location and voila, the application ran. > > Full of confidence I installed Hamachi, the runtime package and the app on a friend's (rather old) > XP and it doesn't run. I dug out my old dev machine which it appears that I wiped and installed XP > on some time ago. I installed Hamachi, got it on the VPN for this database etc. I then installed the > runtime there and tried to run. No go. > > Of course the runtime doesn't report errors or at least reliably. So I built a little test version > with just a single table and an autoform which automatically opened when the app opened. It actually > reported that it couldn't talk to the odbc native sql driver (not the exact words of course). So I > started Googling and soon discovered that this driver is installed by SQL Server as it installs. > However an installer called sqlncli.msi is out there for download. > > I downloaded that and tried to run. No joy. I turned the machine off, and ran a network cable so it > is directly on my network and voila, it runs. I then disabled the cable so that it is connecting to > my network via my wireless and it runs. > > So it appears that the SQL driver is not installed as part of XP, which makes sense since this is > SQL Server 2008 Express version I am connecting to. However I would have expected this odbc SQL > driver to be part of the Access 2007 runtime, but it seems not. > > It does appear that a download / install / reboot of the machine will allow me to hook up to the sql > server instance. > > So I am now running Hamachi, joined to this specific network, with the sqlncli.msi installer > installing the ODBC SQL Server native driver, plus the Access 2007 runtime, running the full on app, > hitting the sql server over the vpn on 5.203.167.79 (the Hamachi IP for the server). > > And it works on this machine of mine. > > Now I have to install the SQL Server drivers on the friend's old machine (also running XP but on 512 > megs of ram) and ensure that will run. > > From Gustav at cactus.dk Mon Mar 14 05:38:03 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 14 Mar 2011 11:38:03 +0100 Subject: [dba-VB] Visual Studio 2010 SP1 Message-ID: Hi all OK, don't blame the child. At one time it even asked for the VS2010 dvd - to roll back the installation. Another install was in progress, or at least I hadn't rebooted the machine. So I did and ran the SP1 again. This time - as everything was downloaded - it took about an hour to finish. /gustav >>> gustav at cactus.dk 11-03-2011 18:59 >>> Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-475d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav From jwcolby at colbyconsulting.com Mon Mar 14 13:16:52 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 14:16:52 -0400 Subject: [dba-VB] C# combo boxes Message-ID: <4D7E5B94.2020805@colbyconsulting.com> I am trying to set up a combo box bound to a field in the form such that the combo populates with the PKID and a text field, and when the form opens, selecting something from the combo drops the PKID into the field in the form. I am not getting what to set the properties of the combo box to. I am pulling the ID and the string to display into the combo but I cannot cause the combo to store the id and release control to the next control on the form. -- John W. Colby www.ColbyConsulting.com From Gustav at cactus.dk Mon Mar 14 14:02:27 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 14 Mar 2011 20:02:27 +0100 Subject: [dba-VB] C# combo boxes Message-ID: Hi John If you (typically) have this in InitializeComponent() of the .Designer.cs file: // comboBoxCountry // this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); this.comboBoxCountry.DisplayMember = "Id"; this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; this.comboBoxCountry.FormattingEnabled = true; this.comboBoxCountry.ItemHeight = 13; this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); this.comboBoxCountry.Name = "comboBoxCountry"; this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); this.comboBoxCountry.Sorted = true; this.comboBoxCountry.TabIndex = 7; this.comboBoxCountry.ValueMember = "Id"; You can do something like this in the Load event: this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); Then: private void FillComboBoxCountryCode() { // Order the country names of countryDataTable. string filterExpression = ""; string sortExpression = "CountryNameUK asc"; DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); this.comboBoxCountry.DataSource = countryList; this.comboBoxCountry.ValueMember = "Id"; this.comboBoxCountry.DisplayMember = "CountryNameUK"; } And: private void ChangeCountrySelection() { if (this.comboBoxCountry.SelectedValue != null) { _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); this.comboBoxPostalCode.Text = String.Empty; // Read postal codes of selected country. FillComboBoxPostalCode(); } } This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. /gustav >>> jwcolby at colbyconsulting.com 14-03-2011 19:16 >>> I am trying to set up a combo box bound to a field in the form such that the combo populates with the PKID and a text field, and when the form opens, selecting something from the combo drops the PKID into the field in the form. I am not getting what to set the properties of the combo box to. I am pulling the ID and the string to display into the combo but I cannot cause the combo to store the id and release control to the next control on the form. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Mon Mar 14 16:36:55 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 17:36:55 -0400 Subject: [dba-VB] C# combo boxes In-Reply-To: References: Message-ID: <4D7E8A77.7070009@colbyconsulting.com> Thanks Gustav, I don't know why you are taking me into code in code. I have data connections. This particular data connection has a single view that allows me to write into a table. It has a view that filters a combo. I drag and drop the view for the table onto the form. I have told that table object that specific fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am trying to set up that combo to 1) Display a text value 2) Contain the PKID from the table the combo draws from 3) When I select an item, place the selected ID into the field the combo is bound to. It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or going into code). Actually I have a kinda sorta related question. When I look at server explorer I see a data connection object, however there are two objects underneath, neither of which are for this solution. Furthermore the objects under the data connection are things found on an entirely unrelated server, for other projects that I did work on in the past. The question then is why is the server explorer choosing to display objects out on a virtual machine instead of the database that was originally selected for this solution? We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound form stuff. I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values in most cases) the developer, the product, the type of work, then a couple of fields for date time values for start / stop work. The table the data is going into has a few thousand records already in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound forms is a natural for this. A long time ago I made this (kind of thing) work. I remember that getting the properties of the combo set correctly was non-intuitive but once I figured it out it just worked. John W. Colby www.ColbyConsulting.com On 3/14/2011 3:02 PM, Gustav Brock wrote: > Hi John > > If you (typically) have this in InitializeComponent() of the .Designer.cs file: > > // comboBoxCountry > // > this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; > this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; > this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; > this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); > this.comboBoxCountry.DisplayMember = "Id"; > this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; > this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); > this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; > this.comboBoxCountry.FormattingEnabled = true; > this.comboBoxCountry.ItemHeight = 13; > this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); > this.comboBoxCountry.Name = "comboBoxCountry"; > this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); > this.comboBoxCountry.Sorted = true; > this.comboBoxCountry.TabIndex = 7; > this.comboBoxCountry.ValueMember = "Id"; > > You can do something like this in the Load event: > > this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); > > Then: > private void FillComboBoxCountryCode() > { > // Order the country names of countryDataTable. > string filterExpression = ""; > string sortExpression = "CountryNameUK asc"; > DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); > > this.comboBoxCountry.DataSource = countryList; > this.comboBoxCountry.ValueMember = "Id"; > this.comboBoxCountry.DisplayMember = "CountryNameUK"; > } > > And: > private void ChangeCountrySelection() > { > if (this.comboBoxCountry.SelectedValue != null) > { > _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); > this.comboBoxPostalCode.Text = String.Empty; > // Read postal codes of selected country. > FillComboBoxPostalCode(); > } > } > > This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> > I am trying to set up a combo box bound to a field in the form such that the combo populates with > the PKID and a text field, and when the form opens, selecting something from the combo drops the > PKID into the field in the form. > > I am not getting what to set the properties of the combo box to. I am pulling the ID and the string > to display into the combo but I cannot cause the combo to store the id and release control to the > next control on the form. From gustav at cactus.dk Mon Mar 14 17:15:35 2011 From: gustav at cactus.dk (Gustav Brock) Date: Mon, 14 Mar 2011 23:15:35 +0100 Subject: [dba-VB] C# combo boxes Message-ID: Hi John Eh, are you commenting or? Not quite clear to me. I thought you were looking for some code samples ... I have connections too but I use them only to feed the datatableadapters. I refrain from working with SqlCommand and the like; waste of time in my opinion. /gustav >>> jwcolby at colbyconsulting.com 14-03-2011 22:36 >>> Thanks Gustav, I don't know why you are taking me into code in code. I have data connections. This particular data connection has a single view that allows me to write into a table. It has a view that filters a combo. I drag and drop the view for the table onto the form. I have told that table object that specific fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am trying to set up that combo to 1) Display a text value 2) Contain the PKID from the table the combo draws from 3) When I select an item, place the selected ID into the field the combo is bound to. It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or going into code). Actually I have a kinda sorta related question. When I look at server explorer I see a data connection object, however there are two objects underneath, neither of which are for this solution. Furthermore the objects under the data connection are things found on an entirely unrelated server, for other projects that I did work on in the past. The question then is why is the server explorer choosing to display objects out on a virtual machine instead of the database that was originally selected for this solution? We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound form stuff. I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values in most cases) the developer, the product, the type of work, then a couple of fields for date time values for start / stop work. The table the data is going into has a few thousand records already in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound forms is a natural for this. A long time ago I made this (kind of thing) work. I remember that getting the properties of the combo set correctly was non-intuitive but once I figured it out it just worked. John W. Colby www.ColbyConsulting.com On 3/14/2011 3:02 PM, Gustav Brock wrote: > Hi John > > If you (typically) have this in InitializeComponent() of the .Designer.cs file: > > // comboBoxCountry > // > this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; > this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; > this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; > this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); > this.comboBoxCountry.DisplayMember = "Id"; > this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; > this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); > this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; > this.comboBoxCountry.FormattingEnabled = true; > this.comboBoxCountry.ItemHeight = 13; > this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); > this.comboBoxCountry.Name = "comboBoxCountry"; > this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); > this.comboBoxCountry.Sorted = true; > this.comboBoxCountry.TabIndex = 7; > this.comboBoxCountry.ValueMember = "Id"; > > You can do something like this in the Load event: > > this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); > > Then: > private void FillComboBoxCountryCode() > { > // Order the country names of countryDataTable. > string filterExpression = ""; > string sortExpression = "CountryNameUK asc"; > DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); > > this.comboBoxCountry.DataSource = countryList; > this.comboBoxCountry.ValueMember = "Id"; > this.comboBoxCountry.DisplayMember = "CountryNameUK"; > } > > And: > private void ChangeCountrySelection() > { > if (this.comboBoxCountry.SelectedValue != null) > { > _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); > this.comboBoxPostalCode.Text = String.Empty; > // Read postal codes of selected country. > FillComboBoxPostalCode(); > } > } > > This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> > I am trying to set up a combo box bound to a field in the form such that the combo populates with > the PKID and a text field, and when the form opens, selecting something from the combo drops the > PKID into the field in the form. > > I am not getting what to set the properties of the combo box to. I am pulling the ID and the string > to display into the combo but I cannot cause the combo to store the id and release control to the > next control on the form. From jwcolby at colbyconsulting.com Mon Mar 14 17:25:08 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 18:25:08 -0400 Subject: [dba-VB] C# combo boxes In-Reply-To: References: Message-ID: <4D7E95C4.9020203@colbyconsulting.com> I am trying (unsuccessfully) to do an "Access like bound form". And I always comment. ;) ATM the form displays the values if dropped down, but if not dropped down it just displays an integer (the PKID of the data in the combo is my guess, or perhaps the PKID in the field in the form. So... How do I cause the combo to display Paul's name instead of 6. John W. Colby www.ColbyConsulting.com On 3/14/2011 6:15 PM, Gustav Brock wrote: > Hi John > > Eh, are you commenting or? Not quite clear to me. I thought you were looking for some code samples ... > > I have connections too but I use them only to feed the datatableadapters. I refrain from working with SqlCommand and the like; waste of time in my opinion. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> > Thanks Gustav, > I don't know why you are taking me into code in code. > > I have data connections. This particular data connection has a single view that allows me to write > into a table. It has a view that filters a combo. > > I drag and drop the view for the table onto the form. I have told that table object that specific > fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am > trying to set up that combo to > > 1) Display a text value > 2) Contain the PKID from the table the combo draws from > 3) When I select an item, place the selected ID into the field the combo is bound to. > > It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or > going into code). > > Actually I have a kinda sorta related question. > > When I look at server explorer I see a data connection object, however there are two objects > underneath, neither of which are for this solution. Furthermore the objects under the data > connection are things found on an entirely unrelated server, for other projects that I did work on > in the past. > > The question then is why is the server explorer choosing to display objects out on a virtual machine > instead of the database that was originally selected for this solution? > > We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound > form stuff. > > I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values > in most cases) the developer, the product, the type of work, then a couple of fields for date time > values for start / stop work. The table the data is going into has a few thousand records already > in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound > forms is a natural for this. > > A long time ago I made this (kind of thing) work. I remember that getting the properties of the > combo set correctly was non-intuitive but once I figured it out it just worked. > > > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 3:02 PM, Gustav Brock wrote: >> Hi John >> >> If you (typically) have this in InitializeComponent() of the .Designer.cs file: >> >> // comboBoxCountry >> // >> this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; >> this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; >> this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; >> this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); >> this.comboBoxCountry.DisplayMember = "Id"; >> this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; >> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); >> this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; >> this.comboBoxCountry.FormattingEnabled = true; >> this.comboBoxCountry.ItemHeight = 13; >> this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); >> this.comboBoxCountry.Name = "comboBoxCountry"; >> this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); >> this.comboBoxCountry.Sorted = true; >> this.comboBoxCountry.TabIndex = 7; >> this.comboBoxCountry.ValueMember = "Id"; >> >> You can do something like this in the Load event: >> >> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); >> >> Then: >> private void FillComboBoxCountryCode() >> { >> // Order the country names of countryDataTable. >> string filterExpression = ""; >> string sortExpression = "CountryNameUK asc"; >> DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); >> >> this.comboBoxCountry.DataSource = countryList; >> this.comboBoxCountry.ValueMember = "Id"; >> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >> } >> >> And: >> private void ChangeCountrySelection() >> { >> if (this.comboBoxCountry.SelectedValue != null) >> { >> _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); >> this.comboBoxPostalCode.Text = String.Empty; >> // Read postal codes of selected country. >> FillComboBoxPostalCode(); >> } >> } >> >> This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >> I am trying to set up a combo box bound to a field in the form such that the combo populates with >> the PKID and a text field, and when the form opens, selecting something from the combo drops the >> PKID into the field in the form. >> >> I am not getting what to set the properties of the combo box to. I am pulling the ID and the string >> to display into the combo but I cannot cause the combo to store the id and release control to the >> next control on the form. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From accessd at shaw.ca Mon Mar 14 18:59:26 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Mon, 14 Mar 2011 16:59:26 -0700 Subject: [dba-VB] C# combo boxes In-Reply-To: <4D7E95C4.9020203@colbyconsulting.com> References: <4D7E95C4.9020203@colbyconsulting.com> Message-ID: Hi John: Try and use ADO. It is on every Windows based computer already and requires no additional ODBC to be setup. In other words, you do not even have to see the computers on which the application is going to have a sable connection. Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, March 14, 2011 3:25 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# combo boxes I am trying (unsuccessfully) to do an "Access like bound form". And I always comment. ;) ATM the form displays the values if dropped down, but if not dropped down it just displays an integer (the PKID of the data in the combo is my guess, or perhaps the PKID in the field in the form. So... How do I cause the combo to display Paul's name instead of 6. John W. Colby www.ColbyConsulting.com On 3/14/2011 6:15 PM, Gustav Brock wrote: > Hi John > > Eh, are you commenting or? Not quite clear to me. I thought you were looking for some code samples ... > > I have connections too but I use them only to feed the datatableadapters. I refrain from working with SqlCommand and the like; waste of time in my opinion. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> > Thanks Gustav, > I don't know why you are taking me into code in code. > > I have data connections. This particular data connection has a single view that allows me to write > into a table. It has a view that filters a combo. > > I drag and drop the view for the table onto the form. I have told that table object that specific > fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am > trying to set up that combo to > > 1) Display a text value > 2) Contain the PKID from the table the combo draws from > 3) When I select an item, place the selected ID into the field the combo is bound to. > > It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or > going into code). > > Actually I have a kinda sorta related question. > > When I look at server explorer I see a data connection object, however there are two objects > underneath, neither of which are for this solution. Furthermore the objects under the data > connection are things found on an entirely unrelated server, for other projects that I did work on > in the past. > > The question then is why is the server explorer choosing to display objects out on a virtual machine > instead of the database that was originally selected for this solution? > > We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound > form stuff. > > I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values > in most cases) the developer, the product, the type of work, then a couple of fields for date time > values for start / stop work. The table the data is going into has a few thousand records already > in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound > forms is a natural for this. > > A long time ago I made this (kind of thing) work. I remember that getting the properties of the > combo set correctly was non-intuitive but once I figured it out it just worked. > > > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 3:02 PM, Gustav Brock wrote: >> Hi John >> >> If you (typically) have this in InitializeComponent() of the .Designer.cs file: >> >> // comboBoxCountry >> // >> this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; >> this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; >> this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; >> this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); >> this.comboBoxCountry.DisplayMember = "Id"; >> this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; >> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); >> this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; >> this.comboBoxCountry.FormattingEnabled = true; >> this.comboBoxCountry.ItemHeight = 13; >> this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); >> this.comboBoxCountry.Name = "comboBoxCountry"; >> this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); >> this.comboBoxCountry.Sorted = true; >> this.comboBoxCountry.TabIndex = 7; >> this.comboBoxCountry.ValueMember = "Id"; >> >> You can do something like this in the Load event: >> >> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); >> >> Then: >> private void FillComboBoxCountryCode() >> { >> // Order the country names of countryDataTable. >> string filterExpression = ""; >> string sortExpression = "CountryNameUK asc"; >> DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); >> >> this.comboBoxCountry.DataSource = countryList; >> this.comboBoxCountry.ValueMember = "Id"; >> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >> } >> >> And: >> private void ChangeCountrySelection() >> { >> if (this.comboBoxCountry.SelectedValue != null) >> { >> _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); >> this.comboBoxPostalCode.Text = String.Empty; >> // Read postal codes of selected country. >> FillComboBoxPostalCode(); >> } >> } >> >> This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >> I am trying to set up a combo box bound to a field in the form such that the combo populates with >> the PKID and a text field, and when the form opens, selecting something from the combo drops the >> PKID into the field in the form. >> >> I am not getting what to set the properties of the combo box to. I am pulling the ID and the string >> to display into the combo but I cannot cause the combo to store the id and release control to the >> next control on the form. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Mon Mar 14 21:13:25 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 22:13:25 -0400 Subject: [dba-VB] C# combo boxes In-Reply-To: References: <4D7E95C4.9020203@colbyconsulting.com> Message-ID: <4D7ECB45.8060609@colbyconsulting.com> I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com On 3/14/2011 7:59 PM, Jim Lawrence wrote: > Hi John: > > Try and use ADO. It is on every Windows based computer already and requires > no additional ODBC to be setup. In other words, you do not even have to see > the computers on which the application is going to have a sable connection. > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Monday, March 14, 2011 3:25 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] C# combo boxes > > I am trying (unsuccessfully) to do an "Access like bound form". > > And I always comment. ;) > > ATM the form displays the values if dropped down, but if not dropped down it > just displays an > integer (the PKID of the data in the combo is my guess, or perhaps the PKID > in the field in the form. > > So... > > How do I cause the combo to display Paul's name instead of 6. > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 6:15 PM, Gustav Brock wrote: >> Hi John >> >> Eh, are you commenting or? Not quite clear to me. I thought you were > looking for some code samples ... >> >> I have connections too but I use them only to feed the datatableadapters. > I refrain from working with SqlCommand and the like; waste of time in my > opinion. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> >> Thanks Gustav, >> I don't know why you are taking me into code in code. >> >> I have data connections. This particular data connection has a single > view that allows me to write >> into a table. It has a view that filters a combo. >> >> I drag and drop the view for the table onto the form. I have told that > table object that specific >> fields are combos so when I drag it out, I have combos (for the FKs - PKs > from other tables). I am >> trying to set up that combo to >> >> 1) Display a text value >> 2) Contain the PKID from the table the combo draws from >> 3) When I select an item, place the selected ID into the field the combo > is bound to. >> >> It seems this is a normal daily requirement in building forms, and > shouldn't be rocket science (or >> going into code). >> >> Actually I have a kinda sorta related question. >> >> When I look at server explorer I see a data connection object, however > there are two objects >> underneath, neither of which are for this solution. Furthermore the > objects under the data >> connection are things found on an entirely unrelated server, for other > projects that I did work on >> in the past. >> >> The question then is why is the server explorer choosing to display > objects out on a virtual machine >> instead of the database that was originally selected for this solution? >> >> We have done a ton of coding sql server stuff here but we have not yet > learned how to do this bound >> form stuff. >> >> I am trying to build a timeclock kind of thing. Very simple, a few combos > to select (default values >> in most cases) the developer, the product, the type of work, then a couple > of fields for date time >> values for start / stop work. The table the data is going into has a few > thousand records already >> in it, the combos pull from tables with 20-30 records etc. It seems (to > me) that just using bound >> forms is a natural for this. >> >> A long time ago I made this (kind of thing) work. I remember that getting > the properties of the >> combo set correctly was non-intuitive but once I figured it out it just > worked. >> >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/14/2011 3:02 PM, Gustav Brock wrote: >>> Hi John >>> >>> If you (typically) have this in InitializeComponent() of the .Designer.cs > file: >>> >>> // comboBoxCountry >>> // >>> this.comboBoxCountry.AutoCompleteMode = > System.Windows.Forms.AutoCompleteMode.Suggest; >>> this.comboBoxCountry.AutoCompleteSource = > System.Windows.Forms.AutoCompleteSource.ListItems; >>> this.comboBoxCountry.BackColor = > System.Drawing.Color.LavenderBlush; >>> this.comboBoxCountry.DataBindings.Add(new > System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, > "CountryId", true)); >>> this.comboBoxCountry.DisplayMember = "Id"; >>> this.comboBoxCountry.FlatStyle = > System.Windows.Forms.FlatStyle.Flat; >>> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe > UI", 8F, System.Drawing.FontStyle.Regular, > System.Drawing.GraphicsUnit.Point, ((byte)(0))); >>> this.comboBoxCountry.ForeColor = > System.Drawing.Color.DarkMagenta; >>> this.comboBoxCountry.FormattingEnabled = true; >>> this.comboBoxCountry.ItemHeight = 13; >>> this.comboBoxCountry.Location = new > System.Drawing.Point(21, 136); >>> this.comboBoxCountry.Name = "comboBoxCountry"; >>> this.comboBoxCountry.Size = new System.Drawing.Size(117, > 21); >>> this.comboBoxCountry.Sorted = true; >>> this.comboBoxCountry.TabIndex = 7; >>> this.comboBoxCountry.ValueMember = "Id"; >>> >>> You can do something like this in the Load event: >>> >>> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => > ChangeCountrySelection(); >>> >>> Then: >>> private void FillComboBoxCountryCode() >>> { >>> // Order the country names of countryDataTable. >>> string filterExpression = ""; >>> string sortExpression = "CountryNameUK asc"; >>> DataRow[] countryList = > _countryDataTable.Select(filterExpression, sortExpression); >>> >>> this.comboBoxCountry.DataSource = countryList; >>> this.comboBoxCountry.ValueMember = "Id"; >>> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >>> } >>> >>> And: >>> private void ChangeCountrySelection() >>> { >>> if (this.comboBoxCountry.SelectedValue != null) >>> { >>> _countryId = > Convert.ToInt32(this.comboBoxCountry.SelectedValue); >>> this.comboBoxPostalCode.Text = String.Empty; >>> // Read postal codes of selected country. >>> FillComboBoxPostalCode(); >>> } >>> } >>> >>> This is pretty standard using tableadapters and bindingnavigator. Nothing > fancy. >>> >>> /gustav >>> >>> >>>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >>> I am trying to set up a combo box bound to a field in the form such that > the combo populates with >>> the PKID and a text field, and when the form opens, selecting something > from the combo drops the >>> PKID into the field in the form. >>> >>> I am not getting what to set the properties of the combo box to. I am > pulling the ID and the string >>> to display into the combo but I cannot cause the combo to store the id > and release control to the >>> next control on the form. >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From accessd at shaw.ca Tue Mar 15 02:19:11 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Tue, 15 Mar 2011 00:19:11 -0700 Subject: [dba-VB] C# combo boxes In-Reply-To: <4D7ECB45.8060609@colbyconsulting.com> References: <4D7E95C4.9020203@colbyconsulting.com> <4D7ECB45.8060609@colbyconsulting.com> Message-ID: Congratulation John. Keep us posted on it stability. Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, March 14, 2011 7:13 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# combo boxes I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com On 3/14/2011 7:59 PM, Jim Lawrence wrote: > Hi John: > > Try and use ADO. It is on every Windows based computer already and requires > no additional ODBC to be setup. In other words, you do not even have to see > the computers on which the application is going to have a sable connection. > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Monday, March 14, 2011 3:25 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] C# combo boxes > > I am trying (unsuccessfully) to do an "Access like bound form". > > And I always comment. ;) > > ATM the form displays the values if dropped down, but if not dropped down it > just displays an > integer (the PKID of the data in the combo is my guess, or perhaps the PKID > in the field in the form. > > So... > > How do I cause the combo to display Paul's name instead of 6. > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 6:15 PM, Gustav Brock wrote: >> Hi John >> >> Eh, are you commenting or? Not quite clear to me. I thought you were > looking for some code samples ... >> >> I have connections too but I use them only to feed the datatableadapters. > I refrain from working with SqlCommand and the like; waste of time in my > opinion. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> >> Thanks Gustav, >> I don't know why you are taking me into code in code. >> >> I have data connections. This particular data connection has a single > view that allows me to write >> into a table. It has a view that filters a combo. >> >> I drag and drop the view for the table onto the form. I have told that > table object that specific >> fields are combos so when I drag it out, I have combos (for the FKs - PKs > from other tables). I am >> trying to set up that combo to >> >> 1) Display a text value >> 2) Contain the PKID from the table the combo draws from >> 3) When I select an item, place the selected ID into the field the combo > is bound to. >> >> It seems this is a normal daily requirement in building forms, and > shouldn't be rocket science (or >> going into code). >> >> Actually I have a kinda sorta related question. >> >> When I look at server explorer I see a data connection object, however > there are two objects >> underneath, neither of which are for this solution. Furthermore the > objects under the data >> connection are things found on an entirely unrelated server, for other > projects that I did work on >> in the past. >> >> The question then is why is the server explorer choosing to display > objects out on a virtual machine >> instead of the database that was originally selected for this solution? >> >> We have done a ton of coding sql server stuff here but we have not yet > learned how to do this bound >> form stuff. >> >> I am trying to build a timeclock kind of thing. Very simple, a few combos > to select (default values >> in most cases) the developer, the product, the type of work, then a couple > of fields for date time >> values for start / stop work. The table the data is going into has a few > thousand records already >> in it, the combos pull from tables with 20-30 records etc. It seems (to > me) that just using bound >> forms is a natural for this. >> >> A long time ago I made this (kind of thing) work. I remember that getting > the properties of the >> combo set correctly was non-intuitive but once I figured it out it just > worked. >> >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/14/2011 3:02 PM, Gustav Brock wrote: >>> Hi John >>> >>> If you (typically) have this in InitializeComponent() of the .Designer.cs > file: >>> >>> // comboBoxCountry >>> // >>> this.comboBoxCountry.AutoCompleteMode = > System.Windows.Forms.AutoCompleteMode.Suggest; >>> this.comboBoxCountry.AutoCompleteSource = > System.Windows.Forms.AutoCompleteSource.ListItems; >>> this.comboBoxCountry.BackColor = > System.Drawing.Color.LavenderBlush; >>> this.comboBoxCountry.DataBindings.Add(new > System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, > "CountryId", true)); >>> this.comboBoxCountry.DisplayMember = "Id"; >>> this.comboBoxCountry.FlatStyle = > System.Windows.Forms.FlatStyle.Flat; >>> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe > UI", 8F, System.Drawing.FontStyle.Regular, > System.Drawing.GraphicsUnit.Point, ((byte)(0))); >>> this.comboBoxCountry.ForeColor = > System.Drawing.Color.DarkMagenta; >>> this.comboBoxCountry.FormattingEnabled = true; >>> this.comboBoxCountry.ItemHeight = 13; >>> this.comboBoxCountry.Location = new > System.Drawing.Point(21, 136); >>> this.comboBoxCountry.Name = "comboBoxCountry"; >>> this.comboBoxCountry.Size = new System.Drawing.Size(117, > 21); >>> this.comboBoxCountry.Sorted = true; >>> this.comboBoxCountry.TabIndex = 7; >>> this.comboBoxCountry.ValueMember = "Id"; >>> >>> You can do something like this in the Load event: >>> >>> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => > ChangeCountrySelection(); >>> >>> Then: >>> private void FillComboBoxCountryCode() >>> { >>> // Order the country names of countryDataTable. >>> string filterExpression = ""; >>> string sortExpression = "CountryNameUK asc"; >>> DataRow[] countryList = > _countryDataTable.Select(filterExpression, sortExpression); >>> >>> this.comboBoxCountry.DataSource = countryList; >>> this.comboBoxCountry.ValueMember = "Id"; >>> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >>> } >>> >>> And: >>> private void ChangeCountrySelection() >>> { >>> if (this.comboBoxCountry.SelectedValue != null) >>> { >>> _countryId = > Convert.ToInt32(this.comboBoxCountry.SelectedValue); >>> this.comboBoxPostalCode.Text = String.Empty; >>> // Read postal codes of selected country. >>> FillComboBoxPostalCode(); >>> } >>> } >>> >>> This is pretty standard using tableadapters and bindingnavigator. Nothing > fancy. >>> >>> /gustav >>> >>> >>>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >>> I am trying to set up a combo box bound to a field in the form such that > the combo populates with >>> the PKID and a text field, and when the form opens, selecting something > from the combo drops the >>> PKID into the field in the form. >>> >>> I am not getting what to set the properties of the combo box to. I am > pulling the ID and the string >>> to display into the combo but I cannot cause the combo to store the id > and release control to the >>> next control on the form. >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From Gustav at cactus.dk Tue Mar 15 04:55:02 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Tue, 15 Mar 2011 10:55:02 +0100 Subject: [dba-VB] C# combo boxes Message-ID: Hi John Good! Yes, the content of Text is handled by DisplayMember: this.comboBoxCountry.DataSource = countryList; this.comboBoxCountry.ValueMember = "Id"; this.comboBoxCountry.DisplayMember = "CountryNameUK"; /gustav >>> jwcolby at colbyconsulting.com 15-03-2011 03:13 >>> I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com From df.waters at comcast.net Tue Mar 15 08:54:00 2011 From: df.waters at comcast.net (Dan Waters) Date: Tue, 15 Mar 2011 08:54:00 -0500 Subject: [dba-VB] C# combo boxes In-Reply-To: <4D7ECB45.8060609@colbyconsulting.com> References: <4D7E95C4.9020203@colbyconsulting.com> <4D7ECB45.8060609@colbyconsulting.com> Message-ID: <002101cbe318$713acbb0$53b06310$@comcast.net> Hi John, Take a look at www.updatecontrols.net. This has a free download which integrates into visual studio and provides a way to simply databinding. I haven't used it yet but it looks promising. Note: At the top of the screens is a combobox which selects either VB or C# for the code examples. Dan -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, March 14, 2011 9:13 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# combo boxes I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com On 3/14/2011 7:59 PM, Jim Lawrence wrote: > Hi John: > > Try and use ADO. It is on every Windows based computer already and > requires no additional ODBC to be setup. In other words, you do not > even have to see the computers on which the application is going to have a sable connection. > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Monday, March 14, 2011 3:25 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] C# combo boxes > > I am trying (unsuccessfully) to do an "Access like bound form". > > And I always comment. ;) > > ATM the form displays the values if dropped down, but if not dropped > down it just displays an integer (the PKID of the data in the combo is > my guess, or perhaps the PKID in the field in the form. > > So... > > How do I cause the combo to display Paul's name instead of 6. > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 6:15 PM, Gustav Brock wrote: >> Hi John >> >> Eh, are you commenting or? Not quite clear to me. I thought you were > looking for some code samples ... >> >> I have connections too but I use them only to feed the datatableadapters. > I refrain from working with SqlCommand and the like; waste of time in > my opinion. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> >> Thanks Gustav, >> I don't know why you are taking me into code in code. >> >> I have data connections. This particular data connection has a >> single > view that allows me to write >> into a table. It has a view that filters a combo. >> >> I drag and drop the view for the table onto the form. I have told >> that > table object that specific >> fields are combos so when I drag it out, I have combos (for the FKs - >> PKs > from other tables). I am >> trying to set up that combo to >> >> 1) Display a text value >> 2) Contain the PKID from the table the combo draws from >> 3) When I select an item, place the selected ID into the field the >> combo > is bound to. >> >> It seems this is a normal daily requirement in building forms, and > shouldn't be rocket science (or >> going into code). >> >> Actually I have a kinda sorta related question. >> >> When I look at server explorer I see a data connection object, >> however > there are two objects >> underneath, neither of which are for this solution. Furthermore the > objects under the data >> connection are things found on an entirely unrelated server, for >> other > projects that I did work on >> in the past. >> >> The question then is why is the server explorer choosing to display > objects out on a virtual machine >> instead of the database that was originally selected for this solution? >> >> We have done a ton of coding sql server stuff here but we have not >> yet > learned how to do this bound >> form stuff. >> >> I am trying to build a timeclock kind of thing. Very simple, a few >> combos > to select (default values >> in most cases) the developer, the product, the type of work, then a >> couple > of fields for date time >> values for start / stop work. The table the data is going into has a >> few > thousand records already >> in it, the combos pull from tables with 20-30 records etc. It seems >> (to > me) that just using bound >> forms is a natural for this. >> >> A long time ago I made this (kind of thing) work. I remember that >> getting > the properties of the >> combo set correctly was non-intuitive but once I figured it out it >> just > worked. >> >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/14/2011 3:02 PM, Gustav Brock wrote: >>> Hi John >>> >>> If you (typically) have this in InitializeComponent() of the >>> .Designer.cs > file: >>> >>> // comboBoxCountry >>> // >>> this.comboBoxCountry.AutoCompleteMode = > System.Windows.Forms.AutoCompleteMode.Suggest; >>> this.comboBoxCountry.AutoCompleteSource = > System.Windows.Forms.AutoCompleteSource.ListItems; >>> this.comboBoxCountry.BackColor = > System.Drawing.Color.LavenderBlush; >>> this.comboBoxCountry.DataBindings.Add(new > System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, > "CountryId", true)); >>> this.comboBoxCountry.DisplayMember = "Id"; >>> this.comboBoxCountry.FlatStyle = > System.Windows.Forms.FlatStyle.Flat; >>> this.comboBoxCountry.Font = new >>> System.Drawing.Font("Segoe > UI", 8F, System.Drawing.FontStyle.Regular, > System.Drawing.GraphicsUnit.Point, ((byte)(0))); >>> this.comboBoxCountry.ForeColor = > System.Drawing.Color.DarkMagenta; >>> this.comboBoxCountry.FormattingEnabled = true; >>> this.comboBoxCountry.ItemHeight = 13; >>> this.comboBoxCountry.Location = new > System.Drawing.Point(21, 136); >>> this.comboBoxCountry.Name = "comboBoxCountry"; >>> this.comboBoxCountry.Size = new >>> System.Drawing.Size(117, > 21); >>> this.comboBoxCountry.Sorted = true; >>> this.comboBoxCountry.TabIndex = 7; >>> this.comboBoxCountry.ValueMember = "Id"; >>> >>> You can do something like this in the Load event: >>> >>> this.comboBoxCountry.SelectedValueChanged += (oo, ee) >>> => > ChangeCountrySelection(); >>> >>> Then: >>> private void FillComboBoxCountryCode() >>> { >>> // Order the country names of countryDataTable. >>> string filterExpression = ""; >>> string sortExpression = "CountryNameUK asc"; >>> DataRow[] countryList = > _countryDataTable.Select(filterExpression, sortExpression); >>> >>> this.comboBoxCountry.DataSource = countryList; >>> this.comboBoxCountry.ValueMember = "Id"; >>> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >>> } >>> >>> And: >>> private void ChangeCountrySelection() >>> { >>> if (this.comboBoxCountry.SelectedValue != null) >>> { >>> _countryId = > Convert.ToInt32(this.comboBoxCountry.SelectedValue); >>> this.comboBoxPostalCode.Text = String.Empty; >>> // Read postal codes of selected country. >>> FillComboBoxPostalCode(); >>> } >>> } >>> >>> This is pretty standard using tableadapters and bindingnavigator. >>> Nothing > fancy. >>> >>> /gustav >>> >>> >>>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >>> I am trying to set up a combo box bound to a field in the form such >>> that > the combo populates with >>> the PKID and a text field, and when the form opens, selecting >>> something > from the combo drops the >>> PKID into the field in the form. >>> >>> I am not getting what to set the properties of the combo box to. I >>> am > pulling the ID and the string >>> to display into the combo but I cannot cause the combo to store the >>> id > and release control to the >>> next control on the form. >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From dbdoug at gmail.com Wed Mar 16 10:28:18 2011 From: dbdoug at gmail.com (Doug Steele) Date: Wed, 16 Mar 2011 08:28:18 -0700 Subject: [dba-VB] VS2010 install Message-ID: Hello all: I just finished doing a clean install of Windows 7 then VS2010. To my surprise, I also appear to have a functioning install of VS2008, loading with the name 'Shell' where my old VS2008 says 'Professional Edition'. I couldn't see anything on the net about this; does anyone have any further information? Thanks, Doug From DWUTKA at Marlow.com Fri Mar 18 14:12:52 2011 From: DWUTKA at Marlow.com (Drew Wutka) Date: Fri, 18 Mar 2011 14:12:52 -0500 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: Message-ID: 500 megs... that's about 10 minutes... wouldn't have time for a full dinner.... Drew -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-810 7-475d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com The information contained in this transmission is intended only for the person or entity to which it is addressed and may contain II-VI Proprietary and/or II-VI Business Sensitive material. If you are not the intended recipient, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. You are notified that any review, retransmission, copying, disclosure, dissemination, or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. From stuart at lexacorp.com.pg Fri Mar 18 18:50:01 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 09:50:01 +1000 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: , Message-ID: <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg> That's about 10 hours and US$35 for me :-( -- Stuart On 18 Mar 2011 at 14:12, Drew Wutka wrote: > 500 megs... that's about 10 minutes... wouldn't have time for a full > dinner.... > > Drew > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com > Subject: [dba-VB] Visual Studio 2010 SP1 > > Hi all > > If you go for this: > http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8 > 10 7-475d-948a-ef22627e57a5&displaylang=en > > it's typically a 500MB+ download with a streaming install. > > Don't go for coffee during this, don't go for lunch, not even a > single-course dinner, don't settle for less than a full menu dinner > followed by coffee and brandy. Or book an evening in town! > > /gustav > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > The information contained in this transmission is intended only for > the person or entity to which it is addressed and may contain II-VI > Proprietary and/or II-VI Business Sensitive material. If you are not > the intended recipient, please contact the sender immediately and > destroy the material in its entirety, whether electronic or hard copy. > You are notified that any review, retransmission, copying, disclosure, > dissemination, or other use of, or taking of any action in reliance > upon this information by persons or entities other than the intended > recipient is prohibited. > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Mar 18 19:02:30 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 03:02:30 +0300 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg> References: , <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg> Message-ID: Hi Stuart -- <<< That's about 10 hours and US$35 for me :-( >>> But you never have a snow there, and your local police doesn't mind if you drive home your car after a couple of beers? :) I mean 500megs download takes just a couple of minutes here and near to zero expences but we still have a snow here and Finnish Gulf is covered by 1 meter+ wide ice and I'm skiing here almost everyday, and if stopped by local road police after a couple beers I would definitely loose my car driving license for 1.5 years, at least... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart McLachlan Sent: 19 ????? 2011 ?. 2:50 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Visual Studio 2010 SP1 That's about 10 hours and US$35 for me :-( -- Stuart On 18 Mar 2011 at 14:12, Drew Wutka wrote: > 500 megs... that's about 10 minutes... wouldn't have time for a full > dinner.... > > Drew > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com > Subject: [dba-VB] Visual Studio 2010 SP1 > > Hi all > > If you go for this: > http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8 > 10 7-475d-948a-ef22627e57a5&displaylang=en > > it's typically a 500MB+ download with a streaming install. > > Don't go for coffee during this, don't go for lunch, not even a > single-course dinner, don't settle for less than a full menu dinner > followed by coffee and brandy. Or book an evening in town! > > /gustav > > From stuart at lexacorp.com.pg Fri Mar 18 19:09:18 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 10:09:18 +1000 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: , <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg>, Message-ID: <4D83F42E.13556.8558BC@stuart.lexacorp.com.pg> Yep, The good points still outweigh the bad. :-) -- Stuart On 19 Mar 2011 at 3:02, Shamil Salakhetdinov wrote: > Hi Stuart -- > > <<< > That's about 10 hours and US$35 for me :-( > >>> > But you never have a snow there, and your local police doesn't mind if > you drive home your car after a couple of beers? :) > > I mean 500megs download takes just a couple of minutes here and near > to zero expences but we still have a snow here and Finnish Gulf is > covered by 1 meter+ wide ice and I'm skiing here almost everyday, and > if stopped by local road police after a couple beers I would > definitely loose my car driving license for 1.5 years, at least... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart > McLachlan Sent: 19 ????? 2011 ?. 2:50 To: Discussion concerning Visual > Basic and related programming issues. Subject: Re: [dba-VB] Visual > Studio 2010 SP1 > > That's about 10 hours and US$35 for me :-( > > -- > Stuart > > On 18 Mar 2011 at 14:12, Drew Wutka wrote: > > > 500 megs... that's about 10 minutes... wouldn't have time for a full > > dinner.... > > > > Drew > > > > -----Original Message----- > > From: dba-vb-bounces at databaseadvisors.com > > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav > > Brock Sent: Friday, March 11, 2011 11:59 AM To: > > dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 > > > > Hi all > > > > If you go for this: > > http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6 > > -8 10 7-475d-948a-ef22627e57a5&displaylang=en > > > > it's typically a 500MB+ download with a streaming install. > > > > Don't go for coffee during this, don't go for lunch, not even a > > single-course dinner, don't settle for less than a full menu dinner > > followed by coffee and brandy. Or book an evening in town! > > > > /gustav > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Mar 18 21:08:01 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 18 Mar 2011 22:08:01 -0400 Subject: [dba-VB] SHA1 to compute a hash Message-ID: <4D841001.2060200@colbyconsulting.com> In my databases I create SHA1 hashes to enable joining between tables and pull identical records (identical for the fields hashed). I create: 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the three values and feed them into SHa1 and out pops a number which I store in a field in my table. 2) A HashFamily of the Zip5, Zip4, Addr ad LName. 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. I am getting known collisions between different addresses (I have discovered and investigated collisions) in my HashAddr when I have many millions of addresses. I need to address this. Back when I made my design decisions (2004) my hardware consisted of single core processors, 4 gigs ram, Windows x32 etc. Now I have 8 cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great extent constrained by my hardware "back in the day" whereas I am much less so now. I am about to redesign my process. I am considering simply appending in the city and state strings to all of the inputs: Addr, City, St, Zip5, Zip4 as the address base and then the same with LName and FName for the other two respective hashes. The objective is to minimize hash collisions, not prevent some crypto attack. I use these hash fields to join between multi million record tables so If I need to discover info in TableA where the HashAddr is the same as in TableB, I need the probability of a collision between different addresses (family/Person) to be as close to zero as I can get it. My questions are: 1) Whether anyone out there is using a hash in this manner? 2) Has anyone seen a table of collision probability between messages of a given (short) message length. My message is 9 digits for the zip5/4 and the address could be something as short as PO Box 1, or Apt 1. IOW the total message length of 14 is pretty common. Adding the state would give me minimum message lengths of only 16 and City would only add a few more characters. 3) Does anyone know if just adding the same data back in again would decrease the collision probability. IOW Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. Any experience out there? -- John W. Colby www.ColbyConsulting.com From stuart at lexacorp.com.pg Sat Mar 19 05:20:16 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 20:20:16 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D841001.2060200@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com> Message-ID: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> How are you creating your hash? Can you post a few examples of different data strings and colliding SHA1 hashes. I can probably make a lot of money out of them. AFAIK, no one other than you has found any. -- Stuart On 18 Mar 2011 at 22:08, jwcolby wrote: > In my databases I create SHA1 hashes to enable joining between tables > and pull identical records (identical for the fields hashed). I > create: > > 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the > three values and feed them into SHa1 and out pops a number which I > store in a field in my table. > > 2) A HashFamily of the Zip5, Zip4, Addr ad LName. > > 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. > > I am getting known collisions between different addresses (I have > discovered and investigated collisions) in my HashAddr when I have > many millions of addresses. I need to address this. > > Back when I made my design decisions (2004) my hardware consisted of > single core processors, 4 gigs ram, Windows x32 etc. Now I have 8 > cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great extent > constrained by my hardware "back in the day" whereas I am much less so > now. > > I am about to redesign my process. > > I am considering simply appending in the city and state strings to all > of the inputs: Addr, City, St, Zip5, Zip4 as the address base and then > the same with LName and FName for the other two respective hashes. > > The objective is to minimize hash collisions, not prevent some crypto > attack. I use these hash fields to join between multi million record > tables so If I need to discover info in TableA where the HashAddr is > the same as in TableB, I need the probability of a collision between > different addresses (family/Person) to be as close to zero as I can > get it. > > My questions are: > > 1) Whether anyone out there is using a hash in this manner? > 2) Has anyone seen a table of collision probability between messages > of a given (short) message length. My message is 9 digits for the > zip5/4 and the address could be something as short as PO Box 1, or Apt > 1. IOW the total message length of 14 is pretty common. Adding the > state would give me minimum message lengths of only 16 and City would > only add a few more characters. 3) Does anyone know if just adding the > same data back in again would decrease the collision probability. IOW > Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. > > Any experience out there? > > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 06:20:26 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 21:20:26 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> Using your SHA1 function, what message digests do you get for the standard test cases: 1. abc 2. abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq These should return: 1. A9993E36 4706816A BA3E2571 7850C26C 9CD0D89D 2. 84983E44 1C3BD26E BAAE4AA1 F95129E5 E54670F1 See Examples 1 and 2 at http://www.itl.nist.gov/fipspubs/fip180-1.htm -- Stuart On 19 Mar 2011 at 20:20, Stuart McLachlan wrote: > How are you creating your hash? > > Can you post a few examples of different data strings and colliding > SHA1 hashes. I can probably make a lot of money out of them. > AFAIK, no one other than you has found any. > > -- > Stuart > > > On 18 Mar 2011 at 22:08, jwcolby wrote: > > > In my databases I create SHA1 hashes to enable joining between > > tables and pull identical records (identical for the fields hashed). > > I create: > > > > 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the > > three values and feed them into SHa1 and out pops a number which I > > store in a field in my table. > > > > 2) A HashFamily of the Zip5, Zip4, Addr ad LName. > > > > 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. > > > > I am getting known collisions between different addresses (I have > > discovered and investigated collisions) in my HashAddr when I have > > many millions of addresses. I need to address this. > > > > Back when I made my design decisions (2004) my hardware consisted of > > single core processors, 4 gigs ram, Windows x32 etc. Now I have 8 > > cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great extent > > constrained by my hardware "back in the day" whereas I am much less > > so now. > > > > I am about to redesign my process. > > > > I am considering simply appending in the city and state strings to > > all of the inputs: Addr, City, St, Zip5, Zip4 as the address base > > and then the same with LName and FName for the other two respective > > hashes. > > > > The objective is to minimize hash collisions, not prevent some > > crypto attack. I use these hash fields to join between multi > > million record tables so If I need to discover info in TableA where > > the HashAddr is the same as in TableB, I need the probability of a > > collision between different addresses (family/Person) to be as close > > to zero as I can get it. > > > > My questions are: > > > > 1) Whether anyone out there is using a hash in this manner? > > 2) Has anyone seen a table of collision probability between messages > > of a given (short) message length. My message is 9 digits for the > > zip5/4 and the address could be something as short as PO Box 1, or > > Apt 1. IOW the total message length of 14 is pretty common. Adding > > the state would give me minimum message lengths of only 16 and City > > would only add a few more characters. 3) Does anyone know if just > > adding the same data back in again would decrease the collision > > probability. IOW Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. > > > > Any experience out there? > > > > > > -- > > John W. Colby > > www.ColbyConsulting.com > > _______________________________________________ > > dba-VB mailing list > > dba-VB at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > http://www.databaseadvisors.com > > > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 07:06:47 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 22:06:47 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> Message-ID: <4D849C57.3867.31639E4@stuart.lexacorp.com.pg> That got my interest piqued. So I've just built a small DLL (13KB) to calculate SHA1 digests for either strings or files. All you need to use it is to put the DLL somewhere in your PATH such as %SYSTEMROOT\System32 and include a couple of declarations in your application. Declare Function SHA1StringDigest Lib "LXSHA1.DLL" Alias "SHA1STRINGDIGEST" (Message As String) As String Declare Function SHA1FileDigest Lib "LXSHA1.DLL" Alias "SHA1FILEDIGEST" (FileName As String) As String It correctly returns the test results below for either the string or a file containing the same data. If anyone wants a copy, let me know. -- Stuart On 19 Mar 2011 at 21:20, Stuart McLachlan wrote: > Using your SHA1 function, what message digests do you get for the > standard test cases: > > 1. abc > 2. abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq > > These should return: > > 1. A9993E36 4706816A BA3E2571 7850C26C 9CD0D89D > 2. 84983E44 1C3BD26E BAAE4AA1 F95129E5 E54670F1 > > See Examples 1 and 2 at > http://www.itl.nist.gov/fipspubs/fip180-1.htm > > > > -- > Stuart > > > On 19 Mar 2011 at 20:20, Stuart McLachlan wrote: > > > How are you creating your hash? > > > > Can you post a few examples of different data strings and colliding > > SHA1 hashes. I can probably make a lot of money out of them. > > AFAIK, no one other than you has found any. > > > > -- > > Stuart > > > > > > On 18 Mar 2011 at 22:08, jwcolby wrote: > > > > > In my databases I create SHA1 hashes to enable joining between > > > tables and pull identical records (identical for the fields > > > hashed). > > > I create: > > > > > > 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the > > > three values and feed them into SHa1 and out pops a number which I > > > store in a field in my table. > > > > > > 2) A HashFamily of the Zip5, Zip4, Addr ad LName. > > > > > > 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. > > > > > > I am getting known collisions between different addresses (I have > > > discovered and investigated collisions) in my HashAddr when I have > > > many millions of addresses. I need to address this. > > > > > > Back when I made my design decisions (2004) my hardware consisted > > > of single core processors, 4 gigs ram, Windows x32 etc. Now I > > > have 8 cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great > > > extent constrained by my hardware "back in the day" whereas I am > > > much less so now. > > > > > > I am about to redesign my process. > > > > > > I am considering simply appending in the city and state strings to > > > all of the inputs: Addr, City, St, Zip5, Zip4 as the address base > > > and then the same with LName and FName for the other two > > > respective hashes. > > > > > > The objective is to minimize hash collisions, not prevent some > > > crypto attack. I use these hash fields to join between multi > > > million record tables so If I need to discover info in TableA > > > where the HashAddr is the same as in TableB, I need the > > > probability of a collision between different addresses > > > (family/Person) to be as close to zero as I can get it. > > > > > > My questions are: > > > > > > 1) Whether anyone out there is using a hash in this manner? > > > 2) Has anyone seen a table of collision probability between > > > messages of a given (short) message length. My message is 9 > > > digits for the zip5/4 and the address could be something as short > > > as PO Box 1, or Apt 1. IOW the total message length of 14 is > > > pretty common. Adding the state would give me minimum message > > > lengths of only 16 and City would only add a few more characters. > > > 3) Does anyone know if just adding the same data back in again > > > would decrease the collision probability. IOW > > > Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. > > > > > > Any experience out there? > > > > > > > > > -- > > > John W. Colby > > > www.ColbyConsulting.com > > > _______________________________________________ > > > dba-VB mailing list > > > dba-VB at databaseadvisors.com > > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > > http://www.databaseadvisors.com > > > > > > > > > > > > > > _______________________________________________ > > dba-VB mailing list > > dba-VB at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > http://www.databaseadvisors.com > > > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 07:09:56 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 08:09:56 -0400 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com> <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: <4D849D14.70906@colbyconsulting.com> >AFAIK, no one other than you has found any. LOL. You are thinking of hashing messages with thousands of characters. here's the deal. Let's assume that I hash X characters in a single field. The message length is the key, not the number of fields. Assume also that I am really only hashing letters, numbers and special characters - the characters that are in a name or address. If I hash 100 1 character messages, my probability of a collision if extremely high since there are only about 128 such alphanumeric characters (even less). Now hash a thousand 1 character messages. The probably just climbed immensely. Now hash a million such 1 character messages. What I am really calculating is the probability that I will repeat the same character string. However if I up the number of characters, let's say 10 character messages. *Now* the hash is really trying to prevent the same output for two *different* inputs. Well that is always what a hash is trying to do. But the point is that the probability of a collision decreases with message length increase. The probability of a collision between any two messages of 10 random characters is much higher than the probability of a collision between any two 100 character messages, and the probability of a collision between any two 1000 character messages is much higher still. That is just the way hash functions work. Nobody claims that hashes don't create collisions, and everyone agrees that the longer the message, the lower the probability of a collision between two messages (to a point). So, my address "messages" currently look like 89364 4456 PO Box 1 76543 9876 Apt 2 97867 3546 1723 Twin Pines Dr The point here is that all address "messages" are short. Now turn the address message into 89364 4456 PO Box 1 CA San Diego 76543 9876 Apt 2 TX Dallas 97867 3546 1723 Twin Pines Dr NC Hudson And the length of the message increases fairly dramatically, decreasing the probability of a collision. Remember too that I am hashing hundreds of millions of records - about 350 million addresses in various tables so far. The other thing that affects the probability of a hash collision is the number of messages hashed. Hash enough records and you *will* 100% probability create a hash that is the same for two different input strings - even for long messages. That is just the nature of the business. The probability of a collision is determined by three things. 1) The (strength of the) hash algorithm. 2) The message length 3) The quantity of messages. 1) I am using a fairly strong hash algorithm - SHA1 2) I am using a fairly short hash message - Zip5 / zip4 / Addr 3) I am hashing a fairly high number of messages - 350 million (and counting) messages (addresses) Believe me, hashes have already occurred, I have seen them. It is an easy thing to test. Join two tables on the same hash and in the query compare that each address field is the same between the two tables. Zip5 <> zip5 *or* zip4 <> zip4 *or* addr <> addr. If the hash is the same (inner join) AND one of the three fields is different then you have a collision. I have done this test and I have found collisions between my address hash. I have never tested my last name or first name hash fields. The problem with finding hashes is that they are rare, and in order to do an exhaustive test I would need to pull all the hash and data fields into a single table and compare that table against itself. My data is contained in a bunch of databases. 11 million "dogs and cats", 21 million "kids", 7 million "smokers", 23 million (and about to double) emails. 65 million "all adults", 100 million "mortgage" etc. So it is difficult to find hash collisions because (at least in the small tables) the probability of a collision is low entirely within a table. Believe me, it *does* happen though (at least for the address hash). John W. Colby www.ColbyConsulting.com On 3/19/2011 6:20 AM, Stuart McLachlan wrote: > How are you creating your hash? > > Can you post a few examples of different data strings and colliding SHA1 hashes. I can > probably make a lot of money out of them. AFAIK, no one other than you has found any. > From jwcolby at colbyconsulting.com Sat Mar 19 07:11:32 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 08:11:32 -0400 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D849C57.3867.31639E4@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> <4D849C57.3867.31639E4@stuart.lexacorp.com.pg> Message-ID: <4D849D74.2040104@colbyconsulting.com> I will do a collision test in my email file. AFAICT I have some collisions in there. If I find any I will send you the message and hash so that you can see what collisions look like. John W. Colby www.ColbyConsulting.com On 3/19/2011 8:06 AM, Stuart McLachlan wrote: > That got my interest piqued. > > So I've just built a small DLL (13KB) to calculate SHA1 digests for either strings or files. > All you need to use it is to put the DLL somewhere in your PATH such as > %SYSTEMROOT\System32 and include a couple of declarations in your application. > > Declare Function SHA1StringDigest Lib "LXSHA1.DLL" Alias "SHA1STRINGDIGEST" > (Message As String) As String > > Declare Function SHA1FileDigest Lib "LXSHA1.DLL" Alias "SHA1FILEDIGEST" (FileName > As String) As String > > It correctly returns the test results below for either the string or a file containing the same > data. > > If anyone wants a copy, let me know. > From harlan at powerzone.com Sat Mar 19 07:17:25 2011 From: harlan at powerzone.com (Harlan Koehn) Date: Sat, 19 Mar 2011 06:17:25 -0600 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D849D14.70906@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com> <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> <4D849D14.70906@colbyconsulting.com> Message-ID: <76EEB091F7CED543A80B5C2BC153975E0165A7C108@PZSERVER01.powerzone.local> John, I'm fairly ignorant when it comes to hashing algorithms but I'm curious if you have considered using a hashing algorithm that would generate a longer hash, which I'm assuming could reduce the likelihood of a collision. Harlan From stuart at lexacorp.com.pg Sat Mar 19 07:51:13 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 22:51:13 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D849D14.70906@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D849D14.70906@colbyconsulting.com> Message-ID: <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> In the case of 1000 single character messages, you are bound to get collisions since there are only 256 possible original messages. You will be hashing the same value multiple times Apart from that, your understanding is incorrect. It doesn't matter how long the string is. The chances of a collision with two different messages remains the same. The message is hashed using padded blocks of a fixed length. There is no more chance of a collision between "a" and "b" than there is between "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". Specifically, the chance of a collision within n different messages, using b bits of encryption is (n*(n-1)/2) * (1/2^b). Note that the length of the message doesn't come into that equation. The probability of a collision is determined by only TWO things. 1) The length of the digest. 2) The quantity of messages. As for "everyone agrees". Noone who understands how it works, agrees. -- Stuart On 19 Mar 2011 at 8:09, jwcolby wrote: > >AFAIK, no one other than you has found any. > > LOL. > > You are thinking of hashing messages with thousands of characters. > here's the deal. > > Let's assume that I hash X characters in a single field. The message > length is the key, not the number of fields. Assume also that I am > really only hashing letters, numbers and special characters - the > characters that are in a name or address. > > > If I hash 100 1 character messages, my probability of a collision if > extremely high since there are only about 128 such alphanumeric > characters (even less). > > Now hash a thousand 1 character messages. The probably just climbed > immensely. Now hash a million such 1 character messages. > What I am really calculating is the probability that I will repeat the > same character string. > > However if I up the number of characters, let's say 10 character > messages. *Now* the hash is really trying to prevent the same output > for two *different* inputs. Well that is always what a hash is trying > to do. But the point is that the probability of a collision decreases > with message length increase. > > The probability of a collision between any two messages of 10 random > characters is much higher than the probability of a collision between > any two 100 character messages, and the probability of a collision > between any two 1000 character messages is much higher still. > > That is just the way hash functions work. Nobody claims that hashes > don't create collisions, and everyone agrees that the longer the > message, the lower the probability of a collision between two messages > (to a point). > > So, my address "messages" currently look like > > 89364 4456 PO Box 1 > 76543 9876 Apt 2 > 97867 3546 1723 Twin Pines Dr > > The point here is that all address "messages" are short. Now turn the > address message into > > 89364 4456 PO Box 1 CA San Diego > 76543 9876 Apt 2 TX Dallas > 97867 3546 1723 Twin Pines Dr NC Hudson > > And the length of the message increases fairly dramatically, > decreasing the probability of a collision. > > Remember too that I am hashing hundreds of millions of records - about > 350 million addresses in various tables so far. The other thing that > affects the probability of a hash collision is the number of messages > hashed. Hash enough records and you *will* 100% probability create a > hash that is the same for two different input strings - even for long > messages. That is just the nature of the business. > > The probability of a collision is determined by three things. > > 1) The (strength of the) hash algorithm. > 2) The message length > 3) The quantity of messages. ... From shamil at smsconsulting.spb.ru Sat Mar 19 08:03:36 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 16:03:36 +0300 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com> <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: Hi Stuart and John, <<< AFAIK, no one other than you has found any >>> Yes, I was also confused how it comes to get SHA1 hashes collisions but then I have thought that John probably "compacts" SHA1 hashes into 8 bytes surrogate keys? Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions for 6 millions cycles for 8 bytes keys. Please correct me if you find some errors in my sample code Thank you. -- Shamil P.S. Code: public class GenerateHashes { private static Dictionary _generatedHashes; private static Dictionary _generatedLongHashes; public static void Run(int minHashedStringLength, int maxHashedStringLength, long testCasesQty) { System.Console.WriteLine("Started at: " + System.DateTime.Now); _generatedHashes = new Dictionary(); _generatedLongHashes = new Dictionary(); int collisionsQty = 0; int longHashCollisionsQty = 0; decimal totalLengthOfProcessedStrings = 0; decimal averageStringLength = 0; Random random = new Random((int)System.DateTime.Now.Ticks); for (long i = 1; i <= testCasesQty; i++) { int stringLength = random.Next(minHashedStringLength, maxHashedStringLength); byte[] plainTextBytes = (byte[])Array.CreateInstance(typeof(byte), stringLength); random.NextBytes(plainTextBytes); System.Security.Cryptography.HashAlgorithm hash = new SHA1Managed(); byte[] hashBytes = hash.ComputeHash(plainTextBytes); string hashValue = Convert.ToBase64String(hashBytes); string testValue; if (_generatedHashes.TryGetValue(hashValue, out testValue)) collisionsQty++; else _generatedHashes.Add(hashValue, hashValue); if (i % 1000000 == 0) System.Console.WriteLine( "{0}: i={1}, cs={2}, cu={3}", DateTime.Now, i, collisionsQty, longHashCollisionsQty); ulong longHash = 0xFFFFFFFF; foreach (byte b in hashValue) { longHash ^= b; longHash = longHash << 8 + 0xFF; } ulong longHashTestValue; if (_generatedLongHashes.TryGetValue(longHash, out longHashTestValue)) { //Long Hash Collision longHashCollisionsQty++; System.Console.WriteLine(" * step {0}, hash '{1}', longHash: {2:X}", i, hashValue, longHash); } else _generatedLongHashes.Add(longHash, longHash); totalLengthOfProcessedStrings += stringLength; averageStringLength = totalLengthOfProcessedStrings / i; } System.Console.WriteLine( "\nTOTALS:\n" + "Min String Length = {0}\n" + "Max String Length = {1}\n" + "Avg String Length = {2}\n" + "Test cases Qty = {3}\n" + "Collisions Qty = {4}\n" + "ulong Collisions Qty = {5}", minHashedStringLength, maxHashedStringLength, averageStringLength, testCasesQty, collisionsQty, longHashCollisionsQty); System.Console.WriteLine("Finished at: " + System.DateTime.Now); } } P.P.S. Stats Started at: 19/03/2011 15:51:04 19/03/2011 15:51:16: i=1000000, cs=0, cu=0 * step 1712197, hash 'Rt4NY5FHg0uMDszVql+l6ksx6Tg=', longHash: 36D7CFC36A99DE80 19/03/2011 15:51:29: i=2000000, cs=0, cu=1 * step 2040324, hash 'laoPHdqlfeZOx9DI8RDaQZjhKZs=', longHash: D1B5AB44BB5CDE80 * step 2043449, hash '1m+tgKu7/l9fl8nUNuz5I/OWF5E=', longHash: C95F3EBC66B15E80 19/03/2011 15:51:44: i=3000000, cs=0, cu=3 19/03/2011 15:52:00: i=4000000, cs=0, cu=3 * step 4369446, hash '2sZrglXDp3NnNPuqdILqwmUKq30=', longHash: F7DB565F166C1E80 19/03/2011 15:52:17: i=5000000, cs=0, cu=4 TOTALS: Min String Length = 7 Max String Length = 25 Avg String Length = 15.4981354 Test cases Qty = 5,000,000 Collisions Qty = 0 ulong Collisions Qty = 4 Finished at: 19/03/2011 15:52:17 -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart McLachlan Sent: 19 ????? 2011 ?. 13:20 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SHA1 to compute a hash How are you creating your hash? Can you post a few examples of different data strings and colliding SHA1 hashes. I can probably make a lot of money out of them. AFAIK, no one other than you has found any. -- Stuart <<< snip >>> From stuart at lexacorp.com.pg Sat Mar 19 08:10:47 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 23:10:47 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D849D14.70906@colbyconsulting.com>, <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> Message-ID: <4D84AB57.31948.350CF8F@stuart.lexacorp.com.pg> I've just realised where the confusion comes in. When you talk about the chance of collisions increasing decreasing message length, you are talking about general hashing algorithms such as CRC etc. You need to undersand the difference between simple and cryptographic hashing. Cryptographic hashing algorithms are specifically designed to be "collision resistant". I repeat, to date no one has found a collision in SHA1 (except you apparently ) -- Stuart On 19 Mar 2011 at 22:51, Stuart McLachlan wrote: > In the case of 1000 single character messages, you are bound to get > collisions since there are only 256 possible original messages. You > will be hashing the same value multiple times > > Apart from that, your understanding is incorrect. It doesn't matter > how long the string is. The chances of a collision with two different > messages remains the same. > > The message is hashed using padded blocks of a fixed length. > There is no more chance of a collision between "a" and "b" than there > is between "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". > > Specifically, the chance of a collision within n different messages, > using b bits of encryption is (n*(n-1)/2) * (1/2^b). > > Note that the length of the message doesn't come into that equation. > > The probability of a collision is determined by only TWO things. > > 1) The length of the digest. > 2) The quantity of messages. > > As for "everyone agrees". Noone who understands how it works, agrees. > > -- > Stuart > > On 19 Mar 2011 at 8:09, jwcolby wrote: > > > >AFAIK, no one other than you has found any. > > > > LOL. > > > > You are thinking of hashing messages with thousands of characters. > > here's the deal. > > > > Let's assume that I hash X characters in a single field. The > > message length is the key, not the number of fields. Assume also > > that I am really only hashing letters, numbers and special > > characters - the characters that are in a name or address. > > > > > > If I hash 100 1 character messages, my probability of a collision if > > extremely high since there are only about 128 such alphanumeric > > characters (even less). > > > > Now hash a thousand 1 character messages. The probably just climbed > > immensely. Now hash a million such 1 character messages. What I am > > really calculating is the probability that I will repeat the same > > character string. > > > > However if I up the number of characters, let's say 10 character > > messages. *Now* the hash is really trying to prevent the same > > output for two *different* inputs. Well that is always what a hash > > is trying to do. But the point is that the probability of a > > collision decreases with message length increase. > > > > The probability of a collision between any two messages of 10 random > > characters is much higher than the probability of a collision > > between any two 100 character messages, and the probability of a > > collision between any two 1000 character messages is much higher > > still. > > > > That is just the way hash functions work. Nobody claims that hashes > > don't create collisions, and everyone agrees that the longer the > > message, the lower the probability of a collision between two > > messages (to a point). > > > > So, my address "messages" currently look like > > > > 89364 4456 PO Box 1 > > 76543 9876 Apt 2 > > 97867 3546 1723 Twin Pines Dr > > > > The point here is that all address "messages" are short. Now turn > > the address message into > > > > 89364 4456 PO Box 1 CA San Diego > > 76543 9876 Apt 2 TX Dallas > > 97867 3546 1723 Twin Pines Dr NC Hudson > > > > And the length of the message increases fairly dramatically, > > decreasing the probability of a collision. > > > > Remember too that I am hashing hundreds of millions of records - > > about 350 million addresses in various tables so far. The other > > thing that affects the probability of a hash collision is the number > > of messages hashed. Hash enough records and you *will* 100% > > probability create a hash that is the same for two different input > > strings - even for long messages. That is just the nature of the > > business. > > > > The probability of a collision is determined by three things. > > > > 1) The (strength of the) hash algorithm. > > 2) The message length > > 3) The quantity of messages. > ... > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 08:20:11 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 23:20:11 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, Message-ID: <4D84AD8B.14064.3596D15@stuart.lexacorp.com.pg> If he is compacting a 160 bit hash to a 64 bit hash, he is certainly increasing the odds of a collision - but I suspect that even at 64 bits, the chances of a collision would be very low. But that does not appear to be what you are testing. It looks as though you created 6 million random strings and four of them turned out to be identical. Therefore also had the same SHA1 digest. What happens if you try it again, making sure that all of your strings are unique. Alternatively, what do you get if you add some code to show the pairs of strings. Are they different? -- Stuart On 19 Mar 2011 at 16:03, Shamil Salakhetdinov wrote: > Hi Stuart and John, > > <<< > AFAIK, no one other than you has found any > >>> > Yes, I was also confused how it comes to get SHA1 hashes collisions > but then I have thought that John probably "compacts" SHA1 hashes into > 8 bytes surrogate keys? > > Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions > for 6 millions cycles for 8 bytes keys. > > Please correct me if you find some errors in my sample code > > Thank you. > > -- > Shamil > > P.S. Code: > > public class GenerateHashes > { > private static Dictionary _generatedHashes; > private static Dictionary _generatedLongHashes; > > public static void Run(int minHashedStringLength, int > maxHashedStringLength, long testCasesQty) > { > System.Console.WriteLine("Started at: " + > System.DateTime.Now); > > _generatedHashes = new Dictionary(); > _generatedLongHashes = new Dictionary(); > > int collisionsQty = 0; > int longHashCollisionsQty = 0; > > decimal totalLengthOfProcessedStrings = 0; > decimal averageStringLength = 0; > > > Random random = new > Random((int)System.DateTime.Now.Ticks); for (long i = 1; i > <= testCasesQty; i++) { > int stringLength = random.Next(minHashedStringLength, > maxHashedStringLength); > byte[] plainTextBytes = > (byte[])Array.CreateInstance(typeof(byte), stringLength); > random.NextBytes(plainTextBytes); > System.Security.Cryptography.HashAlgorithm hash = new > SHA1Managed(); > byte[] hashBytes = hash.ComputeHash(plainTextBytes); > string hashValue = Convert.ToBase64String(hashBytes); > string testValue; if > (_generatedHashes.TryGetValue(hashValue, out > testValue)) > collisionsQty++; > else _generatedHashes.Add(hashValue, hashValue); > if (i % 1000000 == 0) > System.Console.WriteLine( > "{0}: i={1}, cs={2}, cu={3}", > DateTime.Now, i, collisionsQty, > longHashCollisionsQty); > > ulong longHash = 0xFFFFFFFF; > foreach (byte b in hashValue) > { > longHash ^= b; > longHash = longHash << 8 + 0xFF; > } > ulong longHashTestValue; > if (_generatedLongHashes.TryGetValue(longHash, out > longHashTestValue)) > { > //Long Hash Collision > longHashCollisionsQty++; > System.Console.WriteLine(" * step {0}, hash '{1}', > longHash: {2:X}", > i, hashValue, longHash); > } > else _generatedLongHashes.Add(longHash, longHash); > > totalLengthOfProcessedStrings += stringLength; > averageStringLength = totalLengthOfProcessedStrings / > i; > } > > System.Console.WriteLine( > "\nTOTALS:\n" + > "Min String Length = {0}\n" + > "Max String Length = {1}\n" + > "Avg String Length = {2}\n" + > "Test cases Qty = {3}\n" + > "Collisions Qty = {4}\n" + > "ulong Collisions Qty = {5}", > minHashedStringLength, maxHashedStringLength, > averageStringLength, testCasesQty, > collisionsQty, > longHashCollisionsQty); > > System.Console.WriteLine("Finished at: " + > System.DateTime.Now); > > } > } > > > P.P.S. Stats > > Started at: 19/03/2011 15:51:04 > 19/03/2011 15:51:16: i=1000000, cs=0, cu=0 > * step 1712197, hash 'Rt4NY5FHg0uMDszVql+l6ksx6Tg=', longHash: > 36D7CFC36A99DE80 > 19/03/2011 15:51:29: i=2000000, cs=0, cu=1 > * step 2040324, hash 'laoPHdqlfeZOx9DI8RDaQZjhKZs=', longHash: > D1B5AB44BB5CDE80 > * step 2043449, hash '1m+tgKu7/l9fl8nUNuz5I/OWF5E=', longHash: > C95F3EBC66B15E80 > 19/03/2011 15:51:44: i=3000000, cs=0, cu=3 > 19/03/2011 15:52:00: i=4000000, cs=0, cu=3 > * step 4369446, hash '2sZrglXDp3NnNPuqdILqwmUKq30=', longHash: > F7DB565F166C1E80 > 19/03/2011 15:52:17: i=5000000, cs=0, cu=4 > > TOTALS: > Min String Length = 7 > Max String Length = 25 > Avg String Length = 15.4981354 > Test cases Qty = 5,000,000 > Collisions Qty = 0 > ulong Collisions Qty = 4 > Finished at: 19/03/2011 15:52:17 > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart > McLachlan Sent: 19 ????? 2011 ?. 13:20 To: Discussion concerning > Visual Basic and related programming issues. Subject: Re: [dba-VB] > SHA1 to compute a hash > > How are you creating your hash? > > Can you post a few examples of different data strings and colliding > SHA1 hashes. I can probably make a lot of money out of them. > AFAIK, no one other than you has found any. > > -- > Stuart > > <<< snip >>> > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 08:21:58 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 09:21:58 -0400 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D849D14.70906@colbyconsulting.com> <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> Message-ID: <4D84ADF6.4000509@colbyconsulting.com> Stuart, Thanks for that. I am not a cryptologist and all I can do is try to use a tool I do not understand to solve a problem I do understand. If my hashes really are that "unique / secure" then I will stop worrying about it. I still need to do a check of actual data though because I went looking one day and discovered "collisions". Given that we believe the collision is not in the key itself, it must have been in my method of finding the collision. In any case, I am not going to worry any more so thanks for the peace of mind! ;) John W. Colby www.ColbyConsulting.com On 3/19/2011 8:51 AM, Stuart McLachlan wrote: > In the case of 1000 single character messages, you are bound to get collisions since there > are only 256 possible original messages. You will be hashing the same value multiple times > > Apart from that, your understanding is incorrect. It doesn't matter how long the string is. The > chances of a collision with two different messages remains the same. > > The message is hashed using padded blocks of a fixed length. > There is no more chance of a collision between "a" and "b" than there is between > "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". > > Specifically, the chance of a collision within n different messages, using b bits of encryption is > (n*(n-1)/2) * (1/2^b). > > Note that the length of the message doesn't come into that equation. > > The probability of a collision is determined by only TWO things. > > 1) The length of the digest. > 2) The quantity of messages. > > As for "everyone agrees". Noone who understands how it works, agrees. > From shamil at smsconsulting.spb.ru Sat Mar 19 08:21:26 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 16:21:26 +0300 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: References: <4D841001.2060200@colbyconsulting.com><4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: <49B2F00A3FD3442685D8283AE7678544@nant> Hi All -- Sorry I have mentioned 6 millions cycles but my sample was for just 5 millions, and I have got some collisions for 8 bytes long surrogate keys. Here I'm posting a patch for hashing part of the code - and no collisions now for 8 bytes long hashes for 5 millions cycles. .... UInt64 longHashResult = 0xFFFFFFFFFFFFFFFF; UInt64 longHash = 0xFFFFFFFFFFFFFFFF; int shiftCounter = 0; foreach (byte b in hashValue) { longHash ^= b; longHash = longHash << 8 + 0xFF; if (++shiftCounter == 8) { longHashResult ^= longHash; shiftCounter = 0; } } longHash = longHashResult; ... Started at: 19/03/2011 16:08:53 19/03/2011 16:09:04: i=1000000, cs=0, cu=0 19/03/2011 16:09:17: i=2000000, cs=0, cu=0 19/03/2011 16:09:33: i=3000000, cs=0, cu=0 19/03/2011 16:09:49: i=4000000, cs=0, cu=0 19/03/2011 16:10:06: i=5000000, cs=0, cu=0 TOTALS: Min String Length = 7 Max String Length = 25 Avg String Length = 15.4966972 Test cases Qty = 5000000 Collisions Qty = 0 ulong Collisions Qty = 0 Finished at: 19/03/2011 16:10:06 Please correct me if I've got wrong somewhere. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 19 ????? 2011 ?. 16:04 To: stuart at lexacorp.com.pg; 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] SHA1 to compute a hash Hi Stuart and John, <<< AFAIK, no one other than you has found any >>> Yes, I was also confused how it comes to get SHA1 hashes collisions but then I have thought that John probably "compacts" SHA1 hashes into 8 bytes surrogate keys? Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions for 6 millions cycles for 8 bytes keys. Please correct me if you find some errors in my sample code Thank you. -- Shamil <<< snip >>> From jwcolby at colbyconsulting.com Sat Mar 19 08:24:54 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 09:24:54 -0400 Subject: [dba-VB] On a lighter note Message-ID: <4D84AEA6.5060107@colbyconsulting.com> I rooted my droid X last night, bought a "freezer" program, froze all the bloatware, and downloaded and installed a tether program. Here at my house I get about 1.25 mbit down and .5 mbit up via the tether. Not bad considering it is over cell. -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sat Mar 19 08:24:33 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 16:24:33 +0300 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84AD8B.14064.3596D15@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D84AD8B.14064.3596D15@stuart.lexacorp.com.pg> Message-ID: <14BB8E00F24E4EBAB5842E5D75537D2F@nant> Hi Stuart -- <<< It looks as though you created 6 million random strings and four of them turned out to be identical. >>> No, all SHA1 strings are different - these are 64 bit hashes of that strings, which have got some collisions. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart McLachlan Sent: 19 ????? 2011 ?. 16:20 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SHA1 to compute a hash If he is compacting a 160 bit hash to a 64 bit hash, he is certainly increasing the odds of a collision - but I suspect that even at 64 bits, the chances of a collision would be very low. But that does not appear to be what you are testing. It looks as though you created 6 million random strings and four of them turned out to be identical. Therefore also had the same SHA1 digest. What happens if you try it again, making sure that all of your strings are unique. Alternatively, what do you get if you add some code to show the pairs of strings. Are they different? -- Stuart On 19 Mar 2011 at 16:03, Shamil Salakhetdinov wrote: > Hi Stuart and John, > > <<< > AFAIK, no one other than you has found any > >>> > Yes, I was also confused how it comes to get SHA1 hashes collisions > but then I have thought that John probably "compacts" SHA1 hashes into > 8 bytes surrogate keys? > > Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions > for 6 millions cycles for 8 bytes keys. > > Please correct me if you find some errors in my sample code > > Thank you. > > -- > Shamil > <<< snip >>> From stuart at lexacorp.com.pg Sat Mar 19 08:29:55 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 23:29:55 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84ADF6.4000509@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com>, <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg>, <4D84ADF6.4000509@colbyconsulting.com> Message-ID: <4D84AFD3.25991.3625487@stuart.lexacorp.com.pg> Using a 160 bit SHA1, the odds of getting a collision are 50% when you have approximately 2^80 messages. If you ever get near that number of records, I suggest you switch to SHA2 which is 256 bit :-) -- Stuart On 19 Mar 2011 at 9:21, jwcolby wrote: > Stuart, > > Thanks for that. I am not a cryptologist and all I can do is try to > use a tool I do not understand to solve a problem I do understand. If > my hashes really are that "unique / secure" then I will stop worrying > about it. I still need to do a check of actual data though because I > went looking one day and discovered "collisions". Given that we > believe the collision is not in the key itself, it must have been in > my method of finding the collision. > > In any case, I am not going to worry any more so thanks for the peace > of mind! ;) > > John W. Colby > www.ColbyConsulting.com > > On 3/19/2011 8:51 AM, Stuart McLachlan wrote: > > In the case of 1000 single character messages, you are bound to get > > collisions since there are only 256 possible original messages. You > > will be hashing the same value multiple times > > > > Apart from that, your understanding is incorrect. It doesn't matter > > how long the string is. The chances of a collision with two > > different messages remains the same. > > > > The message is hashed using padded blocks of a fixed length. > > There is no more chance of a collision between "a" and "b" than > > there is between "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". > > > > Specifically, the chance of a collision within n different messages, > > using b bits of encryption is (n*(n-1)/2) * (1/2^b). > > > > Note that the length of the message doesn't come into that equation. > > > > The probability of a collision is determined by only TWO things. > > > > 1) The length of the digest. > > 2) The quantity of messages. > > > > As for "everyone agrees". Noone who understands how it works, > > agrees. > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 08:55:51 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 09:55:51 -0400 Subject: [dba-VB] Using the hash for real world problems Message-ID: <4D84B5E7.3010204@colbyconsulting.com> I got into the hashing business in order to create a surrogate key of sorts, in fact a trio of them. I have an address hash, a family hash and a person hash. I compute these three hashes for every table of persons. Now that I have these surrogate keys I have "identical" values in a single field where the input is identical in multiple fields. Thus I do not have to do multi-field joins where I have a hash for all of those fields. I am happy to hear that I am secure in my surrogate hash algorithm choice. OK, so I have these hashes. Every record has these three hashes. Now some definitions: HashAddr: A unique address is addr/zip5/zip4 different HashFamily: A unique family is LName/Addr/Zip5/Zip4 different HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. Using this information, I need to calculate the count of addresses with a single person, with two persons etc for as many combinations as I have. Addresses People 22,538,240 1 780,462,346 2 52,234 3 people etc etc till every quantity of people is found. What is the simplest way to accomplish this with SQL. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 19 09:13:51 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 10:13:51 -0400 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84B5E7.3010204@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com> Message-ID: <4D84BA1F.2060101@colbyconsulting.com> Ooops. My examples were hosed. Let's try again Addresses People 22,538,240 1 00,708,462 2 00,052,234 3 and so forth. What the client really wants to know is "how many people with emails are at a specific address" "how many people with emails are in a specific family" etc. I tie my address table to my email table using an auto increment long increment "I must be careful not to call this a surrogate" key and then perform the counts using the hash fields in a Group By / count query. John W. Colby www.ColbyConsulting.com On 3/19/2011 9:55 AM, jwcolby wrote: > I got into the hashing business in order to create a surrogate key of sorts, in fact a trio of them. > > I have an address hash, a family hash and a person hash. I compute these three hashes for every > table of persons. Now that I have these surrogate keys I have "identical" values in a single field > where the input is identical in multiple fields. Thus I do not have to do multi-field joins where I > have a hash for all of those fields. > > I am happy to hear that I am secure in my surrogate hash algorithm choice. > > OK, so I have these hashes. Every record has these three hashes. > > Now some definitions: > > HashAddr: A unique address is addr/zip5/zip4 different > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > Using this information, I need to calculate the count of addresses with a single person, with two > persons etc for as many combinations as I have. > > Addresses People > 22,538,240 1 > 780,462,346 2 > 52,234 3 people > etc > etc till every quantity of people is found. > > What is the simplest way to accomplish this with SQL. > > > > From shamil at smsconsulting.spb.ru Sat Mar 19 10:07:08 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 18:07:08 +0300 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84BA1F.2060101@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com> <4D84BA1F.2060101@colbyconsulting.com> Message-ID: Hi John -- I suppose you can reuse the following SQL expression for your case: select count(Occurrence) QtyTheSameOccurrences, Occurrence from ( select Country, Count(Country) Occurrence from Customer Group by Country ) o group by o.Occurrence order by o.Occurrence Just replace Country name columns with hashAddress column and Country table name with your table name ... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 19 ????? 2011 ?. 17:14 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Using the hash for real world problems Ooops. My examples were hosed. Let's try again Addresses People 22,538,240 1 00,708,462 2 00,052,234 3 and so forth. What the client really wants to know is "how many people with emails are at a specific address" "how many people with emails are in a specific family" etc. I tie my address table to my email table using an auto increment long increment "I must be careful not to call this a surrogate" key and then perform the counts using the hash fields in a Group By / count query. John W. Colby www.ColbyConsulting.com On 3/19/2011 9:55 AM, jwcolby wrote: > I got into the hashing business in order to create a surrogate key of sorts, in fact a trio of them. > > I have an address hash, a family hash and a person hash. I compute > these three hashes for every table of persons. Now that I have these > surrogate keys I have "identical" values in a single field where the > input is identical in multiple fields. Thus I do not have to do multi-field joins where I have a hash for all of those fields. > > I am happy to hear that I am secure in my surrogate hash algorithm choice. > > OK, so I have these hashes. Every record has these three hashes. > > Now some definitions: > > HashAddr: A unique address is addr/zip5/zip4 different > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > Using this information, I need to calculate the count of addresses > with a single person, with two persons etc for as many combinations as I have. > > Addresses People > 22,538,240 1 > 780,462,346 2 > 52,234 3 people > etc > etc till every quantity of people is found. > > What is the simplest way to accomplish this with SQL. > > > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Mar 19 14:50:55 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 15:50:55 -0400 Subject: [dba-VB] Most useful droid apps Message-ID: <4D85091F.5060807@colbyconsulting.com> What are the most useful apps you run on your droid? -- John W. Colby www.ColbyConsulting.com From davidmcafee at gmail.com Sat Mar 19 15:53:32 2011 From: davidmcafee at gmail.com (David McAfee) Date: Sat, 19 Mar 2011 13:53:32 -0700 Subject: [dba-VB] Most useful droid apps Message-ID: Advanced task killer Weather channel Lookout antivirus AK Notepad Bank of America Acar auto maintenance Barcode scanner Bubble level Bump Droid light Google goggles Google translate IMDB Key ring Print share Trapster Sent from my Droid phone. On Mar 19, 2011 12:52 PM, "jwcolby" wrote: From stuart at lexacorp.com.pg Sat Mar 19 18:11:26 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sun, 20 Mar 2011 09:11:26 +1000 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84BA1F.2060101@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com>, <4D84BA1F.2060101@colbyconsulting.com> Message-ID: <4D85381E.9973.576BE3C@stuart.lexacorp.com.pg> Caaution. All aircode below: To get the list output below, I would use something like Select Distinct Count(HashAddr) as Addresses, People >From (Select distinct HashAddr,Count(HashAddr) as People From tblPeople Group By HashAddr) Group By People Order by People To find out details for a *specific* Address Find the Hash for the desired Address and then pass it to something like this: Select Distinct StreetNo,Street,City,Zip,State,Count(HashAddr) as Address >From tblPeople P Inner Join tblEmails E on P.PK = E.PeopleFK Group By HashAddr StreetNo,Street,City,Zip,State Where HashAddr = @SearchHash On 19 Mar 2011 at 10:13, jwcolby wrote: > Ooops. My examples were hosed. > > Let's try again > > Addresses People > 22,538,240 1 > 00,708,462 2 > 00,052,234 3 > > and so forth. > > What the client really wants to know is > > "how many people with emails are at a specific address" > "how many people with emails are in a specific family" > > etc. > > I tie my address table to my email table using an auto increment long > increment "I must be careful not to call this a surrogate" key and > then perform the counts using the hash fields in a Group By / count > query. > > John W. Colby > www.ColbyConsulting.com > > On 3/19/2011 9:55 AM, jwcolby wrote: > > I got into the hashing business in order to create a surrogate key > > of sorts, in fact a trio of them. > > > > I have an address hash, a family hash and a person hash. I compute > > these three hashes for every table of persons. Now that I have these > > surrogate keys I have "identical" values in a single field where the > > input is identical in multiple fields. Thus I do not have to do > > multi-field joins where I have a hash for all of those fields. > > > > I am happy to hear that I am secure in my surrogate hash algorithm > > choice. > > > > OK, so I have these hashes. Every record has these three hashes. > > > > Now some definitions: > > > > HashAddr: A unique address is addr/zip5/zip4 different > > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > > > Using this information, I need to calculate the count of addresses > > with a single person, with two persons etc for as many combinations > > as I have. > > > > Addresses People > > 22,538,240 1 > > 780,462,346 2 > > 52,234 3 people > > etc > > etc till every quantity of people is found. > > > > What is the simplest way to accomplish this with SQL. > > > > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 18:21:18 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sun, 20 Mar 2011 09:21:18 +1000 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84B5E7.3010204@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com> Message-ID: <4D853A6E.20716.57FC618@stuart.lexacorp.com.pg> Answered on VB list. On 19 Mar 2011 at 9:55, jwcolby wrote: > I got into the hashing business in order to create a surrogate key of > sorts, in fact a trio of them. > > I have an address hash, a family hash and a person hash. I compute > these three hashes for every table of persons. Now that I have these > surrogate keys I have "identical" values in a single field where the > input is identical in multiple fields. Thus I do not have to do > multi-field joins where I have a hash for all of those fields. > > I am happy to hear that I am secure in my surrogate hash algorithm > choice. > > OK, so I have these hashes. Every record has these three hashes. > > Now some definitions: > > HashAddr: A unique address is addr/zip5/zip4 different > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > Using this information, I need to calculate the count of addresses > with a single person, with two persons etc for as many combinations as > I have. > > Addresses People > 22,538,240 1 > 780,462,346 2 > 52,234 3 people > etc > etc till every quantity of people is found. > > What is the simplest way to accomplish this with SQL. > > > > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 21:01:04 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 22:01:04 -0400 Subject: [dba-VB] Most useful droid apps In-Reply-To: References: Message-ID: <4D855FE0.9070004@colbyconsulting.com> I am trying various tethering solutions. My laptop is vista, my ip range is 192.168.122.x. I got WiFi Tether to work, but only without encryption. I was getting about 1.5 mbit down and .5 mbit up. As soon as I turned on encryption I was unable to set up the laptop password to match (that was the only thing that changed). No encryption is unacceptable to me as I would only really use this in public places, exactly where you would most want encryption. I tried barnacle. That one just didn't work at all. I am now trying PDANet which is a USB tether. Since the PC is tied to the phone by a usb, encryption seems to be unnecessary? Unfortunately this one is not working either. The PDANet desktop app says it is connected and the droid pdanet app says it is connected but the PC just can't see the internet. I can't ping anything etc. Any suggestions for this beast? John W. Colby www.ColbyConsulting.com On 3/19/2011 4:53 PM, David McAfee wrote: > Advanced task killer > Weather channel > Lookout antivirus > AK Notepad > Bank of America > Acar auto maintenance > Barcode scanner > Bubble level > Bump > Droid light > Google goggles > Google translate > IMDB > Key ring > Print share > Trapster > > Sent from my Droid phone. > On Mar 19, 2011 12:52 PM, "jwcolby" wrote: > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 21:34:03 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sun, 20 Mar 2011 12:34:03 +1000 Subject: [dba-VB] Most useful droid apps In-Reply-To: <4D855FE0.9070004@colbyconsulting.com> References: , <4D855FE0.9070004@colbyconsulting.com> Message-ID: <4D85679B.30595.6303F8F@stuart.lexacorp.com.pg> Are you on AT&T? If so, you may be interested in this: http://www.theregister.co.uk/2011/03/18/new_att_tethering_policy/ On 19 Mar 2011 at 22:01, jwcolby wrote: > I am trying various tethering solutions. My laptop is vista, my ip > range is 192.168.122.x. > > I got WiFi Tether to work, but only without encryption. I was getting > about 1.5 mbit down and .5 mbit up. > > As soon as I turned on encryption I was unable to set up the laptop > password to match (that was the only thing that changed). No > encryption is unacceptable to me as I would only really use this in > public places, exactly where you would most want encryption. > > I tried barnacle. That one just didn't work at all. > > I am now trying PDANet which is a USB tether. Since the PC is tied to > the phone by a usb, encryption seems to be unnecessary? Unfortunately > this one is not working either. The PDANet desktop app says it is > connected and the droid pdanet app says it is connected but the PC > just can't see the internet. I can't ping anything etc. > > Any suggestions for this beast? > > > John W. Colby > www.ColbyConsulting.com > > On 3/19/2011 4:53 PM, David McAfee wrote: > > Advanced task killer > > Weather channel > > Lookout antivirus > > AK Notepad > > Bank of America > > Acar auto maintenance > > Barcode scanner > > Bubble level > > Bump > > Droid light > > Google goggles > > Google translate > > IMDB > > Key ring > > Print share > > Trapster > > > > Sent from my Droid phone. > > On Mar 19, 2011 12:52 PM, "jwcolby" > > wrote: _______________________________________________ dba-VB > > mailing list dba-VB at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > http://www.databaseadvisors.com > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 21:45:39 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 22:45:39 -0400 Subject: [dba-VB] Most useful droid apps In-Reply-To: <4D85679B.30595.6303F8F@stuart.lexacorp.com.pg> References: , <4D855FE0.9070004@colbyconsulting.com> <4D85679B.30595.6303F8F@stuart.lexacorp.com.pg> Message-ID: <4D856A53.7010507@colbyconsulting.com> Nope, Verizon John W. Colby www.ColbyConsulting.com On 3/19/2011 10:34 PM, Stuart McLachlan wrote: > Are you on AT&T? > > If so, you may be interested in this: > > http://www.theregister.co.uk/2011/03/18/new_att_tethering_policy/ > > > On 19 Mar 2011 at 22:01, jwcolby wrote: > >> I am trying various tethering solutions. My laptop is vista, my ip >> range is 192.168.122.x. >> >> I got WiFi Tether to work, but only without encryption. I was getting >> about 1.5 mbit down and .5 mbit up. >> >> As soon as I turned on encryption I was unable to set up the laptop >> password to match (that was the only thing that changed). No >> encryption is unacceptable to me as I would only really use this in >> public places, exactly where you would most want encryption. >> >> I tried barnacle. That one just didn't work at all. >> >> I am now trying PDANet which is a USB tether. Since the PC is tied to >> the phone by a usb, encryption seems to be unnecessary? Unfortunately >> this one is not working either. The PDANet desktop app says it is >> connected and the droid pdanet app says it is connected but the PC >> just can't see the internet. I can't ping anything etc. >> >> Any suggestions for this beast? >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/19/2011 4:53 PM, David McAfee wrote: >>> Advanced task killer >>> Weather channel >>> Lookout antivirus >>> AK Notepad >>> Bank of America >>> Acar auto maintenance >>> Barcode scanner >>> Bubble level >>> Bump >>> Droid light >>> Google goggles >>> Google translate >>> IMDB >>> Key ring >>> Print share >>> Trapster >>> >>> Sent from my Droid phone. >>> On Mar 19, 2011 12:52 PM, "jwcolby" >>> wrote: _______________________________________________ dba-VB >>> mailing list dba-VB at databaseadvisors.com >>> http://databaseadvisors.com/mailman/listinfo/dba-vb >>> http://www.databaseadvisors.com >>> >>> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From listmaster at databaseadvisors.com Wed Mar 23 14:34:46 2011 From: listmaster at databaseadvisors.com (Bryan Carbonnell) Date: Wed, 23 Mar 2011 15:34:46 -0400 Subject: [dba-VB] Administrivia - Spam Filtering on DBA Servers In-Reply-To: References: Message-ID: We have made a change to the way our mailservers deal with spam filtering. We were getting too many false positive reports from one of the spam black list services we use, so we have discontinued using them. Hopefully your bounce troubles should be over, or at the very least greatly minimized. If you continue to have issues surrounding bounces, PLEASE, PLEASE, PLEASE get in touch with me (listmaster at databaseadvisors OR carbonnb at gmail.com if the listmaster address bounces). -- Bryan Carbonnell - listmaster at databaseadvisors.com Life's journey is not to arrive at the grave safely in a well preserved body, but rather to skid in sideways, totally worn out, shouting "What a great ride!" From jwcolby at colbyconsulting.com Thu Mar 24 13:07:19 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 24 Mar 2011 14:07:19 -0400 Subject: [dba-VB] C# protected Message-ID: <4D8B8857.60604@colbyconsulting.com> I am trying to create a method of a ClassA which can be seen from the object ClassB that instantiates ClassA but not by objectClassC which instantiates ClassB. IOW ClassA "belongs to" ClassB so specific methods / properties of ClassB should be manipulateable by ClassB. However those properties / methods should not be visible to anything outside of ClassA and ClassB. Possible? Oh, and BTW I want ClassA to be out in a class library, external to the project using it. -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Thu Mar 24 15:14:59 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 24 Mar 2011 23:14:59 +0300 Subject: [dba-VB] C# protected In-Reply-To: <4D8B8857.60604@colbyconsulting.com> References: <4D8B8857.60604@colbyconsulting.com> Message-ID: Hi John -- Put ClassA and ClassB into a class library. Use internal access modifier with their methods and properties to be seen within their class library but not from the "outer world". Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 24 ????? 2011 ?. 21:07 To: VBA Subject: [dba-VB] C# protected I am trying to create a method of a ClassA which can be seen from the object ClassB that instantiates ClassA but not by objectClassC which instantiates ClassB. IOW ClassA "belongs to" ClassB so specific methods / properties of ClassB should be manipulateable by ClassB. However those properties / methods should not be visible to anything outside of ClassA and ClassB. Possible? Oh, and BTW I want ClassA to be out in a class library, external to the project using it. -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Thu Mar 24 15:28:03 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 24 Mar 2011 16:28:03 -0400 Subject: [dba-VB] C# protected In-Reply-To: References: <4D8B8857.60604@colbyconsulting.com> Message-ID: <4D8BA953.4030602@colbyconsulting.com> ClassA can go in the class library, but classB is part of the application. John W. Colby www.ColbyConsulting.com On 3/24/2011 4:14 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Put ClassA and ClassB into a class library. > Use > > internal > > access modifier with their methods and properties to be seen within their > class library but not from the "outer world". > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 24 ????? 2011 ?. 21:07 > To: VBA > Subject: [dba-VB] C# protected > > I am trying to create a method of a ClassA which can be seen from the object > ClassB that instantiates ClassA but not by objectClassC which instantiates > ClassB. > > IOW ClassA "belongs to" ClassB so specific methods / properties of ClassB > should be manipulateable by ClassB. However those properties / methods > should not be visible to anything outside of ClassA and ClassB. > > Possible? > > Oh, and BTW I want ClassA to be out in a class library, external to the > project using it. > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Thu Mar 24 16:37:22 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 25 Mar 2011 00:37:22 +0300 Subject: [dba-VB] C# protected In-Reply-To: <4D8BA953.4030602@colbyconsulting.com> References: <4D8B8857.60604@colbyconsulting.com> <4D8BA953.4030602@colbyconsulting.com> Message-ID: Hi John -- Then use protected access modifier for methods and properties of Class A which is inherited by class B (IOW no need to instantiate class A explicitly)... Class C which will instantiate Class B will no see/have access to protected properties and methods of class A inherithed by class B. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 24 ????? 2011 ?. 23:28 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# protected ClassA can go in the class library, but classB is part of the application. John W. Colby www.ColbyConsulting.com On 3/24/2011 4:14 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Put ClassA and ClassB into a class library. > Use > > internal > > access modifier with their methods and properties to be seen within > their class library but not from the "outer world". > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 24 ????? 2011 ?. 21:07 > To: VBA > Subject: [dba-VB] C# protected > > I am trying to create a method of a ClassA which can be seen from the > object ClassB that instantiates ClassA but not by objectClassC which > instantiates ClassB. > > IOW ClassA "belongs to" ClassB so specific methods / properties of > ClassB should be manipulateable by ClassB. However those properties / > methods should not be visible to anything outside of ClassA and ClassB. > > Possible? > > Oh, and BTW I want ClassA to be out in a class library, external to > the project using it. > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 25 14:58:27 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 25 Mar 2011 15:58:27 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <3B9DE744CEA244829179A81CA818F92F@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com> <3B9DE744CEA244829179A81CA818F92F@nant> Message-ID: <4D8CF3E3.3050608@colbyconsulting.com> Shamil, Does this allow using a single "lock object" to lock various operations in a class. For example I want to lock a date flag variable while using a property to either get or set the variable. private Object thisLock = new Object(); public DateTime pFlagDte { get { lock (thisLock) { return dteFlag; } } set { lock (thisLock) { dteFlag = value; } } } The issue is not multiple threads trying to write to the date variable but rather one thread trying to read it while another is writing it. John W. Colby www.ColbyConsulting.com On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Do you you need add, remove and iterate a list to get an object or you need > to add, remove and get an object by its key? - if the latter - here is the > code you can use: > > private Dictionary _myObjects = new Dictionary MyObject>(); > private static object _myObjectsLocker = new object(); > public void Add(MyObject myObject) > { > lock (_myObjectsLocker) > { > _myObjects.Add(myObject.ID, myObject); > _keys.Add(myObject.ID); > } > } > public MyObject Remove(int id) > { > MyObject myobject = null; > lock (_myObjectsLocker) > { > myobject = _myObjects[id]; > _myObjects.Remove(id); > } > return myobject; > } > public MyObject this[int key] > { > get > { > MyObject myObject = null; > lock (_myObjectsLocker) > { > _myObjects.TryGetValue(key, out myObject); > } > return myObject; > } > } > > for the case of iteration just replace: > > _myObjects.TryGetValue(key, out myObject); > > with your iteration code. > > As for the sample code - changing qty of threads and changing total computer > workload (outer context for the test code) you can see how different could > be final results when heavy multi-threading is used. And even if it's not > heavy - it still can result in weird output if special multi-threading > coding technique aren't used... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 11 ????? 2011 ?. 1:27 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Thanks for the code but I think it is overkill for my needs so far, though I > will put it in the archives for when I need it. > > In my system so far I have (or can arrange to have ) only two threads > trying to access a list. > > Basically what I am doing is reading records out of sql server into a class > factory, and placing the class instances into a list. > > Another object with its own thread is coming by and looking at the objects > in the list to see if there is any work for it to do. If it finds an object > to work on, it grabs a pointer to that item in the list and stores it in a > pointer and releases the list. > > One of the ways I saw this handled was to create factory which creates a > brand new list object, grabs a pointer to the list, locks that and then > copies the objects in the list pointers to classes) into that new list, > unlocks the list and hands back a pointer to this brand new list. thus the > new list could be iterated at will. > > I am just trying to lock the list while one thread adds objects to the list > or another thread gets objects out of the list. > > So if I have object A which contains a list and is adding / deleting objects > to that list... > and object B which needs to iterate through the list looking for an object > of interest in the list, and grabbing a pointer to that object, there is no > way to do this without all of the code you wrote? > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Does that in fact lock the list to other objects doing the same thing >> (getting the list through that property)? >>>>> >> You have to lock/synchronize/serialize not the list itself but >> simultaneous access to the list from different threads. >> You'd better use special static object variable to implement locking. >> >> I will post sample code in my following posting here... >> >> Thank you. >> >> -- >> Shamil > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Mar 25 16:05:57 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 26 Mar 2011 00:05:57 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8CF3E3.3050608@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com> Message-ID: <3F0DA84A905C4947819126F43CFC39DB@nant> Hi John -- Your sample is a correct usage of locking - just use static thisLock variable private static Object thisLock = new Object() as non-static variable *is not* thread safe AFAIU. Yes, such locking can be done for various operations of that class - just make sure you'll not get dead-locked... Also use *one lock object* to lock *one resource* in a class - otherwise "deadlock" will become your "everyday guest"... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 25 ????? 2011 ?. 22:58 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, Does this allow using a single "lock object" to lock various operations in a class. For example I want to lock a date flag variable while using a property to either get or set the variable. private Object thisLock = new Object(); public DateTime pFlagDte { get { lock (thisLock) { return dteFlag; } } set { lock (thisLock) { dteFlag = value; } } } The issue is not multiple threads trying to write to the date variable but rather one thread trying to read it while another is writing it. John W. Colby www.ColbyConsulting.com On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Do you you need add, remove and iterate a list to get an object or you > need to add, remove and get an object by its key? - if the latter - > here is the code you can use: > > private Dictionary _myObjects = new Dictionary MyObject>(); > private static object _myObjectsLocker = new object(); public void > Add(MyObject myObject) { > lock (_myObjectsLocker) > { > _myObjects.Add(myObject.ID, myObject); > _keys.Add(myObject.ID); > } > } > public MyObject Remove(int id) > { > MyObject myobject = null; > lock (_myObjectsLocker) > { > myobject = _myObjects[id]; > _myObjects.Remove(id); > } > return myobject; > } > public MyObject this[int key] > { > get > { > MyObject myObject = null; > lock (_myObjectsLocker) > { > _myObjects.TryGetValue(key, out myObject); > } > return myObject; > } > } > > for the case of iteration just replace: > > _myObjects.TryGetValue(key, out myObject); > > with your iteration code. > > As for the sample code - changing qty of threads and changing total > computer workload (outer context for the test code) you can see how > different could be final results when heavy multi-threading is used. > And even if it's not heavy - it still can result in weird output if > special multi-threading coding technique aren't used... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 11 ????? 2011 ?. 1:27 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Thanks for the code but I think it is overkill for my needs so far, > though I will put it in the archives for when I need it. > > In my system so far I have (or can arrange to have ) only two threads > trying to access a list. > > Basically what I am doing is reading records out of sql server into a > class factory, and placing the class instances into a list. > > Another object with its own thread is coming by and looking at the > objects in the list to see if there is any work for it to do. If it > finds an object to work on, it grabs a pointer to that item in the > list and stores it in a pointer and releases the list. > > One of the ways I saw this handled was to create factory which creates > a brand new list object, grabs a pointer to the list, locks that and > then copies the objects in the list pointers to classes) into that new > list, unlocks the list and hands back a pointer to this brand new > list. thus the new list could be iterated at will. > > I am just trying to lock the list while one thread adds objects to the > list or another thread gets objects out of the list. > > So if I have object A which contains a list and is adding / deleting > objects to that list... > and object B which needs to iterate through the list looking for an > object of interest in the list, and grabbing a pointer to that object, > there is no way to do this without all of the code you wrote? > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Does that in fact lock the list to other objects doing the same thing >> (getting the list through that property)? >>>>> >> You have to lock/synchronize/serialize not the list itself but >> simultaneous access to the list from different threads. >> You'd better use special static object variable to implement locking. >> >> I will post sample code in my following posting here... >> >> Thank you. >> >> -- >> Shamil > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 25 16:38:19 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 25 Mar 2011 17:38:19 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <3F0DA84A905C4947819126F43CFC39DB@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com> <3F0DA84A905C4947819126F43CFC39DB@nant> Message-ID: <4D8D0B4B.9070004@colbyconsulting.com> When you say resource, you mean variable? IOW use private static Object LockBln = new Object(); private static Object LockDte = new Object(); and use the LockBln to lock the boolean object while using LockDte to lock the date object? John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - just > make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - otherwise > "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various operations in a > class. > > For example I want to lock a date flag variable while using a property to > either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable but > rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Do you you need add, remove and iterate a list to get an object or you >> need to add, remove and get an object by its key? - if the latter - >> here is the code you can use: >> >> private Dictionary _myObjects = new Dictionary> MyObject>(); >> private static object _myObjectsLocker = new object(); public void >> Add(MyObject myObject) { >> lock (_myObjectsLocker) >> { >> _myObjects.Add(myObject.ID, myObject); >> _keys.Add(myObject.ID); >> } >> } >> public MyObject Remove(int id) >> { >> MyObject myobject = null; >> lock (_myObjectsLocker) >> { >> myobject = _myObjects[id]; >> _myObjects.Remove(id); >> } >> return myobject; >> } >> public MyObject this[int key] >> { >> get >> { >> MyObject myObject = null; >> lock (_myObjectsLocker) >> { >> _myObjects.TryGetValue(key, out myObject); >> } >> return myObject; >> } >> } >> >> for the case of iteration just replace: >> >> _myObjects.TryGetValue(key, out myObject); >> >> with your iteration code. >> >> As for the sample code - changing qty of threads and changing total >> computer workload (outer context for the test code) you can see how >> different could be final results when heavy multi-threading is used. >> And even if it's not heavy - it still can result in weird output if >> special multi-threading coding technique aren't used... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 11 ????? 2011 ?. 1:27 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Thanks for the code but I think it is overkill for my needs so far, >> though I will put it in the archives for when I need it. >> >> In my system so far I have (or can arrange to have ) only two threads >> trying to access a list. >> >> Basically what I am doing is reading records out of sql server into a >> class factory, and placing the class instances into a list. >> >> Another object with its own thread is coming by and looking at the >> objects in the list to see if there is any work for it to do. If it >> finds an object to work on, it grabs a pointer to that item in the >> list and stores it in a pointer and releases the list. >> >> One of the ways I saw this handled was to create factory which creates >> a brand new list object, grabs a pointer to the list, locks that and >> then copies the objects in the list pointers to classes) into that new >> list, unlocks the list and hands back a pointer to this brand new >> list. thus the new list could be iterated at will. >> >> I am just trying to lock the list while one thread adds objects to the >> list or another thread gets objects out of the list. >> >> So if I have object A which contains a list and is adding / deleting >> objects to that list... >> and object B which needs to iterate through the list looking for an >> object of interest in the list, and grabbing a pointer to that object, >> there is no way to do this without all of the code you wrote? >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >>> Hi John -- >>> >>> <<< >>> Does that in fact lock the list to other objects doing the same thing >>> (getting the list through that property)? >>>>>> >>> You have to lock/synchronize/serialize not the list itself but >>> simultaneous access to the list from different threads. >>> You'd better use special static object variable to implement locking. >>> >>> I will post sample code in my following posting here... >>> >>> Thank you. >>> >>> -- >>> Shamil >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Mar 25 17:46:44 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 26 Mar 2011 01:46:44 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8D0B4B.9070004@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8D0B4B.9070004@colbyconsulting.com> Message-ID: Hi John -- By resource I mean access to a read/write resource - it could be not only a variable... If you need two or more class level variables (fields, properties) within one class instance to be locked then keep them in a class/structure and lock that class/structure instance... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 26 ????? 2011 ?. 0:38 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() When you say resource, you mean variable? IOW use private static Object LockBln = new Object(); private static Object LockDte = new Object(); and use the LockBln to lock the boolean object while using LockDte to lock the date object? John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - > just make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - > otherwise "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various > operations in a class. > > For example I want to lock a date flag variable while using a property > to either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable > but rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Do you you need add, remove and iterate a list to get an object or >> you need to add, remove and get an object by its key? - if the latter >> - here is the code you can use: >> >> private Dictionary _myObjects = new Dictionary> MyObject>(); >> private static object _myObjectsLocker = new object(); public void >> Add(MyObject myObject) { >> lock (_myObjectsLocker) >> { >> _myObjects.Add(myObject.ID, myObject); >> _keys.Add(myObject.ID); >> } >> } >> public MyObject Remove(int id) >> { >> MyObject myobject = null; >> lock (_myObjectsLocker) >> { >> myobject = _myObjects[id]; >> _myObjects.Remove(id); >> } >> return myobject; >> } >> public MyObject this[int key] >> { >> get >> { >> MyObject myObject = null; >> lock (_myObjectsLocker) >> { >> _myObjects.TryGetValue(key, out myObject); >> } >> return myObject; >> } >> } >> >> for the case of iteration just replace: >> >> _myObjects.TryGetValue(key, out myObject); >> >> with your iteration code. >> >> As for the sample code - changing qty of threads and changing total >> computer workload (outer context for the test code) you can see how >> different could be final results when heavy multi-threading is used. >> And even if it's not heavy - it still can result in weird output if >> special multi-threading coding technique aren't used... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 11 ????? 2011 ?. 1:27 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Thanks for the code but I think it is overkill for my needs so far, >> though I will put it in the archives for when I need it. >> >> In my system so far I have (or can arrange to have ) only two >> threads trying to access a list. >> >> Basically what I am doing is reading records out of sql server into a >> class factory, and placing the class instances into a list. >> >> Another object with its own thread is coming by and looking at the >> objects in the list to see if there is any work for it to do. If it >> finds an object to work on, it grabs a pointer to that item in the >> list and stores it in a pointer and releases the list. >> >> One of the ways I saw this handled was to create factory which >> creates a brand new list object, grabs a pointer to the list, locks >> that and then copies the objects in the list pointers to classes) >> into that new list, unlocks the list and hands back a pointer to this >> brand new list. thus the new list could be iterated at will. >> >> I am just trying to lock the list while one thread adds objects to >> the list or another thread gets objects out of the list. >> >> So if I have object A which contains a list and is adding / deleting >> objects to that list... >> and object B which needs to iterate through the list looking for an >> object of interest in the list, and grabbing a pointer to that >> object, there is no way to do this without all of the code you wrote? >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >>> Hi John -- >>> >>> <<< >>> Does that in fact lock the list to other objects doing the same >>> thing (getting the list through that property)? >>>>>> >>> You have to lock/synchronize/serialize not the list itself but >>> simultaneous access to the list from different threads. >>> You'd better use special static object variable to implement locking. >>> >>> I will post sample code in my following posting here... >>> >>> Thank you. >>> >>> -- >>> Shamil >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 25 21:57:22 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 25 Mar 2011 22:57:22 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8D0B4B.9070004@colbyconsulting.com> Message-ID: <4D8D5612.6050701@colbyconsulting.com> Shamil, I have processes that log results to flags. For example, make a database (log that it was made), build a table (log that it was built), pull umpteen million records in sorted order (log that it was filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build another chunk table (log that it was filled), BCP out (log that it was exported). The objective is to be able to sustain interruptions and pick up where we left off. These processes can take minutes (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). So each thing I do represents a step in the process and each step is logged in a field in a record in SQL server using a datetime. There are so many of these flags that I am trying to standardize the process by building a class that can be instantiated, filled with data and log itself to SQL Server by one thread and be checked by another thread. These flag class instances will be checked by multiple threads, each thread trying to decide whether it should be doing the next step because another thread has finished it's part. IOW if a file has been written to disk, then the next thread will write it to a VM for processing. If it moved to the VM the next thread will watch the VM's output directory for a file to pop out and move it back to a directory on the server. If the file (a couple of files actually) successfully copied back to the server staging then another thread will import it back into a chunk table in an input database. If the file successfully imported then another thread will... In general one thread will "own" the flag and use it to log its status and one other thread will be checking the status of the flag to determine that it can go to work on that work chunk. You get the picture. I am trying to build an entirely asynchronous highly threaded process which exports a huge table into multiple files, processes every file through a third party app and gets the results back into SQL Server. All while logging each and every step so that no piece can possibly be dropped at any stage, even if the server goes down (or the VM goes down). Eventually this process will run on my server 24/7. It has been working for some time but I am getting threading issues, and I need to work on the high level control so that all of the processes can cleanly start up and shut down and every stage can pick back up when the program restarts should a shutdown occur. A single database can be up to a hundred million records (the biggest so far), and the external program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour depending on many different things so that example will take 50 chunks and could take 40 to 50 hours to complete. It takes about 20 processing steps to handle each file from end to end. It needs to just work, and I need to be able to view status in a meaningful way. And I need to process that and a dozen other files every single month, automatically, with no manual intervention required. John W. Colby www.ColbyConsulting.com On 3/25/2011 6:46 PM, Shamil Salakhetdinov wrote: > Hi John -- > > By resource I mean access to a read/write resource - it could be not only a > variable... > > If you need two or more class level variables (fields, properties) within > one class instance to be locked then keep them in a class/structure and lock > that class/structure instance... > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Sat Mar 26 00:37:25 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 26 Mar 2011 08:37:25 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8D5612.6050701@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8D0B4B.9070004@colbyconsulting.com> <4D8D5612.6050701@colbyconsulting.com> Message-ID: <4D87466503764D98871B609159033CB6@nant> John -- But you can try to use several instances of .NET Framework Queue class http://msdn.microsoft.com/en-us/library/system.collections.queue.aspx ? As far as I understand you have a set of sequential tasks to be performed on every (input/start) chunk so you can make a dedicated Queue instance for every task, and that Queue instance will be handled by dedicated "processor" classes: - TaskA Queue items will be processed by TaskAQueueProcessor, - TaksB .... by TaskBQueueProcessor, -- - TaskZ... by TaskZQueueProcessor There could be many such TaskA(B, ....Z)QueueProcessor instances "floating around" looking for an item from their dedicated tasks' queue to process: when they find it, they pick it up, process and put into another Queue based on "workflow plan"... You can also try to consider using Windows Workflow Foundation(WF) if that wouldn't be an "overkill" for your case... http://msdn.microsoft.com/en-us/netframework/wf-screencasts.aspx#introwf4 Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 26 ????? 2011 ?. 5:57 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, I have processes that log results to flags. For example, make a database (log that it was made), build a table (log that it was built), pull umpteen million records in sorted order (log that it was filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build another chunk table (log that it was filled), BCP out (log that it was exported). The objective is to be able to sustain interruptions and pick up where we left off. These processes can take minutes (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). So each thing I do represents a step in the process and each step is logged in a field in a record in SQL server using a datetime. There are so many of these flags that I am trying to standardize the process by building a class that can be instantiated, filled with data and log itself to SQL Server by one thread and be checked by another thread. These flag class instances will be checked by multiple threads, each thread trying to decide whether it should be doing the next step because another thread has finished it's part. IOW if a file has been written to disk, then the next thread will write it to a VM for processing. If it moved to the VM the next thread will watch the VM's output directory for a file to pop out and move it back to a directory on the server. If the file (a couple of files actually) successfully copied back to the server staging then another thread will import it back into a chunk table in an input database. If the file successfully imported then another thread will... In general one thread will "own" the flag and use it to log its status and one other thread will be checking the status of the flag to determine that it can go to work on that work chunk. You get the picture. I am trying to build an entirely asynchronous highly threaded process which exports a huge table into multiple files, processes every file through a third party app and gets the results back into SQL Server. All while logging each and every step so that no piece can possibly be dropped at any stage, even if the server goes down (or the VM goes down). Eventually this process will run on my server 24/7. It has been working for some time but I am getting threading issues, and I need to work on the high level control so that all of the processes can cleanly start up and shut down and every stage can pick back up when the program restarts should a shutdown occur. A single database can be up to a hundred million records (the biggest so far), and the external program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour depending on many different things so that example will take 50 chunks and could take 40 to 50 hours to complete. It takes about 20 processing steps to handle each file from end to end. It needs to just work, and I need to be able to view status in a meaningful way. And I need to process that and a dozen other files every single month, automatically, with no manual intervention required. John W. Colby www.ColbyConsulting.com On 3/25/2011 6:46 PM, Shamil Salakhetdinov wrote: > Hi John -- > > By resource I mean access to a read/write resource - it could be not > only a variable... > > If you need two or more class level variables (fields, properties) > within one class instance to be locked then keep them in a > class/structure and lock that class/structure instance... > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From gustav at cactus.dk Sat Mar 26 07:41:44 2011 From: gustav at cactus.dk (Gustav Brock) Date: Sat, 26 Mar 2011 13:41:44 +0100 Subject: [dba-VB] c# lock() Message-ID: Hi John My first thought was Windows Workflow Foundation(WF) but, as Shamil mentions, it may be overkill. On the other hand it is exactly for controlling scenarios where "if task 1 is done, start task 2 and task 3, wait for task 2 and task 3 to finish, check something, then start task 4, etc.". However, that reminded me about a series of articles about the Task Parallel Library of .Net: http://www.codeproject.com/KB/cs/TPL1.aspx which I found very interesting, though I haven't had any use for it yet. Among other topics it discusses carefully canceling and error handling which I guess is quite important for your purpose. /gustav >>> jwcolby at colbyconsulting.com 26-03-2011 03:57 >>> Shamil, I have processes that log results to flags. For example, make a database (log that it was made), build a table (log that it was built), pull umpteen million records in sorted order (log that it was filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build another chunk table (log that it was filled), BCP out (log that it was exported). The objective is to be able to sustain interruptions and pick up where we left off. These processes can take minutes (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). So each thing I do represents a step in the process and each step is logged in a field in a record in SQL server using a datetime. There are so many of these flags that I am trying to standardize the process by building a class that can be instantiated, filled with data and log itself to SQL Server by one thread and be checked by another thread. These flag class instances will be checked by multiple threads, each thread trying to decide whether it should be doing the next step because another thread has finished it's part. IOW if a file has been written to disk, then the next thread will write it to a VM for processing. If it moved to the VM the next thread will watch the VM's output directory for a file to pop out and move it back to a directory on the server. If the file (a couple of files actually) successfully copied back to the server staging then another thread will import it back into a chunk table in an input database. If the file successfully imported then another thread will... In general one thread will "own" the flag and use it to log its status and one other thread will be checking the status of the flag to determine that it can go to work on that work chunk. You get the picture. I am trying to build an entirely asynchronous highly threaded process which exports a huge table into multiple files, processes every file through a third party app and gets the results back into SQL Server. All while logging each and every step so that no piece can possibly be dropped at any stage, even if the server goes down (or the VM goes down). Eventually this process will run on my server 24/7. It has been working for some time but I am getting threading issues, and I need to work on the high level control so that all of the processes can cleanly start up and shut down and every stage can pick back up when the program restarts should a shutdown occur. A single database can be up to a hundred million records (the biggest so far), and the external program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour depending on many different things so that example will take 50 chunks and could take 40 to 50 hours to complete. It takes about 20 processing steps to handle each file from end to end. It needs to just work, and I need to be able to view status in a meaningful way. And I need to process that and a dozen other files every single month, automatically, with no manual intervention required. John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 26 07:44:54 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 08:44:54 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: Message-ID: <4D8DDFC6.4040004@colbyconsulting.com> Well... I haven't migrated to VS2010 yet. I tried it about 6 months ago and it was universally a pig on my systems. I thought I would wait until MS polished it a bit. John W. Colby www.ColbyConsulting.com On 3/26/2011 8:41 AM, Gustav Brock wrote: > Hi John > > My first thought was Windows Workflow Foundation(WF) but, as Shamil mentions, it may be overkill. On the other hand it is exactly for controlling scenarios where "if task 1 is done, start task 2 and task 3, wait for task 2 and task 3 to finish, check something, then start task 4, etc.". > > However, that reminded me about a series of articles about the Task Parallel Library of .Net: > > http://www.codeproject.com/KB/cs/TPL1.aspx > > which I found very interesting, though I haven't had any use for it yet. Among other topics it discusses carefully canceling and error handling which I guess is quite important for your purpose. > > /gustav > > >>>> jwcolby at colbyconsulting.com 26-03-2011 03:57>>> > Shamil, > > I have processes that log results to flags. For example, make a database (log that it was made), > build a table (log that it was built), pull umpteen million records in sorted order (log that it was > filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build > another chunk table (log that it was filled), BCP out (log that it was exported). The objective is > to be able to sustain interruptions and pick up where we left off. These processes can take minutes > (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). > > So each thing I do represents a step in the process and each step is logged in a field in a record > in SQL server using a datetime. There are so many of these flags that I am trying to standardize > the process by building a class that can be instantiated, filled with data and log itself to SQL > Server by one thread and be checked by another thread. > > These flag class instances will be checked by multiple threads, each thread trying to decide whether > it should be doing the next step because another thread has finished it's part. IOW if a file has > been written to disk, then the next thread will write it to a VM for processing. If it moved to the > VM the next thread will watch the VM's output directory for a file to pop out and move it back to a > directory on the server. If the file (a couple of files actually) successfully copied back to the > server staging then another thread will import it back into a chunk table in an input database. If > the file successfully imported then another thread will... In general one thread will "own" the > flag and use it to log its status and one other thread will be checking the status of the flag to > determine that it can go to work on that work chunk. > > You get the picture. > > I am trying to build an entirely asynchronous highly threaded process which exports a huge table > into multiple files, processes every file through a third party app and gets the results back into > SQL Server. All while logging each and every step so that no piece can possibly be dropped at any > stage, even if the server goes down (or the VM goes down). Eventually this process will run on my > server 24/7. > > It has been working for some time but I am getting threading issues, and I need to work on the high > level control so that all of the processes can cleanly start up and shut down and every stage can > pick back up when the program restarts should a shutdown occur. > > A single database can be up to a hundred million records (the biggest so far), and the external > program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour > depending on many different things so that example will take 50 chunks and could take 40 to 50 hours > to complete. It takes about 20 processing steps to handle each file from end to end. It needs to > just work, and I need to be able to view status in a meaningful way. And I need to process that and > a dozen other files every single month, automatically, with no manual intervention required. > > > > John W. Colby > www.ColbyConsulting.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From gustav at cactus.dk Sat Mar 26 08:08:10 2011 From: gustav at cactus.dk (Gustav Brock) Date: Sat, 26 Mar 2011 14:08:10 +0100 Subject: [dba-VB] c# lock() Message-ID: Hi John Oh, I recall your trouble now that you mention it. Strange, as it runs fine on my antique Pentium 4 IBM IntelliStation with 2 GB ram. However, VS2010 _has_ been polished: The Service Pack 1 was released a few weeks ago. /gustav >>> jwcolby at colbyconsulting.com 26-03-2011 13:44 >>> Well... I haven't migrated to VS2010 yet. I tried it about 6 months ago and it was universally a pig on my systems. I thought I would wait until MS polished it a bit. John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 26 08:26:26 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 09:26:26 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: Message-ID: <4D8DE982.7060707@colbyconsulting.com> When I had my speed issues I Googled around and found tons with speed issues, and tons without. There are entire threads bitterly complaining about what a dog it is. There was no explanations of when or why. Mine would take forever to do anything. I will definitely try again now that SP1 is out. John W. Colby www.ColbyConsulting.com On 3/26/2011 9:08 AM, Gustav Brock wrote: > Hi John > > Oh, I recall your trouble now that you mention it. Strange, as it runs fine on my antique Pentium 4 IBM IntelliStation with 2 GB ram. > However, VS2010 _has_ been polished: The Service Pack 1 was released a few weeks ago. > > /gustav > > >>>> jwcolby at colbyconsulting.com 26-03-2011 13:44>>> > Well... I haven't migrated to VS2010 yet. > > I tried it about 6 months ago and it was universally a pig on my systems. I thought I would wait > until MS polished it a bit. > > John W. Colby > www.ColbyConsulting.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 26 16:53:15 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 17:53:15 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <3F0DA84A905C4947819126F43CFC39DB@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com> <3F0DA84A905C4947819126F43CFC39DB@nant> Message-ID: <4D8E604B.5040001@colbyconsulting.com> Shamil, I am trying to build a base class which contains a set of lock objects and some standard variables. Having defined the lock objects static per your code below, the derived class cannot access the lock objects even though I set them protected: protected static Object LockBln = new Object(); In the derived class when I try to access it: lock(this.LockBln) { } I get a compile error: "Member cannot be accesses with an instance reference; Qualify it with a type name instead." What does this mean? Reading on the internet it says that static properties are shared between all instances of the class. I don't see how I can share a property between 50 instances and then use a lock(MyStaticProperty) {}. I'm so confused... John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - just > make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - otherwise > "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various operations in a > class. > > For example I want to lock a date flag variable while using a property to > either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable but > rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sat Mar 26 17:27:17 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sun, 27 Mar 2011 01:27:17 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8E604B.5040001@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8E604B.5040001@colbyconsulting.com> Message-ID: <00D0A03B172743B6868A8BE736F7E560@nant> John -- Let's define your target (application) system architecture on conceptual level - I guess that using "micro-level locking" you're trying (?) to achieve isn't needed... 1) you have a huge input data set; 2) you split input data set into X chunks; 3) every chunk gets processed using the same several - Y - steps; 4) on completion of step Yn input/semi-processed chunk Xn is submitted to the Y(n+1) processor class; 5) when all X chunks are processed in Y steps they get collected into a final "huge" output set to produce Z reports. That's it from conceptual high level point of view? Something important missing? Too simplified to be true? John, please take into account that I can't reply promptly here - it's weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to sleep, and then I have quite some work to do starting Monday - so I expect this thread will be supported by other AccessD/dba-VB members... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 0:53 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, I am trying to build a base class which contains a set of lock objects and some standard variables. Having defined the lock objects static per your code below, the derived class cannot access the lock objects even though I set them protected: protected static Object LockBln = new Object(); In the derived class when I try to access it: lock(this.LockBln) { } I get a compile error: "Member cannot be accesses with an instance reference; Qualify it with a type name instead." What does this mean? Reading on the internet it says that static properties are shared between all instances of the class. I don't see how I can share a property between 50 instances and then use a lock(MyStaticProperty) {}. I'm so confused... John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - > just make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - > otherwise "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various > operations in a class. > > For example I want to lock a date flag variable while using a property > to either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable > but rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Mar 26 18:31:31 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 19:31:31 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <00D0A03B172743B6868A8BE736F7E560@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant> Message-ID: <4D8E7753.8090006@colbyconsulting.com> I'm just trying to get a handle on locking things (in general). I am seeing things that say "just lock the object itself", and then other things that say "use a static object (the way you were showing me). If locking the object itself works, why do the static thing? John W. Colby www.ColbyConsulting.com On 3/26/2011 6:27 PM, Shamil Salakhetdinov wrote: > John -- > > Let's define your target (application) system architecture on conceptual > level - I guess that using "micro-level locking" you're trying (?) to > achieve isn't needed... > > 1) you have a huge input data set; > 2) you split input data set into X chunks; > 3) every chunk gets processed using the same several - Y - steps; > 4) on completion of step Yn input/semi-processed chunk Xn is submitted to > the Y(n+1) processor class; > 5) when all X chunks are processed in Y steps they get collected into a > final "huge" output set to produce Z reports. > > That's it from conceptual high level point of view? > Something important missing? > Too simplified to be true? > > John, please take into account that I can't reply promptly here - it's > weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to sleep, > and then I have quite some work to do starting Monday - so I expect this > thread will be supported by other AccessD/dba-VB members... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 27 ????? 2011 ?. 0:53 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > I am trying to build a base class which contains a set of lock objects and > some standard variables. > Having defined the lock objects static per your code below, the derived > class cannot access the lock objects even though I set them protected: > > protected static Object LockBln = new Object(); > > In the derived class when I try to access it: > > lock(this.LockBln) > { > } > > I get a compile error: > > "Member cannot be accesses with an instance reference; Qualify it with a > type name instead." > > What does this mean? > > Reading on the internet it says that static properties are shared between > all instances of the class. I don't see how I can share a property between > 50 instances and then use a > lock(MyStaticProperty) {}. > > I'm so confused... > > John W. Colby > www.ColbyConsulting.com > > On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Your sample is a correct usage of locking - just use static thisLock >> variable >> >> private static Object thisLock = new Object() >> >> as non-static variable *is not* thread safe AFAIU. >> >> Yes, such locking can be done for various operations of that class - >> just make sure you'll not get dead-locked... >> Also use *one lock object* to lock *one resource* in a class - >> otherwise "deadlock" will become your "everyday guest"... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 25 ????? 2011 ?. 22:58 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Does this allow using a single "lock object" to lock various >> operations in a class. >> >> For example I want to lock a date flag variable while using a property >> to either get or set the variable. >> >> private Object thisLock = new Object(); >> >> public DateTime pFlagDte { >> get { >> lock (thisLock) >> { >> return dteFlag; >> } >> } >> set >> { >> lock (thisLock) >> { >> dteFlag = value; >> } >> } >> } >> >> The issue is not multiple threads trying to write to the date variable >> but rather one thread trying to read it while another is writing it. >> >> John W. Colby >> www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Mar 26 19:08:10 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sun, 27 Mar 2011 04:08:10 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <00D0A03B172743B6868A8BE736F7E560@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant> Message-ID: <0CD7EF3475CB43C69334942027697B63@nant> John -- I'm falling asleep here now - would that be correct assumption that you can generalize your global task like that - you have, say: - split input data into 50 Chunks; - every chunk should be processed in 10 steps; - every step's average processing time is 100 minutes. Then in total it is needed 50*10*100 = 50,000 minutes or ~833.(3) hours to process all your input data sequentially, chunk after chunk step after step, say, you don't have enough computer power to process some chunks in parallel, and you can't process the whole not split input data as it's too large for your system... But you have "50 small computers" and you can process every of 50 input chunks in parallel - then you'll need just 10*100 = 1,000 minutes (~16.67 hours) to complete the job. I assume that all the chunks processing is independent - then you can: 1) Create Scheduler class instance; 2) Schedule class creates 10 TaskQueue class instances - 1 queue for every processing step type; 3) Scheduler class defines splitting criteria to split input data into 50 chunks; 4) Schedule defines completion criteria - when all 50 chunks get collected in 11th FinalQueue; 5) Schedule seeds the first InputChunkSplitTaskQueue with 50 SplitTasksDescriptors; 4) From time to time - say every half a minute Scheduler class instance creates 500 worker classes in 500 threads (500 worker classes => 50 chunks * 10 steps.), which "hunt for the job" in their dedicate Queues: if they find Job they work on it, if not - they "die"... 5) When a worker class completes processing its job it queues its "task results" into the next step Queue, if a worker class fails then it puts its incomplete task descriptor back into its dedicated queue and "dies", or that could be Scheduler's (or SchedulerAssitant's) task job to find failed workers' "trap" and to resubmit their failed work to the proper queue according to the custom workflow descriptors attached to every chunk... 500 worker classes is an overkill as only 50 worker classes can have job every time but that seems to be an easy "brute force" and "lazy parallel programming" approach - and it should work... Or you can make worker class instances production processes smarter: Scheduler class can start special thread for WorkerClassGenerator instance, which will monitor all the 10 TaskQueues, and if it finds an item in a Queque, it will pick it up, it will create corresponding Worker class in parallel thread and it will pass WorkItem to the Worker class fro processing... When that described above approach will work there then you can easily(?) scale it splitting your (constant size) input data into 100 chunks, and then if every chunk can be processes in half time - in average (50 minutes) - then all job can be completed in 500 minutes = ~8.33(3) hours etc... Please correct me if I oversimplified your application business area... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 27 ????? 2011 ?. 1:27 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] c# lock() John -- Let's define your target (application) system architecture on conceptual level - I guess that using "micro-level locking" you're trying (?) to achieve isn't needed... 1) you have a huge input data set; 2) you split input data set into X chunks; 3) every chunk gets processed using the same several - Y - steps; 4) on completion of step Yn input/semi-processed chunk Xn is submitted to the Y(n+1) processor class; 5) when all X chunks are processed in Y steps they get collected into a final "huge" output set to produce Z reports. That's it from conceptual high level point of view? Something important missing? Too simplified to be true? John, please take into account that I can't reply promptly here - it's weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to sleep, and then I have quite some work to do starting Monday - so I expect this thread will be supported by other AccessD/dba-VB members... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 0:53 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, I am trying to build a base class which contains a set of lock objects and some standard variables. Having defined the lock objects static per your code below, the derived class cannot access the lock objects even though I set them protected: protected static Object LockBln = new Object(); In the derived class when I try to access it: lock(this.LockBln) { } I get a compile error: "Member cannot be accesses with an instance reference; Qualify it with a type name instead." What does this mean? Reading on the internet it says that static properties are shared between all instances of the class. I don't see how I can share a property between 50 instances and then use a lock(MyStaticProperty) {}. I'm so confused... John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - > just make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - > otherwise "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various > operations in a class. > > For example I want to lock a date flag variable while using a property > to either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable > but rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Sat Mar 26 19:38:28 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sun, 27 Mar 2011 04:38:28 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <4D8E7753.8090006@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8E604B.5040001@colbyconsulting.com><00D0A03B172743B6868A8BE736F7E560@nant> <4D8E7753.8090006@colbyconsulting.com> Message-ID: Hi John -- <<< If locking the object itself works, why do the static thing? >>> Static is used to lock shared resources - in your case there are no such resources at all? Log writer is usually a static shared resource... And yes, you can use your locking code to lock parallel access to your class instances' local variables: >> private Object thisLock = new Object(); >> >> public DateTime pFlagDte { >> get { >> lock (thisLock) >> { >> return dteFlag; >> } >> } >> set >> { >> lock (thisLock) >> { >> dteFlag = value; >> } >> } >> } But as I noted I'm not sure if that "micro-level locking" is needed at all (and you mentioned you have several such shared variables within a class instance) - using Queues seems to be more simple and (thread-)safe approach without almost any custom locking schemes.... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 3:32 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() I'm just trying to get a handle on locking things (in general). I am seeing things that say "just lock the object itself", and then other things that say "use a static object (the way you were showing me). If locking the object itself works, why do the static thing? John W. Colby www.ColbyConsulting.com On 3/26/2011 6:27 PM, Shamil Salakhetdinov wrote: > John -- > > Let's define your target (application) system architecture on > conceptual level - I guess that using "micro-level locking" you're > trying (?) to achieve isn't needed... > > 1) you have a huge input data set; > 2) you split input data set into X chunks; > 3) every chunk gets processed using the same several - Y - steps; > 4) on completion of step Yn input/semi-processed chunk Xn is submitted > to the Y(n+1) processor class; > 5) when all X chunks are processed in Y steps they get collected into > a final "huge" output set to produce Z reports. > > That's it from conceptual high level point of view? > Something important missing? > Too simplified to be true? > > John, please take into account that I can't reply promptly here - it's > weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to > sleep, and then I have quite some work to do starting Monday - so I > expect this thread will be supported by other AccessD/dba-VB members... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 27 ????? 2011 ?. 0:53 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > I am trying to build a base class which contains a set of lock objects > and some standard variables. > Having defined the lock objects static per your code below, the > derived class cannot access the lock objects even though I set them protected: > > protected static Object LockBln = new Object(); > > In the derived class when I try to access it: > > lock(this.LockBln) > { > } > > I get a compile error: > > "Member cannot be accesses with an instance reference; Qualify it with > a type name instead." > > What does this mean? > > Reading on the internet it says that static properties are shared > between all instances of the class. I don't see how I can share a > property between 50 instances and then use a > lock(MyStaticProperty) {}. > > I'm so confused... > > John W. Colby > www.ColbyConsulting.com > > On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Your sample is a correct usage of locking - just use static thisLock >> variable >> >> private static Object thisLock = new Object() >> >> as non-static variable *is not* thread safe AFAIU. >> >> Yes, such locking can be done for various operations of that class - >> just make sure you'll not get dead-locked... >> Also use *one lock object* to lock *one resource* in a class - >> otherwise "deadlock" will become your "everyday guest"... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 25 ????? 2011 ?. 22:58 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Does this allow using a single "lock object" to lock various >> operations in a class. >> >> For example I want to lock a date flag variable while using a >> property to either get or set the variable. >> >> private Object thisLock = new Object(); >> >> public DateTime pFlagDte { >> get { >> lock (thisLock) >> { >> return dteFlag; >> } >> } >> set >> { >> lock (thisLock) >> { >> dteFlag = value; >> } >> } >> } >> >> The issue is not multiple threads trying to write to the date >> variable but rather one thread trying to read it while another is writing it. >> >> John W. Colby >> www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 26 23:12:20 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 27 Mar 2011 00:12:20 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <0CD7EF3475CB43C69334942027697B63@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant> <0CD7EF3475CB43C69334942027697B63@nant> Message-ID: <4D8EB924.5080708@colbyconsulting.com> Well... In programming you can do any task 50 different ways. The way I did it is to create a supervisor and ProcessFile table in SQL Server. The supervisor represents a table in SQL Server that contains an address table which needs to be processed. The ProcessFile records are child to tblSupervisor and represent chunks (files). The supervisor record stores all of the high level flags about the database (address table) being processed. The ProcessFile contains all of the flags about each chunk being processed. I have four different high level "stages" as I call them that a supervisor goes through. AccuzipOut - table to csv files in staging out (stage 1), Staging to VM (stage 2), VM to Staging in (stage 3, Staging in to SQL Server (stage 4) 1) Export all chunks (files) to a SQL Out staging directory. Each chunk ends up being a CSV file with PKID, FName, LName, Addr, City, St, Zip5, Zip4. The ProcessFile needs to be sorted by Zip5 / Zip4 order because the third party application that processes these files is immensely faster if the file is presorted. 2) One by one, copy each chunk file to a virtual machine - AccuzipIn directory. The Accuzip virtual machine runs on a Windows 2008 server running Hyper-V manager. I currently run 3 different VMs, only one of which runs Accuzip. For awhile I actually ran 3 VMs running Accuzip, and I am designing such that I can add additional Accuzip VMs in the future. What this means is that each file could go to any valid Accuzip VM, and when the chunk file is placed on a VM it has to log which VM it went to because... 3) The Accuzip software automatically senses and processes the file in the InputTo directory. A file of 2 million records takes about 25 minutes to CASS process locally to the VM, then is uploaded to a server in California for further NCOA processing. The server in California belongs to Accuzip and they stage big files so it can take awhile before it even begins to process. Once it processes, the NCOA server sends a small file back to my VM where Address change information is integrated back in to the local Accuzip database file (foxpro). After that it is exported back to CSV with a bunch of additional fields and is placed in the Accuzip OutputFrom directory along with a PDF which contains the CASS / NCOA statistics. This PDF is a "proof of processing" document that I have to keep forever, and I have to send to the client. I have a Directory Watcher which is watching the OutputFrom directory. When *all* of the resulting Accuzip output files are in the OutputFrom directory, my Directory Watcher event triggers my program to copy all of these files back to the SQL Server In staging directory. 4) Once all of the Accuzip files for a single chunk are back on the SQL Server In directory, my application BCPs the CSV into a input chunk table. From there decisions are made whether I need to import every address back in or only the moves. In any event some or all of the address records (with the additional fields from Accuzip) are pulled into an AZData table back in the database that the address data originally came from. So... 1) A supervisor object creates a clsDbAZOut instance. 2) clsDBAZOut creates an actual database dbXyzAZOut temporary database. 3) clsDBAZOut pulls every record needing to be accuzipped into a table with a super PK, sorted on Zip5/4 4) clsDBAZOut counts the records in tblAZOut, and starts to build clsProcessFile instances which holds the information about the chunk processing. clsProcessFile can write its info back to the ProcessFile records on SQL Server. 5) clsDbAZOut has a thread which builds tblChunk, gets ChunkSize records into the chunk table and BCPs tblChunk to CSV files on a staging out directory. 6) clsDbAZOut BCPs the chunk to file 7) At this point the clsProcessFile is handed off to clsVM. clsSupervisor has a strongly typed dictionary which stores the clsProcessFile instances created by clsDbAZOut. clsSupervisor can read ProcessFile instances from SQL Server into a clsProcessFile factory or clsDbAZOut can use a class factory to create new ones. In all cases they are stored in the dictionary in clsSupervisor. clsProcessfile has flags which document that: a) The chunk table was created b) The chunk table exported to file c) The chunk went to a VM (and which one) d) The chunk file was (eventually) found in the Accuzip OutputFrom directory e) The resulting AZData file (and PDF) were moved from the VM back to SQL ServerIn staging f) The file in SQL Server In staging was pulled back in to a chunk table in a second database dbXyzAZIn temporary database g) That the chunk table was imported back into the original database. As you can see, clsProcessFile is the center of the universe for a single chunk. clsSupervisor is the center of the universe for the entire process. It is a bit difficult to discuss how the entire program works in a single email. A clsManager loads one or a dozen clsSupervisor instances. clsManager loads clsVM which supervises the (a) virtual machine which runs a third party process. clsVM moves files from SQL Staging Out into the VM one chunk at a time, updating clsProcessTable flags. clsVM moves files from the the VM back to the SQL Staging IN, updating flags. The VM may not be available, or the Accuzip program may be down etc. clsSupervisor loads one to N clsProcessTable instances. clsSupervisor loads a clsDBExp and clsDBImp. clsDBExp creates new clsProcessTable instances, one for each chunk it exports - storing them in clsSupervisor. clsDBExp supervises the export process. clsDBImp supervises the import process getting the chunks back into SQL Server into a temp database, and from there into the live database. I am trying to design the system such that it is asynchronous. Stage1, stage 2/3 and stage 4 can all do their thing independently of each other. Each step of each stage is logged (dateTime flags written) so that any piece can just pick up where it left off. That means any Supervisor and any ProcessFile (chunk) can pick up from any point. In the meantime, the clsManager is watching the tblSupervisor for supervisor records ready to process - a thread. clsSupervisor is watching for ProcessTable records and dispatching them as required to the clsDBExp, clsDBImp or clsVM - a thread in each supervisor. clsDBExp is processing chunks, creating CSV files - a thread. clsVM is watching for clsProcessTable instances ready to go to the VM. It is also watching the output directory and sending files back to the SQL Server - a thread and two file watchers (and two events). clsDBImp is watching for clsProcessTable instances ready to import back into SQL Server - a thread. Most of these threads are writing flags (as they finish their specific process) and reading flags in clsProcessTable to decide what chunk has to be processed in which class. Oh, and I have three list controls on a form which the various classes write to to display the status for Stage1, Stage 2/3 and Stage 4. So pretty much a single supervisor is processing at a time, exporting files, sending them to the vm, importing them back in etc. It is possible for an order supervisor to cut in at the head of the line however. Not a simple task co-ordinating all of this stuff. I used to co-ordinate it all manually! ;) John W. Colby www.ColbyConsulting.com On 3/26/2011 8:08 PM, Shamil Salakhetdinov wrote: > John -- > > I'm falling asleep here now - would that be correct assumption that you can > generalize your global task like that - you have, say: > > - split input data into 50 Chunks; > - every chunk should be processed in 10 steps; > - every step's average processing time is 100 minutes. > > Then in total it is needed 50*10*100 = 50,000 minutes or ~833.(3) hours to > process all your input data sequentially, chunk after chunk step after step, > say, you don't have enough computer power to process some chunks in > parallel, and you can't process the whole not split input data as it's too > large for your system... > > But you have "50 small computers" and you can process every of 50 input > chunks in parallel - then you'll need just 10*100 = 1,000 minutes (~16.67 > hours) to complete the job. > > I assume that all the chunks processing is independent - then you can: > > 1) Create Scheduler class instance; > 2) Schedule class creates 10 TaskQueue class instances - 1 queue for every > processing step type; > 3) Scheduler class defines splitting criteria to split input data into 50 > chunks; > 4) Schedule defines completion criteria - when all 50 chunks get collected > in 11th FinalQueue; > 5) Schedule seeds the first InputChunkSplitTaskQueue with 50 > SplitTasksDescriptors; > 4) From time to time - say every half a minute Scheduler class instance > creates 500 worker classes in 500 threads (500 worker classes => 50 chunks * > 10 steps.), which "hunt for the job" in their dedicate Queues: if they find > Job they work on it, if not - they "die"... > 5) When a worker class completes processing its job it queues its "task > results" into the next step Queue, if a worker class fails then it puts its > incomplete task descriptor back into its dedicated queue and "dies", or that > could be Scheduler's (or SchedulerAssitant's) task job to find failed > workers' "trap" and to resubmit their failed work to the proper queue > according to the custom workflow descriptors attached to every chunk... > > 500 worker classes is an overkill as only 50 worker classes can have job > every time but that seems to be an easy "brute force" and "lazy parallel > programming" approach - and it should work... > Or you can make worker class instances production processes smarter: > Scheduler class can start special thread for WorkerClassGenerator instance, > which will monitor all the 10 TaskQueues, and if it finds an item in a > Queque, it will pick it up, it will create corresponding Worker class in > parallel thread and it will pass WorkItem to the Worker class fro > processing... > > When that described above approach will work there then you can easily(?) > scale it splitting your (constant size) input data into 100 chunks, and then > if every chunk can be processes in half time - in average (50 minutes) - > then all job can be completed in 500 minutes = ~8.33(3) hours etc... > > Please correct me if I oversimplified your application business area... > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Sun Mar 27 19:08:25 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 28 Mar 2011 04:08:25 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <4D8EB924.5080708@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> Message-ID: Hi John -- Yes, I see: you tend to build a multi-threaded application/utility system with centralized control - and I tend to build an anarchical free-market model based application/utility system :) Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 8:12 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Well... In programming you can do any task 50 different ways. The way I did it is to create a supervisor and ProcessFile table in SQL Server. The supervisor represents a table in SQL Server that contains an address table which needs to be processed. The ProcessFile records are child to tblSupervisor and represent chunks (files). The supervisor record stores all of the high level flags about the database (address table) being processed. The ProcessFile contains all of the flags about each chunk being processed. I have four different high level "stages" as I call them that a supervisor goes through. AccuzipOut - table to csv files in staging out (stage 1), Staging to VM (stage 2), VM to Staging in (stage 3, Staging in to SQL Server (stage 4) 1) Export all chunks (files) to a SQL Out staging directory. Each chunk ends up being a CSV file with PKID, FName, LName, Addr, City, St, Zip5, Zip4. The ProcessFile needs to be sorted by Zip5 / Zip4 order because the third party application that processes these files is immensely faster if the file is presorted. 2) One by one, copy each chunk file to a virtual machine - AccuzipIn directory. The Accuzip virtual machine runs on a Windows 2008 server running Hyper-V manager. I currently run 3 different VMs, only one of which runs Accuzip. For awhile I actually ran 3 VMs running Accuzip, and I am designing such that I can add additional Accuzip VMs in the future. What this means is that each file could go to any valid Accuzip VM, and when the chunk file is placed on a VM it has to log which VM it went to because... 3) The Accuzip software automatically senses and processes the file in the InputTo directory. A file of 2 million records takes about 25 minutes to CASS process locally to the VM, then is uploaded to a server in California for further NCOA processing. The server in California belongs to Accuzip and they stage big files so it can take awhile before it even begins to process. Once it processes, the NCOA server sends a small file back to my VM where Address change information is integrated back in to the local Accuzip database file (foxpro). After that it is exported back to CSV with a bunch of additional fields and is placed in the Accuzip OutputFrom directory along with a PDF which contains the CASS / NCOA statistics. This PDF is a "proof of processing" document that I have to keep forever, and I have to send to the client. I have a Directory Watcher which is watching the OutputFrom directory. When *all* of the resulting Accuzip output files are in the OutputFrom directory, my Directory Watcher event triggers my program to copy all of these files back to the SQL Server In staging directory. 4) Once all of the Accuzip files for a single chunk are back on the SQL Server In directory, my application BCPs the CSV into a input chunk table. >From there decisions are made whether I need to import every address back in or only the moves. In any event some or all of the address records (with the additional fields from Accuzip) are pulled into an AZData table back in the database that the address data originally came from. So... 1) A supervisor object creates a clsDbAZOut instance. 2) clsDBAZOut creates an actual database dbXyzAZOut temporary database. 3) clsDBAZOut pulls every record needing to be accuzipped into a table with a super PK, sorted on Zip5/4 4) clsDBAZOut counts the records in tblAZOut, and starts to build clsProcessFile instances which holds the information about the chunk processing. clsProcessFile can write its info back to the ProcessFile records on SQL Server. 5) clsDbAZOut has a thread which builds tblChunk, gets ChunkSize records into the chunk table and BCPs tblChunk to CSV files on a staging out directory. 6) clsDbAZOut BCPs the chunk to file 7) At this point the clsProcessFile is handed off to clsVM. clsSupervisor has a strongly typed dictionary which stores the clsProcessFile instances created by clsDbAZOut. clsSupervisor can read ProcessFile instances from SQL Server into a clsProcessFile factory or clsDbAZOut can use a class factory to create new ones. In all cases they are stored in the dictionary in clsSupervisor. clsProcessfile has flags which document that: a) The chunk table was created b) The chunk table exported to file c) The chunk went to a VM (and which one) d) The chunk file was (eventually) found in the Accuzip OutputFrom directory e) The resulting AZData file (and PDF) were moved from the VM back to SQL ServerIn staging f) The file in SQL Server In staging was pulled back in to a chunk table in a second database dbXyzAZIn temporary database g) That the chunk table was imported back into the original database. As you can see, clsProcessFile is the center of the universe for a single chunk. clsSupervisor is the center of the universe for the entire process. It is a bit difficult to discuss how the entire program works in a single email. A clsManager loads one or a dozen clsSupervisor instances. clsManager loads clsVM which supervises the (a) virtual machine which runs a third party process. clsVM moves files from SQL Staging Out into the VM one chunk at a time, updating clsProcessTable flags. clsVM moves files from the the VM back to the SQL Staging IN, updating flags. The VM may not be available, or the Accuzip program may be down etc. clsSupervisor loads one to N clsProcessTable instances. clsSupervisor loads a clsDBExp and clsDBImp. clsDBExp creates new clsProcessTable instances, one for each chunk it exports - storing them in clsSupervisor. clsDBExp supervises the export process. clsDBImp supervises the import process getting the chunks back into SQL Server into a temp database, and from there into the live database. I am trying to design the system such that it is asynchronous. Stage1, stage 2/3 and stage 4 can all do their thing independently of each other. Each step of each stage is logged (dateTime flags written) so that any piece can just pick up where it left off. That means any Supervisor and any ProcessFile (chunk) can pick up from any point. In the meantime, the clsManager is watching the tblSupervisor for supervisor records ready to process - a thread. clsSupervisor is watching for ProcessTable records and dispatching them as required to the clsDBExp, clsDBImp or clsVM - a thread in each supervisor. clsDBExp is processing chunks, creating CSV files - a thread. clsVM is watching for clsProcessTable instances ready to go to the VM. It is also watching the output directory and sending files back to the SQL Server - a thread and two file watchers (and two events). clsDBImp is watching for clsProcessTable instances ready to import back into SQL Server - a thread. Most of these threads are writing flags (as they finish their specific process) and reading flags in clsProcessTable to decide what chunk has to be processed in which class. Oh, and I have three list controls on a form which the various classes write to to display the status for Stage1, Stage 2/3 and Stage 4. So pretty much a single supervisor is processing at a time, exporting files, sending them to the vm, importing them back in etc. It is possible for an order supervisor to cut in at the head of the line however. Not a simple task co-ordinating all of this stuff. I used to co-ordinate it all manually! ;) John W. Colby www.ColbyConsulting.com On 3/26/2011 8:08 PM, Shamil Salakhetdinov wrote: > John -- > > I'm falling asleep here now - would that be correct assumption that > you can generalize your global task like that - you have, say: > > - split input data into 50 Chunks; > - every chunk should be processed in 10 steps; > - every step's average processing time is 100 minutes. > > Then in total it is needed 50*10*100 = 50,000 minutes or ~833.(3) > hours to process all your input data sequentially, chunk after chunk > step after step, say, you don't have enough computer power to process > some chunks in parallel, and you can't process the whole not split > input data as it's too large for your system... > > But you have "50 small computers" and you can process every of 50 > input chunks in parallel - then you'll need just 10*100 = 1,000 > minutes (~16.67 > hours) to complete the job. > > I assume that all the chunks processing is independent - then you can: > > 1) Create Scheduler class instance; > 2) Schedule class creates 10 TaskQueue class instances - 1 queue for > every processing step type; > 3) Scheduler class defines splitting criteria to split input data into > 50 chunks; > 4) Schedule defines completion criteria - when all 50 chunks get > collected in 11th FinalQueue; > 5) Schedule seeds the first InputChunkSplitTaskQueue with 50 > SplitTasksDescriptors; > 4) From time to time - say every half a minute Scheduler class > instance creates 500 worker classes in 500 threads (500 worker classes > => 50 chunks * 10 steps.), which "hunt for the job" in their dedicate > Queues: if they find Job they work on it, if not - they "die"... > 5) When a worker class completes processing its job it queues its > "task results" into the next step Queue, if a worker class fails then > it puts its incomplete task descriptor back into its dedicated queue > and "dies", or that could be Scheduler's (or SchedulerAssitant's) task > job to find failed workers' "trap" and to resubmit their failed work > to the proper queue according to the custom workflow descriptors attached to every chunk... > > 500 worker classes is an overkill as only 50 worker classes can have > job every time but that seems to be an easy "brute force" and "lazy > parallel programming" approach - and it should work... > Or you can make worker class instances production processes smarter: > Scheduler class can start special thread for WorkerClassGenerator > instance, which will monitor all the 10 TaskQueues, and if it finds an > item in a Queque, it will pick it up, it will create corresponding > Worker class in parallel thread and it will pass WorkItem to the > Worker class fro processing... > > When that described above approach will work there then you can > easily(?) scale it splitting your (constant size) input data into 100 > chunks, and then if every chunk can be processes in half time - in > average (50 minutes) - then all job can be completed in 500 minutes = ~8.33(3) hours etc... > > Please correct me if I oversimplified your application business area... > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sun Mar 27 22:01:44 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 27 Mar 2011 23:01:44 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> Message-ID: <4D8FFA18.9020902@colbyconsulting.com> Shamil, > Yes, I see: you tend to build a multi-threaded application/utility system > with centralized control - and I tend to build an anarchical free-market > model based application/utility system :) LOL, I guess. To be quite honest I didn't understand exactly what you were saying. Remember that my C# skills are still quite rudimentary. Accuzip Processing system basically has developed over time. I didn't sit down to develop an app from scratch to do this. I started in late 2004 to just get one single list into SQL server and serve up counts to the client. I did the whole thing manually, building queries to import the data and clean it up using only SQL Server Management Server. When I started I had never touched SQL Server. Then I was asked to research an application for doing CASS http://en.wikipedia.org/wiki/Coding_Accuracy_Support_System and NCOA http://en.wikipedia.org/wiki/National_Change_Of_Address on the addresses in the list. There was only one list in the beginning which I originally labeled "the database from hell". So I researched and discovered a vendor (Accuzip) that would allow us a fixed price license against which I could process as many records as I wanted. All other vendors (that I found) charge on a sliding scale Cents / record kind of thing, and we had a LOT of records - 65 million in that first list. So I started building a system of stored procedures which performed the export out to Accuzip and then import the Accuzipped records back in. It took roughly 25 stored procedures to perform this process. I had a paper list of instructions on how to run the stored procedures, parameters to feed in etc, plus manually copying the files out and back in. I would manually type EXEC commands into a query window reading from my paper instructions. It was *sloooow* and *error prone*. Next came a stored procedure that ran the stored procedures. Etc. Etc. It just morphed over time. Plus I got more lists. The client was so impressed that I managed to do all this that I got actual orders (export the data and send to the clients) rather than just counts, and the business grew. I started using Access ( the tool I knew ) to automate some of it but it was a very poor fit. Since every operation ran in a single thread the entire application would lock up when SQL Server went out for 30 minutes to do some operation. And of course my hardware was underpowered. In the summer of 2009 I decided I really needed to automate this. The business had grown and the client was suddenly 50% of my income and it was a huge struggle to process these lists manually, even using stored procedures of lists of stored procedures. TSQL as a programming environment is crude to say the least. So I went to the local community college in Sept. 2009 and took the first of two available C# courses. By Christmas I was launching in to my automation attempt. Understand my C# skills were rudimentary at best but the language was robust and I could see the power. I really just started building a method of moving the paper list of stored procedures into C#, IOW C# would execute the stored procedures and monitor their completion. I was not focused on a top down "build a system" but rather let's get my manual process automated ASAP. I was overworked, and wasn't able to handle the load. It turns out that C# is very capable in this regard and the program (such as it was) started working and working well. At the end of the second semester (May 2010) I hired Paul, a young college student who was graduating from the community college to come to work for me part time. I started teaching him what I know about general programming and together we worked on this program. And it continued to morph. We started using NLog for logging, Visual SVN for source control etc. At some point I decided to just move the Stored Procedures out of SQL Server and into dynamic code generated directly in C#. Each step had to keep the program running because I actually used the program to fill orders and do counts. And remember our C# skills are still rudimentary. Then we "discovered" threads and soon thereafter events. Last fall I decided to remodel the program into a manager / supervisor / worker model where the manager tells the supervisor to do something, the supervisor tells other workers to do something and basically just check results. In this model, each level uses threads to scan it's tasks (watch flags) and keep its workers busy. So the dbExport class has a thread which build a database, pulls records, builds chunks, exports to disk. That is it's job. When the file is on disk it is done and control for that chunk is passed to another class (the Virtual Machine). The dbExport class uses flags to store each step of the process. The Virtual Machine class picks up the files and pulls them over to the VMs directory, monitors that Accuzip processes them, tracks errors (if any) from Accuzip, and places the resulting files back on a staging directory (as I call it) for another class to take over. The dbImport class is tasked with creating a temp database, pulling each chunk file back in from staging and getting the data back into the original list database. So the manager class (a single instance) loads supervisor classes, one per list database that needs processing. The Manager class also loads the VM class (only one of these at the moment but there could be more, with more licenses from Accuzip). Each supervisor loads a dbExport and dbImport class to process the records in its list. dbExport creates Process records and stores them in SQL Server and creates a process class for each process record. Each process record represents a chunk of data from a specific list database. the process class doesn't really do anything it is just a place for the flags to be loaded / manipulated, and it knows how to persist itself to SQL Server. dbImport basically gets Process class instances which are finished with stage 2/3 (out of Accuzip) and starts importing them back in to SQL Server. My concept was a factory kind of application. There can be none or dozens of databases needing to be processed. SQL Server and my disk system can only efficiently handle one at a time (at any stage) and Accuzip is the biggest bottleneck so the VM class sits in the middle of an export and an import class. Every class has its own job, every class has at least one thread for its use. dbExport has a thread to use in exporting data to a table and another thread to bcp the data out while the next chunk is created in the export database. dbImport has a thread to BCP data back into chunk tables, and another thread to take the chunk tables and merge them back into live data. The VM has a thread to monitor the input directory and another thread to monitor the output directory, moving files into and out of the VM. Every step has to be logged back to SQL Server as it completes, but the clsSupervisor keeps the progress flags in memory as long as a specific list database is being processed, and the supervisor holds a list of process classes which hold all of the flags for the steps for a single chunk. It is a busy system. I did not originally use threads, and everything ran on the main program thread. And I was back to my user interface locking up as long running processes happened. Now with threading, each stage (export / vm / import) has its own list control which is used to display the status of the list database / chunks as they progress through the system. Everything is logged to flags, but also to NLog tables. Each class has threads to keep its own processes fed and running, and each class does its own piece of the whole to process a list table through Accuzip. BTW, there is another entire C# program which processes orders for the client. It is also threaded, and one of its steps is to create a supervisor record to tell the Accuzip Monitor to Accuzip the addresses being processed for the order. Orders are the Accuzip processing which I mentioned have to cut in at the head of the line in Accuzip Monitoring. As you have seen, I could be processing a 65 million record list database when an order comes in. 65 million records is going to take at least 50 hours to process, and the order has to go out ASAP so the Accuzip Monitor (the Manager class actually) has to watch for orders and interrupt the data supervisor to allow the Order supervisor to cut in. Once Accuzip starts a process (chunk) I don't interrupt it, but when that process (chunk) finishes Accuzip, only Order chunks will process until the order is finished. As I said, it is a busy system. And remember that our C# skills are still rudimentary. I started coding in C# in Sept 2009 and started migrating this process to c# about December 2009. By May 2010 when Paul started work I was actually running an Accuzip process and doing orders using C#. Paul is a youngster right out of school. So much of the stuff you talk about goes right over my head Shamil. John W. Colby www.ColbyConsulting.com On 3/27/2011 8:08 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I see: you tend to build a multi-threaded application/utility system > with centralized control - and I tend to build an anarchical free-market > model based application/utility system :) > > Thank you. > > -- > Shamil From jwcolby at colbyconsulting.com Sun Mar 27 22:20:36 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 27 Mar 2011 23:20:36 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> Message-ID: <4D8FFE84.5090609@colbyconsulting.com> OK, so I have explained the process. As you can see I am using threads pretty heavily. In general only one thread writes to any given flag field in a class and in general only one (other) thread reads from that flag. However threads are peeking at flags all the time "checking" whether the chunk (in particular) is ready to proceed to the next stage. This weekend I decided to try a dedicated flag class. My idea is simply that each flag represents a field in a table. Each flag represents a specific data element from a specific record. It is only written once (generally) but potentially read often. If I build a dedicated flag class it can be initialized with the table / field / pkid etc for its specific field / record and then when the thread writes the flag, the flag class can write its value out to the right field / record table. Again it is normally only ever written once. Most of my flags are a date / time that the process step occurred. So I am trying to wrap the whole flag concept into a class to allow me to process (read / write / test) flags in an identical way in all of my process classes. It was this flag class which I was discussing the end of last week. It will be accessed by at least two threads so it needs to be thread safe. John W. Colby www.ColbyConsulting.com On 3/27/2011 8:08 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I see: you tend to build a multi-threaded application/utility system > with centralized control - and I tend to build an anarchical free-market > model based application/utility system :) > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Mon Mar 28 17:39:57 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Tue, 29 Mar 2011 02:39:57 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <4D8FFE84.5090609@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> <4D8FFE84.5090609@colbyconsulting.com> Message-ID: <5FD8C68EEEF142DA8632253753C51323@nant> Hi John -- <<< It was this flag class which I was discussing the end of last week. It will be accessed by at least two threads so it needs to be thread safe. >>> Yes, I see - I do wish it worked efficiently there real soon... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 28 ????? 2011 ?. 7:21 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() OK, so I have explained the process. As you can see I am using threads pretty heavily. In general only one thread writes to any given flag field in a class and in general only one (other) thread reads from that flag. However threads are peeking at flags all the time "checking" whether the chunk (in particular) is ready to proceed to the next stage. This weekend I decided to try a dedicated flag class. My idea is simply that each flag represents a field in a table. Each flag represents a specific data element from a specific record. It is only written once (generally) but potentially read often. If I build a dedicated flag class it can be initialized with the table / field / pkid etc for its specific field / record and then when the thread writes the flag, the flag class can write its value out to the right field / record table. Again it is normally only ever written once. Most of my flags are a date / time that the process step occurred. So I am trying to wrap the whole flag concept into a class to allow me to process (read / write / test) flags in an identical way in all of my process classes. It was this flag class which I was discussing the end of last week. It will be accessed by at least two threads so it needs to be thread safe. John W. Colby www.ColbyConsulting.com On 3/27/2011 8:08 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I see: you tend to build a multi-threaded application/utility > system with centralized control - and I tend to build an anarchical > free-market model based application/utility system :) > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From DWUTKA at Marlow.com Thu Mar 31 10:34:13 2011 From: DWUTKA at Marlow.com (Drew Wutka) Date: Thu, 31 Mar 2011 10:34:13 -0500 Subject: [dba-VB] Most useful droid apps In-Reply-To: References: Message-ID: AndroXplorer Very handy for transferring files around between Droid, Network and SD Card. Drew -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of David McAfee Sent: Saturday, March 19, 2011 3:54 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Most useful droid apps Advanced task killer Weather channel Lookout antivirus AK Notepad Bank of America Acar auto maintenance Barcode scanner Bubble level Bump Droid light Google goggles Google translate IMDB Key ring Print share Trapster Sent from my Droid phone. On Mar 19, 2011 12:52 PM, "jwcolby" wrote: _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com The information contained in this transmission is intended only for the person or entity to which it is addressed and may contain II-VI Proprietary and/or II-VI Business Sensitive material. If you are not the intended recipient, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. You are notified that any review, retransmission, copying, disclosure, dissemination, or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. From jwcolby at colbyconsulting.com Wed Mar 2 20:11:33 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 02 Mar 2011 21:11:33 -0500 Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews Message-ID: <4D6EF8D5.7060804@colbyconsulting.com> -- John W. Colby www.ColbyConsulting.com http://www.legitreviews.com/news/10164/ From accessd at shaw.ca Thu Mar 3 14:55:06 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Thu, 3 Mar 2011 12:55:06 -0800 Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews In-Reply-To: <4D6EF8D5.7060804@colbyconsulting.com> References: <4D6EF8D5.7060804@colbyconsulting.com> Message-ID: <7AF8F6950EC14685B928DE79F836CB0D@creativesystemdesigns.com> Soon to be into John's office. ;-) Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, March 02, 2011 6:12 PM To: Access Developers discussion and problem solving; Sqlserver-Dba; VBA Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews -- John W. Colby www.ColbyConsulting.com http://www.legitreviews.com/news/10164/ _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Thu Mar 3 15:05:20 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 03 Mar 2011 16:05:20 -0500 Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews In-Reply-To: <7AF8F6950EC14685B928DE79F836CB0D@creativesystemdesigns.com> References: <4D6EF8D5.7060804@colbyconsulting.com> <7AF8F6950EC14685B928DE79F836CB0D@creativesystemdesigns.com> Message-ID: <4D700290.4010308@colbyconsulting.com> I wish. It will most likely be beyond my means for some time. John W. Colby www.ColbyConsulting.com On 3/3/2011 3:55 PM, Jim Lawrence wrote: > Soon to be into John's office. ;-) > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, March 02, 2011 6:12 PM > To: Access Developers discussion and problem solving; Sqlserver-Dba; VBA > Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - > Legit Reviews > > From michael at ddisolutions.com.au Sun Mar 6 19:26:08 2011 From: michael at ddisolutions.com.au (Michael Maddison) Date: Mon, 7 Mar 2011 12:26:08 +1100 Subject: [dba-VB] Java Help? Message-ID: <99266C61B516644D9727F983FAFAB465046ABC@remote.ddisolutions.com.au> Hi Guys, A while ago I asked about creating a COM dll to be called from Java. Does anyone have the Java skills to see if you can call the dll to test for me? Cheers Michael M From dbdoug at gmail.com Sun Mar 6 21:04:47 2011 From: dbdoug at gmail.com (Doug Steele) Date: Sun, 6 Mar 2011 19:04:47 -0800 Subject: [dba-VB] ASP.Net development Message-ID: Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele From Gustav at cactus.dk Mon Mar 7 04:08:58 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 07 Mar 2011 11:08:58 +0100 Subject: [dba-VB] ASP.Net development Message-ID: Hi Doug I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. /gustav >>> dbdoug at gmail.com 07-03-2011 04:04 >>> Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele From accessd at shaw.ca Mon Mar 7 09:22:24 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Mon, 7 Mar 2011 07:22:24 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: Hi Gustav: Showing my ignorance about Winforms; does Winforms create browser based application or create a Windows based applications that uses a browser? Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Monday, March 07, 2011 2:09 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] ASP.Net development Hi Doug I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. /gustav >>> dbdoug at gmail.com 07-03-2011 04:04 >>> Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From dbdoug at gmail.com Mon Mar 7 09:55:51 2011 From: dbdoug at gmail.com (Doug Steele) Date: Mon, 7 Mar 2011 07:55:51 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: Hi Gustav: Thanks for the reply. If you build a table adapter which consists of data from more than one table (like a one to many relationship) in the design surface, VS2008 will refuse to create any CRUD query code - you have to build it yourself. Maybe you only used one-table adapters? Doug On Mon, Mar 7, 2011 at 2:08 AM, Gustav Brock wrote: > Hi Doug > > I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. > > I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. > You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. > > As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. > > /gustav > > >>>> dbdoug at gmail.com 07-03-2011 04:04 >>> > Hello All: > > A lot of the Microsoft sample code for asp.net data access is built > using strongly typed tableadapters with the .xsd design surface. ?I've > built two apps using these in the last year, but they seem to be kind > of fiddly - they don't work well with multi table update queries, for > instance, and they generate a HUGE amount of background source code. > It is nice to have the strong typing, however. > > I have never found any non-Microsoft sample code which uses this > technique. ?Admittedly, most of the sample code in the wild consists > of short snippets built to illustrate a single technique, not a > complete application. > > Two questions: > 1. Are there developers out there using this approach for real life apps? > 2. If not, what do you recommend? > > > Thanks, > > Doug Steele > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Mon Mar 7 11:04:59 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 07 Mar 2011 18:04:59 +0100 Subject: [dba-VB] ASP.Net development Message-ID: Hi Jim WinForms are true Windows desktop apps (I realise I should have changed the subject line, sorry). If you wish a browser based app, that's ASP.NET or Silverlight if we talk about Visual Studio and dotNet. /gustav >>> accessd at shaw.ca 07-03-2011 16:22 >>> Hi Gustav: Showing my ignorance about Winforms; does Winforms create browser based application or create a Windows based applications that uses a browser? Jim From Gustav at cactus.dk Mon Mar 7 11:17:36 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 07 Mar 2011 18:17:36 +0100 Subject: [dba-VB] ASP.Net development Message-ID: Hi Doug Now I see. Yes, for updates I use one-table tableadapters. Multi-table tableadapters are mainly used as source for reports and list-type forms where nothing is updated - and as source for combo- and listboxes. /gustav >>> dbdoug at gmail.com 07-03-2011 16:55 >>> Hi Gustav: Thanks for the reply. If you build a table adapter which consists of data from more than one table (like a one to many relationship) in the design surface, VS2008 will refuse to create any CRUD query code - you have to build it yourself. Maybe you only used one-table adapters? Doug On Mon, Mar 7, 2011 at 2:08 AM, Gustav Brock wrote: > Hi Doug > > I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. > > I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. > You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. > > As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. > > /gustav > > >>>> dbdoug at gmail.com 07-03-2011 04:04 >>> > Hello All: > > A lot of the Microsoft sample code for asp.net data access is built > using strongly typed tableadapters with the .xsd design surface. I've > built two apps using these in the last year, but they seem to be kind > of fiddly - they don't work well with multi table update queries, for > instance, and they generate a HUGE amount of background source code. > It is nice to have the strong typing, however. > > I have never found any non-Microsoft sample code which uses this > technique. Admittedly, most of the sample code in the wild consists > of short snippets built to illustrate a single technique, not a > complete application. > > Two questions: > 1. Are there developers out there using this approach for real life apps? > 2. If not, what do you recommend? > > > Thanks, > > Doug Steele From accessd at shaw.ca Mon Mar 7 11:52:58 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Mon, 7 Mar 2011 09:52:58 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: <9F8E42F58F6F45B18FBDE74842047BF9@creativesystemdesigns.com> Hi Gustav: Thanks for the information. Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Monday, March 07, 2011 9:05 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] ASP.Net development Hi Jim WinForms are true Windows desktop apps (I realise I should have changed the subject line, sorry). If you wish a browser based app, that's ASP.NET or Silverlight if we talk about Visual Studio and dotNet. /gustav >>> accessd at shaw.ca 07-03-2011 16:22 >>> Hi Gustav: Showing my ignorance about Winforms; does Winforms create browser based application or create a Windows based applications that uses a browser? Jim _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Mon Mar 7 12:45:14 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 7 Mar 2011 21:45:14 +0300 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: <5C52BAB72E1246CFBBFB629233371E72@nant> Hi Doug -- I think I have seen an ASP.NET book with sources where strongly typed datasets were used but I can't find it now. But maybe that was this book: Pro ASP.NET 2.0 E-Commerce in C# 2005 (Expert's Voice in .NET) http://www.amazon.com/ASP-NET-Commerce-2005-Experts-Voice/dp/1590597249/ref= pd_sim_b_8 It does have complete source code and the DAL code does use ADO.NET DataSets but not strongly typed. I have got programmed rather large ASP.NET application - more than 150 webforms with I suppose more than 70-80 heavily related MS SQL tables with millions rows in some of tables, hundreds SPs, ... several thousands C# custom classes, ORM layer etc. and this web application does work well forth year in row with a few support work - the latter mainly being adding new functionality. I have used asp.net development approach described in this book: ASP.NET 2.0 Website Programming: Problem - Design - Solution (Programmer to Programmer) http://www.amazon.com/ASP-NET-2-0-Website-Programming-Programmer/dp/07645846 42/ref=sr_1_2?s=books&ie=UTF8&qid=1299518125&sr=1-2 BTW, the same approach is used in DotNetNuke (but I didn't know that when I have got started to develop my customer web application I mentioned above.)) I have had to develop custom ORM mapping + SQL code generator to implement this approach. But it should be possible to use some existing code generators. This approach looks "low level" and it needs quite some coding (code generators help here) but good news is that it will never get outdated. And here is a book on how to use LINQ 2 SQL for ASP.NET applications (I haven't read and I haven't tried this approach) for the same sample ASP.NET application, which is used in the previously referred book: LINQ-ifying TheBeerHouse: An N-Tier LINQ Web Application with ASP.NET 2.0 Website Programming Problem Design Solution (Wrox Briefs) http://www.amazon.com/LINQ-ifying-TheBeerHouse-Application-Programming-Solut ion/dp/B0026A6AQU/ref=sr_1_106?s=books&ie=UTF8&qid=1299518458&sr=1-106 The most modern approach is to use ADO.NET Entity Framework for DAL as Gustav noted, and when ADO.NET EF is used then it should be possible to map logical/business layer objects to the custom stored procedures, which can insert, update, delete, select data from one or database tables... I haven't yet seen any books where ADO.NET Entity Framework is used with classical WebForms based ASP.NET applications - anybody? Some more information on ADO.NET EF and ASP.NET: - here seems to be a good book on ADO.NET EF: Entity Framework 4.0 Recipes: A Problem-Solution Approach (Expert's Voice in .NET) http://www.amazon.com/Entity-Framework-4-0-Recipes-Problem-Solution/dp/14302 27036/ref=sr_1_168?s=books&ie=UTF8&qid=1299519467&sr=1-168 - and this one "opens secretes" on how to build ultra-fast and robust asp.net applications: Ultra-Fast ASP.NET: Build Ultra-Fast and Ultra-Scalable web sites using ASP.NET and SQL Server [Paperback] http://www.amazon.com/Ultra-Fast-ASP-NET-Build-Ultra-Scalable-Server/dp/1430 223839/ref=sr_1_121?s=books&ie=UTF8&qid=1299518863&sr=1-121 Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Doug Steele Sent: 7 ????? 2011 ?. 6:05 To: dba-VB at databaseadvisors.com Subject: [dba-VB] ASP.Net development Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From dbdoug at gmail.com Mon Mar 7 23:34:51 2011 From: dbdoug at gmail.com (Doug Steele) Date: Mon, 7 Mar 2011 21:34:51 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: <5C52BAB72E1246CFBBFB629233371E72@nant> References: <5C52BAB72E1246CFBBFB629233371E72@nant> Message-ID: Hi Shamil: Thanks very much for your reply. I'll be checking your references out.... Doug On Mon, Mar 7, 2011 at 10:45 AM, Shamil Salakhetdinov wrote: > Hi Doug -- > > I think I have seen an ASP.NET book with sources where strongly typed > datasets were used but I can't find it now. > But maybe that was this book: > > Pro ASP.NET 2.0 E-Commerce in C# 2005 (Expert's Voice in .NET) > http://www.amazon.com/ASP-NET-Commerce-2005-Experts-Voice/dp/1590597249/ref= > pd_sim_b_8 > > It does have complete source code and the DAL code does use ADO.NET DataSets > but not strongly typed. > > I have got programmed rather large ASP.NET application - more than 150 > webforms with I suppose more than 70-80 heavily related MS SQL tables with > millions rows in some of tables, hundreds SPs, ... several thousands C# > custom classes, ORM layer etc. and this web application does work well forth > year in row with a few support work - the latter mainly being adding new > functionality. I have used asp.net development approach described in this > book: > > ASP.NET 2.0 Website Programming: Problem - Design - Solution (Programmer to > Programmer) > http://www.amazon.com/ASP-NET-2-0-Website-Programming-Programmer/dp/07645846 > 42/ref=sr_1_2?s=books&ie=UTF8&qid=1299518125&sr=1-2 > > BTW, the same approach is used in DotNetNuke (but I didn't know that when I > have got started to develop my customer web application I mentioned above.)) > > I have had to develop custom ORM mapping + SQL code generator to implement > this approach. > But it should be possible to use some existing code generators. > > This approach looks "low level" and it needs quite some coding (code > generators help here) but good news is that it will never get outdated. > > And here is a book on how to use LINQ 2 SQL for ASP.NET applications (I > haven't read and I haven't tried this approach) for the same sample ASP.NET > application, which is used in the previously referred book: > > LINQ-ifying TheBeerHouse: An N-Tier LINQ Web Application with ASP.NET 2.0 > Website Programming Problem Design Solution (Wrox Briefs) > http://www.amazon.com/LINQ-ifying-TheBeerHouse-Application-Programming-Solut > ion/dp/B0026A6AQU/ref=sr_1_106?s=books&ie=UTF8&qid=1299518458&sr=1-106 > > The most modern approach is to use ADO.NET Entity Framework for DAL as > Gustav noted, and when ADO.NET EF is used then it should be possible to map > logical/business layer objects to the custom stored procedures, which can > insert, update, delete, select data from one or database tables... > > I haven't yet seen any books where ADO.NET Entity Framework is used with > classical WebForms based ASP.NET applications - anybody? > > > Some more information on ADO.NET EF and ASP.NET: > > - here seems to be a good book on ADO.NET EF: > > Entity Framework 4.0 Recipes: A Problem-Solution Approach (Expert's Voice in > .NET) > http://www.amazon.com/Entity-Framework-4-0-Recipes-Problem-Solution/dp/14302 > 27036/ref=sr_1_168?s=books&ie=UTF8&qid=1299519467&sr=1-168 > > - and this one "opens secretes" on how to build ultra-fast and robust > asp.net applications: > > Ultra-Fast ASP.NET: Build Ultra-Fast and Ultra-Scalable web sites using > ASP.NET and SQL Server [Paperback] > http://www.amazon.com/Ultra-Fast-ASP-NET-Build-Ultra-Scalable-Server/dp/1430 > 223839/ref=sr_1_121?s=books&ie=UTF8&qid=1299518863&sr=1-121 > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Doug Steele > Sent: 7 ????? 2011 ?. 6:05 > To: dba-VB at databaseadvisors.com > Subject: [dba-VB] ASP.Net development > > Hello All: > > A lot of the Microsoft sample code for asp.net data access is built using > strongly typed tableadapters with the .xsd design surface. ?I've built two > apps using these in the last year, but they seem to be kind of fiddly - they > don't work well with multi table update queries, for instance, and they > generate a HUGE amount of background source code. > It is nice to have the strong typing, however. > > I have never found any non-Microsoft sample code which uses this technique. > Admittedly, most of the sample code in the wild consists of short snippets > built to illustrate a single technique, not a complete application. > > Two questions: > 1. Are there developers out there using this approach for real life apps? > 2. If not, what do you recommend? > > > Thanks, > > Doug Steele > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From raibeart at gmail.com Tue Mar 8 11:44:25 2011 From: raibeart at gmail.com (Robert Stewart) Date: Tue, 08 Mar 2011 11:44:25 -0600 Subject: [dba-VB] I'm getting nowhere In-Reply-To: References: Message-ID: <4d766a81.cac4e60a.3ab2.255f@mx.google.com> John, Logins: If you are in mixed mode, which for your application you should be, every "group" will have it's own login with a password. User: References a login. A user is given rights to a database and specific objects in a database. Roles: Generally, you can assign DataReader and DataWriter roles to a User in a database to give them CRUD rights to tables. You will need to GRANT EXECUTE rights to stored procedures. Does that help? Robert At 09:22 AM 3/7/2011, you wrote: >From: dba-vb-bounces at databaseadvisors.com >[mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >Sent: Saturday, February 19, 2011 2:54 PM >To: Sqlserver-Dba; VBA; Access Developers discussion and problem solving >Subject: [dba-VB] I'm getting nowhere > >I am getting nowhere on understanding SQL Server security. Microsoft >provides us with SQL Server Express which implies that joe blow (me) is >going to install / maintain it. > >I am not a SQL Server Admin and I cannot afford to spend the time to be one. > >Google is my friend. BOL is not. > >Except that Google is taking me to these places where I am expected to >already know how this stuff works, and then wants to make me a *better* >administrator. Which of course is useless because I am not an administrator >at all. > >OTOH I am not stupid. If I could find something that started at the "This >is SQL Server security" >basics I could learn this stuff. Before anyone says "RTFM (BOL)" let me >simply say, "not happening". I have tried BOL and it simply sucks for my >level of expertise (my opinion of course). > If that is your advice, simply stay out of this thread. Thanks! > >So... my needs: > >I need to set up several SQL Server databases for use by different, very >small groups (5-20 people) of entirely unrelated people. What I mean by >that is that each DB is for a different "company" if you will. I need to >access these databases from C#. I understand the group / user paradigm. I >would like to create groups and users. Specific groups can do specific >things in the database, some can see data but not modify it. Some can add >records in specific tables but not others. Some can run reports (view). > >I do *NOT* want to create windows level groups and users if I can avoid it. >These are people that I do not necessarily know and I do not want to give >them any rights at the machine level, and I prefer to not maintain such >lists at the machine level. > >Unfortunately SQL Server does not seem to model Groups / users. I go into >SQL Server and see a security tab. It has "logins". Is that a user? A >specific ability to log in with a password? To what? The server itself? A >specific database? Groups of databases? > >I see "roles" but these appear to be aimed at the server and none of these >people are going to be doing anything at the server level. > >Can I safely ignore everything under the server security tab? > >I go to a database and I see a security tab. It has users and roles. >Hmm... better (I would think). I would like to add users "under" the >specific database that the user will access. > >So I try to add a new user but I do not see anywhere to require a password. >Hmmm... > >I go into roles and I do not see any predefined role that looks like it >would be useful to me in meeting my needs described above. If I look at >"add new role" it asks for a password. The User / group model does nto >assign passwords at the group level which implies that a role is not a group >at the user / group paradigm. > >Is it just me, or is SQL Server security just... different? Am I correct in >assuming that it doesn't implement a user / group paradigm? > >And more importantly, where can I go to get a plain, simple, English >description of how this mess works? > >And please excuse the tone that results from my frustration. The only help >documents that I have found (and I have extensive lists of bookmarked web >pages) so far assume that I am an administrator. > I am not, and cannot afford to become one. And yet MS pushes SQL Express >as if I (non-admin) should be able to use this as a data store pool. > >Help! > >-- >John W. Colby >www.ColbyConsulting.com >_______________________________________________ From jwcolby at colbyconsulting.com Thu Mar 10 07:12:44 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 08:12:44 -0500 Subject: [dba-VB] c# lock() Message-ID: <4D78CE4C.5000308@colbyconsulting.com> I am using multi-threading and need to lock list objects so that two different threads can access them. Do I even need to lock the list inside of the property as follows: public Dictionary pLstFiles { get { lock (lstFiles) { return lstFiles; } } } or only when I actually access the list? If I create a property to return the list (without locking it in the property) public Dictionary pLstFiles { get { return lstFiles; } } and I then get a pointer to the list using that property Dictionary LstFiles; LstFiles = SomeObj.pLstFiles; and I then lock the pointer while using it lock(LstFiles) { do stuff to list here... } Does that in fact lock the list to other objects doing the same thing (getting the list through that property)? IOW is the lock attribute affixed to the list at the list itself so that anything trying to use the list anywhere in the program can see that the list is locked? -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Thu Mar 10 10:29:39 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 11:29:39 -0500 Subject: [dba-VB] split an SVN database Message-ID: <4D78FC73.6000202@colbyconsulting.com> I use SVN. I am looking at doing a major split of a project into two projects. I want to leave the existing solution until I have one half of the current project carved out, debugged and running and in it's own SVN database. However I don't think I want to carry along the baggage of the old revisions into the new database. How do I go about this? -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Thu Mar 10 14:37:01 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 15:37:01 -0500 Subject: [dba-VB] c# - controlling the run state / status of your threaded classes Message-ID: <4D79366D.8060905@colbyconsulting.com> As I write classes that consume threads, I need to be able to start and stop these threads and the class that consumes the threads. For example, I open a form and load a class instance that starts performing some process - exporting sets of records to csv files for example. Once that class is loaded and processing, I need to be able to ask that class to stop exporting files. However there may be a time lag between asking and the currently processing file finishing it's export. That is precisely the purpose of a thread - tell it to go and do something else. So (for example) I have a check box that I check that tells the class to start exporting. When it starts it sets a flag saying that it is running and places text into the check box label saying that it is exporting (busy). If I then uncheck the check box I am telling the class to stop exporting. The class is monitoring the check box and will not process the next file but it can't just exit. So it needs to set a flag saying that it is stopping and display "stopping" on the check box label. When it finishes exporting the file, it sets a stopped flag and says displays "terminated" in the check box label. So we have a couple of "commands" (start / stop) and a few statuses (stopped, stopping, running). I would like to "standardize" this into a class so that any class which needs this interface does it in the same way. Some controlling object can tell the object to start and stop, and the class can tell the controlling object that is is stopped, stopping or running. this class does not6 need to do any updating of labels or anything like that, just hold the variables, properties, methods and code to perform this in a standard way. Is anyone doing this? Have existing code you would like to share? Have existing experience trying to do this you would like to share? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Thu Mar 10 15:35:24 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 00:35:24 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D78CE4C.5000308@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com> Message-ID: <3693FCAC03E94BCA9C150F6E7AFE4A71@nant> Hi John -- <<< Does that in fact lock the list to other objects doing the same thing (getting the list through that property)? >>> You have to lock/synchronize/serialize not the list itself but simultaneous access to the list from different threads. You'd better use special static object variable to implement locking. I will post sample code in my following posting here... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 16:13 To: VBA Subject: [dba-VB] c# lock() I am using multi-threading and need to lock list objects so that two different threads can access them. Do I even need to lock the list inside of the property as follows: public Dictionary pLstFiles { get { lock (lstFiles) { return lstFiles; } } } or only when I actually access the list? If I create a property to return the list (without locking it in the property) public Dictionary pLstFiles { get { return lstFiles; } } and I then get a pointer to the list using that property Dictionary LstFiles; LstFiles = SomeObj.pLstFiles; and I then lock the pointer while using it lock(LstFiles) { do stuff to list here... } Does that in fact lock the list to other objects doing the same thing (getting the list through that property)? IOW is the lock attribute affixed to the list at the list itself so that anything trying to use the list anywhere in the program can see that the list is locked? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Thu Mar 10 15:36:58 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 00:36:58 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D78CE4C.5000308@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com> Message-ID: <4889C56E6FF246D6AB908FEFD1069A68@nant> Hi John -- Here is sample code I promised to post. using System; using System.Collections.Generic; namespace ConsoleApplication1 { class Log { public static bool On { get; set; } public static void WriteLine(string message) { WriteLine(message, false); } public static void WriteLine(string message, bool forceLogging) { if (Log.On || forceLogging) System.Console.WriteLine(message); } } class MyObject { static int _newId; public MyObject() { _id = System.Threading.Interlocked.Increment(ref _newId); _timeStamp = DateTime.Now; } int _id; public int ID { get { return _id; } set { _id = value; } } DateTime _timeStamp; public DateTime TimeStamp { get { return _timeStamp; } set { _timeStamp = value; } } } class MyObjectsStore { private List _keys = new List(); private Dictionary _myObjects = new Dictionary(); public MyObject this[int key] { get { MyObject myObject = null; lock (_myObjectsLocker) { _myObjects.TryGetValue(key, out myObject); } return myObject; } } public int Count { get { lock (_myObjectsLocker) { return _myObjects.Count; } } } private static object _myObjectsLocker = new object(); public void Add(MyObject myObject) { lock (_myObjectsLocker) { _myObjects.Add(myObject.ID, myObject); _keys.Add(myObject.ID); } } public MyObject Remove(int id) { MyObject myobject = null; lock (_myObjectsLocker) { myobject = _myObjects[id]; _myObjects.Remove(id); } return myobject; } static Random _rnd = new Random(); public MyObject RemoveRandomObject() { MyObject myobject = null; int index = -1; int key = -1; try { lock (_myObjectsLocker) { index = _rnd.Next(_keys.Count - 1); key = _keys[index]; myobject = Remove(key); _keys.Remove(key); } } catch (Exception ex) { Log.WriteLine(string.Format("[{0}]: RemoveRandomObject: {1}, index = {2}, key = {3}", System.Threading.Thread.CurrentThread.ManagedThreadId, ex.Message, index , key)); } return myobject; } } class Program { public const int OBJECTS_PER_THREAD_COUNT = 7; public const int REMOVAL_MAX_RETRIES_COUNT = 5; public static void AddObjectsTest() { for (int i = 1; i <= OBJECTS_PER_THREAD_COUNT; i++) { MyObject myObject = new MyObject(); _store.Add(myObject); Log.WriteLine(string.Format("[{0}]: Object Added - ID = {1}, TS = {2}", System.Threading.Thread.CurrentThread.ManagedThreadId, myObject.ID, myObject.TimeStamp.Ticks)); System.Threading.Thread.Sleep(2); System.Threading.Interlocked.Increment(ref _createdObjectsCount); } System.Threading.Interlocked.Decrement(ref _threadsCount); } public static void RemoveObjectsTest() { for (int i = 1; i <= OBJECTS_PER_THREAD_COUNT; i++) { int retryCount = REMOVAL_MAX_RETRIES_COUNT; retry: MyObject myObject = _store.RemoveRandomObject(); if (myObject != null) { Log.WriteLine(string.Format("[{0}]: Object Removed - ID = {1}, TS = {2}", System.Threading.Thread.CurrentThread.ManagedThreadId, myObject.ID, myObject.TimeStamp.Ticks)); System.Threading.Thread.Sleep(2); } System.Threading.Thread.Sleep(2); if (myObject == null && --retryCount >= 0) { Log.WriteLine(string.Format("[{0}]: Object Removal failed, retry attempt# {1}", System.Threading.Thread.CurrentThread.ManagedThreadId, REMOVAL_MAX_RETRIES_COUNT-retryCount)); System.Threading.Interlocked.Increment(ref _totalRetriesCount); goto retry; } else if (myObject == null) { Log.WriteLine(string.Format("[{0}]: Object Removal failed", System.Threading.Thread.CurrentThread.ManagedThreadId)); System.Threading.Interlocked.Increment(ref _failedObjectRemovalsCount); } else System.Threading.Interlocked.Increment(ref _removedObjectsCount); } System.Threading.Interlocked.Decrement(ref _threadsCount); } static int _createdObjectsCount; static int _removedObjectsCount; static int _totalRetriesCount; static int _failedObjectRemovalsCount; static int _threadsCount; static MyObjectsStore _store; const int THREADS_COUNT = 20; //000; static void Main(string[] args) { Log.On = true; Log.WriteLine(string.Format("{0}: Test Started.", DateTime.Now)); try { Log.On = false; _store = new MyObjectsStore(); for (int i = 1; i <= THREADS_COUNT; i++) { if (i % 2 == 1) new System.Threading.Thread( new System.Threading.ThreadStart(AddObjectsTest)) .Start(); else new System.Threading.Thread( new System.Threading.ThreadStart(RemoveObjectsTest)) .Start(); _threadsCount++; } while (_threadsCount > 0) System.Threading.Thread.Sleep(100); System.Threading.Thread.Sleep(100); Log.On = true; Log.WriteLine(string.Format("Total Threads Count = {0}", THREADS_COUNT)); Log.WriteLine(string.Format("Store has {0} objects, Created = {1}, Removed = {2}", _store.Count, _createdObjectsCount, _removedObjectsCount)); Log.WriteLine(string.Format("RemovalRetriesCount = {0}, FailedRemovalsCount = {1}", _totalRetriesCount, _failedObjectRemovalsCount)); } catch (Exception ex) { Log.On = true; Log.WriteLine(string.Format("Program.Main: " + ex.Message)); } finally { Log.On = true; } Log.WriteLine(string.Format("{0}: Test Finished.", DateTime.Now)); } } } Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 16:13 To: VBA Subject: [dba-VB] c# lock() <<< snip >>> From shamil at smsconsulting.spb.ru Thu Mar 10 15:38:46 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 00:38:46 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D78CE4C.5000308@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com> Message-ID: <50A8FE7A468B41D48B9CC4F729CB02E3@nant> Hi John -- Here are my test results for sample code I've just posted (hopefully it get through AccessD - it was a bit lengthy) //11/03/2011 0:32:50: Test Started. //Total Threads Count = 20 //Store has 0 objects, Created = 70, Removed = 70 //RemovalRetriesCount = 0, FailedRemovalsCount = 0 //11/03/2011 0:32:50: Test Finished. //11/03/2011 0:32:25: Test Started. //Total Threads Count = 200 //Store has 0 objects, Created = 700, Removed = 700 //RemovalRetriesCount = 0, FailedRemovalsCount = 0 //11/03/2011 0:32:25: Test Finished. //11/03/2011 0:31:32: Test Started. //Total Threads Count = 20000 //Store has 19 objects, Created = 70000, Removed = 69981 //RemovalRetriesCount = 295, FailedRemovalsCount = 19 //11/03/2011 0:31:43: Test Finished. //11/03/2011 0:33:32: Test Started. //[3]: Object Added - ID = 1, TS = 634354004122968750 //[5]: Object Added - ID = 2, TS = 634354004122968750 //[3]: Object Added - ID = 3, TS = 634354004122968750 //[4]: Object Removed - ID = 2, TS = 634354004122968750 //[5]: Object Added - ID = 4, TS = 634354004122968750 //[6]: Object Removed - ID = 3, TS = 634354004122968750 //[7]: Object Added - ID = 5, TS = 634354004122968750 //[8]: Object Removed - ID = 4, TS = 634354004122968750 //[9]: Object Added - ID = 6, TS = 634354004122968750 //[10]: Object Removed - ID = 1, TS = 634354004122968750 //[11]: Object Added - ID = 7, TS = 634354004122968750 //[3]: Object Added - ID = 8, TS = 634354004122968750 //[12]: Object Removed - ID = 7, TS = 634354004122968750 //[5]: Object Added - ID = 9, TS = 634354004122968750 //[7]: Object Added - ID = 11, TS = 634354004122968750 //[9]: Object Added - ID = 10, TS = 634354004122968750 //[4]: Object Removed - ID = 6, TS = 634354004122968750 //[11]: Object Added - ID = 12, TS = 634354004122968750 //[3]: Object Added - ID = 13, TS = 634354004122968750 //[7]: Object Added - ID = 16, TS = 634354004123125000 //[6]: Object Removed - ID = 13, TS = 634354004122968750 //[11]: Object Added - ID = 17, TS = 634354004123125000 //[9]: Object Added - ID = 15, TS = 634354004123125000 //[4]: Object Removed - ID = 16, TS = 634354004123125000 //[8]: Object Removed - ID = 11, TS = 634354004122968750 //[10]: Object Removed - ID = 8, TS = 634354004122968750 //[5]: Object Added - ID = 14, TS = 634354004122968750 //[11]: Object Added - ID = 21, TS = 634354004123125000 //[3]: Object Added - ID = 18, TS = 634354004123125000 //[7]: Object Added - ID = 19, TS = 634354004123125000 //[10]: Object Removed - ID = 15, TS = 634354004123125000 //[6]: Object Removed - ID = 5, TS = 634354004122968750 //[4]: Object Removed - ID = 19, TS = 634354004123125000 //[8]: Object Removed - ID = 9, TS = 634354004122968750 //[10]: Object Removed - ID = 21, TS = 634354004123125000 //[6]: Object Removed - ID = 17, TS = 634354004123125000 //[8]: Object Removed - ID = 12, TS = 634354004122968750 //[4]: Object Removed - ID = 22, TS = 634354004123281250 //[12]: Object Removed - ID = 10, TS = 634354004122968750 //[6]: Object Removed - ID = 18, TS = 634354004123125000 //[4]: Object Removed - ID = 24, TS = 634354004123437500 //[8]: Object Removed - ID = 20, TS = 634354004123125000 //[6]: Object Removed - ID = 23, TS = 634354004123437500 //[12]: Object Removed - ID = 25, TS = 634354004123437500 //[10]: Object Removed - ID = 14, TS = 634354004122968750 //[9]: Object Added - ID = 20, TS = 634354004123125000 //[5]: Object Added - ID = 22, TS = 634354004123281250 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[10]: Object Removed - ID = 26, TS = 634354004124062500 //[6]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 1 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[10]: Object Removed - ID = 27, TS = 634354004124062500 //[3]: Object Added - ID = 24, TS = 634354004123437500 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[6]: Object Removal failed, retry attempt# 1 //[6]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 2 //[4]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[8]: Object Removal failed, retry attempt# 1 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[11]: Object Added - ID = 23, TS = 634354004123437500 //[7]: Object Added - ID = 25, TS = 634354004123437500 //[6]: Object Removal failed, retry attempt# 2 //[12]: Object Removed - ID = 28, TS = 634354004124531250 //[4]: Object Removal failed, retry attempt# 1 //[8]: Object Removal failed, retry attempt# 2 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[6]: Object Removed - ID = 29, TS = 634354004125000000 //[4]: Object Removed - ID = 30, TS = 634354004125000000 //[8]: Object Removal failed, retry attempt# 3 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 1 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[8]: Object Removal failed, retry attempt# 4 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 2 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[3]: Object Added - ID = 28, TS = 634354004124531250 //[12]: Object Removal failed, retry attempt# 3 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[8]: Object Removal failed, retry attempt# 5 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 4 //[7]: Object Added - ID = 30, TS = 634354004125000000 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[5]: Object Added - ID = 27, TS = 634354004124062500 //[9]: Object Added - ID = 26, TS = 634354004124062500 //[12]: Object Removal failed, retry attempt# 5 //[12]: Object Removed - ID = 31, TS = 634354004126093750 //[8]: Object Removal failed //[8]: Object Removed - ID = 32, TS = 634354004126250000 //[9]: Object Added - ID = 33, TS = 634354004126250000 //[12]: Object Removed - ID = 33, TS = 634354004126250000 //[11]: Object Added - ID = 29, TS = 634354004125000000 //[9]: Object Added - ID = 34, TS = 634354004126406250 //[12]: Object Removed - ID = 34, TS = 634354004126406250 //[7]: Object Added - ID = 31, TS = 634354004126093750 //[5]: Object Added - ID = 32, TS = 634354004126250000 //[11]: Object Added - ID = 35, TS = 634354004126406250 //Total Threads Count = 10 //Store has 1 objects, Created = 35, Removed = 34 //RemovalRetriesCount = 15, FailedRemovalsCount = 1 //11/03/2011 0:33:32: Test Finished. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 16:13 To: VBA Subject: [dba-VB] c# lock() <<< snip >>> From jwcolby at colbyconsulting.com Thu Mar 10 16:27:16 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 17:27:16 -0500 Subject: [dba-VB] c# lock() In-Reply-To: <3693FCAC03E94BCA9C150F6E7AFE4A71@nant> References: <4D78CE4C.5000308@colbyconsulting.com> <3693FCAC03E94BCA9C150F6E7AFE4A71@nant> Message-ID: <4D795044.6080808@colbyconsulting.com> Shamil, Thanks for the code but I think it is overkill for my needs so far, though I will put it in the archives for when I need it. In my system so far I have (or can arrange to have ) only two threads trying to access a list. Basically what I am doing is reading records out of sql server into a class factory, and placing the class instances into a list. Another object with its own thread is coming by and looking at the objects in the list to see if there is any work for it to do. If it finds an object to work on, it grabs a pointer to that item in the list and stores it in a pointer and releases the list. One of the ways I saw this handled was to create factory which creates a brand new list object, grabs a pointer to the list, locks that and then copies the objects in the list pointers to classes) into that new list, unlocks the list and hands back a pointer to this brand new list. thus the new list could be iterated at will. I am just trying to lock the list while one thread adds objects to the list or another thread gets objects out of the list. So if I have object A which contains a list and is adding / deleting objects to that list... and object B which needs to iterate through the list looking for an object of interest in the list, and grabbing a pointer to that object, there is no way to do this without all of the code you wrote? John W. Colby www.ColbyConsulting.com On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: > Hi John -- > > <<< > Does that in fact lock the list to other objects doing the same thing > (getting the list through that property)? >>>> > You have to lock/synchronize/serialize not the list itself but simultaneous > access to the list from different threads. > You'd better use special static object variable to implement locking. > > I will post sample code in my following posting here... > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Thu Mar 10 16:51:26 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 01:51:26 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D795044.6080808@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com> Message-ID: <3B9DE744CEA244829179A81CA818F92F@nant> Hi John -- Do you you need add, remove and iterate a list to get an object or you need to add, remove and get an object by its key? - if the latter - here is the code you can use: private Dictionary _myObjects = new Dictionary(); private static object _myObjectsLocker = new object(); public void Add(MyObject myObject) { lock (_myObjectsLocker) { _myObjects.Add(myObject.ID, myObject); _keys.Add(myObject.ID); } } public MyObject Remove(int id) { MyObject myobject = null; lock (_myObjectsLocker) { myobject = _myObjects[id]; _myObjects.Remove(id); } return myobject; } public MyObject this[int key] { get { MyObject myObject = null; lock (_myObjectsLocker) { _myObjects.TryGetValue(key, out myObject); } return myObject; } } for the case of iteration just replace: _myObjects.TryGetValue(key, out myObject); with your iteration code. As for the sample code - changing qty of threads and changing total computer workload (outer context for the test code) you can see how different could be final results when heavy multi-threading is used. And even if it's not heavy - it still can result in weird output if special multi-threading coding technique aren't used... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 1:27 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, Thanks for the code but I think it is overkill for my needs so far, though I will put it in the archives for when I need it. In my system so far I have (or can arrange to have ) only two threads trying to access a list. Basically what I am doing is reading records out of sql server into a class factory, and placing the class instances into a list. Another object with its own thread is coming by and looking at the objects in the list to see if there is any work for it to do. If it finds an object to work on, it grabs a pointer to that item in the list and stores it in a pointer and releases the list. One of the ways I saw this handled was to create factory which creates a brand new list object, grabs a pointer to the list, locks that and then copies the objects in the list pointers to classes) into that new list, unlocks the list and hands back a pointer to this brand new list. thus the new list could be iterated at will. I am just trying to lock the list while one thread adds objects to the list or another thread gets objects out of the list. So if I have object A which contains a list and is adding / deleting objects to that list... and object B which needs to iterate through the list looking for an object of interest in the list, and grabbing a pointer to that object, there is no way to do this without all of the code you wrote? John W. Colby www.ColbyConsulting.com On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: > Hi John -- > > <<< > Does that in fact lock the list to other objects doing the same thing > (getting the list through that property)? >>>> > You have to lock/synchronize/serialize not the list itself but > simultaneous access to the list from different threads. > You'd better use special static object variable to implement locking. > > I will post sample code in my following posting here... > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Thu Mar 10 17:11:33 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 02:11:33 +0300 Subject: [dba-VB] c# - controlling the run state / status of your threadedclasses In-Reply-To: <4D79366D.8060905@colbyconsulting.com> References: <4D79366D.8060905@colbyconsulting.com> Message-ID: <3B44AE877A14441CA13A51D1BA80FF65@nant> Hi John -- <<< So we have a couple of "commands" (start / stop) and a few statuses (stopped, stopping, running). >>> You'd better also add 'paused' (and maybe 'pausing' and 'resuming') for your list of states. I must note it's not easy to "standardize" such a coding - as with every framework/standard there immediately come non-standard situations to code into framework/standard - that's endless story... I have such a coding but it's rather complicated, several nesting levels etc.etc. - and it was programmed for a customer, I have reused it already in three projects of that customer but every time I do reuse it I have to do some additions/changes. It tested with up to hundred heavy duty threads talking to web services, MS Access(!) be databases, informing FE UI on threads' status changes etc. - and FE is able to work for days without restarting but that is a result of a very big work - it took two years+ to get from first version to the current one. Of course I didn't program it for two years, but getting all that experience to make it as it's now - that was rather time consuming - at least for me... If I ever will make that code less complicated I will share it but that will not happen soon, sorry... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 23:37 To: VBA Subject: [dba-VB] c# - controlling the run state / status of your threadedclasses As I write classes that consume threads, I need to be able to start and stop these threads and the class that consumes the threads. For example, I open a form and load a class instance that starts performing some process - exporting sets of records to csv files for example. Once that class is loaded and processing, I need to be able to ask that class to stop exporting files. However there may be a time lag between asking and the currently processing file finishing it's export. That is precisely the purpose of a thread - tell it to go and do something else. So (for example) I have a check box that I check that tells the class to start exporting. When it starts it sets a flag saying that it is running and places text into the check box label saying that it is exporting (busy). If I then uncheck the check box I am telling the class to stop exporting. The class is monitoring the check box and will not process the next file but it can't just exit. So it needs to set a flag saying that it is stopping and display "stopping" on the check box label. When it finishes exporting the file, it sets a stopped flag and says displays "terminated" in the check box label. So we have a couple of "commands" (start / stop) and a few statuses (stopped, stopping, running). I would like to "standardize" this into a class so that any class which needs this interface does it in the same way. Some controlling object can tell the object to start and stop, and the class can tell the controlling object that is is stopped, stopping or running. this class does not6 need to do any updating of labels or anything like that, just hold the variables, properties, methods and code to perform this in a standard way. Is anyone doing this? Have existing code you would like to share? Have existing experience trying to do this you would like to share? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 11 08:26:06 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 09:26:06 -0500 Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath Message-ID: <4D7A30FE.8050500@colbyconsulting.com> I am trying to copy a solution to a new location, strip all version control from it, delete a ton of projects within that solution and then version control the new solution. I copied the entire solution from my laptop to a VM and under my user / projects there. The copy still appears to be under version control. I looked in help and it says to copy the directory to itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the top level directory (the solution) but left the check mark on all of the subdirectories (projects and stuff). Does anyone have a clue how to completely and entirely remove a directory from version control - remove all green check marks from every subdirectory etc.? -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Fri Mar 11 08:59:23 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 09:59:23 -0500 Subject: [dba-VB] [dba-SQLServer] SVN - removing all version control for a directory and all files underneath In-Reply-To: <4D7A30FE.8050500@colbyconsulting.com> References: <4D7A30FE.8050500@colbyconsulting.com> Message-ID: <4D7A38CB.7090807@colbyconsulting.com> As it turns out, doing the "export to same directory" seems to actually "unregister" the directory from source control. It leaves the green check marks in explorer - not sure what that means. However if I go into the solution itself, it no longer has the version control dots next to the items in the solution explorer. So it appears to have removed that local copy of the solution from source control. John W. Colby www.ColbyConsulting.com On 3/11/2011 9:26 AM, jwcolby wrote: > I am trying to copy a solution to a new location, strip all version control from it, delete a ton of > projects within that solution and then version control the new solution. > > I copied the entire solution from my laptop to a VM and under my user / projects there. The copy > still appears to be under version control. I looked in help and it says to copy the directory to > itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the > top level directory (the solution) but left the check mark on all of the subdirectories (projects > and stuff). > > Does anyone have a clue how to completely and entirely remove a directory from version control - > remove all green check marks from every subdirectory etc.? From shamil at smsconsulting.spb.ru Fri Mar 11 09:00:05 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 18:00:05 +0300 Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath In-Reply-To: <4D7A30FE.8050500@colbyconsulting.com> References: <4D7A30FE.8050500@colbyconsulting.com> Message-ID: <9192624AA1BA4A6BB323E053411525B8@nant> Hi John -- 1. Just delete hidden .svn subfolder from root folder and all subfolders. 2. Right-click folder in MS Windows Explorer window and use Tortoise SVN Export... command from popup-menu 3. Use command line SVN utility with export command http://svnbook.red-bean.com/nightly/en/svn.ref.svn.c.export.html Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 17:26 To: Access Developers discussion and problem solving; VBA; Sqlserver-Dba Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath I am trying to copy a solution to a new location, strip all version control from it, delete a ton of projects within that solution and then version control the new solution. I copied the entire solution from my laptop to a VM and under my user / projects there. The copy still appears to be under version control. I looked in help and it says to copy the directory to itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the top level directory (the solution) but left the check mark on all of the subdirectories (projects and stuff). Does anyone have a clue how to completely and entirely remove a directory from version control - remove all green check marks from every subdirectory etc.? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 11 10:56:44 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 11:56:44 -0500 Subject: [dba-VB] c# - Creating and using a shared module Message-ID: <4D7A544C.7090904@colbyconsulting.com> I have split my solution into two solutions. However each of those two solutions uses a shared project which I call "base objects". Obviously I want to split that off and then reference that solution from both of the main solutions. I have never done this before. Any suggestions, tips and tricks, warnings, how to etc.? -- John W. Colby www.ColbyConsulting.com From gustav at cactus.dk Fri Mar 11 11:59:11 2011 From: gustav at cactus.dk (Gustav Brock) Date: Fri, 11 Mar 2011 18:59:11 +0100 Subject: [dba-VB] Visual Studio 2010 SP1 Message-ID: Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-475d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav From shamil at smsconsulting.spb.ru Fri Mar 11 11:52:31 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 20:52:31 +0300 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7A544C.7090904@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> Message-ID: <1573E00753994C19BE24FA9E50093615@nant> Hi John -- <<< Any suggestions, tips and tricks, warnings, how to etc.? >>> Just do it! :) I mean if you managed to handle MS Access library databases then handling .NET class libraries would be a "breeze" work for you. Create class library project - you can keep it within its own solution having main (console) project of that solution as a (unit) test one. Put all three solutions - two application ones and class library one in subfolders of the same root folder. (You can do it differently but this is how it's done usually AFAIK) Add existing class library project to application solutions. Reference class library projects from application projects. if you use VS2008 and you have a good PC with plenty of memory (and you have one don't you?) then you can keep all three solutions open in three instances of VS2008. Just keep tack from which solution you have made class library code changes last time - when you'll switch to the other solution and if you have the same source code file opened in it you'll get a message like that "Source file changed - would you like to reload it?" - reply yes... ... Use SVN or Mercurial for source code control. Do commit source code changes to the source code repository from time to time... ... I have 58 projects in one of the solutions here - that results in more than 5 apps and quite some class libs compiled and built - AFAIK folks do have more than hundred projects within solutions sometimes... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 19:57 To: VBA Subject: [dba-VB] c# - Creating and using a shared module I have split my solution into two solutions. However each of those two solutions uses a shared project which I call "base objects". Obviously I want to split that off and then reference that solution from both of the main solutions. I have never done this before. Any suggestions, tips and tricks, warnings, how to etc.? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From df.waters at comcast.net Fri Mar 11 13:46:10 2011 From: df.waters at comcast.net (Dan Waters) Date: Fri, 11 Mar 2011 13:46:10 -0600 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: Message-ID: <003001cbe024$f9c71750$ed5545f0$@comcast.net> I did this yesterday. The download iso image was actually about 1.5 Gb, but I downloaded, burned to disc, and installed in about an hour. Your mileage may vary! Dan -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-47 5d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 11 20:37:10 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 21:37:10 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <1573E00753994C19BE24FA9E50093615@nant> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant> Message-ID: <4D7ADC56.1040806@colbyconsulting.com> You make it sound so easy, but it is an existing project already which was not created as a class library project. I need to create the class library project then somehow get the directories and files into that project and get it to compile. At that point I can reference it by other solutions. John W. Colby www.ColbyConsulting.com On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: > Hi John -- > > <<< > Any suggestions, tips and tricks, warnings, how to etc.? >>>> > Just do it! :) > > I mean if you managed to handle MS Access library databases then handling > .NET class libraries would be a "breeze" work for you. > > Create class library project - you can keep it within its own solution > having main (console) project of that solution as a (unit) test one. > Put all three solutions - two application ones and class library one in > subfolders of the same root folder. (You can do it differently but this is > how it's done usually AFAIK) > Add existing class library project to application solutions. > Reference class library projects from application projects. > if you use VS2008 and you have a good PC with plenty of memory (and you have > one don't you?) then you can keep all three solutions open in three > instances of VS2008. > Just keep tack from which solution you have made class library code changes > last time - when you'll switch to the other solution and if you have the > same source code file opened in it you'll get a message like that "Source > file changed - would you like to reload it?" - reply yes... > > ... > > Use SVN or Mercurial for source code control. Do commit source code changes > to the source code repository from time to time... > ... > > I have 58 projects in one of the solutions here - that results in more than > 5 apps and quite some class libs compiled and built - AFAIK folks do have > more than hundred projects within solutions sometimes... > > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 11 ????? 2011 ?. 19:57 > To: VBA > Subject: [dba-VB] c# - Creating and using a shared module > > I have split my solution into two solutions. However each of those two > solutions uses a shared project which I call "base objects". Obviously I > want to split that off and then reference that solution from both of the > main solutions. > > I have never done this before. Any suggestions, tips and tricks, warnings, > how to etc.? > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From michael at ddisolutions.com.au Fri Mar 11 20:39:50 2011 From: michael at ddisolutions.com.au (Michael Maddison) Date: Sat, 12 Mar 2011 13:39:50 +1100 Subject: [dba-VB] SVN - removing all version control for a directory andall files underneath References: <4D7A30FE.8050500@colbyconsulting.com> <9192624AA1BA4A6BB323E053411525B8@nant> Message-ID: <99266C61B516644D9727F983FAFAB46508623F@remote.ddisolutions.com.au> Hi Guys, I found a reg file that adds a right click menu item to remove all hidden svn dirs. And files from the selected dir. Create a .reg file, Paste this code in, Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Folder\shell\DeleteSVN] @="Delete SVN Folders" [HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Folder\shell\DeleteSVN\command] @="cmd.exe /c \"TITLE Removing SVN Folders in %1 && COLOR 9A && FOR /r \"%1\" %%f IN (.svn) DO RD /s /q \"%%f\" \"" Save it, Merge it to your reg. Works for me. Cheers Michael From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: Saturday, 12 March 2011 2:00 AM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] SVN - removing all version control for a directory andall files underneath Hi John -- 1. Just delete hidden .svn subfolder from root folder and all subfolders. 2. Right-click folder in MS Windows Explorer window and use Tortoise SVN Export... command from popup-menu 3. Use command line SVN utility with export command http://svnbook.red-bean.com/nightly/en/svn.ref.svn.c.export.html Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 17:26 To: Access Developers discussion and problem solving; VBA; Sqlserver-Dba Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath I am trying to copy a solution to a new location, strip all version control from it, delete a ton of projects within that solution and then version control the new solution. I copied the entire solution from my laptop to a VM and under my user / projects there. The copy still appears to be under version control. I looked in help and it says to copy the directory to itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the top level directory (the solution) but left the check mark on all of the subdirectories (projects and stuff). Does anyone have a clue how to completely and entirely remove a directory from version control - remove all green check marks from every subdirectory etc.? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com ________________________________ No virus found in this message. Checked by AVG - www.avg.com Version: 10.0.1204 / Virus Database: 1498/3501 - Release Date: 03/11/11 From jwcolby at colbyconsulting.com Fri Mar 11 21:09:49 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 22:09:49 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7ADC56.1040806@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant> <4D7ADC56.1040806@colbyconsulting.com> Message-ID: <4D7AE3FD.1040206@colbyconsulting.com> I found a "make project from existing files. When I did that it offered to use a selected directory so I dragged the project directory into a new directory and told it to use that. I selected a class library type of project and it went to work "converting" my existing project into a class library. 112 compile errors, most having to do with the fact that it no longer understands using System.Windows.XYZ Many of the common classes did things like accept a combo loaded data into it in a standard way. Stuff like that. So there we are. I "just did it"! ;) John W. Colby www.ColbyConsulting.com On 3/11/2011 9:37 PM, jwcolby wrote: > You make it sound so easy, but it is an existing project already which was not created as a class > library project. I need to create the class library project then somehow get the directories and > files into that project and get it to compile. > > At that point I can reference it by other solutions. > > John W. Colby > www.ColbyConsulting.com > > On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Any suggestions, tips and tricks, warnings, how to etc.? >>>>> >> Just do it! :) >> >> I mean if you managed to handle MS Access library databases then handling >> .NET class libraries would be a "breeze" work for you. >> >> Create class library project - you can keep it within its own solution >> having main (console) project of that solution as a (unit) test one. >> Put all three solutions - two application ones and class library one in >> subfolders of the same root folder. (You can do it differently but this is >> how it's done usually AFAIK) >> Add existing class library project to application solutions. >> Reference class library projects from application projects. >> if you use VS2008 and you have a good PC with plenty of memory (and you have >> one don't you?) then you can keep all three solutions open in three >> instances of VS2008. >> Just keep tack from which solution you have made class library code changes >> last time - when you'll switch to the other solution and if you have the >> same source code file opened in it you'll get a message like that "Source >> file changed - would you like to reload it?" - reply yes... >> >> ... >> >> Use SVN or Mercurial for source code control. Do commit source code changes >> to the source code repository from time to time... >> ... >> >> I have 58 projects in one of the solutions here - that results in more than >> 5 apps and quite some class libs compiled and built - AFAIK folks do have >> more than hundred projects within solutions sometimes... >> >> >> Thank you. >> >> -- >> Shamil From shamil at smsconsulting.spb.ru Sat Mar 12 03:34:15 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 12 Mar 2011 12:34:15 +0300 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7AE3FD.1040206@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant><4D7ADC56.1040806@colbyconsulting.com> <4D7AE3FD.1040206@colbyconsulting.com> Message-ID: <8458BA03716A4820980078DE24EC3D06@nant> Hi John -- You have to add references to some of System.Windows.XYZ: - System.Windows.Forms - ... Just compare references of your source project with a classlib project created from it - and add missing references... <<< I "just did it"! ;) >>> But you did it, didn't you? ;) Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 12 ????? 2011 ?. 6:10 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# - Creating and using a shared module I found a "make project from existing files. When I did that it offered to use a selected directory so I dragged the project directory into a new directory and told it to use that. I selected a class library type of project and it went to work "converting" my existing project into a class library. 112 compile errors, most having to do with the fact that it no longer understands using System.Windows.XYZ Many of the common classes did things like accept a combo loaded data into it in a standard way. Stuff like that. So there we are. I "just did it"! ;) John W. Colby www.ColbyConsulting.com On 3/11/2011 9:37 PM, jwcolby wrote: > You make it sound so easy, but it is an existing project already which > was not created as a class library project. I need to create the class > library project then somehow get the directories and files into that project and get it to compile. > > At that point I can reference it by other solutions. > > John W. Colby > www.ColbyConsulting.com > > On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Any suggestions, tips and tricks, warnings, how to etc.? >>>>> >> Just do it! :) >> >> I mean if you managed to handle MS Access library databases then >> handling .NET class libraries would be a "breeze" work for you. >> >> Create class library project - you can keep it within its own >> solution having main (console) project of that solution as a (unit) test one. >> Put all three solutions - two application ones and class library one >> in subfolders of the same root folder. (You can do it differently but >> this is how it's done usually AFAIK) Add existing class library >> project to application solutions. >> Reference class library projects from application projects. >> if you use VS2008 and you have a good PC with plenty of memory (and >> you have one don't you?) then you can keep all three solutions open >> in three instances of VS2008. >> Just keep tack from which solution you have made class library code >> changes last time - when you'll switch to the other solution and if >> you have the same source code file opened in it you'll get a message >> like that "Source file changed - would you like to reload it?" - reply yes... >> >> ... >> >> Use SVN or Mercurial for source code control. Do commit source code >> changes to the source code repository from time to time... >> ... >> >> I have 58 projects in one of the solutions here - that results in >> more than >> 5 apps and quite some class libs compiled and built - AFAIK folks do >> have more than hundred projects within solutions sometimes... >> >> >> Thank you. >> >> -- >> Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Mar 12 06:27:48 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 12 Mar 2011 07:27:48 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <8458BA03716A4820980078DE24EC3D06@nant> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant><4D7ADC56.1040806@colbyconsulting.com> <4D7AE3FD.1040206@colbyconsulting.com> <8458BA03716A4820980078DE24EC3D06@nant> Message-ID: <4D7B66C4.6040404@colbyconsulting.com> Well... the whole thing compiled (the project) when it was part of the bigger solution. I'll give that a try. John W. Colby www.ColbyConsulting.com On 3/12/2011 4:34 AM, Shamil Salakhetdinov wrote: > Hi John -- > > You have to add references to some of System.Windows.XYZ: > > - System.Windows.Forms > - ... > > Just compare references of your source project with a classlib project > created from it - and add missing references... > > <<< > I "just did it"! ;) >>>> > But you did it, didn't you? ;) > > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 12 ????? 2011 ?. 6:10 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# - Creating and using a shared module > > I found a "make project from existing files. When I did that it offered to > use a selected directory so I dragged the project directory into a new > directory and told it to use that. I selected a class library type of > project and it went to work "converting" my existing project into a class > library. > > 112 compile errors, most having to do with the fact that it no longer > understands > > using System.Windows.XYZ > > Many of the common classes did things like accept a combo loaded data into > it in a standard way. > Stuff like that. > > So there we are. > > I "just did it"! ;) > > John W. Colby > www.ColbyConsulting.com > > On 3/11/2011 9:37 PM, jwcolby wrote: >> You make it sound so easy, but it is an existing project already which >> was not created as a class library project. I need to create the class >> library project then somehow get the directories and files into that > project and get it to compile. >> >> At that point I can reference it by other solutions. >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >>> Hi John -- >>> >>> <<< >>> Any suggestions, tips and tricks, warnings, how to etc.? >>>>>> >>> Just do it! :) >>> >>> I mean if you managed to handle MS Access library databases then >>> handling .NET class libraries would be a "breeze" work for you. >>> >>> Create class library project - you can keep it within its own >>> solution having main (console) project of that solution as a (unit) test > one. >>> Put all three solutions - two application ones and class library one >>> in subfolders of the same root folder. (You can do it differently but >>> this is how it's done usually AFAIK) Add existing class library >>> project to application solutions. >>> Reference class library projects from application projects. >>> if you use VS2008 and you have a good PC with plenty of memory (and >>> you have one don't you?) then you can keep all three solutions open >>> in three instances of VS2008. >>> Just keep tack from which solution you have made class library code >>> changes last time - when you'll switch to the other solution and if >>> you have the same source code file opened in it you'll get a message >>> like that "Source file changed - would you like to reload it?" - reply > yes... >>> >>> ... >>> >>> Use SVN or Mercurial for source code control. Do commit source code >>> changes to the source code repository from time to time... >>> ... >>> >>> I have 58 projects in one of the solutions here - that results in >>> more than >>> 5 apps and quite some class libs compiled and built - AFAIK folks do >>> have more than hundred projects within solutions sometimes... >>> >>> >>> Thank you. >>> >>> -- >>> Shamil > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 12 15:00:17 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 12 Mar 2011 16:00:17 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7B66C4.6040404@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant><4D7ADC56.1040806@colbyconsulting.com> <4D7AE3FD.1040206@colbyconsulting.com> <8458BA03716A4820980078DE24EC3D06@nant> <4D7B66C4.6040404@colbyconsulting.com> Message-ID: <4D7BDEE1.40400@colbyconsulting.com> OK, I got all of the references fixed up and got it checked into VSN. John W. Colby www.ColbyConsulting.com On 3/12/2011 7:27 AM, jwcolby wrote: > Well... the whole thing compiled (the project) when it was part of the bigger solution. > > I'll give that a try. > > John W. Colby > www.ColbyConsulting.com > > On 3/12/2011 4:34 AM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> You have to add references to some of System.Windows.XYZ: >> >> - System.Windows.Forms >> - ... >> >> Just compare references of your source project with a classlib project >> created from it - and add missing references... >> >> <<< >> I "just did it"! ;) >>>>> >> But you did it, didn't you? ;) >> >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 12 ????? 2011 ?. 6:10 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# - Creating and using a shared module >> >> I found a "make project from existing files. When I did that it offered to >> use a selected directory so I dragged the project directory into a new >> directory and told it to use that. I selected a class library type of >> project and it went to work "converting" my existing project into a class >> library. >> >> 112 compile errors, most having to do with the fact that it no longer >> understands >> >> using System.Windows.XYZ >> >> Many of the common classes did things like accept a combo loaded data into >> it in a standard way. >> Stuff like that. >> >> So there we are. >> >> I "just did it"! ;) >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/11/2011 9:37 PM, jwcolby wrote: >>> You make it sound so easy, but it is an existing project already which >>> was not created as a class library project. I need to create the class >>> library project then somehow get the directories and files into that >> project and get it to compile. >>> >>> At that point I can reference it by other solutions. >>> >>> John W. Colby >>> www.ColbyConsulting.com >>> >>> On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >>>> Hi John -- >>>> >>>> <<< >>>> Any suggestions, tips and tricks, warnings, how to etc.? >>>>>>> >>>> Just do it! :) >>>> >>>> I mean if you managed to handle MS Access library databases then >>>> handling .NET class libraries would be a "breeze" work for you. >>>> >>>> Create class library project - you can keep it within its own >>>> solution having main (console) project of that solution as a (unit) test >> one. >>>> Put all three solutions - two application ones and class library one >>>> in subfolders of the same root folder. (You can do it differently but >>>> this is how it's done usually AFAIK) Add existing class library >>>> project to application solutions. >>>> Reference class library projects from application projects. >>>> if you use VS2008 and you have a good PC with plenty of memory (and >>>> you have one don't you?) then you can keep all three solutions open >>>> in three instances of VS2008. >>>> Just keep tack from which solution you have made class library code >>>> changes last time - when you'll switch to the other solution and if >>>> you have the same source code file opened in it you'll get a message >>>> like that "Source file changed - would you like to reload it?" - reply >> yes... >>>> >>>> ... >>>> >>>> Use SVN or Mercurial for source code control. Do commit source code >>>> changes to the source code repository from time to time... >>>> ... >>>> >>>> I have 58 projects in one of the solutions here - that results in >>>> more than >>>> 5 apps and quite some class libs compiled and built - AFAIK folks do >>>> have more than hundred projects within solutions sometimes... >>>> >>>> >>>> Thank you. >>>> >>>> -- >>>> Shamil >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From bheid at sc.rr.com Sun Mar 13 09:26:07 2011 From: bheid at sc.rr.com (Bobby Heid) Date: Sun, 13 Mar 2011 10:26:07 -0400 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: Message-ID: <005d01cbe18a$977c6700$c6753500$@rr.com> Only took me about 40 minutes on my machine (Win 7 Ultimate, VS22010 Ultimate, and SQL 2008 Standard). Bobby -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Friday, March 11, 2011 12:59 PM To: dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-47 5d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sun Mar 13 21:58:15 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 13 Mar 2011 22:58:15 -0400 Subject: [dba-VB] Access runtime (an maybe full install?) Message-ID: <4D7D8447.4000003@colbyconsulting.com> I am trying to get an Access 2007 runtime to run reliably anywhere I install it. I had a Windows 2003 virtual machine hanging around so I cleaned Office off of it, and I installed the runtime. I got the directory path declared a safe location and voila, the application ran. Full of confidence I installed Hamachi, the runtime package and the app on a friend's (rather old) XP and it doesn't run. I dug out my old dev machine which it appears that I wiped and installed XP on some time ago. I installed Hamachi, got it on the VPN for this database etc. I then installed the runtime there and tried to run. No go. Of course the runtime doesn't report errors or at least reliably. So I built a little test version with just a single table and an autoform which automatically opened when the app opened. It actually reported that it couldn't talk to the odbc native sql driver (not the exact words of course). So I started Googling and soon discovered that this driver is installed by SQL Server as it installs. However an installer called sqlncli.msi is out there for download. I downloaded that and tried to run. No joy. I turned the machine off, and ran a network cable so it is directly on my network and voila, it runs. I then disabled the cable so that it is connecting to my network via my wireless and it runs. So it appears that the SQL driver is not installed as part of XP, which makes sense since this is SQL Server 2008 Express version I am connecting to. However I would have expected this odbc SQL driver to be part of the Access 2007 runtime, but it seems not. It does appear that a download / install / reboot of the machine will allow me to hook up to the sql server instance. So I am now running Hamachi, joined to this specific network, with the sqlncli.msi installer installing the ODBC SQL Server native driver, plus the Access 2007 runtime, running the full on app, hitting the sql server over the vpn on 5.203.167.79 (the Hamachi IP for the server). And it works on this machine of mine. Now I have to install the SQL Server drivers on the friend's old machine (also running XP but on 512 megs of ram) and ensure that will run. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sun Mar 13 22:27:45 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 13 Mar 2011 23:27:45 -0400 Subject: [dba-VB] [AccessD] Access runtime (an maybe full install?) In-Reply-To: <4D7D8447.4000003@colbyconsulting.com> References: <4D7D8447.4000003@colbyconsulting.com> Message-ID: <4D7D8B31.4080405@colbyconsulting.com> And it runs on the friend's old machine as well. It feels like I might be working now. I have to say this has been one of the most complex jobs I have taken on, with Hamachi VPN networks, a virtual machine running the sql server database, and then the client machine running Hamachi and an Access 2007 runtime. Safe zones, sql server drivers not installed. It has been a challenge getting this running. John W. Colby www.ColbyConsulting.com On 3/13/2011 10:58 PM, jwcolby wrote: > I am trying to get an Access 2007 runtime to run reliably anywhere I install it. I had a Windows > 2003 virtual machine hanging around so I cleaned Office off of it, and I installed the runtime. I > got the directory path declared a safe location and voila, the application ran. > > Full of confidence I installed Hamachi, the runtime package and the app on a friend's (rather old) > XP and it doesn't run. I dug out my old dev machine which it appears that I wiped and installed XP > on some time ago. I installed Hamachi, got it on the VPN for this database etc. I then installed the > runtime there and tried to run. No go. > > Of course the runtime doesn't report errors or at least reliably. So I built a little test version > with just a single table and an autoform which automatically opened when the app opened. It actually > reported that it couldn't talk to the odbc native sql driver (not the exact words of course). So I > started Googling and soon discovered that this driver is installed by SQL Server as it installs. > However an installer called sqlncli.msi is out there for download. > > I downloaded that and tried to run. No joy. I turned the machine off, and ran a network cable so it > is directly on my network and voila, it runs. I then disabled the cable so that it is connecting to > my network via my wireless and it runs. > > So it appears that the SQL driver is not installed as part of XP, which makes sense since this is > SQL Server 2008 Express version I am connecting to. However I would have expected this odbc SQL > driver to be part of the Access 2007 runtime, but it seems not. > > It does appear that a download / install / reboot of the machine will allow me to hook up to the sql > server instance. > > So I am now running Hamachi, joined to this specific network, with the sqlncli.msi installer > installing the ODBC SQL Server native driver, plus the Access 2007 runtime, running the full on app, > hitting the sql server over the vpn on 5.203.167.79 (the Hamachi IP for the server). > > And it works on this machine of mine. > > Now I have to install the SQL Server drivers on the friend's old machine (also running XP but on 512 > megs of ram) and ensure that will run. > > From Gustav at cactus.dk Mon Mar 14 05:38:03 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 14 Mar 2011 11:38:03 +0100 Subject: [dba-VB] Visual Studio 2010 SP1 Message-ID: Hi all OK, don't blame the child. At one time it even asked for the VS2010 dvd - to roll back the installation. Another install was in progress, or at least I hadn't rebooted the machine. So I did and ran the SP1 again. This time - as everything was downloaded - it took about an hour to finish. /gustav >>> gustav at cactus.dk 11-03-2011 18:59 >>> Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-475d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav From jwcolby at colbyconsulting.com Mon Mar 14 13:16:52 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 14:16:52 -0400 Subject: [dba-VB] C# combo boxes Message-ID: <4D7E5B94.2020805@colbyconsulting.com> I am trying to set up a combo box bound to a field in the form such that the combo populates with the PKID and a text field, and when the form opens, selecting something from the combo drops the PKID into the field in the form. I am not getting what to set the properties of the combo box to. I am pulling the ID and the string to display into the combo but I cannot cause the combo to store the id and release control to the next control on the form. -- John W. Colby www.ColbyConsulting.com From Gustav at cactus.dk Mon Mar 14 14:02:27 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 14 Mar 2011 20:02:27 +0100 Subject: [dba-VB] C# combo boxes Message-ID: Hi John If you (typically) have this in InitializeComponent() of the .Designer.cs file: // comboBoxCountry // this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); this.comboBoxCountry.DisplayMember = "Id"; this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; this.comboBoxCountry.FormattingEnabled = true; this.comboBoxCountry.ItemHeight = 13; this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); this.comboBoxCountry.Name = "comboBoxCountry"; this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); this.comboBoxCountry.Sorted = true; this.comboBoxCountry.TabIndex = 7; this.comboBoxCountry.ValueMember = "Id"; You can do something like this in the Load event: this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); Then: private void FillComboBoxCountryCode() { // Order the country names of countryDataTable. string filterExpression = ""; string sortExpression = "CountryNameUK asc"; DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); this.comboBoxCountry.DataSource = countryList; this.comboBoxCountry.ValueMember = "Id"; this.comboBoxCountry.DisplayMember = "CountryNameUK"; } And: private void ChangeCountrySelection() { if (this.comboBoxCountry.SelectedValue != null) { _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); this.comboBoxPostalCode.Text = String.Empty; // Read postal codes of selected country. FillComboBoxPostalCode(); } } This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. /gustav >>> jwcolby at colbyconsulting.com 14-03-2011 19:16 >>> I am trying to set up a combo box bound to a field in the form such that the combo populates with the PKID and a text field, and when the form opens, selecting something from the combo drops the PKID into the field in the form. I am not getting what to set the properties of the combo box to. I am pulling the ID and the string to display into the combo but I cannot cause the combo to store the id and release control to the next control on the form. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Mon Mar 14 16:36:55 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 17:36:55 -0400 Subject: [dba-VB] C# combo boxes In-Reply-To: References: Message-ID: <4D7E8A77.7070009@colbyconsulting.com> Thanks Gustav, I don't know why you are taking me into code in code. I have data connections. This particular data connection has a single view that allows me to write into a table. It has a view that filters a combo. I drag and drop the view for the table onto the form. I have told that table object that specific fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am trying to set up that combo to 1) Display a text value 2) Contain the PKID from the table the combo draws from 3) When I select an item, place the selected ID into the field the combo is bound to. It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or going into code). Actually I have a kinda sorta related question. When I look at server explorer I see a data connection object, however there are two objects underneath, neither of which are for this solution. Furthermore the objects under the data connection are things found on an entirely unrelated server, for other projects that I did work on in the past. The question then is why is the server explorer choosing to display objects out on a virtual machine instead of the database that was originally selected for this solution? We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound form stuff. I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values in most cases) the developer, the product, the type of work, then a couple of fields for date time values for start / stop work. The table the data is going into has a few thousand records already in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound forms is a natural for this. A long time ago I made this (kind of thing) work. I remember that getting the properties of the combo set correctly was non-intuitive but once I figured it out it just worked. John W. Colby www.ColbyConsulting.com On 3/14/2011 3:02 PM, Gustav Brock wrote: > Hi John > > If you (typically) have this in InitializeComponent() of the .Designer.cs file: > > // comboBoxCountry > // > this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; > this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; > this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; > this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); > this.comboBoxCountry.DisplayMember = "Id"; > this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; > this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); > this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; > this.comboBoxCountry.FormattingEnabled = true; > this.comboBoxCountry.ItemHeight = 13; > this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); > this.comboBoxCountry.Name = "comboBoxCountry"; > this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); > this.comboBoxCountry.Sorted = true; > this.comboBoxCountry.TabIndex = 7; > this.comboBoxCountry.ValueMember = "Id"; > > You can do something like this in the Load event: > > this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); > > Then: > private void FillComboBoxCountryCode() > { > // Order the country names of countryDataTable. > string filterExpression = ""; > string sortExpression = "CountryNameUK asc"; > DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); > > this.comboBoxCountry.DataSource = countryList; > this.comboBoxCountry.ValueMember = "Id"; > this.comboBoxCountry.DisplayMember = "CountryNameUK"; > } > > And: > private void ChangeCountrySelection() > { > if (this.comboBoxCountry.SelectedValue != null) > { > _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); > this.comboBoxPostalCode.Text = String.Empty; > // Read postal codes of selected country. > FillComboBoxPostalCode(); > } > } > > This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> > I am trying to set up a combo box bound to a field in the form such that the combo populates with > the PKID and a text field, and when the form opens, selecting something from the combo drops the > PKID into the field in the form. > > I am not getting what to set the properties of the combo box to. I am pulling the ID and the string > to display into the combo but I cannot cause the combo to store the id and release control to the > next control on the form. From gustav at cactus.dk Mon Mar 14 17:15:35 2011 From: gustav at cactus.dk (Gustav Brock) Date: Mon, 14 Mar 2011 23:15:35 +0100 Subject: [dba-VB] C# combo boxes Message-ID: Hi John Eh, are you commenting or? Not quite clear to me. I thought you were looking for some code samples ... I have connections too but I use them only to feed the datatableadapters. I refrain from working with SqlCommand and the like; waste of time in my opinion. /gustav >>> jwcolby at colbyconsulting.com 14-03-2011 22:36 >>> Thanks Gustav, I don't know why you are taking me into code in code. I have data connections. This particular data connection has a single view that allows me to write into a table. It has a view that filters a combo. I drag and drop the view for the table onto the form. I have told that table object that specific fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am trying to set up that combo to 1) Display a text value 2) Contain the PKID from the table the combo draws from 3) When I select an item, place the selected ID into the field the combo is bound to. It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or going into code). Actually I have a kinda sorta related question. When I look at server explorer I see a data connection object, however there are two objects underneath, neither of which are for this solution. Furthermore the objects under the data connection are things found on an entirely unrelated server, for other projects that I did work on in the past. The question then is why is the server explorer choosing to display objects out on a virtual machine instead of the database that was originally selected for this solution? We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound form stuff. I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values in most cases) the developer, the product, the type of work, then a couple of fields for date time values for start / stop work. The table the data is going into has a few thousand records already in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound forms is a natural for this. A long time ago I made this (kind of thing) work. I remember that getting the properties of the combo set correctly was non-intuitive but once I figured it out it just worked. John W. Colby www.ColbyConsulting.com On 3/14/2011 3:02 PM, Gustav Brock wrote: > Hi John > > If you (typically) have this in InitializeComponent() of the .Designer.cs file: > > // comboBoxCountry > // > this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; > this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; > this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; > this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); > this.comboBoxCountry.DisplayMember = "Id"; > this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; > this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); > this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; > this.comboBoxCountry.FormattingEnabled = true; > this.comboBoxCountry.ItemHeight = 13; > this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); > this.comboBoxCountry.Name = "comboBoxCountry"; > this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); > this.comboBoxCountry.Sorted = true; > this.comboBoxCountry.TabIndex = 7; > this.comboBoxCountry.ValueMember = "Id"; > > You can do something like this in the Load event: > > this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); > > Then: > private void FillComboBoxCountryCode() > { > // Order the country names of countryDataTable. > string filterExpression = ""; > string sortExpression = "CountryNameUK asc"; > DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); > > this.comboBoxCountry.DataSource = countryList; > this.comboBoxCountry.ValueMember = "Id"; > this.comboBoxCountry.DisplayMember = "CountryNameUK"; > } > > And: > private void ChangeCountrySelection() > { > if (this.comboBoxCountry.SelectedValue != null) > { > _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); > this.comboBoxPostalCode.Text = String.Empty; > // Read postal codes of selected country. > FillComboBoxPostalCode(); > } > } > > This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> > I am trying to set up a combo box bound to a field in the form such that the combo populates with > the PKID and a text field, and when the form opens, selecting something from the combo drops the > PKID into the field in the form. > > I am not getting what to set the properties of the combo box to. I am pulling the ID and the string > to display into the combo but I cannot cause the combo to store the id and release control to the > next control on the form. From jwcolby at colbyconsulting.com Mon Mar 14 17:25:08 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 18:25:08 -0400 Subject: [dba-VB] C# combo boxes In-Reply-To: References: Message-ID: <4D7E95C4.9020203@colbyconsulting.com> I am trying (unsuccessfully) to do an "Access like bound form". And I always comment. ;) ATM the form displays the values if dropped down, but if not dropped down it just displays an integer (the PKID of the data in the combo is my guess, or perhaps the PKID in the field in the form. So... How do I cause the combo to display Paul's name instead of 6. John W. Colby www.ColbyConsulting.com On 3/14/2011 6:15 PM, Gustav Brock wrote: > Hi John > > Eh, are you commenting or? Not quite clear to me. I thought you were looking for some code samples ... > > I have connections too but I use them only to feed the datatableadapters. I refrain from working with SqlCommand and the like; waste of time in my opinion. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> > Thanks Gustav, > I don't know why you are taking me into code in code. > > I have data connections. This particular data connection has a single view that allows me to write > into a table. It has a view that filters a combo. > > I drag and drop the view for the table onto the form. I have told that table object that specific > fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am > trying to set up that combo to > > 1) Display a text value > 2) Contain the PKID from the table the combo draws from > 3) When I select an item, place the selected ID into the field the combo is bound to. > > It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or > going into code). > > Actually I have a kinda sorta related question. > > When I look at server explorer I see a data connection object, however there are two objects > underneath, neither of which are for this solution. Furthermore the objects under the data > connection are things found on an entirely unrelated server, for other projects that I did work on > in the past. > > The question then is why is the server explorer choosing to display objects out on a virtual machine > instead of the database that was originally selected for this solution? > > We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound > form stuff. > > I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values > in most cases) the developer, the product, the type of work, then a couple of fields for date time > values for start / stop work. The table the data is going into has a few thousand records already > in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound > forms is a natural for this. > > A long time ago I made this (kind of thing) work. I remember that getting the properties of the > combo set correctly was non-intuitive but once I figured it out it just worked. > > > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 3:02 PM, Gustav Brock wrote: >> Hi John >> >> If you (typically) have this in InitializeComponent() of the .Designer.cs file: >> >> // comboBoxCountry >> // >> this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; >> this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; >> this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; >> this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); >> this.comboBoxCountry.DisplayMember = "Id"; >> this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; >> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); >> this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; >> this.comboBoxCountry.FormattingEnabled = true; >> this.comboBoxCountry.ItemHeight = 13; >> this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); >> this.comboBoxCountry.Name = "comboBoxCountry"; >> this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); >> this.comboBoxCountry.Sorted = true; >> this.comboBoxCountry.TabIndex = 7; >> this.comboBoxCountry.ValueMember = "Id"; >> >> You can do something like this in the Load event: >> >> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); >> >> Then: >> private void FillComboBoxCountryCode() >> { >> // Order the country names of countryDataTable. >> string filterExpression = ""; >> string sortExpression = "CountryNameUK asc"; >> DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); >> >> this.comboBoxCountry.DataSource = countryList; >> this.comboBoxCountry.ValueMember = "Id"; >> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >> } >> >> And: >> private void ChangeCountrySelection() >> { >> if (this.comboBoxCountry.SelectedValue != null) >> { >> _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); >> this.comboBoxPostalCode.Text = String.Empty; >> // Read postal codes of selected country. >> FillComboBoxPostalCode(); >> } >> } >> >> This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >> I am trying to set up a combo box bound to a field in the form such that the combo populates with >> the PKID and a text field, and when the form opens, selecting something from the combo drops the >> PKID into the field in the form. >> >> I am not getting what to set the properties of the combo box to. I am pulling the ID and the string >> to display into the combo but I cannot cause the combo to store the id and release control to the >> next control on the form. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From accessd at shaw.ca Mon Mar 14 18:59:26 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Mon, 14 Mar 2011 16:59:26 -0700 Subject: [dba-VB] C# combo boxes In-Reply-To: <4D7E95C4.9020203@colbyconsulting.com> References: <4D7E95C4.9020203@colbyconsulting.com> Message-ID: Hi John: Try and use ADO. It is on every Windows based computer already and requires no additional ODBC to be setup. In other words, you do not even have to see the computers on which the application is going to have a sable connection. Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, March 14, 2011 3:25 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# combo boxes I am trying (unsuccessfully) to do an "Access like bound form". And I always comment. ;) ATM the form displays the values if dropped down, but if not dropped down it just displays an integer (the PKID of the data in the combo is my guess, or perhaps the PKID in the field in the form. So... How do I cause the combo to display Paul's name instead of 6. John W. Colby www.ColbyConsulting.com On 3/14/2011 6:15 PM, Gustav Brock wrote: > Hi John > > Eh, are you commenting or? Not quite clear to me. I thought you were looking for some code samples ... > > I have connections too but I use them only to feed the datatableadapters. I refrain from working with SqlCommand and the like; waste of time in my opinion. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> > Thanks Gustav, > I don't know why you are taking me into code in code. > > I have data connections. This particular data connection has a single view that allows me to write > into a table. It has a view that filters a combo. > > I drag and drop the view for the table onto the form. I have told that table object that specific > fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am > trying to set up that combo to > > 1) Display a text value > 2) Contain the PKID from the table the combo draws from > 3) When I select an item, place the selected ID into the field the combo is bound to. > > It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or > going into code). > > Actually I have a kinda sorta related question. > > When I look at server explorer I see a data connection object, however there are two objects > underneath, neither of which are for this solution. Furthermore the objects under the data > connection are things found on an entirely unrelated server, for other projects that I did work on > in the past. > > The question then is why is the server explorer choosing to display objects out on a virtual machine > instead of the database that was originally selected for this solution? > > We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound > form stuff. > > I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values > in most cases) the developer, the product, the type of work, then a couple of fields for date time > values for start / stop work. The table the data is going into has a few thousand records already > in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound > forms is a natural for this. > > A long time ago I made this (kind of thing) work. I remember that getting the properties of the > combo set correctly was non-intuitive but once I figured it out it just worked. > > > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 3:02 PM, Gustav Brock wrote: >> Hi John >> >> If you (typically) have this in InitializeComponent() of the .Designer.cs file: >> >> // comboBoxCountry >> // >> this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; >> this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; >> this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; >> this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); >> this.comboBoxCountry.DisplayMember = "Id"; >> this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; >> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); >> this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; >> this.comboBoxCountry.FormattingEnabled = true; >> this.comboBoxCountry.ItemHeight = 13; >> this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); >> this.comboBoxCountry.Name = "comboBoxCountry"; >> this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); >> this.comboBoxCountry.Sorted = true; >> this.comboBoxCountry.TabIndex = 7; >> this.comboBoxCountry.ValueMember = "Id"; >> >> You can do something like this in the Load event: >> >> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); >> >> Then: >> private void FillComboBoxCountryCode() >> { >> // Order the country names of countryDataTable. >> string filterExpression = ""; >> string sortExpression = "CountryNameUK asc"; >> DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); >> >> this.comboBoxCountry.DataSource = countryList; >> this.comboBoxCountry.ValueMember = "Id"; >> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >> } >> >> And: >> private void ChangeCountrySelection() >> { >> if (this.comboBoxCountry.SelectedValue != null) >> { >> _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); >> this.comboBoxPostalCode.Text = String.Empty; >> // Read postal codes of selected country. >> FillComboBoxPostalCode(); >> } >> } >> >> This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >> I am trying to set up a combo box bound to a field in the form such that the combo populates with >> the PKID and a text field, and when the form opens, selecting something from the combo drops the >> PKID into the field in the form. >> >> I am not getting what to set the properties of the combo box to. I am pulling the ID and the string >> to display into the combo but I cannot cause the combo to store the id and release control to the >> next control on the form. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Mon Mar 14 21:13:25 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 22:13:25 -0400 Subject: [dba-VB] C# combo boxes In-Reply-To: References: <4D7E95C4.9020203@colbyconsulting.com> Message-ID: <4D7ECB45.8060609@colbyconsulting.com> I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com On 3/14/2011 7:59 PM, Jim Lawrence wrote: > Hi John: > > Try and use ADO. It is on every Windows based computer already and requires > no additional ODBC to be setup. In other words, you do not even have to see > the computers on which the application is going to have a sable connection. > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Monday, March 14, 2011 3:25 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] C# combo boxes > > I am trying (unsuccessfully) to do an "Access like bound form". > > And I always comment. ;) > > ATM the form displays the values if dropped down, but if not dropped down it > just displays an > integer (the PKID of the data in the combo is my guess, or perhaps the PKID > in the field in the form. > > So... > > How do I cause the combo to display Paul's name instead of 6. > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 6:15 PM, Gustav Brock wrote: >> Hi John >> >> Eh, are you commenting or? Not quite clear to me. I thought you were > looking for some code samples ... >> >> I have connections too but I use them only to feed the datatableadapters. > I refrain from working with SqlCommand and the like; waste of time in my > opinion. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> >> Thanks Gustav, >> I don't know why you are taking me into code in code. >> >> I have data connections. This particular data connection has a single > view that allows me to write >> into a table. It has a view that filters a combo. >> >> I drag and drop the view for the table onto the form. I have told that > table object that specific >> fields are combos so when I drag it out, I have combos (for the FKs - PKs > from other tables). I am >> trying to set up that combo to >> >> 1) Display a text value >> 2) Contain the PKID from the table the combo draws from >> 3) When I select an item, place the selected ID into the field the combo > is bound to. >> >> It seems this is a normal daily requirement in building forms, and > shouldn't be rocket science (or >> going into code). >> >> Actually I have a kinda sorta related question. >> >> When I look at server explorer I see a data connection object, however > there are two objects >> underneath, neither of which are for this solution. Furthermore the > objects under the data >> connection are things found on an entirely unrelated server, for other > projects that I did work on >> in the past. >> >> The question then is why is the server explorer choosing to display > objects out on a virtual machine >> instead of the database that was originally selected for this solution? >> >> We have done a ton of coding sql server stuff here but we have not yet > learned how to do this bound >> form stuff. >> >> I am trying to build a timeclock kind of thing. Very simple, a few combos > to select (default values >> in most cases) the developer, the product, the type of work, then a couple > of fields for date time >> values for start / stop work. The table the data is going into has a few > thousand records already >> in it, the combos pull from tables with 20-30 records etc. It seems (to > me) that just using bound >> forms is a natural for this. >> >> A long time ago I made this (kind of thing) work. I remember that getting > the properties of the >> combo set correctly was non-intuitive but once I figured it out it just > worked. >> >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/14/2011 3:02 PM, Gustav Brock wrote: >>> Hi John >>> >>> If you (typically) have this in InitializeComponent() of the .Designer.cs > file: >>> >>> // comboBoxCountry >>> // >>> this.comboBoxCountry.AutoCompleteMode = > System.Windows.Forms.AutoCompleteMode.Suggest; >>> this.comboBoxCountry.AutoCompleteSource = > System.Windows.Forms.AutoCompleteSource.ListItems; >>> this.comboBoxCountry.BackColor = > System.Drawing.Color.LavenderBlush; >>> this.comboBoxCountry.DataBindings.Add(new > System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, > "CountryId", true)); >>> this.comboBoxCountry.DisplayMember = "Id"; >>> this.comboBoxCountry.FlatStyle = > System.Windows.Forms.FlatStyle.Flat; >>> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe > UI", 8F, System.Drawing.FontStyle.Regular, > System.Drawing.GraphicsUnit.Point, ((byte)(0))); >>> this.comboBoxCountry.ForeColor = > System.Drawing.Color.DarkMagenta; >>> this.comboBoxCountry.FormattingEnabled = true; >>> this.comboBoxCountry.ItemHeight = 13; >>> this.comboBoxCountry.Location = new > System.Drawing.Point(21, 136); >>> this.comboBoxCountry.Name = "comboBoxCountry"; >>> this.comboBoxCountry.Size = new System.Drawing.Size(117, > 21); >>> this.comboBoxCountry.Sorted = true; >>> this.comboBoxCountry.TabIndex = 7; >>> this.comboBoxCountry.ValueMember = "Id"; >>> >>> You can do something like this in the Load event: >>> >>> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => > ChangeCountrySelection(); >>> >>> Then: >>> private void FillComboBoxCountryCode() >>> { >>> // Order the country names of countryDataTable. >>> string filterExpression = ""; >>> string sortExpression = "CountryNameUK asc"; >>> DataRow[] countryList = > _countryDataTable.Select(filterExpression, sortExpression); >>> >>> this.comboBoxCountry.DataSource = countryList; >>> this.comboBoxCountry.ValueMember = "Id"; >>> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >>> } >>> >>> And: >>> private void ChangeCountrySelection() >>> { >>> if (this.comboBoxCountry.SelectedValue != null) >>> { >>> _countryId = > Convert.ToInt32(this.comboBoxCountry.SelectedValue); >>> this.comboBoxPostalCode.Text = String.Empty; >>> // Read postal codes of selected country. >>> FillComboBoxPostalCode(); >>> } >>> } >>> >>> This is pretty standard using tableadapters and bindingnavigator. Nothing > fancy. >>> >>> /gustav >>> >>> >>>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >>> I am trying to set up a combo box bound to a field in the form such that > the combo populates with >>> the PKID and a text field, and when the form opens, selecting something > from the combo drops the >>> PKID into the field in the form. >>> >>> I am not getting what to set the properties of the combo box to. I am > pulling the ID and the string >>> to display into the combo but I cannot cause the combo to store the id > and release control to the >>> next control on the form. >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From accessd at shaw.ca Tue Mar 15 02:19:11 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Tue, 15 Mar 2011 00:19:11 -0700 Subject: [dba-VB] C# combo boxes In-Reply-To: <4D7ECB45.8060609@colbyconsulting.com> References: <4D7E95C4.9020203@colbyconsulting.com> <4D7ECB45.8060609@colbyconsulting.com> Message-ID: Congratulation John. Keep us posted on it stability. Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, March 14, 2011 7:13 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# combo boxes I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com On 3/14/2011 7:59 PM, Jim Lawrence wrote: > Hi John: > > Try and use ADO. It is on every Windows based computer already and requires > no additional ODBC to be setup. In other words, you do not even have to see > the computers on which the application is going to have a sable connection. > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Monday, March 14, 2011 3:25 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] C# combo boxes > > I am trying (unsuccessfully) to do an "Access like bound form". > > And I always comment. ;) > > ATM the form displays the values if dropped down, but if not dropped down it > just displays an > integer (the PKID of the data in the combo is my guess, or perhaps the PKID > in the field in the form. > > So... > > How do I cause the combo to display Paul's name instead of 6. > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 6:15 PM, Gustav Brock wrote: >> Hi John >> >> Eh, are you commenting or? Not quite clear to me. I thought you were > looking for some code samples ... >> >> I have connections too but I use them only to feed the datatableadapters. > I refrain from working with SqlCommand and the like; waste of time in my > opinion. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> >> Thanks Gustav, >> I don't know why you are taking me into code in code. >> >> I have data connections. This particular data connection has a single > view that allows me to write >> into a table. It has a view that filters a combo. >> >> I drag and drop the view for the table onto the form. I have told that > table object that specific >> fields are combos so when I drag it out, I have combos (for the FKs - PKs > from other tables). I am >> trying to set up that combo to >> >> 1) Display a text value >> 2) Contain the PKID from the table the combo draws from >> 3) When I select an item, place the selected ID into the field the combo > is bound to. >> >> It seems this is a normal daily requirement in building forms, and > shouldn't be rocket science (or >> going into code). >> >> Actually I have a kinda sorta related question. >> >> When I look at server explorer I see a data connection object, however > there are two objects >> underneath, neither of which are for this solution. Furthermore the > objects under the data >> connection are things found on an entirely unrelated server, for other > projects that I did work on >> in the past. >> >> The question then is why is the server explorer choosing to display > objects out on a virtual machine >> instead of the database that was originally selected for this solution? >> >> We have done a ton of coding sql server stuff here but we have not yet > learned how to do this bound >> form stuff. >> >> I am trying to build a timeclock kind of thing. Very simple, a few combos > to select (default values >> in most cases) the developer, the product, the type of work, then a couple > of fields for date time >> values for start / stop work. The table the data is going into has a few > thousand records already >> in it, the combos pull from tables with 20-30 records etc. It seems (to > me) that just using bound >> forms is a natural for this. >> >> A long time ago I made this (kind of thing) work. I remember that getting > the properties of the >> combo set correctly was non-intuitive but once I figured it out it just > worked. >> >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/14/2011 3:02 PM, Gustav Brock wrote: >>> Hi John >>> >>> If you (typically) have this in InitializeComponent() of the .Designer.cs > file: >>> >>> // comboBoxCountry >>> // >>> this.comboBoxCountry.AutoCompleteMode = > System.Windows.Forms.AutoCompleteMode.Suggest; >>> this.comboBoxCountry.AutoCompleteSource = > System.Windows.Forms.AutoCompleteSource.ListItems; >>> this.comboBoxCountry.BackColor = > System.Drawing.Color.LavenderBlush; >>> this.comboBoxCountry.DataBindings.Add(new > System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, > "CountryId", true)); >>> this.comboBoxCountry.DisplayMember = "Id"; >>> this.comboBoxCountry.FlatStyle = > System.Windows.Forms.FlatStyle.Flat; >>> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe > UI", 8F, System.Drawing.FontStyle.Regular, > System.Drawing.GraphicsUnit.Point, ((byte)(0))); >>> this.comboBoxCountry.ForeColor = > System.Drawing.Color.DarkMagenta; >>> this.comboBoxCountry.FormattingEnabled = true; >>> this.comboBoxCountry.ItemHeight = 13; >>> this.comboBoxCountry.Location = new > System.Drawing.Point(21, 136); >>> this.comboBoxCountry.Name = "comboBoxCountry"; >>> this.comboBoxCountry.Size = new System.Drawing.Size(117, > 21); >>> this.comboBoxCountry.Sorted = true; >>> this.comboBoxCountry.TabIndex = 7; >>> this.comboBoxCountry.ValueMember = "Id"; >>> >>> You can do something like this in the Load event: >>> >>> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => > ChangeCountrySelection(); >>> >>> Then: >>> private void FillComboBoxCountryCode() >>> { >>> // Order the country names of countryDataTable. >>> string filterExpression = ""; >>> string sortExpression = "CountryNameUK asc"; >>> DataRow[] countryList = > _countryDataTable.Select(filterExpression, sortExpression); >>> >>> this.comboBoxCountry.DataSource = countryList; >>> this.comboBoxCountry.ValueMember = "Id"; >>> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >>> } >>> >>> And: >>> private void ChangeCountrySelection() >>> { >>> if (this.comboBoxCountry.SelectedValue != null) >>> { >>> _countryId = > Convert.ToInt32(this.comboBoxCountry.SelectedValue); >>> this.comboBoxPostalCode.Text = String.Empty; >>> // Read postal codes of selected country. >>> FillComboBoxPostalCode(); >>> } >>> } >>> >>> This is pretty standard using tableadapters and bindingnavigator. Nothing > fancy. >>> >>> /gustav >>> >>> >>>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >>> I am trying to set up a combo box bound to a field in the form such that > the combo populates with >>> the PKID and a text field, and when the form opens, selecting something > from the combo drops the >>> PKID into the field in the form. >>> >>> I am not getting what to set the properties of the combo box to. I am > pulling the ID and the string >>> to display into the combo but I cannot cause the combo to store the id > and release control to the >>> next control on the form. >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From Gustav at cactus.dk Tue Mar 15 04:55:02 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Tue, 15 Mar 2011 10:55:02 +0100 Subject: [dba-VB] C# combo boxes Message-ID: Hi John Good! Yes, the content of Text is handled by DisplayMember: this.comboBoxCountry.DataSource = countryList; this.comboBoxCountry.ValueMember = "Id"; this.comboBoxCountry.DisplayMember = "CountryNameUK"; /gustav >>> jwcolby at colbyconsulting.com 15-03-2011 03:13 >>> I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com From df.waters at comcast.net Tue Mar 15 08:54:00 2011 From: df.waters at comcast.net (Dan Waters) Date: Tue, 15 Mar 2011 08:54:00 -0500 Subject: [dba-VB] C# combo boxes In-Reply-To: <4D7ECB45.8060609@colbyconsulting.com> References: <4D7E95C4.9020203@colbyconsulting.com> <4D7ECB45.8060609@colbyconsulting.com> Message-ID: <002101cbe318$713acbb0$53b06310$@comcast.net> Hi John, Take a look at www.updatecontrols.net. This has a free download which integrates into visual studio and provides a way to simply databinding. I haven't used it yet but it looks promising. Note: At the top of the screens is a combobox which selects either VB or C# for the code examples. Dan -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, March 14, 2011 9:13 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# combo boxes I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com On 3/14/2011 7:59 PM, Jim Lawrence wrote: > Hi John: > > Try and use ADO. It is on every Windows based computer already and > requires no additional ODBC to be setup. In other words, you do not > even have to see the computers on which the application is going to have a sable connection. > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Monday, March 14, 2011 3:25 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] C# combo boxes > > I am trying (unsuccessfully) to do an "Access like bound form". > > And I always comment. ;) > > ATM the form displays the values if dropped down, but if not dropped > down it just displays an integer (the PKID of the data in the combo is > my guess, or perhaps the PKID in the field in the form. > > So... > > How do I cause the combo to display Paul's name instead of 6. > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 6:15 PM, Gustav Brock wrote: >> Hi John >> >> Eh, are you commenting or? Not quite clear to me. I thought you were > looking for some code samples ... >> >> I have connections too but I use them only to feed the datatableadapters. > I refrain from working with SqlCommand and the like; waste of time in > my opinion. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> >> Thanks Gustav, >> I don't know why you are taking me into code in code. >> >> I have data connections. This particular data connection has a >> single > view that allows me to write >> into a table. It has a view that filters a combo. >> >> I drag and drop the view for the table onto the form. I have told >> that > table object that specific >> fields are combos so when I drag it out, I have combos (for the FKs - >> PKs > from other tables). I am >> trying to set up that combo to >> >> 1) Display a text value >> 2) Contain the PKID from the table the combo draws from >> 3) When I select an item, place the selected ID into the field the >> combo > is bound to. >> >> It seems this is a normal daily requirement in building forms, and > shouldn't be rocket science (or >> going into code). >> >> Actually I have a kinda sorta related question. >> >> When I look at server explorer I see a data connection object, >> however > there are two objects >> underneath, neither of which are for this solution. Furthermore the > objects under the data >> connection are things found on an entirely unrelated server, for >> other > projects that I did work on >> in the past. >> >> The question then is why is the server explorer choosing to display > objects out on a virtual machine >> instead of the database that was originally selected for this solution? >> >> We have done a ton of coding sql server stuff here but we have not >> yet > learned how to do this bound >> form stuff. >> >> I am trying to build a timeclock kind of thing. Very simple, a few >> combos > to select (default values >> in most cases) the developer, the product, the type of work, then a >> couple > of fields for date time >> values for start / stop work. The table the data is going into has a >> few > thousand records already >> in it, the combos pull from tables with 20-30 records etc. It seems >> (to > me) that just using bound >> forms is a natural for this. >> >> A long time ago I made this (kind of thing) work. I remember that >> getting > the properties of the >> combo set correctly was non-intuitive but once I figured it out it >> just > worked. >> >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/14/2011 3:02 PM, Gustav Brock wrote: >>> Hi John >>> >>> If you (typically) have this in InitializeComponent() of the >>> .Designer.cs > file: >>> >>> // comboBoxCountry >>> // >>> this.comboBoxCountry.AutoCompleteMode = > System.Windows.Forms.AutoCompleteMode.Suggest; >>> this.comboBoxCountry.AutoCompleteSource = > System.Windows.Forms.AutoCompleteSource.ListItems; >>> this.comboBoxCountry.BackColor = > System.Drawing.Color.LavenderBlush; >>> this.comboBoxCountry.DataBindings.Add(new > System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, > "CountryId", true)); >>> this.comboBoxCountry.DisplayMember = "Id"; >>> this.comboBoxCountry.FlatStyle = > System.Windows.Forms.FlatStyle.Flat; >>> this.comboBoxCountry.Font = new >>> System.Drawing.Font("Segoe > UI", 8F, System.Drawing.FontStyle.Regular, > System.Drawing.GraphicsUnit.Point, ((byte)(0))); >>> this.comboBoxCountry.ForeColor = > System.Drawing.Color.DarkMagenta; >>> this.comboBoxCountry.FormattingEnabled = true; >>> this.comboBoxCountry.ItemHeight = 13; >>> this.comboBoxCountry.Location = new > System.Drawing.Point(21, 136); >>> this.comboBoxCountry.Name = "comboBoxCountry"; >>> this.comboBoxCountry.Size = new >>> System.Drawing.Size(117, > 21); >>> this.comboBoxCountry.Sorted = true; >>> this.comboBoxCountry.TabIndex = 7; >>> this.comboBoxCountry.ValueMember = "Id"; >>> >>> You can do something like this in the Load event: >>> >>> this.comboBoxCountry.SelectedValueChanged += (oo, ee) >>> => > ChangeCountrySelection(); >>> >>> Then: >>> private void FillComboBoxCountryCode() >>> { >>> // Order the country names of countryDataTable. >>> string filterExpression = ""; >>> string sortExpression = "CountryNameUK asc"; >>> DataRow[] countryList = > _countryDataTable.Select(filterExpression, sortExpression); >>> >>> this.comboBoxCountry.DataSource = countryList; >>> this.comboBoxCountry.ValueMember = "Id"; >>> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >>> } >>> >>> And: >>> private void ChangeCountrySelection() >>> { >>> if (this.comboBoxCountry.SelectedValue != null) >>> { >>> _countryId = > Convert.ToInt32(this.comboBoxCountry.SelectedValue); >>> this.comboBoxPostalCode.Text = String.Empty; >>> // Read postal codes of selected country. >>> FillComboBoxPostalCode(); >>> } >>> } >>> >>> This is pretty standard using tableadapters and bindingnavigator. >>> Nothing > fancy. >>> >>> /gustav >>> >>> >>>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >>> I am trying to set up a combo box bound to a field in the form such >>> that > the combo populates with >>> the PKID and a text field, and when the form opens, selecting >>> something > from the combo drops the >>> PKID into the field in the form. >>> >>> I am not getting what to set the properties of the combo box to. I >>> am > pulling the ID and the string >>> to display into the combo but I cannot cause the combo to store the >>> id > and release control to the >>> next control on the form. >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From dbdoug at gmail.com Wed Mar 16 10:28:18 2011 From: dbdoug at gmail.com (Doug Steele) Date: Wed, 16 Mar 2011 08:28:18 -0700 Subject: [dba-VB] VS2010 install Message-ID: Hello all: I just finished doing a clean install of Windows 7 then VS2010. To my surprise, I also appear to have a functioning install of VS2008, loading with the name 'Shell' where my old VS2008 says 'Professional Edition'. I couldn't see anything on the net about this; does anyone have any further information? Thanks, Doug From DWUTKA at Marlow.com Fri Mar 18 14:12:52 2011 From: DWUTKA at Marlow.com (Drew Wutka) Date: Fri, 18 Mar 2011 14:12:52 -0500 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: Message-ID: 500 megs... that's about 10 minutes... wouldn't have time for a full dinner.... Drew -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-810 7-475d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com The information contained in this transmission is intended only for the person or entity to which it is addressed and may contain II-VI Proprietary and/or II-VI Business Sensitive material. If you are not the intended recipient, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. You are notified that any review, retransmission, copying, disclosure, dissemination, or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. From stuart at lexacorp.com.pg Fri Mar 18 18:50:01 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 09:50:01 +1000 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: , Message-ID: <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg> That's about 10 hours and US$35 for me :-( -- Stuart On 18 Mar 2011 at 14:12, Drew Wutka wrote: > 500 megs... that's about 10 minutes... wouldn't have time for a full > dinner.... > > Drew > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com > Subject: [dba-VB] Visual Studio 2010 SP1 > > Hi all > > If you go for this: > http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8 > 10 7-475d-948a-ef22627e57a5&displaylang=en > > it's typically a 500MB+ download with a streaming install. > > Don't go for coffee during this, don't go for lunch, not even a > single-course dinner, don't settle for less than a full menu dinner > followed by coffee and brandy. Or book an evening in town! > > /gustav > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > The information contained in this transmission is intended only for > the person or entity to which it is addressed and may contain II-VI > Proprietary and/or II-VI Business Sensitive material. If you are not > the intended recipient, please contact the sender immediately and > destroy the material in its entirety, whether electronic or hard copy. > You are notified that any review, retransmission, copying, disclosure, > dissemination, or other use of, or taking of any action in reliance > upon this information by persons or entities other than the intended > recipient is prohibited. > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Mar 18 19:02:30 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 03:02:30 +0300 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg> References: , <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg> Message-ID: Hi Stuart -- <<< That's about 10 hours and US$35 for me :-( >>> But you never have a snow there, and your local police doesn't mind if you drive home your car after a couple of beers? :) I mean 500megs download takes just a couple of minutes here and near to zero expences but we still have a snow here and Finnish Gulf is covered by 1 meter+ wide ice and I'm skiing here almost everyday, and if stopped by local road police after a couple beers I would definitely loose my car driving license for 1.5 years, at least... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart McLachlan Sent: 19 ????? 2011 ?. 2:50 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Visual Studio 2010 SP1 That's about 10 hours and US$35 for me :-( -- Stuart On 18 Mar 2011 at 14:12, Drew Wutka wrote: > 500 megs... that's about 10 minutes... wouldn't have time for a full > dinner.... > > Drew > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com > Subject: [dba-VB] Visual Studio 2010 SP1 > > Hi all > > If you go for this: > http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8 > 10 7-475d-948a-ef22627e57a5&displaylang=en > > it's typically a 500MB+ download with a streaming install. > > Don't go for coffee during this, don't go for lunch, not even a > single-course dinner, don't settle for less than a full menu dinner > followed by coffee and brandy. Or book an evening in town! > > /gustav > > From stuart at lexacorp.com.pg Fri Mar 18 19:09:18 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 10:09:18 +1000 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: , <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg>, Message-ID: <4D83F42E.13556.8558BC@stuart.lexacorp.com.pg> Yep, The good points still outweigh the bad. :-) -- Stuart On 19 Mar 2011 at 3:02, Shamil Salakhetdinov wrote: > Hi Stuart -- > > <<< > That's about 10 hours and US$35 for me :-( > >>> > But you never have a snow there, and your local police doesn't mind if > you drive home your car after a couple of beers? :) > > I mean 500megs download takes just a couple of minutes here and near > to zero expences but we still have a snow here and Finnish Gulf is > covered by 1 meter+ wide ice and I'm skiing here almost everyday, and > if stopped by local road police after a couple beers I would > definitely loose my car driving license for 1.5 years, at least... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart > McLachlan Sent: 19 ????? 2011 ?. 2:50 To: Discussion concerning Visual > Basic and related programming issues. Subject: Re: [dba-VB] Visual > Studio 2010 SP1 > > That's about 10 hours and US$35 for me :-( > > -- > Stuart > > On 18 Mar 2011 at 14:12, Drew Wutka wrote: > > > 500 megs... that's about 10 minutes... wouldn't have time for a full > > dinner.... > > > > Drew > > > > -----Original Message----- > > From: dba-vb-bounces at databaseadvisors.com > > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav > > Brock Sent: Friday, March 11, 2011 11:59 AM To: > > dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 > > > > Hi all > > > > If you go for this: > > http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6 > > -8 10 7-475d-948a-ef22627e57a5&displaylang=en > > > > it's typically a 500MB+ download with a streaming install. > > > > Don't go for coffee during this, don't go for lunch, not even a > > single-course dinner, don't settle for less than a full menu dinner > > followed by coffee and brandy. Or book an evening in town! > > > > /gustav > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Mar 18 21:08:01 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 18 Mar 2011 22:08:01 -0400 Subject: [dba-VB] SHA1 to compute a hash Message-ID: <4D841001.2060200@colbyconsulting.com> In my databases I create SHA1 hashes to enable joining between tables and pull identical records (identical for the fields hashed). I create: 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the three values and feed them into SHa1 and out pops a number which I store in a field in my table. 2) A HashFamily of the Zip5, Zip4, Addr ad LName. 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. I am getting known collisions between different addresses (I have discovered and investigated collisions) in my HashAddr when I have many millions of addresses. I need to address this. Back when I made my design decisions (2004) my hardware consisted of single core processors, 4 gigs ram, Windows x32 etc. Now I have 8 cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great extent constrained by my hardware "back in the day" whereas I am much less so now. I am about to redesign my process. I am considering simply appending in the city and state strings to all of the inputs: Addr, City, St, Zip5, Zip4 as the address base and then the same with LName and FName for the other two respective hashes. The objective is to minimize hash collisions, not prevent some crypto attack. I use these hash fields to join between multi million record tables so If I need to discover info in TableA where the HashAddr is the same as in TableB, I need the probability of a collision between different addresses (family/Person) to be as close to zero as I can get it. My questions are: 1) Whether anyone out there is using a hash in this manner? 2) Has anyone seen a table of collision probability between messages of a given (short) message length. My message is 9 digits for the zip5/4 and the address could be something as short as PO Box 1, or Apt 1. IOW the total message length of 14 is pretty common. Adding the state would give me minimum message lengths of only 16 and City would only add a few more characters. 3) Does anyone know if just adding the same data back in again would decrease the collision probability. IOW Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. Any experience out there? -- John W. Colby www.ColbyConsulting.com From stuart at lexacorp.com.pg Sat Mar 19 05:20:16 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 20:20:16 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D841001.2060200@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com> Message-ID: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> How are you creating your hash? Can you post a few examples of different data strings and colliding SHA1 hashes. I can probably make a lot of money out of them. AFAIK, no one other than you has found any. -- Stuart On 18 Mar 2011 at 22:08, jwcolby wrote: > In my databases I create SHA1 hashes to enable joining between tables > and pull identical records (identical for the fields hashed). I > create: > > 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the > three values and feed them into SHa1 and out pops a number which I > store in a field in my table. > > 2) A HashFamily of the Zip5, Zip4, Addr ad LName. > > 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. > > I am getting known collisions between different addresses (I have > discovered and investigated collisions) in my HashAddr when I have > many millions of addresses. I need to address this. > > Back when I made my design decisions (2004) my hardware consisted of > single core processors, 4 gigs ram, Windows x32 etc. Now I have 8 > cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great extent > constrained by my hardware "back in the day" whereas I am much less so > now. > > I am about to redesign my process. > > I am considering simply appending in the city and state strings to all > of the inputs: Addr, City, St, Zip5, Zip4 as the address base and then > the same with LName and FName for the other two respective hashes. > > The objective is to minimize hash collisions, not prevent some crypto > attack. I use these hash fields to join between multi million record > tables so If I need to discover info in TableA where the HashAddr is > the same as in TableB, I need the probability of a collision between > different addresses (family/Person) to be as close to zero as I can > get it. > > My questions are: > > 1) Whether anyone out there is using a hash in this manner? > 2) Has anyone seen a table of collision probability between messages > of a given (short) message length. My message is 9 digits for the > zip5/4 and the address could be something as short as PO Box 1, or Apt > 1. IOW the total message length of 14 is pretty common. Adding the > state would give me minimum message lengths of only 16 and City would > only add a few more characters. 3) Does anyone know if just adding the > same data back in again would decrease the collision probability. IOW > Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. > > Any experience out there? > > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 06:20:26 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 21:20:26 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> Using your SHA1 function, what message digests do you get for the standard test cases: 1. abc 2. abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq These should return: 1. A9993E36 4706816A BA3E2571 7850C26C 9CD0D89D 2. 84983E44 1C3BD26E BAAE4AA1 F95129E5 E54670F1 See Examples 1 and 2 at http://www.itl.nist.gov/fipspubs/fip180-1.htm -- Stuart On 19 Mar 2011 at 20:20, Stuart McLachlan wrote: > How are you creating your hash? > > Can you post a few examples of different data strings and colliding > SHA1 hashes. I can probably make a lot of money out of them. > AFAIK, no one other than you has found any. > > -- > Stuart > > > On 18 Mar 2011 at 22:08, jwcolby wrote: > > > In my databases I create SHA1 hashes to enable joining between > > tables and pull identical records (identical for the fields hashed). > > I create: > > > > 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the > > three values and feed them into SHa1 and out pops a number which I > > store in a field in my table. > > > > 2) A HashFamily of the Zip5, Zip4, Addr ad LName. > > > > 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. > > > > I am getting known collisions between different addresses (I have > > discovered and investigated collisions) in my HashAddr when I have > > many millions of addresses. I need to address this. > > > > Back when I made my design decisions (2004) my hardware consisted of > > single core processors, 4 gigs ram, Windows x32 etc. Now I have 8 > > cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great extent > > constrained by my hardware "back in the day" whereas I am much less > > so now. > > > > I am about to redesign my process. > > > > I am considering simply appending in the city and state strings to > > all of the inputs: Addr, City, St, Zip5, Zip4 as the address base > > and then the same with LName and FName for the other two respective > > hashes. > > > > The objective is to minimize hash collisions, not prevent some > > crypto attack. I use these hash fields to join between multi > > million record tables so If I need to discover info in TableA where > > the HashAddr is the same as in TableB, I need the probability of a > > collision between different addresses (family/Person) to be as close > > to zero as I can get it. > > > > My questions are: > > > > 1) Whether anyone out there is using a hash in this manner? > > 2) Has anyone seen a table of collision probability between messages > > of a given (short) message length. My message is 9 digits for the > > zip5/4 and the address could be something as short as PO Box 1, or > > Apt 1. IOW the total message length of 14 is pretty common. Adding > > the state would give me minimum message lengths of only 16 and City > > would only add a few more characters. 3) Does anyone know if just > > adding the same data back in again would decrease the collision > > probability. IOW Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. > > > > Any experience out there? > > > > > > -- > > John W. Colby > > www.ColbyConsulting.com > > _______________________________________________ > > dba-VB mailing list > > dba-VB at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > http://www.databaseadvisors.com > > > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 07:06:47 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 22:06:47 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> Message-ID: <4D849C57.3867.31639E4@stuart.lexacorp.com.pg> That got my interest piqued. So I've just built a small DLL (13KB) to calculate SHA1 digests for either strings or files. All you need to use it is to put the DLL somewhere in your PATH such as %SYSTEMROOT\System32 and include a couple of declarations in your application. Declare Function SHA1StringDigest Lib "LXSHA1.DLL" Alias "SHA1STRINGDIGEST" (Message As String) As String Declare Function SHA1FileDigest Lib "LXSHA1.DLL" Alias "SHA1FILEDIGEST" (FileName As String) As String It correctly returns the test results below for either the string or a file containing the same data. If anyone wants a copy, let me know. -- Stuart On 19 Mar 2011 at 21:20, Stuart McLachlan wrote: > Using your SHA1 function, what message digests do you get for the > standard test cases: > > 1. abc > 2. abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq > > These should return: > > 1. A9993E36 4706816A BA3E2571 7850C26C 9CD0D89D > 2. 84983E44 1C3BD26E BAAE4AA1 F95129E5 E54670F1 > > See Examples 1 and 2 at > http://www.itl.nist.gov/fipspubs/fip180-1.htm > > > > -- > Stuart > > > On 19 Mar 2011 at 20:20, Stuart McLachlan wrote: > > > How are you creating your hash? > > > > Can you post a few examples of different data strings and colliding > > SHA1 hashes. I can probably make a lot of money out of them. > > AFAIK, no one other than you has found any. > > > > -- > > Stuart > > > > > > On 18 Mar 2011 at 22:08, jwcolby wrote: > > > > > In my databases I create SHA1 hashes to enable joining between > > > tables and pull identical records (identical for the fields > > > hashed). > > > I create: > > > > > > 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the > > > three values and feed them into SHa1 and out pops a number which I > > > store in a field in my table. > > > > > > 2) A HashFamily of the Zip5, Zip4, Addr ad LName. > > > > > > 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. > > > > > > I am getting known collisions between different addresses (I have > > > discovered and investigated collisions) in my HashAddr when I have > > > many millions of addresses. I need to address this. > > > > > > Back when I made my design decisions (2004) my hardware consisted > > > of single core processors, 4 gigs ram, Windows x32 etc. Now I > > > have 8 cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great > > > extent constrained by my hardware "back in the day" whereas I am > > > much less so now. > > > > > > I am about to redesign my process. > > > > > > I am considering simply appending in the city and state strings to > > > all of the inputs: Addr, City, St, Zip5, Zip4 as the address base > > > and then the same with LName and FName for the other two > > > respective hashes. > > > > > > The objective is to minimize hash collisions, not prevent some > > > crypto attack. I use these hash fields to join between multi > > > million record tables so If I need to discover info in TableA > > > where the HashAddr is the same as in TableB, I need the > > > probability of a collision between different addresses > > > (family/Person) to be as close to zero as I can get it. > > > > > > My questions are: > > > > > > 1) Whether anyone out there is using a hash in this manner? > > > 2) Has anyone seen a table of collision probability between > > > messages of a given (short) message length. My message is 9 > > > digits for the zip5/4 and the address could be something as short > > > as PO Box 1, or Apt 1. IOW the total message length of 14 is > > > pretty common. Adding the state would give me minimum message > > > lengths of only 16 and City would only add a few more characters. > > > 3) Does anyone know if just adding the same data back in again > > > would decrease the collision probability. IOW > > > Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. > > > > > > Any experience out there? > > > > > > > > > -- > > > John W. Colby > > > www.ColbyConsulting.com > > > _______________________________________________ > > > dba-VB mailing list > > > dba-VB at databaseadvisors.com > > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > > http://www.databaseadvisors.com > > > > > > > > > > > > > > _______________________________________________ > > dba-VB mailing list > > dba-VB at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > http://www.databaseadvisors.com > > > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 07:09:56 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 08:09:56 -0400 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com> <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: <4D849D14.70906@colbyconsulting.com> >AFAIK, no one other than you has found any. LOL. You are thinking of hashing messages with thousands of characters. here's the deal. Let's assume that I hash X characters in a single field. The message length is the key, not the number of fields. Assume also that I am really only hashing letters, numbers and special characters - the characters that are in a name or address. If I hash 100 1 character messages, my probability of a collision if extremely high since there are only about 128 such alphanumeric characters (even less). Now hash a thousand 1 character messages. The probably just climbed immensely. Now hash a million such 1 character messages. What I am really calculating is the probability that I will repeat the same character string. However if I up the number of characters, let's say 10 character messages. *Now* the hash is really trying to prevent the same output for two *different* inputs. Well that is always what a hash is trying to do. But the point is that the probability of a collision decreases with message length increase. The probability of a collision between any two messages of 10 random characters is much higher than the probability of a collision between any two 100 character messages, and the probability of a collision between any two 1000 character messages is much higher still. That is just the way hash functions work. Nobody claims that hashes don't create collisions, and everyone agrees that the longer the message, the lower the probability of a collision between two messages (to a point). So, my address "messages" currently look like 89364 4456 PO Box 1 76543 9876 Apt 2 97867 3546 1723 Twin Pines Dr The point here is that all address "messages" are short. Now turn the address message into 89364 4456 PO Box 1 CA San Diego 76543 9876 Apt 2 TX Dallas 97867 3546 1723 Twin Pines Dr NC Hudson And the length of the message increases fairly dramatically, decreasing the probability of a collision. Remember too that I am hashing hundreds of millions of records - about 350 million addresses in various tables so far. The other thing that affects the probability of a hash collision is the number of messages hashed. Hash enough records and you *will* 100% probability create a hash that is the same for two different input strings - even for long messages. That is just the nature of the business. The probability of a collision is determined by three things. 1) The (strength of the) hash algorithm. 2) The message length 3) The quantity of messages. 1) I am using a fairly strong hash algorithm - SHA1 2) I am using a fairly short hash message - Zip5 / zip4 / Addr 3) I am hashing a fairly high number of messages - 350 million (and counting) messages (addresses) Believe me, hashes have already occurred, I have seen them. It is an easy thing to test. Join two tables on the same hash and in the query compare that each address field is the same between the two tables. Zip5 <> zip5 *or* zip4 <> zip4 *or* addr <> addr. If the hash is the same (inner join) AND one of the three fields is different then you have a collision. I have done this test and I have found collisions between my address hash. I have never tested my last name or first name hash fields. The problem with finding hashes is that they are rare, and in order to do an exhaustive test I would need to pull all the hash and data fields into a single table and compare that table against itself. My data is contained in a bunch of databases. 11 million "dogs and cats", 21 million "kids", 7 million "smokers", 23 million (and about to double) emails. 65 million "all adults", 100 million "mortgage" etc. So it is difficult to find hash collisions because (at least in the small tables) the probability of a collision is low entirely within a table. Believe me, it *does* happen though (at least for the address hash). John W. Colby www.ColbyConsulting.com On 3/19/2011 6:20 AM, Stuart McLachlan wrote: > How are you creating your hash? > > Can you post a few examples of different data strings and colliding SHA1 hashes. I can > probably make a lot of money out of them. AFAIK, no one other than you has found any. > From jwcolby at colbyconsulting.com Sat Mar 19 07:11:32 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 08:11:32 -0400 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D849C57.3867.31639E4@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> <4D849C57.3867.31639E4@stuart.lexacorp.com.pg> Message-ID: <4D849D74.2040104@colbyconsulting.com> I will do a collision test in my email file. AFAICT I have some collisions in there. If I find any I will send you the message and hash so that you can see what collisions look like. John W. Colby www.ColbyConsulting.com On 3/19/2011 8:06 AM, Stuart McLachlan wrote: > That got my interest piqued. > > So I've just built a small DLL (13KB) to calculate SHA1 digests for either strings or files. > All you need to use it is to put the DLL somewhere in your PATH such as > %SYSTEMROOT\System32 and include a couple of declarations in your application. > > Declare Function SHA1StringDigest Lib "LXSHA1.DLL" Alias "SHA1STRINGDIGEST" > (Message As String) As String > > Declare Function SHA1FileDigest Lib "LXSHA1.DLL" Alias "SHA1FILEDIGEST" (FileName > As String) As String > > It correctly returns the test results below for either the string or a file containing the same > data. > > If anyone wants a copy, let me know. > From harlan at powerzone.com Sat Mar 19 07:17:25 2011 From: harlan at powerzone.com (Harlan Koehn) Date: Sat, 19 Mar 2011 06:17:25 -0600 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D849D14.70906@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com> <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> <4D849D14.70906@colbyconsulting.com> Message-ID: <76EEB091F7CED543A80B5C2BC153975E0165A7C108@PZSERVER01.powerzone.local> John, I'm fairly ignorant when it comes to hashing algorithms but I'm curious if you have considered using a hashing algorithm that would generate a longer hash, which I'm assuming could reduce the likelihood of a collision. Harlan From stuart at lexacorp.com.pg Sat Mar 19 07:51:13 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 22:51:13 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D849D14.70906@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D849D14.70906@colbyconsulting.com> Message-ID: <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> In the case of 1000 single character messages, you are bound to get collisions since there are only 256 possible original messages. You will be hashing the same value multiple times Apart from that, your understanding is incorrect. It doesn't matter how long the string is. The chances of a collision with two different messages remains the same. The message is hashed using padded blocks of a fixed length. There is no more chance of a collision between "a" and "b" than there is between "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". Specifically, the chance of a collision within n different messages, using b bits of encryption is (n*(n-1)/2) * (1/2^b). Note that the length of the message doesn't come into that equation. The probability of a collision is determined by only TWO things. 1) The length of the digest. 2) The quantity of messages. As for "everyone agrees". Noone who understands how it works, agrees. -- Stuart On 19 Mar 2011 at 8:09, jwcolby wrote: > >AFAIK, no one other than you has found any. > > LOL. > > You are thinking of hashing messages with thousands of characters. > here's the deal. > > Let's assume that I hash X characters in a single field. The message > length is the key, not the number of fields. Assume also that I am > really only hashing letters, numbers and special characters - the > characters that are in a name or address. > > > If I hash 100 1 character messages, my probability of a collision if > extremely high since there are only about 128 such alphanumeric > characters (even less). > > Now hash a thousand 1 character messages. The probably just climbed > immensely. Now hash a million such 1 character messages. > What I am really calculating is the probability that I will repeat the > same character string. > > However if I up the number of characters, let's say 10 character > messages. *Now* the hash is really trying to prevent the same output > for two *different* inputs. Well that is always what a hash is trying > to do. But the point is that the probability of a collision decreases > with message length increase. > > The probability of a collision between any two messages of 10 random > characters is much higher than the probability of a collision between > any two 100 character messages, and the probability of a collision > between any two 1000 character messages is much higher still. > > That is just the way hash functions work. Nobody claims that hashes > don't create collisions, and everyone agrees that the longer the > message, the lower the probability of a collision between two messages > (to a point). > > So, my address "messages" currently look like > > 89364 4456 PO Box 1 > 76543 9876 Apt 2 > 97867 3546 1723 Twin Pines Dr > > The point here is that all address "messages" are short. Now turn the > address message into > > 89364 4456 PO Box 1 CA San Diego > 76543 9876 Apt 2 TX Dallas > 97867 3546 1723 Twin Pines Dr NC Hudson > > And the length of the message increases fairly dramatically, > decreasing the probability of a collision. > > Remember too that I am hashing hundreds of millions of records - about > 350 million addresses in various tables so far. The other thing that > affects the probability of a hash collision is the number of messages > hashed. Hash enough records and you *will* 100% probability create a > hash that is the same for two different input strings - even for long > messages. That is just the nature of the business. > > The probability of a collision is determined by three things. > > 1) The (strength of the) hash algorithm. > 2) The message length > 3) The quantity of messages. ... From shamil at smsconsulting.spb.ru Sat Mar 19 08:03:36 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 16:03:36 +0300 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com> <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: Hi Stuart and John, <<< AFAIK, no one other than you has found any >>> Yes, I was also confused how it comes to get SHA1 hashes collisions but then I have thought that John probably "compacts" SHA1 hashes into 8 bytes surrogate keys? Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions for 6 millions cycles for 8 bytes keys. Please correct me if you find some errors in my sample code Thank you. -- Shamil P.S. Code: public class GenerateHashes { private static Dictionary _generatedHashes; private static Dictionary _generatedLongHashes; public static void Run(int minHashedStringLength, int maxHashedStringLength, long testCasesQty) { System.Console.WriteLine("Started at: " + System.DateTime.Now); _generatedHashes = new Dictionary(); _generatedLongHashes = new Dictionary(); int collisionsQty = 0; int longHashCollisionsQty = 0; decimal totalLengthOfProcessedStrings = 0; decimal averageStringLength = 0; Random random = new Random((int)System.DateTime.Now.Ticks); for (long i = 1; i <= testCasesQty; i++) { int stringLength = random.Next(minHashedStringLength, maxHashedStringLength); byte[] plainTextBytes = (byte[])Array.CreateInstance(typeof(byte), stringLength); random.NextBytes(plainTextBytes); System.Security.Cryptography.HashAlgorithm hash = new SHA1Managed(); byte[] hashBytes = hash.ComputeHash(plainTextBytes); string hashValue = Convert.ToBase64String(hashBytes); string testValue; if (_generatedHashes.TryGetValue(hashValue, out testValue)) collisionsQty++; else _generatedHashes.Add(hashValue, hashValue); if (i % 1000000 == 0) System.Console.WriteLine( "{0}: i={1}, cs={2}, cu={3}", DateTime.Now, i, collisionsQty, longHashCollisionsQty); ulong longHash = 0xFFFFFFFF; foreach (byte b in hashValue) { longHash ^= b; longHash = longHash << 8 + 0xFF; } ulong longHashTestValue; if (_generatedLongHashes.TryGetValue(longHash, out longHashTestValue)) { //Long Hash Collision longHashCollisionsQty++; System.Console.WriteLine(" * step {0}, hash '{1}', longHash: {2:X}", i, hashValue, longHash); } else _generatedLongHashes.Add(longHash, longHash); totalLengthOfProcessedStrings += stringLength; averageStringLength = totalLengthOfProcessedStrings / i; } System.Console.WriteLine( "\nTOTALS:\n" + "Min String Length = {0}\n" + "Max String Length = {1}\n" + "Avg String Length = {2}\n" + "Test cases Qty = {3}\n" + "Collisions Qty = {4}\n" + "ulong Collisions Qty = {5}", minHashedStringLength, maxHashedStringLength, averageStringLength, testCasesQty, collisionsQty, longHashCollisionsQty); System.Console.WriteLine("Finished at: " + System.DateTime.Now); } } P.P.S. Stats Started at: 19/03/2011 15:51:04 19/03/2011 15:51:16: i=1000000, cs=0, cu=0 * step 1712197, hash 'Rt4NY5FHg0uMDszVql+l6ksx6Tg=', longHash: 36D7CFC36A99DE80 19/03/2011 15:51:29: i=2000000, cs=0, cu=1 * step 2040324, hash 'laoPHdqlfeZOx9DI8RDaQZjhKZs=', longHash: D1B5AB44BB5CDE80 * step 2043449, hash '1m+tgKu7/l9fl8nUNuz5I/OWF5E=', longHash: C95F3EBC66B15E80 19/03/2011 15:51:44: i=3000000, cs=0, cu=3 19/03/2011 15:52:00: i=4000000, cs=0, cu=3 * step 4369446, hash '2sZrglXDp3NnNPuqdILqwmUKq30=', longHash: F7DB565F166C1E80 19/03/2011 15:52:17: i=5000000, cs=0, cu=4 TOTALS: Min String Length = 7 Max String Length = 25 Avg String Length = 15.4981354 Test cases Qty = 5,000,000 Collisions Qty = 0 ulong Collisions Qty = 4 Finished at: 19/03/2011 15:52:17 -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart McLachlan Sent: 19 ????? 2011 ?. 13:20 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SHA1 to compute a hash How are you creating your hash? Can you post a few examples of different data strings and colliding SHA1 hashes. I can probably make a lot of money out of them. AFAIK, no one other than you has found any. -- Stuart <<< snip >>> From stuart at lexacorp.com.pg Sat Mar 19 08:10:47 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 23:10:47 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D849D14.70906@colbyconsulting.com>, <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> Message-ID: <4D84AB57.31948.350CF8F@stuart.lexacorp.com.pg> I've just realised where the confusion comes in. When you talk about the chance of collisions increasing decreasing message length, you are talking about general hashing algorithms such as CRC etc. You need to undersand the difference between simple and cryptographic hashing. Cryptographic hashing algorithms are specifically designed to be "collision resistant". I repeat, to date no one has found a collision in SHA1 (except you apparently ) -- Stuart On 19 Mar 2011 at 22:51, Stuart McLachlan wrote: > In the case of 1000 single character messages, you are bound to get > collisions since there are only 256 possible original messages. You > will be hashing the same value multiple times > > Apart from that, your understanding is incorrect. It doesn't matter > how long the string is. The chances of a collision with two different > messages remains the same. > > The message is hashed using padded blocks of a fixed length. > There is no more chance of a collision between "a" and "b" than there > is between "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". > > Specifically, the chance of a collision within n different messages, > using b bits of encryption is (n*(n-1)/2) * (1/2^b). > > Note that the length of the message doesn't come into that equation. > > The probability of a collision is determined by only TWO things. > > 1) The length of the digest. > 2) The quantity of messages. > > As for "everyone agrees". Noone who understands how it works, agrees. > > -- > Stuart > > On 19 Mar 2011 at 8:09, jwcolby wrote: > > > >AFAIK, no one other than you has found any. > > > > LOL. > > > > You are thinking of hashing messages with thousands of characters. > > here's the deal. > > > > Let's assume that I hash X characters in a single field. The > > message length is the key, not the number of fields. Assume also > > that I am really only hashing letters, numbers and special > > characters - the characters that are in a name or address. > > > > > > If I hash 100 1 character messages, my probability of a collision if > > extremely high since there are only about 128 such alphanumeric > > characters (even less). > > > > Now hash a thousand 1 character messages. The probably just climbed > > immensely. Now hash a million such 1 character messages. What I am > > really calculating is the probability that I will repeat the same > > character string. > > > > However if I up the number of characters, let's say 10 character > > messages. *Now* the hash is really trying to prevent the same > > output for two *different* inputs. Well that is always what a hash > > is trying to do. But the point is that the probability of a > > collision decreases with message length increase. > > > > The probability of a collision between any two messages of 10 random > > characters is much higher than the probability of a collision > > between any two 100 character messages, and the probability of a > > collision between any two 1000 character messages is much higher > > still. > > > > That is just the way hash functions work. Nobody claims that hashes > > don't create collisions, and everyone agrees that the longer the > > message, the lower the probability of a collision between two > > messages (to a point). > > > > So, my address "messages" currently look like > > > > 89364 4456 PO Box 1 > > 76543 9876 Apt 2 > > 97867 3546 1723 Twin Pines Dr > > > > The point here is that all address "messages" are short. Now turn > > the address message into > > > > 89364 4456 PO Box 1 CA San Diego > > 76543 9876 Apt 2 TX Dallas > > 97867 3546 1723 Twin Pines Dr NC Hudson > > > > And the length of the message increases fairly dramatically, > > decreasing the probability of a collision. > > > > Remember too that I am hashing hundreds of millions of records - > > about 350 million addresses in various tables so far. The other > > thing that affects the probability of a hash collision is the number > > of messages hashed. Hash enough records and you *will* 100% > > probability create a hash that is the same for two different input > > strings - even for long messages. That is just the nature of the > > business. > > > > The probability of a collision is determined by three things. > > > > 1) The (strength of the) hash algorithm. > > 2) The message length > > 3) The quantity of messages. > ... > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 08:20:11 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 23:20:11 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, Message-ID: <4D84AD8B.14064.3596D15@stuart.lexacorp.com.pg> If he is compacting a 160 bit hash to a 64 bit hash, he is certainly increasing the odds of a collision - but I suspect that even at 64 bits, the chances of a collision would be very low. But that does not appear to be what you are testing. It looks as though you created 6 million random strings and four of them turned out to be identical. Therefore also had the same SHA1 digest. What happens if you try it again, making sure that all of your strings are unique. Alternatively, what do you get if you add some code to show the pairs of strings. Are they different? -- Stuart On 19 Mar 2011 at 16:03, Shamil Salakhetdinov wrote: > Hi Stuart and John, > > <<< > AFAIK, no one other than you has found any > >>> > Yes, I was also confused how it comes to get SHA1 hashes collisions > but then I have thought that John probably "compacts" SHA1 hashes into > 8 bytes surrogate keys? > > Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions > for 6 millions cycles for 8 bytes keys. > > Please correct me if you find some errors in my sample code > > Thank you. > > -- > Shamil > > P.S. Code: > > public class GenerateHashes > { > private static Dictionary _generatedHashes; > private static Dictionary _generatedLongHashes; > > public static void Run(int minHashedStringLength, int > maxHashedStringLength, long testCasesQty) > { > System.Console.WriteLine("Started at: " + > System.DateTime.Now); > > _generatedHashes = new Dictionary(); > _generatedLongHashes = new Dictionary(); > > int collisionsQty = 0; > int longHashCollisionsQty = 0; > > decimal totalLengthOfProcessedStrings = 0; > decimal averageStringLength = 0; > > > Random random = new > Random((int)System.DateTime.Now.Ticks); for (long i = 1; i > <= testCasesQty; i++) { > int stringLength = random.Next(minHashedStringLength, > maxHashedStringLength); > byte[] plainTextBytes = > (byte[])Array.CreateInstance(typeof(byte), stringLength); > random.NextBytes(plainTextBytes); > System.Security.Cryptography.HashAlgorithm hash = new > SHA1Managed(); > byte[] hashBytes = hash.ComputeHash(plainTextBytes); > string hashValue = Convert.ToBase64String(hashBytes); > string testValue; if > (_generatedHashes.TryGetValue(hashValue, out > testValue)) > collisionsQty++; > else _generatedHashes.Add(hashValue, hashValue); > if (i % 1000000 == 0) > System.Console.WriteLine( > "{0}: i={1}, cs={2}, cu={3}", > DateTime.Now, i, collisionsQty, > longHashCollisionsQty); > > ulong longHash = 0xFFFFFFFF; > foreach (byte b in hashValue) > { > longHash ^= b; > longHash = longHash << 8 + 0xFF; > } > ulong longHashTestValue; > if (_generatedLongHashes.TryGetValue(longHash, out > longHashTestValue)) > { > //Long Hash Collision > longHashCollisionsQty++; > System.Console.WriteLine(" * step {0}, hash '{1}', > longHash: {2:X}", > i, hashValue, longHash); > } > else _generatedLongHashes.Add(longHash, longHash); > > totalLengthOfProcessedStrings += stringLength; > averageStringLength = totalLengthOfProcessedStrings / > i; > } > > System.Console.WriteLine( > "\nTOTALS:\n" + > "Min String Length = {0}\n" + > "Max String Length = {1}\n" + > "Avg String Length = {2}\n" + > "Test cases Qty = {3}\n" + > "Collisions Qty = {4}\n" + > "ulong Collisions Qty = {5}", > minHashedStringLength, maxHashedStringLength, > averageStringLength, testCasesQty, > collisionsQty, > longHashCollisionsQty); > > System.Console.WriteLine("Finished at: " + > System.DateTime.Now); > > } > } > > > P.P.S. Stats > > Started at: 19/03/2011 15:51:04 > 19/03/2011 15:51:16: i=1000000, cs=0, cu=0 > * step 1712197, hash 'Rt4NY5FHg0uMDszVql+l6ksx6Tg=', longHash: > 36D7CFC36A99DE80 > 19/03/2011 15:51:29: i=2000000, cs=0, cu=1 > * step 2040324, hash 'laoPHdqlfeZOx9DI8RDaQZjhKZs=', longHash: > D1B5AB44BB5CDE80 > * step 2043449, hash '1m+tgKu7/l9fl8nUNuz5I/OWF5E=', longHash: > C95F3EBC66B15E80 > 19/03/2011 15:51:44: i=3000000, cs=0, cu=3 > 19/03/2011 15:52:00: i=4000000, cs=0, cu=3 > * step 4369446, hash '2sZrglXDp3NnNPuqdILqwmUKq30=', longHash: > F7DB565F166C1E80 > 19/03/2011 15:52:17: i=5000000, cs=0, cu=4 > > TOTALS: > Min String Length = 7 > Max String Length = 25 > Avg String Length = 15.4981354 > Test cases Qty = 5,000,000 > Collisions Qty = 0 > ulong Collisions Qty = 4 > Finished at: 19/03/2011 15:52:17 > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart > McLachlan Sent: 19 ????? 2011 ?. 13:20 To: Discussion concerning > Visual Basic and related programming issues. Subject: Re: [dba-VB] > SHA1 to compute a hash > > How are you creating your hash? > > Can you post a few examples of different data strings and colliding > SHA1 hashes. I can probably make a lot of money out of them. > AFAIK, no one other than you has found any. > > -- > Stuart > > <<< snip >>> > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 08:21:58 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 09:21:58 -0400 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D849D14.70906@colbyconsulting.com> <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> Message-ID: <4D84ADF6.4000509@colbyconsulting.com> Stuart, Thanks for that. I am not a cryptologist and all I can do is try to use a tool I do not understand to solve a problem I do understand. If my hashes really are that "unique / secure" then I will stop worrying about it. I still need to do a check of actual data though because I went looking one day and discovered "collisions". Given that we believe the collision is not in the key itself, it must have been in my method of finding the collision. In any case, I am not going to worry any more so thanks for the peace of mind! ;) John W. Colby www.ColbyConsulting.com On 3/19/2011 8:51 AM, Stuart McLachlan wrote: > In the case of 1000 single character messages, you are bound to get collisions since there > are only 256 possible original messages. You will be hashing the same value multiple times > > Apart from that, your understanding is incorrect. It doesn't matter how long the string is. The > chances of a collision with two different messages remains the same. > > The message is hashed using padded blocks of a fixed length. > There is no more chance of a collision between "a" and "b" than there is between > "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". > > Specifically, the chance of a collision within n different messages, using b bits of encryption is > (n*(n-1)/2) * (1/2^b). > > Note that the length of the message doesn't come into that equation. > > The probability of a collision is determined by only TWO things. > > 1) The length of the digest. > 2) The quantity of messages. > > As for "everyone agrees". Noone who understands how it works, agrees. > From shamil at smsconsulting.spb.ru Sat Mar 19 08:21:26 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 16:21:26 +0300 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: References: <4D841001.2060200@colbyconsulting.com><4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: <49B2F00A3FD3442685D8283AE7678544@nant> Hi All -- Sorry I have mentioned 6 millions cycles but my sample was for just 5 millions, and I have got some collisions for 8 bytes long surrogate keys. Here I'm posting a patch for hashing part of the code - and no collisions now for 8 bytes long hashes for 5 millions cycles. .... UInt64 longHashResult = 0xFFFFFFFFFFFFFFFF; UInt64 longHash = 0xFFFFFFFFFFFFFFFF; int shiftCounter = 0; foreach (byte b in hashValue) { longHash ^= b; longHash = longHash << 8 + 0xFF; if (++shiftCounter == 8) { longHashResult ^= longHash; shiftCounter = 0; } } longHash = longHashResult; ... Started at: 19/03/2011 16:08:53 19/03/2011 16:09:04: i=1000000, cs=0, cu=0 19/03/2011 16:09:17: i=2000000, cs=0, cu=0 19/03/2011 16:09:33: i=3000000, cs=0, cu=0 19/03/2011 16:09:49: i=4000000, cs=0, cu=0 19/03/2011 16:10:06: i=5000000, cs=0, cu=0 TOTALS: Min String Length = 7 Max String Length = 25 Avg String Length = 15.4966972 Test cases Qty = 5000000 Collisions Qty = 0 ulong Collisions Qty = 0 Finished at: 19/03/2011 16:10:06 Please correct me if I've got wrong somewhere. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 19 ????? 2011 ?. 16:04 To: stuart at lexacorp.com.pg; 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] SHA1 to compute a hash Hi Stuart and John, <<< AFAIK, no one other than you has found any >>> Yes, I was also confused how it comes to get SHA1 hashes collisions but then I have thought that John probably "compacts" SHA1 hashes into 8 bytes surrogate keys? Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions for 6 millions cycles for 8 bytes keys. Please correct me if you find some errors in my sample code Thank you. -- Shamil <<< snip >>> From jwcolby at colbyconsulting.com Sat Mar 19 08:24:54 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 09:24:54 -0400 Subject: [dba-VB] On a lighter note Message-ID: <4D84AEA6.5060107@colbyconsulting.com> I rooted my droid X last night, bought a "freezer" program, froze all the bloatware, and downloaded and installed a tether program. Here at my house I get about 1.25 mbit down and .5 mbit up via the tether. Not bad considering it is over cell. -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sat Mar 19 08:24:33 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 16:24:33 +0300 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84AD8B.14064.3596D15@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D84AD8B.14064.3596D15@stuart.lexacorp.com.pg> Message-ID: <14BB8E00F24E4EBAB5842E5D75537D2F@nant> Hi Stuart -- <<< It looks as though you created 6 million random strings and four of them turned out to be identical. >>> No, all SHA1 strings are different - these are 64 bit hashes of that strings, which have got some collisions. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart McLachlan Sent: 19 ????? 2011 ?. 16:20 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SHA1 to compute a hash If he is compacting a 160 bit hash to a 64 bit hash, he is certainly increasing the odds of a collision - but I suspect that even at 64 bits, the chances of a collision would be very low. But that does not appear to be what you are testing. It looks as though you created 6 million random strings and four of them turned out to be identical. Therefore also had the same SHA1 digest. What happens if you try it again, making sure that all of your strings are unique. Alternatively, what do you get if you add some code to show the pairs of strings. Are they different? -- Stuart On 19 Mar 2011 at 16:03, Shamil Salakhetdinov wrote: > Hi Stuart and John, > > <<< > AFAIK, no one other than you has found any > >>> > Yes, I was also confused how it comes to get SHA1 hashes collisions > but then I have thought that John probably "compacts" SHA1 hashes into > 8 bytes surrogate keys? > > Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions > for 6 millions cycles for 8 bytes keys. > > Please correct me if you find some errors in my sample code > > Thank you. > > -- > Shamil > <<< snip >>> From stuart at lexacorp.com.pg Sat Mar 19 08:29:55 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 23:29:55 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84ADF6.4000509@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com>, <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg>, <4D84ADF6.4000509@colbyconsulting.com> Message-ID: <4D84AFD3.25991.3625487@stuart.lexacorp.com.pg> Using a 160 bit SHA1, the odds of getting a collision are 50% when you have approximately 2^80 messages. If you ever get near that number of records, I suggest you switch to SHA2 which is 256 bit :-) -- Stuart On 19 Mar 2011 at 9:21, jwcolby wrote: > Stuart, > > Thanks for that. I am not a cryptologist and all I can do is try to > use a tool I do not understand to solve a problem I do understand. If > my hashes really are that "unique / secure" then I will stop worrying > about it. I still need to do a check of actual data though because I > went looking one day and discovered "collisions". Given that we > believe the collision is not in the key itself, it must have been in > my method of finding the collision. > > In any case, I am not going to worry any more so thanks for the peace > of mind! ;) > > John W. Colby > www.ColbyConsulting.com > > On 3/19/2011 8:51 AM, Stuart McLachlan wrote: > > In the case of 1000 single character messages, you are bound to get > > collisions since there are only 256 possible original messages. You > > will be hashing the same value multiple times > > > > Apart from that, your understanding is incorrect. It doesn't matter > > how long the string is. The chances of a collision with two > > different messages remains the same. > > > > The message is hashed using padded blocks of a fixed length. > > There is no more chance of a collision between "a" and "b" than > > there is between "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". > > > > Specifically, the chance of a collision within n different messages, > > using b bits of encryption is (n*(n-1)/2) * (1/2^b). > > > > Note that the length of the message doesn't come into that equation. > > > > The probability of a collision is determined by only TWO things. > > > > 1) The length of the digest. > > 2) The quantity of messages. > > > > As for "everyone agrees". Noone who understands how it works, > > agrees. > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 08:55:51 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 09:55:51 -0400 Subject: [dba-VB] Using the hash for real world problems Message-ID: <4D84B5E7.3010204@colbyconsulting.com> I got into the hashing business in order to create a surrogate key of sorts, in fact a trio of them. I have an address hash, a family hash and a person hash. I compute these three hashes for every table of persons. Now that I have these surrogate keys I have "identical" values in a single field where the input is identical in multiple fields. Thus I do not have to do multi-field joins where I have a hash for all of those fields. I am happy to hear that I am secure in my surrogate hash algorithm choice. OK, so I have these hashes. Every record has these three hashes. Now some definitions: HashAddr: A unique address is addr/zip5/zip4 different HashFamily: A unique family is LName/Addr/Zip5/Zip4 different HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. Using this information, I need to calculate the count of addresses with a single person, with two persons etc for as many combinations as I have. Addresses People 22,538,240 1 780,462,346 2 52,234 3 people etc etc till every quantity of people is found. What is the simplest way to accomplish this with SQL. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 19 09:13:51 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 10:13:51 -0400 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84B5E7.3010204@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com> Message-ID: <4D84BA1F.2060101@colbyconsulting.com> Ooops. My examples were hosed. Let's try again Addresses People 22,538,240 1 00,708,462 2 00,052,234 3 and so forth. What the client really wants to know is "how many people with emails are at a specific address" "how many people with emails are in a specific family" etc. I tie my address table to my email table using an auto increment long increment "I must be careful not to call this a surrogate" key and then perform the counts using the hash fields in a Group By / count query. John W. Colby www.ColbyConsulting.com On 3/19/2011 9:55 AM, jwcolby wrote: > I got into the hashing business in order to create a surrogate key of sorts, in fact a trio of them. > > I have an address hash, a family hash and a person hash. I compute these three hashes for every > table of persons. Now that I have these surrogate keys I have "identical" values in a single field > where the input is identical in multiple fields. Thus I do not have to do multi-field joins where I > have a hash for all of those fields. > > I am happy to hear that I am secure in my surrogate hash algorithm choice. > > OK, so I have these hashes. Every record has these three hashes. > > Now some definitions: > > HashAddr: A unique address is addr/zip5/zip4 different > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > Using this information, I need to calculate the count of addresses with a single person, with two > persons etc for as many combinations as I have. > > Addresses People > 22,538,240 1 > 780,462,346 2 > 52,234 3 people > etc > etc till every quantity of people is found. > > What is the simplest way to accomplish this with SQL. > > > > From shamil at smsconsulting.spb.ru Sat Mar 19 10:07:08 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 18:07:08 +0300 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84BA1F.2060101@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com> <4D84BA1F.2060101@colbyconsulting.com> Message-ID: Hi John -- I suppose you can reuse the following SQL expression for your case: select count(Occurrence) QtyTheSameOccurrences, Occurrence from ( select Country, Count(Country) Occurrence from Customer Group by Country ) o group by o.Occurrence order by o.Occurrence Just replace Country name columns with hashAddress column and Country table name with your table name ... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 19 ????? 2011 ?. 17:14 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Using the hash for real world problems Ooops. My examples were hosed. Let's try again Addresses People 22,538,240 1 00,708,462 2 00,052,234 3 and so forth. What the client really wants to know is "how many people with emails are at a specific address" "how many people with emails are in a specific family" etc. I tie my address table to my email table using an auto increment long increment "I must be careful not to call this a surrogate" key and then perform the counts using the hash fields in a Group By / count query. John W. Colby www.ColbyConsulting.com On 3/19/2011 9:55 AM, jwcolby wrote: > I got into the hashing business in order to create a surrogate key of sorts, in fact a trio of them. > > I have an address hash, a family hash and a person hash. I compute > these three hashes for every table of persons. Now that I have these > surrogate keys I have "identical" values in a single field where the > input is identical in multiple fields. Thus I do not have to do multi-field joins where I have a hash for all of those fields. > > I am happy to hear that I am secure in my surrogate hash algorithm choice. > > OK, so I have these hashes. Every record has these three hashes. > > Now some definitions: > > HashAddr: A unique address is addr/zip5/zip4 different > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > Using this information, I need to calculate the count of addresses > with a single person, with two persons etc for as many combinations as I have. > > Addresses People > 22,538,240 1 > 780,462,346 2 > 52,234 3 people > etc > etc till every quantity of people is found. > > What is the simplest way to accomplish this with SQL. > > > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Mar 19 14:50:55 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 15:50:55 -0400 Subject: [dba-VB] Most useful droid apps Message-ID: <4D85091F.5060807@colbyconsulting.com> What are the most useful apps you run on your droid? -- John W. Colby www.ColbyConsulting.com From davidmcafee at gmail.com Sat Mar 19 15:53:32 2011 From: davidmcafee at gmail.com (David McAfee) Date: Sat, 19 Mar 2011 13:53:32 -0700 Subject: [dba-VB] Most useful droid apps Message-ID: Advanced task killer Weather channel Lookout antivirus AK Notepad Bank of America Acar auto maintenance Barcode scanner Bubble level Bump Droid light Google goggles Google translate IMDB Key ring Print share Trapster Sent from my Droid phone. On Mar 19, 2011 12:52 PM, "jwcolby" wrote: From stuart at lexacorp.com.pg Sat Mar 19 18:11:26 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sun, 20 Mar 2011 09:11:26 +1000 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84BA1F.2060101@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com>, <4D84BA1F.2060101@colbyconsulting.com> Message-ID: <4D85381E.9973.576BE3C@stuart.lexacorp.com.pg> Caaution. All aircode below: To get the list output below, I would use something like Select Distinct Count(HashAddr) as Addresses, People >From (Select distinct HashAddr,Count(HashAddr) as People From tblPeople Group By HashAddr) Group By People Order by People To find out details for a *specific* Address Find the Hash for the desired Address and then pass it to something like this: Select Distinct StreetNo,Street,City,Zip,State,Count(HashAddr) as Address >From tblPeople P Inner Join tblEmails E on P.PK = E.PeopleFK Group By HashAddr StreetNo,Street,City,Zip,State Where HashAddr = @SearchHash On 19 Mar 2011 at 10:13, jwcolby wrote: > Ooops. My examples were hosed. > > Let's try again > > Addresses People > 22,538,240 1 > 00,708,462 2 > 00,052,234 3 > > and so forth. > > What the client really wants to know is > > "how many people with emails are at a specific address" > "how many people with emails are in a specific family" > > etc. > > I tie my address table to my email table using an auto increment long > increment "I must be careful not to call this a surrogate" key and > then perform the counts using the hash fields in a Group By / count > query. > > John W. Colby > www.ColbyConsulting.com > > On 3/19/2011 9:55 AM, jwcolby wrote: > > I got into the hashing business in order to create a surrogate key > > of sorts, in fact a trio of them. > > > > I have an address hash, a family hash and a person hash. I compute > > these three hashes for every table of persons. Now that I have these > > surrogate keys I have "identical" values in a single field where the > > input is identical in multiple fields. Thus I do not have to do > > multi-field joins where I have a hash for all of those fields. > > > > I am happy to hear that I am secure in my surrogate hash algorithm > > choice. > > > > OK, so I have these hashes. Every record has these three hashes. > > > > Now some definitions: > > > > HashAddr: A unique address is addr/zip5/zip4 different > > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > > > Using this information, I need to calculate the count of addresses > > with a single person, with two persons etc for as many combinations > > as I have. > > > > Addresses People > > 22,538,240 1 > > 780,462,346 2 > > 52,234 3 people > > etc > > etc till every quantity of people is found. > > > > What is the simplest way to accomplish this with SQL. > > > > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 18:21:18 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sun, 20 Mar 2011 09:21:18 +1000 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84B5E7.3010204@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com> Message-ID: <4D853A6E.20716.57FC618@stuart.lexacorp.com.pg> Answered on VB list. On 19 Mar 2011 at 9:55, jwcolby wrote: > I got into the hashing business in order to create a surrogate key of > sorts, in fact a trio of them. > > I have an address hash, a family hash and a person hash. I compute > these three hashes for every table of persons. Now that I have these > surrogate keys I have "identical" values in a single field where the > input is identical in multiple fields. Thus I do not have to do > multi-field joins where I have a hash for all of those fields. > > I am happy to hear that I am secure in my surrogate hash algorithm > choice. > > OK, so I have these hashes. Every record has these three hashes. > > Now some definitions: > > HashAddr: A unique address is addr/zip5/zip4 different > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > Using this information, I need to calculate the count of addresses > with a single person, with two persons etc for as many combinations as > I have. > > Addresses People > 22,538,240 1 > 780,462,346 2 > 52,234 3 people > etc > etc till every quantity of people is found. > > What is the simplest way to accomplish this with SQL. > > > > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 21:01:04 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 22:01:04 -0400 Subject: [dba-VB] Most useful droid apps In-Reply-To: References: Message-ID: <4D855FE0.9070004@colbyconsulting.com> I am trying various tethering solutions. My laptop is vista, my ip range is 192.168.122.x. I got WiFi Tether to work, but only without encryption. I was getting about 1.5 mbit down and .5 mbit up. As soon as I turned on encryption I was unable to set up the laptop password to match (that was the only thing that changed). No encryption is unacceptable to me as I would only really use this in public places, exactly where you would most want encryption. I tried barnacle. That one just didn't work at all. I am now trying PDANet which is a USB tether. Since the PC is tied to the phone by a usb, encryption seems to be unnecessary? Unfortunately this one is not working either. The PDANet desktop app says it is connected and the droid pdanet app says it is connected but the PC just can't see the internet. I can't ping anything etc. Any suggestions for this beast? John W. Colby www.ColbyConsulting.com On 3/19/2011 4:53 PM, David McAfee wrote: > Advanced task killer > Weather channel > Lookout antivirus > AK Notepad > Bank of America > Acar auto maintenance > Barcode scanner > Bubble level > Bump > Droid light > Google goggles > Google translate > IMDB > Key ring > Print share > Trapster > > Sent from my Droid phone. > On Mar 19, 2011 12:52 PM, "jwcolby" wrote: > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 21:34:03 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sun, 20 Mar 2011 12:34:03 +1000 Subject: [dba-VB] Most useful droid apps In-Reply-To: <4D855FE0.9070004@colbyconsulting.com> References: , <4D855FE0.9070004@colbyconsulting.com> Message-ID: <4D85679B.30595.6303F8F@stuart.lexacorp.com.pg> Are you on AT&T? If so, you may be interested in this: http://www.theregister.co.uk/2011/03/18/new_att_tethering_policy/ On 19 Mar 2011 at 22:01, jwcolby wrote: > I am trying various tethering solutions. My laptop is vista, my ip > range is 192.168.122.x. > > I got WiFi Tether to work, but only without encryption. I was getting > about 1.5 mbit down and .5 mbit up. > > As soon as I turned on encryption I was unable to set up the laptop > password to match (that was the only thing that changed). No > encryption is unacceptable to me as I would only really use this in > public places, exactly where you would most want encryption. > > I tried barnacle. That one just didn't work at all. > > I am now trying PDANet which is a USB tether. Since the PC is tied to > the phone by a usb, encryption seems to be unnecessary? Unfortunately > this one is not working either. The PDANet desktop app says it is > connected and the droid pdanet app says it is connected but the PC > just can't see the internet. I can't ping anything etc. > > Any suggestions for this beast? > > > John W. Colby > www.ColbyConsulting.com > > On 3/19/2011 4:53 PM, David McAfee wrote: > > Advanced task killer > > Weather channel > > Lookout antivirus > > AK Notepad > > Bank of America > > Acar auto maintenance > > Barcode scanner > > Bubble level > > Bump > > Droid light > > Google goggles > > Google translate > > IMDB > > Key ring > > Print share > > Trapster > > > > Sent from my Droid phone. > > On Mar 19, 2011 12:52 PM, "jwcolby" > > wrote: _______________________________________________ dba-VB > > mailing list dba-VB at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > http://www.databaseadvisors.com > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 21:45:39 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 22:45:39 -0400 Subject: [dba-VB] Most useful droid apps In-Reply-To: <4D85679B.30595.6303F8F@stuart.lexacorp.com.pg> References: , <4D855FE0.9070004@colbyconsulting.com> <4D85679B.30595.6303F8F@stuart.lexacorp.com.pg> Message-ID: <4D856A53.7010507@colbyconsulting.com> Nope, Verizon John W. Colby www.ColbyConsulting.com On 3/19/2011 10:34 PM, Stuart McLachlan wrote: > Are you on AT&T? > > If so, you may be interested in this: > > http://www.theregister.co.uk/2011/03/18/new_att_tethering_policy/ > > > On 19 Mar 2011 at 22:01, jwcolby wrote: > >> I am trying various tethering solutions. My laptop is vista, my ip >> range is 192.168.122.x. >> >> I got WiFi Tether to work, but only without encryption. I was getting >> about 1.5 mbit down and .5 mbit up. >> >> As soon as I turned on encryption I was unable to set up the laptop >> password to match (that was the only thing that changed). No >> encryption is unacceptable to me as I would only really use this in >> public places, exactly where you would most want encryption. >> >> I tried barnacle. That one just didn't work at all. >> >> I am now trying PDANet which is a USB tether. Since the PC is tied to >> the phone by a usb, encryption seems to be unnecessary? Unfortunately >> this one is not working either. The PDANet desktop app says it is >> connected and the droid pdanet app says it is connected but the PC >> just can't see the internet. I can't ping anything etc. >> >> Any suggestions for this beast? >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/19/2011 4:53 PM, David McAfee wrote: >>> Advanced task killer >>> Weather channel >>> Lookout antivirus >>> AK Notepad >>> Bank of America >>> Acar auto maintenance >>> Barcode scanner >>> Bubble level >>> Bump >>> Droid light >>> Google goggles >>> Google translate >>> IMDB >>> Key ring >>> Print share >>> Trapster >>> >>> Sent from my Droid phone. >>> On Mar 19, 2011 12:52 PM, "jwcolby" >>> wrote: _______________________________________________ dba-VB >>> mailing list dba-VB at databaseadvisors.com >>> http://databaseadvisors.com/mailman/listinfo/dba-vb >>> http://www.databaseadvisors.com >>> >>> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From listmaster at databaseadvisors.com Wed Mar 23 14:34:46 2011 From: listmaster at databaseadvisors.com (Bryan Carbonnell) Date: Wed, 23 Mar 2011 15:34:46 -0400 Subject: [dba-VB] Administrivia - Spam Filtering on DBA Servers In-Reply-To: References: Message-ID: We have made a change to the way our mailservers deal with spam filtering. We were getting too many false positive reports from one of the spam black list services we use, so we have discontinued using them. Hopefully your bounce troubles should be over, or at the very least greatly minimized. If you continue to have issues surrounding bounces, PLEASE, PLEASE, PLEASE get in touch with me (listmaster at databaseadvisors OR carbonnb at gmail.com if the listmaster address bounces). -- Bryan Carbonnell - listmaster at databaseadvisors.com Life's journey is not to arrive at the grave safely in a well preserved body, but rather to skid in sideways, totally worn out, shouting "What a great ride!" From jwcolby at colbyconsulting.com Thu Mar 24 13:07:19 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 24 Mar 2011 14:07:19 -0400 Subject: [dba-VB] C# protected Message-ID: <4D8B8857.60604@colbyconsulting.com> I am trying to create a method of a ClassA which can be seen from the object ClassB that instantiates ClassA but not by objectClassC which instantiates ClassB. IOW ClassA "belongs to" ClassB so specific methods / properties of ClassB should be manipulateable by ClassB. However those properties / methods should not be visible to anything outside of ClassA and ClassB. Possible? Oh, and BTW I want ClassA to be out in a class library, external to the project using it. -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Thu Mar 24 15:14:59 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 24 Mar 2011 23:14:59 +0300 Subject: [dba-VB] C# protected In-Reply-To: <4D8B8857.60604@colbyconsulting.com> References: <4D8B8857.60604@colbyconsulting.com> Message-ID: Hi John -- Put ClassA and ClassB into a class library. Use internal access modifier with their methods and properties to be seen within their class library but not from the "outer world". Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 24 ????? 2011 ?. 21:07 To: VBA Subject: [dba-VB] C# protected I am trying to create a method of a ClassA which can be seen from the object ClassB that instantiates ClassA but not by objectClassC which instantiates ClassB. IOW ClassA "belongs to" ClassB so specific methods / properties of ClassB should be manipulateable by ClassB. However those properties / methods should not be visible to anything outside of ClassA and ClassB. Possible? Oh, and BTW I want ClassA to be out in a class library, external to the project using it. -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Thu Mar 24 15:28:03 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 24 Mar 2011 16:28:03 -0400 Subject: [dba-VB] C# protected In-Reply-To: References: <4D8B8857.60604@colbyconsulting.com> Message-ID: <4D8BA953.4030602@colbyconsulting.com> ClassA can go in the class library, but classB is part of the application. John W. Colby www.ColbyConsulting.com On 3/24/2011 4:14 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Put ClassA and ClassB into a class library. > Use > > internal > > access modifier with their methods and properties to be seen within their > class library but not from the "outer world". > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 24 ????? 2011 ?. 21:07 > To: VBA > Subject: [dba-VB] C# protected > > I am trying to create a method of a ClassA which can be seen from the object > ClassB that instantiates ClassA but not by objectClassC which instantiates > ClassB. > > IOW ClassA "belongs to" ClassB so specific methods / properties of ClassB > should be manipulateable by ClassB. However those properties / methods > should not be visible to anything outside of ClassA and ClassB. > > Possible? > > Oh, and BTW I want ClassA to be out in a class library, external to the > project using it. > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Thu Mar 24 16:37:22 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 25 Mar 2011 00:37:22 +0300 Subject: [dba-VB] C# protected In-Reply-To: <4D8BA953.4030602@colbyconsulting.com> References: <4D8B8857.60604@colbyconsulting.com> <4D8BA953.4030602@colbyconsulting.com> Message-ID: Hi John -- Then use protected access modifier for methods and properties of Class A which is inherited by class B (IOW no need to instantiate class A explicitly)... Class C which will instantiate Class B will no see/have access to protected properties and methods of class A inherithed by class B. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 24 ????? 2011 ?. 23:28 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# protected ClassA can go in the class library, but classB is part of the application. John W. Colby www.ColbyConsulting.com On 3/24/2011 4:14 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Put ClassA and ClassB into a class library. > Use > > internal > > access modifier with their methods and properties to be seen within > their class library but not from the "outer world". > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 24 ????? 2011 ?. 21:07 > To: VBA > Subject: [dba-VB] C# protected > > I am trying to create a method of a ClassA which can be seen from the > object ClassB that instantiates ClassA but not by objectClassC which > instantiates ClassB. > > IOW ClassA "belongs to" ClassB so specific methods / properties of > ClassB should be manipulateable by ClassB. However those properties / > methods should not be visible to anything outside of ClassA and ClassB. > > Possible? > > Oh, and BTW I want ClassA to be out in a class library, external to > the project using it. > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 25 14:58:27 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 25 Mar 2011 15:58:27 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <3B9DE744CEA244829179A81CA818F92F@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com> <3B9DE744CEA244829179A81CA818F92F@nant> Message-ID: <4D8CF3E3.3050608@colbyconsulting.com> Shamil, Does this allow using a single "lock object" to lock various operations in a class. For example I want to lock a date flag variable while using a property to either get or set the variable. private Object thisLock = new Object(); public DateTime pFlagDte { get { lock (thisLock) { return dteFlag; } } set { lock (thisLock) { dteFlag = value; } } } The issue is not multiple threads trying to write to the date variable but rather one thread trying to read it while another is writing it. John W. Colby www.ColbyConsulting.com On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Do you you need add, remove and iterate a list to get an object or you need > to add, remove and get an object by its key? - if the latter - here is the > code you can use: > > private Dictionary _myObjects = new Dictionary MyObject>(); > private static object _myObjectsLocker = new object(); > public void Add(MyObject myObject) > { > lock (_myObjectsLocker) > { > _myObjects.Add(myObject.ID, myObject); > _keys.Add(myObject.ID); > } > } > public MyObject Remove(int id) > { > MyObject myobject = null; > lock (_myObjectsLocker) > { > myobject = _myObjects[id]; > _myObjects.Remove(id); > } > return myobject; > } > public MyObject this[int key] > { > get > { > MyObject myObject = null; > lock (_myObjectsLocker) > { > _myObjects.TryGetValue(key, out myObject); > } > return myObject; > } > } > > for the case of iteration just replace: > > _myObjects.TryGetValue(key, out myObject); > > with your iteration code. > > As for the sample code - changing qty of threads and changing total computer > workload (outer context for the test code) you can see how different could > be final results when heavy multi-threading is used. And even if it's not > heavy - it still can result in weird output if special multi-threading > coding technique aren't used... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 11 ????? 2011 ?. 1:27 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Thanks for the code but I think it is overkill for my needs so far, though I > will put it in the archives for when I need it. > > In my system so far I have (or can arrange to have ) only two threads > trying to access a list. > > Basically what I am doing is reading records out of sql server into a class > factory, and placing the class instances into a list. > > Another object with its own thread is coming by and looking at the objects > in the list to see if there is any work for it to do. If it finds an object > to work on, it grabs a pointer to that item in the list and stores it in a > pointer and releases the list. > > One of the ways I saw this handled was to create factory which creates a > brand new list object, grabs a pointer to the list, locks that and then > copies the objects in the list pointers to classes) into that new list, > unlocks the list and hands back a pointer to this brand new list. thus the > new list could be iterated at will. > > I am just trying to lock the list while one thread adds objects to the list > or another thread gets objects out of the list. > > So if I have object A which contains a list and is adding / deleting objects > to that list... > and object B which needs to iterate through the list looking for an object > of interest in the list, and grabbing a pointer to that object, there is no > way to do this without all of the code you wrote? > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Does that in fact lock the list to other objects doing the same thing >> (getting the list through that property)? >>>>> >> You have to lock/synchronize/serialize not the list itself but >> simultaneous access to the list from different threads. >> You'd better use special static object variable to implement locking. >> >> I will post sample code in my following posting here... >> >> Thank you. >> >> -- >> Shamil > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Mar 25 16:05:57 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 26 Mar 2011 00:05:57 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8CF3E3.3050608@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com> Message-ID: <3F0DA84A905C4947819126F43CFC39DB@nant> Hi John -- Your sample is a correct usage of locking - just use static thisLock variable private static Object thisLock = new Object() as non-static variable *is not* thread safe AFAIU. Yes, such locking can be done for various operations of that class - just make sure you'll not get dead-locked... Also use *one lock object* to lock *one resource* in a class - otherwise "deadlock" will become your "everyday guest"... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 25 ????? 2011 ?. 22:58 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, Does this allow using a single "lock object" to lock various operations in a class. For example I want to lock a date flag variable while using a property to either get or set the variable. private Object thisLock = new Object(); public DateTime pFlagDte { get { lock (thisLock) { return dteFlag; } } set { lock (thisLock) { dteFlag = value; } } } The issue is not multiple threads trying to write to the date variable but rather one thread trying to read it while another is writing it. John W. Colby www.ColbyConsulting.com On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Do you you need add, remove and iterate a list to get an object or you > need to add, remove and get an object by its key? - if the latter - > here is the code you can use: > > private Dictionary _myObjects = new Dictionary MyObject>(); > private static object _myObjectsLocker = new object(); public void > Add(MyObject myObject) { > lock (_myObjectsLocker) > { > _myObjects.Add(myObject.ID, myObject); > _keys.Add(myObject.ID); > } > } > public MyObject Remove(int id) > { > MyObject myobject = null; > lock (_myObjectsLocker) > { > myobject = _myObjects[id]; > _myObjects.Remove(id); > } > return myobject; > } > public MyObject this[int key] > { > get > { > MyObject myObject = null; > lock (_myObjectsLocker) > { > _myObjects.TryGetValue(key, out myObject); > } > return myObject; > } > } > > for the case of iteration just replace: > > _myObjects.TryGetValue(key, out myObject); > > with your iteration code. > > As for the sample code - changing qty of threads and changing total > computer workload (outer context for the test code) you can see how > different could be final results when heavy multi-threading is used. > And even if it's not heavy - it still can result in weird output if > special multi-threading coding technique aren't used... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 11 ????? 2011 ?. 1:27 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Thanks for the code but I think it is overkill for my needs so far, > though I will put it in the archives for when I need it. > > In my system so far I have (or can arrange to have ) only two threads > trying to access a list. > > Basically what I am doing is reading records out of sql server into a > class factory, and placing the class instances into a list. > > Another object with its own thread is coming by and looking at the > objects in the list to see if there is any work for it to do. If it > finds an object to work on, it grabs a pointer to that item in the > list and stores it in a pointer and releases the list. > > One of the ways I saw this handled was to create factory which creates > a brand new list object, grabs a pointer to the list, locks that and > then copies the objects in the list pointers to classes) into that new > list, unlocks the list and hands back a pointer to this brand new > list. thus the new list could be iterated at will. > > I am just trying to lock the list while one thread adds objects to the > list or another thread gets objects out of the list. > > So if I have object A which contains a list and is adding / deleting > objects to that list... > and object B which needs to iterate through the list looking for an > object of interest in the list, and grabbing a pointer to that object, > there is no way to do this without all of the code you wrote? > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Does that in fact lock the list to other objects doing the same thing >> (getting the list through that property)? >>>>> >> You have to lock/synchronize/serialize not the list itself but >> simultaneous access to the list from different threads. >> You'd better use special static object variable to implement locking. >> >> I will post sample code in my following posting here... >> >> Thank you. >> >> -- >> Shamil > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 25 16:38:19 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 25 Mar 2011 17:38:19 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <3F0DA84A905C4947819126F43CFC39DB@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com> <3F0DA84A905C4947819126F43CFC39DB@nant> Message-ID: <4D8D0B4B.9070004@colbyconsulting.com> When you say resource, you mean variable? IOW use private static Object LockBln = new Object(); private static Object LockDte = new Object(); and use the LockBln to lock the boolean object while using LockDte to lock the date object? John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - just > make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - otherwise > "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various operations in a > class. > > For example I want to lock a date flag variable while using a property to > either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable but > rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Do you you need add, remove and iterate a list to get an object or you >> need to add, remove and get an object by its key? - if the latter - >> here is the code you can use: >> >> private Dictionary _myObjects = new Dictionary> MyObject>(); >> private static object _myObjectsLocker = new object(); public void >> Add(MyObject myObject) { >> lock (_myObjectsLocker) >> { >> _myObjects.Add(myObject.ID, myObject); >> _keys.Add(myObject.ID); >> } >> } >> public MyObject Remove(int id) >> { >> MyObject myobject = null; >> lock (_myObjectsLocker) >> { >> myobject = _myObjects[id]; >> _myObjects.Remove(id); >> } >> return myobject; >> } >> public MyObject this[int key] >> { >> get >> { >> MyObject myObject = null; >> lock (_myObjectsLocker) >> { >> _myObjects.TryGetValue(key, out myObject); >> } >> return myObject; >> } >> } >> >> for the case of iteration just replace: >> >> _myObjects.TryGetValue(key, out myObject); >> >> with your iteration code. >> >> As for the sample code - changing qty of threads and changing total >> computer workload (outer context for the test code) you can see how >> different could be final results when heavy multi-threading is used. >> And even if it's not heavy - it still can result in weird output if >> special multi-threading coding technique aren't used... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 11 ????? 2011 ?. 1:27 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Thanks for the code but I think it is overkill for my needs so far, >> though I will put it in the archives for when I need it. >> >> In my system so far I have (or can arrange to have ) only two threads >> trying to access a list. >> >> Basically what I am doing is reading records out of sql server into a >> class factory, and placing the class instances into a list. >> >> Another object with its own thread is coming by and looking at the >> objects in the list to see if there is any work for it to do. If it >> finds an object to work on, it grabs a pointer to that item in the >> list and stores it in a pointer and releases the list. >> >> One of the ways I saw this handled was to create factory which creates >> a brand new list object, grabs a pointer to the list, locks that and >> then copies the objects in the list pointers to classes) into that new >> list, unlocks the list and hands back a pointer to this brand new >> list. thus the new list could be iterated at will. >> >> I am just trying to lock the list while one thread adds objects to the >> list or another thread gets objects out of the list. >> >> So if I have object A which contains a list and is adding / deleting >> objects to that list... >> and object B which needs to iterate through the list looking for an >> object of interest in the list, and grabbing a pointer to that object, >> there is no way to do this without all of the code you wrote? >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >>> Hi John -- >>> >>> <<< >>> Does that in fact lock the list to other objects doing the same thing >>> (getting the list through that property)? >>>>>> >>> You have to lock/synchronize/serialize not the list itself but >>> simultaneous access to the list from different threads. >>> You'd better use special static object variable to implement locking. >>> >>> I will post sample code in my following posting here... >>> >>> Thank you. >>> >>> -- >>> Shamil >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Mar 25 17:46:44 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 26 Mar 2011 01:46:44 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8D0B4B.9070004@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8D0B4B.9070004@colbyconsulting.com> Message-ID: Hi John -- By resource I mean access to a read/write resource - it could be not only a variable... If you need two or more class level variables (fields, properties) within one class instance to be locked then keep them in a class/structure and lock that class/structure instance... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 26 ????? 2011 ?. 0:38 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() When you say resource, you mean variable? IOW use private static Object LockBln = new Object(); private static Object LockDte = new Object(); and use the LockBln to lock the boolean object while using LockDte to lock the date object? John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - > just make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - > otherwise "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various > operations in a class. > > For example I want to lock a date flag variable while using a property > to either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable > but rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Do you you need add, remove and iterate a list to get an object or >> you need to add, remove and get an object by its key? - if the latter >> - here is the code you can use: >> >> private Dictionary _myObjects = new Dictionary> MyObject>(); >> private static object _myObjectsLocker = new object(); public void >> Add(MyObject myObject) { >> lock (_myObjectsLocker) >> { >> _myObjects.Add(myObject.ID, myObject); >> _keys.Add(myObject.ID); >> } >> } >> public MyObject Remove(int id) >> { >> MyObject myobject = null; >> lock (_myObjectsLocker) >> { >> myobject = _myObjects[id]; >> _myObjects.Remove(id); >> } >> return myobject; >> } >> public MyObject this[int key] >> { >> get >> { >> MyObject myObject = null; >> lock (_myObjectsLocker) >> { >> _myObjects.TryGetValue(key, out myObject); >> } >> return myObject; >> } >> } >> >> for the case of iteration just replace: >> >> _myObjects.TryGetValue(key, out myObject); >> >> with your iteration code. >> >> As for the sample code - changing qty of threads and changing total >> computer workload (outer context for the test code) you can see how >> different could be final results when heavy multi-threading is used. >> And even if it's not heavy - it still can result in weird output if >> special multi-threading coding technique aren't used... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 11 ????? 2011 ?. 1:27 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Thanks for the code but I think it is overkill for my needs so far, >> though I will put it in the archives for when I need it. >> >> In my system so far I have (or can arrange to have ) only two >> threads trying to access a list. >> >> Basically what I am doing is reading records out of sql server into a >> class factory, and placing the class instances into a list. >> >> Another object with its own thread is coming by and looking at the >> objects in the list to see if there is any work for it to do. If it >> finds an object to work on, it grabs a pointer to that item in the >> list and stores it in a pointer and releases the list. >> >> One of the ways I saw this handled was to create factory which >> creates a brand new list object, grabs a pointer to the list, locks >> that and then copies the objects in the list pointers to classes) >> into that new list, unlocks the list and hands back a pointer to this >> brand new list. thus the new list could be iterated at will. >> >> I am just trying to lock the list while one thread adds objects to >> the list or another thread gets objects out of the list. >> >> So if I have object A which contains a list and is adding / deleting >> objects to that list... >> and object B which needs to iterate through the list looking for an >> object of interest in the list, and grabbing a pointer to that >> object, there is no way to do this without all of the code you wrote? >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >>> Hi John -- >>> >>> <<< >>> Does that in fact lock the list to other objects doing the same >>> thing (getting the list through that property)? >>>>>> >>> You have to lock/synchronize/serialize not the list itself but >>> simultaneous access to the list from different threads. >>> You'd better use special static object variable to implement locking. >>> >>> I will post sample code in my following posting here... >>> >>> Thank you. >>> >>> -- >>> Shamil >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 25 21:57:22 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 25 Mar 2011 22:57:22 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8D0B4B.9070004@colbyconsulting.com> Message-ID: <4D8D5612.6050701@colbyconsulting.com> Shamil, I have processes that log results to flags. For example, make a database (log that it was made), build a table (log that it was built), pull umpteen million records in sorted order (log that it was filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build another chunk table (log that it was filled), BCP out (log that it was exported). The objective is to be able to sustain interruptions and pick up where we left off. These processes can take minutes (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). So each thing I do represents a step in the process and each step is logged in a field in a record in SQL server using a datetime. There are so many of these flags that I am trying to standardize the process by building a class that can be instantiated, filled with data and log itself to SQL Server by one thread and be checked by another thread. These flag class instances will be checked by multiple threads, each thread trying to decide whether it should be doing the next step because another thread has finished it's part. IOW if a file has been written to disk, then the next thread will write it to a VM for processing. If it moved to the VM the next thread will watch the VM's output directory for a file to pop out and move it back to a directory on the server. If the file (a couple of files actually) successfully copied back to the server staging then another thread will import it back into a chunk table in an input database. If the file successfully imported then another thread will... In general one thread will "own" the flag and use it to log its status and one other thread will be checking the status of the flag to determine that it can go to work on that work chunk. You get the picture. I am trying to build an entirely asynchronous highly threaded process which exports a huge table into multiple files, processes every file through a third party app and gets the results back into SQL Server. All while logging each and every step so that no piece can possibly be dropped at any stage, even if the server goes down (or the VM goes down). Eventually this process will run on my server 24/7. It has been working for some time but I am getting threading issues, and I need to work on the high level control so that all of the processes can cleanly start up and shut down and every stage can pick back up when the program restarts should a shutdown occur. A single database can be up to a hundred million records (the biggest so far), and the external program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour depending on many different things so that example will take 50 chunks and could take 40 to 50 hours to complete. It takes about 20 processing steps to handle each file from end to end. It needs to just work, and I need to be able to view status in a meaningful way. And I need to process that and a dozen other files every single month, automatically, with no manual intervention required. John W. Colby www.ColbyConsulting.com On 3/25/2011 6:46 PM, Shamil Salakhetdinov wrote: > Hi John -- > > By resource I mean access to a read/write resource - it could be not only a > variable... > > If you need two or more class level variables (fields, properties) within > one class instance to be locked then keep them in a class/structure and lock > that class/structure instance... > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Sat Mar 26 00:37:25 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 26 Mar 2011 08:37:25 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8D5612.6050701@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8D0B4B.9070004@colbyconsulting.com> <4D8D5612.6050701@colbyconsulting.com> Message-ID: <4D87466503764D98871B609159033CB6@nant> John -- But you can try to use several instances of .NET Framework Queue class http://msdn.microsoft.com/en-us/library/system.collections.queue.aspx ? As far as I understand you have a set of sequential tasks to be performed on every (input/start) chunk so you can make a dedicated Queue instance for every task, and that Queue instance will be handled by dedicated "processor" classes: - TaskA Queue items will be processed by TaskAQueueProcessor, - TaksB .... by TaskBQueueProcessor, -- - TaskZ... by TaskZQueueProcessor There could be many such TaskA(B, ....Z)QueueProcessor instances "floating around" looking for an item from their dedicated tasks' queue to process: when they find it, they pick it up, process and put into another Queue based on "workflow plan"... You can also try to consider using Windows Workflow Foundation(WF) if that wouldn't be an "overkill" for your case... http://msdn.microsoft.com/en-us/netframework/wf-screencasts.aspx#introwf4 Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 26 ????? 2011 ?. 5:57 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, I have processes that log results to flags. For example, make a database (log that it was made), build a table (log that it was built), pull umpteen million records in sorted order (log that it was filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build another chunk table (log that it was filled), BCP out (log that it was exported). The objective is to be able to sustain interruptions and pick up where we left off. These processes can take minutes (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). So each thing I do represents a step in the process and each step is logged in a field in a record in SQL server using a datetime. There are so many of these flags that I am trying to standardize the process by building a class that can be instantiated, filled with data and log itself to SQL Server by one thread and be checked by another thread. These flag class instances will be checked by multiple threads, each thread trying to decide whether it should be doing the next step because another thread has finished it's part. IOW if a file has been written to disk, then the next thread will write it to a VM for processing. If it moved to the VM the next thread will watch the VM's output directory for a file to pop out and move it back to a directory on the server. If the file (a couple of files actually) successfully copied back to the server staging then another thread will import it back into a chunk table in an input database. If the file successfully imported then another thread will... In general one thread will "own" the flag and use it to log its status and one other thread will be checking the status of the flag to determine that it can go to work on that work chunk. You get the picture. I am trying to build an entirely asynchronous highly threaded process which exports a huge table into multiple files, processes every file through a third party app and gets the results back into SQL Server. All while logging each and every step so that no piece can possibly be dropped at any stage, even if the server goes down (or the VM goes down). Eventually this process will run on my server 24/7. It has been working for some time but I am getting threading issues, and I need to work on the high level control so that all of the processes can cleanly start up and shut down and every stage can pick back up when the program restarts should a shutdown occur. A single database can be up to a hundred million records (the biggest so far), and the external program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour depending on many different things so that example will take 50 chunks and could take 40 to 50 hours to complete. It takes about 20 processing steps to handle each file from end to end. It needs to just work, and I need to be able to view status in a meaningful way. And I need to process that and a dozen other files every single month, automatically, with no manual intervention required. John W. Colby www.ColbyConsulting.com On 3/25/2011 6:46 PM, Shamil Salakhetdinov wrote: > Hi John -- > > By resource I mean access to a read/write resource - it could be not > only a variable... > > If you need two or more class level variables (fields, properties) > within one class instance to be locked then keep them in a > class/structure and lock that class/structure instance... > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From gustav at cactus.dk Sat Mar 26 07:41:44 2011 From: gustav at cactus.dk (Gustav Brock) Date: Sat, 26 Mar 2011 13:41:44 +0100 Subject: [dba-VB] c# lock() Message-ID: Hi John My first thought was Windows Workflow Foundation(WF) but, as Shamil mentions, it may be overkill. On the other hand it is exactly for controlling scenarios where "if task 1 is done, start task 2 and task 3, wait for task 2 and task 3 to finish, check something, then start task 4, etc.". However, that reminded me about a series of articles about the Task Parallel Library of .Net: http://www.codeproject.com/KB/cs/TPL1.aspx which I found very interesting, though I haven't had any use for it yet. Among other topics it discusses carefully canceling and error handling which I guess is quite important for your purpose. /gustav >>> jwcolby at colbyconsulting.com 26-03-2011 03:57 >>> Shamil, I have processes that log results to flags. For example, make a database (log that it was made), build a table (log that it was built), pull umpteen million records in sorted order (log that it was filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build another chunk table (log that it was filled), BCP out (log that it was exported). The objective is to be able to sustain interruptions and pick up where we left off. These processes can take minutes (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). So each thing I do represents a step in the process and each step is logged in a field in a record in SQL server using a datetime. There are so many of these flags that I am trying to standardize the process by building a class that can be instantiated, filled with data and log itself to SQL Server by one thread and be checked by another thread. These flag class instances will be checked by multiple threads, each thread trying to decide whether it should be doing the next step because another thread has finished it's part. IOW if a file has been written to disk, then the next thread will write it to a VM for processing. If it moved to the VM the next thread will watch the VM's output directory for a file to pop out and move it back to a directory on the server. If the file (a couple of files actually) successfully copied back to the server staging then another thread will import it back into a chunk table in an input database. If the file successfully imported then another thread will... In general one thread will "own" the flag and use it to log its status and one other thread will be checking the status of the flag to determine that it can go to work on that work chunk. You get the picture. I am trying to build an entirely asynchronous highly threaded process which exports a huge table into multiple files, processes every file through a third party app and gets the results back into SQL Server. All while logging each and every step so that no piece can possibly be dropped at any stage, even if the server goes down (or the VM goes down). Eventually this process will run on my server 24/7. It has been working for some time but I am getting threading issues, and I need to work on the high level control so that all of the processes can cleanly start up and shut down and every stage can pick back up when the program restarts should a shutdown occur. A single database can be up to a hundred million records (the biggest so far), and the external program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour depending on many different things so that example will take 50 chunks and could take 40 to 50 hours to complete. It takes about 20 processing steps to handle each file from end to end. It needs to just work, and I need to be able to view status in a meaningful way. And I need to process that and a dozen other files every single month, automatically, with no manual intervention required. John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 26 07:44:54 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 08:44:54 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: Message-ID: <4D8DDFC6.4040004@colbyconsulting.com> Well... I haven't migrated to VS2010 yet. I tried it about 6 months ago and it was universally a pig on my systems. I thought I would wait until MS polished it a bit. John W. Colby www.ColbyConsulting.com On 3/26/2011 8:41 AM, Gustav Brock wrote: > Hi John > > My first thought was Windows Workflow Foundation(WF) but, as Shamil mentions, it may be overkill. On the other hand it is exactly for controlling scenarios where "if task 1 is done, start task 2 and task 3, wait for task 2 and task 3 to finish, check something, then start task 4, etc.". > > However, that reminded me about a series of articles about the Task Parallel Library of .Net: > > http://www.codeproject.com/KB/cs/TPL1.aspx > > which I found very interesting, though I haven't had any use for it yet. Among other topics it discusses carefully canceling and error handling which I guess is quite important for your purpose. > > /gustav > > >>>> jwcolby at colbyconsulting.com 26-03-2011 03:57>>> > Shamil, > > I have processes that log results to flags. For example, make a database (log that it was made), > build a table (log that it was built), pull umpteen million records in sorted order (log that it was > filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build > another chunk table (log that it was filled), BCP out (log that it was exported). The objective is > to be able to sustain interruptions and pick up where we left off. These processes can take minutes > (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). > > So each thing I do represents a step in the process and each step is logged in a field in a record > in SQL server using a datetime. There are so many of these flags that I am trying to standardize > the process by building a class that can be instantiated, filled with data and log itself to SQL > Server by one thread and be checked by another thread. > > These flag class instances will be checked by multiple threads, each thread trying to decide whether > it should be doing the next step because another thread has finished it's part. IOW if a file has > been written to disk, then the next thread will write it to a VM for processing. If it moved to the > VM the next thread will watch the VM's output directory for a file to pop out and move it back to a > directory on the server. If the file (a couple of files actually) successfully copied back to the > server staging then another thread will import it back into a chunk table in an input database. If > the file successfully imported then another thread will... In general one thread will "own" the > flag and use it to log its status and one other thread will be checking the status of the flag to > determine that it can go to work on that work chunk. > > You get the picture. > > I am trying to build an entirely asynchronous highly threaded process which exports a huge table > into multiple files, processes every file through a third party app and gets the results back into > SQL Server. All while logging each and every step so that no piece can possibly be dropped at any > stage, even if the server goes down (or the VM goes down). Eventually this process will run on my > server 24/7. > > It has been working for some time but I am getting threading issues, and I need to work on the high > level control so that all of the processes can cleanly start up and shut down and every stage can > pick back up when the program restarts should a shutdown occur. > > A single database can be up to a hundred million records (the biggest so far), and the external > program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour > depending on many different things so that example will take 50 chunks and could take 40 to 50 hours > to complete. It takes about 20 processing steps to handle each file from end to end. It needs to > just work, and I need to be able to view status in a meaningful way. And I need to process that and > a dozen other files every single month, automatically, with no manual intervention required. > > > > John W. Colby > www.ColbyConsulting.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From gustav at cactus.dk Sat Mar 26 08:08:10 2011 From: gustav at cactus.dk (Gustav Brock) Date: Sat, 26 Mar 2011 14:08:10 +0100 Subject: [dba-VB] c# lock() Message-ID: Hi John Oh, I recall your trouble now that you mention it. Strange, as it runs fine on my antique Pentium 4 IBM IntelliStation with 2 GB ram. However, VS2010 _has_ been polished: The Service Pack 1 was released a few weeks ago. /gustav >>> jwcolby at colbyconsulting.com 26-03-2011 13:44 >>> Well... I haven't migrated to VS2010 yet. I tried it about 6 months ago and it was universally a pig on my systems. I thought I would wait until MS polished it a bit. John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 26 08:26:26 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 09:26:26 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: Message-ID: <4D8DE982.7060707@colbyconsulting.com> When I had my speed issues I Googled around and found tons with speed issues, and tons without. There are entire threads bitterly complaining about what a dog it is. There was no explanations of when or why. Mine would take forever to do anything. I will definitely try again now that SP1 is out. John W. Colby www.ColbyConsulting.com On 3/26/2011 9:08 AM, Gustav Brock wrote: > Hi John > > Oh, I recall your trouble now that you mention it. Strange, as it runs fine on my antique Pentium 4 IBM IntelliStation with 2 GB ram. > However, VS2010 _has_ been polished: The Service Pack 1 was released a few weeks ago. > > /gustav > > >>>> jwcolby at colbyconsulting.com 26-03-2011 13:44>>> > Well... I haven't migrated to VS2010 yet. > > I tried it about 6 months ago and it was universally a pig on my systems. I thought I would wait > until MS polished it a bit. > > John W. Colby > www.ColbyConsulting.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 26 16:53:15 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 17:53:15 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <3F0DA84A905C4947819126F43CFC39DB@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com> <3F0DA84A905C4947819126F43CFC39DB@nant> Message-ID: <4D8E604B.5040001@colbyconsulting.com> Shamil, I am trying to build a base class which contains a set of lock objects and some standard variables. Having defined the lock objects static per your code below, the derived class cannot access the lock objects even though I set them protected: protected static Object LockBln = new Object(); In the derived class when I try to access it: lock(this.LockBln) { } I get a compile error: "Member cannot be accesses with an instance reference; Qualify it with a type name instead." What does this mean? Reading on the internet it says that static properties are shared between all instances of the class. I don't see how I can share a property between 50 instances and then use a lock(MyStaticProperty) {}. I'm so confused... John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - just > make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - otherwise > "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various operations in a > class. > > For example I want to lock a date flag variable while using a property to > either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable but > rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sat Mar 26 17:27:17 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sun, 27 Mar 2011 01:27:17 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8E604B.5040001@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8E604B.5040001@colbyconsulting.com> Message-ID: <00D0A03B172743B6868A8BE736F7E560@nant> John -- Let's define your target (application) system architecture on conceptual level - I guess that using "micro-level locking" you're trying (?) to achieve isn't needed... 1) you have a huge input data set; 2) you split input data set into X chunks; 3) every chunk gets processed using the same several - Y - steps; 4) on completion of step Yn input/semi-processed chunk Xn is submitted to the Y(n+1) processor class; 5) when all X chunks are processed in Y steps they get collected into a final "huge" output set to produce Z reports. That's it from conceptual high level point of view? Something important missing? Too simplified to be true? John, please take into account that I can't reply promptly here - it's weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to sleep, and then I have quite some work to do starting Monday - so I expect this thread will be supported by other AccessD/dba-VB members... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 0:53 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, I am trying to build a base class which contains a set of lock objects and some standard variables. Having defined the lock objects static per your code below, the derived class cannot access the lock objects even though I set them protected: protected static Object LockBln = new Object(); In the derived class when I try to access it: lock(this.LockBln) { } I get a compile error: "Member cannot be accesses with an instance reference; Qualify it with a type name instead." What does this mean? Reading on the internet it says that static properties are shared between all instances of the class. I don't see how I can share a property between 50 instances and then use a lock(MyStaticProperty) {}. I'm so confused... John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - > just make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - > otherwise "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various > operations in a class. > > For example I want to lock a date flag variable while using a property > to either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable > but rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Mar 26 18:31:31 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 19:31:31 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <00D0A03B172743B6868A8BE736F7E560@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant> Message-ID: <4D8E7753.8090006@colbyconsulting.com> I'm just trying to get a handle on locking things (in general). I am seeing things that say "just lock the object itself", and then other things that say "use a static object (the way you were showing me). If locking the object itself works, why do the static thing? John W. Colby www.ColbyConsulting.com On 3/26/2011 6:27 PM, Shamil Salakhetdinov wrote: > John -- > > Let's define your target (application) system architecture on conceptual > level - I guess that using "micro-level locking" you're trying (?) to > achieve isn't needed... > > 1) you have a huge input data set; > 2) you split input data set into X chunks; > 3) every chunk gets processed using the same several - Y - steps; > 4) on completion of step Yn input/semi-processed chunk Xn is submitted to > the Y(n+1) processor class; > 5) when all X chunks are processed in Y steps they get collected into a > final "huge" output set to produce Z reports. > > That's it from conceptual high level point of view? > Something important missing? > Too simplified to be true? > > John, please take into account that I can't reply promptly here - it's > weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to sleep, > and then I have quite some work to do starting Monday - so I expect this > thread will be supported by other AccessD/dba-VB members... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 27 ????? 2011 ?. 0:53 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > I am trying to build a base class which contains a set of lock objects and > some standard variables. > Having defined the lock objects static per your code below, the derived > class cannot access the lock objects even though I set them protected: > > protected static Object LockBln = new Object(); > > In the derived class when I try to access it: > > lock(this.LockBln) > { > } > > I get a compile error: > > "Member cannot be accesses with an instance reference; Qualify it with a > type name instead." > > What does this mean? > > Reading on the internet it says that static properties are shared between > all instances of the class. I don't see how I can share a property between > 50 instances and then use a > lock(MyStaticProperty) {}. > > I'm so confused... > > John W. Colby > www.ColbyConsulting.com > > On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Your sample is a correct usage of locking - just use static thisLock >> variable >> >> private static Object thisLock = new Object() >> >> as non-static variable *is not* thread safe AFAIU. >> >> Yes, such locking can be done for various operations of that class - >> just make sure you'll not get dead-locked... >> Also use *one lock object* to lock *one resource* in a class - >> otherwise "deadlock" will become your "everyday guest"... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 25 ????? 2011 ?. 22:58 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Does this allow using a single "lock object" to lock various >> operations in a class. >> >> For example I want to lock a date flag variable while using a property >> to either get or set the variable. >> >> private Object thisLock = new Object(); >> >> public DateTime pFlagDte { >> get { >> lock (thisLock) >> { >> return dteFlag; >> } >> } >> set >> { >> lock (thisLock) >> { >> dteFlag = value; >> } >> } >> } >> >> The issue is not multiple threads trying to write to the date variable >> but rather one thread trying to read it while another is writing it. >> >> John W. Colby >> www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Mar 26 19:08:10 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sun, 27 Mar 2011 04:08:10 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <00D0A03B172743B6868A8BE736F7E560@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant> Message-ID: <0CD7EF3475CB43C69334942027697B63@nant> John -- I'm falling asleep here now - would that be correct assumption that you can generalize your global task like that - you have, say: - split input data into 50 Chunks; - every chunk should be processed in 10 steps; - every step's average processing time is 100 minutes. Then in total it is needed 50*10*100 = 50,000 minutes or ~833.(3) hours to process all your input data sequentially, chunk after chunk step after step, say, you don't have enough computer power to process some chunks in parallel, and you can't process the whole not split input data as it's too large for your system... But you have "50 small computers" and you can process every of 50 input chunks in parallel - then you'll need just 10*100 = 1,000 minutes (~16.67 hours) to complete the job. I assume that all the chunks processing is independent - then you can: 1) Create Scheduler class instance; 2) Schedule class creates 10 TaskQueue class instances - 1 queue for every processing step type; 3) Scheduler class defines splitting criteria to split input data into 50 chunks; 4) Schedule defines completion criteria - when all 50 chunks get collected in 11th FinalQueue; 5) Schedule seeds the first InputChunkSplitTaskQueue with 50 SplitTasksDescriptors; 4) From time to time - say every half a minute Scheduler class instance creates 500 worker classes in 500 threads (500 worker classes => 50 chunks * 10 steps.), which "hunt for the job" in their dedicate Queues: if they find Job they work on it, if not - they "die"... 5) When a worker class completes processing its job it queues its "task results" into the next step Queue, if a worker class fails then it puts its incomplete task descriptor back into its dedicated queue and "dies", or that could be Scheduler's (or SchedulerAssitant's) task job to find failed workers' "trap" and to resubmit their failed work to the proper queue according to the custom workflow descriptors attached to every chunk... 500 worker classes is an overkill as only 50 worker classes can have job every time but that seems to be an easy "brute force" and "lazy parallel programming" approach - and it should work... Or you can make worker class instances production processes smarter: Scheduler class can start special thread for WorkerClassGenerator instance, which will monitor all the 10 TaskQueues, and if it finds an item in a Queque, it will pick it up, it will create corresponding Worker class in parallel thread and it will pass WorkItem to the Worker class fro processing... When that described above approach will work there then you can easily(?) scale it splitting your (constant size) input data into 100 chunks, and then if every chunk can be processes in half time - in average (50 minutes) - then all job can be completed in 500 minutes = ~8.33(3) hours etc... Please correct me if I oversimplified your application business area... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 27 ????? 2011 ?. 1:27 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] c# lock() John -- Let's define your target (application) system architecture on conceptual level - I guess that using "micro-level locking" you're trying (?) to achieve isn't needed... 1) you have a huge input data set; 2) you split input data set into X chunks; 3) every chunk gets processed using the same several - Y - steps; 4) on completion of step Yn input/semi-processed chunk Xn is submitted to the Y(n+1) processor class; 5) when all X chunks are processed in Y steps they get collected into a final "huge" output set to produce Z reports. That's it from conceptual high level point of view? Something important missing? Too simplified to be true? John, please take into account that I can't reply promptly here - it's weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to sleep, and then I have quite some work to do starting Monday - so I expect this thread will be supported by other AccessD/dba-VB members... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 0:53 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, I am trying to build a base class which contains a set of lock objects and some standard variables. Having defined the lock objects static per your code below, the derived class cannot access the lock objects even though I set them protected: protected static Object LockBln = new Object(); In the derived class when I try to access it: lock(this.LockBln) { } I get a compile error: "Member cannot be accesses with an instance reference; Qualify it with a type name instead." What does this mean? Reading on the internet it says that static properties are shared between all instances of the class. I don't see how I can share a property between 50 instances and then use a lock(MyStaticProperty) {}. I'm so confused... John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - > just make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - > otherwise "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various > operations in a class. > > For example I want to lock a date flag variable while using a property > to either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable > but rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Sat Mar 26 19:38:28 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sun, 27 Mar 2011 04:38:28 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <4D8E7753.8090006@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8E604B.5040001@colbyconsulting.com><00D0A03B172743B6868A8BE736F7E560@nant> <4D8E7753.8090006@colbyconsulting.com> Message-ID: Hi John -- <<< If locking the object itself works, why do the static thing? >>> Static is used to lock shared resources - in your case there are no such resources at all? Log writer is usually a static shared resource... And yes, you can use your locking code to lock parallel access to your class instances' local variables: >> private Object thisLock = new Object(); >> >> public DateTime pFlagDte { >> get { >> lock (thisLock) >> { >> return dteFlag; >> } >> } >> set >> { >> lock (thisLock) >> { >> dteFlag = value; >> } >> } >> } But as I noted I'm not sure if that "micro-level locking" is needed at all (and you mentioned you have several such shared variables within a class instance) - using Queues seems to be more simple and (thread-)safe approach without almost any custom locking schemes.... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 3:32 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() I'm just trying to get a handle on locking things (in general). I am seeing things that say "just lock the object itself", and then other things that say "use a static object (the way you were showing me). If locking the object itself works, why do the static thing? John W. Colby www.ColbyConsulting.com On 3/26/2011 6:27 PM, Shamil Salakhetdinov wrote: > John -- > > Let's define your target (application) system architecture on > conceptual level - I guess that using "micro-level locking" you're > trying (?) to achieve isn't needed... > > 1) you have a huge input data set; > 2) you split input data set into X chunks; > 3) every chunk gets processed using the same several - Y - steps; > 4) on completion of step Yn input/semi-processed chunk Xn is submitted > to the Y(n+1) processor class; > 5) when all X chunks are processed in Y steps they get collected into > a final "huge" output set to produce Z reports. > > That's it from conceptual high level point of view? > Something important missing? > Too simplified to be true? > > John, please take into account that I can't reply promptly here - it's > weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to > sleep, and then I have quite some work to do starting Monday - so I > expect this thread will be supported by other AccessD/dba-VB members... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 27 ????? 2011 ?. 0:53 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > I am trying to build a base class which contains a set of lock objects > and some standard variables. > Having defined the lock objects static per your code below, the > derived class cannot access the lock objects even though I set them protected: > > protected static Object LockBln = new Object(); > > In the derived class when I try to access it: > > lock(this.LockBln) > { > } > > I get a compile error: > > "Member cannot be accesses with an instance reference; Qualify it with > a type name instead." > > What does this mean? > > Reading on the internet it says that static properties are shared > between all instances of the class. I don't see how I can share a > property between 50 instances and then use a > lock(MyStaticProperty) {}. > > I'm so confused... > > John W. Colby > www.ColbyConsulting.com > > On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Your sample is a correct usage of locking - just use static thisLock >> variable >> >> private static Object thisLock = new Object() >> >> as non-static variable *is not* thread safe AFAIU. >> >> Yes, such locking can be done for various operations of that class - >> just make sure you'll not get dead-locked... >> Also use *one lock object* to lock *one resource* in a class - >> otherwise "deadlock" will become your "everyday guest"... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 25 ????? 2011 ?. 22:58 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Does this allow using a single "lock object" to lock various >> operations in a class. >> >> For example I want to lock a date flag variable while using a >> property to either get or set the variable. >> >> private Object thisLock = new Object(); >> >> public DateTime pFlagDte { >> get { >> lock (thisLock) >> { >> return dteFlag; >> } >> } >> set >> { >> lock (thisLock) >> { >> dteFlag = value; >> } >> } >> } >> >> The issue is not multiple threads trying to write to the date >> variable but rather one thread trying to read it while another is writing it. >> >> John W. Colby >> www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 26 23:12:20 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 27 Mar 2011 00:12:20 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <0CD7EF3475CB43C69334942027697B63@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant> <0CD7EF3475CB43C69334942027697B63@nant> Message-ID: <4D8EB924.5080708@colbyconsulting.com> Well... In programming you can do any task 50 different ways. The way I did it is to create a supervisor and ProcessFile table in SQL Server. The supervisor represents a table in SQL Server that contains an address table which needs to be processed. The ProcessFile records are child to tblSupervisor and represent chunks (files). The supervisor record stores all of the high level flags about the database (address table) being processed. The ProcessFile contains all of the flags about each chunk being processed. I have four different high level "stages" as I call them that a supervisor goes through. AccuzipOut - table to csv files in staging out (stage 1), Staging to VM (stage 2), VM to Staging in (stage 3, Staging in to SQL Server (stage 4) 1) Export all chunks (files) to a SQL Out staging directory. Each chunk ends up being a CSV file with PKID, FName, LName, Addr, City, St, Zip5, Zip4. The ProcessFile needs to be sorted by Zip5 / Zip4 order because the third party application that processes these files is immensely faster if the file is presorted. 2) One by one, copy each chunk file to a virtual machine - AccuzipIn directory. The Accuzip virtual machine runs on a Windows 2008 server running Hyper-V manager. I currently run 3 different VMs, only one of which runs Accuzip. For awhile I actually ran 3 VMs running Accuzip, and I am designing such that I can add additional Accuzip VMs in the future. What this means is that each file could go to any valid Accuzip VM, and when the chunk file is placed on a VM it has to log which VM it went to because... 3) The Accuzip software automatically senses and processes the file in the InputTo directory. A file of 2 million records takes about 25 minutes to CASS process locally to the VM, then is uploaded to a server in California for further NCOA processing. The server in California belongs to Accuzip and they stage big files so it can take awhile before it even begins to process. Once it processes, the NCOA server sends a small file back to my VM where Address change information is integrated back in to the local Accuzip database file (foxpro). After that it is exported back to CSV with a bunch of additional fields and is placed in the Accuzip OutputFrom directory along with a PDF which contains the CASS / NCOA statistics. This PDF is a "proof of processing" document that I have to keep forever, and I have to send to the client. I have a Directory Watcher which is watching the OutputFrom directory. When *all* of the resulting Accuzip output files are in the OutputFrom directory, my Directory Watcher event triggers my program to copy all of these files back to the SQL Server In staging directory. 4) Once all of the Accuzip files for a single chunk are back on the SQL Server In directory, my application BCPs the CSV into a input chunk table. From there decisions are made whether I need to import every address back in or only the moves. In any event some or all of the address records (with the additional fields from Accuzip) are pulled into an AZData table back in the database that the address data originally came from. So... 1) A supervisor object creates a clsDbAZOut instance. 2) clsDBAZOut creates an actual database dbXyzAZOut temporary database. 3) clsDBAZOut pulls every record needing to be accuzipped into a table with a super PK, sorted on Zip5/4 4) clsDBAZOut counts the records in tblAZOut, and starts to build clsProcessFile instances which holds the information about the chunk processing. clsProcessFile can write its info back to the ProcessFile records on SQL Server. 5) clsDbAZOut has a thread which builds tblChunk, gets ChunkSize records into the chunk table and BCPs tblChunk to CSV files on a staging out directory. 6) clsDbAZOut BCPs the chunk to file 7) At this point the clsProcessFile is handed off to clsVM. clsSupervisor has a strongly typed dictionary which stores the clsProcessFile instances created by clsDbAZOut. clsSupervisor can read ProcessFile instances from SQL Server into a clsProcessFile factory or clsDbAZOut can use a class factory to create new ones. In all cases they are stored in the dictionary in clsSupervisor. clsProcessfile has flags which document that: a) The chunk table was created b) The chunk table exported to file c) The chunk went to a VM (and which one) d) The chunk file was (eventually) found in the Accuzip OutputFrom directory e) The resulting AZData file (and PDF) were moved from the VM back to SQL ServerIn staging f) The file in SQL Server In staging was pulled back in to a chunk table in a second database dbXyzAZIn temporary database g) That the chunk table was imported back into the original database. As you can see, clsProcessFile is the center of the universe for a single chunk. clsSupervisor is the center of the universe for the entire process. It is a bit difficult to discuss how the entire program works in a single email. A clsManager loads one or a dozen clsSupervisor instances. clsManager loads clsVM which supervises the (a) virtual machine which runs a third party process. clsVM moves files from SQL Staging Out into the VM one chunk at a time, updating clsProcessTable flags. clsVM moves files from the the VM back to the SQL Staging IN, updating flags. The VM may not be available, or the Accuzip program may be down etc. clsSupervisor loads one to N clsProcessTable instances. clsSupervisor loads a clsDBExp and clsDBImp. clsDBExp creates new clsProcessTable instances, one for each chunk it exports - storing them in clsSupervisor. clsDBExp supervises the export process. clsDBImp supervises the import process getting the chunks back into SQL Server into a temp database, and from there into the live database. I am trying to design the system such that it is asynchronous. Stage1, stage 2/3 and stage 4 can all do their thing independently of each other. Each step of each stage is logged (dateTime flags written) so that any piece can just pick up where it left off. That means any Supervisor and any ProcessFile (chunk) can pick up from any point. In the meantime, the clsManager is watching the tblSupervisor for supervisor records ready to process - a thread. clsSupervisor is watching for ProcessTable records and dispatching them as required to the clsDBExp, clsDBImp or clsVM - a thread in each supervisor. clsDBExp is processing chunks, creating CSV files - a thread. clsVM is watching for clsProcessTable instances ready to go to the VM. It is also watching the output directory and sending files back to the SQL Server - a thread and two file watchers (and two events). clsDBImp is watching for clsProcessTable instances ready to import back into SQL Server - a thread. Most of these threads are writing flags (as they finish their specific process) and reading flags in clsProcessTable to decide what chunk has to be processed in which class. Oh, and I have three list controls on a form which the various classes write to to display the status for Stage1, Stage 2/3 and Stage 4. So pretty much a single supervisor is processing at a time, exporting files, sending them to the vm, importing them back in etc. It is possible for an order supervisor to cut in at the head of the line however. Not a simple task co-ordinating all of this stuff. I used to co-ordinate it all manually! ;) John W. Colby www.ColbyConsulting.com On 3/26/2011 8:08 PM, Shamil Salakhetdinov wrote: > John -- > > I'm falling asleep here now - would that be correct assumption that you can > generalize your global task like that - you have, say: > > - split input data into 50 Chunks; > - every chunk should be processed in 10 steps; > - every step's average processing time is 100 minutes. > > Then in total it is needed 50*10*100 = 50,000 minutes or ~833.(3) hours to > process all your input data sequentially, chunk after chunk step after step, > say, you don't have enough computer power to process some chunks in > parallel, and you can't process the whole not split input data as it's too > large for your system... > > But you have "50 small computers" and you can process every of 50 input > chunks in parallel - then you'll need just 10*100 = 1,000 minutes (~16.67 > hours) to complete the job. > > I assume that all the chunks processing is independent - then you can: > > 1) Create Scheduler class instance; > 2) Schedule class creates 10 TaskQueue class instances - 1 queue for every > processing step type; > 3) Scheduler class defines splitting criteria to split input data into 50 > chunks; > 4) Schedule defines completion criteria - when all 50 chunks get collected > in 11th FinalQueue; > 5) Schedule seeds the first InputChunkSplitTaskQueue with 50 > SplitTasksDescriptors; > 4) From time to time - say every half a minute Scheduler class instance > creates 500 worker classes in 500 threads (500 worker classes => 50 chunks * > 10 steps.), which "hunt for the job" in their dedicate Queues: if they find > Job they work on it, if not - they "die"... > 5) When a worker class completes processing its job it queues its "task > results" into the next step Queue, if a worker class fails then it puts its > incomplete task descriptor back into its dedicated queue and "dies", or that > could be Scheduler's (or SchedulerAssitant's) task job to find failed > workers' "trap" and to resubmit their failed work to the proper queue > according to the custom workflow descriptors attached to every chunk... > > 500 worker classes is an overkill as only 50 worker classes can have job > every time but that seems to be an easy "brute force" and "lazy parallel > programming" approach - and it should work... > Or you can make worker class instances production processes smarter: > Scheduler class can start special thread for WorkerClassGenerator instance, > which will monitor all the 10 TaskQueues, and if it finds an item in a > Queque, it will pick it up, it will create corresponding Worker class in > parallel thread and it will pass WorkItem to the Worker class fro > processing... > > When that described above approach will work there then you can easily(?) > scale it splitting your (constant size) input data into 100 chunks, and then > if every chunk can be processes in half time - in average (50 minutes) - > then all job can be completed in 500 minutes = ~8.33(3) hours etc... > > Please correct me if I oversimplified your application business area... > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Sun Mar 27 19:08:25 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 28 Mar 2011 04:08:25 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <4D8EB924.5080708@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> Message-ID: Hi John -- Yes, I see: you tend to build a multi-threaded application/utility system with centralized control - and I tend to build an anarchical free-market model based application/utility system :) Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 8:12 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Well... In programming you can do any task 50 different ways. The way I did it is to create a supervisor and ProcessFile table in SQL Server. The supervisor represents a table in SQL Server that contains an address table which needs to be processed. The ProcessFile records are child to tblSupervisor and represent chunks (files). The supervisor record stores all of the high level flags about the database (address table) being processed. The ProcessFile contains all of the flags about each chunk being processed. I have four different high level "stages" as I call them that a supervisor goes through. AccuzipOut - table to csv files in staging out (stage 1), Staging to VM (stage 2), VM to Staging in (stage 3, Staging in to SQL Server (stage 4) 1) Export all chunks (files) to a SQL Out staging directory. Each chunk ends up being a CSV file with PKID, FName, LName, Addr, City, St, Zip5, Zip4. The ProcessFile needs to be sorted by Zip5 / Zip4 order because the third party application that processes these files is immensely faster if the file is presorted. 2) One by one, copy each chunk file to a virtual machine - AccuzipIn directory. The Accuzip virtual machine runs on a Windows 2008 server running Hyper-V manager. I currently run 3 different VMs, only one of which runs Accuzip. For awhile I actually ran 3 VMs running Accuzip, and I am designing such that I can add additional Accuzip VMs in the future. What this means is that each file could go to any valid Accuzip VM, and when the chunk file is placed on a VM it has to log which VM it went to because... 3) The Accuzip software automatically senses and processes the file in the InputTo directory. A file of 2 million records takes about 25 minutes to CASS process locally to the VM, then is uploaded to a server in California for further NCOA processing. The server in California belongs to Accuzip and they stage big files so it can take awhile before it even begins to process. Once it processes, the NCOA server sends a small file back to my VM where Address change information is integrated back in to the local Accuzip database file (foxpro). After that it is exported back to CSV with a bunch of additional fields and is placed in the Accuzip OutputFrom directory along with a PDF which contains the CASS / NCOA statistics. This PDF is a "proof of processing" document that I have to keep forever, and I have to send to the client. I have a Directory Watcher which is watching the OutputFrom directory. When *all* of the resulting Accuzip output files are in the OutputFrom directory, my Directory Watcher event triggers my program to copy all of these files back to the SQL Server In staging directory. 4) Once all of the Accuzip files for a single chunk are back on the SQL Server In directory, my application BCPs the CSV into a input chunk table. >From there decisions are made whether I need to import every address back in or only the moves. In any event some or all of the address records (with the additional fields from Accuzip) are pulled into an AZData table back in the database that the address data originally came from. So... 1) A supervisor object creates a clsDbAZOut instance. 2) clsDBAZOut creates an actual database dbXyzAZOut temporary database. 3) clsDBAZOut pulls every record needing to be accuzipped into a table with a super PK, sorted on Zip5/4 4) clsDBAZOut counts the records in tblAZOut, and starts to build clsProcessFile instances which holds the information about the chunk processing. clsProcessFile can write its info back to the ProcessFile records on SQL Server. 5) clsDbAZOut has a thread which builds tblChunk, gets ChunkSize records into the chunk table and BCPs tblChunk to CSV files on a staging out directory. 6) clsDbAZOut BCPs the chunk to file 7) At this point the clsProcessFile is handed off to clsVM. clsSupervisor has a strongly typed dictionary which stores the clsProcessFile instances created by clsDbAZOut. clsSupervisor can read ProcessFile instances from SQL Server into a clsProcessFile factory or clsDbAZOut can use a class factory to create new ones. In all cases they are stored in the dictionary in clsSupervisor. clsProcessfile has flags which document that: a) The chunk table was created b) The chunk table exported to file c) The chunk went to a VM (and which one) d) The chunk file was (eventually) found in the Accuzip OutputFrom directory e) The resulting AZData file (and PDF) were moved from the VM back to SQL ServerIn staging f) The file in SQL Server In staging was pulled back in to a chunk table in a second database dbXyzAZIn temporary database g) That the chunk table was imported back into the original database. As you can see, clsProcessFile is the center of the universe for a single chunk. clsSupervisor is the center of the universe for the entire process. It is a bit difficult to discuss how the entire program works in a single email. A clsManager loads one or a dozen clsSupervisor instances. clsManager loads clsVM which supervises the (a) virtual machine which runs a third party process. clsVM moves files from SQL Staging Out into the VM one chunk at a time, updating clsProcessTable flags. clsVM moves files from the the VM back to the SQL Staging IN, updating flags. The VM may not be available, or the Accuzip program may be down etc. clsSupervisor loads one to N clsProcessTable instances. clsSupervisor loads a clsDBExp and clsDBImp. clsDBExp creates new clsProcessTable instances, one for each chunk it exports - storing them in clsSupervisor. clsDBExp supervises the export process. clsDBImp supervises the import process getting the chunks back into SQL Server into a temp database, and from there into the live database. I am trying to design the system such that it is asynchronous. Stage1, stage 2/3 and stage 4 can all do their thing independently of each other. Each step of each stage is logged (dateTime flags written) so that any piece can just pick up where it left off. That means any Supervisor and any ProcessFile (chunk) can pick up from any point. In the meantime, the clsManager is watching the tblSupervisor for supervisor records ready to process - a thread. clsSupervisor is watching for ProcessTable records and dispatching them as required to the clsDBExp, clsDBImp or clsVM - a thread in each supervisor. clsDBExp is processing chunks, creating CSV files - a thread. clsVM is watching for clsProcessTable instances ready to go to the VM. It is also watching the output directory and sending files back to the SQL Server - a thread and two file watchers (and two events). clsDBImp is watching for clsProcessTable instances ready to import back into SQL Server - a thread. Most of these threads are writing flags (as they finish their specific process) and reading flags in clsProcessTable to decide what chunk has to be processed in which class. Oh, and I have three list controls on a form which the various classes write to to display the status for Stage1, Stage 2/3 and Stage 4. So pretty much a single supervisor is processing at a time, exporting files, sending them to the vm, importing them back in etc. It is possible for an order supervisor to cut in at the head of the line however. Not a simple task co-ordinating all of this stuff. I used to co-ordinate it all manually! ;) John W. Colby www.ColbyConsulting.com On 3/26/2011 8:08 PM, Shamil Salakhetdinov wrote: > John -- > > I'm falling asleep here now - would that be correct assumption that > you can generalize your global task like that - you have, say: > > - split input data into 50 Chunks; > - every chunk should be processed in 10 steps; > - every step's average processing time is 100 minutes. > > Then in total it is needed 50*10*100 = 50,000 minutes or ~833.(3) > hours to process all your input data sequentially, chunk after chunk > step after step, say, you don't have enough computer power to process > some chunks in parallel, and you can't process the whole not split > input data as it's too large for your system... > > But you have "50 small computers" and you can process every of 50 > input chunks in parallel - then you'll need just 10*100 = 1,000 > minutes (~16.67 > hours) to complete the job. > > I assume that all the chunks processing is independent - then you can: > > 1) Create Scheduler class instance; > 2) Schedule class creates 10 TaskQueue class instances - 1 queue for > every processing step type; > 3) Scheduler class defines splitting criteria to split input data into > 50 chunks; > 4) Schedule defines completion criteria - when all 50 chunks get > collected in 11th FinalQueue; > 5) Schedule seeds the first InputChunkSplitTaskQueue with 50 > SplitTasksDescriptors; > 4) From time to time - say every half a minute Scheduler class > instance creates 500 worker classes in 500 threads (500 worker classes > => 50 chunks * 10 steps.), which "hunt for the job" in their dedicate > Queues: if they find Job they work on it, if not - they "die"... > 5) When a worker class completes processing its job it queues its > "task results" into the next step Queue, if a worker class fails then > it puts its incomplete task descriptor back into its dedicated queue > and "dies", or that could be Scheduler's (or SchedulerAssitant's) task > job to find failed workers' "trap" and to resubmit their failed work > to the proper queue according to the custom workflow descriptors attached to every chunk... > > 500 worker classes is an overkill as only 50 worker classes can have > job every time but that seems to be an easy "brute force" and "lazy > parallel programming" approach - and it should work... > Or you can make worker class instances production processes smarter: > Scheduler class can start special thread for WorkerClassGenerator > instance, which will monitor all the 10 TaskQueues, and if it finds an > item in a Queque, it will pick it up, it will create corresponding > Worker class in parallel thread and it will pass WorkItem to the > Worker class fro processing... > > When that described above approach will work there then you can > easily(?) scale it splitting your (constant size) input data into 100 > chunks, and then if every chunk can be processes in half time - in > average (50 minutes) - then all job can be completed in 500 minutes = ~8.33(3) hours etc... > > Please correct me if I oversimplified your application business area... > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sun Mar 27 22:01:44 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 27 Mar 2011 23:01:44 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> Message-ID: <4D8FFA18.9020902@colbyconsulting.com> Shamil, > Yes, I see: you tend to build a multi-threaded application/utility system > with centralized control - and I tend to build an anarchical free-market > model based application/utility system :) LOL, I guess. To be quite honest I didn't understand exactly what you were saying. Remember that my C# skills are still quite rudimentary. Accuzip Processing system basically has developed over time. I didn't sit down to develop an app from scratch to do this. I started in late 2004 to just get one single list into SQL server and serve up counts to the client. I did the whole thing manually, building queries to import the data and clean it up using only SQL Server Management Server. When I started I had never touched SQL Server. Then I was asked to research an application for doing CASS http://en.wikipedia.org/wiki/Coding_Accuracy_Support_System and NCOA http://en.wikipedia.org/wiki/National_Change_Of_Address on the addresses in the list. There was only one list in the beginning which I originally labeled "the database from hell". So I researched and discovered a vendor (Accuzip) that would allow us a fixed price license against which I could process as many records as I wanted. All other vendors (that I found) charge on a sliding scale Cents / record kind of thing, and we had a LOT of records - 65 million in that first list. So I started building a system of stored procedures which performed the export out to Accuzip and then import the Accuzipped records back in. It took roughly 25 stored procedures to perform this process. I had a paper list of instructions on how to run the stored procedures, parameters to feed in etc, plus manually copying the files out and back in. I would manually type EXEC commands into a query window reading from my paper instructions. It was *sloooow* and *error prone*. Next came a stored procedure that ran the stored procedures. Etc. Etc. It just morphed over time. Plus I got more lists. The client was so impressed that I managed to do all this that I got actual orders (export the data and send to the clients) rather than just counts, and the business grew. I started using Access ( the tool I knew ) to automate some of it but it was a very poor fit. Since every operation ran in a single thread the entire application would lock up when SQL Server went out for 30 minutes to do some operation. And of course my hardware was underpowered. In the summer of 2009 I decided I really needed to automate this. The business had grown and the client was suddenly 50% of my income and it was a huge struggle to process these lists manually, even using stored procedures of lists of stored procedures. TSQL as a programming environment is crude to say the least. So I went to the local community college in Sept. 2009 and took the first of two available C# courses. By Christmas I was launching in to my automation attempt. Understand my C# skills were rudimentary at best but the language was robust and I could see the power. I really just started building a method of moving the paper list of stored procedures into C#, IOW C# would execute the stored procedures and monitor their completion. I was not focused on a top down "build a system" but rather let's get my manual process automated ASAP. I was overworked, and wasn't able to handle the load. It turns out that C# is very capable in this regard and the program (such as it was) started working and working well. At the end of the second semester (May 2010) I hired Paul, a young college student who was graduating from the community college to come to work for me part time. I started teaching him what I know about general programming and together we worked on this program. And it continued to morph. We started using NLog for logging, Visual SVN for source control etc. At some point I decided to just move the Stored Procedures out of SQL Server and into dynamic code generated directly in C#. Each step had to keep the program running because I actually used the program to fill orders and do counts. And remember our C# skills are still rudimentary. Then we "discovered" threads and soon thereafter events. Last fall I decided to remodel the program into a manager / supervisor / worker model where the manager tells the supervisor to do something, the supervisor tells other workers to do something and basically just check results. In this model, each level uses threads to scan it's tasks (watch flags) and keep its workers busy. So the dbExport class has a thread which build a database, pulls records, builds chunks, exports to disk. That is it's job. When the file is on disk it is done and control for that chunk is passed to another class (the Virtual Machine). The dbExport class uses flags to store each step of the process. The Virtual Machine class picks up the files and pulls them over to the VMs directory, monitors that Accuzip processes them, tracks errors (if any) from Accuzip, and places the resulting files back on a staging directory (as I call it) for another class to take over. The dbImport class is tasked with creating a temp database, pulling each chunk file back in from staging and getting the data back into the original list database. So the manager class (a single instance) loads supervisor classes, one per list database that needs processing. The Manager class also loads the VM class (only one of these at the moment but there could be more, with more licenses from Accuzip). Each supervisor loads a dbExport and dbImport class to process the records in its list. dbExport creates Process records and stores them in SQL Server and creates a process class for each process record. Each process record represents a chunk of data from a specific list database. the process class doesn't really do anything it is just a place for the flags to be loaded / manipulated, and it knows how to persist itself to SQL Server. dbImport basically gets Process class instances which are finished with stage 2/3 (out of Accuzip) and starts importing them back in to SQL Server. My concept was a factory kind of application. There can be none or dozens of databases needing to be processed. SQL Server and my disk system can only efficiently handle one at a time (at any stage) and Accuzip is the biggest bottleneck so the VM class sits in the middle of an export and an import class. Every class has its own job, every class has at least one thread for its use. dbExport has a thread to use in exporting data to a table and another thread to bcp the data out while the next chunk is created in the export database. dbImport has a thread to BCP data back into chunk tables, and another thread to take the chunk tables and merge them back into live data. The VM has a thread to monitor the input directory and another thread to monitor the output directory, moving files into and out of the VM. Every step has to be logged back to SQL Server as it completes, but the clsSupervisor keeps the progress flags in memory as long as a specific list database is being processed, and the supervisor holds a list of process classes which hold all of the flags for the steps for a single chunk. It is a busy system. I did not originally use threads, and everything ran on the main program thread. And I was back to my user interface locking up as long running processes happened. Now with threading, each stage (export / vm / import) has its own list control which is used to display the status of the list database / chunks as they progress through the system. Everything is logged to flags, but also to NLog tables. Each class has threads to keep its own processes fed and running, and each class does its own piece of the whole to process a list table through Accuzip. BTW, there is another entire C# program which processes orders for the client. It is also threaded, and one of its steps is to create a supervisor record to tell the Accuzip Monitor to Accuzip the addresses being processed for the order. Orders are the Accuzip processing which I mentioned have to cut in at the head of the line in Accuzip Monitoring. As you have seen, I could be processing a 65 million record list database when an order comes in. 65 million records is going to take at least 50 hours to process, and the order has to go out ASAP so the Accuzip Monitor (the Manager class actually) has to watch for orders and interrupt the data supervisor to allow the Order supervisor to cut in. Once Accuzip starts a process (chunk) I don't interrupt it, but when that process (chunk) finishes Accuzip, only Order chunks will process until the order is finished. As I said, it is a busy system. And remember that our C# skills are still rudimentary. I started coding in C# in Sept 2009 and started migrating this process to c# about December 2009. By May 2010 when Paul started work I was actually running an Accuzip process and doing orders using C#. Paul is a youngster right out of school. So much of the stuff you talk about goes right over my head Shamil. John W. Colby www.ColbyConsulting.com On 3/27/2011 8:08 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I see: you tend to build a multi-threaded application/utility system > with centralized control - and I tend to build an anarchical free-market > model based application/utility system :) > > Thank you. > > -- > Shamil From jwcolby at colbyconsulting.com Sun Mar 27 22:20:36 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 27 Mar 2011 23:20:36 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> Message-ID: <4D8FFE84.5090609@colbyconsulting.com> OK, so I have explained the process. As you can see I am using threads pretty heavily. In general only one thread writes to any given flag field in a class and in general only one (other) thread reads from that flag. However threads are peeking at flags all the time "checking" whether the chunk (in particular) is ready to proceed to the next stage. This weekend I decided to try a dedicated flag class. My idea is simply that each flag represents a field in a table. Each flag represents a specific data element from a specific record. It is only written once (generally) but potentially read often. If I build a dedicated flag class it can be initialized with the table / field / pkid etc for its specific field / record and then when the thread writes the flag, the flag class can write its value out to the right field / record table. Again it is normally only ever written once. Most of my flags are a date / time that the process step occurred. So I am trying to wrap the whole flag concept into a class to allow me to process (read / write / test) flags in an identical way in all of my process classes. It was this flag class which I was discussing the end of last week. It will be accessed by at least two threads so it needs to be thread safe. John W. Colby www.ColbyConsulting.com On 3/27/2011 8:08 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I see: you tend to build a multi-threaded application/utility system > with centralized control - and I tend to build an anarchical free-market > model based application/utility system :) > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Mon Mar 28 17:39:57 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Tue, 29 Mar 2011 02:39:57 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <4D8FFE84.5090609@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> <4D8FFE84.5090609@colbyconsulting.com> Message-ID: <5FD8C68EEEF142DA8632253753C51323@nant> Hi John -- <<< It was this flag class which I was discussing the end of last week. It will be accessed by at least two threads so it needs to be thread safe. >>> Yes, I see - I do wish it worked efficiently there real soon... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 28 ????? 2011 ?. 7:21 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() OK, so I have explained the process. As you can see I am using threads pretty heavily. In general only one thread writes to any given flag field in a class and in general only one (other) thread reads from that flag. However threads are peeking at flags all the time "checking" whether the chunk (in particular) is ready to proceed to the next stage. This weekend I decided to try a dedicated flag class. My idea is simply that each flag represents a field in a table. Each flag represents a specific data element from a specific record. It is only written once (generally) but potentially read often. If I build a dedicated flag class it can be initialized with the table / field / pkid etc for its specific field / record and then when the thread writes the flag, the flag class can write its value out to the right field / record table. Again it is normally only ever written once. Most of my flags are a date / time that the process step occurred. So I am trying to wrap the whole flag concept into a class to allow me to process (read / write / test) flags in an identical way in all of my process classes. It was this flag class which I was discussing the end of last week. It will be accessed by at least two threads so it needs to be thread safe. John W. Colby www.ColbyConsulting.com On 3/27/2011 8:08 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I see: you tend to build a multi-threaded application/utility > system with centralized control - and I tend to build an anarchical > free-market model based application/utility system :) > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From DWUTKA at Marlow.com Thu Mar 31 10:34:13 2011 From: DWUTKA at Marlow.com (Drew Wutka) Date: Thu, 31 Mar 2011 10:34:13 -0500 Subject: [dba-VB] Most useful droid apps In-Reply-To: References: Message-ID: AndroXplorer Very handy for transferring files around between Droid, Network and SD Card. Drew -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of David McAfee Sent: Saturday, March 19, 2011 3:54 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Most useful droid apps Advanced task killer Weather channel Lookout antivirus AK Notepad Bank of America Acar auto maintenance Barcode scanner Bubble level Bump Droid light Google goggles Google translate IMDB Key ring Print share Trapster Sent from my Droid phone. On Mar 19, 2011 12:52 PM, "jwcolby" wrote: _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com The information contained in this transmission is intended only for the person or entity to which it is addressed and may contain II-VI Proprietary and/or II-VI Business Sensitive material. If you are not the intended recipient, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. You are notified that any review, retransmission, copying, disclosure, dissemination, or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. From jwcolby at colbyconsulting.com Wed Mar 2 20:11:33 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 02 Mar 2011 21:11:33 -0500 Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews Message-ID: <4D6EF8D5.7060804@colbyconsulting.com> -- John W. Colby www.ColbyConsulting.com http://www.legitreviews.com/news/10164/ From accessd at shaw.ca Thu Mar 3 14:55:06 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Thu, 3 Mar 2011 12:55:06 -0800 Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews In-Reply-To: <4D6EF8D5.7060804@colbyconsulting.com> References: <4D6EF8D5.7060804@colbyconsulting.com> Message-ID: <7AF8F6950EC14685B928DE79F836CB0D@creativesystemdesigns.com> Soon to be into John's office. ;-) Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, March 02, 2011 6:12 PM To: Access Developers discussion and problem solving; Sqlserver-Dba; VBA Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews -- John W. Colby www.ColbyConsulting.com http://www.legitreviews.com/news/10164/ _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Thu Mar 3 15:05:20 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 03 Mar 2011 16:05:20 -0500 Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews In-Reply-To: <7AF8F6950EC14685B928DE79F836CB0D@creativesystemdesigns.com> References: <4D6EF8D5.7060804@colbyconsulting.com> <7AF8F6950EC14685B928DE79F836CB0D@creativesystemdesigns.com> Message-ID: <4D700290.4010308@colbyconsulting.com> I wish. It will most likely be beyond my means for some time. John W. Colby www.ColbyConsulting.com On 3/3/2011 3:55 PM, Jim Lawrence wrote: > Soon to be into John's office. ;-) > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, March 02, 2011 6:12 PM > To: Access Developers discussion and problem solving; Sqlserver-Dba; VBA > Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - > Legit Reviews > > From michael at ddisolutions.com.au Sun Mar 6 19:26:08 2011 From: michael at ddisolutions.com.au (Michael Maddison) Date: Mon, 7 Mar 2011 12:26:08 +1100 Subject: [dba-VB] Java Help? Message-ID: <99266C61B516644D9727F983FAFAB465046ABC@remote.ddisolutions.com.au> Hi Guys, A while ago I asked about creating a COM dll to be called from Java. Does anyone have the Java skills to see if you can call the dll to test for me? Cheers Michael M From dbdoug at gmail.com Sun Mar 6 21:04:47 2011 From: dbdoug at gmail.com (Doug Steele) Date: Sun, 6 Mar 2011 19:04:47 -0800 Subject: [dba-VB] ASP.Net development Message-ID: Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele From Gustav at cactus.dk Mon Mar 7 04:08:58 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 07 Mar 2011 11:08:58 +0100 Subject: [dba-VB] ASP.Net development Message-ID: Hi Doug I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. /gustav >>> dbdoug at gmail.com 07-03-2011 04:04 >>> Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele From accessd at shaw.ca Mon Mar 7 09:22:24 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Mon, 7 Mar 2011 07:22:24 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: Hi Gustav: Showing my ignorance about Winforms; does Winforms create browser based application or create a Windows based applications that uses a browser? Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Monday, March 07, 2011 2:09 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] ASP.Net development Hi Doug I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. /gustav >>> dbdoug at gmail.com 07-03-2011 04:04 >>> Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From dbdoug at gmail.com Mon Mar 7 09:55:51 2011 From: dbdoug at gmail.com (Doug Steele) Date: Mon, 7 Mar 2011 07:55:51 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: Hi Gustav: Thanks for the reply. If you build a table adapter which consists of data from more than one table (like a one to many relationship) in the design surface, VS2008 will refuse to create any CRUD query code - you have to build it yourself. Maybe you only used one-table adapters? Doug On Mon, Mar 7, 2011 at 2:08 AM, Gustav Brock wrote: > Hi Doug > > I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. > > I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. > You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. > > As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. > > /gustav > > >>>> dbdoug at gmail.com 07-03-2011 04:04 >>> > Hello All: > > A lot of the Microsoft sample code for asp.net data access is built > using strongly typed tableadapters with the .xsd design surface. ?I've > built two apps using these in the last year, but they seem to be kind > of fiddly - they don't work well with multi table update queries, for > instance, and they generate a HUGE amount of background source code. > It is nice to have the strong typing, however. > > I have never found any non-Microsoft sample code which uses this > technique. ?Admittedly, most of the sample code in the wild consists > of short snippets built to illustrate a single technique, not a > complete application. > > Two questions: > 1. Are there developers out there using this approach for real life apps? > 2. If not, what do you recommend? > > > Thanks, > > Doug Steele > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Mon Mar 7 11:04:59 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 07 Mar 2011 18:04:59 +0100 Subject: [dba-VB] ASP.Net development Message-ID: Hi Jim WinForms are true Windows desktop apps (I realise I should have changed the subject line, sorry). If you wish a browser based app, that's ASP.NET or Silverlight if we talk about Visual Studio and dotNet. /gustav >>> accessd at shaw.ca 07-03-2011 16:22 >>> Hi Gustav: Showing my ignorance about Winforms; does Winforms create browser based application or create a Windows based applications that uses a browser? Jim From Gustav at cactus.dk Mon Mar 7 11:17:36 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 07 Mar 2011 18:17:36 +0100 Subject: [dba-VB] ASP.Net development Message-ID: Hi Doug Now I see. Yes, for updates I use one-table tableadapters. Multi-table tableadapters are mainly used as source for reports and list-type forms where nothing is updated - and as source for combo- and listboxes. /gustav >>> dbdoug at gmail.com 07-03-2011 16:55 >>> Hi Gustav: Thanks for the reply. If you build a table adapter which consists of data from more than one table (like a one to many relationship) in the design surface, VS2008 will refuse to create any CRUD query code - you have to build it yourself. Maybe you only used one-table adapters? Doug On Mon, Mar 7, 2011 at 2:08 AM, Gustav Brock wrote: > Hi Doug > > I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. > > I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. > You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. > > As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. > > /gustav > > >>>> dbdoug at gmail.com 07-03-2011 04:04 >>> > Hello All: > > A lot of the Microsoft sample code for asp.net data access is built > using strongly typed tableadapters with the .xsd design surface. I've > built two apps using these in the last year, but they seem to be kind > of fiddly - they don't work well with multi table update queries, for > instance, and they generate a HUGE amount of background source code. > It is nice to have the strong typing, however. > > I have never found any non-Microsoft sample code which uses this > technique. Admittedly, most of the sample code in the wild consists > of short snippets built to illustrate a single technique, not a > complete application. > > Two questions: > 1. Are there developers out there using this approach for real life apps? > 2. If not, what do you recommend? > > > Thanks, > > Doug Steele From accessd at shaw.ca Mon Mar 7 11:52:58 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Mon, 7 Mar 2011 09:52:58 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: <9F8E42F58F6F45B18FBDE74842047BF9@creativesystemdesigns.com> Hi Gustav: Thanks for the information. Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Monday, March 07, 2011 9:05 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] ASP.Net development Hi Jim WinForms are true Windows desktop apps (I realise I should have changed the subject line, sorry). If you wish a browser based app, that's ASP.NET or Silverlight if we talk about Visual Studio and dotNet. /gustav >>> accessd at shaw.ca 07-03-2011 16:22 >>> Hi Gustav: Showing my ignorance about Winforms; does Winforms create browser based application or create a Windows based applications that uses a browser? Jim _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Mon Mar 7 12:45:14 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 7 Mar 2011 21:45:14 +0300 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: <5C52BAB72E1246CFBBFB629233371E72@nant> Hi Doug -- I think I have seen an ASP.NET book with sources where strongly typed datasets were used but I can't find it now. But maybe that was this book: Pro ASP.NET 2.0 E-Commerce in C# 2005 (Expert's Voice in .NET) http://www.amazon.com/ASP-NET-Commerce-2005-Experts-Voice/dp/1590597249/ref= pd_sim_b_8 It does have complete source code and the DAL code does use ADO.NET DataSets but not strongly typed. I have got programmed rather large ASP.NET application - more than 150 webforms with I suppose more than 70-80 heavily related MS SQL tables with millions rows in some of tables, hundreds SPs, ... several thousands C# custom classes, ORM layer etc. and this web application does work well forth year in row with a few support work - the latter mainly being adding new functionality. I have used asp.net development approach described in this book: ASP.NET 2.0 Website Programming: Problem - Design - Solution (Programmer to Programmer) http://www.amazon.com/ASP-NET-2-0-Website-Programming-Programmer/dp/07645846 42/ref=sr_1_2?s=books&ie=UTF8&qid=1299518125&sr=1-2 BTW, the same approach is used in DotNetNuke (but I didn't know that when I have got started to develop my customer web application I mentioned above.)) I have had to develop custom ORM mapping + SQL code generator to implement this approach. But it should be possible to use some existing code generators. This approach looks "low level" and it needs quite some coding (code generators help here) but good news is that it will never get outdated. And here is a book on how to use LINQ 2 SQL for ASP.NET applications (I haven't read and I haven't tried this approach) for the same sample ASP.NET application, which is used in the previously referred book: LINQ-ifying TheBeerHouse: An N-Tier LINQ Web Application with ASP.NET 2.0 Website Programming Problem Design Solution (Wrox Briefs) http://www.amazon.com/LINQ-ifying-TheBeerHouse-Application-Programming-Solut ion/dp/B0026A6AQU/ref=sr_1_106?s=books&ie=UTF8&qid=1299518458&sr=1-106 The most modern approach is to use ADO.NET Entity Framework for DAL as Gustav noted, and when ADO.NET EF is used then it should be possible to map logical/business layer objects to the custom stored procedures, which can insert, update, delete, select data from one or database tables... I haven't yet seen any books where ADO.NET Entity Framework is used with classical WebForms based ASP.NET applications - anybody? Some more information on ADO.NET EF and ASP.NET: - here seems to be a good book on ADO.NET EF: Entity Framework 4.0 Recipes: A Problem-Solution Approach (Expert's Voice in .NET) http://www.amazon.com/Entity-Framework-4-0-Recipes-Problem-Solution/dp/14302 27036/ref=sr_1_168?s=books&ie=UTF8&qid=1299519467&sr=1-168 - and this one "opens secretes" on how to build ultra-fast and robust asp.net applications: Ultra-Fast ASP.NET: Build Ultra-Fast and Ultra-Scalable web sites using ASP.NET and SQL Server [Paperback] http://www.amazon.com/Ultra-Fast-ASP-NET-Build-Ultra-Scalable-Server/dp/1430 223839/ref=sr_1_121?s=books&ie=UTF8&qid=1299518863&sr=1-121 Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Doug Steele Sent: 7 ????? 2011 ?. 6:05 To: dba-VB at databaseadvisors.com Subject: [dba-VB] ASP.Net development Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From dbdoug at gmail.com Mon Mar 7 23:34:51 2011 From: dbdoug at gmail.com (Doug Steele) Date: Mon, 7 Mar 2011 21:34:51 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: <5C52BAB72E1246CFBBFB629233371E72@nant> References: <5C52BAB72E1246CFBBFB629233371E72@nant> Message-ID: Hi Shamil: Thanks very much for your reply. I'll be checking your references out.... Doug On Mon, Mar 7, 2011 at 10:45 AM, Shamil Salakhetdinov wrote: > Hi Doug -- > > I think I have seen an ASP.NET book with sources where strongly typed > datasets were used but I can't find it now. > But maybe that was this book: > > Pro ASP.NET 2.0 E-Commerce in C# 2005 (Expert's Voice in .NET) > http://www.amazon.com/ASP-NET-Commerce-2005-Experts-Voice/dp/1590597249/ref= > pd_sim_b_8 > > It does have complete source code and the DAL code does use ADO.NET DataSets > but not strongly typed. > > I have got programmed rather large ASP.NET application - more than 150 > webforms with I suppose more than 70-80 heavily related MS SQL tables with > millions rows in some of tables, hundreds SPs, ... several thousands C# > custom classes, ORM layer etc. and this web application does work well forth > year in row with a few support work - the latter mainly being adding new > functionality. I have used asp.net development approach described in this > book: > > ASP.NET 2.0 Website Programming: Problem - Design - Solution (Programmer to > Programmer) > http://www.amazon.com/ASP-NET-2-0-Website-Programming-Programmer/dp/07645846 > 42/ref=sr_1_2?s=books&ie=UTF8&qid=1299518125&sr=1-2 > > BTW, the same approach is used in DotNetNuke (but I didn't know that when I > have got started to develop my customer web application I mentioned above.)) > > I have had to develop custom ORM mapping + SQL code generator to implement > this approach. > But it should be possible to use some existing code generators. > > This approach looks "low level" and it needs quite some coding (code > generators help here) but good news is that it will never get outdated. > > And here is a book on how to use LINQ 2 SQL for ASP.NET applications (I > haven't read and I haven't tried this approach) for the same sample ASP.NET > application, which is used in the previously referred book: > > LINQ-ifying TheBeerHouse: An N-Tier LINQ Web Application with ASP.NET 2.0 > Website Programming Problem Design Solution (Wrox Briefs) > http://www.amazon.com/LINQ-ifying-TheBeerHouse-Application-Programming-Solut > ion/dp/B0026A6AQU/ref=sr_1_106?s=books&ie=UTF8&qid=1299518458&sr=1-106 > > The most modern approach is to use ADO.NET Entity Framework for DAL as > Gustav noted, and when ADO.NET EF is used then it should be possible to map > logical/business layer objects to the custom stored procedures, which can > insert, update, delete, select data from one or database tables... > > I haven't yet seen any books where ADO.NET Entity Framework is used with > classical WebForms based ASP.NET applications - anybody? > > > Some more information on ADO.NET EF and ASP.NET: > > - here seems to be a good book on ADO.NET EF: > > Entity Framework 4.0 Recipes: A Problem-Solution Approach (Expert's Voice in > .NET) > http://www.amazon.com/Entity-Framework-4-0-Recipes-Problem-Solution/dp/14302 > 27036/ref=sr_1_168?s=books&ie=UTF8&qid=1299519467&sr=1-168 > > - and this one "opens secretes" on how to build ultra-fast and robust > asp.net applications: > > Ultra-Fast ASP.NET: Build Ultra-Fast and Ultra-Scalable web sites using > ASP.NET and SQL Server [Paperback] > http://www.amazon.com/Ultra-Fast-ASP-NET-Build-Ultra-Scalable-Server/dp/1430 > 223839/ref=sr_1_121?s=books&ie=UTF8&qid=1299518863&sr=1-121 > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Doug Steele > Sent: 7 ????? 2011 ?. 6:05 > To: dba-VB at databaseadvisors.com > Subject: [dba-VB] ASP.Net development > > Hello All: > > A lot of the Microsoft sample code for asp.net data access is built using > strongly typed tableadapters with the .xsd design surface. ?I've built two > apps using these in the last year, but they seem to be kind of fiddly - they > don't work well with multi table update queries, for instance, and they > generate a HUGE amount of background source code. > It is nice to have the strong typing, however. > > I have never found any non-Microsoft sample code which uses this technique. > Admittedly, most of the sample code in the wild consists of short snippets > built to illustrate a single technique, not a complete application. > > Two questions: > 1. Are there developers out there using this approach for real life apps? > 2. If not, what do you recommend? > > > Thanks, > > Doug Steele > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From raibeart at gmail.com Tue Mar 8 11:44:25 2011 From: raibeart at gmail.com (Robert Stewart) Date: Tue, 08 Mar 2011 11:44:25 -0600 Subject: [dba-VB] I'm getting nowhere In-Reply-To: References: Message-ID: <4d766a81.cac4e60a.3ab2.255f@mx.google.com> John, Logins: If you are in mixed mode, which for your application you should be, every "group" will have it's own login with a password. User: References a login. A user is given rights to a database and specific objects in a database. Roles: Generally, you can assign DataReader and DataWriter roles to a User in a database to give them CRUD rights to tables. You will need to GRANT EXECUTE rights to stored procedures. Does that help? Robert At 09:22 AM 3/7/2011, you wrote: >From: dba-vb-bounces at databaseadvisors.com >[mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >Sent: Saturday, February 19, 2011 2:54 PM >To: Sqlserver-Dba; VBA; Access Developers discussion and problem solving >Subject: [dba-VB] I'm getting nowhere > >I am getting nowhere on understanding SQL Server security. Microsoft >provides us with SQL Server Express which implies that joe blow (me) is >going to install / maintain it. > >I am not a SQL Server Admin and I cannot afford to spend the time to be one. > >Google is my friend. BOL is not. > >Except that Google is taking me to these places where I am expected to >already know how this stuff works, and then wants to make me a *better* >administrator. Which of course is useless because I am not an administrator >at all. > >OTOH I am not stupid. If I could find something that started at the "This >is SQL Server security" >basics I could learn this stuff. Before anyone says "RTFM (BOL)" let me >simply say, "not happening". I have tried BOL and it simply sucks for my >level of expertise (my opinion of course). > If that is your advice, simply stay out of this thread. Thanks! > >So... my needs: > >I need to set up several SQL Server databases for use by different, very >small groups (5-20 people) of entirely unrelated people. What I mean by >that is that each DB is for a different "company" if you will. I need to >access these databases from C#. I understand the group / user paradigm. I >would like to create groups and users. Specific groups can do specific >things in the database, some can see data but not modify it. Some can add >records in specific tables but not others. Some can run reports (view). > >I do *NOT* want to create windows level groups and users if I can avoid it. >These are people that I do not necessarily know and I do not want to give >them any rights at the machine level, and I prefer to not maintain such >lists at the machine level. > >Unfortunately SQL Server does not seem to model Groups / users. I go into >SQL Server and see a security tab. It has "logins". Is that a user? A >specific ability to log in with a password? To what? The server itself? A >specific database? Groups of databases? > >I see "roles" but these appear to be aimed at the server and none of these >people are going to be doing anything at the server level. > >Can I safely ignore everything under the server security tab? > >I go to a database and I see a security tab. It has users and roles. >Hmm... better (I would think). I would like to add users "under" the >specific database that the user will access. > >So I try to add a new user but I do not see anywhere to require a password. >Hmmm... > >I go into roles and I do not see any predefined role that looks like it >would be useful to me in meeting my needs described above. If I look at >"add new role" it asks for a password. The User / group model does nto >assign passwords at the group level which implies that a role is not a group >at the user / group paradigm. > >Is it just me, or is SQL Server security just... different? Am I correct in >assuming that it doesn't implement a user / group paradigm? > >And more importantly, where can I go to get a plain, simple, English >description of how this mess works? > >And please excuse the tone that results from my frustration. The only help >documents that I have found (and I have extensive lists of bookmarked web >pages) so far assume that I am an administrator. > I am not, and cannot afford to become one. And yet MS pushes SQL Express >as if I (non-admin) should be able to use this as a data store pool. > >Help! > >-- >John W. Colby >www.ColbyConsulting.com >_______________________________________________ From jwcolby at colbyconsulting.com Thu Mar 10 07:12:44 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 08:12:44 -0500 Subject: [dba-VB] c# lock() Message-ID: <4D78CE4C.5000308@colbyconsulting.com> I am using multi-threading and need to lock list objects so that two different threads can access them. Do I even need to lock the list inside of the property as follows: public Dictionary pLstFiles { get { lock (lstFiles) { return lstFiles; } } } or only when I actually access the list? If I create a property to return the list (without locking it in the property) public Dictionary pLstFiles { get { return lstFiles; } } and I then get a pointer to the list using that property Dictionary LstFiles; LstFiles = SomeObj.pLstFiles; and I then lock the pointer while using it lock(LstFiles) { do stuff to list here... } Does that in fact lock the list to other objects doing the same thing (getting the list through that property)? IOW is the lock attribute affixed to the list at the list itself so that anything trying to use the list anywhere in the program can see that the list is locked? -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Thu Mar 10 10:29:39 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 11:29:39 -0500 Subject: [dba-VB] split an SVN database Message-ID: <4D78FC73.6000202@colbyconsulting.com> I use SVN. I am looking at doing a major split of a project into two projects. I want to leave the existing solution until I have one half of the current project carved out, debugged and running and in it's own SVN database. However I don't think I want to carry along the baggage of the old revisions into the new database. How do I go about this? -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Thu Mar 10 14:37:01 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 15:37:01 -0500 Subject: [dba-VB] c# - controlling the run state / status of your threaded classes Message-ID: <4D79366D.8060905@colbyconsulting.com> As I write classes that consume threads, I need to be able to start and stop these threads and the class that consumes the threads. For example, I open a form and load a class instance that starts performing some process - exporting sets of records to csv files for example. Once that class is loaded and processing, I need to be able to ask that class to stop exporting files. However there may be a time lag between asking and the currently processing file finishing it's export. That is precisely the purpose of a thread - tell it to go and do something else. So (for example) I have a check box that I check that tells the class to start exporting. When it starts it sets a flag saying that it is running and places text into the check box label saying that it is exporting (busy). If I then uncheck the check box I am telling the class to stop exporting. The class is monitoring the check box and will not process the next file but it can't just exit. So it needs to set a flag saying that it is stopping and display "stopping" on the check box label. When it finishes exporting the file, it sets a stopped flag and says displays "terminated" in the check box label. So we have a couple of "commands" (start / stop) and a few statuses (stopped, stopping, running). I would like to "standardize" this into a class so that any class which needs this interface does it in the same way. Some controlling object can tell the object to start and stop, and the class can tell the controlling object that is is stopped, stopping or running. this class does not6 need to do any updating of labels or anything like that, just hold the variables, properties, methods and code to perform this in a standard way. Is anyone doing this? Have existing code you would like to share? Have existing experience trying to do this you would like to share? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Thu Mar 10 15:35:24 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 00:35:24 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D78CE4C.5000308@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com> Message-ID: <3693FCAC03E94BCA9C150F6E7AFE4A71@nant> Hi John -- <<< Does that in fact lock the list to other objects doing the same thing (getting the list through that property)? >>> You have to lock/synchronize/serialize not the list itself but simultaneous access to the list from different threads. You'd better use special static object variable to implement locking. I will post sample code in my following posting here... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 16:13 To: VBA Subject: [dba-VB] c# lock() I am using multi-threading and need to lock list objects so that two different threads can access them. Do I even need to lock the list inside of the property as follows: public Dictionary pLstFiles { get { lock (lstFiles) { return lstFiles; } } } or only when I actually access the list? If I create a property to return the list (without locking it in the property) public Dictionary pLstFiles { get { return lstFiles; } } and I then get a pointer to the list using that property Dictionary LstFiles; LstFiles = SomeObj.pLstFiles; and I then lock the pointer while using it lock(LstFiles) { do stuff to list here... } Does that in fact lock the list to other objects doing the same thing (getting the list through that property)? IOW is the lock attribute affixed to the list at the list itself so that anything trying to use the list anywhere in the program can see that the list is locked? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Thu Mar 10 15:36:58 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 00:36:58 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D78CE4C.5000308@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com> Message-ID: <4889C56E6FF246D6AB908FEFD1069A68@nant> Hi John -- Here is sample code I promised to post. using System; using System.Collections.Generic; namespace ConsoleApplication1 { class Log { public static bool On { get; set; } public static void WriteLine(string message) { WriteLine(message, false); } public static void WriteLine(string message, bool forceLogging) { if (Log.On || forceLogging) System.Console.WriteLine(message); } } class MyObject { static int _newId; public MyObject() { _id = System.Threading.Interlocked.Increment(ref _newId); _timeStamp = DateTime.Now; } int _id; public int ID { get { return _id; } set { _id = value; } } DateTime _timeStamp; public DateTime TimeStamp { get { return _timeStamp; } set { _timeStamp = value; } } } class MyObjectsStore { private List _keys = new List(); private Dictionary _myObjects = new Dictionary(); public MyObject this[int key] { get { MyObject myObject = null; lock (_myObjectsLocker) { _myObjects.TryGetValue(key, out myObject); } return myObject; } } public int Count { get { lock (_myObjectsLocker) { return _myObjects.Count; } } } private static object _myObjectsLocker = new object(); public void Add(MyObject myObject) { lock (_myObjectsLocker) { _myObjects.Add(myObject.ID, myObject); _keys.Add(myObject.ID); } } public MyObject Remove(int id) { MyObject myobject = null; lock (_myObjectsLocker) { myobject = _myObjects[id]; _myObjects.Remove(id); } return myobject; } static Random _rnd = new Random(); public MyObject RemoveRandomObject() { MyObject myobject = null; int index = -1; int key = -1; try { lock (_myObjectsLocker) { index = _rnd.Next(_keys.Count - 1); key = _keys[index]; myobject = Remove(key); _keys.Remove(key); } } catch (Exception ex) { Log.WriteLine(string.Format("[{0}]: RemoveRandomObject: {1}, index = {2}, key = {3}", System.Threading.Thread.CurrentThread.ManagedThreadId, ex.Message, index , key)); } return myobject; } } class Program { public const int OBJECTS_PER_THREAD_COUNT = 7; public const int REMOVAL_MAX_RETRIES_COUNT = 5; public static void AddObjectsTest() { for (int i = 1; i <= OBJECTS_PER_THREAD_COUNT; i++) { MyObject myObject = new MyObject(); _store.Add(myObject); Log.WriteLine(string.Format("[{0}]: Object Added - ID = {1}, TS = {2}", System.Threading.Thread.CurrentThread.ManagedThreadId, myObject.ID, myObject.TimeStamp.Ticks)); System.Threading.Thread.Sleep(2); System.Threading.Interlocked.Increment(ref _createdObjectsCount); } System.Threading.Interlocked.Decrement(ref _threadsCount); } public static void RemoveObjectsTest() { for (int i = 1; i <= OBJECTS_PER_THREAD_COUNT; i++) { int retryCount = REMOVAL_MAX_RETRIES_COUNT; retry: MyObject myObject = _store.RemoveRandomObject(); if (myObject != null) { Log.WriteLine(string.Format("[{0}]: Object Removed - ID = {1}, TS = {2}", System.Threading.Thread.CurrentThread.ManagedThreadId, myObject.ID, myObject.TimeStamp.Ticks)); System.Threading.Thread.Sleep(2); } System.Threading.Thread.Sleep(2); if (myObject == null && --retryCount >= 0) { Log.WriteLine(string.Format("[{0}]: Object Removal failed, retry attempt# {1}", System.Threading.Thread.CurrentThread.ManagedThreadId, REMOVAL_MAX_RETRIES_COUNT-retryCount)); System.Threading.Interlocked.Increment(ref _totalRetriesCount); goto retry; } else if (myObject == null) { Log.WriteLine(string.Format("[{0}]: Object Removal failed", System.Threading.Thread.CurrentThread.ManagedThreadId)); System.Threading.Interlocked.Increment(ref _failedObjectRemovalsCount); } else System.Threading.Interlocked.Increment(ref _removedObjectsCount); } System.Threading.Interlocked.Decrement(ref _threadsCount); } static int _createdObjectsCount; static int _removedObjectsCount; static int _totalRetriesCount; static int _failedObjectRemovalsCount; static int _threadsCount; static MyObjectsStore _store; const int THREADS_COUNT = 20; //000; static void Main(string[] args) { Log.On = true; Log.WriteLine(string.Format("{0}: Test Started.", DateTime.Now)); try { Log.On = false; _store = new MyObjectsStore(); for (int i = 1; i <= THREADS_COUNT; i++) { if (i % 2 == 1) new System.Threading.Thread( new System.Threading.ThreadStart(AddObjectsTest)) .Start(); else new System.Threading.Thread( new System.Threading.ThreadStart(RemoveObjectsTest)) .Start(); _threadsCount++; } while (_threadsCount > 0) System.Threading.Thread.Sleep(100); System.Threading.Thread.Sleep(100); Log.On = true; Log.WriteLine(string.Format("Total Threads Count = {0}", THREADS_COUNT)); Log.WriteLine(string.Format("Store has {0} objects, Created = {1}, Removed = {2}", _store.Count, _createdObjectsCount, _removedObjectsCount)); Log.WriteLine(string.Format("RemovalRetriesCount = {0}, FailedRemovalsCount = {1}", _totalRetriesCount, _failedObjectRemovalsCount)); } catch (Exception ex) { Log.On = true; Log.WriteLine(string.Format("Program.Main: " + ex.Message)); } finally { Log.On = true; } Log.WriteLine(string.Format("{0}: Test Finished.", DateTime.Now)); } } } Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 16:13 To: VBA Subject: [dba-VB] c# lock() <<< snip >>> From shamil at smsconsulting.spb.ru Thu Mar 10 15:38:46 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 00:38:46 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D78CE4C.5000308@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com> Message-ID: <50A8FE7A468B41D48B9CC4F729CB02E3@nant> Hi John -- Here are my test results for sample code I've just posted (hopefully it get through AccessD - it was a bit lengthy) //11/03/2011 0:32:50: Test Started. //Total Threads Count = 20 //Store has 0 objects, Created = 70, Removed = 70 //RemovalRetriesCount = 0, FailedRemovalsCount = 0 //11/03/2011 0:32:50: Test Finished. //11/03/2011 0:32:25: Test Started. //Total Threads Count = 200 //Store has 0 objects, Created = 700, Removed = 700 //RemovalRetriesCount = 0, FailedRemovalsCount = 0 //11/03/2011 0:32:25: Test Finished. //11/03/2011 0:31:32: Test Started. //Total Threads Count = 20000 //Store has 19 objects, Created = 70000, Removed = 69981 //RemovalRetriesCount = 295, FailedRemovalsCount = 19 //11/03/2011 0:31:43: Test Finished. //11/03/2011 0:33:32: Test Started. //[3]: Object Added - ID = 1, TS = 634354004122968750 //[5]: Object Added - ID = 2, TS = 634354004122968750 //[3]: Object Added - ID = 3, TS = 634354004122968750 //[4]: Object Removed - ID = 2, TS = 634354004122968750 //[5]: Object Added - ID = 4, TS = 634354004122968750 //[6]: Object Removed - ID = 3, TS = 634354004122968750 //[7]: Object Added - ID = 5, TS = 634354004122968750 //[8]: Object Removed - ID = 4, TS = 634354004122968750 //[9]: Object Added - ID = 6, TS = 634354004122968750 //[10]: Object Removed - ID = 1, TS = 634354004122968750 //[11]: Object Added - ID = 7, TS = 634354004122968750 //[3]: Object Added - ID = 8, TS = 634354004122968750 //[12]: Object Removed - ID = 7, TS = 634354004122968750 //[5]: Object Added - ID = 9, TS = 634354004122968750 //[7]: Object Added - ID = 11, TS = 634354004122968750 //[9]: Object Added - ID = 10, TS = 634354004122968750 //[4]: Object Removed - ID = 6, TS = 634354004122968750 //[11]: Object Added - ID = 12, TS = 634354004122968750 //[3]: Object Added - ID = 13, TS = 634354004122968750 //[7]: Object Added - ID = 16, TS = 634354004123125000 //[6]: Object Removed - ID = 13, TS = 634354004122968750 //[11]: Object Added - ID = 17, TS = 634354004123125000 //[9]: Object Added - ID = 15, TS = 634354004123125000 //[4]: Object Removed - ID = 16, TS = 634354004123125000 //[8]: Object Removed - ID = 11, TS = 634354004122968750 //[10]: Object Removed - ID = 8, TS = 634354004122968750 //[5]: Object Added - ID = 14, TS = 634354004122968750 //[11]: Object Added - ID = 21, TS = 634354004123125000 //[3]: Object Added - ID = 18, TS = 634354004123125000 //[7]: Object Added - ID = 19, TS = 634354004123125000 //[10]: Object Removed - ID = 15, TS = 634354004123125000 //[6]: Object Removed - ID = 5, TS = 634354004122968750 //[4]: Object Removed - ID = 19, TS = 634354004123125000 //[8]: Object Removed - ID = 9, TS = 634354004122968750 //[10]: Object Removed - ID = 21, TS = 634354004123125000 //[6]: Object Removed - ID = 17, TS = 634354004123125000 //[8]: Object Removed - ID = 12, TS = 634354004122968750 //[4]: Object Removed - ID = 22, TS = 634354004123281250 //[12]: Object Removed - ID = 10, TS = 634354004122968750 //[6]: Object Removed - ID = 18, TS = 634354004123125000 //[4]: Object Removed - ID = 24, TS = 634354004123437500 //[8]: Object Removed - ID = 20, TS = 634354004123125000 //[6]: Object Removed - ID = 23, TS = 634354004123437500 //[12]: Object Removed - ID = 25, TS = 634354004123437500 //[10]: Object Removed - ID = 14, TS = 634354004122968750 //[9]: Object Added - ID = 20, TS = 634354004123125000 //[5]: Object Added - ID = 22, TS = 634354004123281250 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[10]: Object Removed - ID = 26, TS = 634354004124062500 //[6]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 1 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[10]: Object Removed - ID = 27, TS = 634354004124062500 //[3]: Object Added - ID = 24, TS = 634354004123437500 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[6]: Object Removal failed, retry attempt# 1 //[6]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 2 //[4]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[8]: Object Removal failed, retry attempt# 1 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[11]: Object Added - ID = 23, TS = 634354004123437500 //[7]: Object Added - ID = 25, TS = 634354004123437500 //[6]: Object Removal failed, retry attempt# 2 //[12]: Object Removed - ID = 28, TS = 634354004124531250 //[4]: Object Removal failed, retry attempt# 1 //[8]: Object Removal failed, retry attempt# 2 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[6]: Object Removed - ID = 29, TS = 634354004125000000 //[4]: Object Removed - ID = 30, TS = 634354004125000000 //[8]: Object Removal failed, retry attempt# 3 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 1 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[8]: Object Removal failed, retry attempt# 4 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 2 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[3]: Object Added - ID = 28, TS = 634354004124531250 //[12]: Object Removal failed, retry attempt# 3 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[8]: Object Removal failed, retry attempt# 5 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 4 //[7]: Object Added - ID = 30, TS = 634354004125000000 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[5]: Object Added - ID = 27, TS = 634354004124062500 //[9]: Object Added - ID = 26, TS = 634354004124062500 //[12]: Object Removal failed, retry attempt# 5 //[12]: Object Removed - ID = 31, TS = 634354004126093750 //[8]: Object Removal failed //[8]: Object Removed - ID = 32, TS = 634354004126250000 //[9]: Object Added - ID = 33, TS = 634354004126250000 //[12]: Object Removed - ID = 33, TS = 634354004126250000 //[11]: Object Added - ID = 29, TS = 634354004125000000 //[9]: Object Added - ID = 34, TS = 634354004126406250 //[12]: Object Removed - ID = 34, TS = 634354004126406250 //[7]: Object Added - ID = 31, TS = 634354004126093750 //[5]: Object Added - ID = 32, TS = 634354004126250000 //[11]: Object Added - ID = 35, TS = 634354004126406250 //Total Threads Count = 10 //Store has 1 objects, Created = 35, Removed = 34 //RemovalRetriesCount = 15, FailedRemovalsCount = 1 //11/03/2011 0:33:32: Test Finished. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 16:13 To: VBA Subject: [dba-VB] c# lock() <<< snip >>> From jwcolby at colbyconsulting.com Thu Mar 10 16:27:16 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 17:27:16 -0500 Subject: [dba-VB] c# lock() In-Reply-To: <3693FCAC03E94BCA9C150F6E7AFE4A71@nant> References: <4D78CE4C.5000308@colbyconsulting.com> <3693FCAC03E94BCA9C150F6E7AFE4A71@nant> Message-ID: <4D795044.6080808@colbyconsulting.com> Shamil, Thanks for the code but I think it is overkill for my needs so far, though I will put it in the archives for when I need it. In my system so far I have (or can arrange to have ) only two threads trying to access a list. Basically what I am doing is reading records out of sql server into a class factory, and placing the class instances into a list. Another object with its own thread is coming by and looking at the objects in the list to see if there is any work for it to do. If it finds an object to work on, it grabs a pointer to that item in the list and stores it in a pointer and releases the list. One of the ways I saw this handled was to create factory which creates a brand new list object, grabs a pointer to the list, locks that and then copies the objects in the list pointers to classes) into that new list, unlocks the list and hands back a pointer to this brand new list. thus the new list could be iterated at will. I am just trying to lock the list while one thread adds objects to the list or another thread gets objects out of the list. So if I have object A which contains a list and is adding / deleting objects to that list... and object B which needs to iterate through the list looking for an object of interest in the list, and grabbing a pointer to that object, there is no way to do this without all of the code you wrote? John W. Colby www.ColbyConsulting.com On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: > Hi John -- > > <<< > Does that in fact lock the list to other objects doing the same thing > (getting the list through that property)? >>>> > You have to lock/synchronize/serialize not the list itself but simultaneous > access to the list from different threads. > You'd better use special static object variable to implement locking. > > I will post sample code in my following posting here... > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Thu Mar 10 16:51:26 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 01:51:26 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D795044.6080808@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com> Message-ID: <3B9DE744CEA244829179A81CA818F92F@nant> Hi John -- Do you you need add, remove and iterate a list to get an object or you need to add, remove and get an object by its key? - if the latter - here is the code you can use: private Dictionary _myObjects = new Dictionary(); private static object _myObjectsLocker = new object(); public void Add(MyObject myObject) { lock (_myObjectsLocker) { _myObjects.Add(myObject.ID, myObject); _keys.Add(myObject.ID); } } public MyObject Remove(int id) { MyObject myobject = null; lock (_myObjectsLocker) { myobject = _myObjects[id]; _myObjects.Remove(id); } return myobject; } public MyObject this[int key] { get { MyObject myObject = null; lock (_myObjectsLocker) { _myObjects.TryGetValue(key, out myObject); } return myObject; } } for the case of iteration just replace: _myObjects.TryGetValue(key, out myObject); with your iteration code. As for the sample code - changing qty of threads and changing total computer workload (outer context for the test code) you can see how different could be final results when heavy multi-threading is used. And even if it's not heavy - it still can result in weird output if special multi-threading coding technique aren't used... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 1:27 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, Thanks for the code but I think it is overkill for my needs so far, though I will put it in the archives for when I need it. In my system so far I have (or can arrange to have ) only two threads trying to access a list. Basically what I am doing is reading records out of sql server into a class factory, and placing the class instances into a list. Another object with its own thread is coming by and looking at the objects in the list to see if there is any work for it to do. If it finds an object to work on, it grabs a pointer to that item in the list and stores it in a pointer and releases the list. One of the ways I saw this handled was to create factory which creates a brand new list object, grabs a pointer to the list, locks that and then copies the objects in the list pointers to classes) into that new list, unlocks the list and hands back a pointer to this brand new list. thus the new list could be iterated at will. I am just trying to lock the list while one thread adds objects to the list or another thread gets objects out of the list. So if I have object A which contains a list and is adding / deleting objects to that list... and object B which needs to iterate through the list looking for an object of interest in the list, and grabbing a pointer to that object, there is no way to do this without all of the code you wrote? John W. Colby www.ColbyConsulting.com On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: > Hi John -- > > <<< > Does that in fact lock the list to other objects doing the same thing > (getting the list through that property)? >>>> > You have to lock/synchronize/serialize not the list itself but > simultaneous access to the list from different threads. > You'd better use special static object variable to implement locking. > > I will post sample code in my following posting here... > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Thu Mar 10 17:11:33 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 02:11:33 +0300 Subject: [dba-VB] c# - controlling the run state / status of your threadedclasses In-Reply-To: <4D79366D.8060905@colbyconsulting.com> References: <4D79366D.8060905@colbyconsulting.com> Message-ID: <3B44AE877A14441CA13A51D1BA80FF65@nant> Hi John -- <<< So we have a couple of "commands" (start / stop) and a few statuses (stopped, stopping, running). >>> You'd better also add 'paused' (and maybe 'pausing' and 'resuming') for your list of states. I must note it's not easy to "standardize" such a coding - as with every framework/standard there immediately come non-standard situations to code into framework/standard - that's endless story... I have such a coding but it's rather complicated, several nesting levels etc.etc. - and it was programmed for a customer, I have reused it already in three projects of that customer but every time I do reuse it I have to do some additions/changes. It tested with up to hundred heavy duty threads talking to web services, MS Access(!) be databases, informing FE UI on threads' status changes etc. - and FE is able to work for days without restarting but that is a result of a very big work - it took two years+ to get from first version to the current one. Of course I didn't program it for two years, but getting all that experience to make it as it's now - that was rather time consuming - at least for me... If I ever will make that code less complicated I will share it but that will not happen soon, sorry... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 23:37 To: VBA Subject: [dba-VB] c# - controlling the run state / status of your threadedclasses As I write classes that consume threads, I need to be able to start and stop these threads and the class that consumes the threads. For example, I open a form and load a class instance that starts performing some process - exporting sets of records to csv files for example. Once that class is loaded and processing, I need to be able to ask that class to stop exporting files. However there may be a time lag between asking and the currently processing file finishing it's export. That is precisely the purpose of a thread - tell it to go and do something else. So (for example) I have a check box that I check that tells the class to start exporting. When it starts it sets a flag saying that it is running and places text into the check box label saying that it is exporting (busy). If I then uncheck the check box I am telling the class to stop exporting. The class is monitoring the check box and will not process the next file but it can't just exit. So it needs to set a flag saying that it is stopping and display "stopping" on the check box label. When it finishes exporting the file, it sets a stopped flag and says displays "terminated" in the check box label. So we have a couple of "commands" (start / stop) and a few statuses (stopped, stopping, running). I would like to "standardize" this into a class so that any class which needs this interface does it in the same way. Some controlling object can tell the object to start and stop, and the class can tell the controlling object that is is stopped, stopping or running. this class does not6 need to do any updating of labels or anything like that, just hold the variables, properties, methods and code to perform this in a standard way. Is anyone doing this? Have existing code you would like to share? Have existing experience trying to do this you would like to share? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 11 08:26:06 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 09:26:06 -0500 Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath Message-ID: <4D7A30FE.8050500@colbyconsulting.com> I am trying to copy a solution to a new location, strip all version control from it, delete a ton of projects within that solution and then version control the new solution. I copied the entire solution from my laptop to a VM and under my user / projects there. The copy still appears to be under version control. I looked in help and it says to copy the directory to itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the top level directory (the solution) but left the check mark on all of the subdirectories (projects and stuff). Does anyone have a clue how to completely and entirely remove a directory from version control - remove all green check marks from every subdirectory etc.? -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Fri Mar 11 08:59:23 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 09:59:23 -0500 Subject: [dba-VB] [dba-SQLServer] SVN - removing all version control for a directory and all files underneath In-Reply-To: <4D7A30FE.8050500@colbyconsulting.com> References: <4D7A30FE.8050500@colbyconsulting.com> Message-ID: <4D7A38CB.7090807@colbyconsulting.com> As it turns out, doing the "export to same directory" seems to actually "unregister" the directory from source control. It leaves the green check marks in explorer - not sure what that means. However if I go into the solution itself, it no longer has the version control dots next to the items in the solution explorer. So it appears to have removed that local copy of the solution from source control. John W. Colby www.ColbyConsulting.com On 3/11/2011 9:26 AM, jwcolby wrote: > I am trying to copy a solution to a new location, strip all version control from it, delete a ton of > projects within that solution and then version control the new solution. > > I copied the entire solution from my laptop to a VM and under my user / projects there. The copy > still appears to be under version control. I looked in help and it says to copy the directory to > itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the > top level directory (the solution) but left the check mark on all of the subdirectories (projects > and stuff). > > Does anyone have a clue how to completely and entirely remove a directory from version control - > remove all green check marks from every subdirectory etc.? From shamil at smsconsulting.spb.ru Fri Mar 11 09:00:05 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 18:00:05 +0300 Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath In-Reply-To: <4D7A30FE.8050500@colbyconsulting.com> References: <4D7A30FE.8050500@colbyconsulting.com> Message-ID: <9192624AA1BA4A6BB323E053411525B8@nant> Hi John -- 1. Just delete hidden .svn subfolder from root folder and all subfolders. 2. Right-click folder in MS Windows Explorer window and use Tortoise SVN Export... command from popup-menu 3. Use command line SVN utility with export command http://svnbook.red-bean.com/nightly/en/svn.ref.svn.c.export.html Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 17:26 To: Access Developers discussion and problem solving; VBA; Sqlserver-Dba Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath I am trying to copy a solution to a new location, strip all version control from it, delete a ton of projects within that solution and then version control the new solution. I copied the entire solution from my laptop to a VM and under my user / projects there. The copy still appears to be under version control. I looked in help and it says to copy the directory to itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the top level directory (the solution) but left the check mark on all of the subdirectories (projects and stuff). Does anyone have a clue how to completely and entirely remove a directory from version control - remove all green check marks from every subdirectory etc.? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 11 10:56:44 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 11:56:44 -0500 Subject: [dba-VB] c# - Creating and using a shared module Message-ID: <4D7A544C.7090904@colbyconsulting.com> I have split my solution into two solutions. However each of those two solutions uses a shared project which I call "base objects". Obviously I want to split that off and then reference that solution from both of the main solutions. I have never done this before. Any suggestions, tips and tricks, warnings, how to etc.? -- John W. Colby www.ColbyConsulting.com From gustav at cactus.dk Fri Mar 11 11:59:11 2011 From: gustav at cactus.dk (Gustav Brock) Date: Fri, 11 Mar 2011 18:59:11 +0100 Subject: [dba-VB] Visual Studio 2010 SP1 Message-ID: Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-475d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav From shamil at smsconsulting.spb.ru Fri Mar 11 11:52:31 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 20:52:31 +0300 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7A544C.7090904@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> Message-ID: <1573E00753994C19BE24FA9E50093615@nant> Hi John -- <<< Any suggestions, tips and tricks, warnings, how to etc.? >>> Just do it! :) I mean if you managed to handle MS Access library databases then handling .NET class libraries would be a "breeze" work for you. Create class library project - you can keep it within its own solution having main (console) project of that solution as a (unit) test one. Put all three solutions - two application ones and class library one in subfolders of the same root folder. (You can do it differently but this is how it's done usually AFAIK) Add existing class library project to application solutions. Reference class library projects from application projects. if you use VS2008 and you have a good PC with plenty of memory (and you have one don't you?) then you can keep all three solutions open in three instances of VS2008. Just keep tack from which solution you have made class library code changes last time - when you'll switch to the other solution and if you have the same source code file opened in it you'll get a message like that "Source file changed - would you like to reload it?" - reply yes... ... Use SVN or Mercurial for source code control. Do commit source code changes to the source code repository from time to time... ... I have 58 projects in one of the solutions here - that results in more than 5 apps and quite some class libs compiled and built - AFAIK folks do have more than hundred projects within solutions sometimes... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 19:57 To: VBA Subject: [dba-VB] c# - Creating and using a shared module I have split my solution into two solutions. However each of those two solutions uses a shared project which I call "base objects". Obviously I want to split that off and then reference that solution from both of the main solutions. I have never done this before. Any suggestions, tips and tricks, warnings, how to etc.? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From df.waters at comcast.net Fri Mar 11 13:46:10 2011 From: df.waters at comcast.net (Dan Waters) Date: Fri, 11 Mar 2011 13:46:10 -0600 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: Message-ID: <003001cbe024$f9c71750$ed5545f0$@comcast.net> I did this yesterday. The download iso image was actually about 1.5 Gb, but I downloaded, burned to disc, and installed in about an hour. Your mileage may vary! Dan -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-47 5d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 11 20:37:10 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 21:37:10 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <1573E00753994C19BE24FA9E50093615@nant> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant> Message-ID: <4D7ADC56.1040806@colbyconsulting.com> You make it sound so easy, but it is an existing project already which was not created as a class library project. I need to create the class library project then somehow get the directories and files into that project and get it to compile. At that point I can reference it by other solutions. John W. Colby www.ColbyConsulting.com On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: > Hi John -- > > <<< > Any suggestions, tips and tricks, warnings, how to etc.? >>>> > Just do it! :) > > I mean if you managed to handle MS Access library databases then handling > .NET class libraries would be a "breeze" work for you. > > Create class library project - you can keep it within its own solution > having main (console) project of that solution as a (unit) test one. > Put all three solutions - two application ones and class library one in > subfolders of the same root folder. (You can do it differently but this is > how it's done usually AFAIK) > Add existing class library project to application solutions. > Reference class library projects from application projects. > if you use VS2008 and you have a good PC with plenty of memory (and you have > one don't you?) then you can keep all three solutions open in three > instances of VS2008. > Just keep tack from which solution you have made class library code changes > last time - when you'll switch to the other solution and if you have the > same source code file opened in it you'll get a message like that "Source > file changed - would you like to reload it?" - reply yes... > > ... > > Use SVN or Mercurial for source code control. Do commit source code changes > to the source code repository from time to time... > ... > > I have 58 projects in one of the solutions here - that results in more than > 5 apps and quite some class libs compiled and built - AFAIK folks do have > more than hundred projects within solutions sometimes... > > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 11 ????? 2011 ?. 19:57 > To: VBA > Subject: [dba-VB] c# - Creating and using a shared module > > I have split my solution into two solutions. However each of those two > solutions uses a shared project which I call "base objects". Obviously I > want to split that off and then reference that solution from both of the > main solutions. > > I have never done this before. Any suggestions, tips and tricks, warnings, > how to etc.? > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From michael at ddisolutions.com.au Fri Mar 11 20:39:50 2011 From: michael at ddisolutions.com.au (Michael Maddison) Date: Sat, 12 Mar 2011 13:39:50 +1100 Subject: [dba-VB] SVN - removing all version control for a directory andall files underneath References: <4D7A30FE.8050500@colbyconsulting.com> <9192624AA1BA4A6BB323E053411525B8@nant> Message-ID: <99266C61B516644D9727F983FAFAB46508623F@remote.ddisolutions.com.au> Hi Guys, I found a reg file that adds a right click menu item to remove all hidden svn dirs. And files from the selected dir. Create a .reg file, Paste this code in, Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Folder\shell\DeleteSVN] @="Delete SVN Folders" [HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Folder\shell\DeleteSVN\command] @="cmd.exe /c \"TITLE Removing SVN Folders in %1 && COLOR 9A && FOR /r \"%1\" %%f IN (.svn) DO RD /s /q \"%%f\" \"" Save it, Merge it to your reg. Works for me. Cheers Michael From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: Saturday, 12 March 2011 2:00 AM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] SVN - removing all version control for a directory andall files underneath Hi John -- 1. Just delete hidden .svn subfolder from root folder and all subfolders. 2. Right-click folder in MS Windows Explorer window and use Tortoise SVN Export... command from popup-menu 3. Use command line SVN utility with export command http://svnbook.red-bean.com/nightly/en/svn.ref.svn.c.export.html Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 17:26 To: Access Developers discussion and problem solving; VBA; Sqlserver-Dba Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath I am trying to copy a solution to a new location, strip all version control from it, delete a ton of projects within that solution and then version control the new solution. I copied the entire solution from my laptop to a VM and under my user / projects there. The copy still appears to be under version control. I looked in help and it says to copy the directory to itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the top level directory (the solution) but left the check mark on all of the subdirectories (projects and stuff). Does anyone have a clue how to completely and entirely remove a directory from version control - remove all green check marks from every subdirectory etc.? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com ________________________________ No virus found in this message. Checked by AVG - www.avg.com Version: 10.0.1204 / Virus Database: 1498/3501 - Release Date: 03/11/11 From jwcolby at colbyconsulting.com Fri Mar 11 21:09:49 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 22:09:49 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7ADC56.1040806@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant> <4D7ADC56.1040806@colbyconsulting.com> Message-ID: <4D7AE3FD.1040206@colbyconsulting.com> I found a "make project from existing files. When I did that it offered to use a selected directory so I dragged the project directory into a new directory and told it to use that. I selected a class library type of project and it went to work "converting" my existing project into a class library. 112 compile errors, most having to do with the fact that it no longer understands using System.Windows.XYZ Many of the common classes did things like accept a combo loaded data into it in a standard way. Stuff like that. So there we are. I "just did it"! ;) John W. Colby www.ColbyConsulting.com On 3/11/2011 9:37 PM, jwcolby wrote: > You make it sound so easy, but it is an existing project already which was not created as a class > library project. I need to create the class library project then somehow get the directories and > files into that project and get it to compile. > > At that point I can reference it by other solutions. > > John W. Colby > www.ColbyConsulting.com > > On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Any suggestions, tips and tricks, warnings, how to etc.? >>>>> >> Just do it! :) >> >> I mean if you managed to handle MS Access library databases then handling >> .NET class libraries would be a "breeze" work for you. >> >> Create class library project - you can keep it within its own solution >> having main (console) project of that solution as a (unit) test one. >> Put all three solutions - two application ones and class library one in >> subfolders of the same root folder. (You can do it differently but this is >> how it's done usually AFAIK) >> Add existing class library project to application solutions. >> Reference class library projects from application projects. >> if you use VS2008 and you have a good PC with plenty of memory (and you have >> one don't you?) then you can keep all three solutions open in three >> instances of VS2008. >> Just keep tack from which solution you have made class library code changes >> last time - when you'll switch to the other solution and if you have the >> same source code file opened in it you'll get a message like that "Source >> file changed - would you like to reload it?" - reply yes... >> >> ... >> >> Use SVN or Mercurial for source code control. Do commit source code changes >> to the source code repository from time to time... >> ... >> >> I have 58 projects in one of the solutions here - that results in more than >> 5 apps and quite some class libs compiled and built - AFAIK folks do have >> more than hundred projects within solutions sometimes... >> >> >> Thank you. >> >> -- >> Shamil From shamil at smsconsulting.spb.ru Sat Mar 12 03:34:15 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 12 Mar 2011 12:34:15 +0300 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7AE3FD.1040206@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant><4D7ADC56.1040806@colbyconsulting.com> <4D7AE3FD.1040206@colbyconsulting.com> Message-ID: <8458BA03716A4820980078DE24EC3D06@nant> Hi John -- You have to add references to some of System.Windows.XYZ: - System.Windows.Forms - ... Just compare references of your source project with a classlib project created from it - and add missing references... <<< I "just did it"! ;) >>> But you did it, didn't you? ;) Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 12 ????? 2011 ?. 6:10 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# - Creating and using a shared module I found a "make project from existing files. When I did that it offered to use a selected directory so I dragged the project directory into a new directory and told it to use that. I selected a class library type of project and it went to work "converting" my existing project into a class library. 112 compile errors, most having to do with the fact that it no longer understands using System.Windows.XYZ Many of the common classes did things like accept a combo loaded data into it in a standard way. Stuff like that. So there we are. I "just did it"! ;) John W. Colby www.ColbyConsulting.com On 3/11/2011 9:37 PM, jwcolby wrote: > You make it sound so easy, but it is an existing project already which > was not created as a class library project. I need to create the class > library project then somehow get the directories and files into that project and get it to compile. > > At that point I can reference it by other solutions. > > John W. Colby > www.ColbyConsulting.com > > On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Any suggestions, tips and tricks, warnings, how to etc.? >>>>> >> Just do it! :) >> >> I mean if you managed to handle MS Access library databases then >> handling .NET class libraries would be a "breeze" work for you. >> >> Create class library project - you can keep it within its own >> solution having main (console) project of that solution as a (unit) test one. >> Put all three solutions - two application ones and class library one >> in subfolders of the same root folder. (You can do it differently but >> this is how it's done usually AFAIK) Add existing class library >> project to application solutions. >> Reference class library projects from application projects. >> if you use VS2008 and you have a good PC with plenty of memory (and >> you have one don't you?) then you can keep all three solutions open >> in three instances of VS2008. >> Just keep tack from which solution you have made class library code >> changes last time - when you'll switch to the other solution and if >> you have the same source code file opened in it you'll get a message >> like that "Source file changed - would you like to reload it?" - reply yes... >> >> ... >> >> Use SVN or Mercurial for source code control. Do commit source code >> changes to the source code repository from time to time... >> ... >> >> I have 58 projects in one of the solutions here - that results in >> more than >> 5 apps and quite some class libs compiled and built - AFAIK folks do >> have more than hundred projects within solutions sometimes... >> >> >> Thank you. >> >> -- >> Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Mar 12 06:27:48 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 12 Mar 2011 07:27:48 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <8458BA03716A4820980078DE24EC3D06@nant> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant><4D7ADC56.1040806@colbyconsulting.com> <4D7AE3FD.1040206@colbyconsulting.com> <8458BA03716A4820980078DE24EC3D06@nant> Message-ID: <4D7B66C4.6040404@colbyconsulting.com> Well... the whole thing compiled (the project) when it was part of the bigger solution. I'll give that a try. John W. Colby www.ColbyConsulting.com On 3/12/2011 4:34 AM, Shamil Salakhetdinov wrote: > Hi John -- > > You have to add references to some of System.Windows.XYZ: > > - System.Windows.Forms > - ... > > Just compare references of your source project with a classlib project > created from it - and add missing references... > > <<< > I "just did it"! ;) >>>> > But you did it, didn't you? ;) > > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 12 ????? 2011 ?. 6:10 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# - Creating and using a shared module > > I found a "make project from existing files. When I did that it offered to > use a selected directory so I dragged the project directory into a new > directory and told it to use that. I selected a class library type of > project and it went to work "converting" my existing project into a class > library. > > 112 compile errors, most having to do with the fact that it no longer > understands > > using System.Windows.XYZ > > Many of the common classes did things like accept a combo loaded data into > it in a standard way. > Stuff like that. > > So there we are. > > I "just did it"! ;) > > John W. Colby > www.ColbyConsulting.com > > On 3/11/2011 9:37 PM, jwcolby wrote: >> You make it sound so easy, but it is an existing project already which >> was not created as a class library project. I need to create the class >> library project then somehow get the directories and files into that > project and get it to compile. >> >> At that point I can reference it by other solutions. >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >>> Hi John -- >>> >>> <<< >>> Any suggestions, tips and tricks, warnings, how to etc.? >>>>>> >>> Just do it! :) >>> >>> I mean if you managed to handle MS Access library databases then >>> handling .NET class libraries would be a "breeze" work for you. >>> >>> Create class library project - you can keep it within its own >>> solution having main (console) project of that solution as a (unit) test > one. >>> Put all three solutions - two application ones and class library one >>> in subfolders of the same root folder. (You can do it differently but >>> this is how it's done usually AFAIK) Add existing class library >>> project to application solutions. >>> Reference class library projects from application projects. >>> if you use VS2008 and you have a good PC with plenty of memory (and >>> you have one don't you?) then you can keep all three solutions open >>> in three instances of VS2008. >>> Just keep tack from which solution you have made class library code >>> changes last time - when you'll switch to the other solution and if >>> you have the same source code file opened in it you'll get a message >>> like that "Source file changed - would you like to reload it?" - reply > yes... >>> >>> ... >>> >>> Use SVN or Mercurial for source code control. Do commit source code >>> changes to the source code repository from time to time... >>> ... >>> >>> I have 58 projects in one of the solutions here - that results in >>> more than >>> 5 apps and quite some class libs compiled and built - AFAIK folks do >>> have more than hundred projects within solutions sometimes... >>> >>> >>> Thank you. >>> >>> -- >>> Shamil > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 12 15:00:17 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 12 Mar 2011 16:00:17 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7B66C4.6040404@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant><4D7ADC56.1040806@colbyconsulting.com> <4D7AE3FD.1040206@colbyconsulting.com> <8458BA03716A4820980078DE24EC3D06@nant> <4D7B66C4.6040404@colbyconsulting.com> Message-ID: <4D7BDEE1.40400@colbyconsulting.com> OK, I got all of the references fixed up and got it checked into VSN. John W. Colby www.ColbyConsulting.com On 3/12/2011 7:27 AM, jwcolby wrote: > Well... the whole thing compiled (the project) when it was part of the bigger solution. > > I'll give that a try. > > John W. Colby > www.ColbyConsulting.com > > On 3/12/2011 4:34 AM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> You have to add references to some of System.Windows.XYZ: >> >> - System.Windows.Forms >> - ... >> >> Just compare references of your source project with a classlib project >> created from it - and add missing references... >> >> <<< >> I "just did it"! ;) >>>>> >> But you did it, didn't you? ;) >> >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 12 ????? 2011 ?. 6:10 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# - Creating and using a shared module >> >> I found a "make project from existing files. When I did that it offered to >> use a selected directory so I dragged the project directory into a new >> directory and told it to use that. I selected a class library type of >> project and it went to work "converting" my existing project into a class >> library. >> >> 112 compile errors, most having to do with the fact that it no longer >> understands >> >> using System.Windows.XYZ >> >> Many of the common classes did things like accept a combo loaded data into >> it in a standard way. >> Stuff like that. >> >> So there we are. >> >> I "just did it"! ;) >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/11/2011 9:37 PM, jwcolby wrote: >>> You make it sound so easy, but it is an existing project already which >>> was not created as a class library project. I need to create the class >>> library project then somehow get the directories and files into that >> project and get it to compile. >>> >>> At that point I can reference it by other solutions. >>> >>> John W. Colby >>> www.ColbyConsulting.com >>> >>> On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >>>> Hi John -- >>>> >>>> <<< >>>> Any suggestions, tips and tricks, warnings, how to etc.? >>>>>>> >>>> Just do it! :) >>>> >>>> I mean if you managed to handle MS Access library databases then >>>> handling .NET class libraries would be a "breeze" work for you. >>>> >>>> Create class library project - you can keep it within its own >>>> solution having main (console) project of that solution as a (unit) test >> one. >>>> Put all three solutions - two application ones and class library one >>>> in subfolders of the same root folder. (You can do it differently but >>>> this is how it's done usually AFAIK) Add existing class library >>>> project to application solutions. >>>> Reference class library projects from application projects. >>>> if you use VS2008 and you have a good PC with plenty of memory (and >>>> you have one don't you?) then you can keep all three solutions open >>>> in three instances of VS2008. >>>> Just keep tack from which solution you have made class library code >>>> changes last time - when you'll switch to the other solution and if >>>> you have the same source code file opened in it you'll get a message >>>> like that "Source file changed - would you like to reload it?" - reply >> yes... >>>> >>>> ... >>>> >>>> Use SVN or Mercurial for source code control. Do commit source code >>>> changes to the source code repository from time to time... >>>> ... >>>> >>>> I have 58 projects in one of the solutions here - that results in >>>> more than >>>> 5 apps and quite some class libs compiled and built - AFAIK folks do >>>> have more than hundred projects within solutions sometimes... >>>> >>>> >>>> Thank you. >>>> >>>> -- >>>> Shamil >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From bheid at sc.rr.com Sun Mar 13 09:26:07 2011 From: bheid at sc.rr.com (Bobby Heid) Date: Sun, 13 Mar 2011 10:26:07 -0400 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: Message-ID: <005d01cbe18a$977c6700$c6753500$@rr.com> Only took me about 40 minutes on my machine (Win 7 Ultimate, VS22010 Ultimate, and SQL 2008 Standard). Bobby -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Friday, March 11, 2011 12:59 PM To: dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-47 5d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sun Mar 13 21:58:15 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 13 Mar 2011 22:58:15 -0400 Subject: [dba-VB] Access runtime (an maybe full install?) Message-ID: <4D7D8447.4000003@colbyconsulting.com> I am trying to get an Access 2007 runtime to run reliably anywhere I install it. I had a Windows 2003 virtual machine hanging around so I cleaned Office off of it, and I installed the runtime. I got the directory path declared a safe location and voila, the application ran. Full of confidence I installed Hamachi, the runtime package and the app on a friend's (rather old) XP and it doesn't run. I dug out my old dev machine which it appears that I wiped and installed XP on some time ago. I installed Hamachi, got it on the VPN for this database etc. I then installed the runtime there and tried to run. No go. Of course the runtime doesn't report errors or at least reliably. So I built a little test version with just a single table and an autoform which automatically opened when the app opened. It actually reported that it couldn't talk to the odbc native sql driver (not the exact words of course). So I started Googling and soon discovered that this driver is installed by SQL Server as it installs. However an installer called sqlncli.msi is out there for download. I downloaded that and tried to run. No joy. I turned the machine off, and ran a network cable so it is directly on my network and voila, it runs. I then disabled the cable so that it is connecting to my network via my wireless and it runs. So it appears that the SQL driver is not installed as part of XP, which makes sense since this is SQL Server 2008 Express version I am connecting to. However I would have expected this odbc SQL driver to be part of the Access 2007 runtime, but it seems not. It does appear that a download / install / reboot of the machine will allow me to hook up to the sql server instance. So I am now running Hamachi, joined to this specific network, with the sqlncli.msi installer installing the ODBC SQL Server native driver, plus the Access 2007 runtime, running the full on app, hitting the sql server over the vpn on 5.203.167.79 (the Hamachi IP for the server). And it works on this machine of mine. Now I have to install the SQL Server drivers on the friend's old machine (also running XP but on 512 megs of ram) and ensure that will run. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sun Mar 13 22:27:45 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 13 Mar 2011 23:27:45 -0400 Subject: [dba-VB] [AccessD] Access runtime (an maybe full install?) In-Reply-To: <4D7D8447.4000003@colbyconsulting.com> References: <4D7D8447.4000003@colbyconsulting.com> Message-ID: <4D7D8B31.4080405@colbyconsulting.com> And it runs on the friend's old machine as well. It feels like I might be working now. I have to say this has been one of the most complex jobs I have taken on, with Hamachi VPN networks, a virtual machine running the sql server database, and then the client machine running Hamachi and an Access 2007 runtime. Safe zones, sql server drivers not installed. It has been a challenge getting this running. John W. Colby www.ColbyConsulting.com On 3/13/2011 10:58 PM, jwcolby wrote: > I am trying to get an Access 2007 runtime to run reliably anywhere I install it. I had a Windows > 2003 virtual machine hanging around so I cleaned Office off of it, and I installed the runtime. I > got the directory path declared a safe location and voila, the application ran. > > Full of confidence I installed Hamachi, the runtime package and the app on a friend's (rather old) > XP and it doesn't run. I dug out my old dev machine which it appears that I wiped and installed XP > on some time ago. I installed Hamachi, got it on the VPN for this database etc. I then installed the > runtime there and tried to run. No go. > > Of course the runtime doesn't report errors or at least reliably. So I built a little test version > with just a single table and an autoform which automatically opened when the app opened. It actually > reported that it couldn't talk to the odbc native sql driver (not the exact words of course). So I > started Googling and soon discovered that this driver is installed by SQL Server as it installs. > However an installer called sqlncli.msi is out there for download. > > I downloaded that and tried to run. No joy. I turned the machine off, and ran a network cable so it > is directly on my network and voila, it runs. I then disabled the cable so that it is connecting to > my network via my wireless and it runs. > > So it appears that the SQL driver is not installed as part of XP, which makes sense since this is > SQL Server 2008 Express version I am connecting to. However I would have expected this odbc SQL > driver to be part of the Access 2007 runtime, but it seems not. > > It does appear that a download / install / reboot of the machine will allow me to hook up to the sql > server instance. > > So I am now running Hamachi, joined to this specific network, with the sqlncli.msi installer > installing the ODBC SQL Server native driver, plus the Access 2007 runtime, running the full on app, > hitting the sql server over the vpn on 5.203.167.79 (the Hamachi IP for the server). > > And it works on this machine of mine. > > Now I have to install the SQL Server drivers on the friend's old machine (also running XP but on 512 > megs of ram) and ensure that will run. > > From Gustav at cactus.dk Mon Mar 14 05:38:03 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 14 Mar 2011 11:38:03 +0100 Subject: [dba-VB] Visual Studio 2010 SP1 Message-ID: Hi all OK, don't blame the child. At one time it even asked for the VS2010 dvd - to roll back the installation. Another install was in progress, or at least I hadn't rebooted the machine. So I did and ran the SP1 again. This time - as everything was downloaded - it took about an hour to finish. /gustav >>> gustav at cactus.dk 11-03-2011 18:59 >>> Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-475d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav From jwcolby at colbyconsulting.com Mon Mar 14 13:16:52 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 14:16:52 -0400 Subject: [dba-VB] C# combo boxes Message-ID: <4D7E5B94.2020805@colbyconsulting.com> I am trying to set up a combo box bound to a field in the form such that the combo populates with the PKID and a text field, and when the form opens, selecting something from the combo drops the PKID into the field in the form. I am not getting what to set the properties of the combo box to. I am pulling the ID and the string to display into the combo but I cannot cause the combo to store the id and release control to the next control on the form. -- John W. Colby www.ColbyConsulting.com From Gustav at cactus.dk Mon Mar 14 14:02:27 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 14 Mar 2011 20:02:27 +0100 Subject: [dba-VB] C# combo boxes Message-ID: Hi John If you (typically) have this in InitializeComponent() of the .Designer.cs file: // comboBoxCountry // this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); this.comboBoxCountry.DisplayMember = "Id"; this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; this.comboBoxCountry.FormattingEnabled = true; this.comboBoxCountry.ItemHeight = 13; this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); this.comboBoxCountry.Name = "comboBoxCountry"; this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); this.comboBoxCountry.Sorted = true; this.comboBoxCountry.TabIndex = 7; this.comboBoxCountry.ValueMember = "Id"; You can do something like this in the Load event: this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); Then: private void FillComboBoxCountryCode() { // Order the country names of countryDataTable. string filterExpression = ""; string sortExpression = "CountryNameUK asc"; DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); this.comboBoxCountry.DataSource = countryList; this.comboBoxCountry.ValueMember = "Id"; this.comboBoxCountry.DisplayMember = "CountryNameUK"; } And: private void ChangeCountrySelection() { if (this.comboBoxCountry.SelectedValue != null) { _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); this.comboBoxPostalCode.Text = String.Empty; // Read postal codes of selected country. FillComboBoxPostalCode(); } } This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. /gustav >>> jwcolby at colbyconsulting.com 14-03-2011 19:16 >>> I am trying to set up a combo box bound to a field in the form such that the combo populates with the PKID and a text field, and when the form opens, selecting something from the combo drops the PKID into the field in the form. I am not getting what to set the properties of the combo box to. I am pulling the ID and the string to display into the combo but I cannot cause the combo to store the id and release control to the next control on the form. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Mon Mar 14 16:36:55 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 17:36:55 -0400 Subject: [dba-VB] C# combo boxes In-Reply-To: References: Message-ID: <4D7E8A77.7070009@colbyconsulting.com> Thanks Gustav, I don't know why you are taking me into code in code. I have data connections. This particular data connection has a single view that allows me to write into a table. It has a view that filters a combo. I drag and drop the view for the table onto the form. I have told that table object that specific fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am trying to set up that combo to 1) Display a text value 2) Contain the PKID from the table the combo draws from 3) When I select an item, place the selected ID into the field the combo is bound to. It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or going into code). Actually I have a kinda sorta related question. When I look at server explorer I see a data connection object, however there are two objects underneath, neither of which are for this solution. Furthermore the objects under the data connection are things found on an entirely unrelated server, for other projects that I did work on in the past. The question then is why is the server explorer choosing to display objects out on a virtual machine instead of the database that was originally selected for this solution? We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound form stuff. I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values in most cases) the developer, the product, the type of work, then a couple of fields for date time values for start / stop work. The table the data is going into has a few thousand records already in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound forms is a natural for this. A long time ago I made this (kind of thing) work. I remember that getting the properties of the combo set correctly was non-intuitive but once I figured it out it just worked. John W. Colby www.ColbyConsulting.com On 3/14/2011 3:02 PM, Gustav Brock wrote: > Hi John > > If you (typically) have this in InitializeComponent() of the .Designer.cs file: > > // comboBoxCountry > // > this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; > this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; > this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; > this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); > this.comboBoxCountry.DisplayMember = "Id"; > this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; > this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); > this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; > this.comboBoxCountry.FormattingEnabled = true; > this.comboBoxCountry.ItemHeight = 13; > this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); > this.comboBoxCountry.Name = "comboBoxCountry"; > this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); > this.comboBoxCountry.Sorted = true; > this.comboBoxCountry.TabIndex = 7; > this.comboBoxCountry.ValueMember = "Id"; > > You can do something like this in the Load event: > > this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); > > Then: > private void FillComboBoxCountryCode() > { > // Order the country names of countryDataTable. > string filterExpression = ""; > string sortExpression = "CountryNameUK asc"; > DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); > > this.comboBoxCountry.DataSource = countryList; > this.comboBoxCountry.ValueMember = "Id"; > this.comboBoxCountry.DisplayMember = "CountryNameUK"; > } > > And: > private void ChangeCountrySelection() > { > if (this.comboBoxCountry.SelectedValue != null) > { > _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); > this.comboBoxPostalCode.Text = String.Empty; > // Read postal codes of selected country. > FillComboBoxPostalCode(); > } > } > > This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> > I am trying to set up a combo box bound to a field in the form such that the combo populates with > the PKID and a text field, and when the form opens, selecting something from the combo drops the > PKID into the field in the form. > > I am not getting what to set the properties of the combo box to. I am pulling the ID and the string > to display into the combo but I cannot cause the combo to store the id and release control to the > next control on the form. From gustav at cactus.dk Mon Mar 14 17:15:35 2011 From: gustav at cactus.dk (Gustav Brock) Date: Mon, 14 Mar 2011 23:15:35 +0100 Subject: [dba-VB] C# combo boxes Message-ID: Hi John Eh, are you commenting or? Not quite clear to me. I thought you were looking for some code samples ... I have connections too but I use them only to feed the datatableadapters. I refrain from working with SqlCommand and the like; waste of time in my opinion. /gustav >>> jwcolby at colbyconsulting.com 14-03-2011 22:36 >>> Thanks Gustav, I don't know why you are taking me into code in code. I have data connections. This particular data connection has a single view that allows me to write into a table. It has a view that filters a combo. I drag and drop the view for the table onto the form. I have told that table object that specific fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am trying to set up that combo to 1) Display a text value 2) Contain the PKID from the table the combo draws from 3) When I select an item, place the selected ID into the field the combo is bound to. It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or going into code). Actually I have a kinda sorta related question. When I look at server explorer I see a data connection object, however there are two objects underneath, neither of which are for this solution. Furthermore the objects under the data connection are things found on an entirely unrelated server, for other projects that I did work on in the past. The question then is why is the server explorer choosing to display objects out on a virtual machine instead of the database that was originally selected for this solution? We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound form stuff. I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values in most cases) the developer, the product, the type of work, then a couple of fields for date time values for start / stop work. The table the data is going into has a few thousand records already in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound forms is a natural for this. A long time ago I made this (kind of thing) work. I remember that getting the properties of the combo set correctly was non-intuitive but once I figured it out it just worked. John W. Colby www.ColbyConsulting.com On 3/14/2011 3:02 PM, Gustav Brock wrote: > Hi John > > If you (typically) have this in InitializeComponent() of the .Designer.cs file: > > // comboBoxCountry > // > this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; > this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; > this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; > this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); > this.comboBoxCountry.DisplayMember = "Id"; > this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; > this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); > this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; > this.comboBoxCountry.FormattingEnabled = true; > this.comboBoxCountry.ItemHeight = 13; > this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); > this.comboBoxCountry.Name = "comboBoxCountry"; > this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); > this.comboBoxCountry.Sorted = true; > this.comboBoxCountry.TabIndex = 7; > this.comboBoxCountry.ValueMember = "Id"; > > You can do something like this in the Load event: > > this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); > > Then: > private void FillComboBoxCountryCode() > { > // Order the country names of countryDataTable. > string filterExpression = ""; > string sortExpression = "CountryNameUK asc"; > DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); > > this.comboBoxCountry.DataSource = countryList; > this.comboBoxCountry.ValueMember = "Id"; > this.comboBoxCountry.DisplayMember = "CountryNameUK"; > } > > And: > private void ChangeCountrySelection() > { > if (this.comboBoxCountry.SelectedValue != null) > { > _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); > this.comboBoxPostalCode.Text = String.Empty; > // Read postal codes of selected country. > FillComboBoxPostalCode(); > } > } > > This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> > I am trying to set up a combo box bound to a field in the form such that the combo populates with > the PKID and a text field, and when the form opens, selecting something from the combo drops the > PKID into the field in the form. > > I am not getting what to set the properties of the combo box to. I am pulling the ID and the string > to display into the combo but I cannot cause the combo to store the id and release control to the > next control on the form. From jwcolby at colbyconsulting.com Mon Mar 14 17:25:08 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 18:25:08 -0400 Subject: [dba-VB] C# combo boxes In-Reply-To: References: Message-ID: <4D7E95C4.9020203@colbyconsulting.com> I am trying (unsuccessfully) to do an "Access like bound form". And I always comment. ;) ATM the form displays the values if dropped down, but if not dropped down it just displays an integer (the PKID of the data in the combo is my guess, or perhaps the PKID in the field in the form. So... How do I cause the combo to display Paul's name instead of 6. John W. Colby www.ColbyConsulting.com On 3/14/2011 6:15 PM, Gustav Brock wrote: > Hi John > > Eh, are you commenting or? Not quite clear to me. I thought you were looking for some code samples ... > > I have connections too but I use them only to feed the datatableadapters. I refrain from working with SqlCommand and the like; waste of time in my opinion. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> > Thanks Gustav, > I don't know why you are taking me into code in code. > > I have data connections. This particular data connection has a single view that allows me to write > into a table. It has a view that filters a combo. > > I drag and drop the view for the table onto the form. I have told that table object that specific > fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am > trying to set up that combo to > > 1) Display a text value > 2) Contain the PKID from the table the combo draws from > 3) When I select an item, place the selected ID into the field the combo is bound to. > > It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or > going into code). > > Actually I have a kinda sorta related question. > > When I look at server explorer I see a data connection object, however there are two objects > underneath, neither of which are for this solution. Furthermore the objects under the data > connection are things found on an entirely unrelated server, for other projects that I did work on > in the past. > > The question then is why is the server explorer choosing to display objects out on a virtual machine > instead of the database that was originally selected for this solution? > > We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound > form stuff. > > I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values > in most cases) the developer, the product, the type of work, then a couple of fields for date time > values for start / stop work. The table the data is going into has a few thousand records already > in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound > forms is a natural for this. > > A long time ago I made this (kind of thing) work. I remember that getting the properties of the > combo set correctly was non-intuitive but once I figured it out it just worked. > > > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 3:02 PM, Gustav Brock wrote: >> Hi John >> >> If you (typically) have this in InitializeComponent() of the .Designer.cs file: >> >> // comboBoxCountry >> // >> this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; >> this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; >> this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; >> this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); >> this.comboBoxCountry.DisplayMember = "Id"; >> this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; >> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); >> this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; >> this.comboBoxCountry.FormattingEnabled = true; >> this.comboBoxCountry.ItemHeight = 13; >> this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); >> this.comboBoxCountry.Name = "comboBoxCountry"; >> this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); >> this.comboBoxCountry.Sorted = true; >> this.comboBoxCountry.TabIndex = 7; >> this.comboBoxCountry.ValueMember = "Id"; >> >> You can do something like this in the Load event: >> >> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); >> >> Then: >> private void FillComboBoxCountryCode() >> { >> // Order the country names of countryDataTable. >> string filterExpression = ""; >> string sortExpression = "CountryNameUK asc"; >> DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); >> >> this.comboBoxCountry.DataSource = countryList; >> this.comboBoxCountry.ValueMember = "Id"; >> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >> } >> >> And: >> private void ChangeCountrySelection() >> { >> if (this.comboBoxCountry.SelectedValue != null) >> { >> _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); >> this.comboBoxPostalCode.Text = String.Empty; >> // Read postal codes of selected country. >> FillComboBoxPostalCode(); >> } >> } >> >> This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >> I am trying to set up a combo box bound to a field in the form such that the combo populates with >> the PKID and a text field, and when the form opens, selecting something from the combo drops the >> PKID into the field in the form. >> >> I am not getting what to set the properties of the combo box to. I am pulling the ID and the string >> to display into the combo but I cannot cause the combo to store the id and release control to the >> next control on the form. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From accessd at shaw.ca Mon Mar 14 18:59:26 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Mon, 14 Mar 2011 16:59:26 -0700 Subject: [dba-VB] C# combo boxes In-Reply-To: <4D7E95C4.9020203@colbyconsulting.com> References: <4D7E95C4.9020203@colbyconsulting.com> Message-ID: Hi John: Try and use ADO. It is on every Windows based computer already and requires no additional ODBC to be setup. In other words, you do not even have to see the computers on which the application is going to have a sable connection. Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, March 14, 2011 3:25 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# combo boxes I am trying (unsuccessfully) to do an "Access like bound form". And I always comment. ;) ATM the form displays the values if dropped down, but if not dropped down it just displays an integer (the PKID of the data in the combo is my guess, or perhaps the PKID in the field in the form. So... How do I cause the combo to display Paul's name instead of 6. John W. Colby www.ColbyConsulting.com On 3/14/2011 6:15 PM, Gustav Brock wrote: > Hi John > > Eh, are you commenting or? Not quite clear to me. I thought you were looking for some code samples ... > > I have connections too but I use them only to feed the datatableadapters. I refrain from working with SqlCommand and the like; waste of time in my opinion. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> > Thanks Gustav, > I don't know why you are taking me into code in code. > > I have data connections. This particular data connection has a single view that allows me to write > into a table. It has a view that filters a combo. > > I drag and drop the view for the table onto the form. I have told that table object that specific > fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am > trying to set up that combo to > > 1) Display a text value > 2) Contain the PKID from the table the combo draws from > 3) When I select an item, place the selected ID into the field the combo is bound to. > > It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or > going into code). > > Actually I have a kinda sorta related question. > > When I look at server explorer I see a data connection object, however there are two objects > underneath, neither of which are for this solution. Furthermore the objects under the data > connection are things found on an entirely unrelated server, for other projects that I did work on > in the past. > > The question then is why is the server explorer choosing to display objects out on a virtual machine > instead of the database that was originally selected for this solution? > > We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound > form stuff. > > I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values > in most cases) the developer, the product, the type of work, then a couple of fields for date time > values for start / stop work. The table the data is going into has a few thousand records already > in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound > forms is a natural for this. > > A long time ago I made this (kind of thing) work. I remember that getting the properties of the > combo set correctly was non-intuitive but once I figured it out it just worked. > > > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 3:02 PM, Gustav Brock wrote: >> Hi John >> >> If you (typically) have this in InitializeComponent() of the .Designer.cs file: >> >> // comboBoxCountry >> // >> this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; >> this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; >> this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; >> this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); >> this.comboBoxCountry.DisplayMember = "Id"; >> this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; >> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); >> this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; >> this.comboBoxCountry.FormattingEnabled = true; >> this.comboBoxCountry.ItemHeight = 13; >> this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); >> this.comboBoxCountry.Name = "comboBoxCountry"; >> this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); >> this.comboBoxCountry.Sorted = true; >> this.comboBoxCountry.TabIndex = 7; >> this.comboBoxCountry.ValueMember = "Id"; >> >> You can do something like this in the Load event: >> >> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); >> >> Then: >> private void FillComboBoxCountryCode() >> { >> // Order the country names of countryDataTable. >> string filterExpression = ""; >> string sortExpression = "CountryNameUK asc"; >> DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); >> >> this.comboBoxCountry.DataSource = countryList; >> this.comboBoxCountry.ValueMember = "Id"; >> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >> } >> >> And: >> private void ChangeCountrySelection() >> { >> if (this.comboBoxCountry.SelectedValue != null) >> { >> _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); >> this.comboBoxPostalCode.Text = String.Empty; >> // Read postal codes of selected country. >> FillComboBoxPostalCode(); >> } >> } >> >> This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >> I am trying to set up a combo box bound to a field in the form such that the combo populates with >> the PKID and a text field, and when the form opens, selecting something from the combo drops the >> PKID into the field in the form. >> >> I am not getting what to set the properties of the combo box to. I am pulling the ID and the string >> to display into the combo but I cannot cause the combo to store the id and release control to the >> next control on the form. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Mon Mar 14 21:13:25 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 22:13:25 -0400 Subject: [dba-VB] C# combo boxes In-Reply-To: References: <4D7E95C4.9020203@colbyconsulting.com> Message-ID: <4D7ECB45.8060609@colbyconsulting.com> I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com On 3/14/2011 7:59 PM, Jim Lawrence wrote: > Hi John: > > Try and use ADO. It is on every Windows based computer already and requires > no additional ODBC to be setup. In other words, you do not even have to see > the computers on which the application is going to have a sable connection. > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Monday, March 14, 2011 3:25 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] C# combo boxes > > I am trying (unsuccessfully) to do an "Access like bound form". > > And I always comment. ;) > > ATM the form displays the values if dropped down, but if not dropped down it > just displays an > integer (the PKID of the data in the combo is my guess, or perhaps the PKID > in the field in the form. > > So... > > How do I cause the combo to display Paul's name instead of 6. > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 6:15 PM, Gustav Brock wrote: >> Hi John >> >> Eh, are you commenting or? Not quite clear to me. I thought you were > looking for some code samples ... >> >> I have connections too but I use them only to feed the datatableadapters. > I refrain from working with SqlCommand and the like; waste of time in my > opinion. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> >> Thanks Gustav, >> I don't know why you are taking me into code in code. >> >> I have data connections. This particular data connection has a single > view that allows me to write >> into a table. It has a view that filters a combo. >> >> I drag and drop the view for the table onto the form. I have told that > table object that specific >> fields are combos so when I drag it out, I have combos (for the FKs - PKs > from other tables). I am >> trying to set up that combo to >> >> 1) Display a text value >> 2) Contain the PKID from the table the combo draws from >> 3) When I select an item, place the selected ID into the field the combo > is bound to. >> >> It seems this is a normal daily requirement in building forms, and > shouldn't be rocket science (or >> going into code). >> >> Actually I have a kinda sorta related question. >> >> When I look at server explorer I see a data connection object, however > there are two objects >> underneath, neither of which are for this solution. Furthermore the > objects under the data >> connection are things found on an entirely unrelated server, for other > projects that I did work on >> in the past. >> >> The question then is why is the server explorer choosing to display > objects out on a virtual machine >> instead of the database that was originally selected for this solution? >> >> We have done a ton of coding sql server stuff here but we have not yet > learned how to do this bound >> form stuff. >> >> I am trying to build a timeclock kind of thing. Very simple, a few combos > to select (default values >> in most cases) the developer, the product, the type of work, then a couple > of fields for date time >> values for start / stop work. The table the data is going into has a few > thousand records already >> in it, the combos pull from tables with 20-30 records etc. It seems (to > me) that just using bound >> forms is a natural for this. >> >> A long time ago I made this (kind of thing) work. I remember that getting > the properties of the >> combo set correctly was non-intuitive but once I figured it out it just > worked. >> >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/14/2011 3:02 PM, Gustav Brock wrote: >>> Hi John >>> >>> If you (typically) have this in InitializeComponent() of the .Designer.cs > file: >>> >>> // comboBoxCountry >>> // >>> this.comboBoxCountry.AutoCompleteMode = > System.Windows.Forms.AutoCompleteMode.Suggest; >>> this.comboBoxCountry.AutoCompleteSource = > System.Windows.Forms.AutoCompleteSource.ListItems; >>> this.comboBoxCountry.BackColor = > System.Drawing.Color.LavenderBlush; >>> this.comboBoxCountry.DataBindings.Add(new > System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, > "CountryId", true)); >>> this.comboBoxCountry.DisplayMember = "Id"; >>> this.comboBoxCountry.FlatStyle = > System.Windows.Forms.FlatStyle.Flat; >>> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe > UI", 8F, System.Drawing.FontStyle.Regular, > System.Drawing.GraphicsUnit.Point, ((byte)(0))); >>> this.comboBoxCountry.ForeColor = > System.Drawing.Color.DarkMagenta; >>> this.comboBoxCountry.FormattingEnabled = true; >>> this.comboBoxCountry.ItemHeight = 13; >>> this.comboBoxCountry.Location = new > System.Drawing.Point(21, 136); >>> this.comboBoxCountry.Name = "comboBoxCountry"; >>> this.comboBoxCountry.Size = new System.Drawing.Size(117, > 21); >>> this.comboBoxCountry.Sorted = true; >>> this.comboBoxCountry.TabIndex = 7; >>> this.comboBoxCountry.ValueMember = "Id"; >>> >>> You can do something like this in the Load event: >>> >>> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => > ChangeCountrySelection(); >>> >>> Then: >>> private void FillComboBoxCountryCode() >>> { >>> // Order the country names of countryDataTable. >>> string filterExpression = ""; >>> string sortExpression = "CountryNameUK asc"; >>> DataRow[] countryList = > _countryDataTable.Select(filterExpression, sortExpression); >>> >>> this.comboBoxCountry.DataSource = countryList; >>> this.comboBoxCountry.ValueMember = "Id"; >>> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >>> } >>> >>> And: >>> private void ChangeCountrySelection() >>> { >>> if (this.comboBoxCountry.SelectedValue != null) >>> { >>> _countryId = > Convert.ToInt32(this.comboBoxCountry.SelectedValue); >>> this.comboBoxPostalCode.Text = String.Empty; >>> // Read postal codes of selected country. >>> FillComboBoxPostalCode(); >>> } >>> } >>> >>> This is pretty standard using tableadapters and bindingnavigator. Nothing > fancy. >>> >>> /gustav >>> >>> >>>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >>> I am trying to set up a combo box bound to a field in the form such that > the combo populates with >>> the PKID and a text field, and when the form opens, selecting something > from the combo drops the >>> PKID into the field in the form. >>> >>> I am not getting what to set the properties of the combo box to. I am > pulling the ID and the string >>> to display into the combo but I cannot cause the combo to store the id > and release control to the >>> next control on the form. >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From accessd at shaw.ca Tue Mar 15 02:19:11 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Tue, 15 Mar 2011 00:19:11 -0700 Subject: [dba-VB] C# combo boxes In-Reply-To: <4D7ECB45.8060609@colbyconsulting.com> References: <4D7E95C4.9020203@colbyconsulting.com> <4D7ECB45.8060609@colbyconsulting.com> Message-ID: Congratulation John. Keep us posted on it stability. Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, March 14, 2011 7:13 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# combo boxes I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com On 3/14/2011 7:59 PM, Jim Lawrence wrote: > Hi John: > > Try and use ADO. It is on every Windows based computer already and requires > no additional ODBC to be setup. In other words, you do not even have to see > the computers on which the application is going to have a sable connection. > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Monday, March 14, 2011 3:25 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] C# combo boxes > > I am trying (unsuccessfully) to do an "Access like bound form". > > And I always comment. ;) > > ATM the form displays the values if dropped down, but if not dropped down it > just displays an > integer (the PKID of the data in the combo is my guess, or perhaps the PKID > in the field in the form. > > So... > > How do I cause the combo to display Paul's name instead of 6. > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 6:15 PM, Gustav Brock wrote: >> Hi John >> >> Eh, are you commenting or? Not quite clear to me. I thought you were > looking for some code samples ... >> >> I have connections too but I use them only to feed the datatableadapters. > I refrain from working with SqlCommand and the like; waste of time in my > opinion. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> >> Thanks Gustav, >> I don't know why you are taking me into code in code. >> >> I have data connections. This particular data connection has a single > view that allows me to write >> into a table. It has a view that filters a combo. >> >> I drag and drop the view for the table onto the form. I have told that > table object that specific >> fields are combos so when I drag it out, I have combos (for the FKs - PKs > from other tables). I am >> trying to set up that combo to >> >> 1) Display a text value >> 2) Contain the PKID from the table the combo draws from >> 3) When I select an item, place the selected ID into the field the combo > is bound to. >> >> It seems this is a normal daily requirement in building forms, and > shouldn't be rocket science (or >> going into code). >> >> Actually I have a kinda sorta related question. >> >> When I look at server explorer I see a data connection object, however > there are two objects >> underneath, neither of which are for this solution. Furthermore the > objects under the data >> connection are things found on an entirely unrelated server, for other > projects that I did work on >> in the past. >> >> The question then is why is the server explorer choosing to display > objects out on a virtual machine >> instead of the database that was originally selected for this solution? >> >> We have done a ton of coding sql server stuff here but we have not yet > learned how to do this bound >> form stuff. >> >> I am trying to build a timeclock kind of thing. Very simple, a few combos > to select (default values >> in most cases) the developer, the product, the type of work, then a couple > of fields for date time >> values for start / stop work. The table the data is going into has a few > thousand records already >> in it, the combos pull from tables with 20-30 records etc. It seems (to > me) that just using bound >> forms is a natural for this. >> >> A long time ago I made this (kind of thing) work. I remember that getting > the properties of the >> combo set correctly was non-intuitive but once I figured it out it just > worked. >> >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/14/2011 3:02 PM, Gustav Brock wrote: >>> Hi John >>> >>> If you (typically) have this in InitializeComponent() of the .Designer.cs > file: >>> >>> // comboBoxCountry >>> // >>> this.comboBoxCountry.AutoCompleteMode = > System.Windows.Forms.AutoCompleteMode.Suggest; >>> this.comboBoxCountry.AutoCompleteSource = > System.Windows.Forms.AutoCompleteSource.ListItems; >>> this.comboBoxCountry.BackColor = > System.Drawing.Color.LavenderBlush; >>> this.comboBoxCountry.DataBindings.Add(new > System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, > "CountryId", true)); >>> this.comboBoxCountry.DisplayMember = "Id"; >>> this.comboBoxCountry.FlatStyle = > System.Windows.Forms.FlatStyle.Flat; >>> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe > UI", 8F, System.Drawing.FontStyle.Regular, > System.Drawing.GraphicsUnit.Point, ((byte)(0))); >>> this.comboBoxCountry.ForeColor = > System.Drawing.Color.DarkMagenta; >>> this.comboBoxCountry.FormattingEnabled = true; >>> this.comboBoxCountry.ItemHeight = 13; >>> this.comboBoxCountry.Location = new > System.Drawing.Point(21, 136); >>> this.comboBoxCountry.Name = "comboBoxCountry"; >>> this.comboBoxCountry.Size = new System.Drawing.Size(117, > 21); >>> this.comboBoxCountry.Sorted = true; >>> this.comboBoxCountry.TabIndex = 7; >>> this.comboBoxCountry.ValueMember = "Id"; >>> >>> You can do something like this in the Load event: >>> >>> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => > ChangeCountrySelection(); >>> >>> Then: >>> private void FillComboBoxCountryCode() >>> { >>> // Order the country names of countryDataTable. >>> string filterExpression = ""; >>> string sortExpression = "CountryNameUK asc"; >>> DataRow[] countryList = > _countryDataTable.Select(filterExpression, sortExpression); >>> >>> this.comboBoxCountry.DataSource = countryList; >>> this.comboBoxCountry.ValueMember = "Id"; >>> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >>> } >>> >>> And: >>> private void ChangeCountrySelection() >>> { >>> if (this.comboBoxCountry.SelectedValue != null) >>> { >>> _countryId = > Convert.ToInt32(this.comboBoxCountry.SelectedValue); >>> this.comboBoxPostalCode.Text = String.Empty; >>> // Read postal codes of selected country. >>> FillComboBoxPostalCode(); >>> } >>> } >>> >>> This is pretty standard using tableadapters and bindingnavigator. Nothing > fancy. >>> >>> /gustav >>> >>> >>>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >>> I am trying to set up a combo box bound to a field in the form such that > the combo populates with >>> the PKID and a text field, and when the form opens, selecting something > from the combo drops the >>> PKID into the field in the form. >>> >>> I am not getting what to set the properties of the combo box to. I am > pulling the ID and the string >>> to display into the combo but I cannot cause the combo to store the id > and release control to the >>> next control on the form. >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From Gustav at cactus.dk Tue Mar 15 04:55:02 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Tue, 15 Mar 2011 10:55:02 +0100 Subject: [dba-VB] C# combo boxes Message-ID: Hi John Good! Yes, the content of Text is handled by DisplayMember: this.comboBoxCountry.DataSource = countryList; this.comboBoxCountry.ValueMember = "Id"; this.comboBoxCountry.DisplayMember = "CountryNameUK"; /gustav >>> jwcolby at colbyconsulting.com 15-03-2011 03:13 >>> I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com From df.waters at comcast.net Tue Mar 15 08:54:00 2011 From: df.waters at comcast.net (Dan Waters) Date: Tue, 15 Mar 2011 08:54:00 -0500 Subject: [dba-VB] C# combo boxes In-Reply-To: <4D7ECB45.8060609@colbyconsulting.com> References: <4D7E95C4.9020203@colbyconsulting.com> <4D7ECB45.8060609@colbyconsulting.com> Message-ID: <002101cbe318$713acbb0$53b06310$@comcast.net> Hi John, Take a look at www.updatecontrols.net. This has a free download which integrates into visual studio and provides a way to simply databinding. I haven't used it yet but it looks promising. Note: At the top of the screens is a combobox which selects either VB or C# for the code examples. Dan -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, March 14, 2011 9:13 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# combo boxes I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com On 3/14/2011 7:59 PM, Jim Lawrence wrote: > Hi John: > > Try and use ADO. It is on every Windows based computer already and > requires no additional ODBC to be setup. In other words, you do not > even have to see the computers on which the application is going to have a sable connection. > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Monday, March 14, 2011 3:25 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] C# combo boxes > > I am trying (unsuccessfully) to do an "Access like bound form". > > And I always comment. ;) > > ATM the form displays the values if dropped down, but if not dropped > down it just displays an integer (the PKID of the data in the combo is > my guess, or perhaps the PKID in the field in the form. > > So... > > How do I cause the combo to display Paul's name instead of 6. > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 6:15 PM, Gustav Brock wrote: >> Hi John >> >> Eh, are you commenting or? Not quite clear to me. I thought you were > looking for some code samples ... >> >> I have connections too but I use them only to feed the datatableadapters. > I refrain from working with SqlCommand and the like; waste of time in > my opinion. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> >> Thanks Gustav, >> I don't know why you are taking me into code in code. >> >> I have data connections. This particular data connection has a >> single > view that allows me to write >> into a table. It has a view that filters a combo. >> >> I drag and drop the view for the table onto the form. I have told >> that > table object that specific >> fields are combos so when I drag it out, I have combos (for the FKs - >> PKs > from other tables). I am >> trying to set up that combo to >> >> 1) Display a text value >> 2) Contain the PKID from the table the combo draws from >> 3) When I select an item, place the selected ID into the field the >> combo > is bound to. >> >> It seems this is a normal daily requirement in building forms, and > shouldn't be rocket science (or >> going into code). >> >> Actually I have a kinda sorta related question. >> >> When I look at server explorer I see a data connection object, >> however > there are two objects >> underneath, neither of which are for this solution. Furthermore the > objects under the data >> connection are things found on an entirely unrelated server, for >> other > projects that I did work on >> in the past. >> >> The question then is why is the server explorer choosing to display > objects out on a virtual machine >> instead of the database that was originally selected for this solution? >> >> We have done a ton of coding sql server stuff here but we have not >> yet > learned how to do this bound >> form stuff. >> >> I am trying to build a timeclock kind of thing. Very simple, a few >> combos > to select (default values >> in most cases) the developer, the product, the type of work, then a >> couple > of fields for date time >> values for start / stop work. The table the data is going into has a >> few > thousand records already >> in it, the combos pull from tables with 20-30 records etc. It seems >> (to > me) that just using bound >> forms is a natural for this. >> >> A long time ago I made this (kind of thing) work. I remember that >> getting > the properties of the >> combo set correctly was non-intuitive but once I figured it out it >> just > worked. >> >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/14/2011 3:02 PM, Gustav Brock wrote: >>> Hi John >>> >>> If you (typically) have this in InitializeComponent() of the >>> .Designer.cs > file: >>> >>> // comboBoxCountry >>> // >>> this.comboBoxCountry.AutoCompleteMode = > System.Windows.Forms.AutoCompleteMode.Suggest; >>> this.comboBoxCountry.AutoCompleteSource = > System.Windows.Forms.AutoCompleteSource.ListItems; >>> this.comboBoxCountry.BackColor = > System.Drawing.Color.LavenderBlush; >>> this.comboBoxCountry.DataBindings.Add(new > System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, > "CountryId", true)); >>> this.comboBoxCountry.DisplayMember = "Id"; >>> this.comboBoxCountry.FlatStyle = > System.Windows.Forms.FlatStyle.Flat; >>> this.comboBoxCountry.Font = new >>> System.Drawing.Font("Segoe > UI", 8F, System.Drawing.FontStyle.Regular, > System.Drawing.GraphicsUnit.Point, ((byte)(0))); >>> this.comboBoxCountry.ForeColor = > System.Drawing.Color.DarkMagenta; >>> this.comboBoxCountry.FormattingEnabled = true; >>> this.comboBoxCountry.ItemHeight = 13; >>> this.comboBoxCountry.Location = new > System.Drawing.Point(21, 136); >>> this.comboBoxCountry.Name = "comboBoxCountry"; >>> this.comboBoxCountry.Size = new >>> System.Drawing.Size(117, > 21); >>> this.comboBoxCountry.Sorted = true; >>> this.comboBoxCountry.TabIndex = 7; >>> this.comboBoxCountry.ValueMember = "Id"; >>> >>> You can do something like this in the Load event: >>> >>> this.comboBoxCountry.SelectedValueChanged += (oo, ee) >>> => > ChangeCountrySelection(); >>> >>> Then: >>> private void FillComboBoxCountryCode() >>> { >>> // Order the country names of countryDataTable. >>> string filterExpression = ""; >>> string sortExpression = "CountryNameUK asc"; >>> DataRow[] countryList = > _countryDataTable.Select(filterExpression, sortExpression); >>> >>> this.comboBoxCountry.DataSource = countryList; >>> this.comboBoxCountry.ValueMember = "Id"; >>> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >>> } >>> >>> And: >>> private void ChangeCountrySelection() >>> { >>> if (this.comboBoxCountry.SelectedValue != null) >>> { >>> _countryId = > Convert.ToInt32(this.comboBoxCountry.SelectedValue); >>> this.comboBoxPostalCode.Text = String.Empty; >>> // Read postal codes of selected country. >>> FillComboBoxPostalCode(); >>> } >>> } >>> >>> This is pretty standard using tableadapters and bindingnavigator. >>> Nothing > fancy. >>> >>> /gustav >>> >>> >>>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >>> I am trying to set up a combo box bound to a field in the form such >>> that > the combo populates with >>> the PKID and a text field, and when the form opens, selecting >>> something > from the combo drops the >>> PKID into the field in the form. >>> >>> I am not getting what to set the properties of the combo box to. I >>> am > pulling the ID and the string >>> to display into the combo but I cannot cause the combo to store the >>> id > and release control to the >>> next control on the form. >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From dbdoug at gmail.com Wed Mar 16 10:28:18 2011 From: dbdoug at gmail.com (Doug Steele) Date: Wed, 16 Mar 2011 08:28:18 -0700 Subject: [dba-VB] VS2010 install Message-ID: Hello all: I just finished doing a clean install of Windows 7 then VS2010. To my surprise, I also appear to have a functioning install of VS2008, loading with the name 'Shell' where my old VS2008 says 'Professional Edition'. I couldn't see anything on the net about this; does anyone have any further information? Thanks, Doug From DWUTKA at Marlow.com Fri Mar 18 14:12:52 2011 From: DWUTKA at Marlow.com (Drew Wutka) Date: Fri, 18 Mar 2011 14:12:52 -0500 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: Message-ID: 500 megs... that's about 10 minutes... wouldn't have time for a full dinner.... Drew -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-810 7-475d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com The information contained in this transmission is intended only for the person or entity to which it is addressed and may contain II-VI Proprietary and/or II-VI Business Sensitive material. If you are not the intended recipient, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. You are notified that any review, retransmission, copying, disclosure, dissemination, or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. From stuart at lexacorp.com.pg Fri Mar 18 18:50:01 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 09:50:01 +1000 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: , Message-ID: <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg> That's about 10 hours and US$35 for me :-( -- Stuart On 18 Mar 2011 at 14:12, Drew Wutka wrote: > 500 megs... that's about 10 minutes... wouldn't have time for a full > dinner.... > > Drew > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com > Subject: [dba-VB] Visual Studio 2010 SP1 > > Hi all > > If you go for this: > http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8 > 10 7-475d-948a-ef22627e57a5&displaylang=en > > it's typically a 500MB+ download with a streaming install. > > Don't go for coffee during this, don't go for lunch, not even a > single-course dinner, don't settle for less than a full menu dinner > followed by coffee and brandy. Or book an evening in town! > > /gustav > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > The information contained in this transmission is intended only for > the person or entity to which it is addressed and may contain II-VI > Proprietary and/or II-VI Business Sensitive material. If you are not > the intended recipient, please contact the sender immediately and > destroy the material in its entirety, whether electronic or hard copy. > You are notified that any review, retransmission, copying, disclosure, > dissemination, or other use of, or taking of any action in reliance > upon this information by persons or entities other than the intended > recipient is prohibited. > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Mar 18 19:02:30 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 03:02:30 +0300 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg> References: , <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg> Message-ID: Hi Stuart -- <<< That's about 10 hours and US$35 for me :-( >>> But you never have a snow there, and your local police doesn't mind if you drive home your car after a couple of beers? :) I mean 500megs download takes just a couple of minutes here and near to zero expences but we still have a snow here and Finnish Gulf is covered by 1 meter+ wide ice and I'm skiing here almost everyday, and if stopped by local road police after a couple beers I would definitely loose my car driving license for 1.5 years, at least... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart McLachlan Sent: 19 ????? 2011 ?. 2:50 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Visual Studio 2010 SP1 That's about 10 hours and US$35 for me :-( -- Stuart On 18 Mar 2011 at 14:12, Drew Wutka wrote: > 500 megs... that's about 10 minutes... wouldn't have time for a full > dinner.... > > Drew > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com > Subject: [dba-VB] Visual Studio 2010 SP1 > > Hi all > > If you go for this: > http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8 > 10 7-475d-948a-ef22627e57a5&displaylang=en > > it's typically a 500MB+ download with a streaming install. > > Don't go for coffee during this, don't go for lunch, not even a > single-course dinner, don't settle for less than a full menu dinner > followed by coffee and brandy. Or book an evening in town! > > /gustav > > From stuart at lexacorp.com.pg Fri Mar 18 19:09:18 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 10:09:18 +1000 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: , <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg>, Message-ID: <4D83F42E.13556.8558BC@stuart.lexacorp.com.pg> Yep, The good points still outweigh the bad. :-) -- Stuart On 19 Mar 2011 at 3:02, Shamil Salakhetdinov wrote: > Hi Stuart -- > > <<< > That's about 10 hours and US$35 for me :-( > >>> > But you never have a snow there, and your local police doesn't mind if > you drive home your car after a couple of beers? :) > > I mean 500megs download takes just a couple of minutes here and near > to zero expences but we still have a snow here and Finnish Gulf is > covered by 1 meter+ wide ice and I'm skiing here almost everyday, and > if stopped by local road police after a couple beers I would > definitely loose my car driving license for 1.5 years, at least... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart > McLachlan Sent: 19 ????? 2011 ?. 2:50 To: Discussion concerning Visual > Basic and related programming issues. Subject: Re: [dba-VB] Visual > Studio 2010 SP1 > > That's about 10 hours and US$35 for me :-( > > -- > Stuart > > On 18 Mar 2011 at 14:12, Drew Wutka wrote: > > > 500 megs... that's about 10 minutes... wouldn't have time for a full > > dinner.... > > > > Drew > > > > -----Original Message----- > > From: dba-vb-bounces at databaseadvisors.com > > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav > > Brock Sent: Friday, March 11, 2011 11:59 AM To: > > dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 > > > > Hi all > > > > If you go for this: > > http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6 > > -8 10 7-475d-948a-ef22627e57a5&displaylang=en > > > > it's typically a 500MB+ download with a streaming install. > > > > Don't go for coffee during this, don't go for lunch, not even a > > single-course dinner, don't settle for less than a full menu dinner > > followed by coffee and brandy. Or book an evening in town! > > > > /gustav > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Mar 18 21:08:01 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 18 Mar 2011 22:08:01 -0400 Subject: [dba-VB] SHA1 to compute a hash Message-ID: <4D841001.2060200@colbyconsulting.com> In my databases I create SHA1 hashes to enable joining between tables and pull identical records (identical for the fields hashed). I create: 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the three values and feed them into SHa1 and out pops a number which I store in a field in my table. 2) A HashFamily of the Zip5, Zip4, Addr ad LName. 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. I am getting known collisions between different addresses (I have discovered and investigated collisions) in my HashAddr when I have many millions of addresses. I need to address this. Back when I made my design decisions (2004) my hardware consisted of single core processors, 4 gigs ram, Windows x32 etc. Now I have 8 cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great extent constrained by my hardware "back in the day" whereas I am much less so now. I am about to redesign my process. I am considering simply appending in the city and state strings to all of the inputs: Addr, City, St, Zip5, Zip4 as the address base and then the same with LName and FName for the other two respective hashes. The objective is to minimize hash collisions, not prevent some crypto attack. I use these hash fields to join between multi million record tables so If I need to discover info in TableA where the HashAddr is the same as in TableB, I need the probability of a collision between different addresses (family/Person) to be as close to zero as I can get it. My questions are: 1) Whether anyone out there is using a hash in this manner? 2) Has anyone seen a table of collision probability between messages of a given (short) message length. My message is 9 digits for the zip5/4 and the address could be something as short as PO Box 1, or Apt 1. IOW the total message length of 14 is pretty common. Adding the state would give me minimum message lengths of only 16 and City would only add a few more characters. 3) Does anyone know if just adding the same data back in again would decrease the collision probability. IOW Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. Any experience out there? -- John W. Colby www.ColbyConsulting.com From stuart at lexacorp.com.pg Sat Mar 19 05:20:16 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 20:20:16 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D841001.2060200@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com> Message-ID: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> How are you creating your hash? Can you post a few examples of different data strings and colliding SHA1 hashes. I can probably make a lot of money out of them. AFAIK, no one other than you has found any. -- Stuart On 18 Mar 2011 at 22:08, jwcolby wrote: > In my databases I create SHA1 hashes to enable joining between tables > and pull identical records (identical for the fields hashed). I > create: > > 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the > three values and feed them into SHa1 and out pops a number which I > store in a field in my table. > > 2) A HashFamily of the Zip5, Zip4, Addr ad LName. > > 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. > > I am getting known collisions between different addresses (I have > discovered and investigated collisions) in my HashAddr when I have > many millions of addresses. I need to address this. > > Back when I made my design decisions (2004) my hardware consisted of > single core processors, 4 gigs ram, Windows x32 etc. Now I have 8 > cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great extent > constrained by my hardware "back in the day" whereas I am much less so > now. > > I am about to redesign my process. > > I am considering simply appending in the city and state strings to all > of the inputs: Addr, City, St, Zip5, Zip4 as the address base and then > the same with LName and FName for the other two respective hashes. > > The objective is to minimize hash collisions, not prevent some crypto > attack. I use these hash fields to join between multi million record > tables so If I need to discover info in TableA where the HashAddr is > the same as in TableB, I need the probability of a collision between > different addresses (family/Person) to be as close to zero as I can > get it. > > My questions are: > > 1) Whether anyone out there is using a hash in this manner? > 2) Has anyone seen a table of collision probability between messages > of a given (short) message length. My message is 9 digits for the > zip5/4 and the address could be something as short as PO Box 1, or Apt > 1. IOW the total message length of 14 is pretty common. Adding the > state would give me minimum message lengths of only 16 and City would > only add a few more characters. 3) Does anyone know if just adding the > same data back in again would decrease the collision probability. IOW > Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. > > Any experience out there? > > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 06:20:26 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 21:20:26 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> Using your SHA1 function, what message digests do you get for the standard test cases: 1. abc 2. abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq These should return: 1. A9993E36 4706816A BA3E2571 7850C26C 9CD0D89D 2. 84983E44 1C3BD26E BAAE4AA1 F95129E5 E54670F1 See Examples 1 and 2 at http://www.itl.nist.gov/fipspubs/fip180-1.htm -- Stuart On 19 Mar 2011 at 20:20, Stuart McLachlan wrote: > How are you creating your hash? > > Can you post a few examples of different data strings and colliding > SHA1 hashes. I can probably make a lot of money out of them. > AFAIK, no one other than you has found any. > > -- > Stuart > > > On 18 Mar 2011 at 22:08, jwcolby wrote: > > > In my databases I create SHA1 hashes to enable joining between > > tables and pull identical records (identical for the fields hashed). > > I create: > > > > 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the > > three values and feed them into SHa1 and out pops a number which I > > store in a field in my table. > > > > 2) A HashFamily of the Zip5, Zip4, Addr ad LName. > > > > 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. > > > > I am getting known collisions between different addresses (I have > > discovered and investigated collisions) in my HashAddr when I have > > many millions of addresses. I need to address this. > > > > Back when I made my design decisions (2004) my hardware consisted of > > single core processors, 4 gigs ram, Windows x32 etc. Now I have 8 > > cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great extent > > constrained by my hardware "back in the day" whereas I am much less > > so now. > > > > I am about to redesign my process. > > > > I am considering simply appending in the city and state strings to > > all of the inputs: Addr, City, St, Zip5, Zip4 as the address base > > and then the same with LName and FName for the other two respective > > hashes. > > > > The objective is to minimize hash collisions, not prevent some > > crypto attack. I use these hash fields to join between multi > > million record tables so If I need to discover info in TableA where > > the HashAddr is the same as in TableB, I need the probability of a > > collision between different addresses (family/Person) to be as close > > to zero as I can get it. > > > > My questions are: > > > > 1) Whether anyone out there is using a hash in this manner? > > 2) Has anyone seen a table of collision probability between messages > > of a given (short) message length. My message is 9 digits for the > > zip5/4 and the address could be something as short as PO Box 1, or > > Apt 1. IOW the total message length of 14 is pretty common. Adding > > the state would give me minimum message lengths of only 16 and City > > would only add a few more characters. 3) Does anyone know if just > > adding the same data back in again would decrease the collision > > probability. IOW Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. > > > > Any experience out there? > > > > > > -- > > John W. Colby > > www.ColbyConsulting.com > > _______________________________________________ > > dba-VB mailing list > > dba-VB at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > http://www.databaseadvisors.com > > > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 07:06:47 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 22:06:47 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> Message-ID: <4D849C57.3867.31639E4@stuart.lexacorp.com.pg> That got my interest piqued. So I've just built a small DLL (13KB) to calculate SHA1 digests for either strings or files. All you need to use it is to put the DLL somewhere in your PATH such as %SYSTEMROOT\System32 and include a couple of declarations in your application. Declare Function SHA1StringDigest Lib "LXSHA1.DLL" Alias "SHA1STRINGDIGEST" (Message As String) As String Declare Function SHA1FileDigest Lib "LXSHA1.DLL" Alias "SHA1FILEDIGEST" (FileName As String) As String It correctly returns the test results below for either the string or a file containing the same data. If anyone wants a copy, let me know. -- Stuart On 19 Mar 2011 at 21:20, Stuart McLachlan wrote: > Using your SHA1 function, what message digests do you get for the > standard test cases: > > 1. abc > 2. abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq > > These should return: > > 1. A9993E36 4706816A BA3E2571 7850C26C 9CD0D89D > 2. 84983E44 1C3BD26E BAAE4AA1 F95129E5 E54670F1 > > See Examples 1 and 2 at > http://www.itl.nist.gov/fipspubs/fip180-1.htm > > > > -- > Stuart > > > On 19 Mar 2011 at 20:20, Stuart McLachlan wrote: > > > How are you creating your hash? > > > > Can you post a few examples of different data strings and colliding > > SHA1 hashes. I can probably make a lot of money out of them. > > AFAIK, no one other than you has found any. > > > > -- > > Stuart > > > > > > On 18 Mar 2011 at 22:08, jwcolby wrote: > > > > > In my databases I create SHA1 hashes to enable joining between > > > tables and pull identical records (identical for the fields > > > hashed). > > > I create: > > > > > > 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the > > > three values and feed them into SHa1 and out pops a number which I > > > store in a field in my table. > > > > > > 2) A HashFamily of the Zip5, Zip4, Addr ad LName. > > > > > > 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. > > > > > > I am getting known collisions between different addresses (I have > > > discovered and investigated collisions) in my HashAddr when I have > > > many millions of addresses. I need to address this. > > > > > > Back when I made my design decisions (2004) my hardware consisted > > > of single core processors, 4 gigs ram, Windows x32 etc. Now I > > > have 8 cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great > > > extent constrained by my hardware "back in the day" whereas I am > > > much less so now. > > > > > > I am about to redesign my process. > > > > > > I am considering simply appending in the city and state strings to > > > all of the inputs: Addr, City, St, Zip5, Zip4 as the address base > > > and then the same with LName and FName for the other two > > > respective hashes. > > > > > > The objective is to minimize hash collisions, not prevent some > > > crypto attack. I use these hash fields to join between multi > > > million record tables so If I need to discover info in TableA > > > where the HashAddr is the same as in TableB, I need the > > > probability of a collision between different addresses > > > (family/Person) to be as close to zero as I can get it. > > > > > > My questions are: > > > > > > 1) Whether anyone out there is using a hash in this manner? > > > 2) Has anyone seen a table of collision probability between > > > messages of a given (short) message length. My message is 9 > > > digits for the zip5/4 and the address could be something as short > > > as PO Box 1, or Apt 1. IOW the total message length of 14 is > > > pretty common. Adding the state would give me minimum message > > > lengths of only 16 and City would only add a few more characters. > > > 3) Does anyone know if just adding the same data back in again > > > would decrease the collision probability. IOW > > > Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. > > > > > > Any experience out there? > > > > > > > > > -- > > > John W. Colby > > > www.ColbyConsulting.com > > > _______________________________________________ > > > dba-VB mailing list > > > dba-VB at databaseadvisors.com > > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > > http://www.databaseadvisors.com > > > > > > > > > > > > > > _______________________________________________ > > dba-VB mailing list > > dba-VB at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > http://www.databaseadvisors.com > > > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 07:09:56 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 08:09:56 -0400 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com> <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: <4D849D14.70906@colbyconsulting.com> >AFAIK, no one other than you has found any. LOL. You are thinking of hashing messages with thousands of characters. here's the deal. Let's assume that I hash X characters in a single field. The message length is the key, not the number of fields. Assume also that I am really only hashing letters, numbers and special characters - the characters that are in a name or address. If I hash 100 1 character messages, my probability of a collision if extremely high since there are only about 128 such alphanumeric characters (even less). Now hash a thousand 1 character messages. The probably just climbed immensely. Now hash a million such 1 character messages. What I am really calculating is the probability that I will repeat the same character string. However if I up the number of characters, let's say 10 character messages. *Now* the hash is really trying to prevent the same output for two *different* inputs. Well that is always what a hash is trying to do. But the point is that the probability of a collision decreases with message length increase. The probability of a collision between any two messages of 10 random characters is much higher than the probability of a collision between any two 100 character messages, and the probability of a collision between any two 1000 character messages is much higher still. That is just the way hash functions work. Nobody claims that hashes don't create collisions, and everyone agrees that the longer the message, the lower the probability of a collision between two messages (to a point). So, my address "messages" currently look like 89364 4456 PO Box 1 76543 9876 Apt 2 97867 3546 1723 Twin Pines Dr The point here is that all address "messages" are short. Now turn the address message into 89364 4456 PO Box 1 CA San Diego 76543 9876 Apt 2 TX Dallas 97867 3546 1723 Twin Pines Dr NC Hudson And the length of the message increases fairly dramatically, decreasing the probability of a collision. Remember too that I am hashing hundreds of millions of records - about 350 million addresses in various tables so far. The other thing that affects the probability of a hash collision is the number of messages hashed. Hash enough records and you *will* 100% probability create a hash that is the same for two different input strings - even for long messages. That is just the nature of the business. The probability of a collision is determined by three things. 1) The (strength of the) hash algorithm. 2) The message length 3) The quantity of messages. 1) I am using a fairly strong hash algorithm - SHA1 2) I am using a fairly short hash message - Zip5 / zip4 / Addr 3) I am hashing a fairly high number of messages - 350 million (and counting) messages (addresses) Believe me, hashes have already occurred, I have seen them. It is an easy thing to test. Join two tables on the same hash and in the query compare that each address field is the same between the two tables. Zip5 <> zip5 *or* zip4 <> zip4 *or* addr <> addr. If the hash is the same (inner join) AND one of the three fields is different then you have a collision. I have done this test and I have found collisions between my address hash. I have never tested my last name or first name hash fields. The problem with finding hashes is that they are rare, and in order to do an exhaustive test I would need to pull all the hash and data fields into a single table and compare that table against itself. My data is contained in a bunch of databases. 11 million "dogs and cats", 21 million "kids", 7 million "smokers", 23 million (and about to double) emails. 65 million "all adults", 100 million "mortgage" etc. So it is difficult to find hash collisions because (at least in the small tables) the probability of a collision is low entirely within a table. Believe me, it *does* happen though (at least for the address hash). John W. Colby www.ColbyConsulting.com On 3/19/2011 6:20 AM, Stuart McLachlan wrote: > How are you creating your hash? > > Can you post a few examples of different data strings and colliding SHA1 hashes. I can > probably make a lot of money out of them. AFAIK, no one other than you has found any. > From jwcolby at colbyconsulting.com Sat Mar 19 07:11:32 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 08:11:32 -0400 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D849C57.3867.31639E4@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> <4D849C57.3867.31639E4@stuart.lexacorp.com.pg> Message-ID: <4D849D74.2040104@colbyconsulting.com> I will do a collision test in my email file. AFAICT I have some collisions in there. If I find any I will send you the message and hash so that you can see what collisions look like. John W. Colby www.ColbyConsulting.com On 3/19/2011 8:06 AM, Stuart McLachlan wrote: > That got my interest piqued. > > So I've just built a small DLL (13KB) to calculate SHA1 digests for either strings or files. > All you need to use it is to put the DLL somewhere in your PATH such as > %SYSTEMROOT\System32 and include a couple of declarations in your application. > > Declare Function SHA1StringDigest Lib "LXSHA1.DLL" Alias "SHA1STRINGDIGEST" > (Message As String) As String > > Declare Function SHA1FileDigest Lib "LXSHA1.DLL" Alias "SHA1FILEDIGEST" (FileName > As String) As String > > It correctly returns the test results below for either the string or a file containing the same > data. > > If anyone wants a copy, let me know. > From harlan at powerzone.com Sat Mar 19 07:17:25 2011 From: harlan at powerzone.com (Harlan Koehn) Date: Sat, 19 Mar 2011 06:17:25 -0600 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D849D14.70906@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com> <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> <4D849D14.70906@colbyconsulting.com> Message-ID: <76EEB091F7CED543A80B5C2BC153975E0165A7C108@PZSERVER01.powerzone.local> John, I'm fairly ignorant when it comes to hashing algorithms but I'm curious if you have considered using a hashing algorithm that would generate a longer hash, which I'm assuming could reduce the likelihood of a collision. Harlan From stuart at lexacorp.com.pg Sat Mar 19 07:51:13 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 22:51:13 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D849D14.70906@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D849D14.70906@colbyconsulting.com> Message-ID: <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> In the case of 1000 single character messages, you are bound to get collisions since there are only 256 possible original messages. You will be hashing the same value multiple times Apart from that, your understanding is incorrect. It doesn't matter how long the string is. The chances of a collision with two different messages remains the same. The message is hashed using padded blocks of a fixed length. There is no more chance of a collision between "a" and "b" than there is between "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". Specifically, the chance of a collision within n different messages, using b bits of encryption is (n*(n-1)/2) * (1/2^b). Note that the length of the message doesn't come into that equation. The probability of a collision is determined by only TWO things. 1) The length of the digest. 2) The quantity of messages. As for "everyone agrees". Noone who understands how it works, agrees. -- Stuart On 19 Mar 2011 at 8:09, jwcolby wrote: > >AFAIK, no one other than you has found any. > > LOL. > > You are thinking of hashing messages with thousands of characters. > here's the deal. > > Let's assume that I hash X characters in a single field. The message > length is the key, not the number of fields. Assume also that I am > really only hashing letters, numbers and special characters - the > characters that are in a name or address. > > > If I hash 100 1 character messages, my probability of a collision if > extremely high since there are only about 128 such alphanumeric > characters (even less). > > Now hash a thousand 1 character messages. The probably just climbed > immensely. Now hash a million such 1 character messages. > What I am really calculating is the probability that I will repeat the > same character string. > > However if I up the number of characters, let's say 10 character > messages. *Now* the hash is really trying to prevent the same output > for two *different* inputs. Well that is always what a hash is trying > to do. But the point is that the probability of a collision decreases > with message length increase. > > The probability of a collision between any two messages of 10 random > characters is much higher than the probability of a collision between > any two 100 character messages, and the probability of a collision > between any two 1000 character messages is much higher still. > > That is just the way hash functions work. Nobody claims that hashes > don't create collisions, and everyone agrees that the longer the > message, the lower the probability of a collision between two messages > (to a point). > > So, my address "messages" currently look like > > 89364 4456 PO Box 1 > 76543 9876 Apt 2 > 97867 3546 1723 Twin Pines Dr > > The point here is that all address "messages" are short. Now turn the > address message into > > 89364 4456 PO Box 1 CA San Diego > 76543 9876 Apt 2 TX Dallas > 97867 3546 1723 Twin Pines Dr NC Hudson > > And the length of the message increases fairly dramatically, > decreasing the probability of a collision. > > Remember too that I am hashing hundreds of millions of records - about > 350 million addresses in various tables so far. The other thing that > affects the probability of a hash collision is the number of messages > hashed. Hash enough records and you *will* 100% probability create a > hash that is the same for two different input strings - even for long > messages. That is just the nature of the business. > > The probability of a collision is determined by three things. > > 1) The (strength of the) hash algorithm. > 2) The message length > 3) The quantity of messages. ... From shamil at smsconsulting.spb.ru Sat Mar 19 08:03:36 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 16:03:36 +0300 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com> <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: Hi Stuart and John, <<< AFAIK, no one other than you has found any >>> Yes, I was also confused how it comes to get SHA1 hashes collisions but then I have thought that John probably "compacts" SHA1 hashes into 8 bytes surrogate keys? Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions for 6 millions cycles for 8 bytes keys. Please correct me if you find some errors in my sample code Thank you. -- Shamil P.S. Code: public class GenerateHashes { private static Dictionary _generatedHashes; private static Dictionary _generatedLongHashes; public static void Run(int minHashedStringLength, int maxHashedStringLength, long testCasesQty) { System.Console.WriteLine("Started at: " + System.DateTime.Now); _generatedHashes = new Dictionary(); _generatedLongHashes = new Dictionary(); int collisionsQty = 0; int longHashCollisionsQty = 0; decimal totalLengthOfProcessedStrings = 0; decimal averageStringLength = 0; Random random = new Random((int)System.DateTime.Now.Ticks); for (long i = 1; i <= testCasesQty; i++) { int stringLength = random.Next(minHashedStringLength, maxHashedStringLength); byte[] plainTextBytes = (byte[])Array.CreateInstance(typeof(byte), stringLength); random.NextBytes(plainTextBytes); System.Security.Cryptography.HashAlgorithm hash = new SHA1Managed(); byte[] hashBytes = hash.ComputeHash(plainTextBytes); string hashValue = Convert.ToBase64String(hashBytes); string testValue; if (_generatedHashes.TryGetValue(hashValue, out testValue)) collisionsQty++; else _generatedHashes.Add(hashValue, hashValue); if (i % 1000000 == 0) System.Console.WriteLine( "{0}: i={1}, cs={2}, cu={3}", DateTime.Now, i, collisionsQty, longHashCollisionsQty); ulong longHash = 0xFFFFFFFF; foreach (byte b in hashValue) { longHash ^= b; longHash = longHash << 8 + 0xFF; } ulong longHashTestValue; if (_generatedLongHashes.TryGetValue(longHash, out longHashTestValue)) { //Long Hash Collision longHashCollisionsQty++; System.Console.WriteLine(" * step {0}, hash '{1}', longHash: {2:X}", i, hashValue, longHash); } else _generatedLongHashes.Add(longHash, longHash); totalLengthOfProcessedStrings += stringLength; averageStringLength = totalLengthOfProcessedStrings / i; } System.Console.WriteLine( "\nTOTALS:\n" + "Min String Length = {0}\n" + "Max String Length = {1}\n" + "Avg String Length = {2}\n" + "Test cases Qty = {3}\n" + "Collisions Qty = {4}\n" + "ulong Collisions Qty = {5}", minHashedStringLength, maxHashedStringLength, averageStringLength, testCasesQty, collisionsQty, longHashCollisionsQty); System.Console.WriteLine("Finished at: " + System.DateTime.Now); } } P.P.S. Stats Started at: 19/03/2011 15:51:04 19/03/2011 15:51:16: i=1000000, cs=0, cu=0 * step 1712197, hash 'Rt4NY5FHg0uMDszVql+l6ksx6Tg=', longHash: 36D7CFC36A99DE80 19/03/2011 15:51:29: i=2000000, cs=0, cu=1 * step 2040324, hash 'laoPHdqlfeZOx9DI8RDaQZjhKZs=', longHash: D1B5AB44BB5CDE80 * step 2043449, hash '1m+tgKu7/l9fl8nUNuz5I/OWF5E=', longHash: C95F3EBC66B15E80 19/03/2011 15:51:44: i=3000000, cs=0, cu=3 19/03/2011 15:52:00: i=4000000, cs=0, cu=3 * step 4369446, hash '2sZrglXDp3NnNPuqdILqwmUKq30=', longHash: F7DB565F166C1E80 19/03/2011 15:52:17: i=5000000, cs=0, cu=4 TOTALS: Min String Length = 7 Max String Length = 25 Avg String Length = 15.4981354 Test cases Qty = 5,000,000 Collisions Qty = 0 ulong Collisions Qty = 4 Finished at: 19/03/2011 15:52:17 -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart McLachlan Sent: 19 ????? 2011 ?. 13:20 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SHA1 to compute a hash How are you creating your hash? Can you post a few examples of different data strings and colliding SHA1 hashes. I can probably make a lot of money out of them. AFAIK, no one other than you has found any. -- Stuart <<< snip >>> From stuart at lexacorp.com.pg Sat Mar 19 08:10:47 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 23:10:47 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D849D14.70906@colbyconsulting.com>, <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> Message-ID: <4D84AB57.31948.350CF8F@stuart.lexacorp.com.pg> I've just realised where the confusion comes in. When you talk about the chance of collisions increasing decreasing message length, you are talking about general hashing algorithms such as CRC etc. You need to undersand the difference between simple and cryptographic hashing. Cryptographic hashing algorithms are specifically designed to be "collision resistant". I repeat, to date no one has found a collision in SHA1 (except you apparently ) -- Stuart On 19 Mar 2011 at 22:51, Stuart McLachlan wrote: > In the case of 1000 single character messages, you are bound to get > collisions since there are only 256 possible original messages. You > will be hashing the same value multiple times > > Apart from that, your understanding is incorrect. It doesn't matter > how long the string is. The chances of a collision with two different > messages remains the same. > > The message is hashed using padded blocks of a fixed length. > There is no more chance of a collision between "a" and "b" than there > is between "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". > > Specifically, the chance of a collision within n different messages, > using b bits of encryption is (n*(n-1)/2) * (1/2^b). > > Note that the length of the message doesn't come into that equation. > > The probability of a collision is determined by only TWO things. > > 1) The length of the digest. > 2) The quantity of messages. > > As for "everyone agrees". Noone who understands how it works, agrees. > > -- > Stuart > > On 19 Mar 2011 at 8:09, jwcolby wrote: > > > >AFAIK, no one other than you has found any. > > > > LOL. > > > > You are thinking of hashing messages with thousands of characters. > > here's the deal. > > > > Let's assume that I hash X characters in a single field. The > > message length is the key, not the number of fields. Assume also > > that I am really only hashing letters, numbers and special > > characters - the characters that are in a name or address. > > > > > > If I hash 100 1 character messages, my probability of a collision if > > extremely high since there are only about 128 such alphanumeric > > characters (even less). > > > > Now hash a thousand 1 character messages. The probably just climbed > > immensely. Now hash a million such 1 character messages. What I am > > really calculating is the probability that I will repeat the same > > character string. > > > > However if I up the number of characters, let's say 10 character > > messages. *Now* the hash is really trying to prevent the same > > output for two *different* inputs. Well that is always what a hash > > is trying to do. But the point is that the probability of a > > collision decreases with message length increase. > > > > The probability of a collision between any two messages of 10 random > > characters is much higher than the probability of a collision > > between any two 100 character messages, and the probability of a > > collision between any two 1000 character messages is much higher > > still. > > > > That is just the way hash functions work. Nobody claims that hashes > > don't create collisions, and everyone agrees that the longer the > > message, the lower the probability of a collision between two > > messages (to a point). > > > > So, my address "messages" currently look like > > > > 89364 4456 PO Box 1 > > 76543 9876 Apt 2 > > 97867 3546 1723 Twin Pines Dr > > > > The point here is that all address "messages" are short. Now turn > > the address message into > > > > 89364 4456 PO Box 1 CA San Diego > > 76543 9876 Apt 2 TX Dallas > > 97867 3546 1723 Twin Pines Dr NC Hudson > > > > And the length of the message increases fairly dramatically, > > decreasing the probability of a collision. > > > > Remember too that I am hashing hundreds of millions of records - > > about 350 million addresses in various tables so far. The other > > thing that affects the probability of a hash collision is the number > > of messages hashed. Hash enough records and you *will* 100% > > probability create a hash that is the same for two different input > > strings - even for long messages. That is just the nature of the > > business. > > > > The probability of a collision is determined by three things. > > > > 1) The (strength of the) hash algorithm. > > 2) The message length > > 3) The quantity of messages. > ... > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 08:20:11 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 23:20:11 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, Message-ID: <4D84AD8B.14064.3596D15@stuart.lexacorp.com.pg> If he is compacting a 160 bit hash to a 64 bit hash, he is certainly increasing the odds of a collision - but I suspect that even at 64 bits, the chances of a collision would be very low. But that does not appear to be what you are testing. It looks as though you created 6 million random strings and four of them turned out to be identical. Therefore also had the same SHA1 digest. What happens if you try it again, making sure that all of your strings are unique. Alternatively, what do you get if you add some code to show the pairs of strings. Are they different? -- Stuart On 19 Mar 2011 at 16:03, Shamil Salakhetdinov wrote: > Hi Stuart and John, > > <<< > AFAIK, no one other than you has found any > >>> > Yes, I was also confused how it comes to get SHA1 hashes collisions > but then I have thought that John probably "compacts" SHA1 hashes into > 8 bytes surrogate keys? > > Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions > for 6 millions cycles for 8 bytes keys. > > Please correct me if you find some errors in my sample code > > Thank you. > > -- > Shamil > > P.S. Code: > > public class GenerateHashes > { > private static Dictionary _generatedHashes; > private static Dictionary _generatedLongHashes; > > public static void Run(int minHashedStringLength, int > maxHashedStringLength, long testCasesQty) > { > System.Console.WriteLine("Started at: " + > System.DateTime.Now); > > _generatedHashes = new Dictionary(); > _generatedLongHashes = new Dictionary(); > > int collisionsQty = 0; > int longHashCollisionsQty = 0; > > decimal totalLengthOfProcessedStrings = 0; > decimal averageStringLength = 0; > > > Random random = new > Random((int)System.DateTime.Now.Ticks); for (long i = 1; i > <= testCasesQty; i++) { > int stringLength = random.Next(minHashedStringLength, > maxHashedStringLength); > byte[] plainTextBytes = > (byte[])Array.CreateInstance(typeof(byte), stringLength); > random.NextBytes(plainTextBytes); > System.Security.Cryptography.HashAlgorithm hash = new > SHA1Managed(); > byte[] hashBytes = hash.ComputeHash(plainTextBytes); > string hashValue = Convert.ToBase64String(hashBytes); > string testValue; if > (_generatedHashes.TryGetValue(hashValue, out > testValue)) > collisionsQty++; > else _generatedHashes.Add(hashValue, hashValue); > if (i % 1000000 == 0) > System.Console.WriteLine( > "{0}: i={1}, cs={2}, cu={3}", > DateTime.Now, i, collisionsQty, > longHashCollisionsQty); > > ulong longHash = 0xFFFFFFFF; > foreach (byte b in hashValue) > { > longHash ^= b; > longHash = longHash << 8 + 0xFF; > } > ulong longHashTestValue; > if (_generatedLongHashes.TryGetValue(longHash, out > longHashTestValue)) > { > //Long Hash Collision > longHashCollisionsQty++; > System.Console.WriteLine(" * step {0}, hash '{1}', > longHash: {2:X}", > i, hashValue, longHash); > } > else _generatedLongHashes.Add(longHash, longHash); > > totalLengthOfProcessedStrings += stringLength; > averageStringLength = totalLengthOfProcessedStrings / > i; > } > > System.Console.WriteLine( > "\nTOTALS:\n" + > "Min String Length = {0}\n" + > "Max String Length = {1}\n" + > "Avg String Length = {2}\n" + > "Test cases Qty = {3}\n" + > "Collisions Qty = {4}\n" + > "ulong Collisions Qty = {5}", > minHashedStringLength, maxHashedStringLength, > averageStringLength, testCasesQty, > collisionsQty, > longHashCollisionsQty); > > System.Console.WriteLine("Finished at: " + > System.DateTime.Now); > > } > } > > > P.P.S. Stats > > Started at: 19/03/2011 15:51:04 > 19/03/2011 15:51:16: i=1000000, cs=0, cu=0 > * step 1712197, hash 'Rt4NY5FHg0uMDszVql+l6ksx6Tg=', longHash: > 36D7CFC36A99DE80 > 19/03/2011 15:51:29: i=2000000, cs=0, cu=1 > * step 2040324, hash 'laoPHdqlfeZOx9DI8RDaQZjhKZs=', longHash: > D1B5AB44BB5CDE80 > * step 2043449, hash '1m+tgKu7/l9fl8nUNuz5I/OWF5E=', longHash: > C95F3EBC66B15E80 > 19/03/2011 15:51:44: i=3000000, cs=0, cu=3 > 19/03/2011 15:52:00: i=4000000, cs=0, cu=3 > * step 4369446, hash '2sZrglXDp3NnNPuqdILqwmUKq30=', longHash: > F7DB565F166C1E80 > 19/03/2011 15:52:17: i=5000000, cs=0, cu=4 > > TOTALS: > Min String Length = 7 > Max String Length = 25 > Avg String Length = 15.4981354 > Test cases Qty = 5,000,000 > Collisions Qty = 0 > ulong Collisions Qty = 4 > Finished at: 19/03/2011 15:52:17 > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart > McLachlan Sent: 19 ????? 2011 ?. 13:20 To: Discussion concerning > Visual Basic and related programming issues. Subject: Re: [dba-VB] > SHA1 to compute a hash > > How are you creating your hash? > > Can you post a few examples of different data strings and colliding > SHA1 hashes. I can probably make a lot of money out of them. > AFAIK, no one other than you has found any. > > -- > Stuart > > <<< snip >>> > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 08:21:58 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 09:21:58 -0400 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D849D14.70906@colbyconsulting.com> <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> Message-ID: <4D84ADF6.4000509@colbyconsulting.com> Stuart, Thanks for that. I am not a cryptologist and all I can do is try to use a tool I do not understand to solve a problem I do understand. If my hashes really are that "unique / secure" then I will stop worrying about it. I still need to do a check of actual data though because I went looking one day and discovered "collisions". Given that we believe the collision is not in the key itself, it must have been in my method of finding the collision. In any case, I am not going to worry any more so thanks for the peace of mind! ;) John W. Colby www.ColbyConsulting.com On 3/19/2011 8:51 AM, Stuart McLachlan wrote: > In the case of 1000 single character messages, you are bound to get collisions since there > are only 256 possible original messages. You will be hashing the same value multiple times > > Apart from that, your understanding is incorrect. It doesn't matter how long the string is. The > chances of a collision with two different messages remains the same. > > The message is hashed using padded blocks of a fixed length. > There is no more chance of a collision between "a" and "b" than there is between > "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". > > Specifically, the chance of a collision within n different messages, using b bits of encryption is > (n*(n-1)/2) * (1/2^b). > > Note that the length of the message doesn't come into that equation. > > The probability of a collision is determined by only TWO things. > > 1) The length of the digest. > 2) The quantity of messages. > > As for "everyone agrees". Noone who understands how it works, agrees. > From shamil at smsconsulting.spb.ru Sat Mar 19 08:21:26 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 16:21:26 +0300 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: References: <4D841001.2060200@colbyconsulting.com><4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: <49B2F00A3FD3442685D8283AE7678544@nant> Hi All -- Sorry I have mentioned 6 millions cycles but my sample was for just 5 millions, and I have got some collisions for 8 bytes long surrogate keys. Here I'm posting a patch for hashing part of the code - and no collisions now for 8 bytes long hashes for 5 millions cycles. .... UInt64 longHashResult = 0xFFFFFFFFFFFFFFFF; UInt64 longHash = 0xFFFFFFFFFFFFFFFF; int shiftCounter = 0; foreach (byte b in hashValue) { longHash ^= b; longHash = longHash << 8 + 0xFF; if (++shiftCounter == 8) { longHashResult ^= longHash; shiftCounter = 0; } } longHash = longHashResult; ... Started at: 19/03/2011 16:08:53 19/03/2011 16:09:04: i=1000000, cs=0, cu=0 19/03/2011 16:09:17: i=2000000, cs=0, cu=0 19/03/2011 16:09:33: i=3000000, cs=0, cu=0 19/03/2011 16:09:49: i=4000000, cs=0, cu=0 19/03/2011 16:10:06: i=5000000, cs=0, cu=0 TOTALS: Min String Length = 7 Max String Length = 25 Avg String Length = 15.4966972 Test cases Qty = 5000000 Collisions Qty = 0 ulong Collisions Qty = 0 Finished at: 19/03/2011 16:10:06 Please correct me if I've got wrong somewhere. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 19 ????? 2011 ?. 16:04 To: stuart at lexacorp.com.pg; 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] SHA1 to compute a hash Hi Stuart and John, <<< AFAIK, no one other than you has found any >>> Yes, I was also confused how it comes to get SHA1 hashes collisions but then I have thought that John probably "compacts" SHA1 hashes into 8 bytes surrogate keys? Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions for 6 millions cycles for 8 bytes keys. Please correct me if you find some errors in my sample code Thank you. -- Shamil <<< snip >>> From jwcolby at colbyconsulting.com Sat Mar 19 08:24:54 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 09:24:54 -0400 Subject: [dba-VB] On a lighter note Message-ID: <4D84AEA6.5060107@colbyconsulting.com> I rooted my droid X last night, bought a "freezer" program, froze all the bloatware, and downloaded and installed a tether program. Here at my house I get about 1.25 mbit down and .5 mbit up via the tether. Not bad considering it is over cell. -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sat Mar 19 08:24:33 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 16:24:33 +0300 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84AD8B.14064.3596D15@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D84AD8B.14064.3596D15@stuart.lexacorp.com.pg> Message-ID: <14BB8E00F24E4EBAB5842E5D75537D2F@nant> Hi Stuart -- <<< It looks as though you created 6 million random strings and four of them turned out to be identical. >>> No, all SHA1 strings are different - these are 64 bit hashes of that strings, which have got some collisions. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart McLachlan Sent: 19 ????? 2011 ?. 16:20 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SHA1 to compute a hash If he is compacting a 160 bit hash to a 64 bit hash, he is certainly increasing the odds of a collision - but I suspect that even at 64 bits, the chances of a collision would be very low. But that does not appear to be what you are testing. It looks as though you created 6 million random strings and four of them turned out to be identical. Therefore also had the same SHA1 digest. What happens if you try it again, making sure that all of your strings are unique. Alternatively, what do you get if you add some code to show the pairs of strings. Are they different? -- Stuart On 19 Mar 2011 at 16:03, Shamil Salakhetdinov wrote: > Hi Stuart and John, > > <<< > AFAIK, no one other than you has found any > >>> > Yes, I was also confused how it comes to get SHA1 hashes collisions > but then I have thought that John probably "compacts" SHA1 hashes into > 8 bytes surrogate keys? > > Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions > for 6 millions cycles for 8 bytes keys. > > Please correct me if you find some errors in my sample code > > Thank you. > > -- > Shamil > <<< snip >>> From stuart at lexacorp.com.pg Sat Mar 19 08:29:55 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 23:29:55 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84ADF6.4000509@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com>, <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg>, <4D84ADF6.4000509@colbyconsulting.com> Message-ID: <4D84AFD3.25991.3625487@stuart.lexacorp.com.pg> Using a 160 bit SHA1, the odds of getting a collision are 50% when you have approximately 2^80 messages. If you ever get near that number of records, I suggest you switch to SHA2 which is 256 bit :-) -- Stuart On 19 Mar 2011 at 9:21, jwcolby wrote: > Stuart, > > Thanks for that. I am not a cryptologist and all I can do is try to > use a tool I do not understand to solve a problem I do understand. If > my hashes really are that "unique / secure" then I will stop worrying > about it. I still need to do a check of actual data though because I > went looking one day and discovered "collisions". Given that we > believe the collision is not in the key itself, it must have been in > my method of finding the collision. > > In any case, I am not going to worry any more so thanks for the peace > of mind! ;) > > John W. Colby > www.ColbyConsulting.com > > On 3/19/2011 8:51 AM, Stuart McLachlan wrote: > > In the case of 1000 single character messages, you are bound to get > > collisions since there are only 256 possible original messages. You > > will be hashing the same value multiple times > > > > Apart from that, your understanding is incorrect. It doesn't matter > > how long the string is. The chances of a collision with two > > different messages remains the same. > > > > The message is hashed using padded blocks of a fixed length. > > There is no more chance of a collision between "a" and "b" than > > there is between "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". > > > > Specifically, the chance of a collision within n different messages, > > using b bits of encryption is (n*(n-1)/2) * (1/2^b). > > > > Note that the length of the message doesn't come into that equation. > > > > The probability of a collision is determined by only TWO things. > > > > 1) The length of the digest. > > 2) The quantity of messages. > > > > As for "everyone agrees". Noone who understands how it works, > > agrees. > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 08:55:51 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 09:55:51 -0400 Subject: [dba-VB] Using the hash for real world problems Message-ID: <4D84B5E7.3010204@colbyconsulting.com> I got into the hashing business in order to create a surrogate key of sorts, in fact a trio of them. I have an address hash, a family hash and a person hash. I compute these three hashes for every table of persons. Now that I have these surrogate keys I have "identical" values in a single field where the input is identical in multiple fields. Thus I do not have to do multi-field joins where I have a hash for all of those fields. I am happy to hear that I am secure in my surrogate hash algorithm choice. OK, so I have these hashes. Every record has these three hashes. Now some definitions: HashAddr: A unique address is addr/zip5/zip4 different HashFamily: A unique family is LName/Addr/Zip5/Zip4 different HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. Using this information, I need to calculate the count of addresses with a single person, with two persons etc for as many combinations as I have. Addresses People 22,538,240 1 780,462,346 2 52,234 3 people etc etc till every quantity of people is found. What is the simplest way to accomplish this with SQL. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 19 09:13:51 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 10:13:51 -0400 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84B5E7.3010204@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com> Message-ID: <4D84BA1F.2060101@colbyconsulting.com> Ooops. My examples were hosed. Let's try again Addresses People 22,538,240 1 00,708,462 2 00,052,234 3 and so forth. What the client really wants to know is "how many people with emails are at a specific address" "how many people with emails are in a specific family" etc. I tie my address table to my email table using an auto increment long increment "I must be careful not to call this a surrogate" key and then perform the counts using the hash fields in a Group By / count query. John W. Colby www.ColbyConsulting.com On 3/19/2011 9:55 AM, jwcolby wrote: > I got into the hashing business in order to create a surrogate key of sorts, in fact a trio of them. > > I have an address hash, a family hash and a person hash. I compute these three hashes for every > table of persons. Now that I have these surrogate keys I have "identical" values in a single field > where the input is identical in multiple fields. Thus I do not have to do multi-field joins where I > have a hash for all of those fields. > > I am happy to hear that I am secure in my surrogate hash algorithm choice. > > OK, so I have these hashes. Every record has these three hashes. > > Now some definitions: > > HashAddr: A unique address is addr/zip5/zip4 different > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > Using this information, I need to calculate the count of addresses with a single person, with two > persons etc for as many combinations as I have. > > Addresses People > 22,538,240 1 > 780,462,346 2 > 52,234 3 people > etc > etc till every quantity of people is found. > > What is the simplest way to accomplish this with SQL. > > > > From shamil at smsconsulting.spb.ru Sat Mar 19 10:07:08 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 18:07:08 +0300 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84BA1F.2060101@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com> <4D84BA1F.2060101@colbyconsulting.com> Message-ID: Hi John -- I suppose you can reuse the following SQL expression for your case: select count(Occurrence) QtyTheSameOccurrences, Occurrence from ( select Country, Count(Country) Occurrence from Customer Group by Country ) o group by o.Occurrence order by o.Occurrence Just replace Country name columns with hashAddress column and Country table name with your table name ... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 19 ????? 2011 ?. 17:14 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Using the hash for real world problems Ooops. My examples were hosed. Let's try again Addresses People 22,538,240 1 00,708,462 2 00,052,234 3 and so forth. What the client really wants to know is "how many people with emails are at a specific address" "how many people with emails are in a specific family" etc. I tie my address table to my email table using an auto increment long increment "I must be careful not to call this a surrogate" key and then perform the counts using the hash fields in a Group By / count query. John W. Colby www.ColbyConsulting.com On 3/19/2011 9:55 AM, jwcolby wrote: > I got into the hashing business in order to create a surrogate key of sorts, in fact a trio of them. > > I have an address hash, a family hash and a person hash. I compute > these three hashes for every table of persons. Now that I have these > surrogate keys I have "identical" values in a single field where the > input is identical in multiple fields. Thus I do not have to do multi-field joins where I have a hash for all of those fields. > > I am happy to hear that I am secure in my surrogate hash algorithm choice. > > OK, so I have these hashes. Every record has these three hashes. > > Now some definitions: > > HashAddr: A unique address is addr/zip5/zip4 different > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > Using this information, I need to calculate the count of addresses > with a single person, with two persons etc for as many combinations as I have. > > Addresses People > 22,538,240 1 > 780,462,346 2 > 52,234 3 people > etc > etc till every quantity of people is found. > > What is the simplest way to accomplish this with SQL. > > > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Mar 19 14:50:55 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 15:50:55 -0400 Subject: [dba-VB] Most useful droid apps Message-ID: <4D85091F.5060807@colbyconsulting.com> What are the most useful apps you run on your droid? -- John W. Colby www.ColbyConsulting.com From davidmcafee at gmail.com Sat Mar 19 15:53:32 2011 From: davidmcafee at gmail.com (David McAfee) Date: Sat, 19 Mar 2011 13:53:32 -0700 Subject: [dba-VB] Most useful droid apps Message-ID: Advanced task killer Weather channel Lookout antivirus AK Notepad Bank of America Acar auto maintenance Barcode scanner Bubble level Bump Droid light Google goggles Google translate IMDB Key ring Print share Trapster Sent from my Droid phone. On Mar 19, 2011 12:52 PM, "jwcolby" wrote: From stuart at lexacorp.com.pg Sat Mar 19 18:11:26 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sun, 20 Mar 2011 09:11:26 +1000 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84BA1F.2060101@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com>, <4D84BA1F.2060101@colbyconsulting.com> Message-ID: <4D85381E.9973.576BE3C@stuart.lexacorp.com.pg> Caaution. All aircode below: To get the list output below, I would use something like Select Distinct Count(HashAddr) as Addresses, People >From (Select distinct HashAddr,Count(HashAddr) as People From tblPeople Group By HashAddr) Group By People Order by People To find out details for a *specific* Address Find the Hash for the desired Address and then pass it to something like this: Select Distinct StreetNo,Street,City,Zip,State,Count(HashAddr) as Address >From tblPeople P Inner Join tblEmails E on P.PK = E.PeopleFK Group By HashAddr StreetNo,Street,City,Zip,State Where HashAddr = @SearchHash On 19 Mar 2011 at 10:13, jwcolby wrote: > Ooops. My examples were hosed. > > Let's try again > > Addresses People > 22,538,240 1 > 00,708,462 2 > 00,052,234 3 > > and so forth. > > What the client really wants to know is > > "how many people with emails are at a specific address" > "how many people with emails are in a specific family" > > etc. > > I tie my address table to my email table using an auto increment long > increment "I must be careful not to call this a surrogate" key and > then perform the counts using the hash fields in a Group By / count > query. > > John W. Colby > www.ColbyConsulting.com > > On 3/19/2011 9:55 AM, jwcolby wrote: > > I got into the hashing business in order to create a surrogate key > > of sorts, in fact a trio of them. > > > > I have an address hash, a family hash and a person hash. I compute > > these three hashes for every table of persons. Now that I have these > > surrogate keys I have "identical" values in a single field where the > > input is identical in multiple fields. Thus I do not have to do > > multi-field joins where I have a hash for all of those fields. > > > > I am happy to hear that I am secure in my surrogate hash algorithm > > choice. > > > > OK, so I have these hashes. Every record has these three hashes. > > > > Now some definitions: > > > > HashAddr: A unique address is addr/zip5/zip4 different > > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > > > Using this information, I need to calculate the count of addresses > > with a single person, with two persons etc for as many combinations > > as I have. > > > > Addresses People > > 22,538,240 1 > > 780,462,346 2 > > 52,234 3 people > > etc > > etc till every quantity of people is found. > > > > What is the simplest way to accomplish this with SQL. > > > > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 18:21:18 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sun, 20 Mar 2011 09:21:18 +1000 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84B5E7.3010204@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com> Message-ID: <4D853A6E.20716.57FC618@stuart.lexacorp.com.pg> Answered on VB list. On 19 Mar 2011 at 9:55, jwcolby wrote: > I got into the hashing business in order to create a surrogate key of > sorts, in fact a trio of them. > > I have an address hash, a family hash and a person hash. I compute > these three hashes for every table of persons. Now that I have these > surrogate keys I have "identical" values in a single field where the > input is identical in multiple fields. Thus I do not have to do > multi-field joins where I have a hash for all of those fields. > > I am happy to hear that I am secure in my surrogate hash algorithm > choice. > > OK, so I have these hashes. Every record has these three hashes. > > Now some definitions: > > HashAddr: A unique address is addr/zip5/zip4 different > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > Using this information, I need to calculate the count of addresses > with a single person, with two persons etc for as many combinations as > I have. > > Addresses People > 22,538,240 1 > 780,462,346 2 > 52,234 3 people > etc > etc till every quantity of people is found. > > What is the simplest way to accomplish this with SQL. > > > > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 21:01:04 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 22:01:04 -0400 Subject: [dba-VB] Most useful droid apps In-Reply-To: References: Message-ID: <4D855FE0.9070004@colbyconsulting.com> I am trying various tethering solutions. My laptop is vista, my ip range is 192.168.122.x. I got WiFi Tether to work, but only without encryption. I was getting about 1.5 mbit down and .5 mbit up. As soon as I turned on encryption I was unable to set up the laptop password to match (that was the only thing that changed). No encryption is unacceptable to me as I would only really use this in public places, exactly where you would most want encryption. I tried barnacle. That one just didn't work at all. I am now trying PDANet which is a USB tether. Since the PC is tied to the phone by a usb, encryption seems to be unnecessary? Unfortunately this one is not working either. The PDANet desktop app says it is connected and the droid pdanet app says it is connected but the PC just can't see the internet. I can't ping anything etc. Any suggestions for this beast? John W. Colby www.ColbyConsulting.com On 3/19/2011 4:53 PM, David McAfee wrote: > Advanced task killer > Weather channel > Lookout antivirus > AK Notepad > Bank of America > Acar auto maintenance > Barcode scanner > Bubble level > Bump > Droid light > Google goggles > Google translate > IMDB > Key ring > Print share > Trapster > > Sent from my Droid phone. > On Mar 19, 2011 12:52 PM, "jwcolby" wrote: > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 21:34:03 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sun, 20 Mar 2011 12:34:03 +1000 Subject: [dba-VB] Most useful droid apps In-Reply-To: <4D855FE0.9070004@colbyconsulting.com> References: , <4D855FE0.9070004@colbyconsulting.com> Message-ID: <4D85679B.30595.6303F8F@stuart.lexacorp.com.pg> Are you on AT&T? If so, you may be interested in this: http://www.theregister.co.uk/2011/03/18/new_att_tethering_policy/ On 19 Mar 2011 at 22:01, jwcolby wrote: > I am trying various tethering solutions. My laptop is vista, my ip > range is 192.168.122.x. > > I got WiFi Tether to work, but only without encryption. I was getting > about 1.5 mbit down and .5 mbit up. > > As soon as I turned on encryption I was unable to set up the laptop > password to match (that was the only thing that changed). No > encryption is unacceptable to me as I would only really use this in > public places, exactly where you would most want encryption. > > I tried barnacle. That one just didn't work at all. > > I am now trying PDANet which is a USB tether. Since the PC is tied to > the phone by a usb, encryption seems to be unnecessary? Unfortunately > this one is not working either. The PDANet desktop app says it is > connected and the droid pdanet app says it is connected but the PC > just can't see the internet. I can't ping anything etc. > > Any suggestions for this beast? > > > John W. Colby > www.ColbyConsulting.com > > On 3/19/2011 4:53 PM, David McAfee wrote: > > Advanced task killer > > Weather channel > > Lookout antivirus > > AK Notepad > > Bank of America > > Acar auto maintenance > > Barcode scanner > > Bubble level > > Bump > > Droid light > > Google goggles > > Google translate > > IMDB > > Key ring > > Print share > > Trapster > > > > Sent from my Droid phone. > > On Mar 19, 2011 12:52 PM, "jwcolby" > > wrote: _______________________________________________ dba-VB > > mailing list dba-VB at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > http://www.databaseadvisors.com > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 21:45:39 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 22:45:39 -0400 Subject: [dba-VB] Most useful droid apps In-Reply-To: <4D85679B.30595.6303F8F@stuart.lexacorp.com.pg> References: , <4D855FE0.9070004@colbyconsulting.com> <4D85679B.30595.6303F8F@stuart.lexacorp.com.pg> Message-ID: <4D856A53.7010507@colbyconsulting.com> Nope, Verizon John W. Colby www.ColbyConsulting.com On 3/19/2011 10:34 PM, Stuart McLachlan wrote: > Are you on AT&T? > > If so, you may be interested in this: > > http://www.theregister.co.uk/2011/03/18/new_att_tethering_policy/ > > > On 19 Mar 2011 at 22:01, jwcolby wrote: > >> I am trying various tethering solutions. My laptop is vista, my ip >> range is 192.168.122.x. >> >> I got WiFi Tether to work, but only without encryption. I was getting >> about 1.5 mbit down and .5 mbit up. >> >> As soon as I turned on encryption I was unable to set up the laptop >> password to match (that was the only thing that changed). No >> encryption is unacceptable to me as I would only really use this in >> public places, exactly where you would most want encryption. >> >> I tried barnacle. That one just didn't work at all. >> >> I am now trying PDANet which is a USB tether. Since the PC is tied to >> the phone by a usb, encryption seems to be unnecessary? Unfortunately >> this one is not working either. The PDANet desktop app says it is >> connected and the droid pdanet app says it is connected but the PC >> just can't see the internet. I can't ping anything etc. >> >> Any suggestions for this beast? >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/19/2011 4:53 PM, David McAfee wrote: >>> Advanced task killer >>> Weather channel >>> Lookout antivirus >>> AK Notepad >>> Bank of America >>> Acar auto maintenance >>> Barcode scanner >>> Bubble level >>> Bump >>> Droid light >>> Google goggles >>> Google translate >>> IMDB >>> Key ring >>> Print share >>> Trapster >>> >>> Sent from my Droid phone. >>> On Mar 19, 2011 12:52 PM, "jwcolby" >>> wrote: _______________________________________________ dba-VB >>> mailing list dba-VB at databaseadvisors.com >>> http://databaseadvisors.com/mailman/listinfo/dba-vb >>> http://www.databaseadvisors.com >>> >>> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From listmaster at databaseadvisors.com Wed Mar 23 14:34:46 2011 From: listmaster at databaseadvisors.com (Bryan Carbonnell) Date: Wed, 23 Mar 2011 15:34:46 -0400 Subject: [dba-VB] Administrivia - Spam Filtering on DBA Servers In-Reply-To: References: Message-ID: We have made a change to the way our mailservers deal with spam filtering. We were getting too many false positive reports from one of the spam black list services we use, so we have discontinued using them. Hopefully your bounce troubles should be over, or at the very least greatly minimized. If you continue to have issues surrounding bounces, PLEASE, PLEASE, PLEASE get in touch with me (listmaster at databaseadvisors OR carbonnb at gmail.com if the listmaster address bounces). -- Bryan Carbonnell - listmaster at databaseadvisors.com Life's journey is not to arrive at the grave safely in a well preserved body, but rather to skid in sideways, totally worn out, shouting "What a great ride!" From jwcolby at colbyconsulting.com Thu Mar 24 13:07:19 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 24 Mar 2011 14:07:19 -0400 Subject: [dba-VB] C# protected Message-ID: <4D8B8857.60604@colbyconsulting.com> I am trying to create a method of a ClassA which can be seen from the object ClassB that instantiates ClassA but not by objectClassC which instantiates ClassB. IOW ClassA "belongs to" ClassB so specific methods / properties of ClassB should be manipulateable by ClassB. However those properties / methods should not be visible to anything outside of ClassA and ClassB. Possible? Oh, and BTW I want ClassA to be out in a class library, external to the project using it. -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Thu Mar 24 15:14:59 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 24 Mar 2011 23:14:59 +0300 Subject: [dba-VB] C# protected In-Reply-To: <4D8B8857.60604@colbyconsulting.com> References: <4D8B8857.60604@colbyconsulting.com> Message-ID: Hi John -- Put ClassA and ClassB into a class library. Use internal access modifier with their methods and properties to be seen within their class library but not from the "outer world". Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 24 ????? 2011 ?. 21:07 To: VBA Subject: [dba-VB] C# protected I am trying to create a method of a ClassA which can be seen from the object ClassB that instantiates ClassA but not by objectClassC which instantiates ClassB. IOW ClassA "belongs to" ClassB so specific methods / properties of ClassB should be manipulateable by ClassB. However those properties / methods should not be visible to anything outside of ClassA and ClassB. Possible? Oh, and BTW I want ClassA to be out in a class library, external to the project using it. -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Thu Mar 24 15:28:03 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 24 Mar 2011 16:28:03 -0400 Subject: [dba-VB] C# protected In-Reply-To: References: <4D8B8857.60604@colbyconsulting.com> Message-ID: <4D8BA953.4030602@colbyconsulting.com> ClassA can go in the class library, but classB is part of the application. John W. Colby www.ColbyConsulting.com On 3/24/2011 4:14 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Put ClassA and ClassB into a class library. > Use > > internal > > access modifier with their methods and properties to be seen within their > class library but not from the "outer world". > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 24 ????? 2011 ?. 21:07 > To: VBA > Subject: [dba-VB] C# protected > > I am trying to create a method of a ClassA which can be seen from the object > ClassB that instantiates ClassA but not by objectClassC which instantiates > ClassB. > > IOW ClassA "belongs to" ClassB so specific methods / properties of ClassB > should be manipulateable by ClassB. However those properties / methods > should not be visible to anything outside of ClassA and ClassB. > > Possible? > > Oh, and BTW I want ClassA to be out in a class library, external to the > project using it. > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Thu Mar 24 16:37:22 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 25 Mar 2011 00:37:22 +0300 Subject: [dba-VB] C# protected In-Reply-To: <4D8BA953.4030602@colbyconsulting.com> References: <4D8B8857.60604@colbyconsulting.com> <4D8BA953.4030602@colbyconsulting.com> Message-ID: Hi John -- Then use protected access modifier for methods and properties of Class A which is inherited by class B (IOW no need to instantiate class A explicitly)... Class C which will instantiate Class B will no see/have access to protected properties and methods of class A inherithed by class B. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 24 ????? 2011 ?. 23:28 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# protected ClassA can go in the class library, but classB is part of the application. John W. Colby www.ColbyConsulting.com On 3/24/2011 4:14 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Put ClassA and ClassB into a class library. > Use > > internal > > access modifier with their methods and properties to be seen within > their class library but not from the "outer world". > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 24 ????? 2011 ?. 21:07 > To: VBA > Subject: [dba-VB] C# protected > > I am trying to create a method of a ClassA which can be seen from the > object ClassB that instantiates ClassA but not by objectClassC which > instantiates ClassB. > > IOW ClassA "belongs to" ClassB so specific methods / properties of > ClassB should be manipulateable by ClassB. However those properties / > methods should not be visible to anything outside of ClassA and ClassB. > > Possible? > > Oh, and BTW I want ClassA to be out in a class library, external to > the project using it. > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 25 14:58:27 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 25 Mar 2011 15:58:27 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <3B9DE744CEA244829179A81CA818F92F@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com> <3B9DE744CEA244829179A81CA818F92F@nant> Message-ID: <4D8CF3E3.3050608@colbyconsulting.com> Shamil, Does this allow using a single "lock object" to lock various operations in a class. For example I want to lock a date flag variable while using a property to either get or set the variable. private Object thisLock = new Object(); public DateTime pFlagDte { get { lock (thisLock) { return dteFlag; } } set { lock (thisLock) { dteFlag = value; } } } The issue is not multiple threads trying to write to the date variable but rather one thread trying to read it while another is writing it. John W. Colby www.ColbyConsulting.com On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Do you you need add, remove and iterate a list to get an object or you need > to add, remove and get an object by its key? - if the latter - here is the > code you can use: > > private Dictionary _myObjects = new Dictionary MyObject>(); > private static object _myObjectsLocker = new object(); > public void Add(MyObject myObject) > { > lock (_myObjectsLocker) > { > _myObjects.Add(myObject.ID, myObject); > _keys.Add(myObject.ID); > } > } > public MyObject Remove(int id) > { > MyObject myobject = null; > lock (_myObjectsLocker) > { > myobject = _myObjects[id]; > _myObjects.Remove(id); > } > return myobject; > } > public MyObject this[int key] > { > get > { > MyObject myObject = null; > lock (_myObjectsLocker) > { > _myObjects.TryGetValue(key, out myObject); > } > return myObject; > } > } > > for the case of iteration just replace: > > _myObjects.TryGetValue(key, out myObject); > > with your iteration code. > > As for the sample code - changing qty of threads and changing total computer > workload (outer context for the test code) you can see how different could > be final results when heavy multi-threading is used. And even if it's not > heavy - it still can result in weird output if special multi-threading > coding technique aren't used... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 11 ????? 2011 ?. 1:27 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Thanks for the code but I think it is overkill for my needs so far, though I > will put it in the archives for when I need it. > > In my system so far I have (or can arrange to have ) only two threads > trying to access a list. > > Basically what I am doing is reading records out of sql server into a class > factory, and placing the class instances into a list. > > Another object with its own thread is coming by and looking at the objects > in the list to see if there is any work for it to do. If it finds an object > to work on, it grabs a pointer to that item in the list and stores it in a > pointer and releases the list. > > One of the ways I saw this handled was to create factory which creates a > brand new list object, grabs a pointer to the list, locks that and then > copies the objects in the list pointers to classes) into that new list, > unlocks the list and hands back a pointer to this brand new list. thus the > new list could be iterated at will. > > I am just trying to lock the list while one thread adds objects to the list > or another thread gets objects out of the list. > > So if I have object A which contains a list and is adding / deleting objects > to that list... > and object B which needs to iterate through the list looking for an object > of interest in the list, and grabbing a pointer to that object, there is no > way to do this without all of the code you wrote? > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Does that in fact lock the list to other objects doing the same thing >> (getting the list through that property)? >>>>> >> You have to lock/synchronize/serialize not the list itself but >> simultaneous access to the list from different threads. >> You'd better use special static object variable to implement locking. >> >> I will post sample code in my following posting here... >> >> Thank you. >> >> -- >> Shamil > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Mar 25 16:05:57 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 26 Mar 2011 00:05:57 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8CF3E3.3050608@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com> Message-ID: <3F0DA84A905C4947819126F43CFC39DB@nant> Hi John -- Your sample is a correct usage of locking - just use static thisLock variable private static Object thisLock = new Object() as non-static variable *is not* thread safe AFAIU. Yes, such locking can be done for various operations of that class - just make sure you'll not get dead-locked... Also use *one lock object* to lock *one resource* in a class - otherwise "deadlock" will become your "everyday guest"... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 25 ????? 2011 ?. 22:58 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, Does this allow using a single "lock object" to lock various operations in a class. For example I want to lock a date flag variable while using a property to either get or set the variable. private Object thisLock = new Object(); public DateTime pFlagDte { get { lock (thisLock) { return dteFlag; } } set { lock (thisLock) { dteFlag = value; } } } The issue is not multiple threads trying to write to the date variable but rather one thread trying to read it while another is writing it. John W. Colby www.ColbyConsulting.com On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Do you you need add, remove and iterate a list to get an object or you > need to add, remove and get an object by its key? - if the latter - > here is the code you can use: > > private Dictionary _myObjects = new Dictionary MyObject>(); > private static object _myObjectsLocker = new object(); public void > Add(MyObject myObject) { > lock (_myObjectsLocker) > { > _myObjects.Add(myObject.ID, myObject); > _keys.Add(myObject.ID); > } > } > public MyObject Remove(int id) > { > MyObject myobject = null; > lock (_myObjectsLocker) > { > myobject = _myObjects[id]; > _myObjects.Remove(id); > } > return myobject; > } > public MyObject this[int key] > { > get > { > MyObject myObject = null; > lock (_myObjectsLocker) > { > _myObjects.TryGetValue(key, out myObject); > } > return myObject; > } > } > > for the case of iteration just replace: > > _myObjects.TryGetValue(key, out myObject); > > with your iteration code. > > As for the sample code - changing qty of threads and changing total > computer workload (outer context for the test code) you can see how > different could be final results when heavy multi-threading is used. > And even if it's not heavy - it still can result in weird output if > special multi-threading coding technique aren't used... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 11 ????? 2011 ?. 1:27 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Thanks for the code but I think it is overkill for my needs so far, > though I will put it in the archives for when I need it. > > In my system so far I have (or can arrange to have ) only two threads > trying to access a list. > > Basically what I am doing is reading records out of sql server into a > class factory, and placing the class instances into a list. > > Another object with its own thread is coming by and looking at the > objects in the list to see if there is any work for it to do. If it > finds an object to work on, it grabs a pointer to that item in the > list and stores it in a pointer and releases the list. > > One of the ways I saw this handled was to create factory which creates > a brand new list object, grabs a pointer to the list, locks that and > then copies the objects in the list pointers to classes) into that new > list, unlocks the list and hands back a pointer to this brand new > list. thus the new list could be iterated at will. > > I am just trying to lock the list while one thread adds objects to the > list or another thread gets objects out of the list. > > So if I have object A which contains a list and is adding / deleting > objects to that list... > and object B which needs to iterate through the list looking for an > object of interest in the list, and grabbing a pointer to that object, > there is no way to do this without all of the code you wrote? > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Does that in fact lock the list to other objects doing the same thing >> (getting the list through that property)? >>>>> >> You have to lock/synchronize/serialize not the list itself but >> simultaneous access to the list from different threads. >> You'd better use special static object variable to implement locking. >> >> I will post sample code in my following posting here... >> >> Thank you. >> >> -- >> Shamil > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 25 16:38:19 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 25 Mar 2011 17:38:19 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <3F0DA84A905C4947819126F43CFC39DB@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com> <3F0DA84A905C4947819126F43CFC39DB@nant> Message-ID: <4D8D0B4B.9070004@colbyconsulting.com> When you say resource, you mean variable? IOW use private static Object LockBln = new Object(); private static Object LockDte = new Object(); and use the LockBln to lock the boolean object while using LockDte to lock the date object? John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - just > make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - otherwise > "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various operations in a > class. > > For example I want to lock a date flag variable while using a property to > either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable but > rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Do you you need add, remove and iterate a list to get an object or you >> need to add, remove and get an object by its key? - if the latter - >> here is the code you can use: >> >> private Dictionary _myObjects = new Dictionary> MyObject>(); >> private static object _myObjectsLocker = new object(); public void >> Add(MyObject myObject) { >> lock (_myObjectsLocker) >> { >> _myObjects.Add(myObject.ID, myObject); >> _keys.Add(myObject.ID); >> } >> } >> public MyObject Remove(int id) >> { >> MyObject myobject = null; >> lock (_myObjectsLocker) >> { >> myobject = _myObjects[id]; >> _myObjects.Remove(id); >> } >> return myobject; >> } >> public MyObject this[int key] >> { >> get >> { >> MyObject myObject = null; >> lock (_myObjectsLocker) >> { >> _myObjects.TryGetValue(key, out myObject); >> } >> return myObject; >> } >> } >> >> for the case of iteration just replace: >> >> _myObjects.TryGetValue(key, out myObject); >> >> with your iteration code. >> >> As for the sample code - changing qty of threads and changing total >> computer workload (outer context for the test code) you can see how >> different could be final results when heavy multi-threading is used. >> And even if it's not heavy - it still can result in weird output if >> special multi-threading coding technique aren't used... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 11 ????? 2011 ?. 1:27 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Thanks for the code but I think it is overkill for my needs so far, >> though I will put it in the archives for when I need it. >> >> In my system so far I have (or can arrange to have ) only two threads >> trying to access a list. >> >> Basically what I am doing is reading records out of sql server into a >> class factory, and placing the class instances into a list. >> >> Another object with its own thread is coming by and looking at the >> objects in the list to see if there is any work for it to do. If it >> finds an object to work on, it grabs a pointer to that item in the >> list and stores it in a pointer and releases the list. >> >> One of the ways I saw this handled was to create factory which creates >> a brand new list object, grabs a pointer to the list, locks that and >> then copies the objects in the list pointers to classes) into that new >> list, unlocks the list and hands back a pointer to this brand new >> list. thus the new list could be iterated at will. >> >> I am just trying to lock the list while one thread adds objects to the >> list or another thread gets objects out of the list. >> >> So if I have object A which contains a list and is adding / deleting >> objects to that list... >> and object B which needs to iterate through the list looking for an >> object of interest in the list, and grabbing a pointer to that object, >> there is no way to do this without all of the code you wrote? >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >>> Hi John -- >>> >>> <<< >>> Does that in fact lock the list to other objects doing the same thing >>> (getting the list through that property)? >>>>>> >>> You have to lock/synchronize/serialize not the list itself but >>> simultaneous access to the list from different threads. >>> You'd better use special static object variable to implement locking. >>> >>> I will post sample code in my following posting here... >>> >>> Thank you. >>> >>> -- >>> Shamil >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Mar 25 17:46:44 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 26 Mar 2011 01:46:44 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8D0B4B.9070004@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8D0B4B.9070004@colbyconsulting.com> Message-ID: Hi John -- By resource I mean access to a read/write resource - it could be not only a variable... If you need two or more class level variables (fields, properties) within one class instance to be locked then keep them in a class/structure and lock that class/structure instance... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 26 ????? 2011 ?. 0:38 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() When you say resource, you mean variable? IOW use private static Object LockBln = new Object(); private static Object LockDte = new Object(); and use the LockBln to lock the boolean object while using LockDte to lock the date object? John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - > just make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - > otherwise "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various > operations in a class. > > For example I want to lock a date flag variable while using a property > to either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable > but rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Do you you need add, remove and iterate a list to get an object or >> you need to add, remove and get an object by its key? - if the latter >> - here is the code you can use: >> >> private Dictionary _myObjects = new Dictionary> MyObject>(); >> private static object _myObjectsLocker = new object(); public void >> Add(MyObject myObject) { >> lock (_myObjectsLocker) >> { >> _myObjects.Add(myObject.ID, myObject); >> _keys.Add(myObject.ID); >> } >> } >> public MyObject Remove(int id) >> { >> MyObject myobject = null; >> lock (_myObjectsLocker) >> { >> myobject = _myObjects[id]; >> _myObjects.Remove(id); >> } >> return myobject; >> } >> public MyObject this[int key] >> { >> get >> { >> MyObject myObject = null; >> lock (_myObjectsLocker) >> { >> _myObjects.TryGetValue(key, out myObject); >> } >> return myObject; >> } >> } >> >> for the case of iteration just replace: >> >> _myObjects.TryGetValue(key, out myObject); >> >> with your iteration code. >> >> As for the sample code - changing qty of threads and changing total >> computer workload (outer context for the test code) you can see how >> different could be final results when heavy multi-threading is used. >> And even if it's not heavy - it still can result in weird output if >> special multi-threading coding technique aren't used... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 11 ????? 2011 ?. 1:27 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Thanks for the code but I think it is overkill for my needs so far, >> though I will put it in the archives for when I need it. >> >> In my system so far I have (or can arrange to have ) only two >> threads trying to access a list. >> >> Basically what I am doing is reading records out of sql server into a >> class factory, and placing the class instances into a list. >> >> Another object with its own thread is coming by and looking at the >> objects in the list to see if there is any work for it to do. If it >> finds an object to work on, it grabs a pointer to that item in the >> list and stores it in a pointer and releases the list. >> >> One of the ways I saw this handled was to create factory which >> creates a brand new list object, grabs a pointer to the list, locks >> that and then copies the objects in the list pointers to classes) >> into that new list, unlocks the list and hands back a pointer to this >> brand new list. thus the new list could be iterated at will. >> >> I am just trying to lock the list while one thread adds objects to >> the list or another thread gets objects out of the list. >> >> So if I have object A which contains a list and is adding / deleting >> objects to that list... >> and object B which needs to iterate through the list looking for an >> object of interest in the list, and grabbing a pointer to that >> object, there is no way to do this without all of the code you wrote? >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >>> Hi John -- >>> >>> <<< >>> Does that in fact lock the list to other objects doing the same >>> thing (getting the list through that property)? >>>>>> >>> You have to lock/synchronize/serialize not the list itself but >>> simultaneous access to the list from different threads. >>> You'd better use special static object variable to implement locking. >>> >>> I will post sample code in my following posting here... >>> >>> Thank you. >>> >>> -- >>> Shamil >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 25 21:57:22 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 25 Mar 2011 22:57:22 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8D0B4B.9070004@colbyconsulting.com> Message-ID: <4D8D5612.6050701@colbyconsulting.com> Shamil, I have processes that log results to flags. For example, make a database (log that it was made), build a table (log that it was built), pull umpteen million records in sorted order (log that it was filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build another chunk table (log that it was filled), BCP out (log that it was exported). The objective is to be able to sustain interruptions and pick up where we left off. These processes can take minutes (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). So each thing I do represents a step in the process and each step is logged in a field in a record in SQL server using a datetime. There are so many of these flags that I am trying to standardize the process by building a class that can be instantiated, filled with data and log itself to SQL Server by one thread and be checked by another thread. These flag class instances will be checked by multiple threads, each thread trying to decide whether it should be doing the next step because another thread has finished it's part. IOW if a file has been written to disk, then the next thread will write it to a VM for processing. If it moved to the VM the next thread will watch the VM's output directory for a file to pop out and move it back to a directory on the server. If the file (a couple of files actually) successfully copied back to the server staging then another thread will import it back into a chunk table in an input database. If the file successfully imported then another thread will... In general one thread will "own" the flag and use it to log its status and one other thread will be checking the status of the flag to determine that it can go to work on that work chunk. You get the picture. I am trying to build an entirely asynchronous highly threaded process which exports a huge table into multiple files, processes every file through a third party app and gets the results back into SQL Server. All while logging each and every step so that no piece can possibly be dropped at any stage, even if the server goes down (or the VM goes down). Eventually this process will run on my server 24/7. It has been working for some time but I am getting threading issues, and I need to work on the high level control so that all of the processes can cleanly start up and shut down and every stage can pick back up when the program restarts should a shutdown occur. A single database can be up to a hundred million records (the biggest so far), and the external program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour depending on many different things so that example will take 50 chunks and could take 40 to 50 hours to complete. It takes about 20 processing steps to handle each file from end to end. It needs to just work, and I need to be able to view status in a meaningful way. And I need to process that and a dozen other files every single month, automatically, with no manual intervention required. John W. Colby www.ColbyConsulting.com On 3/25/2011 6:46 PM, Shamil Salakhetdinov wrote: > Hi John -- > > By resource I mean access to a read/write resource - it could be not only a > variable... > > If you need two or more class level variables (fields, properties) within > one class instance to be locked then keep them in a class/structure and lock > that class/structure instance... > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Sat Mar 26 00:37:25 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 26 Mar 2011 08:37:25 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8D5612.6050701@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8D0B4B.9070004@colbyconsulting.com> <4D8D5612.6050701@colbyconsulting.com> Message-ID: <4D87466503764D98871B609159033CB6@nant> John -- But you can try to use several instances of .NET Framework Queue class http://msdn.microsoft.com/en-us/library/system.collections.queue.aspx ? As far as I understand you have a set of sequential tasks to be performed on every (input/start) chunk so you can make a dedicated Queue instance for every task, and that Queue instance will be handled by dedicated "processor" classes: - TaskA Queue items will be processed by TaskAQueueProcessor, - TaksB .... by TaskBQueueProcessor, -- - TaskZ... by TaskZQueueProcessor There could be many such TaskA(B, ....Z)QueueProcessor instances "floating around" looking for an item from their dedicated tasks' queue to process: when they find it, they pick it up, process and put into another Queue based on "workflow plan"... You can also try to consider using Windows Workflow Foundation(WF) if that wouldn't be an "overkill" for your case... http://msdn.microsoft.com/en-us/netframework/wf-screencasts.aspx#introwf4 Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 26 ????? 2011 ?. 5:57 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, I have processes that log results to flags. For example, make a database (log that it was made), build a table (log that it was built), pull umpteen million records in sorted order (log that it was filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build another chunk table (log that it was filled), BCP out (log that it was exported). The objective is to be able to sustain interruptions and pick up where we left off. These processes can take minutes (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). So each thing I do represents a step in the process and each step is logged in a field in a record in SQL server using a datetime. There are so many of these flags that I am trying to standardize the process by building a class that can be instantiated, filled with data and log itself to SQL Server by one thread and be checked by another thread. These flag class instances will be checked by multiple threads, each thread trying to decide whether it should be doing the next step because another thread has finished it's part. IOW if a file has been written to disk, then the next thread will write it to a VM for processing. If it moved to the VM the next thread will watch the VM's output directory for a file to pop out and move it back to a directory on the server. If the file (a couple of files actually) successfully copied back to the server staging then another thread will import it back into a chunk table in an input database. If the file successfully imported then another thread will... In general one thread will "own" the flag and use it to log its status and one other thread will be checking the status of the flag to determine that it can go to work on that work chunk. You get the picture. I am trying to build an entirely asynchronous highly threaded process which exports a huge table into multiple files, processes every file through a third party app and gets the results back into SQL Server. All while logging each and every step so that no piece can possibly be dropped at any stage, even if the server goes down (or the VM goes down). Eventually this process will run on my server 24/7. It has been working for some time but I am getting threading issues, and I need to work on the high level control so that all of the processes can cleanly start up and shut down and every stage can pick back up when the program restarts should a shutdown occur. A single database can be up to a hundred million records (the biggest so far), and the external program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour depending on many different things so that example will take 50 chunks and could take 40 to 50 hours to complete. It takes about 20 processing steps to handle each file from end to end. It needs to just work, and I need to be able to view status in a meaningful way. And I need to process that and a dozen other files every single month, automatically, with no manual intervention required. John W. Colby www.ColbyConsulting.com On 3/25/2011 6:46 PM, Shamil Salakhetdinov wrote: > Hi John -- > > By resource I mean access to a read/write resource - it could be not > only a variable... > > If you need two or more class level variables (fields, properties) > within one class instance to be locked then keep them in a > class/structure and lock that class/structure instance... > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From gustav at cactus.dk Sat Mar 26 07:41:44 2011 From: gustav at cactus.dk (Gustav Brock) Date: Sat, 26 Mar 2011 13:41:44 +0100 Subject: [dba-VB] c# lock() Message-ID: Hi John My first thought was Windows Workflow Foundation(WF) but, as Shamil mentions, it may be overkill. On the other hand it is exactly for controlling scenarios where "if task 1 is done, start task 2 and task 3, wait for task 2 and task 3 to finish, check something, then start task 4, etc.". However, that reminded me about a series of articles about the Task Parallel Library of .Net: http://www.codeproject.com/KB/cs/TPL1.aspx which I found very interesting, though I haven't had any use for it yet. Among other topics it discusses carefully canceling and error handling which I guess is quite important for your purpose. /gustav >>> jwcolby at colbyconsulting.com 26-03-2011 03:57 >>> Shamil, I have processes that log results to flags. For example, make a database (log that it was made), build a table (log that it was built), pull umpteen million records in sorted order (log that it was filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build another chunk table (log that it was filled), BCP out (log that it was exported). The objective is to be able to sustain interruptions and pick up where we left off. These processes can take minutes (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). So each thing I do represents a step in the process and each step is logged in a field in a record in SQL server using a datetime. There are so many of these flags that I am trying to standardize the process by building a class that can be instantiated, filled with data and log itself to SQL Server by one thread and be checked by another thread. These flag class instances will be checked by multiple threads, each thread trying to decide whether it should be doing the next step because another thread has finished it's part. IOW if a file has been written to disk, then the next thread will write it to a VM for processing. If it moved to the VM the next thread will watch the VM's output directory for a file to pop out and move it back to a directory on the server. If the file (a couple of files actually) successfully copied back to the server staging then another thread will import it back into a chunk table in an input database. If the file successfully imported then another thread will... In general one thread will "own" the flag and use it to log its status and one other thread will be checking the status of the flag to determine that it can go to work on that work chunk. You get the picture. I am trying to build an entirely asynchronous highly threaded process which exports a huge table into multiple files, processes every file through a third party app and gets the results back into SQL Server. All while logging each and every step so that no piece can possibly be dropped at any stage, even if the server goes down (or the VM goes down). Eventually this process will run on my server 24/7. It has been working for some time but I am getting threading issues, and I need to work on the high level control so that all of the processes can cleanly start up and shut down and every stage can pick back up when the program restarts should a shutdown occur. A single database can be up to a hundred million records (the biggest so far), and the external program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour depending on many different things so that example will take 50 chunks and could take 40 to 50 hours to complete. It takes about 20 processing steps to handle each file from end to end. It needs to just work, and I need to be able to view status in a meaningful way. And I need to process that and a dozen other files every single month, automatically, with no manual intervention required. John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 26 07:44:54 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 08:44:54 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: Message-ID: <4D8DDFC6.4040004@colbyconsulting.com> Well... I haven't migrated to VS2010 yet. I tried it about 6 months ago and it was universally a pig on my systems. I thought I would wait until MS polished it a bit. John W. Colby www.ColbyConsulting.com On 3/26/2011 8:41 AM, Gustav Brock wrote: > Hi John > > My first thought was Windows Workflow Foundation(WF) but, as Shamil mentions, it may be overkill. On the other hand it is exactly for controlling scenarios where "if task 1 is done, start task 2 and task 3, wait for task 2 and task 3 to finish, check something, then start task 4, etc.". > > However, that reminded me about a series of articles about the Task Parallel Library of .Net: > > http://www.codeproject.com/KB/cs/TPL1.aspx > > which I found very interesting, though I haven't had any use for it yet. Among other topics it discusses carefully canceling and error handling which I guess is quite important for your purpose. > > /gustav > > >>>> jwcolby at colbyconsulting.com 26-03-2011 03:57>>> > Shamil, > > I have processes that log results to flags. For example, make a database (log that it was made), > build a table (log that it was built), pull umpteen million records in sorted order (log that it was > filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build > another chunk table (log that it was filled), BCP out (log that it was exported). The objective is > to be able to sustain interruptions and pick up where we left off. These processes can take minutes > (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). > > So each thing I do represents a step in the process and each step is logged in a field in a record > in SQL server using a datetime. There are so many of these flags that I am trying to standardize > the process by building a class that can be instantiated, filled with data and log itself to SQL > Server by one thread and be checked by another thread. > > These flag class instances will be checked by multiple threads, each thread trying to decide whether > it should be doing the next step because another thread has finished it's part. IOW if a file has > been written to disk, then the next thread will write it to a VM for processing. If it moved to the > VM the next thread will watch the VM's output directory for a file to pop out and move it back to a > directory on the server. If the file (a couple of files actually) successfully copied back to the > server staging then another thread will import it back into a chunk table in an input database. If > the file successfully imported then another thread will... In general one thread will "own" the > flag and use it to log its status and one other thread will be checking the status of the flag to > determine that it can go to work on that work chunk. > > You get the picture. > > I am trying to build an entirely asynchronous highly threaded process which exports a huge table > into multiple files, processes every file through a third party app and gets the results back into > SQL Server. All while logging each and every step so that no piece can possibly be dropped at any > stage, even if the server goes down (or the VM goes down). Eventually this process will run on my > server 24/7. > > It has been working for some time but I am getting threading issues, and I need to work on the high > level control so that all of the processes can cleanly start up and shut down and every stage can > pick back up when the program restarts should a shutdown occur. > > A single database can be up to a hundred million records (the biggest so far), and the external > program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour > depending on many different things so that example will take 50 chunks and could take 40 to 50 hours > to complete. It takes about 20 processing steps to handle each file from end to end. It needs to > just work, and I need to be able to view status in a meaningful way. And I need to process that and > a dozen other files every single month, automatically, with no manual intervention required. > > > > John W. Colby > www.ColbyConsulting.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From gustav at cactus.dk Sat Mar 26 08:08:10 2011 From: gustav at cactus.dk (Gustav Brock) Date: Sat, 26 Mar 2011 14:08:10 +0100 Subject: [dba-VB] c# lock() Message-ID: Hi John Oh, I recall your trouble now that you mention it. Strange, as it runs fine on my antique Pentium 4 IBM IntelliStation with 2 GB ram. However, VS2010 _has_ been polished: The Service Pack 1 was released a few weeks ago. /gustav >>> jwcolby at colbyconsulting.com 26-03-2011 13:44 >>> Well... I haven't migrated to VS2010 yet. I tried it about 6 months ago and it was universally a pig on my systems. I thought I would wait until MS polished it a bit. John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 26 08:26:26 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 09:26:26 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: Message-ID: <4D8DE982.7060707@colbyconsulting.com> When I had my speed issues I Googled around and found tons with speed issues, and tons without. There are entire threads bitterly complaining about what a dog it is. There was no explanations of when or why. Mine would take forever to do anything. I will definitely try again now that SP1 is out. John W. Colby www.ColbyConsulting.com On 3/26/2011 9:08 AM, Gustav Brock wrote: > Hi John > > Oh, I recall your trouble now that you mention it. Strange, as it runs fine on my antique Pentium 4 IBM IntelliStation with 2 GB ram. > However, VS2010 _has_ been polished: The Service Pack 1 was released a few weeks ago. > > /gustav > > >>>> jwcolby at colbyconsulting.com 26-03-2011 13:44>>> > Well... I haven't migrated to VS2010 yet. > > I tried it about 6 months ago and it was universally a pig on my systems. I thought I would wait > until MS polished it a bit. > > John W. Colby > www.ColbyConsulting.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 26 16:53:15 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 17:53:15 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <3F0DA84A905C4947819126F43CFC39DB@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com> <3F0DA84A905C4947819126F43CFC39DB@nant> Message-ID: <4D8E604B.5040001@colbyconsulting.com> Shamil, I am trying to build a base class which contains a set of lock objects and some standard variables. Having defined the lock objects static per your code below, the derived class cannot access the lock objects even though I set them protected: protected static Object LockBln = new Object(); In the derived class when I try to access it: lock(this.LockBln) { } I get a compile error: "Member cannot be accesses with an instance reference; Qualify it with a type name instead." What does this mean? Reading on the internet it says that static properties are shared between all instances of the class. I don't see how I can share a property between 50 instances and then use a lock(MyStaticProperty) {}. I'm so confused... John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - just > make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - otherwise > "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various operations in a > class. > > For example I want to lock a date flag variable while using a property to > either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable but > rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sat Mar 26 17:27:17 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sun, 27 Mar 2011 01:27:17 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8E604B.5040001@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8E604B.5040001@colbyconsulting.com> Message-ID: <00D0A03B172743B6868A8BE736F7E560@nant> John -- Let's define your target (application) system architecture on conceptual level - I guess that using "micro-level locking" you're trying (?) to achieve isn't needed... 1) you have a huge input data set; 2) you split input data set into X chunks; 3) every chunk gets processed using the same several - Y - steps; 4) on completion of step Yn input/semi-processed chunk Xn is submitted to the Y(n+1) processor class; 5) when all X chunks are processed in Y steps they get collected into a final "huge" output set to produce Z reports. That's it from conceptual high level point of view? Something important missing? Too simplified to be true? John, please take into account that I can't reply promptly here - it's weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to sleep, and then I have quite some work to do starting Monday - so I expect this thread will be supported by other AccessD/dba-VB members... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 0:53 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, I am trying to build a base class which contains a set of lock objects and some standard variables. Having defined the lock objects static per your code below, the derived class cannot access the lock objects even though I set them protected: protected static Object LockBln = new Object(); In the derived class when I try to access it: lock(this.LockBln) { } I get a compile error: "Member cannot be accesses with an instance reference; Qualify it with a type name instead." What does this mean? Reading on the internet it says that static properties are shared between all instances of the class. I don't see how I can share a property between 50 instances and then use a lock(MyStaticProperty) {}. I'm so confused... John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - > just make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - > otherwise "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various > operations in a class. > > For example I want to lock a date flag variable while using a property > to either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable > but rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Mar 26 18:31:31 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 19:31:31 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <00D0A03B172743B6868A8BE736F7E560@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant> Message-ID: <4D8E7753.8090006@colbyconsulting.com> I'm just trying to get a handle on locking things (in general). I am seeing things that say "just lock the object itself", and then other things that say "use a static object (the way you were showing me). If locking the object itself works, why do the static thing? John W. Colby www.ColbyConsulting.com On 3/26/2011 6:27 PM, Shamil Salakhetdinov wrote: > John -- > > Let's define your target (application) system architecture on conceptual > level - I guess that using "micro-level locking" you're trying (?) to > achieve isn't needed... > > 1) you have a huge input data set; > 2) you split input data set into X chunks; > 3) every chunk gets processed using the same several - Y - steps; > 4) on completion of step Yn input/semi-processed chunk Xn is submitted to > the Y(n+1) processor class; > 5) when all X chunks are processed in Y steps they get collected into a > final "huge" output set to produce Z reports. > > That's it from conceptual high level point of view? > Something important missing? > Too simplified to be true? > > John, please take into account that I can't reply promptly here - it's > weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to sleep, > and then I have quite some work to do starting Monday - so I expect this > thread will be supported by other AccessD/dba-VB members... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 27 ????? 2011 ?. 0:53 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > I am trying to build a base class which contains a set of lock objects and > some standard variables. > Having defined the lock objects static per your code below, the derived > class cannot access the lock objects even though I set them protected: > > protected static Object LockBln = new Object(); > > In the derived class when I try to access it: > > lock(this.LockBln) > { > } > > I get a compile error: > > "Member cannot be accesses with an instance reference; Qualify it with a > type name instead." > > What does this mean? > > Reading on the internet it says that static properties are shared between > all instances of the class. I don't see how I can share a property between > 50 instances and then use a > lock(MyStaticProperty) {}. > > I'm so confused... > > John W. Colby > www.ColbyConsulting.com > > On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Your sample is a correct usage of locking - just use static thisLock >> variable >> >> private static Object thisLock = new Object() >> >> as non-static variable *is not* thread safe AFAIU. >> >> Yes, such locking can be done for various operations of that class - >> just make sure you'll not get dead-locked... >> Also use *one lock object* to lock *one resource* in a class - >> otherwise "deadlock" will become your "everyday guest"... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 25 ????? 2011 ?. 22:58 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Does this allow using a single "lock object" to lock various >> operations in a class. >> >> For example I want to lock a date flag variable while using a property >> to either get or set the variable. >> >> private Object thisLock = new Object(); >> >> public DateTime pFlagDte { >> get { >> lock (thisLock) >> { >> return dteFlag; >> } >> } >> set >> { >> lock (thisLock) >> { >> dteFlag = value; >> } >> } >> } >> >> The issue is not multiple threads trying to write to the date variable >> but rather one thread trying to read it while another is writing it. >> >> John W. Colby >> www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Mar 26 19:08:10 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sun, 27 Mar 2011 04:08:10 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <00D0A03B172743B6868A8BE736F7E560@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant> Message-ID: <0CD7EF3475CB43C69334942027697B63@nant> John -- I'm falling asleep here now - would that be correct assumption that you can generalize your global task like that - you have, say: - split input data into 50 Chunks; - every chunk should be processed in 10 steps; - every step's average processing time is 100 minutes. Then in total it is needed 50*10*100 = 50,000 minutes or ~833.(3) hours to process all your input data sequentially, chunk after chunk step after step, say, you don't have enough computer power to process some chunks in parallel, and you can't process the whole not split input data as it's too large for your system... But you have "50 small computers" and you can process every of 50 input chunks in parallel - then you'll need just 10*100 = 1,000 minutes (~16.67 hours) to complete the job. I assume that all the chunks processing is independent - then you can: 1) Create Scheduler class instance; 2) Schedule class creates 10 TaskQueue class instances - 1 queue for every processing step type; 3) Scheduler class defines splitting criteria to split input data into 50 chunks; 4) Schedule defines completion criteria - when all 50 chunks get collected in 11th FinalQueue; 5) Schedule seeds the first InputChunkSplitTaskQueue with 50 SplitTasksDescriptors; 4) From time to time - say every half a minute Scheduler class instance creates 500 worker classes in 500 threads (500 worker classes => 50 chunks * 10 steps.), which "hunt for the job" in their dedicate Queues: if they find Job they work on it, if not - they "die"... 5) When a worker class completes processing its job it queues its "task results" into the next step Queue, if a worker class fails then it puts its incomplete task descriptor back into its dedicated queue and "dies", or that could be Scheduler's (or SchedulerAssitant's) task job to find failed workers' "trap" and to resubmit their failed work to the proper queue according to the custom workflow descriptors attached to every chunk... 500 worker classes is an overkill as only 50 worker classes can have job every time but that seems to be an easy "brute force" and "lazy parallel programming" approach - and it should work... Or you can make worker class instances production processes smarter: Scheduler class can start special thread for WorkerClassGenerator instance, which will monitor all the 10 TaskQueues, and if it finds an item in a Queque, it will pick it up, it will create corresponding Worker class in parallel thread and it will pass WorkItem to the Worker class fro processing... When that described above approach will work there then you can easily(?) scale it splitting your (constant size) input data into 100 chunks, and then if every chunk can be processes in half time - in average (50 minutes) - then all job can be completed in 500 minutes = ~8.33(3) hours etc... Please correct me if I oversimplified your application business area... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 27 ????? 2011 ?. 1:27 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] c# lock() John -- Let's define your target (application) system architecture on conceptual level - I guess that using "micro-level locking" you're trying (?) to achieve isn't needed... 1) you have a huge input data set; 2) you split input data set into X chunks; 3) every chunk gets processed using the same several - Y - steps; 4) on completion of step Yn input/semi-processed chunk Xn is submitted to the Y(n+1) processor class; 5) when all X chunks are processed in Y steps they get collected into a final "huge" output set to produce Z reports. That's it from conceptual high level point of view? Something important missing? Too simplified to be true? John, please take into account that I can't reply promptly here - it's weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to sleep, and then I have quite some work to do starting Monday - so I expect this thread will be supported by other AccessD/dba-VB members... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 0:53 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, I am trying to build a base class which contains a set of lock objects and some standard variables. Having defined the lock objects static per your code below, the derived class cannot access the lock objects even though I set them protected: protected static Object LockBln = new Object(); In the derived class when I try to access it: lock(this.LockBln) { } I get a compile error: "Member cannot be accesses with an instance reference; Qualify it with a type name instead." What does this mean? Reading on the internet it says that static properties are shared between all instances of the class. I don't see how I can share a property between 50 instances and then use a lock(MyStaticProperty) {}. I'm so confused... John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - > just make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - > otherwise "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various > operations in a class. > > For example I want to lock a date flag variable while using a property > to either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable > but rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Sat Mar 26 19:38:28 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sun, 27 Mar 2011 04:38:28 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <4D8E7753.8090006@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8E604B.5040001@colbyconsulting.com><00D0A03B172743B6868A8BE736F7E560@nant> <4D8E7753.8090006@colbyconsulting.com> Message-ID: Hi John -- <<< If locking the object itself works, why do the static thing? >>> Static is used to lock shared resources - in your case there are no such resources at all? Log writer is usually a static shared resource... And yes, you can use your locking code to lock parallel access to your class instances' local variables: >> private Object thisLock = new Object(); >> >> public DateTime pFlagDte { >> get { >> lock (thisLock) >> { >> return dteFlag; >> } >> } >> set >> { >> lock (thisLock) >> { >> dteFlag = value; >> } >> } >> } But as I noted I'm not sure if that "micro-level locking" is needed at all (and you mentioned you have several such shared variables within a class instance) - using Queues seems to be more simple and (thread-)safe approach without almost any custom locking schemes.... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 3:32 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() I'm just trying to get a handle on locking things (in general). I am seeing things that say "just lock the object itself", and then other things that say "use a static object (the way you were showing me). If locking the object itself works, why do the static thing? John W. Colby www.ColbyConsulting.com On 3/26/2011 6:27 PM, Shamil Salakhetdinov wrote: > John -- > > Let's define your target (application) system architecture on > conceptual level - I guess that using "micro-level locking" you're > trying (?) to achieve isn't needed... > > 1) you have a huge input data set; > 2) you split input data set into X chunks; > 3) every chunk gets processed using the same several - Y - steps; > 4) on completion of step Yn input/semi-processed chunk Xn is submitted > to the Y(n+1) processor class; > 5) when all X chunks are processed in Y steps they get collected into > a final "huge" output set to produce Z reports. > > That's it from conceptual high level point of view? > Something important missing? > Too simplified to be true? > > John, please take into account that I can't reply promptly here - it's > weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to > sleep, and then I have quite some work to do starting Monday - so I > expect this thread will be supported by other AccessD/dba-VB members... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 27 ????? 2011 ?. 0:53 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > I am trying to build a base class which contains a set of lock objects > and some standard variables. > Having defined the lock objects static per your code below, the > derived class cannot access the lock objects even though I set them protected: > > protected static Object LockBln = new Object(); > > In the derived class when I try to access it: > > lock(this.LockBln) > { > } > > I get a compile error: > > "Member cannot be accesses with an instance reference; Qualify it with > a type name instead." > > What does this mean? > > Reading on the internet it says that static properties are shared > between all instances of the class. I don't see how I can share a > property between 50 instances and then use a > lock(MyStaticProperty) {}. > > I'm so confused... > > John W. Colby > www.ColbyConsulting.com > > On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Your sample is a correct usage of locking - just use static thisLock >> variable >> >> private static Object thisLock = new Object() >> >> as non-static variable *is not* thread safe AFAIU. >> >> Yes, such locking can be done for various operations of that class - >> just make sure you'll not get dead-locked... >> Also use *one lock object* to lock *one resource* in a class - >> otherwise "deadlock" will become your "everyday guest"... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 25 ????? 2011 ?. 22:58 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Does this allow using a single "lock object" to lock various >> operations in a class. >> >> For example I want to lock a date flag variable while using a >> property to either get or set the variable. >> >> private Object thisLock = new Object(); >> >> public DateTime pFlagDte { >> get { >> lock (thisLock) >> { >> return dteFlag; >> } >> } >> set >> { >> lock (thisLock) >> { >> dteFlag = value; >> } >> } >> } >> >> The issue is not multiple threads trying to write to the date >> variable but rather one thread trying to read it while another is writing it. >> >> John W. Colby >> www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 26 23:12:20 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 27 Mar 2011 00:12:20 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <0CD7EF3475CB43C69334942027697B63@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant> <0CD7EF3475CB43C69334942027697B63@nant> Message-ID: <4D8EB924.5080708@colbyconsulting.com> Well... In programming you can do any task 50 different ways. The way I did it is to create a supervisor and ProcessFile table in SQL Server. The supervisor represents a table in SQL Server that contains an address table which needs to be processed. The ProcessFile records are child to tblSupervisor and represent chunks (files). The supervisor record stores all of the high level flags about the database (address table) being processed. The ProcessFile contains all of the flags about each chunk being processed. I have four different high level "stages" as I call them that a supervisor goes through. AccuzipOut - table to csv files in staging out (stage 1), Staging to VM (stage 2), VM to Staging in (stage 3, Staging in to SQL Server (stage 4) 1) Export all chunks (files) to a SQL Out staging directory. Each chunk ends up being a CSV file with PKID, FName, LName, Addr, City, St, Zip5, Zip4. The ProcessFile needs to be sorted by Zip5 / Zip4 order because the third party application that processes these files is immensely faster if the file is presorted. 2) One by one, copy each chunk file to a virtual machine - AccuzipIn directory. The Accuzip virtual machine runs on a Windows 2008 server running Hyper-V manager. I currently run 3 different VMs, only one of which runs Accuzip. For awhile I actually ran 3 VMs running Accuzip, and I am designing such that I can add additional Accuzip VMs in the future. What this means is that each file could go to any valid Accuzip VM, and when the chunk file is placed on a VM it has to log which VM it went to because... 3) The Accuzip software automatically senses and processes the file in the InputTo directory. A file of 2 million records takes about 25 minutes to CASS process locally to the VM, then is uploaded to a server in California for further NCOA processing. The server in California belongs to Accuzip and they stage big files so it can take awhile before it even begins to process. Once it processes, the NCOA server sends a small file back to my VM where Address change information is integrated back in to the local Accuzip database file (foxpro). After that it is exported back to CSV with a bunch of additional fields and is placed in the Accuzip OutputFrom directory along with a PDF which contains the CASS / NCOA statistics. This PDF is a "proof of processing" document that I have to keep forever, and I have to send to the client. I have a Directory Watcher which is watching the OutputFrom directory. When *all* of the resulting Accuzip output files are in the OutputFrom directory, my Directory Watcher event triggers my program to copy all of these files back to the SQL Server In staging directory. 4) Once all of the Accuzip files for a single chunk are back on the SQL Server In directory, my application BCPs the CSV into a input chunk table. From there decisions are made whether I need to import every address back in or only the moves. In any event some or all of the address records (with the additional fields from Accuzip) are pulled into an AZData table back in the database that the address data originally came from. So... 1) A supervisor object creates a clsDbAZOut instance. 2) clsDBAZOut creates an actual database dbXyzAZOut temporary database. 3) clsDBAZOut pulls every record needing to be accuzipped into a table with a super PK, sorted on Zip5/4 4) clsDBAZOut counts the records in tblAZOut, and starts to build clsProcessFile instances which holds the information about the chunk processing. clsProcessFile can write its info back to the ProcessFile records on SQL Server. 5) clsDbAZOut has a thread which builds tblChunk, gets ChunkSize records into the chunk table and BCPs tblChunk to CSV files on a staging out directory. 6) clsDbAZOut BCPs the chunk to file 7) At this point the clsProcessFile is handed off to clsVM. clsSupervisor has a strongly typed dictionary which stores the clsProcessFile instances created by clsDbAZOut. clsSupervisor can read ProcessFile instances from SQL Server into a clsProcessFile factory or clsDbAZOut can use a class factory to create new ones. In all cases they are stored in the dictionary in clsSupervisor. clsProcessfile has flags which document that: a) The chunk table was created b) The chunk table exported to file c) The chunk went to a VM (and which one) d) The chunk file was (eventually) found in the Accuzip OutputFrom directory e) The resulting AZData file (and PDF) were moved from the VM back to SQL ServerIn staging f) The file in SQL Server In staging was pulled back in to a chunk table in a second database dbXyzAZIn temporary database g) That the chunk table was imported back into the original database. As you can see, clsProcessFile is the center of the universe for a single chunk. clsSupervisor is the center of the universe for the entire process. It is a bit difficult to discuss how the entire program works in a single email. A clsManager loads one or a dozen clsSupervisor instances. clsManager loads clsVM which supervises the (a) virtual machine which runs a third party process. clsVM moves files from SQL Staging Out into the VM one chunk at a time, updating clsProcessTable flags. clsVM moves files from the the VM back to the SQL Staging IN, updating flags. The VM may not be available, or the Accuzip program may be down etc. clsSupervisor loads one to N clsProcessTable instances. clsSupervisor loads a clsDBExp and clsDBImp. clsDBExp creates new clsProcessTable instances, one for each chunk it exports - storing them in clsSupervisor. clsDBExp supervises the export process. clsDBImp supervises the import process getting the chunks back into SQL Server into a temp database, and from there into the live database. I am trying to design the system such that it is asynchronous. Stage1, stage 2/3 and stage 4 can all do their thing independently of each other. Each step of each stage is logged (dateTime flags written) so that any piece can just pick up where it left off. That means any Supervisor and any ProcessFile (chunk) can pick up from any point. In the meantime, the clsManager is watching the tblSupervisor for supervisor records ready to process - a thread. clsSupervisor is watching for ProcessTable records and dispatching them as required to the clsDBExp, clsDBImp or clsVM - a thread in each supervisor. clsDBExp is processing chunks, creating CSV files - a thread. clsVM is watching for clsProcessTable instances ready to go to the VM. It is also watching the output directory and sending files back to the SQL Server - a thread and two file watchers (and two events). clsDBImp is watching for clsProcessTable instances ready to import back into SQL Server - a thread. Most of these threads are writing flags (as they finish their specific process) and reading flags in clsProcessTable to decide what chunk has to be processed in which class. Oh, and I have three list controls on a form which the various classes write to to display the status for Stage1, Stage 2/3 and Stage 4. So pretty much a single supervisor is processing at a time, exporting files, sending them to the vm, importing them back in etc. It is possible for an order supervisor to cut in at the head of the line however. Not a simple task co-ordinating all of this stuff. I used to co-ordinate it all manually! ;) John W. Colby www.ColbyConsulting.com On 3/26/2011 8:08 PM, Shamil Salakhetdinov wrote: > John -- > > I'm falling asleep here now - would that be correct assumption that you can > generalize your global task like that - you have, say: > > - split input data into 50 Chunks; > - every chunk should be processed in 10 steps; > - every step's average processing time is 100 minutes. > > Then in total it is needed 50*10*100 = 50,000 minutes or ~833.(3) hours to > process all your input data sequentially, chunk after chunk step after step, > say, you don't have enough computer power to process some chunks in > parallel, and you can't process the whole not split input data as it's too > large for your system... > > But you have "50 small computers" and you can process every of 50 input > chunks in parallel - then you'll need just 10*100 = 1,000 minutes (~16.67 > hours) to complete the job. > > I assume that all the chunks processing is independent - then you can: > > 1) Create Scheduler class instance; > 2) Schedule class creates 10 TaskQueue class instances - 1 queue for every > processing step type; > 3) Scheduler class defines splitting criteria to split input data into 50 > chunks; > 4) Schedule defines completion criteria - when all 50 chunks get collected > in 11th FinalQueue; > 5) Schedule seeds the first InputChunkSplitTaskQueue with 50 > SplitTasksDescriptors; > 4) From time to time - say every half a minute Scheduler class instance > creates 500 worker classes in 500 threads (500 worker classes => 50 chunks * > 10 steps.), which "hunt for the job" in their dedicate Queues: if they find > Job they work on it, if not - they "die"... > 5) When a worker class completes processing its job it queues its "task > results" into the next step Queue, if a worker class fails then it puts its > incomplete task descriptor back into its dedicated queue and "dies", or that > could be Scheduler's (or SchedulerAssitant's) task job to find failed > workers' "trap" and to resubmit their failed work to the proper queue > according to the custom workflow descriptors attached to every chunk... > > 500 worker classes is an overkill as only 50 worker classes can have job > every time but that seems to be an easy "brute force" and "lazy parallel > programming" approach - and it should work... > Or you can make worker class instances production processes smarter: > Scheduler class can start special thread for WorkerClassGenerator instance, > which will monitor all the 10 TaskQueues, and if it finds an item in a > Queque, it will pick it up, it will create corresponding Worker class in > parallel thread and it will pass WorkItem to the Worker class fro > processing... > > When that described above approach will work there then you can easily(?) > scale it splitting your (constant size) input data into 100 chunks, and then > if every chunk can be processes in half time - in average (50 minutes) - > then all job can be completed in 500 minutes = ~8.33(3) hours etc... > > Please correct me if I oversimplified your application business area... > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Sun Mar 27 19:08:25 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 28 Mar 2011 04:08:25 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <4D8EB924.5080708@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> Message-ID: Hi John -- Yes, I see: you tend to build a multi-threaded application/utility system with centralized control - and I tend to build an anarchical free-market model based application/utility system :) Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 8:12 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Well... In programming you can do any task 50 different ways. The way I did it is to create a supervisor and ProcessFile table in SQL Server. The supervisor represents a table in SQL Server that contains an address table which needs to be processed. The ProcessFile records are child to tblSupervisor and represent chunks (files). The supervisor record stores all of the high level flags about the database (address table) being processed. The ProcessFile contains all of the flags about each chunk being processed. I have four different high level "stages" as I call them that a supervisor goes through. AccuzipOut - table to csv files in staging out (stage 1), Staging to VM (stage 2), VM to Staging in (stage 3, Staging in to SQL Server (stage 4) 1) Export all chunks (files) to a SQL Out staging directory. Each chunk ends up being a CSV file with PKID, FName, LName, Addr, City, St, Zip5, Zip4. The ProcessFile needs to be sorted by Zip5 / Zip4 order because the third party application that processes these files is immensely faster if the file is presorted. 2) One by one, copy each chunk file to a virtual machine - AccuzipIn directory. The Accuzip virtual machine runs on a Windows 2008 server running Hyper-V manager. I currently run 3 different VMs, only one of which runs Accuzip. For awhile I actually ran 3 VMs running Accuzip, and I am designing such that I can add additional Accuzip VMs in the future. What this means is that each file could go to any valid Accuzip VM, and when the chunk file is placed on a VM it has to log which VM it went to because... 3) The Accuzip software automatically senses and processes the file in the InputTo directory. A file of 2 million records takes about 25 minutes to CASS process locally to the VM, then is uploaded to a server in California for further NCOA processing. The server in California belongs to Accuzip and they stage big files so it can take awhile before it even begins to process. Once it processes, the NCOA server sends a small file back to my VM where Address change information is integrated back in to the local Accuzip database file (foxpro). After that it is exported back to CSV with a bunch of additional fields and is placed in the Accuzip OutputFrom directory along with a PDF which contains the CASS / NCOA statistics. This PDF is a "proof of processing" document that I have to keep forever, and I have to send to the client. I have a Directory Watcher which is watching the OutputFrom directory. When *all* of the resulting Accuzip output files are in the OutputFrom directory, my Directory Watcher event triggers my program to copy all of these files back to the SQL Server In staging directory. 4) Once all of the Accuzip files for a single chunk are back on the SQL Server In directory, my application BCPs the CSV into a input chunk table. >From there decisions are made whether I need to import every address back in or only the moves. In any event some or all of the address records (with the additional fields from Accuzip) are pulled into an AZData table back in the database that the address data originally came from. So... 1) A supervisor object creates a clsDbAZOut instance. 2) clsDBAZOut creates an actual database dbXyzAZOut temporary database. 3) clsDBAZOut pulls every record needing to be accuzipped into a table with a super PK, sorted on Zip5/4 4) clsDBAZOut counts the records in tblAZOut, and starts to build clsProcessFile instances which holds the information about the chunk processing. clsProcessFile can write its info back to the ProcessFile records on SQL Server. 5) clsDbAZOut has a thread which builds tblChunk, gets ChunkSize records into the chunk table and BCPs tblChunk to CSV files on a staging out directory. 6) clsDbAZOut BCPs the chunk to file 7) At this point the clsProcessFile is handed off to clsVM. clsSupervisor has a strongly typed dictionary which stores the clsProcessFile instances created by clsDbAZOut. clsSupervisor can read ProcessFile instances from SQL Server into a clsProcessFile factory or clsDbAZOut can use a class factory to create new ones. In all cases they are stored in the dictionary in clsSupervisor. clsProcessfile has flags which document that: a) The chunk table was created b) The chunk table exported to file c) The chunk went to a VM (and which one) d) The chunk file was (eventually) found in the Accuzip OutputFrom directory e) The resulting AZData file (and PDF) were moved from the VM back to SQL ServerIn staging f) The file in SQL Server In staging was pulled back in to a chunk table in a second database dbXyzAZIn temporary database g) That the chunk table was imported back into the original database. As you can see, clsProcessFile is the center of the universe for a single chunk. clsSupervisor is the center of the universe for the entire process. It is a bit difficult to discuss how the entire program works in a single email. A clsManager loads one or a dozen clsSupervisor instances. clsManager loads clsVM which supervises the (a) virtual machine which runs a third party process. clsVM moves files from SQL Staging Out into the VM one chunk at a time, updating clsProcessTable flags. clsVM moves files from the the VM back to the SQL Staging IN, updating flags. The VM may not be available, or the Accuzip program may be down etc. clsSupervisor loads one to N clsProcessTable instances. clsSupervisor loads a clsDBExp and clsDBImp. clsDBExp creates new clsProcessTable instances, one for each chunk it exports - storing them in clsSupervisor. clsDBExp supervises the export process. clsDBImp supervises the import process getting the chunks back into SQL Server into a temp database, and from there into the live database. I am trying to design the system such that it is asynchronous. Stage1, stage 2/3 and stage 4 can all do their thing independently of each other. Each step of each stage is logged (dateTime flags written) so that any piece can just pick up where it left off. That means any Supervisor and any ProcessFile (chunk) can pick up from any point. In the meantime, the clsManager is watching the tblSupervisor for supervisor records ready to process - a thread. clsSupervisor is watching for ProcessTable records and dispatching them as required to the clsDBExp, clsDBImp or clsVM - a thread in each supervisor. clsDBExp is processing chunks, creating CSV files - a thread. clsVM is watching for clsProcessTable instances ready to go to the VM. It is also watching the output directory and sending files back to the SQL Server - a thread and two file watchers (and two events). clsDBImp is watching for clsProcessTable instances ready to import back into SQL Server - a thread. Most of these threads are writing flags (as they finish their specific process) and reading flags in clsProcessTable to decide what chunk has to be processed in which class. Oh, and I have three list controls on a form which the various classes write to to display the status for Stage1, Stage 2/3 and Stage 4. So pretty much a single supervisor is processing at a time, exporting files, sending them to the vm, importing them back in etc. It is possible for an order supervisor to cut in at the head of the line however. Not a simple task co-ordinating all of this stuff. I used to co-ordinate it all manually! ;) John W. Colby www.ColbyConsulting.com On 3/26/2011 8:08 PM, Shamil Salakhetdinov wrote: > John -- > > I'm falling asleep here now - would that be correct assumption that > you can generalize your global task like that - you have, say: > > - split input data into 50 Chunks; > - every chunk should be processed in 10 steps; > - every step's average processing time is 100 minutes. > > Then in total it is needed 50*10*100 = 50,000 minutes or ~833.(3) > hours to process all your input data sequentially, chunk after chunk > step after step, say, you don't have enough computer power to process > some chunks in parallel, and you can't process the whole not split > input data as it's too large for your system... > > But you have "50 small computers" and you can process every of 50 > input chunks in parallel - then you'll need just 10*100 = 1,000 > minutes (~16.67 > hours) to complete the job. > > I assume that all the chunks processing is independent - then you can: > > 1) Create Scheduler class instance; > 2) Schedule class creates 10 TaskQueue class instances - 1 queue for > every processing step type; > 3) Scheduler class defines splitting criteria to split input data into > 50 chunks; > 4) Schedule defines completion criteria - when all 50 chunks get > collected in 11th FinalQueue; > 5) Schedule seeds the first InputChunkSplitTaskQueue with 50 > SplitTasksDescriptors; > 4) From time to time - say every half a minute Scheduler class > instance creates 500 worker classes in 500 threads (500 worker classes > => 50 chunks * 10 steps.), which "hunt for the job" in their dedicate > Queues: if they find Job they work on it, if not - they "die"... > 5) When a worker class completes processing its job it queues its > "task results" into the next step Queue, if a worker class fails then > it puts its incomplete task descriptor back into its dedicated queue > and "dies", or that could be Scheduler's (or SchedulerAssitant's) task > job to find failed workers' "trap" and to resubmit their failed work > to the proper queue according to the custom workflow descriptors attached to every chunk... > > 500 worker classes is an overkill as only 50 worker classes can have > job every time but that seems to be an easy "brute force" and "lazy > parallel programming" approach - and it should work... > Or you can make worker class instances production processes smarter: > Scheduler class can start special thread for WorkerClassGenerator > instance, which will monitor all the 10 TaskQueues, and if it finds an > item in a Queque, it will pick it up, it will create corresponding > Worker class in parallel thread and it will pass WorkItem to the > Worker class fro processing... > > When that described above approach will work there then you can > easily(?) scale it splitting your (constant size) input data into 100 > chunks, and then if every chunk can be processes in half time - in > average (50 minutes) - then all job can be completed in 500 minutes = ~8.33(3) hours etc... > > Please correct me if I oversimplified your application business area... > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sun Mar 27 22:01:44 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 27 Mar 2011 23:01:44 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> Message-ID: <4D8FFA18.9020902@colbyconsulting.com> Shamil, > Yes, I see: you tend to build a multi-threaded application/utility system > with centralized control - and I tend to build an anarchical free-market > model based application/utility system :) LOL, I guess. To be quite honest I didn't understand exactly what you were saying. Remember that my C# skills are still quite rudimentary. Accuzip Processing system basically has developed over time. I didn't sit down to develop an app from scratch to do this. I started in late 2004 to just get one single list into SQL server and serve up counts to the client. I did the whole thing manually, building queries to import the data and clean it up using only SQL Server Management Server. When I started I had never touched SQL Server. Then I was asked to research an application for doing CASS http://en.wikipedia.org/wiki/Coding_Accuracy_Support_System and NCOA http://en.wikipedia.org/wiki/National_Change_Of_Address on the addresses in the list. There was only one list in the beginning which I originally labeled "the database from hell". So I researched and discovered a vendor (Accuzip) that would allow us a fixed price license against which I could process as many records as I wanted. All other vendors (that I found) charge on a sliding scale Cents / record kind of thing, and we had a LOT of records - 65 million in that first list. So I started building a system of stored procedures which performed the export out to Accuzip and then import the Accuzipped records back in. It took roughly 25 stored procedures to perform this process. I had a paper list of instructions on how to run the stored procedures, parameters to feed in etc, plus manually copying the files out and back in. I would manually type EXEC commands into a query window reading from my paper instructions. It was *sloooow* and *error prone*. Next came a stored procedure that ran the stored procedures. Etc. Etc. It just morphed over time. Plus I got more lists. The client was so impressed that I managed to do all this that I got actual orders (export the data and send to the clients) rather than just counts, and the business grew. I started using Access ( the tool I knew ) to automate some of it but it was a very poor fit. Since every operation ran in a single thread the entire application would lock up when SQL Server went out for 30 minutes to do some operation. And of course my hardware was underpowered. In the summer of 2009 I decided I really needed to automate this. The business had grown and the client was suddenly 50% of my income and it was a huge struggle to process these lists manually, even using stored procedures of lists of stored procedures. TSQL as a programming environment is crude to say the least. So I went to the local community college in Sept. 2009 and took the first of two available C# courses. By Christmas I was launching in to my automation attempt. Understand my C# skills were rudimentary at best but the language was robust and I could see the power. I really just started building a method of moving the paper list of stored procedures into C#, IOW C# would execute the stored procedures and monitor their completion. I was not focused on a top down "build a system" but rather let's get my manual process automated ASAP. I was overworked, and wasn't able to handle the load. It turns out that C# is very capable in this regard and the program (such as it was) started working and working well. At the end of the second semester (May 2010) I hired Paul, a young college student who was graduating from the community college to come to work for me part time. I started teaching him what I know about general programming and together we worked on this program. And it continued to morph. We started using NLog for logging, Visual SVN for source control etc. At some point I decided to just move the Stored Procedures out of SQL Server and into dynamic code generated directly in C#. Each step had to keep the program running because I actually used the program to fill orders and do counts. And remember our C# skills are still rudimentary. Then we "discovered" threads and soon thereafter events. Last fall I decided to remodel the program into a manager / supervisor / worker model where the manager tells the supervisor to do something, the supervisor tells other workers to do something and basically just check results. In this model, each level uses threads to scan it's tasks (watch flags) and keep its workers busy. So the dbExport class has a thread which build a database, pulls records, builds chunks, exports to disk. That is it's job. When the file is on disk it is done and control for that chunk is passed to another class (the Virtual Machine). The dbExport class uses flags to store each step of the process. The Virtual Machine class picks up the files and pulls them over to the VMs directory, monitors that Accuzip processes them, tracks errors (if any) from Accuzip, and places the resulting files back on a staging directory (as I call it) for another class to take over. The dbImport class is tasked with creating a temp database, pulling each chunk file back in from staging and getting the data back into the original list database. So the manager class (a single instance) loads supervisor classes, one per list database that needs processing. The Manager class also loads the VM class (only one of these at the moment but there could be more, with more licenses from Accuzip). Each supervisor loads a dbExport and dbImport class to process the records in its list. dbExport creates Process records and stores them in SQL Server and creates a process class for each process record. Each process record represents a chunk of data from a specific list database. the process class doesn't really do anything it is just a place for the flags to be loaded / manipulated, and it knows how to persist itself to SQL Server. dbImport basically gets Process class instances which are finished with stage 2/3 (out of Accuzip) and starts importing them back in to SQL Server. My concept was a factory kind of application. There can be none or dozens of databases needing to be processed. SQL Server and my disk system can only efficiently handle one at a time (at any stage) and Accuzip is the biggest bottleneck so the VM class sits in the middle of an export and an import class. Every class has its own job, every class has at least one thread for its use. dbExport has a thread to use in exporting data to a table and another thread to bcp the data out while the next chunk is created in the export database. dbImport has a thread to BCP data back into chunk tables, and another thread to take the chunk tables and merge them back into live data. The VM has a thread to monitor the input directory and another thread to monitor the output directory, moving files into and out of the VM. Every step has to be logged back to SQL Server as it completes, but the clsSupervisor keeps the progress flags in memory as long as a specific list database is being processed, and the supervisor holds a list of process classes which hold all of the flags for the steps for a single chunk. It is a busy system. I did not originally use threads, and everything ran on the main program thread. And I was back to my user interface locking up as long running processes happened. Now with threading, each stage (export / vm / import) has its own list control which is used to display the status of the list database / chunks as they progress through the system. Everything is logged to flags, but also to NLog tables. Each class has threads to keep its own processes fed and running, and each class does its own piece of the whole to process a list table through Accuzip. BTW, there is another entire C# program which processes orders for the client. It is also threaded, and one of its steps is to create a supervisor record to tell the Accuzip Monitor to Accuzip the addresses being processed for the order. Orders are the Accuzip processing which I mentioned have to cut in at the head of the line in Accuzip Monitoring. As you have seen, I could be processing a 65 million record list database when an order comes in. 65 million records is going to take at least 50 hours to process, and the order has to go out ASAP so the Accuzip Monitor (the Manager class actually) has to watch for orders and interrupt the data supervisor to allow the Order supervisor to cut in. Once Accuzip starts a process (chunk) I don't interrupt it, but when that process (chunk) finishes Accuzip, only Order chunks will process until the order is finished. As I said, it is a busy system. And remember that our C# skills are still rudimentary. I started coding in C# in Sept 2009 and started migrating this process to c# about December 2009. By May 2010 when Paul started work I was actually running an Accuzip process and doing orders using C#. Paul is a youngster right out of school. So much of the stuff you talk about goes right over my head Shamil. John W. Colby www.ColbyConsulting.com On 3/27/2011 8:08 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I see: you tend to build a multi-threaded application/utility system > with centralized control - and I tend to build an anarchical free-market > model based application/utility system :) > > Thank you. > > -- > Shamil From jwcolby at colbyconsulting.com Sun Mar 27 22:20:36 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 27 Mar 2011 23:20:36 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> Message-ID: <4D8FFE84.5090609@colbyconsulting.com> OK, so I have explained the process. As you can see I am using threads pretty heavily. In general only one thread writes to any given flag field in a class and in general only one (other) thread reads from that flag. However threads are peeking at flags all the time "checking" whether the chunk (in particular) is ready to proceed to the next stage. This weekend I decided to try a dedicated flag class. My idea is simply that each flag represents a field in a table. Each flag represents a specific data element from a specific record. It is only written once (generally) but potentially read often. If I build a dedicated flag class it can be initialized with the table / field / pkid etc for its specific field / record and then when the thread writes the flag, the flag class can write its value out to the right field / record table. Again it is normally only ever written once. Most of my flags are a date / time that the process step occurred. So I am trying to wrap the whole flag concept into a class to allow me to process (read / write / test) flags in an identical way in all of my process classes. It was this flag class which I was discussing the end of last week. It will be accessed by at least two threads so it needs to be thread safe. John W. Colby www.ColbyConsulting.com On 3/27/2011 8:08 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I see: you tend to build a multi-threaded application/utility system > with centralized control - and I tend to build an anarchical free-market > model based application/utility system :) > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Mon Mar 28 17:39:57 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Tue, 29 Mar 2011 02:39:57 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <4D8FFE84.5090609@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> <4D8FFE84.5090609@colbyconsulting.com> Message-ID: <5FD8C68EEEF142DA8632253753C51323@nant> Hi John -- <<< It was this flag class which I was discussing the end of last week. It will be accessed by at least two threads so it needs to be thread safe. >>> Yes, I see - I do wish it worked efficiently there real soon... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 28 ????? 2011 ?. 7:21 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() OK, so I have explained the process. As you can see I am using threads pretty heavily. In general only one thread writes to any given flag field in a class and in general only one (other) thread reads from that flag. However threads are peeking at flags all the time "checking" whether the chunk (in particular) is ready to proceed to the next stage. This weekend I decided to try a dedicated flag class. My idea is simply that each flag represents a field in a table. Each flag represents a specific data element from a specific record. It is only written once (generally) but potentially read often. If I build a dedicated flag class it can be initialized with the table / field / pkid etc for its specific field / record and then when the thread writes the flag, the flag class can write its value out to the right field / record table. Again it is normally only ever written once. Most of my flags are a date / time that the process step occurred. So I am trying to wrap the whole flag concept into a class to allow me to process (read / write / test) flags in an identical way in all of my process classes. It was this flag class which I was discussing the end of last week. It will be accessed by at least two threads so it needs to be thread safe. John W. Colby www.ColbyConsulting.com On 3/27/2011 8:08 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I see: you tend to build a multi-threaded application/utility > system with centralized control - and I tend to build an anarchical > free-market model based application/utility system :) > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From DWUTKA at Marlow.com Thu Mar 31 10:34:13 2011 From: DWUTKA at Marlow.com (Drew Wutka) Date: Thu, 31 Mar 2011 10:34:13 -0500 Subject: [dba-VB] Most useful droid apps In-Reply-To: References: Message-ID: AndroXplorer Very handy for transferring files around between Droid, Network and SD Card. Drew -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of David McAfee Sent: Saturday, March 19, 2011 3:54 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Most useful droid apps Advanced task killer Weather channel Lookout antivirus AK Notepad Bank of America Acar auto maintenance Barcode scanner Bubble level Bump Droid light Google goggles Google translate IMDB Key ring Print share Trapster Sent from my Droid phone. On Mar 19, 2011 12:52 PM, "jwcolby" wrote: _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com The information contained in this transmission is intended only for the person or entity to which it is addressed and may contain II-VI Proprietary and/or II-VI Business Sensitive material. If you are not the intended recipient, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. You are notified that any review, retransmission, copying, disclosure, dissemination, or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. From jwcolby at colbyconsulting.com Wed Mar 2 20:11:33 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Wed, 02 Mar 2011 21:11:33 -0500 Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews Message-ID: <4D6EF8D5.7060804@colbyconsulting.com> -- John W. Colby www.ColbyConsulting.com http://www.legitreviews.com/news/10164/ From accessd at shaw.ca Thu Mar 3 14:55:06 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Thu, 3 Mar 2011 12:55:06 -0800 Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews In-Reply-To: <4D6EF8D5.7060804@colbyconsulting.com> References: <4D6EF8D5.7060804@colbyconsulting.com> Message-ID: <7AF8F6950EC14685B928DE79F836CB0D@creativesystemdesigns.com> Soon to be into John's office. ;-) Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Wednesday, March 02, 2011 6:12 PM To: Access Developers discussion and problem solving; Sqlserver-Dba; VBA Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews -- John W. Colby www.ColbyConsulting.com http://www.legitreviews.com/news/10164/ _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Thu Mar 3 15:05:20 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 03 Mar 2011 16:05:20 -0500 Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - Legit Reviews In-Reply-To: <7AF8F6950EC14685B928DE79F836CB0D@creativesystemdesigns.com> References: <4D6EF8D5.7060804@colbyconsulting.com> <7AF8F6950EC14685B928DE79F836CB0D@creativesystemdesigns.com> Message-ID: <4D700290.4010308@colbyconsulting.com> I wish. It will most likely be beyond my means for some time. John W. Colby www.ColbyConsulting.com On 3/3/2011 3:55 PM, Jim Lawrence wrote: > Soon to be into John's office. ;-) > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Wednesday, March 02, 2011 6:12 PM > To: Access Developers discussion and problem solving; Sqlserver-Dba; VBA > Subject: [dba-VB] AMD Shows Off 16-Core 'Interlagos' Reference Design - > Legit Reviews > > From michael at ddisolutions.com.au Sun Mar 6 19:26:08 2011 From: michael at ddisolutions.com.au (Michael Maddison) Date: Mon, 7 Mar 2011 12:26:08 +1100 Subject: [dba-VB] Java Help? Message-ID: <99266C61B516644D9727F983FAFAB465046ABC@remote.ddisolutions.com.au> Hi Guys, A while ago I asked about creating a COM dll to be called from Java. Does anyone have the Java skills to see if you can call the dll to test for me? Cheers Michael M From dbdoug at gmail.com Sun Mar 6 21:04:47 2011 From: dbdoug at gmail.com (Doug Steele) Date: Sun, 6 Mar 2011 19:04:47 -0800 Subject: [dba-VB] ASP.Net development Message-ID: Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele From Gustav at cactus.dk Mon Mar 7 04:08:58 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 07 Mar 2011 11:08:58 +0100 Subject: [dba-VB] ASP.Net development Message-ID: Hi Doug I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. /gustav >>> dbdoug at gmail.com 07-03-2011 04:04 >>> Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele From accessd at shaw.ca Mon Mar 7 09:22:24 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Mon, 7 Mar 2011 07:22:24 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: Hi Gustav: Showing my ignorance about Winforms; does Winforms create browser based application or create a Windows based applications that uses a browser? Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Monday, March 07, 2011 2:09 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] ASP.Net development Hi Doug I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. /gustav >>> dbdoug at gmail.com 07-03-2011 04:04 >>> Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From dbdoug at gmail.com Mon Mar 7 09:55:51 2011 From: dbdoug at gmail.com (Doug Steele) Date: Mon, 7 Mar 2011 07:55:51 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: Hi Gustav: Thanks for the reply. If you build a table adapter which consists of data from more than one table (like a one to many relationship) in the design surface, VS2008 will refuse to create any CRUD query code - you have to build it yourself. Maybe you only used one-table adapters? Doug On Mon, Mar 7, 2011 at 2:08 AM, Gustav Brock wrote: > Hi Doug > > I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. > > I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. > You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. > > As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. > > /gustav > > >>>> dbdoug at gmail.com 07-03-2011 04:04 >>> > Hello All: > > A lot of the Microsoft sample code for asp.net data access is built > using strongly typed tableadapters with the .xsd design surface. ?I've > built two apps using these in the last year, but they seem to be kind > of fiddly - they don't work well with multi table update queries, for > instance, and they generate a HUGE amount of background source code. > It is nice to have the strong typing, however. > > I have never found any non-Microsoft sample code which uses this > technique. ?Admittedly, most of the sample code in the wild consists > of short snippets built to illustrate a single technique, not a > complete application. > > Two questions: > 1. Are there developers out there using this approach for real life apps? > 2. If not, what do you recommend? > > > Thanks, > > Doug Steele > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From Gustav at cactus.dk Mon Mar 7 11:04:59 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 07 Mar 2011 18:04:59 +0100 Subject: [dba-VB] ASP.Net development Message-ID: Hi Jim WinForms are true Windows desktop apps (I realise I should have changed the subject line, sorry). If you wish a browser based app, that's ASP.NET or Silverlight if we talk about Visual Studio and dotNet. /gustav >>> accessd at shaw.ca 07-03-2011 16:22 >>> Hi Gustav: Showing my ignorance about Winforms; does Winforms create browser based application or create a Windows based applications that uses a browser? Jim From Gustav at cactus.dk Mon Mar 7 11:17:36 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 07 Mar 2011 18:17:36 +0100 Subject: [dba-VB] ASP.Net development Message-ID: Hi Doug Now I see. Yes, for updates I use one-table tableadapters. Multi-table tableadapters are mainly used as source for reports and list-type forms where nothing is updated - and as source for combo- and listboxes. /gustav >>> dbdoug at gmail.com 07-03-2011 16:55 >>> Hi Gustav: Thanks for the reply. If you build a table adapter which consists of data from more than one table (like a one to many relationship) in the design surface, VS2008 will refuse to create any CRUD query code - you have to build it yourself. Maybe you only used one-table adapters? Doug On Mon, Mar 7, 2011 at 2:08 AM, Gustav Brock wrote: > Hi Doug > > I have built one major business app using these, though not using ASP.NET but WinForms. It has run for nearly two years now solid as a rock. > > I'm not quite sure what you have in mind with "multi table update" queries ... without browsing the entire code I can't recall a situation where more than one table is updated at any one time - maybe because the table schema is heavily normalized - beyond the level where most are satisfied. Further, the app has no one-to-one table joins. > You are right about the strong typing. It is a pleasure to work with, and I know for sure that it prevents many bugs and traps at the earliest stage: while you are coding. This has convinced me that life is too short for the "easy" alternative - falling back to SqlConnection and SqlCommand. > > As for the generated SQL, I have heard too that it should be quite "elaborate" but - to be honest - I don't care as long as it works, and it does for me. That said, for new development we tend to use the Entity Framework. I love the abstraction layer it offers, but I'm aware that it may be a bit slower and may have other drawbacks. But, as it often is the case, it may depend on your actual data. > > /gustav > > >>>> dbdoug at gmail.com 07-03-2011 04:04 >>> > Hello All: > > A lot of the Microsoft sample code for asp.net data access is built > using strongly typed tableadapters with the .xsd design surface. I've > built two apps using these in the last year, but they seem to be kind > of fiddly - they don't work well with multi table update queries, for > instance, and they generate a HUGE amount of background source code. > It is nice to have the strong typing, however. > > I have never found any non-Microsoft sample code which uses this > technique. Admittedly, most of the sample code in the wild consists > of short snippets built to illustrate a single technique, not a > complete application. > > Two questions: > 1. Are there developers out there using this approach for real life apps? > 2. If not, what do you recommend? > > > Thanks, > > Doug Steele From accessd at shaw.ca Mon Mar 7 11:52:58 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Mon, 7 Mar 2011 09:52:58 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: <9F8E42F58F6F45B18FBDE74842047BF9@creativesystemdesigns.com> Hi Gustav: Thanks for the information. Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Monday, March 07, 2011 9:05 AM To: dba-vb at databaseadvisors.com Subject: Re: [dba-VB] ASP.Net development Hi Jim WinForms are true Windows desktop apps (I realise I should have changed the subject line, sorry). If you wish a browser based app, that's ASP.NET or Silverlight if we talk about Visual Studio and dotNet. /gustav >>> accessd at shaw.ca 07-03-2011 16:22 >>> Hi Gustav: Showing my ignorance about Winforms; does Winforms create browser based application or create a Windows based applications that uses a browser? Jim _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Mon Mar 7 12:45:14 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 7 Mar 2011 21:45:14 +0300 Subject: [dba-VB] ASP.Net development In-Reply-To: References: Message-ID: <5C52BAB72E1246CFBBFB629233371E72@nant> Hi Doug -- I think I have seen an ASP.NET book with sources where strongly typed datasets were used but I can't find it now. But maybe that was this book: Pro ASP.NET 2.0 E-Commerce in C# 2005 (Expert's Voice in .NET) http://www.amazon.com/ASP-NET-Commerce-2005-Experts-Voice/dp/1590597249/ref= pd_sim_b_8 It does have complete source code and the DAL code does use ADO.NET DataSets but not strongly typed. I have got programmed rather large ASP.NET application - more than 150 webforms with I suppose more than 70-80 heavily related MS SQL tables with millions rows in some of tables, hundreds SPs, ... several thousands C# custom classes, ORM layer etc. and this web application does work well forth year in row with a few support work - the latter mainly being adding new functionality. I have used asp.net development approach described in this book: ASP.NET 2.0 Website Programming: Problem - Design - Solution (Programmer to Programmer) http://www.amazon.com/ASP-NET-2-0-Website-Programming-Programmer/dp/07645846 42/ref=sr_1_2?s=books&ie=UTF8&qid=1299518125&sr=1-2 BTW, the same approach is used in DotNetNuke (but I didn't know that when I have got started to develop my customer web application I mentioned above.)) I have had to develop custom ORM mapping + SQL code generator to implement this approach. But it should be possible to use some existing code generators. This approach looks "low level" and it needs quite some coding (code generators help here) but good news is that it will never get outdated. And here is a book on how to use LINQ 2 SQL for ASP.NET applications (I haven't read and I haven't tried this approach) for the same sample ASP.NET application, which is used in the previously referred book: LINQ-ifying TheBeerHouse: An N-Tier LINQ Web Application with ASP.NET 2.0 Website Programming Problem Design Solution (Wrox Briefs) http://www.amazon.com/LINQ-ifying-TheBeerHouse-Application-Programming-Solut ion/dp/B0026A6AQU/ref=sr_1_106?s=books&ie=UTF8&qid=1299518458&sr=1-106 The most modern approach is to use ADO.NET Entity Framework for DAL as Gustav noted, and when ADO.NET EF is used then it should be possible to map logical/business layer objects to the custom stored procedures, which can insert, update, delete, select data from one or database tables... I haven't yet seen any books where ADO.NET Entity Framework is used with classical WebForms based ASP.NET applications - anybody? Some more information on ADO.NET EF and ASP.NET: - here seems to be a good book on ADO.NET EF: Entity Framework 4.0 Recipes: A Problem-Solution Approach (Expert's Voice in .NET) http://www.amazon.com/Entity-Framework-4-0-Recipes-Problem-Solution/dp/14302 27036/ref=sr_1_168?s=books&ie=UTF8&qid=1299519467&sr=1-168 - and this one "opens secretes" on how to build ultra-fast and robust asp.net applications: Ultra-Fast ASP.NET: Build Ultra-Fast and Ultra-Scalable web sites using ASP.NET and SQL Server [Paperback] http://www.amazon.com/Ultra-Fast-ASP-NET-Build-Ultra-Scalable-Server/dp/1430 223839/ref=sr_1_121?s=books&ie=UTF8&qid=1299518863&sr=1-121 Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Doug Steele Sent: 7 ????? 2011 ?. 6:05 To: dba-VB at databaseadvisors.com Subject: [dba-VB] ASP.Net development Hello All: A lot of the Microsoft sample code for asp.net data access is built using strongly typed tableadapters with the .xsd design surface. I've built two apps using these in the last year, but they seem to be kind of fiddly - they don't work well with multi table update queries, for instance, and they generate a HUGE amount of background source code. It is nice to have the strong typing, however. I have never found any non-Microsoft sample code which uses this technique. Admittedly, most of the sample code in the wild consists of short snippets built to illustrate a single technique, not a complete application. Two questions: 1. Are there developers out there using this approach for real life apps? 2. If not, what do you recommend? Thanks, Doug Steele _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From dbdoug at gmail.com Mon Mar 7 23:34:51 2011 From: dbdoug at gmail.com (Doug Steele) Date: Mon, 7 Mar 2011 21:34:51 -0800 Subject: [dba-VB] ASP.Net development In-Reply-To: <5C52BAB72E1246CFBBFB629233371E72@nant> References: <5C52BAB72E1246CFBBFB629233371E72@nant> Message-ID: Hi Shamil: Thanks very much for your reply. I'll be checking your references out.... Doug On Mon, Mar 7, 2011 at 10:45 AM, Shamil Salakhetdinov wrote: > Hi Doug -- > > I think I have seen an ASP.NET book with sources where strongly typed > datasets were used but I can't find it now. > But maybe that was this book: > > Pro ASP.NET 2.0 E-Commerce in C# 2005 (Expert's Voice in .NET) > http://www.amazon.com/ASP-NET-Commerce-2005-Experts-Voice/dp/1590597249/ref= > pd_sim_b_8 > > It does have complete source code and the DAL code does use ADO.NET DataSets > but not strongly typed. > > I have got programmed rather large ASP.NET application - more than 150 > webforms with I suppose more than 70-80 heavily related MS SQL tables with > millions rows in some of tables, hundreds SPs, ... several thousands C# > custom classes, ORM layer etc. and this web application does work well forth > year in row with a few support work - the latter mainly being adding new > functionality. I have used asp.net development approach described in this > book: > > ASP.NET 2.0 Website Programming: Problem - Design - Solution (Programmer to > Programmer) > http://www.amazon.com/ASP-NET-2-0-Website-Programming-Programmer/dp/07645846 > 42/ref=sr_1_2?s=books&ie=UTF8&qid=1299518125&sr=1-2 > > BTW, the same approach is used in DotNetNuke (but I didn't know that when I > have got started to develop my customer web application I mentioned above.)) > > I have had to develop custom ORM mapping + SQL code generator to implement > this approach. > But it should be possible to use some existing code generators. > > This approach looks "low level" and it needs quite some coding (code > generators help here) but good news is that it will never get outdated. > > And here is a book on how to use LINQ 2 SQL for ASP.NET applications (I > haven't read and I haven't tried this approach) for the same sample ASP.NET > application, which is used in the previously referred book: > > LINQ-ifying TheBeerHouse: An N-Tier LINQ Web Application with ASP.NET 2.0 > Website Programming Problem Design Solution (Wrox Briefs) > http://www.amazon.com/LINQ-ifying-TheBeerHouse-Application-Programming-Solut > ion/dp/B0026A6AQU/ref=sr_1_106?s=books&ie=UTF8&qid=1299518458&sr=1-106 > > The most modern approach is to use ADO.NET Entity Framework for DAL as > Gustav noted, and when ADO.NET EF is used then it should be possible to map > logical/business layer objects to the custom stored procedures, which can > insert, update, delete, select data from one or database tables... > > I haven't yet seen any books where ADO.NET Entity Framework is used with > classical WebForms based ASP.NET applications - anybody? > > > Some more information on ADO.NET EF and ASP.NET: > > - here seems to be a good book on ADO.NET EF: > > Entity Framework 4.0 Recipes: A Problem-Solution Approach (Expert's Voice in > .NET) > http://www.amazon.com/Entity-Framework-4-0-Recipes-Problem-Solution/dp/14302 > 27036/ref=sr_1_168?s=books&ie=UTF8&qid=1299519467&sr=1-168 > > - and this one "opens secretes" on how to build ultra-fast and robust > asp.net applications: > > Ultra-Fast ASP.NET: Build Ultra-Fast and Ultra-Scalable web sites using > ASP.NET and SQL Server [Paperback] > http://www.amazon.com/Ultra-Fast-ASP-NET-Build-Ultra-Scalable-Server/dp/1430 > 223839/ref=sr_1_121?s=books&ie=UTF8&qid=1299518863&sr=1-121 > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Doug Steele > Sent: 7 ????? 2011 ?. 6:05 > To: dba-VB at databaseadvisors.com > Subject: [dba-VB] ASP.Net development > > Hello All: > > A lot of the Microsoft sample code for asp.net data access is built using > strongly typed tableadapters with the .xsd design surface. ?I've built two > apps using these in the last year, but they seem to be kind of fiddly - they > don't work well with multi table update queries, for instance, and they > generate a HUGE amount of background source code. > It is nice to have the strong typing, however. > > I have never found any non-Microsoft sample code which uses this technique. > Admittedly, most of the sample code in the wild consists of short snippets > built to illustrate a single technique, not a complete application. > > Two questions: > 1. Are there developers out there using this approach for real life apps? > 2. If not, what do you recommend? > > > Thanks, > > Doug Steele > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From raibeart at gmail.com Tue Mar 8 11:44:25 2011 From: raibeart at gmail.com (Robert Stewart) Date: Tue, 08 Mar 2011 11:44:25 -0600 Subject: [dba-VB] I'm getting nowhere In-Reply-To: References: Message-ID: <4d766a81.cac4e60a.3ab2.255f@mx.google.com> John, Logins: If you are in mixed mode, which for your application you should be, every "group" will have it's own login with a password. User: References a login. A user is given rights to a database and specific objects in a database. Roles: Generally, you can assign DataReader and DataWriter roles to a User in a database to give them CRUD rights to tables. You will need to GRANT EXECUTE rights to stored procedures. Does that help? Robert At 09:22 AM 3/7/2011, you wrote: >From: dba-vb-bounces at databaseadvisors.com >[mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >Sent: Saturday, February 19, 2011 2:54 PM >To: Sqlserver-Dba; VBA; Access Developers discussion and problem solving >Subject: [dba-VB] I'm getting nowhere > >I am getting nowhere on understanding SQL Server security. Microsoft >provides us with SQL Server Express which implies that joe blow (me) is >going to install / maintain it. > >I am not a SQL Server Admin and I cannot afford to spend the time to be one. > >Google is my friend. BOL is not. > >Except that Google is taking me to these places where I am expected to >already know how this stuff works, and then wants to make me a *better* >administrator. Which of course is useless because I am not an administrator >at all. > >OTOH I am not stupid. If I could find something that started at the "This >is SQL Server security" >basics I could learn this stuff. Before anyone says "RTFM (BOL)" let me >simply say, "not happening". I have tried BOL and it simply sucks for my >level of expertise (my opinion of course). > If that is your advice, simply stay out of this thread. Thanks! > >So... my needs: > >I need to set up several SQL Server databases for use by different, very >small groups (5-20 people) of entirely unrelated people. What I mean by >that is that each DB is for a different "company" if you will. I need to >access these databases from C#. I understand the group / user paradigm. I >would like to create groups and users. Specific groups can do specific >things in the database, some can see data but not modify it. Some can add >records in specific tables but not others. Some can run reports (view). > >I do *NOT* want to create windows level groups and users if I can avoid it. >These are people that I do not necessarily know and I do not want to give >them any rights at the machine level, and I prefer to not maintain such >lists at the machine level. > >Unfortunately SQL Server does not seem to model Groups / users. I go into >SQL Server and see a security tab. It has "logins". Is that a user? A >specific ability to log in with a password? To what? The server itself? A >specific database? Groups of databases? > >I see "roles" but these appear to be aimed at the server and none of these >people are going to be doing anything at the server level. > >Can I safely ignore everything under the server security tab? > >I go to a database and I see a security tab. It has users and roles. >Hmm... better (I would think). I would like to add users "under" the >specific database that the user will access. > >So I try to add a new user but I do not see anywhere to require a password. >Hmmm... > >I go into roles and I do not see any predefined role that looks like it >would be useful to me in meeting my needs described above. If I look at >"add new role" it asks for a password. The User / group model does nto >assign passwords at the group level which implies that a role is not a group >at the user / group paradigm. > >Is it just me, or is SQL Server security just... different? Am I correct in >assuming that it doesn't implement a user / group paradigm? > >And more importantly, where can I go to get a plain, simple, English >description of how this mess works? > >And please excuse the tone that results from my frustration. The only help >documents that I have found (and I have extensive lists of bookmarked web >pages) so far assume that I am an administrator. > I am not, and cannot afford to become one. And yet MS pushes SQL Express >as if I (non-admin) should be able to use this as a data store pool. > >Help! > >-- >John W. Colby >www.ColbyConsulting.com >_______________________________________________ From jwcolby at colbyconsulting.com Thu Mar 10 07:12:44 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 08:12:44 -0500 Subject: [dba-VB] c# lock() Message-ID: <4D78CE4C.5000308@colbyconsulting.com> I am using multi-threading and need to lock list objects so that two different threads can access them. Do I even need to lock the list inside of the property as follows: public Dictionary pLstFiles { get { lock (lstFiles) { return lstFiles; } } } or only when I actually access the list? If I create a property to return the list (without locking it in the property) public Dictionary pLstFiles { get { return lstFiles; } } and I then get a pointer to the list using that property Dictionary LstFiles; LstFiles = SomeObj.pLstFiles; and I then lock the pointer while using it lock(LstFiles) { do stuff to list here... } Does that in fact lock the list to other objects doing the same thing (getting the list through that property)? IOW is the lock attribute affixed to the list at the list itself so that anything trying to use the list anywhere in the program can see that the list is locked? -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Thu Mar 10 10:29:39 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 11:29:39 -0500 Subject: [dba-VB] split an SVN database Message-ID: <4D78FC73.6000202@colbyconsulting.com> I use SVN. I am looking at doing a major split of a project into two projects. I want to leave the existing solution until I have one half of the current project carved out, debugged and running and in it's own SVN database. However I don't think I want to carry along the baggage of the old revisions into the new database. How do I go about this? -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Thu Mar 10 14:37:01 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 15:37:01 -0500 Subject: [dba-VB] c# - controlling the run state / status of your threaded classes Message-ID: <4D79366D.8060905@colbyconsulting.com> As I write classes that consume threads, I need to be able to start and stop these threads and the class that consumes the threads. For example, I open a form and load a class instance that starts performing some process - exporting sets of records to csv files for example. Once that class is loaded and processing, I need to be able to ask that class to stop exporting files. However there may be a time lag between asking and the currently processing file finishing it's export. That is precisely the purpose of a thread - tell it to go and do something else. So (for example) I have a check box that I check that tells the class to start exporting. When it starts it sets a flag saying that it is running and places text into the check box label saying that it is exporting (busy). If I then uncheck the check box I am telling the class to stop exporting. The class is monitoring the check box and will not process the next file but it can't just exit. So it needs to set a flag saying that it is stopping and display "stopping" on the check box label. When it finishes exporting the file, it sets a stopped flag and says displays "terminated" in the check box label. So we have a couple of "commands" (start / stop) and a few statuses (stopped, stopping, running). I would like to "standardize" this into a class so that any class which needs this interface does it in the same way. Some controlling object can tell the object to start and stop, and the class can tell the controlling object that is is stopped, stopping or running. this class does not6 need to do any updating of labels or anything like that, just hold the variables, properties, methods and code to perform this in a standard way. Is anyone doing this? Have existing code you would like to share? Have existing experience trying to do this you would like to share? -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Thu Mar 10 15:35:24 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 00:35:24 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D78CE4C.5000308@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com> Message-ID: <3693FCAC03E94BCA9C150F6E7AFE4A71@nant> Hi John -- <<< Does that in fact lock the list to other objects doing the same thing (getting the list through that property)? >>> You have to lock/synchronize/serialize not the list itself but simultaneous access to the list from different threads. You'd better use special static object variable to implement locking. I will post sample code in my following posting here... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 16:13 To: VBA Subject: [dba-VB] c# lock() I am using multi-threading and need to lock list objects so that two different threads can access them. Do I even need to lock the list inside of the property as follows: public Dictionary pLstFiles { get { lock (lstFiles) { return lstFiles; } } } or only when I actually access the list? If I create a property to return the list (without locking it in the property) public Dictionary pLstFiles { get { return lstFiles; } } and I then get a pointer to the list using that property Dictionary LstFiles; LstFiles = SomeObj.pLstFiles; and I then lock the pointer while using it lock(LstFiles) { do stuff to list here... } Does that in fact lock the list to other objects doing the same thing (getting the list through that property)? IOW is the lock attribute affixed to the list at the list itself so that anything trying to use the list anywhere in the program can see that the list is locked? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Thu Mar 10 15:36:58 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 00:36:58 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D78CE4C.5000308@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com> Message-ID: <4889C56E6FF246D6AB908FEFD1069A68@nant> Hi John -- Here is sample code I promised to post. using System; using System.Collections.Generic; namespace ConsoleApplication1 { class Log { public static bool On { get; set; } public static void WriteLine(string message) { WriteLine(message, false); } public static void WriteLine(string message, bool forceLogging) { if (Log.On || forceLogging) System.Console.WriteLine(message); } } class MyObject { static int _newId; public MyObject() { _id = System.Threading.Interlocked.Increment(ref _newId); _timeStamp = DateTime.Now; } int _id; public int ID { get { return _id; } set { _id = value; } } DateTime _timeStamp; public DateTime TimeStamp { get { return _timeStamp; } set { _timeStamp = value; } } } class MyObjectsStore { private List _keys = new List(); private Dictionary _myObjects = new Dictionary(); public MyObject this[int key] { get { MyObject myObject = null; lock (_myObjectsLocker) { _myObjects.TryGetValue(key, out myObject); } return myObject; } } public int Count { get { lock (_myObjectsLocker) { return _myObjects.Count; } } } private static object _myObjectsLocker = new object(); public void Add(MyObject myObject) { lock (_myObjectsLocker) { _myObjects.Add(myObject.ID, myObject); _keys.Add(myObject.ID); } } public MyObject Remove(int id) { MyObject myobject = null; lock (_myObjectsLocker) { myobject = _myObjects[id]; _myObjects.Remove(id); } return myobject; } static Random _rnd = new Random(); public MyObject RemoveRandomObject() { MyObject myobject = null; int index = -1; int key = -1; try { lock (_myObjectsLocker) { index = _rnd.Next(_keys.Count - 1); key = _keys[index]; myobject = Remove(key); _keys.Remove(key); } } catch (Exception ex) { Log.WriteLine(string.Format("[{0}]: RemoveRandomObject: {1}, index = {2}, key = {3}", System.Threading.Thread.CurrentThread.ManagedThreadId, ex.Message, index , key)); } return myobject; } } class Program { public const int OBJECTS_PER_THREAD_COUNT = 7; public const int REMOVAL_MAX_RETRIES_COUNT = 5; public static void AddObjectsTest() { for (int i = 1; i <= OBJECTS_PER_THREAD_COUNT; i++) { MyObject myObject = new MyObject(); _store.Add(myObject); Log.WriteLine(string.Format("[{0}]: Object Added - ID = {1}, TS = {2}", System.Threading.Thread.CurrentThread.ManagedThreadId, myObject.ID, myObject.TimeStamp.Ticks)); System.Threading.Thread.Sleep(2); System.Threading.Interlocked.Increment(ref _createdObjectsCount); } System.Threading.Interlocked.Decrement(ref _threadsCount); } public static void RemoveObjectsTest() { for (int i = 1; i <= OBJECTS_PER_THREAD_COUNT; i++) { int retryCount = REMOVAL_MAX_RETRIES_COUNT; retry: MyObject myObject = _store.RemoveRandomObject(); if (myObject != null) { Log.WriteLine(string.Format("[{0}]: Object Removed - ID = {1}, TS = {2}", System.Threading.Thread.CurrentThread.ManagedThreadId, myObject.ID, myObject.TimeStamp.Ticks)); System.Threading.Thread.Sleep(2); } System.Threading.Thread.Sleep(2); if (myObject == null && --retryCount >= 0) { Log.WriteLine(string.Format("[{0}]: Object Removal failed, retry attempt# {1}", System.Threading.Thread.CurrentThread.ManagedThreadId, REMOVAL_MAX_RETRIES_COUNT-retryCount)); System.Threading.Interlocked.Increment(ref _totalRetriesCount); goto retry; } else if (myObject == null) { Log.WriteLine(string.Format("[{0}]: Object Removal failed", System.Threading.Thread.CurrentThread.ManagedThreadId)); System.Threading.Interlocked.Increment(ref _failedObjectRemovalsCount); } else System.Threading.Interlocked.Increment(ref _removedObjectsCount); } System.Threading.Interlocked.Decrement(ref _threadsCount); } static int _createdObjectsCount; static int _removedObjectsCount; static int _totalRetriesCount; static int _failedObjectRemovalsCount; static int _threadsCount; static MyObjectsStore _store; const int THREADS_COUNT = 20; //000; static void Main(string[] args) { Log.On = true; Log.WriteLine(string.Format("{0}: Test Started.", DateTime.Now)); try { Log.On = false; _store = new MyObjectsStore(); for (int i = 1; i <= THREADS_COUNT; i++) { if (i % 2 == 1) new System.Threading.Thread( new System.Threading.ThreadStart(AddObjectsTest)) .Start(); else new System.Threading.Thread( new System.Threading.ThreadStart(RemoveObjectsTest)) .Start(); _threadsCount++; } while (_threadsCount > 0) System.Threading.Thread.Sleep(100); System.Threading.Thread.Sleep(100); Log.On = true; Log.WriteLine(string.Format("Total Threads Count = {0}", THREADS_COUNT)); Log.WriteLine(string.Format("Store has {0} objects, Created = {1}, Removed = {2}", _store.Count, _createdObjectsCount, _removedObjectsCount)); Log.WriteLine(string.Format("RemovalRetriesCount = {0}, FailedRemovalsCount = {1}", _totalRetriesCount, _failedObjectRemovalsCount)); } catch (Exception ex) { Log.On = true; Log.WriteLine(string.Format("Program.Main: " + ex.Message)); } finally { Log.On = true; } Log.WriteLine(string.Format("{0}: Test Finished.", DateTime.Now)); } } } Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 16:13 To: VBA Subject: [dba-VB] c# lock() <<< snip >>> From shamil at smsconsulting.spb.ru Thu Mar 10 15:38:46 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 00:38:46 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D78CE4C.5000308@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com> Message-ID: <50A8FE7A468B41D48B9CC4F729CB02E3@nant> Hi John -- Here are my test results for sample code I've just posted (hopefully it get through AccessD - it was a bit lengthy) //11/03/2011 0:32:50: Test Started. //Total Threads Count = 20 //Store has 0 objects, Created = 70, Removed = 70 //RemovalRetriesCount = 0, FailedRemovalsCount = 0 //11/03/2011 0:32:50: Test Finished. //11/03/2011 0:32:25: Test Started. //Total Threads Count = 200 //Store has 0 objects, Created = 700, Removed = 700 //RemovalRetriesCount = 0, FailedRemovalsCount = 0 //11/03/2011 0:32:25: Test Finished. //11/03/2011 0:31:32: Test Started. //Total Threads Count = 20000 //Store has 19 objects, Created = 70000, Removed = 69981 //RemovalRetriesCount = 295, FailedRemovalsCount = 19 //11/03/2011 0:31:43: Test Finished. //11/03/2011 0:33:32: Test Started. //[3]: Object Added - ID = 1, TS = 634354004122968750 //[5]: Object Added - ID = 2, TS = 634354004122968750 //[3]: Object Added - ID = 3, TS = 634354004122968750 //[4]: Object Removed - ID = 2, TS = 634354004122968750 //[5]: Object Added - ID = 4, TS = 634354004122968750 //[6]: Object Removed - ID = 3, TS = 634354004122968750 //[7]: Object Added - ID = 5, TS = 634354004122968750 //[8]: Object Removed - ID = 4, TS = 634354004122968750 //[9]: Object Added - ID = 6, TS = 634354004122968750 //[10]: Object Removed - ID = 1, TS = 634354004122968750 //[11]: Object Added - ID = 7, TS = 634354004122968750 //[3]: Object Added - ID = 8, TS = 634354004122968750 //[12]: Object Removed - ID = 7, TS = 634354004122968750 //[5]: Object Added - ID = 9, TS = 634354004122968750 //[7]: Object Added - ID = 11, TS = 634354004122968750 //[9]: Object Added - ID = 10, TS = 634354004122968750 //[4]: Object Removed - ID = 6, TS = 634354004122968750 //[11]: Object Added - ID = 12, TS = 634354004122968750 //[3]: Object Added - ID = 13, TS = 634354004122968750 //[7]: Object Added - ID = 16, TS = 634354004123125000 //[6]: Object Removed - ID = 13, TS = 634354004122968750 //[11]: Object Added - ID = 17, TS = 634354004123125000 //[9]: Object Added - ID = 15, TS = 634354004123125000 //[4]: Object Removed - ID = 16, TS = 634354004123125000 //[8]: Object Removed - ID = 11, TS = 634354004122968750 //[10]: Object Removed - ID = 8, TS = 634354004122968750 //[5]: Object Added - ID = 14, TS = 634354004122968750 //[11]: Object Added - ID = 21, TS = 634354004123125000 //[3]: Object Added - ID = 18, TS = 634354004123125000 //[7]: Object Added - ID = 19, TS = 634354004123125000 //[10]: Object Removed - ID = 15, TS = 634354004123125000 //[6]: Object Removed - ID = 5, TS = 634354004122968750 //[4]: Object Removed - ID = 19, TS = 634354004123125000 //[8]: Object Removed - ID = 9, TS = 634354004122968750 //[10]: Object Removed - ID = 21, TS = 634354004123125000 //[6]: Object Removed - ID = 17, TS = 634354004123125000 //[8]: Object Removed - ID = 12, TS = 634354004122968750 //[4]: Object Removed - ID = 22, TS = 634354004123281250 //[12]: Object Removed - ID = 10, TS = 634354004122968750 //[6]: Object Removed - ID = 18, TS = 634354004123125000 //[4]: Object Removed - ID = 24, TS = 634354004123437500 //[8]: Object Removed - ID = 20, TS = 634354004123125000 //[6]: Object Removed - ID = 23, TS = 634354004123437500 //[12]: Object Removed - ID = 25, TS = 634354004123437500 //[10]: Object Removed - ID = 14, TS = 634354004122968750 //[9]: Object Added - ID = 20, TS = 634354004123125000 //[5]: Object Added - ID = 22, TS = 634354004123281250 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[10]: Object Removed - ID = 26, TS = 634354004124062500 //[6]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 1 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[10]: Object Removed - ID = 27, TS = 634354004124062500 //[3]: Object Added - ID = 24, TS = 634354004123437500 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[6]: Object Removal failed, retry attempt# 1 //[6]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 2 //[4]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[8]: Object Removal failed, retry attempt# 1 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[11]: Object Added - ID = 23, TS = 634354004123437500 //[7]: Object Added - ID = 25, TS = 634354004123437500 //[6]: Object Removal failed, retry attempt# 2 //[12]: Object Removed - ID = 28, TS = 634354004124531250 //[4]: Object Removal failed, retry attempt# 1 //[8]: Object Removal failed, retry attempt# 2 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[6]: Object Removed - ID = 29, TS = 634354004125000000 //[4]: Object Removed - ID = 30, TS = 634354004125000000 //[8]: Object Removal failed, retry attempt# 3 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 1 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[8]: Object Removal failed, retry attempt# 4 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 2 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[3]: Object Added - ID = 28, TS = 634354004124531250 //[12]: Object Removal failed, retry attempt# 3 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[8]: Object Removal failed, retry attempt# 5 //[8]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[12]: Object Removal failed, retry attempt# 4 //[7]: Object Added - ID = 30, TS = 634354004125000000 //[12]: RemoveRandomObject: 'maxValue' must be greater than zero. //Parameter name: maxValue, index = -1, key = -1 //[5]: Object Added - ID = 27, TS = 634354004124062500 //[9]: Object Added - ID = 26, TS = 634354004124062500 //[12]: Object Removal failed, retry attempt# 5 //[12]: Object Removed - ID = 31, TS = 634354004126093750 //[8]: Object Removal failed //[8]: Object Removed - ID = 32, TS = 634354004126250000 //[9]: Object Added - ID = 33, TS = 634354004126250000 //[12]: Object Removed - ID = 33, TS = 634354004126250000 //[11]: Object Added - ID = 29, TS = 634354004125000000 //[9]: Object Added - ID = 34, TS = 634354004126406250 //[12]: Object Removed - ID = 34, TS = 634354004126406250 //[7]: Object Added - ID = 31, TS = 634354004126093750 //[5]: Object Added - ID = 32, TS = 634354004126250000 //[11]: Object Added - ID = 35, TS = 634354004126406250 //Total Threads Count = 10 //Store has 1 objects, Created = 35, Removed = 34 //RemovalRetriesCount = 15, FailedRemovalsCount = 1 //11/03/2011 0:33:32: Test Finished. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 16:13 To: VBA Subject: [dba-VB] c# lock() <<< snip >>> From jwcolby at colbyconsulting.com Thu Mar 10 16:27:16 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 10 Mar 2011 17:27:16 -0500 Subject: [dba-VB] c# lock() In-Reply-To: <3693FCAC03E94BCA9C150F6E7AFE4A71@nant> References: <4D78CE4C.5000308@colbyconsulting.com> <3693FCAC03E94BCA9C150F6E7AFE4A71@nant> Message-ID: <4D795044.6080808@colbyconsulting.com> Shamil, Thanks for the code but I think it is overkill for my needs so far, though I will put it in the archives for when I need it. In my system so far I have (or can arrange to have ) only two threads trying to access a list. Basically what I am doing is reading records out of sql server into a class factory, and placing the class instances into a list. Another object with its own thread is coming by and looking at the objects in the list to see if there is any work for it to do. If it finds an object to work on, it grabs a pointer to that item in the list and stores it in a pointer and releases the list. One of the ways I saw this handled was to create factory which creates a brand new list object, grabs a pointer to the list, locks that and then copies the objects in the list pointers to classes) into that new list, unlocks the list and hands back a pointer to this brand new list. thus the new list could be iterated at will. I am just trying to lock the list while one thread adds objects to the list or another thread gets objects out of the list. So if I have object A which contains a list and is adding / deleting objects to that list... and object B which needs to iterate through the list looking for an object of interest in the list, and grabbing a pointer to that object, there is no way to do this without all of the code you wrote? John W. Colby www.ColbyConsulting.com On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: > Hi John -- > > <<< > Does that in fact lock the list to other objects doing the same thing > (getting the list through that property)? >>>> > You have to lock/synchronize/serialize not the list itself but simultaneous > access to the list from different threads. > You'd better use special static object variable to implement locking. > > I will post sample code in my following posting here... > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Thu Mar 10 16:51:26 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 01:51:26 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D795044.6080808@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com> Message-ID: <3B9DE744CEA244829179A81CA818F92F@nant> Hi John -- Do you you need add, remove and iterate a list to get an object or you need to add, remove and get an object by its key? - if the latter - here is the code you can use: private Dictionary _myObjects = new Dictionary(); private static object _myObjectsLocker = new object(); public void Add(MyObject myObject) { lock (_myObjectsLocker) { _myObjects.Add(myObject.ID, myObject); _keys.Add(myObject.ID); } } public MyObject Remove(int id) { MyObject myobject = null; lock (_myObjectsLocker) { myobject = _myObjects[id]; _myObjects.Remove(id); } return myobject; } public MyObject this[int key] { get { MyObject myObject = null; lock (_myObjectsLocker) { _myObjects.TryGetValue(key, out myObject); } return myObject; } } for the case of iteration just replace: _myObjects.TryGetValue(key, out myObject); with your iteration code. As for the sample code - changing qty of threads and changing total computer workload (outer context for the test code) you can see how different could be final results when heavy multi-threading is used. And even if it's not heavy - it still can result in weird output if special multi-threading coding technique aren't used... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 1:27 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, Thanks for the code but I think it is overkill for my needs so far, though I will put it in the archives for when I need it. In my system so far I have (or can arrange to have ) only two threads trying to access a list. Basically what I am doing is reading records out of sql server into a class factory, and placing the class instances into a list. Another object with its own thread is coming by and looking at the objects in the list to see if there is any work for it to do. If it finds an object to work on, it grabs a pointer to that item in the list and stores it in a pointer and releases the list. One of the ways I saw this handled was to create factory which creates a brand new list object, grabs a pointer to the list, locks that and then copies the objects in the list pointers to classes) into that new list, unlocks the list and hands back a pointer to this brand new list. thus the new list could be iterated at will. I am just trying to lock the list while one thread adds objects to the list or another thread gets objects out of the list. So if I have object A which contains a list and is adding / deleting objects to that list... and object B which needs to iterate through the list looking for an object of interest in the list, and grabbing a pointer to that object, there is no way to do this without all of the code you wrote? John W. Colby www.ColbyConsulting.com On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: > Hi John -- > > <<< > Does that in fact lock the list to other objects doing the same thing > (getting the list through that property)? >>>> > You have to lock/synchronize/serialize not the list itself but > simultaneous access to the list from different threads. > You'd better use special static object variable to implement locking. > > I will post sample code in my following posting here... > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Thu Mar 10 17:11:33 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 02:11:33 +0300 Subject: [dba-VB] c# - controlling the run state / status of your threadedclasses In-Reply-To: <4D79366D.8060905@colbyconsulting.com> References: <4D79366D.8060905@colbyconsulting.com> Message-ID: <3B44AE877A14441CA13A51D1BA80FF65@nant> Hi John -- <<< So we have a couple of "commands" (start / stop) and a few statuses (stopped, stopping, running). >>> You'd better also add 'paused' (and maybe 'pausing' and 'resuming') for your list of states. I must note it's not easy to "standardize" such a coding - as with every framework/standard there immediately come non-standard situations to code into framework/standard - that's endless story... I have such a coding but it's rather complicated, several nesting levels etc.etc. - and it was programmed for a customer, I have reused it already in three projects of that customer but every time I do reuse it I have to do some additions/changes. It tested with up to hundred heavy duty threads talking to web services, MS Access(!) be databases, informing FE UI on threads' status changes etc. - and FE is able to work for days without restarting but that is a result of a very big work - it took two years+ to get from first version to the current one. Of course I didn't program it for two years, but getting all that experience to make it as it's now - that was rather time consuming - at least for me... If I ever will make that code less complicated I will share it but that will not happen soon, sorry... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 10 ????? 2011 ?. 23:37 To: VBA Subject: [dba-VB] c# - controlling the run state / status of your threadedclasses As I write classes that consume threads, I need to be able to start and stop these threads and the class that consumes the threads. For example, I open a form and load a class instance that starts performing some process - exporting sets of records to csv files for example. Once that class is loaded and processing, I need to be able to ask that class to stop exporting files. However there may be a time lag between asking and the currently processing file finishing it's export. That is precisely the purpose of a thread - tell it to go and do something else. So (for example) I have a check box that I check that tells the class to start exporting. When it starts it sets a flag saying that it is running and places text into the check box label saying that it is exporting (busy). If I then uncheck the check box I am telling the class to stop exporting. The class is monitoring the check box and will not process the next file but it can't just exit. So it needs to set a flag saying that it is stopping and display "stopping" on the check box label. When it finishes exporting the file, it sets a stopped flag and says displays "terminated" in the check box label. So we have a couple of "commands" (start / stop) and a few statuses (stopped, stopping, running). I would like to "standardize" this into a class so that any class which needs this interface does it in the same way. Some controlling object can tell the object to start and stop, and the class can tell the controlling object that is is stopped, stopping or running. this class does not6 need to do any updating of labels or anything like that, just hold the variables, properties, methods and code to perform this in a standard way. Is anyone doing this? Have existing code you would like to share? Have existing experience trying to do this you would like to share? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 11 08:26:06 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 09:26:06 -0500 Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath Message-ID: <4D7A30FE.8050500@colbyconsulting.com> I am trying to copy a solution to a new location, strip all version control from it, delete a ton of projects within that solution and then version control the new solution. I copied the entire solution from my laptop to a VM and under my user / projects there. The copy still appears to be under version control. I looked in help and it says to copy the directory to itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the top level directory (the solution) but left the check mark on all of the subdirectories (projects and stuff). Does anyone have a clue how to completely and entirely remove a directory from version control - remove all green check marks from every subdirectory etc.? -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Fri Mar 11 08:59:23 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 09:59:23 -0500 Subject: [dba-VB] [dba-SQLServer] SVN - removing all version control for a directory and all files underneath In-Reply-To: <4D7A30FE.8050500@colbyconsulting.com> References: <4D7A30FE.8050500@colbyconsulting.com> Message-ID: <4D7A38CB.7090807@colbyconsulting.com> As it turns out, doing the "export to same directory" seems to actually "unregister" the directory from source control. It leaves the green check marks in explorer - not sure what that means. However if I go into the solution itself, it no longer has the version control dots next to the items in the solution explorer. So it appears to have removed that local copy of the solution from source control. John W. Colby www.ColbyConsulting.com On 3/11/2011 9:26 AM, jwcolby wrote: > I am trying to copy a solution to a new location, strip all version control from it, delete a ton of > projects within that solution and then version control the new solution. > > I copied the entire solution from my laptop to a VM and under my user / projects there. The copy > still appears to be under version control. I looked in help and it says to copy the directory to > itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the > top level directory (the solution) but left the check mark on all of the subdirectories (projects > and stuff). > > Does anyone have a clue how to completely and entirely remove a directory from version control - > remove all green check marks from every subdirectory etc.? From shamil at smsconsulting.spb.ru Fri Mar 11 09:00:05 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 18:00:05 +0300 Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath In-Reply-To: <4D7A30FE.8050500@colbyconsulting.com> References: <4D7A30FE.8050500@colbyconsulting.com> Message-ID: <9192624AA1BA4A6BB323E053411525B8@nant> Hi John -- 1. Just delete hidden .svn subfolder from root folder and all subfolders. 2. Right-click folder in MS Windows Explorer window and use Tortoise SVN Export... command from popup-menu 3. Use command line SVN utility with export command http://svnbook.red-bean.com/nightly/en/svn.ref.svn.c.export.html Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 17:26 To: Access Developers discussion and problem solving; VBA; Sqlserver-Dba Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath I am trying to copy a solution to a new location, strip all version control from it, delete a ton of projects within that solution and then version control the new solution. I copied the entire solution from my laptop to a VM and under my user / projects there. The copy still appears to be under version control. I looked in help and it says to copy the directory to itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the top level directory (the solution) but left the check mark on all of the subdirectories (projects and stuff). Does anyone have a clue how to completely and entirely remove a directory from version control - remove all green check marks from every subdirectory etc.? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 11 10:56:44 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 11:56:44 -0500 Subject: [dba-VB] c# - Creating and using a shared module Message-ID: <4D7A544C.7090904@colbyconsulting.com> I have split my solution into two solutions. However each of those two solutions uses a shared project which I call "base objects". Obviously I want to split that off and then reference that solution from both of the main solutions. I have never done this before. Any suggestions, tips and tricks, warnings, how to etc.? -- John W. Colby www.ColbyConsulting.com From gustav at cactus.dk Fri Mar 11 11:59:11 2011 From: gustav at cactus.dk (Gustav Brock) Date: Fri, 11 Mar 2011 18:59:11 +0100 Subject: [dba-VB] Visual Studio 2010 SP1 Message-ID: Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-475d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav From shamil at smsconsulting.spb.ru Fri Mar 11 11:52:31 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 11 Mar 2011 20:52:31 +0300 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7A544C.7090904@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> Message-ID: <1573E00753994C19BE24FA9E50093615@nant> Hi John -- <<< Any suggestions, tips and tricks, warnings, how to etc.? >>> Just do it! :) I mean if you managed to handle MS Access library databases then handling .NET class libraries would be a "breeze" work for you. Create class library project - you can keep it within its own solution having main (console) project of that solution as a (unit) test one. Put all three solutions - two application ones and class library one in subfolders of the same root folder. (You can do it differently but this is how it's done usually AFAIK) Add existing class library project to application solutions. Reference class library projects from application projects. if you use VS2008 and you have a good PC with plenty of memory (and you have one don't you?) then you can keep all three solutions open in three instances of VS2008. Just keep tack from which solution you have made class library code changes last time - when you'll switch to the other solution and if you have the same source code file opened in it you'll get a message like that "Source file changed - would you like to reload it?" - reply yes... ... Use SVN or Mercurial for source code control. Do commit source code changes to the source code repository from time to time... ... I have 58 projects in one of the solutions here - that results in more than 5 apps and quite some class libs compiled and built - AFAIK folks do have more than hundred projects within solutions sometimes... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 19:57 To: VBA Subject: [dba-VB] c# - Creating and using a shared module I have split my solution into two solutions. However each of those two solutions uses a shared project which I call "base objects". Obviously I want to split that off and then reference that solution from both of the main solutions. I have never done this before. Any suggestions, tips and tricks, warnings, how to etc.? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From df.waters at comcast.net Fri Mar 11 13:46:10 2011 From: df.waters at comcast.net (Dan Waters) Date: Fri, 11 Mar 2011 13:46:10 -0600 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: Message-ID: <003001cbe024$f9c71750$ed5545f0$@comcast.net> I did this yesterday. The download iso image was actually about 1.5 Gb, but I downloaded, burned to disc, and installed in about an hour. Your mileage may vary! Dan -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-47 5d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 11 20:37:10 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 21:37:10 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <1573E00753994C19BE24FA9E50093615@nant> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant> Message-ID: <4D7ADC56.1040806@colbyconsulting.com> You make it sound so easy, but it is an existing project already which was not created as a class library project. I need to create the class library project then somehow get the directories and files into that project and get it to compile. At that point I can reference it by other solutions. John W. Colby www.ColbyConsulting.com On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: > Hi John -- > > <<< > Any suggestions, tips and tricks, warnings, how to etc.? >>>> > Just do it! :) > > I mean if you managed to handle MS Access library databases then handling > .NET class libraries would be a "breeze" work for you. > > Create class library project - you can keep it within its own solution > having main (console) project of that solution as a (unit) test one. > Put all three solutions - two application ones and class library one in > subfolders of the same root folder. (You can do it differently but this is > how it's done usually AFAIK) > Add existing class library project to application solutions. > Reference class library projects from application projects. > if you use VS2008 and you have a good PC with plenty of memory (and you have > one don't you?) then you can keep all three solutions open in three > instances of VS2008. > Just keep tack from which solution you have made class library code changes > last time - when you'll switch to the other solution and if you have the > same source code file opened in it you'll get a message like that "Source > file changed - would you like to reload it?" - reply yes... > > ... > > Use SVN or Mercurial for source code control. Do commit source code changes > to the source code repository from time to time... > ... > > I have 58 projects in one of the solutions here - that results in more than > 5 apps and quite some class libs compiled and built - AFAIK folks do have > more than hundred projects within solutions sometimes... > > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 11 ????? 2011 ?. 19:57 > To: VBA > Subject: [dba-VB] c# - Creating and using a shared module > > I have split my solution into two solutions. However each of those two > solutions uses a shared project which I call "base objects". Obviously I > want to split that off and then reference that solution from both of the > main solutions. > > I have never done this before. Any suggestions, tips and tricks, warnings, > how to etc.? > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From michael at ddisolutions.com.au Fri Mar 11 20:39:50 2011 From: michael at ddisolutions.com.au (Michael Maddison) Date: Sat, 12 Mar 2011 13:39:50 +1100 Subject: [dba-VB] SVN - removing all version control for a directory andall files underneath References: <4D7A30FE.8050500@colbyconsulting.com> <9192624AA1BA4A6BB323E053411525B8@nant> Message-ID: <99266C61B516644D9727F983FAFAB46508623F@remote.ddisolutions.com.au> Hi Guys, I found a reg file that adds a right click menu item to remove all hidden svn dirs. And files from the selected dir. Create a .reg file, Paste this code in, Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Folder\shell\DeleteSVN] @="Delete SVN Folders" [HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Folder\shell\DeleteSVN\command] @="cmd.exe /c \"TITLE Removing SVN Folders in %1 && COLOR 9A && FOR /r \"%1\" %%f IN (.svn) DO RD /s /q \"%%f\" \"" Save it, Merge it to your reg. Works for me. Cheers Michael From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: Saturday, 12 March 2011 2:00 AM To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] SVN - removing all version control for a directory andall files underneath Hi John -- 1. Just delete hidden .svn subfolder from root folder and all subfolders. 2. Right-click folder in MS Windows Explorer window and use Tortoise SVN Export... command from popup-menu 3. Use command line SVN utility with export command http://svnbook.red-bean.com/nightly/en/svn.ref.svn.c.export.html Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 11 ????? 2011 ?. 17:26 To: Access Developers discussion and problem solving; VBA; Sqlserver-Dba Subject: [dba-VB] SVN - removing all version control for a directory and all files underneath I am trying to copy a solution to a new location, strip all version control from it, delete a ton of projects within that solution and then version control the new solution. I copied the entire solution from my laptop to a VM and under my user / projects there. The copy still appears to be under version control. I looked in help and it says to copy the directory to itself using the vsn right click menu. I did that and it removed the vsn green checkmark from the top level directory (the solution) but left the check mark on all of the subdirectories (projects and stuff). Does anyone have a clue how to completely and entirely remove a directory from version control - remove all green check marks from every subdirectory etc.? -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com ________________________________ No virus found in this message. Checked by AVG - www.avg.com Version: 10.0.1204 / Virus Database: 1498/3501 - Release Date: 03/11/11 From jwcolby at colbyconsulting.com Fri Mar 11 21:09:49 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 11 Mar 2011 22:09:49 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7ADC56.1040806@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant> <4D7ADC56.1040806@colbyconsulting.com> Message-ID: <4D7AE3FD.1040206@colbyconsulting.com> I found a "make project from existing files. When I did that it offered to use a selected directory so I dragged the project directory into a new directory and told it to use that. I selected a class library type of project and it went to work "converting" my existing project into a class library. 112 compile errors, most having to do with the fact that it no longer understands using System.Windows.XYZ Many of the common classes did things like accept a combo loaded data into it in a standard way. Stuff like that. So there we are. I "just did it"! ;) John W. Colby www.ColbyConsulting.com On 3/11/2011 9:37 PM, jwcolby wrote: > You make it sound so easy, but it is an existing project already which was not created as a class > library project. I need to create the class library project then somehow get the directories and > files into that project and get it to compile. > > At that point I can reference it by other solutions. > > John W. Colby > www.ColbyConsulting.com > > On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Any suggestions, tips and tricks, warnings, how to etc.? >>>>> >> Just do it! :) >> >> I mean if you managed to handle MS Access library databases then handling >> .NET class libraries would be a "breeze" work for you. >> >> Create class library project - you can keep it within its own solution >> having main (console) project of that solution as a (unit) test one. >> Put all three solutions - two application ones and class library one in >> subfolders of the same root folder. (You can do it differently but this is >> how it's done usually AFAIK) >> Add existing class library project to application solutions. >> Reference class library projects from application projects. >> if you use VS2008 and you have a good PC with plenty of memory (and you have >> one don't you?) then you can keep all three solutions open in three >> instances of VS2008. >> Just keep tack from which solution you have made class library code changes >> last time - when you'll switch to the other solution and if you have the >> same source code file opened in it you'll get a message like that "Source >> file changed - would you like to reload it?" - reply yes... >> >> ... >> >> Use SVN or Mercurial for source code control. Do commit source code changes >> to the source code repository from time to time... >> ... >> >> I have 58 projects in one of the solutions here - that results in more than >> 5 apps and quite some class libs compiled and built - AFAIK folks do have >> more than hundred projects within solutions sometimes... >> >> >> Thank you. >> >> -- >> Shamil From shamil at smsconsulting.spb.ru Sat Mar 12 03:34:15 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 12 Mar 2011 12:34:15 +0300 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7AE3FD.1040206@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant><4D7ADC56.1040806@colbyconsulting.com> <4D7AE3FD.1040206@colbyconsulting.com> Message-ID: <8458BA03716A4820980078DE24EC3D06@nant> Hi John -- You have to add references to some of System.Windows.XYZ: - System.Windows.Forms - ... Just compare references of your source project with a classlib project created from it - and add missing references... <<< I "just did it"! ;) >>> But you did it, didn't you? ;) Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 12 ????? 2011 ?. 6:10 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# - Creating and using a shared module I found a "make project from existing files. When I did that it offered to use a selected directory so I dragged the project directory into a new directory and told it to use that. I selected a class library type of project and it went to work "converting" my existing project into a class library. 112 compile errors, most having to do with the fact that it no longer understands using System.Windows.XYZ Many of the common classes did things like accept a combo loaded data into it in a standard way. Stuff like that. So there we are. I "just did it"! ;) John W. Colby www.ColbyConsulting.com On 3/11/2011 9:37 PM, jwcolby wrote: > You make it sound so easy, but it is an existing project already which > was not created as a class library project. I need to create the class > library project then somehow get the directories and files into that project and get it to compile. > > At that point I can reference it by other solutions. > > John W. Colby > www.ColbyConsulting.com > > On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Any suggestions, tips and tricks, warnings, how to etc.? >>>>> >> Just do it! :) >> >> I mean if you managed to handle MS Access library databases then >> handling .NET class libraries would be a "breeze" work for you. >> >> Create class library project - you can keep it within its own >> solution having main (console) project of that solution as a (unit) test one. >> Put all three solutions - two application ones and class library one >> in subfolders of the same root folder. (You can do it differently but >> this is how it's done usually AFAIK) Add existing class library >> project to application solutions. >> Reference class library projects from application projects. >> if you use VS2008 and you have a good PC with plenty of memory (and >> you have one don't you?) then you can keep all three solutions open >> in three instances of VS2008. >> Just keep tack from which solution you have made class library code >> changes last time - when you'll switch to the other solution and if >> you have the same source code file opened in it you'll get a message >> like that "Source file changed - would you like to reload it?" - reply yes... >> >> ... >> >> Use SVN or Mercurial for source code control. Do commit source code >> changes to the source code repository from time to time... >> ... >> >> I have 58 projects in one of the solutions here - that results in >> more than >> 5 apps and quite some class libs compiled and built - AFAIK folks do >> have more than hundred projects within solutions sometimes... >> >> >> Thank you. >> >> -- >> Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Mar 12 06:27:48 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 12 Mar 2011 07:27:48 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <8458BA03716A4820980078DE24EC3D06@nant> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant><4D7ADC56.1040806@colbyconsulting.com> <4D7AE3FD.1040206@colbyconsulting.com> <8458BA03716A4820980078DE24EC3D06@nant> Message-ID: <4D7B66C4.6040404@colbyconsulting.com> Well... the whole thing compiled (the project) when it was part of the bigger solution. I'll give that a try. John W. Colby www.ColbyConsulting.com On 3/12/2011 4:34 AM, Shamil Salakhetdinov wrote: > Hi John -- > > You have to add references to some of System.Windows.XYZ: > > - System.Windows.Forms > - ... > > Just compare references of your source project with a classlib project > created from it - and add missing references... > > <<< > I "just did it"! ;) >>>> > But you did it, didn't you? ;) > > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 12 ????? 2011 ?. 6:10 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# - Creating and using a shared module > > I found a "make project from existing files. When I did that it offered to > use a selected directory so I dragged the project directory into a new > directory and told it to use that. I selected a class library type of > project and it went to work "converting" my existing project into a class > library. > > 112 compile errors, most having to do with the fact that it no longer > understands > > using System.Windows.XYZ > > Many of the common classes did things like accept a combo loaded data into > it in a standard way. > Stuff like that. > > So there we are. > > I "just did it"! ;) > > John W. Colby > www.ColbyConsulting.com > > On 3/11/2011 9:37 PM, jwcolby wrote: >> You make it sound so easy, but it is an existing project already which >> was not created as a class library project. I need to create the class >> library project then somehow get the directories and files into that > project and get it to compile. >> >> At that point I can reference it by other solutions. >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >>> Hi John -- >>> >>> <<< >>> Any suggestions, tips and tricks, warnings, how to etc.? >>>>>> >>> Just do it! :) >>> >>> I mean if you managed to handle MS Access library databases then >>> handling .NET class libraries would be a "breeze" work for you. >>> >>> Create class library project - you can keep it within its own >>> solution having main (console) project of that solution as a (unit) test > one. >>> Put all three solutions - two application ones and class library one >>> in subfolders of the same root folder. (You can do it differently but >>> this is how it's done usually AFAIK) Add existing class library >>> project to application solutions. >>> Reference class library projects from application projects. >>> if you use VS2008 and you have a good PC with plenty of memory (and >>> you have one don't you?) then you can keep all three solutions open >>> in three instances of VS2008. >>> Just keep tack from which solution you have made class library code >>> changes last time - when you'll switch to the other solution and if >>> you have the same source code file opened in it you'll get a message >>> like that "Source file changed - would you like to reload it?" - reply > yes... >>> >>> ... >>> >>> Use SVN or Mercurial for source code control. Do commit source code >>> changes to the source code repository from time to time... >>> ... >>> >>> I have 58 projects in one of the solutions here - that results in >>> more than >>> 5 apps and quite some class libs compiled and built - AFAIK folks do >>> have more than hundred projects within solutions sometimes... >>> >>> >>> Thank you. >>> >>> -- >>> Shamil > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 12 15:00:17 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 12 Mar 2011 16:00:17 -0500 Subject: [dba-VB] c# - Creating and using a shared module In-Reply-To: <4D7B66C4.6040404@colbyconsulting.com> References: <4D7A544C.7090904@colbyconsulting.com> <1573E00753994C19BE24FA9E50093615@nant><4D7ADC56.1040806@colbyconsulting.com> <4D7AE3FD.1040206@colbyconsulting.com> <8458BA03716A4820980078DE24EC3D06@nant> <4D7B66C4.6040404@colbyconsulting.com> Message-ID: <4D7BDEE1.40400@colbyconsulting.com> OK, I got all of the references fixed up and got it checked into VSN. John W. Colby www.ColbyConsulting.com On 3/12/2011 7:27 AM, jwcolby wrote: > Well... the whole thing compiled (the project) when it was part of the bigger solution. > > I'll give that a try. > > John W. Colby > www.ColbyConsulting.com > > On 3/12/2011 4:34 AM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> You have to add references to some of System.Windows.XYZ: >> >> - System.Windows.Forms >> - ... >> >> Just compare references of your source project with a classlib project >> created from it - and add missing references... >> >> <<< >> I "just did it"! ;) >>>>> >> But you did it, didn't you? ;) >> >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 12 ????? 2011 ?. 6:10 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# - Creating and using a shared module >> >> I found a "make project from existing files. When I did that it offered to >> use a selected directory so I dragged the project directory into a new >> directory and told it to use that. I selected a class library type of >> project and it went to work "converting" my existing project into a class >> library. >> >> 112 compile errors, most having to do with the fact that it no longer >> understands >> >> using System.Windows.XYZ >> >> Many of the common classes did things like accept a combo loaded data into >> it in a standard way. >> Stuff like that. >> >> So there we are. >> >> I "just did it"! ;) >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/11/2011 9:37 PM, jwcolby wrote: >>> You make it sound so easy, but it is an existing project already which >>> was not created as a class library project. I need to create the class >>> library project then somehow get the directories and files into that >> project and get it to compile. >>> >>> At that point I can reference it by other solutions. >>> >>> John W. Colby >>> www.ColbyConsulting.com >>> >>> On 3/11/2011 12:52 PM, Shamil Salakhetdinov wrote: >>>> Hi John -- >>>> >>>> <<< >>>> Any suggestions, tips and tricks, warnings, how to etc.? >>>>>>> >>>> Just do it! :) >>>> >>>> I mean if you managed to handle MS Access library databases then >>>> handling .NET class libraries would be a "breeze" work for you. >>>> >>>> Create class library project - you can keep it within its own >>>> solution having main (console) project of that solution as a (unit) test >> one. >>>> Put all three solutions - two application ones and class library one >>>> in subfolders of the same root folder. (You can do it differently but >>>> this is how it's done usually AFAIK) Add existing class library >>>> project to application solutions. >>>> Reference class library projects from application projects. >>>> if you use VS2008 and you have a good PC with plenty of memory (and >>>> you have one don't you?) then you can keep all three solutions open >>>> in three instances of VS2008. >>>> Just keep tack from which solution you have made class library code >>>> changes last time - when you'll switch to the other solution and if >>>> you have the same source code file opened in it you'll get a message >>>> like that "Source file changed - would you like to reload it?" - reply >> yes... >>>> >>>> ... >>>> >>>> Use SVN or Mercurial for source code control. Do commit source code >>>> changes to the source code repository from time to time... >>>> ... >>>> >>>> I have 58 projects in one of the solutions here - that results in >>>> more than >>>> 5 apps and quite some class libs compiled and built - AFAIK folks do >>>> have more than hundred projects within solutions sometimes... >>>> >>>> >>>> Thank you. >>>> >>>> -- >>>> Shamil >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From bheid at sc.rr.com Sun Mar 13 09:26:07 2011 From: bheid at sc.rr.com (Bobby Heid) Date: Sun, 13 Mar 2011 10:26:07 -0400 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: Message-ID: <005d01cbe18a$977c6700$c6753500$@rr.com> Only took me about 40 minutes on my machine (Win 7 Ultimate, VS22010 Ultimate, and SQL 2008 Standard). Bobby -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Friday, March 11, 2011 12:59 PM To: dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-47 5d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sun Mar 13 21:58:15 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 13 Mar 2011 22:58:15 -0400 Subject: [dba-VB] Access runtime (an maybe full install?) Message-ID: <4D7D8447.4000003@colbyconsulting.com> I am trying to get an Access 2007 runtime to run reliably anywhere I install it. I had a Windows 2003 virtual machine hanging around so I cleaned Office off of it, and I installed the runtime. I got the directory path declared a safe location and voila, the application ran. Full of confidence I installed Hamachi, the runtime package and the app on a friend's (rather old) XP and it doesn't run. I dug out my old dev machine which it appears that I wiped and installed XP on some time ago. I installed Hamachi, got it on the VPN for this database etc. I then installed the runtime there and tried to run. No go. Of course the runtime doesn't report errors or at least reliably. So I built a little test version with just a single table and an autoform which automatically opened when the app opened. It actually reported that it couldn't talk to the odbc native sql driver (not the exact words of course). So I started Googling and soon discovered that this driver is installed by SQL Server as it installs. However an installer called sqlncli.msi is out there for download. I downloaded that and tried to run. No joy. I turned the machine off, and ran a network cable so it is directly on my network and voila, it runs. I then disabled the cable so that it is connecting to my network via my wireless and it runs. So it appears that the SQL driver is not installed as part of XP, which makes sense since this is SQL Server 2008 Express version I am connecting to. However I would have expected this odbc SQL driver to be part of the Access 2007 runtime, but it seems not. It does appear that a download / install / reboot of the machine will allow me to hook up to the sql server instance. So I am now running Hamachi, joined to this specific network, with the sqlncli.msi installer installing the ODBC SQL Server native driver, plus the Access 2007 runtime, running the full on app, hitting the sql server over the vpn on 5.203.167.79 (the Hamachi IP for the server). And it works on this machine of mine. Now I have to install the SQL Server drivers on the friend's old machine (also running XP but on 512 megs of ram) and ensure that will run. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sun Mar 13 22:27:45 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 13 Mar 2011 23:27:45 -0400 Subject: [dba-VB] [AccessD] Access runtime (an maybe full install?) In-Reply-To: <4D7D8447.4000003@colbyconsulting.com> References: <4D7D8447.4000003@colbyconsulting.com> Message-ID: <4D7D8B31.4080405@colbyconsulting.com> And it runs on the friend's old machine as well. It feels like I might be working now. I have to say this has been one of the most complex jobs I have taken on, with Hamachi VPN networks, a virtual machine running the sql server database, and then the client machine running Hamachi and an Access 2007 runtime. Safe zones, sql server drivers not installed. It has been a challenge getting this running. John W. Colby www.ColbyConsulting.com On 3/13/2011 10:58 PM, jwcolby wrote: > I am trying to get an Access 2007 runtime to run reliably anywhere I install it. I had a Windows > 2003 virtual machine hanging around so I cleaned Office off of it, and I installed the runtime. I > got the directory path declared a safe location and voila, the application ran. > > Full of confidence I installed Hamachi, the runtime package and the app on a friend's (rather old) > XP and it doesn't run. I dug out my old dev machine which it appears that I wiped and installed XP > on some time ago. I installed Hamachi, got it on the VPN for this database etc. I then installed the > runtime there and tried to run. No go. > > Of course the runtime doesn't report errors or at least reliably. So I built a little test version > with just a single table and an autoform which automatically opened when the app opened. It actually > reported that it couldn't talk to the odbc native sql driver (not the exact words of course). So I > started Googling and soon discovered that this driver is installed by SQL Server as it installs. > However an installer called sqlncli.msi is out there for download. > > I downloaded that and tried to run. No joy. I turned the machine off, and ran a network cable so it > is directly on my network and voila, it runs. I then disabled the cable so that it is connecting to > my network via my wireless and it runs. > > So it appears that the SQL driver is not installed as part of XP, which makes sense since this is > SQL Server 2008 Express version I am connecting to. However I would have expected this odbc SQL > driver to be part of the Access 2007 runtime, but it seems not. > > It does appear that a download / install / reboot of the machine will allow me to hook up to the sql > server instance. > > So I am now running Hamachi, joined to this specific network, with the sqlncli.msi installer > installing the ODBC SQL Server native driver, plus the Access 2007 runtime, running the full on app, > hitting the sql server over the vpn on 5.203.167.79 (the Hamachi IP for the server). > > And it works on this machine of mine. > > Now I have to install the SQL Server drivers on the friend's old machine (also running XP but on 512 > megs of ram) and ensure that will run. > > From Gustav at cactus.dk Mon Mar 14 05:38:03 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 14 Mar 2011 11:38:03 +0100 Subject: [dba-VB] Visual Studio 2010 SP1 Message-ID: Hi all OK, don't blame the child. At one time it even asked for the VS2010 dvd - to roll back the installation. Another install was in progress, or at least I hadn't rebooted the machine. So I did and ran the SP1 again. This time - as everything was downloaded - it took about an hour to finish. /gustav >>> gustav at cactus.dk 11-03-2011 18:59 >>> Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8107-475d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav From jwcolby at colbyconsulting.com Mon Mar 14 13:16:52 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 14:16:52 -0400 Subject: [dba-VB] C# combo boxes Message-ID: <4D7E5B94.2020805@colbyconsulting.com> I am trying to set up a combo box bound to a field in the form such that the combo populates with the PKID and a text field, and when the form opens, selecting something from the combo drops the PKID into the field in the form. I am not getting what to set the properties of the combo box to. I am pulling the ID and the string to display into the combo but I cannot cause the combo to store the id and release control to the next control on the form. -- John W. Colby www.ColbyConsulting.com From Gustav at cactus.dk Mon Mar 14 14:02:27 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Mon, 14 Mar 2011 20:02:27 +0100 Subject: [dba-VB] C# combo boxes Message-ID: Hi John If you (typically) have this in InitializeComponent() of the .Designer.cs file: // comboBoxCountry // this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); this.comboBoxCountry.DisplayMember = "Id"; this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; this.comboBoxCountry.FormattingEnabled = true; this.comboBoxCountry.ItemHeight = 13; this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); this.comboBoxCountry.Name = "comboBoxCountry"; this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); this.comboBoxCountry.Sorted = true; this.comboBoxCountry.TabIndex = 7; this.comboBoxCountry.ValueMember = "Id"; You can do something like this in the Load event: this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); Then: private void FillComboBoxCountryCode() { // Order the country names of countryDataTable. string filterExpression = ""; string sortExpression = "CountryNameUK asc"; DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); this.comboBoxCountry.DataSource = countryList; this.comboBoxCountry.ValueMember = "Id"; this.comboBoxCountry.DisplayMember = "CountryNameUK"; } And: private void ChangeCountrySelection() { if (this.comboBoxCountry.SelectedValue != null) { _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); this.comboBoxPostalCode.Text = String.Empty; // Read postal codes of selected country. FillComboBoxPostalCode(); } } This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. /gustav >>> jwcolby at colbyconsulting.com 14-03-2011 19:16 >>> I am trying to set up a combo box bound to a field in the form such that the combo populates with the PKID and a text field, and when the form opens, selecting something from the combo drops the PKID into the field in the form. I am not getting what to set the properties of the combo box to. I am pulling the ID and the string to display into the combo but I cannot cause the combo to store the id and release control to the next control on the form. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Mon Mar 14 16:36:55 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 17:36:55 -0400 Subject: [dba-VB] C# combo boxes In-Reply-To: References: Message-ID: <4D7E8A77.7070009@colbyconsulting.com> Thanks Gustav, I don't know why you are taking me into code in code. I have data connections. This particular data connection has a single view that allows me to write into a table. It has a view that filters a combo. I drag and drop the view for the table onto the form. I have told that table object that specific fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am trying to set up that combo to 1) Display a text value 2) Contain the PKID from the table the combo draws from 3) When I select an item, place the selected ID into the field the combo is bound to. It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or going into code). Actually I have a kinda sorta related question. When I look at server explorer I see a data connection object, however there are two objects underneath, neither of which are for this solution. Furthermore the objects under the data connection are things found on an entirely unrelated server, for other projects that I did work on in the past. The question then is why is the server explorer choosing to display objects out on a virtual machine instead of the database that was originally selected for this solution? We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound form stuff. I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values in most cases) the developer, the product, the type of work, then a couple of fields for date time values for start / stop work. The table the data is going into has a few thousand records already in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound forms is a natural for this. A long time ago I made this (kind of thing) work. I remember that getting the properties of the combo set correctly was non-intuitive but once I figured it out it just worked. John W. Colby www.ColbyConsulting.com On 3/14/2011 3:02 PM, Gustav Brock wrote: > Hi John > > If you (typically) have this in InitializeComponent() of the .Designer.cs file: > > // comboBoxCountry > // > this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; > this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; > this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; > this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); > this.comboBoxCountry.DisplayMember = "Id"; > this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; > this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); > this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; > this.comboBoxCountry.FormattingEnabled = true; > this.comboBoxCountry.ItemHeight = 13; > this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); > this.comboBoxCountry.Name = "comboBoxCountry"; > this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); > this.comboBoxCountry.Sorted = true; > this.comboBoxCountry.TabIndex = 7; > this.comboBoxCountry.ValueMember = "Id"; > > You can do something like this in the Load event: > > this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); > > Then: > private void FillComboBoxCountryCode() > { > // Order the country names of countryDataTable. > string filterExpression = ""; > string sortExpression = "CountryNameUK asc"; > DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); > > this.comboBoxCountry.DataSource = countryList; > this.comboBoxCountry.ValueMember = "Id"; > this.comboBoxCountry.DisplayMember = "CountryNameUK"; > } > > And: > private void ChangeCountrySelection() > { > if (this.comboBoxCountry.SelectedValue != null) > { > _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); > this.comboBoxPostalCode.Text = String.Empty; > // Read postal codes of selected country. > FillComboBoxPostalCode(); > } > } > > This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> > I am trying to set up a combo box bound to a field in the form such that the combo populates with > the PKID and a text field, and when the form opens, selecting something from the combo drops the > PKID into the field in the form. > > I am not getting what to set the properties of the combo box to. I am pulling the ID and the string > to display into the combo but I cannot cause the combo to store the id and release control to the > next control on the form. From gustav at cactus.dk Mon Mar 14 17:15:35 2011 From: gustav at cactus.dk (Gustav Brock) Date: Mon, 14 Mar 2011 23:15:35 +0100 Subject: [dba-VB] C# combo boxes Message-ID: Hi John Eh, are you commenting or? Not quite clear to me. I thought you were looking for some code samples ... I have connections too but I use them only to feed the datatableadapters. I refrain from working with SqlCommand and the like; waste of time in my opinion. /gustav >>> jwcolby at colbyconsulting.com 14-03-2011 22:36 >>> Thanks Gustav, I don't know why you are taking me into code in code. I have data connections. This particular data connection has a single view that allows me to write into a table. It has a view that filters a combo. I drag and drop the view for the table onto the form. I have told that table object that specific fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am trying to set up that combo to 1) Display a text value 2) Contain the PKID from the table the combo draws from 3) When I select an item, place the selected ID into the field the combo is bound to. It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or going into code). Actually I have a kinda sorta related question. When I look at server explorer I see a data connection object, however there are two objects underneath, neither of which are for this solution. Furthermore the objects under the data connection are things found on an entirely unrelated server, for other projects that I did work on in the past. The question then is why is the server explorer choosing to display objects out on a virtual machine instead of the database that was originally selected for this solution? We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound form stuff. I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values in most cases) the developer, the product, the type of work, then a couple of fields for date time values for start / stop work. The table the data is going into has a few thousand records already in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound forms is a natural for this. A long time ago I made this (kind of thing) work. I remember that getting the properties of the combo set correctly was non-intuitive but once I figured it out it just worked. John W. Colby www.ColbyConsulting.com On 3/14/2011 3:02 PM, Gustav Brock wrote: > Hi John > > If you (typically) have this in InitializeComponent() of the .Designer.cs file: > > // comboBoxCountry > // > this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; > this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; > this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; > this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); > this.comboBoxCountry.DisplayMember = "Id"; > this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; > this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); > this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; > this.comboBoxCountry.FormattingEnabled = true; > this.comboBoxCountry.ItemHeight = 13; > this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); > this.comboBoxCountry.Name = "comboBoxCountry"; > this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); > this.comboBoxCountry.Sorted = true; > this.comboBoxCountry.TabIndex = 7; > this.comboBoxCountry.ValueMember = "Id"; > > You can do something like this in the Load event: > > this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); > > Then: > private void FillComboBoxCountryCode() > { > // Order the country names of countryDataTable. > string filterExpression = ""; > string sortExpression = "CountryNameUK asc"; > DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); > > this.comboBoxCountry.DataSource = countryList; > this.comboBoxCountry.ValueMember = "Id"; > this.comboBoxCountry.DisplayMember = "CountryNameUK"; > } > > And: > private void ChangeCountrySelection() > { > if (this.comboBoxCountry.SelectedValue != null) > { > _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); > this.comboBoxPostalCode.Text = String.Empty; > // Read postal codes of selected country. > FillComboBoxPostalCode(); > } > } > > This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> > I am trying to set up a combo box bound to a field in the form such that the combo populates with > the PKID and a text field, and when the form opens, selecting something from the combo drops the > PKID into the field in the form. > > I am not getting what to set the properties of the combo box to. I am pulling the ID and the string > to display into the combo but I cannot cause the combo to store the id and release control to the > next control on the form. From jwcolby at colbyconsulting.com Mon Mar 14 17:25:08 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 18:25:08 -0400 Subject: [dba-VB] C# combo boxes In-Reply-To: References: Message-ID: <4D7E95C4.9020203@colbyconsulting.com> I am trying (unsuccessfully) to do an "Access like bound form". And I always comment. ;) ATM the form displays the values if dropped down, but if not dropped down it just displays an integer (the PKID of the data in the combo is my guess, or perhaps the PKID in the field in the form. So... How do I cause the combo to display Paul's name instead of 6. John W. Colby www.ColbyConsulting.com On 3/14/2011 6:15 PM, Gustav Brock wrote: > Hi John > > Eh, are you commenting or? Not quite clear to me. I thought you were looking for some code samples ... > > I have connections too but I use them only to feed the datatableadapters. I refrain from working with SqlCommand and the like; waste of time in my opinion. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> > Thanks Gustav, > I don't know why you are taking me into code in code. > > I have data connections. This particular data connection has a single view that allows me to write > into a table. It has a view that filters a combo. > > I drag and drop the view for the table onto the form. I have told that table object that specific > fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am > trying to set up that combo to > > 1) Display a text value > 2) Contain the PKID from the table the combo draws from > 3) When I select an item, place the selected ID into the field the combo is bound to. > > It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or > going into code). > > Actually I have a kinda sorta related question. > > When I look at server explorer I see a data connection object, however there are two objects > underneath, neither of which are for this solution. Furthermore the objects under the data > connection are things found on an entirely unrelated server, for other projects that I did work on > in the past. > > The question then is why is the server explorer choosing to display objects out on a virtual machine > instead of the database that was originally selected for this solution? > > We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound > form stuff. > > I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values > in most cases) the developer, the product, the type of work, then a couple of fields for date time > values for start / stop work. The table the data is going into has a few thousand records already > in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound > forms is a natural for this. > > A long time ago I made this (kind of thing) work. I remember that getting the properties of the > combo set correctly was non-intuitive but once I figured it out it just worked. > > > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 3:02 PM, Gustav Brock wrote: >> Hi John >> >> If you (typically) have this in InitializeComponent() of the .Designer.cs file: >> >> // comboBoxCountry >> // >> this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; >> this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; >> this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; >> this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); >> this.comboBoxCountry.DisplayMember = "Id"; >> this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; >> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); >> this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; >> this.comboBoxCountry.FormattingEnabled = true; >> this.comboBoxCountry.ItemHeight = 13; >> this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); >> this.comboBoxCountry.Name = "comboBoxCountry"; >> this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); >> this.comboBoxCountry.Sorted = true; >> this.comboBoxCountry.TabIndex = 7; >> this.comboBoxCountry.ValueMember = "Id"; >> >> You can do something like this in the Load event: >> >> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); >> >> Then: >> private void FillComboBoxCountryCode() >> { >> // Order the country names of countryDataTable. >> string filterExpression = ""; >> string sortExpression = "CountryNameUK asc"; >> DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); >> >> this.comboBoxCountry.DataSource = countryList; >> this.comboBoxCountry.ValueMember = "Id"; >> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >> } >> >> And: >> private void ChangeCountrySelection() >> { >> if (this.comboBoxCountry.SelectedValue != null) >> { >> _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); >> this.comboBoxPostalCode.Text = String.Empty; >> // Read postal codes of selected country. >> FillComboBoxPostalCode(); >> } >> } >> >> This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >> I am trying to set up a combo box bound to a field in the form such that the combo populates with >> the PKID and a text field, and when the form opens, selecting something from the combo drops the >> PKID into the field in the form. >> >> I am not getting what to set the properties of the combo box to. I am pulling the ID and the string >> to display into the combo but I cannot cause the combo to store the id and release control to the >> next control on the form. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From accessd at shaw.ca Mon Mar 14 18:59:26 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Mon, 14 Mar 2011 16:59:26 -0700 Subject: [dba-VB] C# combo boxes In-Reply-To: <4D7E95C4.9020203@colbyconsulting.com> References: <4D7E95C4.9020203@colbyconsulting.com> Message-ID: Hi John: Try and use ADO. It is on every Windows based computer already and requires no additional ODBC to be setup. In other words, you do not even have to see the computers on which the application is going to have a sable connection. Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, March 14, 2011 3:25 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# combo boxes I am trying (unsuccessfully) to do an "Access like bound form". And I always comment. ;) ATM the form displays the values if dropped down, but if not dropped down it just displays an integer (the PKID of the data in the combo is my guess, or perhaps the PKID in the field in the form. So... How do I cause the combo to display Paul's name instead of 6. John W. Colby www.ColbyConsulting.com On 3/14/2011 6:15 PM, Gustav Brock wrote: > Hi John > > Eh, are you commenting or? Not quite clear to me. I thought you were looking for some code samples ... > > I have connections too but I use them only to feed the datatableadapters. I refrain from working with SqlCommand and the like; waste of time in my opinion. > > /gustav > > >>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> > Thanks Gustav, > I don't know why you are taking me into code in code. > > I have data connections. This particular data connection has a single view that allows me to write > into a table. It has a view that filters a combo. > > I drag and drop the view for the table onto the form. I have told that table object that specific > fields are combos so when I drag it out, I have combos (for the FKs - PKs from other tables). I am > trying to set up that combo to > > 1) Display a text value > 2) Contain the PKID from the table the combo draws from > 3) When I select an item, place the selected ID into the field the combo is bound to. > > It seems this is a normal daily requirement in building forms, and shouldn't be rocket science (or > going into code). > > Actually I have a kinda sorta related question. > > When I look at server explorer I see a data connection object, however there are two objects > underneath, neither of which are for this solution. Furthermore the objects under the data > connection are things found on an entirely unrelated server, for other projects that I did work on > in the past. > > The question then is why is the server explorer choosing to display objects out on a virtual machine > instead of the database that was originally selected for this solution? > > We have done a ton of coding sql server stuff here but we have not yet learned how to do this bound > form stuff. > > I am trying to build a timeclock kind of thing. Very simple, a few combos to select (default values > in most cases) the developer, the product, the type of work, then a couple of fields for date time > values for start / stop work. The table the data is going into has a few thousand records already > in it, the combos pull from tables with 20-30 records etc. It seems (to me) that just using bound > forms is a natural for this. > > A long time ago I made this (kind of thing) work. I remember that getting the properties of the > combo set correctly was non-intuitive but once I figured it out it just worked. > > > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 3:02 PM, Gustav Brock wrote: >> Hi John >> >> If you (typically) have this in InitializeComponent() of the .Designer.cs file: >> >> // comboBoxCountry >> // >> this.comboBoxCountry.AutoCompleteMode = System.Windows.Forms.AutoCompleteMode.Suggest; >> this.comboBoxCountry.AutoCompleteSource = System.Windows.Forms.AutoCompleteSource.ListItems; >> this.comboBoxCountry.BackColor = System.Drawing.Color.LavenderBlush; >> this.comboBoxCountry.DataBindings.Add(new System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, "CountryId", true)); >> this.comboBoxCountry.DisplayMember = "Id"; >> this.comboBoxCountry.FlatStyle = System.Windows.Forms.FlatStyle.Flat; >> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe UI", 8F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); >> this.comboBoxCountry.ForeColor = System.Drawing.Color.DarkMagenta; >> this.comboBoxCountry.FormattingEnabled = true; >> this.comboBoxCountry.ItemHeight = 13; >> this.comboBoxCountry.Location = new System.Drawing.Point(21, 136); >> this.comboBoxCountry.Name = "comboBoxCountry"; >> this.comboBoxCountry.Size = new System.Drawing.Size(117, 21); >> this.comboBoxCountry.Sorted = true; >> this.comboBoxCountry.TabIndex = 7; >> this.comboBoxCountry.ValueMember = "Id"; >> >> You can do something like this in the Load event: >> >> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => ChangeCountrySelection(); >> >> Then: >> private void FillComboBoxCountryCode() >> { >> // Order the country names of countryDataTable. >> string filterExpression = ""; >> string sortExpression = "CountryNameUK asc"; >> DataRow[] countryList = _countryDataTable.Select(filterExpression, sortExpression); >> >> this.comboBoxCountry.DataSource = countryList; >> this.comboBoxCountry.ValueMember = "Id"; >> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >> } >> >> And: >> private void ChangeCountrySelection() >> { >> if (this.comboBoxCountry.SelectedValue != null) >> { >> _countryId = Convert.ToInt32(this.comboBoxCountry.SelectedValue); >> this.comboBoxPostalCode.Text = String.Empty; >> // Read postal codes of selected country. >> FillComboBoxPostalCode(); >> } >> } >> >> This is pretty standard using tableadapters and bindingnavigator. Nothing fancy. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >> I am trying to set up a combo box bound to a field in the form such that the combo populates with >> the PKID and a text field, and when the form opens, selecting something from the combo drops the >> PKID into the field in the form. >> >> I am not getting what to set the properties of the combo box to. I am pulling the ID and the string >> to display into the combo but I cannot cause the combo to store the id and release control to the >> next control on the form. > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Mon Mar 14 21:13:25 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Mon, 14 Mar 2011 22:13:25 -0400 Subject: [dba-VB] C# combo boxes In-Reply-To: References: <4D7E95C4.9020203@colbyconsulting.com> Message-ID: <4D7ECB45.8060609@colbyconsulting.com> I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com On 3/14/2011 7:59 PM, Jim Lawrence wrote: > Hi John: > > Try and use ADO. It is on every Windows based computer already and requires > no additional ODBC to be setup. In other words, you do not even have to see > the computers on which the application is going to have a sable connection. > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Monday, March 14, 2011 3:25 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] C# combo boxes > > I am trying (unsuccessfully) to do an "Access like bound form". > > And I always comment. ;) > > ATM the form displays the values if dropped down, but if not dropped down it > just displays an > integer (the PKID of the data in the combo is my guess, or perhaps the PKID > in the field in the form. > > So... > > How do I cause the combo to display Paul's name instead of 6. > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 6:15 PM, Gustav Brock wrote: >> Hi John >> >> Eh, are you commenting or? Not quite clear to me. I thought you were > looking for some code samples ... >> >> I have connections too but I use them only to feed the datatableadapters. > I refrain from working with SqlCommand and the like; waste of time in my > opinion. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> >> Thanks Gustav, >> I don't know why you are taking me into code in code. >> >> I have data connections. This particular data connection has a single > view that allows me to write >> into a table. It has a view that filters a combo. >> >> I drag and drop the view for the table onto the form. I have told that > table object that specific >> fields are combos so when I drag it out, I have combos (for the FKs - PKs > from other tables). I am >> trying to set up that combo to >> >> 1) Display a text value >> 2) Contain the PKID from the table the combo draws from >> 3) When I select an item, place the selected ID into the field the combo > is bound to. >> >> It seems this is a normal daily requirement in building forms, and > shouldn't be rocket science (or >> going into code). >> >> Actually I have a kinda sorta related question. >> >> When I look at server explorer I see a data connection object, however > there are two objects >> underneath, neither of which are for this solution. Furthermore the > objects under the data >> connection are things found on an entirely unrelated server, for other > projects that I did work on >> in the past. >> >> The question then is why is the server explorer choosing to display > objects out on a virtual machine >> instead of the database that was originally selected for this solution? >> >> We have done a ton of coding sql server stuff here but we have not yet > learned how to do this bound >> form stuff. >> >> I am trying to build a timeclock kind of thing. Very simple, a few combos > to select (default values >> in most cases) the developer, the product, the type of work, then a couple > of fields for date time >> values for start / stop work. The table the data is going into has a few > thousand records already >> in it, the combos pull from tables with 20-30 records etc. It seems (to > me) that just using bound >> forms is a natural for this. >> >> A long time ago I made this (kind of thing) work. I remember that getting > the properties of the >> combo set correctly was non-intuitive but once I figured it out it just > worked. >> >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/14/2011 3:02 PM, Gustav Brock wrote: >>> Hi John >>> >>> If you (typically) have this in InitializeComponent() of the .Designer.cs > file: >>> >>> // comboBoxCountry >>> // >>> this.comboBoxCountry.AutoCompleteMode = > System.Windows.Forms.AutoCompleteMode.Suggest; >>> this.comboBoxCountry.AutoCompleteSource = > System.Windows.Forms.AutoCompleteSource.ListItems; >>> this.comboBoxCountry.BackColor = > System.Drawing.Color.LavenderBlush; >>> this.comboBoxCountry.DataBindings.Add(new > System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, > "CountryId", true)); >>> this.comboBoxCountry.DisplayMember = "Id"; >>> this.comboBoxCountry.FlatStyle = > System.Windows.Forms.FlatStyle.Flat; >>> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe > UI", 8F, System.Drawing.FontStyle.Regular, > System.Drawing.GraphicsUnit.Point, ((byte)(0))); >>> this.comboBoxCountry.ForeColor = > System.Drawing.Color.DarkMagenta; >>> this.comboBoxCountry.FormattingEnabled = true; >>> this.comboBoxCountry.ItemHeight = 13; >>> this.comboBoxCountry.Location = new > System.Drawing.Point(21, 136); >>> this.comboBoxCountry.Name = "comboBoxCountry"; >>> this.comboBoxCountry.Size = new System.Drawing.Size(117, > 21); >>> this.comboBoxCountry.Sorted = true; >>> this.comboBoxCountry.TabIndex = 7; >>> this.comboBoxCountry.ValueMember = "Id"; >>> >>> You can do something like this in the Load event: >>> >>> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => > ChangeCountrySelection(); >>> >>> Then: >>> private void FillComboBoxCountryCode() >>> { >>> // Order the country names of countryDataTable. >>> string filterExpression = ""; >>> string sortExpression = "CountryNameUK asc"; >>> DataRow[] countryList = > _countryDataTable.Select(filterExpression, sortExpression); >>> >>> this.comboBoxCountry.DataSource = countryList; >>> this.comboBoxCountry.ValueMember = "Id"; >>> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >>> } >>> >>> And: >>> private void ChangeCountrySelection() >>> { >>> if (this.comboBoxCountry.SelectedValue != null) >>> { >>> _countryId = > Convert.ToInt32(this.comboBoxCountry.SelectedValue); >>> this.comboBoxPostalCode.Text = String.Empty; >>> // Read postal codes of selected country. >>> FillComboBoxPostalCode(); >>> } >>> } >>> >>> This is pretty standard using tableadapters and bindingnavigator. Nothing > fancy. >>> >>> /gustav >>> >>> >>>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >>> I am trying to set up a combo box bound to a field in the form such that > the combo populates with >>> the PKID and a text field, and when the form opens, selecting something > from the combo drops the >>> PKID into the field in the form. >>> >>> I am not getting what to set the properties of the combo box to. I am > pulling the ID and the string >>> to display into the combo but I cannot cause the combo to store the id > and release control to the >>> next control on the form. >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From accessd at shaw.ca Tue Mar 15 02:19:11 2011 From: accessd at shaw.ca (Jim Lawrence) Date: Tue, 15 Mar 2011 00:19:11 -0700 Subject: [dba-VB] C# combo boxes In-Reply-To: <4D7ECB45.8060609@colbyconsulting.com> References: <4D7E95C4.9020203@colbyconsulting.com> <4D7ECB45.8060609@colbyconsulting.com> Message-ID: Congratulation John. Keep us posted on it stability. Jim -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, March 14, 2011 7:13 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# combo boxes I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com On 3/14/2011 7:59 PM, Jim Lawrence wrote: > Hi John: > > Try and use ADO. It is on every Windows based computer already and requires > no additional ODBC to be setup. In other words, you do not even have to see > the computers on which the application is going to have a sable connection. > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Monday, March 14, 2011 3:25 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] C# combo boxes > > I am trying (unsuccessfully) to do an "Access like bound form". > > And I always comment. ;) > > ATM the form displays the values if dropped down, but if not dropped down it > just displays an > integer (the PKID of the data in the combo is my guess, or perhaps the PKID > in the field in the form. > > So... > > How do I cause the combo to display Paul's name instead of 6. > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 6:15 PM, Gustav Brock wrote: >> Hi John >> >> Eh, are you commenting or? Not quite clear to me. I thought you were > looking for some code samples ... >> >> I have connections too but I use them only to feed the datatableadapters. > I refrain from working with SqlCommand and the like; waste of time in my > opinion. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> >> Thanks Gustav, >> I don't know why you are taking me into code in code. >> >> I have data connections. This particular data connection has a single > view that allows me to write >> into a table. It has a view that filters a combo. >> >> I drag and drop the view for the table onto the form. I have told that > table object that specific >> fields are combos so when I drag it out, I have combos (for the FKs - PKs > from other tables). I am >> trying to set up that combo to >> >> 1) Display a text value >> 2) Contain the PKID from the table the combo draws from >> 3) When I select an item, place the selected ID into the field the combo > is bound to. >> >> It seems this is a normal daily requirement in building forms, and > shouldn't be rocket science (or >> going into code). >> >> Actually I have a kinda sorta related question. >> >> When I look at server explorer I see a data connection object, however > there are two objects >> underneath, neither of which are for this solution. Furthermore the > objects under the data >> connection are things found on an entirely unrelated server, for other > projects that I did work on >> in the past. >> >> The question then is why is the server explorer choosing to display > objects out on a virtual machine >> instead of the database that was originally selected for this solution? >> >> We have done a ton of coding sql server stuff here but we have not yet > learned how to do this bound >> form stuff. >> >> I am trying to build a timeclock kind of thing. Very simple, a few combos > to select (default values >> in most cases) the developer, the product, the type of work, then a couple > of fields for date time >> values for start / stop work. The table the data is going into has a few > thousand records already >> in it, the combos pull from tables with 20-30 records etc. It seems (to > me) that just using bound >> forms is a natural for this. >> >> A long time ago I made this (kind of thing) work. I remember that getting > the properties of the >> combo set correctly was non-intuitive but once I figured it out it just > worked. >> >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/14/2011 3:02 PM, Gustav Brock wrote: >>> Hi John >>> >>> If you (typically) have this in InitializeComponent() of the .Designer.cs > file: >>> >>> // comboBoxCountry >>> // >>> this.comboBoxCountry.AutoCompleteMode = > System.Windows.Forms.AutoCompleteMode.Suggest; >>> this.comboBoxCountry.AutoCompleteSource = > System.Windows.Forms.AutoCompleteSource.ListItems; >>> this.comboBoxCountry.BackColor = > System.Drawing.Color.LavenderBlush; >>> this.comboBoxCountry.DataBindings.Add(new > System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, > "CountryId", true)); >>> this.comboBoxCountry.DisplayMember = "Id"; >>> this.comboBoxCountry.FlatStyle = > System.Windows.Forms.FlatStyle.Flat; >>> this.comboBoxCountry.Font = new System.Drawing.Font("Segoe > UI", 8F, System.Drawing.FontStyle.Regular, > System.Drawing.GraphicsUnit.Point, ((byte)(0))); >>> this.comboBoxCountry.ForeColor = > System.Drawing.Color.DarkMagenta; >>> this.comboBoxCountry.FormattingEnabled = true; >>> this.comboBoxCountry.ItemHeight = 13; >>> this.comboBoxCountry.Location = new > System.Drawing.Point(21, 136); >>> this.comboBoxCountry.Name = "comboBoxCountry"; >>> this.comboBoxCountry.Size = new System.Drawing.Size(117, > 21); >>> this.comboBoxCountry.Sorted = true; >>> this.comboBoxCountry.TabIndex = 7; >>> this.comboBoxCountry.ValueMember = "Id"; >>> >>> You can do something like this in the Load event: >>> >>> this.comboBoxCountry.SelectedValueChanged += (oo, ee) => > ChangeCountrySelection(); >>> >>> Then: >>> private void FillComboBoxCountryCode() >>> { >>> // Order the country names of countryDataTable. >>> string filterExpression = ""; >>> string sortExpression = "CountryNameUK asc"; >>> DataRow[] countryList = > _countryDataTable.Select(filterExpression, sortExpression); >>> >>> this.comboBoxCountry.DataSource = countryList; >>> this.comboBoxCountry.ValueMember = "Id"; >>> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >>> } >>> >>> And: >>> private void ChangeCountrySelection() >>> { >>> if (this.comboBoxCountry.SelectedValue != null) >>> { >>> _countryId = > Convert.ToInt32(this.comboBoxCountry.SelectedValue); >>> this.comboBoxPostalCode.Text = String.Empty; >>> // Read postal codes of selected country. >>> FillComboBoxPostalCode(); >>> } >>> } >>> >>> This is pretty standard using tableadapters and bindingnavigator. Nothing > fancy. >>> >>> /gustav >>> >>> >>>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >>> I am trying to set up a combo box bound to a field in the form such that > the combo populates with >>> the PKID and a text field, and when the form opens, selecting something > from the combo drops the >>> PKID into the field in the form. >>> >>> I am not getting what to set the properties of the combo box to. I am > pulling the ID and the string >>> to display into the combo but I cannot cause the combo to store the id > and release control to the >>> next control on the form. >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From Gustav at cactus.dk Tue Mar 15 04:55:02 2011 From: Gustav at cactus.dk (Gustav Brock) Date: Tue, 15 Mar 2011 10:55:02 +0100 Subject: [dba-VB] C# combo boxes Message-ID: Hi John Good! Yes, the content of Text is handled by DisplayMember: this.comboBoxCountry.DataSource = countryList; this.comboBoxCountry.ValueMember = "Id"; this.comboBoxCountry.DisplayMember = "CountryNameUK"; /gustav >>> jwcolby at colbyconsulting.com 15-03-2011 03:13 >>> I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com From df.waters at comcast.net Tue Mar 15 08:54:00 2011 From: df.waters at comcast.net (Dan Waters) Date: Tue, 15 Mar 2011 08:54:00 -0500 Subject: [dba-VB] C# combo boxes In-Reply-To: <4D7ECB45.8060609@colbyconsulting.com> References: <4D7E95C4.9020203@colbyconsulting.com> <4D7ECB45.8060609@colbyconsulting.com> Message-ID: <002101cbe318$713acbb0$53b06310$@comcast.net> Hi John, Take a look at www.updatecontrols.net. This has a free download which integrates into visual studio and provides a way to simply databinding. I haven't used it yet but it looks promising. Note: At the top of the screens is a combobox which selects either VB or C# for the code examples. Dan -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: Monday, March 14, 2011 9:13 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# combo boxes I got it working. I had to not only set the properties in the datasource property dialog of the combo box, but I also had to set the data bindings properties in the combo, specifically setting the text property there to NONE. She's a wurkin. John W. Colby www.ColbyConsulting.com On 3/14/2011 7:59 PM, Jim Lawrence wrote: > Hi John: > > Try and use ADO. It is on every Windows based computer already and > requires no additional ODBC to be setup. In other words, you do not > even have to see the computers on which the application is going to have a sable connection. > > Jim > > > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: Monday, March 14, 2011 3:25 PM > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] C# combo boxes > > I am trying (unsuccessfully) to do an "Access like bound form". > > And I always comment. ;) > > ATM the form displays the values if dropped down, but if not dropped > down it just displays an integer (the PKID of the data in the combo is > my guess, or perhaps the PKID in the field in the form. > > So... > > How do I cause the combo to display Paul's name instead of 6. > > John W. Colby > www.ColbyConsulting.com > > On 3/14/2011 6:15 PM, Gustav Brock wrote: >> Hi John >> >> Eh, are you commenting or? Not quite clear to me. I thought you were > looking for some code samples ... >> >> I have connections too but I use them only to feed the datatableadapters. > I refrain from working with SqlCommand and the like; waste of time in > my opinion. >> >> /gustav >> >> >>>>> jwcolby at colbyconsulting.com 14-03-2011 22:36>>> >> Thanks Gustav, >> I don't know why you are taking me into code in code. >> >> I have data connections. This particular data connection has a >> single > view that allows me to write >> into a table. It has a view that filters a combo. >> >> I drag and drop the view for the table onto the form. I have told >> that > table object that specific >> fields are combos so when I drag it out, I have combos (for the FKs - >> PKs > from other tables). I am >> trying to set up that combo to >> >> 1) Display a text value >> 2) Contain the PKID from the table the combo draws from >> 3) When I select an item, place the selected ID into the field the >> combo > is bound to. >> >> It seems this is a normal daily requirement in building forms, and > shouldn't be rocket science (or >> going into code). >> >> Actually I have a kinda sorta related question. >> >> When I look at server explorer I see a data connection object, >> however > there are two objects >> underneath, neither of which are for this solution. Furthermore the > objects under the data >> connection are things found on an entirely unrelated server, for >> other > projects that I did work on >> in the past. >> >> The question then is why is the server explorer choosing to display > objects out on a virtual machine >> instead of the database that was originally selected for this solution? >> >> We have done a ton of coding sql server stuff here but we have not >> yet > learned how to do this bound >> form stuff. >> >> I am trying to build a timeclock kind of thing. Very simple, a few >> combos > to select (default values >> in most cases) the developer, the product, the type of work, then a >> couple > of fields for date time >> values for start / stop work. The table the data is going into has a >> few > thousand records already >> in it, the combos pull from tables with 20-30 records etc. It seems >> (to > me) that just using bound >> forms is a natural for this. >> >> A long time ago I made this (kind of thing) work. I remember that >> getting > the properties of the >> combo set correctly was non-intuitive but once I figured it out it >> just > worked. >> >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/14/2011 3:02 PM, Gustav Brock wrote: >>> Hi John >>> >>> If you (typically) have this in InitializeComponent() of the >>> .Designer.cs > file: >>> >>> // comboBoxCountry >>> // >>> this.comboBoxCountry.AutoCompleteMode = > System.Windows.Forms.AutoCompleteMode.Suggest; >>> this.comboBoxCountry.AutoCompleteSource = > System.Windows.Forms.AutoCompleteSource.ListItems; >>> this.comboBoxCountry.BackColor = > System.Drawing.Color.LavenderBlush; >>> this.comboBoxCountry.DataBindings.Add(new > System.Windows.Forms.Binding("SelectedValue", this.mediaBindingSource, > "CountryId", true)); >>> this.comboBoxCountry.DisplayMember = "Id"; >>> this.comboBoxCountry.FlatStyle = > System.Windows.Forms.FlatStyle.Flat; >>> this.comboBoxCountry.Font = new >>> System.Drawing.Font("Segoe > UI", 8F, System.Drawing.FontStyle.Regular, > System.Drawing.GraphicsUnit.Point, ((byte)(0))); >>> this.comboBoxCountry.ForeColor = > System.Drawing.Color.DarkMagenta; >>> this.comboBoxCountry.FormattingEnabled = true; >>> this.comboBoxCountry.ItemHeight = 13; >>> this.comboBoxCountry.Location = new > System.Drawing.Point(21, 136); >>> this.comboBoxCountry.Name = "comboBoxCountry"; >>> this.comboBoxCountry.Size = new >>> System.Drawing.Size(117, > 21); >>> this.comboBoxCountry.Sorted = true; >>> this.comboBoxCountry.TabIndex = 7; >>> this.comboBoxCountry.ValueMember = "Id"; >>> >>> You can do something like this in the Load event: >>> >>> this.comboBoxCountry.SelectedValueChanged += (oo, ee) >>> => > ChangeCountrySelection(); >>> >>> Then: >>> private void FillComboBoxCountryCode() >>> { >>> // Order the country names of countryDataTable. >>> string filterExpression = ""; >>> string sortExpression = "CountryNameUK asc"; >>> DataRow[] countryList = > _countryDataTable.Select(filterExpression, sortExpression); >>> >>> this.comboBoxCountry.DataSource = countryList; >>> this.comboBoxCountry.ValueMember = "Id"; >>> this.comboBoxCountry.DisplayMember = "CountryNameUK"; >>> } >>> >>> And: >>> private void ChangeCountrySelection() >>> { >>> if (this.comboBoxCountry.SelectedValue != null) >>> { >>> _countryId = > Convert.ToInt32(this.comboBoxCountry.SelectedValue); >>> this.comboBoxPostalCode.Text = String.Empty; >>> // Read postal codes of selected country. >>> FillComboBoxPostalCode(); >>> } >>> } >>> >>> This is pretty standard using tableadapters and bindingnavigator. >>> Nothing > fancy. >>> >>> /gustav >>> >>> >>>>>> jwcolby at colbyconsulting.com 14-03-2011 19:16>>> >>> I am trying to set up a combo box bound to a field in the form such >>> that > the combo populates with >>> the PKID and a text field, and when the form opens, selecting >>> something > from the combo drops the >>> PKID into the field in the form. >>> >>> I am not getting what to set the properties of the combo box to. I >>> am > pulling the ID and the string >>> to display into the combo but I cannot cause the combo to store the >>> id > and release control to the >>> next control on the form. >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From dbdoug at gmail.com Wed Mar 16 10:28:18 2011 From: dbdoug at gmail.com (Doug Steele) Date: Wed, 16 Mar 2011 08:28:18 -0700 Subject: [dba-VB] VS2010 install Message-ID: Hello all: I just finished doing a clean install of Windows 7 then VS2010. To my surprise, I also appear to have a functioning install of VS2008, loading with the name 'Shell' where my old VS2008 says 'Professional Edition'. I couldn't see anything on the net about this; does anyone have any further information? Thanks, Doug From DWUTKA at Marlow.com Fri Mar 18 14:12:52 2011 From: DWUTKA at Marlow.com (Drew Wutka) Date: Fri, 18 Mar 2011 14:12:52 -0500 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: Message-ID: 500 megs... that's about 10 minutes... wouldn't have time for a full dinner.... Drew -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 Hi all If you go for this: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-810 7-475d-948a-ef22627e57a5&displaylang=en it's typically a 500MB+ download with a streaming install. Don't go for coffee during this, don't go for lunch, not even a single-course dinner, don't settle for less than a full menu dinner followed by coffee and brandy. Or book an evening in town! /gustav _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com The information contained in this transmission is intended only for the person or entity to which it is addressed and may contain II-VI Proprietary and/or II-VI Business Sensitive material. If you are not the intended recipient, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. You are notified that any review, retransmission, copying, disclosure, dissemination, or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. From stuart at lexacorp.com.pg Fri Mar 18 18:50:01 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 09:50:01 +1000 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: , Message-ID: <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg> That's about 10 hours and US$35 for me :-( -- Stuart On 18 Mar 2011 at 14:12, Drew Wutka wrote: > 500 megs... that's about 10 minutes... wouldn't have time for a full > dinner.... > > Drew > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com > Subject: [dba-VB] Visual Studio 2010 SP1 > > Hi all > > If you go for this: > http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8 > 10 7-475d-948a-ef22627e57a5&displaylang=en > > it's typically a 500MB+ download with a streaming install. > > Don't go for coffee during this, don't go for lunch, not even a > single-course dinner, don't settle for less than a full menu dinner > followed by coffee and brandy. Or book an evening in town! > > /gustav > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > The information contained in this transmission is intended only for > the person or entity to which it is addressed and may contain II-VI > Proprietary and/or II-VI Business Sensitive material. If you are not > the intended recipient, please contact the sender immediately and > destroy the material in its entirety, whether electronic or hard copy. > You are notified that any review, retransmission, copying, disclosure, > dissemination, or other use of, or taking of any action in reliance > upon this information by persons or entities other than the intended > recipient is prohibited. > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Mar 18 19:02:30 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 03:02:30 +0300 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg> References: , <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg> Message-ID: Hi Stuart -- <<< That's about 10 hours and US$35 for me :-( >>> But you never have a snow there, and your local police doesn't mind if you drive home your car after a couple of beers? :) I mean 500megs download takes just a couple of minutes here and near to zero expences but we still have a snow here and Finnish Gulf is covered by 1 meter+ wide ice and I'm skiing here almost everyday, and if stopped by local road police after a couple beers I would definitely loose my car driving license for 1.5 years, at least... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart McLachlan Sent: 19 ????? 2011 ?. 2:50 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Visual Studio 2010 SP1 That's about 10 hours and US$35 for me :-( -- Stuart On 18 Mar 2011 at 14:12, Drew Wutka wrote: > 500 megs... that's about 10 minutes... wouldn't have time for a full > dinner.... > > Drew > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav Brock > Sent: Friday, March 11, 2011 11:59 AM To: dba-vb at databaseadvisors.com > Subject: [dba-VB] Visual Studio 2010 SP1 > > Hi all > > If you go for this: > http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6-8 > 10 7-475d-948a-ef22627e57a5&displaylang=en > > it's typically a 500MB+ download with a streaming install. > > Don't go for coffee during this, don't go for lunch, not even a > single-course dinner, don't settle for less than a full menu dinner > followed by coffee and brandy. Or book an evening in town! > > /gustav > > From stuart at lexacorp.com.pg Fri Mar 18 19:09:18 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 10:09:18 +1000 Subject: [dba-VB] Visual Studio 2010 SP1 In-Reply-To: References: , <4D83EFA9.2386.73B0D9@stuart.lexacorp.com.pg>, Message-ID: <4D83F42E.13556.8558BC@stuart.lexacorp.com.pg> Yep, The good points still outweigh the bad. :-) -- Stuart On 19 Mar 2011 at 3:02, Shamil Salakhetdinov wrote: > Hi Stuart -- > > <<< > That's about 10 hours and US$35 for me :-( > >>> > But you never have a snow there, and your local police doesn't mind if > you drive home your car after a couple of beers? :) > > I mean 500megs download takes just a couple of minutes here and near > to zero expences but we still have a snow here and Finnish Gulf is > covered by 1 meter+ wide ice and I'm skiing here almost everyday, and > if stopped by local road police after a couple beers I would > definitely loose my car driving license for 1.5 years, at least... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart > McLachlan Sent: 19 ????? 2011 ?. 2:50 To: Discussion concerning Visual > Basic and related programming issues. Subject: Re: [dba-VB] Visual > Studio 2010 SP1 > > That's about 10 hours and US$35 for me :-( > > -- > Stuart > > On 18 Mar 2011 at 14:12, Drew Wutka wrote: > > > 500 megs... that's about 10 minutes... wouldn't have time for a full > > dinner.... > > > > Drew > > > > -----Original Message----- > > From: dba-vb-bounces at databaseadvisors.com > > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Gustav > > Brock Sent: Friday, March 11, 2011 11:59 AM To: > > dba-vb at databaseadvisors.com Subject: [dba-VB] Visual Studio 2010 SP1 > > > > Hi all > > > > If you go for this: > > http://www.microsoft.com/downloads/en/details.aspx?FamilyID=75568aa6 > > -8 10 7-475d-948a-ef22627e57a5&displaylang=en > > > > it's typically a 500MB+ download with a streaming install. > > > > Don't go for coffee during this, don't go for lunch, not even a > > single-course dinner, don't settle for less than a full menu dinner > > followed by coffee and brandy. Or book an evening in town! > > > > /gustav > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Fri Mar 18 21:08:01 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 18 Mar 2011 22:08:01 -0400 Subject: [dba-VB] SHA1 to compute a hash Message-ID: <4D841001.2060200@colbyconsulting.com> In my databases I create SHA1 hashes to enable joining between tables and pull identical records (identical for the fields hashed). I create: 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the three values and feed them into SHa1 and out pops a number which I store in a field in my table. 2) A HashFamily of the Zip5, Zip4, Addr ad LName. 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. I am getting known collisions between different addresses (I have discovered and investigated collisions) in my HashAddr when I have many millions of addresses. I need to address this. Back when I made my design decisions (2004) my hardware consisted of single core processors, 4 gigs ram, Windows x32 etc. Now I have 8 cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great extent constrained by my hardware "back in the day" whereas I am much less so now. I am about to redesign my process. I am considering simply appending in the city and state strings to all of the inputs: Addr, City, St, Zip5, Zip4 as the address base and then the same with LName and FName for the other two respective hashes. The objective is to minimize hash collisions, not prevent some crypto attack. I use these hash fields to join between multi million record tables so If I need to discover info in TableA where the HashAddr is the same as in TableB, I need the probability of a collision between different addresses (family/Person) to be as close to zero as I can get it. My questions are: 1) Whether anyone out there is using a hash in this manner? 2) Has anyone seen a table of collision probability between messages of a given (short) message length. My message is 9 digits for the zip5/4 and the address could be something as short as PO Box 1, or Apt 1. IOW the total message length of 14 is pretty common. Adding the state would give me minimum message lengths of only 16 and City would only add a few more characters. 3) Does anyone know if just adding the same data back in again would decrease the collision probability. IOW Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. Any experience out there? -- John W. Colby www.ColbyConsulting.com From stuart at lexacorp.com.pg Sat Mar 19 05:20:16 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 20:20:16 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D841001.2060200@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com> Message-ID: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> How are you creating your hash? Can you post a few examples of different data strings and colliding SHA1 hashes. I can probably make a lot of money out of them. AFAIK, no one other than you has found any. -- Stuart On 18 Mar 2011 at 22:08, jwcolby wrote: > In my databases I create SHA1 hashes to enable joining between tables > and pull identical records (identical for the fields hashed). I > create: > > 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the > three values and feed them into SHa1 and out pops a number which I > store in a field in my table. > > 2) A HashFamily of the Zip5, Zip4, Addr ad LName. > > 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. > > I am getting known collisions between different addresses (I have > discovered and investigated collisions) in my HashAddr when I have > many millions of addresses. I need to address this. > > Back when I made my design decisions (2004) my hardware consisted of > single core processors, 4 gigs ram, Windows x32 etc. Now I have 8 > cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great extent > constrained by my hardware "back in the day" whereas I am much less so > now. > > I am about to redesign my process. > > I am considering simply appending in the city and state strings to all > of the inputs: Addr, City, St, Zip5, Zip4 as the address base and then > the same with LName and FName for the other two respective hashes. > > The objective is to minimize hash collisions, not prevent some crypto > attack. I use these hash fields to join between multi million record > tables so If I need to discover info in TableA where the HashAddr is > the same as in TableB, I need the probability of a collision between > different addresses (family/Person) to be as close to zero as I can > get it. > > My questions are: > > 1) Whether anyone out there is using a hash in this manner? > 2) Has anyone seen a table of collision probability between messages > of a given (short) message length. My message is 9 digits for the > zip5/4 and the address could be something as short as PO Box 1, or Apt > 1. IOW the total message length of 14 is pretty common. Adding the > state would give me minimum message lengths of only 16 and City would > only add a few more characters. 3) Does anyone know if just adding the > same data back in again would decrease the collision probability. IOW > Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. > > Any experience out there? > > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 06:20:26 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 21:20:26 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> Using your SHA1 function, what message digests do you get for the standard test cases: 1. abc 2. abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq These should return: 1. A9993E36 4706816A BA3E2571 7850C26C 9CD0D89D 2. 84983E44 1C3BD26E BAAE4AA1 F95129E5 E54670F1 See Examples 1 and 2 at http://www.itl.nist.gov/fipspubs/fip180-1.htm -- Stuart On 19 Mar 2011 at 20:20, Stuart McLachlan wrote: > How are you creating your hash? > > Can you post a few examples of different data strings and colliding > SHA1 hashes. I can probably make a lot of money out of them. > AFAIK, no one other than you has found any. > > -- > Stuart > > > On 18 Mar 2011 at 22:08, jwcolby wrote: > > > In my databases I create SHA1 hashes to enable joining between > > tables and pull identical records (identical for the fields hashed). > > I create: > > > > 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the > > three values and feed them into SHa1 and out pops a number which I > > store in a field in my table. > > > > 2) A HashFamily of the Zip5, Zip4, Addr ad LName. > > > > 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. > > > > I am getting known collisions between different addresses (I have > > discovered and investigated collisions) in my HashAddr when I have > > many millions of addresses. I need to address this. > > > > Back when I made my design decisions (2004) my hardware consisted of > > single core processors, 4 gigs ram, Windows x32 etc. Now I have 8 > > cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great extent > > constrained by my hardware "back in the day" whereas I am much less > > so now. > > > > I am about to redesign my process. > > > > I am considering simply appending in the city and state strings to > > all of the inputs: Addr, City, St, Zip5, Zip4 as the address base > > and then the same with LName and FName for the other two respective > > hashes. > > > > The objective is to minimize hash collisions, not prevent some > > crypto attack. I use these hash fields to join between multi > > million record tables so If I need to discover info in TableA where > > the HashAddr is the same as in TableB, I need the probability of a > > collision between different addresses (family/Person) to be as close > > to zero as I can get it. > > > > My questions are: > > > > 1) Whether anyone out there is using a hash in this manner? > > 2) Has anyone seen a table of collision probability between messages > > of a given (short) message length. My message is 9 digits for the > > zip5/4 and the address could be something as short as PO Box 1, or > > Apt 1. IOW the total message length of 14 is pretty common. Adding > > the state would give me minimum message lengths of only 16 and City > > would only add a few more characters. 3) Does anyone know if just > > adding the same data back in again would decrease the collision > > probability. IOW Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. > > > > Any experience out there? > > > > > > -- > > John W. Colby > > www.ColbyConsulting.com > > _______________________________________________ > > dba-VB mailing list > > dba-VB at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > http://www.databaseadvisors.com > > > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 07:06:47 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 22:06:47 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> Message-ID: <4D849C57.3867.31639E4@stuart.lexacorp.com.pg> That got my interest piqued. So I've just built a small DLL (13KB) to calculate SHA1 digests for either strings or files. All you need to use it is to put the DLL somewhere in your PATH such as %SYSTEMROOT\System32 and include a couple of declarations in your application. Declare Function SHA1StringDigest Lib "LXSHA1.DLL" Alias "SHA1STRINGDIGEST" (Message As String) As String Declare Function SHA1FileDigest Lib "LXSHA1.DLL" Alias "SHA1FILEDIGEST" (FileName As String) As String It correctly returns the test results below for either the string or a file containing the same data. If anyone wants a copy, let me know. -- Stuart On 19 Mar 2011 at 21:20, Stuart McLachlan wrote: > Using your SHA1 function, what message digests do you get for the > standard test cases: > > 1. abc > 2. abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq > > These should return: > > 1. A9993E36 4706816A BA3E2571 7850C26C 9CD0D89D > 2. 84983E44 1C3BD26E BAAE4AA1 F95129E5 E54670F1 > > See Examples 1 and 2 at > http://www.itl.nist.gov/fipspubs/fip180-1.htm > > > > -- > Stuart > > > On 19 Mar 2011 at 20:20, Stuart McLachlan wrote: > > > How are you creating your hash? > > > > Can you post a few examples of different data strings and colliding > > SHA1 hashes. I can probably make a lot of money out of them. > > AFAIK, no one other than you has found any. > > > > -- > > Stuart > > > > > > On 18 Mar 2011 at 22:08, jwcolby wrote: > > > > > In my databases I create SHA1 hashes to enable joining between > > > tables and pull identical records (identical for the fields > > > hashed). > > > I create: > > > > > > 1) A HashAddr of the zip5, zip4 and addr. IOW I simply append the > > > three values and feed them into SHa1 and out pops a number which I > > > store in a field in my table. > > > > > > 2) A HashFamily of the Zip5, Zip4, Addr ad LName. > > > > > > 3) A HashPerson of Zip5, Zip4, Addr, LName and FName. > > > > > > I am getting known collisions between different addresses (I have > > > discovered and investigated collisions) in my HashAddr when I have > > > many millions of addresses. I need to address this. > > > > > > Back when I made my design decisions (2004) my hardware consisted > > > of single core processors, 4 gigs ram, Windows x32 etc. Now I > > > have 8 cores, 32 gigs Ram, Windows X64 etc. IOW I was to a great > > > extent constrained by my hardware "back in the day" whereas I am > > > much less so now. > > > > > > I am about to redesign my process. > > > > > > I am considering simply appending in the city and state strings to > > > all of the inputs: Addr, City, St, Zip5, Zip4 as the address base > > > and then the same with LName and FName for the other two > > > respective hashes. > > > > > > The objective is to minimize hash collisions, not prevent some > > > crypto attack. I use these hash fields to join between multi > > > million record tables so If I need to discover info in TableA > > > where the HashAddr is the same as in TableB, I need the > > > probability of a collision between different addresses > > > (family/Person) to be as close to zero as I can get it. > > > > > > My questions are: > > > > > > 1) Whether anyone out there is using a hash in this manner? > > > 2) Has anyone seen a table of collision probability between > > > messages of a given (short) message length. My message is 9 > > > digits for the zip5/4 and the address could be something as short > > > as PO Box 1, or Apt 1. IOW the total message length of 14 is > > > pretty common. Adding the state would give me minimum message > > > lengths of only 16 and City would only add a few more characters. > > > 3) Does anyone know if just adding the same data back in again > > > would decrease the collision probability. IOW > > > Zip5,Zip4,Addr,City,St,Zip5,Zip4Etc. > > > > > > Any experience out there? > > > > > > > > > -- > > > John W. Colby > > > www.ColbyConsulting.com > > > _______________________________________________ > > > dba-VB mailing list > > > dba-VB at databaseadvisors.com > > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > > http://www.databaseadvisors.com > > > > > > > > > > > > > > _______________________________________________ > > dba-VB mailing list > > dba-VB at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > http://www.databaseadvisors.com > > > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 07:09:56 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 08:09:56 -0400 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com> <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: <4D849D14.70906@colbyconsulting.com> >AFAIK, no one other than you has found any. LOL. You are thinking of hashing messages with thousands of characters. here's the deal. Let's assume that I hash X characters in a single field. The message length is the key, not the number of fields. Assume also that I am really only hashing letters, numbers and special characters - the characters that are in a name or address. If I hash 100 1 character messages, my probability of a collision if extremely high since there are only about 128 such alphanumeric characters (even less). Now hash a thousand 1 character messages. The probably just climbed immensely. Now hash a million such 1 character messages. What I am really calculating is the probability that I will repeat the same character string. However if I up the number of characters, let's say 10 character messages. *Now* the hash is really trying to prevent the same output for two *different* inputs. Well that is always what a hash is trying to do. But the point is that the probability of a collision decreases with message length increase. The probability of a collision between any two messages of 10 random characters is much higher than the probability of a collision between any two 100 character messages, and the probability of a collision between any two 1000 character messages is much higher still. That is just the way hash functions work. Nobody claims that hashes don't create collisions, and everyone agrees that the longer the message, the lower the probability of a collision between two messages (to a point). So, my address "messages" currently look like 89364 4456 PO Box 1 76543 9876 Apt 2 97867 3546 1723 Twin Pines Dr The point here is that all address "messages" are short. Now turn the address message into 89364 4456 PO Box 1 CA San Diego 76543 9876 Apt 2 TX Dallas 97867 3546 1723 Twin Pines Dr NC Hudson And the length of the message increases fairly dramatically, decreasing the probability of a collision. Remember too that I am hashing hundreds of millions of records - about 350 million addresses in various tables so far. The other thing that affects the probability of a hash collision is the number of messages hashed. Hash enough records and you *will* 100% probability create a hash that is the same for two different input strings - even for long messages. That is just the nature of the business. The probability of a collision is determined by three things. 1) The (strength of the) hash algorithm. 2) The message length 3) The quantity of messages. 1) I am using a fairly strong hash algorithm - SHA1 2) I am using a fairly short hash message - Zip5 / zip4 / Addr 3) I am hashing a fairly high number of messages - 350 million (and counting) messages (addresses) Believe me, hashes have already occurred, I have seen them. It is an easy thing to test. Join two tables on the same hash and in the query compare that each address field is the same between the two tables. Zip5 <> zip5 *or* zip4 <> zip4 *or* addr <> addr. If the hash is the same (inner join) AND one of the three fields is different then you have a collision. I have done this test and I have found collisions between my address hash. I have never tested my last name or first name hash fields. The problem with finding hashes is that they are rare, and in order to do an exhaustive test I would need to pull all the hash and data fields into a single table and compare that table against itself. My data is contained in a bunch of databases. 11 million "dogs and cats", 21 million "kids", 7 million "smokers", 23 million (and about to double) emails. 65 million "all adults", 100 million "mortgage" etc. So it is difficult to find hash collisions because (at least in the small tables) the probability of a collision is low entirely within a table. Believe me, it *does* happen though (at least for the address hash). John W. Colby www.ColbyConsulting.com On 3/19/2011 6:20 AM, Stuart McLachlan wrote: > How are you creating your hash? > > Can you post a few examples of different data strings and colliding SHA1 hashes. I can > probably make a lot of money out of them. AFAIK, no one other than you has found any. > From jwcolby at colbyconsulting.com Sat Mar 19 07:11:32 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 08:11:32 -0400 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D849C57.3867.31639E4@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D84917A.28865.2EBCA7E@stuart.lexacorp.com.pg> <4D849C57.3867.31639E4@stuart.lexacorp.com.pg> Message-ID: <4D849D74.2040104@colbyconsulting.com> I will do a collision test in my email file. AFAICT I have some collisions in there. If I find any I will send you the message and hash so that you can see what collisions look like. John W. Colby www.ColbyConsulting.com On 3/19/2011 8:06 AM, Stuart McLachlan wrote: > That got my interest piqued. > > So I've just built a small DLL (13KB) to calculate SHA1 digests for either strings or files. > All you need to use it is to put the DLL somewhere in your PATH such as > %SYSTEMROOT\System32 and include a couple of declarations in your application. > > Declare Function SHA1StringDigest Lib "LXSHA1.DLL" Alias "SHA1STRINGDIGEST" > (Message As String) As String > > Declare Function SHA1FileDigest Lib "LXSHA1.DLL" Alias "SHA1FILEDIGEST" (FileName > As String) As String > > It correctly returns the test results below for either the string or a file containing the same > data. > > If anyone wants a copy, let me know. > From harlan at powerzone.com Sat Mar 19 07:17:25 2011 From: harlan at powerzone.com (Harlan Koehn) Date: Sat, 19 Mar 2011 06:17:25 -0600 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D849D14.70906@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com> <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> <4D849D14.70906@colbyconsulting.com> Message-ID: <76EEB091F7CED543A80B5C2BC153975E0165A7C108@PZSERVER01.powerzone.local> John, I'm fairly ignorant when it comes to hashing algorithms but I'm curious if you have considered using a hashing algorithm that would generate a longer hash, which I'm assuming could reduce the likelihood of a collision. Harlan From stuart at lexacorp.com.pg Sat Mar 19 07:51:13 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 22:51:13 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D849D14.70906@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D849D14.70906@colbyconsulting.com> Message-ID: <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> In the case of 1000 single character messages, you are bound to get collisions since there are only 256 possible original messages. You will be hashing the same value multiple times Apart from that, your understanding is incorrect. It doesn't matter how long the string is. The chances of a collision with two different messages remains the same. The message is hashed using padded blocks of a fixed length. There is no more chance of a collision between "a" and "b" than there is between "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". Specifically, the chance of a collision within n different messages, using b bits of encryption is (n*(n-1)/2) * (1/2^b). Note that the length of the message doesn't come into that equation. The probability of a collision is determined by only TWO things. 1) The length of the digest. 2) The quantity of messages. As for "everyone agrees". Noone who understands how it works, agrees. -- Stuart On 19 Mar 2011 at 8:09, jwcolby wrote: > >AFAIK, no one other than you has found any. > > LOL. > > You are thinking of hashing messages with thousands of characters. > here's the deal. > > Let's assume that I hash X characters in a single field. The message > length is the key, not the number of fields. Assume also that I am > really only hashing letters, numbers and special characters - the > characters that are in a name or address. > > > If I hash 100 1 character messages, my probability of a collision if > extremely high since there are only about 128 such alphanumeric > characters (even less). > > Now hash a thousand 1 character messages. The probably just climbed > immensely. Now hash a million such 1 character messages. > What I am really calculating is the probability that I will repeat the > same character string. > > However if I up the number of characters, let's say 10 character > messages. *Now* the hash is really trying to prevent the same output > for two *different* inputs. Well that is always what a hash is trying > to do. But the point is that the probability of a collision decreases > with message length increase. > > The probability of a collision between any two messages of 10 random > characters is much higher than the probability of a collision between > any two 100 character messages, and the probability of a collision > between any two 1000 character messages is much higher still. > > That is just the way hash functions work. Nobody claims that hashes > don't create collisions, and everyone agrees that the longer the > message, the lower the probability of a collision between two messages > (to a point). > > So, my address "messages" currently look like > > 89364 4456 PO Box 1 > 76543 9876 Apt 2 > 97867 3546 1723 Twin Pines Dr > > The point here is that all address "messages" are short. Now turn the > address message into > > 89364 4456 PO Box 1 CA San Diego > 76543 9876 Apt 2 TX Dallas > 97867 3546 1723 Twin Pines Dr NC Hudson > > And the length of the message increases fairly dramatically, > decreasing the probability of a collision. > > Remember too that I am hashing hundreds of millions of records - about > 350 million addresses in various tables so far. The other thing that > affects the probability of a hash collision is the number of messages > hashed. Hash enough records and you *will* 100% probability create a > hash that is the same for two different input strings - even for long > messages. That is just the nature of the business. > > The probability of a collision is determined by three things. > > 1) The (strength of the) hash algorithm. > 2) The message length > 3) The quantity of messages. ... From shamil at smsconsulting.spb.ru Sat Mar 19 08:03:36 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 16:03:36 +0300 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com> <4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: Hi Stuart and John, <<< AFAIK, no one other than you has found any >>> Yes, I was also confused how it comes to get SHA1 hashes collisions but then I have thought that John probably "compacts" SHA1 hashes into 8 bytes surrogate keys? Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions for 6 millions cycles for 8 bytes keys. Please correct me if you find some errors in my sample code Thank you. -- Shamil P.S. Code: public class GenerateHashes { private static Dictionary _generatedHashes; private static Dictionary _generatedLongHashes; public static void Run(int minHashedStringLength, int maxHashedStringLength, long testCasesQty) { System.Console.WriteLine("Started at: " + System.DateTime.Now); _generatedHashes = new Dictionary(); _generatedLongHashes = new Dictionary(); int collisionsQty = 0; int longHashCollisionsQty = 0; decimal totalLengthOfProcessedStrings = 0; decimal averageStringLength = 0; Random random = new Random((int)System.DateTime.Now.Ticks); for (long i = 1; i <= testCasesQty; i++) { int stringLength = random.Next(minHashedStringLength, maxHashedStringLength); byte[] plainTextBytes = (byte[])Array.CreateInstance(typeof(byte), stringLength); random.NextBytes(plainTextBytes); System.Security.Cryptography.HashAlgorithm hash = new SHA1Managed(); byte[] hashBytes = hash.ComputeHash(plainTextBytes); string hashValue = Convert.ToBase64String(hashBytes); string testValue; if (_generatedHashes.TryGetValue(hashValue, out testValue)) collisionsQty++; else _generatedHashes.Add(hashValue, hashValue); if (i % 1000000 == 0) System.Console.WriteLine( "{0}: i={1}, cs={2}, cu={3}", DateTime.Now, i, collisionsQty, longHashCollisionsQty); ulong longHash = 0xFFFFFFFF; foreach (byte b in hashValue) { longHash ^= b; longHash = longHash << 8 + 0xFF; } ulong longHashTestValue; if (_generatedLongHashes.TryGetValue(longHash, out longHashTestValue)) { //Long Hash Collision longHashCollisionsQty++; System.Console.WriteLine(" * step {0}, hash '{1}', longHash: {2:X}", i, hashValue, longHash); } else _generatedLongHashes.Add(longHash, longHash); totalLengthOfProcessedStrings += stringLength; averageStringLength = totalLengthOfProcessedStrings / i; } System.Console.WriteLine( "\nTOTALS:\n" + "Min String Length = {0}\n" + "Max String Length = {1}\n" + "Avg String Length = {2}\n" + "Test cases Qty = {3}\n" + "Collisions Qty = {4}\n" + "ulong Collisions Qty = {5}", minHashedStringLength, maxHashedStringLength, averageStringLength, testCasesQty, collisionsQty, longHashCollisionsQty); System.Console.WriteLine("Finished at: " + System.DateTime.Now); } } P.P.S. Stats Started at: 19/03/2011 15:51:04 19/03/2011 15:51:16: i=1000000, cs=0, cu=0 * step 1712197, hash 'Rt4NY5FHg0uMDszVql+l6ksx6Tg=', longHash: 36D7CFC36A99DE80 19/03/2011 15:51:29: i=2000000, cs=0, cu=1 * step 2040324, hash 'laoPHdqlfeZOx9DI8RDaQZjhKZs=', longHash: D1B5AB44BB5CDE80 * step 2043449, hash '1m+tgKu7/l9fl8nUNuz5I/OWF5E=', longHash: C95F3EBC66B15E80 19/03/2011 15:51:44: i=3000000, cs=0, cu=3 19/03/2011 15:52:00: i=4000000, cs=0, cu=3 * step 4369446, hash '2sZrglXDp3NnNPuqdILqwmUKq30=', longHash: F7DB565F166C1E80 19/03/2011 15:52:17: i=5000000, cs=0, cu=4 TOTALS: Min String Length = 7 Max String Length = 25 Avg String Length = 15.4981354 Test cases Qty = 5,000,000 Collisions Qty = 0 ulong Collisions Qty = 4 Finished at: 19/03/2011 15:52:17 -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart McLachlan Sent: 19 ????? 2011 ?. 13:20 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SHA1 to compute a hash How are you creating your hash? Can you post a few examples of different data strings and colliding SHA1 hashes. I can probably make a lot of money out of them. AFAIK, no one other than you has found any. -- Stuart <<< snip >>> From stuart at lexacorp.com.pg Sat Mar 19 08:10:47 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 23:10:47 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D849D14.70906@colbyconsulting.com>, <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> Message-ID: <4D84AB57.31948.350CF8F@stuart.lexacorp.com.pg> I've just realised where the confusion comes in. When you talk about the chance of collisions increasing decreasing message length, you are talking about general hashing algorithms such as CRC etc. You need to undersand the difference between simple and cryptographic hashing. Cryptographic hashing algorithms are specifically designed to be "collision resistant". I repeat, to date no one has found a collision in SHA1 (except you apparently ) -- Stuart On 19 Mar 2011 at 22:51, Stuart McLachlan wrote: > In the case of 1000 single character messages, you are bound to get > collisions since there are only 256 possible original messages. You > will be hashing the same value multiple times > > Apart from that, your understanding is incorrect. It doesn't matter > how long the string is. The chances of a collision with two different > messages remains the same. > > The message is hashed using padded blocks of a fixed length. > There is no more chance of a collision between "a" and "b" than there > is between "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". > > Specifically, the chance of a collision within n different messages, > using b bits of encryption is (n*(n-1)/2) * (1/2^b). > > Note that the length of the message doesn't come into that equation. > > The probability of a collision is determined by only TWO things. > > 1) The length of the digest. > 2) The quantity of messages. > > As for "everyone agrees". Noone who understands how it works, agrees. > > -- > Stuart > > On 19 Mar 2011 at 8:09, jwcolby wrote: > > > >AFAIK, no one other than you has found any. > > > > LOL. > > > > You are thinking of hashing messages with thousands of characters. > > here's the deal. > > > > Let's assume that I hash X characters in a single field. The > > message length is the key, not the number of fields. Assume also > > that I am really only hashing letters, numbers and special > > characters - the characters that are in a name or address. > > > > > > If I hash 100 1 character messages, my probability of a collision if > > extremely high since there are only about 128 such alphanumeric > > characters (even less). > > > > Now hash a thousand 1 character messages. The probably just climbed > > immensely. Now hash a million such 1 character messages. What I am > > really calculating is the probability that I will repeat the same > > character string. > > > > However if I up the number of characters, let's say 10 character > > messages. *Now* the hash is really trying to prevent the same > > output for two *different* inputs. Well that is always what a hash > > is trying to do. But the point is that the probability of a > > collision decreases with message length increase. > > > > The probability of a collision between any two messages of 10 random > > characters is much higher than the probability of a collision > > between any two 100 character messages, and the probability of a > > collision between any two 1000 character messages is much higher > > still. > > > > That is just the way hash functions work. Nobody claims that hashes > > don't create collisions, and everyone agrees that the longer the > > message, the lower the probability of a collision between two > > messages (to a point). > > > > So, my address "messages" currently look like > > > > 89364 4456 PO Box 1 > > 76543 9876 Apt 2 > > 97867 3546 1723 Twin Pines Dr > > > > The point here is that all address "messages" are short. Now turn > > the address message into > > > > 89364 4456 PO Box 1 CA San Diego > > 76543 9876 Apt 2 TX Dallas > > 97867 3546 1723 Twin Pines Dr NC Hudson > > > > And the length of the message increases fairly dramatically, > > decreasing the probability of a collision. > > > > Remember too that I am hashing hundreds of millions of records - > > about 350 million addresses in various tables so far. The other > > thing that affects the probability of a hash collision is the number > > of messages hashed. Hash enough records and you *will* 100% > > probability create a hash that is the same for two different input > > strings - even for long messages. That is just the nature of the > > business. > > > > The probability of a collision is determined by three things. > > > > 1) The (strength of the) hash algorithm. > > 2) The message length > > 3) The quantity of messages. > ... > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 08:20:11 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 23:20:11 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, Message-ID: <4D84AD8B.14064.3596D15@stuart.lexacorp.com.pg> If he is compacting a 160 bit hash to a 64 bit hash, he is certainly increasing the odds of a collision - but I suspect that even at 64 bits, the chances of a collision would be very low. But that does not appear to be what you are testing. It looks as though you created 6 million random strings and four of them turned out to be identical. Therefore also had the same SHA1 digest. What happens if you try it again, making sure that all of your strings are unique. Alternatively, what do you get if you add some code to show the pairs of strings. Are they different? -- Stuart On 19 Mar 2011 at 16:03, Shamil Salakhetdinov wrote: > Hi Stuart and John, > > <<< > AFAIK, no one other than you has found any > >>> > Yes, I was also confused how it comes to get SHA1 hashes collisions > but then I have thought that John probably "compacts" SHA1 hashes into > 8 bytes surrogate keys? > > Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions > for 6 millions cycles for 8 bytes keys. > > Please correct me if you find some errors in my sample code > > Thank you. > > -- > Shamil > > P.S. Code: > > public class GenerateHashes > { > private static Dictionary _generatedHashes; > private static Dictionary _generatedLongHashes; > > public static void Run(int minHashedStringLength, int > maxHashedStringLength, long testCasesQty) > { > System.Console.WriteLine("Started at: " + > System.DateTime.Now); > > _generatedHashes = new Dictionary(); > _generatedLongHashes = new Dictionary(); > > int collisionsQty = 0; > int longHashCollisionsQty = 0; > > decimal totalLengthOfProcessedStrings = 0; > decimal averageStringLength = 0; > > > Random random = new > Random((int)System.DateTime.Now.Ticks); for (long i = 1; i > <= testCasesQty; i++) { > int stringLength = random.Next(minHashedStringLength, > maxHashedStringLength); > byte[] plainTextBytes = > (byte[])Array.CreateInstance(typeof(byte), stringLength); > random.NextBytes(plainTextBytes); > System.Security.Cryptography.HashAlgorithm hash = new > SHA1Managed(); > byte[] hashBytes = hash.ComputeHash(plainTextBytes); > string hashValue = Convert.ToBase64String(hashBytes); > string testValue; if > (_generatedHashes.TryGetValue(hashValue, out > testValue)) > collisionsQty++; > else _generatedHashes.Add(hashValue, hashValue); > if (i % 1000000 == 0) > System.Console.WriteLine( > "{0}: i={1}, cs={2}, cu={3}", > DateTime.Now, i, collisionsQty, > longHashCollisionsQty); > > ulong longHash = 0xFFFFFFFF; > foreach (byte b in hashValue) > { > longHash ^= b; > longHash = longHash << 8 + 0xFF; > } > ulong longHashTestValue; > if (_generatedLongHashes.TryGetValue(longHash, out > longHashTestValue)) > { > //Long Hash Collision > longHashCollisionsQty++; > System.Console.WriteLine(" * step {0}, hash '{1}', > longHash: {2:X}", > i, hashValue, longHash); > } > else _generatedLongHashes.Add(longHash, longHash); > > totalLengthOfProcessedStrings += stringLength; > averageStringLength = totalLengthOfProcessedStrings / > i; > } > > System.Console.WriteLine( > "\nTOTALS:\n" + > "Min String Length = {0}\n" + > "Max String Length = {1}\n" + > "Avg String Length = {2}\n" + > "Test cases Qty = {3}\n" + > "Collisions Qty = {4}\n" + > "ulong Collisions Qty = {5}", > minHashedStringLength, maxHashedStringLength, > averageStringLength, testCasesQty, > collisionsQty, > longHashCollisionsQty); > > System.Console.WriteLine("Finished at: " + > System.DateTime.Now); > > } > } > > > P.P.S. Stats > > Started at: 19/03/2011 15:51:04 > 19/03/2011 15:51:16: i=1000000, cs=0, cu=0 > * step 1712197, hash 'Rt4NY5FHg0uMDszVql+l6ksx6Tg=', longHash: > 36D7CFC36A99DE80 > 19/03/2011 15:51:29: i=2000000, cs=0, cu=1 > * step 2040324, hash 'laoPHdqlfeZOx9DI8RDaQZjhKZs=', longHash: > D1B5AB44BB5CDE80 > * step 2043449, hash '1m+tgKu7/l9fl8nUNuz5I/OWF5E=', longHash: > C95F3EBC66B15E80 > 19/03/2011 15:51:44: i=3000000, cs=0, cu=3 > 19/03/2011 15:52:00: i=4000000, cs=0, cu=3 > * step 4369446, hash '2sZrglXDp3NnNPuqdILqwmUKq30=', longHash: > F7DB565F166C1E80 > 19/03/2011 15:52:17: i=5000000, cs=0, cu=4 > > TOTALS: > Min String Length = 7 > Max String Length = 25 > Avg String Length = 15.4981354 > Test cases Qty = 5,000,000 > Collisions Qty = 0 > ulong Collisions Qty = 4 > Finished at: 19/03/2011 15:52:17 > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart > McLachlan Sent: 19 ????? 2011 ?. 13:20 To: Discussion concerning > Visual Basic and related programming issues. Subject: Re: [dba-VB] > SHA1 to compute a hash > > How are you creating your hash? > > Can you post a few examples of different data strings and colliding > SHA1 hashes. I can probably make a lot of money out of them. > AFAIK, no one other than you has found any. > > -- > Stuart > > <<< snip >>> > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 08:21:58 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 09:21:58 -0400 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D849D14.70906@colbyconsulting.com> <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg> Message-ID: <4D84ADF6.4000509@colbyconsulting.com> Stuart, Thanks for that. I am not a cryptologist and all I can do is try to use a tool I do not understand to solve a problem I do understand. If my hashes really are that "unique / secure" then I will stop worrying about it. I still need to do a check of actual data though because I went looking one day and discovered "collisions". Given that we believe the collision is not in the key itself, it must have been in my method of finding the collision. In any case, I am not going to worry any more so thanks for the peace of mind! ;) John W. Colby www.ColbyConsulting.com On 3/19/2011 8:51 AM, Stuart McLachlan wrote: > In the case of 1000 single character messages, you are bound to get collisions since there > are only 256 possible original messages. You will be hashing the same value multiple times > > Apart from that, your understanding is incorrect. It doesn't matter how long the string is. The > chances of a collision with two different messages remains the same. > > The message is hashed using padded blocks of a fixed length. > There is no more chance of a collision between "a" and "b" than there is between > "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". > > Specifically, the chance of a collision within n different messages, using b bits of encryption is > (n*(n-1)/2) * (1/2^b). > > Note that the length of the message doesn't come into that equation. > > The probability of a collision is determined by only TWO things. > > 1) The length of the digest. > 2) The quantity of messages. > > As for "everyone agrees". Noone who understands how it works, agrees. > From shamil at smsconsulting.spb.ru Sat Mar 19 08:21:26 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 16:21:26 +0300 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: References: <4D841001.2060200@colbyconsulting.com><4D848360.6537.2B4B316@stuart.lexacorp.com.pg> Message-ID: <49B2F00A3FD3442685D8283AE7678544@nant> Hi All -- Sorry I have mentioned 6 millions cycles but my sample was for just 5 millions, and I have got some collisions for 8 bytes long surrogate keys. Here I'm posting a patch for hashing part of the code - and no collisions now for 8 bytes long hashes for 5 millions cycles. .... UInt64 longHashResult = 0xFFFFFFFFFFFFFFFF; UInt64 longHash = 0xFFFFFFFFFFFFFFFF; int shiftCounter = 0; foreach (byte b in hashValue) { longHash ^= b; longHash = longHash << 8 + 0xFF; if (++shiftCounter == 8) { longHashResult ^= longHash; shiftCounter = 0; } } longHash = longHashResult; ... Started at: 19/03/2011 16:08:53 19/03/2011 16:09:04: i=1000000, cs=0, cu=0 19/03/2011 16:09:17: i=2000000, cs=0, cu=0 19/03/2011 16:09:33: i=3000000, cs=0, cu=0 19/03/2011 16:09:49: i=4000000, cs=0, cu=0 19/03/2011 16:10:06: i=5000000, cs=0, cu=0 TOTALS: Min String Length = 7 Max String Length = 25 Avg String Length = 15.4966972 Test cases Qty = 5000000 Collisions Qty = 0 ulong Collisions Qty = 0 Finished at: 19/03/2011 16:10:06 Please correct me if I've got wrong somewhere. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 19 ????? 2011 ?. 16:04 To: stuart at lexacorp.com.pg; 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] SHA1 to compute a hash Hi Stuart and John, <<< AFAIK, no one other than you has found any >>> Yes, I was also confused how it comes to get SHA1 hashes collisions but then I have thought that John probably "compacts" SHA1 hashes into 8 bytes surrogate keys? Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions for 6 millions cycles for 8 bytes keys. Please correct me if you find some errors in my sample code Thank you. -- Shamil <<< snip >>> From jwcolby at colbyconsulting.com Sat Mar 19 08:24:54 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 09:24:54 -0400 Subject: [dba-VB] On a lighter note Message-ID: <4D84AEA6.5060107@colbyconsulting.com> I rooted my droid X last night, bought a "freezer" program, froze all the bloatware, and downloaded and installed a tether program. Here at my house I get about 1.25 mbit down and .5 mbit up via the tether. Not bad considering it is over cell. -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sat Mar 19 08:24:33 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 16:24:33 +0300 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84AD8B.14064.3596D15@stuart.lexacorp.com.pg> References: <4D841001.2060200@colbyconsulting.com>, <4D848360.6537.2B4B316@stuart.lexacorp.com.pg>, <4D84AD8B.14064.3596D15@stuart.lexacorp.com.pg> Message-ID: <14BB8E00F24E4EBAB5842E5D75537D2F@nant> Hi Stuart -- <<< It looks as though you created 6 million random strings and four of them turned out to be identical. >>> No, all SHA1 strings are different - these are 64 bit hashes of that strings, which have got some collisions. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Stuart McLachlan Sent: 19 ????? 2011 ?. 16:20 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] SHA1 to compute a hash If he is compacting a 160 bit hash to a 64 bit hash, he is certainly increasing the odds of a collision - but I suspect that even at 64 bits, the chances of a collision would be very low. But that does not appear to be what you are testing. It looks as though you created 6 million random strings and four of them turned out to be identical. Therefore also had the same SHA1 digest. What happens if you try it again, making sure that all of your strings are unique. Alternatively, what do you get if you add some code to show the pairs of strings. Are they different? -- Stuart On 19 Mar 2011 at 16:03, Shamil Salakhetdinov wrote: > Hi Stuart and John, > > <<< > AFAIK, no one other than you has found any > >>> > Yes, I was also confused how it comes to get SHA1 hashes collisions > but then I have thought that John probably "compacts" SHA1 hashes into > 8 bytes surrogate keys? > > Here (P.S.) is a "quick&dirty" "brute force" test code - 4 collisions > for 6 millions cycles for 8 bytes keys. > > Please correct me if you find some errors in my sample code > > Thank you. > > -- > Shamil > <<< snip >>> From stuart at lexacorp.com.pg Sat Mar 19 08:29:55 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sat, 19 Mar 2011 23:29:55 +1000 Subject: [dba-VB] SHA1 to compute a hash In-Reply-To: <4D84ADF6.4000509@colbyconsulting.com> References: <4D841001.2060200@colbyconsulting.com>, <4D84A6C1.21571.33EE56F@stuart.lexacorp.com.pg>, <4D84ADF6.4000509@colbyconsulting.com> Message-ID: <4D84AFD3.25991.3625487@stuart.lexacorp.com.pg> Using a 160 bit SHA1, the odds of getting a collision are 50% when you have approximately 2^80 messages. If you ever get near that number of records, I suggest you switch to SHA2 which is 256 bit :-) -- Stuart On 19 Mar 2011 at 9:21, jwcolby wrote: > Stuart, > > Thanks for that. I am not a cryptologist and all I can do is try to > use a tool I do not understand to solve a problem I do understand. If > my hashes really are that "unique / secure" then I will stop worrying > about it. I still need to do a check of actual data though because I > went looking one day and discovered "collisions". Given that we > believe the collision is not in the key itself, it must have been in > my method of finding the collision. > > In any case, I am not going to worry any more so thanks for the peace > of mind! ;) > > John W. Colby > www.ColbyConsulting.com > > On 3/19/2011 8:51 AM, Stuart McLachlan wrote: > > In the case of 1000 single character messages, you are bound to get > > collisions since there are only 256 possible original messages. You > > will be hashing the same value multiple times > > > > Apart from that, your understanding is incorrect. It doesn't matter > > how long the string is. The chances of a collision with two > > different messages remains the same. > > > > The message is hashed using padded blocks of a fixed length. > > There is no more chance of a collision between "a" and "b" than > > there is between "aaaaaaaaaaaaaaaaaaa" and "baaaaaaaaaaaaaaaaaaa". > > > > Specifically, the chance of a collision within n different messages, > > using b bits of encryption is (n*(n-1)/2) * (1/2^b). > > > > Note that the length of the message doesn't come into that equation. > > > > The probability of a collision is determined by only TWO things. > > > > 1) The length of the digest. > > 2) The quantity of messages. > > > > As for "everyone agrees". Noone who understands how it works, > > agrees. > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 08:55:51 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 09:55:51 -0400 Subject: [dba-VB] Using the hash for real world problems Message-ID: <4D84B5E7.3010204@colbyconsulting.com> I got into the hashing business in order to create a surrogate key of sorts, in fact a trio of them. I have an address hash, a family hash and a person hash. I compute these three hashes for every table of persons. Now that I have these surrogate keys I have "identical" values in a single field where the input is identical in multiple fields. Thus I do not have to do multi-field joins where I have a hash for all of those fields. I am happy to hear that I am secure in my surrogate hash algorithm choice. OK, so I have these hashes. Every record has these three hashes. Now some definitions: HashAddr: A unique address is addr/zip5/zip4 different HashFamily: A unique family is LName/Addr/Zip5/Zip4 different HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. Using this information, I need to calculate the count of addresses with a single person, with two persons etc for as many combinations as I have. Addresses People 22,538,240 1 780,462,346 2 52,234 3 people etc etc till every quantity of people is found. What is the simplest way to accomplish this with SQL. -- John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 19 09:13:51 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 10:13:51 -0400 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84B5E7.3010204@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com> Message-ID: <4D84BA1F.2060101@colbyconsulting.com> Ooops. My examples were hosed. Let's try again Addresses People 22,538,240 1 00,708,462 2 00,052,234 3 and so forth. What the client really wants to know is "how many people with emails are at a specific address" "how many people with emails are in a specific family" etc. I tie my address table to my email table using an auto increment long increment "I must be careful not to call this a surrogate" key and then perform the counts using the hash fields in a Group By / count query. John W. Colby www.ColbyConsulting.com On 3/19/2011 9:55 AM, jwcolby wrote: > I got into the hashing business in order to create a surrogate key of sorts, in fact a trio of them. > > I have an address hash, a family hash and a person hash. I compute these three hashes for every > table of persons. Now that I have these surrogate keys I have "identical" values in a single field > where the input is identical in multiple fields. Thus I do not have to do multi-field joins where I > have a hash for all of those fields. > > I am happy to hear that I am secure in my surrogate hash algorithm choice. > > OK, so I have these hashes. Every record has these three hashes. > > Now some definitions: > > HashAddr: A unique address is addr/zip5/zip4 different > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > Using this information, I need to calculate the count of addresses with a single person, with two > persons etc for as many combinations as I have. > > Addresses People > 22,538,240 1 > 780,462,346 2 > 52,234 3 people > etc > etc till every quantity of people is found. > > What is the simplest way to accomplish this with SQL. > > > > From shamil at smsconsulting.spb.ru Sat Mar 19 10:07:08 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 19 Mar 2011 18:07:08 +0300 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84BA1F.2060101@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com> <4D84BA1F.2060101@colbyconsulting.com> Message-ID: Hi John -- I suppose you can reuse the following SQL expression for your case: select count(Occurrence) QtyTheSameOccurrences, Occurrence from ( select Country, Count(Country) Occurrence from Customer Group by Country ) o group by o.Occurrence order by o.Occurrence Just replace Country name columns with hashAddress column and Country table name with your table name ... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 19 ????? 2011 ?. 17:14 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Using the hash for real world problems Ooops. My examples were hosed. Let's try again Addresses People 22,538,240 1 00,708,462 2 00,052,234 3 and so forth. What the client really wants to know is "how many people with emails are at a specific address" "how many people with emails are in a specific family" etc. I tie my address table to my email table using an auto increment long increment "I must be careful not to call this a surrogate" key and then perform the counts using the hash fields in a Group By / count query. John W. Colby www.ColbyConsulting.com On 3/19/2011 9:55 AM, jwcolby wrote: > I got into the hashing business in order to create a surrogate key of sorts, in fact a trio of them. > > I have an address hash, a family hash and a person hash. I compute > these three hashes for every table of persons. Now that I have these > surrogate keys I have "identical" values in a single field where the > input is identical in multiple fields. Thus I do not have to do multi-field joins where I have a hash for all of those fields. > > I am happy to hear that I am secure in my surrogate hash algorithm choice. > > OK, so I have these hashes. Every record has these three hashes. > > Now some definitions: > > HashAddr: A unique address is addr/zip5/zip4 different > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > Using this information, I need to calculate the count of addresses > with a single person, with two persons etc for as many combinations as I have. > > Addresses People > 22,538,240 1 > 780,462,346 2 > 52,234 3 people > etc > etc till every quantity of people is found. > > What is the simplest way to accomplish this with SQL. > > > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Mar 19 14:50:55 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 15:50:55 -0400 Subject: [dba-VB] Most useful droid apps Message-ID: <4D85091F.5060807@colbyconsulting.com> What are the most useful apps you run on your droid? -- John W. Colby www.ColbyConsulting.com From davidmcafee at gmail.com Sat Mar 19 15:53:32 2011 From: davidmcafee at gmail.com (David McAfee) Date: Sat, 19 Mar 2011 13:53:32 -0700 Subject: [dba-VB] Most useful droid apps Message-ID: Advanced task killer Weather channel Lookout antivirus AK Notepad Bank of America Acar auto maintenance Barcode scanner Bubble level Bump Droid light Google goggles Google translate IMDB Key ring Print share Trapster Sent from my Droid phone. On Mar 19, 2011 12:52 PM, "jwcolby" wrote: From stuart at lexacorp.com.pg Sat Mar 19 18:11:26 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sun, 20 Mar 2011 09:11:26 +1000 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84BA1F.2060101@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com>, <4D84BA1F.2060101@colbyconsulting.com> Message-ID: <4D85381E.9973.576BE3C@stuart.lexacorp.com.pg> Caaution. All aircode below: To get the list output below, I would use something like Select Distinct Count(HashAddr) as Addresses, People >From (Select distinct HashAddr,Count(HashAddr) as People From tblPeople Group By HashAddr) Group By People Order by People To find out details for a *specific* Address Find the Hash for the desired Address and then pass it to something like this: Select Distinct StreetNo,Street,City,Zip,State,Count(HashAddr) as Address >From tblPeople P Inner Join tblEmails E on P.PK = E.PeopleFK Group By HashAddr StreetNo,Street,City,Zip,State Where HashAddr = @SearchHash On 19 Mar 2011 at 10:13, jwcolby wrote: > Ooops. My examples were hosed. > > Let's try again > > Addresses People > 22,538,240 1 > 00,708,462 2 > 00,052,234 3 > > and so forth. > > What the client really wants to know is > > "how many people with emails are at a specific address" > "how many people with emails are in a specific family" > > etc. > > I tie my address table to my email table using an auto increment long > increment "I must be careful not to call this a surrogate" key and > then perform the counts using the hash fields in a Group By / count > query. > > John W. Colby > www.ColbyConsulting.com > > On 3/19/2011 9:55 AM, jwcolby wrote: > > I got into the hashing business in order to create a surrogate key > > of sorts, in fact a trio of them. > > > > I have an address hash, a family hash and a person hash. I compute > > these three hashes for every table of persons. Now that I have these > > surrogate keys I have "identical" values in a single field where the > > input is identical in multiple fields. Thus I do not have to do > > multi-field joins where I have a hash for all of those fields. > > > > I am happy to hear that I am secure in my surrogate hash algorithm > > choice. > > > > OK, so I have these hashes. Every record has these three hashes. > > > > Now some definitions: > > > > HashAddr: A unique address is addr/zip5/zip4 different > > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > > > Using this information, I need to calculate the count of addresses > > with a single person, with two persons etc for as many combinations > > as I have. > > > > Addresses People > > 22,538,240 1 > > 780,462,346 2 > > 52,234 3 people > > etc > > etc till every quantity of people is found. > > > > What is the simplest way to accomplish this with SQL. > > > > > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 18:21:18 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sun, 20 Mar 2011 09:21:18 +1000 Subject: [dba-VB] Using the hash for real world problems In-Reply-To: <4D84B5E7.3010204@colbyconsulting.com> References: <4D84B5E7.3010204@colbyconsulting.com> Message-ID: <4D853A6E.20716.57FC618@stuart.lexacorp.com.pg> Answered on VB list. On 19 Mar 2011 at 9:55, jwcolby wrote: > I got into the hashing business in order to create a surrogate key of > sorts, in fact a trio of them. > > I have an address hash, a family hash and a person hash. I compute > these three hashes for every table of persons. Now that I have these > surrogate keys I have "identical" values in a single field where the > input is identical in multiple fields. Thus I do not have to do > multi-field joins where I have a hash for all of those fields. > > I am happy to hear that I am secure in my surrogate hash algorithm > choice. > > OK, so I have these hashes. Every record has these three hashes. > > Now some definitions: > > HashAddr: A unique address is addr/zip5/zip4 different > HashFamily: A unique family is LName/Addr/Zip5/Zip4 different > HahsPerson: A unique person is FName/LName/Addr/Zip5/Zip4 different. > > Using this information, I need to calculate the count of addresses > with a single person, with two persons etc for as many combinations as > I have. > > Addresses People > 22,538,240 1 > 780,462,346 2 > 52,234 3 people > etc > etc till every quantity of people is found. > > What is the simplest way to accomplish this with SQL. > > > > > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 21:01:04 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 22:01:04 -0400 Subject: [dba-VB] Most useful droid apps In-Reply-To: References: Message-ID: <4D855FE0.9070004@colbyconsulting.com> I am trying various tethering solutions. My laptop is vista, my ip range is 192.168.122.x. I got WiFi Tether to work, but only without encryption. I was getting about 1.5 mbit down and .5 mbit up. As soon as I turned on encryption I was unable to set up the laptop password to match (that was the only thing that changed). No encryption is unacceptable to me as I would only really use this in public places, exactly where you would most want encryption. I tried barnacle. That one just didn't work at all. I am now trying PDANet which is a USB tether. Since the PC is tied to the phone by a usb, encryption seems to be unnecessary? Unfortunately this one is not working either. The PDANet desktop app says it is connected and the droid pdanet app says it is connected but the PC just can't see the internet. I can't ping anything etc. Any suggestions for this beast? John W. Colby www.ColbyConsulting.com On 3/19/2011 4:53 PM, David McAfee wrote: > Advanced task killer > Weather channel > Lookout antivirus > AK Notepad > Bank of America > Acar auto maintenance > Barcode scanner > Bubble level > Bump > Droid light > Google goggles > Google translate > IMDB > Key ring > Print share > Trapster > > Sent from my Droid phone. > On Mar 19, 2011 12:52 PM, "jwcolby" wrote: > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From stuart at lexacorp.com.pg Sat Mar 19 21:34:03 2011 From: stuart at lexacorp.com.pg (Stuart McLachlan) Date: Sun, 20 Mar 2011 12:34:03 +1000 Subject: [dba-VB] Most useful droid apps In-Reply-To: <4D855FE0.9070004@colbyconsulting.com> References: , <4D855FE0.9070004@colbyconsulting.com> Message-ID: <4D85679B.30595.6303F8F@stuart.lexacorp.com.pg> Are you on AT&T? If so, you may be interested in this: http://www.theregister.co.uk/2011/03/18/new_att_tethering_policy/ On 19 Mar 2011 at 22:01, jwcolby wrote: > I am trying various tethering solutions. My laptop is vista, my ip > range is 192.168.122.x. > > I got WiFi Tether to work, but only without encryption. I was getting > about 1.5 mbit down and .5 mbit up. > > As soon as I turned on encryption I was unable to set up the laptop > password to match (that was the only thing that changed). No > encryption is unacceptable to me as I would only really use this in > public places, exactly where you would most want encryption. > > I tried barnacle. That one just didn't work at all. > > I am now trying PDANet which is a USB tether. Since the PC is tied to > the phone by a usb, encryption seems to be unnecessary? Unfortunately > this one is not working either. The PDANet desktop app says it is > connected and the droid pdanet app says it is connected but the PC > just can't see the internet. I can't ping anything etc. > > Any suggestions for this beast? > > > John W. Colby > www.ColbyConsulting.com > > On 3/19/2011 4:53 PM, David McAfee wrote: > > Advanced task killer > > Weather channel > > Lookout antivirus > > AK Notepad > > Bank of America > > Acar auto maintenance > > Barcode scanner > > Bubble level > > Bump > > Droid light > > Google goggles > > Google translate > > IMDB > > Key ring > > Print share > > Trapster > > > > Sent from my Droid phone. > > On Mar 19, 2011 12:52 PM, "jwcolby" > > wrote: _______________________________________________ dba-VB > > mailing list dba-VB at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-vb > > http://www.databaseadvisors.com > > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 19 21:45:39 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 19 Mar 2011 22:45:39 -0400 Subject: [dba-VB] Most useful droid apps In-Reply-To: <4D85679B.30595.6303F8F@stuart.lexacorp.com.pg> References: , <4D855FE0.9070004@colbyconsulting.com> <4D85679B.30595.6303F8F@stuart.lexacorp.com.pg> Message-ID: <4D856A53.7010507@colbyconsulting.com> Nope, Verizon John W. Colby www.ColbyConsulting.com On 3/19/2011 10:34 PM, Stuart McLachlan wrote: > Are you on AT&T? > > If so, you may be interested in this: > > http://www.theregister.co.uk/2011/03/18/new_att_tethering_policy/ > > > On 19 Mar 2011 at 22:01, jwcolby wrote: > >> I am trying various tethering solutions. My laptop is vista, my ip >> range is 192.168.122.x. >> >> I got WiFi Tether to work, but only without encryption. I was getting >> about 1.5 mbit down and .5 mbit up. >> >> As soon as I turned on encryption I was unable to set up the laptop >> password to match (that was the only thing that changed). No >> encryption is unacceptable to me as I would only really use this in >> public places, exactly where you would most want encryption. >> >> I tried barnacle. That one just didn't work at all. >> >> I am now trying PDANet which is a USB tether. Since the PC is tied to >> the phone by a usb, encryption seems to be unnecessary? Unfortunately >> this one is not working either. The PDANet desktop app says it is >> connected and the droid pdanet app says it is connected but the PC >> just can't see the internet. I can't ping anything etc. >> >> Any suggestions for this beast? >> >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/19/2011 4:53 PM, David McAfee wrote: >>> Advanced task killer >>> Weather channel >>> Lookout antivirus >>> AK Notepad >>> Bank of America >>> Acar auto maintenance >>> Barcode scanner >>> Bubble level >>> Bump >>> Droid light >>> Google goggles >>> Google translate >>> IMDB >>> Key ring >>> Print share >>> Trapster >>> >>> Sent from my Droid phone. >>> On Mar 19, 2011 12:52 PM, "jwcolby" >>> wrote: _______________________________________________ dba-VB >>> mailing list dba-VB at databaseadvisors.com >>> http://databaseadvisors.com/mailman/listinfo/dba-vb >>> http://www.databaseadvisors.com >>> >>> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From listmaster at databaseadvisors.com Wed Mar 23 14:34:46 2011 From: listmaster at databaseadvisors.com (Bryan Carbonnell) Date: Wed, 23 Mar 2011 15:34:46 -0400 Subject: [dba-VB] Administrivia - Spam Filtering on DBA Servers In-Reply-To: References: Message-ID: We have made a change to the way our mailservers deal with spam filtering. We were getting too many false positive reports from one of the spam black list services we use, so we have discontinued using them. Hopefully your bounce troubles should be over, or at the very least greatly minimized. If you continue to have issues surrounding bounces, PLEASE, PLEASE, PLEASE get in touch with me (listmaster at databaseadvisors OR carbonnb at gmail.com if the listmaster address bounces). -- Bryan Carbonnell - listmaster at databaseadvisors.com Life's journey is not to arrive at the grave safely in a well preserved body, but rather to skid in sideways, totally worn out, shouting "What a great ride!" From jwcolby at colbyconsulting.com Thu Mar 24 13:07:19 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 24 Mar 2011 14:07:19 -0400 Subject: [dba-VB] C# protected Message-ID: <4D8B8857.60604@colbyconsulting.com> I am trying to create a method of a ClassA which can be seen from the object ClassB that instantiates ClassA but not by objectClassC which instantiates ClassB. IOW ClassA "belongs to" ClassB so specific methods / properties of ClassB should be manipulateable by ClassB. However those properties / methods should not be visible to anything outside of ClassA and ClassB. Possible? Oh, and BTW I want ClassA to be out in a class library, external to the project using it. -- John W. Colby www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Thu Mar 24 15:14:59 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Thu, 24 Mar 2011 23:14:59 +0300 Subject: [dba-VB] C# protected In-Reply-To: <4D8B8857.60604@colbyconsulting.com> References: <4D8B8857.60604@colbyconsulting.com> Message-ID: Hi John -- Put ClassA and ClassB into a class library. Use internal access modifier with their methods and properties to be seen within their class library but not from the "outer world". Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 24 ????? 2011 ?. 21:07 To: VBA Subject: [dba-VB] C# protected I am trying to create a method of a ClassA which can be seen from the object ClassB that instantiates ClassA but not by objectClassC which instantiates ClassB. IOW ClassA "belongs to" ClassB so specific methods / properties of ClassB should be manipulateable by ClassB. However those properties / methods should not be visible to anything outside of ClassA and ClassB. Possible? Oh, and BTW I want ClassA to be out in a class library, external to the project using it. -- John W. Colby www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Thu Mar 24 15:28:03 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Thu, 24 Mar 2011 16:28:03 -0400 Subject: [dba-VB] C# protected In-Reply-To: References: <4D8B8857.60604@colbyconsulting.com> Message-ID: <4D8BA953.4030602@colbyconsulting.com> ClassA can go in the class library, but classB is part of the application. John W. Colby www.ColbyConsulting.com On 3/24/2011 4:14 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Put ClassA and ClassB into a class library. > Use > > internal > > access modifier with their methods and properties to be seen within their > class library but not from the "outer world". > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 24 ????? 2011 ?. 21:07 > To: VBA > Subject: [dba-VB] C# protected > > I am trying to create a method of a ClassA which can be seen from the object > ClassB that instantiates ClassA but not by objectClassC which instantiates > ClassB. > > IOW ClassA "belongs to" ClassB so specific methods / properties of ClassB > should be manipulateable by ClassB. However those properties / methods > should not be visible to anything outside of ClassA and ClassB. > > Possible? > > Oh, and BTW I want ClassA to be out in a class library, external to the > project using it. > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Thu Mar 24 16:37:22 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Fri, 25 Mar 2011 00:37:22 +0300 Subject: [dba-VB] C# protected In-Reply-To: <4D8BA953.4030602@colbyconsulting.com> References: <4D8B8857.60604@colbyconsulting.com> <4D8BA953.4030602@colbyconsulting.com> Message-ID: Hi John -- Then use protected access modifier for methods and properties of Class A which is inherited by class B (IOW no need to instantiate class A explicitly)... Class C which will instantiate Class B will no see/have access to protected properties and methods of class A inherithed by class B. Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 24 ????? 2011 ?. 23:28 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] C# protected ClassA can go in the class library, but classB is part of the application. John W. Colby www.ColbyConsulting.com On 3/24/2011 4:14 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Put ClassA and ClassB into a class library. > Use > > internal > > access modifier with their methods and properties to be seen within > their class library but not from the "outer world". > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 24 ????? 2011 ?. 21:07 > To: VBA > Subject: [dba-VB] C# protected > > I am trying to create a method of a ClassA which can be seen from the > object ClassB that instantiates ClassA but not by objectClassC which > instantiates ClassB. > > IOW ClassA "belongs to" ClassB so specific methods / properties of > ClassB should be manipulateable by ClassB. However those properties / > methods should not be visible to anything outside of ClassA and ClassB. > > Possible? > > Oh, and BTW I want ClassA to be out in a class library, external to > the project using it. > -- > John W. Colby > www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 25 14:58:27 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 25 Mar 2011 15:58:27 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <3B9DE744CEA244829179A81CA818F92F@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com> <3B9DE744CEA244829179A81CA818F92F@nant> Message-ID: <4D8CF3E3.3050608@colbyconsulting.com> Shamil, Does this allow using a single "lock object" to lock various operations in a class. For example I want to lock a date flag variable while using a property to either get or set the variable. private Object thisLock = new Object(); public DateTime pFlagDte { get { lock (thisLock) { return dteFlag; } } set { lock (thisLock) { dteFlag = value; } } } The issue is not multiple threads trying to write to the date variable but rather one thread trying to read it while another is writing it. John W. Colby www.ColbyConsulting.com On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Do you you need add, remove and iterate a list to get an object or you need > to add, remove and get an object by its key? - if the latter - here is the > code you can use: > > private Dictionary _myObjects = new Dictionary MyObject>(); > private static object _myObjectsLocker = new object(); > public void Add(MyObject myObject) > { > lock (_myObjectsLocker) > { > _myObjects.Add(myObject.ID, myObject); > _keys.Add(myObject.ID); > } > } > public MyObject Remove(int id) > { > MyObject myobject = null; > lock (_myObjectsLocker) > { > myobject = _myObjects[id]; > _myObjects.Remove(id); > } > return myobject; > } > public MyObject this[int key] > { > get > { > MyObject myObject = null; > lock (_myObjectsLocker) > { > _myObjects.TryGetValue(key, out myObject); > } > return myObject; > } > } > > for the case of iteration just replace: > > _myObjects.TryGetValue(key, out myObject); > > with your iteration code. > > As for the sample code - changing qty of threads and changing total computer > workload (outer context for the test code) you can see how different could > be final results when heavy multi-threading is used. And even if it's not > heavy - it still can result in weird output if special multi-threading > coding technique aren't used... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 11 ????? 2011 ?. 1:27 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Thanks for the code but I think it is overkill for my needs so far, though I > will put it in the archives for when I need it. > > In my system so far I have (or can arrange to have ) only two threads > trying to access a list. > > Basically what I am doing is reading records out of sql server into a class > factory, and placing the class instances into a list. > > Another object with its own thread is coming by and looking at the objects > in the list to see if there is any work for it to do. If it finds an object > to work on, it grabs a pointer to that item in the list and stores it in a > pointer and releases the list. > > One of the ways I saw this handled was to create factory which creates a > brand new list object, grabs a pointer to the list, locks that and then > copies the objects in the list pointers to classes) into that new list, > unlocks the list and hands back a pointer to this brand new list. thus the > new list could be iterated at will. > > I am just trying to lock the list while one thread adds objects to the list > or another thread gets objects out of the list. > > So if I have object A which contains a list and is adding / deleting objects > to that list... > and object B which needs to iterate through the list looking for an object > of interest in the list, and grabbing a pointer to that object, there is no > way to do this without all of the code you wrote? > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Does that in fact lock the list to other objects doing the same thing >> (getting the list through that property)? >>>>> >> You have to lock/synchronize/serialize not the list itself but >> simultaneous access to the list from different threads. >> You'd better use special static object variable to implement locking. >> >> I will post sample code in my following posting here... >> >> Thank you. >> >> -- >> Shamil > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Mar 25 16:05:57 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 26 Mar 2011 00:05:57 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8CF3E3.3050608@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com> Message-ID: <3F0DA84A905C4947819126F43CFC39DB@nant> Hi John -- Your sample is a correct usage of locking - just use static thisLock variable private static Object thisLock = new Object() as non-static variable *is not* thread safe AFAIU. Yes, such locking can be done for various operations of that class - just make sure you'll not get dead-locked... Also use *one lock object* to lock *one resource* in a class - otherwise "deadlock" will become your "everyday guest"... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 25 ????? 2011 ?. 22:58 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, Does this allow using a single "lock object" to lock various operations in a class. For example I want to lock a date flag variable while using a property to either get or set the variable. private Object thisLock = new Object(); public DateTime pFlagDte { get { lock (thisLock) { return dteFlag; } } set { lock (thisLock) { dteFlag = value; } } } The issue is not multiple threads trying to write to the date variable but rather one thread trying to read it while another is writing it. John W. Colby www.ColbyConsulting.com On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Do you you need add, remove and iterate a list to get an object or you > need to add, remove and get an object by its key? - if the latter - > here is the code you can use: > > private Dictionary _myObjects = new Dictionary MyObject>(); > private static object _myObjectsLocker = new object(); public void > Add(MyObject myObject) { > lock (_myObjectsLocker) > { > _myObjects.Add(myObject.ID, myObject); > _keys.Add(myObject.ID); > } > } > public MyObject Remove(int id) > { > MyObject myobject = null; > lock (_myObjectsLocker) > { > myobject = _myObjects[id]; > _myObjects.Remove(id); > } > return myobject; > } > public MyObject this[int key] > { > get > { > MyObject myObject = null; > lock (_myObjectsLocker) > { > _myObjects.TryGetValue(key, out myObject); > } > return myObject; > } > } > > for the case of iteration just replace: > > _myObjects.TryGetValue(key, out myObject); > > with your iteration code. > > As for the sample code - changing qty of threads and changing total > computer workload (outer context for the test code) you can see how > different could be final results when heavy multi-threading is used. > And even if it's not heavy - it still can result in weird output if > special multi-threading coding technique aren't used... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 11 ????? 2011 ?. 1:27 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Thanks for the code but I think it is overkill for my needs so far, > though I will put it in the archives for when I need it. > > In my system so far I have (or can arrange to have ) only two threads > trying to access a list. > > Basically what I am doing is reading records out of sql server into a > class factory, and placing the class instances into a list. > > Another object with its own thread is coming by and looking at the > objects in the list to see if there is any work for it to do. If it > finds an object to work on, it grabs a pointer to that item in the > list and stores it in a pointer and releases the list. > > One of the ways I saw this handled was to create factory which creates > a brand new list object, grabs a pointer to the list, locks that and > then copies the objects in the list pointers to classes) into that new > list, unlocks the list and hands back a pointer to this brand new > list. thus the new list could be iterated at will. > > I am just trying to lock the list while one thread adds objects to the > list or another thread gets objects out of the list. > > So if I have object A which contains a list and is adding / deleting > objects to that list... > and object B which needs to iterate through the list looking for an > object of interest in the list, and grabbing a pointer to that object, > there is no way to do this without all of the code you wrote? > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> <<< >> Does that in fact lock the list to other objects doing the same thing >> (getting the list through that property)? >>>>> >> You have to lock/synchronize/serialize not the list itself but >> simultaneous access to the list from different threads. >> You'd better use special static object variable to implement locking. >> >> I will post sample code in my following posting here... >> >> Thank you. >> >> -- >> Shamil > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 25 16:38:19 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 25 Mar 2011 17:38:19 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <3F0DA84A905C4947819126F43CFC39DB@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com> <3F0DA84A905C4947819126F43CFC39DB@nant> Message-ID: <4D8D0B4B.9070004@colbyconsulting.com> When you say resource, you mean variable? IOW use private static Object LockBln = new Object(); private static Object LockDte = new Object(); and use the LockBln to lock the boolean object while using LockDte to lock the date object? John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - just > make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - otherwise > "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various operations in a > class. > > For example I want to lock a date flag variable while using a property to > either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable but > rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Do you you need add, remove and iterate a list to get an object or you >> need to add, remove and get an object by its key? - if the latter - >> here is the code you can use: >> >> private Dictionary _myObjects = new Dictionary> MyObject>(); >> private static object _myObjectsLocker = new object(); public void >> Add(MyObject myObject) { >> lock (_myObjectsLocker) >> { >> _myObjects.Add(myObject.ID, myObject); >> _keys.Add(myObject.ID); >> } >> } >> public MyObject Remove(int id) >> { >> MyObject myobject = null; >> lock (_myObjectsLocker) >> { >> myobject = _myObjects[id]; >> _myObjects.Remove(id); >> } >> return myobject; >> } >> public MyObject this[int key] >> { >> get >> { >> MyObject myObject = null; >> lock (_myObjectsLocker) >> { >> _myObjects.TryGetValue(key, out myObject); >> } >> return myObject; >> } >> } >> >> for the case of iteration just replace: >> >> _myObjects.TryGetValue(key, out myObject); >> >> with your iteration code. >> >> As for the sample code - changing qty of threads and changing total >> computer workload (outer context for the test code) you can see how >> different could be final results when heavy multi-threading is used. >> And even if it's not heavy - it still can result in weird output if >> special multi-threading coding technique aren't used... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 11 ????? 2011 ?. 1:27 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Thanks for the code but I think it is overkill for my needs so far, >> though I will put it in the archives for when I need it. >> >> In my system so far I have (or can arrange to have ) only two threads >> trying to access a list. >> >> Basically what I am doing is reading records out of sql server into a >> class factory, and placing the class instances into a list. >> >> Another object with its own thread is coming by and looking at the >> objects in the list to see if there is any work for it to do. If it >> finds an object to work on, it grabs a pointer to that item in the >> list and stores it in a pointer and releases the list. >> >> One of the ways I saw this handled was to create factory which creates >> a brand new list object, grabs a pointer to the list, locks that and >> then copies the objects in the list pointers to classes) into that new >> list, unlocks the list and hands back a pointer to this brand new >> list. thus the new list could be iterated at will. >> >> I am just trying to lock the list while one thread adds objects to the >> list or another thread gets objects out of the list. >> >> So if I have object A which contains a list and is adding / deleting >> objects to that list... >> and object B which needs to iterate through the list looking for an >> object of interest in the list, and grabbing a pointer to that object, >> there is no way to do this without all of the code you wrote? >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >>> Hi John -- >>> >>> <<< >>> Does that in fact lock the list to other objects doing the same thing >>> (getting the list through that property)? >>>>>> >>> You have to lock/synchronize/serialize not the list itself but >>> simultaneous access to the list from different threads. >>> You'd better use special static object variable to implement locking. >>> >>> I will post sample code in my following posting here... >>> >>> Thank you. >>> >>> -- >>> Shamil >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Fri Mar 25 17:46:44 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 26 Mar 2011 01:46:44 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8D0B4B.9070004@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8D0B4B.9070004@colbyconsulting.com> Message-ID: Hi John -- By resource I mean access to a read/write resource - it could be not only a variable... If you need two or more class level variables (fields, properties) within one class instance to be locked then keep them in a class/structure and lock that class/structure instance... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 26 ????? 2011 ?. 0:38 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() When you say resource, you mean variable? IOW use private static Object LockBln = new Object(); private static Object LockDte = new Object(); and use the LockBln to lock the boolean object while using LockDte to lock the date object? John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - > just make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - > otherwise "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various > operations in a class. > > For example I want to lock a date flag variable while using a property > to either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable > but rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com > > On 3/10/2011 5:51 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Do you you need add, remove and iterate a list to get an object or >> you need to add, remove and get an object by its key? - if the latter >> - here is the code you can use: >> >> private Dictionary _myObjects = new Dictionary> MyObject>(); >> private static object _myObjectsLocker = new object(); public void >> Add(MyObject myObject) { >> lock (_myObjectsLocker) >> { >> _myObjects.Add(myObject.ID, myObject); >> _keys.Add(myObject.ID); >> } >> } >> public MyObject Remove(int id) >> { >> MyObject myobject = null; >> lock (_myObjectsLocker) >> { >> myobject = _myObjects[id]; >> _myObjects.Remove(id); >> } >> return myobject; >> } >> public MyObject this[int key] >> { >> get >> { >> MyObject myObject = null; >> lock (_myObjectsLocker) >> { >> _myObjects.TryGetValue(key, out myObject); >> } >> return myObject; >> } >> } >> >> for the case of iteration just replace: >> >> _myObjects.TryGetValue(key, out myObject); >> >> with your iteration code. >> >> As for the sample code - changing qty of threads and changing total >> computer workload (outer context for the test code) you can see how >> different could be final results when heavy multi-threading is used. >> And even if it's not heavy - it still can result in weird output if >> special multi-threading coding technique aren't used... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 11 ????? 2011 ?. 1:27 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Thanks for the code but I think it is overkill for my needs so far, >> though I will put it in the archives for when I need it. >> >> In my system so far I have (or can arrange to have ) only two >> threads trying to access a list. >> >> Basically what I am doing is reading records out of sql server into a >> class factory, and placing the class instances into a list. >> >> Another object with its own thread is coming by and looking at the >> objects in the list to see if there is any work for it to do. If it >> finds an object to work on, it grabs a pointer to that item in the >> list and stores it in a pointer and releases the list. >> >> One of the ways I saw this handled was to create factory which >> creates a brand new list object, grabs a pointer to the list, locks >> that and then copies the objects in the list pointers to classes) >> into that new list, unlocks the list and hands back a pointer to this >> brand new list. thus the new list could be iterated at will. >> >> I am just trying to lock the list while one thread adds objects to >> the list or another thread gets objects out of the list. >> >> So if I have object A which contains a list and is adding / deleting >> objects to that list... >> and object B which needs to iterate through the list looking for an >> object of interest in the list, and grabbing a pointer to that >> object, there is no way to do this without all of the code you wrote? >> >> John W. Colby >> www.ColbyConsulting.com >> >> On 3/10/2011 4:35 PM, Shamil Salakhetdinov wrote: >>> Hi John -- >>> >>> <<< >>> Does that in fact lock the list to other objects doing the same >>> thing (getting the list through that property)? >>>>>> >>> You have to lock/synchronize/serialize not the list itself but >>> simultaneous access to the list from different threads. >>> You'd better use special static object variable to implement locking. >>> >>> I will post sample code in my following posting here... >>> >>> Thank you. >>> >>> -- >>> Shamil >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> _______________________________________________ >> dba-VB mailing list >> dba-VB at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-vb >> http://www.databaseadvisors.com >> >> > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Fri Mar 25 21:57:22 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Fri, 25 Mar 2011 22:57:22 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8D0B4B.9070004@colbyconsulting.com> Message-ID: <4D8D5612.6050701@colbyconsulting.com> Shamil, I have processes that log results to flags. For example, make a database (log that it was made), build a table (log that it was built), pull umpteen million records in sorted order (log that it was filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build another chunk table (log that it was filled), BCP out (log that it was exported). The objective is to be able to sustain interruptions and pick up where we left off. These processes can take minutes (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). So each thing I do represents a step in the process and each step is logged in a field in a record in SQL server using a datetime. There are so many of these flags that I am trying to standardize the process by building a class that can be instantiated, filled with data and log itself to SQL Server by one thread and be checked by another thread. These flag class instances will be checked by multiple threads, each thread trying to decide whether it should be doing the next step because another thread has finished it's part. IOW if a file has been written to disk, then the next thread will write it to a VM for processing. If it moved to the VM the next thread will watch the VM's output directory for a file to pop out and move it back to a directory on the server. If the file (a couple of files actually) successfully copied back to the server staging then another thread will import it back into a chunk table in an input database. If the file successfully imported then another thread will... In general one thread will "own" the flag and use it to log its status and one other thread will be checking the status of the flag to determine that it can go to work on that work chunk. You get the picture. I am trying to build an entirely asynchronous highly threaded process which exports a huge table into multiple files, processes every file through a third party app and gets the results back into SQL Server. All while logging each and every step so that no piece can possibly be dropped at any stage, even if the server goes down (or the VM goes down). Eventually this process will run on my server 24/7. It has been working for some time but I am getting threading issues, and I need to work on the high level control so that all of the processes can cleanly start up and shut down and every stage can pick back up when the program restarts should a shutdown occur. A single database can be up to a hundred million records (the biggest so far), and the external program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour depending on many different things so that example will take 50 chunks and could take 40 to 50 hours to complete. It takes about 20 processing steps to handle each file from end to end. It needs to just work, and I need to be able to view status in a meaningful way. And I need to process that and a dozen other files every single month, automatically, with no manual intervention required. John W. Colby www.ColbyConsulting.com On 3/25/2011 6:46 PM, Shamil Salakhetdinov wrote: > Hi John -- > > By resource I mean access to a read/write resource - it could be not only a > variable... > > If you need two or more class level variables (fields, properties) within > one class instance to be locked then keep them in a class/structure and lock > that class/structure instance... > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Sat Mar 26 00:37:25 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sat, 26 Mar 2011 08:37:25 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8D5612.6050701@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8D0B4B.9070004@colbyconsulting.com> <4D8D5612.6050701@colbyconsulting.com> Message-ID: <4D87466503764D98871B609159033CB6@nant> John -- But you can try to use several instances of .NET Framework Queue class http://msdn.microsoft.com/en-us/library/system.collections.queue.aspx ? As far as I understand you have a set of sequential tasks to be performed on every (input/start) chunk so you can make a dedicated Queue instance for every task, and that Queue instance will be handled by dedicated "processor" classes: - TaskA Queue items will be processed by TaskAQueueProcessor, - TaksB .... by TaskBQueueProcessor, -- - TaskZ... by TaskZQueueProcessor There could be many such TaskA(B, ....Z)QueueProcessor instances "floating around" looking for an item from their dedicated tasks' queue to process: when they find it, they pick it up, process and put into another Queue based on "workflow plan"... You can also try to consider using Windows Workflow Foundation(WF) if that wouldn't be an "overkill" for your case... http://msdn.microsoft.com/en-us/netframework/wf-screencasts.aspx#introwf4 Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 26 ????? 2011 ?. 5:57 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, I have processes that log results to flags. For example, make a database (log that it was made), build a table (log that it was built), pull umpteen million records in sorted order (log that it was filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build another chunk table (log that it was filled), BCP out (log that it was exported). The objective is to be able to sustain interruptions and pick up where we left off. These processes can take minutes (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). So each thing I do represents a step in the process and each step is logged in a field in a record in SQL server using a datetime. There are so many of these flags that I am trying to standardize the process by building a class that can be instantiated, filled with data and log itself to SQL Server by one thread and be checked by another thread. These flag class instances will be checked by multiple threads, each thread trying to decide whether it should be doing the next step because another thread has finished it's part. IOW if a file has been written to disk, then the next thread will write it to a VM for processing. If it moved to the VM the next thread will watch the VM's output directory for a file to pop out and move it back to a directory on the server. If the file (a couple of files actually) successfully copied back to the server staging then another thread will import it back into a chunk table in an input database. If the file successfully imported then another thread will... In general one thread will "own" the flag and use it to log its status and one other thread will be checking the status of the flag to determine that it can go to work on that work chunk. You get the picture. I am trying to build an entirely asynchronous highly threaded process which exports a huge table into multiple files, processes every file through a third party app and gets the results back into SQL Server. All while logging each and every step so that no piece can possibly be dropped at any stage, even if the server goes down (or the VM goes down). Eventually this process will run on my server 24/7. It has been working for some time but I am getting threading issues, and I need to work on the high level control so that all of the processes can cleanly start up and shut down and every stage can pick back up when the program restarts should a shutdown occur. A single database can be up to a hundred million records (the biggest so far), and the external program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour depending on many different things so that example will take 50 chunks and could take 40 to 50 hours to complete. It takes about 20 processing steps to handle each file from end to end. It needs to just work, and I need to be able to view status in a meaningful way. And I need to process that and a dozen other files every single month, automatically, with no manual intervention required. John W. Colby www.ColbyConsulting.com On 3/25/2011 6:46 PM, Shamil Salakhetdinov wrote: > Hi John -- > > By resource I mean access to a read/write resource - it could be not > only a variable... > > If you need two or more class level variables (fields, properties) > within one class instance to be locked then keep them in a > class/structure and lock that class/structure instance... > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From gustav at cactus.dk Sat Mar 26 07:41:44 2011 From: gustav at cactus.dk (Gustav Brock) Date: Sat, 26 Mar 2011 13:41:44 +0100 Subject: [dba-VB] c# lock() Message-ID: Hi John My first thought was Windows Workflow Foundation(WF) but, as Shamil mentions, it may be overkill. On the other hand it is exactly for controlling scenarios where "if task 1 is done, start task 2 and task 3, wait for task 2 and task 3 to finish, check something, then start task 4, etc.". However, that reminded me about a series of articles about the Task Parallel Library of .Net: http://www.codeproject.com/KB/cs/TPL1.aspx which I found very interesting, though I haven't had any use for it yet. Among other topics it discusses carefully canceling and error handling which I guess is quite important for your purpose. /gustav >>> jwcolby at colbyconsulting.com 26-03-2011 03:57 >>> Shamil, I have processes that log results to flags. For example, make a database (log that it was made), build a table (log that it was built), pull umpteen million records in sorted order (log that it was filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build another chunk table (log that it was filled), BCP out (log that it was exported). The objective is to be able to sustain interruptions and pick up where we left off. These processes can take minutes (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). So each thing I do represents a step in the process and each step is logged in a field in a record in SQL server using a datetime. There are so many of these flags that I am trying to standardize the process by building a class that can be instantiated, filled with data and log itself to SQL Server by one thread and be checked by another thread. These flag class instances will be checked by multiple threads, each thread trying to decide whether it should be doing the next step because another thread has finished it's part. IOW if a file has been written to disk, then the next thread will write it to a VM for processing. If it moved to the VM the next thread will watch the VM's output directory for a file to pop out and move it back to a directory on the server. If the file (a couple of files actually) successfully copied back to the server staging then another thread will import it back into a chunk table in an input database. If the file successfully imported then another thread will... In general one thread will "own" the flag and use it to log its status and one other thread will be checking the status of the flag to determine that it can go to work on that work chunk. You get the picture. I am trying to build an entirely asynchronous highly threaded process which exports a huge table into multiple files, processes every file through a third party app and gets the results back into SQL Server. All while logging each and every step so that no piece can possibly be dropped at any stage, even if the server goes down (or the VM goes down). Eventually this process will run on my server 24/7. It has been working for some time but I am getting threading issues, and I need to work on the high level control so that all of the processes can cleanly start up and shut down and every stage can pick back up when the program restarts should a shutdown occur. A single database can be up to a hundred million records (the biggest so far), and the external program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour depending on many different things so that example will take 50 chunks and could take 40 to 50 hours to complete. It takes about 20 processing steps to handle each file from end to end. It needs to just work, and I need to be able to view status in a meaningful way. And I need to process that and a dozen other files every single month, automatically, with no manual intervention required. John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 26 07:44:54 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 08:44:54 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: Message-ID: <4D8DDFC6.4040004@colbyconsulting.com> Well... I haven't migrated to VS2010 yet. I tried it about 6 months ago and it was universally a pig on my systems. I thought I would wait until MS polished it a bit. John W. Colby www.ColbyConsulting.com On 3/26/2011 8:41 AM, Gustav Brock wrote: > Hi John > > My first thought was Windows Workflow Foundation(WF) but, as Shamil mentions, it may be overkill. On the other hand it is exactly for controlling scenarios where "if task 1 is done, start task 2 and task 3, wait for task 2 and task 3 to finish, check something, then start task 4, etc.". > > However, that reminded me about a series of articles about the Task Parallel Library of .Net: > > http://www.codeproject.com/KB/cs/TPL1.aspx > > which I found very interesting, though I haven't had any use for it yet. Among other topics it discusses carefully canceling and error handling which I guess is quite important for your purpose. > > /gustav > > >>>> jwcolby at colbyconsulting.com 26-03-2011 03:57>>> > Shamil, > > I have processes that log results to flags. For example, make a database (log that it was made), > build a table (log that it was built), pull umpteen million records in sorted order (log that it was > filled), build a chunk table (log that it was filled), bcp out (log that it was exported), build > another chunk table (log that it was filled), BCP out (log that it was exported). The objective is > to be able to sustain interruptions and pick up where we left off. These processes can take minutes > (fill chunk table, bcp out) or a half hour (pull umpteen million records in sorted order). > > So each thing I do represents a step in the process and each step is logged in a field in a record > in SQL server using a datetime. There are so many of these flags that I am trying to standardize > the process by building a class that can be instantiated, filled with data and log itself to SQL > Server by one thread and be checked by another thread. > > These flag class instances will be checked by multiple threads, each thread trying to decide whether > it should be doing the next step because another thread has finished it's part. IOW if a file has > been written to disk, then the next thread will write it to a VM for processing. If it moved to the > VM the next thread will watch the VM's output directory for a file to pop out and move it back to a > directory on the server. If the file (a couple of files actually) successfully copied back to the > server staging then another thread will import it back into a chunk table in an input database. If > the file successfully imported then another thread will... In general one thread will "own" the > flag and use it to log its status and one other thread will be checking the status of the flag to > determine that it can go to work on that work chunk. > > You get the picture. > > I am trying to build an entirely asynchronous highly threaded process which exports a huge table > into multiple files, processes every file through a third party app and gets the results back into > SQL Server. All while logging each and every step so that no piece can possibly be dropped at any > stage, even if the server goes down (or the VM goes down). Eventually this process will run on my > server 24/7. > > It has been working for some time but I am getting threading issues, and I need to work on the high > level control so that all of the processes can cleanly start up and shut down and every stage can > pick back up when the program restarts should a shutdown occur. > > A single database can be up to a hundred million records (the biggest so far), and the external > program only handles roughly 2 million records. Each "chunk" takes roughly 45 minutes to an hour > depending on many different things so that example will take 50 chunks and could take 40 to 50 hours > to complete. It takes about 20 processing steps to handle each file from end to end. It needs to > just work, and I need to be able to view status in a meaningful way. And I need to process that and > a dozen other files every single month, automatically, with no manual intervention required. > > > > John W. Colby > www.ColbyConsulting.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From gustav at cactus.dk Sat Mar 26 08:08:10 2011 From: gustav at cactus.dk (Gustav Brock) Date: Sat, 26 Mar 2011 14:08:10 +0100 Subject: [dba-VB] c# lock() Message-ID: Hi John Oh, I recall your trouble now that you mention it. Strange, as it runs fine on my antique Pentium 4 IBM IntelliStation with 2 GB ram. However, VS2010 _has_ been polished: The Service Pack 1 was released a few weeks ago. /gustav >>> jwcolby at colbyconsulting.com 26-03-2011 13:44 >>> Well... I haven't migrated to VS2010 yet. I tried it about 6 months ago and it was universally a pig on my systems. I thought I would wait until MS polished it a bit. John W. Colby www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 26 08:26:26 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 09:26:26 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: Message-ID: <4D8DE982.7060707@colbyconsulting.com> When I had my speed issues I Googled around and found tons with speed issues, and tons without. There are entire threads bitterly complaining about what a dog it is. There was no explanations of when or why. Mine would take forever to do anything. I will definitely try again now that SP1 is out. John W. Colby www.ColbyConsulting.com On 3/26/2011 9:08 AM, Gustav Brock wrote: > Hi John > > Oh, I recall your trouble now that you mention it. Strange, as it runs fine on my antique Pentium 4 IBM IntelliStation with 2 GB ram. > However, VS2010 _has_ been polished: The Service Pack 1 was released a few weeks ago. > > /gustav > > >>>> jwcolby at colbyconsulting.com 26-03-2011 13:44>>> > Well... I haven't migrated to VS2010 yet. > > I tried it about 6 months ago and it was universally a pig on my systems. I thought I would wait > until MS polished it a bit. > > John W. Colby > www.ColbyConsulting.com > > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From jwcolby at colbyconsulting.com Sat Mar 26 16:53:15 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 17:53:15 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <3F0DA84A905C4947819126F43CFC39DB@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com> <3F0DA84A905C4947819126F43CFC39DB@nant> Message-ID: <4D8E604B.5040001@colbyconsulting.com> Shamil, I am trying to build a base class which contains a set of lock objects and some standard variables. Having defined the lock objects static per your code below, the derived class cannot access the lock objects even though I set them protected: protected static Object LockBln = new Object(); In the derived class when I try to access it: lock(this.LockBln) { } I get a compile error: "Member cannot be accesses with an instance reference; Qualify it with a type name instead." What does this mean? Reading on the internet it says that static properties are shared between all instances of the class. I don't see how I can share a property between 50 instances and then use a lock(MyStaticProperty) {}. I'm so confused... John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - just > make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - otherwise > "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various operations in a > class. > > For example I want to lock a date flag variable while using a property to > either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable but > rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com From shamil at smsconsulting.spb.ru Sat Mar 26 17:27:17 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sun, 27 Mar 2011 01:27:17 +0300 Subject: [dba-VB] c# lock() In-Reply-To: <4D8E604B.5040001@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8E604B.5040001@colbyconsulting.com> Message-ID: <00D0A03B172743B6868A8BE736F7E560@nant> John -- Let's define your target (application) system architecture on conceptual level - I guess that using "micro-level locking" you're trying (?) to achieve isn't needed... 1) you have a huge input data set; 2) you split input data set into X chunks; 3) every chunk gets processed using the same several - Y - steps; 4) on completion of step Yn input/semi-processed chunk Xn is submitted to the Y(n+1) processor class; 5) when all X chunks are processed in Y steps they get collected into a final "huge" output set to produce Z reports. That's it from conceptual high level point of view? Something important missing? Too simplified to be true? John, please take into account that I can't reply promptly here - it's weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to sleep, and then I have quite some work to do starting Monday - so I expect this thread will be supported by other AccessD/dba-VB members... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 0:53 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, I am trying to build a base class which contains a set of lock objects and some standard variables. Having defined the lock objects static per your code below, the derived class cannot access the lock objects even though I set them protected: protected static Object LockBln = new Object(); In the derived class when I try to access it: lock(this.LockBln) { } I get a compile error: "Member cannot be accesses with an instance reference; Qualify it with a type name instead." What does this mean? Reading on the internet it says that static properties are shared between all instances of the class. I don't see how I can share a property between 50 instances and then use a lock(MyStaticProperty) {}. I'm so confused... John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - > just make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - > otherwise "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various > operations in a class. > > For example I want to lock a date flag variable while using a property > to either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable > but rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sat Mar 26 18:31:31 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sat, 26 Mar 2011 19:31:31 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <00D0A03B172743B6868A8BE736F7E560@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant> Message-ID: <4D8E7753.8090006@colbyconsulting.com> I'm just trying to get a handle on locking things (in general). I am seeing things that say "just lock the object itself", and then other things that say "use a static object (the way you were showing me). If locking the object itself works, why do the static thing? John W. Colby www.ColbyConsulting.com On 3/26/2011 6:27 PM, Shamil Salakhetdinov wrote: > John -- > > Let's define your target (application) system architecture on conceptual > level - I guess that using "micro-level locking" you're trying (?) to > achieve isn't needed... > > 1) you have a huge input data set; > 2) you split input data set into X chunks; > 3) every chunk gets processed using the same several - Y - steps; > 4) on completion of step Yn input/semi-processed chunk Xn is submitted to > the Y(n+1) processor class; > 5) when all X chunks are processed in Y steps they get collected into a > final "huge" output set to produce Z reports. > > That's it from conceptual high level point of view? > Something important missing? > Too simplified to be true? > > John, please take into account that I can't reply promptly here - it's > weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to sleep, > and then I have quite some work to do starting Monday - so I expect this > thread will be supported by other AccessD/dba-VB members... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 27 ????? 2011 ?. 0:53 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > I am trying to build a base class which contains a set of lock objects and > some standard variables. > Having defined the lock objects static per your code below, the derived > class cannot access the lock objects even though I set them protected: > > protected static Object LockBln = new Object(); > > In the derived class when I try to access it: > > lock(this.LockBln) > { > } > > I get a compile error: > > "Member cannot be accesses with an instance reference; Qualify it with a > type name instead." > > What does this mean? > > Reading on the internet it says that static properties are shared between > all instances of the class. I don't see how I can share a property between > 50 instances and then use a > lock(MyStaticProperty) {}. > > I'm so confused... > > John W. Colby > www.ColbyConsulting.com > > On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Your sample is a correct usage of locking - just use static thisLock >> variable >> >> private static Object thisLock = new Object() >> >> as non-static variable *is not* thread safe AFAIU. >> >> Yes, such locking can be done for various operations of that class - >> just make sure you'll not get dead-locked... >> Also use *one lock object* to lock *one resource* in a class - >> otherwise "deadlock" will become your "everyday guest"... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 25 ????? 2011 ?. 22:58 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Does this allow using a single "lock object" to lock various >> operations in a class. >> >> For example I want to lock a date flag variable while using a property >> to either get or set the variable. >> >> private Object thisLock = new Object(); >> >> public DateTime pFlagDte { >> get { >> lock (thisLock) >> { >> return dteFlag; >> } >> } >> set >> { >> lock (thisLock) >> { >> dteFlag = value; >> } >> } >> } >> >> The issue is not multiple threads trying to write to the date variable >> but rather one thread trying to read it while another is writing it. >> >> John W. Colby >> www.ColbyConsulting.com > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > _______________________________________________ > dba-VB mailing list > dba-VB at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-vb > http://www.databaseadvisors.com > > From shamil at smsconsulting.spb.ru Sat Mar 26 19:08:10 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sun, 27 Mar 2011 04:08:10 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <00D0A03B172743B6868A8BE736F7E560@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant> Message-ID: <0CD7EF3475CB43C69334942027697B63@nant> John -- I'm falling asleep here now - would that be correct assumption that you can generalize your global task like that - you have, say: - split input data into 50 Chunks; - every chunk should be processed in 10 steps; - every step's average processing time is 100 minutes. Then in total it is needed 50*10*100 = 50,000 minutes or ~833.(3) hours to process all your input data sequentially, chunk after chunk step after step, say, you don't have enough computer power to process some chunks in parallel, and you can't process the whole not split input data as it's too large for your system... But you have "50 small computers" and you can process every of 50 input chunks in parallel - then you'll need just 10*100 = 1,000 minutes (~16.67 hours) to complete the job. I assume that all the chunks processing is independent - then you can: 1) Create Scheduler class instance; 2) Schedule class creates 10 TaskQueue class instances - 1 queue for every processing step type; 3) Scheduler class defines splitting criteria to split input data into 50 chunks; 4) Schedule defines completion criteria - when all 50 chunks get collected in 11th FinalQueue; 5) Schedule seeds the first InputChunkSplitTaskQueue with 50 SplitTasksDescriptors; 4) From time to time - say every half a minute Scheduler class instance creates 500 worker classes in 500 threads (500 worker classes => 50 chunks * 10 steps.), which "hunt for the job" in their dedicate Queues: if they find Job they work on it, if not - they "die"... 5) When a worker class completes processing its job it queues its "task results" into the next step Queue, if a worker class fails then it puts its incomplete task descriptor back into its dedicated queue and "dies", or that could be Scheduler's (or SchedulerAssitant's) task job to find failed workers' "trap" and to resubmit their failed work to the proper queue according to the custom workflow descriptors attached to every chunk... 500 worker classes is an overkill as only 50 worker classes can have job every time but that seems to be an easy "brute force" and "lazy parallel programming" approach - and it should work... Or you can make worker class instances production processes smarter: Scheduler class can start special thread for WorkerClassGenerator instance, which will monitor all the 10 TaskQueues, and if it finds an item in a Queque, it will pick it up, it will create corresponding Worker class in parallel thread and it will pass WorkItem to the Worker class fro processing... When that described above approach will work there then you can easily(?) scale it splitting your (constant size) input data into 100 chunks, and then if every chunk can be processes in half time - in average (50 minutes) - then all job can be completed in 500 minutes = ~8.33(3) hours etc... Please correct me if I oversimplified your application business area... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of Shamil Salakhetdinov Sent: 27 ????? 2011 ?. 1:27 To: 'Discussion concerning Visual Basic and related programming issues.' Subject: Re: [dba-VB] c# lock() John -- Let's define your target (application) system architecture on conceptual level - I guess that using "micro-level locking" you're trying (?) to achieve isn't needed... 1) you have a huge input data set; 2) you split input data set into X chunks; 3) every chunk gets processed using the same several - Y - steps; 4) on completion of step Yn input/semi-processed chunk Xn is submitted to the Y(n+1) processor class; 5) when all X chunks are processed in Y steps they get collected into a final "huge" output set to produce Z reports. That's it from conceptual high level point of view? Something important missing? Too simplified to be true? John, please take into account that I can't reply promptly here - it's weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to sleep, and then I have quite some work to do starting Monday - so I expect this thread will be supported by other AccessD/dba-VB members... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 0:53 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Shamil, I am trying to build a base class which contains a set of lock objects and some standard variables. Having defined the lock objects static per your code below, the derived class cannot access the lock objects even though I set them protected: protected static Object LockBln = new Object(); In the derived class when I try to access it: lock(this.LockBln) { } I get a compile error: "Member cannot be accesses with an instance reference; Qualify it with a type name instead." What does this mean? Reading on the internet it says that static properties are shared between all instances of the class. I don't see how I can share a property between 50 instances and then use a lock(MyStaticProperty) {}. I'm so confused... John W. Colby www.ColbyConsulting.com On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Your sample is a correct usage of locking - just use static thisLock > variable > > private static Object thisLock = new Object() > > as non-static variable *is not* thread safe AFAIU. > > Yes, such locking can be done for various operations of that class - > just make sure you'll not get dead-locked... > Also use *one lock object* to lock *one resource* in a class - > otherwise "deadlock" will become your "everyday guest"... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 25 ????? 2011 ?. 22:58 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > Does this allow using a single "lock object" to lock various > operations in a class. > > For example I want to lock a date flag variable while using a property > to either get or set the variable. > > private Object thisLock = new Object(); > > public DateTime pFlagDte { > get { > lock (thisLock) > { > return dteFlag; > } > } > set > { > lock (thisLock) > { > dteFlag = value; > } > } > } > > The issue is not multiple threads trying to write to the date variable > but rather one thread trying to read it while another is writing it. > > John W. Colby > www.ColbyConsulting.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From shamil at smsconsulting.spb.ru Sat Mar 26 19:38:28 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Sun, 27 Mar 2011 04:38:28 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <4D8E7753.8090006@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant> <4D8E604B.5040001@colbyconsulting.com><00D0A03B172743B6868A8BE736F7E560@nant> <4D8E7753.8090006@colbyconsulting.com> Message-ID: Hi John -- <<< If locking the object itself works, why do the static thing? >>> Static is used to lock shared resources - in your case there are no such resources at all? Log writer is usually a static shared resource... And yes, you can use your locking code to lock parallel access to your class instances' local variables: >> private Object thisLock = new Object(); >> >> public DateTime pFlagDte { >> get { >> lock (thisLock) >> { >> return dteFlag; >> } >> } >> set >> { >> lock (thisLock) >> { >> dteFlag = value; >> } >> } >> } But as I noted I'm not sure if that "micro-level locking" is needed at all (and you mentioned you have several such shared variables within a class instance) - using Queues seems to be more simple and (thread-)safe approach without almost any custom locking schemes.... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 3:32 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() I'm just trying to get a handle on locking things (in general). I am seeing things that say "just lock the object itself", and then other things that say "use a static object (the way you were showing me). If locking the object itself works, why do the static thing? John W. Colby www.ColbyConsulting.com On 3/26/2011 6:27 PM, Shamil Salakhetdinov wrote: > John -- > > Let's define your target (application) system architecture on > conceptual level - I guess that using "micro-level locking" you're > trying (?) to achieve isn't needed... > > 1) you have a huge input data set; > 2) you split input data set into X chunks; > 3) every chunk gets processed using the same several - Y - steps; > 4) on completion of step Yn input/semi-processed chunk Xn is submitted > to the Y(n+1) processor class; > 5) when all X chunks are processed in Y steps they get collected into > a final "huge" output set to produce Z reports. > > That's it from conceptual high level point of view? > Something important missing? > Too simplified to be true? > > John, please take into account that I can't reply promptly here - it's > weekend now and I'll be out tomorrow, and it's 1:26 a.m. - time to > sleep, and then I have quite some work to do starting Monday - so I > expect this thread will be supported by other AccessD/dba-VB members... > > Thank you. > > -- > Shamil > > -----Original Message----- > From: dba-vb-bounces at databaseadvisors.com > [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby > Sent: 27 ????? 2011 ?. 0:53 > To: Discussion concerning Visual Basic and related programming issues. > Subject: Re: [dba-VB] c# lock() > > Shamil, > > I am trying to build a base class which contains a set of lock objects > and some standard variables. > Having defined the lock objects static per your code below, the > derived class cannot access the lock objects even though I set them protected: > > protected static Object LockBln = new Object(); > > In the derived class when I try to access it: > > lock(this.LockBln) > { > } > > I get a compile error: > > "Member cannot be accesses with an instance reference; Qualify it with > a type name instead." > > What does this mean? > > Reading on the internet it says that static properties are shared > between all instances of the class. I don't see how I can share a > property between 50 instances and then use a > lock(MyStaticProperty) {}. > > I'm so confused... > > John W. Colby > www.ColbyConsulting.com > > On 3/25/2011 5:05 PM, Shamil Salakhetdinov wrote: >> Hi John -- >> >> Your sample is a correct usage of locking - just use static thisLock >> variable >> >> private static Object thisLock = new Object() >> >> as non-static variable *is not* thread safe AFAIU. >> >> Yes, such locking can be done for various operations of that class - >> just make sure you'll not get dead-locked... >> Also use *one lock object* to lock *one resource* in a class - >> otherwise "deadlock" will become your "everyday guest"... >> >> Thank you. >> >> -- >> Shamil >> >> -----Original Message----- >> From: dba-vb-bounces at databaseadvisors.com >> [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby >> Sent: 25 ????? 2011 ?. 22:58 >> To: Discussion concerning Visual Basic and related programming issues. >> Subject: Re: [dba-VB] c# lock() >> >> Shamil, >> >> Does this allow using a single "lock object" to lock various >> operations in a class. >> >> For example I want to lock a date flag variable while using a >> property to either get or set the variable. >> >> private Object thisLock = new Object(); >> >> public DateTime pFlagDte { >> get { >> lock (thisLock) >> { >> return dteFlag; >> } >> } >> set >> { >> lock (thisLock) >> { >> dteFlag = value; >> } >> } >> } >> >> The issue is not multiple threads trying to write to the date >> variable but rather one thread trying to read it while another is writing it. >> >> John W. Colby >> www.ColbyConsulting.com From jwcolby at colbyconsulting.com Sat Mar 26 23:12:20 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 27 Mar 2011 00:12:20 -0400 Subject: [dba-VB] c# lock() In-Reply-To: <0CD7EF3475CB43C69334942027697B63@nant> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant> <0CD7EF3475CB43C69334942027697B63@nant> Message-ID: <4D8EB924.5080708@colbyconsulting.com> Well... In programming you can do any task 50 different ways. The way I did it is to create a supervisor and ProcessFile table in SQL Server. The supervisor represents a table in SQL Server that contains an address table which needs to be processed. The ProcessFile records are child to tblSupervisor and represent chunks (files). The supervisor record stores all of the high level flags about the database (address table) being processed. The ProcessFile contains all of the flags about each chunk being processed. I have four different high level "stages" as I call them that a supervisor goes through. AccuzipOut - table to csv files in staging out (stage 1), Staging to VM (stage 2), VM to Staging in (stage 3, Staging in to SQL Server (stage 4) 1) Export all chunks (files) to a SQL Out staging directory. Each chunk ends up being a CSV file with PKID, FName, LName, Addr, City, St, Zip5, Zip4. The ProcessFile needs to be sorted by Zip5 / Zip4 order because the third party application that processes these files is immensely faster if the file is presorted. 2) One by one, copy each chunk file to a virtual machine - AccuzipIn directory. The Accuzip virtual machine runs on a Windows 2008 server running Hyper-V manager. I currently run 3 different VMs, only one of which runs Accuzip. For awhile I actually ran 3 VMs running Accuzip, and I am designing such that I can add additional Accuzip VMs in the future. What this means is that each file could go to any valid Accuzip VM, and when the chunk file is placed on a VM it has to log which VM it went to because... 3) The Accuzip software automatically senses and processes the file in the InputTo directory. A file of 2 million records takes about 25 minutes to CASS process locally to the VM, then is uploaded to a server in California for further NCOA processing. The server in California belongs to Accuzip and they stage big files so it can take awhile before it even begins to process. Once it processes, the NCOA server sends a small file back to my VM where Address change information is integrated back in to the local Accuzip database file (foxpro). After that it is exported back to CSV with a bunch of additional fields and is placed in the Accuzip OutputFrom directory along with a PDF which contains the CASS / NCOA statistics. This PDF is a "proof of processing" document that I have to keep forever, and I have to send to the client. I have a Directory Watcher which is watching the OutputFrom directory. When *all* of the resulting Accuzip output files are in the OutputFrom directory, my Directory Watcher event triggers my program to copy all of these files back to the SQL Server In staging directory. 4) Once all of the Accuzip files for a single chunk are back on the SQL Server In directory, my application BCPs the CSV into a input chunk table. From there decisions are made whether I need to import every address back in or only the moves. In any event some or all of the address records (with the additional fields from Accuzip) are pulled into an AZData table back in the database that the address data originally came from. So... 1) A supervisor object creates a clsDbAZOut instance. 2) clsDBAZOut creates an actual database dbXyzAZOut temporary database. 3) clsDBAZOut pulls every record needing to be accuzipped into a table with a super PK, sorted on Zip5/4 4) clsDBAZOut counts the records in tblAZOut, and starts to build clsProcessFile instances which holds the information about the chunk processing. clsProcessFile can write its info back to the ProcessFile records on SQL Server. 5) clsDbAZOut has a thread which builds tblChunk, gets ChunkSize records into the chunk table and BCPs tblChunk to CSV files on a staging out directory. 6) clsDbAZOut BCPs the chunk to file 7) At this point the clsProcessFile is handed off to clsVM. clsSupervisor has a strongly typed dictionary which stores the clsProcessFile instances created by clsDbAZOut. clsSupervisor can read ProcessFile instances from SQL Server into a clsProcessFile factory or clsDbAZOut can use a class factory to create new ones. In all cases they are stored in the dictionary in clsSupervisor. clsProcessfile has flags which document that: a) The chunk table was created b) The chunk table exported to file c) The chunk went to a VM (and which one) d) The chunk file was (eventually) found in the Accuzip OutputFrom directory e) The resulting AZData file (and PDF) were moved from the VM back to SQL ServerIn staging f) The file in SQL Server In staging was pulled back in to a chunk table in a second database dbXyzAZIn temporary database g) That the chunk table was imported back into the original database. As you can see, clsProcessFile is the center of the universe for a single chunk. clsSupervisor is the center of the universe for the entire process. It is a bit difficult to discuss how the entire program works in a single email. A clsManager loads one or a dozen clsSupervisor instances. clsManager loads clsVM which supervises the (a) virtual machine which runs a third party process. clsVM moves files from SQL Staging Out into the VM one chunk at a time, updating clsProcessTable flags. clsVM moves files from the the VM back to the SQL Staging IN, updating flags. The VM may not be available, or the Accuzip program may be down etc. clsSupervisor loads one to N clsProcessTable instances. clsSupervisor loads a clsDBExp and clsDBImp. clsDBExp creates new clsProcessTable instances, one for each chunk it exports - storing them in clsSupervisor. clsDBExp supervises the export process. clsDBImp supervises the import process getting the chunks back into SQL Server into a temp database, and from there into the live database. I am trying to design the system such that it is asynchronous. Stage1, stage 2/3 and stage 4 can all do their thing independently of each other. Each step of each stage is logged (dateTime flags written) so that any piece can just pick up where it left off. That means any Supervisor and any ProcessFile (chunk) can pick up from any point. In the meantime, the clsManager is watching the tblSupervisor for supervisor records ready to process - a thread. clsSupervisor is watching for ProcessTable records and dispatching them as required to the clsDBExp, clsDBImp or clsVM - a thread in each supervisor. clsDBExp is processing chunks, creating CSV files - a thread. clsVM is watching for clsProcessTable instances ready to go to the VM. It is also watching the output directory and sending files back to the SQL Server - a thread and two file watchers (and two events). clsDBImp is watching for clsProcessTable instances ready to import back into SQL Server - a thread. Most of these threads are writing flags (as they finish their specific process) and reading flags in clsProcessTable to decide what chunk has to be processed in which class. Oh, and I have three list controls on a form which the various classes write to to display the status for Stage1, Stage 2/3 and Stage 4. So pretty much a single supervisor is processing at a time, exporting files, sending them to the vm, importing them back in etc. It is possible for an order supervisor to cut in at the head of the line however. Not a simple task co-ordinating all of this stuff. I used to co-ordinate it all manually! ;) John W. Colby www.ColbyConsulting.com On 3/26/2011 8:08 PM, Shamil Salakhetdinov wrote: > John -- > > I'm falling asleep here now - would that be correct assumption that you can > generalize your global task like that - you have, say: > > - split input data into 50 Chunks; > - every chunk should be processed in 10 steps; > - every step's average processing time is 100 minutes. > > Then in total it is needed 50*10*100 = 50,000 minutes or ~833.(3) hours to > process all your input data sequentially, chunk after chunk step after step, > say, you don't have enough computer power to process some chunks in > parallel, and you can't process the whole not split input data as it's too > large for your system... > > But you have "50 small computers" and you can process every of 50 input > chunks in parallel - then you'll need just 10*100 = 1,000 minutes (~16.67 > hours) to complete the job. > > I assume that all the chunks processing is independent - then you can: > > 1) Create Scheduler class instance; > 2) Schedule class creates 10 TaskQueue class instances - 1 queue for every > processing step type; > 3) Scheduler class defines splitting criteria to split input data into 50 > chunks; > 4) Schedule defines completion criteria - when all 50 chunks get collected > in 11th FinalQueue; > 5) Schedule seeds the first InputChunkSplitTaskQueue with 50 > SplitTasksDescriptors; > 4) From time to time - say every half a minute Scheduler class instance > creates 500 worker classes in 500 threads (500 worker classes => 50 chunks * > 10 steps.), which "hunt for the job" in their dedicate Queues: if they find > Job they work on it, if not - they "die"... > 5) When a worker class completes processing its job it queues its "task > results" into the next step Queue, if a worker class fails then it puts its > incomplete task descriptor back into its dedicated queue and "dies", or that > could be Scheduler's (or SchedulerAssitant's) task job to find failed > workers' "trap" and to resubmit their failed work to the proper queue > according to the custom workflow descriptors attached to every chunk... > > 500 worker classes is an overkill as only 50 worker classes can have job > every time but that seems to be an easy "brute force" and "lazy parallel > programming" approach - and it should work... > Or you can make worker class instances production processes smarter: > Scheduler class can start special thread for WorkerClassGenerator instance, > which will monitor all the 10 TaskQueues, and if it finds an item in a > Queque, it will pick it up, it will create corresponding Worker class in > parallel thread and it will pass WorkItem to the Worker class fro > processing... > > When that described above approach will work there then you can easily(?) > scale it splitting your (constant size) input data into 100 chunks, and then > if every chunk can be processes in half time - in average (50 minutes) - > then all job can be completed in 500 minutes = ~8.33(3) hours etc... > > Please correct me if I oversimplified your application business area... > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Sun Mar 27 19:08:25 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Mon, 28 Mar 2011 04:08:25 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <4D8EB924.5080708@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> Message-ID: Hi John -- Yes, I see: you tend to build a multi-threaded application/utility system with centralized control - and I tend to build an anarchical free-market model based application/utility system :) Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 27 ????? 2011 ?. 8:12 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() Well... In programming you can do any task 50 different ways. The way I did it is to create a supervisor and ProcessFile table in SQL Server. The supervisor represents a table in SQL Server that contains an address table which needs to be processed. The ProcessFile records are child to tblSupervisor and represent chunks (files). The supervisor record stores all of the high level flags about the database (address table) being processed. The ProcessFile contains all of the flags about each chunk being processed. I have four different high level "stages" as I call them that a supervisor goes through. AccuzipOut - table to csv files in staging out (stage 1), Staging to VM (stage 2), VM to Staging in (stage 3, Staging in to SQL Server (stage 4) 1) Export all chunks (files) to a SQL Out staging directory. Each chunk ends up being a CSV file with PKID, FName, LName, Addr, City, St, Zip5, Zip4. The ProcessFile needs to be sorted by Zip5 / Zip4 order because the third party application that processes these files is immensely faster if the file is presorted. 2) One by one, copy each chunk file to a virtual machine - AccuzipIn directory. The Accuzip virtual machine runs on a Windows 2008 server running Hyper-V manager. I currently run 3 different VMs, only one of which runs Accuzip. For awhile I actually ran 3 VMs running Accuzip, and I am designing such that I can add additional Accuzip VMs in the future. What this means is that each file could go to any valid Accuzip VM, and when the chunk file is placed on a VM it has to log which VM it went to because... 3) The Accuzip software automatically senses and processes the file in the InputTo directory. A file of 2 million records takes about 25 minutes to CASS process locally to the VM, then is uploaded to a server in California for further NCOA processing. The server in California belongs to Accuzip and they stage big files so it can take awhile before it even begins to process. Once it processes, the NCOA server sends a small file back to my VM where Address change information is integrated back in to the local Accuzip database file (foxpro). After that it is exported back to CSV with a bunch of additional fields and is placed in the Accuzip OutputFrom directory along with a PDF which contains the CASS / NCOA statistics. This PDF is a "proof of processing" document that I have to keep forever, and I have to send to the client. I have a Directory Watcher which is watching the OutputFrom directory. When *all* of the resulting Accuzip output files are in the OutputFrom directory, my Directory Watcher event triggers my program to copy all of these files back to the SQL Server In staging directory. 4) Once all of the Accuzip files for a single chunk are back on the SQL Server In directory, my application BCPs the CSV into a input chunk table. >From there decisions are made whether I need to import every address back in or only the moves. In any event some or all of the address records (with the additional fields from Accuzip) are pulled into an AZData table back in the database that the address data originally came from. So... 1) A supervisor object creates a clsDbAZOut instance. 2) clsDBAZOut creates an actual database dbXyzAZOut temporary database. 3) clsDBAZOut pulls every record needing to be accuzipped into a table with a super PK, sorted on Zip5/4 4) clsDBAZOut counts the records in tblAZOut, and starts to build clsProcessFile instances which holds the information about the chunk processing. clsProcessFile can write its info back to the ProcessFile records on SQL Server. 5) clsDbAZOut has a thread which builds tblChunk, gets ChunkSize records into the chunk table and BCPs tblChunk to CSV files on a staging out directory. 6) clsDbAZOut BCPs the chunk to file 7) At this point the clsProcessFile is handed off to clsVM. clsSupervisor has a strongly typed dictionary which stores the clsProcessFile instances created by clsDbAZOut. clsSupervisor can read ProcessFile instances from SQL Server into a clsProcessFile factory or clsDbAZOut can use a class factory to create new ones. In all cases they are stored in the dictionary in clsSupervisor. clsProcessfile has flags which document that: a) The chunk table was created b) The chunk table exported to file c) The chunk went to a VM (and which one) d) The chunk file was (eventually) found in the Accuzip OutputFrom directory e) The resulting AZData file (and PDF) were moved from the VM back to SQL ServerIn staging f) The file in SQL Server In staging was pulled back in to a chunk table in a second database dbXyzAZIn temporary database g) That the chunk table was imported back into the original database. As you can see, clsProcessFile is the center of the universe for a single chunk. clsSupervisor is the center of the universe for the entire process. It is a bit difficult to discuss how the entire program works in a single email. A clsManager loads one or a dozen clsSupervisor instances. clsManager loads clsVM which supervises the (a) virtual machine which runs a third party process. clsVM moves files from SQL Staging Out into the VM one chunk at a time, updating clsProcessTable flags. clsVM moves files from the the VM back to the SQL Staging IN, updating flags. The VM may not be available, or the Accuzip program may be down etc. clsSupervisor loads one to N clsProcessTable instances. clsSupervisor loads a clsDBExp and clsDBImp. clsDBExp creates new clsProcessTable instances, one for each chunk it exports - storing them in clsSupervisor. clsDBExp supervises the export process. clsDBImp supervises the import process getting the chunks back into SQL Server into a temp database, and from there into the live database. I am trying to design the system such that it is asynchronous. Stage1, stage 2/3 and stage 4 can all do their thing independently of each other. Each step of each stage is logged (dateTime flags written) so that any piece can just pick up where it left off. That means any Supervisor and any ProcessFile (chunk) can pick up from any point. In the meantime, the clsManager is watching the tblSupervisor for supervisor records ready to process - a thread. clsSupervisor is watching for ProcessTable records and dispatching them as required to the clsDBExp, clsDBImp or clsVM - a thread in each supervisor. clsDBExp is processing chunks, creating CSV files - a thread. clsVM is watching for clsProcessTable instances ready to go to the VM. It is also watching the output directory and sending files back to the SQL Server - a thread and two file watchers (and two events). clsDBImp is watching for clsProcessTable instances ready to import back into SQL Server - a thread. Most of these threads are writing flags (as they finish their specific process) and reading flags in clsProcessTable to decide what chunk has to be processed in which class. Oh, and I have three list controls on a form which the various classes write to to display the status for Stage1, Stage 2/3 and Stage 4. So pretty much a single supervisor is processing at a time, exporting files, sending them to the vm, importing them back in etc. It is possible for an order supervisor to cut in at the head of the line however. Not a simple task co-ordinating all of this stuff. I used to co-ordinate it all manually! ;) John W. Colby www.ColbyConsulting.com On 3/26/2011 8:08 PM, Shamil Salakhetdinov wrote: > John -- > > I'm falling asleep here now - would that be correct assumption that > you can generalize your global task like that - you have, say: > > - split input data into 50 Chunks; > - every chunk should be processed in 10 steps; > - every step's average processing time is 100 minutes. > > Then in total it is needed 50*10*100 = 50,000 minutes or ~833.(3) > hours to process all your input data sequentially, chunk after chunk > step after step, say, you don't have enough computer power to process > some chunks in parallel, and you can't process the whole not split > input data as it's too large for your system... > > But you have "50 small computers" and you can process every of 50 > input chunks in parallel - then you'll need just 10*100 = 1,000 > minutes (~16.67 > hours) to complete the job. > > I assume that all the chunks processing is independent - then you can: > > 1) Create Scheduler class instance; > 2) Schedule class creates 10 TaskQueue class instances - 1 queue for > every processing step type; > 3) Scheduler class defines splitting criteria to split input data into > 50 chunks; > 4) Schedule defines completion criteria - when all 50 chunks get > collected in 11th FinalQueue; > 5) Schedule seeds the first InputChunkSplitTaskQueue with 50 > SplitTasksDescriptors; > 4) From time to time - say every half a minute Scheduler class > instance creates 500 worker classes in 500 threads (500 worker classes > => 50 chunks * 10 steps.), which "hunt for the job" in their dedicate > Queues: if they find Job they work on it, if not - they "die"... > 5) When a worker class completes processing its job it queues its > "task results" into the next step Queue, if a worker class fails then > it puts its incomplete task descriptor back into its dedicated queue > and "dies", or that could be Scheduler's (or SchedulerAssitant's) task > job to find failed workers' "trap" and to resubmit their failed work > to the proper queue according to the custom workflow descriptors attached to every chunk... > > 500 worker classes is an overkill as only 50 worker classes can have > job every time but that seems to be an easy "brute force" and "lazy > parallel programming" approach - and it should work... > Or you can make worker class instances production processes smarter: > Scheduler class can start special thread for WorkerClassGenerator > instance, which will monitor all the 10 TaskQueues, and if it finds an > item in a Queque, it will pick it up, it will create corresponding > Worker class in parallel thread and it will pass WorkItem to the > Worker class fro processing... > > When that described above approach will work there then you can > easily(?) scale it splitting your (constant size) input data into 100 > chunks, and then if every chunk can be processes in half time - in > average (50 minutes) - then all job can be completed in 500 minutes = ~8.33(3) hours etc... > > Please correct me if I oversimplified your application business area... > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From jwcolby at colbyconsulting.com Sun Mar 27 22:01:44 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 27 Mar 2011 23:01:44 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> Message-ID: <4D8FFA18.9020902@colbyconsulting.com> Shamil, > Yes, I see: you tend to build a multi-threaded application/utility system > with centralized control - and I tend to build an anarchical free-market > model based application/utility system :) LOL, I guess. To be quite honest I didn't understand exactly what you were saying. Remember that my C# skills are still quite rudimentary. Accuzip Processing system basically has developed over time. I didn't sit down to develop an app from scratch to do this. I started in late 2004 to just get one single list into SQL server and serve up counts to the client. I did the whole thing manually, building queries to import the data and clean it up using only SQL Server Management Server. When I started I had never touched SQL Server. Then I was asked to research an application for doing CASS http://en.wikipedia.org/wiki/Coding_Accuracy_Support_System and NCOA http://en.wikipedia.org/wiki/National_Change_Of_Address on the addresses in the list. There was only one list in the beginning which I originally labeled "the database from hell". So I researched and discovered a vendor (Accuzip) that would allow us a fixed price license against which I could process as many records as I wanted. All other vendors (that I found) charge on a sliding scale Cents / record kind of thing, and we had a LOT of records - 65 million in that first list. So I started building a system of stored procedures which performed the export out to Accuzip and then import the Accuzipped records back in. It took roughly 25 stored procedures to perform this process. I had a paper list of instructions on how to run the stored procedures, parameters to feed in etc, plus manually copying the files out and back in. I would manually type EXEC commands into a query window reading from my paper instructions. It was *sloooow* and *error prone*. Next came a stored procedure that ran the stored procedures. Etc. Etc. It just morphed over time. Plus I got more lists. The client was so impressed that I managed to do all this that I got actual orders (export the data and send to the clients) rather than just counts, and the business grew. I started using Access ( the tool I knew ) to automate some of it but it was a very poor fit. Since every operation ran in a single thread the entire application would lock up when SQL Server went out for 30 minutes to do some operation. And of course my hardware was underpowered. In the summer of 2009 I decided I really needed to automate this. The business had grown and the client was suddenly 50% of my income and it was a huge struggle to process these lists manually, even using stored procedures of lists of stored procedures. TSQL as a programming environment is crude to say the least. So I went to the local community college in Sept. 2009 and took the first of two available C# courses. By Christmas I was launching in to my automation attempt. Understand my C# skills were rudimentary at best but the language was robust and I could see the power. I really just started building a method of moving the paper list of stored procedures into C#, IOW C# would execute the stored procedures and monitor their completion. I was not focused on a top down "build a system" but rather let's get my manual process automated ASAP. I was overworked, and wasn't able to handle the load. It turns out that C# is very capable in this regard and the program (such as it was) started working and working well. At the end of the second semester (May 2010) I hired Paul, a young college student who was graduating from the community college to come to work for me part time. I started teaching him what I know about general programming and together we worked on this program. And it continued to morph. We started using NLog for logging, Visual SVN for source control etc. At some point I decided to just move the Stored Procedures out of SQL Server and into dynamic code generated directly in C#. Each step had to keep the program running because I actually used the program to fill orders and do counts. And remember our C# skills are still rudimentary. Then we "discovered" threads and soon thereafter events. Last fall I decided to remodel the program into a manager / supervisor / worker model where the manager tells the supervisor to do something, the supervisor tells other workers to do something and basically just check results. In this model, each level uses threads to scan it's tasks (watch flags) and keep its workers busy. So the dbExport class has a thread which build a database, pulls records, builds chunks, exports to disk. That is it's job. When the file is on disk it is done and control for that chunk is passed to another class (the Virtual Machine). The dbExport class uses flags to store each step of the process. The Virtual Machine class picks up the files and pulls them over to the VMs directory, monitors that Accuzip processes them, tracks errors (if any) from Accuzip, and places the resulting files back on a staging directory (as I call it) for another class to take over. The dbImport class is tasked with creating a temp database, pulling each chunk file back in from staging and getting the data back into the original list database. So the manager class (a single instance) loads supervisor classes, one per list database that needs processing. The Manager class also loads the VM class (only one of these at the moment but there could be more, with more licenses from Accuzip). Each supervisor loads a dbExport and dbImport class to process the records in its list. dbExport creates Process records and stores them in SQL Server and creates a process class for each process record. Each process record represents a chunk of data from a specific list database. the process class doesn't really do anything it is just a place for the flags to be loaded / manipulated, and it knows how to persist itself to SQL Server. dbImport basically gets Process class instances which are finished with stage 2/3 (out of Accuzip) and starts importing them back in to SQL Server. My concept was a factory kind of application. There can be none or dozens of databases needing to be processed. SQL Server and my disk system can only efficiently handle one at a time (at any stage) and Accuzip is the biggest bottleneck so the VM class sits in the middle of an export and an import class. Every class has its own job, every class has at least one thread for its use. dbExport has a thread to use in exporting data to a table and another thread to bcp the data out while the next chunk is created in the export database. dbImport has a thread to BCP data back into chunk tables, and another thread to take the chunk tables and merge them back into live data. The VM has a thread to monitor the input directory and another thread to monitor the output directory, moving files into and out of the VM. Every step has to be logged back to SQL Server as it completes, but the clsSupervisor keeps the progress flags in memory as long as a specific list database is being processed, and the supervisor holds a list of process classes which hold all of the flags for the steps for a single chunk. It is a busy system. I did not originally use threads, and everything ran on the main program thread. And I was back to my user interface locking up as long running processes happened. Now with threading, each stage (export / vm / import) has its own list control which is used to display the status of the list database / chunks as they progress through the system. Everything is logged to flags, but also to NLog tables. Each class has threads to keep its own processes fed and running, and each class does its own piece of the whole to process a list table through Accuzip. BTW, there is another entire C# program which processes orders for the client. It is also threaded, and one of its steps is to create a supervisor record to tell the Accuzip Monitor to Accuzip the addresses being processed for the order. Orders are the Accuzip processing which I mentioned have to cut in at the head of the line in Accuzip Monitoring. As you have seen, I could be processing a 65 million record list database when an order comes in. 65 million records is going to take at least 50 hours to process, and the order has to go out ASAP so the Accuzip Monitor (the Manager class actually) has to watch for orders and interrupt the data supervisor to allow the Order supervisor to cut in. Once Accuzip starts a process (chunk) I don't interrupt it, but when that process (chunk) finishes Accuzip, only Order chunks will process until the order is finished. As I said, it is a busy system. And remember that our C# skills are still rudimentary. I started coding in C# in Sept 2009 and started migrating this process to c# about December 2009. By May 2010 when Paul started work I was actually running an Accuzip process and doing orders using C#. Paul is a youngster right out of school. So much of the stuff you talk about goes right over my head Shamil. John W. Colby www.ColbyConsulting.com On 3/27/2011 8:08 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I see: you tend to build a multi-threaded application/utility system > with centralized control - and I tend to build an anarchical free-market > model based application/utility system :) > > Thank you. > > -- > Shamil From jwcolby at colbyconsulting.com Sun Mar 27 22:20:36 2011 From: jwcolby at colbyconsulting.com (jwcolby) Date: Sun, 27 Mar 2011 23:20:36 -0400 Subject: [dba-VB] c# lock() In-Reply-To: References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> Message-ID: <4D8FFE84.5090609@colbyconsulting.com> OK, so I have explained the process. As you can see I am using threads pretty heavily. In general only one thread writes to any given flag field in a class and in general only one (other) thread reads from that flag. However threads are peeking at flags all the time "checking" whether the chunk (in particular) is ready to proceed to the next stage. This weekend I decided to try a dedicated flag class. My idea is simply that each flag represents a field in a table. Each flag represents a specific data element from a specific record. It is only written once (generally) but potentially read often. If I build a dedicated flag class it can be initialized with the table / field / pkid etc for its specific field / record and then when the thread writes the flag, the flag class can write its value out to the right field / record table. Again it is normally only ever written once. Most of my flags are a date / time that the process step occurred. So I am trying to wrap the whole flag concept into a class to allow me to process (read / write / test) flags in an identical way in all of my process classes. It was this flag class which I was discussing the end of last week. It will be accessed by at least two threads so it needs to be thread safe. John W. Colby www.ColbyConsulting.com On 3/27/2011 8:08 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I see: you tend to build a multi-threaded application/utility system > with centralized control - and I tend to build an anarchical free-market > model based application/utility system :) > > Thank you. > > -- > Shamil From shamil at smsconsulting.spb.ru Mon Mar 28 17:39:57 2011 From: shamil at smsconsulting.spb.ru (Shamil Salakhetdinov) Date: Tue, 29 Mar 2011 02:39:57 +0400 Subject: [dba-VB] c# lock() In-Reply-To: <4D8FFE84.5090609@colbyconsulting.com> References: <4D78CE4C.5000308@colbyconsulting.com><3693FCAC03E94BCA9C150F6E7AFE4A71@nant> <4D795044.6080808@colbyconsulting.com><3B9DE744CEA244829179A81CA818F92F@nant> <4D8CF3E3.3050608@colbyconsulting.com><3F0DA84A905C4947819126F43CFC39DB@nant><4D8E604B.5040001@colbyconsulting.com> <00D0A03B172743B6868A8BE736F7E560@nant><0CD7EF3475CB43C69334942027697B63@nant> <4D8EB924.5080708@colbyconsulting.com> <4D8FFE84.5090609@colbyconsulting.com> Message-ID: <5FD8C68EEEF142DA8632253753C51323@nant> Hi John -- <<< It was this flag class which I was discussing the end of last week. It will be accessed by at least two threads so it needs to be thread safe. >>> Yes, I see - I do wish it worked efficiently there real soon... Thank you. -- Shamil -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of jwcolby Sent: 28 ????? 2011 ?. 7:21 To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] c# lock() OK, so I have explained the process. As you can see I am using threads pretty heavily. In general only one thread writes to any given flag field in a class and in general only one (other) thread reads from that flag. However threads are peeking at flags all the time "checking" whether the chunk (in particular) is ready to proceed to the next stage. This weekend I decided to try a dedicated flag class. My idea is simply that each flag represents a field in a table. Each flag represents a specific data element from a specific record. It is only written once (generally) but potentially read often. If I build a dedicated flag class it can be initialized with the table / field / pkid etc for its specific field / record and then when the thread writes the flag, the flag class can write its value out to the right field / record table. Again it is normally only ever written once. Most of my flags are a date / time that the process step occurred. So I am trying to wrap the whole flag concept into a class to allow me to process (read / write / test) flags in an identical way in all of my process classes. It was this flag class which I was discussing the end of last week. It will be accessed by at least two threads so it needs to be thread safe. John W. Colby www.ColbyConsulting.com On 3/27/2011 8:08 PM, Shamil Salakhetdinov wrote: > Hi John -- > > Yes, I see: you tend to build a multi-threaded application/utility > system with centralized control - and I tend to build an anarchical > free-market model based application/utility system :) > > Thank you. > > -- > Shamil _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com From DWUTKA at Marlow.com Thu Mar 31 10:34:13 2011 From: DWUTKA at Marlow.com (Drew Wutka) Date: Thu, 31 Mar 2011 10:34:13 -0500 Subject: [dba-VB] Most useful droid apps In-Reply-To: References: Message-ID: AndroXplorer Very handy for transferring files around between Droid, Network and SD Card. Drew -----Original Message----- From: dba-vb-bounces at databaseadvisors.com [mailto:dba-vb-bounces at databaseadvisors.com] On Behalf Of David McAfee Sent: Saturday, March 19, 2011 3:54 PM To: Discussion concerning Visual Basic and related programming issues. Subject: Re: [dba-VB] Most useful droid apps Advanced task killer Weather channel Lookout antivirus AK Notepad Bank of America Acar auto maintenance Barcode scanner Bubble level Bump Droid light Google goggles Google translate IMDB Key ring Print share Trapster Sent from my Droid phone. On Mar 19, 2011 12:52 PM, "jwcolby" wrote: _______________________________________________ dba-VB mailing list dba-VB at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/dba-vb http://www.databaseadvisors.com The information contained in this transmission is intended only for the person or entity to which it is addressed and may contain II-VI Proprietary and/or II-VI Business Sensitive material. If you are not the intended recipient, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. You are notified that any review, retransmission, copying, disclosure, dissemination, or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited.