Mark Breen
marklbreen at gmail.com
Sun Jun 13 06:17:02 CDT 2010
Hello John, Fascinated is not the word, jealous beyond reasonable doubt is more like it. It sounds challanging but it also like incredible fun. I imagine you are enjoying it. Mark On 12 June 2010 14:26, jwcolby <jwcolby at colbyconsulting.com> wrote: > I am doing ETL but I am doing very specific, very repetitive ETL. I am not > trying to migrate the > world in and out. > > What I am writing is an application to handle Stan's entire world. I do > many different jobs for him > and they all revolve around list processing. Internal to my database the > lists all have a common > component (name / address) that is the level that is exported out / in. > > What I am doing is automating what I used to do manually, using C# and SQL > Server stored procedures. > I had over time developed the stored procedures and ran them manually. > Now I run them from my > application. > > I am sure you would be fascinated to see what I do here. > > John W. Colby > www.ColbyConsulting.com > > > Mark Breen wrote: > > Hello John, > > > > I agree with Michael here, only one persons opinion matters here and > that's > > yours as only you really now the true effect. > > > > Is it not possible to do a method A and Method B comparison? > > > > I am presuming that each time you face the decision the results are > > significantly different that it is difficult to identify the true answer. > > If that is the case, the again, only you will really know. > > > > Can I ask, have you read up much on ETL? The works you are doing > nowadays > > is AFAICS, ETL and I believe that there are plenty of ETL tools. In fact > it > > is a subject area in itself and my instinct tells me that traditional > > approaches to data management is different when you are doing ETL. > > > > I only saw it for a few months when I worked for a while in an Oracle > Shop, > > but they had a whole ETL department with their own tools. Those tools > did a > > lot of the heavy lifting that you are coding. > > > > HTH somewhat > > > > Mark > > > > > > On 10 June 2010 01:57, Michael Maddison <michael at ddisolutions.com.au> > wrote: > > > >> Hi John, > >> > >> I think you will only get opinions, if that. > >> You are going to have to test each scenario IMO. > >> Another option for testing is to enable/disable the covering indexes. > >> You probably need to update statistic after enabling if the data has > >> changed. > >> If the data hasn't changed then this 'should' be your best option. > >> > >> Cheers > >> > >> Michael M > >> > >> > >> I have processes which process huge tables (65 million records are > >> common) and update specific > >> fields with the results of user defined functions. For example I have a > >> function which builds a > >> text "code" using four passed in values. These four values come from > >> four fields of the record and > >> the resulting code is placed back in another field of the same record. > >> > >> ATM I build a "cover index" just before updating this field, perform the > >> update, then delete the > >> index when done. > >> > >> I am wondering whether you guys believe that the time to build the cover > >> index will be recovered by > >> less time taken to perform this update. IOW does it take sufficiently > >> less time to get data from an > >> index vs a table scan that it is worth building the index? > >> > >> -- > >> John W. Colby > >> www.ColbyConsulting.com > >> _______________________________________________ > >> dba-SQLServer mailing list > >> dba-SQLServer at databaseadvisors.com > >> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > >> http://www.databaseadvisors.com > >> > >> > >> No virus found in this incoming message. > >> Checked by AVG - www.avg.com > >> Version: 9.0.829 / Virus Database: 271.1.1/2926 - Release Date: 06/10/10 > >> 04:35:00 > >> > >> _______________________________________________ > >> dba-SQLServer mailing list > >> dba-SQLServer at databaseadvisors.com > >> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > >> http://www.databaseadvisors.com > >> > >> > > _______________________________________________ > > dba-SQLServer mailing list > > dba-SQLServer at databaseadvisors.com > > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > > http://www.databaseadvisors.com > > > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > >