jwcolby
jwcolby at colbyconsulting.com
Sun Oct 16 20:40:05 CDT 2011
6. What kind of data is being modeled / stored / analyzed. John W. Colby Colby Consulting Reality is what refuses to go away when you do not believe in it On 10/16/2011 4:57 PM, Jim Lawrence wrote: > Hi Hans: > > Maybe you can give the facts and figures on a NOSQL implementation. These > figures would be: > > 1. The size and resources of the equipment/hardware you have experience > with. > 2. How much data is being handled with this equipment? > 3. Some basic guesstimates of the costs of this implementation. > 4. How successful is NoSQL in retrieving complex data requests. > 5. Anything thing else that you would think is relevant. > > Once the facts and figures could be put together, it would put an end to the > controversy, which at the moment is just hearsay. > > Jim > > -----Original Message----- > From: dba-sqlserver-bounces at databaseadvisors.com > [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of > Hans-Christian Andersen > Sent: Saturday, October 15, 2011 11:35 PM > To: Discussion concerning MS SQL Server > Subject: Re: [dba-SQLServer] Microsoft is moving ahead > > > Hi John, > > I don't think any of us had the intention to offend you. Obviously you know > what your business requirements are better than anyone else. > > - Hans > > > On 2011-10-15, at 6:30 AM, jwcolby wrote: > >> Hans, >> >>>> To be more accurate, NoSQL is intended to be a solution for companies > that are expecting rapid growth and cannot rely on vertical scaling alone in > order to keep up with demands on resources. >> >> I don't see this anywhere. Point me to anywhere that any company is even > thinking about NoSQL to run their *business* side of the house. Show me > *anything* where *anyone* is developing book keeping or banking or > manufacturing kind of databases using NoSQL. >> >> To be more accurate, NOSQL is intended to be a solution for companies > expecting rapid growth in *document storage*, and needing to *search > documents*. >> >>>> It's not nonsense, just because it doesn't apply to you. :) >> >> I did not say "it" (NOSQL) was nonsense, I have been saying that it it > nonsense to keep trying to fit that square peg in this round hole. It is > nonsense to keep telling me I need it when (as you are saying) it doesn't > apply to me! >> >> I read an article by one of the founders of (I believe) Hadoop. What he > said was that NOSQL was *NOT* a replacement for SQL based languages, but a > solution for places where SQL databases don't fit. The things I do demand > relational data. Relationships are the core of my business. My data is > large, but they are not large individual chunks (paragraphs or pages or > documents) but lots of records with lots of attributes. >> >> I have 600 million records in about 30 table pairs. The tables are pairs, > each table related to one other with a pk/fk. One table contains name / > address / hash fields and a PK. The other table has attributes about the > people in that first table. I have (in 15 tables) 300 million records with > first name, last name, addr1, city, state, zip, plus a handful of other > fields discussing the validity of the address itself. I have to index on > and pull addresses based on specific attributes of those addresses. I have > indexes on and pull data about those people records based on attributes > (fields) in the attribute table. >> >> As an example I have to pull those 300 million addresses out every month > and run them through a third party program to track people moving. That > software requires the name / address fields and hands me back those same > fields plus a bunch more that discuss the validity of those addresses (is > the address still valid? Is the address complete?) as well as move > information. That third party program requires the data in CSV and uses > Foxpro to process it. >> >> How in the world does this sound like Hadoop? >> >> I think my friend Jim just has some misconceptions about what my data is. > Given how much I have discussed my "database from hell" with its 600 > attribute fields I am a little puzzled how he could not understand what I > do. >> >> Or perhaps he (or I) misunderstands what NoSQL is. From what I am > reading, NoSQL is not about handling relational data or hundreds of tiny > attributes (fields) of an object and selecting records based on those > attributes. NoSQL (AFAIU) is about storing documents and allowing you to > search those documents. I don't have a single field in all of my data that > stores more than about 80 characters. I have tables, related to other > tables, each of which may have literally hundreds of fields, each field > being anywhere from one character (yes / no) to 60 characters (email > address). In fact the email address is the single biggest field in all of my > data. I have to select small (a few million) record sets based on "where" > clauses examining those fields. I have to join the information in these 15 > table pairs to select records based on commonality. >> >> How does this sound like NoSQL? >> >> Every time my friend Jim comes at me with "you need NoSQL" I spend more > time trying to see what it is about NOSQL that fits my situation. I am not > blithely ignoring him. I have spent hours now reading stuff about this > technology, and every time I keep reading stuff by the very people who > design NoSQL saying that *it is not a replacement for SQL*. These people > say that NOSQL does not do SQL kind of stuff easily. These people say NOSQL > is about spreading the load of searching millions of *documents* across an > entire server farm. These people are saying that it tough (requires entire > new languages, technology and knowledge base) to get the data split out > across that server farm and to reassemble the search results *but* that the > results are worth it *when* you are dealing with billions of *documents*. >> >> I am a one man show. I don't own a server farm. I don't have billions of > documents and I am not going to acquire billions of documents. >> >> I am just tired of being told how my situation is going to be helped by a > technology specifically and intentionally designed to handle the storage and > search of *documents*, when I don't have a single document in my entire > database. >> >> *THIS IS GETTING OLD!!!* >> >> I am thrilled that NoSQL exists and that it helps those that it helps. > What I am *not* seeing is a single case study where they are taking an SAP > process and doing it on NoSQL. Flattening 10 thousand tables in a massive > SQL based data processing system and making it run on NoSQL. What I am > *not* seeing is anyone claiming that in 5 (or even 30) years the SQL > language will cease to exist because of NoSQL. >> >> And what I am not thrilled about is a constant "you need NOSQL" when there > is never any explanation about how this very cool but not applicable > technology applies to me. >> >> IOW, LMTFA!!! >> >> John W. Colby >> Colby Consulting >> >> Reality is what refuses to go away >> when you do not believe in it >> >> On 10/15/2011 2:47 AM, Hans-Christian Andersen wrote: >>> >>> >>> To be more accurate, NoSQL is intended to be a solution for companies > that are expecting rapid growth and cannot rely on vertical scaling alone in > order to keep up with demands on resources. >>> >>> Trust me, John, if you are in a situation like this, no amount of memory, > cpu or compression can save you. It's not nonsense, just because it doesn't > apply to you. :) >>> >>> - Hans >>> >>> >>> >>> On 2011-10-14, at 3:37 PM, jwcolby wrote: >>> >>>> LOL. And maybe you will eventually discover that NoSQL is targeted at > people with a a million blade servers / million dollars in a data center and > will quit haranguing me? ;) >>>> >>>> I see NO NOSQL in my future. >>>> >>>> I have already solved my issues the same way that these guys did. > Jillions of cores and 64 gigabytes of RAM, and compression. My hour long > processes are under a minute or two. >>>> >>>> It would be interesting to actually show you what I do Jim. You would > instantly quit this nonsense. >>>> >>>> John W. Colby >> _______________________________________________ >> dba-SQLServer mailing list >> dba-SQLServer at databaseadvisors.com >> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver >> http://www.databaseadvisors.com >> > > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > > _______________________________________________ > dba-SQLServer mailing list > dba-SQLServer at databaseadvisors.com > http://databaseadvisors.com/mailman/listinfo/dba-sqlserver > http://www.databaseadvisors.com > >