[dba-SQLServer] Microsoft is moving ahead

jwcolby jwcolby at colbyconsulting.com
Sun Oct 16 20:40:05 CDT 2011


6.  What kind of data is being modeled / stored / analyzed.

John W. Colby
Colby Consulting

Reality is what refuses to go away
when you do not believe in it

On 10/16/2011 4:57 PM, Jim Lawrence wrote:
> Hi Hans:
>
> Maybe you can give the facts and figures on a NOSQL implementation. These
> figures would be:
>
> 1. The size and resources of the equipment/hardware you have experience
> with.
> 2. How much data is being handled with this equipment?
> 3. Some basic guesstimates of the costs of this implementation.
> 4. How successful is NoSQL in retrieving complex data requests.
> 5. Anything thing else that you would think is relevant.
>
> Once the facts and figures could be put together, it would put an end to the
> controversy, which at the moment is just hearsay.
>
> Jim
>
> -----Original Message-----
> From: dba-sqlserver-bounces at databaseadvisors.com
> [mailto:dba-sqlserver-bounces at databaseadvisors.com] On Behalf Of
> Hans-Christian Andersen
> Sent: Saturday, October 15, 2011 11:35 PM
> To: Discussion concerning MS SQL Server
> Subject: Re: [dba-SQLServer] Microsoft is moving ahead
>
>
> Hi John,
>
> I don't think any of us had the intention to offend you. Obviously you know
> what your business requirements are better than anyone else.
>
> - Hans
>
>
> On 2011-10-15, at 6:30 AM, jwcolby wrote:
>
>> Hans,
>>
>>>> To be more accurate, NoSQL is intended to be a solution for companies
> that are expecting rapid growth and cannot rely on vertical scaling alone in
> order to keep up with demands on resources.
>>
>> I don't see this anywhere.  Point me to anywhere that any company is even
> thinking about NoSQL to run their *business* side of the house.  Show me
> *anything* where *anyone* is developing book keeping or banking or
> manufacturing kind of databases using NoSQL.
>>
>> To be more accurate, NOSQL is intended to be a solution for companies
> expecting rapid growth in *document storage*, and needing to *search
> documents*.
>>
>>>> It's not nonsense, just because it doesn't apply to you. :)
>>
>> I did not say "it" (NOSQL) was nonsense, I have been saying that it it
> nonsense to keep trying to fit that square peg in this round hole.  It is
> nonsense to keep telling me I need it when (as you are saying) it doesn't
> apply to me!
>>
>> I read an article by one of the founders of (I believe) Hadoop.  What he
> said was that NOSQL was *NOT* a replacement for SQL based languages, but a
> solution for places where SQL databases don't fit.  The things I do demand
> relational data.  Relationships are the core of my business.  My data is
> large, but they are not large individual chunks (paragraphs or pages or
> documents) but lots of records with lots of attributes.
>>
>> I have 600 million records in about 30 table pairs.  The tables are pairs,
> each table related to one other with a pk/fk.  One table contains name /
> address / hash fields and a PK.  The other table has attributes about the
> people in that first table.  I have (in 15 tables) 300 million records with
> first name, last name, addr1, city, state, zip, plus a handful of other
> fields discussing the validity of the address itself.  I have to index on
> and pull addresses based on specific attributes of those addresses.  I have
> indexes on and pull data about those people records based on attributes
> (fields) in the attribute table.
>>
>> As an example I have to pull those 300 million addresses out every month
> and run them through a third party program to track people moving.  That
> software requires the name / address fields and hands me back those same
> fields plus a bunch more that discuss the validity of those addresses (is
> the address still valid?  Is the address complete?) as well as move
> information.  That third party program requires the data in CSV and uses
> Foxpro to process it.
>>
>> How in the world does this sound like Hadoop?
>>
>> I think my friend Jim just has some misconceptions about what my data is.
> Given how much I have discussed my "database from hell" with its 600
> attribute fields I am a little puzzled how he could not understand what I
> do.
>>
>> Or perhaps he (or I) misunderstands what NoSQL is.  From what I am
> reading, NoSQL is not about handling relational data or hundreds of tiny
> attributes (fields) of an object and selecting records based on those
> attributes.  NoSQL (AFAIU) is about storing documents and allowing you to
> search those documents.  I don't have a single field in all of my data that
> stores more than about 80 characters.  I have tables, related to other
> tables, each of which may have literally hundreds of fields, each field
> being anywhere from one character (yes / no) to 60 characters (email
> address). In fact the email address is the single biggest field in all of my
> data.  I have to select small (a few million) record sets based on "where"
> clauses examining those fields.  I have to join the information in these 15
> table pairs to select records based on commonality.
>>
>> How does this sound like NoSQL?
>>
>> Every time my friend Jim comes at me with "you need NoSQL" I spend more
> time trying to see what it is about NOSQL that fits my situation.  I am not
> blithely ignoring him.  I have spent hours now reading stuff about this
> technology, and every time I keep reading stuff by the very people who
> design NoSQL saying that *it is not a replacement for SQL*.  These people
> say that NOSQL does not do SQL kind of stuff easily.  These people say NOSQL
> is about spreading the load of searching millions of *documents* across an
> entire server farm.  These people are saying that it tough (requires entire
> new languages, technology and knowledge base) to get the data split out
> across that server farm and to reassemble the search results *but* that the
> results are worth it *when* you are dealing with billions of *documents*.
>>
>> I am a one man show.  I don't own a server farm.  I don't have billions of
> documents and I am not going to acquire billions of documents.
>>
>> I am just tired of being told how my situation is going to be helped by a
> technology specifically and intentionally designed to handle the storage and
> search of *documents*, when I don't have a single document in my entire
> database.
>>
>> *THIS IS GETTING OLD!!!*
>>
>> I am thrilled that NoSQL exists and that it helps those that it helps.
> What I am *not* seeing is a single case study where they are taking an SAP
> process and doing it on NoSQL.  Flattening 10 thousand tables in a massive
> SQL based data processing system and making it run on NoSQL.  What I am
> *not* seeing is anyone claiming that in 5 (or even 30) years the SQL
> language will cease to exist because of NoSQL.
>>
>> And what I am not thrilled about is a constant "you need NOSQL" when there
> is never any explanation about how this very cool but not applicable
> technology applies to me.
>>
>> IOW, LMTFA!!!
>>
>> John W. Colby
>> Colby Consulting
>>
>> Reality is what refuses to go away
>> when you do not believe in it
>>
>> On 10/15/2011 2:47 AM, Hans-Christian Andersen wrote:
>>>
>>>
>>> To be more accurate, NoSQL is intended to be a solution for companies
> that are expecting rapid growth and cannot rely on vertical scaling alone in
> order to keep up with demands on resources.
>>>
>>> Trust me, John, if you are in a situation like this, no amount of memory,
> cpu or compression can save you. It's not nonsense, just because it doesn't
> apply to you. :)
>>>
>>> - Hans
>>>
>>>
>>>
>>> On 2011-10-14, at 3:37 PM, jwcolby wrote:
>>>
>>>> LOL.  And maybe you will eventually discover that NoSQL is targeted at
> people with a a million blade servers / million dollars in a data center and
> will quit haranguing me?  ;)
>>>>
>>>> I see NO NOSQL in my future.
>>>>
>>>> I have already solved my issues the same way that these guys did.
> Jillions of cores and 64 gigabytes of RAM, and compression.  My hour long
> processes are under a minute or two.
>>>>
>>>> It would be interesting to actually show you what I do Jim.  You would
> instantly quit this nonsense.
>>>>
>>>> John W. Colby
>> _______________________________________________
>> dba-SQLServer mailing list
>> dba-SQLServer at databaseadvisors.com
>> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
>> http://www.databaseadvisors.com
>>
>
>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>
> _______________________________________________
> dba-SQLServer mailing list
> dba-SQLServer at databaseadvisors.com
> http://databaseadvisors.com/mailman/listinfo/dba-sqlserver
> http://www.databaseadvisors.com
>
>



More information about the dba-SQLServer mailing list