[dba-SQLServer] So what is super hot in the tech world?

Arthur Fuller fuller.artful at gmail.com
Wed Dec 28 17:32:34 CST 2011

>From what I've read and what I've installed and cruised, the NoSQL
solutions seem ideal for retrieval of non-hierarchical DB queries (e.g. web
pages requiring lookup of either static pages or templates to be filled
in), and totally wrong for transactional DBs. I am not making a judgement
here, for or against; just saying that on my read these technologies serve
different purposes.

And while we're at it, let's also discuss the concept of partitioning: in
most transactional cases, what occurred a year ago is History (i.e. not
subject to updates), and what occurred more recently is potentially subject
to updates. This leads to the concept of vertical partitioning, in which
all data older than a year (or month, or whatever unit is deemed
appropriate) is stored in a partition, while the Live data is stored in the
Here and Now. This in turn requires at least a few queries that inspect and
select from both partitions; not in itself problematic, but it ought to be
recognized as a distinctly separate task than querying the live data,
however that may be defined.

I do love the concept of a SQL implementation entirely stored in RAM, and
current technologies make most of this possible. For example, let us
suppose, given the partitioning described above, that the current year's
data resides entirely in RAM, and the previous years' data resides on disc.
Then all the transactional events could occur at RAM-speed, with periodic
writes to disc in user-defined periodicity. That in turn implies that the
current stuff exists in one database and the historical stuff in another,
and that scheduled jobs move the data out of Current and into History.

The gains in such an approach increase in reverse-proportion to the
timeline defined as Current, and this, IMO, is significant: suppose that
the time-frontier shrinks to the past 30 days. That would easily enable
most transactional DBs to store everything of interest in RAM, and on
occasion reach deeper into the past (imagine a banking app that lets you
print a statement dating back 30/60/90/120/all: the most frequent queries
could be done from RAM, and the progressively more distant queries require
an increasingly deep reach into disc storage).

Architecturally, this approach is not a problem. However, I don't think
that the NoSQL solutions are designed to solve this. I could be wrong on my
read, but so far I think that NoSQL is designed for different problems. The
up-side is, I also don't think that any vendor has approached this problem
in precisely this way. Vertical partitioning goes 80% of what this demands,
but not 100%, and therein lies an opportunity for some entrepreneur or
major vendor to step up and provide it.

I'll even supply the requirements:

a) all the "current" transactional data resides in RAM (add as much RAM as
you need)
b) scheduled processes move older transactions from the RAM DB into the
History DB
c) queries designed to pull data from both Current and History should be
easy to create
d) History DB is itself partitioned, say by year, so that any given query
wanting the Current and History (previous two years) need no rewriting as
time goes on.

First vendor to supply this in a wizard-like install/define UI owes me 1%
of the revenue -- just a suggestion LOL.


On Wed, Dec 28, 2011 at 5:45 PM, jwcolby <jwcolby at colbyconsulting.com>wrote:

> LOL.
> I like it.  NoAccess...
> John W. Colby
> Colby Consulting
> Reality is what refuses to go away
> when you do not believe in it
> On 12/28/2011 4:55 PM, Stuart McLachlan wrote:
>> Yada, yada, yada.
>> Still evangelising I see :-)
>> 100 million records is not per se a problem for a SQL structured DB.
>> No matter how many times people claim that it is, NoSQL is NOT a
>> replacement for SQL,
>> NoSQL is just a group of solutions to manage types of data that SQL is
>> not ideal for.
>> You might just as well say:
>> With images exceeding several megabytes in size, NoAccess is much better
>> than Access for
>> manipulating graphic data.    Take for instance the NoAcess super
>> application called
>> PhotoShop.
> ______________________________**_________________
> dba-SQLServer mailing list
> dba-SQLServer@**databaseadvisors.com <dba-SQLServer at databaseadvisors.com>
> http://databaseadvisors.com/**mailman/listinfo/dba-sqlserver<http://databaseadvisors.com/mailman/listinfo/dba-sqlserver>
> http://www.databaseadvisors.**com <http://www.databaseadvisors.com>

Cell: 647.710.1314

Thirty spokes converge on a hub
but it's the emptiness
that makes a wheel work
   -- from the Daodejing

More information about the dba-SQLServer mailing list