Derek Comingore Blog

Derek Comingore's Microsoft Business Intelligence Blog

OLAP's Ante to Play: OLTP

I'm starting to read various chapters from 'Programming Microsoft SQL Server 2008' by Leonard Lobel and his co-authors. I found his Chapter 14, 'Data Warehousing', contains a very good section on the solutions and technologies that were 'main stream' prior to the deployment of data warehousing and business intelligence solutions.

This brings up an excellent (debatable) point. In my opinion, anyone wishing to enter the field of DW/BI (regardless of software vendor) should have good solid prior experience in traditional OLTP-based applications and systems. This is my opinion because (as 'Programming Microsoft SQL Server 2008' mentions), OLTP systems have been around long before OLAP and data warehousing. Furthermore, data warehousing requires us to extract data regularly from OLTP systems. Finally, because data warehousing is an evolution that was created to address the many constraints found in traditional OLTP systems for decision support requirements.

What do you think? You could certainly argue that so long as one has a basic understanding of the systems they are collecting data from they should be fine regardles of this prior experience. Let's take this discussion one step further, what about these various DW/BI Training companies who now promote learning DW/BI without considering the individuals background? I am sure the training companies don't care (that much) but are their customers going to be able to obtain real-world DW/BI jobs without having any OLTP experience prior (and probably no DW/BI experience as well)?

YEAH RIGHT! (I am now on a separate 'soap box'). If you truly want to assist and train people in DW/BI you should enforce a course requirement that people have X number of years experience in OLTP. Otherwise, you are teaching people 'Algebra' without them having an understanding of 'Math'!

For those who read my blog, chime in!


Legacy Comments

Joe Harris
re: OLAP's Ante to Play: OLTP
Right on! BUT before we let people touch our OLTP systems they should really have a good few years of mainframe under their belts. How can they appreciate the value of a transaction if they don't understand the all of the various IBM filesystems and the difference between big-endian and little-endian.

Now seriously... this is a natural change. DW/BI will really move on only when the next generation is totally unencumbered by previous practices. Plenty of OLTP people refuse to believe that Netezza (or DatAllegro, etc.) can really run an aggregate over a billion rows in 10 seconds. To them that's not possible.

When BW/BI practitioners take this kind of ability for granted will we begin to see what it can really do.

re: OLAP's Ante to Play: OLTP
I agree with the basic commment that it's good to have an understanding of the tools and methods used in the source systems when implementing a datawarehouse, but in over 15 years of implementing warehouses on Teradata, WhiteCross, Netezza etc I have seen the recurring issue of having to get developers to 'unlearn' some of their treasured approaches when implementing on the newer MPP technologies. Blindly trying to reuse traditional ETL techniques etc on the newer appliances is most definitely NOT the way to get the best out of them.