It’s a very interesting view into what may be coming. My concerns are really echos of what Adrian already mentioned. I wonder if a mapped-metadata layer can do the kind of cleaning and conforming that is needed for an EDW. It’s certainly a logical possibility… but one that doesn’t (as far as I know) exist in reality as of yet. I guess I need to roll-up my sleeves and learn OBIEE backwards and forwards… especially since it may make my skills irrelevant :-)

Knowing what direction to go down when you have Oracle DW, Hyperion Sys 9 and Siebel Analytics (OBIEE) all present in your organization is a tricky business, and with 11g the move is clearly towards db hosting for all data … so where is Hyperion going? Hosting out of Oracle DB’s in 11g, with MV-like refresh capabilities?

Some statements on medium and long-term strategy would be much appreciated.

It’s a typical scenario. The Business approaches IT with the need for a reporting solution. IT say ‘great, lets build a Data Warehouse’. The Business says ‘fine, how long?’ and then faces fall at the answer. ‘OK’, says IT, ‘how about we use the fine features of this new technology to deliver you a rapid application as a short term deliverable and work on the Data Warehouse solution afterwards?’. ‘Excellent’ says IT.

A month or so of constructive co-operation between IT and the Business ensues, resulting in the delivery of a fast, seemingly complete, solution.

The Business is so happy that they bring forward a new requirement. ‘What about the Data Warehouse?’ asks IT. ‘Why do we need that?’ says the Business. ‘The data we need is there, it’s fast and everyone can access it’. ‘Oh but what about slowly changing dimensions, advanced aggregations and future manageability?’ pleads IT. Business Eyes glaze over.

A bit simplistic for sure but reflective of many situations I’ve been in. Now, don’t get me wrong, the Business pays so the Business rules but it does seem to me that IT have a stake in it’s success as well. A more mature approach is required from both parties. IT need to accept that not everything will end up in the DW (some requirements are just too short term) and the Business need to understand that the long term usage of a solution rely in cost effective, consistent warehousing solutions. Flexibility is required from IT, whilst Fidelity is required from the Business.

Technology will help but it’s going to prove difficult for the DW to keep up with the rapid focus changes of the Business if they are designing their own solutions all the time. A co-ordinated approach is still required. I guess then, in answer to your post, I think that, yes, you can start off against source data but that, sooner or later, the need will arise for the Data Warehouse and that that need should be communicated to all at the start of development and the benefits to be gained reinforced throughout.

@Adrian – another good point, and this of course is where the reality of corporate policies (only allowing access to data away from the source system and aggregated) and the difficulties of data aggregation (providing a single consistent data source when data is integrated as needed, and where you are constantly migrating source data into the warehouse) comes in. I guess the “single version of the truth” issue can be addressed by providing all data access through the OBIEE semantic model – what the model points to over time may change, but at any one point, the Sales measure in the business model will provide the same results to all users of the system, whilst with performance, pointing directly towards the source systems is an interim step whilst the warehouse is created and populated, and users may be willing to put up with slow access to a particular subject area if it means they can get data much earlier than normal. Good points though, ones I’ll need to address in the presentation.

Another issue is performance, we often use datawarehouses to integrate data and appy the necessary structure (bitmaps, star schema, aggragates) to ensure that end user reports run quickly. We are working with source dataabases that contain billions of records, which cannot be analysed by Analytics on the fly, and so we usually recommend going down the SAS route for detailed analysis, and aggregates into the warehouse for summary reporting.

Thirdly, we always need to ensure a single version of the truth. For example, when sales managers get their Sales forecast reports for the weekly meeting with the team, everyone needs to have the same numbers. This is best achieved by controlling the published data in the warehouse.