At Rittman Mead we tend to look a Business Intelligence through a series of levels of maturity, these typically go from being able to produce operational reports from a single system with no control and governance, to building an analytic system where data is integrated, dimensions are conformed, metrics are clearly defined, control and governance is applied to the data and power users can do sophisticated analysis of the data and make use of features such as drilling down, drilling across, aggregation and filtering. Reports built in this system could then be arranged on dashboards to provide user end user access to the information.

Users get value by being able to compare data from different business processes, being able to aggregate data and being able to look at data over time. There can be a number of steps between the basic operational reporting and this level of analytic reporting, but this level basically describes the core features of OBIEE’s Answers and Dashboards products.

We find most organisation are at a more basic level of maturity. In order to get traction from Business Intelligence projects, we have to engage at that organisation’s existing level of reporting maturity, there is no point proposing some wonderful high brow analytics system if the organisation is still doing most of its reporting using spreadsheets, I don’t believe you will get traction. You have to grow the maturity of the organisation with the development of the Business Intelligence landscape.

So this leads me to my original question of where the what if analysis has gone, how should organisations move to this? From a product point of view there are some obvious candidates: Oracle Data Mining (ODM), Essbase, Oracle OLAP and Crystal Ball. The questions are: what is the use case for implementing these? Where can an organisation get business value using this type of analysis? How could you construct a ROI case for these?

A meeting with a colleague identified the following scenario which I quite liked and thought made a clear and strong case. The starting point is at a level of maturity where an organisation had a good analytic system. The use case is the organisation wants to change their existing business model to either target another customer segment, or add another value proposition or product.

The most basic way of doing this would be to create a simple spreadsheet that read a bit like a Profit & Loss statement and had a figure for expected revenue and some figures for costs, these estimates would be based on the experience of the spreadsheet author and would at most contain worst case, expected and best case values. They could potentially turn out to be accurate, however they are not really going to carry any weight, especially if significant investment is going to be required to implement this change. What is required is a more qualified approach to modelling this change.

Using the products mentioned above one approach to solving this problem could be to start with ODM. The organisation could use some of the algorithms in ODM to start to predict measures like revenue, based on customers likelihood to churn and sensitivity to price. These calculated measures would be based on historic data the organisation had and would give a much more reliable view or model of the revenue for the new or revised business model.

However so far we have only got one version of the model and it may not feature all the measures we need, for example it may be possible to calculate some revenue figures, however cost may need to be manually input.The next step would be to use Essbase. If the model comprising of the associated dimensions and facts was then loaded to Essbase, along with the ODM calculated data the organisation could then publish this to several users and allow them to input data for measures that couldn’t be calculated and to create different scenarios for the model, based on different interpretations of how successful the model is, for example worst case, best case and most likely. In additional profit would be able to be determined for these scenarios as all the relevant measures would be present – either calculated by the analytics tool, ODM or user input.

Coming back to the question of investment, whether it was either internal or external, the organisation would now have a much more compelling case to persuade a potential investor. However I think it is possible to argue that there is still a missing element from this which is risk. The investor would be well within their rights to ask what risk was associated with their investment. This is where the final piece of the jigsaw comes in, Crystal Ball.

Crystal Ball allows users to run a series of (Monte Carlo) simulations based on high and low water mark parameters that are run against the model, the end result is that a level of confidence can be ascertained about the likelihood of an event. Crystal Ball works as an Excel plug-in, and as such can also work in conjunction with Essbase. The Essbase data can be loaded into Excel through its plug-in, then have the statistical analysis from Crystal Ball performed on it, then potentially read back into Essbase as another scenario.

The end result would be a business model both in real terms and loaded into Essbase that an organisation could go to an investor with understanding of revenue, profit and level of confidence (risk) and be in a much stronger position to secure the finance they need. As a further benefit the organisation may be able to negotiate a significantly lower interest rate on any investment as they can demonstrate low risk.

I think this story sets out a clear picture of how these technologies can be used together to support a genuine business scenario and it is certainly something I will be investigating internally with some business initiatives we have coming up over the remainder of this year.

We are now recruiting an additional business development manager to join our friendly and energetic sales team who has experience in the Oracle BI space.

Initially you will be required to sell Oracle BI Delivery Services, Products, Training, and Managed Services to clients throughout the UK and overseas. More specifically, you will be a proven new business hunter with experience of selling services and software in the Business Intelligence, Performance Management, Data Warehousing, Analytics and Data Mining arena.

The ideal candidate will have a minimum of 5-7 years proven sales experience with a consultative and solution oriented sales approach. In addition we would expect you to have proven experience in account penetration and development strategies, initiating client contact, qualifying prospects, leading our pre-sales team in sales process and strategy, making sales presentations, closing techniques, and developing service and pricing proposals.

We would expect you to work with the senior exec team to ensure a coordinated sales effort in target geographies and sectors and be able to perform and document research on leads, ensuring regular follow up with all assigned leads. Any experience of working in close partnership with the Oracle sales teams would be a distinct advantage.

You must have experience in working in UK & European Markets, and hold a Bachelors degree.

If you are interested please contact us at careers@rittmanmead.com.

Rittman Mead are looking for a strong technical consultant to join our fast growing office in the Benelux area. We have an immediate opening for project work in Brussels.

The applicant must have a solid background in Oracle PL/SQL development and a good understanding of data warehouse design and multi-dimensional modeling. Experience in any of the following is a major asset: Oracle Warehouse Builder (OWB), Oracle Data Integrator (ODI) or Oracle Business Intelligence Enterprise Edition (OBIEE).

The applicant must have excellent written and spoken English. Fluency in French or Flemish is a definite plus.

If you are interested in this position, please send your CV and contact details to careers@rittmanmead.com

The main focus of the event was Exadata (Exadata is the answer to every question, it turns out). With the release of OBIEE 11g there has been a lot of focus on the reporting tools recently, so it was a real pleasure to spend a couple of days listening to and thinking about some of the leading Data Warehouse implementations in EMEA.

We got to see what some pretty large organisations, predominantly in the telco and financial services areas, were doing with Exadata and hear the performance gains, reduction in TCO and general consolidation stories. We were also given an insight into how FIFA were using Data Warehousing to analyse betting fraud in world cup games, which watching the news this evening is pretty relevant. 

Probably the take aways from the event for me were:

• How mature Exadata is becoming; we are now effectively on the third release of the product – the other side of this is that clients were definitely reporting some hardware issues on the V1 boxes – something we have witnessed through a client of ours.
• Exadata genuinely seems to win-out in proof of concepts.  Obviously you don’t see the cases where it did not win, but its not just the Oracle marketing machine making this stuff up.
• Database consolidation is a major win for implementers. Having a consolidated database platform is an relatively obvious gain, however the compression Exadata gives means that a lot of database can be hosted on one server, compression ratios of between 4x and 10x were reported.
• The volume of data that can be processed by Exadata, and the performance gains are undoubtedly impressive. Throughput numbers like 50GB/s, and real time processing of 1m records a second, from this Exadata would really support the big data world.
• There are some very smart people implementing large databases on Oracle in EMEA and I think a lot of them were at this event

So thanks to Reiner and his team, and to everyone we met, it was a great event and we look forward to the next one.

The event is now scheduled for Monday 18th April 2011 at Ambassadors Bloomsbury, London – see http://www.ukoug.org/events/exadata-special-event/ for exact details. The UKOUG has since put together a strong agenda for the event including our very own Stewart Bryson who is giving a paper on how agile development fits in the Exadata world. Stewart’s abstract is:

In Agile project management methodologies, user stories describe desired content and functionality delivered and verified in a short amount of time. But traditional data warehouse projects struggle to deliver completed user stories due to the complexity of the intermingled moving parts: data modeling, ETL, front-end development, and a QA process that needs to validate the data from source to target.

The full agenda is as follows:

• 09:30 - Registration and coffee
• 10:00 - Welcome and Introduction  -  Lisa Dobson, UKOUG Director
• 10:15 - Consolidating on Exadata Database Machine: Managing Resources  -  Dan Norris, Oracle
• 11:15 - Coffe
• 11:35 - Understanding Exadata Performance: Metrics and Wait Events  -  Tanel Poder, E2SN Ltd
• 12:35 - Exadata POC’s – An Oracle Partner Experience  -  David Henderson, e-DBA Limited
• 13:20 - Lunch
• 14:00 - Exadata implementation case study for LinkShare  -  Alex Gorbachev, Pythian
• 15:00 - Agile DW with Exadata and OBIEE  -  Stewart Bryson, Rittman Mead
• 15:45 - AOB
• 16:15 - Close

Please come and join us at the event.

Taking a step back and looking at the ETL process we felt there was a lot of complexity unnecessarily created by decomposing the process into a number of program units or mappings. In our view this process creates the following problems:

• A large amount of processing time was wasted on the inter-communication of these mappings.
• Unnessary temporary storage objects and created and populated in the database.
• A separate technology is required to orchestrate all the mappings.
• It encouraged multiple developers to work on the ETL process thereby increasing the risk of mis-communication and mis-aligned interfaces.

In response to this Rittman Mead have developed the One Mapping Paradigm. We believe that you should put all your ETL code into one mapping, and as such have called this approach the One Mapping Paradigm (OMP). The goal of this approach is to encapsulate your entire ETL routine into one mapping or program unit.

We feel this approach adheres to some of the fundamental tennets of software development: encapsulation (everything is in the one mapping) and decoupling (there are no external dependencies). Further it completely negates the need for old bugbear re-usability, you now don’t even need to re-use code, just use it once, all in the same mapping. Most importantly OMP will also provides a reduction in development costs: you now only need one developer.

Our extensive research has also developed a series of steps you can follow to deliver your One Mapping. You should note that the One Mapping that OMP generates will be extremely complex, only by following these can you address the complexity of the mapping that will be generated.

OMP follows a black hole development approach where it is crucial for the developer to do as much development as possible without any outside interfere from either peers or the business. This allows the developer to focus solely on the development task in hand, which is a must when developing extremely complex code. It is also essential that the developer is allowed to proceed as far through the process as possible without stopping for other distracting activities like testing. In order to follow the OMP I have built the following example using Oracle Warehouse Builder.

• Step 1: source objects – create new mapping a drag all your source objects onto the canvas – it is important to arrange these in a straight line on the left hand side of the canvas.
• Step 2: add all your join operators to combine the data. A couple of tips here, (1) add predicates into the join conditions to avoid using filter operators (2) keep the data transition lines as straight as possible for performance reasons.
• Step 3: add any expression or transformational operators required – these should really be added to the middle of the canvas.
• Step 4: add all your target tables – these are added to the right hand side of your canvas. You are in the home straight now, but you may find this the trickiest part and we recommend using at least a 29″ monitor to complete this process.
• Step 5: unit test – note there is no orchestration or integration required, as you only have One Mapping.
• Step 6: release to production – you can just release you mapping straight into production, overwriting whatever was there before. There is no system or integration testing required as there is only one piece of code. UAT is further bypassed as your unit testing verifies whether the entire ETL process works or not.

We are looking for beta testers for this concept, so if you want to try the OMP for your ETL code, please contact me at omp@rittmanmead.com.

The slides can be downloaded here – note they are about 1.3MB.