Like the previous update, the 11.1.1.6 version of the SampleApp is an incremental release, designed to highlight new features that have come with the 11.1.1.6+ release of OBIEE, as well as incorporating some new examples sourced from the field. One of the most interesting new features in this updated version is Exalytics-enablement; if you download the VirtualBox image which has all the software pre-installed and pre-configured, OBIEE is enabled for Exalytics, with prompt buttons removed, auto-complete and auto-suggest enabled, and the ability to run the Summary Advisor from the BI Administration tool, which you’ll need to ensure is at version 11.1.1.6 or higher too and install on a separate Windows-based environment.

This new SampleApp also comes with scripts to automatically “inflate” the size of your dataset, and an additional subject area designed to analyze large tables and test out aggregation. The VirtualBox image for this release includes TimesTen for Exalytics, but of course it doesn’t come with a free Exalytics box, so this is really for testing and self-training only, not actual performance testing. By default no Summary Advisor aggregates are set up, but you can run the Summary Advisor, create some and then test out the dashboard with all of the “speed limiters” removed.

If you were lucky enough to see Jordan Meyer’s “Designing Eye-Catching and Effective Dashboards” presentation at last week’s BI Forum in Atlanta (if not, here’s an interactive version you can page through using your cursor right key) you may have had your interest piqued in Oracle R, and it’s ability to do statistical analysis on large data sets. This new version of the SampleApp has some R-based visualisations included, which use Oracle R Enterprise, an Oracle Database 11gR2 option that’s pre-installed on the VirtualBox image) and BI Publisher to render the actual visuals.

If you followed some of my postings around the launch of Exalytics, I mentioned that usage tracking has been enhanced in this release in order to capture the physical SQL, as well as logical SQL, associated with an analysis. The new SampleApp includes an example dashboard page that displays this physical SQL information for the most recent queries, and is accessible from a link on any other dashboard page, like this:

There’s also a number of new visualisation and UI enhancement examples in this new release, including what is probably my favourite new front-end feature in 11.1.1.6 – the ability to add selection steps to analyses even after they’ve been published to the dashboard, giving users much more ability to customise analysis views within their own dashboard.

Other new visualisation features include the new Strategy Wheel view in Scorecard and Strategy Management, waterfall charts, heat maps and an interesting new capability, web-style menus on the dashboard page:

Something that caused some gnashing of teeth back at Rittman Mead headquarters when it was announced back at Openworld, was support for Web Catalog auditing from within the next SampleApp (we’d developed, independently, something very similar). Using a set of scripts provided with the SampleApp, you can export the web catalog structure into a set of database tables, and then report against those, along with Usage Tracking data, to work out which catalog objects aren’t being used as well as volumes of objects in each folder, by user and type over time.

You can also perform some impact analysis, helping to identity in which dashboard are a set of select analyses, and quickly identify what list of objects are exposed in each dashboard page. Usefully, you can also use this feature to spot all objects that are using a specific RPD column, and looking at it the other way list out all RPD columns that are used by an analysis or other catalog object.

Something that’ll interest the DBAs amongst us is another new set of dashboard pages, that report against the Oracle Database system views, to display details of user sessions, SQL queries, datafile sizes, database parameters and so on. It’s clearly not meant as a substitute for Enterprise Manager Database Control, but if you’re working on a development system and want to check, for example, whether materialised view query rewrite is enabled, or who else is logged in and reporting against your data, it could be a useful set of reports.

Finally, there’s a whole new set of BI Publisher examples brought across from the new BI Publisher trial release, including lots of ones that show the additional capabilities the BIP brings, such as barcodes, OCR recognition, complex layouts using XPATH, bursting and so on. Here’s one new report, showing how BIP can report against Essbase directly, using MDX queries defined in the report’s data model.

So, quite a few interesting new features. The new 11.1.1.6 SampleApp should be available soon, probably at the same time as the next bundle patchset for OBIEE 11.1.1.6, and will be available in both VirtualBox format, or as a set of install files that you can set up in your own environment. Watch this space, as they say.

Anyway, here’s the invite. Make sure you take part, so that there’s a good sample size for Oracle BI, EPM and OLAP tools in the survey:

“We would appreciate your participation in ‘The BI Survey 11: The Customer Verdict’, the world’s largest survey of business intelligence (BI) and performance management (PM) users.

The BI Survey 11 should take about 25 minutes to complete. For further information, please contact Silke Hopf at BARC (shopf@barc.de).”

• The Rittman Mead BI Forum 2012 : Brighton’s Over, Now off to Atlanta!
• Week Two of the RM BI Forum 2012, in Atlanta

All presentations are in PDF format and, unless otherwise stated, are copyright of their respective authors/companies. Congratulations also to Emiel van Bockel and Jordan Meyer, winners of the best presentation awards for Brighton and Atlanta respectively, and to Antony Heljula and John Minkjan, who tied for best speaker in the 10-minute session category in Brighton.

Kevin McGinley Masterclass slides:

• The Action Framework : A Practical Guide to the Galaxy
• Oracle Scorecard & Strategy Management : The Un-Scorecard Manual
• Oracle BI Mobile : Where are we Now?
• OBIEE 11g Customized : Beyond the Whitepaper…

These are the presentation slides from the rest of the conference, from both Brighton and Atlanta:

• Alan Lee, Matt Bedin, Oracle Corporation : “RM BI Forum 2012 Opening Keynote”
• Steve Devine, Hitachi Consulting : “Effective Dashboard Design”
• Antony Heljula, Peak Indicators, “OBIEE 11g Security – It’s as Easy as 1-2-3″
• Adam Ferarri, Oracle Corporation : “Introducing Endeca Information Discovery”
• Tony Selden, PlayTech (Virtue Fusion) : “Whistlestop tour of a greenfield DW/BI Development (…When you really don’t need an ETL tool)”
• Daan Bakboord, Ebicus : “Oracle BI Applications: Can we make it worth the Purchase?”
• Antony Heljula, Peak Indicators : “Disabling your session logs will make no difference!”
• Mike Vickers, Rittman Mead : “The Rise of the Data Scientist”
• Ayse Oztop, GTech : “You Can’t Improve What You Don’t Measure”
• Adam Seed, James Coyle, John Holifield, Rittman Mead / Nominet : “BI Reporting Pack in the Real World”
• Adam Bloom, Oracle Corporation : “Oracle BI 11g Diagnostics”
• Emiel van Bockel, Centraal Boekhuis : “Go agile all the BI way”
• Mike Durran & Mark Rittman, Oracle Corporation/Rittman Mead : “Exalytics – All you ever wanted to know”
• Chris Webb, Crossjoin Consulting : “Introduction to the Microsoft BI Stack from a Microsoft MVP”
• Michael Wilcke, sumIT AG : “From reports/spreadsheets to OBI”
• Tim/Dan Vlamis, Vlamis Software Solutions, Inc. : “Data Visualization in Oracle BI 11g”
• Sunil S Ranka, FCS : “Deep Dive in OBIEE 11g Performance Tuning”
• Rene Kuipers, VX Company : “Groups, Roles and Issues”
• Stewart Bryson, Rittman Mead : “Reporting Against Transactional Schemas with OBIEE 11g”
• Matt Bedin, Phillipe Lions : “Real-Time / Low Data Latency Analytics with Oracle BI and Oracle Exalytics In-Memory Machine”
• Mark Rittman, Rittman Mead : “Inside Oracle Exalytics and Oracle TimesTen”
• Dan Vlamis, Vlamis Software Solutions, Inc : “OLAP Is Different From What You Think”
• Jordan Meyer, Rittman Mead : “Designing Eye-Catching and Effective Dashboards” and in web-based, interactive format (press right-arrow on keyboard to page through)
• Ramke Ramakrishnan, MarketSphere : “A Deep Dive into OBIEE 11g Security”

Finally, I put together two Flickr photo sets of the Brighton, and Atlanta, events, which you can access here:

• Rittman Mead BI Forum 2012 Photo Set, Brighton
• Rittman Mead BI Forum 2012 Photo Set, Atlanta

Thanks again to everyone who attended, spoke or helped organize the event. Now to start planning 2013!

The Myth: 

If you are having performance problems in OBIEE, you should switch off BI Server logging

The arguments for:

1. Instinct would tell us that writing a log is going to take longer than not writing to a log
2. On a system with high user concurrency, we would expect to see contention for writing to the log file
3. Usage Tracking records report response times, so why do we also need the server logging
4. Log files will cause the disk to fill up, which left uncontrolled could cause system instability

The arguments against:

1. If you have performance problems in OBIEE, then you need logging in place to be able to trace and diagnose them. The BI Server log gives us vital information such as what physical SQL results from a logical query from the front end. If you turn off logging, you lose all visibility of query behaviour, timings, and row counts. 
2. OBIEE writes lots of logs, more so now in 11g. Why only disable one of them? Why not all logs? 
3. If a query takes 30 seconds to run, how much of that 30 seconds is actually going to be in log overhead? You disable logging and now your query runs in 29.999 seconds. It’s still slow, it’s still a performance problem – and now you don’t have the data available with which to diagnose the problem!
4. Usage Tracking doesn’t record the same level of detail around a query’s behaviour (response time profile, row counts) that the server log does.
5. By default, Usage Tracking chops off Logical SQL above 1024 characters in length. 
6. Sometimes you need the log file to confirm that Usage Tracking is reporting correctly (especially in circumstances where report run times seem unusually high)
7. Error messages returned from the database are not captured in Usage Tracking  

It Depends

To a point, I am being contrary in arguing this specific issue, but it is important with this and other broad-stroke pronouncements around performance that get regurgitating without context and caveats that they are understood. In particular, labelling it a “Best Practice” is a dangerous fallacy as it implies that it should be done without much further thought or consideration of its consequences.

If the NFR for a report’s performance is [sub]-second and it is not being met, then profiling of the end-to-end response time breakdown should be done, and it might be that it demonstrates that the logging is impeding performance. But the point is that it is proven rather than done blindly. 

Further reading

Cary Millsap’s paper, Thinking Clearly About Performance, is an excellent starting point for developing an understanding of a logical and methodical approach to performance problem solving. 

James Morle wrote an great blog post on the subject of “Best Practice” and why it is dangerous terminology, entitled “Right Practice”

 
Thanks to Tony for reviewing & making further suggestions for this article.
 

If you’ve wanted to attend one of our courses but your company didn’t want to train up an entire team, here’s your opportunity to learn Oracle BI from the experts!

We are pleased to announce the following new dates for our popular public training courses in the UK, India, and Australia. Dates for America will follow shortly: 

TRN202 : OBIEE 11g Bootcamp (5 days) Prices: £2000 + VAT (UK), US$ 3,250 (USA), Rs. 44,000 (India), AUD 3,250 + GST (Australia)

• 11th – 15th June, Brighton UK
• 11th – 15th June, Bengaluru India
• 9th – 13th July, Bengaluru India
• 23rd – 27th July, Melbourne Australia
• 6th – 10th August, Bengaluru India
• 13th – 17th August, Brighton UK
• 10th – 14th September, Bengaluru India
• 8th – 12th October, Brighton UK
• 8th – 12th October, Bengaluru India
• 12th – 16th November, Bengaluru India
• 10th – 14th December, Brighton UK
• 10th – 14th December, Bengaluru India

TRN205 : Oracle BI EE 11g Create Reports, Dashboards, Alerts and Scorecards (2 days) Prices: £800 + VAT (UK), US$ 1,300 (USA), Rs. 17,600 (India), AUD 1,300 + GST (Australia)

• 23rd August – 24th August, Melbourne, Australia

TRN 403 : Oracle Data Integrator 11g Bootcamp (5 Days) Prices: £2000 + VAT (UK), US$ 3,250 (USA), Rs. 44,000 (India), AUD 3,250 + GST (Australia)

• 11th – 15th June, Atlanta GA
• 16th – 20th July, Bengaluru India
• 30th July – 3rd August, Brighton UK
• 17th – 21st September, Bengaluru India
• 24th – 29th September, Melbourne Australia
• 29th October – 2nd November, Brighton UK

TRN303 : Oracle Business Intelligence Applications 7.9.6.3 Bootcamp (3 Days) Prices: £2000 + VAT (UK), US$ 3,250 (USA), Rs. 44,000 (India), AUD 3,750 + GST (Australia)

• 20th – 22nd August, Bengaluru India
• 15th – 17th December, Bengaluru India

Don’t forget, there is a 10% discount for UKOUG members. For more information, or discuss your training requirements in detail, please email us.

Jon Mead is on the board of directors for the UKOUG, and both Jon and I have chaired the Business Intelligence & Reporting Tools SIG within the UKOUG in previous years. I’m particularly keen that we get some good, new and in-depth content around Oracle BI, DW and ETL for this year’s event, and I’d imagine papers on the following topics might go down well:

• OBIEE development methodologies and best/right practices
• OBIEE metadata modeling techniques, particularly against OLTP sources, OLAP sources and federating disparate data sources
• Dashboard design best practices, and anything around effective / innovative visualizations
• Presentations around data mining, Oracle R, big data and Hadoop
• Tips and techniques around ODI, including migrations from OWB and Informatica
• Papers around mapping, mobile, scorecards or any of the other OBIEE extensions introduced with 11g
• Techniques to get the best out of the BI Apps, including handling upgrades, preparing for the Fusion Apps, and combining packaged data models with custom ones

I’ll be putting a paper in on real-world use of the Exalytics In-Memory Machine, based on testing we’ll be doing over the summer along with feedback from customer PoCs. All speakers get a three-day pass to attend all of the other sessions, and if you get accepted you’ll be presenting at Europe’s biggest Oracle conference as well as knowing you’re contributing to a great event. There’s also a number of great social events and awards over the three days of the conference, and the photo below is of Debra Lilley (UKOUG President) and Lisa Dobson (Vice President) giving the awards whilst Vikki Lira (OTN and the ACE Program) stands in the foreground.

The call for papers website is here, and you’ve got until the 1st of June to get your abstracts in. Hopefully I’ll see some of you attend, and maybe present,

The format of the US BI Forum followed the same structure as the UK one, with Kevin McGinley providing the masterclass on the Tuesday, the main conference running on Wednesday and Thursday, and the NDA BI Developer day on the Friday organized in conjunction with Oracle. Kevin’s session was if anything even better received than in Brighton, with topics ranging from the Action Framework through Oracle Scorecard & Strategy Management, UI customization and of course BI Mobile. Thanks again to Kevin for taking the time to develop the materials, and then join us over two weeks to deliver the masterclass and then take part in the event itself.

The main conference then kicked-off with Tim and Dan Vlamis talking about dashboard best practices, then went on to cover Endeca, OBIEE performance tuning, security, Exalytics, RPD data modeling, big data and the new 11.1.1.6.2 SampleApp. As with Brighton, we ran a number of 10-minute sessions over the two days, including some Ignite-style sessions that had slides that auto-advanced every thirty seconds, and TED-style sessions where the speaker covered a controversial or counter-intuitive topic with minimal slides and sometimes props. Here’s Christian Screen, from Cap Gemini and ArtofBI.com, delivering his Ignite session on how to become an Oracle ACE.

The Best Presentation Award went in the end to our own Jordan Meyer, who talked about the wider world of data visualizations including examples such as the Billion Dollar Gram, a facebook network visualization created using R, and other examples created using Oracle’s R toolkit and embedded in Oracle BI dashboards. Jordan had a great relaxed but engaging presenting manner, covered some hot new technology and even managed to create a visualization based on Stewart’s comments about Apple on our internal mailing list, shown in the photo below along with the subject.

Although the event is primarily community organized, we had some exceptional support from the Oracle BI product management/development team again this year in Atlanta, including Matt Bedin who heads-up developer outreach for Oracle BI, Philippe Lions who demonstrated the new 11.1.1.6.2 SampleApp and provided a beta version for delegates to take away with them, and Pravin Janardanam who recently joined the product management team and is responsible for the metadata elements of the BI Server. We were also privileged to be joined for the second year by Jean-Pierre Dijcks who ran a whiteboarding session on big data, and Adam Ferrari, ex-CTO of Endeca who talked about the Oracle Endeca Information Discovery platform and analyzed, live, the tweet stream from this week’s, and the previous week’s, attendees. Thanks again to everyone from Oracle, especially for staying around for all four days and taking part in all of the sessions.

Friday, as with the Brighton week, was a special BI Developer day organized in conjunction with Oracle BI product development and held under NDA (non-disclosure agreement), where we were taken through the product roadmap in more detail and looked in particular at a couple of significant changes/developments in the OBIEE product architecture. Of course I can’t go into any detail now, but thanks again to Oracle for this and watch this space for insight and analysis once things become public.

So that’s it for now. I’ll do one final blog post early next week to post all of the presentation PDFs, and photos from the US event can be viewed in this BI Forum Atlanta 2012 Flickr set. Thanks again to everyone, and no doubt we’ll start planning the 2013 event very soon!

‘Configurable dimensionality’ is the significant update to this version. We can probably say that this is the long awaited and biggest change that developers made to HFM.

So what is ‘Configurable dimensions’ is all about? There had always been a need for HFM customers to go beyond the four Custom dimensions, which are provided by default for their business needs. For e.g. they worked by stuffing two different details like Balance Sheet movements and Products details into a single Custom dimension. This was being addressed by Oracle in this release. Now, the users can have as many custom dimensions as required for their implementation. Let us see how this can be done starting by creating a new classic application.

Creating application:

Desktop Client: Many existing functions are cut down in Win32 client. Now, it has only Profile Manager and Metadata manager, rest all functions are available via Web including Create Application task. Profile Manager is used to define the application profile (.per). It can be installed in any machine since it can only be used in offline mode and has no dependency on HFM server component.

In addition to defining Year, Periods and Frequencies, we can now define the number of Custom dimensions as a part of the application profile. So basically, when you create application using this profile file, all these custom dimensions are created. The size (Small, Medium or Large) for a custom dimension must be determined depending on the number of members its hierarchy contains. Custom1&2 by default, are Large size.  As shown in the screenshot, we’ve created a total of 6 Custom dimensions.

Login to the Workspace to use this profile and create the application. As you can see, web server URL need not be provided.

I’ve loaded some sample metadata and data for few periods. !CUSTOM_ORDER section is introduced which specifies the order in which the custom dimensions are displayed. This has to be specified in the metadata load file.

Now, let’s skip the complexities and move the data up the Value dimension. At this point, I’m eager to see the application database structure.

Sub-cube Architecture:

Financial Management stores its data in database blocks called subcubes rather than in records. Subcube contains Page and Subcube dimensions. The Page dimensions are Scenario, Year, Entity, Value and the Subcube dimensions contain all the members of ICP, Account, View and Custom dimensions (in our case 6 custom dimensions). So typically, for a single Page dimension intersection, if we have 100 accounts and 10 valid members for each Custom dimension and all intersections are populated, then it would be 100000000 records.

HFM stores all this data for these dimensions in three tables:

• DCE (Currency Subcube)

Stores Entity and Parent Currency values and their adjustments.

• DCN (Parent Subcube)

Stores remaining value dimension members

• DCT (Journal Transactions)

Stores Journal Transactions and when posted, they are transferred to DCE (<Entity Currency Adjs>/<Parent Currency Adjs>) or DCN ([Parent      Adjs]/[Contribution Adjs])

In the below table, DPx_INPUT and DPx_INPUTTRANSTYPE are repeating fields for each period that the application contains. The columns are numbered from zero so for 12 periods, the column names will be DP0_INPUT to DP11_INPUT and DP0_INPUTTRANSTYPE to DP11_INPUTTRANSTYPE.

Compared to the older versions, the column structure almost remains the same. Interestingly, if you’ve noticed, in older versions, there are four LCUSTOMx columns one for each custom dimension. In this version, I anticipated that for each extra custom we add to the application, there would be a new column added to these tables like LCUSTOM5 etc. But the structure doesn’t even contain LCUSTOM3 & LCUSTOM4 and it is tempting to know how data for other Custom dimensions are identified. We are working to find out the same and any comments from the readers are really appreciated.

The supporting products for HFM viz. Financial Reporting, FDM, Application Upgrade Utility and existing HFM rule functions were also being updated to support HFM’s Configurable Dimensionality. Although we have the flexibility of adding any number of custom dimensions, time taken for calculations, consolidation and other performance issues need to be considered and application should be designed accordingly.

UI Enhancements:

The new User Interface for HFM is definitely a notable update when compared to the earlier version.  It is easier to navigate and includes some special features as well. This is an outcome of migrating EPM components like HFM and Planning to Oracle’s ADF (Application Development Framework).

Let’s take a look at all these UI enhancements –

Multiple applications in Workspace:

Different HFM applications can be opened at the same time.

Multiple modules in an application:

Different HFM application modules in each application can be opened simultaneously.

POV enhancements:

There are significant changes to the way we select dimensions to Rows and Columns. We can relate this approach to Hyperion Financial Reporting wherein we ‘drag and drop’ dimensions to Rows and Columns in dimension layout. This approach is simpler and made common to both data grids and data forms. We can also add dimensions manually in data forms.

Data grid enhancements:

Again, we can relate the creation of data grids to how we create the reports in Financial Reporting. In HFR, we create/design the report and run the report to view it. Similarly, we have Grid designer and Grid Viewer in data grids.

Display options were docked to the right hand side thus decreasing the time wasted on navigation.

Another new feature is the indication of cell colors at the bottom of the Grid Viewer. This will be very helpful to the business users and they don’t need to reach support or documentation to understand the Cell Colors.

Data form enhancements:

Member selection process is pretty much similar to what we do in data grids. And there exists designer and viewer in data forms too. Export to/Import from Excel options were disabled.

Favorite members in Member selection:

Frequently used members can be selected and saved as favorites. These are available across other modules of the application as well.

Loads and Extracts page:

Load and Extract tasks like Security, Metadata, Member Lists and Rules are consolidated in one single page.

Journals Module Enhancements:

Journal tasks functionality is pretty much same but with major changes in UI again. Journal reports module exists both as a separate task and as a part of Manage Journals (opens in a new window when clicked).

Intercompany transactions module is not available and will be included in the upcoming patch – 11.1.2.2.101. It might take some time for the old version users to get used to navigating the new interface. Hope we can leverage from these ADF features. There are other enhancements to the HFM Win 32 client and Custom dimension configuration, which will be covered in a different post.

Using the same example, I’m going to provide a simplistic explanation of what’s happening inside the black box that is RTD.  There’s more happening in the box than explained here, but hopefully you’ll get an idea of how the solution works.

Let’s say we have four products available a DVD, Book, CD and Video Game.  Obviously, in reality we may have thousands of different products that will come under different product categories and sub-categories.

A customer visits our website and gets to the point of requiring a product recommendation.  RTD could first check availability of stock, as there is no point recommending products that are not available for purchase (yes, I appreciate you might want to allow people to pre-order for when available, but as I said this is a simple example!).  In this case, the video game is out of stock, leaving RTD to make a decision from the remaining three products.

When RTD is first switched on, it needs to learn before it can make recommendations.  In this example, we are going to have three bits of data, the customer’s age, day of the week and customer’s country of residence.  In the example, there are also a limited number of values for each field.  Day of the week, for instance, can only be Monday or Friday (should have been Saturday and Sunday if I’m going to make it a two day week!).  So the first customer from the UK, comes to the site on Friday and is 18 years old.  This customer purchases the DVD, so as you can see from the diagram below, these customer attributes are stored against the DVD.

The next diagram below shows the second customer, aged 30 from America, visits the site on Friday and purchases the CD.  Again, you can see that the attribute values for the second customer are stored against the CD.

Customer number three then purchases the CD, with their attributes being stored appropriately as shown in the pic below:

So I guess you get idea by now.  So after 50 customers, of which 15 have purchased the DVD, 15 the book and 20 the CD, the attribute values against each product look as follows:

Now RTD has sufficient learning, when the next customer comes to the website it can predict the product they are interested in.  It does this by scoring the customer’s likelihood to purchase each product.  So when the 51^st customer, who is 50 and from the UK, visits the website on Monday, they get the following likelihood to purchase scores:

As you can see from the above the product with the highest likelihood of purchase for this customer on this day is the book.  As well as returning this recommendation to the website, RTD continually learns and hence will self-adjust to changes in customer behaviour.  RTD scores each product or choice, and hence from a front end display point of view you could show the highest ranked product or the top ten in purchase likelihood order.

When new products or new customer attributes are added, RTD will automatically start learning on these and recommending as appropriate.

Hopefully, this has given you some idea of how RTD works.  In my next post, we’ll take a look at RTD components and hardware requirements.