From an OBIEE perspective, we started 2011 with the 11.1.1.3 release on general availability, and the 11.1.1.5 release made available about mid-way through the year. In hindsight, 11.1.1.3 was a bit of an “early adopter” release, with all the main functionality working but a few hassles around installation, upgrades and “fit and finish” issues. Many of our customers and partners adopted OBIEE 11g at this point, but quite a few held back for 11.1.1.5 which is generally considered a “fully-working” version (albeit with the usual bugs, issues etc that you get with any piece of enterprise software). Looking back at 2011, probably the biggest drivers for 11g adoption were Mobile (particularly the iPad client), the new Answers and Dashboards UI, and management via Enterprise Manager and WLST scripting. Without much fanfare, the Action Framework has also been widely adopted, but more for adding “missing functionality” such as Essbase writeback rather than integration with web services and workflows. Something I hear is going down well but don’t see much evidence in the field is Scorecard & Strategy Management, though I would expect to see more of this as Oracle push “Oracle BI Foundation” as the base license deal rather than “Oracle Business Intelligence Enterprise Edition”, as this will remove the price premium for scorecards (and mobile) and remove one of the hurdles for adoption.

As well as OBIEE 11g releases, we saw the 11.1.1.5 release of Oracle Data Integrator (review here, here and here), and confirmation that OWB is now at the end-of-the-line, along with Discoverer, with only maintenance releases coming along in future. Almost every new ETL project I see now is based on ODI (rather than OWB) so in the field, this was a de-facto change anyway, though it’ll be interesting too see what options emerge for OWB customers looking to migrate to ODI – keep an eye on this blog as we’ll be running a seminar in London in February on just this topic.

2011 was also the year we ran our third-ever Rittman Mead BI Forum, running again in Brighton and also this time, in Atlanta in America. I was very pleased to be joined by Tony Heljula for the one-day Masterclass before each event, where we talked about OBIEE 11g topics such as architecture, data modeling, SOA integration and Spatial Integration; Tony’s a great speaker and it was fun to collaborate on the topics and deliver them together over the two locations. Next year, Kevin McGinley from Accenture is delivering the masterclasses, and he’ll be known to our US readers as winner of the “Best Speaker” award at last year’s US BI Forum, and “Best Paper” award at ODTUG KScope’11. The call for papers for next year’s BI Forum is now open, and running until the end of January, so if you want to propose a session for either the UK, US or both events, take a look at the announcement page now!

Rittman Mead were certainly out on our travels in 2011, with our OBIEE 11g Training Days events running in the first half of the year in Atlanta (twice), Bangalore (Twice), London, Johnannesburg and Brussels, and our team speaking at events such as Collaborate’11 in Orlando, UKOUG 2011 in Birmingham, ODTUG KScope’11 in Long Beach, Oracle Open World in San Francisco, ODTUG BI/EPM Symposium in Sydney, AIOUG Sangam’11 in Bangalore, DW Global Leaders’ events in Athens, Rome and San Francisco, and plenty of others I’ve probably forgotten.

We encourage all of our staff to speak at conferences and our policy is that, if you get a paper accepted anywhere, we’ll give you the time-off to speak and cover your expenses. Thanks also to the Oracle ACE and Oracle ACE Director Program and to Vikki, Lillian and Justin at the Oracle Technology Network, who’ve funded a lot of my speaking engagements on behalf of the ACE Director program, and who provide complementary Open World attendee passes for myself as an ACE Director and our Oracle ACEs, Stewart Bryson and Venkatakrishnan J.

It’s also been a year where many of our staff have started to gain a name for themselves as specialists in certain areas. Stewart Bryson, for example, has been speaking and writing all year about real-time BI, and the new development paradigm that you can adopt when combining Exadata, OBIEE and Agile methodologies, whilst Venkat has been building on his reputation as the “go-to” expert for OBIEE and Hyperion Integration. Borkur has spoken at several conferences on Oracle Golden Gate, whilst Mike Vickers is getting a name for himself as a speaker on BI methodologies. Behind all these experts though is our team of BI, DW and EPM enthusiasts, taking this message and their own particular skill-sets to our customers around the world, sharing what we’ve learned and hopefully “raising the bar” for customer implementations.

For Rittman Mead as a company, it’s been an exciting year with lots of growth. We’ve now got offices in the UK, US, India, Brussels and Australia, new training centres in Brighton and Bangalore, and we’ve been winning awards such as the UKOUG BI Partner of the Year award 2011/12. Thanks again to all of our customers, partners and staff, and I’m looking forward to getting out again and visiting our various country offices in 2012.

So what about 2012? What are we likely to see, both from Oracle, from the industry in general, and from Rittman Mead? Well, from an OBIEE perspective, we’ll most likely see the 11.1.1.6 release fairly early in 2012, with probably a minor release (11.1.1.6.1?) sometime later in 2012, going on how Oracle do OBIEE releases at the moment (one major, one minor, each year). Going on from 11.1.1.6, there’s some general themes that Oracle are working to with OBIEE:

• Making Mobile a first-class client – perhaps adding support for Android, re-vamping the UI, maybe making it possible to author reports from mobile, maybe even re-thinking the UI so it’s not just a dashboard on an iPad, but maybe delivering BI in different ways that are perhaps more suitable to a mobile device
• Developer productivity – making it simpler/quicker to develop ETL from a logical model, and to process customizations in source systems, plus better support for team working, version control, more granular metadata stored in middleware repositories, and hopefully running end-to-end from ETL through to the front-end reports
• Appliances – Exalytics will be out in early 2012, combining hardware + software into a “it just works” appliance that also harnesses the “Exa-” engineered systems approach.
• Building out BI innovation – 11g so far has been about Fusion Middleware integration, and supporting the Fusion Apps. Hopefully we’ll see more pure-BI innovation in future releases, to enable OBIEE to take on some of the specialist, niche vendors such as Qlikview and Tibco (Spotfire) whilst providing enterprise features that are Oracle’s traditional strength
• Steady improvement in Hyperion Integration – I think it’s fair to say that Essbase/Hyperion integration for OBIEE is still a “work in progress” so we’re looking forward to seeing improved Essbase integration, metadata access support for Planning, integration of Essbase/HFM with BI Apps, and some replacement solution for integrating the OBIEE dashboard with Hyperion Workspace

There are also some general industry trends that we’re also very excited about for 2012. Real-Time BI (in it’s many forms) is becoming more and more “the expected norm”, but there are lots of challenges in terms of how we do data movement/ETL, how we present a consistent set of numbers to users, how we may meaningful decisions based on data that’s come in a few seconds ago, and how do we present what can potentially be large amounts of sensor-driven, machine-captured “big data” to users without them becoming overwhelmed by data?

Unstructured data will most certainly be something Oracle will be talking about a lot in 2012 following the acquisition of Endeca, with my prediction being that a future version of the OBIEE 11g BI Server will support Endeca as a data source (but how will Endeca’s data be modelled as a star schema?), and Endeca Latitude front-end features being gradually incorporated into OBIEE 11g’s front-end, in the same way that Hyperion WebAnalysis and Oracle Discoverer features have made their way into the 11g release. I’m not sure Endeca / Unstructured data will ever be “mass market”, but it’s an obvious acquisition for Oracle and I’ll be looking forward to seeing unstructured data features making their way into Oracle’s BI platform.

Another area we’ll be focusing on in 2012 will be in-memory BI. Exalytics is just around the corner, and the new version of TimesTen that it’ll ship with, plus the in-memory version of Essbase, should make split-second, lightning-quick analysis of large sets of data a possibility. The biggest barrier I see to user adoption (well, apart from data quality) is slow queries on dashboards, so Exalytics’s in-memory databases plus the 1TB of RAM it’ll ship with will be definitely welcome. But how well will in-memory combine with real-time, and how will the TimesTen in-memory database perform compared to Exadata, OLAP + materialized views, or even the file-based results cache that OBIEE already comes with? Check out our presentations at RMOUG Training Days in February, and the Hotsos Symposium in March, for the results of our testing.

The BI Apps is also an area that’s due for a lot of changes in 2012. BI Apps 11g is already out, but you’d be forgiven for not noticing, as up until recently it’s been on controlled release, and it still only covers the Fusion Applications as a data source, with support for Apps Unlimited (EBS, Siebel, PSFT etc) coming in the next 12 months or so. We should also start to see some innovation in the product itself, with closer integration with OBIEE in terms of pushing through customizations, integration with Essbase and HFM, and general reduction in the workload in terms of upgrades, data loading and team development. Probably for most existing BI Apps 7.9.x + Informatica customers, not much will be happening in the next 12 months, but keep an eye on product announcements and expect lots of activity in 2013.

Cloud-based BI is a particular interest of the company and in particular Jon Mead who presented on the topic at Open World last September, based on trends towards cloud-based applications in the industry and demand from customers to simplify their systems and reduce their costs. We’ll be introducing an “Exalytics in the Cloud” option in 2012, offering customers the ability to access our cloud-based Exalytics machine and replicate, via GoldenGate, their data into our hosted, secure environment and have us manage their BI system. As always, the challenges with BI in the cloud are firstly, replicating large amounts of data securely into the cloud, and secondly, trusting a vendor to manage your data for you, but ETL technology and bandwidth are making the first less of an issue, and for the second, a proper “best practices”, always-patched, 24×7 monitored remote system is often more secure and better managed than something in-house. Look out for lots on this from us over the next 12 months.

We’ll be speaking at lots of events in 2012, including RMOUG Training Days in Denver in February, Hotsos Symposium in Dallas in March, Collaborate’12 in Las Vegas in April and ODTUG KScope’12 in San Antonio in June. Keep an eye on our Events page for dates as they are announced, and also on our Public Scheduled Training Events page for details of OBIEE 11g, ODI 11g and BI Apps 7.9.6.3 courses running regularly at our training centres in Brighton, Atlanta, Bangalore and Melbourne.

Finally – at last, 2012 will be the year that my book is released. I’ve had some excellent support and co-operation from the Oracle Product Development team over the past few months, and so the book will be bang-up-to-date when it comes out, covering 11.1.1.6.x and Exalytics and including all the new visualizations, team development features, security capabilities, scorecard features, and data source support that’ll be around in the second half of 2012 when the book comes out. I’ve now written 12 of the 15 chapters, writing the Answers + Dashboards chapter over Christmas, and I’ve just got Security, Clustering and High Availability and Scorecards & KPIs to write, plus all of the technical edits from Mike Durran and Venkatakrishnan J to incorporate, plus adding new features introduced in 11.1.1.6 since the original chapters were written against 11.1.1.3 and 11.1.1.5. Expect to see the book on the shelves at the Oracle Open World bookstore, and a return to more regular blogging from me once I’ve finished the book and all the NDAs are lifted!

So that’s it from me now – Happy New Year to you all from Rittman Mead in the UK, Europe, USA, India and Australia, and see you all again hopefully in early 2012.

BI EE connects to HFM through JavaHost. So, the setup for configuring the connectivity is to basically make the EPM environment variables available for access to BI EE. Before making the changes, there are a couple of things to be aware of

1. Ensure that DCOM is enabled on the BI Server machine. This is needed for HFM client to work/connect properly with HFM Server.

2. Ensure that HFM Client is installed.

The above assumes EPM 11.1.2.x version. If you are on Hyperion 9.3.3, then you need to have the ADM drivers installed manually. All of this is documented clearly. So, rather than re-iterating that i will just point to the link here. Ensure that you follow all the steps mentioned in the docs else you will have issues in the connectivity. Once all the pre-requisite steps are done, we start off with the metadata import process.

The import process will basically reverse engineer all the HFM hierarchies. In addition, interestingly it also brings in attributes like CellText, CurrencyType etc. So, not only does this provide pure data reporting capabilities it can also report on supporting documents within HFM. A clear screenshot of the physical layer post import is given below

Like Essbase & Oracle OLAP sources, creation of metadata for other layers like BMM and Presentation layer is just a matter of drag & drop.

But the interesting thing to note here is the way hierarchies are imported. If you notice below, all the major dimension hierarchies are imported as parent child hierarchies.

This makes sense as HFM hierarchies rarely are balanced. Let’s now try creating a report containing just the Entity hierarchical column and then look at the query generated.

Interestingly similar to SQL sources (HFM also has a Relational data store – But BI EE uses the ADM driver to extract the data), it does generate separate queries for each level. Also the Queries themselves don’t look to be SQL. Let’s now fire a query to extract the data from HFM.

This looks good. And it does seem to generate the right queries as well. But what are the queries that get generated? They are not normal SQL queries. Also, how does BI Server generate them. Does it delegate all the data extraction to the ADM driver natively or does it let JavaHost extract the data. If it is the latter, it will be very interesting as this is the first time i am noticing JavaHost being used for data extraction from within BI Server. Perhaps this is how the planned BI EE 11g-Planning integration will work in the future. Anyway to answer these queries, i will follow this post with another one with more details.

Of course, Mark Rittman will be there in his capacity as Rittman Mead Technical Director and Evangelist, Oracle ACE Director, and also, as an ODTUG officer. Marks sessions and events are listed here:

Sunday: 8.30am – 4.30pm, Room 202B: BI Symposium (Organizer)
Monday: 1.15pm – 2.15pm, Room 101A: OBIEE 11g Answers, Dashboards, Scorecards & Reporting New Features (Presentation)
Tuesday: 12.15pm – 1.45pm, Room Promenade B&C: BI & EPM Lunch & Learn with the ACE Directors (Panelist)
Wednesday: 9.45am – 10.45am, 101A: OBIEE 11g Architecture & Internals (Presentation)

From Rittman Mead America, we have Charles Elliott, our Senior OBIEE specialist delivering an OBIEE 11g Hands-On Training (HoT) session where attendees will learn to use New Report Prompts, Action Links, Custom Groups, and Hierarchical Columns. If you haven’t yet experimented with the new front-end features in OBIEE 11g, then this is the HoT for you:

Wednesday: 1:45pm – 5:15pm, Hyatt Seaview: HoT Session F, Oracle BI 11g Answers and Dashboards (Hands-On Training)

I’ll be attending as well. I’ll be in the room with Charles for the HoT Session F, but I’m also doing a presentation as well as participating on the EPM/BI Experts Panel moderated by Natalie Delemar. This was interesting to me in that it apparently takes four Hyperion/Essbase panelists to adequately represent that side of the house, but only one OBIEE expert is required: me. All joking aside, I know this panel will be heavily attended by the Hyperion-minded, and I may be a straw man for frustration they’ve had with OBIEE over the years, but I’m ready! There’s a much better story to tell now with OBIEE 11g and OLAP, and hopefully they’ll at least give me a chance to speak before they start throwing the fruit:

Tuesday: 11:15am – 12:15pm, Room 102AB: EPM/BI Expert Panel (Panelist)
Wednesday: 1:45pm – 5:15pm, Hyatt Seaview: HoT Session F, Oracle BI 11g Answers and Dashboards (Hands-On Training)
Thursday: 8:30am – 9:30am (Seriously?), 101A: Agile Data Warehousing with Exadata and OBIEE11g (Presentation)

As usual, we are always happy to speak to attendees, customers or other community members, so if you see Mark, Charles or myself around, feel free to stop us and say hello. In case you’re unsure… I’m the short one.

OBIEE 10g offered a lot of promise for Hyperion customers, in that you could connect Essbase to the Oracle BI Repository and create rich, interactive dashboards and analyses using Essbase data, relational data, or a combination of the two. The BI Server translated the incoming “logical SQL” queries from the Presentation Server into MDX queries that Essbase understood, allowing you to create reports, graphs and other visualizations that worked the same way whether your source data came from an Oracle Database, an Essbase cube, or any other sort of supported data source. You could also integrate the OBIEE dashboard with Hyperion Workspace, so that users signed on once through Shared Services and could then log in to OBIEE using the same credentials, inside a unified query environment, as detailed in this blog post from 2009.

But this integration came with limitations. The 10g version of OBIEE didn’t really have a proper OLAP-style query tool, such that when you drilled-down from one hierarchy level to another, it kept adding columns to the report and placing filters on the column you’ve drilled on, and there was no proper hierarchical selection tool, meaning that Essbase users moving from tools such as SmartView or WebAnalysis found OBIEE 10g to be a very “relational” query environment. In addition, only level-based hierarchies were supported within the Oracle BI Repository, and whilst OBIEE did a good job in “unraveling” the parent-child hierarchies in Essbase into corresponding level-based hierarchy when you imported an Essbase outline into the repository, changes to the hierarchy – particularly adding new levels or generations into the Essbase hierarchy – usually meant that you had to re-import the whole Essbase cube back into the repository, losing any integration work that you’d done beforehand.

So OBIEE 11g offered a bunch of improvements that were particularly aimed at Essbase, and other OLAP server, customers; the new hierarchical columns feature brought proper OLAP analysis to OBIEE, and coupled with OLAP-oriented features such as hierarchical prompts and selection steps, made OBIEE 11g a much more conducive environment for Essbase users.

There’s much better support for Essbase-native features such as value-based hierarchies, alternate hierarchies, multiple alias tables, UDAs and other Essbase artifacts, and the value-based hierarchy feature in particular is very welcome as it can avoid the need to completely re-import your Essbase database if your outline gains additional hierarchy levels.

So coupled with the new and improved visualization capabilities in OBIEE 11g, including maps, scorecards, improved dashboard controls, the whole new UI, improved BI Publisher and so on, there’s a lot to recommend OBIEE 11g to users of the “legacy” Hyperion tools looking for a bit of a refresh of their BI environment.

Some of the new capabilities in OBIEE 11g, such as the action framework, may not immediately obvious to Hyperion users as useful to them, but they actually bring the ability to create additional integration between the platforms. One thing that’s still lacking, for example, is the ability to write-back from the dashboard to Essbase (you can write back to relational sources, but not Essbase, yet), and you can also use it to call actions that, for example, run a calc script or perform an allocation routine. If you take a look at the new 11.1.1.5 version of the SampleApp (due out in a couple of weeks), there’s some working examples “from the field” that perform this function, and there’s a posting on our blog from last year that show how you can use the Action Framework, and the Credential Store within Oracle Fusion Middleware, to provide integration between the OBIEE toolset and the Hyperion toolset.

As you’ll also be on the OBIEE platform, Oracle’s “strategic” BI direction, you’ll also be able to take advantage of features such as Oracle Enterprise Manager, Oracle Platform Security Services, and all the other middleware features that come with 11g. And you’ll be able to also integrate with the Oracle Business Intelligence Applications, giving you the ability to reach through to an integrated data warehouse sourcing data from Oracle E-Business Suite, SAP, Siebel and Peoplesoft.

But with every new feature, you’re going to expect to hit some new issues. Some things haven’t changed in this release compared to 10g; if you do need to make changes to the Essbase database metadata in your Oracle BI Repository and you can’t get around it by switching to value-based hierarchies, you still need to re-import the whole outline as there’s no “sync” or “incremental import” feature in OBIEE 11g. The Oracle BI Server still function-ships very few OBIEE functions down to MDX functions, so you’ll either have to have the BI Server perform analytic-style functions for you, or you’ll need to use the EVALUATE function to pass them down yourself. And we’ve actually lost the integration between Hyperion Workspace and the OBIEE dashboard, so if you want both OBIEE and the Hyperion tools to use the same directory, you’ll need to connect them both to an external LDAP server, and use some other SSO solution over both products rather than the “point-to-point” solution that worked with OBIEE 10g.

Some new features, such as value-based hierarchies, come with one or more “gotchas” that may well catch you out by causing one problem when you’ve just fixed another. If, for example, you’ve moved your Essbase hierarchies in the Oracle BI Repository physical layer to value-based ones, your users benefit from more efficient searches across the hierarchy and you won’t need to re-import the outline if you add a new level or generation. But where you’ll come unstuck now is if you’re planning on setting up drill-to-detail reports, where Essbase provides the aggregated data but the detail comes from a relational database, such as Oracle. In this case, you’ll find it hard to set up the “vertical federation” that’s required as there are no “levels” to designate the granularity of the Essbase and relational sources. There are a number of workarounds (we covered two approaches, here and here, towards the end of 2010), but neither of them are pretty and you’ll have a “facepalm”-type moment when you realise this limitation of value-based hierarchies.

Other, possibly more serious issue, is around the way that OBIEE generates the logical SQL, and physical MDX, for pivot-table style queries involving hierarchical columns. Take for example the following simple pivot-table report with a single product hierarchy and revenue measure, where the user has drilled-down into the data to produce some totals and sub-totals.

Now if this analysis was being generated against a relational source, the BI Server would generate three or four logical SQL requests for each measure “grouping”, which in turn would normally be pushed-down into a single SQL statement that used sub-query factoring (the WITH clause). Again, not pretty, but databases such as Oracle 10gR2 onwards can handle this pretty well.

Where it gets tricky though is when you’ve got an equivalent Essbase source under the covers. In this case, these logical SQLs all get translated into one or more physical MDX statements, each one of which is returning the aggregated values for these sets of grouped measures. If we were using an “MDX-native” tool such as SmartView or WebAnalysis, we’d only need the one MDX query, but in this case our query has generated five of them.

Now where it gets really interesting is if we add a second, or third hierarchical column to the report, and in particular, when we pivot the dimensions around so that one of them runs along the columns, rather than just the rows, of the pivot table, like this:

In this case, what would appear to be a simple pivot table query with a couple of dimensions drilled-in to asymmetrically, ends up generating around fifty-eight separate MDX queries, each of which is relatively simple, but multiply that by a number of concurrent users, and you’ve got a seriously big load on your Essbase server.

Now in practice, there are workarounds, and the new subquery-caching feature in OBIEE 11g means that these MDX sub-queries are often retrieved from cache rather than sent on a round-trip to the Essbase server. You can also limit the effect by keeping all of the dimensions on the rows, rather than rows and columns, of your pivot table, and in practice most Essbase systems aren’t all that heavily loaded, so you may well get away with it – but it’s something to bear in mind.

Another new feature that can also add to this problem is selection steps. Selection steps are very interesting to Essbase customers as they allow you to build up queries that use just certain members from the dimension, combine them together, use them to filter data and do other “OLAP”-style filtering and selections. To take the example below, we’ve used the Sample.Basic cube to initially return the grand total figure, then used selection steps to keep just the Q1 and Q2 figures, add a calculated member for H1, then do something simular for the regions.

All good so far, and light-years ahead of what we could do with OBIEE 10g. The problem comes though with the way that the selection step sub-queries are translated from the logical SQL that the BI Server uses down to the physical MDX queries that Essbase uses. From the tests that Venkat has carried out, the number of logical SQLs that selection steps generates comes to the product of all the individual selection steps, so if you have three dimensions that contain 2, 3 and 3 selection steps each, you’re going to end up with 2 x 3 x 3 = 12 logical SQLs, each one of which translates into 1 or 2 physical MDX queries. Once you add in the usual grand total at the top that brings back the entire cube, and any hierarchical column drills you might add in, you can see how you might end up with a very large number of MDX statements being sent to the Essbase server for what appears to be a very straightforward report.

Now in practice you tend to have workarounds for these issues; we tend to suggest keeping the number of hierarchical columns to a minimum in an analysis, and when you do use them, limit how far down you drill into them. If you use selection steps, try and keep them to one or two per dimension maximum, and overall, size your system to expect large numbers of relatively simple MDX statements.

Another aspect to consider is where your users current make use of the Excel Add-in, or SmartView, to do their analysis. Typically, they’ll use Excel and the “query-by-example” feature of the Excel Add-in to write-in the names of the dimension members they want in the X and Y axis, and then do a retrieval based on the selected members. Using this approach, end-users can create pretty complex analyses where there’s very little limitation on the combination of dimension members they use (selecting budget profit for this year, alongside the same figure for last year, together with actual inventory and forecast margin, for example). Creating analyses using these sorts of member selections is certainly possible using OBIEE 11g, but you’ll need to learn a new set of skills to do it properly.

For example, if you want to create a dynamic time-series reference such as the value of a measure as at this time last year, or over the past three months, or year-to-date, you can use one of the OBIEE time-series functions (AGO, PERIODROLLING, TODATE) to provide an offset value for a measure.

Similarly, OBIEE 11g now by default creates a single measure within your logical fact table which is then dimensioned by the measure (accounts) and scenario dimension, if present. If you want to display on the screen your “budget profit” figure using this approach, you can use the FILTER function in OBIEE to filter the measure by these two dimension member names, an approach you can use to display any arbitrary measure and scenario combination in an analysis.

Another technique that’s useful is being able to make use of native MDX function names through EVALUATE and EVALUATE_AGGR. As I mentioned earlier, one limitation with this current release is that very few OBIEE functions are function-shipped down to MDX, but if there’s an MDX function you want to use, you can easily access it through EVALUATE (for scalar functions, such as TOPCOUNT) or EVALUATE_AGGR (for functions that return an aggregated value, such as STDDEV).

So there’s two things to take away from this last point; firstly, don’t think of OBIEE 11g as a like-for-like replacement for Excel and the Excel-Addin, think of it instead as a complementary tool that covers, say, 60% of its functionality but then provides a whole raft of features that Excel alone can’t provide. Also, be prepared to roll your sleeves up and learn how OBIEE analyses are created, and expect your non-trivial reports to require a bit of Essbase knowledge an understanding of the more exotic features of Answers and OBIEE.

There’s more to the story that I’ll go into in the session, including what’s now possible for Hyperion Planning and Hyperion Financial Management users, and I’ll also run through some detail on how the actions integration works between the OBIEE dashboard and Essbase writeback and calc scripts, and I’ll also go into more detail on how you set up vertical and horizonal federation between Essbase and relational sources, and why you might want to use the new LOOKUP function to do some of this instead. So without jumping the gun too much, it’s a bit of a “qualified Yes”, but I’ll go into more detail on the day.

My session at the OUG Conference Series EPM & Hyperion 2011 is at 9.40am – 11.10am on Day 2 (15th June 2011), and full details on the event, including how UKOUG members can get in free, are on the event website. I’ll also post the slides on our Articles page once the event is over. Hopefully I’ll see you there.

The example that we will be using here involves 2 reports. One report created in BI EE and another report created in Hyperion Financial Report. The idea is to use the Action Framework of BI EE 11g to pass the parameters from the BI EE report to the Hyperion Financial Report.

The BI EE report is shown below. This is a very simple report containing the Yearly sales data from the Demo > Basic Essbase cube. The idea is to navigate to a HFR report by clicking on any Sales number and at the same time passing the context to the HFR report.

The target FR report is given below. This report also goes against the Demo Basic Essbase cube with Year dimension acting as a User POV.

To enable the integration, we need to first setup the ability in BI EE to navigate the EPM catalog. This is done by adding a registry entry in the ActionFrameworkConfig.xml file. This file can be found under {Middleware_Home}\user_projects\domains\bifoundation_domain\config\fmwconfig\biinstances\coreapplication. Open this file in a text editor and add the following XML registry tags

<registry>
<id>HDPreg</id>
 <name>Hyperion Directory Provider</name>
 <content-type>epm</content-type>
 <provider-class>oracle.bi.action.registry.epm.HDPRegistry</provider-class>
 <description>Hyperion Financial Reports Registry</description>
 <location> <path>http://venkatepm:19000/raframework/browse/listxml</path> </location>
 <service-access>
 <account>EPM</account>
 <propagateIdentity>false</propagateIdentity>
 </service-access>
 </registry>

Remember the PropagateIdentity statement determines how the authentication/authorization is done. This is a must and without this tag, the EPM navigation will not work. Also note down the path of the EPM catalog listxml. In EPM 11.1.2, the listxml url has changed to point to the RA Framework. Ensure that while installing EPM, RA Framework is also chosen.

In the above registry XML entry, you will notice that we are using an account called EPM. This is nothing but a pointer to an actual account in the Weblogic Credential framework. So, we need to navigate to the Credentials page in the BI EE FMW Control (http://localhost:7001/em) and setup the EPM account.

In the Credentials page, create a new Credential Map called oracle.bi.actions (if it does not exist already).

Within this Credential Map, create a new key called EPM. In the username and password section of the EPM key, enter the admin username and password that will be used to connect to the EPM catalog(in my case hyperion/welcome123). This username and password will be used only for the catalog navigation.

Now restart the entire managed server so that BI EE can pick up the changes. As you see, the entire process of integration between EPM and BI EE (from BI EE) has been made simpler and also now we can add as many registry entries as we want, depending on the number of EPM targets that we want to navigate to.

After the restart, create a new BI EE report as shown below

Go to the column properties of the Sales measure and add an Action Link navigation.

Create a new Action Link and choose Navigate to EPM content as the target.

This will basically open up the EPM catalog. Lets choose our target FR report.

This will automatically open up a parameter screen where most of the workspace specific parameters will be filled in automatically

Add a new Parameter called Year which is what we will be passing from BI EE to HFR. Make the Year value to be passed from the Year column

In the URL field, enter the URL given below

http://venkatepm:19000/workspace/index.jsp?module=@{module}&repository_path=@{repository_path}&elementType=@{elementType}&repository_name=@{repository_name}&repository_format_id=@{repository_format_id}&run=@{run}&attribute=Year.id.@{Year}&attribute=Product.id.Product&attribute=Accounts.id.Accounts&attribute=Scenario.id.Scenario&attribute=Market.id.Market&mimetype=@{mimetype}

Basically the CONTEXT values can be made to be passed automatically from BI EE to HFR by setting up the dimension mapping in the FR.properties file(we shall see how this works in a future blog entry). But in our case above, we are passing the parameters manually.

Now we should have the action link enabled on the Sales column. By clicking on the FR Report link, the parameters will get passed from BI EE to HFR automatically. With the above approach, there is no SSO between BI EE and EPM. So, the username and passwords will have to be entered again.

If the SSO is enabled, SSO_TOKEN can be passed across BI EE and EPM so that the navigation is seamless.

As you see, the Qtr1 value that was passed from BI EE has now set the context in FR. The same can be done for Planning forms, Web Analysis reports etc. I will cover the Planning forms integration with BI EE in a future post.

1. Dynamic References using CurrentMember, CurrentTupule etc are not supported currently (necessary if we want to run multiple allocations within the same dimension)
2. Hierarchical references like PARENT etc are not allowed in both calculations & allocations. These references are allowed for the parameters but not for the assignment scripts/allocation amounts.

The above 2 are significant drawbacks which i hope will be supported in future releases. Lets now look at a very simple example for allocation. I will be using the ASO version of the Demo->Basic cube for demonstrating this. The screenshot below shows the data that has been loaded into the cube.

If you notice, we have data for the Sales measure for all Regions except South. The idea is to allocate data into the South Region (both Sales & Cogs equally) based on the Total Expense values of the East Region. This is a very simple allocation example where our input data is at Level-0 but we are allocating from non Level-0. One good thing about this example is the fact that this shows that though ASO does not support input to non level-0 members, it supports allocation from them.

An Allocation in a ASO cube does not require a separate calculation script. All allocations are done through MaxLs directly. An allocation MaxL contains four main parts (in addition to other parameters which i shall not cover here).

1. POV – POV or a Point of View provides execution context to an allocation. A POV allows only reference to level-0 members. For every combination in the POV, the allocation will be executed once. POV is specified in the form of a MDX-Set.

2. Amount – This can contain a static member, Tupule or a constant. This does not support MDX expressions. In our example above, the amount will the (Total Expenses,East) Tupule. Ideally, Amount & POV should have all the dimensions in the cube.

3. Target – Target is specified in the form of a MDX Tupule. Basically it represents the region that will act as a target for the allocation. In our case, this will be empty.

4. Region – Region is a MDX set expression. This represents the target region where the data will be allocated. The dimensions specified in POV, Target and Region should be mutually exclusive as they all combine to form the target region. In the example above, Children(Total Expenses) & the Children (South) will form the Region.

5. Basis – This is commonly used when we are doing member based allocations. In our case, since we are using dividing the values equally (Spread allocation), this is optional.

So the final formula that i shall be using for achieving this is given below

execute allocation process on database DemoASO.Basic with
pov "Crossjoin(Descendants([Year],[Year].levels(0)),
Crossjoin(Descendants([Product],[Product].levels(0)),
Descendants([Scenario],[Scenario].levels(0))))"
amount    "([East],[Total_Expenses])"
target    ""
range        "CrossJoin({[Sales],[Cost_of_Goods_Sold]},
Descendants([South],[Market].levels(0)))"
spread;

As you see, Essbase has generated the necessary cells to accomplish this. As i mentioned last time in the Calculation blog post, the allocations/calculations are done separately and then the updated cells are pushed through external data load buffers. So, you would start noticing cells being loaded into incremental slices.

And if you look at the data, the South region would now have data with the values uniformly distributed.

Though this does work there are some inherent drawbacks. As ASO does not accept data at non level-0 members, the way allocation maxL’s are designed it makes it very tough(though possible) to do hierarchical allocations within the same dimension. Hopefully the introduction of dynamic member references and member functions should enable that in the future.

In this release, as i mentioned yesterday, the calculation scripts will have to be written in a separate text editor or the BSO calc script editor. There is no syntax validation available. So, ensure that you get the syntax correct atleast for the POV part.

Lets look at the various aspects that are involved in the creation of an ASO Calculation through MaxL.

1. Calc Script – This basically is a file (like the BSO .csc file) that will contain the actual calculation. The calculation currently does not support MDX member commands like Descendants etc. Only constant Tuples are supported. An example script is given below

([Bud Var],[Jan]) := 100;

In the above example, the calculation script basically assigns a value of 100 to all the cells with the matching Tupule.

2. POV – This basically is used during the execution of a calculation script. This determines the context of the calculation. For example, we can determine the combination of other dimensions not specified in the Tupule above. It is expressed as an MDX Set (not a Tupule). There is a very important difference between an ASO and a BSO calculation. If the POV includes the same dimension as the Tupule members in the calc script, then all the same dimensionality members in the POV will add themselves(can be controlled through Debit & credit members) and overwrite into the Calc Script Tupule Member combination. I will cover it in the examples below.

3. SourceRegion – A calc script can potentially have more than one calculation. For example, if you look at the calculation script below

([Bud Var],[Jan]) := ([Budget],[Jan]) - ([Actual],[Jan]);
([Bud Var],[East]) := ([Actual],[East]) - ([Budget],[East]);

This contains 2 separate calculations. Each tupule contains a different set of the dimensions. A source region basically is a MDX set which superset of all the tupules specified in the calculation script. This is needed as each line in the script is executed in parallel.

There are 3 other optional parameters like CreditMember, DebitMember & Offset. For today’s blog post, i will just cover the above 3.

Lets first start with creating a simple calculation script shown below

Basically all this script does is, it tries to assign a value of 100 to the ([Bud Var],[Jan]) tupule. After creating the calc script, save the script in the {ESSBASE_HOME}/app/DemoASO/BasicASO folder where DemoASO->BasicASO is my application & database respectively. Before running this script lets look at the statistics of the ASO cube.

As you see, this cube basically has no incremental slices. All the data is in the main slice. In the case of Block Storage cubes, when we assign a constant value to a member then the constant value gets assigned to all the existing blocks or will create new blocks depending on the CREATEBLOCKONEQ setting in the calculation script. But in the case of Aggregate Storage cubes, we do not have blocks. So it will be interesting to see what happens when we run the above calculation. To find out, lets run this script using the MaxL shown below

execute calculation on database DemoASO.BasicASO with
local script_file "/u01/app/Middleware/user_projects/epmsystem1/EssbaseServer/essbaseserver1/app/DemoASO/BasicASO/BudVar.csc"
POV "Crossjoin({[Jan]},Crossjoin({[New_York]},Crossjoin({[Stereo]},Crossjoin({[Sales]},Descendants(Scenario,Scenario.Levels(0))))))"
SourceRegion "Crossjoin({[Bud Var]},{[Jan]})";

As you see, all we are doing is we are running the calculation script against a POV context containing 4 cells.

The outline of the DemoASO cube is shown below for reference.

If you notice, in the POV we have 4 members from the same dimension (Scenario). And in the calc script we are assigning a value of 100 to a member in the Scenario dimension. Also, if you look at the data(before calculation) in the cube, we have cells populated for 3 of the Scenario members and not for the 4th one (Bud Var)

Now lets look at the data after running the calculation.

Though we assigned a value of only 100 to the Bud Var member, we now have a value of 400. The reason why this is happening is in our POV we have 4 scenario members included. Since we are assigning to value to only one scenario member, all the values of the 4 members get added and will be stored in the Bud Var intersection. Lets take a look at cube statistics again after running this calculation.

This is interesting. What we have now is 1 incremental cell in the incremental slice and the remaining 3 generated cells are in the main slice. So what has happened is, Essbase looks at the POV context and determines the number of cells to be generated. Out of these cells, it looks at the source of these 4 cells. Out of these 4, 3 already exist in the main slice. But 1 cell does not exist at all. So, it generates the new cell in the incremental slice. To validate this theory, lets change the MaxL as shown below and execute it

execute calculation on database DemoASO.BasicASO with
local script_file "/u01/app/Middleware/user_projects/epmsystem1/EssbaseServer/essbaseserver1/app/DemoASO/BasicASO/BudVar.csc"
POV "Crossjoin({[Jan]},Crossjoin({[New_York],[Boston]},Crossjoin({[Stereo]},Crossjoin({[Sales]},Descendants(Scenario,Scenario.Levels(0))))))"
SourceRegion "Crossjoin({[Bud Var]},{[Jan]})";

What we have basically done is, we have added one more member called Boston to the MaxL. Again, if we look at the Cube stats, we will see that there are 2 incremental cells and each cell will be in its own slice.

These 2 incremental cells are for New_York->Bud Var and Boston->Bud Var members which do not exist in the cube at all. The data in the cube will now have spread the value of 400 across both New York and Boston as shown below

In the case of BSO calculations, the calculations are generally driven through the existing blocks & the FIX’ed members (though we can alter this in different ways). In the case of ASO cube though, the control happens through the POV (and also Offset which i have not covered here) & the calc script. ASO generates new cells depending on whether that cell exists or not in the main slice. Of course, i have not tested this when all the input data in an incremental slice, but i would assume the behavior will more or less be the same. So far, i will have to say the performance is really good. I will be putting this through further test with a larger data set. It would be interesting to see if this can replace level-0 currency conversions that are quite common in implementations. So far, the only way to do that in ASO was either by using BSO as a partition source or by using Dynamic MDX hierarchies. I have covered both of them here and here. My first impression is, this can quite easily be used for currency conversions as well. I will hopefully cover my findings in a future blog post on this.

1. Using Calculation Manager to design the calcs and executing from there
2. Using MaxL and executing the scripts through MaxL

The second option above is actually more flexible since it allows to write custom MDX Tuples, sets easily within the calc scripts. I somehow felt that the calc manager has undergone a significant UI change primarily due to the introduction of ADF based UI. Though it does work, there still are some UI bugs which crept up here and there especially while trying to enter the calculations manually. One significant difference between calculation manager and MaxL is the fact that Calculation Manager follows more of a BSO based member set entries. It does not support MDX, atleast as far as i have tested. MDX is supported only within the calc scripts that run through EAS.

Lets look at a very simple example using both the above options. I shall be using an ASO version of the Demo->Basic cube to demonstrate this

Essbase ASO calculation execution now follows more of a HBR/Calc Manager syntax. That is, we start with defining a calculation script and then during execution time, we can specify the Point Of View of Execution, Member Offset, Credit/Debit members etc. A single calculation script can be executed in different point of views thereby providing different results. ASO calc scripts cannot be executed in isolation directly within EAS. They will have to be executed from within the maxL script editor. Also, the calculation script editor in EAS, does not support MDX. So for now, we will have to ensure that the scripts are absolutely correct as there is no way we can check the syntax validity from within the UI. I believe there is a bug in the EAS console, wherein if you get the MDX set scripts wrong, it will bring down the EAS server (atleast on my 64 bit console).

Lets start with logging into the calculation manager. Though IE 8 is supported, the latest version of Firefox is not supported for any of the ADF based EPM UI(similar to EPM 11.1.1.3). I had to downgrade my Mac firefox to a lower version to get this working.

I will start with creating a very simple rule called Budget Variance.

You will see that now the calc manager has ASO specific icons like Allocation, Calculation, Point Of View etc for executing ASO calculations. As with any calc manager calculation, we can specify a global range and then execute calculations.

With the introduction of ADF components, we can now drag and drop the individual components into the main palette. I will start with defining the Point of View first. I want the calculation to execute for the entire level-0 member combinations of the cube.

After defining the Point of View, lets drag and drop the calculation(Formula) on to the main palette. The calculation will be defined after the Point of View(or before depending on how we want the POV to be applied).

In the formula, lets define a simple calculation where we would like the Bud Var member to be having a value of 100.

We can have more than one formula defined in a calculation. But the difference between a ASO and a BSO calc script is the fact that the formulas are executed in parallel in ASO. We can serialize them by defining a proper flow within the calc manager. Lets now deploy this rule and execute it.

After execution, the cells within the ASO cube will be updated as shown below.

So, this does work & seems to work even for some complex cases within the calc manager. Lets put this to test further by using EAS & MaxL in the next blog entry. I have not tested what happens to the load buffers, data slices etc when we run these calcs. I will hopefully cover those as well as part of the EAS/MaxL based ASO calculation execution blog entry.

1. EPM System Installer
2. EPM Foundation Services (All the 4 parts are needed. 3 Parts are OS specific and the last part is OS generic)
3. EPM Essbase Server (both the parts are needed. The first part is OS generic and 2nd part is OS specific)
4. EPM Essbase Client (OS generic)
5. EPM Static Content (OS generic)

When you unzip the contents there are 2 things that you need to be aware of

1. The file size requirements have significantly increased in this release. So, if you are installing this on a VM, then make sure that you atleast have 30 to 40 GB of free space.
2. In EPM 11.1.1.3, we can just fire the unzip commands (in linux) and overwrite the files. But in the case of 11.1.2, one has to be extremely careful while doing the unzip. Since there are some binaries that are OS generic and some binaries that are OS specific, ensure that you unzip the Essbase Server, Essbase Client and Essbase Static Content files in separate directories. And then copy the assembly files to the main assemblies directory. Any mistake here, the installer will not recognize Essbase.

The folder structure within your assemblies install folder should have the following folders

There are some significant changes to the entire EPM framework. So direct upgrades are not possible. The main changes are

1. Shared Services does not use OpenLDAP anymore. Instead it uses a local database that is part of the Weblogic framework.
2. Apache Web Server has been replaced by Oracle HTTP Server and the Tomcat application server has been replaced by Weblogic server.
3. The base memory footprint required (RAM) has increased significantly due to the introduction of HTTP Server and Weblogic server.
4. There are some ADF based components that are used widely within the workspace. You can actually see how the entire Oracle product is converging using the common ADF framework.
5. The directory structure of the entire EPM 11.1.2 is completely different from the previous EPM 11.1.1.3 release. Essbase base folders have been pushed into the weblogic Middleware home directory.
6. Essbase clustering can be done directly during the installation process itself. There are significant changes in the Essbase ASO functionality as well.
7. Not a significant one, but the admin passwords can be changed as part of the install itself.
8. Data Direct drivers are not used anymore. EPM 11.1.2 now supports OCI by default.

The screenshot below shows the weblogic domain that gets created during the installation. This is the base of the entire EPM release now (Default install, other application servers are still supported)

As part of Essbase integration with Oracle SSO and Oracle Enterprise Manager, Oracle HTTP server is mandatory and should be installed prior to Essbase installation.

As i said above, Essbase by default is installed within the Middleware folder structure. This can be customized. Also, you can see the cluster setup can be done as part of the install process (i will cover this in a later blog entry)

And finally, workspace will be deployed & rendered through the Oracle HTTP server. All the other applications like APS, Planning, EPMA will be accessibly within the workspace through the Oracle HTTP server instead of apache.

You can notice that the entire shared services & workspace UI now have more of a ADF feel to it.

From an Essbase Administration services standpoint, the Java Web Start based console has undergone a small UI change. And of course, the features that have been added in EAS directly correspond to the features added in Essbase. I will cover them in the coming blog entries.

From my standpoint, the amount of changes in this release are so significant that this does not seem to be a .X release. I am just wondering how the EPM team managed to bring in so many changes within a short period of time (after the EPM 11.1.1.3 release) considering the fact that we have been waiting for BI EE 11g for almost a couple of years now. I am guessing this has been in the works for quite a while and i will have to say that the EPM team has done an extraordinary job in bringing this together. Atleast these are my first impressions. I will blog about some of the new features over the course of next week.

Some of the significant new features in this release are

1. Oracle Financial Close Manager
2. Oracle Essbase Analytics Link in HFM (was available in 11.1.1.3 itself but enhanced now)
3. Oracle Hierarchy/Dimension Management from within Oracle Applications
4. New Planning Modules – For Public Sector
5. Smart-View Enhanced – Lot more powerful now like closer Planning forms integration etc
6. Tasks integration with Outlook
7. Hyperion DRM – Completely Web based now
8. Clustering of Essbase
9. SSO with Oracle SSO in Essbase
10. Mgmt packs for Essbase
11. Footnotes & XBRL tagging in HFR
12. Enhancements to Hyperion Planning – Web Based Form Conditional Formatting etc

Not all the features are covered in the list above. But i will start covering them hopefully in the coming weeks.