‘Configurable dimensionality’ is the significant update to this version. We can probably say that this is the long awaited and biggest change that developers made to HFM.

So what is ‘Configurable dimensions’ is all about? There had always been a need for HFM customers to go beyond the four Custom dimensions, which are provided by default for their business needs. For e.g. they worked by stuffing two different details like Balance Sheet movements and Products details into a single Custom dimension. This was being addressed by Oracle in this release. Now, the users can have as many custom dimensions as required for their implementation. Let us see how this can be done starting by creating a new classic application.

Creating application:

Desktop Client: Many existing functions are cut down in Win32 client. Now, it has only Profile Manager and Metadata manager, rest all functions are available via Web including Create Application task. Profile Manager is used to define the application profile (.per). It can be installed in any machine since it can only be used in offline mode and has no dependency on HFM server component.

In addition to defining Year, Periods and Frequencies, we can now define the number of Custom dimensions as a part of the application profile. So basically, when you create application using this profile file, all these custom dimensions are created. The size (Small, Medium or Large) for a custom dimension must be determined depending on the number of members its hierarchy contains. Custom1&2 by default, are Large size.  As shown in the screenshot, we’ve created a total of 6 Custom dimensions.

Login to the Workspace to use this profile and create the application. As you can see, web server URL need not be provided.

I’ve loaded some sample metadata and data for few periods. !CUSTOM_ORDER section is introduced which specifies the order in which the custom dimensions are displayed. This has to be specified in the metadata load file.

Now, let’s skip the complexities and move the data up the Value dimension. At this point, I’m eager to see the application database structure.

Sub-cube Architecture:

Financial Management stores its data in database blocks called subcubes rather than in records. Subcube contains Page and Subcube dimensions. The Page dimensions are Scenario, Year, Entity, Value and the Subcube dimensions contain all the members of ICP, Account, View and Custom dimensions (in our case 6 custom dimensions). So typically, for a single Page dimension intersection, if we have 100 accounts and 10 valid members for each Custom dimension and all intersections are populated, then it would be 100000000 records.

HFM stores all this data for these dimensions in three tables:

• DCE (Currency Subcube)

Stores Entity and Parent Currency values and their adjustments.

• DCN (Parent Subcube)

Stores remaining value dimension members

• DCT (Journal Transactions)

Stores Journal Transactions and when posted, they are transferred to DCE (<Entity Currency Adjs>/<Parent Currency Adjs>) or DCN ([Parent      Adjs]/[Contribution Adjs])

In the below table, DPx_INPUT and DPx_INPUTTRANSTYPE are repeating fields for each period that the application contains. The columns are numbered from zero so for 12 periods, the column names will be DP0_INPUT to DP11_INPUT and DP0_INPUTTRANSTYPE to DP11_INPUTTRANSTYPE.

Compared to the older versions, the column structure almost remains the same. Interestingly, if you’ve noticed, in older versions, there are four LCUSTOMx columns one for each custom dimension. In this version, I anticipated that for each extra custom we add to the application, there would be a new column added to these tables like LCUSTOM5 etc. But the structure doesn’t even contain LCUSTOM3 & LCUSTOM4 and it is tempting to know how data for other Custom dimensions are identified. We are working to find out the same and any comments from the readers are really appreciated.

The supporting products for HFM viz. Financial Reporting, FDM, Application Upgrade Utility and existing HFM rule functions were also being updated to support HFM’s Configurable Dimensionality. Although we have the flexibility of adding any number of custom dimensions, time taken for calculations, consolidation and other performance issues need to be considered and application should be designed accordingly.

UI Enhancements:

The new User Interface for HFM is definitely a notable update when compared to the earlier version.  It is easier to navigate and includes some special features as well. This is an outcome of migrating EPM components like HFM and Planning to Oracle’s ADF (Application Development Framework).

Let’s take a look at all these UI enhancements –

Multiple applications in Workspace:

Different HFM applications can be opened at the same time.

Multiple modules in an application:

Different HFM application modules in each application can be opened simultaneously.

POV enhancements:

There are significant changes to the way we select dimensions to Rows and Columns. We can relate this approach to Hyperion Financial Reporting wherein we ‘drag and drop’ dimensions to Rows and Columns in dimension layout. This approach is simpler and made common to both data grids and data forms. We can also add dimensions manually in data forms.

Data grid enhancements:

Again, we can relate the creation of data grids to how we create the reports in Financial Reporting. In HFR, we create/design the report and run the report to view it. Similarly, we have Grid designer and Grid Viewer in data grids.

Display options were docked to the right hand side thus decreasing the time wasted on navigation.

Another new feature is the indication of cell colors at the bottom of the Grid Viewer. This will be very helpful to the business users and they don’t need to reach support or documentation to understand the Cell Colors.

Data form enhancements:

Member selection process is pretty much similar to what we do in data grids. And there exists designer and viewer in data forms too. Export to/Import from Excel options were disabled.

Favorite members in Member selection:

Frequently used members can be selected and saved as favorites. These are available across other modules of the application as well.

Loads and Extracts page:

Load and Extract tasks like Security, Metadata, Member Lists and Rules are consolidated in one single page.

Journals Module Enhancements:

Journal tasks functionality is pretty much same but with major changes in UI again. Journal reports module exists both as a separate task and as a part of Manage Journals (opens in a new window when clicked).

Intercompany transactions module is not available and will be included in the upcoming patch – 11.1.2.2.101. It might take some time for the old version users to get used to navigating the new interface. Hope we can leverage from these ADF features. There are other enhancements to the HFM Win 32 client and Custom dimension configuration, which will be covered in a different post.

Using the same example, I’m going to provide a simplistic explanation of what’s happening inside the black box that is RTD.  There’s more happening in the box than explained here, but hopefully you’ll get an idea of how the solution works.

Let’s say we have four products available a DVD, Book, CD and Video Game.  Obviously, in reality we may have thousands of different products that will come under different product categories and sub-categories.

A customer visits our website and gets to the point of requiring a product recommendation.  RTD could first check availability of stock, as there is no point recommending products that are not available for purchase (yes, I appreciate you might want to allow people to pre-order for when available, but as I said this is a simple example!).  In this case, the video game is out of stock, leaving RTD to make a decision from the remaining three products.

When RTD is first switched on, it needs to learn before it can make recommendations.  In this example, we are going to have three bits of data, the customer’s age, day of the week and customer’s country of residence.  In the example, there are also a limited number of values for each field.  Day of the week, for instance, can only be Monday or Friday (should have been Saturday and Sunday if I’m going to make it a two day week!).  So the first customer from the UK, comes to the site on Friday and is 18 years old.  This customer purchases the DVD, so as you can see from the diagram below, these customer attributes are stored against the DVD.

The next diagram below shows the second customer, aged 30 from America, visits the site on Friday and purchases the CD.  Again, you can see that the attribute values for the second customer are stored against the CD.

Customer number three then purchases the CD, with their attributes being stored appropriately as shown in the pic below:

So I guess you get idea by now.  So after 50 customers, of which 15 have purchased the DVD, 15 the book and 20 the CD, the attribute values against each product look as follows:

Now RTD has sufficient learning, when the next customer comes to the website it can predict the product they are interested in.  It does this by scoring the customer’s likelihood to purchase each product.  So when the 51^st customer, who is 50 and from the UK, visits the website on Monday, they get the following likelihood to purchase scores:

As you can see from the above the product with the highest likelihood of purchase for this customer on this day is the book.  As well as returning this recommendation to the website, RTD continually learns and hence will self-adjust to changes in customer behaviour.  RTD scores each product or choice, and hence from a front end display point of view you could show the highest ranked product or the top ten in purchase likelihood order.

When new products or new customer attributes are added, RTD will automatically start learning on these and recommending as appropriate.

Hopefully, this has given you some idea of how RTD works.  In my next post, we’ll take a look at RTD components and hardware requirements.

The Rittman Mead BI Forum this year is running for four days at both venues, with the first day an optional OBIEE11g Masterclass by Kevin McGinley, and the final day being a BI Developer Day, held under NDA (non-disclosure agreement) in conjunction with Oracle BI product development. Around thirty people arrived either on Monday night or early Tuesday morning for Kevin’s masterclass, which covered topics such as the OBIEE 11g Action Framework, Oracle Scorecard and Strategy Management and Oracle BI Mobile. Kevin won best speaker award at last year’s US BI Forum event, and leads a team of developers and consultants at Accenture delivering OBIEE solutions around the US. Here’s Kevin delivering his session, talking in this case about use-cases for the Action Framework beyond the typical “invoke marketing campaign from a link on the dashboard”:

A big part of the BI Forum is the social and networking aspect, with in the case of the Brighton event around 75 BI developers from around Europe, the USA and India attending this year. An hour or so after the close of Kevin’s keynote we assembled in the bar for the drinks reception, and then suitably lubricated convened back in the seminar room for the Oracle keynote, delivered this year by Alan Lee (Oracle BI Server PM) and Mike Durran (Oracle BI Principal Product Manager, looking after aspects of Exalytics). It was great to get Alan over as he’s in the past been responsible for Essbase, and now he’s looking after what is arguably the single most important component in the OBIEE architecture. Without breaking too many confidences, there was certainly a lively debate around Exalytics and the upcoming product roadmap, and this was a great venue for candid and informed discussions around Oracle’s product strategy.

The main core sessions for the conference then started on Wednesday morning, with Tony Heljula opening the proceedings with a session on OBIEE 11g security, followed later in the day by Adam Ferrari (ex-Endeca CTO) on Oracle Endeca Information Discovery, Ayse Oztop on Oracle Scorecard and Strategy Management, and John Hollifield (Nominet) along with James Coyle and Adam Seed (Rittman Mead) with a customer case study around reporting packs and OBIEE 11g. One new thing we tried for this year’s event was to run a number of ten-minute sessions, using the Ignite format (10 minutes, 20 slides that auto-advance every 30 seconds) on Wednesday and the TED format (10 minutes, no slides) on the Thursday. I think these worked – of course a lot of it was down to the presenter and the topic – but it was an opportunity to cover a lot of topics, and for speakers who didn’t have the time to create an hour’s worth of material but who had an interesting idea to get across. Thanks again to everyone who presented over the two days, and there’s a Flickr photo set here – Rittman Mead BI Forum 2012, Brighton – if you want to take a look at photos of the presenters and the sessions. Congratulations also to Oracle ACE Emiel van Bockel from Centraal Boekhuis in the Netherlands who won the Best Speaker award for the best one-hour session, and Tony Heljula (Peak Indicators) and John Minkjan (Ciber) who tied as Best Speakers for the 10-minute sessions.

Another new idea we tried this year was to have a couple of guest speakers, coming from outside of our normal Oracle BI technical area. On the Wednesday evening therefore, before the main meal and after our main sessions for the day, we had Debra Lilley, Oracle ACE Director, Oracle Alliance Director for Fujitsu and President of the UK Oracle User Group, present to the forum on BI within the Oracle Fusion Applications. Debra has been working closely with Oracle through leading the Oracle Product Development Committee for the International Oracle Users Group Community (IOUC), and had a great story to tell the audience around OBIEE and Essbase are deeply embedded, and central, to the Fusion Apps suite.

I was keen to have Debra talk for a couple of reasons; firstly, I think it’s good to bring in people who work outside the core OBIEE tech area and Debra is an accomplished speaker and advocate for users. Secondly though, it was particularly interesting to hear how all of the Fusion Middleware components that OBIEE 11g has adopted, such as Oracle Platform Security Services, Oracle WebLogic, Oracle Enterprise Manager and the Action Framework that links into them, are actually there because the Fusion Apps are the primary customer for OBIEE and it’s in their set of requirements, with the spin-off benefit for us in that we get all of these enterprise features as part of the standard product. There is a method to the madness, so to speak, in all these new Fusion Middleware components coming into OBIEE, although it has to be said that there were several calls for a more simplified, cut-down version of OBIEE suitable for SMEs (something like the new BI Publisher trial version launched last week), maybe with a lightweight application server and without some of the complexities around security and so on, to compete with vendors such as Qliktech.

Qliktech, and their Qlikview product, came up in conversation earlier in the day when Adam Ferrari, now a VP within Oracle but previously the Chief Technology Officer for Endeca, presented on Oracle Endeca Information Discovery and the Endeca Server (aka MDEX) that powers it. Adam’s a great speaker and stuck around for the whole conference, and he’s also coming over to Atlanta next week to present the same session for the US audience. The development process for the Endeca Server is quite unlike that for traditional BI projects, with much less time spent on the data modeling part and a set of UI components that let you develop “search/analytical” applications using both structured, semi-structured and unstructured (almost) data.

Adam was also able to share a bit of the Endeca roadmap with the audience, including some initial thoughts on how the Endeca Server can leverage OBIEE’s semantic layer, and how the BI Server might be able to access Endeca Server data, and how Endeca Latitude’s style of search/analytical UI components are being rewritten to use Oracle ADF DVT technology. If you’re interested in the Endeca Information Discovery story, we put together a week-long series of articles on the product a few months ago including this article on our thoughts on how Endeca will fit into the Oracle BI/DW product line, which also has links to all of the other postings.

Wednesday night was the gala meal, this time at the Havana Restaurant in Brighton, with delegates traveling into town after Debra’s keynote and visiting our new offices on the way for a champaign toast. Thanks to Becky, Alex and everyone else back in the office for organizing the events this night, and also the various social events over the rest of the week. Here’s a few photos from the meal, and from events during the rest of the Brighton BI Forum:

Our second guest speaker was on Thursday, and in a bit of a departure from our normal topics we had Chris Webb, a Microsoft MVP and owner of Crossjoin Consulting and Technitrain, to talk to us about the Microsoft BI stack. I’ve followed Chris’ blog for some time and we’ve had kind-of parallel careers, with Chris specializing in Microsoft Analysis Services and Microsoft BI, whereas of course I’ve focused more on Oracle BI. I thought it’d be interesting to have someone come along and show how they do a similar job to us but with a different set of tools, and tell us what they thought was good about the product, what didn’t work so well and how they typically approached projects. It was a bit of a gamble as to whether it would work, but it was actually one of the best received sessions of the week and helped us put Oracle’s product direction, explained in detail during the final session on Friday, in context. I was particularly interested to hear Chris’ take on the various changes happening to Analysis Services metadata over the past few years, introducing more relational and in-memory features, something Oracle of course did with Oracle OLAP / Oracle Express back in the 9i days, with mixed success (to put it diplomatically).

So the main BI Forum sessions finished on Thursday evening, but for around twenty of the delegates plus around five or so Rittman Mead staff, there was an extra day at the end which we’d organized in conjunction with Oracle BI product management. Held under non-disclosure, Alan Lee and the product team took us through a more detailed look at the product roadmap, and looked at a couple of particular focus areas that are going to be key to the product going forward. Apologies to anyone who wanted to come to this day but missed out because of number restrictions, but we’re running the same thing again at the Atlanta event next week, and hopefully on the Friday after this year’s Openworld – thanks again to Oracle for making this possible, it was very well received.

Anyway, we’re off to the States on Monday to do the same thing again, running the US BI Forum at 200 Peachtree in downtown Atlanta. Once both events are over we’ll post all of the presentations for public download on our website, and you can follow the proceedings using the Twitter hashtag #biforum. See you all in Atlanta!

Screen is one of those pieces of software whose name makes it inherently difficult to Google for, so you’ll often see it referred to as GNU screen. It is described as a “Screen Manager” – which for me doesn’t really describe its rich functionality and goodness.

Here are some of the many reasons why screen is great:

Session persistence

Reconnect and pick up exactly where you left off. Whether you’re on a dodgy wifi connection that’s dropped out, or you’re in the office and need to reconnect to your session when you get home, screen keeps your session running regardless of your SSH connection and lets you reconnect to it at any time. Whilst you’re disconnected, the processes associated with your session are still executing.

You may wondering why I wouldn’t just use nohup ... & for keeping processes running to survive a disconnect (a SIGHUP, hence NOHUP). With nohup ... & , you have to decide in advance that you want to run the process in the background, and you can’t easily interact with it once it’s there. To do the same in screen you just run the process, and then reconnect to your screen session when you want to.

Think of it like auto-save on steroids for your ssh session. If you use no other functionality in screen, this alone is reason to use it.

Multiple sessions in one

So you could run multiple copies of puTTY, but why not connect once, and have multiple sessions within that one? Often you’ll want to have a WebLogic process, bi_server process, opmn, maybe top, and sqlplus. If you create five puTTY connections, and they drop, you have to reconnect all five and restart what you were doing. If you are using screen, you just reconnect and resume the screen session, and it is literally as it was when you left it. If you’re halfway through a masterpiece of SQL query development or execution in sqlplus, you’ll appreciate being able to go straight back into it, and not start a new sqlplus session.

Each session can be named, with a “task bar” across the bottom of the screen to show which sessions there are and which you’re currently looking at

Scrollback

Most ssh clients offer scrollback history, but it’s native to the client, and relies on you being connected to the session.

With screen you can scroll up and down a session’s history from the keyboard, and see whatever’s been output whether you were connected to the session at the time or not

What’s this got to do with OBIEE?

OBIEE is not a simple beast, particularly when you are setting it up and deploying it. Being able to have multiple sessions on the server is invaluable. For example, tail the server log in one session and start up a process in the other, whilst running top to monitor system usage in a third.

As you may well know, the WebLogic processes (AdminServer and Managed Server) do not run as daemon (background) processes when started from the command line. On Linux this means you either have to run them with nohup ... & or as a session within screen. If you run it with nohup ... & you would be sensible to redirect output to disk so that you can see the console output. So to see what’s going on with the process, you need to find the output file and tail it. If you’ve run it with screen, you just need to switch to that session and the output’s there to see. In addition, you can see at a glance what is running, without having to start looking at process lists, as you would have to for a process started using nohup.

Sounds good, where do I get it?

screen is installed by default on lots of Linux distributions (including Exalytics’ OEL5) – just type screen -v to see. If it’s not, then a quick Google should turn up how to install it for your particular Linux/Unix distribution. It’s almost certainly available on any Linux/Unix variant (and certainly any on which OBIEE will run!).

What’s the catch?

screen has a steep learning curve, there is no denying it. But when you reach the crest of that curve, the warm glow of enlightenment as the power of it dawns on you is a rewarding one ;-)

The main thing to get your head around is the keyboard navigation. It’s pretty much all based on pressing Ctrl-A following by another key sequence. If you get stuck, do Ctrl-A and press the question mark key ?.

So how do I use it?

First off, I would strongly recommend setting up the .screenrc file – this configures the “toolbar” at the bottom of the window. If you don’t do this, then you don’t have the visual indication that screen is even running, let alone which session you are in. The file goes in your home directory (which is what the tilde ~ means). Note that the first character of the filename is a dot.

Put the following in ~/.screenrc :

# Minimal .screenrc
# RNM 2012-05-11
#
hardstatus alwayslastline "%{= RY}%H %{kG}%{G} Screen(s): %{c}%w %=%{kG}%c %D, %M %d %Y "
startup_message off
msgwait 1
defscrollback 100000 

This is a one-time configuration step. If you look on Google you will find plenty of examples of pimped-out config files doing lots of funky things.

So now type screen, and you should see something like this:

Now I’m going to walk through how I would use it to start up and working with an OBIEE instance. So the first step is start WebLogic AdminServer. Note that I’m not using nohup

 cd user_projects/domains/bifoundation_domain/bin
./startWebLogic.sh 

Press Ctrl-A and then Shift-A. You’ll be prompted to change the title.

 cd user_projects/domains/bifoundation_domain/bin
./startManagedWebLogic.sh bi_server1 

 cd wlserver_10.3/server/bin/
./startNodeManager.sh 

Switch to the opmn window, by pressing Ctrl-A and then Ctrl-N to move to the next window, and start opmn and the system components:

 cd instances/instance1/bin
./opmnctl startall 

Whilst opmn starts, cycle through all of your windows (Ctrl-A and then Ctrl-N to move to the next window) and observe that you can see the status of each process that you’ve started.

Finally, create a fifth window (Ctrl-A, Ctrl-C) and call it “sar”. Type sar 1 600 which will run sar for ten minutes. Note the times on the left column, and switch to the previous screen (Ctrl-A, Ctrl-A). Wait a few seconds, and switch back (Ctrl-A, Ctrl-A). You’ll see that sar has been running all the time, whether you were on the screen or not

To prove the point, close your ssh session and then reopen it, and resume your screen session (screen -x). You’ll see that the processes have been running all the time, whether you are connected or not.

So, to summarise, I now have a screen session with:

• The control of each process available directly from each window
• The output of the processes which write to stdout
• Additional windows for monitoring (sar, collectl, etc), running sqlplus, etc

I can disconnect any time, and I know that the processes are still running, and I can reconnect whenever I want. I don’t have to go furtling through folders to find my nohup.out files. If a colleague needs to take over admin of the server, they too can just connect with screen -x and see the same session.

Finding out more about screen

What I’ve described here is really only scratching the surface of what you can do with screen. It’s a very popular tool, and hence widely written about. Some links which I’ve bookmarked in the past are:

• http://www.gnu.org/software/screen/
• http://www.softpanorama.org/Utilities/screen.shtml
• http://aperiodic.net/screen/quick_reference
• http://magazine.redhat.com/2007/09/27/a-guide-to-gnu-screen/
• http://www.linuxjournal.com/article/6340?page=0,0
• http://www.karan.org/blog/index.php/2010/01/06/this-is-my-screenrc-whats-yours

A typical Exalytics server (in the supported configuration) has the following software installed on it:

• OBIEE 11g in an Enterprise Install with WebLogic Server
• Oracle Essbase Server
• Oracle TimesTen for Exalytics

This allows you to run a single repository (RPD) and catalog within the single BI instance provided by the OBIEE 11g install, with a product architecture that looks like this:

The installation itself took place over two days, with the first day used for racking up the server in the datacenter, and the second for installing the software. The server itself arrives from the factory with just the Oracle Linux OS installed, and Oracle ACS then set up and install the OBIEE, WebLogic, TimesTen (and if you’re using it), Essbase software, configuring an initial TimesTen database and then connecting it to the Oracle BI Server. It’s then down to the customer (or a partner such as ourselves) to connect TimesTen to your BI repository, enable usage tracking and summary statistics, and start running the Summary Advisor to create the in-memory aggregates.

As we’re primarily using the Exalytics Server for customer PoCs, we’ve set ours up a bit differently to the supported configuration, with four OBIEE 11.1.1.6 installations in separate middleware homes so we can run BI instances simultaneously. We’re also planning on installing Endeca Information Discovery on the box, so we can take advantage of the memory on the server and upcoming integration with OBIEE. Essbase will also be going on there shortly, and we’ve also put an Oracle database on there, with one instance for the RCU schemas, and others for customer PoC data as required. Finally, access is through a VPN and firewall, allowing us to lock down the machine but make environments available as required, using a Juniper firewall/router.

Note that that is not a supported configuration, and it’s something we’ve put together to test options for running multiple repositories and catalogs at the same time – don’t take this as Rittman Mead guidance on how to host multiple BI instances on the same box. That said, we’re thinking one of the major use-cases for Exalytics will be server consolidation, so it’ll be useful to test out how practical it is to run multiple BI instances on the same box.

We’ve also got a testing area set aside for ourselves, so look out for blog postings over the next few months as we put Exalytics through its paces. We’re particularly keen to work out the “sweet spot” for TimesTen in-memory aggregations, particularly in comparison to aggregations in the in-memory Essbase cube, and work out just how practical it is to create a “hotspots” cache for a very large data warehouse, and keep those cache entries fresh as new data comes in. As the Exalytics releases come in, we’ll also be trying out the new visualizations and also integration with products such as Endeca Information Discovery.

In the meantime, we’ve been giving the box a spin using some demo datasets, including one that contains all scheduled airline flights in the USA over the past thirty years, and details of any delays that they incurred. The dataset contains around 120 millions rows of data and is analyzed across multiple dimensions including route, airline, origin and destination airport, and time. Even thought there’s a vast amount of data in the back-end database, Exalytics provides a split-second, speed-of-though analysis environment with the TimesTen for Exalytics In-Memory Database working as the mid-tier, in-memory cache.

We’ve also got a bunch of demos built around the new SampleApp to show off the Summary Advisor, and a demo that simulates thousands of concurrent users running random queries with no drop in response time. Keep an eye on our Exalytics page as we post videos of these, and other demos, over the next few months. Also, if you’re coming to either of the Rittman Mead BI Forum events running in Brighton this week, and Atlanta next week, I’ll be running demos at various points.

Finally, if you’re interested in finding out how Exalytics can make your organization’s dashboards run faster and provide a richer, more interactive query environments, drop us a line at exalytics@rittmanmead.com to give your dashboards a spin on our test box!

much, much, later…

Right got one!  Let’s use a well-known use case as hopefully everyone can associate with it.  We have a website from which we sell a range of products.  We want to recommend products to customers on the website at the checkout stage of their purchase.  We target additional products to customers at this point so as not to interrupt their original purchase flow.   The solution using RTD would look something like this:

There are a number of data inputs we could take into consideration for the decision.  Inputs into RTD can come from multiple sources with various refresh rates.  In this example we have a number of inputs providing RTD with the information from which to make a decision:

• Data from the website, providing context information about what the customer is doing on the site in their current session.

• Customer data.  This could be from a transactional database and/or data warehouse.

• Product information, including availability.

You can integrate RTD easily with other systems through the use of Java, .NET and JSP Smart Clients or via Web Services.  So in the above diagram, the Java Smart Client could be used to pass context information from the web session into RTD.  We’d load the customer and product information into the RTD database.  We might choose to load the customer data every 24 hours and the product information every few minutes in order to pick up information on stock levels and any new products that become available.

In the example, the website would use the Smart Client to ask for a recommendation by passing in a customer identifier and would receive back a product reference.  This product reference would then be used to present the relevant image on the website.  The Smart Client would also be used to pass success feedback to RTD for learning purposes, this success feedback could be the purchase of the recommended product or someone clicking on the image of the recommendation.

There’s a few other things we could also do.  From the front end we could pass information to a web analytics tool by tagging the images of recommendations.  In this way we can analysis the impact on customer journeys.  We could also take information from the RTD tables and load it into a BI solution so that we can combine the information from RTD with other BI data.

One final thing, it’s quick, with response times for decisions measured in milliseconds.  Obviously, critical in the example used, when making real-time product recommendations to customers on your website.

In my next blog, we’ll look into the black box that is RTD on the diagram and explain how RTD makes the decision on which products to show a particular customer.

The scenario I’ll use is one that I went through in a previous post on Extreme BI using GoldenGate and ODI. In a typical data warehouse, replication will be performed from the Source to both the Staging and Foundation layer databases. Both databases hold tables that are duplicates of those in the Source, with the addition of several columns in the Foundation tables for tracking history.

The columns used for history capture in the Foundation database are as follows:

EDW_SCN (System Change Number)
EDW_COMMIT_TIMESTAMP (date/time when the change was committed)
EDW_TRANS_TYPE (type of change made)

The example Foundation layer Model, fittingly named FOUNDATION_EXAMPLE, has 45 Datastores which will need these 3 columns added. Not an overwhelming number of tables, but adding columns to each Datastore would definitely be monotonous and time consuming. Monotony is a part of nearly every job (except maybe an air traffic controller), but time, and remaining budget, is precious to every project. This approach will save both, allowing for the addition of columns to any number of tables within seconds.

To get started, open up the ODI 11g client and connect to a work repository. In the menu bar, click on Tools → Groovy → New Script. This will open a new .groovy extension script within the ODI user interface. When saving, you can rename the file and choose a new file location. The final script, found here, can then be pasted into the new script window and executed.

We begin by adding code to allow for the display of all available Models in the repository. With Groovy syntax we create a function named captureInput. This code will use the IOdiModelFinder Interface to get a list of all Models in the repository, adding the list to an array. The array is then displayed in a combobox for user selection.

def captureInput() {

  txnDef = new DefaultTransactionDefinition();
  tm = odiInstance.getTransactionManager()
  txnStatus = tm.getTransaction(txnDef)

  models = []

  modelFinder = (IOdiModelFinder) odiInstance.getTransactionalEntityManager().getFinder(OdiModel.class);
  modelM = modelFinder.findAll();
  conItr = modelM.iterator()

  //add each model to the models array.
   while (conItr.hasNext()) {
     mod = (OdiModel) conItr.next()
     models.add(mod.getCode())
   }

  tm.commit(txnStatus)

  Model m=new Model()
  d = new java.awt.Dimension(205,20)
  m.toItems = new DefaultComboBoxModel(models as Object[])

  def s = new SwingBuilder()
  s.setVariable('myDialog-properties',[:]) 

  def vars = s.variables
  def dial = s.dialog(title:'Add Columns to ODI Model',id:'myDialog',modal:true) { 

    panel() {
        boxLayout(axis:BXL.Y_AXIS)
        //display the models in a combobox.
        panel(alignmentX:0f) {
            flowLayout(alignment:FL.RIGHT)
            label('Model Code:')
            comboBox(id:'modelCode', 'model': bind {m.toItems}, null, preferredSize:d)
        }

        panel(alignmentX:0f) {
            flowLayout(alignment:FL.LEFT)
            button('OK',preferredSize:[80,24],
                   actionPerformed:{
                       vars.dialogResult = 'OK'
                       dispose()
            })

            button('Cancel',preferredSize:[80,24],
                   actionPerformed:{
                       vars.dialogResult = 'cancel'
                       dispose()
            })
        }
    }
  }

  dial.pack()
  dial.show()

  //return the selected model.
  return vars
}

Once the user clicks the OK button, the selected Model Code is passed as a parameter to the the addColumns function. We then find the Model object, loop through the Datastores associated with that Model, and add the 3 EDW_* columns. A future enhancement to this code might be to capture user input on each column to be added, including name, datatype, and length. To take it a step further you could first select from a list of technologies, then choose the technology appropriate datatype for each column. For this example, I’ll keep it simple since I know the specifics of the columns I want to add.

def addColumns(modCode) {
  txnDef = new DefaultTransactionDefinition();
  tm = odiInstance.getTransactionManager()
  txnStatus = tm.getTransaction(txnDef)

  modFinder = (IOdiModelFinder) odiInstance.getTransactionalEntityManager().getFinder(OdiModel.class);
  mod = modFinder.findByCode(modCode);

  Collection dataStores = mod.getGlobalSubModel().getDataStores();
  dsArray = dataStores.toArray(new OdiDataStore[0]);  

  for (int i = 0; i <= dsArray.length - 1; i++) {
    //Get the DataStore.
    OdiDataStore ds = null;
    ds = dsArray[i];

    //Add the columns.

    OdiColumn col = new OdiColumn(ds, "EDW_SCN");
    col.setDataTypeCode("NUMBER");
    col.setMandatory(false);
    col.setLength(20);
    col.setScale(0);

    col = new OdiColumn(ds, "EDW_COMMIT_TIMESTAMP");
    col.setDataTypeCode("DATE");
    col.setLength(23);
    col.setMandatory(false);

    col = new OdiColumn(ds, "EDW_TRANS_TYPE");
    col.setDataTypeCode("VARCHAR2");
    col.setMandatory(false);
    col.setLength(30);

    odiInstance.getTransactionalEntityManager().persist(ds) 

  }
  tm.commit(txnStatus)
  return mod
}

To execute the script, click the green “Play” button on the ODI toolbar. After selecting the appropriate Model from the list, click the OK button. The script should take just a second or two, depending on the number of Datastores in your Model. Once completed, a quick look at the one of the Datastores and we see 3 new EDW columns, ready to capture transactional history.

Enjoy your Groovy scripting!

UI Enhancements:

ADF branding is a lot more apparent with the 11.1.2.2 release. For example, all forms now use the native ADF based prompts, POVs and selectors. Also, the UI has been enhanced to give “Concertina” style menus so that it is easier to access related & relevant objects

Adhoc Grids use more of ADF features like swapping columns from rows to columns, pages to columns etc. Adhoc grids are more like BI EE 11g in UI.

Cell Data History:

This is one feature that planners and people who are part of the approval chain will really like i.e. showcase who has modified what data. Generally when plans are submitted for approval, the approver has to go through all the cells and find out what has changed manually. But with this feature its a lot easier to find out who has modified what cells. This can result in faster approvals and reduced effort spent.

For this to work Auditing at Cell Data level needs to be turned on.

Charts in Planning Forms:

Another key feature is in the ability to display the forms as charts in a Composite form. This way planners can immediately visualize the change in data.

More important drills are available on graphs as well – Now we need this capability in BI EE i.e. here the drills are all native Essbase drills (unlike in BI EE where we have to go through the RPD).

Grid Diagnostics:

Another significant new feature is the ability to find out poorly performing/designed forms by using the Grid Diagnostics feature. This will let us know the page load times, number of row retrieved, suppressed etc. Pretty handy tool especially for Administrators

Substitution Variables Management from Planning:

Another excellent & long-pending feature is the ability to manage & update the substitution variables used per plan type directly from Planning App. There is no more dependency on Essbase/EAS (of course they still reside there but no need to create there first in EAS) to create the variables

Predictive Planning & Crystal Ball:

Another very interesting new feature is the ability to predict future plan values using Crystal Ball (or called as Predictive Planning now). It is natively integrated into Smart View. By just clicking on a cell within a form, one can start predicting what will be the plan values. Crystal Ball provides a comprehensive set of statistics as well while providing a recommended value (on what basis the recommendation was done).

In addition there are other small enhancements like Group Based Approvals, Rolling Forecasts (Setup at form level), Multiple cell document attachments, De-support of Business rules etc. Also, from an architecture standpoint we can now have multiple Planning Managed Servers in a cluster thereby making it truly Highly Available. In the next post, we shall see the new features in HFM 11.1.2.2.

You may have heard of Oracle Real-Time Decisions (RTD), but aren’t fully aware of what it’s about.  Why you are reading my waffle is that I’ve led a number of RTD implementations.  Additionally, I’ve been fortune enough to spend time with other RTD customers and with the RTD product team, so in theory I should have some knowledge to share.

I thought you might be interested in finding out more as the product is going down a storm with customers that have implemented it.  For example, a global financial services organisation with 10 million customers visiting its website in one month generated 100% conversion uplift, and one of the largest US retailers with over 70,000 products achieved a 25% lift in sales rates compared to control.

So, what is this RTD?  Oracle say… “The RTD platform combines both rules and predictive analytics to power solutions that provide real-time automated decisions”.  I think of RTD as a Q&A box, which can answer any question.  In order to do this, I need to provide the system with all the possible answers or choices and input data (for example, customer data).  Each time the system is asked for a decision it will provide the most relevant choice by scoring all of the available choices based on the input data.  I can automate the upload of the inputs and the choices (for example, by integrating with CMS solution for the available banners and associated metadata).  Sounds simple, but under the covers there is a ton of clever stuff going on and lots of clever solutions you can provide through its use.

Having the inside track on this product, I’ll be taking a few articles to share with you how RTD works and why I think it’s so interesting.  Here’s a glimpse of some of the topics we will be covering:

• Potential uses for RTD
• Why Oracle RTD
• When not to use
• Hardware requirements and technical integration
• Project timescale and resource example

We’ll also be getting under the covers of Oracle RTD and showing that you can get inside the magic black box to understand how and why it’s making particular decisions and how you can learn from the data and correlations that RTD makes and use this information within a wider business context.