The most obvious change in the 11g Discoverer release is the move away from Oracle Application Server, and OC4J, to start using Oracle Weblogic Server which is now the standard within Oracle Fusion Middleware. Weblogic performs much the same function as Oracle Application Server in that it’s a J2EE application server, so in that regard things aren’t much different when it comes to the end-user elements of Discoverer – Discoverer is now just running in a different J2EE container than it did before. From the application server administration perspective though things are a fair bit different, and the installation itself differs significantly from the 9iAS/10gR1/10gR2 versions.

Whilst there are potentially a lot of Fusion Middleware 11g components you can download, to get a bare-bones Discoverer 11g system up and running you need to download:

• Oracle Weblogic Server 11gR1 (earlier versions might be supported, I’m not sure)
• Repository Creation Utility
• Portal, Forms, Reports and Discoverer

All of these components are available for Windows, Linux and Unix. Unlike previous releases of Oracle Application Server, you need to install the application server first, like this:

In addition, you’ll need an Oracle 11gR1 database to hold the repository metadata used by the Portal, Forms, Reports and Discoverer components, which needs to be patched up to 11.1.0.7 before you can run the Repository Creation Utility.

Running the Repository Creation Utility gives you the option to create repository schemas for the various components within the wider Fusion Middleware 11g stack, but for now I’m just going to create a schema for the Discoverer application metadata.

If you’ve ever installed Oracle Application Server’s previous repository into a standalone database, rather than use the packaged database that comes with the Application Server installer, this is a very similar process. What Oracle have done here is decouple the application server itself, the repository creation and the database it goes into, into separate parts. Presumably at some point in the future you’ll be able to store Fusion Middleware’s repositories into databases other than Oracle, but for now you’re restricted to an 11.1.0.7 version of the Oracle Database (though Discoverer itself can still report against 9i, 10g and 11g Oracle databases).

Once you’ve installed Weblogic and created the repository schemas, you can start the Portal, Reports, Forms and Discover installer (see what great company we keep). This lets you install all or just a selection of these components, like this:

If you want to install Portal as well, you need to use the 10.1.3.4 Oracle Identity Management installer to create an OID installation, then you need to set up Single Sign-On using either the 10g SSO disks or the 11g ones. So there’s a bit of legacy stuff in this release, you need to keep your OAS 10g disks available as you seem to need them at various points. For now though, I skipped the Portal install and was therefore able to skip all the steps around OID and SSO.

Once the install takes place, you then go through the post-install configuration steps. Now on Linux x86 this went fine, but on Windows Server 2003 the configuration failed at the OHS creation step, according to the logs it couldn’t restart opmnctl and after a lot of fiddling, I still couldn’t fix it. After switching to Linux though it worked fine, with all of the configuration steps completing as normal.

One other thing I noticed, from running Fusion Middleware 11g on both a Windows Server 2003 VMWare virtual machine, and an Oracle Linux 5 virtual machine (each with 2GB of RAM on a 4GB laptop), was that memory consumption on the Linux VM was much less – on the Windows VM, the fan was constantly on when Weblogic was running, whilst on Linux it’s just ticking away in the background with little memory usage. This corresponds with other work I’ve done with OBIEE and EPM Suite running on Weblogic, on a Windows VM the system is constantly swapping whilst on Linux, it runs really nicely. Something to bear in mind if you’re putting a demo system together.

Now once you’ve got everything setup, actually running Discoverer is a bit of a dissapointment as it’s exactly the same as in the 10.1.2.x release. There are one or two minor changes around navigation in Discoverer Viewer, but 99% of the application is exactly the same as in the 10g release.

One thing I did notice though was that, if you’ve not got Discoverer set up with OID and SSO, you get quite a scary looking error message when bringing up the Discoverer login page telling you that it can’t retrieve the list of connections. You can safely ignore this but it does make you wonder at the start as to whether the install has gone OK or not.

Now where there are some significant changes is in the application server control web page, which in the past was provided by Oracle Application Server Control but is now provided by Fusion Middleware Control.

You can see from the screenshot above that the Discoverer application pages are completely different to the 10g release of the product, Discoverer here has an entry in the Weblogic Server domain listing and you can see the new, formally supported and properly documented Web Services interface in the application list that will presumably in future be used to integrate Discoverer worksheets into Oracle BI EE Dashboards and Delivers.

There are also new pages for administering the application server itself, with most of the Oracle Application Server elements having an equivalent in this new product, but still it’ll take you a few hours to get used to the new layout and how things work in Weblogic.

One final thing to bear in mind is that, if you running Discoverer 11g on Linux or Unix, you’ll also need to download the BI Tools 10.1.2 CD from OTN as you still use these old versions to create and administer the End User Layer for Discoverer 11g (shows how much it’s changed in this new release). If you install the Windows version of Portal, Forms, Reports and Discoverer it installs a single copy of Discoverer Administrator and Discoverer Desktop along with the application server files, but if you install the Unix versions you’ll need to download the tools CD separately. And “opmnctl” is still there and available with a command-line interface, so you can still bring up, down and view the status of the various application server components from the command line.

If you look back to say 2002 or 2003, the typical product skills required of an Oracle BI consultant were:

• Discoverer, for ad-hoc analysis
• Oracle Reports (occasionally, though this was more often required if you were a Forms developer)
• Oracle Portal, if you were looking to put Discoverer into a dashboard
• Oracle Express, though even then this was considered “legacy”, or Oracle OLAP, though often more in theory than in practice as there weren’t that many new implementations
• Oracle Warehouse Builder, though most customers were still unconvinced and hand-coded their data warehouse ETL routines

In addition to these product skills, you also needed to have a fair bit of database knowledge, partly because all of these tools worked directly with database data, so you had to know your indexing, materialized views, explain plans and so on, and partly because features such as Oracle OLAP were directly embedded in the database. If you were working with Portal, Discoverer or Reports, you also had to be fairly competent with Application Server development, so that you could use all the OPMN and DCM commands to stop and start the applications, and you had to know technologies such as Oracle Internet Directory, Oracle Containers for Java and so forth as these provided the security and hosting for your applications.

This set of skills would be reflected in postings on this blog, which were as much about database resource plans, setting up virtual private database and so forth as about using the tools themselves. In fact, to be honest much of an Oracle BI project was about actually getting the tools working in the first place, and then understanding the complexities of working with the application server, understanding how to apply row-level security and so on, to the point where actually delivering the bits the customers were interested in often ended up being less of a priority than actually getting it all working and in a reliable state.

Fast forward to 2009, and if I was recruiting an Oracle Business Intelligence developer now, what would I be looking for? Ideally, most if not all of the following product skills:

• Oracle Business Intelligence Enterprise Edition, usually with a specialization in either the back-end (the BI Server) or front-end (Answers, Delivers, BI Publisher), would be mandatory
• Essbase skills would be very desirable, and would be required far more than Oracle OLAP skills in the past.
• Oracle BI Application experience is a definite bonus, with the real skill in being able to apply customizations to the ETL and RPD layers
• Another very desirable skill is being able to implement at least Hyperion Planning, and ideally some more of the Hyperion financial applications. It’s rare though to find people with both good OBIEE and Hyperion skills, those that do can virtually write their own salary cheque.
• In terms of ETL, OWB is now “mainstream” and is a mandatory skill. ODI is rising in importance, but I don’t see many “classic BI” projects that use it, at least at the moment.

Compared to five years ago, skills in the Oracle database, and in particular with the Oracle Application Server, are less of a priority, as security for example is usually handled now by the OBIEE BI Server, and fast access to data is typically done through Essbase. Also, now that Oracle’s BI tools (and Fusion Middleware) are designed to work across heterogeneous platforms, its more useful now for developers to have experience with Active Directory, for example, compared to Oracle Internet Directory, and it’s also useful to have knowledge of other database platforms such as Microsoft SQL Server, Microsoft Analysis Services and the like as Oracle’s BI tools are increasingly being sold to non-Oracle database customers.

What I don’t see much of these days is demand for projects requiring Discoverer skills, or for that case Reports or Portal skills, as these are most definately “legacy” tools only really used by organizations who have no real need (or budget) to upgrade and probably have significant investments in the tools already. These types of customers typically resource new projects in-house or hire in contractors, and so whilst it’s nice to have these in the background I wouldn’t be concerned if someone new came on board who’d never worked with them before.

Going forward, the type of BI developer and consultant who will have the greatest job security, and can command the best salary in the future, will be one that can combine OBIEE skills (particularly around the BI Server and its federated query capabilities), Oracle BI Apps skills and Hyperion skills, particularly as Oracle converge the product lines over time. If I had to learn one ETL tool it’d be Oracle Data Integrator, and if I had to learn one OLAP server, it’d be Essbase.

Whilst there’s less need to know about database security, for example, a skill that’s not always obvious at the time but that can really add value to a project is being able to tune and optimize data warehouse queries, so knowing your way around an execution plan and understanding how concepts like partitioning, compression, materialized views and parallel query can be a bonus.

But certainly, a common observation you can make about the skills required for an Oracle BI developer are that they’re now much less Oracle database-centric, and much more based around what Oracle now call their “Business Intelligence Foundation” layer (the BI Server, Essbase, RTD) coupled with the applications that sit on top of it (the Hyperion Applications and the BI Applications, with EPM workspace now sitting on top). As such, not withstanding the effects of the recession and outsourcing, I’d say that job is even more demanding now and probably attracts a higher salary compared to before, and I certainly know from experience that people with all of these skills, and with real project experience, are still somewhat rare.

The way they’d set their Portal page up was that it contained a number of Discoverer Portlets, which were parameterized and tied to a simple parameter drop-down (see this OTN article for more details). When the drop-down value was changed, the value was sent to each of the Discoverer portlets and in theory, each of them would then display a worksheet for this value. In reality, they were having two problems:

1. Firstly, even when the portlets refreshed to their schedule, only certain parameter value selections seemed to be cached, with some parameter values coming back instantly, others caused the Discoverer portlets to show the “Portlet is refreshing” message. Moreover, it seemed pretty random whether the selection of a parameter value would cause a refresh or come back straight away.
2. Secondly, when a portlet needed refreshing, the user had to click on the refresh link repeatedly until the worksheet came back. Ideally, they’d want the page to refresh itself rather than requiring user intervention.

So that I had a sandbox environment to work with, I recreated the portal page and portlets using the Sales History sample schema, and the latest versions of Discoverer and Portal, versions 10.1.2.2 and 10.1.4 respectively. Whenever I come across this sort of situation my initial recommendation is to make sure you’ve applied the latest patches, we’ll see in a moment that this was actually quite good advice.

Anyway, having a play around with the portal page and the portlets demonstrated the behavoir that the customer was experiencing. I could select some parameter values and the worksheet portlets would display instantly, others would require a portlet refresh.

Webcache was turned on and in theory caching Portal pages, so I thought I’d have a delve around in the internal Discoverer tables, in particular the DISCOVERER5.PTM5_CACHE table, to see what was going on.

Taking a look at this table, I noticed that for each portlet, three parameter value selections were stored in table rows. I could see three rows being created and updated for each portlet, with the CCH_URL column containing the value of each worksheet parameter value. What was apparently happening here was that this table contained cached details of each portlet for the last three parameter values selected, meaning that if I went back to portal and selected the parameter values contained in the table rows, the worksheet came back straight away, if it was a value not in this table, a refresh was needed. So how could I get Discoverer to store more than three values in this table?

I then headed over to metalink and did a search on the PTM5_CACHE table name, and came up with Note 432849.1, “Discoverer 10g 10.1.2.2 Patchset Readme Notes” that described a new setting that you could add to the Discoverer configuration file. Called the “cacheThreshold” configuration setting, you could use it, from this patchset release on, to increase the number of worksheet parameter value settings that Discoverer would remember; moreover, setting it to a higher value will mean that, during the worksheet portlet request, Discoverer will run the query for all of the parameter values for that worksheet in the PTM5_CACHE table and store the results in the cache as well. For a worksheet with a small number of parameter values (six in the example I created above) this means that you can potentially pre-seed the cache with all the different parameter values, and have then all remembered so that worksheet portlets come back instantly. Not bad, and exactly what I was looking for – however you need to be aware that if you increase this value too high, your portlet refreshes can take a proportionately higher amount of time to refresh, as the worksheet runs for every parameter value in the PTM5_CACHE table. I decided to go with a value of 15 which wasn’t too high and would cover all the parameter values that I had.

To increase the cacheThreshold value, locate the $DISCOVERER_HOME/discoverer/config/configuration.xml file and then find the following line:

<portlet logKey="disco.portlet.fatal"
logLevel="error" pickerCompatibleVersions="10.1.2" maxDataRows="1000">

Edit this line to add the cacheThreshold value:

<portlet logKey="disco.portlet.fatal"
logLevel="error" pickerCompatibleVersions="10.1.2" maxDataRows="1000" cacheThreshold="15">

Restart the Discoverer server and then open the Portal page again, and now you should find that once you select a parameter value and the worksheet is displayed (which may require an initial refresh to get the values), the portlet contents are remembered next time, up to the number of parameter values defined in your cacheThreshold value. Then, when the next scheduled refresh happens, all of the parameter values in PTM5_CACHE will be refreshed (up to, again, your cacheThreshold value) such that results come back straight away.

The other fix we wanted to put in place was to remove the need for users to click on the “refresh portlet” link. To do this, I used a bit of Javascript I found on this OTN Forum thread that looks for any part of the portal page showing the refresh message and triggers a page refresh after a short interval.

<script language="JavaScript" type="text/javascript">
<!--Copyright 2006 jason pschar
function refresh_wait()
{
location.reload(true);
}

if (document.getElementById("refreshPageLink"))
{setTimeout("refresh_wait();",10000);}
//-->

I then created a Dynamic Page portlet and added the code to the HTML section, like this:

Then, I added it as the last portlet to be displayed on the page (placing it in the bottom right-hand corner of the page), in it’s own region which I set to not display any region headers, borders and so on. Then, when the page is displayed this new portlet will detect any Discoverer portlet refresh messages, and automatically keep refreshing the page until the message goes away.

Anyway, not a bad new feature, and once you’ve selected each parameter value and there’s an entry for it in the PTM5_CACHE table, the refresh for that value happens automatically. A nice touch is that entries in this table seem to persist across reboots and even different users logging in, as long as you’ve created your portlets using public connections.

Going back to my patching comment, this shows why making sure you’re fully patched up prior to doing some bug-fixing always makes sense, as the cacheThreshold feature only got introduced with 10.1.2.2 and without this, we’d have to settle for a maximum of three worksheet parameter settings being cached. This way, the business can decide how much resources it wants to throw at caching and pre-calculation of portlets based on the required portlet response time, and the issues (chiefly around unpredictability of response time and the need for manual intervention) can be made to go away.

The process starts off with a regular Discoverer worksheet, in this case from the Videostore Video Tutorial Workbook. Discoverer is at version 10.1.2.2.0 whilst BI Publisher is at 10.1.3.3 – see this previous posting on getting everything set up for Discoverer / BI Publisher integration.

Then it’s a case of starting up Microsoft Word, connecting to BI Publisher Enterprise as an OID user that has access to both Discoverer and BI Publisher, and then selecting the Discoverer workbook from the “New Template” dialog.

Once that’s done, the BI Publisher Desktop add-in grabs a sample dataset and presents me with a blank template to start working on. I use the Chart Wizard to create a simple bar chart using “Profit Sum” as the measure, “Deparment” as the label and “Year” as the series, like this:

Then I create a crosstab to accompany it, add a title and some commentary text, and drag and drop a logo onto the top right-hand part of the page, like this:

Now if I’m going to display this template in HTML format (as opposed to say, PDF) I need to make sure this image is available on a web server somewhere, so I copy it to the following location:

$ORACLE_BI_HOME\j2ee\home\applications\xmlpserver\xmlpserver\obi_logo.jpg

which makes it available at the URL:

http://winxpvm:7781/xmlpserver/obi_logo.jpg

and I double-click on the image in the Word document, select “Web” from the “Properties” tab, and enter the following code:

I then save the template as an RTF file, and preview it in HTML form. Everything looks fine.

Then, I use the BI Publisher Desktop menu to upload the template to the BI Publisher Enterprise server, and it’s now ready to run within Enterprise. Before I do that though, I log in as an administrator and set up the Scheduler. This involves creating a database schema then passing the connection details to the Scheduler setup wizard, which then creates all the scheduler tables for me.

After I restart BI Publisher so that the new settings can take effect, I go back in as the administrator and set up email delivery. For the purposes of the demo I’m using a freeware email server that installs on Windows called ArgoSoft Mail Server which lets me demo email integration without an internet connection; I enter the connection details into the BI Publisher Enterprise web page and my email connection is set up.

Now it’s a case of logging back in to BI Publisher Enterprise as the report owner, locating the report and then clicking on the link to schedule it.

This brings up a screen where I can set the frequency of report runs, provide an email address for the report to be sent to, and provide any other details relating to the running of the report and the different destinations it can go to, including FTP, WebDav and the filesystem.

Once the scheduled job is submitted, I can check back on the schedule details and see that the report has run correctly:

and then finally, I can start up my email application and see that the report has been delivered.

As well as BI-style reports, BI Publisher can also produce reports in any format, including mail-merge letters, a feature that’s often required by customers migrating off tools like Cognos Impromptu. Overall, this is a useful feature for customers who’ve got a lot of investment in Discoverer reports and metadata, but want a bit more flexibility in the way they can use the data.

Anyway, I was discussing this with Mike Durran, the Discoverer Product Manager, and Oracle have come up with a detailed setup document for BI Publisher and Discoverer integration, a collaborative effort between a few consultants, which they’ve kindly agreed to allow me to host on this site – you can download it here. It’s got lots of screenshots, explanations and so on, and whilst I’ve not gone through it myself (I’ve already got it up and running) it does seem to cover off all the required parts, including the BI Publisher / OID setup step. Anyway, download a copy if you’re getting this up and running, it should cover off all the steps you need.

To get this feature up and running, you need the following releases of Oracle Application Server and Oracle BI Suite Enterprise Edition

• Oracle Application Server 10.1.2.2.0
• Oracle BI Suite Enterprise Edition 10.1.3.3

In addition, you’ll need a one-off patch, No. 5648158, which gets applied to your Discoverer mid-tier to install the Discoverer Web Service, which BI Publisher uses to get data from Discoverer worksheets.

Getting this all set up was quite tricky, as you’ve got to patch Discoverer to install the Web Service, but before that you need to make sure Application Server is patched up to 10.1.2.2.0, which if you’ve not done so already involves also upgrading the metadata repository database to 10.1.0.5. As well as that, you also need to integrate Discoverer with OID, which involves protecting the Discoverer Viewer and Plus URLs using SSO, and you also need to configure BI Publisher so that it uses the OID (LDAP) security model, something that’s not mentioned in the docs. Starting off with the original requirement to link BI Publisher and Discoverer, this all causes a domino effect of having to upgrade various bits of your BI infrastructure to bring them up the right levels, although of course if you’ve already patched up Application Server and Discoverer to the latest levels, turned on SSO and so on, you’re probably mostly there anyway. Where it did get particularly tricky for me was trying to get all of this running in a VM with 1.5GB of memory – I need to have two Application Server tiers up and running (infrastructure and mid-tier), 2 Oracle databases (the metadata database, and my regular Oracle database with the Discoverer EUL in it), an instance of Application Server 10.1.3.3 and Oracle BI Suite Enterprise Edition, and BI Publisher – I had to be very careful about what’s running at any one time, and all of it worked most of the time unless I tried to run the Application Server Enterprise Manager website, whereapon things mostly seized up.

Anyway, once it’s up and running, there are two ways in which you can use data from a Discoverer worksheet in a BI Publisher report. Firstly, you can use the BI Publisher Enterprise Web site to create a new report, then select “Oracle BI Discoverer” as the data source type, then pick a connection – these have to be public connections as BI Publisher logs in to the Web Service as a trusted OC4J user, but not an authenticated SSO user.

Once you select the connection, you then pick from the list of workbooks available from that connection

When the workbook and then worksheet is selected, you can then save this part of the report definition and go over to the BI Publisher Desktop interface.

Laying out the report template involves firstly connecting to the BI Publisher server using the BI Publisher menu within Microsoft Word, and then bring up the Open Template dialog to locate the report that you’ve just created using BI Publisher Enterprise. Using the dialog, you locate the report, and then double-click on the entry below the report to create a new template.

This then creates a new blank template, with a sample of the data from the Discoverer worksheet available to you to help lay out the tables, crosstabs, graphs and so on. Once the template is complete, you upload it back to BI Publisher Enterprise and you can run it along with all your other BI Publisher reports.

The other way you can create reports using Discoverer data, is to go straight in to the BI Publisher Desktop application, log in to the BI Publisher server and bring up the Open Template dialog. This time though, if you use the drop-down list to select DIscoverer from the available workspaces, like this:

you can then select a connection:

and then a workbook and worksheet:

Then, you are prompted to copy the data source definition into the BI Publisher workspace (catalog), which creates a .XDO report definition and allows you to proceed as if you’d created the report yourself using the BI Publisher web interface.

Now you can create the template as you normally would, using the data supplied by the Discoverer worksheet (which can be an OLAP worksheet, not just a relational Discoverer worksheet) in charts, tables, crosstabs and form letters.

That’s it for now, I’m off on a flight up to Glasgow to run some BI Suite Enterprise Edition training through a partner, then it back again and bags packed for my seminar tour around Melbourne, Sydney and Auckland. Exciting stuff indeed.

When Oracle first announced the roll-out of the enterprise edition of the Oracle BI suite, I was a little worried that the fantastic Discoverer might be ‘gone to meet it’s maker’. But seeing how the (formerly-known-as-Siebel) Oracle Answers is still lacking on supporting all the native Oracle analytic functions, I gladly accept Discoverer as the stronger candidate in ad-hoc querying against the Oracle database. Now, in the SoD they don’t go as far as really suggesting you absolutely should migrate to the Enterprise edition of the suite and drop your Discoverer environment, but the case for doing so is made a lot stronger than the case for staying with it. At least the promise is there to keep it alive and in the loop. The following quote does give some food for thought:

Existing Discoverer customers can continue to use Discoverer with the assurance of Oracle’s Lifetime Support Policy, as well as continued planned enhancements and integration with future releases of Oracle Application Server and Oracle BI EE.

This was quite an interesting article to write; apart from establishing how ODI related to OWB, I also put an example together that takes data, via Asychronous Hotlog Change Data Capture, from an Oracle database, combines it with data from a flat file, and then loads it, in real-time, into a Microsoft SQL Server database. It’s a real working example, and although I’ve obviously had to summarize the steps to keep the article a reasonable length, you should be able to follow it along and get it working yourself, if you’re interested to see what ODI is capable of.

I’ve also uploaded the final copy of the slides that I presented at the UKOUG BIRT SIG earlier this week (“Oracle BI Suite Enterprise Edition for Discoverer Users”), and we’re extending the offer at the end of the presentation to all the readers of this blog in the UK and Ireland (and possibly mainland Europe): if you’re considering migrating from Discoverer to Oracle BI Suite Enterprise Edition, and you’re looking for a partner to help you with the migration process, we’re offering a free day’s initial assessment where either Jon or I will come in, talk to you about what’s involved, give you a tour around the new BI EE software, take a look at your existing Discoverer setup and give you a road-map for making the move.

Oracle Product Development are also keen to help out, so if you go ahead with a migration or prototype, they’ll provide access to an “accelerator” that will do the majority of the work in migrating the EUL and the workbooks, leaving you and us with just the task of working with the new features such as dashboard, alerting, publishing and so on. Drop us an email if you’re interested in taking part.

Brian Duff posted a link earlier this week to the new Oracle Blogs Semantic Web site, that aggregates a much wider set of blogs than the old blogs.oracle.com site, and adds some Ajax, tags and semantic web goodness to allow you to list blogs by author, tag, date of posting and so on. What’s particularly cool is it’s (I presume) parsing of blog content to add it’s own tags, so that I can list out my own postings and view them by tags such as “Identity Management”, “SOA Suite” and so on. Not bad actually, miles better than the old OTN blogs site and it’s good to see something going beyond even Orablogs and OraNA.info.

On the subject of Oracle and blogging, I read and followed with interest the debate on various sites last week on Oracle Corporation and it’s (lack of) respect in the blogosphere. I was a bit snowed under with work at the time so didn’t join in, and I have got a fair bit of sympathy for Justin’s position, but looking back there’s probably two suggestions I could make that I think would significantly increase Oracle’s credibility in at least the technical blogging community:

• Have Oracle staff actually comment on your blog postings, join in the debate, discuss why things work the way they do and how they could be improved/extended or whatever. I do get a bit of this in postings on one or two product areas but overall it’s pretty rare, which gives the impression after a while that most of the activity Oracle does in the blogosphere is broadcasting rather than discussing, if you see what I mean.
• Have more technical staff (i.e. the development team for particular products, or consultants, or pre-sales consultants) blog, AS WELL as the product marketing managers, VPs and so on. I’d love to get in to conversation with the developers of OWB, or of OBI EE, or Discoverer or whatever, not just the PMs (though that’s good as well). Every so often I get the odd email from product developers but I’d love read in a blog what they have to say about why OWB works the way it does, what’s coming up, why they chose this particular architecture and so on.

And on that particular subject, it was through the new OTN BlogsSemantic Web site that I came across the new Oracle Warehouse Builder blog, hosted by Jean-Pierre Djicks and with contributions from … the OWB development team. So there you go, that’s exactly what I’m talking about and here’s hoping we see something similar in time from the team behind OBI EE, Hyperion, Data Integrator and Discoverer (Tim Dexter already does a sterling job for BI Publisher, Mike Durran and Abhinav Argawal post every so often on Discoverer but it’d also be good to get some content from the product developers as well). Taking at look at the OWB blog, there’s already some good postings on Speeding-up the Control Center Manager, how to restore the OWB Design Center panels when you move them all around, and creating an XML view for use with OWB, so take a look when you get a chance as it’s all very good content.

For me, I’ve got a very full and interesting two or three weeks coming up. I’m off to Johannesburg, South Africa on Sunday to present a one-day Oracle BI Suite Enterprise Edition Masterclass on Tuesday for Oracle South Africa, then I’m back in the UK, taking part in an Oracle BI and SOA workshop on the Wednesday, presenting at and chairing the UKOUG BIRT SIG in London on Thursday, off to see a potential new partner on Friday and then off to Lyon on Sunday for the Hyperion Conference. After that, it’s back for a day, then off to the Miracle Scotland Database Forum, then back, then a week’s holiday, then ODTUG in the States. At the UKOUG event I’ll be presenting on “Moving to OBIEE from a Discoverer Users’ Perspective”, where I’ll go through the manual migration process, show how Videostore data and reports can be recreated in OBI EE, and do a demo showing how the process works and what you can do with new technologies such as Delivers, Dashboards and Answers. If anyone’s interested, I’ve uploaded the slides here, there’s a couple of weeks until the talk so if you’ve got any feedback or suggestions, let me know and there might be time to incorporate some changes or additional material.

Finally, if you’ve installed OBIEE 10.1.3.2 and are wondering why you can’t get the scheduler (i.e. Delivers) working, this link might help. There’s one more post-install step that encrypts the Administrator password that you need to carry out, I had the same problem but following the direction in this OTN posting sorted it out for me. Just in case anyone else has the same problem.

So, I guess that if we’re looking at a potential upgrade for Discoverer users, it would be useful to take a moment to think about what’s good, and what’s not so good, about Oracle Discoverer. In the paper I’ll do a quick one slide/one paragraph recap on what Discoverer is, and then go through what I feel are the major plus-points of the tool:

• Easy to use, lots of wizards, familiar look-and-feel, high awareness and exposure within the Oracle user community – many people have exposure to Discoverer through its apps integration, bundling with Oracle Application Server and so on – the “comfort” factor if you like.
• Leveraging of Oracle’s built in calculation, analytic and PL/SQL functions – Discoverer uses the Oracle database as the calculation engine, you get access to all the built-in SQL and PL/SQL features including all the analytic (lag, lead, window, top N etc) functions.
• Integration with Oracle database security and E-Business Suite responsibilities, and pre-built E-Business Suite reports and BI metadata layer.
• Integration with Oracle Warehouse Builder (although this requires the Enterprise ETL Option for Warehouse Builder, at $10k a CPU on the ETL database), and integration with Oracle Portal
• Oracle OLAP access through Discoverer for OLAP
• Lots of functionality around totals, percentages and other report add-ons

If you were honest, the failings of Discoverer could be summed up as the following:

• Oracle database-centric; although Discoverer can connect to non-Oracle databases, this is a fairly complicated DBA task and still requires everything to be routed through the Oracle Database, and the End User Layer’s Oracle database dependency still means you need an Oracle database somewhere, even if all your data is in MS SQL Server, for example
• Although Discoverer integrates with Oracle Portal, in my opinion is not an optimal solution as it’s tricky to get all the report refreshes working properly, the reports in Portal don’t show a real-time view of the data underneath, you can’t drill and analyze in-place, and Portal itself is a bit overkill for just a BI portal
• It’s very hard, if not impossible, to get Discoverer reports to run lightening-fast; typically Discoverer reports take 10, 20 seconds or more to return data, Discoverer itself adds a significant time overhead to queries, it’s just not a fast, snappy environment to report in.
• The report authoring part of Discoverer requires a Java applet to be installed and then run in the client PCs Web browser, which can cause security and installation issues for users not running with admin rights, and requires a higher-spec (memory, CPU) PC to run on.
• Discoverer for OLAP, whilst very similar to regular relational Discoverer in terms of functionality, look and feel, is however still different, has more limited capabilities (no parameters, can’t total by attribute and so on) and has a separate report catalog and security setup to Discoverer relational.
• There’s (currently) no capability to create alerts, distribute reports, add in-context messages to reports giving advice on how to interpret the report.
• There’s also (currently) no way of calling Discoverer reports via an API, or adding workflow to Discoverer reports so that a user clicking on a report area or a link displayed along it can trigger, say, a BPEL or Oracle Workflow process to act on insights provided by the report.
• There is also (currently) no way of displaying Discoverer-generated data in, say, a letter, or printed labels, or in a report that contains more than one dataset.

I say “(currently)” in some of these issues because a few of them are being addressed by planned integration of Discoverer with OBI EE, and I’ll address these future alternatives to a straight upgrade to OBI EE later in the paper. For the time being though, Discoverer’s advantages could be described as its familiarity, leveraging of Oracle database features, support for totaling, percentages and analytic functions and integration with Warehouse Builder and Portal, whilst the drawbacks are this very Oracle database integration, lack of alerting and report distribution features, lack of APIs and interconnectivity with the application development world, limited output options, and performance, which shouldn’t be overlooked as it’s the number one compliant I hear about Discoverer when I visit customer sites.

So, with these points in mind, I’ll do a short introduction to OBI EE and then start to go through its features, placing them in the context of Discoverer and Discoverer’s good and bad points. As I’m conscious that this is only a one hour session, and working to my “golden rule” that people can only take in a maximum of six things during a presentation before it all starts to wash over them, I’ll concentrate on the following key features.

1. The architecture of OBI EE

   

2. What Oracle Answers is, and how it compares to Discoverer. At this point I would build a simple report, and afterwards highlight in summary some of the things Answers does better/differently than Discoverer.

   

3. How Oracle Interactive Dashboards works, how it differs to Oracle Portal and how it integrates with Answers. Again, I’d quickly show off Dashboards by bringing in the previously created report, and point out in summary what else it does different/better than Oracle Portal.

   

4. How the alerting and distribution element works (Oracle BI Delivers)
5. How the metadata layer (Common Enterprise Information Model) works, how it differs from the Discoverer EUL
6. How it integrates in with Web Services, BPEL and SOA
7. How it integrates with Oracle E-Business Suite
8. What Oracle BI Publisher brings to things

(I know that’s 8, rather than 6, things, but hopefully I’ll still have the audience with me at that point).

Now that I’ve gone through the key points of the new products, it’d be worth talking about the migration process and also, what elements of functionality Discoverer has that the new products don’t have, or don’t do as well.

For the migration process, this would have two strands. Firstly, the Oracle Discoverer development team are working on a migration utility that will, to one extent or another, automate the process of creating OBI EE metadata (and possibly reports) from the equivalent Discoverer metadata and workbooks. Now at the time of delivering this paper it’s unlikely the utility will be publicly available, so whilst I’ll make reference to it, I’ll mainly look at migrating Discoverer elements manually. This will consist of a couple of slides on the process of bringing the Videostore dataset in to OBI EE’s metadata layer, creating regular and time dimensions, recreating complex folders in the OBI EE presentation layer and custom folders as SELECT tables the physical layer, applying security and re-implementing workbooks, which of course could be a presentation in itself, so I’ll only have time to cover this superficially in the presentation, although I might have time to cover it in more depth in the accompanying paper.

In terms of what functionality is currently missing in OBI EE, I thought this was an interesting (and important) area to cover as “what’s missing” is not normally something you’d see covered in Oracle sales presentations or product literature, and it’s only something you tend to find out once you try to implement a migration. As I personally haven’t migrated a Discoverer system to OBI EE yet, I’ve based my observations on migrating the Videostore data and workbooks to OBI EE, and so far I’ve come up with the following areas OBI EE still falls short:

• Discoverer is still, in my opinion, more intuitive for first-time or less experienced users to create or open reports. Discoverer provides you with a Workbook wizard when you first start up giving you simple choices around opening a report, creating a report and so on, whilst Answers provides you with a fairly busy, hard-to-fathom web interface with lots of very small buttons, no real guidance on what to do next. Now the Answers development team would no doubt say that, in reality, most users won’t see Answers as you do the vast majority of report querying directly within Dashboard (in Discoverer/Portal, anything more than just viewing a snapshotted Discoverer worksheet portlet requires launching out into Discoverer Viewer) but even so, Answers could be a bit more newbie-friendly.
• The range of analytic functions and other calculations in Answers is limited compared to the Oracle PL/SQL and SQL functions made available from Discoverer. For example, whilst coming with a number of analytic functions such as Rank, TopN and so on, Answers’ time series calculations are limited to prior period (“ago”) and period-to-date (“todate”), whilst Discoverer has access to all the Oracle analytic functions in the relational version, and all the OLAP DML functions (forecasts, allocations, statistical etc) in the OLAP version.
• OLAP in general in Answers is very limited (there’s no ability to select dimension members via their position in a hierarchy, no concept of dimension attributes, no ability to drill up or to related) although Answers does handle the “OLAP as a star-schema performance booster” use case well, as it just presents OLAP data as a relational dataset and handles the issue of fully-solved cubes very well (and it talks MDX and XML/A natively, opening up MS AS, SAP BW and in the future, Essbase, very well).
• There’s currently a lot of duplication at the BI Server level with equivalent Oracle database features – BI Server has it’s own summary management, query rewrite, security layer and so on – which whilst useful in a heterogenous environment (for consistency) leaves you wondering when to use OBI EE features, when to use Oracle RDBMS features.
• Cost, of course. OBI EE on a per-CPU basis is more than ten times the cost of Discoverer, although on a named user basis the cost difference is less marked and the upcoming OBI Standard Edition one is more comparable cost-wise, albeit with limits on the number of server CPUs and end-users

So, in summary, that’s what I’m going to try and cover, Again, considering I’ve only got an hour, I might end up covering less points but in more detail, or just to reduce the pace of the talk, but if you can think of anything else that’s relevant, or questions you’d like answered (I’ll post the presentation and paper on the site when they’re written), just add a comment and let me know.