Rittman Mead - Delivered Intelligence

In yesterday’s article I posted the first half of an email interview with Phil Bates, architect for Oracle Business Intelligence Enterprise Edition based down in Bristol, UK. Phil’s focus is on the vision, direction, architecture and development strategy for Oracle Business Intelligence products, and in today’s posting we ask him about the integration of Oracle Business Intelligence with Service-Orientated Architecture, how the metadata layer in Oracle Business Intelligence Enterprise Edition will evolve over time, and how he sees Oracle’s dashboard and analysis tools developing into the future. Don’t forget, you can put any questions to Phil in the comments to this article, I’ll then gather all of the questions together with the ones for part 1 of the interview and pass them to Phil, who will then answer them in a follow-up interview post.

[Mark Rittman] “Phil, you’ve presented a few times recently at user group events on the convergence of BI and Service-Orientated Architecture, and the benefits of automated decision-making tools like Oracle Real-Time Decisions? What business benefits do you see with these technologies?”

[Phil Bates] “There’s a convergence of interest in BI and SOA technologies from customers who are implementing business process management solutions to gain greater agility. This is perhaps not surprising: SOA enables organizations to deploy and adapt business processes more quickly, orchestrating processes across a company’s existing heterogeneous applications and systems. Business Intelligence enables organizations to understand business process performance and key business goals by integrating the data from a companies existing heterogeneous applications and systems. The ability to deliver on the promise of business agility fundamentally requires good business intelligence to understand what changes to make to business processes. Put another way - business agility requires business insight. Equally the value of business insight itself is also greatly enhanced when we can quickly act upon that insight to drive business performance improvements. So BI and SOA technologies are, I believe, inherently complementary and are increasingly both found at the heart of an enterprise architecture strategy.

A service oriented architecture typically requires some infrastructure to formalize the definition and orchestration of business processes, rules and events. This requires technology such as the Oracle Business Process Architect, Oracle BPEL Process Manager (BPM), Oracle Business Rules Engine (BRE), Oracle Business Activity Monitoring (BAM) and the Oracle Enterprise Service Bus (ESB)- all components of the Oracle SOA suite. The SOA Suite also contains a set of technologies for integrating business processes and events with companies existing infrastructure and applications.

Convergence between BI and SOA enables customers to gain insight into business process performance and drive business performance improvements more effectively. I’ve seen three distinct patterns of integration arising here:

I. The use of BI to gain visibility into business process performance,

SOA infrastructure such as Oracle BPM generates key business process data - e.g. process cycle times, frequency of operations, approval times, delivery delays etc. that can be integrated with data from the transactional systems the processes interact with and viewed and analysed in Oracle Business Intelligence dashboards. In addition we have seen an emergence of innovative customer solutions using Oracle Business Intelligence to monitor, analyse and report against human workflow activities (tasks, assignments, approvals etc.) that are orchestrated using the human workflow capabilities of Oracle BPM. There’s also a set of use cases where it makes sense to use BI and BAM together to integrate historical and realtime information within a common dashboarding and alerting environment. We demonstrated examples of how the integration between Oracle BIEE and Oracle BAM today enables the realization of these use cases at a recent UK OUG event in London.

II. The ability to transform insight into action by interacting with business processes from BI tools

Business Intelligence tools have traditionally been used to analyse and monitor key business data, supporting the ability to drill to detail to understand the cause of an underlying issue. Once an issue is understood, users typically need to take some form of action that is very often an invocation of a business process. For example, a BI dashboard may allow a user to monitor cashflow performance, and drill to detail to understand key issues such as a customer having large number of overdue invoices outstanding. In this scenario the next step is for the controller to take action - by invoking a business process to address the problem - for example to initiate a credit hold or a debt collection process. With Oracle Business Intelligence, these processes can be invoked directly from dashboards, alerts and/or iBots (intelligent sense/respond agents) and orchestrated within the business process engines in the SOA suite. The process may embed human workflows to handle approvals and notifications.

Combined with the ability to gain visibility into business process performance this can provide a much richer, empowering user experience for the Business Intelligence users. For example, the dashboard in the example above can also contain views tracking the actions and state of the business processes the user has initiated.

III. The ability to embed BI as a service into business process workflows and rules.

Oracle’s Business Intelligence products have a rich set of web services to enable the use of BI as a service. This allows business processes orchestrated in the BPM suite to invoke Oracle business intelligence services to, for example, deliver timely, personalized reports and alerts at key points in the process, and to use metrics and KPIs defined in the BI systems to make effective decisons within the business process. For example, a procurement business process can be orchestrated to select a supplier based on supplier performance metrics that are accessed by invoking the Oracle Business Intelligence web service interfaces. This approach has substantial benefits in that it can leverage the enterprise semantic model within the BI server so the definitions of key metrics can be used consistently in dashboards, business processes and reports, as well leveraging the BI EE server to decouple the business process from the complexity of underlying data sources associated with these metrics.

Real Time Decisions further extends these capabilities to allow adaptive decision making within business processes. This is particularly useful in business processes where there are particularly complex and fast changing rules that should determine which branch of a workflow to take. For example consider an insurance company looking to select a supplier - e.g. a plumber - to deal with an emergency. In this scenario, optimal supplier selection may be based on a set of complex, realtime, and fluctuating conditions - locality, time of day, nature of issue, size of supplier, etc. Real Time Decisions can be used to make the decisions within the workflow and learns from the outcomes (happy customers, cost, right first time, supplier availability etc.) to optimize the decision making within the business process.”

[MR] “How would you like to see the metadata layer in Oracle BI Enterprise Edition developing over time? Do you forsee a time in the future when Oracle’s BI tools share a common metadata layer and a common development environment?

[PB] “Development of the BI EE metadata layer is core to our strategy and there are a number of interesting themes to this. We are investing substantially in extending the metadata capabilities to exploit best practices we’ve identified in developing the BI applications and supporting customers building bespoke BI solutions. One of the exciting areas is to extend the model first paradigm to make it easier to manage the full BI application development lifecycle. The model first paradigm builds on our experience that it is better practice to develop BI applications initially from the business user and business analyst perspective to define the key analytic use cases users need, independently of the underlying data storage. Once the semantic model is defined, the physical model and associated ETL can be generated directly from the BI tools. We delivered the first increment of the model first vision in 10.1.3.3 with the aggregate persistance feature - future releases will extend the capabilities in this area, driving down the cost and complexity of developing BI solutions.

Today, the Oracle BI EE components share a common metadata layer across the BI EE components and BI applications. One of the key strategic priorities for us with the integration of Hyperion into the BI Suite is to ensure we can leverage common semantic definition across the product suite and in the near term this will be achieved through a combination of interoperability and metadata exchange. For example, integration between the BI EE server and Essbase enables the BI EE metadata model to be used to query Essbase. Essbase can also access BI EE using the BI EE ODBC interface (and hence semantic model).”

[MR] “In terms of usability and user productivity, how do you see Oracle’s dashboard and analysis tools developing into the future? Will Oracle Answers eventually be developed to cater for OLAP analysis, and perhaps planning and budgeting?”

[PB] “Usability and user productivity is central to our product strategy and we continue to invest strongly in this area. We do believe that pervasive use of BI requires that BI content can be delivered and used in many different environments - portals, dashboards, mobile devices, MS Office, desktop gadgets, search, transactional applications etc. The releases planned over the next 18 months have many features that continue the emphasis on usability, enhanced visualizations, charting, graphics and interactivity.

Answers today supports analysis against a variety of OLAP data sources including Essbase, Oracle OLAP, Microsoft Analysis Services, and SAP BW and we will continue to enhance the ability to leverage the computational capabilities of these data sources. We are also investing substantially to extend the capabilities in Answers to enable rich multi-dimensional analysis capabilities that OLAP users are accustomed to, for example the ability to use an explicitly dimensional query model, define and save selections, work with different kinds of hierarchies and define custom dimension members and aggregates. For planning and budgeting, Hyperion Planning is the strategic product set handling the broader lifecycle of planning processes.”

Thanks Phil. As I said earlier, if you have any questions for Phil on the answers he gave in this interview, add them as a comment to this post. Once they’re all in and Phil is back from Holiday, I’ll be back with a follow-up post with Phil’s answers.

One of the things you might not know about product development for Oracle Business Intelligence Enterprise Edition is that a lot of it happens down the road from us, in Bristol in the UK. Oracle’s team in Bristol and the UK were originally responsible for Oracle Discoverer, and over the past year or so have taken on development of some of the more interesting aspects of Oracle Business Intelligence Enterprise Edition, including integration with Service Orientated Architecture, enhancements to the Presentation Server tools and integration with the Hyperion toolsets.

The team down at Bristol have been excellent supporters of the UK Oracle User Group over the years, with product managers such as Mike Durran and Paolo Fragapone presenting at our last BI & Performance Management Special Event on the Oracle BI Apps roadmap and integration of OBIEE and Business Activity Monitoring. One of the talks I most look forward though to at events such as these are the product updates from Phil Bates, who is also based down in Bristol and works on the architecture and product direction for Oracle Business Intelligence Enterprise Edition. Many of the postings on this blog such as those on OBIEE architecture, development techniques and integration with Fusion Middleware have come after discussions with Phil and Mike, and I was especially pleased when Phil agreed to do a short interview for this blog on what he does and where he sees Oracle BI going in the next few years.

It’s a fairly long interview so I’ll run it in two parts;, with the first part talking about his role, the main development themes for Oracle BI over the next few years, and how he sees the integration with the Hyperion tools progressing. Phil has kindly agreed to answer any questions you might have around Oracle’s BI product strategy and the answers he gives during the interview; if you’ve got a question, leave a comment on this article or the one later on this week, and Phil will reply when he gets back from traveling in a week or so’s time.

[Mark Rittman] “Tell me a bit about what you do and what your responsibilities are regarding Oracle Business Intelligence?”

[Phil Bates] “My focus is principally on the vision, direction, architecture and development strategy for our Business Intelligence products. In practice this involves working across our product development teams to ensure we consistently innovate business intelligence functionality within a coherent, high quality, secure and manageable infrastructure. On a day-to-day basis I find my work involves a pretty varied and interesting set of activities including:

• Working with development teams to explore and design new product features;
• Establishing technical strategies for tighter integration between components and to improve the ease of use and manageability of the products;
• Identifying how we can better integrate business intelligence with middleware and applications;
• Defining how to leverage Oracle and third party technologies and applications within Oracle Business Intelligence;
• Collaborating with our product management and marketing teams to develop and articulate our product vision and priorities.

I also spend a significant amount of time with customers and partners. I see this is an essential part of an architect’s role: working directly with customers and partners enables us to shape our product strategy, define priorities, develop and refine our understanding of requirements.”

[MR] “Looking to the future and in particular over the next couple of years, what do you think the main themes will be in the development of Oracle’s business intelligence tools?”

[PB] “From a business perspective, the key themes will include;

• Enabling customers to better achieve end-to-end enterprise business insight and performance management
• Extending the scope and coverage of prepackaged BI applications (in terms of content, source systems and integration with CRM/ERP/EPM applications)
• Making business intelligence more accessible and actionable through a greater number of delivery vehicles and channels
• Integrating business intelligence more thoroughly into the business processes people are seeking to understand and optimize.
• Making business intelligence easier to manage, scale and secure.

[MR] “How do Oracle intend to incorporate Hyperion’s tools into Oracle Business Intelligence? What will the main customer benefits be?”

[PB] “The strategy here is to protect, extend and evolve. Protect means ensuring customers existing investments in Hyperion, and Oracle technologies are covered by our lifetime support policies. At the same time we are extending the capabilities in the components from Hyperion and Oracle and have a significant set of new releases planned over the next 18 months or so, each packed with new features. Some of these features extend the products to ensure we continue to have best of breed capabilities across our BI product suite and continue to lead the market from a functional perspective. Other features - such as common platform certification, dashboard interoperability and Essbase integration with BI EE - enable us to evolve the product capabilities to make it easier to use the components from Oracle and Hyperion together. These evolution features in turn enable us to incrementally deliver a deeply integrated end-to-end enterprise performance management experience.

So we will continue to see a pragmatic, incremental product strategy that allows customers to benefit from innovative new features with their existing products while at the same time extending their capability to gain consistent insight across the full enterprise performance management lifecyle - i.e. from planning and budgeting, monitoring and analysing the operational business processes (marketing, sales, service, supply chain, HR, finance, etc). and managing the end of cycle reporting and consolidation processes. From a technical perspective we will also see an incremental consolidation of infrastructure and a strong focus on improved systems management and lifecycle capabilities to make all of the Oracle/Hyperion BI components and applications easier to manage, secure, deploy and scale. The intention here is to dramatically lower the total cost of ownership of an enterprise business intelligence and performance management system.

Customers will also benefit from increasing integration of the wider product management and lifecycle processes (e.g. more extensive testing and certification, more comprehensive and consistent documentation, extensive education and training programmes, integrated and common support processes) as well as a larger partner ecosystem of systems integrators, VARs and consulting organizations with skills across the combined Oracle/Hyperion Business Intelligence portfolio.”

Don’t forget, if you’ve got any questions for Phil on Oracle’s BI strategy, add them as comments to the post. I’ll be back tomorrow with part 2 of the interview, where Phil talks about the integration of business intelligence and service-orientated architecture, where the metadata layer in Oracle business intelligence is going, and how the front-end tools are likely to be developed over the next few years.

I’m running a BI Publisher course this week, and one of the areas the client was really interested in was integration of BI Publisher with Oracle ApEx. As the notes on OTN and various blogs are sometimes a bit confusing, I thought I’d do a quick walkthrough of how the integration works. Setting up BI Publisher and ApEx integration consists of two steps; first you have to configure ApEx to work with BI Publisher (and if you’re using Oracle 11g as your ApEx database, configure the database to allow outbound network connections), and then you need to generate your report within ApEx and BI Publisher Desktop.

In this example I’m using Oracle BI Publisher 10.1.3.3.2 (as packaged along with OBIEE 10.1.3.3.2) along with Oracle Database 11.1.0.6 and ApEx 3.0. The first step in configuring ApEx is to log in to the Apex Administration application and bring up the Manage Environment Settings > Instance Configuration page, which we’ll use to tie in BI Publisher to this ApEx installation.

Selecting this administration page allows us to enter the connection details for the BI Publisher server. Using this dialog, you enter the hostname of the BI Publisher server, the port number and the page details of the BI Publisher interface. Note also that you radio button is set to “Advanced Printing” which signifies that we’re going to use BI Publisher; the Standard Support option above it requires you to use an Apache module that also prints to PDF but doesn’t have as much functionality as BI Publisher - you’d use this if you want BI Publisher to print to PDF but you haven’t licensed BI Publisher.

One more thing that you have to do, if you’re using ApEx as part of Oracle 11g, is to configure the database to allow outward network connections, which is disabled by default (on earlier database releases this step isn’t needed. To do this, you run a PL/SQL script that uses the DBMS_NETWORK_ACL_ADMIN built in package to administer the database network access control list. This script assumes you’re using ApEx 3.0; if you’re using 3.1 you’ll need to change the references to FLOWS_030000 to FLOWS_030100

DECLARE
  ACL_PATH  VARCHAR2(4000);
  ACL_ID    RAW(16);
BEGIN
  -- Look for the ACL currently assigned to '*' and give FLOWS_030000
  -- the "connect" privilege if FLOWS_030000 does not have the privilege yet.

  SELECT ACL INTO ACL_PATH FROM DBA_NETWORK_ACLS
   WHERE HOST = '*' AND LOWER_PORT IS NULL AND UPPER_PORT IS NULL;

  -- Before checking the privilege, make sure that the ACL is valid
  -- (for example, does not contain stale references to dropped users).
  -- If it does, the following exception will be raised:
  --
  -- ORA-44416: Invalid ACL: Unresolved principal 'FLOWS_030000'
  -- ORA-06512: at "XDB.DBMS_XDBZ", line ...
  --
  SELECT SYS_OP_R2O(extractValue(P.RES, '/Resource/XMLRef')) INTO ACL_ID
    FROM XDB.XDB$ACL A, PATH_VIEW P
   WHERE extractValue(P.RES, '/Resource/XMLRef') = REF(A) AND
         EQUALS_PATH(P.RES, ACL_PATH) = 1;

  DBMS_XDBZ.ValidateACL(ACL_ID);
   IF DBMS_NETWORK_ACL_ADMIN.CHECK_PRIVILEGE(ACL_PATH, 'FLOWS_030000',
     'connect') IS NULL THEN
      DBMS_NETWORK_ACL_ADMIN.ADD_PRIVILEGE(ACL_PATH,
     'FLOWS_030000', TRUE, 'connect');
  END IF;

EXCEPTION
  -- When no ACL has been assigned to '*'.
  WHEN NO_DATA_FOUND THEN
  DBMS_NETWORK_ACL_ADMIN.CREATE_ACL('power_users.xml',
    'ACL that lets power users to connect to everywhere',
    'FLOWS_030000', TRUE, 'connect');
  DBMS_NETWORK_ACL_ADMIN.ASSIGN_ACL('power_users.xml','*');
END;
/
COMMIT;

Run this script as the SYS or SYSTEM user, and you should then be ready to go.

To create an ApEx report that uses BI Publisher, log on to a workspace and create a new Shared Component > Report Query.

When the new Report Query dialog comes up, press the Create button to create the query, and then enter the SQL for the query you’re going to run. In this case, I’m using a query against the database tables used by the ApEx sample application.

On the next page of the dialog you can test the query - do this and make sure it comes back with some data. Next, you can tie your ApEx parameters to the report, and then a dialog is displayed where you can download the XML for your report and use it, with BI Publisher Desktop, to lay our your report template.

Once you’ve used BI Publisher Desktop to create your template RTF file, you can use the same dialog to upload the RTF file to ApEx so that it can then send it to BI Publisher when your report is displayed. One thing to note here is that the BI Publisher references only appear once you’ve done the configuration steps I mentioned beforehand - if you don’t do these, ApEx just makes reference to FO and XML files which is how it handles PDF printing when BI Publisher isn’t around.

Once you’ve uploaded the RTF file, ApEx on the next page displays a link that you can use when calling the report from elsewhere in your ApEx application, and you can press the Test Report button to try out the report and check it displays OK.

Pressing the Test Report button brings up a dialog asking if you want to open the PDF document, once you accept this the BI Publisher report is displayed as expected.

So all in all, it’s a relatively straightforward process, once you’ve worked out how to do the initial configuration step and if you’re using 11g, how to enable outbound network connections from the database. Thanks to Borkur for working most of this out for our course examples.

As Mark mentioned and illustrated last week, I really did have an audience to my talk on Cube-Organized Materialized Views. He also managed to choose a camera angle for one of the photos that made the conference room look completely empty, but thankfully that was not the photo he published. In fact the presentation went well; I had several questions to answer (that’s a good thing) and I finished up a couple of minutes before the end of the 1 hour slot so my timing was pretty good.

I must have been good enough as I have been asked to think about submitting a paper for Collaborate 09 in Orlando, FL - bet it won’t be snowing there in May!

I have now put a copy of the slides and the conference paper up on the Articles tab of this website, just scroll down to presentations and click on the Cube-Organized Materialized Views links

A Quick re-post as this event is running in a week or so’s time. Rittman Mead Consulting, in conjunction with Oracle, are running a half-day Oracle BI Suite Standard Edition One seminar at Oracle Birmingham on Wednesday, 30th April 2008. Free to attend, this event is aimed at organizations new to BI and will provide an overview of Oracle Business Intelligence Suite with a focus on Standard Edition One, together with tips and techniques on how to implement business intelligence and reporting within your organization

If for example you are an IT Manager charged with implementing a BI tool within your company, a Finance Director looking to add greater transparency to your business processes, an MIS Manager looking to provide a company-wide reporting solution, or a DBA struggling to provide reports manually using SQL and Excel, register now and see how Oracle’s BI tools work.

Anyway, here’s the official invite:

“Business Intelligence Seminar : Too Much Information … Not Enough Insight?

30th April 2008 - Oracle, Birmingham

Oracle Business Intelligence Standard Edition One (BI SE1) is a complete BI system in a box developed for growing businesses. It is easy to configure and install, and is priced to meet even the tightest of IT budgets. It is based on the exact same technology platform as Oracle Business Intelligence Enterprise Edition, and so is simple to scale as your business grows.

Familiar with these challenges?

• Too much data, not enough insight?
• Fragmented view of information and performance across the business?
• Inability to develop new analyses without IT involvement?
• IT overwhelmed with reporting and analysis requests?
• “After-the-fact” insight?
• Primary BI tool is Microsoft Excel?

Why attend?

Gain a better understanding of Business Intelligence and reporting

Learn about Oracle’s complete BI solution and how to apply it. Topics will include:

• Discover how Oracle’s BI enables self-sufficiency in business users with no technical skills.
• Oracle BI Suite Components
• Reporting on business activities
• Improving business process
• Steps to implementation
• How to apply across your business and departments
• Pricing and packaging for the smaller deployment
• Business Intelligence Demonstration :

Venue : Oracle Birmingham, Blythe Valley Park, Shirley, Solihull, West Midlands, B90 8AD

Date & Time : 30th April 2008, Registration 09:30, Event Close : 14:00

Target Audience : IT Managers, DBAs, Financial Managers, Managing Directors, Sales & Marketing

Oracle Partner: Rittman Mead

Rittman Mead Consulting are a preferred Oracle business intelligence and data warehousing partner. They work with organisations throughout the UK and Ireland looking to implement reporting, business intelligence and dashboard projects using Oracle technology. Providing training, development and implementation support, Rittman Mead can work with you to deliver real business benefits in a surprisingly short timescale.

Join us at the Business Intelligence Seminar on Wednesday, 30th April 2008 and let us help you gain a better understanding of reporting and Information insight!

REGISTRATION

Click here to register or contact Emma Bolger on 0870 8768756 and ask for BI seminar registration.”

After the various sessions in the afternoon, I finished off yesterday with a meal with ODTUG, the Oracle Developer Tools User Group (I’m one of their SIG leaders). Apart from Mike Riley, John Jeunette, Jerry Ireland, Jeff Jacobs and John King, Kathleen and Karen (who do a lot of the organizing), we also had Dan Vlamis, Edward Roske and Tim Tow come along. Dan of course is a good friend and we’re generally the two main speakers on Oracle OLAP at the various conferences, however Tim and Edward are new to ODTUG and in fact IOUG/OAUG as they’re from the world of Hyperion, or more precisely Essbase. As you’d expect we all had a great deal in common and I’m looking forward to keeping in touch with Tim and Edward (and Tracey, the co-author on their book) in the future.

As a few others have mentioned we went from sunny weather and temperature in the 80’s to sub-zero temperatures and a blizzard, here’s Dan Vlamis and Kathleen just outside the restaurant in the middle of a snow shower.

It was actually pretty cold and wet and certainly a contrast from earlier in the week where we’d been sitting outside in shirtsleeves in the sun. Today though it’s cleared up and whilst it’s been cold, it’s been sunny with clear skies.

Today we were up bright and early for Pete’s talk on Cube Organized Materialized Views. I’d actually been teasing him over the past day that no-one would come along as it was at 8.30am on the day after the main party (this actually happened to me at Collaborate’06, when my OLAP tuning talk had been put in the JD Edwards stream, and then moved forward a day just in case anyone had actually seen it in the catalog) but in the end he had a good turnout, probably more than came to my talk actually. Pete’s talk looked at Cube Organized Materialized Views, the new OLAP feature in 11g, from a DBA’s perspective rather than a BI developer’s, and in particular looked at the actual performance of the feature and whether it actually delivered the performance boost it was supposed to.

Pete’s slides should be up on our site fairly soon, I’m also trying to talk him in to expanding on the subject and using it for his BIWA Summit submission for later in the year.

Apart from Pete’s talk, that was it for me for Collaborate for this year as I had to get back and start doing some work on a presentation I’m doing at our BI SE One event later in the month. I spent most of the day in my hotel room putting the presentation and demo together, I did however get a chance to pop out for lunch for an hour down 16th St just next to our hotel, this is the mile-long pedestrianized mall with lots of shops and so on, I walked on down to the end and took some photos of the Capitol building and some more of the historic area of Denver.

Tomorrow I’ll be holed up in my hotel room again catching up on work, and working through a BI Publisher course that Borkur’s been putting together that I’m delivering next week. The flight back to London is at 8pm, hopefully there’ll be no delays and in the meantime, if you’re interested, there’s the full set of pictures on Flickr.

I’m currently sitting in the wireless hotspot area of the Denver Convention Center (next to Alex Gorbachev, funnily enough) waiting for the Hyperion Tools “What’s New and What’s Coming” session at 11am. As I’ve got a few minutes spare I thought I’d jot down some notes on the sessions I went to yesterday and today.

The first session I went to yesterday was a bit of a diversion from the BI sessions I’ve been mostly going to. To be honest, I’ve been a bit disappointed with the BI sessions so far - a lot of them have been by the same speakers on more or less the same subject, quite a few are just the Oracle slides with their own company brandings, and the “expert-level” sessions have been fairly basic. There’s been a few exceptions - Dan Vlamis’ talk comparing Essbase and Oracle OLAP was good, so was the Edward Roske one on Essbase internals - but so far I’ve been a bit disappointed. Let’s hope today and tomorrow’s go a bit better.

Anyway, the first session I went to was originally billed as being by Hotsos, but in the end was by Method R, who are of course the spin-off company founded by Cary Millsap who’s recently left Hotsos. Method R (the name is taken from the name of the methodology described in the book “Optimizing Oracle Performance”) look like they’re taking over the development of Method R and the tools around it, and the session itself was good and centered around the idea that the advisors in Oracle 10g and 11g are useful, but you really need to know how they work and how you’d diagnose problems if you didn’t have them to hand. It was interesting to hear a Method R presentation given by someone other than Cary, also it was nice to think about something other than BI for an hour or so.

After the Method R presentation was probably the presentation that I’ve found most useful over the conference so far. It was on enabling real-time data integration using Oracle Change Data Capture, Oracle Database Change Notification and Java messaging, and was of particular interest to me as CDC is something many customers have asked us about. I set up a CDC demo system a couple of years ago - this page on our blog is one of our most popular in terms of page requests - but I’ve not used it in production, this presentation took CDC on one step further and coupled it with Database Change Notification, a feature that’s been in the Oracle database since 9i or 10g and provides mid-tier applications that provide a cache within notifications as to when data has changed. The session talked about a scenario where they had one application running on Oracle that had CDC enabled on it(asynchronous hotlog, which is how we’d set it up), DCN was used to notify the other application that data was waiting, and Java messaging was used to provide the data to the application. DCN can also inform the mid-tier application that the database is up, down or whatever, I’d never heard of this feature but it certainly sounds useful. There were a whole bunch of questions at the end, my one was around resilience, i.e. if the streams subsystem goes down, how easy was it to restart it (reasonably, but there was a lot of learning to do up-front on how this works) and whether any transactions ever got lost (answer: no, it always catches up with itself if streams goes down, the only problem seems to be that CDC seems fairly sensitive to memory issues and so on and will just stop propagating if resources are low). Based on this one bit of good advice was to make sure you build a mechanism to monitor the stream subsystem, if you’re using it on a project make sure you create a new database with CDC and streams (and Java) enabled right from the off rather than try and retrofit it to your existing database, and give CDC and streams a thorough shakedown before you try and add your source application to the new database.

After the CDC and DCN talk was another one I was looking forward to, but in the end found disappointing. It was on “Robust ETL using Oracle Warehouse Builder” and was from the guys at Verisign, with the database being ETL’d holding all the .COM and .NET registrations for the world and was hence rather large. We’ve delivered a similar talk at the last UKOUG conference - slides here - and whilst many of the recommendations were the same - keep your mappings simple, use separate repositories for each environment, use OMB scripting to automate deployment and so on - the problem was that their presentation was based on OWB 10.1 rather that 10.2, which has been out in production for around a couple of years now and is probably the version of OWB that’s most often used by customers now. Several of the recommendations, around say how to handle SCD2 mappings, how to handle errors and so on, were just out of date now and it’d have been better to hear, say, a comparison of manually handling SCD2s vs. the in-built handling of these in OWB10gR2, or manual handling of errors vs. using data rules and the data quality option - in the end I felt for the speakers as it was a well thought out talk, but just not all that relevant now that most people are on OWB10gR2.

And that was more or less it for the second day of Collaborate, at least in terms of sessions. In the evening I hooked up with Nicholas Bonnet, one of the Oracle Real Time Decisions product managers, some RTD partners and customers and also Mike Donahoe (BI Publisher product manager) and Matt Bedin (OBIEE product manager) and went downtown for something to eat, then it was back home, catch up with some emails and get ready for today.

Today’s first session was on the unstructured data support in Oracle 11g, again my interest in this is around the upcoming DW seminars I’m going to do for Oracle University - one of the areas I want to try and cover off in these is adding unstructured, spatial and multimedia data to your warehouse, so this was a useful session. An area I really must try and get in to is adding spatial and map data to OBIEE, you can add maps and the like to OBIEE now but in 11g this will be a standard feature of the tool, linking this back to the Spatial Option, or even the Locator feature which is of course free, in the Oracle database seems a useful thing to do. Keep an eye on the blog for more on this later on.

After that was a product update talk on the Hyperion tools. I seem to have missed most of the product update sessions at Collaborate this year - there was a Joe Thomas (BI product marketing) talk earlier in the week, and annoyingly the main Oracle BI product management update tomorrow is on at the same time as Pete’s talk - but the Hyperion one was useful, both in terms of news on Hyperion Workspace / OBIEE integration (this is due in the EPM 11.1.1 release, previously known as the Kennedy release, or 9.5, which will also need the OBIEE 10.1.3.4 release, both of which are apparently due around June this year), and news on the Essbase Studio (the replacement for Essbase Administration Services and Essbase Integration Services) which will also support OBIEE logical models as a data source (which doesn’t seem to work with the current release of EIS and OBIEE, see this blog post for details).

Anyway, the last session for me today will be the BIWA panel session on whether data warehouse appliance vendors are relevant any more, after that I’m taking the afternoon off and popping down the shops, then meeting up with some of the ODTUG people for something to eat later on.

I am sitting in my hotel room, bathed in beautiful sunshine as the sun dips towards the snow-capped mountains. Better make the most of it for tomorrow is forecast to be a lot (like 19ºC) colder and maybe even a touch of snow. Not that we will see much of the outdoors from the convention center across the street from the hotel.

I am still getting use to the time zone, that means I wake too early, sleep too early and really feel tired by the last conference session of the day, perhaps I might give the city a miss tonight

Yesterday, I said that Mark and I often attended different sessions so that we could see as wide a spread of presentations as possible; but today was an exception, we seemed to want to go to the same sessions - I did start off on my own at a vendor presentation about reporting from EBS with OBIEE then met up with Mark for a rather good talk by Mark Brooks on the deployment of change data capture, change notification and java messaging to implement a realtime feed between an Oracle database and a packaged web application with a SQL Server 2005 database. I don’t know why, but change data capture seems to be a underused feature - it really is a good technique and to my thinking a better methodology than adding masses of horrible triggers to an application to capture data change on a row by row basis - what do you think?

After lunch we both sat in on Scott Rappoport’s dimensional modelling talk - it was nice to hear another take on the topic, you always pick up a new insight, and for me this was quite timely as I am preparing a revised version of my DW design course. The final session of the day (for me) was one on robust ETL using OWB, an interesting talk from a user organization - it is always good to hear practical advice from people that have done things for real, especially if they have tackled in in a similar way to the one I would have done!

Well, that’s Day 1 of Collaborate’08 over and I think I’ve just about got over the jet lag. I’m up early again - today it’s 6am - but at least I’m not suffering from altitude sickness like a couple of people I bumped in to. We’re actually a mile above sea-level in Denver which means the air’s a bit thinner here, so far I’ve been OK but then again I’ve not exactly been running around or climbing lots of stairs.

The first talk I went to yesterday was Edward Roske’s on “How Essbase Thinks”. Seeing Edward was interesting as I’d had him down as a business-type person, in real-life he’s obviously a bit of a techy and certainly knows his stuff around Essbase and the Hyperion tools. The room was packed - around 170 at last count, with people standing at the sides and sitting in the aisle - which is about 160 more than the average Oracle OLAP talk in previous years. Edward’s a bit of a pro when it comes to speaking, a mix of technical stuff, a few bits of gossip and some audience participation, the session was very good and helped me understand a bit more around Essbase’s internals. For example, the Indexfile that Essbase uses roughly corresponds to Composites in Express and Oracle OLAP, whilst blocks roughtly correspond to pages in Express/Oracle OLAP. I was talking with Dan Vlamis after the session and he pointed out that Essbase is actually very much, architecturally-wise, like the old mainframe version of Express, which also had separate files for each of the database data elements, the programs and so on. A good session and good educationally as well.

After that was Vincent Chazhoor’s talk on Oracle 11g data warehousing. This year I’m trying to take in a few more database presentations as I’m due to put some new material together for Oracle University on 11g data warehousing, and I was therefore interested to hear Vincent’s take on this. I’d actually first heard him speak at ODTUG (I think) a couple of years ago on advanced dimensional modeling, again he had some good content although perhaps could have got through it a bit quicker, he ran out of time in the end and had to rush the bit I was most interested in, around summary management. There was some good content however on the new key 11g DW features and whilst most of it I concurred with, there were a couple of points I wasn’t so sure about - the range-range partitioning scheme for solving the old “do you partition on invoice date or load date” question for example - but overall it was a good session and made a few things around, say, virtual columns (main benefit - you can index and gather statistics on them) a bit clearer. One question did come up (OK, I asked it) on the interval partitioning scheme - Vincent thought that when you create an interval partitioned table, Oracle creates the metadata on the partitions upfront but not the segments, so I asked how far forward does Oracle create the metadata - for all time, for a year, for three years or whatever? Vincent wasn’t sure (neither was I), I’ll have to look this up as I suspect that the metadata doesn’t in fact get created upfront, otherwise where would it stop? If anyone has the answer on this I’d be interested to know.

After Vincent’s talk was the BIWA SIG meeting, which really turned in to “ask Matt Bedin (one of the Oracle BI PMs) any questions about Oracle BI product strategy”. Matt was good enough to take the stage for most of the session, some of the questions were around OWB vs. ODI, Oracle OLAP vs. Essbase, and whether Oracle plan to offer OBIEE as part of the Oracle On Demand service. I had to leave the session a bit early as lunch was only for an hour and we only had 15 minutes or so to go. After lunch, it was off to Dan Vlamis’ session on Essbase vs. Oracle OLAP.

Dan did a very good job of explaining the nuances and technical differences between Essbase and Oracle OLAP, and went into some good detail about the differences and similarities between the two products. I did a similar session myself at the Norwegian Oracle User Group (slides are here) and I’m doing something similar at ODTUG in June, it was interesting hearing Dan’s take on the two technologies which generally more or less matched my take on this subject.

After Dan’s session was my one on solving generalized business problems using Oracle Real Time Decisions. Seeing as my session apparently wasn’t listed in the main conference guide the turnout was pretty good, there were a few people in the audience with some good questions and I think I managed to get the point of the session across OK. Again if you’re interested the slides are here and so is the conference paper.

After my session I was just about done for the day, so I ended up missing Mike Donahoe’s talk on BI Publisher futures. Luckily Mike’s paper was up on the Collaborate website and I’d managed to take a look on the flight over, so I saw from the slides that planned features for future releases include integration with Hyperion Workspace and Hyperion SmartSpace (the widgets that ship with the latest version), support for MDX and XML/A data source (Essbase, MS AS and SAP B/W) including an MDX query builder, enhancements to the MS Word Template Builder and a web-based Layout Builder that looks like Microsoft Office 2007 and is an alternative to building report templates using Microsoft Word. If you’ve got access to the Collaborate proceedings there are some interesting screenshots of these new features in the slides.

Anyway, for me, my schedule today looks like this:

• 09:45 : Are you a Money or an Astronaut : The Oracle Advisors from a Different Perspective (a Hotsos talk)
• 11:00 : Real-Time Data Integration via CDC, Oracle Notifications and Java Messaging
• 13:45 : Data Warehouse Design Templates
• 15:30 : Robust ETL using Oracle Warehouse Builder
• 16:45 : Why I use Oracle’s OSWatcher

I’ve got a meal in the evening with some customers and partners who are using Oracle Real Time Decisions, so that should wrap up a busy day.

Up early, as I always am and met Mark for a 6:30AM breakfast. Was criticised for eating “Fruit Loops” as that’s not the sort of thing that the sane consultant eats…

Then back to the hotel room to get a little work out of the way before crossing the street to the Conference Center to get to the first session. Mark and I tend to go to different talks as there are so many to take in and by splitting up we miss less; that said we both attended one talk which seemed highly relevant to our work and of course I listened to Mark’s talk as I wanted to see how a “British” presentation worked with the local audience. In fact Mark’s presentation went well (unlike the photos I took of him speaking) with many good questions from the audience

First up for me this morning was Stephen Rokes and his presentation on building a multi-terabyte claims data warehouse. I love this type of presentation;  I always come away from them with a nugget of information that I might well put to use in one of our DW projects. In this case about adding more steps into an ETL process to minimise the number of joins per step and in Stephen’s example this gave some significant improvement in his batch run time. Another thing I enjoy is to hear about database performance on other platforms, and a two-node RAC with 18TB of storage is quite an exotic beast for me.

Met Mark to sit in on Vincent Chazhoor’s Oracle 11g data warehousing talk. I guess that Mark may write this one up, but is was a real shame that Vincent was so pressed for time.

Readers of my older posts will know that I am not always fully Oracle focused - coming from an IT services company background I was used to customers working with a whole host of technologies - that included both old established names and the newer ideas such as data warehouse appliances. So the two presentations either side of the very filling box lunch fitted well together. Before lunch Chandu Patel presented “When a data warehouse appliance will not work for you” which turned out to be more of a SWOT analysis of the technology compared against the traditional approach and after lunch Oracle’s Hermann Baer spoke about Oracle Optimized Warehouses (and the 11g features that help make data warehouses scalable)

I finished my day with a session on Oracle OLAP presented by Chris Claterbos; I have been making a point of getting to as many OLAP presentations as possible at recent events - my talk this coming Thursday is on Cube-Organized Materialized Views - and to see that my findings are similar to those found in other work is very reassuring.

I really enjoyed today, let’s see what tomorrow brings!