My talk will be about Realtime Data Warehousing – it is an overview of reasons, techniques and pitfalls, but I do cover a lot of material in that hour. Of course, Change Data Capture (CDC) will be a major part of the talk; Oracle has so many options here including their recently acquired GoldenGate product set. As always, the slides will be here on the Rittman Mead site soon after I speak.

My colleague, Stewart Bryson has also had some recent thoughts about change data capture over on the TDWI group at LinkedIn.com (group membership needed); he was quite preceptive (and on the money, in my opinion) with his comment “I would hesitate to let technical limitations dictate user requirements. In today’s BI/DW market, there are very few technical limitations that cannot be solved one way or another.”

One of points I will make in my Realtime DW talk, and perhaps I need a few more slides to do it justice, is the need to profile the change you capture on the source system. Often there is a lot of “noise” that looks like change but you have no real interest in it at the data warehouse. Not all systems are “well behaved”; I have seen systems that always update a record even if nothing has changed and even systems that update each column as separate statement with its own commit.  Of course, even systems that don’t have those vices can still have columns that have no DW significance being updated and see those changes being filtered out on the data warehouse after we had already done a lot of work (processing, network bandwidth and the like) to get the data there.

The more I do this kind of work I feel there is a need to switch CDC on on the live source for a while and see the typical patten of change that occurs in a day, week, period whatever and then make decisions on how to handle this defensively downstream. Do we need to exclude certain columns that are just “noise”? What will be the impact of multiple, rapidly-occurring commits on how we handle SCD-2 dimensions? Of course we can predict what will see and come up with a proposed solution but the real source often has a few surprises up its sleeve – once a customer gave me a sequence of order statuses that an order passed through in its life-cycle except that on the actual source system the order sequence was not the same as their documentation and that would impact our reporting.

With involvement from diverse groups, I wanted to make sure the presentation would be interesting from a lot of perspectives, so involving core database functionality like OWB seemed important. And this particular presentation is directed at DBAs and developers, so the non-BI representation from GOUSER might be interested in what’s going on.

If you’re new to OWB, don’t be alarmed by the focus on the newest release: 11gR2. I’ll explain in general what OWB is and how it’s positioned, moving into some of the new features that make this release the best OWB yet. Also speaking at the event will be Mitch Campbell, Director of Business Intelligence with Oracle, who will be talking about Oracle’s BI product roadmap. Following both talks, I’ll be sitting for a BI panel discussion with Mitch and Greg, to give responses to attendees questions, hopefully representing the perspectives of Oracle, their partners, and their customers.

The event is free, with snacks and refreshments provided, so hopefully we’ll see you there.

This was our first time at RMOUG, probably the biggest and best regional user group conference in the States and very similar to our UK Oracle User Group conference in terms of speakers, technical coverage and friendliness of the speakers. This year RMOUG took the opportunity to broaden their coverage into areas such as Hyperion, SOA and business intelligence, so it was also good to see people such as Edward Roske here delivering a number of Essbase and Hyperion sessions. We took our usual “OWB11gR2 New Features” presentation and gave it a DBA and database developer angle, and whilst most of the audience were new to OWB hopefully the new ODI-derived functionality still made a bit of sense. If you’re interested, the paper and presentation can be downloaded from our articles page.

Now whilst Stewart has now gone back to Atlanta, I’ve stayed on as I’m taking part in my first ever ODTUG Board Meeting. I got elected to the board late last year and whilst I know most of the board members already (having been the BI&DW SIG co-chair for the last few years), it’ll be very interesting to take part and contribute to some of the user group decisions, and to see how we can improve and build on the BI content going forward. Certainly, the BI stream agenda for this year’s Kaleidoscope, due to be held in Washington D.C. in June, is about the best BI agenda I’ve seen for a long time (including lots of sessions by the Oracle PMs responsible for the 11g release), and I’m looking forward to working with the board to increase this coverage in the future. So for me, it’s another three days work going into the weekend, then I fly back to the UK on Sunday ready to deliver a course on Tuesday. Busy times, but all good fun.

Like last year’s event, the 2010 BI Forum is focused on Oracle BI Enterprise Edition and the technologies that support it, such as Essbase, ODI EE and the Oracle Database. We have a number of the world’s top speakers on the subject, including names such as

• Gerard Braat (Oracle Corporation), on “Complex Modelling with OBIEE 11g”
• Antony Heljula (Peak Indicators), on “BI EE Architectures and Sizing”
• Venkatakrishnan J (Rittman Mead), on “Fusion Middleware 11g – ADF Business Components”
• Craig Stewart, (Oracle Corporation), on “ODI 11g: The New Generation of Data Integration”, and
• John Minkjan (Ciber), on “OBIEE Customizations: When You Are In The Kitchen, Learn to Cook”

For a full listing of all the sessions, check out the event website at http://www.rittmanmead.com/biforum2010

As well as general sessions on the 20th and 21st May, we also have a keynote by Phil Bates (OBIEE Architect, Oracle Corporation) at the event reception on the evening of the 19th May, and a special one-day OBIEE Masterclass earlier in the day by none other than Kurt Wolff, one of the original nQuire team and an expert on repository modeling and dashboard design.

The pricing for the event is very reasonable and we’ve also subsidized the cost of two nights in the event hotel, so that we can all stay in the same place over the three days. There’s also the opportunity to stay for an extra night before and after the event, if you want to arrive the night before for Kurt’s masterclass, or your flight back doesn’t leave until the Saturday.

As with last year, we’ve taken a decision to limit the number of attendees to 50, to maximize the opportunities for networking and to allow us to keep the focus on intermediate-to-advanced OBIEE topics. Last year’s attendees have already been able to register for the past week, but we’re now opening up registrations publically and there are around 25 places left. If you are interested in attending, check out the event website and register here, and hopefully we’ll see you in Brighton in May!

http://www.zoomerang.com/Survey/?p=WEB22A5U9QD93G

UPDATE: We missed one off – sorry:

http://www.zoomerang.com/Survey/?p=WEB22A63S9HTWZ

If you’re one of the potential speakers, please don’t vote for your own paper (this keeps scoring consistent across all speakers). We’ll keep the vote open for seven days from now, and contact the successful speakers at the start of February. Once all is in place we’ll then open the event for registration towards the middle of May. Thanks again to everyone who has shown interest, and for the potential speakers who have submitted some excellent abstracts.

I’ll be posting more details on the events and the venue later in January, but if you’re interesting in presenting at this year’s event, we’re looking for presentations in the following areas:

• Focus is on OBIEE+, together with supporting technologies such as Essbase, OLAP, ODI and OWB
• Presentations should be new material, aimed at experienced developers with core knowledge of OBIEE+, pus some or all of the above technologies
• Topics include design, optimization, architecture, performance, data modeling, internals, undocumented features
• Presentations with a live demo are preferred (and definitely no “marketing” sessions!)
• Total attendees will be limited to 50 to maximize networking and interaction opportunities
• There will be 10 presentations, plus “round-table” and “OBIEE development challenge” sessions
• The event will take place over two days, with attendees staying at the venue hotel and lots of social activities
• The venue is the Seattle Hotel, Brighton Marina, May 20th – 21st 2010, with the event fee waived for speakers

If you’re interested in an overview of the first BI Forum we ran last year, the following blog posts will give you a flavour:

• BI Forum 2009 Event Announcement
• The Day Before the BI Forum (Mark Rittman)
• BI Forum Day 1 (Mark Rittman)
• BI Forum Day 2 (Mark Rittman)
• BI Forum in England (Edward Roske)
• Brighton’s Biggest Bi Event (Edward Roske)
• Brighton 2009 (John Minkjan)
• BI Forum Brighton Day 1 (@lex Hansal)
• BI Forum Brighton Day 2 (@lex Hansal)
• Oracle BI Forum – Brighton Conference – Update (Venkat J)
• Oracle BI Forum – Updates – Day 1 (Venkat J)
• Final Event Re-Cap
• Flickr Photo Set

If you’re an experienced OBIEE, Essbase, OLAP, ODI or OWB developer or consultant, and you’d like to share your experiences with the rest of the expert Oracle BI community, this will be a great event to attend this year. If you’d like to put a paper forward, send an abstract including title and the area you’d like to cover (no more than 200 words) to mark.rittman@rittmanmead.com. The deadline for abstracts is January 22nd 2010, and once the papers are in we’ll open up voting to everyone who attended last year, and anyone who registers an interest following our official announcement of the event which we plan to post on January 4th 2010.

As with last year, the model for this event is events such as the Miracle Database Forum and the Hotsos Symposium, events that runs every year in the States and Europe that focus on Oracle database internals and performance tuning; the aim is to create a similar forum for Oracle BI developers who are interested in the technology behind the tools we use. Speakers get their attendance fee waived (you’ll still need to cover your own travel and hotel bill), and to recap the deadline for abstracts is January 22nd 2010 with abstracts to be sent to mark.rittman@rittmanmead.com.

For next year’s Kaleidoscope we’ve taken a leaf out of the Hyperion committee’s book and really tried to source some excellent presentations, with the plan being to organise a chair for each day’s stream and maybe have a social event one of the evenings. Fingers crossed, 2010 should see the release of OBIEE 11g together with ODI 11g, and we’ve recently had the 11gR2 release of Oracle Warehouse Builder. Given the amount of new releases and the interest this will generate, we decided to focus the presentations on new features and best practices, and from my perspective some of the highlights in this stream include:

• Implementing Right-Time Data Loading with Oracle Warehouse Builder – David Allen
• Oracle Data Integrator 11g – New Features and Directions – Francois-Xavier Nicolas

David Allen is one of the architects behind Warehouse Builder, and I co-presented with him at the recent Open World in San Francisco on the new features in OWB 11gR2. This presentation looks as if it will cover some of the change data capture and trickle-feed features in the new release, and it’ll be great to have David presenting at Kaleidoscope and answering questions from the audience. One not to miss for Warehouse Builder fans.

Also, if you’ve been following the news on Warehouse Builder, you’ll know that it’s due to merge functionality with Oracle Data Integrator over the next few years. We’ve recently had a new release of Warehouse Builder that incorporates some ODI functionality, so it’ll be interesting to hear what’s coming up in the next release of ODI proper, to see where Oracle are taking the product and to see if any OWB functionality makes its way into the product. FX is based in France and was one of the original Sunoposis team, so it’ll be great to have him over in DC and to hear first-hand from him what’s coming along for ODI in 2010 and beyond.

• OBIEE 11g Integration with Oracle ADF Business Components – Palaniappan Chidambaram
• Web Services and Application Integration with Oracle Business Intelligence EE Suite – David Granholm

I met Palaniappan at our BI Meetup at Open World, and he’s heavily involved in some of the back-end development for the new OBIEE release. Integration with ADF business components, either as data sources or for write-back, is one of the lesser-known new features in OBIEE 11gR2 and it’ll be great to hear from the developers how this feature can be used.

David is part of the product management team and co-incidentally was the first person I heard present on OBIEE, back at Kaleidoscope in 2006. Like Palaniappan, David’s session will be covering integration with the SOA and Java/.NET stack and will be focusing on web service and application introduction. These session should be of value to both OBIEE and Java developers, and I’ll make sure I’ve loaded up a copy of JDeveloper before I come along so I can get some tips whilst I’m at the conference.

• Best Practices for Performance, Scalability and Reliability with Oracle BI Enterprise Edition – Mike Durran
• Oracle BI EE Metadata Modeling Best Practices and Tips for Concurrent Development – Alan Fuller

One of the bits of feedback we got from last year’s Kaleidoscope was that people wanted to hear more intermediate, advanced and best-practice sessions. Mike, Alan and the team from Oracle took this on-board and have come up with a number of sessions that address developers who already have a good understanding of the various toolsets. This best practice session for OBIEE should be good, with Mike covering some of the design, administration and architecture practices, and Alan covering data modeling and team development practices, recommended by the product and implementation teams at Oracle.

• Essbase and Oracle Business Intelligence Integration – Real World Case Study – Shyam Nath
• Oracle Business Intelligence Applications Essbase Integrator – Alaric Thomas

You may well have missed it at Open World (and my post on the session slipped out during a weekend just after), but Alaric Thomas gave a very interesting session at the last Open World on a project going on within Oracle to provide integration between the BI Applications, Essbase and SmartView. The BI Apps Essbase Integrator provides metadata and data synchronization between the BI Apps data model and an Essbase database, allowing you to start up SmartView from a link in the BI Apps dashboard and analyze your data using an OLAP cube, and should be released at some point in 2010 for use with current versions of the BI Apps and Essbase. Alaric’s talk will be about the technology behind this product, whilst Shyam’s will be talking about a recent project that accomplished a similar thing using custom integration. One for the BI Apps and Essbase fans out there.

Other sessions to look out for are one by Toufic Wakim, SmartView product manager, on the new release of SmartView better designed to work with OBIEE; two sessions by Oracle ACE Directors Edward Roske and Tracey McMullen on integrating OBIEE and Essbase, one my colleague Stewart Bryson on new data warehousing features in Oracle Database 11gR2, and two by me on infrastructure and metadata changes in OBIEE 11g, and the new features in Answers, Dashboards and BI Publisher 11gR2.

As well as the BI stream I’m chairing, there is also a database development stream that features two sessions by Jean-Pierre Dijcks on Oracle and Hadoop, and Oracle Partitioning, and separate streams for Hyperion Applications, Essbase and “Hardcore Hyperion”, making Kaleidoscope the must-attend BI, DW and Hyperion event in the States in 2010. Registration is open now, and numbers are limited to ensure the “community feel” of the event. I’ll be there, and I hope some of you will be able to make it too.

Congratulations also to Gareth Webb who won our prize of an iPod Touch on the last day of the conference, which was presented to him by Chris Raby, our Business Development Manager. Thanks also to the UKOUG for organizing the competition, and for all the rest of the hard work they put in to make the event a success.

It’s back to work for me now, with a day back home today and then back up to London tomorrow. I’ve got a few things on my “to blog” list following the conference, including a proper look at Golden Gate, but for now it’s time to unpack and get things ready for tomorrow.

This year we’ve brought most of the consulting team along to the conference, as a way of getting to see people who are often working away on-site, and so that everyone can get to meet the great delegates, speakers and partners at the conference. Here’s the team at the start of the day, having just set up the demo and waiting for the opening of the exhibition.

And here’s things at the end of the day, during the Exhibition Hall Drinks Reception. We had our famous Rittman Mead beer available again, and it was great to catch up with people such as Paul Vallee, Connor McDonald, Jonathan Lewis, Lisa Dobson and everyone else who stopped by.

The two sessions we ran today were on Essbase and OBIEE Integration, and on the new 11gR2 release of OWB. The Essbase and OBIEE session went well; we had Venkat doing the demos, myself doing the positioning, and then a great questions and answer session at the end where we discussed attendees experiences with the integration and gave some feedback on what worked for us. The slides are available here if you’re interested.

The second session was particularly well attended and was on the 11gR2 version of OWB. Now if you’ve been following our blog, we’ve posted a series of articles on this new release, and we’re particularly excited by the new code template (ODI) features in this release, especially as the integration between the ODI and OWB technologies seems to have been done so well. We were lucky enough to be joined by Craig Stewart, our Oracle ODI representative in EMEA and someone we know well from his attendance at various user group events and indeed our own BI Forum earlier in the year in Brighton. Craig was able provide a “hot off the presses” roadmap slide on OWB and ODI 12g which he’s kindly agreed for me to reproduce here:

What we can take from this are a few points:

• Going forward, the OWB product name will be de-emphasized in favour of “In-Database ETL”, to differentiate itself from Oracle’s strategic data integration platform, ODI
• The 11gR2 version of OWB will be the last in it’s current packaging, and license-wise it’ll now be incorporated into ODI Enterprise Edition. The free OWB functionality will still remain but will be renamed “In-Database ETL” as above.
• ODI 12g is the target version for integration between the ODI and OWB platforms
• There will be an ODI 11g release out in the first half of next year (usual disclaimers apply
• ODI is the strategic data integration platform for Oracle

Again, the slides are available for download here if you want to take a look, and they cover code templates, heterogeneous sources, OBIEE integration, change data capture and web service integration.

Craig is actually going to join me again tomorrow for the ODI and Oracle BI Applications session I’ll be running (14.30 – 15.30, Tuesday, Hall 10b), so again if you’ve got questions on the ETL direction for the BI Applications and the role that ODI will play, come along and we’ll be pleased to answer any questions.

Finally, if you’re around tomorrow (Tuesday) evening, we’re running a BI “Appreciation” drinks session at the Pitcher and Piano, Brindley Place, from 6pm – 9pm. If you’re a BI user, customer, developer, partner etc, or you just know us and would like to join us for drinks and canapes, come down in the evening and it’ll be great to see you.

The evening was organized by Debra Lilly and myself, with Todd from the OTN team kindly bringing along the credit card and passing along messages from Vikki, Lillian, Justin and the team back from HQ. We had an excellent turnout with 25 ACEs and ACE Directors from around the world, including such names as Steven Feuerstein, Bryn Llewellyn, and Mogens Norgaard, and it was a great chance to catch up with people such as Marco, Dimitri, Dan Morgan, Tim and others that I tend to see only at other conferences around the world.

Today’s a busy day for me, with the following two sessions running later today:

• “Oracle BI EE and Essbase Integration Step-by-Step”, Monday 14:45 – 15:45, presented by myself and Venkat Janakiraman
• “Oracle Warehouse Builder 11gR2 New Features”, Monday 16:15 – 17:00, presented by myself

We’ve also got a fun demo running on the stand (Stand 50) where we’re using OBIEE as part of a football management simulation, where you can pretend to be an investor looking to put money into any of the Premier League teams and seeing, through the use of analytics, writeback and the dashboard where this money would be put to best use. I’m trying to persuade Chris to rig the numbers so that Arsenal always get relegated and Spurs win the league, but I think at least the latter is a bit beyond the powers of Oracle BI – perhaps we need to deploy Essbase in the background. Anyway, if you’re at the conference then be sure to pop by stand 50 if you get a moment, otherwise hopefully I’ll see a few people at the various conference sessions.