My talk will be about Realtime Data Warehousing – it is an overview of reasons, techniques and pitfalls, but I do cover a lot of material in that hour. Of course, Change Data Capture (CDC) will be a major part of the talk; Oracle has so many options here including their recently acquired GoldenGate product set. As always, the slides will be here on the Rittman Mead site soon after I speak.

My colleague, Stewart Bryson has also had some recent thoughts about change data capture over on the TDWI group at LinkedIn.com (group membership needed); he was quite preceptive (and on the money, in my opinion) with his comment “I would hesitate to let technical limitations dictate user requirements. In today’s BI/DW market, there are very few technical limitations that cannot be solved one way or another.”

One of points I will make in my Realtime DW talk, and perhaps I need a few more slides to do it justice, is the need to profile the change you capture on the source system. Often there is a lot of “noise” that looks like change but you have no real interest in it at the data warehouse. Not all systems are “well behaved”; I have seen systems that always update a record even if nothing has changed and even systems that update each column as separate statement with its own commit.  Of course, even systems that don’t have those vices can still have columns that have no DW significance being updated and see those changes being filtered out on the data warehouse after we had already done a lot of work (processing, network bandwidth and the like) to get the data there.

The more I do this kind of work I feel there is a need to switch CDC on on the live source for a while and see the typical patten of change that occurs in a day, week, period whatever and then make decisions on how to handle this defensively downstream. Do we need to exclude certain columns that are just “noise”? What will be the impact of multiple, rapidly-occurring commits on how we handle SCD-2 dimensions? Of course we can predict what will see and come up with a proposed solution but the real source often has a few surprises up its sleeve – once a customer gave me a sequence of order statuses that an order passed through in its life-cycle except that on the actual source system the order sequence was not the same as their documentation and that would impact our reporting.

I will be giving a new presentation called “Getting Real – Data Warehouse Loading As It Happens” which will discuss some of the challenges of realtime data acquisition, starting with my premise that Real Time doesn’t really exist….there will always be some lag, even if you query the source directly. I intend to touch on replication, direct query of source, change propagation and of course the things that GoldenGate now brings to the Oracle table. There will also be space for a bit of pragmatic cheating – how to make things look real-time when they are not, and of course fit in the warning that “just because you can do something doesn’t mean that you should”.

Mark will be busy with two sessions – a Deep Dive, all-day session on OBIEE on Sunday 18th and look at OWB 11gR2 the next day; Stewart will talk about DW fault tolerance.

I am looking forward to being in Vegas – the chance to hear some good talks and to meet some old friends.

We’re located at Booth 124, Moscone South, which is on the perimeter of the outer track of booths. Basically, when you walk in through the main exhibition doors, you take an immediate left, and walk all the way down until you see the Tap and Brew, which I should point gives away free beer in the afternoons. So depending on the timing, grab a beer, and then take a right and go about three quarters of the way down until you see the Rittman Mead booth on your left. Most likely, you’ll see either Joe or myself, and probably Venkat as well. Jon and Mark will be in and out of the booth depending on how their schedules of presentations and other things go.

It’s been interesting speaking to attendees about their BI needs, and hearing the different ways that Oracle customers are using the products and the challenges they are facing. It’s also been gratifying when customers seek out our booth with their latest technical dilemma because they think we are the one’s that can help them solve it.

Today is the last day of exhibition hours, so take a minute to stop in and say hello.

It’s an exciting time for the company, as this is the first Open World with the US branch established and operating. Joe Leva and I are here along with a few other guys you may have heard of: Mark Rittman, Jon Mead and Venkatakrishnan Janakiraman. As we all travel so much, and hail from such disparate locations, we don’t often get to be in the same place at the same time, so it should be good fun to see the guys from across the pond (and across the world in Venkat’s case).

Another first for us at Open World: Rittman Mead is exhibiting this year, so come by and see us at Booth 124, Moscone South. Because I’m spending so much time at the booth, speaking to attendees about their BI and data warehousing needs, I haven’t had much time to see many technical sessions, though I did get a chance to see Mark and Venkat’s presentation yesterday… a “Deep Dive” into OBIEE and Essbase integration. There were a lot of good questions from the audience, especially those that stayed behind to speak to us, and it does appear that there are customers out there trying to make this work. It’s also evident that there is a real desire for people to get their hands on OBIEE 11g.

Joe and I will be at the booth all day… and after the Oracle Press event that will last most of the morning, Jon and Mark should be by in the afternoon. So if you have any Oracle product questions, general BI questions, or just want to come by and meet the team, we’d be glad to see you.

Speaking of first class, ODTUG Kaleidoscope never disappoints, the level of the sessions is consistently excellent. People tend to vote with their feet and the word is that attendance is up this year as well. This certainly fits with what we’ve been seeing at the presentations. This is the first conference I’ve gone to in the last 9 or 10 month that I haven’t presented at and it is nice to be able to attend more sessions. This is actually my first trip to Monterey and variation in landscapes in California continues to impress.

Stewart delivered an impressive presentation on considerations for selecting OWB. There were some particularly interesting points regarding the advantages and trade offs of the cube and dimension accelerators. The question and answer session afterward suggests that the new packaging of OWB with ODI has helped to reduce if not eliminate customer concern regarding which tool to use for new development.

Mark moderated the BI/DW panel with Mike Durran, Dan Vlamis, Jean-Pierre Dijcks and Michael Armstrong-Smith (Left to Right). There were some provocative opinions expressed regarding possible future paths for the essbase engine and how it might or might not integrate with the Oracle technology stack. Perhaps I’ll kick that bees nest in some future post. It is a topic that I think would generate a lot of discussion. Mark also presented our OBIEE optimization methodology. This approach focuses on specifically on measuring and managing the performance of the Oracle BI server. It joins our methodology for improving the performance of the Oracle BI applications. If you are reading this before today’s sessions, you will also want to stop by Mark’s presentation “Creating Hybrid Essbase/Relational OBIEE Logical Models”.

I’d like to welcome Venkat to the team. One of the reasons that Stewart and I were drawn to working with Mark and Jon and forming Rittman Mead America was their ability to identify and recruit world class talent. Venkat is a great example of such talent and we are very happy to have him on board.

A highlight of this conference for me was finally meeting Stanley. We may have been roommates at Collaborate, I really can’t say, I was pretty jet lagged at the time because I had flown there directly from Singapore. But this time we got to meet and he was good enough to pose for a picture with me, no autograph though. Stanley is Dan Norris’ Oracle Ace vest, he has his own twitter tag #stanleyace and his own blog (http://www.wtfistheacevest.com/). Apparently, he had gone diving with Dan in Monterey Bay over the weekend, and frankly he did not smell like he had showered since. This is why I only half have my arm over his sholder. Happy travels Stanley.

Though Rittman Mead is positioned squarely in the BI/DW space, in some ways, I’m still a database guy at heart (I grew up as a DBA some years ago), and I  am tempted to step out and see what the rest of the world is doing. Of course, it’s tough to talk about the Oracle Database without mentioning Tom Kyte, but as his first session runs concurrently with mine, I won’t give him too much publicity, as if he needs it. I am looking forward to his later session on SQL analytic functions because of their immense value in the BI space, obviously from the perspective of report generation, but also with what they bring to ETL development. If your ETL code isn’t packed full of SQL analytics, then you still have the database in first gear.

APEX is really a hot topic this year (as it should be), and I may attend one of the two presentations by Scott Spendolini, especially Extreme Makeover: APEX Edition. Come on… it’s a great title. Seriously, I’ve been giving a lot of thought to what part APEX may and should eventually play in the BI space, as more and more companies take advantage of it’s extremely attractive price tag (hint: there isn’t one) to try and build robust BI dashboard applications. Keep watching the blog… perhaps Rittman Mead will get a chance to do just that in the future.

Mostly, I’ll have BI/DW on the mind. Obviously, I’m interested in Mark’s presentations, as most of the BI-minded participants are, but I’m especially curious to see what he has up his sleeves for An Oracle BI EE Optimization Methodology. Sadly, as I’m scheduled to fly out on Wednesday due to client project requirements, I won’t be able to attend his other presentation Creating Hybrid Essbase / Relational OBIEE Logical Models. As with all Rittman Mead presentations, it will be available on our website, so I’ll just take a few spare minutes, download the presentation, and flip through the slides, imagining a distinct British accent as I read the content.

I’m still torn considering all the other BI options to choose from, so I’ll likely see how I’m feeling as each new time-slot approaches. I’d enjoy reading your comments concerning which presentations you are looking forward to (please mention mine even if you don’t mean it). And look for Mark and me throughout the conference; we’d love to chat with you about your own business intelligence implementations. Joe Leva may also be joining us there if his client obligations ease up enough for him to spring loose for a few days. You may have to look hard to spot us as all the attendees will be wearing Rittman Mead apparel by way of the lanyards.

Our session was in the architecture stream and went through the a reference architecture for Oracle BI, DW and EPM technologies. Entitled “Pragmatic Solutions for BI Architecture”, it builds on some of the articles we’ve written recently on the blog and proposes an approach that couples the BI Server with the Oracle database and packaged analytics to deliver reports and dashboards in a much faster timeframe, whilst retaining the flexibility to adapt the design as more requirements come in.

The feedback after the session was pretty encouraging, most of the questions were around the three layer data warehouse design, specifically the process-neutral data store that supports the more traditional star schema, and whether or not the BI Server could actually support metadata models that let you change data sources from your applications to your data warehouse as your system matures. If anyone was at the event or would just be interested in more details on this architecture, just drop me a line.

Other than that, thanks to everyone that came along, and don’t forget that Rittman Mead are running their own, free event in London on April 1st where we go into a lot more detail on this subject. Registration for this event is free and is available using this web page.

Conferences seem to start early here – and coupled with a 15 minute walk from the hotel to the Oracle Conference Center makes for a long day.

First up was the final keynote of the Summit, Jaun Loaiza speaking on Oracle Exadata. The rest of the day was in technical sessions.  Bryan Wise speaking on Securing OBIEE was well worth listening to. Although I have set clients up with LDAP security before I learnt a couple of useful tips including how to use groups within the LDAP service; in the past I shied against this and preferred to manage groups through a database look-up (and that was still Bryan’s preferred method), the idea of writing a database function that uses a call to DBMS_LDAP to return a semicolon separated list of group memberships had not really struck me, he also gave the very sound piece of advice if you are implementing SSO, don’t activate it until you have tested your security model, as soon as SSO is on the repository security model is no longer usable – make a mistake and you, the administrator, can get locked out.

One thing I like doing is looking at things I don’t see on a day-to-day basis, so the talk on “Crystal Ball” was an interesting diversion for me; then thoughts of my current project beckoned so on to a talk on Master Data Management. Lunch was a ‘birds of a feather’ type of thing – I started on one of data warehouse tables before going to join one of the OWB tables.
Stayed in the same seat for another, but this time more technical talk on Exadata and the Oracle/HP DW machine, and still did not move for Maria Colgan’s  talk ‘Oracle 11g Optimizer Uncovered’ – three of my big gripes from pre-11g data warehousing seem to have efficient resolutions in 11g; and to think of the hoops I had to go to get around them in Oracle 9.2 or 10g.  Getting cardinalities right for correlated columns, keeping optimal query plans after patches / upgrades and quickly maintaining global stats on partitioned tables have been a major issue for me when I used to operate (and not just develop) data warehouses. Her other feature, bind peeking on skewed columns is less of a problem for me as mostly I don’t see bind variables in use.

Finished a long day with talks on SOA and unstructured data.   All-in-all a great conference, I am already looking forward to the next one!