Realtime Data Warehouses
April 8th, 2010 by Peter Scott
In a little over two weeks I will be giving my Real Time Data Warehousing talk at Collaborate 10 in Las Vegas. I will cover a variety of techniques and not just change data capture. However, in my opinion, CDC plays a major part in the practical implementation of a realtime data warehouse. It is not the whole story as CDC is about data propagation; we still need an ETL component to consume those changes and publish them to the reporting layer of the DW. Depending on what needs to be done with the data (such as processing slowly changing dimensions and building aggregations) this step can add significantly to the latency of the event to publish process. I will cover this in my talk – and here on the Rittman Mead blog after Collaborate has finished.
No talk including Oracle CDC would be complete without mention of GoldenGate; Mark has recently blogged about consuming GoldenGate captured data with ODI. For those that don’t know, GoldenGate is an Oracle acquired company that specialised in data replication that is very fast, able to handle large data volumes and able to support heterogeneous source and target platforms both in terms of database and operating system. One combination I aim to try in a few weeks time is pushing change from an Oracle database into an Oracle TimesTen in-memory database.
April 9th, 2010 at 3:11 pm
I have been asked off-line why I want GoldenGate to push data into TimesTen and not use it to extract data from TimesTen back to a traditional database. Well the simple answer to that is that of now there is not a GoldenGate agent to extract TimesTen changes. On the otherhand pushing change from MS SQLServer, IBM DB2 and Oracle into GoldenGate is fully supported; this ability to push change from non-Oracle databases is very welcome.
April 15th, 2010 at 10:57 am
I’m very interested in reading more, post-Collaborate.
Real time date warehousing seems to be a very immature area. And I have a lot of trouble seeing past the marketing blurb of the various components in the Oracle DW/BI stack these days.
April 17th, 2010 at 8:52 pm
@Dominic I will try to do the real time DW talk in the UK sometime.
I have not be able to get across the Atlantic – there have been no flights for the past four days because of the Icelandic volcano ash-fall closing most of Northern Europe’s airspace. So I will not be at Collaborate 10. We are investigating whether a colleague would be able present on my behalf. More on that later
August 17th, 2010 at 7:37 am
I would be very interested in realtime analytics; please let me know of any talks you have lined up in the UK.
October 13th, 2010 at 11:03 am
Could Goldengate repilcate from LDAP? and How?