Andy Hayler on Federated Data Warehousing
Apart from a talent for producing criminal geniuses, the UK seems to be good at producing data warehousing experts, and Kalido's Andy Hayler is one of my favourites. His blog is always a good read, and I particularly enjoyed a couple of his recent posts, on data warehouse architectures, and the business payoff of a data warehouse project.
The data warehouse architectures posting is particularly well timed as it looks at the inevitability of federated data warehouse architectures, something that's now possible with an all-Oracle setup when you use the new Siebel Analytics-derived BI Suite. Up until now, the mantra from Redwood Shores has been to build a single, consolidated data warehouse for the entire enterprise, but in Andy's opinion, this is never going to happen due to the sheer scale of the undertaking, and in his opinion:
"... for any large corporation it seems to me that a federated warehouse approach is what you will end up with, whether you like it or not. Few companies will have the energy or resources to deliver the single giant warehouse, and even those few that do will, in reality, have a series of skunk works data marts/warehouses dotted around the corporation since such a behemoth warehouse will be a bottleneck, hard to change and inevitably slow to respond to rapidly changing business needs.
The most pragmatic approach would seem to me to acknowledge this reality and architect for a federated approach, rather than staying in denial. It is practical to build a warehouse for either a country-level subsidiary (or groups of countries) or each business line, let that deal with the needs of that particular country or business line, and then link these together to a global warehouse which deals at the summary level. The global warehouse does not need to store every transaction in the enterprise; at that level you need to know what the sales were in Germany yesterday by product, channel and perhaps customer, but not that a particular customer bought a specific item at 14:25 at a store in Rhine-Westphalia. The detailed information like this is the domain of the country-level warehouse. Because the transaction detail is not needed at the enterprise level, you avoid the problems of technical scale that may otherwise occur, and only deal with the data that makes sense to look at across the enterprise as a whole."
The second posting, entitled "Putting a little glitz into data warehousing", looks at how data warehouses are rarely the first thing you think about building after an acquisition, and yet they can be a perfect way to start delivering synergistic cost savings fairly soon after the merger.
"For example, when HBOS merged, one of the key areas for quick savings was identified as procurement. Yet to just pick one of the existing procurement systems, switch off the other and convert all the data from one to the other was estimated at taking well over a year. Instead, what they did was to implement a packaged data warehouse, map the two sets of data from each bank, and in this way get a single view of the procurement spend across torganizationion without having to convert all the data in the underlying systems. This was achieved in just three months, giving an immediate view of post-merger procurement that allowed huge business savings to achieved. For more on this award-winning project click here."
A pretty interesting couple of postings, and there's another one here about explaining Master Data Management to a business audience. Andy Hayler's got a wise, experienced perspective on data warehousing and you'd do well to bookmark his blog if you're interested in the subject.