Planning for ODTUG Kaleidoscope 2008

May 22nd, 2008 by Mark Rittman

odtug.jpg I’m writing this posting mid-way across the North Sea returning from the Swedish and Finnish User Group Conference, and looking in my diary I’ve noticed it’s only a few weeks until it’s time for the ODTUG Kaleidoscope 2008 conference in New Orleans. As the flight’s a couple of hours I thought I’d put my slides together and take a look at the agenda that I’d downloaded before I came over.

ODTUG has always been my favourite of the two US-based user group conferences as it’s small enough to get to meet everyone, and it’s 100% focused on developers. This year I’ve had a hand in putting together the BI, data warehousing and Essbase tracks and there’s some particularly good sessions on OWB11gR2, Oracle Business Intelligence Enterprise Edition, Oracle BI Publisher, Essbase and of course Discoverer. It’s always good to meet up with an hear such presenters as Jean-Pierre Dijcks (OWB), Dan Vlamis (Oracle OLAP and Essbase), Michael Armstrong-Smith (Discoverer), Tim Tow and Edward Roske (Essbase), and as it’s a general Oracle development conference I can usually slip into some of the more general sessions on products such as ApEx and SOA Suite.

This year I’m running two sessions myself, the first on “A Next-Generation Oracle BI Architecture” where I’ll be putting forward a development methodology for Oracle’s new BI tools, and the second being an extended session on the last day called “Advanced Development using Oracle BI Suite Enterprise Edition” where I’ll be talking about and demonstrating the following OBIEE features:

  1. Model-first development
  2. Adding additional data sources
  3. Time-Series analysis
  4. Fragmentation
  5. Row-Level and Subject-Area security
  6. Aggregate Persistence
  7. Cache Management

Apart from the regular sessions, we’re running an Oracle BI and Essbase Experts Panel on the monday where myself, Michael Armstrong-Smith, Tim Tow and Edward Roske will be answering people’s questions, and on the Sunday before the main conference there’s an all-day Essbase symposium chaired by Tim and Edward that aims to emulate the old “Essbase Top-Gun” event that used to run prior to the acquisition by Oracle.

If you hurry the Early Bird registration offer is still available, and the event itself runs from June 15th - 19th in New Orleans, USA. If you’re going, it’ll be good to see you there.

Essbase Dimension Member Aliases and Unique IDs

May 22nd, 2008 by Mark Rittman

If you’ve read my recent OTN article on Essbase and you’re an old Essbase hand, one of the things you might have picked up on what that I defined my dimension members using the descriptive columns in the SH Sample Schema dimension tables. For example, in the Product dimension that I built, I defined the members as being taken from the SH.PROD_NAME column, and the PROD_SUBCATEGORY_DESC column, and the PROD_TOTAL column. If you’re on the ball and you know the SH schema, you’d have noticed that I missed out the PROD_CATEGORY column, and if you know anything about Essbase or data warehousing you’d normally have used the various ID columns (PROD_ID, PROD_SUBCATEGORY_ID) instead of the descriptions as the descriptions (a) may not have been unique and (b) may have included characters, double quotes (”) for example that aren’t allowed in member names. For the purposes of keeping the article simple though, I used descriptions instead of IDs as using IDs makes things a bit more complicated, and I deliberately missed out columns that I knew would cause uniqueness problems. In this posting though I’ll go through what you’d do to add these in, just in case anyone is interested in how you should do things if you’ve got a bit more time.

If you take the SH schema, you’ve got five dimension tables with a number of columns that form the levels in the dimension hierarchies. Products, for example, has four levels made up of product, product subcategory, product category and total products. If you use the descriptive columns as your dimension member names you’ll hit a “member names not unique” issue if you include all of the levels; you also hit issues when some of the descriptions contain double-quotes and in the article, I demonstrate how you can replace these with the word “inch” so they don’t cause errors.

Now what in reality you should do is to use the IDs as member names instead, and use the “Alias” feature to create descriptive, aliases for these IDs. Most of the Essbase client tools (including OBIEE, funnily enough) will then either show the aliases on demand or instead of the IDs in the first place. In Oracle OLAP terms this is like defining the dimension member as being the ID, and the long and short descriptions as being the description.

When you go down this route though a problem you’ll hit fairly early on is that the ID values aren’t unique across each dimension, and especially not across the whole Essbase database. Now I’m not an expert on this but my findings so far are that dimension member IDs need to be unique across the whole Essbase database, so you need to find some way to ensure that each ID is unique, which you usually do by appending characters to the start of the ID to represent the dimension and the level name.

So let’s take a look at how the SH cube could be correctly loaded, taking all of this into account. I go into Essbase Integration Services and define my initial OLAP Model against the SH schema, like this:

alias1.jpg

Then, I go in to each of the dimensions and start creating the default hierarchies. To take the product dimension, I create the hierarchy using the ID columns rather than the descriptive columns, like this:

alias2.jpg

Now as things stand with this, there are going to be two issues when I come to use the data. Firstly, whilst the IDs used in these columns are unique across the hierarchy levels, they are re-used in other dimensions and your data load would fail because of this. To make your ID values unique you therefore need to press the Transform button and append a character, in my case “P”, to the front of each ID to make the values unique to the product dimension.

alias3.jpg

You can then repeat this process for all the other dimensions except the customer dimension, where the IDs aren’t even unique across all hierarchy levels. To get around this additional problem you need to add another prefix character to each level, so that your prefixes are C1, C2 and so on, which then makes each ID unique across all levels, like this:

alias4.jpg

So at this point you can use Integration Services to derive your OLAP Metaoutline, and then do a member and data load to create your Essbase database. All of this should work fine, but when you come to display your data in a tool such as the Excel add-in, all of your product names, for example, are these IDs, so you can’t tell which product, which product category and so on are which.

alias5.jpg

This is where Aliases come in. If you now go back to Integration Services and load up the Metaoutline again, you can right-click on any dimension member (hierarchy level), select Properties from the context menu and then select the Aliases tab to define the alias. Using this tab, you select another column in the dimension source table that contains the alias (i.e. description), like this:

alias6.jpg

Once you’ve worked through all of your dimensions and hierarchy levels, your metaoutline should look like this, detailing the alias columns that you’ve specified:

alias7.jpg

If you then rebuild and reload your Essbase database using these new definitions, aliases will then be available for all of your dimension members. Then, you can start for example the Excel Essbase add-in and take a look at your data. Initially, it looks exactly the same as before, with the IDs being shown in the spreadsheet, but if you bring up the Options dialog for the Excel add-in, you can tick a box to specify that Aliases are shown instead of member names, like this:

alias8.jpg

Then, when you take a look at your spreadsheet, it’s the aliases that are displayed instead of the member IDs.

alias9.jpg

So there you go. Whereas in the OTN article I kept things simple and just used the description fields as my member names, in a real implementation you’d be better off using the IDs, with suitable prefixes, instead. When you use an Essbase client tool you’ll normally find an option to display dimension member aliases instead of the member names, and Oracle Business Intelligence Enterprise Edition automatically uses the aliases when you add your Essbase database to your semantic model. It’s actually quite straight forward to implement although it adds a few steps to your implementation plan, but once you get it set up it’s a bit more elegant than the simple solution. One further point if you’re particularly interested is that you can in fact define your database so it will allow duplicate member names, but this adds its own complexities and it’s generally accepted that you should try and keep all member names unique, across all dimensions.

OWB 11gR2 beta

May 22nd, 2008 by Jon Mead

We have been accepted onto the OWB 11gR2 beta program, so over the next few months we will be putting the new features (and some of the old ones) through their paces. There are some pretty strong new features with the product, we are unable to provide details of these at the moment, but at a later date we will post articles about the new features and provide some more detailed information and walkthroughs of how they can be used.

UK Higher Education Seminar : Wed 25th June, 2008, Reading

May 21st, 2008 by Mark Rittman

If you’re in the UK Higher Education Sector and are users of Oracle’s business intelligence software, one of the things you’re probably particularly looking forward to is the inclusion of Oracle Business Intelligence Enterprise Edition in the “Campus License” agreement. We’re particularly pleased therefore to be putting together a one-day seminar towards the end of June where details of the new campus agreement are being made available, and we’ll be able to show you about how you can apply Oracle’s new business intelligence technologies to your organization.

We’re running the event in conjunction with Oracle, Griffiths Waite and DSP Global, and the UK Oracle User Group wil be on-hand to talk about the results of an HE survey that they’ve been recently running. It’ll be an excellent opportunity to hear how other HE organizations are using Oracle BI Suite Enterprise Edition, tools such as Warehouse Builder, Essbase and Data Integrator, and how they are applying them to the areas of student admissions and course management.

Places are free but limited to the size of the venue, at Oracle’s offices in Thames Valley Park, Reading. If you’re interested in attending then fill out the form here and we’ll see you later in June.

Synchronizing mappings and process flows using OMB Plus

May 20th, 2008 by Jon Mead

I was recently in a situation where I had to delete a package (transformation) in OWB and then re-import it from the database, because I had deleted it and re-imported it, every mapping and process flow that referenced the package lost its reference to it. I could have gone through each mapping and process flow and synchronized inbound, manually selected the function, but this would have needed to be done hundreds of times, so I looked at automating the problem using OMB Plus. I would write a script and execute it using the OMB Plus window in the Design Center.

Mappings

The mappings were relatively simple, despite losing the reference, the operators retained the function name. The function was bound to a pre-mapping operator, so using the following code I was able to iterate through each operator and synchronize each of the functions:

foreach opName [OMBRETRIEVE MAPPING '$mapName' GET PREMAPPING_PROCESS OPERATORS ] {
    set fncName [OMBRETRIEVE MAPPING '$mapName' \
    OPERATOR '$opName' \
    GET PROPERTIES(FUNCTION_NAME) ];
    switch -regexp $fncName {
    ^.MAP_START.$ {
      set opRef "/$projectName/$ctlModName/$ctlPkgName/MAP_START";
      puts "Synchronizing $opName to $opRef";
      OMBSYNCHRONIZE FUNCTION '$opRef' \
      TO MAPPING '$mapName' OPERATOR '$opName' \
      USE (RECONCILE_STRATEGY 'REPLACE', MATCHING_STRATEGY 'MATCH_BY_OBJECT_ID');}
    default {puts "Function not changed.";}
 }

Note: there are a number of variable in the code that I had already initialised.

Process Flows

Process flows were much more difficult – there was no way of getting back the reference and even if I was able to derive the transformation from the name of the activity I was still getting errors when trying to synchronize. So the only way to do it was to delete the transformation activity and re-create it. However first I would need to take a copy of each transition in the process flow and then delete them – this is because they would be deleted anyway once the transformation activity was deleted.


foreach transName [ OMBRETRIEVE PROCESS_FLOW '$flowName' GET TRANSITIONS ] {
  set src [ OMBRETRIEVE PROCESS_FLOW '$flowName' TRANSITION '$transName' GET SOURCE_ACTIVITY ];
  set dest [ OMBRETRIEVE PROCESS_FLOW '$flowName' TRANSITION '$transName' GET DESTINATION_ACTIVITY ];
  set cond [ OMBRETRIEVE PROCESS_FLOW '$flowName' TRANSITION '$transName' GET PROPERTIES (TRANSITION_CONDITION) ];
  lappend transPropList [ list $transName $src $dest $cond ];
  OMBALTER PROCESS_FLOW '$flowName' DELETE TRANSITION '$transName';
}

From the name of the activity I was then able to derive the required function.



    switch -regexp $actName {
      START_EXECUTION -
      EXECUTE_MAPPING -
      COMPLETE_EXECUTION -
      MAP_START -
      MAP_END -
      DATA_CONSISTENCY_CHECK {set actRef "/$projectName/$ctlModName/$ctlPkgName/$actName"}
      ^EM_.* {set actRef "/$projectName/$ctlModName/$ctlPkgName/EXECUTE_MAPPING"}
      default {set actRef [OMBRETRIEVE PROCESS_FLOW '$flowName' ACTIVITY '$actName' GET REFERENCE]}
     }    

And then for each activity I would have to get all the parameter details.


    foreach paramName [OMBRETRIEVE PROCESS_FLOW '$flowName' ACTIVITY '$actName' GET PARAMETERS ] {
      set paramValue [OMBRETRIEVE PROCESS_FLOW '$flowName' ACTIVITY '$actName' PARAMETER '$paramName' GET PROPERTIES (VALUE) ];
      set paramBinding [OMBRETRIEVE PROCESS_FLOW '$flowName' ACTIVITY '$actName' PARAMETER '$paramName' GET PROPERTIES (BINDING) ];
      lappend paramsPropList [ list $paramName $paramValue $paramBinding ];
    }

I then had to re-create the activity and reinstate all the parameter properties.



    OMBALTER PROCESS_FLOW '$flowName' DELETE ACTIVITY '$actName';
    OMBALTER PROCESS_FLOW '$flowName' ADD TRANSFORMATION ACTIVITY '$actName' SET REFERENCE TRANSFORMATION '$actRef';
    foreach paramProp $paramsPropList {
      set paramName [ lindex $paramProp 0 ];
      set paramValue [ lindex $paramProp 1 ];
      set paramBinding [ lindex $paramProp 2 ];

      if {$paramName != "" } {
        if {$paramValue != ""} {
          OMBALTER PROCESS_FLOW '$flowName' MODIFY TRANSFORMATION ACTIVITY '$actName' \
          MODIFY PARAMETER '$paramName' SET PROPERTIES (VALUE) VALUES ('[OMBToSettableString $paramValue]');
        }
        if {$paramBinding != ""} {
          OMBALTER PROCESS_FLOW '$flowName' MODIFY TRANSFORMATION ACTIVITY '$actName' \
          MODIFY PARAMETER '$paramName' SET PROPERTIES (BINDING) VALUES ('$paramBinding');
        }
      }
    }
    unset paramsPropList    

And finally add all the transitions back in.



    foreach transProp $transPropList {
      set transName [ lindex $transProp 0 ];
      set transSource [ lindex $transProp 1 ];
      set transDest [ lindex $transProp 2 ];
      set transCond [ lindex $transProp 3 ];
      if {$transCond != "" } {
        OMBALTER PROCESS_FLOW '$flowName' \ADD TRANSITION '$transName' FROM ACTIVITY '$transSource' TO '$transDest' \
        SET PROPERTIES (TRANSITION_CONDITION) VALUES ('$transCond');
      } else {
        OMBALTER PROCESS_FLOW '$flowName' ADD TRANSITION '$transName' FROM ACTIVITY '$transSource' TO '$transDest';
      }
    }

    unset transPropList    

As you can see from the code this took to fix it wasn’t a trivial task, but unfortunately there was not another option, so be warned if you are about to do this.

People vs Tools and Techniques

May 17th, 2008 by Jon Mead

One of the things I am doing at the moment is reviewing our delivery methodology, looking at how different techniques and can be used with the current stack of Oracle BI tools to effectively deliver projects. This involves looking the tools currently on offer, such as OBI EE or Warehouse Builder and a number of different design techniques such as dimensional modelling, dashboard design and various agile/RAD methods, and seeing what we think works the best.

My path into IT was a Software Engineering degree, so when setting out on these types of tasks I naturally revert to this subject area to feed into the process. One of the references I was using was book Facts and Fallacies of Software Engineering by Robert Glass (review here – I would also recommend the whole Coding Horror blog).

The book starts with a section about management and a couple of comments struck me. First he quotes Alan M. Davis (here and here – not the comedian):

Good management is more important than good technology.

And the first two facts in the book are:

The most important factor in software work is not the tools and techniques used by the programmers, but rather the quality of the programmers themselves.

and

The best programmers are up to 28 times better than the worst programmers, according to “individual differences” research. Given that their pay is never commensurate, they are the biggest bargains in the software field.

These are long established facts/opinions (maybe controversially) and the cornerstone of books such as Peopleware.

So, I then have to ask myself if we are wasting our time putting a lot of effort into setting out a project methodology, should we just concentrate on recruitment? If we get the right people in, will our projects always be successful? As we are a service provider/consulting company trading on providing an expert service, based on experts we employ, we hope that this is something that we have got right, and (with my selling hat on), I believe that’s one of the reasons we are used.

Why care about tools and techniques?

Taking tools as an example, say an organisation is fully committed to using analytics for competitive advantage and this was driven and supported from the highest level in the organisation, then would it really matter if they use the Oracle stack or Cognos (IBM)? Sure, it may be easier to integrate if they chose the same vendor as their transactional systems, but the choice wouldn’t be the critical factor that determined the success of the project, the implementation of it would be. As a quick aside there is an interesting article here looking at the offerings of each of the ‘big four’ BI vendors.

Similarly with techniques, there are probably a number of approaches that could work, HOWEVER as a service provider/consultancy I think it is important to have a clear approach. We offer services in a niche area, and organisations will often get us in because they do not have not expertise, or enough of that expertise in house. I feel if we can initially give people a roadmap of how the project will progress, what resources will be required when and when deliverables are expected then this has got to be a good starting point, and I think our clients look to us for something like this.

I think the really important thing to realise is that by following this (blindly or otherwise) will NOT just mean the project is automatically successful. It is everything that happens after day one that will determine whether the project is successful, and this comes down to people, how the project is managed (some of which will come from our methodology) and how good the developers/consultants are. Similarly with tools, there will be features and functionality that differentiate the tools and chosing the right one will give you a better chance of success, however what is important is what the development team do with that tool after day one when it has been chosen. That is what will determine the success of the project.

Essbase and OBIEE Article Now Up On OTN

May 17th, 2008 by Mark Rittman

My new article on Essbase and Oracle BI EE has just gone up on OTN. It covers the steps you’d need to go through to build a simple Essbase database based on the SH Sample Schema that comes with Oracle 9i, 10g and 11g, and how you’d then plug that database into Oracle Business Intelligence Enterprise Edition, so that you could report on it using Oracle BI Answers and Oracle BI Dashboards.

All the software it uses is available for download on OTN, and now Essbase is on version 9.3.1 it’s an easy install without any of the license management stuff you used to have to do on earlier releases. Take a look if you’re interested, add comments here if you’ve got any questions after working through the article.

Rapid development using OBI EE (and what about the Data Warehouse?)

May 16th, 2008 by Jon Mead

We are often told that one of the benefit of OBI EE is the speed and ease of development. Data sources can easily be added into the system, users can then quickly build queries and the results are easy to distribute. While I completely support this, to me this leaves a few questions when you go beyond the slick sales demos: do we need a Data Warehouse? How do we deal with data quality? How do we test? How do we ensure our great looking reports get from our development environment to our production environment and still display the same data?

This posting just concentrates on the Data Warehouse. Think of the project if we didn’t need a Data Warehouse, initially in terms of cost: no database license required, no ETL/Data Warehouse development required, no ongoing maintenance and maybe in the eyes of the business no black hole of money and time where the hairy developers go away, grumble about data quality and take longer than everyone thought. Think of all this new way from a business point of view: there is a new reporting requirement, they sit down with a business analyst, add the data source into the physical and then business model, graphically create the joins, and 10 minutes later the data is on the CEO’s dashboard, marvelous, and not a hairy developer type to be found anywhere in the process, so no cost and no hold-ups. But wait a minute: why have we been spending money developing Data Warehouses over the past couple of decades?

What was the Data Warehouse actually doing?

For starters, they prevented queries being run against the live transactional systems. Query and analysis tools can be very powerful, and hence can generate complex queries. If each of these queries was being run against the live systems then performance would be impacted, plus the reporting system would be dependent on all the systems being live all the time. Thus loading all the data into a ‘reporting database’ protected the transactional systems.

But what about our real-time Data Warehouse, managers need up to the minute information to make informed decisions? Real-time Data Warehouses can be a double-edged sword, it is great to have up the minute information, but the downside is that reports keep on changing, it becomes more difficult to reconcile information or to get a consistent view of the organisation. Sometimes it is actually useful to have a static view of the data for 24 hours. Real-time Data Warehousing can also be implemented using replication features like Change Data Capture in the database if required.

Data Warehouses also offer a number of functional advantages:

  • they can store historic data that may have been archived from the transactional system.
  • they can store the history of dimensions, so facts can be correctly categorised when they happened.
  • they can store data from different systems.
  • they can store the data in a way that makes it easy and efficient for users to query, this means that a large volume of data can be accessed and used effectively.
  • they can use features like partitioning, bitmap indexes, materialized views to further speed up those queries.

What were those hairy developers actually doing?

They are writing ETL programs (or using a tool to) transforming the data from a number of disparate systems, and possibly a number of disparate organisations so that it can be presented in a unified model. This process can be easy for a small young company, however for larger company, with a history of acquisition this is far more complex. The reason this process can take such a long time is that it can be very hard to combine data from different systems. In the world of OBI EE we would create a join between the different tables – this is great in theory, but only really works if the keys are aligned between the systems. Generally they are not. Add to this the fact that the data quality maybe of poor quality and suddenly you find that the join is ‘losing’ a large number of records, or they are being incorrectly categorized.

The ETL process, and the hairy developers, will address this process and actually get in a state it can be compared across systems. I have worked on projects where the ETL team have lead/driven the business in the whole data quality and cleansing process.

So to turn the argument on its head, if we can do all this with the ETL process and in our Data Warehouse, and this does add value, why are we hacking things together in OBI EE, surely then everything should be done at the back end? Again there is some truth in this, if we are going to define complex time series calculations, and we can do them during the ETL phase, then it supports the ‘one version of the truth’, it means everyone will be using the same calculation and it can be rigorously tested.

If we had no constraints on time or money we may do everything at the back-end, or in the ETL, however the reality of it is that there just isn’t time (or money), or we often don’t know what the requirements are until later in the project. If we have written the ETL code and it is tested and working correctly and the business decides it needs another calculation then it may be a lot more practical to write it in situ in the report, or in the business model than to rework (and hence retest) the ETL code.

This is where OBI EE and an iterative development methodology come in. In a typical BI/DW project requirements are initially stated at the outset of the project, then they are then added to, changed and refined throughout the process. Users do not know what they want to report against until they see the data in a report in front of them – it is only then can they see the possibilities. To take the example of the calculation and how this could be implemented using this methodology and OBI EE: initially it could be prototyped directly in Answers. It could then be reviewed, refined and tested. Once agreed the calculation could be added to the business model, this way it would be available to all the users and they could add it to reports, this may lead to changes, or finding cases where it doesn’t work correctly. If the calculation becomes a core part of the system, is found to be stable and performance could be improved by materializing it, then it may be added to the Data Warehouse and hence the ETL code. We find that as the requirement becomes more stable it filters from front-end tools back into the database Data Warehouse.

This gives us the foundation of our methodology for approaching projects. It is initially important to gather and then implement/prototype the core or high priority requirements in the first few iterations of development, our understanding of these requirements only needs to be ‘good enough’ at this point and it is important to get them in front of the users. OBI EE can be used to do this. As they are then reviewed and refined they can be incorporated further into the system.

Once we requirements reach a certain stability they can be passed back toward the back-end of the system on future development iterations. This means that the more complex coding tasks, the ones that take time can be carried out on better understood requirements. This is where the Data Warehouse fits in and where it still can and does add value.

Importantly OBI EE gives us the ability to handle these changes seamlessly, we can change the physical objects supporting the business model. Mark wrote a series of postings about this process entitled Migrating OBIEE Logical Models to use a Data Warehouse, these describe how to re-wire the business model to a Data Warehouse.

So this means that old school ways, such as Data Warehousing, are not dead, interesting the latest Kimball Design Tip (I think you have to subscribe to get access – it’s free) discusses how names of the concepts of what we now call BI&DW have changed. What will make a project successful is finding a balance between how to optimally use each of the technologies, depending on user requirements, complexity and quality of source data and a myriad of other factors, and most importantly having a methodology that can stitch these approaches together.

Advert: OBIEE Developer Training

May 12th, 2008 by Mark Rittman

I’ve just updated our training page with details of our Oracle BI Suite Enterprise Edition developer course. This course is aimed at organizations who have recently bought OBIEE and are looking to train their development team up on creating the metadata layer, adding security, using features like caching and aggregate persistence, creating reports and publishing them to the dashboard, and using features such as BI Publisher and Delivers. It’s a fairly intensive three-day course taught either by myself or Borkur, with 35 separate hands-on labs where you really get to grips with how the tools work.

One thing we’ve added recently is a section where we take the various heterogenous sources that we use in the exercises and migrate them to a small data mart using Oracle Warehouse Builder 11g, then “re-wire” the logical model to point to this single data source instead. In my experience this is one of those areas you’re never going to get on a standard public OBIEE course, but it reflects how developers might want to take their system on once they’ve gone beyond the prototyping phase.

Anyway, if you’re interested, drop us a line and we’ll tell you more about it. We’ve also recently put together an Oracle BI Publisher for Developers course that goes through report building but also integration with environments such as ApEx and Oracle Forms, I’ll post more on that when we pull the datasheet together.

“Bashups” : Data Mashups Meet Business Intelligence

May 10th, 2008 by Mark Rittman

Dan Vlamis sent me a link through to a recent article by Mark Whitehorn on “Data Bashups”, an idea that takes business intelligence and uses it in “mashups”:

“For example, suppose you are interested in the effect an advertising spend is having citywide. Your company invested in renting several prominent and expensive billboards in a city to advertise your latest products. Your current BI system can probably return the percentage sales growth by city area over the last month as a list of raw numbers from which you can create a graph and work out whether your billboards are earning their keep. Alternatively, you could mash up the same values as a heat map laid over the city and see instantly whether the high growth correlates with the billboards’ locations.

With services like Virtual Earth and Google Earth, you can do this across one city, the country, the continent or the world. Better yet, all that mapping comes free of charge. And it keeps on getting better.

Now suppose that your BI system tells you that, in the past, sales of hot dogs double whenever the local baseball team plays at home after winning three games in a row. It’s a fascinating factoid, but it doesn’t make you any extra profit unless you can get the extra dogs to the relevant stadium in time for the right game. Your company is very unlikely to store baseball results, but there are plenty of sites out there that do. And there are plenty of shipping companies that provide data about trucking availability and pricing. On its own, your BI system is simply predicting the future. But mash up that prediction with these external data sources and suddenly your BI system is able to predict the future in a way that you can act on quickly to increase profits.”

Of course all this “mashup” talk reminds me of when I used to be a DJ, but it’s no doubt a very interesting idea and one that, as Dan said in his email, links in well with all the talk around BI and Service-Orientated Architecture. I’ll be keeping an eye on this area in future, and it’s got me thinking about putting something like this together for one of my UKOUG conference submissions.