Flying Back from Johannesburg, and Thoughts on BI and the SOA Suite Demo
May 16th, 2007 by Mark Rittman
I’m currently sitting on the flight back from Johannesburg to London Heathrow, after running an Oracle BI Enterprise Edition workshop on behalf of Oracle South Africa. On the way back from the conference centre, I did smile to myself as I saw a road sign for Pretoria out of the car window – it’s not often you go to work for the day in Southern Africa and plan to get back in the morning to do a days work in Brighton. Still, it was very much worth it, and it was a great experience taking a room full of delegates through their first hands-on experience with the BI Server, Answers, Dashboards and BI Publisher.
I was very impressed with the effort everyone had made for the event – we started off with an introductory talk on the platform for around a hundred or so attendees, then forty of so of them stayed on past the introductory talk to take part in the hands-on. This was actually the first hands-on I’ve done with the tools – so far, the seminars around Europe have always been “show-and-tell”, but although it took a lot of effort on Oracle’s part (setting up the room, software and so on) and mine (creating the hands-on examples) I think it went very, very well actually – actually getting your hands on the tools and working through examples is often a lot more fulfilling than just watching someone demo them, and bearing this in mind I’ll probably offer Oracle University the option of hands-on sessions for the next round of seminars I’ll be running from July this year. As I said, it’s a lot of work upfront for me and on the day for the local team, but if it can be done, I think the overall benefit of the seminar is much enhanced. I’ll have to see if I can talk OU in to doing this for the next round of seminars proper.
Anyway, as I said the event went off pretty well, and credit to Oracle in Johannesburg for going to all the trouble of setting up the event, getting me down there, and laying on this “test-drive” for their customers, who were also, I have to say, excellent and picked things up really quickly. Although I could only stay for a day, I’d be keen to go back later this year with the new “BI and SOA” seminar, and maybe get a chance to catch up with some of the people I spoke with who are planning to roll out BI EE on their next project.
As for me now, I’m up and running with the SOA Suite Order Bookings demo, and getting my head around all the new SOA technologies such as Enterprise Service Bus, Business Process Execution Language, Web Services and Rules engines. It’s not completely new to me – I’ve kept an interested eyed on SOA and the new loosely-coupled development paradigm over the past few years, and I’m fairly comfortable with the world of Java application servers, J2EE, JNDI, JDBC and so on – working with Oracle Application Server over the past few years, plus tools such as XML Publisher, Oracle Business Rules and so on gets you exposed to the world of Java and enterprise software fairly quickly. For me though, not being totally familiar with all these SOA tools is the whole point of the next seminar – I’m writing it from the perspective of a business intelligence developer who’s read about all these new opportunities to embed BI in applications and business processes, and wants to know enough about these new SOA tools such that they can plumb Oracle BI Enterprise Edition, Oracle Real-Time Decisions and so on in to existing applications and business processes, i.e. it’s about integration rather than writing applications from scratch. With this in mind, I’m working through the SOA Suite Order Bookings demo, and looking for opportunities to embed BI functionality in it, and also to spot occasions where business processes and other SOA functionality might be callable from a BI dashboard.
So far I’ve got the demo installed, and I’ve worked through the first set of examples where you first browse items for sale using the SOA Demo Web client application, and then create an order for 10 items totaling $1300.

Then, when the user places the order, the Web client sends a message to Enterprise Service Bus saying an order has been initiated, which then calls a BPEL business process to handle the order.
The BPEL business processes then goes off and retrieves a new order ID, stores the order in the database, checks the customers credit card is valid and then determines, using the Oracle Rules engine, whether the order needs manual approval.

Now in the demo, the rule is static – if the order is under $1000 in total, or the customer is a “platinum” customer, then the order goes through automatically, otherwise it requires manual approval. Now here’s somewhere where we could start to use BI functionality – instead of customers statically being platinum, or gold, or whatever, we could dynamically calculate the customer’s worth, perhaps by placing the customers in to segments based on profitability on a real-time basis, or by working out based on the last, say 10 transactions, whether they’re above-average profitable or not. We could work this out using an Oracle Answers report, make the report available as a Web service, and call the report on demand for a given customer ID from the business process.
We could also try and make the business rule itself more dynamic – instead of a hard limit of X dollars or a certain status, we could set more broad business objective rules, such as “keep the order process profitable” or “make sure we keep fraud below X dollars per month”, and have BI Enterprise Edition – perhaps Real-Time Decisions – work out the most appropriate set of order approval rules based on these overall goals.
Moving on in the demo, there’s a manual approval process for those orders that according to the current business rules need them, and then the BPEL process goes and gets quotes from two suppliers for the items ordered by the customer.

Now at the moment, the decision on which supplier to use is purely down to who provides the lowest quote, but of course in reality when you come to select a supplier, other factors come in to play – how reliable are they, what sort of customer satisfaction scores have they ended up with, and so on. Now if could replace this “pick the cheapest supplier” rule with “pick the supplier that overall has contributed most to our profitability”, perhaps with some sort of feedback loop in to the process where we can enhance our decision making with the actual result of assigning the work – did the customer return the order, did they complain, did they repeat order afterwards – we could hone the process further and make it “self-optimizing”.
So, I can think of at least three situations where BI, and Oracle BI Enterprise Edition, could come in to play. Firstly, as a way of providing a better, more real-time measure as to whether the customer is good for a large order, and secondly perhaps as a way of fine-tuning in real-time the rules on whether an order is referred or not. The third opportunity is to use a tool such as Real-Time Decisions to enhance the supplier choice process to take into account other factors, and to make the decision process self-learning and modifying based on the outcome of previous decisions.
The area I haven’t really looked at so far is doing things the other way around – having Oracle BI Suite EE calling a business process, either through an iBot calling a BPEL process, or having the user trigger a BPEL process manually based on some insight they’ve had when working through their dashboard. Ideally, I’ll re-use the Order Bookings demo with this, I think the key to it is working out how to get an Delivers iBot to call a BPEL process, and how to create a link on a report that again calls out to a BPEL process. I’m meeting up with some people the week after next to try and build this all out, once it’s up and running I’ll post another blog article on how it all works.
On a final note, one bit of SOA Suite that I did find interesting was Enterprise Service Bus (ESB). What ESB appears to do is route messages, with the ability to inspect the message payload to determine where to send it, transform the data en-route and have it available to all end-points on the bus at the same time – which to me, sounds a abit like a real-time ETL process. In the demo, ESB is used to route the order through to the an appropriate distribution company based on the order value, and to register that the order has been shipped, like this:

Now I think if you were to compare, say ESB with a real-time ETL process built using Oracle Data Integrator, you’d say that ESB is all about messaging, routing the message based on rules (which can involve inspecting the message payload), and some limited transformation using XSL of the data as it goes across the bus, whilst a real-time ETL process is more event or batch driven, moves the data via (in ODI’s case) an agent and takes it from point-to-point. I wonder though whether ESB might be an alternative to a to trying to build a near real-time ETL process using, for example, ODI, when all you need to do is move data from an application or database to the reporting environment and do a bit of data transformation on the way? I suspect there’s some key reason why ESB isn’t realistically an alternative ETL tool, but it does seem to perform a somewhat similar function and I’ll keep the idea in my back pocket should a suitable situation arise in the future. For now though, it’s 23:30, and time I tried to get a few hours sleep, so I’ll sign off now and hopefully connect up again when I get back to the UK.


May 16th, 2007 at 8:20 am
I think one reason why ESB is not a realistic alternative to ETL is the “bloat” of messaging – in web world these messages are often blocks of XML passing from application to application – they have tags before and /tags after the data – this adds loads of bytes to move and trash on what is esentially a batched process. ESB might be OK for true realtime but the more that records get batched together the less attractive it looks.
Which reminds me – what is “real time” in a BI sense? – it is all down to latency – the gap between the event and be able to report it (which of course includes the time to move data to the reporting table and to execute the query and return results – it is not just the load time)
May 16th, 2007 at 10:15 am
Thanks for the BI EE workshop it was excellent…i’m ready to develop BI reports for my company…have a safe journey
May 16th, 2007 at 7:47 pm
Desmond,
You’re welcome, glad it was a useful day for you. Good luck with starting to use BI EE, let me know how you get on and what aspects of the toolset you’ve found particularly useful.
regards
Mark
May 16th, 2007 at 7:51 pm
Pete – good point, a useful bit of reality interjecting in to the theory there, although if we’re trickle-feeding data through in low volumes, I wonder if it’d still be a significant issue?
Good point about “real-time” as well – even tools such as ODI that make a big deal out of their real-time features, still have some degree of latency or other; either the tool waits for, say, five new rows of data to be logged before starting the process (to reduce the load on the ETL server), or a batch runs every minute or so to load data in. Most people would still think of this in real-time (most people except embedded systems engineers, I guess) but as you say, there’s still an inherent latency of some form in both of these approaches.
regards
Mark
May 17th, 2007 at 7:15 am
I think it’s difficult (and probably self-defeating) to try to define clear boundaries between TLAs like ETL / EII / EAI / ESB. Many times, the acronym chosen indicates the vendor’s vackground and approach to solving the generic “integration” problem, and how they want to present to the market.
I know of message/event-oriented EAI tools that could be described as ESB (or at least “esb”), eg DataMirror’s Transformation Server uses a publish/subscribe model – even though the message bearer is proprietary (or was 5 years ago) because (a) it predated the JMS/ESB wave and (b) performance.
Another way of looking at it is that, as you say, an ESB concentrates on message-level routing and transformation, while EAI/ETL are more focussed on source and target adapters and often bulk data transit. It’s all shades of grey really.
Regards Nigel
May 22nd, 2007 at 1:58 pm
It is useful to point out here that an ESB really brings its worth when used in conjunction with a higher process level orchestration whereby the ESB implements the business rules while the the Process Engine lets you sequence calls to these low level services.If all you want to do is get data from A to B then ESB/ODI/EAI will all do the job.