My first trip to Qatar is now coming to an end. I have been working pretty much every day since I got here and now it’s time to head back home to ‘sunny Brussels’. As a member of the RittmanMead Virtual Team I will be returning out here in a two weeks time with two other consultants to finish the project; a proof of concept involving OWB, Portal and Discoverer.

Hopefully we will see an article from a new author on this blog very soon. My friend Nicolas De Brabanter has been cooking up an interesting how-to on using the ‘act-as’ feature in OBIEE: Allowing authorized users to run reports and view dashboards with ‘borrowed eyes’. Looking forward to reading it.

Week two starts tomorrow. We are working with OWB, Discoverer and Portal to come up with a proof of concept as well as training the client in using these tools. The team is very enthusiastic and eager to learn so the work is very enjoyable. The working hours are rather different from what I am used to; Start at 8 and then take a break at around 15:00 and then start back at around 19:30 and work until about 22:00. In between, it’s back to the hotel, lunch and perhaps go to the gym. With the different rhythm, I seem to have lost close to 5kg here (or is it just because of the lack of beer ? :)) in just one week. Somehow I think I will manage to find these lost kilos when I get back …

Before I took off to day I had finish up on a few things with my client. One thing was to look into some performance problems that we were having on out OBIEE dashboard. The underlying data for the particular reports is a 60M+ records partitioned table containing monthly aggregates. What we have set up in the business layer is an aggregate aware fact table that should use this monthly aggregate at all times in the dashboard (as a sideline – my flights tonight will be showing up in the underlying facts and datamarts come Monday morning :)). What we had done also, a while back, was to set up additional materialized views, only intended for query rewrite purposes. When the client started complaining about the performance issues, my thoughts went immediately to the aforementioned mview. “Ahh, the view is just stale and all I need to do is refresh it and Bob’s your uncle!”. Alas, the view was not the culprit here so I then (and only then :|) took a look at the actual query in the nqQuery.log to take a further look at the execution plan. What I soon noticed then was that instead of using all my nice little bitmap indices, the CBO was doing full range partition scans! I then turned my attention to the bitmaps and found out that there weren’t any! What? But! I am sure I had create a bunch of these before! The datamart has about 9 dimensions and only the date key and two dimension keys were actually bitmapped. So, I saw no point in trying to figure out why these weren’t in place or who’s fault it was they were missing (mainly because it was most likely myself who had dropped these during a back-loading process we had to go through about a month ago …). I quickly wrote a PL/SQL block to generate the missing bitmap indices and after asking the ops DBA to put the tablespaces containing the historical partitions in READ WRITE, the bitmaps got created and then (and only then) Bob was everybody’s uncle :)

Lesson learned? Make sure you have all the bits and pieces in place and keep an eye on your structures. Bad performance can really damage a DW project and don’t count on the end-users to report bad performance.

But the summer did bring me some changes. For the last 5 weeks I have been working as an independent consultant which allows me to diversify my work quite a lot. But it also means I have to secure contracts to keep busy, which is new for me. Mark and Jon have been throwing me some work and I am really looking forward to work more with them in the future. Also, the UKOUG is coming up and I have a paper to present there so I should be pretty busy until the end of the year, with all this and my work here in Belgium.

I started on writing up a short guide on setting up multiple presentation services and BI servers on one machine, as a way of supporting a  development and testing environment  with less hardware. It should be up here in a few days :)

In the past I have blogged about a third party application that my team provides a little support for. Most of my blogs about this application tend to move towards ‘rant’ – the application is poorly designed (no foreign keys, few indexes, inappropriate data types) – in short the kind of app where the developers have written their own RI code in the application and ignored the capabilities of the database. In a more charitable moment I would say that the database components were poorly translated into Oracle, but for now I can’t think of a variant of SQL where they would have been well written. Recently our customer has asked that some new account codes are loaded into the application, and as the original vendor can’t help or indeed has any staff with access to the design documentation our customer has asked us to investigate reverse engineering the database to locate the table that contains this data. Hence my dip into the database and headache of trying to decipher other people’s code.

There is not much in the way of PL/SQL in this application; just two procedures, one to load data and one to extract it. I have already talked about the extraction code so this time I looked at the data load code. I am firmly convinced that if I can not see all of the logic in a single screenful of code editor then I am going to despise the code (and coder responsible) – perhaps my attention span is waning in old age. This code is massive, hundreds and hundreds of lines in one single block inserts, updates, and deletes galore over at least 8 tables. In data warehouse terms the process is a classic; load to staging tables adding a surrogate key, compare data with production then update or insert and log the results (updates, inserts, rejects, and batch duration) to a log table, but the way the developers to do it!

• the emptying of the 6 source stage tables by using DELETE (why, oh why, not TRUNCATE?)
• The insert into the log table of the run number and run start time (a novel use of SELECT 1 + COUNT(*) from logging_table as the source of the run number) – which of course means we can’t purge records from the log table without reducing the run number
• The numerous updates to the logging table (at least 10 of them) using WHERE RUN_NUMBER=(SELECT MAX(RUN_NUMBER) from LOGGING_TABLE) as a filter (haven’t these guys heard of storing the run_number in a variable… it won’t change during a run)
• using those 10 updates where one would have done
• inserting into the stage tables and then updating rows to fix up nulls. Then updating updating again to provide derived values, then updating again to build concatenated text keys. Each update having its own commit. To make matters worse they are flitting between all 6 stage tables. I would have tackled one table at a time and tried to write the code as a single insert with no updates (CASE and NVL() are your friends in ETL coding)
• We then get to the removal of duplicates – I would prefer to copy out the duplicates so that I could report them back to the source data owner. Still, at least they count the duplicates before deleting them.
• We then copy out the surrogate keys from the denormalised target table and insert them into the individual staging tables. Then check for changes and process as updates before finally inserting any new records into the denormalised table.
• And to finish we fix up any Nulls in the denormalised table (as 20 separate update statements with differing where clauses) – and because of a quirk of how we process duplicates we can get nulls in to the denormalised table. And finally record the batch run time (in minutes) using our final update of the logging table and the most odd looking time calculation I have seen in an age.
• Oh, and there is no exception handling code

Today, I aged five years wading through that tosh – Tuesday, I’ll rewrite it so that anyone can understand it, or perhaps I won’t bother…

My original problem was to convert a unix time number (number of seconds since 01 Jan 1970) to an Oracle date. A quick Google brought me to AskTom . He does point out that my timezone might not be supported. His solutions is as simple and clean as expected, but DST doesn’t allow for the quick fix of just adding the “gmt offset”.

The link above, to the Oracle documents, does point to an example of using FROM_TZ,CAST and AT TIME ZONE functions to get this done. But back to the original NEW_TIME function: why not add the rest of the timezones in there? Here is one way of doing this (try using the values 1183680000 and 1168041600)

SELECT FROM_TZ(CAST(TO_DATE(’1970-01-01′,
‘YYYY-MM-DD’)+ 1/24/60/60 * :theTime AS TIMESTAMP), ‘UTC’)
AT TIME ZONE ‘Europe/Brussels’ “Brussels time”
FROM DUAL;

So, now I can at least know if anything happens on a thread I have been watching. There have been several threads on this. The one I watched on the subject (and obviously never got any update on) is here The message count on that thread is up to 290 …

Of course design matters; and getting it right is important for the success of any development project. For data warehousing this means an holistic approach – the user base wants to exploit their data capital for business advantage, so they need a system that:

1) Provides the correct results
2) Provides those results in a reasonable timescale
3) Allows new data to be added to the DW, again in an acceptable batch window

Get any of these wrong and your project will be seen as a failure in the eyes of some or all of your users. Remember, it is these users that ‘pay’ for the systems, the days of IT departments sponsoring a development project to keep staff ‘gainfully employed’ are long gone. Data warehouses should be seen as a strategic asset and therefore be designed for sustainability and longevity – this is not a technology area for the ‘quick fix’ or those who shun best practice.

Oh, and one more part of my rant – vendor neutral database design does not work in a data warehouses – Oracle has a fantastic set of performance enhancing features for large databases, omit using them appropriately and you will struggle to deliver.